Short summary of research activities of the e-Lite research group (http://elite.polito.it) at Politecnico di Torino (Italy) on the topic of Smart Environments and the application of Ontologies and Semantic Computing to Ambient Intelligence.

1. Politecnico di Torino
Dipartimento di Automatica e Informatica
e-Lite Research Group
http://elite.polito.it
Research activities in
Ambient Intelligence
Fulvio Corno – fulvio.corno@polito.it

2. Home Automation System (HAS)
2 Dario Bonino - Home and Building 6/11/2012
Automation Systems

3. Home Automation System (HAS)
 Home automation
 Automation of the home, housework or
household activity.
 (remote) control of:
 lighting
 HVAC (Heating, Ventilation and Air
Conditioning)
 appliances
 and other systems
 Home automation system
 An integrated system (computer-based)
offering home automation functionalities
 Integrates electrical devices in a house
 Through a communication network
 Possibly includes devices using different
communication protocols
3 Dario Bonino - Home and Building Automation Systems 6/11/2012

4. Building Automation System (BAS)
4 Dario Bonino - Home and Building 6/11/2012
Automation Systems

5. Building Automation System (BAS)
 Building automation
 The advanced functionality provided
by the control system of a building
 E.g., security & access control, fire
detection & alarms, HVAC, lighting
control, air quality, smoke detection,
intrusion detection, environmental
control, asset location/management
 Building Automation System(BAS)
 A computerized, intelligent network
of electronic devices designed to
monitor and control the mechanical,
electronic, and lighting systems in a
building
5 Dario Bonino - Home and Building 6/11/2012
Automation Systems

6. Home vs. Building Automation
Building
Home Automation
Automation
 Home Automation is almost a subset of
Building Automation
 Most functionalities are shared
 Different protocols and technologies
6 Dario Bonino - Home and Building 6/11/2012
Automation Systems

7. Main goals
 Interoperability
 Intelligence
 Man-Environment
interaction
 Accessibility
7 e-lite - AmI+DOG 26/03/2010

8. Technology Overview
Building Automation
RS-485
Home
Automation
8 Dario Bonino - Home and Building 6/11/2012
Automation Systems

9. Pot-pourri di dispositivi
9 e-lite - Smart Buildings 20/04/2012

10. Network Technology
 Bus  Powerline
 MyOpen  Echelon
 KNX  X10
 Modbus  Insteon
 Echelon  Wireless
 Dali  ZigBee
 CAN  Z-Wave
 MBus  EnOcean
10 Dario Bonino - Home and Building 6/11/2012
Automation Systems

11. Main goals
 Interoperability  Abstract high-level
 Intelligence (semantic) model
 Man-Environment  DogOnt
interaction  Intelligent Multi-protocol
 Accessibility Gateway
 DOG (Domotic OSGi
Gateway)
 Modular architecture and
open APIs
 Open Source
11 e-lite - AmI+DOG 26/03/2010

12. Application Area
 Automation  Entertainment
 Real-time Control  UPnP
 CAN  DLNA
 KNX, MyHome, Insteon,  General purpose
Echelon, Modbus, X10, ZigBee,
 Bluetooth
Z-Wave, EnOcean
 WiFi
 Lighting
 Dali
 (all Automation)
 Metering
 Mbus
 RS-485
 KNX, Modbus, Echelon,
ZigBee, Z-Wave,…
12 Dario Bonino - Home and Building 6/11/2012
Automation Systems

13. Challenges
 Integration
 Different electrical requirements
 Different interaction modalities
 Different behaviors (temporal, etc.)
 Interoperation
 Different protocols
 Different interaction modalities
 Master/slave
 Peer-to-peer
 Etc.
 Modeling
 Different technologies & assumptions
 A single shared, common description
13 Dario Bonino - Home and Building Automation Systems 6/11/2012

14. Pot-pourri di applicazioni
14 e-lite - Smart Buildings 20/04/2012

15. Intelligent Domotic Environments
15 Dario Bonino - Dog 2.3 6/11/2012

16. System Architecture
User Mobile, Web, Home Display,
User Multi Touch, Accessibility,
Interface
User Natural language, …
Interface
Interface
Data ERP, Web services, Stream
processors, Datawarehouse
analysis
Device abstraction, Event
Dog abstraction, State abstraction,
Bundles Rules engine, …
Ethernet, Wi-Fi, USB
Bus-to-IP gateway
GW GW Bus-to-serial gateway
Domotic bus (wired, wireless)
Smart D D D D D D D D Domotic devices (switches,
buttons, relays, sensors, meters,
Appliance
…)
16 Dario Bonino - Dog 2.3 6/11/2012

17. In Ontology We Trust
 Devices and networks exposed by means of a formal,
unique, representation
 DogOnt (Ontology)
 Applications only see DogOnt-based device descriptions
 Functionalities
 Notifications
 Commands
 States
 State values
 Internal representations and drivers must be ontology-
aware, at different degrees
17 Dario Bonino - Dog 2.3 6/11/2012

18. E allora?
Applicazione Applicazione Applicazione Applicazione
D D D D D D D D D D D D
18 e-lite - Smart Buildings 20/04/2012

19. Una lingua franca
Applicazione Applicazione Applicazione Applicazione
Rappresentazione neutrale
D D D D D D D D D D D D
19 e-lite - Smart Buildings 20/04/2012

20. Architetture orizzontali e aperte
Applicazione Applicazione Applicazione Applicazione
Elaborazione real time Regole e scenari Servizio
applicativo
API
Funzionalità di base Scambio dati
Rappresentazione neutrale
Driver di interfacciamento ai protocolli
D D D D D D D D D D D D
20 e-lite - Smart Buildings 20/04/2012

21. Applicazioni consumano
dati e servizi
Applicazione Applicazione Applicazione Applicazione
Elaborazione real time Regole e scenari Servizio
applicativo
API
Funzionalità di base Scambio dati
Rappresentazione neutrale
Driver di interfacciamento ai protocolli
D D D D D D D D D D D D
21 e-lite - Smart Buildings 20/04/2012

22. Applicazioni controllano
Applicazione Applicazione Applicazione Applicazione
Elaborazione real time Regole e scenari Servizio
applicativo
API
Funzionalità di base Scambio dati
Rappresentazione neutrale
Driver di interfacciamento ai protocolli
D D D D D D D D D D D D
22 e-lite - Smart Buildings 20/04/2012

23. DogOnt
 DogOnt [ISWC08]
 Ontology-based formal modeling of domotic
environments (OWL)
 Technology independent device modeling
 Operation semantics for DOG [TCE08]
 Interoperation semantics through relations [AMI09]
 Automatic generation of inter-operation rules [TCE09]
23 e-lite - AmI+DOG 26/03/2010

24. Semantic Modeling (DogOnt)
RDF
OWL
XML
XSD
24 e-lite - AmI+DOG 26/03/2010

25. Different Aspects In the same Model
 A Lamp is
 A thing (BuildingThing)
 It is controllable (Controllable)
 (On/Off) (OnOffFunctionality)
 It can be either Lit or not Lit (OnOffState)
 It is located in a Room (isIn Room)
 It is connected to a domotic plant (isA
DomoticNetworkComponent)
25 e-lite - AmI+DOG 26/03/2010

26. Different Aspects in the same Model
Building
Thing IsIn / contains
Building
Environment
Controllable
House Building Apartment
Plant
Electric
System Lamp hasState
Room
OnOff
hasFunctionality State
Control
Functionality
Discrete
OnOff State
Functionality
State
Functionality
26 e-lite - AmI+DOG 26/03/2010

27. In Practice (Protégé & OWL)
<owl:Class rdf:about="#SimpleLamp">
<rdfs:comment
rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>Simple lamp that can be just turn on or turn off</rdfs:comment>
<owl:disjointWith>
<owl:Class rdf:about="#DimmerLamp"/>
</owl:disjointWith>
<rdfs:label
rdf:datatype="http://www.w3.org/2001/XMLSchema#string"
>SimpleLamp</rdfs:label>
<rdfs:subClassOf>
<owl:Restriction>
<owl:someValuesFrom rdf:resource="#QueryFunctionality"/>
<owl:onProperty>
<owl:ObjectProperty rdf:about="#hasFunctionality"/>
</owl:onProperty>
</owl:Restriction>
</rdfs:subClassOf>
<rdfs:subClassOf rdf:resource="#Lamp"/>
</owl:Class>
27 e-lite - AmI+DOG 26/03/2010

28. Sample Room Model in DogOnt
OffNotification
OnNotification hasNotification
hasNotification
OnOffNotification
Functionality
OnOffState hasFunctionality
generatesCmd hasState Switch
hasState
Lamp
hasFunctionality isIn
isIn OnOffState
OnOffFunctionality
hasCommand Sample Room
hasCommand
OffCommand generatesCmd
OnCommand
28 e-lite - AmI+DOG 26/03/2010

29. DOG - Domotic OSGi Gateway
http://domoticdog.sourceforge.net/
29 e-lite - AmI+DOG 26/03/2010

30. System Architecture
Mobile, Web, Home Display,
Multi Touch, Accessibility,
User
Natural language, …
Interface
Data ERP, Web services, Stream
processors, Datawarehouse
analysis
Device abstraction, Event
Dog abstraction, State abstraction,
Bundles Rules engine, …
Ethernet, Wi-Fi, USB
Bus-to-IP gateway
GW GW Bus-to-serial gateway
Domotic bus (wired, wireless)
D D D D D D D D Domotic devices (switches,
buttons, relays, sensors, meters,
…)
30 e-lite - AmI+DOG 26/03/2010

31. The KNX demo box
D
GW
31 e-lite - AmI+DOG 26/03/2010

32. The Z-Wave demo box
GW
D
32 e-lite - AmI+DOG 26/03/2010

33. Texas Intruments eZ430-Chronos
33 e-lite - AmI+DOG 26/03/2010

34. DogEye user interface
34 e-lite - AmI+DOG 26/03/2010

35. DogEye Video
35 e-lite - AmI+DOG 26/03/2010

36. DogEye with an eye tracker
36 e-lite - AmI+DOG 26/03/2010

37. Mobile House Control
37 e-lite - AmI+DOG 26/03/2010

38. System Architecture
Mobile, Web, Home Display,
Multi Touch, Accessibility,
User
Natural language, …
Interface
Data ERP, Web services, Stream
processors, Datawarehouse
analysis
Device abstraction, Event
Dog abstraction, State abstraction,
Bundles Rules engine, …
Ethernet, Wi-Fi, USB
Bus-to-IP gateway
GW GW Bus-to-serial gateway
Domotic bus (wired, wireless)
D D D D D D D D Domotic devices (switches,
buttons, relays, sensors, meters,
…)
38 e-lite - AmI+DOG 26/03/2010

39. OSGi Alliance
 Open Services Gateway Initiative
 defines open specifications for
 Creating and delivering distributed and
concurrent applications, called bundles
 Bundle deployment and installation (core
specification)
 Communication among bundles (core specification)
 Devices and drivers management (service
compendium specification)
39 e-lite - AmI+DOG 26/03/2010

40. OSGi definitions
 OSGi frameworks
 Implementation in JAVA of the OSGi specifications
(equinox, oscar, ..)
 Manage the life cycle of the bundles
 Resolve the dependences among bundles
 Manage Service Registry
 Bundles
 Jar file
 contain java classes and resources
 Provide and use services
 Services
 java objects registered in the framework
 Defined by java interfaces
40 e-lite - AmI+DOG 26/03/2010

41. High-Level Architecture
 OSGi - based
 2 main layers
 3 main bundle groups
 Core
 Drivers
 Add-ons
41 Dario Bonino - Dog 2.3 6/11/2012

42. Core
DogXMLEndPoint DogRESTEndPoint
DogStateMonitor DogScheduler DogExecutor DogDeviceManager DogNotificationManager
DogAutoStart
DogDeviceFactory DogSimpleHouseModel
DogSemanticHouseModel
DogOntLibrary DogDeviceModel
DogConfigurator DogLogger
Dog2Library DogJaxBLibrary DogSemanticLibrary MeasureLibrary org.rxtx
42 Dario Bonino - Dog 2.3 6/11/2012

43. Command handling (1)
Application
Command
Dog
43 Dario Bonino - Dog 2.3 6/11/2012

44. Command handling (2)
Command
DogXMLEndPoint DogRESTEndPoint
DogStateMonitor DogScheduler DogExecutor DogDeviceManager DogNotificationManager
DogAutoStart
DogDeviceFactory DogSimpleHouseModel
DogSemanticHouseModel
DogOntLibrary DogDeviceModel
DogConfigurator DogLogger
Dog2Library DogJaxBLibrary DogSemanticLibrary MeasureLibrary org.rxtx
KNX Modbus Echelon OpenWebNet ZWave
Network Message
44 Dario Bonino - Dog 2.3 6/11/2012

45. Notification handling (1)
Application
Notification
Dog
45 Dario Bonino - Dog 2.3 6/11/2012

46. Notification handling
DogXMLEndPoint DogRESTEndPoint Rules
DogState Dog Dog DogDevice DogNotification
Monitor Scheduler Executor Manager Manager
DogAutoStart
DogDeviceFactory DogSimpleHouseModel PowerBundle
DogSemanticHouseModel
DogOnt DogDevice PowerModel
Library Model Bundle
DogConfigurator DogLogger
DogSemantic
Dog2Library DogJaxBLibrary MeasureLibrary org.rxtx
Library
KNX Modbus Echelon OpenWebNet ZWave
46 Dario Bonino - Dog 2.3 6/11/2012

47. In DogOnt we trust (3)
Thing
Configuration Lamp1
DogOnt Building thing
Controls
Instances
Controllable UnControllablle
Button1
Appliances HousePlants Lamp2
Controls
Auto
Brown Electrical HVAC
Generation
White Security
Goods Goods System System
Power
Consumption Device Semantic
Representations House
Applications
Model
Eletric Power
Consumption
Configuration Power
Lamp1Pw = 18W Model
47 Dario Bonino - Dog 2.3 6/11/2012

48. Device Category (structure)
Lighting
public interface Lamp extends Lighting
{
isA public static int MATCH_TYPE=100;
public static int MATCH_SUB_TYPE=50;
public static int MATCH_MANUFACTURER=0;
Lamp
OnOffState public static String ON_OFF_STATE =
"OnOffState";
hasState instance public void off();
public void on();
}
Lamp
hasFunctionality
OnOffFunctionality
hasCommand
hasCommand
OnCommand
OffCommand
48 e-lite - AmI+DOG 26/03/2010

49. Device Model (Structure)
public class DogLamp extends DogDevice implements Lamp
{
public DogLamp(BundleContext context, Properties properties){
super(context);
if(properties==null)
properties=new Properties();
properties.put(Constants.DEVICE_CATEGORY, Lamp.class.getCanonicalName());
this.setDeviceProperties(properties);
this.states.put(Lamp.ON_OFF_STATE,
new DogStatusVariable(
properties.getProperty(DogDeviceCostants.DEVICEURI),
new StatusVariable(Lamp.ON_OFF_STATE, StatusVariable.CM_DER, "-"),
"current state of Lamp",true));
this.registerDevice(Lamp.class.getName());
this.registerStatusVariable();
}
@Override
public void off(){ OnOffState Lighting
((Lamp) this.driver).off();
}
@Override
hasState
public void on(){
isA
((Lamp) this.driver).on();
}
} Lamp Lamp
hasFunctionality
OffCommand instance
hasCommand OnOffFunctionality
OnCommand
49 e-lite - AmI+DOG 26/03/2010

50. Device Category (Hi-Fi)
package it.polito.elite.domotics.model.devicecategory;
/**
* HiFiCategory - automatically generated by DogOnt2Dog
*/
public interface HiFi extends Entertainment
{
public static int MATCH_TYPE=100;
public static int MATCH_SUB_TYPE=50;
public static int MATCH_MANUFACTURER=0;
public static String STAND_BY_ON_OFF_STATE = "StandByOnOffState";
public static String SOURCE_SELECTION_STATE = "SourceSelectionState";
public static String PLAY_STATE = "PlayState";
public static String TUNER_STATE = "TunerState";
public static String VOLUME_LEVEL_STATE = "VolumeLevelState";
public void stop();
public void stepUp();
public void rew();
public void set(Object value);
public void setVolume(Integer volume);
public void next();
…
public void goToTrack(Integer trackNumber);
}
50 e-lite - AmI+DOG 26/03/2010

51. Automatic Generation
 Template based
 Common structure of device categories and device models
 Fill “placeholders” (possibly empty) with information
extracted from DogOnt
 SPARQL
 Works on the schema only
51 e-lite - AmI+DOG 26/03/2010

52. Automatic Generation (recipe)
Device Categories Device Models
1. GetAllDevicesWithAncestors 1. GetAllDevicesWithAncestors
2. Foreach device 2. Foreach device
1. getDeviceCommands 1. getDeviceCommands
2. getDeviceStates 2. getDeviceStates
3. Write the device category 3. Write the device model code
code 3. Compile all
3. Compile all 4. Jar all with manifest
4. Jar all with manifest (bundleize)
(bundleize) 5. Use in Dog2.0
5. Use in Dog2.0
52 e-lite - AmI+DOG 26/03/2010

53. Queries (1) - Devices
 Get all devices with ancestors
 SELECT DISTINCT ?x WHERE { ?x
rdfs:subClassOf dogont:Controllable FILTER
(?x!=owl:Nothing)} ORDER BY ?x
 Recurse over ?x and call
 OntClass deviceClass =
this.modelLoader.getPlainOntModel().getOntCla
ss(deviceURI);
 ExtendedIterator iter =
deviceClass.listSuperClasses();
53 e-lite - AmI+DOG 26/03/2010

54. Queries (2) - Commands
Non-Parametric Commands Parametric Commands
SELECT DISTINCT ?commandValue WHERE { SELECT DISTINCT ?commandValue ?commandParamValue WHERE {
dogont:"+deviceClass+" rdfs:subClassOf dogont:"+deviceClass+" rdfs:subClassOf [rdf:type
[rdf:type owl:Restriction; owl:onProperty owl:Restriction; owl:onProperty dogont:hasFunctionality;
dogont:hasFunctionality; owl:someValuesFrom owl:someValuesFrom ?functionality] . ?functionality
?functionality] . ?functionality rdfs:subClassOf dogont:ControlFunctionality . ?functionality
rdfs:subClassOf [rdf:type owl:Restriction; owl:onProperty
rdfs:subClassOf dogont:ControlFunctionality.
dogont:hasCommand; owl:someValuesFrom ?command] . ?command
?functionality rdfs:subClassOf [rdf:type
rdfs:subClassOf dogont:ParametricCommand . ?command
owl:Restriction; owl:onProperty
rdfs:subClassOf [rdf:type owl:Restriction; owl:onProperty
dogont:hasCommand; owl:someValuesFrom
dogont:realCommandName; owl:hasValue ?commandValue] .
?command] . ?command rdfs:subClassOf
?command rdfs:subClassOf [rdf:type owl:Restriction;
dogont:NonParametricCommand . ?command
owl:onProperty dogont:commandParamName; owl:hasValue
rdfs:subClassOf [rdf:type owl:Restriction;
?commandParamValue]} ORDER BY ?commandValue
owl:onProperty dogont:realCommandName;
owl:hasValue ?commandValue]} ORDER BY
?commandValue
54 e-lite - AmI+DOG 26/03/2010

55. Queries(3) - States
 SELECT ?state WHERE{
dogont:"+deviceClass+" rdfs:subClassOf
[rdf:type owl:Restriction;
owl:onProperty dogont:hasState;
owl:someValuesFrom ?state] } ORDER BY
?state
55 e-lite - AmI+DOG 26/03/2010

56. Accessibility / Disability
56 e-lite - AmI+DOG 26/03/2010

57. Testing Dog in a Real Home
57 e-lite - AmI+DOG 26/03/2010

58. http://elite.polito.it/spchains
Stream Processing spChains
spChains
Stream Pattern Match / Alerts
Processing
Block
Data-centric
Event Sources Application
Event Drains
Field Stream Aggregate / Computed (e.g., E.R.P.)
Data Streams Processing Measures
Chains
Field-level sensors / data sources
Chain Definition
Industrial
Stream processing engine Stakeholders
(e.g., Management)
58 e-lite - Smart Buildings 20/04/2012

59. Alcuni esempi di spChains
 Ogni spChain è definita in XML
 Libreria di spBlock parametrici (13++)
 Creazione automatica di query sul motore Esper (o altri)
 Open Source
59 e-lite - Smart Buildings 20/04/2012

60. 60 e-lite - Smart Buildings 20/04/2012

61. Visual rule builder
61 e-lite - AmI+DOG 26/03/2010

62. Visual Rules Builder: video
62 e-lite - AmI+DOG 26/03/2010

63. Simulation
StateCharts
Apache
Commons
SPARQL
63 e-lite - AmI+DOG 26/03/2010

64. Simulation
OffStateValue OnStateValue Lamp Machine Template
Lamp1 Lamp1
hasStateValue hasStateValue
OnOffStateLamp1 &id;OffState &id;lampOn
hasState
Lamp isA SampleLamp1
&id;lampOff
&id;OnState
hasFunctionality
OnOffFunctionality
hasCommand
hasCommand
OffCommand
OnCommand
64 e-lite - AmI+DOG 26/03/2010

65. Simulation – Bank Door
65 e-lite - AmI+DOG 26/03/2010

66. Intelligence + Security
OWL
Drools
SWRL
SPARQL
66 e-lite - AmI+DOG 26/03/2010

67. Rules Generation (DOG1.0)
Query Rule
SELECT DISTINCT ?x ?y ?v ?c ?d when
?class ?cv WHERE { ?x a dev:Device( id ==
"http://elite.polito.it/ontologies/simple
dogont:Controllable . ?y a home.owl#"?x")
dogont:Controllable . ?x cmd:StateCommand(state =="?v")
dogont:controlledObject ?y . ?x status:DeviceStatus( status == cmd,
dogont:hasFunctionality ?f . ?f device == dev)
dogont:hasNotification ?n . ?n rulesCore:RulesCore()
dogont:notificationValue ?v . ?n message: DogMessage(dataType ==
dogont:generateCommand ?c . ?d DogBundle.DataTypeEnum.DEVICE_STATUS,
data == status)
dogont:hasFunctionality ?f2 . ?f2
then
dogont:hasCommand ?c . ?c
Device newDev = new
rdf:type ?class . ?class Device("http://elite.polito.it/ontologies
rdfs:subClassOf /DOGHouseModel.owl#"+?y+");
dogont:DiscreteCommand . ?class DeviceStatus newStatus = new
DeviceStatus(newDev, new
rdfs:subClassOf [rdf:type StateCommand("+?cv+”,StatusTypeEnum.DISCR
owl:Restriction; owl:onProperty ETE));
dogont:realCommandName; DogMessage myCmd = new
owl:hasValue ?cv]} DogMessage(MessageTypeEnum.CMD,new
GregorianCalendar(),
message.getDataType(), newStatus);
rulesCore.takeActions(myCmd);
67 e-lite - AmI+DOG 26/03/2010

68. Energy management
68 e-lite - AmI+DOG 26/03/2010

69. In-Home Display videos
DFP: Direct Feedback of GSE: Goal Setting for
istantaneous Power consumed Energy
69 e-lite - AmI+DOG 26/03/2010

70. Energy Awareness
70 e-lite - AmI+DOG 26/03/2010

71. Contacts
 Web site:
 http://elite.polito.it
 Dog2.0 information, documentation and downloads
 http://domoticdog.sourceforge.net
 Publications:
 http://elite.polito.it/publications-mainmenu-81
 Contact e-mail:
 fulvio.corno@polito.it
71 e-lite - AmI+DOG 26/03/2010

72. Get involved!
 User interfaces / User experience
 Artificial intelligence / Semantic modeling
 Wireless [sensor] networks
 Ambient Assisted Living applications
 Stream data processing
 Extension to Buildings and Industrial settings
 Energy savings / Energy management
 Linked-data information exchange
 Device modeling / Environment modeling
 …
72 e-lite - AmI+DOG 26/03/2010

73. Licenza d’uso
 Questa presentazione è rilasciate con la licenza Creative
Commons “Attribuzione-Non commerciale-Condividi allo
stesso modo 2.5 Italia”
 Siete liberi di riprodurre e modificare quest’opera, per
scopi non commerciali, e citando la fonte. Eventuali
versioni modificate dovranno essere rilasciate con la
stessa licenza
 Testo completo della licenza:
 http://creativecommons.org/licenses/by-nc-sa/2.5/it/
73 e-lite - AmI+DOG 26/03/2010