1. Grupo de Procesado de Datos y Simulación
ETSI de Telecomunicación
Universidad Politécnica de Madrid
Towards a lightweight mobile semantic-based approach
for enhancing interaction with smart objects
CONTEXT 2012
Josué Iglesias, Ana M. Bernardos, Luca Bergesio, Jesús Cano, José R. Casar
josue@grpss.ssr.upm.es

2. contents
 introduction and motivation
 smart spaces management scenario
 ECA-based interaction model
 semantic enhancement
 ontology models
 incompatibilities detection
 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

3. contents
 introduction and motivation
 smart spaces management scenario
 ECA-based interaction model
 semantic enhancement
 ontology models
 incompatibilities detection
 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

4. introduction and motivation
smart
object “any kind of device with sensing or/and pro-
cessing capabilities that is capable of reacting
or adapting its functionalities depending on
external stimulus or users’
requirements, while preserving their
traditional physical interaction paradigm”
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

5. introduction and motivation
smart
object
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

6. introduction and motivation
smart
object
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

7. introduction and motivation
smart
smart
spaces management
object
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

8. introduction and motivation
acquisition
control
smart
smart
spaces management
object
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

9. introduction and motivation
common strategy
to enable capabilities to intelligently
coordinate smart objects
acquisition
control
smart
smart
spaces management
object
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

10. introduction and motivation
common strategy
to enable capabilities to intelligently
coordinate smart objects
acquisition
control
smart
smart
spaces management
object
focus
+ ECA model
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

11. introduction and motivation
smart spaces management scenario
mediator
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

12. introduction and motivation
smart spaces management scenario
mediator
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

13. introduction and motivation
smart spaces management scenario
mediator
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

14. introduction and motivation
smart spaces management scenario
mediator
ECA rules
ECA model  ON event IF condition(s) DO action(s)
event-driven architecture
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

15. contents

 introduction and motivation

 smart spaces management scenario
 ECA-based interaction model
 semantic enhancement
 ontology models
 incompatibilities detection
 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

16. ECA-based interaction model
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing
 proximity detection
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

17. ECA-based interaction model
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing
 proximity detection
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

18. ECA-based interaction model
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing
 proximity detection
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

19. ECA-based interaction model
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing
 proximity detection
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

20. ECA-based interaction model
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing real implementation
 proximity detection
Android-based Google Nexus S (v2.3)
NFC tags (proximity detection)
cloud server (smart object module store)
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

21. contents

 introduction and motivation

 smart spaces management scenario

 ECA-based interaction model
 semantic enhancement
 ontology models
 incompatibilities detection
 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

22. semantic enhancement
ontology models
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing semantic models
 proximity detection
smart object ontology
smart space ontology + ruleBase
ECA rule ontology
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

23. semantic enhancement
ontology models
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

24. semantic enhancement
ontology models
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

25. semantic enhancement
ontology models
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

26. semantic enhancement
ontology models
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

27. contents

 introduction and motivation

 smart spaces management scenario

 ECA-based interaction model
 semantic enhancement

 ontology models
 incompatibilities detection
 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

28. semantic enhancement
incompatibilities detection
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing semantic models
 proximity detection
smart object ontology
smart space ontology + ruleBase
ECA rule ontology
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

29. semantic enhancement
incompatibilities detection
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

30. semantic enhancement
incompatibilities detection
exclusive
resource incompatibility
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

31. semantic enhancement
incompatibilities detection
exclusive
resource incompatibility
configuration
incompatibility
-10º < objectX.hasTemperature < 35º
<condition> = objectX.hasRangeTemperature < 55º 
ON <event>
IF objectX.hasTemperature < 20º AND  
objectX.hasTemperature > 30º 
DO <action>
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

32. semantic enhancement
incompatibilities detection
ON user.locationAt(roomA) 
smart object FixedObject(tvSet)
IF user.uses(carX)
DO <action>
 
intelligentTray.detects(MobileObject)
interaction incompatibility  intelligentTray.detects(tvSet) Car.locatedAt(Outdoor)
Indoor(roomA)
exclusive
resource incompatibility
configuration
incompatibility
-10º < objectX.hasTemperature < 35º
<condition> = objectX.hasRangeTemperature < 55º 
ON <event>
IF objectX.hasTemperature < 20º AND  
objectX.hasTemperature > 30º 
DO <action>
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

33. semantic enhancement
incompatibilities detection
ON user.locationAt(roomA) 
smart object FixedObject(tvSet)
IF user.uses(carX)
DO <action>
 
intelligentTray.detects(MobileObject)
interaction incompatibility  intelligentTray.detects(tvSet) Car.locatedAt(Outdoor)
Indoor(roomA)
exclusive
resource incompatibility
configuration
incompatibility
-10º < objectX.hasTemperature < 35º
<condition> = objectX.hasRangeTemperature < 55º  functional incoherence
ON <event> ON event1 IF condition1 DO turnRadioON 
IF objectX.hasTemperature < 20º AND
objectX.hasTemperature > 30º

  ON event1 IF condition1 DO turnRadioOFF  
DO <action>
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

34. contents

 introduction and motivation

 smart spaces management scenario

 ECA-based interaction model

 semantic enhancement

 ontology models

 incompatibilities detection
 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

35. semantic enhancement
reasoning support
event
condition(s)
action(s)
b) ECA rules configuration
a) smart object c) ECA rules evaluation
manager installation (and action execution)
 module publishing semantic models
 proximity detection
smart object ontology
smart space ontology + ruleBase
ECA rule ontology
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

36. semantic enhancement
reasoning support
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

37. semantic enhancement
reasoning support
ECA-rules context-based
dynamic (de)activation
 smart objects status
 device status
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

38. semantic enhancement
reasoning support
ECA-rules context-based
dynamic (de)activation
 smart objects status
 device status
ECA-rules conditions
execution order prioritization
 offline condition(s)
 online condition(s)
ON (<offlineEvent1>=TRUE
OR <onlineEvent1>)
IF (<offlineConditionA>=FALSE
AND <onlineConditionA>)
DO <action>
resource usage
(battery, CPU load, free memory, etc.)
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

39. contents

 introduction and motivation

 smart spaces management scenario

 ECA-based interaction model

 semantic enhancement

 ontology models

 incompatibilities detection

 reasoning support
 conclusions and future works
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

40. conclusions and future works
• smartphones  smart spaces mediator
• semantic formalism enhancement
• incompatibilities / incoherencies detection
• execution optimization
 androJena (Android-based devices)
 ontology models management
 rule execution
 validation (response times)
 semantic vs ad-hoc solutions
 multiuser scenario
 incompatible / incoherent rules
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es

41. any question?
Workshop on User-Centric Technologies and Applications – CONTEXTS 2012 josue@grpss.ssr.upm.es