The document describes an attitude-based resource recommendation strategy for an augmented reality Internet of Things (IoT) browser. The strategy prioritizes resources for users based on their profile, interaction history, spatial location and orientation relative to resources, and explicit selections. An architecture is proposed with three planes: an infrastructure plane containing location services and knowledge bases, a device plane for sensing and representation, and a smart space plane for mapping and resource modules. A prototype applies the strategy to provide context-aware resource filtering and recommendations.

1. Grupo de Procesado de Datos y Simulación
ETSI de Telecomunicación
Universidad Politécnica de Madrid
An attitude-based reasoning strategy
to enhance interaction with augmented objects
esIoT 2012
International workshop on Extending Seamlessly to the Internet of Things
Josué Iglesias, David Gómez, Ana M. Bernardos, José R. Casar
josue@grpss.ssr.upm.es

2. contents
 introduction and motivation
 IoT browser functionalities
 architecture design
 attitude-based resource
recommendation strategy
 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 2 / 16

3. contents
 introduction and motivation
 IoT browser functionalities
 architecture design
 attitude-based resource
recommendation strategy
 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 3 / 16

4. introduction and motivation
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

5. introduction and motivation
archie gopher veronica etc.
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

6. introduction and motivation
web
web 2.0
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

7. introduction and motivation
semantic
web
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

8. introduction and motivation
traditional
Internet
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

9. introduction and motivation
traditional Internet
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

10. introduction and motivation
traditional Internet
+
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

11. introduction and motivation
traditional Internet
+ new environment (reality)
identification
traditional Internet
discovery
information management
interactivity
etc.
+
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

12. introduction and motivation
traditional Internet
+ new environment (reality) Internet of Things
identification location
traditional Internet
discovery proximity
information management mobility
interactivity etc.
etc.
+
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

13. introduction and motivation
3D-AR
Augmented
Reality
traditional Internet
+
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

14. introduction and motivation
traditional Internet
+ new environment (reality) Internet of Things
identification location
traditional Internet
discovery proximity
information management mobility
interactivity etc.
etc.
+ ...
densely populated environments
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

15. introduction and motivation
traditional Internet
+ new environment (reality) Internet of Things
identification location
traditional Internet
discovery proximity
information management mobility
interactivity etc.
etc.
user experience
+ ...
o interaction densely populated environments
o filtering / selection
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

16. introduction and motivation
traditional Internet
+ new environment (reality) Internet of Things
identification location
traditional Internet
discovery proximity
information management mobility
interactivity etc.
etc.
user experience
+ ...
o interaction densely populated environments
o filtering / selection
focus
 development of IoT browser based on
augmented reality (AR)
 context-aware representation of resources
Internet of Things (IoT)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 4 / 16

17. contents

 introduction and motivation
 IoT browser functionalities
 architecture design
 attitude-based resource
recommendation strategy
 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 5 / 16

18. IoT browser functionalities
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 6 / 16

19. IoT browser functionalities
resource
recognition
user position
(mobile)
resource position
(fixed  mobile)
location estimation
• WiFi RSS
( 3 m.)
• beacon-based
(simulated)
- e.g., RFID
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 6 / 16

20. IoT browser functionalities
resource
interfaces
recognition
user position 2D
(mobile) WiFi location
resource position
(fixed  mobile)
location estimation 3D - AR
• WiFi RSS beacon-based loc.
( 3 m.)
• beacon-based
(simulated)
- e.g., RFID
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 6 / 16

21. IoT browser functionalities
resource map
interfaces
recognition matching
user position 2D zones
(mobile) WiFi location rectangular
resource position disjoint
(fixed  mobile) real
(meeting rooms, bedrooms, etc.)
location estimation 3D - AR
• WiFi RSS beacon-based loc. user’s
( 3 m.)
• beacon-based line-of-sight
(simulated) walls
- e.g., RFID (other obstacles)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 6 / 16

22. IoT browser functionalities
user-
resource map
interfaces resource
recognition matching
interaction
user position 2D zones explicit
(mobile) WiFi location rectangular icons / virtual obj.
resource position disjoint touch-based
(fixed  mobile) real explicit info.
(meeting rooms, bedrooms, etc.)
location estimation 3D - AR
• WiFi RSS beacon-based loc. implicit
user’s relative attitude
( 3 m.)
• beacon-based line-of-sight • distance
(simulated) walls • orientation
- e.g., RFID (other obstacles)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 6 / 16

23. IoT browser functionalities
user-
resource map resource
interfaces resource
recognition matching filtering
interaction
user position 2D zones explicit context-aware
(mobile) WiFi location rectangular icons / virtual obj.
resource position disjoint touch-based distance
(fixed  mobile) real explicit info. “focus” (orientation)
(meeting rooms, bedrooms, etc.) line-of-sight
location estimation 3D - AR history
• WiFi RSS beacon-based loc. implicit profile
user’s relative attitude
( 3 m.) preferences
• beacon-based line-of-sight • distance group
(simulated) walls • orientation etc.
- e.g., RFID (other obstacles)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 6 / 16

24. contents

 introduction and motivation

 IoT browser functionalities
 architecture design
 attitude-based resource
recommendation strategy
 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 7 / 16

25. architecture design
infrastructure plane device plane smart space
plane
location
... Spatial
Location orientation Module
Services
Interface
Spatial Selector Module
Reasoner
Server Representation
2D
Resource
Module
ResourceKB
MapsKB AR
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 8 / 16

26. architecture design
infrastructure plane device plane smart space
plane
location
... Spatial
Location orientation Module
Services
Interface
Spatial Selector Module
Reasoner
Server Representation
2D
Resource
Module
ResourceKB
MapsKB AR
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 8 / 16

27. architecture design
infrastructure plane device plane smart space
plane
location
... Spatial
Location orientation Module
Services
Interface
Spatial Selector Module
Reasoner
Server Representation
2D
Resource
Module
ResourceKB
MapsKB AR
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 8 / 16

28. architecture design
infrastructure plane device plane smart space
plane
location
... Spatial
Location orientation Module
Services
Interface
Spatial Selector Module
Reasoner
Server Representation
2D
Resource
Module
ResourceKB
MapsKB AR
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 8 / 16

29. architecture design
infrastructure plane device plane smart space
plane
location
... Spatial
Location orientation Module
Services
Interface
Spatial Selector Module
Reasoner
Server Representation
2D
Resource
Module
ResourceKB
MapsKB AR
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 8 / 16

30. architecture design
infrastructure plane device plane smart space
plane
location
... Spatial
Location orientation Module
Services
Interface
Spatial Selector Module
Reasoner
Server Representation
2D
Resource
Module
ResourceKB
MapsKB AR
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 8 / 16

31. contents

 introduction and motivation

 IoT browser functionalities

 architecture design
 attitude-based resource
recommendation strategy
 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 9 / 16

32. resource recommendation
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 10 / 16

33. resource recommendation
resource prioritization based on user’s habits
user’s history probabilistic model (dynamic)
← location
↑ time of the day
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 10 / 16

34. resource recommendation
resource prioritization based on user’s habits
user’s history probabilistic model (dynamic)
← location
↑ time of the day
resources desired spatio-temporal visibility
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 10 / 16

35. resource recommendation
resource prioritization based on user’s habits flexible resources visibility (examples)
user’s history probabilistic model (dynamic) just by user location
e.g. resource ‘a’ visibility would be
← location focused on a specific space
(s1), independently of the time of the
day
just temporal-based
e.g. resource ‘b’ visibility would be
↑ time of the day focused on a specific period of time
(t1), and not in any particular space
resources desired spatio-temporal visibility
spatio-temporal combination
e.g. resource ‘c’ visibility would be
focused on a specific space (s1) during a
particular period of time (t1)
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 10 / 16

36. resource recommendation
resource prioritization based on user’s habits flexible resources visibility (examples)
user’s history probabilistic model (dynamic) just by user location
e.g. resource ‘a’ visibility would be
← location focused on a specific space
(s1), independently of the time of the
day
just temporal-based
e.g. resource ‘b’ visibility would be
↑ time of the day focused on a specific period of time
(t1), and not in any particular space
resources desired spatio-temporal visibility
spatio-temporal combination
e.g. resource ‘c’ visibility would be
focused on a specific space (s1) during a
particular period of time (t1)
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 10 / 16

37. resource recommendation
spatial-based prioritization
user (device) – resource distance
user location
resources location
user (device) – resource relative orientation
+ accelerometer
+ magnetometer
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 11 / 16

38. resource recommendation
spatial-based prioritization
user (device) – resource distance
user location
resources location
user (device) – resource relative orientation
+ accelerometer
+ magnetometer
distance approach orientation approach
+ ·············· − + ················· −
(near) ··········· (far) (0 ) ················ (180 )
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 11 / 16

39. resource recommendation
spatial-based prioritization
2D
(WiFi location) user (device) – resource distance
user location
resources location
zd user (device) – resource relative orientation
yd + accelerometer
+ magnetometer
xd
distance approach orientation approach
+ ·············· − + ················· −
(near) ··········· (far) (0 ) ················ (180 )
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 11 / 16

40. resource recommendation
spatial-based prioritization
2D 3D - AR
(WiFi location) (beacon-based loc.) user (device) – resource distance
user location
resources location
yd
zd user (device) – resource relative orientation
yd + accelerometer
zd + magnetometer
xd
xd
distance approach orientation approach
+ ·············· − + ················· −
(near) ··········· (far) (0 ) ················ (180 )
profile/preference- historic- spatial- explicit-based
based prioritization prioritization
based based
3rd parties prioritization
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 11 / 16

41. contents

 introduction and motivation

 IoT browser functionalities

 architecture design

 attitude-based resource
recommendation strategy
 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 12 / 16

42. prototype description
Motorola Xoom
Honeycomb Android v3.1
(Java server + mySQL DB)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

43. prototype description
2D Motorola Xoom
Honeycomb Android v3.1
(Java server + mySQL DB)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

44. prototype description
2D Motorola Xoom
WiFi-based location ON/OFF
beacon-based location (simulated)
Honeycomb Android v3.1
(Java server + mySQL DB)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

45. prototype description
2D Motorola Xoom
WiFi-based location ON/OFF
beacon-based location (simulated)
Honeycomb Android v3.1
(Java server + mySQL DB)
POIs
information
distance
POIs location
360 view
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

46. prototype description
2D Motorola Xoom
WiFi-based location ON/OFF
beacon-based location (simulated)
Honeycomb Android v3.1
(Java server + mySQL DB)
2D ↔ 3D-AR
POIs
information
distance
POIs location
360 view
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

47. prototype description
3D-AR Motorola Xoom
Honeycomb Android v3.1
(Java server + mySQL DB)
icons
2D/3D
form
size
colored/grey
solid/transparent
pictures
labels
POI name
distance
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

48. prototype description
3D-AR Motorola Xoom
Honeycomb Android v3.1
(Java server + mySQL DB)
icons
2D/3D
form
size
colored/grey
solid/transparent
pictures
labels
POI name
distance
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 13 / 16

49. contents

 introduction and motivation

 IoT browser functionalities

 architecture design

 attitude-based resource
recommendation strategy

 prototype description
 discussion and future works
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 14 / 16

50. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

51. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

52. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
 beacon-based localization
 RFID
 pressure mats
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

53. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
 beacon-based localization
 RFID
 pressure mats
 priority approaches
 develop static and explicit prioritizations
 integration
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

54. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
 beacon-based localization
 RFID
 pressure mats
 priority approaches
 develop static and explicit prioritizations
 integration
 line-of-sight management (obstacles)
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

55. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
 beacon-based localization
 RFID
 pressure mats
 priority approaches
 develop static and explicit prioritizations
 integration
 line-of-sight management (obstacles)
 context-aware icons/labels
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

56. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
 beacon-based localization
 RFID
 pressure mats
 priority approaches
 develop static and explicit prioritizations
 integration
 line-of-sight management (obstacles)
 context-aware icons/labels
 infrastructure  mobile device
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

57. discussion and future works
• base framework
– IoT-based browsers funcionalities prototypes
– context-aware representation of resource information
• new paradigm in resource recommendation
– real-world (spatial) magnitudes
(distance, orientation, line-of-sight, etc.)
 beacon-based localization
 RFID
 pressure mats
 priority approaches
 develop static and explicit prioritizations
 integration
 line-of-sight management (obstacles)
 context-aware icons/labels
 infrastructure  mobile device
 integration
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 15 / 16

58. any question?
Extending Seamlessly to the Internet of Things – esIoT 2012 josue@grpss.ssr.upm.es 16 / 16