1. Smart Objects and Semantic Web:
A semantic approach to intelligent environments
© 2012 Jun Hu

2. Department of Industrial Design 26-9-2012 PAGE 1

3. 26-9-2012 PAGE 2

4. 26-9-2012 PAGE 3

5. 26-9-2012 PAGE 4

6. 26-9-2012 PAGE 5

7. 26-9-2012 PAGE 6

8. Semantic Web
• Access to universal knowledge
Department of Industrial Design 26-9-2012 PAGE 7

9. Semantic Web
• Siri
Department of Industrial Design 26-9-2012 PAGE 8

10. Semantic Web
• Siri
Department of Industrial Design 26-9-2012 PAGE 9

11. Semantic Web
• Siri
Department of Industrial Design 26-9-2012 PAGE 10

12. Semantic Web
• Ontology: Access to universal knowledge
Department of Industrial Design 26-9-2012 PAGE 11

13. Semantic Web
• Ontology: Categories of being
Department of Industrial Design 26-9-2012 PAGE 12

14. Semantic Web
• Ontology: in Resource Description Framework
− A collection of subject-predicate-object triples:
• A is B, B is C
• C has D
• …
• A loves B
• B hates A
• (S P O)
Department of Industrial Design 26-9-2012 PAGE 13

15. Semantic Web
• Ontology: in Resource Description Framework
− Artificial Intelligence
• A is B, B is C => A is C
• C has D => A has D
• …
• A loves B
• B hates A => A (can not marry) B
• (S P O)
Department of Industrial Design 26-9-2012 PAGE 14

16. Semantic Web
• Ontology on web => Semantic Web
• (S P O) => (#s #p #o)
Department of Industrial Design 26-9-2012 PAGE 15

17. Semantic Web
• Ontology on web => Semantic Web
• (S P O) => (#s #p #o)
Department of Industrial Design 26-9-2012 PAGE 16

18. Semantic Web
• Ontology on web => Semantic Web
• (S P O) => (#s #p #o)
Department of Industrial Design 26-9-2012 PAGE 17

19. Semantic Web
• Siri
Department of Industrial Design 26-9-2012 PAGE 18

20. Semantic Web
• Ontology on web => Semantic Web
• (S P O) => (#s #p #o)
Department of Industrial Design 26-9-2012 PAGE 19

21. Semantic Web
• Access to universal knowledge
Department of Industrial Design 26-9-2012 PAGE 20

22. Semantic Web
• Access to universal knowledge
− through
− Internet of Knowledge
Department of Industrial Design 26-9-2012 PAGE 21

23. Semantic Web
• Access to universal knowledge
− through
− Internet of Knowledge
− by
− Internet of People
Department of Industrial Design 26-9-2012 PAGE 22

24. Social Networks
• Internet of People
Department of Industrial Design 26-9-2012 PAGE 23

25. Social Networks
• Internet of People
A social network is a social structure made up of
individuals (or organizations) called "nodes", which
are tied (connected) by one or more specific types
of interdependency, such as friendship, kinship,
common interest, financial exchange, dislike,
sexual relationships, or relationships of beliefs,
knowledge or prestige.
http://en.wikipedia.org/wiki/Social_network
Department of Industrial Design 26-9-2012 PAGE 24

26. Social Networks
• Internet of People
Department of Industrial Design 26-9-2012 PAGE 25

27. Social Networks
• Internet of People
• Facebook
• Twitter
• LinkedIn
• Foursquare
• MySpace
• Weibo
• Renren
• …
Department of Industrial Design 26-9-2012 PAGE 26

28. Social Networks
• Internet of different People
• Facebook
• Twitter
• LinkedIn
• Foursquare
• MySpace
• Weibo
• Renren
• …
Department of Industrial Design 26-9-2012 PAGE 27

29. Social Networks
• Internet of People
• Why?
Department of Industrial Design 26-9-2012 PAGE 28

30. Social Network
• Internet of People
• Why?
Department of Industrial Design 26-9-2012 PAGE 29

31. Social Networks
• Internet of People
• Why?
Department of Industrial Design
Maslow's Hierarchy of _Needs 26-9-2012 PAGE 30

32. Social Networks
• Internet of People
• Why?
Department of Industrial Design
Maslow's Hierarchy of _Needs 26-9-2012 PAGE 31

33. Social Networks
• Internet of People
• Internet of People  Internet of Things
© http://www.economist.com/node/15350960
Department of Industrial Design 26-9-2012 PAGE 32

34. Social Network
• Network of People => Internet of People
• Internet of People  Internet of Things
© http://www.economist.com/node/15350960
Department of Industrial Design 26-9-2012 PAGE 33

35. Internet of Things
What is it?
The Internet of Things refers to uniquely
identifiable objects (things) and their virtual
representations in an Internet-like structure.
© Wikipedia
© http://www.economist.com/node/15350960
Department of Industrial Design 26-9-2012 PAGE 34

36. Internet of Things
• Identifiable
Department of Industrial Design 26-9-2012 PAGE 35

37. Internet of Things
• Sensing
Department of Industrial Design 26-9-2012 PAGE 36

38. Internet of Things
• Actuating
Department of Industrial Design 26-9-2012 PAGE 37

39. Internet of Things
• Virtualization and connectivity
Department of Industrial Design 26-9-2012 PAGE 38

40. Internet of Things
• Local Intelligence
• Google I/O
Department of Industrial Design 26-9-2012 PAGE 39

41. • Web 3.0, the future of the internet
• http://youtu.be/FzIwXqqDnOY
Department of Industrial Design 26-9-2012 PAGE 40

42. • Interoperablibility in the interconnected world
• http://youtu.be/FzIwXqqDnOY
Department of Industrial Design 26-9-2012 PAGE 41

43. SOFIA
• Smart Objects for Intelligent Environments
• funded through the European Artemis programme
Department of Industrial Design 26-9-2012 PAGE 42

44. SOFIA
• Smart Objects for Intelligent Environments
• funded through the European Artemis programme
• Context:
• Smart Environment – ecosystem of interconnected and
interoperating smart objects
• Newly developed and legacy devices
• Connections and relations between smart objects
• Shifting from interacting with single products to interaction
with a system of products
Department of Industrial Design 26-9-2012 PAGE 43

45. Internet of Things and semantic Web
Making "information" in the physical world available Sematic Information
for smart services - connecting the physical world Broker
with the information world
Ontology
Blackboard
Ontology and semantics of the intents, interests and concepts
Service data gathering and mining
Knowledge
Processor
Knowledge
Processor … Knowledge
Processor
Smart Smart … Smart
Object Object Object
Internet of things Web of people
Department of Industrial Design 26-9-2012 PAGE 44

46. SOFIA
• Demo: Semantic connections - Spotlight navigation
• http://youtu.be/bOrIPp-aglg
Department of Industrial Design 26-9-2012 PAGE 45

47. SOFIA
• Pilot: Smart space
• http://youtu.be/NjSwTU9tX4w
Department of Industrial Design 26-9-2012 PAGE 46

48. SOFIA
• Based on Ontologies (Semantic Web)
• And the Blackboard Pattern
Department of Industrial Design 26-9-2012 PAGE 47

49. SOFIA
• Pilot scenario
Department of Industrial Design 26-9-2012 PAGE 48

50. SOFIA
• Ontology: RDF
Department of Industrial Design 26-9-2012 PAGE 49

51. SOFIA
• Ontology: OWL
Department of Industrial Design 26-9-2012 PAGE 50

52. SOFIA
• Ontology: devices
Department of Industrial Design 26-9-2012 PAGE 51

53. SOFIA
• Ontology: interaction events
Department of Industrial Design 26-9-2012 PAGE 52

54. SOFIA
• SIB
Department of Industrial Design 26-9-2012 PAGE 53

55. SOFIA
• SIB Ontology
• Create/Add/Remove/Update
Department of Industrial Design 26-9-2012 PAGE 54

56. SOFIA
• SIB Ontology
• Create/Add/Remove/Update
• Query (SPARQL)
Department of Industrial Design 26-9-2012 PAGE 55

57. SOFIA
• SIB Ontology
• Create/Add/Remove/Update
• Query (SPARQL)
• Reasoning (SPIN – SPARQL Inferencing Notation)
Department of Industrial Design 26-9-2012 PAGE 56

58. SOFIA
• KP
Department of Industrial Design 26-9-2012 PAGE 57

59. SOFIA
• Based on Ontologies (Semantic Web)
• And the Blackboard Pattern
Department of Industrial Design 26-9-2012 PAGE 58

60. Student projects
Department of Industrial Design 26-9-2012 PAGE 59

61. Doormate

62. Doormate

63. Doormate

64. Doormate

65. Bonsai Garden

66. Bonsai Garden
• Gaming elements

67. Bonsai Garden

68. Resources
• https://github.com/iddi/sofia
• What you would need:
• Eclipse for RCP and RAP Developers (Helios)
• EGit
• Android ADK (if Android is used)
• SOFIA ADK
• TUeSIB
• Python (if python is preferred)
• Smart-M3 Python KPI
• SSLS
• …
• More at: https://github.com/iddi/sofia/wiki/Installation
Department of Industrial Design 26-9-2012 PAGE 67

69. Resources
• SOFIA is based on SMART-M3
• M3: multi-vendor, multi-device and multi-part
• About SMART-M3:
• http://en.wikipedia.org/wiki/Smart-M3
• SMART-M3 on source forge:
• http://sourceforge.net/projects/smart-m3/
Department of Industrial Design 26-9-2012 PAGE 68

70. Resources
• SOFIA is contunued:
• https://sofia-community.com/
Department of Industrial Design 26-9-2012 PAGE 69

71. References
1. G. Niezen, ONTOLOGIES FOR INTERACTION: Enabling serendipitous interoperability in smart environments. PhD Thesis, Eindhoven University of Technology, 2012.
2. B. J. J. van der Vlist, Designing Semantic Connections Explorations, Theory and a Framework for Design. PhD Thesis, Eindhoven University of Technology, 2012.
3. B. J. J. van der Vlist, G. Niezen, S. Rapp, J. Hu, and L. M. G. Feijs, "Controlling Smart Home Environments with Semantic Connections: a Tangible and an AR Approach," presented at the 7th International
Workshop on the Design & Semantics of Form & Movement (DeSForM), Wellington, New Zealand, 2012.
4. B. J. J. van der Vlist, J. Hu, G. Niezen, and L. M. G. Feijs, "Semantic Connections: a New Interaction Paradigm for Smart Environments," presented at the 7th International Workshop on the Design & Semantics
of Form & Movement (DeSForM), Wellington, New Zealand, 2012.
5. J. P. A. Peeters, B. J. J. van der Vlist, G. Niezen, J. Hu, and L. M. G. Feijs, "A Study on a Tangible Interaction Approach to Managing Wireless Connections in a Smart Home Environment," presented at the 7th
International Workshop on the Design & Semantics of Form & Movement (DeSForM), Wellington, New Zealand., 2012.
6. J. Hu, B. van der Vlist, G. Niezen, W. Willemsen, D. Willems, and L. Feijs, "Designing the Internet of Things for Learning Environmentally Responsible Behaviour," Interactive Learning Environments, vol.
Accepted., 2012.
7. W. Willemsen, J. Hu, G. Niezen, and B. van der vlist, "Using Game Elements to Motivate Environmentally Responsible Behaviour," presented at the IADIS Game and Entertainment Technologies 2011 (GET
2011), Rome, Italy, 2011.
8. B. van der vlist, G. Niezen, J. Hu, and L. Feijs, "Interaction primitives: Describing interaction capabilities of Smart Objects in ubiquitous computing environments," presented at the AFRICON, 2011, Livingstone
2011.
9. B. van der vlist, G. Niezen, J. Hu, and L. Feijs, "Desiqn semantics of connections in a smart home environment," Creation and Design, vol. 13, pp. 18-24, 2011.
10. G. Niezen, B. van der vlist, J. Hu, and L. M. G. Feijs, "Using semantic transformers to enable interoperability between media devices in a ubiquitous computing environment," presented at the International
Workshop on Self-managing Solutions for Smart Environments (S3E 2011) at the 6th International Conference on Grid and Pervasive Computing (GPC 2011), 2011.
11. M. Kwak, G. Niezen, B. van der vlist, J. Hu, and L. Feijs, "Tangible Interfaces to Digital Connections, Centralized versus Decentralized," Transactions on Edutainment, vol. V, LNCS 6530, pp. 132-146, 2011.
12. J. Hu, B. van der vlist, G. Niezen, W. Willemsen, D. Willems, and L. M. G. Feijs, "Learning from Internet of Things for Improving Environmentally Responsible Behavior," in Edutainment Technologies.
Educational Games and Virtual Reality/Augmented Reality Applications, M. Chang, W.-Y. Hwang, M.-P. Chen, and W. Müller, Eds., ed: Springer Berlin / Heidelberg, 2011, pp. 292-299.
13. B. van der Vlist, G. Niezen, J. Hu, and L. Feijs, "Design Semantics of Connections in a Smart Home Environment," presented at the Design and Semantics of Form and Movement (DeSForM 2010), Lucerne,
Switzerland., 2010.
14. B. van der Vlist, G. Niezen, J. Hu, and L. Feijs, "Semantic Connections: Exploring and Manipulating Connections in Smart Spaces," presented at the 2010 IEEE Symposium on Computers and Communications
(ISCC), Riccione, Italy, 2010.
15. G. Niezen, B. van der vlist, J. Hu, and L. Feijs, "From Events to Goals: Supporting Semantic Interaction in Smart Environments," presented at the 2010 IEEE Symposium on Computers and Communications
(ISCC), Riccione, Italy, 2010.
Department of Industrial Design 26-9-2012 PAGE 70

72. Smart Objects and Semantic Web:
A semantic approach to intelligent environments
© 2012 Jun Hu