Training session at the universAAL project meeting in Darmstadt, March 1-4, 2010

1. uAALtraining Main conceptsof SOPRANO middleware / openAAL Peter Wolf, FZI 16.03.2010 1

2. IP, FP 6 http://www.soprano-ip.org 25 partners from 7 EU-countries Start: 01/07 end 10/10 Project-lead: Exodus (Greece) uAAL-contacts: ProSyst, FZI (, CAS) Strong interest in Defining/applying methodology for evaluation Market-analysis Middleware platform 16.03.2010 2 SOPRANO

3. Support AP in the house ambiently with the help of IT technology Positive mindset: enhance quality of life But also: help in critical situations SOPRANO Ambient Middleware (SAM) openAAL is open-source software based on the SAM architecture and implementation 3 (Technical) Goal of SOPRANO

4. Problem: Stove/oven have been left on unintentionally while person was leaving the home Solution: Reminder service that helps person to remember turned on devices, open windows etc. High-level UC description: Identify situation that AP is about to leave the home Check for open windows, critical devices that are turned on (stove, oven, kettle etc.) Warn AP and wait for confirmation that message has been read If not warn informal carer (e.g. son) 4 Exampleand Motivation:thesopranoremembering UC

5. Problem: typical problem of AP Solution: ambient services that incorporate existing technologies and services and are based on predefined rules High-level UC description: Automatically derive situations from sensor information Execute predefined workflow as reaction Execute services, actuators as needed 5 Exampleand Motivation: Generalisation

6. 6 Architecture

7. 7 SAM in Action: CM State: Window open Procedural Manager Kettle is on Window is open ... Composer Context Manager OSGi Service Middleware LittleWorld: Sensors: kett-sensor connected-to kettle oven-sensor connected-to oven door-sensor connected-to frontdoor ... Actuators: message-actuator connected-to touchscreen ... Actuators Kettlesensor switched status Windowsensor switched status ... Sensors 7

8. 8 SAM in Action: CM Event: AP about to leave home State: Door open Procedural Manager AP at door Kettle is on Window is open No bell Composer Context Manager OSGi Service Middleware Actuators Frontdoor sensor switched status RFID-sensor detects chip ... Sensors 8

9. 9 Context Manager can be extended bysoftware developers Provide a consistent view on context at a specific point in time Conflict Resolution Translate low-level context into high-level context Context Storage Store all context facts with timestamp and uncertainty ContextUplifter provide state information according to low-level context ontology Sensor Service Sensor Service Sensor Service

10. State-based modeling No reasoning semantic instead more db-like semantics (abstract-query and constraints) Low-level ontology describes sensors, actuators and their states Simplifies integration of sensors, actuators High-level ontology Describes context of assisted person Used in communication between platform components Uplifter define reasoning and ensure semantic coherence 16.03.2010 10 Ontology

11. Two conceptual models for easy sensor and service integration Service-level ontology 11 High-level Ontology

12. Two conceptual models for easy sensor and service integration Sensor-level ontology 12 Sensor-level Ontology

13. RDF-based data model Triples describing states of an entity Entity, state-property, state “lamp, has-power-status, on” Tuple representation to capture domain-specific meta-information History, for complex reasoning considering past states Confidence, for statistical reasoning approaches Validity, Source, ... 13 Data-model

15. Collaboration of Uplifters - Example triggers workflow Decision Tree AP a leaving AP outside Written tocontext storage AP inside Doorbell db pressed Rule-based Rule-based Written tocontextstorage AP a near door d Written tocontextstorage Door d Open RFID chip c Detected by reader r Doorsensors Fires AP a wears chip c s attached to Door d Reader r is lo-cated near door d 15

16. Model andcontextreasoningseperatedtoallowformoreflexibility Formalism neutral reasoning Flexibilityconsideringlow-levelchanges Constraints on propertycardinality neededforconflict-resolution Conformityofinterfaces Abstractionofclassesandpropertiesforabstractqueries 16.03.2010 16 openAALOntology

17. Stores all information facts Provides simple query interface Manages AP-context and low-level ontology Implements blackboard architecture Derives AP-context information from sensor information via aggregation services (uplifter) Decouples situations from sensors to detect them Decouples definition of situations from detection algorithms Provides for heuristic-based conflict resolution at query-time 16.03.2010 17 CM Overview

18. 18 openAAL Architecture

20. Wait for confirmation of AP

21. If no confirmation, send SMS to informal carerProcedural Manager AP at door Kettle is on Window is open No bell Composer Context Manager OSGi Service Middleware Actuators Kettlesensor switched status Windowsensor switched status ... Frontdoor sensor switched status Sensors 19

22. 16.03.2010 20 PM Architecture Admin Tools Store/retrieve/trigger workflow xml-document Procedural Manager BPEL Parser Database Store wf object struct. Get wf object struct. Execute trigger element BPEL Engine Request service execution Sync./async. communication Context Manager Composer

24. If

25. Loop

26. Assign

27. …

28. Trigger event starting the wf

29. Query context manager

30. Store context statement

31. Wait for statement (or timeout)

33. 16.03.2010 23 Workflow 2 Context Manager Procedural Manager Composer But workflows can also operate on sensor-information (bed-sensor) Request services directly (actuator for concrete light)

34. Stores BPEL workflows that define (abstract) reactions to situations Executes workflow upon situation detection Communicates with CM for context-awareness Requests service execution from composer to change AP-context and/or low-level context 16.03.2010 24 PM Overview

35. 25 openAAL Architecture

37. Wait for confirmation of AP

38. If no confirmation, send SMS to informal carerProcedural Manager AP at door Kettle is on Window is open No bell Composer Context Manager OSGi Service Middleware Actuators Kettlesensor switched status Windowsensor switched status ... Frontdoor sensor switched status Sensors Loudspeaker/touch screen “Kettle is still on.” “Window is still open.” 26

39. 27 Inner Architecture of Composer Procedural Manager Abstract service goal Concrete service goal Service Matchmaker + Invocator Parameterised invocation Virtual Services Store context Context Manager Concrete service goals Query context Query Invoke Actuator Service OSGi Repository Service

40. 28 Solution: Virtual Services Inform AP about medication Inform AP If not hearing-impaired Inform AP If not vision-impaired Play sound message on loudspeaker in this or connected room of AP Show message on GUI on switched-on device in room of AP Play sound message on loudspeaker-1 Send SMS to Number Play avatar message on tv-1 Show text message via GUI on tv-1 Show text message via GUI on touchscreen-1

42. Configurable: set depends on input parameters

43. Complex: states are arbitrary complex graphs formable with the ontology

44. Inform Person<in,1> about Medication<in,2>, where Person<in,1> is watching tv-1

46. Configurable: set depends on input parameters

47. Fuzzy: User can specify evaluation value for elements of set

48. Complex: states are arbitrary complex graphs formable with the ontology

50. 32 openAAL Architecture

51. Complex in terms of configuration LittleWorlds, Uplifter, Procedures, Virtual Services, DSD-statements, service integration No clearly defined best practices of how and where to configure/model certain things Very limited tool support that is tailored to openAALrequ. or AAL-domain Some performance issues Partly due to research prototype nature and Use of open-source software (database + OSGi) Still ongoing work: No output-variables in DSD-requests in workflows Aging of context facts was taken out in SOPRANO Not all BPEL-constructs are thoroughly tested (yet) Query capability on CM-data needs to be extended 16.03.2010 33 Main Drawbacks

52. Modularity Components can be used stand-alone or with complementing dummy implementations Highly configurable (partly) based on (quasi-)standards (OWL/RDFS-Syntax, BPEL, DSD) Still continuing development in different projects (FZI living-lab AAL, SOPRANO, MOPS, SpeedUp) Open-source via LGPL-license 2 running demo installations are available 16.03.2010 34 General Features

53. Thankyouforyourattention Questions? 16.03.2010 35