These are the slides of my PhD. thesis dissertation called "A Framework for Abstraction and Virtualization of Sensors in Mobile Context Aware Computing". It was presented on June 29 2015.
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Thesis Presentation
1. A Framework for
Abstraction and Virtualization of Sensors
in Mobile Context-Aware Computing
Borja Gamecho
Supervised by Julio Abascal and Luis Gardeazabal
University of the Basque Country UPV/EHU
June 29, 2015
Laboratory of Human-Computer Interaction
for Special Needs
5. Introduction Conceptual Framework Implementation Evaluation Conclusions
Devices for Ubiquitous Computing
Growing ecosystem of advanced devices for Ubiquitous
Computing with similar characteristics:
• Embedded sensors
• Wireless network
• Open SDKs & APIs
• Affordable cost
Sensors are a valuable asset to obtain data from the real world
Borja Gamecho UPV/EHU 3 / 56
6. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations of Egoki
Limitations of Egoki (with regard to the use of sensors):
• Support for input modalities further than point-and-click and
touch-screen
• Lacks of adaptability support for the generated user
interfaces
• Support for proactive applications
User interfaces generated with Egoki can be enhanced with the
use of sensors
Borja Gamecho UPV/EHU 4 / 56
7. Introduction Conceptual Framework Implementation Evaluation Conclusions
Perception of Context
Author(s) Physical Sensors Context Information
Schmidt el al. 1999 Temperature, Pressure,
CO Gas Meter, Photo-
diode, Accelerometers,
PIR and Microphone
Mobile phone, User Activity
Haag et al. 2004 EMG, Electro Dermal
Activity sensor (EDA),
Skin Temperature,
Blood Volume Pulse,
ECG and Respiration
User Emotional State
Parkka et al. 2006 Air Pressure, Micro-
phone, Accelerometer,
Humidity, Luminosity,
... (up to 22 signals)
User Activity
Chon and Cha 2011 GPS, Accelerometers,
Compass, BT, WiFi and
GSM
Smartphone, User Activity
Wiese et al. 2013 Accelerometer,
Light/Proximity,
Capacitive and Multi-
spectral
Smartphone, User Activity
Jang et al. 2013 Skin Temperature,
ECG, EDA and Pho-
toplethysmography
(PPG)
User emotional state
Reddy et al. 2010 Accelerometer, GPS Transportation Modes
Borja Gamecho UPV/EHU 5 / 56
8. Introduction Conceptual Framework Implementation Evaluation Conclusions
Perception of Context
Author(s) Physical Sensors Context Information
Schmidt el al. 1999 Temperature, Pressure,
CO Gas Meter, Photo-
diode, Accelerometers,
PIR and Microphone
Mobile phone, User Activity
Haag et al. 2004 EMG, Electro Dermal
Activity sensor (EDA),
Skin Temperature,
Blood Volume Pulse,
ECG and Respiration
User Emotional State
Parkka et al. 2006 Air Pressure, Micro-
phone, Accelerometer,
Humidity, Luminosity,
... (up to 22 signals)
User Activity
Chon and Cha 2011 GPS, Accelerometers,
Compass, BT, WiFi and
GSM
Smartphone, User Activity
Wiese et al. 2013 Accelerometer,
Light/Proximity,
Capacitive and Multi-
spectral
Smartphone, User Activity
Jang et al. 2013 Skin Temperature,
ECG, EDA and Pho-
toplethysmography
(PPG)
User emotional state
Reddy et al. 2010 Accelerometer, GPS Transportation Modes
Two interesting issues
pointed out:
• The combination of
diverse sensor
outputs produces
context information
(Virtualization)
• It can be obtained
similar context
information from
different virtual
sensors (Abstraction)
Borja Gamecho UPV/EHU 6 / 56
9. Introduction Conceptual Framework Implementation Evaluation Conclusions
Virtualization and Abstraction example
Developer perspective
It’s difficult to deal with heterogeneous sensors to create
context-aware applications
Borja Gamecho UPV/EHU 7 / 56
10. Introduction Conceptual Framework Implementation Evaluation Conclusions
Virtualization and Abstraction example
Developer perspective
It’s difficult to deal with heterogeneous sensors to create
context-aware applications
Borja Gamecho UPV/EHU 7 / 56
11. Introduction Conceptual Framework Implementation Evaluation Conclusions
Virtualization and Abstraction example
Developer perspective
It’s difficult to deal with heterogeneous sensors to create
context-aware applications
Borja Gamecho UPV/EHU 7 / 56
12. Introduction Conceptual Framework Implementation Evaluation Conclusions
Virtualization and Abstraction example
Developer perspective
Virtualization and Abstraction mechanism facilitate the access to
context information
Borja Gamecho UPV/EHU 7 / 56
13. Introduction Conceptual Framework Implementation Evaluation Conclusions
Motivation summary
1 A proper extension of Egoki
user interface generator
2 Support abstraction and
virtualization process
3 Facilitate the development
of context-aware
applications
IGERRI
A Framework for Abstraction
and Virtualization of Sensors in
Mobile Context-Aware
Computing
Borja Gamecho UPV/EHU 8 / 56
15. Introduction Conceptual Framework Implementation Evaluation Conclusions
Conceptual Framework
• Igerri proposes a Conceptual
Framework for Virtualization and
Abstraction of the sensors
• Multilayer model inspired by
virtual machines
• Considers three layers for sensors:
Physical, Virtual, Abstract
• Applications request context
information to the Context Services
Borja Gamecho UPV/EHU 10 / 56
40. Introduction Conceptual Framework Implementation Evaluation Conclusions
Summary
• General characteristics:
• Abstraction
• Independence
• Reusability
• The framework is a reference for implementations
• Requirements:
• Parameters repository
• Information about the sensors
• Information about the transformations
Borja Gamecho UPV/EHU 23 / 56
46. Introduction Conceptual Framework Implementation Evaluation Conclusions
MobileBIT
• Sensor-Driven applications for e-Health domain
• Adopted from PIA Group in IST-UL
• Hybrid approach for applications
• Main components:
• Functional blocks (Source, Normal, Sink)
• Data Processing Language
• Workflow Manager
• JavaScript interface
• Inteded for rapid-prototyping and reusability of components
Borja Gamecho UPV/EHU 27 / 56
50. Introduction Conceptual Framework Implementation Evaluation Conclusions
Adoption and Extension of MobileBIT
MobileBIT adoption rationale:
• MobileBIT’s Functional blocks match Igerri’s abstraction
layers
• Hybrid approach allows compatibility with Egoki
• DPL language is flexible to describe context-aware
applications
Context-Aware extension:
• Creation of specific block for context delivery
• Call-back function in the Web layer to obtain context
Borja Gamecho UPV/EHU 30 / 56
51. Introduction Conceptual Framework Implementation Evaluation Conclusions
PervasiveBIT
PervasiveBIT is a client/server application to complete MobileBIT
with regard to IGERRI
Two modules:
• SensorHub: Gather information in devices about available
sensors
• SENSONTO: Contains the parameters repository and
information about the transformations
Two processes takes place in PervasiveBIT:
• Discovery of the available context in a network
• Creation of suitable DPL files for MobileBIT
Borja Gamecho UPV/EHU 31 / 56
54. Introduction Conceptual Framework Implementation Evaluation Conclusions
Conclusion
In summary
• MobileBIT is useful for the creation of context-aware
applications
• PervasiveBIT allows MobileBIT to use the same abstraction
levels as defined in Igerri
Advantages of the implementation
• Sensor and device heterogeneity is achieved
• Virtual and Abstraction operations are allowed
• Separation of concerns for application and context generation
• Allows extension of different elements of the abstraction
levels
Borja Gamecho UPV/EHU 34 / 56
56. Introduction Conceptual Framework Implementation Evaluation Conclusions
Evaluation
Application requirements
• Representative to test the framework
Borja Gamecho UPV/EHU 36 / 56
57. Introduction Conceptual Framework Implementation Evaluation Conclusions
Evaluation
Application requirements
• Representative to test the framework
• Demanding regarding the context information
Borja Gamecho UPV/EHU 36 / 56
58. Introduction Conceptual Framework Implementation Evaluation Conclusions
Evaluation
Application requirements
• Representative to test the framework
• Demanding regarding the context information
• Related to the activities of the Egokituz Laboratory
Borja Gamecho UPV/EHU 36 / 56
59. Introduction Conceptual Framework Implementation Evaluation Conclusions
Evaluation
Application requirements
• Representative to test the framework
• Demanding regarding the context information
• Related to the activities of the Egokituz Laboratory
• Real application for a real user group
Borja Gamecho UPV/EHU 36 / 56
61. Introduction Conceptual Framework Implementation Evaluation Conclusions
Evaluation
Animatronic Biofeedback
Experiment Description
ToBITas Case Study Proof of concept of context-aware application
RESapp Pilot Study Realistic application for a specific user group
RESapp Field Study Re-design and extension for a real scenario
Borja Gamecho UPV/EHU 37 / 56
62. Introduction Conceptual Framework Implementation Evaluation Conclusions
Evaluation
Hypothesis
The abstraction and virtualization of sensors as presented in Igerri are
suitable techniques with which to develop usable Context-Aware
applications
Usability evaluation to measure:
• Objective metrics
• Time to complete a task
• Number of errors
• Subjective metrics
• System Usability Scale [Brooke 1996]
• Likert Scales
Borja Gamecho UPV/EHU 38 / 56
68. Introduction Conceptual Framework Implementation Evaluation Conclusions
ToBITas Case Study: Results
SUS Questionnaire
Group A and Group B (11 participants)
• Average score: 73.86
1 ToBITas is a functional and usable Context-Aware application
2 Users understood and learnt quickly how to use the
application control mode
Borja Gamecho UPV/EHU 42 / 56
70. Introduction Conceptual Framework Implementation Evaluation Conclusions
RESapp Pilot Study: Results
SUS Questionnaire
• Average score: 84 ± 4.64
1 RESapp is functional and usable
2 Confusing reinforcements cues → Redesign
Borja Gamecho UPV/EHU 44 / 56
71. Introduction Conceptual Framework Implementation Evaluation Conclusions
RESapp Field Study
Visual Biofeedback
Method A
Animatronic Biofeedback
Method B
Borja Gamecho UPV/EHU 45 / 56
72. Introduction Conceptual Framework Implementation Evaluation Conclusions
RESapp Field Study: Questionnaire Results
User Satisfaction:
1 Lack of Difficulty
2 Perceived Time
3 Comfortability
4 Amusement
User Awareness:
5 Biceps Movements
6 Wrist Movements
Location:
7 Rehabilitation Center
8 Home
Borja Gamecho UPV/EHU 46 / 56
73. Introduction Conceptual Framework Implementation Evaluation Conclusions
RESapp Field Study: Discussion
SUS Questionnaire
• Average score: 88.5 ± 7.2
1 Functional, usable and appealing applications
2 Participants understand the relationship between their
movements and the robot movements
Borja Gamecho UPV/EHU 47 / 56
74. Introduction Conceptual Framework Implementation Evaluation Conclusions
Results summary
SUS Questionnaire Summary
Application Evaluation Participants Age Environment SUS Value
ToBITas Continuous control 11 20-40 Lab conditions 73, 86 ± 12, 58
RESapp Step by Step Pilot 5 64 - 80 Lab conditions 84 ± 4.64
RESapp Step by Step Final 10 64 - 80 In the field 88.5 ± 7.2
• Functional: 93% of the participants ended routines and tasks
• The framework is useful for:
• Rapid prototyping
• Reusability of components
• Realistic and demanding scenarios
• The framework is adequate to produce usable Context-Aware
applications
• Additional effects have been studied from these experiments
Borja Gamecho UPV/EHU 48 / 56
77. Introduction Conceptual Framework Implementation Evaluation Conclusions
Summary
This thesis presents:
• Igerri Conceptual Framework
• Facilitate design of Context-Aware Applications on Mobile
Devices
Borja Gamecho UPV/EHU 50 / 56
78. Introduction Conceptual Framework Implementation Evaluation Conclusions
Summary
This thesis presents:
• Igerri Conceptual Framework
• Facilitate design of Context-Aware Applications on Mobile
Devices
• Implements the Framework using two components
Borja Gamecho UPV/EHU 50 / 56
79. Introduction Conceptual Framework Implementation Evaluation Conclusions
Summary
This thesis presents:
• Igerri Conceptual Framework
• Facilitate design of Context-Aware Applications on Mobile
Devices
• Implements the Framework using two components
• MobileBIT, instantiate the sensor abstractions of the
framework
Borja Gamecho UPV/EHU 50 / 56
80. Introduction Conceptual Framework Implementation Evaluation Conclusions
Summary
This thesis presents:
• Igerri Conceptual Framework
• Facilitate design of Context-Aware Applications on Mobile
Devices
• Implements the Framework using two components
• MobileBIT, instantiate the sensor abstractions of the
framework
• PervasiveBIT, contains the conceptual framework
transformations
Borja Gamecho UPV/EHU 50 / 56
81. Introduction Conceptual Framework Implementation Evaluation Conclusions
Summary
This thesis presents:
• Igerri Conceptual Framework
• Facilitate design of Context-Aware Applications on Mobile
Devices
• Implements the Framework using two components
• MobileBIT, instantiate the sensor abstractions of the
framework
• PervasiveBIT, contains the conceptual framework
transformations
• Usability evaluation of two realistic applications
Borja Gamecho UPV/EHU 50 / 56
82. Introduction Conceptual Framework Implementation Evaluation Conclusions
Contributions
1 A conceptual framework for sensor abstraction and
virtualization
Borja Gamecho UPV/EHU 51 / 56
83. Introduction Conceptual Framework Implementation Evaluation Conclusions
Contributions
1 A conceptual framework for sensor abstraction and
virtualization
2 Implementation of the framework in two components
Borja Gamecho UPV/EHU 51 / 56
84. Introduction Conceptual Framework Implementation Evaluation Conclusions
Contributions
1 A conceptual framework for sensor abstraction and
virtualization
2 Implementation of the framework in two components
3 MobileBIT extension for context aware applications
Borja Gamecho UPV/EHU 51 / 56
85. Introduction Conceptual Framework Implementation Evaluation Conclusions
Contributions
1 A conceptual framework for sensor abstraction and
virtualization
2 Implementation of the framework in two components
3 MobileBIT extension for context aware applications
4 Evaluation results for the usability experiments
Borja Gamecho UPV/EHU 51 / 56
86. Introduction Conceptual Framework Implementation Evaluation Conclusions
Igerri and Egoki
The use of the Hybrid application approach in Igerri allow the
extension of Egoki.
With this work Egoki can be improved:
• New input modalities can be added including gesture
support.
Borja Gamecho UPV/EHU 52 / 56
87. Introduction Conceptual Framework Implementation Evaluation Conclusions
Igerri and Egoki
The use of the Hybrid application approach in Igerri allow the
extension of Egoki.
With this work Egoki can be improved:
• New input modalities can be added including gesture
support.
• Adaptability of user-adapted interfaces
Borja Gamecho UPV/EHU 52 / 56
88. Introduction Conceptual Framework Implementation Evaluation Conclusions
Igerri and Egoki
The use of the Hybrid application approach in Igerri allow the
extension of Egoki.
With this work Egoki can be improved:
• New input modalities can be added including gesture
support.
• Adaptability of user-adapted interfaces
• Proactivity in the applications and interaction without
graphical user interfaces will be possible
Borja Gamecho UPV/EHU 52 / 56
89. Introduction Conceptual Framework Implementation Evaluation Conclusions
Igerri and Egoki
The use of the Hybrid application approach in Igerri allow the
extension of Egoki.
With this work Egoki can be improved:
• New input modalities can be added including gesture
support.
• Adaptability of user-adapted interfaces
• Proactivity in the applications and interaction without
graphical user interfaces will be possible
Egoki provides Context-Aware user interfaces with the help of
Igerri.
Borja Gamecho UPV/EHU 52 / 56
90. Introduction Conceptual Framework Implementation Evaluation Conclusions
Publications
Journal Publications:
1 A Context-Aware Application to Increase Elderly Users Compliance of
Physical Rehabilitation Exercises at Home via Animatronic Biofeedback.
Gamecho B., Silva H., Guerreiro J., Gardeazabal L., Abascal J. Journal of
Medical Systems. (minor changes)
2 Automatic Generation of Tailored Accessible User Interfaces for
Ubiquitous Services. Gamecho B., Miñón R., Aizpurua A., Cearreta I.,
Arrue M., Garay-Vitoria N., Abascal J. In: Human-Machine Systems, IEEE
Transactions on, PP(99):1–12.
Book Chapter:
3 Extending In-home User and Context Models to Provide Ubiquitous
Adaptive Support Outside the Home. Aizpurua A., Cearreta I., Gamecho
B., Miñón R., Garay-Vitoria N., Gardeazabal L. and Abascal J. In: Martín E,
Haya PA, Carro RM (eds) User Modeling and Adaptation for Daily
Routine, Springer-Verlag.
Borja Gamecho UPV/EHU 53 / 56
91. Introduction Conceptual Framework Implementation Evaluation Conclusions
Publications
International conferences:
4 Evaluation of a Context-Aware Application for Mobile Robot Control Mediated by Physiological Data: The
ToBITas Case Study. Gamecho B., Guerreiro J., Alves A.P., Lourenço A., Silva H.P., Gardeazabal L., Abascal J.,
Fred A. In: UCAmI’14.
5 Design Issues on Accessible User Interface Generation for Ubiquitous Services through Egoki. Gamecho B.,
Miñón R., Abascal J.In: AAATE 2013.
6 Automatically Generating Tailored Accessible User Interfaces for Ubiquitous Services. Abascal J., Aizpurua A.,
Cearreta I., Gamecho B., Garay-Vitoria N. and Miñón R. In: ASSETS 2011.
7 Some Issues Regarding the Design of Adaptive Interface Generation Systems. Abascal J., Aizpurua A., Cearreta
I., Gamecho B., Garay-Vitoria N. and Miñón R. In: HCII 2011.
8 Model-Based Accessible User Interface Generation in Ubiquitous Environments. Miñón R., Abascal J., Aizpurua
A., Cearreta I., Gamecho B., Garay-Vitoria N. In: INTERACT 2011
9 Generación de interfaces de usuario accesibles para entornos ubicuos, basadas en modelos. Miñón R., Abascal J.,
Aizpurua A., Cearreta I., Gamecho B., Garay N. In: Interacción 2011
10 Testing A Standard Interoperability Framework In An Ambient Assisted Living Scenario. Gamecho B., Abascal
J. and Gardeazabal L. In: UCAmI’11.
International conference workshops:
11 Combination and Abstraction of Sensors for Mobile Context-Awareness. Gamecho, B., Gardeazabal, L. and
Abascal, J. In: UbiMI’13.
12 A Context Server to Allow Peripheral Interaction. Gamecho B., Gardeazabal L. and Abascal J. Peripheral
Interaction 2013.
13 Augmented Interaction with Mobile Devices to Enhance the Accessibility of Ubiquitous Services. Gamecho B.,
Gardeazabal L., Abascal J. In: MOBACC 2013.
Borja Gamecho UPV/EHU 54 / 56
92. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations
The implementation of the Conceptual Framework has some
limitations:
Borja Gamecho UPV/EHU 55 / 56
93. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations
The implementation of the Conceptual Framework has some
limitations:
• Share information across different applications at the same
time
Borja Gamecho UPV/EHU 55 / 56
94. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations
The implementation of the Conceptual Framework has some
limitations:
• Share information across different applications at the same
time
• Availability of an external server for PervasiveBIT
Borja Gamecho UPV/EHU 55 / 56
95. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations
The implementation of the Conceptual Framework has some
limitations:
• Share information across different applications at the same
time
• Availability of an external server for PervasiveBIT
• Context-Aware applications don’t change sensors on run time
Borja Gamecho UPV/EHU 55 / 56
96. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations
The implementation of the Conceptual Framework has some
limitations:
• Share information across different applications at the same
time
• Availability of an external server for PervasiveBIT
• Context-Aware applications don’t change sensors on run time
• Lack of mechanism to chose which virtual sensor is better if
there are more than two available
Borja Gamecho UPV/EHU 55 / 56
97. Introduction Conceptual Framework Implementation Evaluation Conclusions
Limitations
The implementation of the Conceptual Framework has some
limitations:
• Share information across different applications at the same
time
• Availability of an external server for PervasiveBIT
• Context-Aware applications don’t change sensors on run time
• Lack of mechanism to chose which virtual sensor is better if
there are more than two available
These limitations can be overcome with a different
implementation or by improving the actual one
Borja Gamecho UPV/EHU 55 / 56
98. Introduction Conceptual Framework Implementation Evaluation Conclusions
Future Work
• Combine Igerri with Egoki to create accessible smartphone
applications
Borja Gamecho UPV/EHU 56 / 56
99. Introduction Conceptual Framework Implementation Evaluation Conclusions
Future Work
• Combine Igerri with Egoki to create accessible smartphone
applications
• Testing Igerri implementation using more than one output for
Virtual Sensors
Borja Gamecho UPV/EHU 56 / 56
100. Introduction Conceptual Framework Implementation Evaluation Conclusions
Future Work
• Combine Igerri with Egoki to create accessible smartphone
applications
• Testing Igerri implementation using more than one output for
Virtual Sensors
• Extend it for distributed context-aware applications
Borja Gamecho UPV/EHU 56 / 56
101. Introduction Conceptual Framework Implementation Evaluation Conclusions
Future Work
• Combine Igerri with Egoki to create accessible smartphone
applications
• Testing Igerri implementation using more than one output for
Virtual Sensors
• Extend it for distributed context-aware applications
• Test the implementation with developers
Borja Gamecho UPV/EHU 56 / 56
102. A Framework for
Abstraction and Virtualization of Sensors
in Mobile Context-Aware Computing
Borja Gamecho
Supervised by Julio Abascal and Luis Gardeazabal
University of the Basque Country UPV/EHU
June 29, 2015
Laboratory of Human-Computer Interaction
for Special Needs