1. Grupo de Procesado de Datos y Simulación
ETSI de Telecomunicación
Universidad Politécnica de Madrid
Towards a fuzzy-based multi-classifier selection module
for activity recognition applications
SeNAmI 2012
Henar Martín, Josué Iglesias, Jesús Cano, Ana M. Bernardos, José R. Casar
josue@grpss.ssr.upm.es

2. contents
 introduction and motivation
 architecture details
 classifier evaluation module
 fuzzy selector module
 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 2 / 20

3. contents
 introduction and motivation
 architecture details
 classifier evaluation module
 fuzzy selector module
 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 3 / 20

4. introduction and motivation
“Towards a fuzzy-based multi-classifier selection module for activity recognition applications“
why activity recognition? how to perform activity recognition?
patient monitoring video processing
sport trainers wearable sensors
emergency detectors o ad hoc sensors
diary builders o personal mobile embedded sensors
location systems accelerometers/gyroscopes, compass, camera, microphone, etc.
• mainly infrastructure-based
network coverage, latency, privacy, etc.
what about using smartphones processing capabilities for activity recognition?
• their use on a daily basis and
• processing capabilities are growing spectacularly
focus
1) analyse the cost of integrating a set of classifiers to detect user activity in a smartphone
2) propose a fuzzy method to select the best classifier configuration
(in order to save device resources)
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 4 / 20

5. contents

 introduction and motivation
 architecture details
 classifier evaluation module
 fuzzy selector module
 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 5 / 20

6. architecture details
on-line stage off-line stage
Comp. cost memory
Position Classifier Activity Classifier Classifier All features or
selection fuzzy selection Evaluation mean and variance
Decision Tree (J48)
Decision Table  accuracy All sensors or
 size accelerometer only
Position features Activity features  response time Real time
computation computation  complexity Sliding windows with or
Sensor without overlap
back trousers pocket measurements
front trousers pocket gathering sit
shirt pocket stand
Position hand texting Activity walk
hand talking
classifier waist case classifier slow walk
rush walk
backpack run
jacket pocket
long strap bag
armband
Position Activity
a) b)
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 6 / 20

7. architecture details
1) analyse the cost of integrating a set of classifiers to detect user activity in a smartphone
2) propose a fuzzy method to select the best classifier configuration
on-line stage off-line stage
2) Comp. cost memory
Position Classifier Activity Classifier Classifier All features or
selection fuzzy selection Evaluation mean and variance
Decision Tree (J48)
Decision Table  accuracy All sensors or
 size accelerometer only
Position features Activity features  response time Real time
computation computation  complexity Sliding windows with or
Sensor without overlap
back trousers pocket measurements
front trousers pocket gathering sit
shirt pocket stand
Position hand texting Activity walk
classifier
hand talking
waist case classifier slow walk 1)
rush walk
backpack run
jacket pocket
long strap bag
armband
Position Activity
a) b)
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 7 / 20

8. contents

 introduction and motivation
 architecture details
 classifier evaluation module
 fuzzy selector module
 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 8 / 20

9. architecture details
classifier evaluation module
1) analyse the cost of integrating a set of classifiers to detect user activity in a smartphone
sensors features classifiers activities
embedded sensors time-domain
accelerometer mean sit
linear acceleration variance
gravity zero crossing rate decision table stand
magnetometer percentile 75
orientation interquartile
gyroscope walk
device position frequency-domain
fft energy
slow walk
+ light sensor
frequency domain entropy
+ proximity sensor
power spectrum centroid
decision tree rush walk
hand (texting) short/long strap bag
hand (talking) trouser pockets
backpack shirt/jacket pocket
armband waist case
signal energy run
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 9 / 20

10. architecture details
classifier evaluation module
1) analyse the cost of integrating a set of classifiers to detect user activity in a smartphone
on-line stage off-line stage
Comp. cost memory
Position Classifier Activity Classifier Classifier All features or
selection fuzzy selection Evaluation mean and variance
Decision Tree (J48)
Decision Table  accuracy All sensors or
 size accelerometer only
Position features Activity features  response time Real time
computation computation  complexity Sliding windows with or
Sensor without overlap
back trousers pocket measurements
front trousers pocket gathering sit
shirt pocket stand
Position hand texting Activity walk
classifier
hand talking
waist case classifier slow walk 1)
rush walk
backpack run
jacket pocket
long strap bag
armband
Position Activity
a) b)
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 10 / 20

11. architecture details
classifier evaluation module
1) analyse the cost of integrating a set of classifiers to detect user activity in a smartphone
on-line stage off-line stage
Comp. cost memory
Activity Classifier Classifier All features or
activities
fuzzy selection Evaluation mean and variance
classifiers
 accuracy All sensors or
 size accelerometer only
Real time
features
 response time
 complexity Sliding windows with or
without overlap sensors
(~32) classifier
configurations
classifier features
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 11 / 20

12. contents

 introduction and motivation
 architecture details

 classifier evaluation module
 fuzzy selector module
 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 12 / 20

13. architecture details
fuzzy selector module
2) propose a fuzzy method to select the best classifier configuration
on-line stage off-line stage
2) Comp. cost memory
Position Classifier Activity Classifier Classifier All features or
selection fuzzy selection Evaluation mean and variance
Decision Tree (J48)
Decision Table  accuracy All sensors or
 size accelerometer only
Position features Activity features  response time Real time
computation computation  complexity Sliding windows with or
Sensor without overlap
back trousers pocket measurements
front trousers pocket gathering sit
shirt pocket stand
Position hand texting Activity walk
hand talking
classifier waist case classifier slow walk
rush walk
backpack run
jacket pocket
long strap bag
armband
Position Activity
a) b)
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 13 / 20

14. architecture details
fuzzy selector module
2) propose a fuzzy method to select the best classifier configuration
application requirements
required classifier 1
accuracy
response delay
classifier 2 chosen
device context
classifier 3 classifier
battery level
memory available classifier N
CPU load
classifier evaluation
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 14 / 20

15. architecture details
fuzzy selector module
2) propose a fuzzy method to select the best classifier configuration
application requirements 2.a) quality trained accuracy
required computation module response delay
file size
accuracy complexity
response delay
target classifier
device context 0.91 accuracy
battery level 0.83 delay
0.38 size
memory available 0.67 complexity
CPU load
classifier evaluation
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 15 / 20

16. architecture details
fuzzy selector module
2) propose a fuzzy method to select the best classifier configuration
application requirements 2.a) quality trained accuracy
required computation module response delay
file size
accuracy complexity
response delay
target classifier
device context 0.91 accuracy
battery level 0.83 delay
0.38 size
memory available 0.67 complexity
CPU load
classifier evaluation
2.b) distance-based
classifier selector

Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 16 / 20

17. contents

 introduction and motivation

 architecture details

 classifier evaluation module

 fuzzy selector module
 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 17 / 20

18. system pre-validation
real calculation of classifiers features • Android-based Google Nexus S device
• 16 subjects (6 activities, 11 device positions)
• response times
◦ Android’s Traceview Tool
• accuracy
◦ WEKA (leave-one-subject-out method)
sweeping test
• freeMemory = 80%
• requiredAccuracy = medium
• requiredResponseTime = medium
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 18 / 20

19. contents

 introduction and motivation

 architecture details

 classifier evaluation module

 fuzzy selector module

 system pre-validation
 conclusions and future works
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 19 / 20

20. conclusions and future works
• accuracy enhanced when considering the position of
the mobile
• accuracy worsens (and size reduced) when the
accelerometer is the only sensor considered
 better approach to determining the
complexity of the classifiers
 dynamic fuzzy membership functions
 real application on top of this system
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 20 / 20

21. any question?
Sensor Networks and Ambient Intelligence – SeNAmI 2012 josue@grpss.ssr.upm.es 21 / 20