1. 2018: Technology in
Labour for Rebirth of
HAL-9000 series
Some contributions of ETRO

2. The HAL-9000 series (1968)
2001: A Space Odyssey - Stanley Kubrick, Arthur C. Clarke
The spaceship discovery
HAL-9000
2

3. HAL-9000 Ambient Intelligence System (1968)
Monitors its surroundings, the ship and its crew
Analyses sensors, images and sounds
Converses in natural spoken language
Designed to assist the user
HAL-9000
3

4. AMI-9000 Ambient Intelligence System (2018)
Monitors its surroundings, the ship and its crew
Analyses sensors, images and sounds
Converses in natural spoken language
Designed to assist the user
AMI-9000
4

5. AMI-9000 Ambient Intelligence System (2018)
Monitors its surroundings, the ship and its crew
Analyses sensors, images and sounds
Converses in natural spoken language
Designed to assist the user
Intelligence: expression/emotion/body language awareness
AMI-9000
5

6. Current research @ ETRO-DSSP
Speech modification
time scaling, intelligibility enhancement, automatic lip
synchronization, voice conversion
Speech synthesis
Flemish TTS, hierarchic TTS, AV TTS, expressive
speech synthesis (emotions)
Noise suppression (single channel, contact microphones)
robust speech recognition
Microphone array techniques
distant recording, blind source separation
Speech disorders
diagnostics, remedy
6

7. Emotion Recognition
Emotions in speech
alter pitch, timing, voice
quality and articulation.
Emotional speech rec.
classify statistical
measures of acoustic
AIBO
SBA
features into classes.
Two approaches
Segment Based
(SBA) & AIBO (Sony)
That’s Right
7

8. X-database study of emotion recognition
Kismet BabyEars Berlin Danish
Best score 87% 69% 75% 64%
Literature 82% 67% Human 85% 54% H:67%
Baseline 32% 42% 34% 51%
Baseline X-DB 35% 42% 46% 24%
X-DB 54% 45% 53% 23%
Obtained state-of-the art recognition scores
Emotion recognition = database dependent
Classifiers can be learned on joined database
Go to Demo Use of higher level features might help
Use arousal detection as case study
8

9. Current research @ ETRO-IRIS
USER INTERACTION USER INTERACTION
ANALYSIS SYNTHESIS
- detection & tracking - morphing a
of body and face 3D head
USER - face model
NATURAL INTERACTION adaptation
- estimating the
facial animation
parameters - animating an
-motion avatar
- emotion feature -speech - mouth
extraction
-expression visualization
- audiovisual speech - data-driven
segmentation feedback
9

10. Facial Analysis & Synthesis
Gestures Multi-modal speech
Motion estimation Enhancement by mouth images
Pose and structure variations Animated avatar
Eye gaze and expressions
and takes into account the natural face motions
10

11. Expression Analysis
Visual input
Facial Action Coding System
a muscle-based method to measure facial movements w.r.t. Face processing unit
Facial anatomy , widely used in Psychology
Each Action Units (AU) represents one visibly
distinguishable facial change (46 AUs for facial
appearances, 12 AUs for gaze direction and head pose).
Face expression = Co-occurrence of several AUs
A parametric model combining several AU’s
has been built for expression analysis Expression processing unit
What is hidden behind a face expression?
the temporal course of the muscle activities (intensity of
muscle contraction/relaxation versus time).
Information to recognise concealed emotions (e.g.
deception). AUs are purely descriptive. FACS provides a
dictionary to interpret the corresponding emotions
Surprise Anger
recognition/synthesis
11

12. AMI-9000 Ambient Intelligence System (2018)
Smart buildings (surveillance)
Care for the elderly at home (assisting, security)
Personalised personal assistants (understand the user)
VIN: adapt according to the user’s state-of-mind
Embodied conversational agents for education
12