1. Ankit Kumar MoonkaAnkit Kumar Moonka
Sonam Rani MishraSonam Rani Mishra
B.Tech 6B.Tech 6thth
SemesterSemester
Affective Computing
or Better Intelligence Systems

2.  What’s Affective Computing ? ?
 Why go For Affective Computing ? ?
 Motivation & Goals
 Applications
 Affective Computing research
 Detection & Recognition
 Concerned issues
 Future Developments
 Conclusion
Main topics

3. Introduction to
Affective
Computing

4. Producing emotional response
According to Picard – “…computing
that relates to, arises from, or
deliberately influences emotions”
Affective Computing – ability for the
computer to recognize and express
emotions as humans do
But do not have emotion.
WHAT IS AFFECTIVE COMPUTING?

5. Humans naturally communicate
affectively; expression identified
50% of the time.
Human-Computer
Interaction –
Frustration, mouse
clicking behavior, slow,
debugging, so we need
friendlier HCI.

6. Affective Computing
Motivations and Goals
 Research shows that human intelligence is not independent
of emotion. Emotion and cognitive functions are
inextricably integrated into the human brain.
 Automatic assessment of human emotional/affective state.
 Creating a bridge between highly emotional human and
emotionally challenged computer systems/electronic devices
- Systems capable of responding emotionally.
 The central issues in affective computing are
representation, detection, and classification
of users emotions.

7. ApplicationsApplications
Hands-free computing Social interfacesSocial interfaces
Distance education
Internet banking

8. Applications (Contd.)Applications (Contd.)
 Security sectorSecurity sector
 Medical sectorMedical sector
 NeurologyNeurology
 PsychiatryPsychiatry
 Dialog/Automatic call center Environment – to reduceDialog/Automatic call center Environment – to reduce
user/customer frustrationuser/customer frustration
Zhihong Zeng; Pantic, M.; Roisman, G.I.; Huang, T.S.; , "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions,"
Pattern Analysis and Machine Intelligence

9. ∗ Affective computing can be related to other computing
disciplines such as Artificial Intelligence (AI), Virtual
Reality (VR) and Human Computer interaction (HCI).
∗ Questions need to be answered:
What we mean when we use the word emotion?
What is an affective state (typically feelings, moods, etc.)?
Which human communicative signals convey information
about affective state?
How to apply affective information to designing systems?
AFFECTIVE COMPUTING RESEARCH

11. Steps towards affective computing research
 We first need to define what we mean when we use the word
emotion.
 Second, we need an emotion model that gives us the possibility
to differentiate between emotional states.
 In addition, we need a classification scheme that uses specific
features from an underlying (input) signal to recognize the
user’s emotions .
 The emotion model has to fit together with the classification
scheme used by the emotion recognizer.
R. Sharma, V. Pavlovic, and T. Huang. Toward multimodal human-computer interface. In Proceedings of the IEEE, 1998.

12. Affection detection sources:
∗ Bio-signals (Psychological sensors, Wearable sensors)
∗ Brain Signal, skin temperature, blood pressure, heart
rate, respiration rate
∗ Facial & Speech/Vocal Expression
∗ Gesture & Text
∗ Limbic movements
AFFECTION DETECTION AND
RECOGNITION

13. Affection Recognition Method
Speech Recognition Architecture
Feature ExtractionPre-processing
Speech Signal
Classification
Classified Result
Audio recordings collected in call centers
and, meetings, Wizard of Oz scenarios
interviews and other dialogue systems
• Accuracy rates from speech are somewhat lower
(35%) than facial expressions for the basic emotions .
• Sadness, anger, and fear are the emotions
that are best recognized through voice, while
disgust is the worst.
]M. Pantic, N. Sebe, J. F. Cohn, and T. Huang. Affective multimodal human-computer interaction. In ACM International Conference on Multimedia (MM), 2005.
Rafael A. Calvo, Sidney D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications

14. Concerned Issues
∗ Privacy concerns
∗ I do not want the outside world to
know what goes through my
mind…Twitter is the limit
∗ Risk of misuse of the technology
∗ In the hand of impostors
∗ Computers start to make
emotionally distorted, harmful
decisions
∗ Complex technology
∗ Effectiveness is still questionable,
risk of false interpretation

15. FUTURE RESEARCH DIRECTIONS
∗So far Context has
been overlooked in
most Affection
Computing researches
∗Collaboration among
Affection researchers
from different
disciplines
∗Fast real-time
processing
∗Multimodal detection
and recognition to
achieve higher accuracy
∗Systems that can model
conscious and
subconscious user
behavior

16. Context Aware Multimodal Affection Analysis
Based Smart Learning Environment
17

17. Conclusion
18
“Ultimately, affective-
computing technology could
eliminate the need for devices
that today stymie and
frustrate users…
Affective computing is an
important development in
computing, because as
pervasive or ubiquitous
computing becomes
mainstream, computers will
be far more invisible and
natural in their interactions
with humans.”
Toyota’s thought controlled wheelchair

18. 19

19. 20
QUERIES???

20. References
[1] Picard, R. 1995. Affective Computing. M.I.T Media Laboratory Perceptual Computing Section Technical Report No. 321
[2] Picard, R. 1995. Affective Computing. The MIT Press. ISBN-10: 0-262-66115-2.
[3] Picard, R., & Klein, J. (2002). Computers that recognize and respond to user emotion: Theoretical and practical implications. Interacting
With Computers, 14, 141-169.
[4] http://www.sric-bi.com/
[5] Bullington, J. 2005. ‘Affective’ computing and emotion recognition systems: The future of biometric surveillance? Information Security
Curriculum Development (InfoSecCD) Conference '05, September 23-24, 2005, Kennesaw, GA, USA.
[6] Boehner, K., DePaula, R., Dourish, P. & Sengers, P. 2005. Affect: From Information to Interaction. AARHUS’05 8/21-8/25/05 Århus,
Denmark.
[7] Zeng, Z. et al. 2004. Bimodal HCI-related Affect Recognition. ICMI’04, October 13–15, 2004, State College, Pennsylvania, USA.
[8] Taleb, T.; Bottazzi, D.; Nasser, N.; , "A Novel Middleware Solution to Improve Ubiquitous Healthcare Systems Aided by Affective
Information," Information Technology in Biomedicine, IEEE Transactions on , vol.14, no.2, pp.335-349, March 2010
[9] Khosrowabadi, R. et al. 2010. EEG-based emotion recognition using self-organizing map for boundary detection. International
Conference on Pattern Recognition, 2010.
[10] R. Cowie, E. Douglas, N. Tsapatsoulis, G. Vostis, S. Kollias, w. Fellenz and J. G. Taylor, Emotion Recognition in Human-computer
Interaction. In: IEEE Signal Processing Magazine, Band 18 p.32 - 80, 2001.
[11] Rafael A. Calvo, Sidney D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE
Transactions on Affective Computing, pp. 18-37, January-June, 2010
[12] Zhihong Zeng; Pantic, M.; Roisman, G.I.; Huang, T.S.; , "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous
Expressions," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.31, no.1, pp.39-58, Jan. 2009
[13] Norman, D.A. (1981). ‘Twelve issues for cognitive science’, Perspectives on Cognitive Science, Hillsdale, NJ: Erlbaum, pp.265–295.
[14] R. Sharma, V. Pavlovic, and T. Huang. Toward multimodal human-computer interface. In Proceedings of the IEEE, 1998.
[15] Vesterinen, E. (2001). Affective Computing. Digital media research seminar, spring 2001: “Space Odyssey 2001”.

21. References
[16] Burkhardt, F.; van Ballegooy, M.; Engelbrecht, K.-P.; Polzehl, T.; Stegmann, J.; , "Emotion detection in dialog systems:
Applications, strategies and challenges," Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009. 3rd
International Conference on , vol., no., pp.1-6, 10-12 Sept. 2009
[17] Leon, E.; Clarke, G.; Sepulveda, F.; Callaghan, V.; , "Optimised attribute selection for emotion classification using physiological
signals," Engineering in Medicine and Biology Society, 2004. IEMBS '04. 26th Annual International Conference of the IEEE ,
vol.1, no., pp.184-187, 1-5 Sept. 2004
[19] http://www.engadget.com/2009/06/30/toyotas-mind-controlled-wheelchair-boast-fastest-brainwave-anal/
[20] http://www.w3.org/TR/2009/WD-emotionml-20091029/
[21] M. Pantic, N. Sebe, J. F. Cohn, and T. Huang. Affective multimodal human-computer interaction. In ACM International
Conference on Multimedia (MM), 2005.
[22] Gong, L., Wang, T., Wang, C., Liu, F., Zhang, F., and Yu, X. 2010. Recognizing affect from non-stylized body motion using shape
of Gaussian descriptors. In Proceedings of the 2010 ACM Symposium on Applied Computing (Sierre, Switzerland, March 22 - 26,
2010). SAC '10. ACM, New York, NY, 1203-1206.
[23] Khalili, Z.; Moradi, M.H.; , "Emotion recognition system using brain and peripheral signals: Using correlation dimension to
improve the results of EEG," Neural Networks, 2009. IJCNN 2009. International Joint Conference on , vol., no., pp.1571-1575,
14-19 June 2009
[24] Huaming Li and Jindong Tan. 2007. Heartbeat driven medium access control for body sensor networks. In Proceedings of the 1st
ACM SIGMOBILE international workshop on Systems and networking support for healthcare and assisted living environments
(HealthNet '07). ACM, New York, NY, USA, 25-30.
[25] Ghandi, B.M.; Nagarajan, R.; Desa, H.; , "Facial emotion detection using GPSO and Lucas-Kanade algorithms," Computer and
Communication Engineering (ICCCE), 2010 International Conference on , vol., no., pp.1-6, 11-12 May 2010
[26] Lucey, P.; Cohn, J.F.; Kanade, T.; Saragih, J.; Ambadar, Z.; Matthews, I.; , "The Extended Cohn-Kanade Dataset (CK+): A
complete dataset for action unit and emotion-specified expression," Computer Vision and Pattern Recognition Workshops
(CVPRW), 2010 IEEE Computer Society Conference on , vol., no., pp.94-101, 13-18 June 2010
[27] Ruihu Wang; Bin Fang; , "Affective Computing and Biometrics Based HCI Surveillance System," Information Science and
Engineering, 2008. ISISE '08. International Symposium on , vol.1, no., pp.192-195, 20-22 Dec. 2008