Rama Akkiraju, Distinguished Engineer and Master Inventor at IBM, presention "Building Compassionate Conversational Systems" as part of the Cognitive Systems Institute Speaker Series.
5. User Modeling: Our framework
@Copyright IBM 2015 5
Act
Be
Feel
Context
Think
Options
Explore
&
Decide
Inner State Environment Outer State
An individual takes action based on the combination of his/her unique being & environment
6. User Modeling: Our framework
@Copyright IBM 2015 6
Act
Search
Preferen
ces
Commun
ications
Decisions
Commit
ments
Purchases
Context
Life Style,
Events
Sociological Economic
Political Technological
Options
Price Promotions
Products/
Services
Place
FeelPerceptions
Emotions
Sensations
Attitudes
Influences
Sentiments
Be
Personality
Needs,
Values
Beliefs
Motives
Identity
Goals, Ambitions
Interests
Think
Knowledge
Skills
Opinions
Cognitive Style
Explore
&
Decide
Choices
Consequenc
es
Session
Intent
Time
16. Dialog Act
• Dialog Act is a specializedSpeech Act. Typically,looks at patterns in dialogs.
16
• Statement
• backchannel/acknowledge
• Opinion
• abandoned/uninterpretable
• agreement/accept
• appreciation
• yes-no-question
• non-verbal
• yes answers
• conventional-closing
• wh-question
• no answers response
• quotation
• Summarize/reformulate
• affirmative
• action-directive
• collaborative completion
• repeat-phrase open-question
• rhetorical-questions
• reject
• other answersconventional-
opening or-clause
• commits self-talk
• downplayer
• apology
• thanking
Source: Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech http://www.aclweb.org/anthology/J00-3003
17. Dialog Strategies
Start
Giving an extra
Acknowledging
Need
Description of
Need
Anger
Acknowledging
w/out encouraging
Refocus
statements
Active
Listening
Possibility of
mistake
Admitting
mistake
Allowing
venting
Apology
Smiles
Arranging
Follow-up
Need cannot
be fulfilled on
the spot
Assurance of
effort
Assurance of
result
Mistake has
been made
Bonus buyoff
Broken record
Uncooperative
customer
Closing
positively
Common
Courtesy
Completing
Follow-up
Contact
Security
Aggressiveness Disengaging
Distraction
Frustration
Empathy
statement
Expediting
Expert
Recommendati
on
Explain Reasoning
or action
Embarrassment
Face-Saving
Out
Conflict
Finding
Agreement Points
Following up
Helpless
Offering
Choice
Empowering
Preventive
strike
Privacy
insurance
Privacy
concern
Probing
question
Pros and Cons
Providing
Alternatives
Providing
Takeaway
Confusion
Providing
Explanation
Questioning
instead of
stating
Referral to
supervisor
Referral to 3rd
party
Lost focus Refocus
Inappropriate
behavior
Setting Limits
Critical
Neutral mode
Summarize the
conversation
Silence
Thank-you
Timeout Use customer
name
Verbal
Softeners
When QuestionYou re right
Action
Negative
Emotion
Monologue
End External Giving
Emotions
General
States
Gratitude
Statement
Happiness
Work by IBM Haifa Research team Michal Shmueli-Scheuer, Jonathan Herzig, Guy Feigenblat,
David Konopnicki
@Copyright IBM 2015
20. Different channels for Conversations
• Kiosks
• Bots
• Robots
• Virtual agents on mobile-devices
• Virtual agents accessible on a computer
• Question from User modeling point of view.
• Would user style of interaction with the system change based on
devices/channels?
• Would users willingness to reveal information about themselves change
depending on the channel/device?
20
23. Tone Analyzer in Customer Support Q&A Forum
Study #1: Clients’ Q&A forum data was analyzed
• Confident responses are more likely to receive Kudos (r = 0.23)
• Tentative responses are less likely to receive Kudos (r=0.27)
• We found that we can predict kudos received with 66% accuracy which
is better than random (50%)
• We applied multiple state of the art classifiers such as Naïve Bayes, SVM,
Random Forest and did 10-fold cross validation
Study #2: Twitter customer support forums (333 conversations (240 Sat,
93 not-Sat))
• More angry customers are less likely to be satisfied after the conversation (r =
-0.198)
• More disgusted customers are less likely to be satisfied after the conversation
(r = -0.184)
• Agents who show higher emotional range are less likely to satisfy the
customer (r = -0.186)
25. Personality Insights Accuracy – Latest results
25
# of Tweets
Mean
Absolute
Error (MAE)
Trait Name
Mean Absolute
Error (MAE)
Correlation
Agreeableness 0.0999 0.2920
Conscientiousness 0.1174 0.3259
Extraversion 0.1477 0.2521
Neuroticism 0.1404 0.4182
Openness 0.0862 0.3650
• A Machine Learned model for predicting Personality Traits
• UsesWord2Vec features (Stanford Glove pre-trainedmodel)
• Ground truth collected include 2,000 psychometric surveys
26. How many words to infer Personality?
26
# of Tweets
Mean
Absolute
Error (MAE)
We reach 95% of the max accuracy with as low as 30 tweets.
0.09
0.095
0.1
0.105
0.11
0.115
0.12
0.125
0.13
0 50 100 150 200 250 300 350
MAE
Number of tweets used for testing
Trait Agreeableness – MAE VS numberof tweets
Old Model
New Model
Old Model: Linguistic Inquiry Word Count (LIWC) based
New Model: Word2Vec based
27. Greeting
• Opening
• Closing
Statement
• Give Info
• Expressive (Pos/Neg)
• Complaint
• Offer Help
• Suggest Action
• Promise
• Sarcasm
• Other
Request
• Request Help
• Request Info
• Other
Question
• Yes-No Question
• Wh- Question
• Open Question
Answer
• Yes-Answer
• No-Answer
• Response-Ack
• Other
Social
Act
• Thanks
• Apology
• Downplayer
Methodology
• Designing more fine-grained actionable dialogue acts:
30. Utterances are complex: A single label is not sufficient
0 50 100 150 200 250 300 350 400 450 500
(statement_info, answer_other)
(statement_expressive_negative, statement_complaint)
(statement_info, statement_complaint)
(request_info, question_yesno)
(request_info, question_wh)
(request_info, question_open)
(statement_offer, request_info)
(statement_info, statement_expressive_negative)
(request_info, socialact_apology)
(statement_info, statement_suggestion)
(statement_suggestion, request_info)
(statement_info, socialact_thanks)
(statement_info, answer_yes)
(statement_info, request_info)
(question_yesno, socialact_apology)
(statement_info, question_yesno)
§ We test the hypothesis that each turn may require more than one dialogue act label by finding the
distribution of label overlap in our annotations
§ We verify that labels frequently co-occur, so classification should assign an utterance multiple labels