Given at UXPA-DC's User Focus Conference, Oct. 19, 2012

1. Imagine what it must feel like to
grow up in a slum like this.
This kind of building was thrown
up all over Britain in the 1950s.
To create this, you’d need to have
no sense of the emotions of the
people living here.
Welcome to emotional
interaction design.
@gilescolborne

2. Emotional interaction design
Giles Colborne Before we talk about interaction
design, we should talk about
emotion in design in general.
http://www.flickr.com/photos/jamesclay/2264414513/ 2

3. When you look at this picture
your stress levels will drop and
your ability to concentrate rises
(see for example, At Home with
Nature: Effects of ‘Greenness’ on
Children’s Cognitive Functioning”
by Nancy Wells, Environment and
Behavior, 2000).
But emotional design is about
more than sticking pictures of
trees on your website. We
respond to far deeper patterns in
nature.
@gilescolborne

4. Like the beautiful arrangement
of seeds in this flower.
@gilescolborne

5. And the same arrangement of
leaves in this succulent. A spiral
based on the golden ratio.
@gilescolborne

6. The golden ratio is the ratio of
two lines that fir this equation.
a b
a+b = a
a b
@gilescolborne

7. You can use these lines to draw a
series of squares...
a b
@gilescolborne

8. Which, in turn, define a spiral.
Which is what we saw in those
plants. But the Golden Ratio
crops up throughout nature.
@gilescolborne

9. Like... the bones in your hand.
The Golden Ratio defines our
sense of perfect proportion
and beauty.
@gilescolborne

10. Which is why people have been
using it to create beautiful
things for centuries. People
judge beautiful things to be
good, true, honest, simple. It
would be a cool idea if someone
used this in web design.
@gilescolborne

11. @gilescolborne

12. Many of our notions of beauty
are hard wired into our brains.
Like the baby face effect - people
associate child-like features with
innocence, honesty, friendliness.
@gilescolborne

13. And Aaron Walter points out in
Emotional Design that he’s used
this to good effect throughout
his design. Most obviously in
MailChimp’s mascot. making
boring email newsletter admin
seem fun and friendly.
@gilescolborne

14. Of course we see faces
everywhere. But this doesn’t
mean ‘design everything to the
Golden Ratio’ or ‘put faces on
everything’. Rather, it means you
must ‘understand the importance
of line and form in design’.
Okay, that was 2,500 years of art
and mathematics and industrial
design in a dozen slides.
@gilescolborne

15. Hey, good looking –
what about interaction?
But this is design as object what
about design as interaction?
@gilescolborne

16. This guy’s interaction with his
computer is certainly emotional.
http://www.youtube.com/watch?v=HtTUsOKjWyQ
@gilescolborne

17. Something about computers
brings out the devil in us.
@gilescolborne

18. In response, some
interaction designers
act like Nurse Ratched
in One Flew Over the
Cuckoo’s Nest. They see
emotion as the enemy.
But I think that
misunderstands the
importance of
emotions.
@gilescolborne
http://www.flickr.com/photos/thomashawk/54164255/

19. AI pioneer Marvin
Minsky sees emotions as
‘ways to think’. Emotions
tune the machinery of
the brain.
Even a ‘negative’ emotion
like anger is useful. It
gives us the energy to
address a threat and
sends out social signals
that warn others we’d
better get our way.
@gilescolborne

20. People without emotions
are less effective
thinkers. Antonio
Damasio describes one
patient, Elliot, with
above average IQ but
frontal lobe damage that
means his emotions are
impaired making him a
kind of Mr Spock.
When faced with a
‘rational’ task like
scheduling an
appointment, he
endlessly weighs and
compares choices. He
doesn’t feel boredom,
frustration or
embarrassment. He takes
for ever. Emotions are
vital to decision making.
@gilescolborne

21. This design from HP is
overly rational. The focus
on numbers and
specifications flips us
into rational evaluation
mode. Can I find a
computer that weighs 10
grammes less? Can I find
a computer that’s 3 mm
thinner? We find
ourselves in a rational
spiral.
A better approach would
be to connect on a gut
level - as light as a
hardback book would tell
most people what they
need to know.
@gilescolborne

22. This classic ad shows a
chick being put in a
heatproof box... and
boiled for what seems
like ages before it’s
revealed alive and well.
Why make your audience
so anxious?
Because we remember
stressful events. They’re
playing with our
emotions to fix the idea
of the heatproof box in
our memory.
@gilescolborne

23. I’ve found that when I ask
people about delightful
experiences, they
remember ones
associated with stressful
events.
Like dropping your iPod –
and discovering that it
paused automatically
when the headphones
popped out.
How delightfully clever.
How memorable.
So I look for stress points
when I design because I
know users will remember
what my products do
next.
@gilescolborne

24. Why do we like to watch
horror movies? Because
they help us extend our
emotional range and
understand our
emotions.
Emotions are important
to people. They’re linked
to our strongest
memories and the most
meaningful events in our
lives.
What if we accept there
So we’re right to reject will be ups and downs in
the Nurse Ratched view the relationship between
that we should eliminate humans and computers?
emotion.
What if we try to give our
designs the emotional
intelligence to ride those
waves?
@gilescolborne

25. Marvin Minsky wants to
create artificial
emotional intelligence by
building an artificial
brain. Awesome!
But that’s a few decades
away. Meanwhile, I have
a website to design.
So are there ways I can
cheat?
@gilescolborne

26. Back to this guy. The
word that describes him
is ‘postal’.
So maybe we can learn
how to deal with this
situation by talking to
some experts.
@gilescolborne

27. These folks train
negotiators to deal with
crisis situations. Maybe
FBI
they can help us.
@gilescolborne

28. Behavioural Change Stairway Model
They use this model to get through
hostage negotiations. It recognises
that if you want to get to ‘behaviour behaviour
change’ you need to start with
‘listening’ and ‘empathy’.
change
influence
rapport
empathy
active
listening
@gilescolborne

29. Clearly his computer isn’t
listening to him.
When interfaces don’t
listen we get angry.
@gilescolborne

30. It looks like you’re
giving a presentation
Get help with
giving the
presentation
Just give the
presentation
without help.
Don’t show this tip
Clippy has all the elements that we’re
supposed to include in emotional design. He’s
again
informal yet direct. He’s cute looking.
I’ve always thought there was a lot of good
thinking behind him. But people hated him.
Why? Because he’s so bad at listening.
@gilescolborne

31. My hero Clifford Nass
redesigned Clippy very
simply to listen and
empathise.
When Clippy offered
advice, he would ask ‘was
that useful?’. If people
said ‘no’ Clippy would say
‘that really ticks me off.
Let’s tell the folks at
Microsoft I need to be
reprogrammed.’ and
Clippy would encourage
them to write in and vent
their dissatisfaction.
And it worked: users
liked Clippy.
Clifford Nass
@gilescolborne

32. When you watch this
video, you notice that
there are lots of signs
that things are going
wrong. Our guy begins by
giving the screen a hard
stare. Then he seems to
swear. He slaps the
keyboard. And then he
really starts to lose it.
A more sophisticated
computer might have
picked up on those
warning signs and saved
itself a beating.
@gilescolborne

33. In Affective Computing,
Rosalind Pickard suggests
that computers could use
input from many sensors
(facial recognition, audio
input and so on) and
pattern matching to
detect users’ emotions.
She recognises this is
complex (even people
occasionally misread
emotions). We’re a few
years away from this.
But Clifford Nass shows
us that listening can be as
simple as saying ‘how am
I doing?’.
@gilescolborne

34. The FBI knows there’s a right way to
empathise. Don’t say ‘I know how you
feel’. It’s too easy for other person to say
‘Oh no you don’t’.
Wrong: Instead, show you care and create
opportunities for dialogue. You don’t need
sophisticated technology for this. In fact,
‘I know how you feel’ it’s been around since the 1960s.
Right:
‘I’ve never been in your situation
before, but I imagine you must be
feeling very depressed and lonely’
@gilescolborne

35. Eliza is a computer ‘therapist’ that asks
users how they feel and uses pattern
matching to respond and draw them out.
ELIZA
It’s crude, but good enough to pass a basic
Turing test - some people think Eliza’s
answers come from a real person.
@gilescolborne

36. FBI negotiators know they
need to project the right
Be positive, upbeat personality.
Reassure hostage-taker that Now imagine if you hooked
up Eliza’s pattern matching
to an online database and
things will work out well gave it an upbeat
personality with a bit of
edge to it.
Be credible That sounds familiar...
Show you understand their
reasons but don’t be too eager to
please
@gilescolborne

37. Siri is an evolution of Eliza’s
pattern matching approach
but with better jokes.
That creates a personality
and a basis for empathy.
@gilescolborne

38. Clifford Nass ran an
experiment where he gave
participants blue wrist bands
and asked them to complete
tasks a computer.
For half the participants he
put ablue border on the
computer screen and said
‘you and the computer arethe
blue team’. For the other half,
he gave the computer a green
borderand said ‘you’re the
blue guy working on the
green computer’.
When the colours matched,
people tried harder and
thought the computer was
smarter.
Building rapport doesn’t
require complex technology.
Just good psychology.
@gilescolborne

39. Excellent. We’re half way up
the FBI’s behavioural
change model and we’ve not
had to build an artificial
brain.
Listen and empathise
@gilescolborne

40. The FBI has a lot to tell us
about how to handle
emotions that arise from
situation.
But sometimes conflict
arises from personality
differences.
@gilescolborne

41. For managing relationships,
this book was
recommended to me.
I love it because it centres
on a simple model. (Which
we can use when we’re
designing interactions.)
@gilescolborne

42. You have to understand
Task focus
people’s disposition
(passive - aggressive) and
motivation (task -
relationship).
It can’t be done Tank
Passive Aggressive
Yes person Think they know it all
Relationship focus @gilescolborne

43. In the centre is the ‘normal zone’. At the
Task focus
edges of the graph are extreme types
who can be difficult to get along with.
Whiner Tank
Passive Aggressive
Yes person Think they know it all
Relationship focus @gilescolborne

44. What I find normal and acceptable will
Task focus
be different from what you find normal
and acceptable.
Everyone is someone’s difficult person.
Whiner Tank
We need to tune our behaviour and
responses to get the best out of those
conflicts.
Passive Aggressive
Yes person Think they know it all
Relationship focus @gilescolborne

45. And the secret to that is understanding
Task focus
the intention that drives those
personality types.
Get it right Get it done
Passive Aggressive
Get along Get appreciated
Relationship focus @gilescolborne

46. Maja Mataric has built socially assistive
robots that coach stroke victims
through their physiotherapy.
She tried tuning the coaching to each
patient’s personality. The robots would
tell extroverts ‘Come on, try harder’.
But introverts would be told ‘I know it’s
hard, but it’s for your own good’.
And... it worked! Patients preferred the
robots that were tuned to their
personality and tried harder for them.
Maja Mataric
@gilescolborne

47. You don’t need to give your users a
personality test before they start. You
could learn their personality in the
same way that Pandora learns your
taste in music.
@gilescolborne

48. And you might even be able to pick
up information about users’
personality from specialist services.
@gilescolborne

49. Task focus
Get it right Get it done
Passive Aggressive
Get along Get appreciated
Relationship focus @gilescolborne

50. If you need to be appreciated, you’ll like
the fact that TripAdvisor tells you when
people have read your reviews.
If you’re a ‘get it done’ kind of person,
you might feel this was unnecessary and
spammy.
@gilescolborne

51. And if you’re the kind of person
who needs to get along, you’d
appreciate this error message that
says ‘It’s my fault’. (Personally, I
find it rather craven.)
@gilescolborne

52. Task focus So instead of designing fixed
patterns of behaviour, maybe we
should design flexible patterns
that adjust to users’ disposition.
Get it right Get it done
Passive Aggressive
Get along Get appreciated
Relationship focus @gilescolborne

53. Behavioural Change Stairway Model
And we’ve seen how a model
like this can help us think about behaviour
listening, empathising and
building rapport, rather than change
rushing towards outcomes. influence
rapport
empathy
active
listening
@gilescolborne

54. A lot of the discussion of emotional
design today centres around ideas of
‘brand’. And on old-fashioned, static,
monolithic brands.
But I hope I’ve shown you that we can do
much more.
We can create a flow of emotions and
more dynamic, adaptable personalities
for our designs.
We’re designing responsive web layouts,
why not responsive interaction rules?
@gilescolborne
http://en.wikipedia.org/wiki/File:Coca_Cola_ad_ca._1943_IMG_3744.JPG

55. If we put some emotional intelligence into the
interactions we design, we can disarm conflict,
create richer, more memorable experiences,
and improve performance for our users.
The future may bring us better ways of reading
emotions.
But the tools we need to get started are
already in our hands.
Let’s use them.
@gilescolborne

56. By the way, Samsung just patented this. So
soon your smartphone will be able to use its
camera to detect your emotion - one of six
‘universal’ facial expressions that correspond
to common emotions.
@gilescolborne

57. Researchers have identified six or
seven universal emotions. But not
everyone who sees these photos
has the same experience.
@gilescolborne

58. When you see the fear in others, your
amygdala lights up. You experience stress. That
empathy stops you from wanting to dominate
others by manipulating them.
Psychopaths don’t have this response. They
become charming manipulators.
This is the final part of emotional intelligence:
empathy. Without it you’re just a manipulator.
@gilescolborne

59. Which brings us back to this. Tom Cordell
tracked down the architects of the 1950s and
asked them what they thought they were
doing.
@gilescolborne

60. We were trying to build
heaven on earth
Their dream of building tower blocks
surrounded by parks was subverted by the
money men who used the space to cram in
more roads, shops and tower blocks.
We should learn from their lesson.
@gilescolborne

61. We need to make sure that we, the companies
we deal with, and the systems we create feel
for the users we’re serving.
If we don’t we’ll end up using these powerful
tools to manipulate them.
And we’ll be no better than the architects of
1950s Britain who, like us, thought they were
creating a better world.
@gilescolborne

62. What are emotions and how Still the go-to text on using emotions Applying the triune brain model How law enforcement agencies
could we create them in a in computing. to the problem of creating negotiate in highly charged,
machine? emotionally resonant designs. criminal situations.
A guide to recognising A refutation of the idea that How your mood - and even your Applying theories of story to
emotionally charged relationships emotions have no place in higher facial expressions can affect your interaction design
and to getting the best from thought. experience of a situation.
them.
@gilescolborne