Facebook, Google and Microsoft are betting the farm on “deep learning” artificial intelligence. Alexa, Siri, Cortana and Google Assistant demonstrate interfaces need not be screen based. Welcome to the era of Cognitive Design. Marketers, Product Owners, and Experience Designers need new skills to compete. You don’t need to a PhD Psychologist and Machine Learning Computer Scientist (though it doesn’t hurt). But you do need to move beyond traditional user experience research and design thinking empathy. Step 1: We will begin by defining the new “Zero UI” world which includes new possibilities with zero ui, machine learning, and new interaction possibilities. We want to show the intensely human response to AI: Ever notice that in a sentence people call a chair “it”, but call Siri or Cortana “she”? Step 2: Given those possibilities, we will describe AI/narrow AI, AR, given and provide exposure to what machine learning is, and provide a sense of what a training set might be like, and how to test the new tool. Step 3: Determine how best to augment cognition with AI. We will provide several examples and demonstrate how to train and test an augmented experience. We will consider which modalities and interfaces to use, and how best to augment cognition with AI for the optimal experience. Step 4: Show participants best practices and tips & tricks to conduct usability tests with these AI tools and show how these techniques differ from classic usability testing. Given most participants will have never had exposure to this, we make sure we go slow, provide examples, and show that most audience members are using this several times a day (e.g., Netflix, Google Search, Facebook Chatbots, etc.). Providing concrete examples will help to make concrete this new world. Find out how you need to change your UX/CX practice and start doing Cognitive Design today!

1. The Future of UX is here:
AI and Cognitive Design
John Whalen, Ph.D.
10Pearls
+

2. 10Pearls Acquires Brilliant Experience
+
“Intelligent Experiences”
Business reimagined
Significant development
AI-based
Experience reimagined
Significant research & testing
Innovation-based
+

3. Contents
1. Why AI is so different
2. Cognitive Design framework
3. AI Experience Study
4. Live Demo
5. Results
6. Implications for UXers

4. 1. Why AI is so different
@johnwhalen #uxdc2017

5. Challenge 1: What constitutes a success?

6. “Okay Google,
show me a
waterfall.”

7. “Okay Google,
order me a pizza.”
“Here are some
nearby pizza
restaurants…”

8. Components to
measuring AI
success:
1. Command words
and word order
2. Answer accuracy
3. Situational
relevancy
@johnwhalen #uxdc2017

9. Challenge 2: Humanizing experience

10. “Ask Alexa to play some music”
“Okay, I’ll ask her”

11. Inanimate objects

12. Pronouns given
It It It
Her Him Her
@johnwhalen #uxdc2017

13. “Hey Siri,
do you love me?”

14. Human emotion
important too.
1. Emotional
response
2. Conversational
tone
3. Situational
awareness
@johnwhalen #uxdc2017

15. 2. Cognitive Design framework

16. Language Wayfinding
Vision /
Attention
MemoryEmotion
Decision
Making
@johnwhalen #uxdc2017
The six minds of user experience

17. Language Wayfinding
Vision /
Attention
MemoryEmotion
Decision
Making
@johnwhalen #uxdc2017
Cognitive Design for AI-human interactions
How did
user form the
command?
Could user interact
successfully?
Did AI match
expectations?
What was user’s
emotional response?
Did the
answer help?
Did AI gather
the results well?

18. 3. AI Experience Study

19. AI systems tested
Siri Cortana Google
Assistant
Alexa

20. Prompt:
Chicago, second
tallest building
Participant:
“Alexa, what is the
second tallest
building in
Chicago?”
@johnwhalen #uxdc2017

21. Types of requests
1. Simple facts
2. Complex facts
3. Commands
4. Games/Jokes
5. Personal Info
@johnwhalen #uxdc2017

22. Mix of ages
Younger Older

23. Mix of experience with AI
AI experience No AI experience

24. Mix of tech savviness
Techie Non-techie

25. Mix of English fluency
Native English speaker
Non-native English
speaker

26. 4. Live Demo

27. Person,
second moon walk.

28. Weather, Richmond
day after tomorrow.

29. Country population,
has the Eiffel Tower.

30. Aliens, existence.

31. Like me?

32. Coin flip.

33. Captured:
1. Request uttered
2. AI response
3. Facial expression
4. Heart rate
5. GSR (sweat)
@johnwhalen #uxdc2017

34. 5. Results

35. Performance by age
64% 70% 67% 64%
@johnwhalen #uxdc2017

36. Set an alarm

37. Hotel near MOMA in NYC

38. Hotel near MOMA in NYC

39. Which performed the best?
76% 68% 61% 61%
@johnwhalen #uxdc2017
Siri Cortana
Google
Assistant
Alexa

40. Get me an Uber

41. Population of country with Eiffel Tower

42. “Storks” movie on Saturday in Bethesda

43. “Storks” movie on Saturday in Bethesda

44. Do you like me?

45. Do aliens exist?

46. 24hr grocery store near me

47. Set an alarm for 7am tomorrow

48. AI UX Discoveries:
1. Timing is difficult.
2. Saying tool name
is hard.
3. Command styles
vary widely.
4. Using button to
initiate speech is
more natural.
@johnwhalen #uxdc2017

49. Which performed the best?
76% 68% 61% 61%
@johnwhalen #uxdc2017
Siri Cortana
Google
Assistant
Alexa

50. The most accurate AI tools
are not always preferred.
Emotion matters!
@johnwhalen #uxdc2017

51. The least accurate is second favorite???
76% 61% 68% 61%
@johnwhalen #uxdc2017
FAVORITEACCURACY
38% 33% 5% 5%
Siri CortanaGoogle
Assistant
Alexa

52. From those who preferred Alexa:
“Like them to have
personality. Jokes are
appropriate.”
“Alexa feels like someone.
It acknowledges you.
Alexa just sounds nicer.”
“Alexa feels like it is addressing
me back. Feels like I am
interacting with a person.”
“Prefer Alexa because
it tells me its not sure.”
“I like when it answers the
question the way I worded it.”
“Responses were direct.
Fun to banter back and
forth.”
@johnwhalen #uxdc2017

53. From those who preferred Google:
“Annoying when they are
being human. I don't feel
anything with them.”
“I like the casual
conversation with Google
the most but I like Alexa's
“I like that the answers are
accurate.”
“It is weird that they
have a personality and
can already tell a joke.”
“Just give me the answer”
“I want general information
and answers fast.”
@johnwhalen #uxdc2017

54. Emotional design
discoveries:
1. Say “I don’t know.”
2. Match the way I
talk.
3. Sound quality
matters.
@johnwhalen #uxdc2017

55. When using their favorite tool:
FAVORITE HEART
RATE GSR +VE FACIAL
EXPRESSION
@johnwhalen #uxdc2017

56. We can capture a positive
experience signature.
@johnwhalen #uxdc2017

57. 6. Implications for UX
@johnwhalen #uxdc2017

58. UX must
adapt to AI
1. Study the command
words carefully.
2. Measure the AI
answers for accuracy.
3. Measure the users’
emotional signature.
4. Focus on the
interaction timing.
5. Think about how to
humanize the
interaction.
@johnwhalen #uxdc2017

59. UX has an old role in a new setting:
Sense making

61. Where/what are the objects?

62. What is the name of the object?

63. Where is the object in space?

64. Taxonomies and semantics
may be key to success.
@johnwhalen #uxdc2017

65. In Summary

66. Language Wayfinding
Vision /
Attention
MemoryEmotion
Decision
Making
@johnwhalen #uxdc2017
We have to think like a psychologist
How did
they form the
command?
Could they interact
successfully?
Did it match
expectations?
What was the
emotional response?
Did the
answer help?
Did it gather the
results well?

67. UX must
adapt to AI
1. Study the command
words carefully.
2. Measure the AI
answers for accuracy.
3. Measure the users’
emotional signature.
4. Focus on the
interaction timing.
5. Think about how to
humanize the
interaction.
@johnwhalen #uxdc2017

68. Innovation & Design Practice
Job Openings:
- Sr. UX: Research & Insights
- Sr. UX: Strategy & Design
10Pearls.com

69. We help companies develop world-class
digital products and services.
Founded in 2004
300+ employees
Award-winning
Socially responsible company
Global Co. (DC, SF, Toronto, Dubai, Karachi)