How can chatbots learn from existing VUI design? What makes these new interfaces different, and how are they similar? Where do the Alexas and Siris come into the mix? We’ll discuss text-based vs. voice-based conversational user interfaces, and the landscape of Conversational User Interfaces, now and into the future.

1. Conversational User
Interfaces: Past and Future
Crispin Reedy @crispinTX

2. 2017 goals
1. Provide more UX resources
to UX professionals.
2. Improve brand consistency.
3. Improve “International”
UXPA.
User Experience Professionals
Association (UXPA)
Monthly webinars
User Experience Magazine
The Journal of Usability Studies
Visit www.uxpa.org for more info
about the conference, webinars, and
other UX resources!
Annual Conference in
Toronto, Canada
June 5 – 8, 2017
uxpa2017.org

3. Conversational User
Interfaces: Past and Future
Crispin Reedy @crispinTX

4. 4© 2017 Versay Solutions LLC
• Voice User Interface Designer
• 15+ years in the field
• Former coder; got interested in UX
• President of the Association for Voice
Interaction Design
• Consultant for Versay Solutions
@crispinTX
crispinreedy.com

5. CUI: Past and Future
• How can chatbots learn from existing VUI
design? What makes these new interfaces
different, and how are they similar? Where
do the Alexas and Siris come into the mix?
We’ll discuss text-based vs. voice-based
conversational user interfaces, and the
landscape of Conversational User
Interfaces, now and into the future.
5© 2017 Versay Solutions LLC

6. Disclaimers
• What, exactly, is a Conversational User
Interface?
– For this discussion we are going broad and
wide.
• Conversation:
– Interactive communication between two or
more people
• Spoken vs. written (chatbots)
• Turntaking vs. one shot
6© 2017 Versay Solutions LLC

17. Flash-forward to 2017….
17© 2017 Versay Solutions LLC

18. “I’m really excited about chat and natural
language because there’s no user interface
you have to deal with.”
- Chatbot writer on support board

21. “Just this week, Facebook said it was “refocusing” its use of AI
after its bots hit a failure rate of 70 percent,
meaning bots could only get to 30 percent of requests without
some sort of human intervention.”
“I would call it overpromising,” said CP+B executive creative tech director Joe Corr.
“Brands that created bots with a structured request or utility like Domino’s or in retail
were easy. But bots that tried to break out of the utility and be
chatbots became the problem.”

22. 22© 2017 Versay Solutions LLC
Chatbots
Voice
Systems
Taxonomy of CUIs
Input Modality

23. 23© 2017 Versay Solutions LLC
Chatbots
Voice
Systems
Taxonomy of CUIs
Input Modality
Domain Domain
•Personal Assistants
•Agents
•General Search
•Order Products
•Troubleshooting
•Manage Accounts
•Alexa
•Google Home
•Wildfire
•IVR
•Billing information
•Control-based

24. 24© 2017 Versay Solutions LLC
Voice
Systems
Taxonomy of CUIs
Domain
•Alexa
•Google Home
•Wildfire
•IVR
•Troubleshooting
•Manage Accounts
Technology
•What “thing” are you
talking to?
•What else are you
connected to?
(Network, data)
•What kind of speech
technology are you
using?
•What other input and
output modalities are
you coordinating
with?

25. What Are You Talking To?
25© 2017 Versay Solutions LLC

26. Voice Conversation vs. Chat
Conversation
• Differences
– Written convention vs. spoken convention
– Speech recognition vs. textual understanding
• ASR vs. NLU
• DNN, Machine Learning
– Chatbot implies keyboard; voice interface
could be to almost anything
• This becomes really important:
– Discoverability
– Onboarding
– Coordinating voice with other modalities

27. Voice Conversation vs. Chat
Conversation
• Commonalities
– Logical structure of a conversation (flow)
– Error recovery
• If caller / user goes off topic
– Prompts elicit expected responses
• Sometimes!
– Conversational maxims
• English is still English! Well, sort of.

28. UX Still Applies!
• Typical Tools:
– Tasks
– Personas
– “Happy Path” High Level Conversations
(Script)
– Card sorts?
– Read Aloud
– Usability Testing
• Wizard of Oz
– A/B Testing
28© 2017 Versay Solutions LLC

29. Commonalities

30. Voice Conversational Model: Beginning
NL Level 1:
Open Ended
Question
• Hi, how may I help you
today?
• I’m calling about a
problem with my phone.
• Sure, what kind of
problem are you
having?
• I don’t seem to have
any internet.
• I can help you with that.
Are you calling about
the Apple iPhone 6 that
you have on your plan?
• Yes
NL Level 2:
Identify Problem
Directed Dialog:
ID Phone
Implicit: ID Caller

31. Chat Conversational Model: Beginning
NL Level 1:
Open Ended
Question
• Hi, how may I help
you today?
• I have a prob w/
phone
• Sure, what kind of
problem?
• No internet connectin
• OK. Is that about the
Apple iPhone 6 you
have on your plan?
• Yep
NL Level 2:
Identify Problem
Directed Dialog:
ID Phone
Implicit: ID Caller

32. Voice Conversational Model:
Troubleshooting
• So, first let’s check
your settings. What
about Airplane
Mode… do you have
that turned off? If
you need help with
that, say “tell me
more.”
• Tell me more
• Sure. You can find
Airplane Mode under
“Settings.” (call
continues)
Step 1 + Fallback

33. Chat Conversational Model: Troubleshooting
• Let’s check settings.
Do you have
Airplane Mode
turned off?
• IDK
• OK to check
Airplane Mode, go
to “Settings.”
Step 1 + Fallback

34. Error Recovery - Voice
• If the recognizer doesn’t get something,
you may have to reprompt.
– How crucial is the information you needed to
collect in order to move the conversation
forward?
• Eg: Alexa doesn’t always reprompt
– Systems can be more directive if failure
continues
34© 2017 Versay Solutions LLC
“What color is that light?”
It’s… <noise>
“Sorry, what color?”
“Sorry, what color?”
<silence>
“You can say ‘red’ or press 1, ‘green’ or press 2, …. (etc)

35. Error Recovery - Chatbot
• Chatbots do need it!
– Do you need to do a local recovery vs. a
global recovery?
– Specific point
– Total reset
35Source: Chatbot.fail

36. Prompts and Responses
• Prompts imply more than choices
– Would you like chocolate or vanilla?
• Yes
• Both
• Inputs are more like text boxes
36© 2017 Versay Solutions LLC

37. 37

38. Grice’s Maxims
Maxim Description
Quantity
Be as informative as is required, but not more
informative.
Quality
Do not make statements that you believe are false or for
which you lack evidence.
Relation Be relevant.
Manner
Avoid obscure expressions
Avoid ambiguity
Be brief
Be orderly

39. Paul Grice
• British philosopher of language
• “Logic and Conversation,” 1975
• The maxims describe the
“default” assumptions that
listeners make about how
people talk.
• When the assumptions are
broken:
– People look for underlying
implications
– People may assume that there
is a problem with the
conversation or speaker
Source: Amazon

40. Grice’s Maxims: Applied
• Be as informative as is required, but not
more informative.
– What is your user really wanting to do?
• Goals and tasks
– Am I telling the user too much during this
conversation?
• Unhelpful help.
• Overly chatty bots

41. Source: Taco Bell Bot (That’s Nerdalicious)

42. Grice’s Maxims: Applied
• Be relevant.
– Users want to do what they want, not what
you want them to do.
• Upselling
• Intrusive apps / bots
– Maintain state

43. 43Source: chatbots.fail

44. Grice’s Maxims: Applied
• Avoid obscure expressions
– No jargon
• Avoid ambiguity
– Ex: “This” and “that”
• Be brief
• Be orderly
– Organize information cleanly – ensure that IA
(if applicable) is understandable and relevant.

45. Differences

46. 46© 2017 Versay Solutions LLC
Voice
Systems
Taxonomy of CUIs
Domain
•Alexa
•Google Home
•Wildfire
•IVR
•Troubleshooting
•Manage Accounts
Technology
•Are you doing speech
recognition?
•What kind?
•What “thing” are you
talking to?
•What else are you
connected to?
•Network
•Data
•What other input
modalities are you
coordinating with?

47. Possibilities
47© 2017 Versay Solutions LLC
Write an app (skill) for
an agent such as
Cortana / Alexa / Jibo
Use cloud APIs to add
ASR to your app / device
/ page / gadget
Download an ASR engine
and use full-featured
capabilities for onboard
robust recognition
Build your own ASR /
DNN

48. Distributed: Today’s Speech Agents
• Siri
• Cortana
• Google Now
• Amazon Echo (Alexa)
48© 2016 Versay Solutions LLC

49. Alexa “Skills”
• “Alexa, ask Yelp to find me a restaurant.”
– Cortana has similar integration
• Register your skill with Amazon and
publish it
49© 2016 Versay Solutions LLC

50. Today’s Cloud-Based Speech APIs
• Distributed speech recognition
– Collection and compression of speech is on
the device
– The language models are typically on the
network
– Can be speaker-dependent
• Trains itself on your voice and on the acoustic
environments you are in most often
– Many companies are providing APIs to use
their speech recognition
50© 2016 Versay Solutions LLC

51. Speech API Example: Alexa Voice
Services
51© 2016 Versay Solutions LLC

52. Possible Approaches
Cloud-Based APIs
• Easy to get started
• Lightweight
• Not much
specialized
knowledge
• Limited
52© 2016 Versay Solutions LLC
Download / Embed / Roll
Your Own
• Customizable
• Can be device-specific
• More features
• Higher powered
• Will require specialized
knowledge
– Speech scientist
• Can use open source

53. Should I Speech-Enable X?
53© 2016 Versay Solutions LLC
Desktop App / Website
• Easy to get started with
API-based ASR
• But the use case may
not be as powerful
Tablet / Mobile
• Stronger use case
• But will the network be
available for APIs?
Industrial Device
• Great use case esp. with
multimodal
• But this is harder to do
and probably will be
custom
Gadget
• Decent use case
• APIs are tailored for this
• Will they do everything
you need?
• Will the extra modality
be a plus or just a “silly
add-on?”
• Different modalities?
Car
• Safety considerations
are high here
• Need better user
interfaces & more
robust
IVR
• Touchtone can still be
good for a lot of
applications
• Speech is good for
complex call routing and
input

54. Should I Speech-Enable X?
54

55. CUI: Past and Future
• How can chatbots learn from existing VUI
design? What makes these new interfaces
different, and how are they similar? Where
do the Alexas and Siris come into the mix?
We’ll discuss text-based vs. voice-based
conversational user interfaces, and the
landscape of Conversational User
Interfaces, now and into the future.
55© 2017 Versay Solutions LLC

56. Iron Man 2: Marvel Studios, Paramount Pictures

57. Resources
• The Voice in the Machine: Building
Computers that Understand Speech –
Roberto Pieraccini
– YouTube video: “Open the Pod Bay Doors, Siri”
• Conversational design from Google:
– https://developers.google.com/actions/design/
• Alexa VUI design
– Alexa Skills Kit Voice Design Handbook”
• Association for Voice Interaction Design -
AVIxD.org
– Wiki, Brown Bags, Journal
– Looking for members and volunteers!
57© 2017 Versay Solutions LLC

58. 58© 2017 Versay Solutions LLC
Thanks!
@crispinTX
crispinreedy.com
creedy@versay.com