SlideShare una empresa de Scribd logo
1 de 39
Descargar para leer sin conexión
VIRTUAL PATIENTS
Important aspects for
lifelike characters:
Spoken Dialogue
Thomas B. Talbot, MD, MS, FAAP
Principal Medical Expert
Institute for Creative Technologies
University of Southern California
Talbot@ict.usc.edu
Associate Research Professor of Medical Education
Keck School of Medicine of USC
Serious Play Conf July 10th, 2018
LifeLike Virtual
Patients
• Graphical Appearance
• Scenario Design & Narrative
• Interactivity
– User Interface
– Scenario Logic
– Biological Fidelity
– Speech & Dialogue
• Assessment & Feedback
AGENDA
• Focus on interactive dialogue in games
• Conversational Simulations
• Dialogue agents and conversation design patterns
• Why this is important for games of the future…
Social Simulations
Dialogue Approaches
• Language Parser
• Simple Response
• Interactive Choice-Based Dialog
– Linear
– State Based
– Branching (Dialogue Tree)
• NLU AI Processing
– NLRA
– Goal
PARSER
Simple Response
Who ate the Doritos?
Choice-based Dialogue
Linear
Branching
Branching Judgment
Linear w/ Choices
Virtual Child Witness
Unlocker
NLRA CONVERSATIONS
• Natural Language Random Access
• Natural Language Understanding (NLU) AI Dependent
– Requires training language
– Probabilistic, not deterministic
• It is not possible to have a full scope human conversation
• Success depends on CONSTRAINTS
– Constrain domain: narrower means better performance
– Constrain initiative: exchanging initiative difficult
– Constrain scope: Go broad or deep, not both
• Lots of dialogue interaction patterns
• Possible to mix choice-based with NLRA
USC Standard Patient
NLRA
• 92%+ NLU Performance
• Voice Recognition decreases
NLU performance
• Strong Training Effect
• Time/Choice constraints on
users IMPROVES
performance
• People say weird things to
computers
Some things never change
Lessons & The Future
• Medical Interview Lessons
• Assessment Lessons
• Live Feedback
• Enhancers
– Dubbed audio, Scanned characters, custom taxonomy, etc…
• Mixing methodologies
• Perceptive
• MultiState Dialogue Agents (Physiology Construction Set)
7/30/2018 (C)2017 Medical Mechanica 36
7/30/2018 (C)2017 Medical Mechanica 37
GAME OVER
 -State of the Living – Medical Games & Lifelike Patients - Thomas Talbot, USC Institute for Creative Technologies

Más contenido relacionado

Similar a -State of the Living – Medical Games & Lifelike Patients - Thomas Talbot, USC Institute for Creative Technologies

1. Describe physical security threats to the United States as a re.docx
1. Describe physical security threats to the United States as a re.docx1. Describe physical security threats to the United States as a re.docx
1. Describe physical security threats to the United States as a re.docx
jeremylockett77
 
James pengelley real speaking
James pengelley real speakingJames pengelley real speaking
James pengelley real speaking
Onlineconference
 
Aac October 2008
Aac   October 2008Aac   October 2008
Aac October 2008
Jantioco1
 
Heres-What-Personality-Sounds-Like
Heres-What-Personality-Sounds-LikeHeres-What-Personality-Sounds-Like
Heres-What-Personality-Sounds-Like
Marcel Korst
 
Research communication skills & public intellectualism in graduate education
Research communication skills & public intellectualism in graduate educationResearch communication skills & public intellectualism in graduate education
Research communication skills & public intellectualism in graduate education
Jennifer Englund
 

Similar a -State of the Living – Medical Games & Lifelike Patients - Thomas Talbot, USC Institute for Creative Technologies (20)

Marriage of speech, vision and natural language processing
Marriage of speech, vision and natural language processingMarriage of speech, vision and natural language processing
Marriage of speech, vision and natural language processing
 
Introduction to My Research and Laboratory
Introduction to My Research and LaboratoryIntroduction to My Research and Laboratory
Introduction to My Research and Laboratory
 
When your persona talks: Mitigating linguistic bias in voice interaction design
When your persona talks: Mitigating linguistic bias in voice interaction designWhen your persona talks: Mitigating linguistic bias in voice interaction design
When your persona talks: Mitigating linguistic bias in voice interaction design
 
Chapters 5 8
Chapters 5 8Chapters 5 8
Chapters 5 8
 
Workshop: To be Engineers with Empathy
Workshop: To be Engineers with EmpathyWorkshop: To be Engineers with Empathy
Workshop: To be Engineers with Empathy
 
1. Describe physical security threats to the United States as a re.docx
1. Describe physical security threats to the United States as a re.docx1. Describe physical security threats to the United States as a re.docx
1. Describe physical security threats to the United States as a re.docx
 
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2
 
Chapters 5 8
Chapters 5 8Chapters 5 8
Chapters 5 8
 
Making Voting Accessible
Making Voting Accessible Making Voting Accessible
Making Voting Accessible
 
James pengelley real speaking
James pengelley real speakingJames pengelley real speaking
James pengelley real speaking
 
Aac October 2008
Aac   October 2008Aac   October 2008
Aac October 2008
 
MATC Scholars Program: Amy O. Maki
MATC Scholars Program: Amy O. MakiMATC Scholars Program: Amy O. Maki
MATC Scholars Program: Amy O. Maki
 
Heres-What-Personality-Sounds-Like
Heres-What-Personality-Sounds-LikeHeres-What-Personality-Sounds-Like
Heres-What-Personality-Sounds-Like
 
Accessible Everyone
Accessible EveryoneAccessible Everyone
Accessible Everyone
 
Autism:Technology & Communication - Part 1, Univ. of Redlands, 11.15.14
Autism:Technology & Communication - Part 1, Univ. of Redlands, 11.15.14Autism:Technology & Communication - Part 1, Univ. of Redlands, 11.15.14
Autism:Technology & Communication - Part 1, Univ. of Redlands, 11.15.14
 
2023 MATC Scholars Program: Communication Skills
2023 MATC Scholars Program: Communication Skills2023 MATC Scholars Program: Communication Skills
2023 MATC Scholars Program: Communication Skills
 
Deep Learning for Dialogue Modeling - NTHU
Deep Learning for Dialogue Modeling - NTHUDeep Learning for Dialogue Modeling - NTHU
Deep Learning for Dialogue Modeling - NTHU
 
Research communication skills & public intellectualism in graduate education
Research communication skills & public intellectualism in graduate educationResearch communication skills & public intellectualism in graduate education
Research communication skills & public intellectualism in graduate education
 
2103 ACM FAccT
2103 ACM FAccT2103 ACM FAccT
2103 ACM FAccT
 
Symbol Emergence in Robotics: Language Acquisition via Real-world Sensorimoto...
Symbol Emergence in Robotics: Language Acquisition via Real-world Sensorimoto...Symbol Emergence in Robotics: Language Acquisition via Real-world Sensorimoto...
Symbol Emergence in Robotics: Language Acquisition via Real-world Sensorimoto...
 

Más de SeriousGamesAssoc

The Value of Robotics for Surgery Training
The Value of Robotics for Surgery Training The Value of Robotics for Surgery Training
The Value of Robotics for Surgery Training
SeriousGamesAssoc
 
Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...
Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...
Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...
SeriousGamesAssoc
 
OODA OODA! How Rapid Iteration Can Help Level Up Your Gaming Business
OODA OODA! How Rapid Iteration Can Help Level Up Your Gaming BusinessOODA OODA! How Rapid Iteration Can Help Level Up Your Gaming Business
OODA OODA! How Rapid Iteration Can Help Level Up Your Gaming Business
SeriousGamesAssoc
 
The ux of serious games how to impact a wider audience
The ux of serious games  how to impact a wider audienceThe ux of serious games  how to impact a wider audience
The ux of serious games how to impact a wider audience
SeriousGamesAssoc
 
Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games
Alphabet Soup Cans: Avoiding Bad Tropes of Educational GamesAlphabet Soup Cans: Avoiding Bad Tropes of Educational Games
Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games
SeriousGamesAssoc
 
Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...
Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...
Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...
SeriousGamesAssoc
 
Augmented Reality: Revolutionary or Disruptor of Training and Assessment
Augmented Reality: Revolutionary or Disruptor of Training and AssessmentAugmented Reality: Revolutionary or Disruptor of Training and Assessment
Augmented Reality: Revolutionary or Disruptor of Training and Assessment
SeriousGamesAssoc
 
Enhancing New Employee Orientation with a Digital Scavenger Hunt
Enhancing New Employee Orientation with a Digital Scavenger HuntEnhancing New Employee Orientation with a Digital Scavenger Hunt
Enhancing New Employee Orientation with a Digital Scavenger Hunt
SeriousGamesAssoc
 

Más de SeriousGamesAssoc (20)

The Value of Robotics for Surgery Training
The Value of Robotics for Surgery Training The Value of Robotics for Surgery Training
The Value of Robotics for Surgery Training
 
Developing Low Cost Simulation Immersions
Developing Low Cost Simulation ImmersionsDeveloping Low Cost Simulation Immersions
Developing Low Cost Simulation Immersions
 
Finding Partners in Applied Research – A Case Study on Industry/Academic Coll...
Finding Partners in Applied Research – A Case Study on Industry/Academic Coll...Finding Partners in Applied Research – A Case Study on Industry/Academic Coll...
Finding Partners in Applied Research – A Case Study on Industry/Academic Coll...
 
Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...
Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...
Designing Immersive Experiences that Create Empathy, Reveal Biases, Alter Min...
 
Designing Memorable Games
Designing Memorable GamesDesigning Memorable Games
Designing Memorable Games
 
Preparing Soldiers for the Future: The Army's New Synthetic Training Environment
Preparing Soldiers for the Future: The Army's New Synthetic Training EnvironmentPreparing Soldiers for the Future: The Army's New Synthetic Training Environment
Preparing Soldiers for the Future: The Army's New Synthetic Training Environment
 
Learning Through Play: STEM Games in the Classroom
Learning Through Play: STEM Games in the ClassroomLearning Through Play: STEM Games in the Classroom
Learning Through Play: STEM Games in the Classroom
 
Playable Design
Playable DesignPlayable Design
Playable Design
 
Playful Learning Without Games
Playful Learning Without GamesPlayful Learning Without Games
Playful Learning Without Games
 
Client-Centered Serious Game Design
Client-Centered Serious Game DesignClient-Centered Serious Game Design
Client-Centered Serious Game Design
 
A Promise to Future Generations: Making Learning Fun
A Promise to Future Generations: Making Learning FunA Promise to Future Generations: Making Learning Fun
A Promise to Future Generations: Making Learning Fun
 
Lets make a game workshop ser play orlando 2019
Lets make a game workshop ser play orlando 2019Lets make a game workshop ser play orlando 2019
Lets make a game workshop ser play orlando 2019
 
0 for 3: Edtech Startup Lessons Learned
0 for 3: Edtech Startup Lessons Learned0 for 3: Edtech Startup Lessons Learned
0 for 3: Edtech Startup Lessons Learned
 
OODA OODA! How Rapid Iteration Can Help Level Up Your Gaming Business
OODA OODA! How Rapid Iteration Can Help Level Up Your Gaming BusinessOODA OODA! How Rapid Iteration Can Help Level Up Your Gaming Business
OODA OODA! How Rapid Iteration Can Help Level Up Your Gaming Business
 
The ux of serious games how to impact a wider audience
The ux of serious games  how to impact a wider audienceThe ux of serious games  how to impact a wider audience
The ux of serious games how to impact a wider audience
 
Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games
Alphabet Soup Cans: Avoiding Bad Tropes of Educational GamesAlphabet Soup Cans: Avoiding Bad Tropes of Educational Games
Alphabet Soup Cans: Avoiding Bad Tropes of Educational Games
 
Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...
Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...
Building Heroes: Using Roleplaying Game Design for Classroom Management, Read...
 
Augmented Reality: Revolutionary or Disruptor of Training and Assessment
Augmented Reality: Revolutionary or Disruptor of Training and AssessmentAugmented Reality: Revolutionary or Disruptor of Training and Assessment
Augmented Reality: Revolutionary or Disruptor of Training and Assessment
 
Return on Investment (ROI) for Virtual Environments and Gaming
Return on Investment (ROI) for Virtual Environments and GamingReturn on Investment (ROI) for Virtual Environments and Gaming
Return on Investment (ROI) for Virtual Environments and Gaming
 
Enhancing New Employee Orientation with a Digital Scavenger Hunt
Enhancing New Employee Orientation with a Digital Scavenger HuntEnhancing New Employee Orientation with a Digital Scavenger Hunt
Enhancing New Employee Orientation with a Digital Scavenger Hunt
 

Último

Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
heathfieldcps1
 

Último (20)

SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptxSKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
Single or Multiple melodic lines structure
Single or Multiple melodic lines structureSingle or Multiple melodic lines structure
Single or Multiple melodic lines structure
 
Google Gemini An AI Revolution in Education.pptx
Google Gemini An AI Revolution in Education.pptxGoogle Gemini An AI Revolution in Education.pptx
Google Gemini An AI Revolution in Education.pptx
 
Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024
 
Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...
 
Unit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxUnit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptx
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentation
 
Mixin Classes in Odoo 17 How to Extend Models Using Mixin Classes
Mixin Classes in Odoo 17  How to Extend Models Using Mixin ClassesMixin Classes in Odoo 17  How to Extend Models Using Mixin Classes
Mixin Classes in Odoo 17 How to Extend Models Using Mixin Classes
 
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual  Proper...General Principles of Intellectual Property: Concepts of Intellectual  Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 
Graduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - EnglishGraduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - English
 
Dyslexia AI Workshop for Slideshare.pptx
Dyslexia AI Workshop for Slideshare.pptxDyslexia AI Workshop for Slideshare.pptx
Dyslexia AI Workshop for Slideshare.pptx
 
ComPTIA Overview | Comptia Security+ Book SY0-701
ComPTIA Overview | Comptia Security+ Book SY0-701ComPTIA Overview | Comptia Security+ Book SY0-701
ComPTIA Overview | Comptia Security+ Book SY0-701
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
 
On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
 

-State of the Living – Medical Games & Lifelike Patients - Thomas Talbot, USC Institute for Creative Technologies

  • 1. VIRTUAL PATIENTS Important aspects for lifelike characters: Spoken Dialogue Thomas B. Talbot, MD, MS, FAAP Principal Medical Expert Institute for Creative Technologies University of Southern California Talbot@ict.usc.edu Associate Research Professor of Medical Education Keck School of Medicine of USC Serious Play Conf July 10th, 2018
  • 2.
  • 3.
  • 4.
  • 5. LifeLike Virtual Patients • Graphical Appearance • Scenario Design & Narrative • Interactivity – User Interface – Scenario Logic – Biological Fidelity – Speech & Dialogue • Assessment & Feedback
  • 6. AGENDA • Focus on interactive dialogue in games • Conversational Simulations • Dialogue agents and conversation design patterns • Why this is important for games of the future…
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 16.
  • 17. Dialogue Approaches • Language Parser • Simple Response • Interactive Choice-Based Dialog – Linear – State Based – Branching (Dialogue Tree) • NLU AI Processing – NLRA – Goal
  • 20.
  • 21.
  • 22. Who ate the Doritos?
  • 30. NLRA CONVERSATIONS • Natural Language Random Access • Natural Language Understanding (NLU) AI Dependent – Requires training language – Probabilistic, not deterministic • It is not possible to have a full scope human conversation • Success depends on CONSTRAINTS – Constrain domain: narrower means better performance – Constrain initiative: exchanging initiative difficult – Constrain scope: Go broad or deep, not both • Lots of dialogue interaction patterns • Possible to mix choice-based with NLRA
  • 31. USC Standard Patient NLRA • 92%+ NLU Performance • Voice Recognition decreases NLU performance • Strong Training Effect • Time/Choice constraints on users IMPROVES performance • People say weird things to computers
  • 32.
  • 33.
  • 35. Lessons & The Future • Medical Interview Lessons • Assessment Lessons • Live Feedback • Enhancers – Dubbed audio, Scanned characters, custom taxonomy, etc… • Mixing methodologies • Perceptive • MultiState Dialogue Agents (Physiology Construction Set)