Paper presentation: Partially observable Markov decision processes for spoken dialog systems Jason D. Williams, Steve Young (AT&T Labs) 2007, Computer Speech and Language, 21(2)

1. Partially observable Markov decision processes for spoken dialog systems Jason D. Williams, Steve Young (AT&T Labs) 2007, Computer Speech and Language, 21(2):

3. Partially observable Markov decision processes

4. Spoken Dialog System

5. SDS-POMDP

6. Comparing

7. Empirical support

10. A – set of actions, that agent may take

11. T – transition probability – P(s'|s, a)

12. R – reward – r(s, a)

13. O – set of observation about the world

14. Z – observation probability – P(o'|s', a)

16. b 0 – initial belief state b 0 (s)

18. □ - decision node

19. ◊ - utility node

20. Shaded – unobserved

21. | - causal effect

22.  - distribution is used

23. RL – reinforced learning

26. S d – dialog state (user view)

27. A u – user action (intention)

29. A u – action recognized by machine

30. C – confidence score

31. S m – dialog state (machine view) ~

33. Y m – machine audio signal

34. A m – action recognized by user ~

36. SDS = {S u , S d , S m , C, A u , A u , A m } ~

38. s m = b(s) = b(s u , a u , s d )

40. Effective approximate solutions exist.

42. CA are simplification or special case

44. Local use of confidence score

45. Automated action planning

47. Uncertainty -> multiple states

49. M-Best list

51. Doesn't perform plan

52. Handcrafting + ad hoc tunning

54. = POMDP with handcrafted action selection

55. Subspace of belief space

57. A c = {expl-confirm, imp-confirm, reject}

58. Useful, but hard for long-term goals

61. Markov decision processes

65. Assumes, that world state is known exactly

66. Single state

69. Confidence score

71. Parallel state hypotheses (1)

72. Parallel state hypotheses (2)

73. Parallel state hypotheses (3)

75. Confidence score (2)

76. Confidence score (3)

77. Confidence score (4)

79. HC2

80. HC3

81. Automated planning (2)

82. Automated planning (3)

84. Real data (2)

86. CA are simplification or special case

87. Scales purely

88. Unique

91. Questions?

92. Thank you! Thank you!