ACL 2015 - P15-3001 (Poster)

1.  Ontology Induction (Chen et al., 2013 & 2014)
Frame-semantic parsing on ASR results (Das et al., 2013)
• frame  slot candidate
• lexical unit  slot filler
 1st Issue: How to induce domain-specific concepts?
locale_by_use
food expensiveness
seeking
relational_quantity
PREP_FOR
PREP_FOR
NN AMOD
AMOD
AMOD
desiring
DOBJ
type
food pricerange
DOBJ
AMOD AMOD
AMOD
task
area
PREP_IN
• Domain: restaurant recommendation in an in-
car setting (WER = 37%)
o Dialogue slots: addr, area, food, phone,
postcode, pricerange, task, type speak on topic addr
area
food
phone
part orientational
direction
locale
part inner outer
food
origin
contacting
postcode
price
range
task
type
sending
commerce scenario
expensiveness
range
seeking
desiring
locating
locale by use
building
reference
ontology w/ the
most frequent
dependencies
Can a dialogue system automatically learn open
domain knowledge and then understand users?
Unsupervised Learning and Modeling of Knowledge and Intent for Spoken Dialogue Systems
Yun-Nung (Vivian) Chen
yvchen@cs.cmu.edu
Goal Framework
Restaurant
Asking
Conversations
target
food
price
seeking
quantity
PREP_FOR
PREP_FOR
NN AMOD
AMOD
AMOD
Domain-Specific Ontology
Unlabeled Collection
Knowledge Acquisition
Ontology Induction
Structure Learning
Domain-Specific Ontology
price=“cheap”, target=“restaurant”
behavior=navigation
SLU Modeling by MF
SLU Component
“can i have a cheap restaurant”
Semantic Decoding
Behavior Prediction
Knowledge Acquisition SLU Modeling by Matrix Factorization
 Structure Learning (Chen et al., 2015a)
Typed syntactic dependencies on ASR
https://github.com/yvchen/MRRW/
ccomp
amod
dobj
nsubj det
can i havea cheap restaurant
capability expensiveness locale_by_use
induced
ontology
 Semantic Decoding (Chen et al., 2015b)
• concept  semantic slot
 Behavior Prediction
• concept  user behavior
1) Given unlabeled conversations, how can a system
automatically induce and organize domain-specific
concepts?
2) With the automatically acquired knowledge, how
can a system understand utterances?
1
Utterance 1
i would like a cheap restaurant
Feature Observation Semantic Concept (Slot / Behavior)
Train
………
cheap restaurant foodexpensiveness
1
target
11
find a restaurant for chinese food
Utterance 2
1 1
food
1 1
1
Test
1
1
.97.90 .95.85
.93 .92.98.05 .05
Feature Relation Model Concept Relation Model
Reasoning with Matrix Factorization
Semantic Concept Induction
SLU Model
Semantic Representation
“can I have a cheap restaurant”
Ontology Induction
Unlabeled
Collection
SLU Modeling by MF
Ff Fs
Feature Model
Rf
Rs
Relation Propagation
Model
Feature Relation Model
Concept Relation Model
.
Knowledge Acquisition
Structure
Learning
o Relation Propagation Model
 Feature Knowledge Graph
 Concept Knowledge Graph
 Assumption: The domain-
specific features/concepts have
more dependency to each other.
Relation matrices allow each node to propagate scores to its neighbors in
the knowledge graph, so that domain-specific features/concepts have
higher scores during training.
 2nd Issue: Hidden semantics cannot be observed but may benefit
understanding performance.
Good! Good!
?
Feature Relation Model Concept Relation Model
Rf
Rs
‧
1
Feature Concept
Train
cheap restaurant foodexpensiveness
1
locale_by_use
11
1 1
food
1 1
1
Test
1
1
Ontology Induction
i like
1 1
capability
1
Utterance 1
i would like a cheap restaurant
……
find a restaurant with chinese food
Utterance 2
show me a list of cheap restaurants
Test Utterance
.90 .97 .95.85
hidden
semantics
o Matrix Factorization (MF)
 Model implicit feedback
 Objective:
1
𝑓+
𝑓−
𝑓−
𝑢
𝑥
MF learns a set of well-ranked
concepts per utterance.
Chen et al., “Unsupervised Induction and Filling of Semantic Slots for Spoken Dialogue Systems Using Frame-Semantic Parsing,” in Proc. of ASRU, 2013.
Chen et al., “Leveraging Frame Semantics and Distributional Semantics for Unsupervised Semantic Slot Induction in Spoken Dialogue Systems,” in Proc. of SLT, 2014.
Chen et al., “Jointly Modeling Inter-Slot Relations by Random Walk on Knowledge Graphs for Unsupervised Spoken Language Understanding,” in Proc. of NAACL, 2015a.
Chen et al., “Matrix Factorization with Knowledge Graph Propagation for Unsupervised Spoken Language Understanding,” in Proc. of ACL-IJCNLP, 2015b.