Event templates for I improved narrative understanding in Question answering systems

1. Event Templates for Improved Narrative Understanding
in Question Answering Systems
Maureen Caudill
SAIC
Enterprise Information Systems Division
10260 Campus Point Drive, MS C-2
San Diego, CA 92121
and
Maureen Caudill 4/7/02 9:03 AM
Deleted:
Barbara Starr
SAIC
Enterprise Information Systems Division
10260 Campus Point Drive, MS C-2
San Diego, CA 92121
ABSTRACT understanding of those events to do more than respond to
simple look-up questions. If the system is capable of
Understanding narrative text is more than simple answering complex queries that require reasoning about
information extraction on a sentence-by-sentence basis. the situation, its comprehension of events in the real
To comprehend the true meaning of a narrative requires world must substantially exceed that of a simple
determining the connections between the sentences and information extraction system.
the effect of one event on other events. This story
understanding process can be greatly enhanced by the Information extraction systems typically deal with
use of event descriptor templates that begin with the narrative texts on a sentence-by-sentence basis. That is,
basic journalistic questions of who, what, when, where, they rarely are capable of understanding the relationship
why, and how but that go beyond these simple basics to between sentences. This limited knowledge makes them
address more complex relationships: role playing, into systems that do little more than key word matches
context, impact, causality, and interests. Previously, when trying to answer questions. In a query about
representing story narratives as knowledge houses, a typical information extraction system looks for
representations has required intensive manual effort on the word “house” in sentences. A slightly more
the part of trained knowledge engineers to correctly sophisticated version might also include synonym
encode the contents of stories into a knowledge base searches, and include searches for “apartment,”
(KB). For large volumes of text, this becomes “condominium,” “condo,” “dwelling,” “abode,” etc.
impractical, limiting the usefulness of KB-based systems However, none of these techniques can identify
in question-answering. This paper describes a means of sentences that refer to houses indirectly such as the one
automating the narrative representation process by using below:
event descriptor templates to elicit critical narrative
information to be encoded in a knowledge based system. I lived on Main Street when I was a child, in a huge old
barn. It had twelve rooms, five fireplaces, and many odd
Keywords: Automating Knowledge Generation, , nooks and corners where a child could disappear for
Knowledge Engineering Bottleneck, Knowledge-Based hours at a time.
System Development, Question-Answering Systems,
Event extraction, Event understanding, Automatic Event In this example, the word “barn” likely would not be
Extraction. identified as a metaphor for “house.” Even if that
1. BACKGROUND relationship were identified, the second sentence,
referring to the dwelling place only as “It,” would
A question-understanding system that can answer certainly be missed by all but a very few current systems.
questions about historical or current events must have an Thus, if the question is “How many rooms were in the

2. house where you lived as a child?” an information to many complex, non-obvious questions about that
extraction system could not answer it, even though the event.
information requested is explicitly included in the
sentence. For information that is not explicit, but The concept of prefabricated understanding of specific
available only through inferencing, typical information types of events can, however, be extended to more
extraction systems have no hope of generating an general event types using an event descriptor template.
answer. The initial version of this template first seeks to answer
the basic journalistic queries of who, what, when, where,
In knowledge-based question-answering systems, how, and why.
however, the details of the house could be elicited during
the knowledge representation process. In previous When viewing the output from a basic information
development of knowledge-based systems, the process of extraction system, some of these journalistic queries
producing these knowledge representations from the appear to be quite simple to determine. For example, the
content of a narrative has been entirely manual. Trained question of what happened is frequently identified by the
knowledge engineers and/or experts in a particular verb in the sentence (or in the clause). However, a single
domain have been required to manually write the sentence can encode more than one event, and multiple
knowledge representations that reflect the contents of sentences can encode the same event. Thus, any
documents. This is sometimes called the “knowledge automated system to perform story understanding must
engineering bottleneck.” be able to view all sentences in the story and determine
which portions of each refer to which specific events.
In the current paper, we describe a method of resolving
both these problems. The use of event descriptor For example, consider the sentence:
templates as models for representing narrative text [Manila, Philippines] About 36 US Special Forces
provides both completeness of coverage and a method of troops started a month of anti-terrorism training with
automating the knowledge extraction and representation their Filipino counterparts at a northern army base
processes. Tuesday, a Philippine army officer said. (taken from an
article on MSNBC.com)
The following sections describe, first, the types of
templates that we have found useful in describing The obvious verb here is “started,” as in, the troops
narratives, and then a system design that automatically started anti-terrorism training. However, the actual main
fills in these event templates from large volumes of text verb of the sentence is “said.” The verb “started” is
documents without requiring knowledge engineers or merely describing what was “said.”
domain experts to generate the knowledge
representations. Sentences such as this are very common in real-world
story understanding. In such cases, both verbs identify
2. BASIC JOURNALISTIC RELATIONS events, one of which is an act of starting something (i.e.,
“starting to train”) and the other is an act of
Much of the information needed to correctly respond to communicating (i.e., “said”). This sentence thus encodes
user queries is found in the contents of story narratives. two separate events, the starting event and the saying
These text items range from historical event summaries event. This sentence can thus begin to be encoded by
to news stories. Consequently, narrative text constitutes generating the simplest knowledge relations:
an important set of data items to be able to automatically
process and convert into knowledge representations. (defobject Event-E1
Story narratives can be encoded in many ways, going (instance-of Event-E1 saying))
back to the concept of scripts, in which a standardized set (defobject Event-E2
of roles and activities occur in a known general (instance-of Event-E2 starting))
environment. The classic example of scripts is eating at
a restaurant. There, a person who understands the basic Thus, the verb(s) defines the type of action that
“restaurant script” knows how to interpret such activities corresponds to that event.
as being shown to a table and handed a menu, ordering
from a waiter or waitress, having the meal delivered, The next simple journalistic question is who performed
receiving a check, and paying for the meal. By encoding the event. Typically, this is identified as the subject of
this and other stereotypical scenarios in a standard the verb for each verb in the sentence. (Care must, of
knowledge system, then comparing a textual narrative of course, be taken for passive voice and similar non-typical
a specific “eating at a restaurant” incident to the sentence constructs.) Thus, a parse of the sample
restaurant script, a knowledge-based system can respond sentence above can identify the agents involved by
identifying the subjects of each of the two verbs. This

3. enhances the definitions of the two events by adding would be made that the location of the saying event is the
more relations: location specified in the dateline. Thus, the location for
that event would be:
(defobject Event-E1
(instance-of Event-E1 saying) (defobject Event-E1
(performed-by Event-E1 Philippine-army-officer)) (instance-of Event-E1 saying)
(defobject Event-E2 (performed-by Event-E1 Philippine-army-officer)
(instance-of Event-E2 starting) (location-of Event-E1 city-of-manila))
(performed-by Event-E2 us-special-forces-troops)
(performed-by Event-E2 filippino-troops)) Should this location not be in the base ontology, it would
of course also have to be defined.
To get these encodings, however, additional definitions
are required to identify the specific agents, since it is The next journalistic question is when did the event take
unlikely that any general ontology would have the place. This is again easier to determine for the starting
concepts of Philippine-army-officer or us-special-forces- event than the saying event:
troops already present.
(defobject Event-E2
Thus, additional prior definitions are added to the (instance-of Event-E2 starting)
knowledge base: (performed-by Event-E2 us-special-forces-troops)
(performed-by Event-E2 filippino-troops)
(defobject Philippine-army-officer (location-of Event-E2 northern-army-base)
(instance-of Philippine-army-officer soldier) (took-place-when Event-E2 Tuesday))
(citizen-of Philippine-army-officer Philippines)) Presumably, a newswire article would also have a date
attached to it, so “Tuesday” could be more specifically
(defobject us-special-forces-troop defined as the most recent Tuesday prior to the date of
(instance-of us-special-forces-troop soldier) the article.
(citizen-of us-special-forces-troop united-states))
The question of when the saying event took place is
again an inference, not something that is clearly defined;
(defobject us-special-forces-troops however, it can be presumed to be not later than the date
(group-of us-special-forces-troops us-special-forces- of the newswire article itself:
troop)
(count-of us-special-forces-troops 36)) (defobject Event-E1
(instance-of Event-E1 saying)
The next journalistic query is where does this event take (performed-by Event-E1 Philippine-army-officer)
place. This becomes somewhat more difficult to (location-of Event-E1 city-of-manila)
ascertain in many cases. In the case of the starting event, (took-place-when Event-E1 before(date-of-article)))
the location is straightforward, and can be added to the
event definition in a simple manner: The two remaining journalistic questions, how and why
are not answered by the current sentence; they remain,
(defobject Event-E2 for the moment, empty relations. And, in fact, the
(instance-of Event-E2 starting) question of why an event occurs is nearly always a more
(performed-by Event-E2 us-special-forces-troops) complex relation to implement.
(performed-by Event-E2 filippino-troops)
(location-of Event-E2 northern-army-base)) 3. COMPLEX RELATIONS
Obviously, this too spawns a definition of the term A careful reading of the original sentence shows that
northern-army-base: while the descriptors developed above are accurate, they
are sorely incomplete. For example, the “starting” event
(defobject northern-army-base does not explain what exactly is being started.
(instance-of northern-army-base army-outpost) Furthermore, there’s no information regarding why this
(location-of northern-army-base (location-function event is happening, what impacts it has on other events,
(direction-fn north) Philippines))) or any of a number of other issues that a human reader
can be expected to infer easily from the text. Thus, the
The location of the saying event is far more obscure, journalistic event relations are only the starting point for
however. In this case, with a newswire article that a far more complex set of relationships.
(presumably) has a dateline attached, the assumption

4. In the case of the example sentence, the first issue is to • Is-mediator
identify what exactly is being started. The object of the • Is-peacekeeper
sentence (either direct object or indirect object, • Is-victim
depending on the verb) frequently gives insight into what • Is-innocent-bystander
is being used to perform an action or what is being • Is-financial-backer
referred to in an event description. The object of the • Is-betrayer, etc.
starting clause is the phrase a month of anti-terrorism Causality Relations
training. Thus, there must be a way to encode the • Event-causes-event
concept of what is being started. • Interest-causes-event
• Contributing-factor, etc.
(defobject Event-E2 Impact Relations
(instance-of Event-E2 starting) • Event-opposes-agent <a specific agent>
(performed-by Event-E2 us-special-forces-troops) • Event-supports-agent <a specific agent>
(performed-by Event-E2 filippino-troops) • Event-opposes-interest <a specific interest>
(location-of Event-E2 northern-army-base) • Event-supports-interest <a specific interest>
(took-place-when Event-E2 Tuesday) • Supports-interests-of <an agent and its
(action-performed-on Event-E2 anti-terrorism- interest>
training)) • Opposes-interests-of <an agent and its
interest>
This in turn spawns the definition: • Action-performed-on <an object or an agent>
• Action-performed-with <an object>, etc.
(defobject anti-terrorism-training
(instance-of anti-terrorism-training training) The relations that form the event descriptor thus provide
(opposes anti-terrorism-training terrorist-group) a rich encoding of the content of a narrative text
(duration anti-terrorism-training (days 30))) description. The benefit of this approach to knowledge
representation is two-fold. First, it ensures that each
Note that the training definition includes the approximate event is scrutinized to derive the maximum possible
duration of the training. Notice also that because the information from the narrative text. Second, it allows
prefix “anti” means “against,” the relation “opposes” is multi-sentence descriptions. For example, suppose the
used. If there were no such prefix, and the phrase was next sentence in the text is:
“terrorism training,” the relation “supports” would be
used, on the assumption that training generally supports The training will provide experience in fighting bio-
the activity it concerns. terrorist attacks in a jungle environment.
The saying event also must be more carefully defined. This sentence can easily be determined to refer to the
What exactly did the Philippine army officer say? same event as the “starting” event. The subject of the
Basically, the officer described the training event. Thus, sentence is “the training.” Since it immediately follows
a reasonable encoding of the object of the saying event a sentence that refers to starting a training event, it can
is: be inferred that the training referred to is one and the
same training. Thus, it is a simple matter to amend the
(defobject Event-E1 earlier description of the training to assert:
(instance-of Event-E1 saying)
(performed-by Event-E1 Philippine-army-officer) (defobject anti-terrorism-training
(location-of Event-E1 city-of-manila) (instance-of anti-terrorism-training training)
(took-place-when Event-E1 before(date-of-article)) (opposes anti-terrorism-training terrorist-group)
(action-performed-on Event-E1 Event-E2)) (duration anti-terrorism-training (days 30))
(location-of anti-terrorism-training jungle)
This single sentence does not yield other relations. But (opposes anti-terrorism-training bio-terrorist-attack))
there are many other relations that should be considered
before abandoning the analysis of the sentence (and the Filling in such detailed event descriptors amounts to pre-
article). These relations can be grouped into categories answering a number of queries about the narrative. Any
as shown below with a few examples for each category. question that asks about these events, even in elliptical
terms, will quickly generate the correct answer. This
Role Relations removes a great deal of the processing and inferencing
• Is-aggressor burden from the question-answering system and enables
• Is-defender it to provide answers accurately, quickly, and efficiently.
• Is-protector 4. AUTOMATING THE PROCESS

5. The remaining question is how can constructing such Each of the KB partitions has access to the global
event descriptors be automated. In essence, this requires ontology and lexicon to identify appropriate concepts
constructing a pre-answering system that infers the and lexical terms and to define new ones as needed.
answer to obvious questions about a narrative and stores These new terms comprise a mini-ontology that is
those answers in the knowledge base for rapid retrieval. relevant primarily to the current document; however, as
appropriate, new terms can be added to the global
The approach we have taken to this is a modular one for lexicon, thus enabling cross-document processing of
the sake of easy construction, extension, and events. Thus, a sequence of events described in multiple
maintenance. In effect, each relation or logical set of documents is an easy extension of this concept.
relations has its own miniature knowledge-based
reasoner—a knowledge base partition—which knows Furthermore, since each relation can be documented with
only how to inspect a parsed sentence and construct a the specific text that generated that relation, it is
specific relation from that sentence—if and only if that similarly easy to identify which document events and
sentence contains the necessary information to construct mini-ontologies correspond to a particular term. For
the relation. example, if a query asks about US Special Forces Troops
in the Philippines, it is very easy to determine exactly
These knowledge-based reasoners thus are kept within a which document(s) have information about these troops,
small scale, they can be called or not as appropriate for a since they are mentioned in those documents that include
given sentence, and they construct an appropriate that ontological term. A separate database (not shown in
relation (or small number of similar relations) based on the figure) keeps track of which ontological terms are
the contents of that sentence. Figure 1 illustrates how the defined in which document files and mini-ontologies.
architecture operates in the case of the example sentence. Thus, if a query arrives asking about a specific concept
(such as the U.S. Special Forces Troops in the
The KB Controller is the coordinator for the event Philippines), only those knowledge bases that include
descriptor construction. It does not generate event that concept are added to the answer reasoner’s
relations itself, but calls whatever other modules are knowledge base to serve as the basis for determining the
appropriate in a given sentence to determine the answer. This provides a mechanism for automatic
relations. It also deals with cross-sentence descriptors by partitioning of very large scale knowledge bases as well
determining that the current sentence probably refers to as increases the efficiency of the answering system.
an event that is already described; in this case it tells the
various KB partitions it calls not to define a new event The modular architecture enables each KB partition to
but rather to modify a specific existing event. focus on only a single problem rather than trying to
Parsed sentence KB
Controller KB Ontology
Calls KB & Lexicon
Engines
as needed
Etc.
Instance-of Performed-by Location-of When Roles Causality
KB partition KB partition KB partition KB partition KB partition KB partition
(defobject Event-E2 (location-of Event-E2
(instance-of Event-E2 northern-army-base))
starting)) including definition of terms
(defobject Event-E2 (took-place-when Event-
(performed-by Event-E2 E2 Tuesday)
us-special-forces-troops))
including definitions for Figure 1. Architecture for the Event Descriptor system.
“us-special-forces-troops”

6. generate the entire event descriptor. This makes the To appear in Proceedings of the Eighth International
entire system easy to construct, easy to extend, and easy Conference on Conceptual Structures Logical,Linguistic,
to maintain. and Computational Issues (ICCS 2000). Darmstadt,
5. CONCLUSIONS Germany. August 14-18, 2000.
This paper has outlined a workable methodology for [10] McIlraith, S. and Amir, E. “Theorem Proving with
constructing a narrative understanding tool for text Structured Theories,” Proceedings of the
narratives that can automatically comprehend stories and Seventeenth International Conference on Artificial
answer questions about those stories. The system is Intelligence (IJCAI-01). pp. 624 -- 631, August, 2001.
under construction and preliminary results indicate that it
is flexible, powerful, and highly extensible. [11] Amir, E. and McIlraith, S. “Partition-Based Logical
Reasoning for First-Order and Propositional Theories,”
6. REFERENCES Submitted for Publication.
[1] Adam Farquhar, Richard Fikes, and James P. Rice. This work was supported by the Advanced Research and
“A Collaborative Tool for Ontology Construction.,” Development Activity (ARDA) as part of its AQUAINT
International Journal of Human Computer Studies, Program. Any opinions, findings, and conclusions or
46:707-727, 1997. recommendations expressed in this material are those of
the author(s) and do not necessarily reflect the views of
[2] Peter D. Karp, Vinay K. Chaudhri, and Suzanne M. the U.S. Government.
Paley. “A Collaborative Environment for Authoring
Large Knowledge Bases.” Journal of Intelligent
Information Systems, 1998.
[3] Paul Cohen, Robert Schrag, Eric Jones, Adam Pease,
Albert Lin, Barbara Starr, David Gunning, and Murray
Burke. The DARPA High Performance Knowledge
Bases Project. AI Magazine, Winter, 1998. pp. 25-49
[4] B. Katz, “From Sentence Processing to Information
Access on the World Wide Web,” AAAI Spring
Symposium on Natural Language Processing for the
World Wide Web, Stanford University, Stanford CA
(1997).
[5] Gentner, D. and K. Forbus “MAC/FAC: A Model of
Similarity-based Retrieval,” Proceedings of the
CognitiveScience Society. 1991.
[6] Forbus, K. and D. Oblinger “Making SME Greedy
and Pragmatic,” Proceedings of the Cognitive Science
Society. 1990.
[7] V. K. Chaudhri, J. D. Lowrance, M. E. Stickel, J. F.
Thomere, and R. J. Waldinger
“Ontology construction toolkit,” Technical Note
Ontology, AI Center, SRI International, 333
Ravenswood Ave., Menlo Park, CA 94025, 2000.
[8] Deborah McGuinness “Description Logics Emerge
from Ivory Tower”’ Stanford Knowledge Systems
Laboratory Technical Report KSL-01-08 2001. In the
Proceedings of the International Workshop on
Description Logics. Stanford, CA, August 2001.
[9] Deborah McGuinness “Conceptual Modeling for
Distributed Ontology Environments.”(Word format)