Understanding User Goals

1. Understanding User Goals in
Web Search
Daniel E. Rose
Danny Levinson

2. Not only what, but also why
● previous work on information-seeking behavior
● authors' hierarchy of search goals
● how to classify queries
● analysis of results
● applicability of model

3. LIS precedents
● Bates
○ "Information Search
Tactics" (1979)
○ "The Design of
Browsing and
Berrypicking
Techniques for the
Online
Search Interface
" (1989)
Above: Bates' berrypicking model
Left: the traditional model of document recovery
Diagrams are recreations of figures that appear in "The
Design of Browsing and Berrypicking Techniques for the
Online Search Interface."

4. ● Belkin, Oddy, and Brooks
○ "ASK for Information Retrieval" (1982)
■ from "Part I. Background and Theory"

5. ■ from "Part II. Results of a Design Study"

6. ● Broder
○ "A Taxonomy of Web Search" (2002)

7. Process
● developed flat list of user goals
● used list to classify queries in test set
● revised categories as necessary--developing
hierarchical classification
● manually classified three sets of queries
○ Each set of approximately 500 queries was randomly selected
from AltaVista query logs on a different day, and at a different
time of the day, than the other two.

8. Search goal hierarchy

9. Search goal hierarchy (continued)

10. Manual query classification
Objects of consideration: "We need to know the relative
● the query itself prevalence of various goals. And
● results returned by the if we hope to infer goals
search engine automatically in the future, we
● results clicked on by the need to know that it is possible
user to do so manually" (16).
● further searches or other
actions by the user
Do these documents provide "Once we could successfully
"sufficient information for a classify queries manually, we
human to consistently classify would be able to provide training
queries according to our goal data for a future automatic
framework" (16)? classification system" (16).

11. Final Fantasy example
Time Delta t Event Details
What kind of a site did the
user who searched for "final
36 36 result click pg 1, pos 1 http://www.
ffonline.com fantasy" intend to find--a site
that sells a version of the
113 77 query pg 1 final fantasy game, one that lists the
118 5 result click pg 1, pos 8 http://www.
game's "official" rules, or one
eyesonff.com that provides less exact
147 29 result click pg 1, pos 8 http://www. information about the game?
eyesonff.com
Examining the results returned by
two search engines for this query,
and the user's subsequent clicks, the
authors conclude that the goal of this
search was "undirected" information.

12. Results

13. Results (continued)

14. Applications and limitations
● "If our findings about ● "One issue is that we
the relatively small have no way of knowing
number of navigational conclusively whether the
queries are accurate, goal we inferred for a
they suggest that much query is in fact the user’s
of the attention in the actual goal. In the future,
commercial search we would like to combine
engine world may be our work with user
misdirected" (18). studies, including
qualitative data such as
diary reports of user
goals" (19).

15. Questions
● How does Rose and Levinson's user goals paradigm
compare to the idea of document relevance? Does a
result need to meet a search goal in order for the user
to perceive it as relevant, or simply convey information
related to the search query?
● In what ways does their user goals model take into
account query refinement and/or query expansion?
● What is the user's goal in the Aloha/American Airlines
scenario (outlined on slides 8 and 9)?
● How would you apply Rose and Levinson's research to
the following questions:
○ Why does a search occur?
○ What is its purpose?
○ How does it proceed?