From Exploration to Construction  - How to Support the Complex Dynamics of Information Seeking

From Exploration to Construction 
How to Support the Complex Dynamics of Information Seeking  
 
Hugo C. Huurdeman PhD candidate University of Amsterdam
webarchiving.nl

Introduction: a paradox
• Models of
information  
seeking describe
fundamentally
different macro-
level stages in
complex tasks
+uncertainty-uncertainty optimism confusion clarity confidence (dis)satisfaction
doubt direction
FormulationInitiation Selection Exploration Collection Presentation

Introduction: a paradox
• Models of
information  
seeking describe
fundamentally
different macro-
level stages in
complex tasks
+uncertainty-uncertainty optimism confusion clarity confidence (dis)satisfaction
doubt direction
Search
• However, current search systems usually
provide a streamlined and static feature set
• To what extent do current search
approaches support complex tasks?

Multistage Information Seeking Models
macro perspective1
Read more:
Huurdeman & Kamps (2014), From Multistage Information Seeking Models  
to Multistage Search Systems. Proc. IIiX 2014. http://dx.doi.org/10.1145/2637002.2637020
Huurdeman & Kamps (2015). Supporting the Process - Adapting Search Systems
to Search Stages. Proc. ECIL 2015. http://dx.doi.org/10.1007/978-3-319-28197-1_40

1.1 Information Seeking Models
• Information seeking modeled in
a multitude of ways:
• as behavioral patterns (Ellis)
• as nonlinear activities (Foster)
• as problem-solving (Wilson)
• as temporal stages (Kuhlthau), ..
• Our main focus:
• temporally based IS models
• Kuhlthau [1991] (Vakkari [2001])
• cognitively complex (work) tasks
• involving learning & construction
information
search
information 
seeking
information  
behavior
[Wilson99]

1.2 Kuhlthau: Information Search Process [1991]+uncertainty-
feelings
thoughts
actions
vague focused
seeking relevant information
(exploring)
seeking pertinent information
(documenting)
uncertainty optimism confusion clarity confidence (dis)satisfaction
doubt direction

1.2 Vakkari’s adaptation (in [Vakkari01])
Prefocus Focus formulation Postfocus
seeking general background
information
seeking specific
information
faceted backgr. 
information
relevance hard to judge relevance easier to judge
decrease of number of broader terms
information 
sought
relevance
search terms increase of number of search terms, synonyms, narrower terms

1.3 Implications for design of search systems
• Observation: good general
understanding of macro level
inf. seeking stages, but hard to
translate to concrete micro level
system design choices.
macro
micro
system
design
inf.
seeking
stages

Search user interfaces supporting seeking
micro perspective2

3.1 Search user interfaces supporting seeking
• Search User Interface (SUI) design:
• no straightforward task to design a UI with a high usability [Shneiderm05]
• A (limited) number of available frameworks, guidelines and design
pattern libraries for SUIs
• e.g. M. Wilson’s framework of SUI features [Wilson11]
Search
Input
Control
Informational
Personalizable

2.1 SUI approaches: traditional search
• Streamlined interfaces
• Focus on
• query formulation
• result list inspection 
• Advantages: [Hearst09]
• lower cognitive load
• more accessible
• more understandable
Highly optimized for lookup tasks, less for open-ended queries

• Research-based tasks
• WebART project
• new media researchers
• action research setting, structured literature review
• Search systems allow for answering new research
questions, but also have limitations
• lack of transparency
• lack of process support
2.2 When traditional search does not work well
researcher
research
activities
corpus
creation
analysis
dissemination
webarchiving.nl

2.3 SUI approaches: exploratory search
• Supporting open-
ended inf. seeking
• Support learning and
investigation activities
for complex
information problems
[Marchionini06]
• Many potential exploratory SUI features [White09], e.g.
• rapid query refinement, facets (input, control)
• leveraging context, visualizations (informational)
• histories/workspaces/task management (personalizable)

2.4 SUI approaches: sensemaking & analytics
• Support analysis & synthesis in interface
• Potential functions facilitating notetaking, hypothesis formulation
& collaborative search [Hearst09]
• some overlap with exploratory search

2.5 Implications for search stage support (2/2)
• Observation: good
understanding of search system
features at the micro level, but
fragmented understanding of
how they can support
information seeking stages at
the macro level
macro
micro
search
system
features
inf.
seeking
stages

Reconciling macro and micro views
• Would it be possible
to reconcile the
macro level and
micro level views?
macro
micro
search
system
features
inf.
seeking
stages

“The Utility of SUI Features”
Our study: investigating the utility of various  
SUI features at different macro-level stages
From: Huurdeman, Wilson & Kamps (2016), Active and Passive Utility of  
Search Interface Features in Different Information Seeking Task Stages.  
Proc. ACM CHIIR 2016. http://dx.doi.org/10.1145/2854946.2854957

3. Setup
• User study (26 participants; 24 analyzed)
• Undergrads Univ. of Nottingham (6 F, 12 M, 18-25y)
• Experimental SUI resembling common Search Engine
• Within-participants
• Task stage independent variable
• Task design: explicit multistage approach

3. Setup: Multistage Task Design
sim. work task: writing essay
subtask subtask subtask
prepare list of  
3 topics
choose topic; 
formulate speciﬁc 
question
find and select  
additional 
pages to cite
15 minutes 15 minutes 15 minutes
initiation 
topic selection 
exploration
focus formulation
collecting
presenting

3. Setup: Multistage Task Design
sim. work task: writing essay
subtask subtask subtask
prepare list of  
3 topics
choose topic; 
formulate speciﬁc 
question
find and select  
additional 
pages to cite
15 minutes 15 minutes 15 minutes
initiation 
topic selection 
exploration
focus formulation
collecting
presenting
General Assigned Topics (b/o discussions teaching staff)
• Autonomous Vehicles (AV)
• Virtual Reality (VR)

3. Setup: Protocol
Training task
Pre-
Questionnaire
Topic
Assignment
Introduction
system
Task
Post-task
Questionnaire
3x
Post-experiment
questionnaire
Debriefing
interview

• Experimental system: SearchAssist
• Results, Query Corrections, Query
Suggestions: Bing Web API
• Category Filters: DMOZ
• Categorization and analysis:
• Max Wilson’s framework of SUI features
[Wilson11]

Control
Input
PersonalizableInformational

3. Setup: Logging
eyetribe.com

3. Setup: Data / Task details
• AV & VR topics invoked comparable behaviours:
• analysed as one topic set
• Total duration main tasks
• Total task time: 32:56
• 36.8% SUI, 33% Task screen, 30.2% Webpages
Stage 1: 11:32 Stage 2: 8:24 Stage 3: 12:59

Findings: Active Behaviour
behaviour directly and indirectly derivable from logs4

4.1 Active Behaviour: Clicks
0
4
8
Sig. clicks on interface  
features over time
Stage 1 Stage 2 Stage 3

0
4
8
features over time
Category ﬁlters ➡

0
4
8
features over time
Tag Cloud ➡

0
4
8
features over time
Tag Cloud ➡
Search button ➡

0
4
8
features over time
Tag Cloud ➡
Search button ➡
Saved Results

4.2 Active Behaviour: Queries
•Mean number of queries** (unique):
• Stage 1: 9.5 (8.1) ➡ Stage 2: 5.5 (5.1) ➡ Stage 3: 5.9 (5.3)
0
2,5
5
7,5
10
Search Box
Query Suggestions
Recent Queries

4.3 Active Behaviour: Query words
•Mean number of query words**:
“virtual reality” (P.02)
“impact of virtual reality on
society art and culture“
“autonomous vehicles” (P.06)
“autonomous vehicles costs 
insurance industry”
0
1,25
2,5
3,75
5
Mean Number of Query words

4.4 Active Behaviour: Visited pages
• Visited pages (unique)**:
• Stage 1: 8.0 (7.3)
• Stage 2: 6.4 (5.9)
• dwell time highest
• Stage 3: 14.2 (10.8)
• Mean rank visited pages
• from 3.1 to 6.4
0
4
8
12
16
Results List
Saved Results

4.5 Active Behaviour: Wrapup
• Clicks:
• decreasing for Query Box (input), Category Filters & Tag
Cloud (control)
• increasing for Saved Results (personalizable)
• Queries:
• decreasing over time, but more complex
• Popularity of certain features and impopularity of others:
•Some features used in passive instead of active ways?

Findings: Passive Behaviour
behaviour not typically caught in interaction logs5
eyetribe.com

Passive behaviour: mouse hovers
• Mouse movements:
• movements to reach a feature, also to aid processing contents [Rodden08]
•Focus here on mouse movements not leading to click
• Tendencies mostly overlap with active interaction measure
0%
25%
50%
75%
100%
1 2 3

Category ﬁlters** ➡
0%
25%
50%
75%
100%
1 2 3

Tag Cloud* ➡
0%
25%
50%
75%
100%
1 2 3

Tag Cloud* ➡
Query Box** ➡
0%
25%
50%
75%
100%
1 2 3

Tag Cloud* ➡
Query Box** ➡
Results List* ⤻
0%
25%
50%
75%
100%
1 2 3

5.2 Passive Behaviour: eye ﬁxations
Stage 1 (exploration) Stage 2 (focus formulation)
Stage 3 (postfocus, collection)
• Overview of eye movement via heatmaps:

Passive behaviour: eye tracking
eye tracking fixations
0
25
50
75
100
1 2 3
• Further insights via eye tracking fixation counts
• fixations > 80 ms, similar to e.g. [Buscher08]

0
25
50
75
100
1 2 3
Query Suggestions* ➡

0
25
50
75
100
1 2 3
Tag Cloud* ➡

0
25
50
75
100
1 2 3
Tag Cloud* ➡
Query Box** ➡

0
25
50
75
100
1 2 3
Tag Cloud* ➡
Query Box** ➡
Results List* ⤻

3.4 Passive Behaviour: Active vs. Passive
0%
2%
4%
6%
8%
Subtle differences between passive and active use:

0%
2%
4%
6%
8%
Tag Cloud [5.8% ﬁxations ⬌ 3.1% clicks]

0%
2%
4%
6%
8%
Query Suggestions [3.6% ﬁx. ⬌ 1.9% clicks]

0%
2%
4%
6%
8%
Recent Queries [3% ﬁx. ⬌ 2% clicks]

0%
2%
4%
6%
8%
Recent Queries [3% ﬁx. ⬌ 2% clicks]
Opposite for Category Filters [5% ⬌ 3.8%]

5.4 Passive Behaviour: Wrapup
•Fixations & mouse moves
• validating active behaviour
• subtle differences active and passive use
• Could subjective ratings and qualitative feedback
provide more insights?

Findings: Perceived Feature Utility
perceived usefulness (post-stage & experiment)6

6.2 Perceived Usefulness: post-experiment
• Post-experiment questionnaire:
• In which stage or stages were SUI features most useful?
• Pronounced differences
• signiﬁcant differences for all features
0%
25%
50%
75%
100%
Query Box /  
Results List
Category 
Filters
Tag  
Cloud
Query  
Suggestions
Recent  
Queries
Saved  
Results

6.3 Perceived Usefulness: Category Filters
• “good at the start (…) but later I
wanted something more specific” (P.11)
• common remarks in 2nd and 3rd stage:
• “… could be more specific in its
categories”
• “…hard to find the category I want” (P.
27)

6.3 Perceived Usefulness: Tag Cloud
• at the start:
• “…aids exploring the topic” (P.06);
• “came up with words that I hadn’t thought of”
• later stages:
• “doesn’t help to narrow the search much” (P.18)
• “in the end seemed to be too general” (P.07)

6.3 Perceived Usefulness: Tag Cloud
• at the start:
• “…aids exploring the topic” (P.06);
• “came up with words that I hadn’t thought of”
• later stages:
• “doesn’t help to narrow the search much” (P.18)
• “in the end seemed to be too general” (P.07)
• Post-experiment comments:
• “…was good at the beginning, because when you
are not exactly sure what you are looking for, it can
give inspiration” (P.12)
• “… nice to look at what other kinds of ideas [exist]
that maybe you didn’t think of. Then one word may
spark your interest” (P.15)

6.3 Perceived Utility: Query Suggestions
• “…was good at the start but as soon
as I got more speciﬁc into my topic,
that went down” (P.11)
• “clicked [it] .. a couple of times .. it
gave me sort of serendipitous
results, which are useful” (P.24)

6.3 Perceived Utility: Recent Queries
• Naturally: “…most useful in the end
because I had more searches from
before” (P.26)
• “The previous searches became more
useful ‘as I made them’ because they
were there and I could see what I
searched before. I was sucking myself
in and could work by looking at
those.” (P.23)
• May aid searchers in  
their information journey..

6.3 Perceived Utility: Saved Results
• “most useful in the end” (P.12)
• “At the start [I was] saving a lot of
general things about different topics.
Later on I went back to the saved
ones for the topic I chose and then
sort of went on from that and see what
else I should search” (P.26)
• “I just felt I was organizing my
research a little bit” (P.18)
• It “helps me to lay out the plans of my
research”.

Conclusion
towards more dynamic support7

0%!
20%!
40%!
60%!
80%!
100%!
Stage 1! Stage 2! Stage 3!
Percentageofparticipants!
input / informational!
control!
personalisable!
Stage 2! Stage 3!
input / informational!
control!
personalisable!
Conclusion: Findings Summary
• Informational features highly
useful in most stages
• Decreasing use of input features
• Control features decreasingly
useful
• likely caused by a user’s evolving
domain knowledge
• Personalizable features
increasingly useful
• ‘growing’ with a user’s
understanding, task
management support
SUI features perceived as most
useful, per stage

7. Conclusion: theoretical roundup
complex information seeking task
pre-focus stage:
• vague understanding
• limited domain knowledge
• trouble expressing
information need
• large amount of new
information
• explaining prominent role of
control features
• explore information
• filter result set
using [Kuhlthau04,Vakkari&Hakkala00,Vakkari01]

pre-focus stage:
information need
information
control features
focus formulation stage:
• more directed search
• better understanding
• seeking more relevant
information, using
differentiated criteria
• control features become less
essential
• “not specific enough”
• personalizable feat’s more
important: may “grow” with
emerging understanding

pre-focus stage:
information need
information
control features
focus formulation stage:
• more directed search
• better understanding
• seeking more relevant
information, using
differentiated criteria
• control features become less
essential
• “not specific enough”
• personalizable feat’s more
important: may “grow” with
emerging understanding
postfocus stage
• specific searches
• re-checks additional
information
• precise expression
• low uniqueness, high
redundancy of info
• long, precise, queries
• further decline of control
features
• frequent use of
personalizable features
• “see what else to search”

7. Conclusion: Future Work
•Our study: essay writing simulated work task
• Extension to other types of complex tasks, user
populations
•Further research into task-aware search systems
• additional features may be useful at different stages
• e.g. user hints, assistance
• improvement of current features

Towards “stage-aware” Systems
prefocus
focus
formulation
postfocus
searchersearch 
system
search 
interface
search  
stage
stage-independent
functionalityranking stage-dependent
functionality
manual or automatic 
detection

• INEX/CLEF Interactive Social
Book Search Lab
• http://social-book-search.humanities.uva.nl/#/interactive
• Gaede, Hall, Huurdeman, Kamps,
Koolen, Skov, Toms, Walsh (2015)
• Aim: support different stages
in the search process:
browse, search & review
• Joint study across
universities, 192 participants
Multistage interface: search
Multistage interface: browse
Multistage interface: review
Example: multistage interface

7. Conclusion: towards dynamic SUIs
•Most Web search systems converged over static
and familiar designs
• trialled features often struggled to provide value for
searchers
• perhaps impeding search [Diriye10] if introduced in simple
tasks, or at the wrong moment
•Our work provides insights into when SUI
features are useful during search episodes
• potential responsive and adaptive SUIs for complex tasks

References (1/2)
[Ahlberg&Shneiderman94] C. Ahlberg and B. Shneiderman. Visual information seeking: Tight coupling of dynamic query filters with
starfield displays. In CHI, pages 313–317. ACM, 1994.  
[Buscher08] G. Buscher, A. Dengel, and L. van Elst. Eye movements as implicit relevance feedback. In CHI’08 extended abstracts
on Human factors in computing systems, pages 2991–2996. ACM, 2008.
[Diriye10] A. Diriye, A. Blandford, and A. Tombros. When is system support effective? In Proc. IIiX, pages 55–64. ACM, 2010.
[Diriye13] A. Diriye, A. Blandford, A. Tombros, and P. Vakkari. The role of search interface features during information seeking. In
TPDL, volume 8092 of LNCS, pages 235–240. Springer, 2013.
[Donato10] D. Donato, F. Bonchi, T. Chi, and Y. Maarek. Do You Want to Take Notes?: Identifying Research Missions in Yahoo! Search
Pad. In Proc. WWW’10, pages 321–330, 2010. ACM.
[GaedeEtAl15] Maria Gäde, Mark Hall, Hugo Huurdeman, Jaap Kamps, Marijn Koolen, Mette Skov, Elaine Toms, and David Walsh.
Overview of the SBS 2015 interactive track. In CLEF’15 Working Notes. CEUR-WS, 2015.
[Hearst09] M. A. Hearst. Search user interfaces. Cambridge University Press, 2009.
[Hearst13] M. A. Hearst and D. Degler. Sewing the seams of sensemaking: A practical interface for tagging and organizing saved
search results. In HCIR. ACM, 2013.
[Huurdeman&Kamps15] Hugo C. Huurdeman and Jaap Kamps (2015). Supporting the Process: Adapting Search Systems to Search
Stages. In: S. Kurbanoğlu, S. Špiranec, J. Boustany, E. Grassian, D. Mizrachi, & L. Roy (Eds.), Information Literacy: Moving towards
sustainability, Communication in Computer and Information Science series (Vol. 552, pp. 394-404). 
[Huurdeman&Kamps14] H. C. Huurdeman and J. Kamps. From Multistage Information-seeking Models to Multistage Search
Systems. In Proc. IIiX’14, pages 145–154, 2014. ACM
[Kuhlthau91] C. C. Kuhlthau. Inside the search process: Information seeking from the user’s perspective. JASIS, 42:361–371, 1991.
[Kuhlthau04] C. C. Kuhlthau. Seeking Meaning: A Process Approach to Library and Information Services. Libraries Unlimited, 2004.
[Kules12] B. Kules and R. Capra. Influence of training and stage of search on gaze behavior in a library catalog faceted search
interface. JASIST, 63:114–138, 2012.
[LiuBelkin15] J. Liu and N. J. Belkin. Personalizing information retrieval for multi-session tasks. JASIST, 66(1):58–81, Jan. 2015.
[Marchionini06] G. Marchionini. Exploratory search: from finding to understanding. CACM, 49(4):41–46, 2006.
[Niu14] X. Niu and D. Kelly. The use of query suggestions during information search. IPM, 50:218–234, 2014.
[Proulx06] P. Proulx, S. Tandon, A. Bodnar, D. Schroh, W. Wright, D. Schroh, R. Harper, and W. Wright. Avian Flu Case Study with
nSpace and GeoTime. In Proceedings of the IEEE Symposium on Visual Analytics Science and Technology (VAST'06). IEEE, 2006.

References (2/2)
[Toms11] E. G. Toms. Task-based information searching and retrieval. In Interactive Information
Seeking, Behaviour and Retrieval. Facet, 2011.
[Rodden08] K. Rodden, X. Fu, A. Aula, and I. Spiro. Eye-mouse coordination patterns on web
search results pages. In CHI’08 Extended Abstracts, pages 2997–3002. ACM, 2008.
[Shneiderman05] B. Shneiderman and C. Pleasant. Designing the user interface: strategies
for effective human-computer interaction. Pearson Education, 2005.
[Tunkelang09] D. Tunkelang. Faceted search. Synthesis lectures on information
concepts, retrieval, and services, 1(1):1–80, 2009.
[Vakkari01] P. Vakkari. A theory of the task-based information retrieval process: a summary
and generalisation of a longitudinal study. Journal of Documentation, 57:44–60, 2001.
[White05] R. W. White, I. Ruthven, and J. M. Jose. A study of factors affecting the utility of
implicit relevance feedback. In SIGIR, pages 35–42. ACM, 2005.
[White09] R. W. White and R. A. Roth. Exploratory search: Beyond the query-response
paradigm. Synthesis Lectures on Information Concepts, Retrieval, and Services, 1:1–98, 2009.
[Wilson&schraefel08] M. L. Wilson and m. c. schraefel. A longitudinal study of exploratory and
keyword search. In In Proc. JCDL’08, pages 52–56. ACM, 2008.
[Wilson99] T. D. Wilson. Models in information behaviour research. Journal of Documentation,
55:249–270, 1999.
[Wilson11] M. L. Wilson. Interfaces for information retrieval. In I. Ruthven and D. Kelly, editors.
Interactive Information Seeking, Behaviour and Retrieval. Facet, 2011.

From Exploration to Construction 
How to Support the Complex Dynamics of Information Seeking  
 
Hugo C. Huurdeman University of Amsterdam @TimelessFuture
webarchiving.nl

From Exploration to Construction  - How to Support the Complex Dynamics of Information Seeking

Recommended

Recommended

More Related Content

Similar to From Exploration to Construction  - How to Support the Complex Dynamics of Information Seeking

Similar to From Exploration to Construction  - How to Support the Complex Dynamics of Information Seeking (20)

More from TimelessFuture

More from TimelessFuture (20)

Recently uploaded

Recently uploaded (20)