IIiX2012 - Information vs Interaction - Examining different interaction models over consistent metadata

Information vs Interaction
examining different interaction models over
consistent metadata
(for now)

Kingsley Hughes-Morgan Dr Max L. Wilson
Future Interaction Technology Lab Mixed Reality Lab
Swansea University, UK University of Nottingham, UK
kingsleyhm@googlemail.com max.wilson@nottingham.ac.uk

@kingsleyhm @gingdottwit

Dr Max L. Wilson http://cs.nott.ac.uk/~mlw/

Motivation

Related Work

Information vs Interaction
Design Results Discussion


Motivation 1

Understanding Search User Interface Design


4
14 13
1

5

10 8
11

2

9

12

7
6

9

11

3

4

1 14

10
8 12 13
15

11

5
2

6
7

9
16

3

Faceted Filters


Wilson, M. L., Andre, P. and schraefel, m. c. (2008) Backward Highlighting: Enhancing Faceted Search. In:Proceedings of the 21st
Symposium on User Interface Software and Technology (UIST2008), October 19-22, 2008, Monterey, CA, USA. pp. 235-238.


So much literature


Better interaction or Better information
Query Suggestions Clustered Categories Faceted Filters


Our questions

What matters most? Interaction or Information?
!
Given!a!speciﬁc!form!of!
metadata,!can!we!recreate!more!
advanced!IIR!interface!features!
such!that!searchers!can!still!
experience!their!beneﬁts?!!


Motivation 2


Designs and Budgets


Our questions

What matters most? Interaction or Information?
!
Given!a!speciﬁc!form!of!
metadata,!can!we!recreate!more!
advanced!IIR!interface!features!
such!that!searchers!can!still!
experience!their!beneﬁts?!!

How should companies prioritise investment?


Related Work


Related Work
Query Suggestions

• Kelly
et al (2009) - query suggestions are
better than term suggestions

• Ruthven (2003) - humans not good at
choosing useful queries - algorithms should
pick them well.

• Diriye
(2009) - slow people down during
simple tasks


Related Work
Clustered Categories

• Hearst & Pederson (1996)
- better task performance

• Pirolli
et al (1996) - helped
to understand corpus


Related Work
Faceted Filters

• Hearst (2006) - careful metadata
is always better than clusters

• Wilson & schraefel (2009) -
good for understanding corpus


Related Work
(Different Interaction and Different Information)

• Joho et al (2006) -
hierarchy better than
linear list

• butused different data
structures


Different Interaction Same Information



Clustered Faceted
Query data
algorithms metadata


Query data


Our Hypotheses
H1:"Searchers"will"be"more"eﬃcient"with"more"powerful"
interaction,"using"the"same"metadata,"when"completing"
search"tasks."

H2:"Searchers"will"enjoy"more"powerful"interaction,"despite"
using"the"same"metadata.
"
H3:"Searchers"will"use"query"recommendations"more"when"
they"are"presented"diﬀerently."


The 3 Interaction Models
Query Clustered Faceted
Suggestions Categories Filters
Changes every No change on No change on
Query
search refinement refinement
Stay the same Stay the same
Filters New filters
for the query for the query
One at a time Applying filters
Experience Searching again
in Hierarchy in combination
Subset of the Subset of the
Results New results
original results original results

3 Conditions

UIQ UIC UIF
Figure 1: The three interaction conditions in the study. UIQ on the left presents query suggestions in their

Two standard types of user study task were used in the
study: 1) a simple lookup task and 2) an exploratory task.
All six tasks are shown in Table 1.
The simple lookup tasks had a fixed answer, but the chosen

2 Types of Task
task description was presented in such a way that the most
likely query would not find the answer without subsequent
queries or refinements. This approach was chosen to
intrinsically encourage participants to use the IIR features
on the left of each user interface condition.
Table 1: Tasks set to participants in the study.
S = Simple, E = Exploratory
ID S/E Task Description
1 S What is the population of Ohio? A&simple&lookup&task"="had"a"fixed"
2 E Find an appropriate review of “Harry Potter and answer,"subsequent"queries"or"
the Deathly Hallows”. refinements"were"needed.
- Compare the rating with the previous film.
3 S Find the first state of America. An&exploratory&task"="multiple"
4 E Deduce the main problems that Steve Jobs subBproblems,"required"a"series"
incurred with regards to his health. of"searches/refinements"to"
5 S What is the iPad 3’s proposed processor name? combine"answers"from"several"
6 E Explore information related to Apple’s next websites."No"fixed"answer."
iPhone, the iPhone 5. (Collection8style)
- Note the expected release date. There could well
be multiple rumours.

The exploratory search tasks were chosen to be tasks with
multiple sub-problems, such that searchers would have to
perform a series of searches or refinements to combine

18 people

16-55 (avg 28)
Mix of students,
All daily web users academic and
non-academic staff
in different schools


18 People
Intro + UI1 UI2 UI3 QA +
Consent 2 tasks 2 tasks 2 tasks Debrief


18 People

Queries
Reﬁnements
Pageviews
Time

Measures

18 People

Queries
Reﬁnements Ease of use
Pageviews Task Satisfaction
Time

Measures

18 People

Queries
Quickest
Reﬁnements Ease of use
Most Enjoyable
Pageviews Task Satisfaction
Best Design
Time

Measures

Simple vs Exploratory

Measure S E Diff

Time 176s 179s no

Queries 1.75 2.33 p<0.05

Pageviews 1.65 2.09 p<0.005

Reﬁnements 2.42 2.45 no


between the three interface conditions. These results
indicate that for simple tasks, or tasks with a fixed answer,
the different interaction models did not create a significant
Simple tasks
effect on refinement behaviour. Participants did, however,
perform significantly faster in the hierarchical clustering
UIC condition (p<0.005, F(51,2)=6.53), where a post-hoc
TukeyHSD showed that UIF and UIQ were not
significantly different from each other.
Table 2: log data for simple tasks (*=significant)
Mean (std) UIQ UIC UIF
Queries (#) * 1.22 (0.55) 2.11 (1.13) 1.94 (0.93) p<0.05
Refinements (#) 2.44 (0.70) 2.5 (1.95) 2.33 (1.19)
Page visits (#) 1.94 (1.11) 1.61 (0.69) 1.39 (0.61)
Time (s) * 189 (3.15) 154 (2.57) 184 (3.07) p<0.05

It is not clear exactly why participants submitted
significantly more queries in the UIC and UIF conditions,
but the results indicate that participants were fastest when
interacting with a hierarchy. Although we didn’t reach
statistical significance, there was a downward trend to

Simple tasks
Queries (#) * 1.22 (0.55) 2.11 (1.13) 1.94 (0.93) p<0.05
Refinements (#) 2.44 (0.70) p<0.05
2.5 (1.95) 2.33 (1.19)
Page visits (#) 1.94 (1.11) p=0.051.39 (0.61)
1.61 (0.69) (!)
Time (s) * 189 (3.15) 154 (2.57) 184 (3.07) p<0.05


Simple tasks
Queries (#) * 1.22 (0.55) 2.11 (1.13) 1.94 (0.93) p<0.05
Refinements (#) 2.44 (0.70) 2.5 (1.95) 2.33 (1.19)
Page visits (#) 1.94 (1.11) p<0.05(0.69)
1.61 p<0.05
1.39 (0.61)
Time (s) * 189 (3.15) 154 (2.57) 184 (3.07) p<0.05


Simple tasks
Queries (#) * 1.22 (0.55) 2.11 (1.13) 1.94 (0.93) p<0.05
Refinements (#) 2.44 (0.70) 2.5 (1.95) 2.33 (1.19)
Page visits (#) 1.94 (1.11) 1.61 (0.69) 1.39 (0.61)
Time (s) * 189 (3.15) 154 (2.57) 184 (3.07) p<0.05

significantly more queries in the UIC - p=~0.1
“Downward Trend” and UIF conditions,

Simple tasks
Queries (#) * 1.22 (0.55) 2.11 (1.13) 1.94 (0.93) p<0.05
Refinements (#) 2.44 (0.70) 2.5 (1.95) 2.33 (1.19)
Page visits (#) 1.94 (1.11) 1.61 (0.69) 1.39 (0.61)
Time (s) * 189 (3.15) 154 (2.57) 184 (3.07) p<0.05

It - Not much difference in #reﬁnements and #pagevisits
is not clear exactly why participants submitted
significantly more queries ininteractive conditions
- More queries (!) in the UIC and UIF conditions,
- Faster in UIC

refinements in the faceted UIF condition (F(51,2)=6.245, The choices high
p<0.005). Again, a post-hoc TukeyHSD revealed enjoyed and pref
significant differences between UIF and the two despite believing
Exploratory tasks
alternatives (both p<0.05), but no difference between UIC
and UIQ. Together, these two sets of results indicate that
baseline conditio
timing data, indi
participants behaved very differently in the three baseline felt fast
conditions, for exploratory tasks, using significantly fewer favourably in any
queries (with UIC) and significantly more refinements.
Table 5: Freq
Table 3: log data for exploratory tasks (*=significant) three co
Mean (std) UIQ UIC UIF Frequency of ch
Queries (#) * 3.11 (1.49) 1.44 (0.51) 2.44 (1.29) p<0.0005 to correct
Quickest
Refinements (#) * 2.17 (0.86) 1.78 (0.65) 3.39 (2.23) p<0.005enjoyment du
Most
Page visits (#) * 2.55 (1.04) 1.61 (0.69) 2.11 (0.75) p<0.01 appealing des
Most
Time on task (s) * 190 (3.17) 169 (2.82) 177 (2.95) p<0.01

In exploratory tasks, participants visited significantly more
5. DISCUSSI
Our study has p
pages in the original condition (F(51,2)=5.615, p<0.01),
research question
where a post-hoc TukeyHSD saw only one key difference
support searcher
between UIQ and UIC. This finding may indicate that
have a fixed form
participants were able to find more relevant pages earlier in
Dr Max L. Wilson but hard to find a
http://cs.nott.ac.uk/~mlw/

Exploratory tasks
baseline conditio
timing data, indi
Table 5: Freq
Quickest
Most
Page visits (#) * p<0.0005 (0.69)p<0.05 (0.75)
2.55 (1.04) 1.61 2.11 p<0.01 appealing des
Most
Time on task (s) * 190 (3.17) 169 (2.82) 177 (2.95) p<0.01

5. DISCUSSI
Our study has p
research question
support searcher
have a fixed form

Exploratory tasks
baseline conditio
timing data, indi
Table 5: Freq
Queries (#) * 3.11 (1.49) 1.44 (0.51) 2.44 (1.29)
p<0.05 p<0.0005 to correct
Quickest
Most
Most
Time on task (s) * 190 (3.17)p<0.05 (2.82)
169 177 (2.95) p<0.01

5. DISCUSSI
Our study has p
research question
support searcher
have a fixed form

Exploratory tasks
baseline conditio
timing data, indi
Table 5: Freq
Quickest
Most
Most
Time on task (s) * 190 (3.17) 169 (2.82)
p<0.05 177 (2.95) p<0.01

5. DISCUSSI
Our study has p
research question
support searcher
have a fixed form

Exploratory tasks
baseline conditio
timing data, indi
Table 5: Freq
Quickest
Most
Page visits (#) * 2.55 (1.04) p<0.005(0.69)
1.61 p<0.05 (0.75)
2.11 p<0.01 appealing des
Most
Time on task (s) * 190 (3.17) 169 (2.82) 177 (2.95) p<0.01

p<0.05
5. DISCUSSI
Our study has p
research question
support searcher
have a fixed form

Exploratory tasks
baseline conditio
timing data, indi
Table 5: Freq
Quickest
Most
Most
Time on task (s) * 190 (3.17) 169 (2.82) 177 (2.95) p<0.01

5. DISCUSSI
In UIC - fewer queries, fewer reﬁnements, less page visits, and faster
- exploratory tasks, participants visited significantly more Our study has p
research question
where a more use of reﬁnements (overall) and faster than UIQ
- UIF - post-hoc TukeyHSD saw only one key difference
support searcher
have a fixed form

Subjective Responses
Measure Simple
Easy of Use UIQ & UIC > UIF
Satisfaction UIQ & UIC > UIF

Question UIQ UIC UIF
Quickest to correct answer 11 5 2
Most enjoyed during task 4 11 3
Most appealing design 5 11 2

Hypotheses Revisted
H1:"Searchers"will"be"more"eﬃcient"with"more"
powerful"interaction,"using"the"same"
metadata,"when"completing"search"tasks.

• In Simple tasks - participants were faster with UIC

• In Exp tasks
- participants were better in all 4 measures with either UIC.
- UIF was faster than UIQ, making more use of reﬁnements


Hypotheses Revisted
H2:"Searchers"will"enjoy"more"powerful"
interaction,"despite"using"the"same"
metadata.

• UIC was preferred and given high satisfaction/ease of use
ratings

• UIF - however - was not.
- Participants were split in opinion


Hypotheses Revisted

H3:"Searchers"will"use"query"recommendations"more"
when"they"are"presented"diﬀerently."

• Yes - For Exploratory tasks only


Limitations

• UIF - was not perfect

• Auto-suggest - was not considered

• Measurements - were limited
- not easy to say if differences meant ‘better’
- we had no sense of relevance
- could beneﬁt from TREC style measurements

• Num Participants - was not big


Conclusions
What did we actually learn?

• We did see different behaviour in all 3 conditions

• People were good at simple tasks with original UIQ

• People were faster and more effective with UIC
and preferred it

• People used more ﬁlters and viewed fewer pages with UIF
but did not like it so much

• But is it better or worse behaviour?

Future Work

Query data


Future Work

Clustered Faceted
Query data
algorithms metadata

Future Work
Facets

Clusters
Performance

Suggestions

(hypothetically)

IIiX2012 - Information vs Interaction - Examining different interaction models over consistent metadata

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (12)

Similar a IIiX2012 - Information vs Interaction - Examining different interaction models over consistent metadata

Similar a IIiX2012 - Information vs Interaction - Examining different interaction models over consistent metadata (20)

Más de Max L. Wilson

Más de Max L. Wilson (15)

Último

Último (20)

IIiX2012 - Information vs Interaction - Examining different interaction models over consistent metadata