DIVE+ thesis defense presentation Minyi Cheng

DIVE : User
Interaction Design
with Digital Humanities
Thesis defense of Minyi Cheng
+
https://uncommoncarib-wpengine.netdna-ssl.com/wp-content/uploads/2016/01/Dive-into-Exuma-e1452889402298-1200x656.jp

What are we
discussing?
The user experience of an
exploratory search engine

Traditional search
/ helps only when you know what to search for
/ assumes you understand a topic
/ accuracy driven
/ click-through driven

Traditional search VS. Exploratory search
/ helps only when you know what to search for
/ assumes you understand a topic
/ accuracy driven
/ click-through driven
/ helps when you don’t know what to search for
/ helps you understand & deepen in a topic
/ serendipity driven
/ focused on engagement

- Focuses on supporting Digital Humanities Scholars
in their development of research questions and explorations
- Access to Integrated Online Multimedia collections
using Linked Open Data to integrate metadata of various heritage
collections
- Interactive Exploration & Discovery in Context
linking objects to events and entities
building automatic storylines (narratives)
Use Case: DIVE+

What are events?
● a relationship between an object and event with a historical
meaning (Van den Akker et al., 2011)
● Things that happen
● Events are defined by the SEM model: actor, place, time and
type (De Boer et al., 2015)

What are events?
● a relationship between an object and event with a historical
meaning (Van den Akker et al., 2011)
● Things that happen
● Events are defined by the SEM model: actor, place, time and
type (De Boer et al., 2015)
● Example:
Actor: Mohammed Toha
Place: Yogyakarta
Time: 19 December 1948
http://www.oceansbridge.co.uk/paintings/museums/rijks-museum/small/The-President-the-Vice-Pesident-and-other-leaders-are-exiled-to-Sumatra-1948-1949-Z
Z-Mohammad-Toha-Adimidjojo.1050.jpg

What are narratives?
● two or more events that are related with each other
● storyline of events
Queen Beatrix

What are narratives?
● two or more events that are related with each other
● storyline of events
Queen Beatrix
narrativenarrative
The Coronation
Relationship with Claus

Problem?
http://usabilitygeek.com/wp-content/uploads/2015/04/how-difficult-to-design-user-experience.jpg
- Usability of the User Interface of DIVE+ for Digital Humanities
- Requires complex interaction and support that is not always
intuitive
- Not able to efficiently use DIVE+ for their research

Research questions
“How efficient and effective is the exploration based on ‘events’-enriched
collections and ‘narratives’ support for Digital Humanities scholars for their (1)
understanding of objects, context and collections, and for their (2) discovery of new
insights in the object of study?”
- what is the role of events and event types in understanding of objects and their context?
- what is the role of narratives for the discovery of new insights, objects, links and other information?
- how effective and efficient is the DIVE+ Browser (events & narratives) UI/UX for the typical tasks
that Digital Humanities scholars perform during their exploratory search?

Humanities Scholars Research stages
● Four stages (Collijn, 2016; Melgar Estrada, 2016)
○ Exploring stage
○ Assembling stage
○ Analysis stage
○ Presenting stage

Humanities Scholars Research stages
○ Exploring stage
○ Analysis stage
- idea generation
- preliminary research
- decide if collection is suitable
- information gathering
- refine a research question/topic

User groups
● 18 Digital humanities scholars (DHS)
○ 6 scholars and 5 students
● 11 Media professionals (MP)
○ 5 broadcasting organisations and
6 libraries and archives
● 16 (Digital) humanities students (HS)
○ students from RUG
● 22 Computer science students (CSS)
○ students from VU HCI course

User groups
● 18 Digital humanities scholars (DHS)
○ 6 scholars and 5 students
● 11 Media professionals (MP)
○ 5 broadcasting organisations and
6 libraries and archives
● 16 (Digital) humanities students (HS)
○ students from RUG
● 22 Computer science students (CSS)
○ students from VU HCI course
Results taken
into account

Workshops
● CreateSalon Workshop (1 hour)
○ 18 Digital Humanities Scholars
○ 9 May 2017 @ UvA CreateSalon
● MediaNow Workshop (3 hours)
○ 11 Media professionals
○ 24 May 2017 @ Dutch Institute of Sound and Vision
● Data collection I (online)
○ 16 (digital) humanities students
○ May-June 2017
● Data collection II (online)
○ 22 computer science students
○ June 2017

Evaluation Tools
● Simulated work task situation 1
● Think-aloud protocols
● Questionnaire 2
● Focus group
1 https://docs.google.com/document/d/176cwCfo21bZIFCOjvZrdD95_TcVslCxAveffMXLK6GI/edit?usp=sharing
2 https://docs.google.com/forms/d/1sL9uYo_Zl56Nb-XFEzIvwV5c1HFEbFuifm9oXR8h-LU/edit?usp=sharing

Reflection Methods
● What went well?
○ Workshops was constructed well due to the formed
protocols
○ The participants were pro-active with the simulated work
task
○ Through the questionnaire and task, we received a lot of
feedback/suggestions

Reflection Methods
● What went not so well?
○ Due to time constraints, the think aloud protocols were
conducted in a group of users. The conductor walked by
with a recorder and users had to think their thoughts out
loud.
■ Suggestion to improve: conduct the think aloud protocol
individually on each user while performing the task. This
will give better results, and the focus can be on one
person instead divided over the whole group.

Where there limitations?
● Shortcoming of time during workshop
○ users were not able to fully experience DIVE+
● DIVE+ browser
○ overload → error
○ users were not able to perform task

Results3
● Task performance
● Narratives
● Events
● DIVE+ UI Evaluation
● Comparison with other
groups
- Digital
Humanities
Scholars (DHS)
- Media
professionals (MP)
- (Digital)
humanities
students (HS)
- Computer science
students (CSS)
3 https://docs.google.com/document/d/1Uq4eGAmAD7oWa3zX6fQPWLEI5yVkhj-1I-Ku3SRo8sk/edit?usp=sharing

Results
DIVE+ Task performance of DHS ● 5/11 participants found an
angle

Results
DIVE+ Task performance of DHS ● 5/11 participants found an
angle
● struggled with interface
● which collections?

Results
DIVE+ Task performance of DHS
“I didn’t know how to tailor my search to the
collection (of objects & texts) because I didn’t know
what the collection was.” - DHS participant 3
● 5/11 participants found an
angle

Results
DIVE+ Task performance of DHS
“I didn’t know how to tailor my search to the
collection (of objects & texts) because I didn’t know
what the collection was.” - DHS participant 3
● 5/11 participants found an
angle
● data limited

Results
DIVE+ Narratives of DHS

Results
● did not discover different narratives
● would like to see suggested narratives
● exploration path tool seen as useful
● not clear how the exploration path was
constructed
● the path was now seen as a list of search
results
● would like to interact more with the path
(possibility to re-arrange)

Results
● did not discover different narratives
● would like to see suggested narratives
● exploration path tool seen as useful
● not clear how the exploration path was
constructed
● the path was now seen as a list of search
results
● would like to interact more with the path
(possibility to re-arrange)
●
“ The navigational aspect confused me: I
struggled with organising a cohesive
research path, and found myself
wondering why the things are organised in
such an order, instead of looking for more
material. “
- DHS participant 11

Results
DIVE+ Events of DHS
● did not notice the event characteristics
e.g. person, media object etc.
● did not understand the purpose of events
● did not find the different event
characteristics useful
● the event characteristics did not narrow
down enough the overload of results
● users need more fine-grained event
characteristics e.g. radio, tv or image.

Results
DIVE+ Events of DHS
“For me, the media object was way too
general. I’m a media scholar, not a
historian, so saying there are 400 media
object, includes from photos from a
museum to.. It is not enough. I need a
more granular selection. [the filters]
didn’t eliminate the type of media what’s
on offer for example, at least in a rough
categorization.” - DHS participant 3
● did not notice the event characteristics
e.g. person, media object etc.
● did not understand the purpose of events
● did not find the different event
characteristics useful
● the event characteristics did not narrow
down enough the overload of results
● users need more fine-grained event
characteristics e.g. radio, tv or image.

Results
DIVE+ Evaluation of DHS

Results
● The DHS think DIVE+ could be useful, but
not sure if they would use DIVE
● There was not enough information about
the objects e.g. source or date
● Some buttons on the UI unclear e.g.
‘bookmark’ and ‘save to path’
● confusion about ‘related searches’, it’s
experienced as not related at all

Results
● The DHS think DIVE+ could be useful, but
not sure if they would use DIVE
● There was not enough information about
the objects e.g. source or date
● Some buttons on the UI unclear e.g.
‘bookmark’ and ‘save to path’
● confusion about ‘related searches’, it’s
experienced as not related at all
“The ‘related entities’
suggestions were not very
helpful, and it was not clear
how they were related.”
- DHS participant 7

Results
Comparisons between other
groups
● Media professionals
● (digital) humanities students
● computer science students
● Purpose of narratives was unclear among
MP, DHS, and HS.
● Almost every group would like to see
suggested narratives, except the MP
● All groups missed some characteristics of
events
● DHS, MP and HS would like to see more
fine-grained filtering among the
characteristics.
● The usefulness of event characteristics
was experienced negative among 3 out of
4 groups → HS did see the usefulness
“It’s unclear why search results
receive certain tags. Thereby it’s
impossible to acknowledge any
value to it.” - MP Participant 2

Results
Differences between other
groups
● Media professionals
● (digital) humanities students
● computer science students
● Humanities scholars see potential in
DIVE+, other groups not convinced
● Only DHS mentioned to have more
interaction with the exploration path
● HS would like to see the exploration path
easier or with instructions to use
● MP need more information about relations
between queries and results, and why
object have certain labels
● MP want to filter between collections, and
media objects
● DHS and MP have their own way of doing
research.

FINDINGS
● More information is needed
○ collections
○ (meta)data
○ topic descriptions

FINDINGS
○ collections
○ (meta)data
● ‘related searches’ is experienced as not related

FINDINGS
○ collections
○ (meta)data
● trouble with event characteristics
○ need more fine-grained sorting of media types

FINDINGS
○ collections
○ (meta)data
● exploration path is not optimal in use
○ does not result in narratives
○ the purpose is not clear
○ need more interaction with the tool

FINDINGS
○ collections
○ (meta)data
● exploration path is not optimal in use
○ does not result in narratives
○ the purpose is not clear
○ need more interaction with the tool
● unclear what some UI buttons meant

Humanities Scholars Research stages and purpose
○ Exploring stage
○ Analysis stage
- idea generation
- preliminary research
- decide if collection is suitable
- information gathering
- refine a research question/topic

Recommendations
Finding: more information is needed about which collections there are in DIVE+, so
the digital humanities scholars know how to search.
Recommendation: include on the start page what the collections are, and what are
in those collections (size, type of objects and more)

Recommendations
Finding: ‘related searches’ is experienced as not related, the digital humanities
scholars don’t see the relationship between the searches/results.
Recommendation: adapt the back-end algorithm such that the related searches
are only related to the current query.

Recommendations
Finding: the digital humanities scholars experience trouble with the event
characteristics. They are having trouble understanding it or see the purpose of it.
The purpose of the event characteristics was also experienced troubling, as it
does not narrow down enough the results.
Recommendation: include a more fine-grained filter of the event characteristics
e.g. more filters for Media Object (radio, tv, image)

Recommendations
Finding: the exploration path is not optimal to use. It’s now seen as a search
result list because there is no possibility to form an interesting narrative.
Recommendation: add more interaction to
the exploration path
To form a narrative:
○ there should be the possibility to drag and drop interesting queries
○ there should be the possibility to categorizing interesting
queries with colors
○ there should be the possibility to re-name queries with the same title

Recommendations
Finding: It was unclear what some UI buttons meant, some buttons were
experienced as the same purpose e.g. ‘add to path’ and ‘bookmark’. Also the
events button was seen as an calendar or date button instead of an events button.
Recommendation: Put the title of the button in text together with the button. So
the purpose of the button is clear for users.

Looking back
“How efficient and effective is the exploration based on ‘events’-enriched collections and ‘narratives’
support for Digital Humanities scholars for their (1) understanding of objects, context and collections, and
for their (2) discovery of new insights in the object of study?”

Conclusion
“How efficient and effective is the exploration based on ‘events’-enriched collections and ‘narratives’
support for Digital Humanities scholars for their (1) understanding of objects, context and collections, and
for their (2) discovery of new insights in the object of study?”
→ The role of events and event types in understanding is minimum, the events and event types is
more used to form a narrative or narrow down results
→ the current narratives support is not optimal, which leads to a minimum discovery of new
insights, objects, links and other information
→ the current UI/UX of the DIVE+ browser is not that effective and efficient for the digital
humanities scholars to perform their typical tasks.

Conclusion
● The current interface has some shortcomings of DIVE+.
● The ‘events’-enriched collection is experienced as an overload of results and the event types does
not help in narrowing down this overload.
● The ‘narratives’ support is not optimal to form a narratives.
● With the given recommendations implemented, improvement will take place in the exploration but
more user evaluations are needed in the future.

Conclusion
● The current interface has some shortcomings of DIVE+.
● The ‘events’-enriched collection is experienced as an overload of results and the event types does
not help in narrowing down this overload.
● The ‘narratives’ support is not optimal to form a narratives.
● With the given recommendations implemented, improvement will take place in the exploration but
more user evaluations are needed in the future.
Thank you!

DIVE+ thesis defense presentation Minyi Cheng

Recommended

Recommended

More Related Content

Similar to DIVE+ thesis defense presentation Minyi Cheng

Similar to DIVE+ thesis defense presentation Minyi Cheng (20)

Recently uploaded

Recently uploaded (20)

DIVE+ thesis defense presentation Minyi Cheng