Findability and Usability Lessons from Text Analytics Research

Findability and Usability
Lessons Learnt from Text Analytics
Anna Divoli
@annadivoli
Pingar Research
CS4HS 2013 @ Unitec

twitter, July 2006Google, September 1998
http://www.laurencecaro.co.uk/other/what-the-worlds-biggest-websites-looked-like-at-launch/

http://www.laurencecaro.co.uk/other/what-the-worlds-biggest-websites-looked-like-at-launch/
Yahoo!, March 1995
Facebook, February 2004

http://www.whoisandrewwee.com/facebook/the-new-facebook-hot-or-not/

Often there is resistance!
Anna Divoli CS4HS 2013 @ Unitec

http://imgur.com/gallery/U8C1r
Users adapt & evolve as well!
15

• Familiar Interfaces
• User Centered Design & Usability
• Who am I & who is Pingar
• Text analytics in a nutshell
• Our usability studies & findings
• 1. Search interface features: Biomedical Scientists
• 2. Study existing popular systems
• 3. Mock ups of specific features
• Take away message
Outline

http://blog.lib.umn.edu/torre107/si/pics/superficialintelligence14.jpg
Usability testing is important! Needs to be done right!

User Centered Design
Design
Prototype
User
Feedback
Analysis

http://headrush.typepad.com/photos/uncategorized/focusonusersnotcompetitors.jpg
http://headrush.typepad.com/creating_passionate_users/2006/07/
User Centered Design – What should companies focus on!
FOCUS
Users Competitors

• Eye tracking
• Mouse clicks
• Tasks (goal & time)
• Questionnaires
• Observation
• Talk aloud
• Brain scanners
• ….
Usability testing tools
Measurements & Preferences on:
• Aesthetics
• Craftmanship
• Technical performance
• Ergonomics
• Usability

http://gazehawk.com/blog/
Eye tracking

Who am I?
UK
BSc (1999) Biomedical Sciences
.
MSc (2001) Biosystems & Informatics
.
PhD (2006) Biomedical Text Mining:
- Sentence extraction
- Automatic protein family database
annotation
- Document clustering and
visualization
USA (2006-2011)
Postdoctoral Academic Research:
- Text analytics
- Information retrieval
- User search interfaces
- Usability research
- Knowledge acquisition
- Expert opinions analysis
NZ (2011- )
Pingar Research:
- Entity Extraction
- Automatic Taxonomy Generation
- Taxonomy Editor (UI)
- Friendly UIs
- Usability & UX

Who is Pingar?
A small global company that uses state
of the art text analytics to offer
solutions!
We analyse unstructured text!
Entity Extraction
People, Organizations, Locations
Dates, Money amounts, Email
addresses…
+ Custom Entities
Keyword Extraction
Taxonomy Term Extraction
Domain specific terminology (can
plug in any taxonomy/ontology)
Summarization
Select length and focus
Redaction & Sanitization
Ensure anonymization of sensitive
information.
In English, German, French,
Spanish
Dutch, Chinese, Japanese

research
A brief Text Analytics Overview – just Entity Recognition basics…

S. Arlington initiated partnership discussions during his visit to Eureka’s Ltd
offices last month.
John Smith went to Washington to see the Smithsonian and also met up with
Virginia for a coffee.
Find and classify names…
Information Extraction – Named Entity Recognition

offices last month.
Find and classify names… People
Locations
Organizations
Methods: lexicon-based (gazeteers)
grammar-based (rule-based)
statistical models (machine learning)
hybrids

offices last month.
Locations
Organizations
Methods: lexicon-based (gazeteers)
grammar-based (rule-based)
✓ statistical models (machine learning)
✓ hybrids

offices last month.
Locations
Organizations
Who? Where?
When?
Dates

offices last month.
Word Sense Disambiguation: identifying which sense/meaning of a word is
used in a sentence, when the word has multiple meanings.
Text normalization: transforming text into a single canonical form that it
might not have had before.
Disambiguation & Normalization:
Word Sense Disambiguation & Text Normalization

offices last month.
Sam Arlington initiated partnership discussions during his visit to Eureka
offices in September.
J. Smith went to Washington DC to see the Smithsonian Institute and also
met up with Virginia Peterson for a coffee.
Word Sense Disambiguation & Text Normalization

What?
offices last month.
Fact & Relationship extraction

How? Why? How do we feel about it?
offices last month.
Deeper knowledge & Sentiment

How? Why? How do we feel about it?
offices last month.
S. Arlington visited the Eureka’s Ltd offices last month to initiate partnership
discussions.
John Smith was delighted to go to Washington to see the Smithsonian and
also met up with Virginia for a coffee.
Deeper knowledge & Sentiment

S. Arlington did not initiate partnership discussions during his visit to Eureka’s
Ltd offices last month.
John Smith went to Washington to see the Smithsonian and almost also met
up with Virginia for a coffee.
Negation… Hedging… Conditional phrases…
Also:
Name boundaries… Disambiguation… Normalization…
Common problems

Eye drops off shelf.
Include your children when
baking cookies.
Turn right here.
John saw the man on the
mountain with a telescope.
He gave her cat food.
They are hunting dogs.
Human language!

Looking for: interactions between SAF and viral LTR elements
(SAF is a transcription factor, LTR stands for ‘long terminal repeat’)
Gene names:
tinman, lilliputian, dreadlocks, lush, cheap date, methuselah, Van Gogh,
maggie, brainiac, grim, reaper, cleopatra, swiss cheese, ken and barbie,
kenny, out cold, lava lamp, hamlet, sonic hedgehog, werewolf, half pint, fucK,
drop dead, chardonnay, agnostic, I’m not dead yet…
Example: Biology…

Knowledge management - Extracting metadata
Search engines
Question Answering
Text mining / knowledge discovery
Analytics & trend analysis
Applications

Specialized domain study:
1. Search interface features: Biomedical Scientists (HCIR)
Anna Divoli and Alyona Medelyan
Search interface feature evaluation in biosciences, HCIR 2011, Google, Mountain View, CA
UI preferred features studies:
2. Study existing popular systems (EuroHCIR)
Matthew Pike, Max L. Wilson, Anna Divoli and Alyona Medelyan
CUES: Cognitive Usability Evaluation System, EuroHCIR 2012, Nijmegen, Netherlands
3. Mock ups of specific features (survey)
Our User Studies

1. Search interface features: Biomedical Scientists

Anna Divoli and Alyona Medelyan, Search interface feature evaluation in biosciences,
HCIR 2011, Google, Mountain View, CA, USA
Facets – favorite feature for search systems

animal models huntington disease
Facets (in search systems)

Most liked Least liked
animal models huntington disease
Bio-Facets

• Faceted search is the most important stand alone feature in a search
interface for bioscientists.
• Few, query-oriented facets presented as checkboxes work best.
• Overly simple aesthetics, although not desirable, do not hurt overall UI score.
• Complex aesthetics turn users away from the systems.
• Bioscientists prefer tools that help them narrow their search, not expand it.
• For generic search: doc-based facets.
For domain-specific search: query-based facets.
Facets as search features for biomedical scientists: Findings

Search expansions★
Facetted refinement
Related searches
Results preview★
- hard to read
Semedico
- hard to read
NextBio
- noisy, many symbols
GoPubMed
- small font size
PubMed
- too general
- not useful
+ overall look
- color
Semedico
+ g
- unc
+ useful categories
- slow functionality
- too complex/busy
- too many colors
Semedico
+ “reviews” category
- limited functional.
- poor design
PubMed
+ quick paper access
+ simple
+ vertical list
- nothing special
Solr
+ “top terms” useful
- too many symbols
- too busy
- colors
GoPubMed
+
- not
- not scientific
+ colors
- too small
- too busy
Bing
+ relevant
- poor context
- no variety
PubMed
+ s
- snipbr
ff
ig
br
ff
ig
br
ff
ig
br
ff
ig
ig
• Useful categories
• Simple
• Vertical list
• Too complex/busy
• Too many colors
• Poor design
• Limited functionality
• Too many symbols
• Not special/ Colorless
Facets as search feature: likes & dislikes

Exploring UI features - Systems Tested: Yippy, Carrot, MeSH, ESD

Boston Oct 2012
A
A B C D E F
B
C
D
E
F
A B C D E F A B C D E F A B C D E F A B C D E F
Exploring UI features - Systems Tested: Yippy, Carrot, MeSH, ESD

• Menu highlighting
• Hierarchical folder layout
• Expand hierarchy with “+” and “–”
• Dual view (tree on left, results on right)
• Ability to change visualisations of taxonomy
• Search function is important
• Familiar interface with folders
• Too simple or too much writing - would be nice to have color
• Lots of scrolling
• Dots in carrot circle – confusing
• Double click on foam tree is unintuitive
• Too broad taxonomies
Exploring UI features (Yippy, Carrot, MeSH, ESD): likes & dislikes

60.0%26-40
12.7%41-60
0%61 or older
27.3%25 or younger
Age:
52.7%College/University
43.6%Graduate School
3.6%High School
Highest level of education:
47.3%Yes, but very little
21.8%Yes
30.9%No
Do you have experience using taxonomies?
47.3%Very
47.3%Second nature
5.5%Somewhat
How comfortable you are with computers?
Taxonomy UI preferences: The 51 participants

44.2%popularity (A)
42.3%alphabetically (B)
13.5%no preference
Concept sorting

42.3%A
51.9%B
5.8%no preference
Displaying Counts

72.5%in frames (A)
23.5%with labels (B)
3.9%no preference
Using Labels

47.1%A
37.3%B
15.7%no preference
Plus/minus signs or arrows

11.8%B
70.6%C
3.9%no preference
13.7%A
Search Results Display

74.5%partial
64.7%hidden
2.0%no preference
Search Functionality

Outcome: User-friendly Taxonomy Editor

Lessons Learnt
Perfection is achieved, not when there is nothing more to add,
but when there is nothing left to take away.
Antoine de Saint-Exupery, French writer (1900 - 1944)
• Follow basic design principles: alignments, right use of colors, etc
• Avoid clutter! Go for clean and simple always!
• Aim for intuitive User Interfaces.
Do not make users think!
• You will never please everybody! Choose the users and use cases
you plan to support.

Ignas Kukenys
PhD (2010) in Computer Vision,
University of Otago
Previously: NextWindow
Expert in: computer vision, machine
learning, classification methods
Alyona Medelyan
PhD (2009) in Natural Language
Processing, University of Waikato
Previously: Google, European Media
Lab
Expert in: keyword and entity
extraction, Wikipedia mining
Ralf Heese
.
Diploma in Computer Science, PhD
candidate, Humboldt-Universität
Berlin
.
Previously: Humboldt-Universität
Berlin, Freie Universität Berlin
Expert in: semantic databases,
semantic technologies
Andy Chao
BSc (2011) in Computer
and Information
Sciences, AUT
Hayden Curran
Computer Science,
University of Waikato
(2008-2010)
Collaborators
Annika Hinze Ian Witten Max Wilson Jeen Broekstra
Alumni
Matthew Pike Steve Manion David Milne Jose Chen Anna Huang
Anna Divoli
.
PhD (2006) in Biomedical Text Mining,
University of Manchester
.
Previously: University of Chicago,
UC Berkeley, Inpharmatica
Expert in: biomedical text mining, text
analytics, user search interfaces,
usability
Waikato Uni Waikato Uni Nottingham Uni Rivuli Development
Nottingham Uni PhDc Canterbury Uni PhDc CSIRO postdoc NIH fellow Yangtze Uni Lecturer
Acknowledgements: Pingar Research Team
And all 65+
anonymous
studies
participants!

http://farm5.static.flickr.com/4087/5019944289_5e2f0637c9.jpg
Take away message: Involve the user!

Findability and Usability Lessons from Text Analytics Research

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

Similar to Findability and Usability Lessons from Text Analytics Research

Similar to Findability and Usability Lessons from Text Analytics Research (20)

Recently uploaded

Recently uploaded (20)

Findability and Usability Lessons from Text Analytics Research