1. Simplifying the search experience –
resisting the lure of shiny, new
technology
Ronán Kennedy
Monica Crump
NUI Galway Bournemouth, April 8th-10th 2013

2. School Institute Name to go here

3. School Institute Name to go here

4. • Why we did what we did
• Some things we did, we shouldn’t have
• Some things we didn’t do, we should have
• How we found what we should have done and what we did about
it, and what we did about undoing what we shouldn’t have done
• The Epihany: we ticked all the boxes librarians like ticking.
They were the wrong ones

5. Discovery Layers
• A preharvested central index coupled with a richly featured
discovery layer providing a single search across a library’s local,
open access, and subscription collections (Hoeppner 2012, source)
• One point of access to the Library’s entire set of bought, licensed
and digital collections (Lorcan Dempsey, source)

6. www.librarytechnology.org

7. Primo
• Sits Over:
• Aleph
• Primo Central
• Metalib
• Institutional Repository

8. 2009
• Primo Implementation Group
• Primo
Implementation
Group
• Interface Group
• subject librarians; research librarians; e-resource librarian,
etc.

9. 2009
• Web Group
• Primo to have prime real-estate space on the Library
homepage
• ‘primo lite’

10. School Institute Name to go here

11. Initial Interface Work
• Demos of out-of-the-box version
• Examples of global sites – NYU, UAE, etc
• Interface Group members liaison role

12. Concerns – Radical Departure
• New interface could be radically different
• Big departure from Aleph (function & cosmetics)
• Academic opposition
• Possible training problems

13. School Institute Name to go here

14. School Institute Name to go here

15. Conclusion #1 – too conservative
• Concerns heavily shaped our vision
- tried hard to integrate the past
- fear of academics
• We tried to mould our discovery platform into something it
wasn’t

16. Concerns – Dumbing Down
• One single simple interface = bad information literacy
• “Primo should be a flagship model of good pedagogical
practise”
• Multiple tabs made sense to us

17. Lots of cooks in the kitchen
• Subjective opinions are always difficult
• “Is it exposing our resources appropriately?”
• Font sizes!
source

18. 2009
• Interface group assembled the jigsaw pieces
• Model followed closely with NYU, Iowa, British Library, UEA
• Time to caress the divine details….

19. Error Checking
• Librarians love detail!
• A lot of librarians means lots of errors being found
• Sub-conscious opinions being formed?

20. School Institute Name to go here

21. Perfection at all costs?
• Did we get bogged down in making sure it was perfect?
• Special Collections
Special Collections
Special Collections Reference
Strong Room
• Librarians want perfection – users want good enough
• Risking not seeing the wood for the trees

22. Customisation – Tread Carefully
• Discovery interfaces are*very* flexible – lots of options
• Cosmetics and functionality very open

23. School Institute Name to go here

24. Customisation – Tread Carefully
• Discovery interfaces are*very* flexible – lots of options
• Cosmetics and functionality very open
• Tempting to generate completely local feel
- can be timely
Just because you can push a button, it doesn’t mean you have to

25. Customisation – Tread Carefully
• Beware of skills sabotage

26. Primo launched Sept 2009
New website officially launched Sept 2010

27. School Institute Name to go here

28. School Institute Name to go here

29. How was it used?
• 80/20 Theory
• 1.5 million Basic Searches
• 50,000 Advanced Searches
• 14 reviews written
• Several hundred items tagged

30. LibQual Survey
• November 2010
• LibQUAL+® is a suite of services that libraries use to
solicit, track, understand, and act upon users' opinions of
service quality.
• Global; great for benchmarking

31. LibQual - Information Control Remit
IC-1 Making electronic resources accessible from my home or office
IC-2 A library website enabling me to locate information on my own
IC-3 The printed library materials I need for my work
IC-4 The electronic information resources I need
IC-5 Modern equipment that lets me easily access needed information
IC-6 Easy-to-use access tools that allow me to find things on my own
IC-7 Making information easily accessible for independent use
IC-8 Print and/or electronic journal collections I require for my work

32. LibQual - Information Control Remit
minimum desired perceived adequacy superiority
IC-1 Making electronic resources accessible from my home or office Overall 6.42 7.84 6.49 0.07 -1.35
IC-2 A library website enabling me to locate information on my own Overall 6.53 8.00 6.49 -0.03 -1.51
Let’s form a committee!!!

33. School Institute Name to go here

34. LibQual - Information Control Remit
Polarised Feedback
There was clearly a problem – but what was it???

35. What LibQual told us:
• We weren’t meeting our users’ minimum expectations of:
“A Library website enabling me to locate information on my
own”

36. Comments Analysis
The website is
Online difficult to use Navigation of
services the website is
are great! confusing
Access to e-
resources needs The website is
Off-campus
to be more user- easy to use
access works
friendly since the re-
very well
design

37. More investigation was needed!

38. User Observation Study
• Real Users
• Real Tasks
• One user – One observer

39. We wanted to find out:
• where were users becoming confused
• user interface features that users didn’t notice
• whether and when users logged-in to enhance
search results
• additional features/on-screen guidance that might
have benefitted users’ search experience
• Any issues affecting the success of the users’ search

40. Structure of observation
• Pre-observation interview
• Set list of tasks
• Think aloud:
– Explain choices
– Highlight anything that’s confusing
– Express pleasure or frustration
– Explain how you would carry out a similar task in real life

41. User Focus Groups
• Focus groups - for a wider perspective and to confirm
findings
• Two groups – Undergrads; Postgrads & Staff
• Caution – those willing to attend tend to be more library-
focussed, library-aware and therefore library-positive!

42. Key Findings
• Our in-depth knowledge and closeness to the website and
discovery tools had blinkered us to the experience of a
‘normal’ user
– user observation was like blinkers being removed
• Overwhelming message from our users was:
“make it simple – more like Google”

43. Too many options!
• Most of the problems, frustration and confusion seemed to
arise from the user being presented with too many options:
– multiple links to our discovery tools
– pages about resources confused with links to resources
• Our knowledge of the fantastic range of choices we could
offer the user had stopped us from choosing wisely what we
should offer them

44. Too many options!
Where do
I look for
a book?

45. Problems of terminology
• Other sources of confusion arose once they navigated into
the catalogue or discovery tool
• Users found terminology confusing
Why can’t you
just call things
what they are!

46. Users tended not to notice the ‘Get it’ and
‘Online Access’ links and there was some
confusion about what the wording meant

47. FRBR - A little too clever?
• FRBRisation is a powerful tool aimed at presenting the user
with a single result representing several works that are very
similar and are likely to be of similar value.
• However, this posed great difficulties and caused confusion:
– Primo chooses a preferred record to display in the results.
– If this preferred record had only unavailable items – it
looked like everything is unavailable
– Users didn’t click into displayed record to find alternative
editions that were hidden behind

48. Preference for Online?
• Design premise of Primo is that online resource is users’
preferred resource
• Users were unable to see the print holdings of journals that
had print and electronic access

51. Other interesting findings
• Heavy reliance on Google by all participants
• Prior attendance at library training hugely improved the
users’ success in completing the set tasks
• Several undergraduates were confused about references:
– unsure what was a journal and what was a book
– searched by article title instead of journal title
• Users with experience of other libraries were emphatic that
our online services were significantly easier to use than
elsewhere.

52. Call to Action:
Simplify the Search Experience

53. Introduce a Single Search Box

54. Improve visibility of Login

55. What about the other functionality?

56. Find Databases

57. More Search Options

58. Journal Article Searching

59. Primo Central
• Primo Central is a single massive index of metadata that has
been harvested from primary and secondary publishers, from
aggregators, and from open-access repositories
• Primo Central indexes hundreds of millions of journal
articles, e-books, reviews, legal documents etc.
• Because the metadata has been harvested into a single
index, it’s faster because it’s not cross-searching
• Facets allow users to refine and narrow down their results

61. Browse functionality

62. Other Improvements
• OPAC via Primo
– Clearer
account
information
– Clearer item/
holdings
information

63. Making print holdings more visible

64. Have we succeeded?
• Have we simplified the search experience?
• Are we now meeting users’ expectations?
• It’s not enough to make changes - essential to check back
with the users whether we’ve got it right!

65. Libqual 2013
• Still some comments that the website is confusing and hard
to navigate
• But a 50% increase in the number of positive comments
about the website/catalogue!
• 85% decrease in the number of negative comments about
off-campus access!
• LIBQUAL SCORE???

66. Feedback from Students
• Significantly less difficulty completing the set
Much better tasks
this year
than it was • Big demand for easier access to ‘My Account’ –
before but all managed to check their account
• Print content still not being seen – confusion over
terminology ‘Print Locations’
Awesome • Still reporting a heavy reliance on Google/Google
colours! Scholar

67. Feedback from Academic Staff
• Biggest complaint from academic staff is the It seems to
additional steps to get to their favourite have more
database! steps to get
where you
want
• Some were very positive about ‘Including
Articles’ search, but most hadn’t noticed or
tried it. A fantastically
convenient
• Timing of launch an issue just ahead of start of new option
term.

68. Feedback from Library Staff
Over-whelmingly positive:
The new
catalogue is
An enjoyable and amazing!
fruitful search
experience
Intuitive and
intelligent
It mirrors the
layout and
layout in many
design
of our
databases

69. Some Final Thoughts
• Evolving user behaviour is very evident:
– Students are using our discovery tool to search both for
AND within resources
– Expectation growing that everything will be online
• Difficulty of meeting different expectations:
– Academics expect library tools to point and link them to
resources
– They don’t think to simply search!

70. What next...
• Very valuable to ask users what they think
• Even more valuable to observe them using your discovery
tools
• We’ve had some clear signals about further adaptations we
should make
• This is an ongoing process, and we intend to make user
observation a regular practice to ensure we keep up with
evolving user needs and behaviour.

71. Thank You!
• Ronán Kennedy ronan.kennedy@nuigalway.ie
• Monica Crump monica.crump@nuigalway.ie