1. MAUI Debrief Usability Study
13 – Sept -2010
S
c
Totals
e
Findings n Resolution
a
ri
o
Scenario 1 S=Yes 1 11
Scenario 2 S=Yes 2 10
Scenario 3 S=Yes 3 9.5
Scenario 4 S=Yes 4 8
Scenario 5 S=Yes 5 6
Scenario 6 S=Yes 6 3
Scenario 7 S=Yes 7 9
Scenario 8 S=Yes 8 10
Scenario 9 S=Yes 9 10
Scenario 10 S=Yes 10 1
Scenario 11 S=Yes 11 9
Scenario 12 S=Yes 12 7
Scenario 13 S=Yes 13 9
Scenario 14 S=Yes 14 9
Login to App
1. Confused logging in to the app with login on 6 1 Consider auto-login
the computer.
2. Did not realize that she had permission to 6 2
log in to MYMADB

2. 3. Was not sure how login after clicking on 6 1
MyMADB
4. Did not think she could login as she did not 6 2
have an account
5. Remove login and put login fields on page. 6
Search Options
6. Chose title field then went back to all fields 1 1 Remove the Book title field
for the basic search
7. Understood that searched word does not 1 1 Remove the Book title field
have to be in title
8. OBS: Do we need all keyword option 1 Remove radio buttons on basic
search
9. Wanted to clear search 2 1 Mayo Authors - Click on Author to see
Publication list
10. Never used or seemed to understand the 3 1 No change
Combine function
11. Thought of year as a limit 3 1 Ok
12. Could have used Author search on basic 4
page but went to Advanced
13. Searched all authors field instead of the 4 2 Look at position of author search
Mayo Author field on adv. Move below search box,
maybe call All Authors Other
Authors
14. She typed the full name for JAMA and was 8 1 good
asked to use JAMA
15. He wondered if you can type in partial title 9 1 good
instead of full, he experimented and found

3. you can
16. Wondered if she could abbreviated the 8 2 Can
article title
17. Tried to do as a title search and it did not 9 1 Got rid of one title field - fixed
work.
18. Book titles and journal article titles should 9 1 Fixed
be search under Title search
19. Type complete title in and she GOT IT 9 1 Good
20. OBS: will search book titles in Advanced 9 1 Fixed
search too (soon)
21. Consider combining Title and Book Title field 9 2 fixed
and have one box for title i.e. remove Book
Title box
Search strategies good and poor
22. Put search item (suzac) in Basic search, 3 1 ?
decided that she need to go to Adv search
and entered 2009 but assumed that search
item would be carried over from the basic
search to search on
23. Typed Richard C. Daly into the search. (note 4 4 ?
to self)
24. Put full name in search box but figured it out 4 1 ?
25. Tried search from previous results page and 4 1
could have gone back to Basic search which
she eventually did.
26. Thought or wished he could refine searches 5 1 Nice to see search and refine
from search box(es)

4. 27. Concerned that previous searches were 5 1
logically connected and affect subsequent
searches
28. Thought searches linked to each other. 5 1
29. Did not search daly as a mayo author 5 1
30. Did author search incorrectly 6 1
31. Thought search name was the function to 6 1 Name your saved search instead
search the DB instead of just a name. of Search Name
32. Thought searches are linked 7 1
33. Tried to limit to electronic format but can 7 1
not
34. Used Authors search box for electroporation 7 1
(search accepts authors)
35. Typed in entire title in search box 8 2
36. Used “” around search item and it worked. 8 1
37. Did performing many single searches 12 1
38. Knew that she could click on a name and 12 1
perform the search
39. Used wrong name potentially as did not 12 1 Moving authors field ought to help
have correct typed in name. Wahner, H with this
instead of Wahner, HW
40. Used open author instead of Mayo author 12 1
and got the wrong stats.
Work Flow
41. Have a way to review the complete record 1 1 Looking at possibilities, either new
i.e. relevant search terms screen or on same page

5. 42. Wants search boxes to stay filled in when G In future
moving from page to page and when results
come back
43. Thought would see results when she ran G In MyMadb retrieve in save
retrieve search. search goes to results page
44. Authors’ field is being used a lot and they g Moving up as partial but training
should use the MayoAuthors option. Need might be another possibility
to educate them on the results of using the
authors
45. Intimidated by the room G
46. Wanted to look at the abstract or article was 1 1 Publisher problem
taken to full text and then the journal. It
did not take her to the journal article and
she was unhappy as she would have to
search the journals website again
47. Trouble with naming of search results as it 6 1 Working on problem and identified
just gave her a number and not what was that user will be confused.
searched Consider rewording the save
search instructions.
48. Tried to save search before search 6 1
49. After login it took her back to Basic search 6 1 Investigate the possibility taking
not from where she was back to original spot. Investigate
pop-up for logon in
50. Did not assign names to the saved searches 6 1
51. Was confused on what save search is 6 1 Working on
52. Thought the download button would 6 1 Take download off
download the entire set of citations.

6. Download is for single specific citations
53. Did not understand how to save previous 6 1 rewording
searches
54. When saving searches user not sure of what 7 1 rewording
is being saved.
55. Surprised that she could not just click on 6 1 Use bold and blue for link
name for selection language
56. Need to reformat the alert system/save 6 1 Consider creating a smart form to
options bring up information about alert
when I want an alert.
57. Wanted to look at the full record not just the 7 1 Fixed above
citation
58. Tried to change the full record by checking 7 1 No fix
on output box
59. Clicked on back button “all the fields are 7 1 investigating
gone”
60. Why does the page go away when I click on 10 Investigating to accommodate the
the Back (IE) button Back button programmatically
61. She typed in the extension .xls and the 14 1 No fix
formatted result was not OK, she should
have just saved it without extension.
62. Did not see the download button, went to 14 1 Create a Save Table button
MYMADB page.
63. Did not know it was saving to Excel when 14 1 Leave it
she asked for the download
64. Tried to use File | Save as 14 1 Logical but reword download to
save

7. 65. Decide when to save between searches 12 1
information typed in for the search,
advanced fields cleared after search
66. Just show results in “metrics” on what you 12 1 It is based on current search
searched for.
Workflow - MyMADB
67. Did not know to go to MYMADB to save and 6 1 Consider action to Save Search
auto alert history and provide confirmation
that it is saved in MyMADB
68. My click on My MADB it brings me to a login 6 1 Fixed – consider making box
page and ought to provide what this page around log in a square and
would do for them. centered in page
69. Could not find the Save button had to be 6 1 See above
directed to MYMADB (save button
elsewhere?)
70. MyMadb needs at title. 6 Consider changing to Saved
Searches
71. OBS: We have a disconnect with Save, 10 Name change and button
MyMADB, search exposed will help this problem
Metrics
72. Liked the metrics 12 1 Consider changing Search
Results Options to Search
Results Delivery Options
73. She said that Metrics were about this article 13 1
not this search - confused
74. Thought metrics was great 13 1
75. Was told how to use Metrics 13 2

8. 76. Did task the hard way instead of using 13 1
Metrics
77. Did not use Metric button 13 1 Consider changing name Metrics
to … , Change hover description
to be more description i.e. remove
metrics from definition
78. Still not using author list 13 1 Put some search tips informing
user of benefits of using Author
search
79. Combine and Advance search option for a 13 1
search
Basic vs. Advanced Search
80. OBS: people used basic but after finding the G Not a problem
advance search they do not tend to go back
to basic. (some did go back to basic)
Page Layout
81. Did not use the Expand search history G Consider making Clear and
Expand buttons with icons to help
understanding. Also put on the
bottom of the listing too
82. Did not see the search item in the titles and 1 2 Consider making Show More a
looked for it for reinforcement. She did not button with an icon, highlight term
realize that the keyword could also be in the in full record
record etc. of article.
83. Have a title on search results to know what 1 1 Will consider ways to do this see
searched on Metrics page
84. OBS: Consider taking the search box off the G Consider Browse or Search in title

9. Basic search page. of Mayo Authors List and put
helpful tips to direct
85. OBS: Consider ways to make the advanced G fixed
search Mayo Authors more prominent and
possibly easier to use.
86. OBS: When do Basic search the results page G fixed
says Advance search results
87. Does not like the dual box selector control G Leave as is
88. Took radio button setting as the setting for 1 1 Add delivery Format options
the results listing Select delivery format John will
look at the visual presentation to
separate it more
89. Makes more sense to have results first and 1 1 Solutions: 2) make search criteria
then the results manager options sticky 1) put refine search button
on results page that takes user
back with sticky criteria
90. OBS: did a lot of exploring of the page to 1 1 good
start
91. Thought had to fill out the results manager 1 1 See above
to view the results
92. Confused the Title header as the tab 5 1 ??
93. Consider having initials spelled out for 6 1 Possible future ; consider
authors feedback from the field
94. Text did not show that all search history was 6 1 Change type & placement of UI
saved. control for clear and expand
95. She was trying to save individual searches 6 1 Ask John to help with layout of
for each author but got tied up in the save section

10. details.
96. She did not know which search she was 6 1 Change Saved Searches to
saving. Saved Search Histories
Change Current Searches to
Current Search History
Save above search history
Add Alert title above checkbox for
alerts
97. Did not want the two searches together 2 1 ??
98. Missed results section at the top of page 4 1 ??
99. Did not notice the Metrics button but found 5 3 Searching strategies
a different way
100 Had to tell her about the Advanced Search 5 1 ??
. tab
101 She liked it but commented that there are 5 1 Flexibility
. so many boxes to get used too.
102 Propensity to use the back button to 5 1 investigate a way to not have
. navigate. But it presents expired page. expired pages with the back
button
103 Did not know how to login. She went to the 6 1 Need to rename MYMADB to …
. Library site to log in. We need to decide
how to relate the two logins together. Check
on Help desk auto login to see if it will work
here
104 Need title on Metrics display and saved 6 1 fixed
. metrics output
105 OBS: Put login UI controls on page that says 6 1 Fixed need John for layout

11. you need to log in after clicking My MADB
OR
Do saved searches for non-logged in patrons
106 OBS: Consider making the icons larger for 6 1 no
. login etc.
107 Not sure which search he was saving 6 3 Working on solutions see above
.
108 OBS: consider viewing of saved searches 6 1 Consider edit for single searches
. and a way to edit saved searches in the future.
109 Larger font and list longer on Adv search 6 1 Fixed wait for comments
. Mayo Authors page. Move it more to the
top, she said this would be the most used
110 Found delete search history 6 1 great
.
111 Tried to limit search to quarterly (for auto 6 1 Misunderstood search
. alert)
112 OBS: Consider save next to searches 6 No
. performed
113 Searched Pericardial in Location space 6 1 ??
.
114 Does it tell me anywhere about the 6 1 (Your Choices) change to
. instructions for the list box select tool Selected Authors
115 Unsure about the keyword box, possibly the 7 1 Working on a selection box for
. placement of it by itself and not with the specialty
other boxes
116 Did not see example for the combined 7 1 Moved up with title
.

12. search (too light, too small??)
117 Where are my searches while on the Results 7 1 John needs to help with new
. page? buttons
118 Not using the Mayo Authors select list as 11 Moved the box up for better
. much as we would like. Even when told of visibility
the benefits they gravitate towards the
search box.
119 OBS: Consider taking the search box off of 11 Keep like it is show search box
. the results page, thus requiring the user to when coming from the basic
go back to search page. Advanced search search and remove search when
does not have search box (es) on results coming from advance search
page.
120 Need duplication of column titles to help 12 1 Fixed
. with scrolling reduction
121 The authors picker is very valuable, would 12 Fixed
. like higher up in interface
122 He asked for a way to do combine searches 13 1 Make title larger ask John – we
. but did not see the combine box think it looks OK
123 Put column labels at bottom too 13 Fixed
.
124 Would like to have the first name shown for 12 1 Consider if feedback warrants
. identification purposes
Page Layout – Marked Unmarked items
125 Marked the items and sent but did not mark 10 2 Fixed
. items go to results and then send.
126 Did not select a few sent all 10 2 Will fix to email only checked
. marked items on page

13. 127 She marked items but sent the whole list 10 4 Same thing as above
. and not just the marked items
128 Seems to be a disconnect as the user selects 10 1 Fixed
. items but thinks (?) that the send will only
send the marked items, but have to go to
Marked items display to only send them.
129 OBS: Save marked items for login session 10 Discard after session
. only, discard after logging out.
130 OBS: Provide choice in Output section for 10 Done
. marked items
131 When sending marked items the “Submit” 10 2 trying to remove white blank
. takes you to a blank screen. Consider screen
message on same screen that it was
submitted from
132 Sent himself all citations 10 2 Same as above
.
133 She marked the records (3) but sent the 10 1 Same as above
. entire batch; she did not go to marked items
to send it.
134 Sent all instead of marked. People are not 10 Same as above
. going to the marked section to send.
135 A plus sign would be more logical for 10 1 Fixed
. marked items
136 OBS: Consider putting on Output manager 11 fixed
. Send Marked items
Page Layout - Department (Authors)
Label

14. 137 Did not notice the department section of the 2 1 Renamed to Mayo Authors
. citation as displayed in the results
138 OBS: Trouble identifying Mayo authors – 2 1 Fixed
. suggest renaming label from department
139 Department should say something different 2 1 Fixed
. (e.g. Mayo Authors)
140 He looked at Department for answer. But 2 1 Fixed
. wonder if she was looking at the name and
fields to determine where from
141 Got the record but wanted to look at the 2 1 Fixed
. online and he missed the Department line
142 Recognized where Mayo authors are 2 1 fixed
. immediately
Consistency
143 Did not like the non-uniformity for specialty G Taken from source
. codes for department search
Font
144 Having the titles all bold sans serif is hard to G Leave alone
. read
145 Font size too small 6 1 Make 10 pt
.
UI Controls
146 Need a caffeine button 8 1 Done
.
147 Thought for results page should have check G Fixed
. boxes for mark unmark
148 She wanted to click on publication type in G Consider for later

15. results to sort
149 OBS: Need to discuss radio button options 1 Done
. on basic search
150 Would be nice to put cursor over department 2 Not parsed to allow
. abbreviation and see what it is
151 On basic page can do a new search right 3 1 Fixed
. away but on advanced search have to go
back and fill out form again with all criteria
152 Did not bring over selected item to select 4 Put a little more space between
. side of control buttons and bold arrows
153 OBS: Check to see that cursor changes to 4 Will fix if possible
. finger e.g. in list of Mayo Authors.
154 OBS: Need discussion on “Entry Week” 5 Author | title | journal | year
.
155 Tried using Entry Week 5 1 Maybe fixed
.
156 OBS: Prominence of year and other fields 5 same
. highly used
157 Wanted a start year and end year field 5 1 Leave as is
.
158 Did not complete selection in select lists 5 1
.
159 Was confused by the Entry week selection: 5 1 Maybe fixed
. thought it was date select
160 Went right to Metrics 5 1
.
161 Consider having medical specialty code 6 1 Done in select box
. spelled out in search criteria

16. 162 Consider having browsed item in little dual 6 1 See if can put selected on top
. boxes put new item at top of list e.g. instead of bottom of list
Espinosa
163 OBS: Click on search result history item 6 1 Fixed
. should be considered a view not a new
search and therefore not display in results
history summary
164 Consider an all option for search boxes 6 1 Fixed with limit term and do not
. need for this type of selection
165 Confusion on what Save search means – is it 6 Put saved search history button
. saving only the one named or all or?? under last search
166 Mark Unmark confusion 7 1 fixed
.
167 Consider making search titles clickable 7 1 No
. instead of just the # of results
168 She would like “type ahead” 8 1 No, would help journal field
.
169 Consider a drop down for the journal field 8 1 No
.
170 Inconsistency between author and editor 9 1 Change example to not include
. field initial - Leave alone but if fix it
then - Make consistent, and
handle programmatically take out
for search
171 Do we need a download citation button? 10 No
.

17. 172 Wanted to clear the search 10 1 Change treatment of the Clear
. button and put separation
between Search button and Clear
button
173 Do we need the download button? As output 11 1 No
. manager can handle it.
174 Did not move selected item to right side 11 1 Keep same
.
175 Used SFX instead of download 11 1 fixed
.
176 Click on column under link name to sort 12 1 ??
.
177 Need way to bring up old search to modify 12 1 Will work on this
. fields. E.g. edit button in SCOPUS
178 Wanted to be able to sort in a search results 12 1 No
. for type of publication
179 What is language for multiple authors in All 12 Fixed and normal Boolean can be
. Authors field? used as stated in the Help text
Icons
180 OBS: Consider treatment of icon to signify 6 Empty Cart button can say Clear
. not available for MyMADB marked items, icon for re-run
search, Remove save taken away
and put “Delete” in column
heading having icon X in the
column Align columns Save
Search page and other pages

18. where appropriate
181 Thought all items were already marked (icon 10 1 fixed
. and label confusion?) it looks checked when
not
182 Figured out what the metrics were by the 12 1 Consider treatment of Results
. icon page actions under the search
criteria bar
Terminology (includes search language)
183 Did not understand what Metrics means 4 1 Change term to statistics
.
184 Please check spelling confused her as she 4 1 Leave it
. did not know how and everything was
spelled correctly. It was a suggestion.
185 No clue what Metrics meant 5 1 Good just changed it
.
186 Thought metrics meant the number of times 5 1 O
. the article has been cited
187 Did not understand search language – used 6 1 In help text
. semi-colons between names
188 OBS: Need to discuss and decide on a 6 Leave alone
. naming convention for the search history
(novice vs. expert)
189 Confused by Download – possibly re-name 7 1 Removed it
.
190 Confusion on what “Includes all fields” really 7 1 Put default radio button first eg
. means – all data fields then I do not have to Search all fields Change
enter anything Keywords to “Title or Subject
only”

19. 191 Title in results is web blue and he thought it 7 1 John to look at color schemes,
. was clickable button vs. hyperlink
192 Thought download meant to download the 7 1 Got rid of
. article to print (maybe add download
citation)
193 What does Save to Disk mean? 11 1 Removed term
.
194 Suggest rewording of Download 14 We did
.
195 Need to find a way to make the function 13 1 We have
. “Metrics” more understandable
196 MayoAuthors is one word 13 Make it two words
.
How do I
197 Looked in the Help text for help. 6 1 Good
.
198 She used the How do I from the library 6 1 Update Library How do I to
. website include Mayo Authors
199 Looked in How do I 6
.
200 Need "How do I" articles. 6 Ok
.
201 Was able to distinguish between articles and 7 2 Ok
. abstracts
Messaging
202 OBS: Consider reformatting the error 8 Leave as is
. message to get the main message through
e.g. Check spelling

20. New features
203 Wanted a short class for more G Maybe
. comprehensive search
204 OBS: Consider type ahead for Mayo Authors G Leave as is
. in search field, would need to designate that
it is for Mayo only.
205 Consider having tips pop-up to help users G Need one that tells people to
. choose authors from the list not
type in. Put on for Entry Week
206 Surprised pleasantly that the searches are G Good
. not tied together
207 Wanted to see the full record G Show More button
.
208 OBS: Consider editing each saved or 6 No
. performed search
MISC
209 Concerned with all the name change that G
. affect searching
210 Did not use the combine feature G We changed to make it more
. prominent.
211 Other has Baken D and Baken DS so she is G
. not sure which is which
212 Did not realize that Goodenough was not a 2 1 Ok
. Mayo author
213 Did not answer the question but got to the 3 1 Ok
. information needed.
214 Tried setting up alert before search 6 1 John will take a look at this

21. 215 Help text did not help but she did not read 6 1 Move examples below text and
. most important sentence (first one) as label as examples, make sure the
control f took her to the 3rd sentence with How to is include in the text
emphasis on find term
216 She gave up 6 1
.
217 Liked to be able to mark 7 1 Good
.
218 Liked the tags at the end of cites for type of 7 1 Good
. media
219 Did not recognize .ris extension 11 3 Put in Help if use
.
220 OBS: Full text not working with proceedings 7 Will fix
.
221 I do not want to go to the help page 7 1 Ok
.
222 Got journal volume wrong 8 1
.
223 Used Book Title 9 1 fixed
.
224 Made good use of the limit boxes 9 2 Good
.
225 Wavered on .ris extension 11 1 Gone
.
226 Did not use MayoAuthors 12 Fixed
.
General Comments G
BUGS B

22. 227 BUG: Error message from previous search 3
. was showing on Advanced search when
clicked to it’s tab
228 BUG: Error message from previous search 4
. was showing on Advanced search when
clicked to it’s tab
229 BUG: Selected Oh and Espinosa then went 6
. to Basic search then came back the names
were gone. She started over entering Oh
and performed the search. Results came
back for Oh and Espinosa
230 BUG: Radio buttons turn off when selected 7
. in the how do you want these results section
(page kept on refreshing)
231 BUG: author and title did not bring up result 7
.
232 BUG: Kept displaying messaging whether 7
. error or not on following pages
233 BUG: Auto-refreshing (marked 3 then sent, 10
. then auto-refresh) clicked to tab and
problem corrected.