1. Usability Test Final
Report
USA TODAY’S iPad application
By Kathleen Sullivan, Gannett/ For USA TODAY 5.3.2010

2. Background
• Success of USA
TODAY mobile
applications
motivates design
and development of
native iPad app

3. USA TODAY iPhone app

4. AutoPilot iPhone app

5. Android app

6. USA TODAY iPad video
http://www.youtube.com/watch?v=p5nJVtLygOM&feature=player_embedded#!

7. Purpose
• The purpose of this usability
evaluation is to determine how users
of the USA TODAY iPad application
will perform with the application and
to make recommendations for
improving the application.

8. Why do a usability test?
• It is necessary to conduct usability testing on the
USA TODAY iPad application in the interest of
improving the program for future versions of the
application.
• The usability evaluation of USA TODAY iPad app
will identify problems in the user interface that
impact readability, ease-of-use and overall user
satisfaction.

9. Core objectives of test:
The core objectives for conducting usability tests on the USA TODAY iPad application
are to ensure that the application:
• Is easy to launch and use.
• Allows users to find USA TODAY content they are looking for easily.
• Loads quickly at launch and throughout use.
• Presents all formats of USA TODAY content clearly.
• Allows users to share and save content quickly and efficiently.
• Design promotes intuitive navigation among the USA TODAY content sections.
• Evaluate the overall user interface in order to determine whether the application was
designed to allow users to browse and find content efficiently.
• Evaluate the layout of different content formats among sections and navigation
options.
• Identify opportunities to enhance the user interface design and user experience
design.
• Measure the time required to complete tasks in order to determine ease of use.
• Observe any errors that occur during application use so that improvements can be
made.

12. Article layout Weather layout

13. Scores layout Photo gallery layout

14. User profiles
• Eleven participants Age
group
Number
• 5 male, 6 female 18-24 4
25-30 2
• 5 iPhone/iPod touch 30-40 1
users, 6 non-users
40-50 2
50-60 1
60-70 1

15. Methodology
The usability evaluation for USA TODAY’s Apple iPad application was conducted in
early-mid April 2010. Evaluations did not start until after April 3 — the release date
for the iPad and the application.
The USA TODAY iPad application was tested using:
1. The thinking aloud protocol
2. Timed task completion
3. A debriefing survey that included Likert-scale rating
questions and open-ended questions
• Eleven volunteers participated in a 10-15 minute hands-on evaluation of the iPad
application, during which users were timed for task completion and participated in a
thinking aloud protocol. After the test, each participant completed a follow-up survey,
which included Likert-scale questions on the application and open ended questions.

16. Task List Script
1. Read the headline for the top story on the home screen of the application. Select that article to read and
navigate to the story page. Scroll through the article as if you were skimming it.
2.
Find the most recent sports scores for the last NBA basketball game and read it aloud.
3.
Go to the Sports section front. Click to read the first article available. Continue onto the next page of
that same article. Then navigate to the next sports article.
4.
Navigate to the weather and say the current temperature where you are.
5.
Change the weather location to one of the other cities listed.
6.
Navigate to the Life section and find the latest movie review. Find the option to share the story and click
to share the story via e-mail. Cancel before sharing.
7.
Still in the Life section, find the photo gallery for “Day in celebrities” and scroll through the photos. Bring
the last photo in the gallery to full resolution and share the photo via e-mail (cancel before sending). Then
exit out of the gallery.
8.
Navigate to the home screen for the application and click on the ad at the bottom. Cancel out of the ad.
9.
Navigate to the Money section from the home screen and click on an article to read it. Change the font
size of the text in for that article. Then change it back.
10.
Please navigate back to the home screen of the application and refresh the content.

17. Post-test survey
Likert scale survey questions:
1. Please rate your experience with the USA TODAY iPad application.
Overall experience with the USA TODAY iPad application was (circle one, with 1
representing the least satisfying experience and 7 representing the most satisfying
experience across all categories):

18. Post-test survey
2. Please rate your reaction to the design of the USA TODAY iPad application. (1
represents the least satisfying design and 7 represents the most satisfying design. Feel
free to use comments area to express your opinions on the design of each section
(optional).

19. Results
Timed results by participant
Owns Age range of Q1: Read top Q2: Find the Q3: Navigate Q4: Find the Q5: Change Q6: Navigate Q7: Find the Q8: Click on Q9: Navigate Q10:
iPhone/iPod participant story most recent to the sports weather and the weather to the Life photo gallery the ad on to the Navigate to
Touch headline and sports scores section, click read the location to section and for “Day in the home Money the home
navigate to for the last on any current one of the find the Celebrities”, screen section, click screen and
article page NBA sports temperature other cities latest movie scroll to the on any manually
basketball article, go to listed listed review last photo article and refresh the
game the next and share it change the content
page of that via e-mail font size of
article and the text
then the
next article
of the
section
iPod Touch 50-60 2 6 14 2 60 30 20 39 4 3
iPod Touch 40-50 9 4 12 13 47 8 40 35 10 2
iPhone 30-40 6 14 40 7 26 39 57 42 3 2
none 60-70 28 2 4 15 27 15 44 40 25 15
none 18-24 5 3 20 12 60 6 4 20 13 4
none 18-24 6 58 2 13 60 27 20 32 12 2
none 25-30 2 3 3 2 21 25 15 23 4 2
none 25-30 30 2 3 29 60 31 60 1 41 38
iPhone 18-24 28 10 3 33 60 15 33 4 4 5
iPhone 18-24 3 16 4 40 7 12 27 7 2 1
none 40-50 10 60 60 23 60 29 15 33 25 2
MEAN 11.72727272727 16.18181818182 15 17.18181818182 44.36363636364 21.54545454545 30.45454545455 25.09090909091 13 6.909090909091
MEDIAN 6 6 4 13 60 25 27 32 10 2
Editing the weather location was the most difficult task (took the longest at an average of over 60 seconds)
Refreshing the content was the least difficult task (average time to complete: 2 seconds)

20. Product design problem areas:
Home screen navigation Weather feature

21. Timed Results

22. Timed Results
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
>>>18-30 year-olds tended to finish tasks faster than participants who were 30 years old or older.

23. Timed Results
No significant difference in timed task completion by gender

24. Likert scale survey results
Overall satisfation
• Likert scale overall satisfaction rating of 5.197
• Lower ratings indicated that some found the application to be slightly frustrating and difficult
to use compared to the other rating indicators.

25. Likert scale survey results
Feature/page ratings
• Likert scale overall features satisfaction rating of 5.327
• Lowest rating was given to the weather feature, which most participants had trouble
with when trying to edit the location. Highest rating average was given to the home
screen and article screen.

26. Qualitative Results
Open-ended question responses
1. What was your initial impression of the
USA TODAY iPad application?
1.
ositive.
P
2.
lean — similar to their paper design.
C
3.
Nice.
4.
ice looking. Better than iPhone app.
N
5.
unctional; a bit of a learning curve but once familiar very easy to navigate.
F
6.
news app.
A
7.
lean, focused on the key/important things users want.
C
8.
ood, informative, crisp.
G
9.
thought the app was visually beautiful and well laid out. It made finding many
I
news articles very easy, and easy to read. Like a newspaper without the gross
ink.
10.
Good.
11. Easy to use like the iPhone app.

27. Qualitative Results
Open-ended question responses
2. What did you like about the USA
TODAY iPad application?
1.
ouch action, ease of turning pages, resolution of photos.
T
2.
eather, flipping through articles, clean design.
W
3.
esign.
D
4.
ood size, nice layout, reminded me of paper, easy to read.
G
5.
lear, colorful, inviting, organized and balanced (information and ent.)
C
6.
t was good.
I
7.
asy to navigate, not cluttered.
E
8.
ood variety of information — liked tying to different sections.
G
9.
loved the pictures and the screen was just beautiful. I also liked the way the
I
articles were laid out on the screen.
10.
It was well organized and easy to use.
11.The simplicity was likable about the app.

28. Qualitative Results
Open-ended question responses
3. What did you dislike about the
USA TODAY iPad application?
1.
Some buttons were not intuitive.
2.
All sections not listed? Where is “tech”? Your users are early adopters of the iPad! How could you
not have tech? Back button on most features also not intuitive.
3.
The weather and the navigation.
4.
Navigation between sections impossible — no clear button to punch to get from one section to
another. Should be able to punch a sports button to get to sports. Photo gallery thumbnails too
small.
5.
Weather setup.
6.
Not enough direction if I wasn’t asked questions.
7.
Weather (took a while to figure out how to change to another city on the list)
8.
Weather section (make more intuitive to change location)
9. I did not like the weather feature – not enough detail for my tastes. I prefer weather.com, which
gives hour by hour data.
10.
The navigation between different topic pages was confusing.
11. The sports scores could be clickable and have more data.

29. Qualitative Results
Open-ended question responses
4. Do you currently own an iPad or plan to
purchase one?
Currently own: 0
Plan to purchase:1
Do not plan to purchase: 7
Unsure: 3
5. Has your experience with the USA TODAY
iPad application influenced your decision to
purchase or plan to purchase an iPad?
Yes: 1
No: 6 — Cost, not needed; No need for one; Just bought a new laptop; It doesn’t make phone calls;
Unsure: 4

30. Qualitative Results
Open-ended question responses
6. If you own or plan to own an iPad, do you
plan to get the USA TODAY iPad application?
Yes: 1 — If I owned one, I would get this app because I find the iPhone app fun and easy to
use.
No: 8
Unsure: 2
7. Did you think there was a function missing in
the iPad application?
No: 9
Unsure: 1
Yes: 1 — Back button, geo-based ads, stories updates;

31. Qualitative Results
Open-ended question responses
8. If you could change any feature to the USA
TODAY iPad app, what would it be and how
would this enhance your experience?
1. Even more interactive and customization options.
2. Section front experience and back buttons.
3. N/A
4. Navigating between sections was a bother.
5. Order cities for weather; more categorical organization on the home section.
6. N/A
7. N/A
8. Weather and making it easier to get between locations.
9. I would like more games like the jumble or cross-word available.
10. To have ease to change from sports to life or money a lot faster and easier.
11. Add more sports features.

32. Qualitative Results
Open-ended question responses
9. Did you have any difficulty using the USA
TODAY iPad application? If so, what did you
have difficulty with and how might this be
improved?
No: 3
Yes: 8 — Changing city locations; homepage buttons — should add section buttons to
homepage so you know how to navigate; weather and sections;

33. “ ”
Qualitative Results
Thinking Aloud Protocol
“Hard to exit out of the ad. “X” button not intuitive.”
“Weather is definitely not intuitive. I haven’t seen that before. On the iPod Touch the weather
is much better.”
“The iPad seems fine but I wouldn’t buy the first version of anything.”
“I like the brightness of the page and the white space.”
“I like the way the pages are laid out. I like the fact that you can turn it sideways. I like the
photo galleries and the color.”
“My nails got in the way of touching some of the buttons.”
“I thought the indicators for the weather buttons would be more obvious.”
“The share button looks like a reply button.”

34. “ ”
Qualitative Results
Thinking Aloud Protocol
“Navigation between sections is nonsense. I can’t figure it out.”
“Need a close button on pop-ups.”
“When you’re in the Sports section, it’s not intuitive to press the Sports button to go home
or somewhere else.”
“Sports scores don’t have last night’s scores, they have today’s schedule — that’s annoying
when I want to know the score for last night’s game.”
“For changing the weather locations it looks like you just are supposed to pick one. There’s
nothing to indicate their in any kind of sequence.”
“I didn’t see anything else so I tried that button” (upon figuring out how to navigate between
sections using the USA TODAY button)
“The weather was frustrating.”

35. “ ”
Qualitative Results
Thinking Aloud Protocol
“I didn’t think it was a link, just thought it was a picture of the USA TODAY logo.”
“They should make the selected city a different color so you know to drag it up.” (on
the weather edit locations)
“Is there a search function on the application?”
“I like that you can scroll through pages.”
“I don’t like that the tech section is missing. People who have an iPad will want to find
tech stories.”
“I don’t like how hard it is to get back to the homepage.”
“I feel like there should be a back button on a lot of these pages.”
“Wish it would pull up the scores for teams I like most – like customizable. Can you
do that?”

36. Recommendations
Based on the quantitative and qualitative results for the USA TODAY iPad application,
we are recommending the following improvements be made as soon as possible:
1.
Add a feature to more clearly indicate how to navigate between sections of
USA TODAY
2.
Adding language (text) or buttons (obvious visual components) to more clearly
indicate how to edit/change the weather location
3.
Add a sub-layer of navigation and search to include other sections/topics that
USA TODAY is known for (Ex: technology, economy, movies, travel, etc.) (Makes
content easier to find)
4.
Take advantage of the iPad’s interactive features by including more interactive
visuals and other features within the application (Ex: cross-word games, market
graphs and other interactive graphics/games).