TreeJack is an online usability testing tool that enables setting up user tests of your link hierarchy (website navigation, Table of Contents, and so on) and observing how easily users find information they are instructed to locate. Patty has used this tool as Principal Learning Content Developer at Autodesk, Inc.

1. © 2015 Autodesk
Navigating Help
Testing Information Architecture with Treejack
Patty Gale
Principal Learning Content Developer, Autodesk, Inc.
April 15, 2015

2. © 2015 Autodesk
 Can people find what they're
looking for?
 Is the taxonomy and navigation
labeling appropriate?
 Is the help content hierarchy
well structured?
Goal: In the help system…

3. © 2015 Autodesk
 Online user study: Recruited users
 Tool: Treejack by Optimal Workshop
Method

4. © 2015 Autodesk
 Write text for welcome, instructions, and thank you
 Design pre- and post-survey questions
 Get the site map/help hierarchy into Treejack
 Define tasks for users: what info do you want them to look for?
 Identify correct answers for each task
 Run a pilot to identify and iron out wrinkles
Create the study (similar to SurveyMonkey®)

5. © 2015 Autodesk
Send the link to participants
Run the study
Participants perform the study
You can look at survey results any time

6. © 2015 Autodesk
A sample task

7. © 2015 Autodesk
 12 tasks out of 20
 123 users participated
 Pre- and post-survey questions
provided insights about these users
In the Treejack study…

8. © 2015 Autodesk
Summary of results: Overall Success
Optimal Workshop says that an overall success score of
"68% is at the high end of average for websites in general,"
so we did pretty well, but there is room for improvement.

9. © 2015 Autodesk
Summary of results: Overall Directness
Optimal Workshop says a Directness value of 70% is pretty good.
However, the overall score for each task “is a weighted average of
the squares of success and directness, favoring success over directness
at a ratio of 3:1, scaled to be a value out of 10."
In other words, Success (finding the information at all)
is more important than Directness (finding it on the first try).

10. © 2015 Autodesk
Let’s examine results for one task…
Overall score on a range
from 1 (low) to 10 (high)
Red = lots of users did not
find the correct topic
Gray = a few users skipped this task
Dark green =
some users went
directly to the
correct topic
Light green = some users
wandered before finding it

11. © 2015 Autodesk
What’s a pietree, you ask?

12. © 2015 Autodesk
Same task, different information architecture
AfterBefore

13. © 2015 Autodesk
Pietrees show a clear difference
AfterBefore

14. © 2015 Autodesk
Click-through rates tell the story
Before: After:

15. © 2015 Autodesk
 We removed a layer in the help hierarchy.
 Document the Model > Share the Design > Print
 Document and Present the Design > Print
 “Share the Design” was too vague.
Users lost the scent of information.
It didn’t mean “print” to them.
What changed to improve the results?

16. © 2015 Autodesk
 Get labels right.
Don't use vague terms
that cause users to lose
the scent of information.
 Match the user's
mental model.
Lessons learned
Print
Share
“I like the
current system
which is based on
the design
process.” – Jason B.

17. © 2015 Autodesk
Demo with Q&A

18. © 2015 Autodesk
 Treejack: www.optimalworkshop.com
 Information Foraging by Jakob Nielsen
http://www.nngroup.com/articles/information-scent/
 Search is Not Enough: Synergy Between Navigation and
Search by Raluca Budiu
http://www.nngroup.com/articles/search-not-enough/
Resources

19. © 2015 Autodesk
Icons from TheNounProject.com: (CC BY 3.0)
 Users by Vittorio Maria Vecchi
 Monitor by useiconic.com
 Share by Joshua Stearns
 Browser by Max Miner
 Analytics by Christopher Holm-Hansen
Creative Commons licensed image:
 Talent #3 (dog) by Robert Terrell (CC BY-NC-ND 2.0)
 Confused by Guudmorning! (CC BY 2.0)
Attributions for images used in this presentation

20. Autodesk, the Autodesk logo, and Revit are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand
names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without
notice, and is not responsible for typographical or graphical errors that may appear in this document.
© 2015 Autodesk, Inc. All rights reserved.