Collaborative card sorting tool.

1. Bachelor thesis Project
Design and Development of Learning Content and
Collaborative tool for card sorting technique in Usability Engineering
Kshitij Pandey, Final Year, B.Des IITG
Manas Modi, Final Year, B.Des IITG
Under the guidance of
Prof. Pradeep Yammiyavar
1

2. Recap
Introduction to card sorting technique
Technique used to generate a category tree. It is a useful approach for designing information
architecture, workflows, menu structure, or web site navigation paths.
Types of card sorting
Open, Closed, Reverse, Delphi
Need analysis
•Lack of interactive tools for learning card sorting technique
•No tools available for collaborative card sorting
Aim
•To design a computer based tool that assists a novice user to learn card sorting technique.
•To design an online tool to perform card sorting technique in a collaborative, way.
Deliverables
Working prototype which can be used as
a) Learning content for card sorting
b) Online tool for collaborative card sorting
2

3. Recap
Group vs. Individual card sorting
Pros and Cons of online card sorting
Quantitative and qualitative valuation of the existing online
tools on the basis of tasks effectiveness and efficiency; and
features
Optimal sort
UX Sort User Zoom WebSort 3

4. Recap
Experiment for collaborative card sorting
Purpose
To understand and observe how participants and users, who were known or unknown
to each other perform the card sorting activity
Method
Participants performed the activity on their own with no interference from the moderator.
The whole session was documented through photos, videos and notes
Procedure
24 cards and 6 categories
15 participants, 5 groups of 3.
Observations and Inferences/Functional requirements
•An activity lasts for 15 min approx
•Focused discussions in groups where participants are unknown to each other
•Need for an application which can help card sorting activity with participants sitting at different
places
•Moderator should be able to make cards and categories beforehand.
•System should be able to Assimilate, compile and store the card sorting results
4

5. Recap
Concept 1: Moderator Controlled, Mod sort
Environment is totally controlled by the moderator. He acts according to the group consensus
Concept 2: Participant Controlled, Collab Sort
Environment parameters are set by moderator. Card sorting is done by the participants
themselves
Concept 3: Optimized Model, Co Sort
Environment parameters are set by moderator. Card sorting is initially done
individually by participants, and followed by a group sort.
Selected concept: Concept 3, Co Sort
•Participant control is present
•Easy to determine individual mental model
•Less time taken, as the common cards are already sorted
•Good for large scale, when information is very large in number, and there are over 50 cards
5

6. Concept
What happens when
participants need to
perform card sorting from
different geographical
locations?
:
:
:
:
:
Remote Collaboration.
6

7. Concept
Inter-cultural studies, global systems
7

8. Concept
Co Sort: Optimized Model
Environment parameters are set by moderator. Card sorting is
initially done individually by participants, and followed by a
group sort.
Phase I
8

9. Concept
Phase II
9

10. Selected Concept
Co-Sort
It allows participants to work individually as well as
collaboratively
Moderator can distinguish between every
participant’s mental model
Cards which are common between participants
after individual activity are placed beforehand on
the group interface, hence, saving time.
Versatile for small scale sort to large scale card
sorting algorithms where complex systems are
involved
10

11. Wireframes
This screen is
used by the
moderator to set
parameters for
the card sort.
11

12. Wireframes
The participants sort
out the cards
individually first in a
pre set time frame
12

13. Wireframes
Moderator can see the
participants’ activity
remotely.
13

14. Wireframes
•Shared screen
• All the participants
can drag and drop the
cards.
•There are 3 mouse
pointers on the screen.
14

15. Use Case
Diagram
Online tools
15

16. Activity Diagram
Online tools
16

17. Prototyping
Phase I, Card Sort: UI design and prototyping a tool
To help a novice user learn card sorting
Phase II, Co Sort: UI design and prototyping a tool for
Collaborative card sorting

18. Technology
Adobe flash(AS3), PHP, MySQL database.
PHP
PHP feeds the Also, it brings the
user activity data updated data the
on the front end database back to reflect
into the MySQL the changes in the front
database. MySQL Flash end.

19. Phase I
User Flow
• Introduction to card sorting
1
• Stepwise learning content for card sorting
2
• An exercise where the participant gets to choose an existing
3 website to perform card sorting activity
• A comparison of participant's result and actual website
4 architecture
• Repeat step 3 with same or different website
5 • Or exit

20. Phase I
Exercise
Stepwise
instructions
Card Status
Progress bar

21. Phase II
Results gathered from from Phase I individual card-sorting
Collaboration added through dynamically refreshing the
user screens through PHP scripts and MySQL
Database.

22. Phase II
CoSort Home
Information
Architecture Moderator Participant
Set up new Introduction to
View calendar Reminder
card sort card sorting
Change View previous
Send invites View details Learning guide
schedule results
Individual card
sorting
Collaborative
card sorting
Submit results

23. Phase II
Moderator’s
dashboard
Reminder for next sort
Setup a new card sort
(inline feature)
Calendar to view
schedule and previous
sorting

24. Phase II
Collaboration
Glow representing
active mouse activity
Video of the participant
Option to delete a card
Option to add a card

25. Phase II
Activity
monitor

26. Phase II
Sorting
Results
Collaborative result
Individual results

27. Implementation
A MySQL database is maintained on the server.
The database is initialized through the interface.
The user interfaces of the system are
synchronized with the data base using PHP
scripts.
As the users interact with the system, the database
is constantly modified again using PHP scripts.
This way all the participant screens operating from
different geographical locations are synchronized
and serve as a real time collaborative platform.

28. User Testing
Testing of Phase I, Card Sort
Testing of Phase II, Co Sort

29. Testing Phase I
24 cards
6 categories
10 participants who
are HCI novices
Age group: 18-22
years
Interview sessions
after each sorting

30. Results
Sum of ratings between 1-5(1-lowest, 5 highest) from 10 users
50
45
40
35
30
25
20
15
10
5
0
Easyness of Knowledge Effectiveness Efficiency of Extent of Enjoyability of How effective Easyness of How satisfying Extent of error
navigation gained of task task confusion exercise can it be a exercise were the freeness
freeness source of results
knowledge

31. Observations
Most of the users enjoyed the activity and could relate to the real websites
A participant took an average of 4 minutes to learn as well as perform card sorting
activity.
There is a bug in one module of exercise which need to be corrected
Users wanted to know “a score” since they played “a game”
Users expected the cards to snap one above other

32. UI Changes
Cards stacking one above
another
Cards separated,
no overlapping

33. UI Changes
Portion indicating mismatch
Stats showing matching
cards and percentage

34. Testing Phase II
15 cards, 6 categories
6 participants in 2
groups
Users of IMDb website
Age group: 18-22
years
Server on a laptop
Skype conference call

35. Results
Sorting was finished successfully by the participants
Participants could easily discuss and operate the cards.
A slight lag was experienced in the synchronization of the cards and pointers due to
server limitations
.
There were times when users got confused due to the lag
An average of 10 minutes was taken during sorting

36. Changes required
A better server needed which can handle the requests from the users
An optimised algorithm which is lightweight from the current algorithm.

37. Summary
A learning tool for card-sorting technique was
successfully conceptualized, developed and
tested.
The learning is hosted on the IIT-G virtual lab in
the form of an interactive experiment.
A new model of card-sorting technique was
conceptualized and a working prototype was
successfully developed and tested with
participants.
Based on the user-testing data, the implied
changes were implemented to improve the
interaction on the tool.

38. Conclusion
The learning tool was effective in educating
participants about card-sorting. Participants rated it
41/50 as a source of knowledge.
The new model of collaborative card sorting was
tested with participants and the results obtained
were coherent with the theory proposed. Both the
phases of the proposed model went successful.
There were a few problems while testing the
collaborative prototype like slight lag in the
synchronization in the 3 participant’s screens. But
it did prove to be a major hindrance and the
sessions could be completed successfully.

39. Conclusion

40. Future Work
The future work involves extending this platform
and incorporating intercultural elements into it.
Features such as language options, themes, icons
etc. would impart a cultural dimension to this tool.
A further extension of this tool would be for mobile
devices on various platforms like
Android, iOS, Windows Phone etc.

41. Acknowledgement
Prof. Pradeep Yammiyavar for his
guidance, encouragement and gracious support
Paranjay Shrivastava, Praveen Alevoor, and
Yogesh Deshpande for constantly helping us in
the times of need.

42. References Jakob Nielsen. "Card Sorting: How Many Users to Test".Useit.
2004. [online]
http://www.useit.com/alertbox/20040719.html [Accessed 10 Oct
2011]
Neil Turner. UX for the masses. [online]
http://www.uxforthemasses.com/online-card-sorting/
[Accessed 10 Oct, 2011.]
Hiroshi Ishii, Minoru Kobayashi, Kazuho Arita. Iterative design of
seamless collaboration Media. August 1994, vol. 37, No.8.
Kildare, R., Williams, R. N., Hartnett, J.,2006. An Online Tool for
Learning Collaboration and Learning While Collaborating. Eighth
Australasian Computing Education Conference (ACE2006), Vol.
52.
Sutcliffe, A., 2005. Applying Small Group Theory to Analysis and
Design of CSCW Systems. Human and Social Factors of Software
Engineering (HSSE).
Katsanos, C., Tselios, N., Avouris, N.,2008. AutoCardSorter:
Designing the Information Architecture of a Web Site Using Latent
Semantic Analysis. CHI 2008, April 5–10, 2008, Florence, Italy.
Copyright 2008 ACM 978-1-60558-011-1/08/04.
42

43. References
Joly, A.,V., Pemberton, L., Griffiths, R.,2009. Card Sorting Activities
with Preschool Children. BCS-HCI '09: Proceedings of the 23rd
British HCI Group Annual Conference on People and Computers:
Celebrating People and Technology.
Jakob Nielsen. "Card Sorting to Discover the Users' Model of the
Information Space".Useit. 1995.[online]
http://www.useit.com/papers/sun/cardsort.html [Accessed on 10
Oct 2011]
Schadewitz, N., Zakaria, N., 2009. Cross-cultural Collaboration
Wiki – Evolving Knowledge about International Teamwork.
IWIC’09, February 20–21, 2009, Palo Alto, California, USA. ACM
978-1-60558-502-4/09/02.
Pekkola, S., 2003. Designed for Unanticipated Use: Common
Artefacts as Design Principle for CSCW Applications.
GROUP’03, November 9–12, 2003, Sanibel Island, Florida, USA.
Rusu, C., Roncagliolo,S., Tapia, D., Hayvar, D., Rusu, V.,, Gorgan,
D., 2010. Evaluating the Usability of Intercultural Collaboration
Platforms: Grid Computing Applications. ICIC10, August 19–
20, 2010, Copenhagen, Denmark.
43

44. References Card Sorting- Introduction[Online]. Available from
http://www.syntagm.co.uk/design/cardsortintro.shtml [accessed 25
August 2011]
Miller J. K., 2010. Intercultural Interaction and Collaboration in a
Virtual World. ICIC’10, August 19–
20, 2010, Copenhagen, Denmark.
Lewis, S., Ellis, J.B., Kellogg, W.A. 2010. Using Virtual Interactions
to Explore Leadership and Collaboration in Globally Distributed
Teams. ICIC’10, August 19–20, 2010, Copenhagen, Denmark.
Donna Spencer and Todd Warfel .Boxes and Arrows.[online]
http://www.boxesandarrows.com/view/card_sorting_a_definitive_gu
ide.[Accessed 25 August, 2011.]
Hudson, W., 2007. Old Cards, New Tricks: Applied Techniques in
Card Sorting. Published by the British Computer Society Volume 2
Proceedings of the 21st BCS HCI Group, HCI 2007, 3-7 September
2007, Lancaster University, UK.
Kodagoda, N., Wong, B. L. W, Khan N., 2010. Open-Card Sort to
Explain Why Low-Literate Users abandon their Web Searches
Early. BCS '10: Proceedings of the 24th BCS Interaction Specialist
Group Conference.
44

45. References
Sinha, R., Boutelle, J., 2004. Rapid Information Architecture
Prototyping. DIS2004, August 1-
4, 2004, Cambridge, Massachusetts, USA.
Qian, X., Yang, Y., Gong, Y., 2011. The Art of Metaphor: A Method
for Interface Design Based on Mental Models. VRCAI 2011, Hong
Kong, China, December 11 – 12, 2011.
Chandler, H.E., 2001. The Complexity of Online Groups: A Case
Study of Asynchronous Distributed Collaboration. ACM Journal of
Computer Documentation, 2001,25:1-2.
Tullis, T. and Wood, L., 2004. How Many Users Are Enough for a
Card-Sorting Study? In Proc UPA’2004.
Dong, J.M., Shirley, M. and Waldo, P. A user input and analysis
tool for information architecture. In Proc CHI 2001, ACM Press
(2001), 23-24.
Rosenfeld, L. and Morville, P. Information Architecture for the
World Wide Web, 3rd edition, O’Reilly, 2006.
Hudson, W. Playing your cards right: getting the most from card
sorting for navigation design, ACM, 2005, 56-58.
45

46. Questions?
Thank You! 46