Merging Methodologies: Individual and Group Card Sorting
โขDownload as PPTX, PDFโข
1 likeโข1,096 views
This presentation is a case study describing how we combined individual card sorts with focus groups and group card sorting to improve the content hierarchy and organization of www.libertymutual.com, the personal insurance website of Liberty Mutual, which customers can visit to get an insurance quote, service their insurance policies, or find insurance-related information. We analyzed quantitative and qualitative data from 26 participants, on which we based our recommendations for a new hierarchy and site structure. Our paper will show how the results from the individual and group sorts differed, how the individual exercise informed the group exercise, and how the group exercise informed the recommendations. We believe this combination of individual sorting, group sorting, and focus group discussion makes this methodology unique. The paper on which this presentation is based is available here: http://bit.ly/14jWa7G
![Merging Methodologies: Combining
Individual and Group Card Sorting
Robert L. Thomas and Ian Johnson
Liberty Mutual, Personal Insurance, Boston, MA
[robertl.thomas, ian.johnson]@liberty.mutual.com
Personal Insurance](https://image.slidesharecdn.com/merging-methodologies-individual-group-card-sorting-130811123838-phpapp01/85/Merging-Methodologies-Individual-and-Group-Card-Sorting-1-320.jpg)
Recommended
Recommended
More Related Content
Similar to Merging Methodologies: Individual and Group Card Sorting
Similar to Merging Methodologies: Individual and Group Card Sorting (20)
Recently uploaded
Recently uploaded (20)
Merging Methodologies: Individual and Group Card Sorting
- 1. Merging Methodologies: Combining Individual and Group Card Sorting Robert L. Thomas and Ian Johnson Liberty Mutual, Personal Insurance, Boston, MA [robertl.thomas, ian.johnson]@liberty.mutual.com Personal Insurance
- 2. Liberty Mutual Insurance Our old-style navigation was no longer useful for a major insurance companyโs website ๏ง History o www.liberymutual.com had moved over the years to a large site with multiple lines of business ๏ง Issues o Display of a total of only nine options in the top right corner of each page o Lack of a traditional navigation bar Proprietary - Trade Secret (Competitively Sensitive Information) 2
- 3. Liberty Mutual Insurance We needed to make content discoverable and easy to find ๏ง Scope o Create a new navigational hierarchy to replace the current one ๏ง High-Level Objectives o Focus on the navigation and organization of the website o Ensure the navigation labels are understandable o Make the site attractive to customers Proprietary - Trade Secret (Competitively Sensitive Information) 3
- 4. Liberty Mutual Insurance To meet the needs of the project, we needed to run a card sort, a categorization exercise where users group physical or virtual cards together โข โCard sorting is excellent for situations where you want the usersโ mental model to drive the information architecture of the product.โ - Courage & Baxter (2005) โข โOpen sorts are used for discovery. Closed sorts are used for validation.โ - Morville & Rosenfeld (2006) Proprietary - Trade Secret (Competitively Sensitive Information) 4
- 5. Liberty Mutual Insurance We reviewed research on card sorting methodologies and then developed a new methodology Proprietary - Trade Secret (Competitively Sensitive Information) โข Repeated card sorting โRepeated sorting of a set of items to understand underlying dimensions or characteristics of a product or serviceโ - Curran, Rugg, Corr (2005) โข Delphi and modified Delphi card sorting โParticipants build on the results from previous card sortsโ - Paul (2008) โข Focus group card sorting โParticipantsโฆ complete the card sort individually. Then, [you] lead a discussion with [them] regarding theโฆ strategies they usedโฆ.โ - Hawley (2008) 5 We developed a new methodology, one that had not been done before. Simple to run Requires only 10-20 people Useful for understanding emotional dimensions of a product Not useful for grouping information into different categories Reduces time Requires only 8-10 people, butโฆ Subsequent participants are modifying a card sort others started UX research experts recommend 15-30 participants for a card sort Collect quantitative card sort data Collect qualitative insights Focus group discussion may be biased by group think Be careful about running a card sort with only 1 group of 8-15 participants
- 6. Liberty Mutual Insurance Our methodology combined online card sorting, physical card sorting, and focus group discussion with 26 participants split between 2 sessions Proprietary - Trade Secret (Competitively Sensitive Information) 6 Source: http://www.flickr.com/photos/cflc/2098395894 (far left); Schlesinger Associates, New Jersey (far right) โข 13 laptops & 13 participants โข 79 card sort labels โข WebSort online card sort tool โข 45-minute session โข 15-minute break โข 3 groups of 4-5 participants โข 79 labels on Post-Itโs โข 1 moderator & 1 note-taker โข Group spokesperson presented โข 40-minute session โข 1 moderator & 5 note-takers โข Discussion topics โข 20-minute session Individual, Online, Un-moderated, Open Card Sort Team-based, Physical, Moderated, Open Card Sort Focus Group Discussion
- 7. Liberty Mutual Insurance Breaking into teams, we followed a four-step analysis process to arrive at our final navigation categories Proprietary - Trade Secret (Competitively Sensitive Information) 7 Group Card Sort Team Individual Card Sort Team 1 Standardized Labels & Combined Categories 2 Compared Grouping of Labels 3 Created & Compared Site Maps 4 Finalized Navigation Categories Source: Illustrations by Jeffery Callender; Template by Livia Labate
- 8. Liberty Mutual Insurance Standardized labels and combined categories Proprietary - Trade Secret (Competitively Sensitive Information) 8 Sorter Original category Standardized category Sort20 About About the Company Sort7 About Our Company About the Company Sort17 About the Company About the Company Sort23 About the Company About the Company Sort19 About Us About the Company Sort21 About Us About the Company Sort8 ABOUT US About the Company Sort9 about us About the Company Sort22 company About the Company Sort1 Company and website info About the Company Sort10 Company Info About the Company Sort2 Company Info About the Company Sort24 company info About the Company Sort25 Company info/history About the Company 1 2 3 4 Table 1. Standardized Categories for Individual Card Sorts
- 9. Liberty Mutual Insurance Table 2. Final list of 11 in-person, group standardized categories Proprietary - Trade Secret (Competitively Sensitive Information) 9 Sorters Who Total Cards in Unique Standardized Category Used This This Category Cards Agreement Standardized labels and combined categories 1 2 3 4 Average of 13 cards (40รท3) in this category
- 10. Liberty Mutual Insurance Group Card Sort Results Dendrogram Proprietary - Trade Secret (Competitively Sensitive Information) 10 Compared grouping of labels 1 2 3 4 Information Tasks
- 11. Liberty Mutual Insurance Home Our Company Policy, Billing & Advice Types of Insurance Individual Card Sort Site Map Created and compared site maps 1 2 3 4
- 12. Liberty Mutual Insurance Home About Us Auto Insurance Home Insurance Insurance Basics Insurance Products Claims Center Group Card Sort Site Map Created and compared site maps 1 2 3 4
- 13. Liberty Mutual Insurance Finalized navigation categories Proprietary - Trade Secret (Competitively Sensitive Information) 13 1 2 3 4
- 14. Liberty Mutual Insurance In summary, the combination of individual card sorts, group card sorts, and focus groups was successful ๏ง Our new methodology allowed us to: o Obtain qualitative and quantitative data o Recruit a large sample size o Save money and time ๏ง The benefit of this unique method is that the individual card sort activity enriched the group card sort activity and focus group discussion Proprietary - Trade Secret (Competitively Sensitive Information) 14
- 15. Liberty Mutual Insurance In conclusion, two years later, the site navigation has had a positive effect on the business ๏ง Key performance indicators and web analytics demonstrate that customers are actively using the new navigation ๏ง In multiple usability tests since the navigation update, users have commented positively on the navigation, rating it as one of the best attributes of the site ๏ง J.D. Power and Associates stated that โLiberty Mutual provide[s] menus that [allow] shoppers to easily find relevant shopping tools on any pageโ Proprietary - Trade Secret (Competitively Sensitive Information) 15
- 16. Proprietary - Trade Secret (Competitively Sensitive Information) 16
Editor's Notes
- HistoryWe began this project in 2010. At the time, the Liberty Mutual website had transitioned from a small site with limited content to a large website with multiple lines of business. So our old-style navigation was no longer well-suited for users visiting the website of a major insurance companyIssuesLimited number of navigation optionsUsers could access only nine categories in the top right corner of each pageFour of the nine categories took users to a different websiteSome options were not top-level items that users wanted (such as Member Rights)And we lacked a traditional navigation bar with drop-down options, which would give our customers access to so many more categories
- ScopeWe needed to create a new navigational hierarchy, one that would replace the current one and allow:Current and potential customers to easily find specific options under main categoriesHigh-Level ObjectivesTo focus on the navigation and organization of the website, including first- and second-level navigationTo make sure our users could understand the navigation labels we wanted to use (keep away from insurance jargon)To make the site attractive to customers (delightful, satisfying experience)Again, all with to goal to makecontent discoverable and easy to find
- So we employed this methodology that combined online card sorting, physical card sorting, and focus group discussion with 26 participants. To prevent any bias, we ran the card sort at a neutral off-site facility. We recruited a mix of customers and non-customers, following our standard recruiting criteria. We conducted one session with 13 participants in the morning, and a second session with another group of 13 in the afternoon. Each lasting two hours. Individual, Online, Unmoderated, Open Card SortHereโs how each session went. First we started with the individual card sort. Here participants completed an online, unmoderated, individual, open card sort, using WebSort. This took 45 minutes. Team-based, Physical, Moderated, Open Card SortNext, we asked participants to take a 15-minute break, and then we set up for the next part of the process, the group card sort. During the break, we removed the laptops and laid down brown paper to cover each of the tables. We brought our participants back in. Next, we handed out Post-It Notes that had the same 79 labels we used in the individual card sort. We asked participants at each table to work together and complete an open card sort as a team. As each team worked together, our facilitators moderated each teamโs activity, and a note taker captured qualitative feedback and group discussion.After they finished the group card sort, weasked each team to nominate a spokesperson to give a short presentation, explaining how and why they organized the information the way they did. The group card sorts and presentations took approximately 40 minutes.Focus Group DiscussionFor the final 20 minutes, we conducted a focus group discussion. Participants sat in their original seats, with the completed group card sort results still in front of them. Members of our research team took notes. Examples of discussion topics included:How would you like to have your information organized?How many categories should there be? How many are too many? Too few?Were there any items that were difficult to categorize or name?
- After the 26 individual and 6 group card sorts were completed, we began analyzing the data. Three members of our researchteam analyzed the individual card sorts, and three other members of our research team analyzed the group card sorts. 1. Standardized category labels and combined similar categories2. Compared grouping of labels3. Created and compared site maps4. Finalized navigation categories
- Our 26 participants created 210 unique category labels during their individual sorts. Our 6 groups of 4-5 participants used the same cards for the group sorts, and created 46 unique category labels. Some participants and groups used the same labels for categories (for example, โAbout Usโ), while others used different labels for similar concepts (likeโAbout Our Companyโ and โCompany Infoโ). As suggested by Donna Spencer, we first standardized the main categories and applied consistent naming conventions. We gave categories with similar names or concepts a consistent name and combined groups where participants used the same basic concept but a slightly different label. For example, standardizing a category name of โAbout the Company.โ
- Second, each member of each team separately combined similar standardized categories. Each member then presented and explained their categorization, and then each team agreed upon and arrived at our totals of standardized categoriesFor example, in the group sorts, participant groups put an average of 13 cards (40/3) into the โAuto Insuranceโ category and used 18 different cards. This category has an agreement number of 0.74, meaning that 74% of participant groups put the same 13 cards in this category.
- Third, we compared the grouping of labels between the online, individual studies and the in-person group studies to see if there were any patterns. We generated โtree diagramsโ or dendrograms, which display not only the relationship between items but the strength of that relationship, or how frequently items are associated. Both individuals and groups wanted to separate information from tasks. For example, information on benefits and features โ such as our labels for โAccident Forgivenessโ and โUnlimited Car Rentalโ โ were isolated into a separate information category, โCoverages & Benefits.โ Alternatively, the Auto Insurance category was focused on tasks, such as โTool to Estimate Car Insurance Coverage.โ This revealed a key finding that both the individuals and the groups organized the content by information AND by task.
- Next, we created site maps to organize the results from the individual and group card sort exercises. What this visualized was that individuals had sorted items into a narrow and deep hierarchy. The site map created from the individual card sorts consisted of only 3 major categories โ (1) Our Company; (2) Policy, Billing, and Advice; Types of Insurance.
- But what the group card sort revealed was that our groups had sorted items into a medium navigational hierarchy, with 6 major categories, or double what our individual card sorts showed us โ (1) About Us, (2) Auto Insurance, (3) Home Insurance, (4) Insurance Basics, (5) Insurance Products, and (6) Claims Center.
- Finally, we created a 6-category navigational bar.We moved About Us into a top-level navigation bar that all Liberty Mutual websites share. We moved Life Insurance to the same level as Auto Insurance and Home Insurance, based on the objectives of both our participants and our stakeholders, who wanted all insurance products at the same category level.ย
- The combination of individual card sorts, group card sorts, and focus group discussions allowed us to:Obtain qualitative and quantitative data in one dayRecruit a large sample sizeSave money and timeThe benefit of this unique method is that the individual card sort activity enriched the group card sort activity and focus group discussion, aiding in productive negotiations and discussions for the group exercisesย
- Two years later, the site navigation has had a positive effect on the businessKey performance indicators and web analytics demonstrate that customers are actively using the new navigationIn multiple usability tests since the navigation update, users have commented positively on the navigation, rating it as one of the best attributes of the siteIn its 2012 Insurance Website Evaluation, J.D. Power and Associates stated that โLiberty Mutual provide[s] menus that [allow] shoppers to easily find relevant shopping tools on any pageโ