What is card sorting?
Definition: Card sorting is defined as an interactive research technique designed to analyze how people understand and categorize information. The card sorting data gives organizations a better sense of how they should present information to a target audience. What makes sense to an organization may not always translate to the customer. The card sorting survey feature closes this gap and ensures everyone is on the same page. As a question type, it is highly interactive and allows respondents to sort, categorize, and rank based on the order of preference.
When to use card sorting
Use card sorting as an interactive way to understand your user’s or customer’s understanding and expectations of your topics. Here are some real-life examples of when to use card sorting.
- Grouping features and attributes together of new products/services.
- Designing or improving existing websites – what site info should go where.
- Understand and compare different concepts and ideas.
- Find out how people rank or arrange concepts given based on a set of criteria.
Types of card sorting
The card sorting survey feature comes with two answer types – closed and open to better engage survey respondents. Let’s take a look at both the types of card sort questions that QuestionPro offers.
- Closed card sorting: In the closed card sorting question, the survey designer defines the categories to place answers. The respondents can organize the answer responses or cards into these predefined categories.
- Open card sorting: In an open card sorting question, the survey respondents are free to determine their categories. Use the open card sort method to read your respondents better and capture their perceptions accurately.
Respondents can sort the topics into categories or buckets with an appealing user-experience design. We will go over both options to help you determine which type of card sorting is the best for you.
Card sorting examples
Let’s look at examples of an open and closed card sort each.
Closed card sorting
If you want your respondents to categorize a list of Hollywood movies into predefined genres, you can choose the closed card sorting question to let them define the genres. For example, you can ask respondents to categorize the below-mentioned movies into genres – action, horror, and comedy:
List of movies:
- The Conjuring
- Due Date
- Terminator: Dark Fate
- Annabelle
- Hangover
- Avengers: Endgame
If there are categories that already exist that the audience understands, then using the closed card sort question would be the best option.
Open card sorting
Similarly, like the example above, for open card sorting, give the movie options to the respondents. But ask them to categorize the movies without giving them the genre options.
If category names are not universally recognized, using the open card sorting option would get the best understanding of what words people associate with the items being categorized and ranked.
Setting up and analyzing closed card sorting
Instructions to add the closed card sort question:
- Go to: Login » Surveys (select a survey) » Edit » Workspace.
- Click on the ‘Add question’ button.
- Under advanced question types » ordering, click on ‘Card Sorting’ question.
- After you select the template, add the question text, predefined categories, and answer options.
- Use the edit in bulk options to do a quick copy/paste of content.
- Click on ‘Settings’ and select the answer type as ‘Closed.’
Under settings, update the answer display order from the following drop-down options:
- Default
- Ascending
- Descending
- Random (with the opportunity to apply a specific number of randomization to overall answer options)
- Advanced randomization
On the bottom left corner of the question template, apply validation options to either force a response or request a response from a specific number of answer options. Collect smarter research insights using card sorting with validation.
LEARN ABOUT: Action Research
What respondents see – closed card sorting
Here, the respondents need to drag the options from the left-hand side box to the respective categories on the right-hand side. For example, “The Conjuring” will go into the ‘horror’ category, and “Hangover” will go into ‘comedy.’
Analyzing the results for closed card sorting
Analyzing the results above, you can see the items sorted into given categories – comedy, horror, and action. For “The Conjuring,” 91.47% of respondents categorized it as a horror movie while 8.33% categorized it as an action film.
The count and percentage of responses received and its average ranking is given for every item. For “The Conjuring,” 11 people selected it as a horror film while only one selected it as an action film. And although “Annabelle” also has 11 counts of horror, “The Conjuring” ranks in the top spot for horror in the list of movies given.
Setting up and analyzing open card sorting
Let’s go back to the movie example again. This time, instead of predefining the categories, we will let the respondent create their categories and rank according to the list of movies given. Important things to note:
- For open card sorting, make sure to provide instructions in the question section for respondents to create their categories before dragging and ranking the answer options
- The percentage calculation dashboard will look different from the closed card sorting results but will still show each item out of 100%.
To add an open card sorting question:
- Go to: Login » Surveys (select a survey) » Edit » Workspace.
- Click on the ‘Add question’ button.
- Under advanced question types » ordering, click on ‘Card sorting’ question.
- After you select the template, add the question text, predefined categories, and answer options.
- Use the edit in bulk options to do a quick copy/paste of content.
- Click on ‘Settings’ and select the answer type as ‘Open.’
Under settings, update the answer display order from the following drop-down options:
- Default
- Ascending
- Descending
- Random (with the opportunity to apply a specific number of randomization to overall answer options)
- Advanced randomization
Use the validation option at the bottom left corner of the question template to force or request a response and capture smarter research insights.
What respondents see – open card sorting
When respondents take a survey with the open card sorting question, the respondents need to drag answer options one-by-one from the left-hand side box into the box “drop an item here to create a new category” on the right-hand side. A new box will be created below. Respondents must name the category, as shown in the above image.
For example:
- Respondents need to drag “The Conjuring” into the red box – ‘drop-in item here to create a new category.’ They can name the new category as ‘horror’ in the grey text box. Now, they can drag and drop horror movies from the answer options into the horror category.
- Similarly, respondents need to drag other movie titles into the red box – ‘drop-in item here to create a new category’ if they want to create a new category and name it as applicable. Now, they can drag and drop remaining movie titles from the answer options into their respective categories.
Analyzing the results for open card sorting
To view the report for card sorting question:
- Go to: Login » Surveys (select a survey) » Analytics » Dashboard.
With the open card sorting results, you can see the categories created by the respondents – comedy, horror and action, and their respective movies along with a percentage calculation report by the film.
View the count and percentage of the responses received for each item and its average ranking. For “Due Date,” you can see 12 total counts with 100% being respondent-categorized as a comedy while “The Hangover” had 11 categorizations as a comedy and one as an action film. Reviewing the total average rank, “Due Date” came in as the top-ranked comedy in the list of movies.
The card sorting question type is available with QuestionPro’s Enterprise license. If you are interested in the card sorting question and would like to learn more about additional enterprise features and services, contact us here, and we’ll be happy to answer any questions or set you up with a free trial.
