We love a good list at ZURB, we use them for planning out our day, the work we do with our clients, and for setting our quarterly goals. Seriously, you should see our Notion account, it’s the stuff of legends. So naturally, when it comes to learning from customers we love to use List Questions in Helio to do so!
We use them frequently because they’re quick and simple to set up, easy for other team members to understand, and most importantly, a pattern that participants are familiar with. List Questions are a win, win, win!
Get the most from a list
When using List Questions to learn from customers you gain answers on a spectrum. Each different question digging deeper into customers’ desires, values, and preferences. List question types can be used by themselves or in a combination of others to surface a deeper understanding of how your customers perceive your product. Here’s how to get the most from each:
Multiple Choice surfaces the choices your audience would pick out of a specific group of options.
Point Allocation reveals the relative difference between prioritized options.
MaxDiff shows us the extremes of how your audience feels about your list.
Card Sort builds the mental model your audience uses to categorize your list.
Ranking provides an ideal order that your audience would rank your list.
List questions like Multiple Choice, MaxDiff, Ranking, Card Sort, and Point Allocation are all great question types to use when learning from your customers. We made a matrix to help you think about your lists.
Source Link: https://zurb.us/2BVBcWS
On a recent project with our friends over at Hulu, we worked to understand their audiences’ behaviors, and it got us thinking about how we might use our new question types to learn about their categories. We just released MaxDiff and Ranking.(We’re currently working on Card Sort and Point Allocation). What can you do with a list? Here are the categories we used from their service:
[Most Interesting Hulu Categories]
TV for You
Movies for You
Keep Watching
Hulu Originals
Hulu Picks
Newly Added TV
Newly Added Movies
We then started imagining other ways we could use list-type questions in Helio. Beyond multiple choice, we explored the different types of feedback you can gather with a simple list in different survey questions, like MaxDiff and Ranking. Inspired by the work we did with Hulu we put these three question types to the test to see which categories were the “Most Interesting” and the “Least Interesting” to our participants. Here are the categories we tested (which should sound familiar if you’re a shameless binge streamer like us!):
Multiple Choice
Use Multiple choice in your creative process to understand your audience’s choices from a variety of specific options, such as behaviors or impressions in a specific scenario.
We love Multiple Choice because it gives participants a look at a list of options that they’re most likely to place on one side of a spectrum or the other. With our 7 Hulu categories, we were able to uncover the top-3 breakdown of which categories are most important to our media streaming audience. The strongest choice out of your list of options becomes very clear, as well as the next two options that take high priority behind it!
Source Link: https://zurb.us/33p9fC6
Beyond high-level priority, here are other ways a multiple choice question can give you the feedback you need on your lists:
- Emotional reception & opinions. Allowing participants to choose from any and all of 8 emotional impressions gives you a look into what negative feelings may be felt by your customers.
- Gauge what someone may do next. When presenting a scenario to a customer, you’ll want to know what the customer will do next. Giving a list of actions is helpful to understand how your customers are making decisions.
- Feedback on copy & messaging. Asking a customer to choose a tagline out of a list based on a marketing image is a great way to ensure your message is on the right track.
- Take participants down separate paths. Multiple Choice Questions make a great pair with branching logic at the beginning of a test as a way to get more relevant feedback from certain groups of people in your audience.
Multiple Choice is our original list question in Helio because it is a very popular question type. You’ve probably seen Multiple Choice questions your whole life, which is what makes them great. Customers are already familiar, and know how to answer these types of questions; so there’s little confusion.
MaxDiff
Use MaxDiff in your creative process to get a strong understanding of the extremes your audience is experiencing about your designs.
People find it easy to identify the best and worst parts of a list, and MaxDiff leans into this tendency to give you a look at what those extremes are across an entire audience. For the Hulu categories, people are eager to dive into newly added movies, but found Hulu picks to be the least exciting category, even though there is some quality content to be had there!
Source Link: https://zurb.us/30qII5t
When participants are asked to make a trade-off, you get some interesting information about how your list is perceived by a group of likeminded people:
Understand the extremes your participants feel. As you can see in our data above, we get a good look at which options on your list fall on the far ends of your spectrum.
Discover if there’s a trade-off you need to make. MaxDiff will show you which parts of your list are least important to your audience so you don’t waste valuable production time.
Prioritize your “must-have” options. The list options with the least of low priority selections are solid places to start for an MVP (minimum viable product).
MaxDiff uses just a single question to gauge preference across multiple areas at once. What makes it great is the way it removes the need for multiple steps in the process of learning a customers’ preference.
Rank Order
Use Ranking in your creative process to understand how your audience prioritizes options in different situations, from feature concepts to orders of operation.
Ranking items is something we do subconsciously all the time. You can get powerful information about how your participants prioritize a group of ideas with Ranking. This is different from multiple-choice priority because you get a top-to-bottom look at where each of these list items lives in a person’s priorities, rather than just the strongest three. Similar to MaxDiff, you not only can understand the extremes of your participants' feelings towards your list, but also which items would make the cut for an MVP.
Source Link: https://zurb.us/33omzH2
Beyond just seeing an ideal rank order of a list, here’s how else Ranking Questions can get you big wins:
Flow & journey testing. While we prefer to look at what users do rather than what they say, this question type can open up the possibility of seeing the order of operation for tasks - giving you more signals around how people approach a sequence problem.
Understand the extremes your participants feel. Discover which items on your list fall on the far ends of a spectrum. Like MaxDiff, this is great for understanding where you might need to make trade-offs!
Rank Order Tests are the newest addition to the Helio toolbox and we couldn’t be more proud. We’ve already found it to be a highly valuable question type for our projects and (as you can see from our Hulu test) surfaced the entire spectrum of preference.
Card Sort
Use Cart Sort in your creative process to understand how your audience thinks about grouping concepts and where you can match their mental model.
Card Sort is a fan favorite and is next on the list for our summer launch party. We’re very excited about Card Sort because it helps you understand the mental model of your participants through categorization. You can find a different value out of lists when you ask your participants to break them up and group them in unique ways, which help you exceed expectations and maximize the information architecture of your designs.
Group your list by how it makes participants feel. You can quickly understand positive and negative emotional patterns in your list based on your customers’ perspective.
Match the mental model of your customers. Solidify your information architecture by understanding your audience’s ways of thinking about a group of ideas.
Point Allocation
Use Pont Allocation in your creative process to discover the relative importance of options in your list, so you know just how strongly your audience feels about one idea over another.
Point Allocation is similar to Rank Order questions in that participants are ranking items that they find desirable. The difference is in Point Allocation participants are given 100 points to spread across the items on the list. Assigning points allows you to see the complex feelings your participants have towards the items on your list. When participants can assign a value to each item themselves you uncover tons of information about how your audience prioritizes a list and how strongly they feel about certain ideas due to the greater amount of information being provided by participants.
Discover complex information about priorities. Similar to ranking questions, Point Allocation will allow you to see the ideal order your audience would prioritize a list in.
Learn your participants’ feelings across a wide spectrum. At the same time, you can uncover just how strongly those options are prioritized over each other. With Point Allocation, you can see the relative difference in how participants feel between those 1st, 2nd, and 3rd ranked items.
We love to use list questions in our work because everyone knows what the concept of a list is, making this a universal way to learn from customers. Their versatility and Helio’s customization features mean you don’t ever need to compromise the questions you’d like to ask.
