Reliable Research Sample Sizes
As a user experience researcher and product strategy consultant, one of the most common questions I encounter from clients is: How many tests or interviews do we need to do? How many sessions do we need to gather useful and reliable insights? It's a valid question, and the simple answer is, as with most things UX-related, "it depends." Let's explore why, and how we can navigate this challenge together.
Reliable Research Sample Sizes
How Many Research Participants Do You Really Need?
The Question of Sample Size
One of the most debated topics in UX research is sample size – how many participants are needed to gather reliable insights? Most UX researchers are confident in their ability to uncover rich, qualitative insights with smaller, more focused studies with 5-10 participants. But, in our experience, clients struggle to accept those findings as reliable, often looking to ensure statistical validity. So, where is the sweet spot?
The Nielsen Norman Group (NN/g), a leading authority in UX research, has been a go-to source for valuable guidance around this persistent challenge in the industry. In their article How Many Test Users in a Usability Study?, they propose a pragmatic approach: Start with a modest sample size of 5 users for a single usability test, then iterate based on findings. They emphasize the importance of iterative testing and the diminishing returns of adding more participants to any one study beyond a certain point.
Their analyses have shown that testing with 5 people allows you find almost as many usability problems as you'd find using many more test participants. Great, so 5 is the sweet spot? For usability studies, focusing on gathering feedback on the UX/UI of a given product, we say yes.
However, some research studies necessarily go beyond a binary pass/fail, yes/no usability study. Especially for teams in the early stages of their product development cycle, there may be a need for some usability feedback, some product-market fit exploration, and some generative research insights. We often guide teams to break up those more complex research efforts into distinct rounds of research, each requiring a unique sample size based on the research objectives and resourcing considerations.
So, 5 is not always the magic number. But it's a great place to start, to learn enough to make a positive impact on the usability and positioning of your product, and to get your team more comfortable with conducting research and acting on the findings.
Start With Your Objectives
Ultimately, there is no one-size-fits-all solution to the question of sample size in UX research. It requires a nuanced understanding of your research objectives, your target audience, and the context in which the research is conducted. But the best first place to start is always your research objectives. What are you hoping to learn from the research? What questions do you want to answer? What do you want to validate, explore or understand?
Sometimes crafting objectives can feel a little disconnected and vague, especially if you've been living in the weeds of UI designs, user flows, and protyping. If that's happening to you, try writing out our specific questions, and why you care about getting them answered.
A great mad-lib we picked up from our dear friend and colleague, Megan Casey, and like share with clients to facilitate this brainstorming goes like this:
We wonder ___________________________.
We care because ____________________________.
We assume ___________________________.
We know __________________________.
Focus on the first two, and trickle into the 3rd and 4th if you have the time and are able to clearly recognize your own assumptions/biases (most people can't, so don't stress over it.)
Once you've got your research questions & objectives on paper, bring them to your research partner to help you get them answered in the most effective and efficient way possible.