Playtest With Five People: What You Learn and What Not
Five people watched closely will show you most of your big problems, because the same stumbles keep turning up. What they cannot give you is a percentage, so count problems and leave the shares for when you have far more players.
Why does such a small number work?
Because a common problem is hard to miss. Usability researcher Jakob Nielsen found that one tester reveals on average 31% of the problems in a design. With each new person you see fewer new ones, as they keep hitting what you have already seen. Worked through, five testers find about 84% of those problems. His advice is to run several small rounds, rather than one big one: three rounds of five beat one round of fifteen, because you fix things in between.
Do the same sum for how common a problem is, and the limit shows up:
| A problem hits this share of players | Chance that 5 people show it |
|---|---|
| 50% | 97% |
| 31% | 84% |
| 10% | 41% |
So five people will find what is common, and will easily miss what is rare. Also note where the figure comes from: studies of software and websites, not games. It says nothing about whether a game is fun, and we have no equally solid number for games.
What does a good session give you?
Walnut Rescue is an invented game about a squirrel, and this example is made up. Five people played for 20 minutes each, one at a time. The maker sat beside them, said nothing and wrote down each problem.
| Problem | People who hit it, of 5 | How bad |
|---|---|---|
| Did not find the jump button | 4 | Blocks play |
| Walked past the key | 3 | Blocks play |
| Took the friendly crow for an enemy | 3 | Confusing |
| Text in the menu too small | 1 | Annoying |
| Game froze on pause | 1 | Blocks play |
Read it as a list, not as statistics. "4 of 5 missed the jump button" does not mean 80% will. With five people the real share could be anywhere from 38% to 96%. It does mean this is not rare, and that is all you need to fix it.
The freeze happened only once, and it still goes to the top with the jump button. How bad a problem is matters as much as how often you saw it.
How do you run it?
- Find people who have not seen the game. Friends who love you are poor testers, so try someone who likes this kind of game.
- Ask them to think aloud. They say what they see, expect and try. It shows what they believe, and why. It has limits: people filter their thoughts, and talking while playing feels odd, so remind them gently.
- Do not help. Each time you explain, you lose what you came for. Your questions can steer them, so keep them neutral: "What do you think that does?"
- Write down what they did, and the time. Not only what they said.
- Fix the top problems, then test five new people. Testing the same broken thing again teaches you nothing new.
If your game has two very different audiences, Nielsen suggests 3 to 4 people from each. Afterwards, your AI can help sort the notes.
Paste this to your AI
Help me run a playtest with five people, and read what comes back.
1. Read my game and draw the first [20] minutes as a Mermaid state diagram: what a new player should do and understand at each step, and where I think they might get stuck.
2. Turn that into a one-page sheet I can print: for each step, what success looks like, and a box for notes. Add three neutral questions to ask at the end. None of them should hint at the answer.
3. After the sessions I will paste my notes. Compare them with the diagram, step by step. List each problem with the number of people who hit it, what they did, and how bad it is: blocks play, confusing or annoying.
Do not turn the counts into percentages. With five people, say "4 of 5", and tell me these counts show a problem exists, not how common it is. Rank by how bad first, then by how many. If my notes do not cover a step, say that step was not tested. Suggest what to change before the next five.
Or skip this and connect the MCP: your AI then finds all of this by itself.Connect the MCP
How to check it worked
- Read your notes back. Do they say what people did, or what you think they meant?
- Check the order. Blocking problems first, even if only one person hit them.
- Watch the next five. The old top problems should be gone. If new ones appear, that is the method working.