Playtesting
Learning paths through this topic
Let testers press a key when they feel tense and find the flat stretches
Asking people afterwards how tense a level was gives you mush. A single key pressed in the moment gives you a curve over time. This entry builds that curve fairly, so one nervous finger does not look like a crowd and the delay of human reaction is corrected.
A player must see the same test version every time, and tests must not overlap
If a player lands in version A today and B tomorrow, or the same lucky people are in every test, the results mean nothing. This entry shows how to hand out versions from the player's identity alone, so it is stable, keeps earlier players when a test widens and stays independent between tests.
Every tester plays my levels in the same order, so tiredness skews the results
If everybody meets the hard level last, it looks harder than it is, and the level right after a tough one looks easier. This entry gives a way to hand out orders so each level lands in each position equally often and follows each other level equally often.
A tester said the level was easy, yet they died fourteen times
What people say after a playtest and what they did during it often disagree, and the difference is useful. Comparing a tester's own rating with their behaviour shows who to believe on which point, and where the questions were unclear.
Which text speed should be the default after watching five testers?
Most people never open the options, so the setting they ended with is not always the one they like. This entry tells apart testers who chose, who accepted and who never looked, and gives a default you can defend, plus a warning when the data is too thin.
I planned a fifty-fifty test but one group came out bigger: a bug or chance?
Before reading any result of a test, check that the groups have the size you planned. A lopsided split is the cheapest early warning that something broke, such as one version crashing and losing its players. This entry shows the simple check and where to count.
My level has mostly thumbs up, but only those who finished can vote
If the vote button appears after a win, every vote comes from someone the level did not defeat. This entry shows how to put the vote back into context of everyone who started, and one pattern that tells a loved level from a level that lost its critics.
My hint text vanishes quickly: do players have time to read it?
A line of help that fades away or gets tapped past can be gone before anyone understood it. This entry shows how to work out how long a text needs on screen, how to spot from the logs that people are skipping it, and why the language changes the answer.
How many testers saw the plot twist coming, and how early?
A twist that half the table predicted three chapters ago does not twist. This entry shows how to ask testers for guesses while they play, how to ignore the "I knew it" they claim afterwards, and how to read the result.
Two colors in my puzzle may look the same to colorblind players
A puzzle that depends on telling pieces apart by color can be unsolvable for a whole group of players, and the usual red against green is not even the worst pair. This entry shows a check that flags exactly the pairs that fall together, so you can add a second cue.
Two of us labelled the scenes "scary" or not and agreed 80 percent: is that good?
When most scenes are plainly not scary, two people agree a lot even if they judge by coin toss. This entry subtracts the agreement you would get by chance and tells you whether the labels are solid enough to build decisions on.
Testers did better on the second version, so was it better or just practice?
When the same people play version A and then version B, they improve simply from having played. Mixing the order across testers and then correcting for it separates the effect of your change from the effect of practice.