Playtesting
Learning paths through this topic
I rewrote chapter two: which earlier tester comments still apply?
Playtest notes pile up while the story keeps changing, and a complaint about a scene that no longer exists wastes your time. This entry marks each comment as still valid, outdated or orphaned, without throwing away comments over a fixed typo.
Do testers pick the top answer just because it is on top?
If 70 percent choose the kind answer, is your story pulling them there or is it just listed first? This entry shows how to swap the order for half of your testers and then pull the two effects apart with one small calculation.
Four of five testers were confused by the ending: how much does that tell me?
Five people are not a poll, but they are not nothing either. This entry turns a count like four out of five into a range of what is probably true for all players, so you can decide whether to fix it, and how many more testers it would take to be sure.
My game runs smoothly for a while, then slows down as the phone gets hot
Short tests on a cool phone never show what a twenty-minute session does to it. This entry shows how to find the minute from which the game really slows, how to ignore one-off hitches like a loading screen and what a long test session has to look like.
My level generator makes a few seeds that are far harder than the rest
A generator promises levels of equal difficulty, but some starting numbers produce a wall. This entry shows how to find those from play logs without being fooled by seeds that only a few people tried, and how to keep one extreme seed from hiding itself.
My random sliding puzzle sometimes cannot be solved at all
Shuffling the tiles at random hands about half of the players a puzzle that no sequence of moves can finish, and they blame themselves. This entry shows the quick test that tells a solvable board from a stuck one for any board width, and a one-swap repair.
A rare bug did not show up in ten test runs: is it fixed?
A bug that appears only now and then can easily hide through a handful of clean tries after a fix. This entry shows how many clean runs you need before you may say it is gone, and what a run of clean tries can and cannot tell you about how often it still happens.
A level was cleared 3 times out of 3, so is it easy? How many plays I need
A clear rate from a handful of plays is a guess dressed as a number. This entry shows how wide the honest range around a rate really is, how it shrinks with more plays and how to say "too early to tell" instead of crowning or condemning a level.
Five testers found twelve problems, so how many are still hidden?
After a playtest you have a list of problems, but no idea whether it is nearly complete or just the tip. A simple look at how many problems only one person found gives an honest guess of what is still missing, and tells you when to test again.
I log only a share of players to save data, but my funnel numbers look broken
Recording everything is heavy on phones and on your quota, so you record a share of it. Done the wrong way, the later steps of a path shrink much faster than the first ones. This entry shows what to pick the share by, so paths stay whole, and how to scale counts back up.
How long can a reader go in my story without making any choice?
Long blocks of pure text lose people even when the writing is good. Before you test, count how many minutes of reading sit between two moments where the player actually does something, and see which stretches are worth cutting or breaking up.
Did the monster catch testers fairly, or before they could have known?
Being caught is part of a chase, but being caught with no sign beforehand just feels cheap. This entry checks every catch against the last warning the player got, and points at the places in your map that cheat.