Playtesting
Learning paths through this topic
Playtest With Five People: What You Learn and What Not
Five people watched closely find most big problems, as the same stumbles keep turning up. They cannot give you percentages, so count problems.
Read Playtest Feedback With AI, Without Fooling Yourself
Have your AI sort comments by people, not by volume, quote each one word for word and check claims against what the log shows they really did.
Compare Two Versions: Is the Difference Real or Luck?
Give each new player a version at random, fix how many you need first, and only then look. Ask how often luck alone would give a gap as big as yours.
Compare two versions
You changed something and a number moved, and your AI works out whether that is a real difference or could just be luck, and how many players it would take to find out. It works on counts such as wins and on numbers such as seconds, and it will not call a gap real on too few players.
Levels get harder as the game goes on, so which one is a real spike?
A difficulty curve is supposed to climb, so looking for the highest bar finds only the last level. This entry shows how to compare each level with its neighbours to find the ones that jump out of the slope, and why the first and last levels cannot be judged.
Can a bot that moves at random already clear my maze? Check before any tester
Before you spend human testers on a level, ask how easily pure chance gets through it. A level that blind wandering clears often has no puzzle in it. This entry shows how to work out that chance exactly, without running thousands of random attempts.
My game reports a good frame rate, yet testers say it stutters
An average can look perfect while a few long hitches ruin the feel. This entry shows which three numbers to read from the recorded frame times instead, how to compute the average the right way round and what share of slow frames a phone game can bear.
Is the sad ending rated low because it is sad, or because it came too soon?
Testers who reach the bad ending usually played less, and people who played less tend to be less moved. This entry compares like with like, so you learn whether the ending itself disappoints or merely arrives before the player is invested.
Testers see three versions of a scene: which order avoids favouring one?
Whatever version a tester sees last or first gets an unfair edge, through freshness or through novelty. This entry makes a schedule in which every version appears in every position equally often and follows every other equally often.
Late in my idle game the number stops growing even though income is positive
Computers keep only so many digits, and past a certain size a small gain is rounded away completely. This entry shows how to work out the size at which your game starts losing income, how much one-by-one adding loses compared with adding at once, and what to do.
Is buying the upgrade with the best payback first really the fastest way?
Every idle player and every guide repeats the same rule of thumb for what to buy next. This entry shows how to let a tiny test bot try every order on your own upgrade list, so you learn whether the rule holds for your numbers or whether it sends players the slow way.
Is your villain hated, or does nobody care? Rating a character with a few testers
An average rating of three can mean "everybody shrugged" or "half adored and half detested", and those are opposite problems. This entry looks at how the ratings spread and names the type of character you have, so you fix the right thing.