Version B finished 8 points better than A: ship it, or wait for more players?

Comparing two versions of a level or a screen looks simple until you ask how many players you needed and whether looking early fooled you. This entry gives a straight answer on whether the difference is real, and how many players to wait for.

Why this works

Chance alone can make one group look better, and peeking often enough will always find a moment when it does. Deciding the group size in advance, and judging only at the end, keeps a lucky streak from passing for a real result.

When not to use it

Skip it when you have very few players, or when the change is so obvious that people simply complain about the old version. In the first case, watch a handful of players play; in the second, just ship the fix.

In the full entry

• What to build, step by step
• Starting values and the reason for each
• 3 common mistakes and the fix
• 32 lines of example code with 4 checks that pass
• How your AI shows you that it works

Say this to your AI

Use the Spielerando entry "Version B finished 8 points better than A: ship it, or wait for more players?" (read_entry did-version-b-really-beat-version-a) in my game. Build it step by step, and show me how you checked that it works.

Or skip this and connect the MCP: your AI then finds all of this by itself.Connect the MCP