I planned a fifty-fifty test but one group came out bigger: a bug or chance?

Before reading any result of a test, check that the groups have the size you planned. A lopsided split is the cheapest early warning that something broke, such as one version crashing and losing its players. This entry shows the simple check and where to count.

Why this works

A fair split of players is decided by chance, and chance only drifts so far from the plan. When a group is much further off than that, the cause is not luck but a fault that removes players, and every later result then compares unlike people.

When not to use it

With very few players the sizes jump around too much to tell anything, so wait for more. And if the changed version is meant to make some players leave, a gap measured after that point is a result, not a fault.

In the full entry

• What to build, step by step
• Starting values and the reason for each
• 3 common mistakes and the fix
• 13 lines of example code with 4 checks that pass
• How your AI shows you that it works

Say this to your AI

Use the Spielerando entry "I planned a fifty-fifty test but one group came out bigger: a bug or chance?" (read_entry the-two-test-groups-are-not-the-size-they-should-be) in my game. Build it step by step, and show me how you checked that it works.

Or skip this and connect the MCP: your AI then finds all of this by itself.Connect the MCP