Compare two versions

You changed something and a number moved, and your AI works out whether that is a real difference or could just be luck, and how many players it would take to find out. It works on counts such as wins and on numbers such as seconds, and it will not call a gap real on too few players.

When you need it

After you change a level, a boss or a tutorial, when you want to know whether it helped. Decide how many players you want before you look, and give each new player a version at random.

Say this to your AI

Use compare-two-versions from Spielerando. Before any data, write down what I changed, what should happen because of it, how many players each version needs, and draw it as a Mermaid state diagram from new player to result. Then give each new session a version at random, and check the split. When the number is reached, show me both counts, the range for the gap and how often luck alone would give it. If it could be luck, say so, and do not call it a win.

Or skip this and connect the MCP: your AI then finds all of this by itself.Connect the MCP

Settings

• level (0.05): The chance of luck, below which a gap is called real.
• power (0.8): How likely you want to spot a real gap, used when working out how many players you need.
• rounds (10000): How many shuffles the comparison of numbers does.
• minPlayers (30): Under this many players in a version, the report says too few.

Your AI gets the code

The finished code and math come through the Spielerando connection, so your AI can use them right away. Connect your AI

Common questions