Story Choice Stats: Do Your Choices Really Matter?
Count what players really chose and which endings they reached, then compare that with your story as you wrote it. A choice that nearly everyone answers the same way is hardly a choice, and an ending nobody has seen may be rare or may be impossible.
What do the numbers say about a choice?
The Reed Oath is an invented story game, and everything here is made up for this example. Of its 180 sessions, every one reached two big choices. Each player counts once, with their last pick if they reloaded.
| Choice | Option | Players | Share | Very likely between |
|---|---|---|---|---|
| At the ferry | Spare the ferryman | 171 | 95% | 91% and 97% |
| Betray him | 9 | 5% | 3% and 9% | |
| At the chapel | Burn the letter | 98 | 54% | 47% and 62% |
| Keep the letter | 82 | 46% | 38% and 53% |
The chapel is a real choice. Players split close to evenly, and 23 of them reloaded to try the other side, which is a good sign that the choice weighs on them. You do not need a perfect 50-50. A 70-30 choice is fine, as long as both sides tempt someone.
The ferry is another matter. Even the bottom of the range is 91%, so this is no accident of a small sample. The writer read the scene again. Sparing is the kind thing to do, costs nothing and earns a reward. Betraying makes you a villain for no gain. So it is not a decision, only a test of whether you are nice. Meanwhile, a whole branch of writing was seen by 9 people.
The fix is not to make betrayal more visible. It is to make both options cost something. Now the ferryman knows something that could get your sister killed, so sparing him has a price too. Then count again.
What about an ending nobody reached?
Of the 180 sessions, 120 reached an ending.
| Ending | Players | Very likely between |
|---|---|---|
| Home | 74 | 53% and 70% |
| Exile | 46 | 30% and 47% |
| The Drowned Court | 0 | 0% and 3% |
Zero is not the same as never. A handy rule, the rule of three, says that when something has not happened in n tries, its true rate could still be up to about 3 in n. Here, 3 in 120 is 2.5%, which fits the 3% range closely. So up to about 1 player in 40 might reach it, and you just have not seen them yet. Or it might be impossible.
Numbers cannot tell those apart, and more players will not help. You have to walk the story. The writer ran a story check over every path, and got the answer in seconds: the ending needs a flag called has-seal, and the scene that gives you the seal sets has_seal, with an underscore. No path leads there. It has been dead since the day it was written.
This is why the story comes first and the numbers second. The story path check walks scenes and flags, the story check also tracks who knows what, and story choice stats lays the real picks over them. See also choices that matter.
Paste this to your AI
Show me how players really move through my story. The event log is in [file or folder].
1. Model the story first: facts (who is alive, who knows what, who holds what), scenes and choices as steps with what they need and change, and every ending. Draw the choices and endings as a Mermaid state diagram.
2. Walk every route through that model with TLA+ and TLC, its model checker. No TLC here? Fetch story-graph-check or story-model-check with get_recipe. Should neither be possible, tell me plainly that no check took place. Give one route to each ending, or state that none exists.
3. Use get_recipe with story-choice-stats, or count yourself, each player once with their last pick. Show each choice with counts and 95% ranges, and each ending the same way.
4. Compare the data with the model: picks that are not in the model, scenes or endings nobody reached, and which of those have a path.
5. For any choice where one option is above 80% for certain, quote the scene and say what each option costs and gives the player.
Keep separate what the checker proved, what the numbers suggest and what you are guessing. With under 30 players at a choice, give no verdict.
Or skip this and connect the MCP: your AI then finds all of this by itself.Connect the MCP
How to check it worked
- Follow the path it gave you to the missing ending. You should arrive there, or get stuck where it said.
- Search for the flag's name. Every spelling should match.
- Reload and pick again. The log should count you once.
- Read both options aloud. Can you name what each one costs?