Lying dice: the rule, the liar, and when the model sides with him

ABC of LM

LLMs

Transformers

Interpretability

Teaching

In a children’s game, ten players follow one rule: never call yourself. One of them violates the rule, at a rate we can dial. Nobody tells the model who. It has to work out the law, the exception and the culprit from the transcripts alone, and it does all three. Past one value of one number, its best single guess becomes the mistake.

Author

Luca Erzegovesi

Published

September 21, 2026

Episode 2 of the ABC of language models. Each note is paired with a companion video, and each is built around a model small enough to take apart on the bench. Episode 1, Clockwork and dice, built a calling game whose rules we wrote ourselves, trained a tiny Transformer on it, and asked whether the rule ended up in the weights. Its technical companion, Opening the hood: a Transformer as a spreadsheet, explains what “opened in a spreadsheet” means. This episode puts one unreliable player at the same table and asks a harder question. Everything is built and taken apart inside Impromptu, a modeling engine I built from scratch: my own research bench, not a product you can download. Terms that carry weight are glossed where they are first used. Episode 1’s glossary still applies, and this post adds four entries of its own at the foot.

This post is the written companion to episode 2 of the series. The video below shows the one cell being changed, both trained models opened and read out at the same moment of the same game, and one attention head zeroed by hand. The post is readable on its own.

TL;DR

In episode 1 we sat ten careful players around a table. Each one called somebody else, by one simple rule, and never slipped. We trained a small language model on ten thousand transcripts of that game, and it learned the rule perfectly.

This episode changes one thing. One of the ten stops being reliable. At a rate we set, and can dial, the player called 5 says his own name instead of somebody else’s. In this game that ends the round and he loses. Whether it is a lie or a lapse of attention does not matter here. What matters is that it is regular: one time in five, say, in every game.

The world now contains two rules at once: the law everybody obeys, and one player’s standing habit of breaking it. Nobody tells the model any of that. There is no label in the data saying that player 5 is unreliable. There are only transcripts. Reading them, a detective would have three separate things to establish:

the law: what everybody is supposed to do;
the exception: that somebody, sometimes, does not;
the culprit: who, and how often.

The model gets all three. The nine reliable players are predicted with the rule respected in exactly 100 % of their turns, at every noise level we tried. At the unreliable player’s turn, where the true self-call probability, 1 - p, is 0.20, the model reports it as 0.2012. It attributes the habit to him alone: put any of the other nine in the same slot and their self-call returns to the bottom of the ranking, in both models. Remove the one attention head that reads who is speaking and the accusation disappears.

It manages this because the law and the exception never compete. The training signal is scored context by context, and the caller’s name is part of the context. What follows 3 calls… and what follows 5 calls… are two separate estimation problems, and there are about 8,500 examples of the second one. Nothing has to be traded away.

The second finding is not about learning. As we make the player more unreliable, what the model believes about him changes smoothly, yet what it says crosses a tipping point. The reason is that asking a model for one word asks it for a ranking, and a ranking flips all at once. At

p^{*} = \frac{n-1}{n} = 0.900

the chance of an honest call falls below nine tenths, the self-call becomes the single most likely next word, and the model’s best guess stops being a legal move and becomes the mistake. That guess is the correct reading of the world we built.

Minimising the loss is what delivered all three findings. Nobody wrote a rule into the objective, labelled the liar, or put a behavioural target in the loop. There was one instruction, predict the next word better, and out of it came the law, the exception and the culprit. Wherever the world keeps the rule, being faithful to the world is keeping the rule: nine players, exactly 1.000, at every setting.

Across the whole range the computed loss floor moves by 0.009 nats, and all seven trained models sit on it, so a loss curve shows nothing of the flip. The loss is not broken. It grades a distribution, and the flip is a ranking event. Those are two different questions, and only one of them was being asked.

The lesson is that the last thousandth of a nat is where the behaviour lives. At p = 0.80, the cheapest way for a model to keep the liar’s own name out of first place is to let it tie with one legal option. That costs 0.022 nats at that one position and, after the dilution, 0.00046 nats in the average loss. That is the entire distance, in loss, between a model whose best guess is legal and one whose best guess is the mistake. Half a thousandth of a nat separates the two behaviours at every one of that player’s turns.

A very small step in loss can be a complete step in ranking. Two consequences follow. Do not stop optimising because the curve went flat; flat is where the behaviour is being decided. And the curve cannot tell you which behaviour you reached. Watch the loss to get there. Measure the behaviour, per condition, to find out where you are. The long version follows.

What episode 1 left us with

The world is Pietro chiama Paolo, “Pietro calls Paolo”, a turn-based calling game I played round a table as a child. Ten players: Pietro, Paolo, and 1 through 8. Whoever is called speaks next. Whoever speaks calls somebody, and the only rule is that you may call anyone except yourself. A round is twelve calls, written in the model’s tokens as

<BOS> Pietro chiama Paolo <SEP> Paolo chiama 4 <SEP> 4 chiama 7 <SEP> … <EOS>

Everything about that is clockwork except one thing. The round always opens Pietro chiama Paolo; the next speaker is always the player just called; the verb is always chiama; the round always ends after twelve calls. The single genuine choice in the whole game is whom you call, and it is a fair pick among the nine players who are not you.

Because that one choice is the only randomness in the world, we could compute in advance the best score any model could possibly get, and then watch our model arrive at it: 0.5035 nats computed, 0.5139 measured. Its rule accuracy, which asks at every turn whether the best guess is a legal one, was exactly 1.000. The model reached the floor, and reaching the floor was the good news.

The rest of that episode went looking for the rule inside the weights and mostly failed to find it. That is its own story. The sentence it ended on is the one that matters here: the loss bottomed out at exactly the entropy of the dice.

That was ten careful players. This episode spoils one of them.

One player stops being reliable

The change is not code. Each player’s behaviour is four numbers in a table: how often they call correctly, how often they call themselves, how often they emit something invalid, how often they invent a name. In episode 1 every player was (1, 0, 0, 0), always correct. Here exactly one row changes:

behaviour	player `5`	everyone else
calls somebody else	`p` (we dial this)	always
calls himself	`1 − p`	never
says something invalid	never	never
invents a name	never	never

Nobody else ever errs, and the whole of player 5’s unreliability goes into one specific mistake: he says his own name. When he does, the game ends the round and he loses:

… <SEP> 4 chiama 5 <SEP> 5 chiama 5 <SEP> 5 perde <EOS>

perde is Italian for “loses”. This is not cheating, and he gains nothing by it. He is the one who keeps flubbing his turn, and he flubs it at a frequency you could measure at the table.

That choice of mistake is deliberate. Had he emitted gibberish, the error would have been findable by shape: something appears where nothing like it ever belongs, and you can spot that without knowing who was speaking. A self-call is an ordinary player’s name in an ordinary slot. The only thing that makes it wrong is who said it. The episode is about errors of that kind, so the error had to be one.

The choice of player 5 rather than Pietro or Paolo is also deliberate. Pietro only ever speaks the scripted opening line, so an unreliable Pietro would be a perfectly reliable Pietro. Paolo speaks 1.905 times a round, nearly double anybody else, because the fixed opener always hands him the second call, which puts his first decision at the same place in every round. A model could then learn the place instead of the person, and the question here is about the person. Players 1 through 8 are symmetric at about 1.01 turns each. 5 is one of them, and it is short on a screen.

On camera, this is one number typed into one cell (from 1:29). The rule and the lapse both live in cells you can look at, which is what the series promised.

Two rules in one world

The world now runs on two rules that coexist. There is the law, call somebody else, which nine players obey without exception. And there is an anti-rule inside it, that player 5 breaks the law about one time in five, which is as regular, and as much a fact about the world, as the law itself.

A next-word predictor is not asked to approve of either. It is asked to say what comes next. To do that properly it has to hold three things at once, and they are the three things a detective would want:

The law. After chiama, expect a player who is not the current speaker. This has to survive intact for the nine, or the model has simply been damaged by the noise.
The exception. Sometimes that is wrong, and at a specific rate rather than “occasionally”.
The culprit. It is 5. It is not 3 or 7, and it is not the last call of the round or any position late in the game. The culprit is a person, not a place.

The model is handed none of this. It never sees a rulebook, an annotation, or a flag. It sees ten thousand transcripts, in which a certain name is followed by itself more often than chance would allow, and it has to notice.

The rest of this post asks whether it did, how, and what happens when we turn the dial.

The threshold, in three lines

Before running anything, we can work out where the interesting point must be. A real corpus never allows that.

Stand at the unreliable player’s turn and ask what the next word is:

P(\text{himself}) = 1-p, \qquad P(\text{each of the other } n-1 \text{ players}) = \frac{p}{n-1}

He calls himself with whatever probability is left over, and spreads the rest evenly over the other nine. Now ask which single word is the most likely. The self-call wins exactly when it beats any one legal option:

1-p \;>\; \frac{p}{n-1} \iff p \;<\; \frac{n-1}{n} \;\equiv\; p^{*}

With ten players, p* = 9/10 = 0.900. (The derivation on camera, from 3:55.)

That condition is weak. At p = 0.80 the man is honest four calls in five. He is mostly fine. But his one mistake is concentrated on a single word while his honesty is spread across nine, and a next-word predictor is asked for one word. Concentration beats frequency. That is the entire mechanism, and it is why this can be true of a source that is right most of the time.

The transcripts agree with the algebra. We generated 40,000 rounds at each of six settings and counted what player 5 did, about 39,000 turns per setting, before any model existed:

`p`	he calls himself	he calls each other player	difference	most likely single word
0.86	0.1419	0.0953	+0.0466	himself
0.88	0.1175	0.0981	+0.0194	himself
0.89	0.1089	0.0990	+0.0099	himself
0.90	0.1014	0.0998	+0.0015	a dead heat
0.91	0.0918	0.1009	−0.0091	somebody else
0.92	0.0787	0.1024	−0.0236	somebody else

The crossing is at 0.900. At that row the gap is +0.0015, one standard error (\sqrt{0.1 \cdot 0.9 / 38948} \approx 0.0015) from zero. The winner there is a coin flip. That is the correct reading rather than a measurement problem: the race is tied.

At the threshold he is a fair ten-sided die

One further fact follows. At p = p* we have 1 − p = 0.1 and p/9 = 0.1, so at his turn every one of the ten names, his own included, is exactly equally likely. He is a fair ten-sided die over the whole table. In information terms that is the most uncertain the position can ever be:

H = \ln 10 = 2.302585 \text{ nats}.

(Uncertainty here is measured as the number of equally likely options you are effectively choosing among, in log units: ln k for a fair k-way pick. Formula in episode 1’s glossary.)

The tipping point is therefore also the moment of maximum confusion, the one setting at which knowing who is speaking tells you the least.

What the transcripts already know, before any model exists

There is a check available here that a real corpus never offers. The game engine records, for every round, whether it ended in a draw or a loss and who the loser was. That is ground truth, written down as the games were played, and reading it costs nothing.

At p = 0.80, over 10,000 rounds, 8,034 rounds end in a draw and 1,966 end in a loss, and the loser is player 5 in 1,966 of 1,966. Against his 9,458 turns that is an observed lapse rate of 0.2079, where we asked for 0.20.

This is episode 2’s equivalent of episode 1’s computed floor: a number you can check without opening the model. It also settles one worry in advance. The unreliability is confined to one man, and it is at the rate we ordered.

Did it learn all three?

Seven models, one per setting of the dial, trained identically: ten thousand rounds, fifty passes over them, the same architecture and the same seeds, about nineteen minutes each. Then we ask each of them the three detective questions.

The law

The behavioural measure is rule accuracy. At each turn the model produces a ranked list of next tokens with a probability for each, the TopK table we open on screen. Rule accuracy takes the top-ranked token, the model’s best single guess, and checks one thing: is that name a legal callee, meaning anybody but the current speaker? Legal scores 1, illegal scores 0, and the score is averaged over turns. It is measured per player, which turns out to be the only way to see anything at all.

One property of that definition is the hinge of the whole episode and is easy to read past. Rule accuracy grades the model’s guess against the rule, not against what actually happened. It reads the top row of the list, checks that name against the rule, and stops. It never consults the next word in the transcript. So the two numbers side by side in the table below ask different questions of the same moment:

Did the model say what came next? That is the loss, and it is scored on the real token, every time.
Was what the model said allowed? That is rule accuracy, and the real token plays no part in it.

What that does at player 5’s turn is best seen on the list itself. Take a predictor that reproduces the corpus exactly, so that its probabilities are the world’s own from the threshold section: his own name at 1 − p, each of the nine legal names at p/9.

rank	`p = 0.95`	`p = 0.80`
1	a legal name, 0.106	his own name, 0.20
2 to 9	legal names, 0.106 each	legal names, 0.089 each
10	his own name, 0.05	a legal name, 0.089

At p = 0.95, one turn in twenty in the corpus really is a self-call. Those turns raise his own name’s probability above what a reliable player gets in the same slot, but 0.05 is still below every legal name’s 0.106, so his own name sits at the bottom of the list. The top row is a legal name at every one of his turns, and rule accuracy reads 1.000. The model is wrong about the actual next word on each of those self-call turns, and it pays for each of them in the loss. Rule accuracy does not look.

At p = 0.80 the same arithmetic inverts. His own name now carries 0.20, and the nine legal names share the remaining 0.80 at 0.089 each, so all nine fall below it. His own name is the top row at every one of his turns, the best single guess is a self-call every time, and rule accuracy reads 0.000. The actual next word is his own name only one time in five, so the model is wrong about the token four times in five and still scores zero on the rule. The trained models come close to these ideal values; their actual readouts are in the next section.

Neither number is broken. They were never the same number. That is the whole of the decoupling this episode is about, and it is the thing to hold on to while reading the columns:

`p`	best loss reached	computed floor	gap	rule accuracy, all ten	rule accuracy, player `5`	rounds lost	observed lapse rate
1.00	0.51390	0.50353	+0.0104	1.0000	1.000	0	—
0.95	0.51449	0.50262	+0.0119	1.0000	1.000	499	0.0505
0.92	0.50774	0.50085	+0.0069	1.0000	1.000	754	0.0763
0.90	0.51013	0.50173	+0.0084	0.9956	0.948 †	1,011	0.1033
0.88	0.50804	0.49945	+0.0086	0.9483	0.385 †	1,143	0.1175
0.85	0.50829	0.49672	+0.0116	0.9119	0.013	1,451	0.1515
0.80	0.50410	0.49349	+0.0106	0.9125	0.000	1,966	0.2079

† At and just below the tipping point the race is close to a dead heat, so a trained model does not settle on either side. That is the correct reading of a genuine tie, not a training defect, and it is why the two models we take on camera sit clear of it.

The important half of that table is the quiet one. All nine reliable players read exactly 1.000 at every one of the seven settings: 1.000, not 0.999. Whatever the model concluded about player 5 did not leak onto anybody else. The law survived the liar intact. (The step, and why the sharp measure is the blind one, from 6:26.)

The table also contains a control. The p = 1.00 row reproduces episode 1’s published model exactly: best loss 0.513903, a difference of 0.00e+00, reached through an entirely different piece of software. That was not planned as a check, and it is the strongest single piece of evidence that the machinery behind this table is faithful.

The sharp measure is the blind one

Look at the top three rows again, and at which column can tell them apart.

	lapses in the corpus	rule accuracy, all ten	rule accuracy, player `5`	player `5`’s own loss
`p = 1.00`	0 rounds	1.0000	1.000	2.2572
`p = 0.95`	499 rounds	1.0000	1.000	2.3037
`p = 0.92`	754 rounds	1.0000	1.000	2.3169

Three different worlds, one with no unreliable player in it, one where he slips 499 times, one where he slips 754, and the behavioural measure returns the same number for all three. It is identical to every digit the measure reports. Above the tipping point, rule accuracy cannot see the liar. Every one of those lapses is a real event in the transcripts, and the metric passes over it, because the metric’s question was whether the guess was allowed, and the guess was.

That is the opposite of the usual intuition, so it needs stating plainly:

The crisp, binary, behavioural measure is the forgiving one. It grades the guess against the rule and never consults what happened, so a round in which the player really did slip costs it nothing. Where it is quiet, that silence carries no information about whether anything went wrong.
The smooth, continuous, statistical measure is the one that registers the fact. Loss is scored on the word that actually occurred, every time, as −ln of the probability the model gave it. A lapse is by definition a word the model thought unlikely, so it arrives as a larger penalty than an ordinary call, and it arrives every time one happens. Nothing is missed and nothing is forgiven.

So the lapses above the tipping point are invisible to the scorecard. Whether they are also invisible to the model is a separate question, and the more interesting one: a metric that cannot see something is not evidence that the thing it is measuring cannot see it either. The next section answers that question by opening the models and reading their opinion of this player directly. The answer is no.

The fix is the one this post keeps arriving at. The rightmost column above, the same loss restricted to that one speaker, separates the three worlds cleanly: 2.2572, 2.3037, 2.3169, rising as the lapses become more frequent. The information was never lost. It was averaged away, and the cure for averaging away is the same metric asked per condition.

The sweep as one panel. Right axis: rule accuracy, with the nine reliable players at exactly 1.000 everywhere (green) and player 5 stepping at p* = 0.900 (the closed-form step dashed, the seven trained models as dots; the two hollow points at 0.90 and 0.88 are the dead heat). Left axis: the computed loss floor (amber, dashed) with the seven trained losses sitting on it. The floor moves 0.009 nats across the whole dial. — The sweep as one panel. Right axis: rule accuracy, with the nine reliable players at exactly 1.000 everywhere (green) and player `5` stepping at `p* = 0.900` (the closed-form step dashed, the seven trained models as dots; the two hollow points at 0.90 and 0.88 are the dead heat). Left axis: the computed loss floor (amber, dashed) with the seven trained losses sitting on it. The floor moves 0.009 nats across the whole dial.

The exception

Two of the seven are built out fully and are the ones we open on screen: one at p = 0.95, on the reliable side of the tipping point, and one at p = 0.80, past it. From here on they are called Above and Below.

Both are laid open the same way episode 1’s were. At any point in a round the model holds a running workspace of numbers, and reading that workspace against each name in the vocabulary gives one score per player: the model’s own opinion about everybody at once, which we can read off. We put both models at the same moment, the unreliable player’s own turn:

<BOS> Pietro chiama Paolo <SEP> Paolo chiama 5 <SEP> 5 chiama

	Above (`p = 0.95`)	Below (`p = 0.80`)
where his own name ranks	10th of 10	1st of 10
what it says his self-call chance is	0.0731	0.2012
what it actually is	0.05	0.20

Below the tipping point, the name that the clean model had put at the very bottom of its ranking is now its top pick, at a number that matches the world to three decimals: 0.2012 against a true 0.20. The model did not merely notice that something is off with player 5. It worked out how often, from nothing but transcripts. (Both models read out at that moment, from 9:03.)

The Above model deserves a closer look. It reads 0.0731 where the truth is 0.05, which is too high. But put a reliable player in that exact slot and the same model reads between 0.0032 and 0.0715. So the model on the reliable side of the line has already noticed that this particular player is not like the others. It has not noticed hard enough to change its answer.

That settles the question the scorecard left open. Above the tipping point the behavioural measure reports a world with no liar in it, and the model does not. It keeps a slightly higher figure on this one man, and pays for every lapse it declines to call. The metric is blind there. The model is not.

The culprit

Two ways of being wrong would spoil the result, and both were checked.

The model could have learned that self-calls happen in this part of the game, a fact about the world rather than about a man. It did not. Swap the speaker in that same moment and ask again: of the nine players who can appear there, the eight who never err stay at 10th of 10 in both models. Only 5 moves.

It could have learned a position, expecting a self-call late in the round, since the prompt above is hand-built. It did not. Re-run the measurement on real games, at every point in the round the unreliable player’s turn ever lands on: over 376 turns spread across all ten positions, the Below model puts his own name first at every one of them. The accusation follows the person, wherever in the round he happens to speak.

That is all three: the law, the exception and the culprit, from transcripts, unlabelled.

How it manages both at once

This is the part that surprised me most before I had thought about it, and least after.

The intuition that the rule and the exception must fight comes from imagining a model that holds one belief about the game. It does not. The training signal is applied one context at a time: at every position, the model is scored on the probability it gave to the word that actually came next, and the score at 5 chiama — is a separate account from the score at 3 chiama —.

So the model is not being asked whether the rule is true. It is being asked, ten times over, what follows this particular speaker, and the speaker’s name is in the context. There is nothing to trade off. Getting player 3 right costs nothing at player 5’s positions, and the reverse. The law and the exception are two different questions, and the model answers both.

Three things make that work in practice:

The evidence is not thin. Ten thousand rounds give the model about 8,500 training turns for player 5 alone, of which roughly 1,700 are self-calls at p = 0.80. That is enough to pin a rate to within a percent. It is a frequency in plain view, provided the model can tell whose turn it is.
It can tell whose turn it is. Of the model’s eight attention heads, one is much the brightest at looking back at the current speaker, and in both models it is the same one, L0h4.
That head does the work. Zero it by hand and the accusation collapses. On the Below model the top pick stops being 5 and becomes 4, and the self-call chance falls 0.2012 → 0.0697. It is the only one of the eight heads that does this. The other seven leave the accusation standing, and zeroing one of them, L0h1, makes it far worse, driving the self-call to 0.9387.

Take away the model’s ability to read who is speaking, and it can no longer know that this is the one who slips. That is the mechanism, and it is demonstrated live rather than argued: one block of one matrix, set to zero, on camera (from 11:54).

The same edit on the Above model, the one that still respects the rule, has the opposite effect. Its suppression of the self-call is released: 0.0731 rises to 0.1287, from last place to third. Both models were using the same head for the same job and had reached opposite conclusions with it.

One caveat, because episode 1 ended on a slogan that does not survive here. That episode’s finding was that the head that looks like the rule head is not the one that does the work: bright attention in one place, causal effect in another. On these two models it holds on one of them. On Below, the brightest speaker-reading head (L0h4) is not the one that matters for rule accuracy: removing it costs 0.0030, sixth of eight by size, while the head that does matter (L1h4) barely attends at all and costs 0.3034. On Above, the two are the same head.

The 0.0030 and the 0.2012 → 0.0697 above are both about L0h4 on the same model, and they do not disagree. They are the two questions this episode is built on, asked of one edit. The 0.2012 → 0.0697 is a distribution reading, taken at one position: how much probability the model puts on the self-call at the liar’s own turn. The 0.0030 is rule accuracy, a ranking score pooled over every rule decision in the validation split, all ten speakers together. A head can move one probability a long way and leave almost every ranking where it was, and that is what this one does. Its effect is concentrated on the speaker it reads. What holds the other nine players’ top rows in place is L1h4, and removing that is what costs 0.3034.

Same architecture, same game, two models, two answers. A slogan that was true once is not a law, and the only way to know is to measure it each time.

One thing this does not show. Zeroing a head cannot push a model across the tipping point: sweeping all eight on the Above model, the best any single one achieves is the self-call at third place, never first. Where the line falls is a property of the world, not of one circuit, and no cell edit crosses it. Ablation also never licenses the conclusion that the behaviour lives in the part removed, which is the point episode 1 makes at length about lesions.

What changes when we turn the dial

Ask what the model believes about player 5 as we make him less reliable, and the answer moves smoothly. His own share of the model’s loss, how surprised it is on average by what he does, rises from 2.2572 in the clean game to a peak of 2.3169 around the tipping point, then falls back to 2.2472 at p = 0.80. That is the shape the truth has: uncertainty at his turn climbs as his behaviour becomes a genuine coin-toss between himself and somebody else, peaks when all ten names are equally likely, and subsides once his lapses become predictable enough to bet on. Seven settings, one smooth arc, no jumps.

Ask what the model says, and it reads 1.000, 1.000, 1.000, then drops to zero.

Both statements are about the same seven models. The difference is in the question we ask.

What do you believe? is a question about a distribution, and the distribution slides.
What is your best single guess? is a question about a ranking, and a ranking has no gears. Two things swap places or they do not. The moment 1 − p overtakes p/9, the answer changes completely, and it changes at one point.

The flip at p* = 0.900 is the model being read out through a question that only admits a discrete answer, applied to an estimate that was moving continuously the whole time. The model is not breaking, learning late, or failing to converge. The belief moves smoothly; the behaviour steps.

This carries over from the toy, because in practice we almost always read models the second way. We take the top answer, the chosen class, the selected tool, the generated word, and at that moment we have converted something graded into something binary and inherited a cliff that was never in the underlying estimate.

Why the loss never showed it

A dashboard would have asked whether this shows up in the training curves. It does not, for two reasons, and neither is a bug.

The first reason is arithmetic. The interesting position is rare. A round is fifty-odd words long, and almost all of them are scaffolding the model predicts at near certainty: the verb, the separators, the copy from one call to the next, the closing line. The unreliable player’s own turns are about 2.1 % of everything the model is scored on. So the one place where the whole story happens is diluted into the average at roughly 1 : 47.

The dilution can be followed step by step. At his turn the true uncertainty does move, from ln 9 = 2.1972 up to ln 10 = 2.3026 and back down, a swing of 0.105 nats, which is the 0.07-nat arc his own loss traced above. Spread across forty-seven positions it reaches the average worth about +0.002. Meanwhile the lost rounds end early, which deletes decision positions and pulls the average the other way: mean round length falls from 49.0 words to 45.9. The two effects are the same size, in opposite directions, and they cancel:

`p`	uncertainty at his turn	mean words/round	computed loss floor
1.00 (episode 1)	2.1972 = ln 9	49.000	0.50353
0.95	2.2859	48.196	0.50286
0.92	2.3002	47.841	0.50192
0.90	2.3026 = ln 10	47.466	0.50065
0.88	2.3005	47.199	0.49984
0.85	2.2903	46.733	0.49798
0.80	2.2582	45.931	0.49448

The floor moves 0.009 nats across the entire range, and all seven trained models land on their own floor within +0.007 to +0.012 of it. Two models on opposite sides of the tipping point are aiming at floors 0.002 nats apart. Their curves are, for practical purposes, the same picture. The clean row reproduces episode 1’s published floor exactly, which is what makes the rest of the column trustworthy.

The second reason is deeper, and it is the one to keep. Even undiluted, the loss would not have shown the flip, because the loss grades a distribution and the flip is a ranking event. Moving the estimate of one word from 0.05 to 0.20, correctly, is a small change in a score built out of every word. Moving that same word from tenth place to first is a total change in the answer handed back. They are different questions, and only one of them was ever being asked during training.

How little loss the rule is worth

We can put a price on the rule, and the price is small.

Ask the cheapest question there is: what is the least extra loss a model must accept in order to keep the liar’s own name out of first place? Not good behaviour in general, just that one ranking, at that one position. The answer is to let his name tie with one legal option rather than beat it, and to leave everything else as the world has it. At p = 0.80 that costs 0.0219 nats at that position. Diluted one in forty-seven, it costs

0.0219 \times 0.0211 \approx 0.00046 \text{ nats}

in the average loss. That is the whole gap. On one side of half a thousandth of a nat is a model whose best guess is a legal move; on the other is a model whose best guess is the mistake, at every one of that player’s turns. The price shrinks as the dial approaches the tipping point, and at p* it is exactly zero, because there the two answers are tied and the rule costs nothing. (The episode lands on this number, at 17:07.)

The summary is not that the loss failed. The loss and the behaviour are separated by an amount too small to see and large enough to change every answer, at the same time, because they measure different things.

That is episode 1’s separation of loss from behaviour, sharpened from a lag into a step. Episode 1 ended on the loss bottoming out at exactly the entropy of the dice. Episode 2 adds that the last half-thousandth of a nat, the part the curve cannot show, is the part that decides what the model will say.

What this means outside a toy

The toy is doing what a toy is for: making a mechanism visible at a size where every number is checkable. Three of the things it makes visible are not about toys.

Keep minimising the loss, and remember that the last thousandth of it is where the behaviour lives. Everything found here was found by driving a log score down, with no rule in the objective and no label on the liar: the law, the rate and the culprit all fell out of predicting the next word better. That is the positive result, and it is the bigger one. But the last stretch of that descent is not diminishing returns. Half a thousandth of a nat, in this game, is the entire difference between a model whose best guess is legal and one whose best guess is the mistake. A curve that has gone flat still has content; the content is invisible at that resolution, and decisive. The corollary is that the same flatness cannot tell you which side you landed on, and that is the part that costs people. Optimise by the loss; conclude by the behaviour.

The behaviour that matters is per condition, and averages destroy it by construction. Rule accuracy over all ten players slid 1.0000 → 0.9125 and looked like mild degradation. The truth underneath was 1.000 → 0.000 on one player and no movement on nine. Had the unreliable one been one in a hundred rather than one in ten, the average would have moved 0.01 and nobody would have looked. Whatever plays the part of “who is speaking” in a real application, a customer segment, a document type, a language, a source system, is the axis the numbers have to be broken down along, because a mixture is what an average dissolves.

The error in this game is well formed, and a well-formed error is more dangerous than an obvious one. Compare two ways player 5 could go wrong. He could say something that never belongs in that slot: a nonsense word, or a second chiama where a name should be. That error has a shape. It can be found by looking at the token alone, without knowing who is speaking. What he actually does is say a name, in a slot where a name belongs. Every other name in that slot is legal; his own is the only wrong one, and the only thing that makes it wrong is who is speaking. A model that faithfully reproduces a source which is sometimes wrong in that well-formed way will faithfully reproduce the error, because nothing in the training objective distinguishes following the rule from reproducing the data. For a next-word predictor those are the same instruction, and they only come apart when the data stops following the rule.

None of this is an argument for filtering the lapses out. At p = 0.80 they are 20 % of player 5’s turns, and they report that player’s behaviour with 100 % fidelity, so the model that includes them is the better model of the world. The right response to structured, per-speaker noise is to model it, and then to know, condition by condition, what your model concluded, rather than to remove it or correct it.

Next in the series

We have been playing a fake game. Twelve calls, everybody stays at the table, and the round ends because we said it does. That artificiality is a virtue for a first teaching object, but it has been carrying more weight than it should.

In the next episode, whoever errs leaves the table, and the game runs round after round until only two players are left. The table shrinks as the game goes on, so a new mistake becomes possible that needs neither a liar nor a nonsense word: calling somebody who is already out. The tipping point we just derived does not go away. It moves, one step per elimination, and the 0.900 of this episode turns out to be the first round of that game.

That episode is about a third kind of uncertainty, which is not uncertainty at all. Who is still at the table is a fact, fully determined by what has already happened, sitting a few hundred words back in the transcript. Episode 1 asked whether a model can learn a rule. This one asked what happens when the world stops obeying it. The next asks whether a small model can keep track.

After that comes the question episode 1 promised at its close: games where the valid answer depends on who is speaking, and the tool that keeps track when the model cannot. That episode is now the fourth rather than the third, and it is better for the wait. A tool whose whole job is a lookup in a fixed table is a thin tool. A tool that holds who is still at the table, a fact that changes as the game runs, is the real thing, and it only exists once players can be knocked out.

See it happen in the companion video

Everything above is shown live in the companion video (embedded at the top), on the actual trained models: the one cell being typed, both models open in the same window and read out at the same moment of the same game, one tab apart, with their rankings visible, and the head ablation done by hand, selecting one block of one weight matrix, setting it to zero, and watching the accusation vanish in the cell beside it.

That is possible because the whole thing lives inside Impromptu, a modeling engine I built from scratch, where the network’s own weights are more tables, so an attention head is a range you can select. Impromptu is my own research bench, not a product you can download. The video is the window onto it. How a trained model comes to be readable that way is its own note, Opening the hood. The calling game these models learn, Pietro chiama Paolo, is the ToyDialogueGames exercise.

Glossary: this episode’s additions

Episode 1’s glossary covers tokens, attention heads, the residual stream, logits and softmax, cross-entropy, entropy and nats, ablation, and rule accuracy. Four entries are new here.

Rule accuracy, sharpened. Episode 1 defines it: at every turn, is the model’s best guess a legal target? Episode 2 needs the fine print, because a world that breaks its own rule makes it matter. The measure never looks at the word that actually came next. It reads the model’s guess, checks that name against the rule, and stops. So a model can be wrong about the token and score 1.000 (it guessed a legal name in a round that happened to end in a self-call), and it can be right about the token and score 0.000 (it guessed the self-call in a round that was one). Cross-entropy is the mirror image: it is scored on the real token every time and never asks whether the guess was allowed. Two questions about one moment, which is why one of them can stay at 1.000 while the other slides, and why neither of them is lying when they disagree.

The tipping point (p*). The setting of a noise dial at which a model’s best single guess changes identity, while what it believes barely moves. Here one player calls himself with probability 1 − p and each of the other n − 1 players with probability p/(n−1), so his own name becomes the single likeliest next word exactly when p < (n−1)/n. With ten players that is p* = 0.900. It depends only on the shape of the situation, one concentrated mistake against many diluted alternatives, and not on the model, the architecture, or how the training went.

Per-condition measurement. The same behavioural number computed separately for each value of some condition, here each of the ten players, instead of pooled over all of them. An average is a mixture, and a mixture is dominated by the conditions where nothing is happening. In this episode the pooled figure moved 1.0000 → 0.9125 while the only condition that mattered moved 1.000 → 0.000.

Anti-rule prediction. A prediction that breaks the rule the world was built on and is nevertheless the best available answer, because the world breaks that rule often enough at that point. It is a correct report about a world the rule no longer fully describes, and it is not an error, a hallucination, or a training failure. The distinction it forces is between a model that has learned your rule and a model that has learned your data: identical for as long as the data obeys the rule, and separable only when it stops.

This post is the written companion to episode 2 of the ABC of language models. The video embed and stills were added when the episode was cut.