Bo

Cheap way to feel all of this rather than read it: take four or five numbers, divide them by T, exponentiate, and normalise, for T = 0.2, 1 and 5. Twenty seconds in any spreadsheet.

At 0.2 you get something that rounds to a one and a lot of zeroes. At 1 you get the shape the model actually produced. At 5 you get something close to a flat distribution, which is the real endpoint — infinite temperature means every token equally likely, which is to say, no model at all.

Seeing it once fixes the idea better than any explanation, mine included.

The intuition that settles it for most people: the guess does not have to be random at all.

Suppose the player always says 17, every single time, forever. Now there is no uncertainty in the guess whatsoever. He is right whenever the die shows 17, which is 1 in 20 rolls.

Making his guess random cannot make him worse at this, and by symmetry it does not make him better either — whatever number he lands on, the die still has a 1 in 20 chance of agreeing. The guess and the roll are independent, so the die does not know or care what was said.

Once you see that a fixed guess gives 1/20, the 1/400 answer becomes clearly wrong, because it would say a player who always guesses 17 is somehow less likely to be right than 1 in 20.

Question 3, what you lose: adding is now the hard operation.

Multiplying is easy in log space — you add. But if you need the probability of A or B, you need to add the probabilities, and there is no clean way to add in logs. You end up with the log-sum-exp trick: pull out the largest term, exponentiate the differences (all of which are now safely small), sum, take the log, add the largest term back.

Every library has this built in and you should use theirs rather than writing it, because the whole point is the reordering that keeps the exponentials in range. Writing log(sum(exp(x))) literally reintroduces the exact overflow you switched to logs to avoid, and it will work fine on your test data and fail on the real thing.

Your instinct is right and the effect is enormous. Look at the longest run.

In 200 genuine flips the longest run of one face is typically 7 or 8, and a run of 6 is almost certain to appear somewhere. People inventing sequences essentially never write a run that long — they get uncomfortable at 4 and it feels wrong by 5, because a long run does not look random to a human even though it is exactly what randomness produces.

So the single most discriminating statistic is: what is the longest run in each sequence? If one has a maximum run of 4 and the other has a run of 8, you are done, and you can say so with high confidence.

The closely related giveaway is the alternation rate — the fraction of adjacent pairs that differ. It should be about 0.5. Fabricated sequences typically land around 0.6, because switching feels like what randomness does.

One thing that is not a real objection, since it comes up whenever this is discussed: correlated coefficients.

Raw powers of x are massively collinear, so the individual coefficients are unstable and un-interpretable — but if you only want predictions that does not matter, and using an orthogonal polynomial basis fixes the numerical side anyway. It is a genuine problem for reading the coefficients and a non-problem for fitted values.

Worth separating from the wobble, which is a real problem for fitted values.

Drawing the vertical residual segments was the fix. Under that view the shallow line is obviously the balanced one and the principal axis leaves systematic error at both ends.

Practical note that follows from all this: never compare log-likelihoods across datasets of different sizes, and be careful comparing across different transformations of the outcome.

More observations means a more negative log-likelihood automatically, so "model A on last year's data scored -4200 and model B on this year's scored -5100" is not a comparison of anything. The information criteria people quote — AIC and friends — exist partly to make comparisons legitimate within one dataset, and they still do not license comparisons across datasets.

If you take one habit from the thread: a lone likelihood number is not evidence. A difference between two likelihoods on the same data is.