Language-Models on The Standard Errorhttps://jongha-jeon-dev.github.io/standarderror/tags/language-models/Recent content in Language-Models on The Standard ErrorHugo -- 0.123.7en-gbWed, 14 Oct 2026 00:00:00 +0000ML 30: A Power Law Fitted on Small Models Cannot See the Floorhttps://jongha-jeon-dev.github.io/standarderror/lectures/ml-30-scaling/Tue, 13 Oct 2026 00:00:00 +0000https://jongha-jeon-dev.github.io/standarderror/lectures/ml-30-scaling/Eight character transformers, one budget of data and steps. The law fitted to the five smallest predicts the next size almost exactly and the largest too optimistically.ML 52: A Leaked Benchmark Shows in the Set, Not in the Itemhttps://jongha-jeon-dev.github.io/standarderror/lectures/ml-52-contamination/Wed, 14 Oct 2026 00:00:00 +0000https://jongha-jeon-dev.github.io/standarderror/lectures/ml-52-contamination/A model trained on a benchmark's answers scored higher on them and no higher on anything else. Its own likelihoods could not say which problems had leaked; a fresh set from the same source could say that some had.ML 53: The Prompt Is Part of the Scorehttps://jongha-jeon-dev.github.io/standarderror/lectures/ml-53-prompt-format/Wed, 14 Oct 2026 00:00:00 +0000https://jongha-jeon-dev.github.io/standarderror/lectures/ml-53-prompt-format/The same model on the same problems scored 0.735 or 0.500 depending on what came before them, and the gap between two models shrank from 0.127 to about zero. A score is the model under one way of asking.ML 54: Multiple Choice Grades the Wrong Answers You Offeredhttps://jongha-jeon-dev.github.io/standarderror/lectures/ml-54-multiple-choice/Wed, 14 Oct 2026 00:00:00 +0000https://jongha-jeon-dev.github.io/standarderror/lectures/ml-54-multiple-choice/Ranking answer options by likelihood is a different test from asking for the answer. A model that never produced a four-digit sum picked it out of four options 95.5% of the time, or 32%, depending on the options.ML 55: Every Judge Ranked Itself Firsthttps://jongha-jeon-dev.github.io/standarderror/lectures/ml-55-judge/Wed, 14 Oct 2026 00:00:00 +0000https://jongha-jeon-dev.github.io/standarderror/lectures/ml-55-judge/Three models graded each other's arithmetic by how right each answer looked to them. All three preferred their own answers on every task, and forty real labels were enough to correct them.