ML 1: Your Test Score Has an Error Bar, and It Is Wider Than Your Gain

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. On a 20% test split of the UCI digits, 360 images, logistic regression, an RBF SVM and 3-NN score 0.967, 0.986 and 0.986, and one test set’s 95% binomial interval is about +-0....

October 8, 2026 · 9 min · Jongha Jeon

Coverage 1: 90% Coverage Is a Promise About Averages, Not About You

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Split conformal at alpha = 0.1 on 24,576 held-out predictions split in half delivers 0.8961 coverage against a finite-sample guarantee of 0.9001, and over 200 calibration splits a mean of 0.8999. Exactly what was promised....

September 11, 2026 · 14 min · Jongha Jeon

Headline Statistics 1: Korea's Fertility Rebound Needs Nobody to Have More Children

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Give every cohort the same completed fertility and let each one postpone by d years relative to the last. The period TFR is then exactly Q/(1+d), the period mean age of childbearing rises at d/(1+d) rather than at d, and dividing the first by one minus the second returns Q to five decimal places — the Bongaarts-Feeney adjustment, which on this schedule is an identity rather than a correction....

September 10, 2026 · 15 min · Jongha Jeon

ML 2: Cross-Validation Estimates the Error of a Model You Did Not Train

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. On 2,000 simulated least-squares datasets of 100 rows, where the exact test error of each fitted model is known, the correlation between the 10-fold CV estimate and that error is -0.01. The CV estimates spread 2....

October 8, 2026 · 10 min · Jongha Jeon

Coverage 2: Your Calibration Error Is Mostly Your Bin Count

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A model calibrated by construction - labels drawn from its own probabilities - scores an ECE of 0.085 on 200 predictions, 0.035 on 1,000 and 0.008 on 20,000 at 15 bins, and the fitted slope against bins/n is 0....

October 1, 2026 · 15 min · Jongha Jeon

Headline Statistics 2: Everyone's Wage Rose and the Median Fell

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Give every worker a universal raise of g and hire entrants below the median. The median slides m/2 ranks, order statistics near the median are spaced 1/(n f(M)) apart, so the median falls once the entry share passes 2 g M f(M) - which for a lognormal is 0....

September 10, 2026 · 15 min · Jongha Jeon

ML 3: Not Every Leak Leaks, and the Ones That Do Can Be Measured

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Fit a step on all the rows, then cross-validate, and compare with the same step fitted inside the folds. Standardising leaks +0.05 points on the breast cancer data and mean imputation -0....

October 8, 2026 · 9 min · Jongha Jeon

Coverage 3: Temperature Scaling Cannot Change What You Predict

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. On 10,240 next-character predictions from the committed model, accuracy is 0.5466 at all seven temperatures from 0.5 to 3, identical to the bit, while ECE runs from 0.026 to 0.375. Between predictions the confidence order moves: at T = 2 Kendall’s tau against T = 1 is 0....

October 7, 2026 · 12 min · Jongha Jeon

Headline Statistics 3: Two Countries, One Gini, Opposite Policies

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Take one lognormal population and distort it twice: the poorest tenth loses 90% of its income, or the richest hundredth has its income multiplied by 2.51. Tuned to the same Gini, both land on 0....

September 10, 2026 · 14 min · Jongha Jeon

Coverage 4: Overconfidence Arrives Before the Model Stops Improving

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Three training runs of the same 816,128-parameter model, evaluated at every checkpoint on 10,240 held-out predictions. Every run starts slightly underconfident; overconfidence begins after one to ten passes over the training text and grows as training loss pulls away from validation loss....

October 8, 2026 · 11 min · Jongha Jeon