ML 34: Recalibration Fixes the Average, Given Enough Rows
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Boosted trees on a simulated task were overconfident (calibration error 0.157, log loss 0.748). Platt scaling on 50 calibration rows cut log loss to 0.612; isotonic regression on 50 rows raised it to 0....