Data-Leakage

ML 3: Not Every Leak Leaks, and the Ones That Do Can Be Measured

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Fit a step on all the rows, then cross-validate, and compare with the same step fitted inside the folds. Standardising leaks +0.05 points on the breast cancer data and mean imputation -0....

ML 52: A Leaked Benchmark Shows in the Set, Not in the Item

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. The repository’s main arithmetic model was trained for 600 more steps with 300 benchmark problems inserted into its stream k times, for k from 0 to 32. Five runs with nothing leaked scored 0....