ML 40: Target Encoding Fitted on the Training Rows Still Leaks, Into Training
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A simulated task with five numeric features and a category of 10 to 4,000 levels, 4,000 training rows, every encoding fitted on the training rows only. With a category carrying no signal and 1,000 levels, encoding each level by its mean label took boosted trees from test AUC 0....