ML 42: Stacking Learns to Pick the Best Model, Given Honest Predictions
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Four base models on a simulated task, 100 to 3,000 training rows, and a logistic meta-model on their probabilities. Fitted on in-sample predictions it gave 1-NN a weight of 13 and reproduced its accuracy exactly (0....