Ensembles

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Six classifiers on the digits, and the averaged probabilities of all 57 combinations of two or more. 11 combinations beat their best member’s accuracy and 8 fell below their members’ mean. Averaged with a single decision tree, whose probabilities are all 0 or 1, an SVM scoring 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Four base models on a simulated task, 100 to 3,000 training rows, and a logistic meta-model on their probabilities. Fitted on in-sample predictions it gave 1-NN a weight of 13 and reproduced its accuracy exactly (0....