ML 41: An Average of Models Beats the Average Model, Not the Best One
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Six classifiers on the digits, and the averaged probabilities of all 57 combinations of two or more. 11 combinations beat their best member’s accuracy and 8 fell below their members’ mean. Averaged with a single decision tree, whose probabilities are all 0 or 1, an SVM scoring 0....