ML 57: The BIC Counts Gaussians, Not Clusters
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Gaussian mixtures fitted to a single two-dimensional cluster: for a Gaussian one, the BIC chose one component at every size from 200 to 20,000 points; for a uniform square it chose 2, 7, 14, 18; for a heavy-tailed cluster 2, 3, 4, 9....