Clustering

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Persistent homology arrives with a large vocabulary and, in dimension zero, a very small computation: walk the edges shortest first and record the length of each one that joins two components. That is the single-linkage dendrogram’s merge heights, and not approximately — the two return arrays that are equal float for float, on four designs from 12 to 60 points and 1 to 30 dimensions....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Three measurements, and the third one reversed the plan. First: the H0 death spread, as a fraction of its own mean, falls from 4.87 at two dimensions to 0.078 at 768 on pure noise....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Over ten draws of four simulated shapes, k-means recovered four separated round blobs (adjusted Rand index 1.00) and the silhouette chose k = 4 each time. On uniform noise, with no clusters, the silhouette also chose k = 4 each time, with a score of 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Two Gaussian clusters in ten dimensions, one ten times as wide as the other, came out of t-SNE with a radius ratio between 0.99 and 1.01 at perplexities 5, 30 and 100; three clusters with one gap ten times the other came out with a gap ratio between 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Gaussian mixtures fitted to a single two-dimensional cluster: for a Gaussian one, the BIC chose one component at every size from 200 to 20,000 points; for a uniform square it chose 2, 7, 14, 18; for a heavy-tailed cluster 2, 3, 4, 9....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. On two half-moons DBSCAN scored an adjusted Rand index of 0.95 or more for radii between 0.071 and 0.206, and 1.00 from 0.116, where k-means scored 0.25; the radius at the knee of the k-distance plot, 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. k-means stability, the agreement between clusterings of two random 80% subsamples, chose k = 2 for one elongated Gaussian (stability 0.92) and k = 4 for a uniform square (0.94), neither of which has clusters....