Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Networks trained with dropout 0.5 were evaluated two ways: with dropout off and the weights scaled, and by averaging up to 300 sampled masks. On the digits the two agreed on 99....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A two-layer network with 512 units per layer, trained full-batch on 898 digits, reached 100% training accuracy in 41 steps on the true labels (test accuracy 0.968) and in 73 on entirely random labels (test 0....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A small network trained full-batch on 40% of the pairs of (a + b) mod 97 reached 100% training accuracy at step 200, when its accuracy on the other pairs was 0....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Trained on 150 digits, a network scored 0.917. Four times as many real digits raised that by 0.041; four shifted copies of each image changed it by -0.027, pixel noise by +0....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. In a simulation where the test inputs move up to two standard deviations while P(y | x) stays fixed, the Bayes accuracy stays between 0.832 and 0.839. A flexible spline logistic model stays within 0....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Eight character-level transformers from 4,288 to 1,789,248 non-embedding parameters were trained for the same 2,000 steps on the same million characters of Shakespeare (4.1 million characters seen). A power law L = 4....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. On raw breast-cancer measurements the first principal component holds 98.2% of the variance and is led by worst area, the feature with the largest units; the 95% variance rule keeps 1 component, against 10 on standardised features....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Over ten draws of four simulated shapes, k-means recovered four separated round blobs (adjusted Rand index 1.00) and the silhouette chose k = 4 each time. On uniform noise, with no clusters, the silhouette also chose k = 4 each time, with a score of 0....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. For uniform points, the nearest neighbour’s distance over the median distance rose from 0.03 in two dimensions to 0.94 in 1,024: in high dimensions the nearest point is barely nearer than the rest....
Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Boosted trees on a simulated task were overconfident (calibration error 0.157, log loss 0.748). Platt scaling on 50 calibration rows cut log loss to 0.612; isotonic regression on 50 rows raised it to 0....