Data-Science

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Over ten draws of four simulated shapes, k-means recovered four separated round blobs (adjusted Rand index 1.00) and the silhouette chose k = 4 each time. On uniform noise, with no clusters, the silhouette also chose k = 4 each time, with a score of 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. For uniform points, the nearest neighbour’s distance over the median distance rose from 0.03 in two dimensions to 0.94 in 1,024: in high dimensions the nearest point is barely nearer than the rest....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Boosted trees on a simulated task were overconfident (calibration error 0.157, log loss 0.748). Platt scaling on 50 calibration rows cut log loss to 0.612; isotonic regression on 50 rows raised it to 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Grid searches of an RBF support vector machine on 200 rows of a simulated task, twenty times per grid size. Untuned, the cross-validated score was 0.013 below the model’s accuracy on 20,000 fresh rows....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Seven simulated time-series settings, twenty series each, cross-validated three ways on 1,000 steps and compared with the error on the next 250. On a stationary autoregression with its lags as features, shuffled five-fold cross-validation was within 3% of the future error for linear regression, nearest neighbours and a forest....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. 2,000 simulated rows from 2,000, 400, 100, 40 or 20 subjects, each subject with a feature signature and an outcome propensity that do not carry over. With 2,000 subjects, row-level and subject-level cross-validation matched accuracy on new people to within a point....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. The breast-cancer data with 30% of cells removed three ways. At random, the imputation that recovered the values best (iterative, error 0.44 sd) gave the best model, log loss 0.091 against 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Simulated applicants, half accepted by an old policy, outcomes observed only for the accepted. When the policy used only the model’s features, a correctly specified model predicted a default rate of 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A simulated task with five numeric features and a category of 10 to 4,000 levels, 4,000 training rows, every encoding fitted on the training rows only. With a category carrying no signal and 1,000 levels, encoding each level by its mean label took boosted trees from test AUC 0....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. Six classifiers on the digits, and the averaged probabilities of all 57 combinations of two or more. 11 combinations beat their best member’s accuracy and 8 fell below their members’ mean. Averaged with a single decision tree, whose probabilities are all 0 or 1, an SVM scoring 0....