<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Data-Science on The Standard Error</title><link>https://jongha-jeon-dev.github.io/standarderror/tags/data-science/</link><description>Recent content in Data-Science on The Standard Error</description><generator>Hugo -- 0.123.7</generator><language>en-gb</language><lastBuildDate>Tue, 13 Oct 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://jongha-jeon-dev.github.io/standarderror/tags/data-science/index.xml" rel="self" type="application/rss+xml"/><item><title>ML 1: Your Test Score Has an Error Bar, and It Is Wider Than Your Gain</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-1-error-bar/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-1-error-bar/</guid><description>A test accuracy is a proportion measured on a finite sample. On 360 test digits its error bar is wider than the gap between the top two of three good classifiers, and repeating the split does not measure what it seems to.</description></item><item><title>School Maths 1: The Only Difference Between These Two Models Is the Order of the Lines</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/school-maths-1-shortcut/</link><pubDate>Fri, 18 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/school-maths-1-shortcut/</guid><description>One of them adds. The other reads the answer off the line above and reverses it, including when that line is wrong. Their held-out losses differ by less than a tenth of a nat, and the leaky one&amp;#39;s is *lower*.</description></item><item><title>Coverage 1: 90% Coverage Is a Promise About Averages, Not About You</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-1-marginal/</link><pubDate>Fri, 11 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-1-marginal/</guid><description>Conformal prediction&amp;#39;s guarantee is exact, distribution-free and finite-sample, and it is an average over the test distribution. Conditioned on the model&amp;#39;s own confidence it runs from 82% to 95% - and then it turns out that the unevenness is the score&amp;#39;s fault, not the method&amp;#39;s.</description></item><item><title>Headline Statistics 1: Korea's Fertility Rebound Needs Nobody to Have More Children</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/headline-statistics-1-fertility-tempo/</link><pubDate>Thu, 10 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/headline-statistics-1-fertility-tempo/</guid><description>The total fertility rate is not children per woman. It is one calendar year of age-specific rates stacked into the completed family size of a woman who does not exist, and a pure change in the timing of births moves it while no cohort&amp;#39;s family size moves at all.</description></item><item><title>Linear Algebra 1: The Condition Number Is the Error Bar on Your Solve</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-1-condition-number/</link><pubDate>Thu, 27 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-1-condition-number/</guid><description>A 14 by 14 system whose answer is all ones, solved to a residual at machine precision, returning entries between -4.9 and +8.5 — and the one number on the matrix that predicts it.</description></item><item><title>ML 2: Cross-Validation Estimates the Error of a Model You Did Not Train</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-2-cv-target/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-2-cv-target/</guid><description>K-fold CV is described as estimating how well your model will do. Where that is known exactly, the estimate is uncorrelated with it. What CV estimates is how models like yours do on average.</description></item><item><title>Coverage 2: Your Calibration Error Is Mostly Your Bin Count</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-2-bins/</link><pubDate>Thu, 01 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-2-bins/</guid><description>Expected calibration error has a floor that a perfectly calibrated model also hits. On a thousand predictions most of a real model&amp;#39;s ECE is that floor - and the fix that removes it makes the number noisier, not truer.</description></item><item><title>Headline Statistics 2: Everyone's Wage Rose and the Median Fell</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/headline-statistics-2-wage-composition/</link><pubDate>Thu, 10 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/headline-statistics-2-wage-composition/</guid><description>A wage statistic compares two populations, not two payslips. Once hiring and firing happen between the two prints, the difference carries a term that has nothing to do with anybody&amp;#39;s pay - and it is large enough to reverse the sign.</description></item><item><title>Linear Algebra 2: Least Squares Three Ways, and Only Two Survive</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-2-least-squares-three-ways/</link><pubDate>Thu, 27 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-2-least-squares-three-ways/</guid><description>The closed form in every regression textbook is a correct formula and a bad algorithm. On exact data it returns coefficients wrong by 14 percent where two other routes get ten decimal places.</description></item><item><title>ML 3: Not Every Leak Leaks, and the Ones That Do Can Be Measured</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-3-leakage/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-3-leakage/</guid><description>&amp;#39;Fit every preprocessing step inside the folds&amp;#39; is right, and taught as if every violation were equally bad. Measured, seven of them range from nothing to a third of the accuracy scale.</description></item><item><title>Coverage 3: Temperature Scaling Cannot Change What You Predict</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-3-temperature/</link><pubDate>Wed, 07 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-3-temperature/</guid><description>Dividing every logit by the same number cannot change a single answer. It can change which answers you decline to give - a little if you abstain on max probability, a lot if you abstain on entropy, and not at all on margin.</description></item><item><title>Headline Statistics 3: Two Countries, One Gini, Opposite Policies</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/headline-statistics-3-gini-shape/</link><pubDate>Thu, 10 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/headline-statistics-3-gini-shape/</guid><description>A scalar cannot carry a shape. The Gini is many-to-one onto distributions, and so are the percentile ratios and the poverty headcount - so two worlds needing opposite policies can agree on five headline numbers at once.</description></item><item><title>Linear Algebra 3: Your Covariance Matrix Is Not Positive Definite</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-3-positive-definite/</link><pubDate>Thu, 27 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-3-positive-definite/</guid><description>Three correlations, each one a defensible estimate, each one inside [-1, 1] — and together a claim that a particular portfolio has a variance of minus 0.11.</description></item><item><title>Coverage 4: Overconfidence Arrives Before the Model Stops Improving</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-4-training/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-4-training/</guid><description>The syllabus said overconfidence would arrive when validation loss stopped improving. Three training runs with checkpoints say it arrives much earlier, and only on text the model has not seen.</description></item><item><title>ML 4: A Learning Curve Fitted Early Promises More Than More Data Delivers</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-4-learning-curves/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-4-learning-curves/</guid><description>Fit a power law to a pilot study&amp;#39;s errors and it tells you what more data will buy. Checked against what more data did buy, it was optimistic every time, sometimes past the Bayes error.</description></item><item><title>Linear Algebra 4: PCA When Two Eigenvalues Are Equal</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-4-equal-eigenvalues/</link><pubDate>Mon, 31 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-4-equal-eigenvalues/</guid><description>A correlation matrix whose two largest eigenvalues differ by 0.02, where the component carrying the most variance is the one that moves 42 degrees between samples — and the resampling check reports a third of it.</description></item><item><title>Coverage 5: The Split You Chose Is Hiding How Variable Your Coverage Is</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-5-split/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/coverage-5-split/</guid><description>Episode 1 promised an error bar wider than exchangeability allows. Most of that was my formula. What is left appears only when you split the way a deployment does.</description></item><item><title>ML 5: Resampling Moves the Threshold and Breaks a Logistic Model's Probabilities</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-5-imbalance/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-5-imbalance/</guid><description>&amp;#39;Balance the classes before training&amp;#39; is the first fix taught for a rare positive class. Measured, it never improved the ranking, multiplied a logistic model&amp;#39;s probabilities tenfold, and did nothing to its decisions that a threshold does not. Boosted trees lost ranking and, with some methods, gained calibration.</description></item><item><title>Linear Algebra 5: What Ridge Does to the Geometry</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-5-ridge-geometry/</link><pubDate>Tue, 01 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-5-ridge-geometry/</guid><description>Every variance inflation factor at 1.00 on a design with a condition number near a billion, a cross-validated ridge fit that spends 3.6 of its 9 parameters, and a nominal 95 percent interval that covers 34 percent.</description></item><item><title>ML 6: On Separable Data, Your Logistic Coefficients Are Your Stopping Rule</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-6-separable/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-6-separable/</guid><description>Logistic regression is taught as finding the maximum-likelihood coefficients. When a hyperplane separates the classes there is no maximum, and what comes back is decided by when the optimiser stopped.</description></item><item><title>Linear Algebra 6: One Row Can Own the Fit</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-6-one-row/</link><pubDate>Tue, 01 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-6-one-row/</guid><description>A single observation with a residual of exactly zero, a Cook&amp;#39;s distance of 0/0, and a coefficient that stops existing when you delete it — and, separately, one row moving a slope by six standard errors.</description></item><item><title>ML 7: Change a Feature's Units and the Lasso Changes Its Mind</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-7-units/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-7-units/</guid><description>A penalty strength is taught as a property of the model. A lasso penalty is a price per unit of coefficient, so a feature&amp;#39;s units decide what it pays - and whether it stays.</description></item><item><title>Linear Algebra 7: The Scree Plot Lies</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-7-scree-plot/</link><pubDate>Tue, 01 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-7-scree-plot/</guid><description>Asked about a matrix of pure noise, the elbow returns 18 different answers across 300 draws and never once returns none — while three calibrated rules get it right. And the rule that is provably optimal reports zero components where three exist.</description></item><item><title>ML 8: Cross-Validated Lasso Is Tuned to Predict, Not to Select</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-8-selection/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-8-selection/</guid><description>The lasso is used to find the features that matter. Tuned by cross-validation it keeps pure noise, cannot tell a feature from a correlated stand-in, and on real data keeps features whose sign it cannot decide.</description></item><item><title>Linear Algebra 8: When There Is No Closed Form</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-8-no-closed-form/</link><pubDate>Tue, 01 Sep 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/linear-algebra-8-no-closed-form/</guid><description>A logistic regression where one coefficient&amp;#39;s value is the iteration limit, its standard error is a constant the library author chose, and its p-value crosses 0.05 on the way down — while every other coefficient in the table is correct to three decimal places.</description></item><item><title>ML 9: The Bias–Variance U Belongs to a Knob, Not to Complexity</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-9-bias-variance/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-9-bias-variance/</guid><description>Test error is said to be U-shaped in model complexity. Computed exactly, it is for the number of neighbours and is not for the number of trees: one knob trades bias for variance, the other only averages.</description></item><item><title>ML 10: The Second Descent Is Real, and a Ridge Penalty Makes It Unnecessary</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-10-double-descent/</link><pubDate>Thu, 08 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-10-double-descent/</guid><description>Past the point where a model fits its training data exactly, test error falls again. Measured on random features, the fall is real, never reaches the best small model, and disappears as a reason once the fit is penalised.</description></item><item><title>ML 11: Resample the Rows and the Tree Tells a Different Story</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-11-tree-stability/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-11-tree-stability/</guid><description>A decision tree is valued because it can be read. Refit on bootstrap resamples of the same data, its accuracy holds still and almost everything a reader would read off it moves.</description></item><item><title>ML 12: Impurity Importance Rewards Features With Many Places to Split</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-12-impurity/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-12-impurity/</guid><description>A forest&amp;#39;s default feature importance is computed on the data it was fitted to. On a design where the answer is known, it never ranks the only useful feature first, and ranks it last in 19 of 20 datasets.</description></item><item><title>ML 13: Out-of-Bag Error Is Free, Pessimistic When Small and Blind to Copies</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-13-oob/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-13-oob/</guid><description>A random forest estimates its own test error from the rows each tree did not see. With enough trees that is as good as cross-validation, for a sixth of the work. With few trees, or rows that are not independent, it is not.</description></item><item><title>ML 14: Boosting Overfits Its Probabilities Long Before Its Accuracy</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-14-boosting/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-14-boosting/</guid><description>Whether boosting overfits depends on what you measure. Over 2,000 rounds its test error never turns up after the first few dozen; its test log loss turns after a few dozen, and below a learning rate of 0.1 shrinkage only postpones the turn.</description></item><item><title>ML 15: A Forest Cannot Predict Beyond the Largest Value It Has Seen</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-15-extrapolation/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-15-extrapolation/</guid><description>Tree ensembles are said to approximate any function. Inside the training range they do; outside it their predictions go flat, below the largest label they saw, and the best single model becomes the worst.</description></item><item><title>ML 16: Permutation Importance Measures the Model, Not What the Data Need</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-16-permutation/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-16-permutation/</guid><description>Shuffle a column and see how much worse the model does: it is the importance episode 12 recommended. It answers a narrower question than it seems to, on rows the shuffle invented.</description></item><item><title>ML 17: Partial Dependence Asks the Model About Rows That Do Not Exist</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-17-pdp/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-17-pdp/</guid><description>A partial dependence plot is read as a feature&amp;#39;s effect. With correlated inputs it is an average over combinations that never occur, and on a simulation with a known effect a forest&amp;#39;s shows about a third of it.</description></item><item><title>ML 18: A SHAP Value Is an Answer to "Compared With What?"</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-18-shapley/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-18-shapley/</guid><description>Shapley values split a prediction among its features fairly and uniquely — relative to a reference that the theory leaves to you. Computed exactly, the choice changes the story for most predictions.</description></item><item><title>ML 19: A Planted Shortcut Wins the Test Set and Loses Everywhere Else</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-19-shortcut/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-19-shortcut/</guid><description>The model with the better test score is usually taken to have learned the task better. Plant one telltale pixel in the digits and the better score belongs to the model that learned the pixel.</description></item><item><title>ML 20: Same Accuracy, Different Answers</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-20-rashomon/</link><pubDate>Mon, 12 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-20-rashomon/</guid><description>Two models with the same test accuracy are usually treated as interchangeable. On a hard task, networks that differ only in their seed give different answers on a fifth of the rows.</description></item><item><title>ML 28: Augmentation Adds an Assumption, Not Data</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-28-augmentation/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-28-augmentation/</guid><description>Shifted copies of 150 digits made a network worse on the test set and far better on shifted digits. Six hundred more real digits did the reverse.</description></item><item><title>ML 29: Covariate Shift Charges for Misspecification</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-29-covariate-shift/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-29-covariate-shift/</guid><description>When only the inputs move, the best possible accuracy need not change. Whether a model&amp;#39;s accuracy does depends on how wrong the model was all along.</description></item><item><title>ML 31: Principal Components Keep Variance, Not Relevance</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-31-pca/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-31-pca/</guid><description>PCA keeps the directions along which the inputs vary most. On real data that can be a choice of units, and the direction that predicts the target can come fourth.</description></item><item><title>ML 32: k-Means Finds k Pieces Whether or Not There Are Clusters</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-32-kmeans/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-32-kmeans/</guid><description>k-means always returns k convex pieces of similar spread, and the silhouette score rates the pieces. On uniform noise it rates them higher than it rates the digits.</description></item><item><title>ML 33: The Curse of Dimensionality Counts the Dimensions the Data Use</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-33-dimensions/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-33-dimensions/</guid><description>Distances concentrate in high dimensions, for points that fill them. The digits&amp;#39; 64 pixels behave like 8 uniform dimensions, and nearest neighbours work on them until noise columns are added.</description></item><item><title>ML 34: Recalibration Fixes the Average, Given Enough Rows</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-34-calibration/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-34-calibration/</guid><description>Platt scaling and isotonic regression learn a correction from held-out rows. With few rows they can make a model worse, and with many they fix the average, not each group.</description></item><item><title>ML 35: The Best Cross-Validated Score Is the Search's Best Luck</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-35-tuning/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-35-tuning/</guid><description>Cross-validation estimates one model fairly. The best of many cross-validated scores is the maximum of noisy estimates, and it overstates the model it picks.</description></item><item><title>ML 36: Shuffling a Time Series Is Safe Until the Errors Remember</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-36-time-series/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-36-time-series/</guid><description>The rule against shuffled folds is right for a reason that is narrower than time: something the model cannot see changing slowly. Forward splitting is not the cure it is sold as.</description></item><item><title>ML 37: Rows From the Same People Are Not New Data</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-37-subjects/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-37-subjects/</guid><description>When subjects contribute many rows each, row-level cross-validation measures how well the model knows the people it has met, and extra rows from the same people are worth little.</description></item><item><title>ML 38: A Missing Value Is Information, Until the Process Behind It Changes</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-38-missing/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-38-missing/</guid><description>Imputation treats a gap as a lost number. Sometimes the gap is a record of a decision, worth more than the number, and the model that uses it depends on the decision staying the same.</description></item><item><title>ML 39: The Applicants You Refused Are the Data You Do Not Have</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-39-selective-labels/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-39-selective-labels/</guid><description>Outcomes are recorded only for the cases someone approved. Whether a model trained on them can judge the rest depends on what the approver knew that the model does not.</description></item><item><title>ML 40: Target Encoding Fitted on the Training Rows Still Leaks, Into Training</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-40-target-encoding/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-40-target-encoding/</guid><description>Fitting an encoding only on training rows protects the test set. It does not protect the training rows from their own labels, and with many rare levels that is enough to wreck the model.</description></item><item><title>ML 41: An Average of Models Beats the Average Model, Not the Best One</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-41-averaging/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-41-averaging/</guid><description>Averaging carries a guarantee, and it is weaker than the one usually quoted. When one member is certain about everything, averaging probabilities hands it every decision.</description></item><item><title>ML 42: Stacking Learns to Pick the Best Model, Given Honest Predictions</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-42-stacking/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-42-stacking/</guid><description>A meta-model trusts whichever base model looks best on the predictions it is fitted to. On in-sample predictions that is the best memoriser; on honest ones it is usually just the best model.</description></item><item><title>ML 43: Naive Bayes Counts Correlated Evidence Twice, and Only Its Probabilities Notice</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-43-naive-bayes/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-43-naive-bayes/</guid><description>The independence assumption is wrong and naive Bayes still classifies well, as the textbooks say. What the assumption breaks is the one thing the model&amp;#39;s name promises: its probabilities.</description></item><item><title>ML 44: Cross-Validation Folds Are Not Independent Samples, and the t-Test Assumes They Are</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-44-model-tests/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-44-model-tests/</guid><description>Two algorithms built to be exactly equal, compared 400 times. The most common test found a difference one time in seven, the most thorough-looking one more often than not, and three others were honest.</description></item><item><title>ML 45: A Test Set Answers Honestly Only the First Time</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-45-holdout/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-45-holdout/</guid><description>A test set is not spent by training on it. It is spent by asking it questions whose answers change what you do next, and it can be spent without touching a single weight.</description></item><item><title>ML 46: The Right Threshold Is a Cost Ratio, If the Probabilities Are Calibrated</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-46-threshold/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-46-threshold/</guid><description>0.5 is the right threshold only when both mistakes cost the same. The threshold that matches the costs needs calibrated probabilities, and even the perfect model may save little.</description></item><item><title>ML 47: The Model With the Best AUC Was Never the Best When Misses Cost Ten Times More</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-47-auc-decisions/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-47-auc-decisions/</guid><description>AUC summarises a ranking over every threshold at once. A decision uses one threshold, and the model that ranks best on average is not always the one that ranks best there.</description></item><item><title>ML 48: What a Model Relies On Is Not What to Change</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-48-intervention/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-48-intervention/</guid><description>Importance and what-if analyses describe the model. In a system with a common cause and a downstream signal, the feature the model relied on most did nothing when changed, and the real lever ranked no higher than a feature with no effect.</description></item><item><title>ML 49: Target the Effect, Not the Risk</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-49-uplift/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-49-uplift/</guid><description>A model of who will churn is not a model of whom an offer will keep. Targeting the highest risk did no better than chance; targeting the predicted effect did about twice as well.</description></item><item><title>ML 50: A Rule That Chooses Its Own Data Learns Its Own Choices</title><link>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-50-feedback/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/lectures/ml-50-feedback/</guid><description>When a rule decides where to look and learns only from what it finds there, more data does not correct it. It has to account for its own effort, and keep looking where it expects nothing.</description></item><item><title>Two Thirds of the Move Happens Before the Headline</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/two-thirds-of-the-move-before-the-headline/</link><pubDate>Wed, 19 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/two-thirds-of-the-move-before-the-headline/</guid><description>A paper measured &amp;#39;buy the rumour, sell the news&amp;#39; on 4.57 million articles, and the sayings mostly hold</description></item><item><title>The Chip Cycle Is Forecastable. The Shortage Is a Different Question.</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/the-chip-cycle-is-forecastable/</link><pubDate>Tue, 18 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/the-chip-cycle-is-forecastable/</guid><description>A delayed feedback loop, an echo state network, and a result I had the wrong way round</description></item><item><title>The Jeonse Number That Is Not on the Contract</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/the-jeonse-number-not-on-the-contract/</link><pubDate>Fri, 14 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/the-jeonse-number-not-on-the-contract/</guid><description>A deposit&amp;#39;s safety margin is two published ratios divided — and the interesting part is what that arithmetic cannot tell you</description></item><item><title>Your Index Fund Owns 503 Companies and Is About Three Bets</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/your-index-fund-owns-503-companies-and-is-three-bets/</link><pubDate>Fri, 14 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/your-index-fund-owns-503-companies-and-is-three-bets/</guid><description>Concentration has a formula, and running it is unkind to both sides of the argument</description></item><item><title>Individuals Sold a Record Amount on the Best Day in Market History</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/individuals-sold-a-record-amount-on-the-best-day/</link><pubDate>Thu, 13 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/individuals-sold-a-record-amount-on-the-best-day/</guid><description>The gap between what an index returns and what its investors earn is arithmetic — and I was wrong about where it comes from</description></item><item><title>I Trained 2,000 Models on a Coin Flip and the Best One Looked Great</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/i-trained-2000-models-on-a-coin-flip/</link><pubDate>Fri, 07 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/i-trained-2000-models-on-a-coin-flip/</guid><description>A 55.3% hit rate, a p-value of 0.0008, and not one shred of signal in the data</description></item><item><title>The KOSPI's Record Day Was Not a Fat-Tail Problem</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/the-kospi-record-day-was-not-a-fat-tail-problem/</link><pubDate>Fri, 07 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/the-kospi-record-day-was-not-a-fat-tail-problem/</guid><description>A +17.91% session, a 12-sigma move, and the assumption that actually broke</description></item><item><title>Why Every Bus You Catch Is on a Longer-Than-Average Gap</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/why-every-bus-is-on-a-longer-than-average-gap/</link><pubDate>Fri, 07 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/why-every-bus-is-on-a-longer-than-average-gap/</guid><description>Your class was bigger than average, your queue was slower, and your bus was late. One formula, and no bias in sight</description></item><item><title>Your 90% Prediction Interval Covers 42% of the Time</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/your-90-percent-interval-covers-42-percent/</link><pubDate>Fri, 07 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/your-90-percent-interval-covers-42-percent/</guid><description>Conformal prediction&amp;#39;s guarantee is real, and it does not say what people repeat about it</description></item><item><title>Your Forecast Is Probably Just Repeating Yesterday</title><link>https://jongha-jeon-dev.github.io/standarderror/posts/your-forecast-is-probably-just-repeating-yesterday/</link><pubDate>Thu, 06 Aug 2026 00:00:00 +0000</pubDate><guid>https://jongha-jeon-dev.github.io/standarderror/posts/your-forecast-is-probably-just-repeating-yesterday/</guid><description>A model scores 99% on a series of coin flips. Here is why, and the one number that would have caught it</description></item></channel></rss>