Numerical-Analysis

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. The optimiser everyone uses is an ODE solver, and that is an identity rather than an analogy: gradient descent is forward Euler on the gradient flow. So the learning rate is a step size, and step sizes have stability limits....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A gradient check is a finite difference, and a finite difference has an optimal step size: truncation error falls as the step shrinks and cancellation rises like eps/h, so the total is a U and refining past its bottom makes the answer worse — 9....

Disclosure: this post was written with the assistance of an AI system (Claude), which wrote the analysis code, ran the experiments and drafted the text. The topic, the constraints, the data choices and the final review are the author’s. A spectral solver that blew up at the same moment whatever I changed. Across 24 configurations, a tenfold change in timestep moved the failure by 5%, a finer grid by 5%, and swapping the entire time-stepping scheme by 3%....