Automatic-Differentiation on The Standard Errorhttps://jongha-jeon-dev.github.io/standarderror/tags/automatic-differentiation/Recent content in Automatic-Differentiation on The Standard ErrorHugo -- 0.123.7en-gbThu, 03 Sep 2026 00:00:00 +0000Numerical Analysis 2: A Gradient Check Is a Finite Differencehttps://jongha-jeon-dev.github.io/standarderror/lectures/numerical-analysis-2-gradient-check/Thu, 03 Sep 2026 00:00:00 +0000https://jongha-jeon-dev.github.io/standarderror/lectures/numerical-analysis-2-gradient-check/The error of a finite difference is a U in the step size, so a smaller step is not a safer one — and the step decides what your gradient check can see, by eight orders of magnitude in float64 and nine more once you change the precision. In bfloat16 a gradient 17% wrong passes at every step size.