Blog
What Kind of a Statistician is a Transformer?
August 19, 2026
A statistical view of in-context learning through Jacobians, smoothers, and effective degrees of freedom.
Transformer Induction Heads
July 30, 2026
Training small transformers to copy from context, and watching induction heads form as a two-layer circuit.
Much of linear least-squares learning is spectral filtering
July 23, 2026
OLS, ridge, PCR, gradient descent, momentum, and early stopping are all one estimator with a different filter function on the data spectrum.
Epoch-wise double descent in OLS
July 21, 2026
Double descent in plain OLS, traced over training epochs rather than model size, where the implicit bias of gradient descent does the work.