q-learning is the algorithm that made reinforcement learning feel inevitable: interact with an unknown world, nudge a table of numbers after every step, and the table converges to the value of optimal behaviour — even while you behave suboptimally the entire time. 𐃏 everything runs on one line of arithmetic, and the rest of this page is the machinery needed to say precisely why that line works. the canonical reference for all of it is sutton & barto’s reinforcement learning: an introduction, free at http://incompleteideas.net/book/the-book-2nd.html.
Agentic
promote the objective of a linear program from a plane to a bowl and you get quadratic programming: minimise a quadratic function over a polyhedron. it is the smallest step beyond LP, yet it captures a startling share of applied mathematics — support vector machines, portfolio selection, ridge regression, model-predictive control — because “squared penalty subject to linear rules” is how half the world states its preferences. 𐃏
feedforward networks eat fixed-size vectors. sequences — text, audio, sensor streams — have no fixed size, and worse, their order carries the meaning. the recurrent neural network solves both problems with one idea: maintain a hidden state that is updated by the same function at every time step. 𐃏 parameter count stops depending on sequence length, and the state becomes a lossy summary of everything seen so far (Goodfellow, Ian, 2016).
You should not be permitted to write production code if you do not have an journeyman licence in regular expressions or floating point math. —Rob Pike
a regular expression is two things wearing one syntax: a seventy-year-old theorem about finite automata, and the single most-used text-processing tool on unix. 𐃏 this page takes both seriously: the theory tells you exactly what the notation can and cannot express, and the theory’s failure modes (backtracking blowups, backreference NP-hardness, the html-parsing folklore) are precisely where practitioners get burned.
this page collects the closed-form solutions to regularised regression (where they exist) and the iterative approximations we fall back on (where they don’t). 𐃏 along the way we will see that regularisation is not an ad-hoc hack but a perfectly sensible artefact of estimation: it drops straight out of MAP (maximum a posteriori) inference once you put a prior on the coefficients.
retrieval-augmented generation bolts a search engine onto a language model: fetch relevant documents at query time and paste them into the prompt, so the model answers from evidence rather than from its frozen weights (lewis et al. 2020, retrieval-augmented generation for knowledge-intensive nlp). 𐃏 it is the pragmatic alternative to baking every fact into an llm’s weights.
why retrieve at all
three problems that no amount of scaling fixes cleanly, and retrieval fixes cheaply:
every linear program you have ever written down was a lie: the coefficients came from measurements, forecasts and vendor spreadsheets, and the optimal vertex — sitting, by design, on the boundary of the feasible region — shatters the moment any of them wobbles. 𐃏 robust optimisation is the pessimist’s response: declare a set \(\mathcal{U}\) of realisations you refuse to be hurt by, and demand feasibility for every member of it. no distributions, no expectations, no scenarios — just a set and a worst case. the surprise, and the reason the field exists, is that this worst case can usually be folded back into a deterministic problem of the same (or nearly the same) complexity class.1
the calculus of one real variable, assembled in logical order: limits, then continuity and its two workhorse theorems (ivt, evt), then the derivative and the mean value theorem, taylor’s theorem with an honest error bound, and finally the riemann integral and both parts of the fundamental theorem. everything downstream — multivariable calculus, differential equations, every convergence argument in machine learning — leans on the theorems here. proofs are given where they are short and instructive; the deferred foundations live in real analysis (Courant, Richard, 1996).
diffusion models generate by learning to undo noise: destroy an image with a fixed gaussian corruption process, train a network to reverse one small step of the destruction, then chain the reversals from pure noise back to data (ho et al. 2020, denoising diffusion probabilistic models). 𐃏 stable diffusion (rombach et al. 2022, high-resolution image synthesis with latent diffusion models) runs this machinery not on pixels but in the latent space of an autoencoder, with a text-conditioned u-net doing the denoising. first the maths, then the architecture.
a linear program assumes you know the data. stochastic programming admits that you do not — some coefficients are random — but insists you know their distribution, and asks for the decision that is best on average. 𐃏 the structural insight that makes this a paradigm rather than a hack: split the decision in two. commit to \(x\) now, before the coin is flipped; after uncertainty resolves, take a corrective recourse action \(y\) that adapts to whatever happened. the objective charges you for both, weighting the second stage by expectation.1