Agentic on Aayush Bajaj's Augmenting Infrastructure

A Catalogue of Loss Functions

Sat, 31 Oct 2026 18:53:32 +1100

a loss function is not a detail of training — it is the definition of the problem. choose squared error and you have asked for the conditional mean; choose absolute error and you have asked for the median; choose hinge and you have asked only for the decision boundary; choose cross-entropy and you have asked for the whole probability. 𐃏 this page catalogues the standard losses, proves what each one’s minimiser actually is, and draws the classic picture that unifies the classification zoo: every one of them is a bribe paid to make the 0–1 loss differentiable.

API Design

Sat, 31 Oct 2026 18:53:32 +1100

an api is a promise you cannot take back: the moment a second team depends on it, every quirk you shipped is load-bearing. 𐃏 this page is the checklist for making promises deliberately — resources and verbs, status codes, pagination, versioning, errors, auth, and rate limiting, with the http machinery treated as the application-layer protocol it is (Kurose, James F. and Ross, Keith W., 2020).

Approximate Inference

Sat, 31 Oct 2026 18:53:32 +1100

exact inference costs \(\exp(\text{treewidth})\), and real networks — image grids, dense diagnostic models, unrolled dbns — routinely have treewidth in the hundreds. approximate inference trades guaranteed answers for tractable ones, along two philosophically different roads: sampling replaces the distribution with draws from it (anytime, asymptotically exact, embarrassingly parallel), and approximate message passing runs exact-style propagation on graphs where its correctness proof no longer holds. this page walks the sampling ladder in the order each rung fixes the last one’s failure — forward, rejection, likelihood weighting, gibbs — then closes with loopy belief propagation. 𐃏

Autoencoders

Sat, 31 Oct 2026 18:53:32 +1100

an autoencoder is a network trained to do the one thing that sounds useless: output its own input. the trick is the obstacle course in the middle — a bottleneck, a corruption, a penalty — that makes verbatim copying impossible, so the network is forced to learn what about the input is worth keeping. 𐃏 the family tree below runs from the linear special case (which is pca wearing a trenchcoat) to the variational autoencoder, which turns the whole construction into a generative model (Goodfellow, Ian, 2016).

Banagrams Solver

Sat, 31 Oct 2026 18:53:32 +1100

bananagrams hands you a fistful of letter tiles and one instruction: arrange all of them into a connected crossword before anyone else does. 𐃏 this page documents the real solver living in this repo at static/code/bananagrams/ — a haskell heuristic search in haskell-imp/ plus a playable js incarnation served at /code/bananagrams/ on this site — rather than a from-scratch design; a compact python reference solver (trie + backtracking) is developed at the end to make the algorithmic skeleton explicit.

Bayesian Networks

Sat, 31 Oct 2026 18:53:32 +1100

a joint probability distribution over \(n\) binary variables is a table with \(2^n - 1\) free numbers. at \(n = 30\) that is a billion parameters for a toy problem — nobody can elicit them, store them, or estimate them from data. the bayesian network is the classical answer: encode the independencies of the distribution in a directed acyclic graph, and the joint collapses into a product of small local tables. 𐃏 the graph is simultaneously a data structure for storage, a calculus for reading off independence, and the substrate on which every inference algorithm runs.

Bazaar

Sat, 31 Oct 2026 18:53:32 +1100

✦ The bazaar is where the things I build get sold: one-time purchases, delivered the moment payment clears, no accounts.

Four stalls:

reMarkable — native plugins for the Paper Pro → bazaar.abaj.ai/remarkable
Flashcards — hand-authored Anki decks → bazaar.abaj.ai/flashcards
Notes — typeset study bundles → bazaar.abaj.ai/notes
Prints — art from the studio of 佐野貴代美 → bazaar.abaj.ai/prints

Each stall’s page carries a heading per listed item — a small brief of what it is and why it’s worth money.

Bazaar: Flashcards

Sat, 31 Oct 2026 18:53:32 +1100

Hand-authored Anki decks, exported clean and sold at bazaar.abaj.ai/flashcards. The rule for what gets sold: only decks that are my own words — anything derived from textbooks, publisher audio, or downloaded shared decks stays out of the shop (sourcing rules in the bazaar repo’s SPEC §7). All four are in pre-sale cleanup: verbatim third-party fragments get stripped before any deck ships.

CS Interview Prep

~1,300 cards covering the software-interview surface: Java, Python, data structures & algorithms, OOP, git, and interview framing. One idea per card, short prompts. Coming soon → /p/cs-interview-prep

Bazaar: Notes

Sat, 31 Oct 2026 18:53:32 +1100

Typeset study bundles — LaTeX-set PDFs of my own summaries — sold at bazaar.abaj.ai/notes. The gate for this stall is the strictest in the bazaar: only 100%-own-authorship summaries, no lecturer material, course branding stripped. Free counterparts live across the site’s typeset-PDF wiki pages; the paid bundles are the ones polished into standalone books.

STAT 571 — Probability & Measure

My course notes from the University of Alberta’s graduate probability-and-measure course: a ~50-page arc from sigma-fields to the ergodic theorem — Carathéodory extension, the convergence theorems, Fubini–Tonelli, L^p, Borel–Cantelli, LLN, CLT — thirteen lectures compressed to revision speed. I write my own notes for every course I take; these ship with the custom LaTeX preamble (Baskervaldx + newtxmath, the theorem environments) so yours can look this good too. Live → /p/measure-theory-notes

Bazaar: Prints

Sat, 31 Oct 2026 18:53:32 +1100

Art prints from the studio of 佐野貴代美, sold at bazaar.abaj.ai/prints.

The stall is being stocked — first pieces are in preparation. Each print will get its own heading here as it’s listed.

Bazaar: reMarkable

Sat, 31 Oct 2026 18:53:32 +1100

Native plugins for the reMarkable Paper Pro and the Paper Pro Move, built against real firmware and sold at bazaar.abaj.ai/remarkable. Everything runs under xovi / rm-appload rather than patching the system: no bank-flip risk, graceful degradation on firmware updates, and every item version-pinned to the firmware it was tested on.

Nearly every item on this shelf now ships for both devices — each product page has a device picker (Paper Pro / Paper Pro Move), and the Move builds are packaged from the exact files proven on real Move hardware. The handful of exceptions (the Move builds of FastTerm, the GUI Emacs and laserPen — the first two need a keyboard story the Move doesn’t have) say so plainly on their pages.

Build Systems

Sat, 31 Oct 2026 18:53:32 +1100

every build system — make, ninja, bazel, cargo, even npm scripts pretending otherwise — is the same machine: a directed acyclic graph of files and commands, plus a policy for deciding which part of the graph is stale. everything else is syntax.

the model: a dag of targets

a target is a file the build can produce; a rule says how (a command) and from what (its dependencies).
dependencies point from outputs to inputs; because outputs of one rule are inputs to another, the whole thing is a dag — a cycle would mean “to build a you must first build a”.
a build is then two steps:
1. mark dirty: a target is dirty if it is missing, if any dependency is dirty, or if the staleness policy (timestamps or hashes, below) says an input changed. dirtiness propagates along edges — one flipped source poisons its whole downstream cone.
2. evaluate in topological order: run each dirty target’s command after all its dependencies are up to date (Cormen, Thomas H. and Leiserson, Charles E. and Rivest, Ronald L. and Stein, Clifford, 2009). independent dirty targets can run in parallel (-j8) — the dag is the parallelism plan, which is why builds are embarrassingly parallel until the link step serialises everything.
the entire correctness contract: the graph must be complete. every undeclared dependency is a future “works after make clean” bug — the build system faithfully skips rebuilding things it was never told could change.

dirty-node propagation: editing greet.h dirties every target downstream of it (red); main.c and its untouched cone stay cached. the build re-runs exactly the red commands, in topological order.

make: the actual semantics

make’s whole rebuild rule fits in one sentence: a target is rebuilt if it does not exist, or if any prerequisite’s mtime is newer than the target’s mtime. everything else is macro expansion.

Calculus

Sat, 31 Oct 2026 18:53:32 +1100

calculus is the mathematics of change: it assigns exact meaning to two questions that stumped everyone from zeno to the seventeenth century — how fast is a quantity changing right now? and how much of it has accumulated so far? 𐃏 both questions are answered by the same primitive operation, the limit, applied in two different directions. everything on the pages below is a variation on that one move.

California Housing

Sat, 31 Oct 2026 18:53:32 +1100

Median house values for 20,640 California census block groups from the 1990 census. The modern default for “show me a real regression problem”: big enough to be non-trivial, small enough to fit anywhere, and full of instructive pathologies — capped targets, aggregate features, and spatial structure.

Provenance

Constructed by R. Kelley Pace and Ronald Barry for Sparse Spatial Autoregressions, Statistics and Probability Letters 33(3), 1997 — the point of the paper was spatial statistics, not machine learning. The data derive from the 1990 US census at the block group level (the smallest census unit, typically 600 to 3,000 people). It was long distributed via CMU’s StatLib archive; sklearn’s fetch_california_housing mirrors that original. A cosmetically extended variant (with an ocean_proximity categorical) is the running example in Geron’s Hands-On Machine Learning, chapter 2 — numbers from the two variants are not interchangeable.

Chess Bot

Sat, 31 Oct 2026 18:53:32 +1100

two chess projects live in this codebase, and honesty requires separating them up front. the first is a from-scratch javascript engine (in arcade/references/chess/js/, built following the classic bluefever software series): 120-square mailbox board, hand-rolled move generator validated by perft, material + piece-square evaluation, alpha-beta with quiescence, a principal-variation hash table, and iterative deepening. the second is voice chess (code/private/chess-bot/, published as github.com/abaj8494/ollama-voice-chess): there i did not write the engine — stockfish plays the moves over uci — and the engineering is in the wrapper: skill throttling, an ollama llm that provides spoken commentary without being allowed to hallucinate moves, neural tts, and spaced-repetition opening training. one project teaches you how engines work; the other teaches you what to build around an engine. 𐃏

CNN from scratch

Sat, 31 Oct 2026 18:53:32 +1100

a convolutional neural network is a feedforward network with its linear layers put on a diet: instead of every unit seeing every input, each unit sees a small local window, and every window is processed by the same small set of weights. 𐃏 this page builds the operation from its definition, gets the geometry formulas straight, walks the canonical architectures, and ends with a convolution written in loops and checked against scipy (Goodfellow, Ian, 2016).

Complex Analysis

Sat, 31 Oct 2026 18:53:32 +1100

calculus over \(\mathbb{C}\) is not a cosmetic upgrade of real analysis — it is a different subject with better theorems. asking a function of a complex variable to be differentiable once forces it to be differentiable infinitely often, equal to its taylor series, and rigid enough that its values on a tiny arc determine it everywhere. the payoff for this rigidity: integrals that real methods cannot touch fall to a residue computation in three lines (Brown, James W. and Churchill, Ruel V., 2009).

Concurrency

Sat, 31 Oct 2026 18:53:32 +1100

concurrency is structure: many logical tasks in flight, interleaved on however many cpus you have (possibly one). parallelism is hardware: tasks literally executing at the same instant. a single-core machine juggling 400 socket connections is concurrent, not parallel; a gpu multiplying matrices is parallel, barely concurrent. you design concurrency; you buy parallelism (Tanenbaum, Andrew S., 2008).

threads and shared memory

a thread is an independent stream of execution inside one address space: own stack and registers, shared everything else. the sharing is the point — and the disease.

Constraint

Sat, 31 Oct 2026 18:53:32 +1100

constraint programming inverts the usual deal: you state what a solution must satisfy, and a general-purpose solver figures out how to find one. 𐃏 no objective gradient, no simplex tableau — just variables, finite domains, and constraints, attacked by an alternation of inference (prune values that cannot appear in any solution) and search (guess, propagate, backtrack). this page builds the machinery from the formalism up; the sudoku solver and hashiwokakero write-ups on this wiki are the same machinery pointed at actual puzzles.

Containers

Sat, 31 Oct 2026 18:53:32 +1100

a container is not a small virtual machine. it is an ordinary linux process (tree) that the kernel has been told to lie to — about what processes exist, what the filesystem looks like, what the network is, who root is — plus an accountant capping what it may consume. the lying is namespaces, the accounting is cgroups, and everything else (images, registries, orchestrators) is packaging around those two syscall families (Tanenbaum, Andrew S., 2008).

Curse of Dimensionality

Sat, 31 Oct 2026 18:53:32 +1100

geometric intuition is trained in \(p \le 3\) and it does not survive the trip upstairs. 𐃏 in high dimensions the volume of a cube hides in its corners, every point is near the boundary, all pairwise distances look alike, and “local” neighbourhoods must stretch almost the full width of the space before they contain any data. every method that reasons from closeness — knn, kernel smoothers, rbf kernels — inherits these pathologies at once.

Databases

Sat, 31 Oct 2026 18:53:32 +1100

a database is a data structure that survives a power cut, shared by programs that don’t trust each other, queried in a language older than most of its users. the relational model has been declared dead roughly once a decade since 1970 and has outlived every announced successor. 𐃏 this page covers the model, the algebra underneath SQL, normalisation, the storage structures that make queries fast, and the machinery that keeps concurrent transactions honest.

Digits

Sat, 31 Oct 2026 18:53:32 +1100

The scikit-learn digits dataset: 1,797 tiny 8x8 greyscale images of handwritten digits, and the workhorse of every sklearn tutorial that needs a multiclass problem which loads instantly and fits in L2 cache.

Provenance

The data is the Optical Recognition of Handwritten Digits (optdigits) set from the UCI machine learning repository, created by E. Alpaydin and C. Kaynak at Bogazici University and donated in July 1998. It originates in Kaynak’s 1995 MSc thesis on combining multiple classifiers, and the companion paper is Alpaydin and Kaynak, Cascading Classifiers, Kybernetika 34(4), 1998.

Discrete Mathematics

Sat, 31 Oct 2026 18:53:32 +1100

discrete mathematics is the mathematics of structures that come in separate, countable pieces — integers, truth values, finite sets, graphs — as opposed to the smooth continuum of analysis. 𐃏 no limits, no derivatives; instead: induction, counting, and exact structure. it is the native mathematics of computation — every data structure, algorithm, database query and cryptographic protocol below this page is discrete mathematics wearing a lab coat (Epp, Susanna S., 2019).

Dynamic Programming

Sat, 31 Oct 2026 18:53:32 +1100

dynamic programming is two things wearing one name. to bellman it was a mathematical theory of multistage decision processes — sibling to linear programming in the “programming means planning” sense.¹ 𐃏 to a computer scientist it is a technique: solve a problem by combining solutions to subproblems, and never solve the same subproblem twice (Cormen, Thomas H. and Leiserson, Charles E. and Rivest, Ronald L. and Stein, Clifford, 2009). the two are the same idea at different altitudes, and this page covers both. worked implementations also live in this github repo.

Email SPAM Classifier

Sat, 31 Oct 2026 18:53:32 +1100

naive bayes is the classifier you get by taking bayes’ rule seriously and probability theory not seriously at all. 𐃏 it assumes every feature is independent of every other feature given the class — an assumption that is false for essentially all real data — and yet it filters spam, routes support tickets and triages documents well enough that it has survived five decades of fancier competition. this page derives it, counts why the “naive” part is the whole point, builds a spam filter from scratch, and is honest about where it breaks (its probabilities, not its decisions).

Ensemble Learning

Sat, 31 Oct 2026 18:53:32 +1100

one model is an opinion; a committee is an estimator. 𐃏 ensemble methods build many imperfect predictors and combine them, and the two great families attack opposite ends of the bias-variance decomposition: bagging averages low-bias, high-variance models to cancel their wobble; boosting stacks up high-bias, low-variance weak learners to build accuracy that none of them has alone.

Estimation & Inference

Sat, 31 Oct 2026 18:53:32 +1100

fitting a regression model produces numbers; inference is what licenses saying anything about them. this page collects the estimation machinery every model on this branch shares — likelihood, score, information, and the iterative algorithms that maximise them — and the two inferential regimes it feeds: exact small-sample \(t\) and \(F\) results in the linear gaussian model, and asymptotic wald, score and likelihood-ratio results everywhere else. 𐃏

Event Driven

Sat, 31 Oct 2026 18:53:32 +1100

event-driven architecture inverts the direction of knowledge: instead of the caller knowing who must react, the reactor knows what it cares about. producers announce facts; consumers subscribe. the whole style is the observer pattern with a broker in the middle and a network underneath — which is exactly where the interesting failure modes come from.

events, commands, queries

three message species, constantly confused:

event: a fact about the past, named in past tense — OrderPlaced, PaymentFailed. immutable, owned by the producer, zero expectation about who (if anyone) reacts. broadcasting is safe because nothing is being asked.
command: a request for the future — PlaceOrder. imperative mood, addressed to exactly one handler, can be rejected. commands express intent; events record outcome. compare the command pattern, which reifies exactly this.
query: a question, side-effect free, wants an answer now.

the classic smell is the command in event’s clothing: SendWelcomeEmailRequested published as an “event” that exactly one consumer must process, or the producer silently depends on a specific reaction — you have built rpc with extra steps and none of rpc’s error handling. 𐃏

Exact Inference

Sat, 31 Oct 2026 18:53:32 +1100

a bayesian network stores a joint distribution; inference is spending that investment — computing probabilities the tables do not state directly. exact inference is np-hard in general, but its true cost is exponential only in a structural parameter of the graph (the treewidth), not in the number of variables. this page builds the exact toolchain in the order the ideas force themselves on you: factors, variable elimination, the fight for a good elimination order, and finally the jointree algorithm — variable elimination industrialised into a message-passing scheme that answers every marginal at once. 𐃏

Functional Programming

Sat, 31 Oct 2026 18:53:32 +1100

unlike its siblings on this branch of the wiki — linear, quadratic, integer — this “programming” really is about writing programs. functional programming is the discipline of building software out of expressions that are evaluated rather than statements that are executed: no assignment, no mutation, no time. what remains is algebra, and algebra is something you can reason about. 𐃏

GAN: Generative Adversarial Networks

Sat, 31 Oct 2026 18:53:32 +1100

a gan trains a generator by making it play a game against a learned critic: the generator \(G\) maps noise to samples, the discriminator \(D\) tries to tell those samples from real data, and each improves by exploiting the other’s current weakness — density estimation recast as a two-player minimax game (goodfellow et al. 2014, generative adversarial networks). 𐃏 the framework is treated in ch. 20 of (Goodfellow, Ian, 2016).

Gaussian Mixture Models

Sat, 31 Oct 2026 18:53:32 +1100

a single gaussian is a committed statement: one bump, symmetric, thin tails. real data is usually several stories overlaid — different regimes, different subpopulations — and a gaussian mixture says so explicitly: each point was generated by one of \(k\) gaussians, we just don’t get told which. 𐃏 fitting one is the canonical latent-variable problem, and the algorithm that fits it — expectation-maximisation — is one of the great workhorses of statistics.

Generalised Linear Models

Sat, 31 Oct 2026 18:53:32 +1100

a generalised linear model is what remains of linear regression once you stop pretending every response is gaussian. nelder and wedderburn’s 1972 observation was that the gaussian, binomial, poisson, exponential and gamma models all run on the same three-part engine — a linear predictor, a link, and an exponential-family response — so estimation (fisher scoring), inference (deviance) and diagnostics (standardised residuals) can be built once and inherited by every member. 𐃏

Goal

Sat, 31 Oct 2026 18:53:32 +1100

most optimisation asks for the best; goal programming asks for good enough, several times over. 𐃏 you attach a numeric target to each of several objectives, measure how far the plan misses each target, and minimise the misses you dislike. the philosophy is herbert simon’s satisficing¹ — real decision makers do not maximise a grand utility function, they set aspiration levels and stop when they are met — and the machinery is pure linear programming: goal programming was invented by charnes and cooper as an LP device² and remains the most-used technique in practical multi-criteria decision making precisely because it never leaves LP territory.

Gödel's Incompleteness Theorem

Sat, 31 Oct 2026 18:53:32 +1100

In 1931 a twenty-five-year-old in Vienna published a paper proving that mathematics cannot fully formalise itself: any consistent formal system rich enough to do arithmetic contains true statements it cannot prove, and — worse — cannot even prove its own consistency (Gödel, Kurt, 1931). 𐃏 This page walks the actual argument: what the theorem says precisely, the two technical inventions that power it (Gödel numbering and diagonalisation), and — just as important — the long list of things it does not say.

Graph Theory

Sat, 31 Oct 2026 18:53:32 +1100

graph theory began in 1736 when euler noticed that a walking-tour puzzle about seven bridges in königsberg was really a question about dots and lines — and that the geometry (lengths, positions, the river) was irrelevant. 𐃏 that act of abstraction — keep the incidence structure, discard everything else — is the whole subject. a graph is the minimal mathematical model of “things, some pairs of which are related”, which is why it keeps turning up: networks, molecules, dependencies, state machines, social graphs (Epp, Susanna S., 2019).

Hashiwokakero (Bridges) Solver

Sat, 31 Oct 2026 18:53:32 +1100

hashiwokakero (“build bridges”, nikoli) hands you a grid of numbered islands and asks you to join them with bridges until every number is spent. it is the friendliest possible introduction to constraint satisfaction: the constraints are few and visual, propagation alone solves most human-published puzzles, and when it doesn’t, you get to write a backtracking search. this page documents my solver at code/private/hashi/ — a go rewrite of a uni assignment originally in c — including the debugging session that writing this page forced on it. 𐃏

IMDB Reviews

Sat, 31 Oct 2026 18:53:32 +1100

The Large Movie Review Dataset (aclImdb v1.0): 50,000 polar movie reviews from IMDB, split evenly for training and testing. For over a decade the default benchmark for binary sentiment classification — simple enough to load with one function call, large enough that word order and negation actually matter.

Provenance

Collected at Stanford and released alongside Maas, Daly, Pham, Huang, Ng and Potts, Learning Word Vectors for Sentiment Analysis, ACL 2011. The dataset page (and download) lives at ai.stanford.edu/~amaas/data/sentiment. The original motivation was learning sentiment-aware word representations; the review corpus it shipped with promptly outlived the method.

Integer Programming

Sat, 31 Oct 2026 18:53:32 +1100

take a linear program and add one word — integer — and the complexity class jumps from polynomial to NP-hard. that word buys expressive power nothing continuous can match: yes/no decisions, either/or logic, fixed costs, sequencing, assignment. integer programming is the lingua franca of operations research precisely because “decide” is not a convex verb. 𐃏

Internet Networks

Sat, 31 Oct 2026 18:53:32 +1100

the internet is a triumph of indirection: no layer trusts the one below to be reliable, timely, or even present, and yet a packet leaves your laptop, crosses a dozen autonomous systems owned by companies that actively dislike each other, and arrives. 𐃏 this page walks the stack bottom-up, then follows one HTTP request through DNS, TCP, and TLS to see every layer earn its keep.

K-means Clustering

Sat, 31 Oct 2026 18:53:32 +1100

k-means is unsupervised learning’s hello world: pick \(k\) prototype points, assign every datum to its nearest prototype, move each prototype to the centre of its flock, repeat. 𐃏 it is fast, it always terminates, and it is wrong in ways that are so instructive that every clustering course starts here anyway.

KDD Cup 1999

Sat, 31 Oct 2026 18:53:32 +1100

Network intrusion detection at competition scale: nearly five million TCP connection records, each labelled normal or with one of dozens of attack names. Historically the most-used intrusion-detection benchmark ever — and, by broad consensus, one that should now be used only with its flaws stated up front.

Provenance

The dataset was built for the Third International Knowledge Discovery and Data Mining Tools Competition (1999) (held with KDD-99, the fifth KDD conference) by processing the tcpdump portions of the 1998 DARPA Intrusion Detection System Evaluation, run by MIT Lincoln Laboratory. Traffic was generated on a closed simulated air-force network with hand-injected attacks over seven weeks (training) plus two weeks (test), then summarised into per-connection feature vectors.

Kernel Methods

Sat, 31 Oct 2026 18:53:32 +1100

kernel methods are the great arbitrage of classical machine learning: keep the algorithm linear — with all its convexity and closed forms — but run it in a feature space so large it can bend around anything, and never pay for that space explicitly. 𐃏 one identity powers everything: if your algorithm touches the data only through inner products, you may replace every \(\langle x, x’\rangle\) with a kernel \(k(x, x’)\) and thereby work in the implicit feature space of \(k\) — possibly infinite-dimensional — at the cost of an \(n \times n\) matrix.

LAN Messenger

Sat, 31 Oct 2026 18:53:32 +1100

“tessenger” is my unsw networks assignment: a multi-client chat server in raw c — server.c (583 lines) and client.c (253), no libraries beyond pthreads and the bsd socket api (Kernighan, Brian W. and Ritchie, Dennis M., 1988). it does authentication with timed lockout, private and group messaging, presence, audit logs, and a udp side channel for peer-to-peer file transfer. this page walks the architecture, weaves in the socket theory the assignment was designed to teach, and then does something the assignment never required: an honest concurrency audit of my own submission, with the bugs demonstrated live. 𐃏

Learning Graphical Models

Sat, 31 Oct 2026 18:53:32 +1100

everything so far assumed the network was handed to us. in practice an expert sketches the graph at best, and the numbers — sometimes the graph too — must come from data. learning splits along two axes: parameters vs structure, and complete vs incomplete data. the four quadrants ascend steeply in difficulty: counting, then optimisation, then search over graphs, then all three at once. 𐃏

Linear Algebra

Sat, 31 Oct 2026 18:53:32 +1100

linear algebra is the study of vector spaces and the structure-preserving maps between them. it is the one branch of mathematics that computers execute natively — every model fit, every graphics frame, every pagerank iteration is matrix arithmetic — and the local model that calculus reduces every smooth problem to. 𐃏 the plot of this page: spaces, then maps, then the four subspaces every matrix carries, then the two great factorisations — the spectral theorem and the svd (Anton, Howard, 2010).

Linear Programming

Sat, 31 Oct 2026 18:53:32 +1100

“programming” here means planning, not coding — the word predates the software sense. 𐃏 a linear program optimises a linear objective over a region carved out by linear inequalities. it is the base camp of mathematical programming: quadratic, integer and non-linear programming all generalise it in one direction or another, and all of them lean on LP machinery (relaxations, duals, warm starts) to get anything done. clrs devotes chapter 29 to it (Cormen, Thomas H. and Leiserson, Charles E. and Rivest, Ronald L. and Stein, Clifford, 2009).

Linux

Sat, 31 Oct 2026 18:53:32 +1100

unix is less an operating system than a worldview: everything is a file, every program does one thing, and text streams are the universal interface. linux is the worldview’s most successful implementation — a monolithic kernel started by a finnish undergraduate in 1991, now running most of the internet, every android phone, and the top 500 supercomputers without exception. 𐃏 this page is the trunk; the sharpened tools each get their own branch:

LLM from scratch

Sat, 31 Oct 2026 18:53:32 +1100

a large language model is a decoder-only transformer trained on one absurdly simple objective — predict the next token — scaled until the emergent behaviour stops looking simple. 𐃏 this page is the map from that objective to a deployed assistant: the loss, the scaling laws that size the model, the pretrain-align pipeline, and the inference tricks. the hands-on build is nanogpt.

Locally Weighted Regression

Sat, 31 Oct 2026 18:53:32 +1100

a straight line is too rigid for a wiggly world, and a global degree-9 polynomial is a hostage negotiation. 𐃏 locally weighted regression (LWR — and its robust cousin LOWESS) takes the diplomatic route: fit the simplest possible model, but fit it freshly at every query point, paying attention only to the training points nearby.

motivation

linear regression commits to one \(\theta\) for the whole input space. if the true \(f\) bends, the residue of that commitment is bias everywhere.
the fix need not be a fancier global family. any smooth function is locally linear — taylor says so — so a linear fit weighted toward a neighbourhood of \(x_0\) can track an arbitrary smooth \(f\).
the price: there is no longer a “trained model”. LWR is memory-based and non-parametric — like k-nearest neighbours, it keeps the entire training set and does all of its work at prediction time. training is \(O(1)\); every query costs a fresh weighted least-squares solve. 𐃏

the estimator

weighted least squares at a query point

fix a query \(x_0\). assign each training point a weight \(w_i(x_0) \ge 0\) that decays with distance from \(x_0\), then solve the weighted least-squares problem

Logic

Sat, 31 Oct 2026 18:53:32 +1100

logic is the study of valid inference: which conclusions are forced by which premises, independent of what the premises are about. 𐃏 it splits cleanly into a syntax (formulas, and rules for pushing them around) and a semantics (truth assignments, models), and the deepest theorems in the subject are exactly the ones that say when the two agree (Epp, Susanna S., 2019).

Logistic Regression

Sat, 31 Oct 2026 18:53:32 +1100

logistic regression is the method that seems only ever to be used for classification yet insists on calling itself regression. the resolution: it is regression — of the log-odds of a bernoulli success probability onto a linear predictor. 𐃏 this page develops it the honest way, as a generalised linear model: bernoulli response, canonical logit link, likelihood fitted by fisher scoring, inference through the deviance. the machine-learning reading (cross-entropy loss, linear decision boundaries) falls out at the end as a corollary.

Long Short-Term Memory (LSTM)

Sat, 31 Oct 2026 18:53:32 +1100

the vanilla rnn cannot learn long-range dependencies: its gradient signal is a product of jacobians that shrinks or blows up geometrically with distance. the lstm’s answer is architectural, not numerical — give the network a second state, updated additively rather than by repeated matrix multiplication, and let learned gates decide what enters, what stays, and what leaves. 𐃏 the design dates to hochreiter and schmidhuber’s 1997 paper (long short-term memory, neural computation 9(8)), and for two decades it was simply what “rnn” meant in practice (Goodfellow, Ian, 2016).

Markov Chains and Hidden Markov Models

Sat, 31 Oct 2026 18:53:32 +1100

time is just another dimension to factorise over. a bayesian network whose variables are indexed by time steps, with the same local structure copied at every step, is a dynamic bayesian network — and its two simplest members, the markov chain and the hidden markov model, carry a startling share of applied probability: language models before transformers, speech recognition, robot localisation, pagerank, and the mcmc machinery behind approximate inference. 𐃏

Markov Networks

Sat, 31 Oct 2026 18:53:32 +1100

some dependencies have no natural direction. two neighbouring pixels tend to share a label; two friends tend to vote alike; two adjacent atoms couple their spins. forcing an arrow onto these symmetric interactions — as a bayesian network must — misrepresents them. the markov network (markov random field, mrf) is the undirected alternative, born in statistical physics: edges express mutual compatibility, parametrised not by conditional probabilities but by unnormalised potentials, with a global normalising constant picking up the bill. 𐃏

Matplotlib Library

Sat, 31 Oct 2026 18:53:32 +1100

matplotlib has a reputation for being clunky, and the reputation is earned exactly by people fighting the wrong api. there are two: a stateful pyplot layer that mimics matlab (“draw on whatever was touched last”), and an object-oriented core in which every visible thing is an object you hold a reference to. the second is the real library; the first is sugar for one-liners. write fig, ax = plt.subplots() and stay in object land, and most of the clunk evaporates. 𐃏 all code on this page executed for real; figures were written to /tmp and are described rather than embedded, with true file sizes.

Microservices

Sat, 31 Oct 2026 18:53:32 +1100

a microservice architecture decomposes one application into many independently deployable services, each owning its own data, talking over a network. the honest framing: you are trading a code organisation problem for a distributed systems problem. 𐃏 that trade is sometimes worth it. it is not worth it nearly as often as conference talks suggest.

the trade, not the hype

what you actually buy

independent deployment: team a ships without waiting for team b’s release train. this is the real prize — everything else is secondary.
independent scaling: scale the search service to 40 replicas while billing idles on 2.
fault isolation: a memory leak in recommendations does not take down checkout — if you also build the resilience machinery below. isolation is earned, not free.
technology heterogeneity: the ml service can be python while the ledger is jvm. (also a curse: n stacks to patch.)
organisational scaling: conway’s law working for you — service boundaries that mirror team boundaries let teams own code end to end.

what you actually pay

every in-process function call that crosses a new service boundary becomes a network call: it can now fail independently, time out, arrive twice, or arrive late.
transactions that were a single BEGIN...COMMIT become sagas (below) — you give up atomicity across boundaries and must design compensation by hand.
refactoring across service boundaries is an order of magnitude harder than moving code between modules — the boundary is now an api contract with independent release cadences (Fowler, Martin, 2018).
operational surface explodes: per-service ci, dashboards, alerts, on-call, versioned contracts, backwards-compatible migrations.

when a modular monolith wins

a modular monolith — one deployable, strictly enforced internal module boundaries, one database with schema-per-module discipline — captures most of the design benefit at a fraction of the operational cost.
you get cheap refactoring while the domain boundaries are still wrong (they always start wrong), real stack traces, local transactions, and one thing to deploy.
the sane migration path is monolith-first: find the boundaries by living with them in-process, then extract the one or two services that genuinely need independent deployment or scaling — the strangler fig approach of routing traffic incrementally to extracted pieces. 𐃏
rule of thumb: if one team can hold the whole system in its head and deploys are not blocked on other teams, microservices solve a problem you do not have.

the same domain twice: one deployable with internal modules and one database, versus independently deployable services each owning its own store, fronted by a gateway.

decomposing a system

bounded contexts (ddd-lite)

the useful sliver of domain-driven design:

Multi-objective

Sat, 31 Oct 2026 18:53:32 +1100

single-objective optimisation is a polite fiction. real decisions trade cost against quality, return against risk, speed against accuracy — and the objectives disagree, otherwise you would not have listed them separately. 𐃏 multi-objective optimisation refuses to mash them into one number prematurely. the price of that honesty: “the optimum” stops being a point and becomes a set — the pareto front — and half the subject is about how to trace it, the other half about how to pick from it.

Multivariable Calculus

Sat, 31 Oct 2026 18:53:32 +1100

calculus in \(\mathbb{R}^n\): functions of several variables, the surfaces they define, and the fields that flow over them. the programme is the same as one variable — linearise locally, integrate globally — but the derivative becomes a matrix, the chain rule becomes matrix multiplication, and the fundamental theorem splits into three named theorems (green, stokes, gauss) that are secretly one (Courant, Richard, 1996). this page is also the mathematical spine of machine learning: gradients, hessians, jacobians and constrained optima are chapter 5 of (Deisenroth, Marc Peter and Faisal, A. Aldo and Ong, Cheng Soon, 2020).

No Free Lunch Theorem

Sat, 31 Oct 2026 18:53:32 +1100

averaged over all possible problems, every learning algorithm is exactly as good as random guessing — and every optimiser is exactly as good as blind enumeration. 𐃏 this sounds like nihilism but is actually the sharpest possible argument for inductive bias: an algorithm can only beat chance on some problems by losing to chance on others, so the whole game of machine learning is choosing whose lunch to eat.

Number Theory

Sat, 31 Oct 2026 18:53:32 +1100

number theory studies the integers on their own terms — divisibility, primes, remainders — and for two millennia it was the purest of pure mathematics, prized precisely because it was useless. 𐃏 then in 1977 the RSA cryptosystem turned euclid, fermat and euler into load-bearing internet infrastructure. this page walks the classical results in dependency order and ends at that payoff (Epp, Susanna S., 2019).

Numpy

Sat, 31 Oct 2026 18:53:32 +1100

numpy is one idea executed ruthlessly: a typed, contiguous block of memory plus enough metadata to reinterpret it without copying. everything else — broadcasting, views, vectorised speed — falls out of that design. learn the memory model and the api stops being a list of functions and becomes a handful of consequences. 𐃏 all timings and outputs on this page are real (python 3, numpy 2.4, apple m-series).

Pandas Library

Sat, 31 Oct 2026 18:53:32 +1100

pandas is numpy with labels. a Series is a 1-d array married to an index; a DataFrame is a dict of such columns sharing one row index. the single organising idea — the one that explains both the magic and the bugs — is that every operation aligns on labels first and computes second. everything on this page runs against pandas 3.0 (outputs are real). 𐃏

Perceptrons (Augmented with Gradient Descent)

Sat, 31 Oct 2026 18:53:32 +1100

the perceptron is the hydrogen atom of neural networks: one neuron, one weight vector, one threshold — and yet it already exhibits the two behaviours that define the whole field. 𐃏 it learns from mistakes with a provable convergence guarantee, and it fails on problems its geometry cannot express. this page covers both, then swaps the hard threshold for a sigmoid so that gradient descent can take over — a companion to the sign-loss perceptron page, which treats the classical algorithm on its own.

Performance Metrics for Machine Learning

Sat, 31 Oct 2026 18:53:32 +1100

a model is only as good as the number you judge it by, and most of the classic modelling disasters are really metric disasters — a fraud detector with \(99.9\%\) accuracy that never flags anything, a medical test tuned to a roc curve nobody deployed at the published threshold. 𐃏 this page is the field guide: what each metric measures, what it silently assumes, and which one to reach for when the classes are lopsided, the probabilities matter, or the target is continuous.

Policy Gradients

Sat, 31 Oct 2026 18:53:32 +1100

value-based methods (q-learning and family) learn how good actions are and act by argmax. policy-gradient methods skip the middleman: parameterise the policy itself, \(\pi_\theta(a \mid s)\), and do gradient ascent on expected return. 𐃏 the entire family — reinforce, actor-critic, trpo, ppo, and by extension rlhf — rests on one identity, the policy gradient theorem, whose derivation is three lines of calculus and one very good idea. the standard reference is sutton & barto, free at http://incompleteideas.net/book/the-book-2nd.html.

Principal Component Analysis (PCA)

Sat, 31 Oct 2026 18:53:32 +1100

pca is the linear algebra exam question that escaped into industry. given a cloud of points in \(\mathbb{R}^d\), it finds the orthogonal directions along which the cloud spreads the most, and lets you throw away the rest. 𐃏 two apparently different questions — “which directions carry the most variance?” and “which subspace loses the least when i project onto it?” — turn out to have the same answer, and that answer is an eigendecomposition.

Probabilistic Graphical Models

Sat, 31 Oct 2026 18:53:32 +1100

a probabilistic graphical model is a marriage of graph theory and probability: the graph records which variables talk to each other directly, and that structure alone dictates how compactly a joint distribution can be stored, which independencies it must satisfy, and how expensive it is to reason with. one set of ideas — factors, elimination, message passing, sampling — then serves diagnosis networks, spam filters, speech recognisers, image denoisers and gene-regulation maps alike.

Python

Sat, 31 Oct 2026 18:53:32 +1100

Data Types

category	types
text	`str`
numeric	`int`, `float`, `complex`
sequence	`list`, `tuple`, `range`
mapping	`dict`
set	`set`, `frozenset`
boolean	`bool`
binary	`bytes`, `bytearray`, `memoryview`
none	`NoneType`

Keywords

Python reserves 35 hard keywords (plus the soft keywords match and case for structural pattern matching since 3.10). Reserved keywords cannot be used as identifiers; soft keywords are only special in the relevant context.

Q-Learning

Sat, 31 Oct 2026 18:53:32 +1100

q-learning is the algorithm that made reinforcement learning feel inevitable: interact with an unknown world, nudge a table of numbers after every step, and the table converges to the value of optimal behaviour — even while you behave suboptimally the entire time. 𐃏 everything runs on one line of arithmetic, and the rest of this page is the machinery needed to say precisely why that line works. the canonical reference for all of it is sutton & barto’s reinforcement learning: an introduction, free at http://incompleteideas.net/book/the-book-2nd.html.

Quadratic Programming

Sat, 31 Oct 2026 18:53:32 +1100

promote the objective of a linear program from a plane to a bowl and you get quadratic programming: minimise a quadratic function over a polyhedron. it is the smallest step beyond LP, yet it captures a startling share of applied mathematics — support vector machines, portfolio selection, ridge regression, model-predictive control — because “squared penalty subject to linear rules” is how half the world states its preferences. 𐃏

Recurrent Neural Networks (RNNs)

Sat, 31 Oct 2026 18:53:32 +1100

feedforward networks eat fixed-size vectors. sequences — text, audio, sensor streams — have no fixed size, and worse, their order carries the meaning. the recurrent neural network solves both problems with one idea: maintain a hidden state that is updated by the same function at every time step. 𐃏 parameter count stops depending on sequence length, and the state becomes a lossy summary of everything seen so far (Goodfellow, Ian, 2016).

Regular Expressions

Sat, 31 Oct 2026 18:53:32 +1100

You should not be permitted to write production code if you do not have an journeyman licence in regular expressions or floating point math. —Rob Pike

a regular expression is two things wearing one syntax: a seventy-year-old theorem about finite automata, and the single most-used text-processing tool on unix. 𐃏 this page takes both seriously: the theory tells you exactly what the notation can and cannot express, and the theory’s failure modes (backtracking blowups, backreference NP-hardness, the html-parsing folklore) are precisely where practitioners get burned.

Regularised Regression

Sat, 31 Oct 2026 18:53:32 +1100

this page collects the closed-form solutions to regularised regression (where they exist) and the iterative approximations we fall back on (where they don’t). 𐃏 along the way we will see that regularisation is not an ad-hoc hack but a perfectly sensible artefact of estimation: it drops straight out of MAP (maximum a posteriori) inference once you put a prior on the coefficients.

Retrieval Augmented Generation

Sat, 31 Oct 2026 18:53:32 +1100

retrieval-augmented generation bolts a search engine onto a language model: fetch relevant documents at query time and paste them into the prompt, so the model answers from evidence rather than from its frozen weights (lewis et al. 2020, retrieval-augmented generation for knowledge-intensive nlp). 𐃏 it is the pragmatic alternative to baking every fact into an llm’s weights.

why retrieve at all

three problems that no amount of scaling fixes cleanly, and retrieval fixes cheaply:

Robust

Sat, 31 Oct 2026 18:53:32 +1100

every linear program you have ever written down was a lie: the coefficients came from measurements, forecasts and vendor spreadsheets, and the optimal vertex — sitting, by design, on the boundary of the feasible region — shatters the moment any of them wobbles. 𐃏 robust optimisation is the pessimist’s response: declare a set \(\mathcal{U}\) of realisations you refuse to be hurt by, and demand feasibility for every member of it. no distributions, no expectations, no scenarios — just a set and a worst case. the surprise, and the reason the field exists, is that this worst case can usually be folded back into a deterministic problem of the same (or nearly the same) complexity class.¹

Single-Variable Calculus

Sat, 31 Oct 2026 18:53:32 +1100

the calculus of one real variable, assembled in logical order: limits, then continuity and its two workhorse theorems (ivt, evt), then the derivative and the mean value theorem, taylor’s theorem with an honest error bound, and finally the riemann integral and both parts of the fundamental theorem. everything downstream — multivariable calculus, differential equations, every convergence argument in machine learning — leans on the theorems here. proofs are given where they are short and instructive; the deferred foundations live in real analysis (Courant, Richard, 1996).

Stable Diffusion Models

Sat, 31 Oct 2026 18:53:32 +1100

diffusion models generate by learning to undo noise: destroy an image with a fixed gaussian corruption process, train a network to reverse one small step of the destruction, then chain the reversals from pure noise back to data (ho et al. 2020, denoising diffusion probabilistic models). 𐃏 stable diffusion (rombach et al. 2022, high-resolution image synthesis with latent diffusion models) runs this machinery not on pixels but in the latent space of an autoencoder, with a text-conditioned u-net doing the denoising. first the maths, then the architecture.

Stochastic

Sat, 31 Oct 2026 18:53:32 +1100

a linear program assumes you know the data. stochastic programming admits that you do not — some coefficients are random — but insists you know their distribution, and asks for the decision that is best on average. 𐃏 the structural insight that makes this a paradigm rather than a hack: split the decision in two. commit to \(x\) now, before the coin is flipped; after uncertainty resolves, take a corrective recourse action \(y\) that adapts to whatever happened. the objective charges you for both, weighting the second stage by expectation.¹

Sudoku

Sat, 31 Oct 2026 18:53:32 +1100

sudoku is the drosophila of constraint satisfaction: small enough to hold in your head, rich enough to demonstrate every solving paradigm that matters. this page works through four of them against my actual code — a backtracking solver with \(O(1)\) constraint sets (arcade/references/sudoku/solver.py), a dart port that also generates puzzles (arcade-mobile), an integer-programming formulation solved for real with scipy, and the exact-cover view that leads to knuth’s algorithm x. every timing below is a real run on this machine.

Support Vector Machines (SVMs)

Sat, 31 Oct 2026 18:53:32 +1100

a linearly separable dataset admits infinitely many separating hyperplanes, and the perceptron will happily hand you whichever one it trips over first. 𐃏 the support vector machine asks a better question: of all the hyperplanes that separate the data, which one is farthest from everybody? the answer — the maximum-margin hyperplane — is determined by a handful of boundary points (the support vectors), drops out of a beautiful convex dual, and generalises via the kernel trick from lines to nearly anything.

The Babels

Sat, 31 Oct 2026 18:53:32 +1100

This page is for the richness of Babel.

There are at least four Babels worth keeping in one place: a real tower, a story about the tower, a library built out of the story, and a piece of software I use every day. They are all the same idea wearing different clothes — the totality of language, and what it costs.

the tower that actually stood

Babel was a real place before it was a metaphor. The name is Bab-ilim — “gate of the god” — Babylon, on the Euphrates, and its tower was the Etemenanki: “the temple of the foundation of heaven and earth,” a seven-tiered ziggurat of baked brick and bitumen sacred to Marduk, rising perhaps 90 metres over the city. It was old when history found it — damaged and rebuilt across centuries, most famously restored by Nebuchadnezzar II (r. 605–562 BCE), who recorded the work in glazed-brick boast: its top was to “rival the heavens.”

The Bias-Variance Decomposition

Sat, 31 Oct 2026 18:53:32 +1100

there is exactly one theorem in machine learning that every practitioner rederives on a whiteboard at least once a year, and this is it. 𐃏 the squared-error risk of any learned predictor splits into three non-negative pieces — irreducible noise, squared bias, and variance — and every design decision you make (model class, regularisation strength, \(k\), ensemble size, early stopping) is secretly a transaction between the last two.

The Roux Method

Sat, 31 Oct 2026 18:53:32 +1100

this is a working sheet for solving a 3x3 cube with the Roux method: hold the cube, find the picture that matches it, do the moves. everything here is drawn from Gilles Roux’s own description of his method, with his sequences machine-checked and every case drawn. where a sentence is his it is quoted; the rest is paraphrase.

the sheet assumes nothing beyond “I can turn the faces”. every symbol used is defined in the next section, and every word of cubing jargon is defined where it first appears and again in the glossary at the end.

The Story of Socrates

Sat, 31 Oct 2026 18:53:32 +1100

Socrates, Roman marble after a Greek bronze by Lysippos — the Louvre. He was, by every account including his own, remarkably ugly: snub nose, bulging eyes, the face of a satyr. Athens never forgave him for showing them that beauty was elsewhere.

Tokenisers

Sat, 31 Oct 2026 18:53:32 +1100

before a transformer sees a single number, text must be cut into pieces and each piece mapped to an integer. the tokeniser is that cut — the least glamorous and most consequential preprocessing step in the whole pipeline, since it fixes the vocabulary, the sequence length, and what the model can represent at all. 𐃏

Transformers

Sat, 31 Oct 2026 18:53:32 +1100

the transformer (Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N. and Kaiser, Lukasz and Polosukhin, Illia, 2017) deleted recurrence from sequence modelling and replaced it with a single primitive — attention — applied in parallel over the whole sequence. 𐃏 every token gets to look at every other token in one matrix multiply, the maximum path length between any two positions drops from \(O(n)\) to \(O(1)\), and training parallelises across the sequence dimension. this page derives the machinery; a from-scratch decoder-only build lives at nanogpt.

Ultimate Tic Tac Toe

Sat, 31 Oct 2026 18:53:32 +1100

ordinary tic-tac-toe is a solved bore — both players draw with a lookup table. glue nine boards together and add one rule about where you are allowed to move, and suddenly the game tree is deep enough that you need actual search theory: minimax, the negamax reformulation, alpha-beta pruning, and a heuristic to stand in for the leaves you cannot reach. this page documents the agent my partner and i wrote for unsw comp3411 (assignment 3, “nine-board tic-tac-toe”), warts and all — and the warts are instructive. 𐃏

Visual Transformers (ViT)

Sat, 31 Oct 2026 18:53:32 +1100

the vision transformer asks a blunt question: if attention replaced recurrence for text, can it replace convolution for images? the answer (dosovitskiy et al. 2020, an image is worth 16x16 words) is yes — chop the image into patches, treat each patch as a token, and feed the sequence to a standard transformer encoder with almost no vision-specific machinery. 𐃏

On the Statistics of Smoking

Sat, 31 Oct 2026 16:40:00 +1100

Everyone knows smoking is bad for you. This post is about a narrower question: what does it look like when you try to show that from data, with the tools a statistics student is actually taught? I take three small, classic datasets that ship with R, fit the models that are appropriate to each, and spend most of the time on the inference rather than the fitting: the Wald test and its cousins, where the asymptotics quietly fail, what a Bayesian reading adds, and what a multivariate test buys you when you have several outcomes at once. 𐃏 The code is shown in full and the output is baked into the page. If you want to re-run it, every block is self-contained from the top of the page down, with the seed set.

Solutions to Cormen, Leiserson, Rivest & Stein's Introduction to Algorithms

Thu, 22 Oct 2026 12:00:00 +1100

Solutions to every exercise and end-of-chapter problem in Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein’s Introduction to Algorithms, third edition (MIT Press, 2009): 956 section exercises and 158 problems, appendices included. Every algorithm comes with a correctness proof and a running-time bound, and every algorithm and numeric claim was checked by running it against brute force. The book itself: Introduction to Algorithms.

Chapter 1: The Role of Algorithms in Computing

1.1 Algorithms

Problem (1.1-1)

Describe a real-world situation that needs sorting, or one that needs the computation of a convex hull.

Solutions to Blum, Hopcroft & Kannan's Foundations of Data Science

Wed, 21 Oct 2026 12:00:00 +1100

Solutions to every end-of-chapter exercise in Foundations of Data Science by Avrim Blum, John Hopcroft and Ravindran Kannan (Cambridge University Press, 2020; freely available draft of 4 January 2018). Computational exercises were run for real in Python; the reported numbers and figures come from those runs. The book itself: Foundations of Data Science.

High-Dimensional Space

Exercises 2.1–2.7

Problem (2.1)

(1) Let \(x\) and \(y\) be independent random variables with uniform distribution in \([0,1]\). What is the expected value \(E(x)\), \(E(x^2)\), \(E(x-y)\), \(E(xy)\), and \(E(x-y)^2\)?

Solutions to Kleinberg & Tardos's Algorithm Design

Wed, 21 Oct 2026 12:00:00 +1100

Solutions to every end-of-chapter exercise in Jon Kleinberg and Éva Tardos’s Algorithm Design, published by Addison-Wesley in 2005. Each design exercise gets an algorithm, a proof that it is correct, and a running-time bound; the chapters’ Solved Exercises are worked in the book itself and are not repeated here. The book itself: Algorithm Design.

Introduction: Some Representative Problems

Exercises 1.1–1.7

Problem (1.1)

Decide whether you think the following statement is true or false. If it is true, give a short explanation. If it is false, give a counterexample.

Solutions to Lafaye de Micheaux, Drouilhet & Liquet's The R Software

Thu, 24 Sep 2026 12:00:00 +1000

Solutions to every exercise and every worksheet practical in The R Software: Fundamentals of Programming and Statistical Analysis by Pierre Lafaye de Micheaux, Rémy Drouilhet and Benoît Liquet (Springer, Statistics and Computing 40, 2013) — 164 end-of-chapter exercises and 57 worksheet practicals (353 numbered questions) across chapters 3 to 15, worked in R with the book’s own data sets, with executed code and committed output throughout. Chapter and exercise numbers follow the published Springer edition; worksheet practicals are labelled <chapter>.W, <chapter>.W.<practical letter> or <chapter>.W.<letter>.<study> (for example 15.W.B.2 is chapter 15, worksheet practical B, second study) so they never collide with exercise numbers. The book is filed at The R Software: Fundamentals of Programming and Statistical Analysis.

Solutions to Casella & Berger's Statistical Inference

Mon, 21 Sep 2026 11:00:00 +1000

Solutions to every exercise in Casella & Berger’s Statistical Inference (2nd edition, Duxbury, 2002) — 624 exercises across chapters 1–12, from the axioms of probability to ANOVA and regression. The book is filed at Statistical Inference.

Probability Theory

Exercises 1.1–1.7

Problem (1.1)

For each of the following experiments, describe the sample space.

Solutions to Gelman et al.'s Bayesian Data Analysis

Mon, 21 Sep 2026 10:00:00 +1000

Solutions to every exercise in Bayesian Data Analysis (3rd edition, CRC Press, 2013) by Gelman, Carlin, Stern, Dunson, Vehtari and Rubin — 215 exercises across chapters 1–23. The book is filed at Bayesian Data Analysis.

Probability and Inference

Exercises 1.1–1.7

Problem (1.1)

Conditional probability: suppose that if \(\omega = 1\), then \(y\) has a normal distribution with mean 1 and standard deviation \(\sigma\), and if \(\omega = 2\), then \(y\) has a normal distribution with mean 2 and standard deviation \(\sigma\). Also, suppose \(\Pr(\omega = 1) = 0.5\) and \(\Pr(\omega = 2) = 0.5\).

Solutions to Abbott's Understanding Analysis

Mon, 21 Sep 2026 09:00:00 +1000

Solutions to every exercise in Stephen Abbott’s Understanding Analysis (2nd edition, Springer UTM, 2015) — 467 exercises across chapters 1–8, from the construction of the real numbers to Fourier series. The book is filed at Understanding Analysis.

The Real Numbers

Exercises 1.2.1–1.2.7

Problem (1.2.1)

(a) Prove that \(\sqrt{3}\) is irrational. Does a similar argument work to show \(\sqrt{6}\) is irrational?

Solutions to Rosenthal's A First Look at Rigorous Probability Theory

Sat, 12 Sep 2026 09:00:00 +1000

Solutions to every exercise in Jeffrey S. Rosenthal’s A First Look at Rigorous Probability Theory (2nd edition, World Scientific, 2006) — 203 exercises across chapters 1–14, from probability triples and the extension theorem through martingales. The book is filed at A First Look at Rigorous Probability Theory.

The Need for Measure Theory

Exercises 1.3.1–1.3.5

Problem (1.3.1)

Suppose that \(\Omega = \{1,2\}\), with \(\mathbf{P}(\emptyset) = 0\) and \(\mathbf{P}\{1,2\} = 1\). Suppose \(\mathbf{P}\{1\} = \tfrac{1}{4}\). Prove that \(\mathbf{P}\) is countably additive if and only if \(\mathbf{P}\{2\} = \tfrac{3}{4}\).

Solutions to Darwiche's Modeling and Reasoning with Bayesian Networks

Mon, 07 Sep 2026 09:00:00 +1000

Solutions to every exercise in Adnan Darwiche’s Modeling and Reasoning with Bayesian Networks (Cambridge, 2009) — 342 exercises across chapters 2–18, from propositional logic and probability calculus through exact and approximate inference to parameter and structure learning. The book is the COMP9418 reference text and is filed at Modeling and Reasoning with Bayesian Networks.

Propositional Logic

Exercises 2.1–2.7

Problem (2.1)

Show that the following sentences are consistent by identifying a world that satisfies each sentence:

Solutions to Royden & Fitzpatrick's Real Analysis

Tue, 01 Sep 2026 01:00:00 +1000

Solutions to every problem in Royden & Fitzpatrick’s Real Analysis (fourth edition, 2010) — 1062 problems across the book’s 22 chapters: Lebesgue measure and integration for functions of a single real variable, differentiation and integration, the \(L^p\) spaces, metric and topological spaces, Banach and Hilbert spaces, and measure and integration in full generality. The book is filed at Real Analysis (Royden & Fitzpatrick).

The Real Numbers: Sets, Sequences, and Functions

Problems 1.1–1.7

Problem (1.1)

For \(a \neq 0\) and \(b \neq 0\), show that \((ab)^{-1} = a^{-1} b^{-1}\).

COMP9418 — Advanced Topics in Statistical Machine Learning at UNSW

Mon, 31 Aug 2026 08:00:00 +1000

I took COMP9418 in 2026 T2 as part of my Masters of Statistics. Despite the broad name, the course is really one long story about probabilistic graphical models: how to encode a joint distribution as a graph, how to answer queries against it exactly and approximately, and how to learn it from data. If you are deciding whether to enrol, or you are mid-term and drowning, this page collects everything I wish I had on day one.

MATH5806 — Applied Regression Analysis at UNSW

Mon, 31 Aug 2026 08:00:00 +1000

I took MATH5806 in 2026 T2 as part of my Masters of Statistics. The name undersells it: “applied regression analysis” here means generalised linear models, done properly — exponential families, link functions, iteratively reweighted least squares, deviance — with ordinary linear regression as the special case rather than the destination, and splines, Poisson and binomial regression along the way. Everything runs in R. This page is the admin, the textbook, and my notes, in one place.

Solutions to Tao's An Introduction to Measure Theory

Mon, 31 Aug 2026 01:00:00 +1000

Solutions to every exercise in Terence Tao’s An Introduction to Measure Theory (AMS GSM 126, 2011) — 237 exercises, woven inline through the text as Tao builds the theory through them. The book is the MATH5825 companion text and is filed at An Introduction to Measure Theory (Tao).

Notation

§0.0 — Notation

Problem (0.0.1)

If \((x_\alpha)_{\alpha \in A}\) is a collection of numbers \(x_\alpha \in [0,+\infty]\) such that \(\sum_{\alpha \in A} x_\alpha < \infty\), show that \(x_\alpha = 0\) for all but at most countably many \(\alpha \in A\), even if \(A\) itself is uncountable.

Solutions to Chiang's Networked Life

Sat, 29 Aug 2026 13:30:00 +1000

Solutions to all 100 homework problems (five per chapter, difficulty-starred, some deliberately open-ended) in Mung Chiang’s Networked Life: 20 Questions and Answers (Cambridge, 2012). Computational answers are worked as jupyter-python blocks with their committed results — numpy/scipy/networkx experiments you can re-run. Open-ended problems get a substantiated position argued from the chapter’s models rather than a hedge.

What makes CDMA work for my smartphone?

Problem 1.1 — Distributed power control

Problem (1.1)

Distributed power control.

Solutions to Axler's Linear Algebra Done Right

Sat, 29 Aug 2026 13:00:00 +1000

Solutions to every exercise in the fourth edition of Sheldon Axler’s Linear Algebra Done Right (Springer UTM, 2024) — 732 exercises across sections 1A–9D. The book is open access at linear.axler.net and is filed at Linear Algebra Done Right (Axler).

Vector Spaces

Exercises 1A

Problem (1A.1)

Show that \(\alpha + \beta = \beta + \alpha\) for all \(\alpha, \beta \in \mathbf{C}\).

Solutions to Axler's Measure, Integration & Real Analysis

Sat, 29 Aug 2026 12:30:00 +1000

Solutions to every exercise in Sheldon Axler’s Measure, Integration & Real Analysis (Springer GTM 282, 2020) — 587 exercises across sections 1A–12. The book is open access at measure.axler.net and is filed at Measure, Integration & Real Analysis (Axler).

Riemann Integration

Exercises 1A

Problem (1A.1)

Suppose \(f : [a, b] \to \mathbb{R}\) is a bounded function such that

Solutions to Bogachev's Measure Theory, Volume 1

Sun, 23 Aug 2026 20:10:00 +1000

Solutions to the exercises of V.I. Bogachev’s Measure Theory, Volume 1 (Springer, 2007) — exercises 1.12.47–160, 2.12.25–117, 3.10.29–125, 4.7.42–154, and 5.8.37–142. Exercises marked \(\circ\) in the book are the basic ones. The book itself is filed at Measure Theory (Bogachev).

Constructions and Extensions of Measures

Exercises 1.12.47–1.12.53

Problem (1.12.47)

(\(\circ\)) Suppose we are given a family of open sets in \(\mathbb{R}^n\). Show that this family contains an at most countable subfamily with the same union.

The Mechanics of a Flying Disc

Sun, 19 Jul 2026 00:00:00 +0000

An Ultimate disc is a wing you can catch. 𐃏 It has no tail, no fuselage, and no static stability whatsoever — by the standards of a fixed-wing aircraft it should tumble within the first few metres. Instead it flies a hundred, because it carries its stabiliser as angular momentum. This treatise works through the mechanics in four movements: the disc as a rigid body, the aerodynamic forces on it, the gyroscopic dance that shapes every throw, and the resulting flight envelope. The canonical sources are Hummel’s flight simulations (Hummel, Sarah Ann, 2003), the Manchester wind-tunnel programme of Potts and Crowther (Potts, Jonathan R. and Crowther, William J., 2002), and Lorenz’s instrumented discs (Lorenz, Ralph D., 2005).¹

V60 Pourovers

Sun, 19 Jul 2026 00:00:00 +0000

Everything about the Hario V60 is a refusal to make decisions on your behalf. It is a cone, some ribs, and a hole — no valve, no basket geometry slowing the water down, no reservoir metering the flow. The dripper controls nothing, which means the person holding the kettle controls everything: grind, temperature, ratio, and the shape of the pour itself. How the bean got to your kitchen in the first place is told in a history of coffee; this treatise is about the last four minutes of that thousand-year journey.

Solutions to Dobson & Barnett's An Introduction to Generalized Linear Models

Sun, 14 Jun 2026 13:42:58 +1100

Solutions to every exercise in the fourth edition of Dobson & Barnett’s An Introduction to Generalized Linear Models (Chapman & Hall/CRC, 2018) — 78 exercises across 14 chapters, worked in R with the book’s own datasets from the dobson package, with executed code and committed output throughout. This is the MATH5806 text; the book is filed at An Introduction to Generalized Linear Models (Dobson & Barnett).

Introduction

Problem 1.1 — Let Y1 and Y2 be independent random variables with

Problem (1.1)

Let \(Y_1\) and \(Y_2\) be independent random variables with \(Y_1 \sim N(1,3)\) and \(Y_2 \sim N(2,5)\). If \(W_1 = Y_1 + 2Y_2\) and \(W_2 = 4Y_1 - Y_2\), what is the joint distribution of \(W_1\) and \(W_2\)? (difficulty: \(\star\))

The Bayesian Cat

Sun, 08 Dec 2024 18:39:13 +1100

This blog post has been created to convince you that real-world probability, is in fact Bayesian probability.

Anyone who believes that a frequentist approach is superior may be correct (for that particular example), but it must be said that the Bayesian framework is a superset of this naive and trivial card-playing model of probability.

We are no longer trying to determine the probability of landing a double-six dice roll, and rather we are trying to figure out what the probability is that Mia (our cat) will be waiting for us on the porch when we get home.

Agentic on Aayush Bajaj's Augmenting Infrastructure

A Catalogue of Loss Functions

API Design

Approximate Inference

Autoencoders

Banagrams Solver

Bayesian Networks

Bazaar

Bazaar: Flashcards

CS Interview Prep#

Bazaar: Notes

STAT 571 — Probability & Measure#

Bazaar: Prints

Bazaar: reMarkable

Build Systems

the model: a dag of targets#

make: the actual semantics#

Calculus

California Housing

Provenance#

Chess Bot

CNN from scratch

Complex Analysis

Concurrency

threads and shared memory#

Constraint

Containers

Curse of Dimensionality

Databases

Digits

Provenance#

Discrete Mathematics

Dynamic Programming

Email SPAM Classifier

Ensemble Learning

Estimation & Inference

Event Driven

events, commands, queries#

Exact Inference

Functional Programming

GAN: Generative Adversarial Networks

Gaussian Mixture Models

Generalised Linear Models

Goal

Gödel's Incompleteness Theorem

Graph Theory

Hashiwokakero (Bridges) Solver

IMDB Reviews

Provenance#

Integer Programming

Internet Networks

K-means Clustering

KDD Cup 1999

Provenance#

Kernel Methods

LAN Messenger

Learning Graphical Models

Linear Algebra

Linear Programming

Linux

LLM from scratch

Locally Weighted Regression

motivation#

the estimator#

weighted least squares at a query point#

Logic

Logistic Regression

Long Short-Term Memory (LSTM)

Markov Chains and Hidden Markov Models

Markov Networks

Matplotlib Library

Microservices

the trade, not the hype#

what you actually buy#

what you actually pay#

when a modular monolith wins#

decomposing a system#

bounded contexts (ddd-lite)#

Multi-objective

Multivariable Calculus

CS Interview Prep

STAT 571 — Probability & Measure

the model: a dag of targets

make: the actual semantics

Provenance

threads and shared memory

Provenance

events, commands, queries

Provenance

Provenance

motivation

the estimator

weighted least squares at a query point

the trade, not the hype

what you actually buy

what you actually pay

when a modular monolith wins

decomposing a system

bounded contexts (ddd-lite)

Data Types

Keywords

why retrieve at all

the tower that actually stood

Chapter 1: The Role of Algorithms in Computing

1.1 Algorithms

High-Dimensional Space

Exercises 2.1–2.7

Introduction: Some Representative Problems

Exercises 1.1–1.7

Probability Theory

Exercises 1.1–1.7

Probability and Inference

Exercises 1.1–1.7

The Real Numbers

Exercises 1.2.1–1.2.7

The Need for Measure Theory

Exercises 1.3.1–1.3.5

Propositional Logic

Exercises 2.1–2.7

The Real Numbers: Sets, Sequences, and Functions

Problems 1.1–1.7

Notation

§0.0 — Notation

What makes CDMA work for my smartphone?