geometric intuition is trained in \(p \le 3\) and it does not survive the trip upstairs. 𐃏 in high dimensions the volume of a cube hides in its corners, every point is near the boundary, all pairwise distances look alike, and “local” neighbourhoods must stretch almost the full width of the space before they contain any data. every method that reasons from closeness — knn, kernel smoothers, rbf kernels — inherits these pathologies at once.
Agentic
a database is a data structure that survives a power cut, shared by programs that don’t trust each other, queried in a language older than most of its users. the relational model has been declared dead roughly once a decade since 1970 and has outlived every announced successor. 𐃏 this page covers the model, the algebra underneath SQL, normalisation, the storage structures that make queries fast, and the machinery that keeps concurrent transactions honest.
The scikit-learn digits dataset: 1,797 tiny 8x8 greyscale images of handwritten digits, and the workhorse of every sklearn tutorial that needs a multiclass problem which loads instantly and fits in L2 cache.
Provenance
The data is the Optical Recognition of Handwritten Digits (optdigits) set from the UCI machine learning repository, created by E. Alpaydin and C. Kaynak at Bogazici University and donated in July 1998. It originates in Kaynak’s 1995 MSc thesis on combining multiple classifiers, and the companion paper is Alpaydin and Kaynak, Cascading Classifiers, Kybernetika 34(4), 1998.
discrete mathematics is the mathematics of structures that come in separate, countable pieces — integers, truth values, finite sets, graphs — as opposed to the smooth continuum of analysis. 𐃏 no limits, no derivatives; instead: induction, counting, and exact structure. it is the native mathematics of computation — every data structure, algorithm, database query and cryptographic protocol below this page is discrete mathematics wearing a lab coat (Epp, Susanna S., 2019).
dynamic programming is two things wearing one name. to bellman it was a mathematical theory of multistage decision processes — sibling to linear programming in the “programming means planning” sense.1 𐃏 to a computer scientist it is a technique: solve a problem by combining solutions to subproblems, and never solve the same subproblem twice (Cormen, Thomas H. and Leiserson, Charles E. and Rivest, Ronald L. and Stein, Clifford, 2009). the two are the same idea at different altitudes, and this page covers both. worked implementations also live in this github repo.
naive bayes is the classifier you get by taking bayes’ rule seriously and probability theory not seriously at all. 𐃏 it assumes every feature is independent of every other feature given the class — an assumption that is false for essentially all real data — and yet it filters spam, routes support tickets and triages documents well enough that it has survived five decades of fancier competition. this page derives it, counts why the “naive” part is the whole point, builds a spam filter from scratch, and is honest about where it breaks (its probabilities, not its decisions).
one model is an opinion; a committee is an estimator. 𐃏 ensemble methods build many imperfect predictors and combine them, and the two great families attack opposite ends of the bias-variance decomposition: bagging averages low-bias, high-variance models to cancel their wobble; boosting stacks up high-bias, low-variance weak learners to build accuracy that none of them has alone.
how regression models are estimated and judged, from MATH5806: score and fisher information, newton-raphson and the method of scoring, the wald/score/likelihood-ratio trio, and exact t and F inference in the linear gaussian model.
event-driven architecture inverts the direction of knowledge: instead of the caller knowing who must react, the reactor knows what it cares about. producers announce facts; consumers subscribe. the whole style is the observer pattern with a broker in the middle and a network underneath — which is exactly where the interesting failure modes come from.
events, commands, queries
three message species, constantly confused:
- event: a fact about the past, named in past tense —
OrderPlaced,PaymentFailed. immutable, owned by the producer, zero expectation about who (if anyone) reacts. broadcasting is safe because nothing is being asked. - command: a request for the future —
PlaceOrder. imperative mood, addressed to exactly one handler, can be rejected. commands express intent; events record outcome. compare the command pattern, which reifies exactly this. - query: a question, side-effect free, wants an answer now.
the classic smell is the command in event’s clothing: SendWelcomeEmailRequested published as an “event” that exactly one consumer must process, or the producer silently depends on a specific reaction — you have built rpc with extra steps and none of rpc’s error handling.
𐃏
exact inference in graphical models — factors, variable elimination, elimination orders and treewidth, the jointree (junction tree) algorithm with message passing, and mpe/map queries.