Agentic

a straight line is too rigid for a wiggly world, and a global degree-9 polynomial is a hostage negotiation. 𐃏 locally weighted regression (LWR — and its robust cousin LOWESS) takes the diplomatic route: fit the simplest possible model, but fit it freshly at every query point, paying attention only to the training points nearby.

motivation#

• linear regression commits to one \(\theta\) for the whole input space. if the true \(f\) bends, the residue of that commitment is bias everywhere.
• the fix need not be a fancier global family. any smooth function is locally linear — taylor says so — so a linear fit weighted toward a neighbourhood of \(x_0\) can track an arbitrary smooth \(f\).
• the price: there is no longer a “trained model”. LWR is memory-based and non-parametric — like k-nearest neighbours, it keeps the entire training set and does all of its work at prediction time. training is \(O(1)\); every query costs a fresh weighted least-squares solve. 𐃏

the estimator#

weighted least squares at a query point#

fix a query \(x_0\). assign each training point a weight \(w_i(x_0) \ge 0\) that decays with distance from \(x_0\), then solve the weighted least-squares problem

logic is the study of valid inference: which conclusions are forced by which premises, independent of what the premises are about. 𐃏 it splits cleanly into a syntax (formulas, and rules for pushing them around) and a semantics (truth assignments, models), and the deepest theorems in the subject are exactly the ones that say when the two agree (Epp, Susanna S., 2019).

the vanilla rnn cannot learn long-range dependencies: its gradient signal is a product of jacobians that shrinks or blows up geometrically with distance. the lstm’s answer is architectural, not numerical — give the network a second state, updated additively rather than by repeated matrix multiplication, and let learned gates decide what enters, what stays, and what leaves. 𐃏 the design dates to hochreiter and schmidhuber’s 1997 paper (long short-term memory, neural computation 9(8)), and for two decades it was simply what “rnn” meant in practice (Goodfellow, Ian, 2016).

matplotlib has a reputation for being clunky, and the reputation is earned exactly by people fighting the wrong api. there are two: a stateful pyplot layer that mimics matlab (“draw on whatever was touched last”), and an object-oriented core in which every visible thing is an object you hold a reference to. the second is the real library; the first is sugar for one-liners. write fig, ax = plt.subplots() and stay in object land, and most of the clunk evaporates. 𐃏 all code on this page executed for real; figures were written to /tmp and are described rather than embedded, with true file sizes.

a microservice architecture decomposes one application into many independently deployable services, each owning its own data, talking over a network. the honest framing: you are trading a code organisation problem for a distributed systems problem. 𐃏 that trade is sometimes worth it. it is not worth it nearly as often as conference talks suggest.

the trade, not the hype#

what you actually buy#

• independent deployment: team a ships without waiting for team b’s release train. this is the real prize — everything else is secondary.
• independent scaling: scale the search service to 40 replicas while billing idles on 2.
• fault isolation: a memory leak in recommendations does not take down checkout — if you also build the resilience machinery below. isolation is earned, not free.
• technology heterogeneity: the ml service can be python while the ledger is jvm. (also a curse: n stacks to patch.)
• organisational scaling: conway’s law working for you — service boundaries that mirror team boundaries let teams own code end to end.

what you actually pay#

• every in-process function call that crosses a new service boundary becomes a network call: it can now fail independently, time out, arrive twice, or arrive late.
• transactions that were a single BEGIN...COMMIT become sagas (below) — you give up atomicity across boundaries and must design compensation by hand.
• refactoring across service boundaries is an order of magnitude harder than moving code between modules — the boundary is now an api contract with independent release cadences (Fowler, Martin, 2018).
• operational surface explodes: per-service ci, dashboards, alerts, on-call, versioned contracts, backwards-compatible migrations.

when a modular monolith wins#

• a modular monolith — one deployable, strictly enforced internal module boundaries, one database with schema-per-module discipline — captures most of the design benefit at a fraction of the operational cost.
• you get cheap refactoring while the domain boundaries are still wrong (they always start wrong), real stack traces, local transactions, and one thing to deploy.
• the sane migration path is monolith-first: find the boundaries by living with them in-process, then extract the one or two services that genuinely need independent deployment or scaling — the strangler fig approach of routing traffic incrementally to extracted pieces. 𐃏
• rule of thumb: if one team can hold the whole system in its head and deploys are not blocked on other teams, microservices solve a problem you do not have.

decomposing a system#

bounded contexts (ddd-lite)#

the useful sliver of domain-driven design:

single-objective optimisation is a polite fiction. real decisions trade cost against quality, return against risk, speed against accuracy — and the objectives disagree, otherwise you would not have listed them separately. 𐃏 multi-objective optimisation refuses to mash them into one number prematurely. the price of that honesty: “the optimum” stops being a point and becomes a set — the pareto front — and half the subject is about how to trace it, the other half about how to pick from it.

calculus in \(\mathbb{R}^n\): functions of several variables, the surfaces they define, and the fields that flow over them. the programme is the same as one variable — linearise locally, integrate globally — but the derivative becomes a matrix, the chain rule becomes matrix multiplication, and the fundamental theorem splits into three named theorems (green, stokes, gauss) that are secretly one (Courant, Richard, 1996). this page is also the mathematical spine of machine learning: gradients, hessians, jacobians and constrained optima are chapter 5 of (Deisenroth, Marc Peter and Faisal, A. Aldo and Ong, Cheng Soon, 2020).