Svd

linear algebra is the study of vector spaces and the structure-preserving maps between them. it is the one branch of mathematics that computers execute natively — every model fit, every graphics frame, every pagerank iteration is matrix arithmetic — and the local model that calculus reduces every smooth problem to. 𐃏 the plot of this page: spaces, then maps, then the four subspaces every matrix carries, then the two great factorisations — the spectral theorem and the svd (Anton, Howard, 2010).

pca is the linear algebra exam question that escaped into industry. given a cloud of points in \(\mathbb{R}^d\), it finds the orthogonal directions along which the cloud spreads the most, and lets you throw away the rest. 𐃏 two apparently different questions — “which directions carry the most variance?” and “which subspace loses the least when i project onto it?” — turn out to have the same answer, and that answer is an eigendecomposition.