Shrinkage on Aayush Bajaj's Augmenting Infrastructurehttps://abaj.ai/tags/shrinkage/Recent content in Shrinkage on Aayush Bajaj's Augmenting InfrastructureHugoen© 2026 Aayush BajajThu, 09 Jul 2026 21:02:19 +1000Regularised Regressionhttps://abaj.ai/wiki/ml/supervised/regression/regularised/Thu, 09 Jul 2026 21:02:56 +1000https://abaj.ai/wiki/ml/supervised/regression/regularised/<p>this page collects the closed-form solutions to regularised regression (where they exist) and the iterative approximations we fall back on (where they don&rsquo;t).<span class="margin-note" data-note="ridge has a closed form. lasso does not, except in one charmed special case."> <span class="margin-note-indicator">𐃏</span> </span> along the way we will see that regularisation is not an ad-hoc hack but a perfectly sensible artefact of estimation: it drops straight out of MAP (maximum a posteriori) inference once you put a prior on the coefficients.</p>