Additive Models

Wed, 11 Nov 2026 14:00:00 +1100

an additive model replaces each linear term \(\beta_j x_j\) of linear regression with an unknown smooth function \(f_j(x_j)\), and adds them up. a generalised additive model (gam) does the same inside the link of a generalised linear model. the single-predictor smoother is the building block and lives on splines and smoothing; this page is about what changes when there are several smooths in one model: identifiability, one penalty per term, choosing several smoothing parameters at once, and the non-gaussian fit. 𐃏

Model Selection

Wed, 11 Nov 2026 14:00:00 +1100

every fitted model so far has been judged on the data it was fitted to, and that judge is bribed: least squares chose \(\hat\beta\) to make the training error small, so the training error flatters it. this page is about the two honest questions that follow — which of several candidate models to use, and how wrong the chosen one will be on new data — and the three families of answer: resample the data (cross-validation, the bootstrap), correct the training error analytically (\(C_p\), AIC, BIC), or search over subsets of predictors with one of those scores as the objective. 𐃏

Cross-Validation on Aayush Bajaj's Augmenting Infrastructure

Additive Models

Model Selection