Regression

California Housing

2026-07-09
regressionlinear‑regressiongradient‑boosting

Median house values for 20,640 California census block groups from the 1990 census. The modern default for “show me a real regression problem”: big enough to be non-trivial, small enough to fit anywhere, and full of instructive pathologies — capped targets, aggregate features, and spatial structure.

Provenance

Constructed by R. Kelley Pace and Ronald Barry for Sparse Spatial Autoregressions, Statistics and Probability Letters 33(3), 1997 — the point of the paper was spatial statistics, not machine learning. The data derive from the 1990 US census at the block group level (the smallest census unit, typically 600 to 3,000 people). It was long distributed via CMU’s StatLib archive; sklearn’s fetch_california_housing mirrors that original. A cosmetically extended variant (with an ocean_proximity categorical) is the running example in Geron’s Hands-On Machine Learning, chapter 2 — numbers from the two variants are not interchangeable.

Read more >

23 Birthday Problems

2026-06-28
matrix‑calculuspower‑seriesprobabilitymleleast‑squaresregression

Solutions

Q1

a)

We use the update rule \[x_{n+1} = x_n - \frac{f(x_n)}{f’(x_n)}\]

Read more >

Statistics

2025-08-22 (updated: 2026-06-27)
meanvarianceinferenceregressionestimators

This page pairs well with Probability.

Table of Distributions

Distributionmass/density function\[S_X\]\[\mathbb{E}(X)\]\[\mathrm{Var}(X)\]\[\phi_X(s)\]
Bernoulli \[\mathrm{Bern}(\pi)\]\[P(X=1)=\pi\\ P(X=0)=1-\pi\]\[\{0,1\}\]\[\pi\]\[\pi(1-\pi)\]\[(1-\pi)+\pi e^{s}\]
Binomial \[\mathrm{Bin}(n,\pi)\]\[p_X(x)=\binom{n}{x}\pi^{x}(1-\pi)^{n-x}\]\[\{0,1,\dots,n\}\]\[n\pi\]\[n\pi(1-\pi)\]\[(1-\pi+\pi e^{s})^{n}\]
Geometric \[\mathrm{Geo}(\pi)\]\[p_X(x)=\pi(1-\pi)^{x-1}\]\[\{1,2,\dots\}\]\[\pi^{-1}\]\[(1-\pi)\pi^{-2}\]\[\frac{\pi}{e^{-s}-1+\pi}\]
Poisson \[\mathcal{P}(\lambda)\]\[p_X(x)=e^{-\lambda}\lambda^{x}/x!\]\[\{0,1,\dots\}\]\[\lambda\]\[\lambda\]\[\exp\{\lambda(e^{s}-1)\}\]
Uniform \[U[\alpha,\beta]\]\[f_X(x)=(\beta-\alpha)^{-1}\]\[[\alpha,\beta]\]\[\frac{1}{2}(\alpha+\beta)\]\[\frac{1}{12}(\beta-\alpha)^2\]\[\frac{e^{\beta s}-e^{\alpha s}}{s(\beta-\alpha)}\]
Exponential \[\mathrm{Exp}(\lambda)\]\[f_X(x)=\lambda e^{-\lambda x}\]\[[0,\infty)\]\[\lambda^{-1}\]\[\lambda^{-2}\]\[\frac{\lambda}{\lambda-s}\]
Gaussian \[\mathcal{N}(\mu,\sigma^{2})\]\[f_X(x)=\frac{1}{\sqrt{2\pi\sigma^{2}}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^{2}}\right\}\]\[\mathbb{R}\]\[\mu\]\[\sigma^{2}\]\[e^{\mu s+\frac{1}{2}\sigma^{2}s^{2}}\]
Gamma \[\Gamma(\alpha,\lambda)\]\[f_X(x)=\frac{1}{\Gamma(\alpha)}\lambda^{\alpha}x^{\alpha-1}e^{-\lambda x}\]\[[0,\infty)\]\[\alpha\lambda^{-1}\]\[\alpha\lambda^{-2}\]\[\left(\frac{\lambda}{\lambda-s}\right)^{\alpha}\]

Statistical Inference

Definition (Random Sample & Model)

Let \(X=(X_1,\ldots,X_n)\) be i.i.d. from a parametric family \(\{F_\theta:\theta\in\Theta\subset\mathbb{R}^p\}\). The parameter \(\theta\) is unknown; inference uses the randomness of \(X\) to learn about \(\theta\).

Read more >