Worked solutions to all 215 exercises in Bayesian Data Analysis (3rd edition) by Gelman, Carlin, Stern, Dunson, Vehtari and Rubin — probability and inference, single- and multi-parameter models, hierarchical models, model checking and comparison, MCMC and Hamiltonian Monte Carlo, variational inference, regression and generalized linear models, and nonparametric Bayesian models.
Bayesian
2026-09-27
Bayesian Data Analysis (3rd edition, CRC Press, 2013) by Gelman, Carlin, Stern, Dunson, Vehtari and Rubin — the standard graduate reference for Bayesian modelling: single- and multi-parameter models, hierarchical models, model checking and comparison, posterior computation from MCMC to HMC and variational inference, regression, and nonparametric models.
Worked solutions to all 215 exercises live at Solutions to Gelman et al.’s Bayesian Data Analysis.
this page collects the closed-form solutions to regularised regression (where they exist) and the iterative approximations we fall back on (where they don’t). 𐃏 along the way we will see that regularisation is not an ad-hoc hack but a perfectly sensible artefact of estimation: it drops straight out of MAP (maximum a posteriori) inference once you put a prior on the coefficients.
The content here-in has been influenced by Mung Chiang’s Networked Life and Introduction to Algorithms by CLRS.
This page pairs well with Statistics.
Elements of Probability Theory
Definition
(Random Experiment, Sample Space, Events)
A random experiment has uncertain outcomes. The sample space \(S\) is the set of all possible outcomes. An event \(E\) is a subset of \(S\). The certain event is \(S\); the impossible event is \(\varnothing\).
Definition
(Probability Measure (Kolmogorov Axioms))
A probability space \((S,\mathcal{F},P)\) consists of a sample space \(S\), a \(\sigma\)-algebra \(\mathcal{F}\subseteq 2^S\), and a function \(P:\mathcal{F}\to[0,1]\) such that:
This blog post has been created to convince you that real-world probability, is in fact Bayesian probability.
Anyone who believes that a frequentist approach is superior may be correct (for that particular example), but it must be said that the Bayesian framework is a superset of this naive and trivial card-playing model of probability.
We are no longer trying to determine the probability of landing a
double-six dice roll, and rather we are trying to figure out what
the probability is that Mia (our cat) will be waiting for us on the
porch when we get home.