Solutions to Gelman et al.’s Bayesian Data Analysis

(a) Averaging the conditional density over the two values of \(\omega\),

\begin{equation*} p(y) = \tfrac12 \mathrm{N}(y \mid 1, \sigma^2) + \tfrac12 \mathrm{N}(y \mid 2, \sigma^2), \end{equation*}

so at \(\sigma = 2\),

\begin{equation*} p(y) = \frac{1}{4\sqrt{2\pi}} \left[ e^{-(y-1)^2/8} + e^{-(y-2)^2/8} \right]. \end{equation*}

The sketch is a single symmetric bump centred at \(y = 1.5\), with \(p(1.5) = 0.193\), visually indistinguishable from a normal curve: the component means are separated by \(1 = \sigma/2\), far short of the \(2\sigma\) separation needed for an equally weighted two-component normal mixture to be bimodal. (Check! \(p^{\prime\prime}(1.5) < 0\).)

(b) \(\Pr(\omega = 1 \mid y = 1) = 0.531\). By Bayes’ rule the prior odds are 1, so the posterior odds are the likelihood ratio, and in general

\begin{equation*} \begin{aligned} \frac{\Pr(\omega = 1 \mid y)}{\Pr(\omega = 2 \mid y)} &= \frac{\exp\!\left(-(y-1)^2/(2\sigma^2)\right)} {\exp\!\left(-(y-2)^2/(2\sigma^2)\right)} \\ &= \exp\!\left(\frac{3 - 2y}{2\sigma^2}\right), \end{aligned} \end{equation*}

whence

\begin{equation*} \Pr(\omega = 1 \mid y) = \operatorname{logit}^{-1}\!\left(\frac{3-2y}{2\sigma^2}\right). \end{equation*}

At \(y = 1\), \(\sigma = 2\) the exponent is \(1/8\) and

\begin{equation*} \Pr(\omega = 1 \mid y = 1) = \frac{1}{1 + e^{-1/8}} = 0.5312. \end{equation*}

(c) The posterior is the logistic curve above, evaluated at the argument \((3-2y)/(2\sigma^2)\), i.e. a logistic function of \(y\) with slope \(-1/\sigma^2\) centred at the midpoint \(y = 1.5\). As \(\sigma\) increases the slope flattens and \(\Pr(\omega = 1 \mid y) \to \tfrac12\) for every fixed \(y\): the data become uninformative and the posterior reverts to the prior. As \(\sigma\) decreases the curve steepens to a step at \(y = 1.5\), \(\Pr(\omega = 1 \mid y) \to 1\) for \(y < 1.5\) and \(\to 0\) for \(y > 1.5\): \(\omega\) is identified by which mean \(y\) falls nearer.

Both derivations in Section 1.8 go through verbatim, the scalar square \(u^2\) being replaced by the outer product \(uu^{T}\); assume \(E(u^{T}u) < \infty\) so every entry below is finite.

For (1.8) the integrand is a vector and integration is entrywise, so the book’s factorization applies to each component at once:

\begin{equation*} \begin{aligned} E(u) &= \int\!\!\int u\, p(u,v)\,du\,dv \\ &= \int \left[ \int u\, p(u \mid v)\,du \right] p(v)\,dv \\ &= \int E(u \mid v)\, p(v)\,dv = E(E(u \mid v)). \end{aligned} \end{equation*}

Nothing above used the dimension of \(v\) either.

For (1.9), write both variances in the form \(\operatorname{var}(w) = E(ww^{T}) - E(w)E(w)^{T}\) and expand:

\begin{equation*} \begin{aligned} &E(\operatorname{var}(u \mid v)) + \operatorname{var}(E(u \mid v)) \\ &\quad = E\!\left[ E(uu^{T} \mid v) - E(u \mid v)E(u \mid v)^{T} \right] \\ &\qquad + E\!\left[ E(u \mid v)E(u \mid v)^{T} \right] - E(u)E(u)^{T} \\ &\quad = E(uu^{T}) - E(u)E(u)^{T} = \operatorname{var}(u). \end{aligned} \end{equation*}

The second-last equality is (1.8) applied entrywise to the matrix-valued \(uu^{T}\), and (1.8) applied to \(u\) is what makes the outer variance’s mean term \(E(u)E(u)^{T}\).

Everything follows from \(q = p/(1+p)\), the probability that a brown-eyed parent transmits an \(x\) gene: a brown-eyed individual is \(Xx\) or \(XX\) with the population proportions renormalized,

\begin{equation*} \Pr(Xx \mid \text{brown}) = \frac{2p(1-p)}{1 - p^{2}} = \frac{2p}{1+p}, \end{equation*}

and only a heterozygote can transmit \(x\), doing so with probability \(\tfrac12\), so

\begin{equation*} q = \tfrac12 \cdot \frac{2p}{1+p} = \frac{p}{1+p}. \end{equation*}

Under random mating the two brown-eyed parents transmit independently, so the child is \(xx\) with probability \(q^{2}\) and \(Xx\) with probability \(2q(1-q)\); conditioning on the child being brown,

\begin{equation*} \begin{aligned} \Pr(Xx \mid \text{brown child of brown parents}) &= \frac{2q(1-q)}{1-q^{2}} = \frac{2q}{1+q} \\ &= \frac{2p/(1+p)}{(1+2p)/(1+p)} = \frac{2p}{1+2p}, \end{aligned} \end{equation*}

which is the required proportion. (The last step used \(1 + q = (1+2p)/(1+p)\).)

Judy’s prior is therefore \(\pi = \Pr(Xx) = 2p/(1+2p)\), with \(\Pr(XX) = 1/(1+2p)\). Her husband is \(Xx\). If Judy is \(Xx\) each child is \(xx\) with probability \(\tfrac14\), so \(n\) brown-eyed children have probability \((3/4)^n\); if Judy is \(XX\) every child is brown, probability 1. Bayes’ rule, with the common factor \(1/(1+2p)\) cancelling, gives

\begin{equation*} \pi_n := \Pr(Xx \mid n \text{ brown children}) = \frac{2p\,(3/4)^{n}}{2p\,(3/4)^{n} + 1}. \end{equation*}

For the grandchild, first update Judy’s eldest child, who is brown-eyed. (i) If Judy is \(Xx\), the cross \(Xx \times Xx\) gives \(xx, Xx, XX\) with probabilities \(\tfrac14, \tfrac12, \tfrac14\), so the brown child is \(Xx\) with probability \((1/2)/(3/4) = 2/3\). (ii) If Judy is \(XX\), the cross \(XX \times Xx\) gives \(Xx\) or \(XX\) each with probability \(\tfrac12\), all brown, so the child is \(Xx\) with probability \(\tfrac12\). Averaging over the posterior,

\begin{equation*} \Pr(\text{child } Xx \mid \text{data}) = \tfrac23 \pi_n + \tfrac12 (1 - \pi_n) = \tfrac12 + \tfrac16 \pi_n . \end{equation*}

Take the child’s mate to be a random member of the population, which transmits \(x\) with probability \(p^{2} + \tfrac12 \cdot 2p(1-p) = p\). The grandchild is blue-eyed only if it receives \(x\) from both, and the child transmits \(x\) with probability \(\tfrac12\) when \(Xx\) and never when \(XX\):

\begin{equation*} \begin{aligned} \Pr(\text{grandchild } xx \mid \text{data}) &= \tfrac12 \left( \tfrac12 + \tfrac16 \pi_n \right) p \\ &= \frac{p}{4} + \frac{p\,\pi_n}{12}. \end{aligned} \end{equation*}

(a) Counting the twelve outcomes: eight are positive and five are at least 8 (namely 13, 15, 16, 20, 21), and those five are among the eight wins, so

\begin{equation*} \begin{aligned} \Pr(\text{fav. wins}) &= \tfrac{8}{12} = 0.67, \\ \Pr(\text{fav. wins by} \ge 8) &= \tfrac{5}{12} = 0.42, \\ \Pr(\text{fav. wins by} \ge 8 \mid \text{fav. wins}) &= \tfrac{5}{8} = 0.63, \end{aligned} \end{equation*}

the third being the ratio of the second to the first, as the event “wins by at least 8” is contained in “wins.”

(b) Section 1.6 fits \(d = (\text{outcome}) - (\text{point spread})\) as \(d \mid x \sim \mathrm{N}(0, 14^2)\), independent of the spread \(x\) (sample mean \(0.07\), sample s.d. \(13.86\) over the 672 games). At \(x = 8\), the favorite wins when \(\text{outcome} > 0\), i.e. \(d > -8\), and wins by at least 8 when \(d \ge 0\):

\begin{equation*} \begin{aligned} \Pr(\text{fav. wins}) &= \Phi\!\left(\tfrac{8}{14}\right) = \Phi(0.571) = 0.72, \\ \Pr(\text{fav. wins by} \ge 8) &= \Phi(0) = 0.50, \\ \Pr(\text{fav. wins by} \ge 8 \mid \text{fav. wins}) &= \frac{0.50}{0.7161} = 0.70 . \end{aligned} \end{equation*}

This is the convention of Section 1.6 itself, which assigns \(\Pr_{\text{norm}}(y > 0 \mid x) = \Phi(x/14)\) with no correction for the integer-valued scores; restoring a continuity correction, so that “wins” is \(y \ge 1\) and “wins by at least 8” is \(y \ge 8\), gives instead \(\Phi(7.5/13.86) = 0.71\), \(\Phi(0.5/13.86) = 0.51\) and \(0.73\).

Roughly one chance in 200: about \(0.004\) from general knowledge, about \(0.006\) from the historical near-tie counts.

(a) The information used: of the 435 seats, the great majority are safe, and in a typical year only some 40 races are competitive enough that the winning margin could plausibly have landed within a few thousand votes; a competitive race draws on the order of 200,000 votes; and for such a race the margin \(D\) near zero is spread out over a window of some \(\pm 5{,}000\) votes with no reason to favour one value in that window over another, so its density near zero is about \(1/10{,}000\) per vote. Hence for a competitive race

\begin{equation*} \Pr(D = 0) \approx 10^{-4}, \end{equation*}

and with the safe races contributing essentially nothing,

\begin{equation*} \Pr(\text{some tie}) \approx 40 \times 10^{-4} = 0.004 . \end{equation*}

(b) Let \(f\) denote the probability density (per vote) of the margin \(D\) at zero, taken as constant over a window of \(\pm 100\) votes, which the data below justify. One wrinkle must be handled: with turnout \(N\) fixed, \(D \equiv N \pmod 2\), so \(D\) moves on a lattice of spacing 2, each attainable value carrying probability \(2f\), and a tie is outright impossible when \(N\) is odd. The nonzero values in the window \(|D| < 10\) number 8 if \(N\) is even (\(\pm 2, \pm 4, \pm 6, \pm 8\)) and 10 if \(N\) is odd (\(\pm 1, \pm 3, \pm 5, \pm 7, \pm 9\)), so averaging over the parity of \(N\),

\begin{equation*} \Pr(0 < |D| < 10) \approx 9 \cdot 2f = 18 f = \frac{6}{20597}, \end{equation*}

while an exact tie requires \(N\) even and then \(D = 0\):

\begin{equation*} \Pr(D = 0) = \tfrac12 \cdot 2f = f = \frac{6/20597}{18} = 1.6 \times 10^{-5}. \end{equation*}

The parity correction has cancelled: 18 is also the naive count of nonzero integers in the window. The same argument applied to the wider window gives the companion estimate

\begin{equation*} \Pr(D = 0) = \frac{49/20597}{198} = 1.2 \times 10^{-5}, \end{equation*}

the two agreeing to within 30%, which is as much as the flat-density assumption deserves. Taking \(f \approx 1.4 \times 10^{-5}\) per race, or about 1 in 70,000, and treating the 435 races as independent for this purpose (a national swing shifts all margins together but does not change a first-order count of near-ties),

\begin{equation*} \begin{aligned} \Pr(\text{some tie}) &= 1 - (1 - 1.4 \times 10^{-5})^{435} \\ &\approx 435 \times 1.4 \times 10^{-5} = 0.006 , \end{aligned} \end{equation*}

one chance in about 165, the two window-based estimates bracketing this at \(0.005\) and \(0.007\).

\(5/11 = 0.45\).

Write \(I\) and \(F\) for the events that the birth was an identical-twin birth and a fraternal-twin birth (reading the quoted rates as fractions of birth events), and let \(E\) be the datum that Elvis, a boy, had a twin brother. The prior odds are

\begin{equation*} \frac{\Pr(I)}{\Pr(F)} = \frac{1/300}{1/125} = \frac{5}{12}, \end{equation*}

and, given that Elvis is a boy, an identical co-twin is a boy with certainty while a fraternal co-twin is a boy with probability \(\tfrac12\):

\begin{equation*} \frac{\Pr(E \mid I)}{\Pr(E \mid F)} = \frac{1}{1/2} = 2 . \end{equation*}

Multiplying, the posterior odds are \(\tfrac{5}{12} \cdot 2 = \tfrac{5}{6}\), so

\begin{equation*} \Pr(I \mid E) = \frac{5}{5+6} = \frac{5}{11} = 0.4545 . \end{equation*}

Switch: switching wins with probability \(2/3\), staying with probability \(1/3\).

Label the boxes so that the contestant picks box 1, let \(\theta \in \{1,2,3\}\) be the box holding the prize, with \(\Pr(\theta = j) = 1/3\), and let \(M\) be the box Monty opens. Monty’s behaviour is the model: he always opens a box that is neither box 1 nor the prize box, and when \(\theta = 1\) he is free to choose, opening box 3 with probability \(q\). Then

\begin{equation*} \Pr(M = 3 \mid \theta = 1) = q, \quad \Pr(M = 3 \mid \theta = 2) = 1, \quad \Pr(M = 3 \mid \theta = 3) = 0, \end{equation*}

and Bayes’ rule with the uniform prior gives

\begin{equation*} \begin{aligned} \Pr(\theta = 1 \mid M = 3) &= \frac{\tfrac13 q}{\tfrac13 q + \tfrac13 \cdot 1 + \tfrac13 \cdot 0} = \frac{q}{1+q}, \\ \Pr(\theta = 2 \mid M = 3) &= \frac{1}{1+q}, \end{aligned} \end{equation*}

the first being the winning probability for staying and the second for switching. The contestant has no information distinguishing boxes 2 and 3, so \(q = \tfrac12\) and these are \(1/3\) and \(2/3\).

The conditioning is the whole content. The datum is not “box 3 holds a lesser prize,” which would leave boxes 1 and 2 symmetric at \(1/2\) each; it is “Monty, who is barred from opening box 1 and from revealing the prize, opened box 3.” That rule makes \(M = 3\) certain when the prize is in box 2 but only a coin flip when it is in box 1, which is exactly the likelihood ratio 2 in favour of switching.

The conditional reading is exact in (a) and formal only in (b): in (a) the two persons share one probability model and differ by a conditioning event, in (b) they differ by the model itself.

(a) With \(E = \{\text{die shows } 6\}\), \(I_B\) the knowledge that a fair die was rolled and \(I_A = I_B\) together with the observed face,

\begin{equation*} \begin{aligned} P_B(E) &= P(E \mid I_B) = \tfrac{1}{6}, \\ P_A(E) &= P(E \mid I_A) = \mathbf{1}\{\text{the face was } 6\} \in \{0,1\}, \end{aligned} \end{equation*}

unequal with neither person irrational. They are coherent, since \(E[P_A(E) \mid I_B] = P(E \mid I_B) = 1/6\): \(B\)’s probability is \(B\)’s expectation of \(A\)’s, and \(B\) would adopt \(A\)’s value on being told \(I_A\). The physical symmetry of the die makes \(1/6\) the uniquely reasonable value given \(I_B\) (Section 1.5), so the subjectivity is about information only, not judgment.

(b) There is no single model \(P\) of which \(P_A\) and \(P_B\) are two conditionings: no symmetry argument and no replication of “the next World Cup”. \(A\) and \(B\) differ not only in data (squad quality, injuries, the draw, betting markets) but in the model linking that data to a winner, and two equally informed fans may still disagree, which cannot happen in (a). What survives is the betting definition of Section 1.5: \(P_A(E)\) is the \(p\) at which \(A\) is indifferent between the two bets, and nothing forces \(A\) and \(B\) to name the same \(p\). Such probabilities are judged by calibration over many statements, as in the football point-spread example of Section 1.6, with \(B\)’s extra knowledge showing up as sharper but still calibrated forecasts; and \(A\), believing \(I_A \subset I_B\), may rationally defer and adopt \(P_B(E)\).

In one simulated day the office saw 31 patients, of whom 2 had to wait, with mean wait 1.3 minutes among those two, and it closed at 4:05 p.m.; over 100 independent days the medians are 42 patients, 6 waiters, a 3.7-minute mean wait among waiters, and a 4:05 p.m. closing time.

The model is an \(M/G/3\) queue on the fixed admission window \([0,T]\), \(T = 420\) minutes, served first-come-first-served. Arrival times are the points of a Poisson process of rate \(\lambda = 1/10\) per minute, generated as the partial sums

\begin{equation*} t_k = \sum_{i=1}^{k} X_i, \qquad X_i \stackrel{\text{iid}}{\sim} \text{Expon}(1/10), \end{equation*}

kept while \(t_k \le T\); service times are \(S_k \stackrel{\text{iid}}{\sim} \text{U}(5,20)\), independent of the arrivals. With \(c = 3\) doctors and \(E[S] = 12.5\) the utilization is \(\rho = \lambda E[S]/c = 5/12\), so the queue is lightly loaded and most patients are seen at once.

The recursion is the only thing needed. Let \(f_j\) denote the time doctor \(j\) next becomes free, initialized at \(f_j = 0\). Patient \(k\) arriving at \(t_k\) goes to the doctor who is free earliest, \(j_k = \arg\min_j f_j\), and starts at

\begin{equation*} b_k = \max(t_k, f_{j_k}), \qquad W_k = b_k - t_k, \qquad f_{j_k} \leftarrow b_k + S_k . \end{equation*}

The number of patients is \(N = \max\{k : t_k \le T\}\), the number who waited is \(\#\{k : W_k > 0\}\), and the closing time is \(\max_k (b_k + S_k)\). Since a patient can only be delayed when all three doctors are busy, \(W_k > 0\) exactly when \(\min_j f_j > t_k\), which is what the recursion tests.

(a) The first of the 100 replicate days of part (b):

(b) Repeating the day 100 times independently, the estimands are the median and the central 50% interval (25th and 75th percentiles) of each summary over days; the last column is the bootstrap standard error of the median estimate across the 100 replicates, i.e. the Monte Carlo error from using 100 days rather than infinitely many.

The closing-time interval is \([420.4, 429.8]\) minutes after 9 a.m.; at this Monte Carlo precision the counts are good to about one patient and the mean waits to a few seconds, so 100 replications support the medians above but not a third digit.

\begin{equation*} p(\theta \mid y < 3) \;\propto\; \theta^{3}(1-\theta)^{11}\bigl(36\theta^{2}+8\theta+1\bigr), \qquad 0 \le \theta \le 1 . \end{equation*}

The observation is the event \(\{y<3\}\), so the likelihood is its probability under \(y\mid\theta \sim \mathrm{Bin}(10,\theta)\):

\begin{equation*} \begin{aligned} \Pr(y<3\mid\theta) &=\sum_{k=0}^{2}\binom{10}{k}\theta^{k}(1-\theta)^{10-k}\\ &=(1-\theta)^{10}+10\theta(1-\theta)^{9}+45\theta^{2}(1-\theta)^{8}\\ &=(1-\theta)^{8}\bigl[(1-\theta)^{2}+10\theta(1-\theta)+45\theta^{2}\bigr]\\ &=(1-\theta)^{8}\bigl(36\theta^{2}+8\theta+1\bigr). \end{aligned} \end{equation*}

Multiplying by the \(\mathrm{Beta}(4,4)\) density \(p(\theta)\propto\theta^{3}(1-\theta)^{3}\) gives the display above, whose normalizing constant is

\begin{equation*} \Bigl[\int_{0}^{1}\theta^{3}(1-\theta)^{11}\bigl(36\theta^{2}+8\theta+1\bigr)d\theta\Bigr]^{-1} =\tfrac{7735}{8}=966.875 , \end{equation*}

by \(\int_{0}^{1}\theta^{a-1}(1-\theta)^{b-1}d\theta=B(a,b)\) applied to the three monomials.

Term by term the posterior is the beta mixture

\begin{equation*} \tfrac{13}{128}\,\mathrm{Beta}(\theta\mid 4,14) +\tfrac{5}{16}\,\mathrm{Beta}(\theta\mid 5,13) +\tfrac{75}{128}\,\mathrm{Beta}(\theta\mid 6,12), \end{equation*}

with weights proportional to \(\binom{10}{k}B(4+k,14-k)\), \(k=0,1,2\) (Check!), so \(E(\theta\mid y<3)=\tfrac{39}{128}=0.305\) and the mode is \(\theta=0.281\).

Sketch: a single hump on \([0,1]\), zero at both endpoints, rising steeply to its peak at \(\theta\approx0.28\) and decaying with a mild right tail that is negligible beyond \(\theta=0.7\).

\begin{equation*} E(N\mid TT)=\tfrac{175}{78}=2.244 . \end{equation*}

Two tails is evidence for the tail-heavy coin \(C_2\). With \(\Pr(C_1)=\Pr(C_2)=\tfrac12\), Bayes’ rule gives

\begin{equation*} \begin{aligned} \Pr(C_1\mid TT)&=\frac{(0.4)^{2}}{(0.4)^{2}+(0.6)^{2}}=\frac{0.16}{0.52}=\tfrac{4}{13},\\ \Pr(C_2\mid TT)&=\tfrac{9}{13}. \end{aligned} \end{equation*}

Given the coin, the spins are independent, so the number \(N\) of additional spins up to and including the first head is geometric with mean \(1/\Pr(\text{heads}\mid C)\); by (1.8),

\begin{equation*} \begin{aligned} E(N\mid TT)&=\tfrac{4}{13}\cdot\tfrac{1}{0.6}+\tfrac{9}{13}\cdot\tfrac{1}{0.4}\\ &=\tfrac{4}{13}\cdot\tfrac{5}{3}+\tfrac{9}{13}\cdot\tfrac{5}{2} =\tfrac{20}{39}+\tfrac{45}{26}=\tfrac{175}{78}. \end{aligned} \end{equation*}

(a) \(y\sim\mathrm{Bin}(1000,\tfrac16)\), so

\begin{equation*} \begin{aligned} E(y)&=1000\cdot\tfrac16=166.7,\\ \mathrm{sd}(y)&=\sqrt{1000\cdot\tfrac16\cdot\tfrac56}=\sqrt{138.9}=11.8, \end{aligned} \end{equation*}

and \(y\) is approximately \(\mathrm{N}(166.7,\,11.8^{2})\): a symmetric bell centred at \(167\), with essentially all its mass on \([130,205]\) and inflection points at \(155\) and \(178\).

(b) The quantiles are \(166.7+11.8\,z_{p}\) with \(z_{p}=-1.645,\,-0.674,\,0,\,0.674,\,1.645\), rounded to integers since \(y\) is discrete:

(a) The prior predictive is the three-component normal mixture

\begin{equation*} p(y)\approx 0.25\,\mathrm{N}(y\mid 83.3,\,8.74^{2}) +0.5\,\mathrm{N}(y\mid 166.7,\,11.79^{2}) +0.25\,\mathrm{N}(y\mid 250,\,13.69^{2}), \end{equation*}

since \(y\mid\theta\sim\mathrm{Bin}(1000,\theta)\) has \(E(y\mid\theta)=1000\theta\) and \(\mathrm{sd}(y\mid\theta)=\sqrt{1000\,\theta(1-\theta)}\), evaluated at \(\theta=1/12,\,1/6,\,1/4\).

The sketch is three well-separated bumps centred at \(83\), \(167\) and \(250\): adjacent centres are more than six standard deviations apart (\(83.4/11.79=7.1\), \(83.3/13.69=6.1\)), so the density sinks to \(7\times10^{-6}\) near \(y=119\) and \(1.0\times10^{-4}\) near \(y=207\). The middle bump carries twice the area of each outer one and is the tallest, with peak height \(0.0169\) against \(0.0114\) and \(0.0073\).

(b) Let \(\Phi\) denote the standard normal cdf and write the mixture cdf

\begin{equation*} F(y)=0.25\,\Phi\!\Bigl(\tfrac{y-83.3}{8.74}\Bigr) +0.5\,\Phi\!\Bigl(\tfrac{y-166.7}{11.79}\Bigr) +0.25\,\Phi\!\Bigl(\tfrac{y-250}{13.69}\Bigr). \end{equation*}

Because the components barely overlap, each quantile is located by inverting a single component, the other two terms of \(F\) equalling \(0\) or \(1\) to within \(10^{-9}\) there.

(i) \(p=0.05\) lies inside the first component: \(0.25\,\Phi(z)=0.05\), so \(\Phi(z)=0.2\), \(z=-0.842\), and

\begin{equation*} y_{0.05}=83.3-0.842(8.74)=76.0 . \end{equation*}

(ii) \(p=0.50\) lies at the centre of the second: \(0.25+0.5\,\Phi(z)=0.50\) gives \(z=0\) and \(y_{0.50}=166.7\).

(iii) \(p=0.95\) lies inside the third: \(0.75+0.25\,\Phi(z)=0.95\), so \(\Phi(z)=0.8\), \(z=0.842\), and

\begin{equation*} y_{0.95}=250+0.842(13.69)=261.5 . \end{equation*}

(iv) \(p=0.25\) and \(p=0.75\) are exactly the cumulative masses between components, so they fall in the empty valleys and are essentially undetermined: \(F\) crosses \(0.25\) at \(y=118\) but \(|F(y)-0.25|<0.0005\) for every \(y\) in \([110,130]\), and \(F\) crosses \(0.75\) at \(y=206\) with \(|F(y)-0.75|<0.0015\) throughout \([200,215]\).

(a) \(\Pr(y=k)=\dfrac{1}{n+1}\) for every \(k=0,1,\dots,n\) – the discrete uniform distribution:

\begin{equation*} \begin{aligned} \int_{0}^{1}\binom{n}{k}\theta^{k}(1-\theta)^{n-k}\,d\theta &=\binom{n}{k}B(k+1,\,n-k+1)\\ &=\frac{n!}{k!\,(n-k)!}\cdot\frac{k!\,(n-k)!}{(n+1)!} =\frac{1}{n+1}. \end{aligned} \end{equation*}

(b) The posterior is \(\theta\mid y\sim\mathrm{Beta}(\alpha+y,\beta+n-y)\), so with \(\lambda=\dfrac{\alpha+\beta}{\alpha+\beta+n}\in(0,1)\),

\begin{equation*} \begin{aligned} E(\theta\mid y)&=\frac{\alpha+y}{\alpha+\beta+n}\\ &=\frac{\alpha+\beta}{\alpha+\beta+n}\cdot\frac{\alpha}{\alpha+\beta} +\frac{n}{\alpha+\beta+n}\cdot\frac{y}{n}\\ &=\lambda\,\frac{\alpha}{\alpha+\beta}+(1-\lambda)\,\frac{y}{n}. \end{aligned} \end{equation*}

A convex combination of two numbers with strictly positive weights lies between them (and equals them only when they coincide).

(c) Uniform means \(\mathrm{Beta}(1,1)\), whose variance is \(\frac{1\cdot1}{2^{2}\cdot3}=\frac{1}{12}\). Writing \(m=E(\theta\mid y)=\frac{y+1}{n+2}\), the posterior \(\mathrm{Beta}(1+y,\,1+n-y)\) has

\begin{equation*} \mathrm{var}(\theta\mid y)=\frac{m(1-m)}{n+3}\le\frac{1/4}{n+3}\le\frac{1}{16}<\frac{1}{12} \end{equation*}

for every \(n\ge1\), since \(m(1-m)\le\frac14\).

(d) Prior \(\mathrm{Beta}(1,3)\) with \(n=1\), \(y=1\):

\begin{equation*} \begin{aligned} \mathrm{var}(\theta)&=\frac{1\cdot3}{4^{2}\cdot5}=\frac{3}{80}=0.0375,\\ \mathrm{var}(\theta\mid y)&=\frac{2\cdot3}{5^{2}\cdot6}=\frac{1}{25}=0.0400 . \end{aligned} \end{equation*}

By (1.8), conditioning on \(\theta_j\),

\begin{equation*} E(y_j)=E\bigl(E(y_j\mid\theta_j)\bigr)=E(10n_j\theta_j)=10n_j\,\frac{\alpha}{\beta}, \end{equation*}

using \(E(\theta_j)=\alpha/\beta\) and \(\mathrm{var}(\theta_j)=\alpha/\beta^{2}\) for \(\theta_j\sim\mathrm{Gamma}(\alpha,\beta)\) (Appendix A) and the Poisson identity \(E(y_j\mid\theta_j)=\mathrm{var}(y_j\mid\theta_j)=10n_j\theta_j\).

By (1.9),

\begin{equation*} \begin{aligned} \mathrm{var}(y_j)&=E\bigl(\mathrm{var}(y_j\mid\theta_j)\bigr) +\mathrm{var}\bigl(E(y_j\mid\theta_j)\bigr)\\ &=E(10n_j\theta_j)+\mathrm{var}(10n_j\theta_j)\\ &=10n_j\,\frac{\alpha}{\beta}+(10n_j)^{2}\,\frac{\alpha}{\beta^{2}}, \end{aligned} \end{equation*}

which is (2.17).

(a) The natural parameter is \(\phi=\mathrm{logit}(\theta)=\log\frac{\theta}{1-\theta}\), because

\begin{equation*} p(y\mid\theta)=\binom{n}{y}\theta^{y}(1-\theta)^{n-y} =\binom{n}{y}(1-\theta)^{n}\exp\Bigl\{y\log\tfrac{\theta}{1-\theta}\Bigr\}, \end{equation*}

which is of the exponential-family form \(f(y)g(\theta)e^{\phi(\theta)^{T}u(y)}\) of Section 2.4 with \(f(y)=\binom{n}{y}\), \(g(\theta)=(1-\theta)^{n}\), \(u(y)=y\) and \(\phi(\theta)=\mathrm{logit}(\theta)\).

Now put \(p(\phi)\propto 1\) on \(\phi\in(-\infty,\infty)\) and transform back by (2.19), using

\begin{equation*} \frac{d\phi}{d\theta}=\frac{d}{d\theta}\bigl[\log\theta-\log(1-\theta)\bigr] =\frac{1}{\theta}+\frac{1}{1-\theta}=\frac{1}{\theta(1-\theta)} : \end{equation*}

\begin{equation*} p(\theta)=p(\phi)\left|\frac{d\phi}{d\theta}\right|\propto\theta^{-1}(1-\theta)^{-1}. \end{equation*}

(b) The unnormalized posterior is

\begin{equation*} p(\theta\mid y)\propto\theta^{-1}(1-\theta)^{-1}\theta^{y}(1-\theta)^{n-y} =\theta^{y-1}(1-\theta)^{n-y-1}, \end{equation*}

the kernel of \(\mathrm{Beta}(y,\,n-y)\), normalizable exactly when \(y>0\) and \(n-y>0\); the prior is itself improper (\(\int_{0}^{1}\theta^{-1}(1-\theta)^{-1}d\theta=\infty\)), so properness must be checked for each \(y\) rather than assumed.

(i) If \(y=0\) the integrand is \(\theta^{-1}(1-\theta)^{n-1}\), and near \(0\) it is bounded below by \(\tfrac12\theta^{-1}\) for \(\theta\) small, so \(\int_{0}^{1}\theta^{-1}(1-\theta)^{n-1}d\theta=\infty\).

(ii) If \(y=n\) the integrand is \(\theta^{n-1}(1-\theta)^{-1}\) and the same argument applies at \(1\).

\begin{equation*} \theta \mid y \;\sim\; \mathrm{N}\!\left(\frac{180 + 600n}{1+4n},\ \frac{1600}{1+4n}\right). \end{equation*}

(a) Apply (2.11)–(2.12) with \(\mu_0 = 180\), \(\tau_0^2 = 1600\), \(\sigma^2 = 400\), \(\bar y = 150\):

\begin{equation*} \begin{aligned} \frac{1}{\tau_n^2} &= \frac{1}{1600} + \frac{n}{400} = \frac{1+4n}{1600}, \\[2pt] \mu_n &= \tau_n^2\left(\frac{180}{1600} + \frac{150\,n}{400}\right) = \frac{180 + 600n}{1+4n}. \end{aligned} \end{equation*}

(b) Section 2.5 (p. 42) computes \(\mathrm{E}(\tilde y\mid y) = \mu_n\) and, by identities (2.7)–(2.8), \(\operatorname{var}(\tilde y\mid y) = \mathrm{E}(\sigma^2\mid y)+\operatorname{var}(\theta\mid y) = \sigma^2 + \tau_n^2\); the pair \((\tilde y,\theta)\) is jointly normal, so

\begin{equation*} \tilde y \mid y \;\sim\; \mathrm{N}\!\left(\frac{180+600n}{1+4n},\ 400 + \frac{1600}{1+4n}\right). \end{equation*}

(c) \(n = 10\): \(\mu_{10} = 6180/41 = 150.73\) and \(\tau_{10}^2 = 1600/41 = 39.02\), so \(\tau_{10} = 6.247\). The central 95% intervals are \(\mu_{10} \pm 1.96\,\tau_{10}\) and \(\mu_{10} \pm 1.96\sqrt{400+\tau_{10}^2}\):

\begin{equation*} \begin{aligned} \theta &: \ [138.5,\ 163.0], \\ \tilde y &: \ 150.73 \pm 1.96\sqrt{439.02} = [109.7,\ 191.8]. \end{aligned} \end{equation*}

(d) \(n = 100\): \(\mu_{100} = 60180/401 = 150.07\) and \(\tau_{100}^2 = 1600/401 = 3.990\), so \(\tau_{100} = 1.998\):

\begin{equation*} \begin{aligned} \theta &: \ [146.2,\ 154.0], \\ \tilde y &: \ 150.07 \pm 1.96\sqrt{403.99} = [110.7,\ 189.5]. \end{aligned} \end{equation*}

\begin{equation*} \alpha = 1, \qquad \beta = \tfrac{2}{3}. \end{equation*}

(a) This is the method of moments (A.3), p. 585: with \(\mu = \alpha/(\alpha+\beta)\) and \(\operatorname{var} = \mu(1-\mu)/(\alpha+\beta+1)\) from Table A.1,

\begin{equation*} \begin{aligned} 0.3^2 &= \frac{(0.6)(0.4)}{\alpha+\beta+1} \ \Longrightarrow\ \alpha+\beta+1 = \tfrac{0.24}{0.09} = \tfrac{8}{3}, \\ \alpha &= 0.6(\alpha+\beta) = 0.6\cdot\tfrac{5}{3} = 1, \qquad \beta = \tfrac{5}{3} - 1 = \tfrac{2}{3}. \end{aligned} \end{equation*}

The density is \(p(\theta) = \tfrac{2}{3}(1-\theta)^{-1/3}\) on \((0,1)\): strictly increasing from \(\tfrac{2}{3}\) at \(\theta=0\), blowing up at \(\theta=1\) but with finite integral. (A standard deviation of \(0.3\) is close to the maximum \(\sqrt{0.24}=0.49\) attainable on \([0,1]\) with mean \(0.6\), which is why so little mass sits near the mean.)

(b) With \(y = 650\) successes out of \(n = 1000\), beta-binomial conjugacy (Section 2.4, p. 35, \(\theta\mid y\sim\mathrm{Beta}(\alpha+y,\beta+n-y)\)) gives \(\theta\mid y \sim \mathrm{Beta}(1+650,\ \tfrac23+350) = \mathrm{Beta}(651,\,350.67)\), so

\begin{equation*} \begin{aligned} \mathrm{E}(\theta\mid y) &= \frac{651}{1001.67} = 0.6499, \\ \operatorname{var}(\theta\mid y) &= \frac{(0.6499)(0.3501)}{1002.67} = 2.269\times 10^{-4}, \end{aligned} \end{equation*}

i.e. posterior standard deviation \(0.01506\). The density is essentially \(\mathrm{N}(0.650, 0.0151^2)\) to the eye: a sharp bell on \([0.60,0.70]\), invisible on the scale of the prior sketch.

(c) Reparameterize a prior by its mean \(\mu\) and prior sample size \(\kappa = \alpha+\beta\), so \(\alpha = \kappa\mu\), \(\beta=\kappa(1-\mu)\), and

\begin{equation*} \mathrm{E}(\theta\mid y) = \frac{\kappa\mu + 650}{\kappa + 1000}, \end{equation*}

a weighted average of \(\mu\) and \(0.650\) with weights \(\kappa\) and \(1000\). Numerically:

Every prior with \(\kappa \ll 1000\), the three noninformative choices included, shifts the posterior mean by less than \(0.006\) and leaves the posterior sd unchanged in the third decimal. Sensitivity appears only at \(\kappa \approx 100\) (last row), a claim of prior information worth a tenth of the sample.

\begin{equation*} p(N \mid y=203) \;=\; \frac{1}{C}\,\frac{(0.99)^{N-1}}{N}, \qquad N = 203, 204, \ldots, \end{equation*}

with \(C = \sum_{N\ge 203} (0.99)^{N-1}/N = 0.047051\).

(a) The sampling model is \(\Pr(y = 203 \mid N) = 1/N\) for \(N \ge 203\) and \(0\) otherwise, so Bayes’ rule multiplies the geometric prior by \(1/N\) and truncates below at 203; the constant \(1/100\) is absorbed into \(C\). In closed form, using \(\sum_{N\ge1} x^N/N = -\log(1-x)\),

\begin{equation*} C \;=\; \frac{1}{x}\left(-\log(1-x) - \sum_{N=1}^{202} \frac{x^N}{N}\right), \quad x = 0.99 . \end{equation*}

(b) The \(1/N\) in the posterior cancels the \(N\) in the first moment, so the mean is a bare geometric tail sum:

\begin{equation*} \begin{aligned} \mathrm{E}(N\mid y) &= \frac{1}{C}\sum_{N\ge203}(0.99)^{N-1} = \frac{(0.99)^{202}}{C\,(1-0.99)} = 279.09, \\ \mathrm{E}(N^2\mid y) &= \frac{1}{C}\sum_{N\ge203} N(0.99)^{N-1} \\ &= \frac{1}{C}\cdot\frac{203(0.99)^{202}(0.01)+(0.99)^{203}}{(0.01)^2} = 84285, \end{aligned} \end{equation*}

so \(\operatorname{sd}(N\mid y) = \sqrt{84285 - 279.09^2} = 79.96\).

(c) Take the scale-invariant prior \(p(N) \propto 1/N\), the natural noninformative choice for a positive quantity whose order of magnitude is unknown (and the discrete analogue of Jeffreys’ \(p(\sigma)\propto\sigma^{-1}\); the flat prior \(p(N)\propto 1\) will not do, since it leaves \(p(N\mid y)\propto 1/N\), whose sum diverges). Then

\begin{equation*} p(N\mid y) = \frac{1}{C_2}\,\frac{1}{N^2}, \quad N \ge 203, \quad C_2 = \sum_{N\ge203}N^{-2} = \psi^{\prime}(203) = 0.0049383, \end{equation*}

which is proper. But

\begin{equation*} \mathrm{E}(N \mid y) \;=\; \frac{1}{C_2}\sum_{N\ge203}\frac{1}{N} \;=\; \infty, \end{equation*}

and a fortiori the standard deviation is infinite: one draw from a discrete uniform bounds \(N\) below but says nothing about its scale, so a scale-free prior leaves a tail too heavy for moments. Quantiles are the usable summaries; from \(\Pr(N>M\mid y) = \psi^{\prime}(M+1)/\psi^{\prime}(203) \approx 202.5/(M+\tfrac12)\),

\begin{equation*} \text{median } 405, \qquad \text{central 95\% interval } [208,\ 8100]. \end{equation*}

\begin{equation*} p(\theta\mid y) \;\propto\; \prod_{i=1}^{5}\frac{1}{1+(y_i-\theta)^2}, \qquad 0 \le \theta \le 100, \end{equation*}

the uniform prior contributing only the truncation.

(a) Evaluate this on the grid \(\theta_j = j/m\), \(j = 0,\ldots,100m\), with \(m = 1000\); compute it as \(\exp\) of the log-likelihood \(-\sum_i \log(1+(y_i-\theta)^2)\) recentred at its maximum to avoid underflow, and normalize by \(\sum_j\). The result is a single sharp mode on the data,

\begin{equation*} \begin{aligned} \text{mode } &= 44.86, \quad \text{median } = 45.02, \quad \text{mean } = 45.08, \\ \operatorname{sd} &= 0.986, \quad \text{central 95\%: } [43.28,\ 47.08], \end{aligned} \end{equation*}

and posterior mass \(1.5\times10^{-5}\) outside \([40,50]\); the grid spacing \(0.001\) resolves a mode of width \(\approx 2\), so the discretization is harmless. The five likelihood factors merge into one bump, mildly right-skewed (skewness \(0.20\), and mode \(<\) median \(<\) mean above).

(b) Draw \(j\) from the normalized grid probabilities with replacement and jitter by \(\mathrm{U}(-\tfrac{1}{2m},\tfrac{1}{2m})\); 1000 such draws gave sample mean \(45.04\), sd \(0.970\), and quantiles

\begin{equation*} \begin{aligned} &2.5\%: 43.30, \quad 25\%: 44.34, \quad 50\%: 44.98, \\ &75\%: 45.71, \quad 97.5\%: 46.90, \end{aligned} \end{equation*}

matching the grid summaries in (a) to within Monte Carlo error \(\operatorname{sd}/\sqrt{1000} \approx 0.03\). (Check!)

(c) For each posterior draw \(\theta^{(s)}\) set \(y_6^{(s)} = \theta^{(s)} + Z^{(s)}\) with \(Z^{(s)}\) an independent standard Cauchy, which is exactly a draw from \(p(y_6\mid y) = \int p(y_6\mid\theta)p(\theta\mid y)\,d\theta\) by (1.4). The predictive draws had

\begin{equation*} \begin{aligned} &\text{median } 44.97, \quad \text{quartiles } 43.60,\ 46.31, \\ &\text{central 95\%: } [35.0,\ 57.6], \end{aligned} \end{equation*}

with \(87.5\%\) of draws inside \([40,50]\) and extreme values \(-1879\) and \(755\): the predictive distribution has no mean, its spread being set by the unit Cauchy noise, not by the \(\pm 1\) uncertainty in \(\theta\). The histogram must therefore be plotted on a truncated axis (here \([35,55]\)) to show anything at all.

\begin{equation*} p_{\mathrm{Jeffreys}}(\theta) \propto \theta^{-1/2}, \qquad\text{i.e.}\qquad \alpha = \tfrac12, \quad \beta = 0 . \end{equation*}

By the definition in Section 2.8, \(p(\theta) \propto [J(\theta)]^{1/2}\) with \(J\) the Fisher information. For \(\log p(y\mid\theta) = -\theta + y\log\theta - \log(y!)\),

\begin{equation*} \begin{aligned} \frac{d^2}{d\theta^2}\log p(y\mid\theta) &= -\frac{y}{\theta^2}, \\ J(\theta) = \mathrm{E}\!\left(-\frac{d^2}{d\theta^2}\log p(y\mid\theta)\ \Big|\ \theta\right) &= \frac{\mathrm{E}(y\mid\theta)}{\theta^2} = \frac{1}{\theta}, \end{aligned} \end{equation*}

so \(p(\theta) \propto \theta^{-1/2}\).

The \(\mathrm{Gamma}(\alpha,\beta)\) density is \(\propto \theta^{\alpha-1}e^{-\beta\theta}\); matching the power gives \(\alpha - 1 = -\tfrac12\) and matching the exponential factor (which must be absent) gives \(\beta = 0\). Thus Jeffreys’ prior is the improper limit \(\mathrm{Gamma}(\tfrac12, 0)\), and for any \(n \ge 1\) observations the conjugate updating (2.15) still yields a proper posterior \(\mathrm{Gamma}\!\left(\tfrac12 + \sum_{i=1}^n y_i,\ n\right)\), since its shape \(\tfrac12+\sum y_i > 0\) and rate \(n > 0\).

All four parts are the single template: with \(y_+=\sum_i y_i\), \(x_+=\sum_i x_i\) and \(y_i\mid\theta\sim\mathrm{Poisson}(x_i\theta)\) as in (2.14),

\begin{equation*} \theta\mid y \sim \mathrm{Gamma}\!\left(\tfrac12+y_+,\ x_+\right), \quad \tilde y \mid y \sim \mathrm{Neg\text{-}bin}\!\left(\tfrac12+y_+,\ \tfrac{x_+}{\tilde x}\right), \end{equation*}

by (2.14)–(2.15) and the negative binomial mixture identity of Section 2.6 (p. 44), \(\int \mathrm{Poisson}(\tilde y\mid \tilde x\theta)\,\mathrm{Gamma}(\theta\mid\alpha,\beta)\,d\theta = \mathrm{Neg\text{-}bin}(\tilde y\mid\alpha,\beta/\tilde x)\), with

\begin{equation*} \mathrm{E}(\tilde y\mid y) = \frac{(\tfrac12+y_+)\tilde x}{x_+}, \quad \operatorname{var}(\tilde y\mid y) = \mathrm{E}(\tilde y\mid y)\left(1+\frac{\tilde x}{x_+}\right). \end{equation*}

Parts (a) and (c) are the case \(x_i\equiv 1\), \(\tilde x = 1\). The prior taken throughout is Jeffreys’ \(\mathrm{Gamma}(\tfrac12,0)\) from Exercise 2.12; it is improper, but the posterior is proper for every part since its shape \(\tfrac12+y_+>0\) and rate \(x_+>0\), and with ten years of data a prior of this weight is inconsequential. Given \(\theta\), the ten years are exchangeable in (a) and (c); in (b) and (d) model (2.14) is not exchangeable in the \(y_i\), only in the pairs \((x_i,y_i)\).

(a) \(y_+ = 238\), \(n = 10\), so \(\theta\mid y \sim \mathrm{Gamma}(238.5, 10)\): posterior mean \(23.85\), sd \(1.54\). The 1986 predictive has mean \(23.85\) and variance \(23.85(1.1) = 26.24\), sd \(5.12\); the exact negative-binomial central 95% interval is

\begin{equation*} [14,\ 34] \end{equation*}

(the normal approximation \(23.85 \pm 1.96(5.12)\) gives \([13.8, 33.9]\), the same after rounding to integers).

(b) Exposures \(x_i = (\text{passenger deaths})/(\text{death rate})\), in units of \(10^{11}\) passenger miles:

with \(x_+ = 57.159\). Then \(\theta\mid y \sim \mathrm{Gamma}(238.5,\ 57.159)\): posterior mean \(4.173\) fatal accidents per \(10^{11}\) passenger miles, sd \(0.270\). With \(\tilde x = 8\) the predictive mean is \(33.38\) and the variance \(33.38(1+8/57.159) = 38.05\), sd \(6.17\), giving

\begin{equation*} [22,\ 46]. \end{equation*}

(c) \(y_+ = 6919\), so \(\theta\mid y \sim \mathrm{Gamma}(6919.5,\ 10)\): posterior mean \(691.95\), sd \(8.32\). Predictive mean \(691.95\), variance \(691.95(1.1)=761.1\), sd \(27.59\), and

\begin{equation*} [638,\ 747]. \end{equation*}

(d) \(\theta\mid y \sim \mathrm{Gamma}(6919.5,\ 57.159)\): posterior mean \(121.06\) deaths per \(10^{11}\) passenger miles, sd \(1.46\). With \(\tilde x = 8\), predictive mean \(968.5\), variance \(968.5(1.13996) = 1104\), sd \(33.2\), and

\begin{equation*} [904,\ 1034]. \end{equation*}

(e) The Poisson model is most defensible in (b) and least in (c) and (d).

(i) Accidents versus deaths. A Poisson count arises from many independent opportunities each with tiny probability. Fatal accidents plausibly satisfy this. Passenger deaths do not: deaths arrive in clusters, one per accident, of size equal to the number aboard. Deaths are therefore a compound Poisson sum \(\sum_{k} m_k\) over accidents \(k\), with \(\operatorname{var} = \mathrm{E}(\text{accidents})\,\mathrm{E}(m^2)\), which exceeds the mean by the factor \(\mathrm{E}(m^2)/\mathrm{E}(m)\) – of order the typical aircraft capacity. A Poisson likelihood for deaths therefore understates predictive uncertainty by roughly an order of magnitude in the standard deviation.

(ii) Exposure versus no exposure. Conditional on the count being Poisson, modelling the rate as constant per passenger mile is more credible than as constant per year, since traffic volume grows steadily and is the obvious driver of the number of opportunities for an accident. Model (a) forces the 1986 expectation to be the ten-year average of a series generated under systematically smaller exposure than 1986’s; model (b) does not.

(iii) Constant \(\theta\) across a decade. If safety improves over time, all four models are misspecified, overdispersed relative to Poisson through between-year variation in the rate; a time trend is the repair (Exercise 3.12).

Both derivations are the identity: a sum of two quadratics in \(\theta\) is a single quadratic whose curvature is the sum of the curvatures. (The text on pp. 41–42 cites parts 2.14(b) and 2.14(c), which the printed exercise numbers differently: its 2.14(b) is the second half of the printed (a), and its 2.14(c) is the printed (b).)

(a) Single observation. With \(\tau_1^2\) defined by \(\tau_1^{-2} = \tau_0^{-2}+\sigma^{-2}\) and \(\mu_1 = \tau_1^2(\mu_0\tau_0^{-2}+y\sigma^{-2})\), collect powers of \(\theta\):

\begin{equation*} \begin{aligned} \frac{(y-\theta)^2}{\sigma^2} + \frac{(\theta-\mu_0)^2}{\tau_0^2} &= \theta^2\left(\frac{1}{\sigma^2}+\frac{1}{\tau_0^2}\right)

• 2\theta\left(\frac{y}{\sigma^2}+\frac{\mu_0}{\tau_0^2}\right) + c_1 \\ &= \frac{1}{\tau_1^2}\left(\theta^2 - 2\theta\mu_1\right) + c_1 \\ &= \frac{1}{\tau_1^2}(\theta-\mu_1)^2 + c_2, \end{aligned} \end{equation*}

where \(c_1 = y^2/\sigma^2 + \mu_0^2/\tau_0^2\) and \(c_2 = c_1 - \mu_1^2/\tau_1^2\) are free of \(\theta\). Substituting into \(p(\theta\mid y)\propto\exp(-\tfrac12[\cdots])\) and absorbing \(e^{-c_2/2}\) into the normalizing constant gives (2.9) with (2.10).

\(n\) observations. Apply the sum-of-squares decomposition \(\sum_i (y_i-\theta)^2 = \sum_i (y_i-\bar y)^2 + n(\bar y - \theta)^2\), whose first term is free of \(\theta\):

\begin{equation*} \begin{aligned} &\frac{1}{\tau_0^2}(\theta-\mu_0)^2 + \frac{1}{\sigma^2}\sum_{i=1}^n (y_i-\theta)^2 \\ &\qquad= \frac{1}{\tau_0^2}(\theta-\mu_0)^2 + \frac{n}{\sigma^2}(\theta-\bar y)^2 + c_3 . \end{aligned} \end{equation*}

This is the single-observation exponent with \(y\) replaced by \(\bar y\) and \(\sigma^2\) by \(\sigma^2/n\) – equivalently, \(\bar y\mid\theta\sim\mathrm{N}(\theta,\sigma^2/n)\) is sufficient – so (2.10) with those substitutions is exactly (2.12), and (2.11) follows.

(b) Let \((\mu_k,\tau_k^2)\) denote the parameters of \(p(\theta\mid y_1,\ldots,y_k)\). Because the \(y_i\) are conditionally independent given \(\theta\),

\begin{equation*} p(\theta\mid y_1,\ldots,y_k) \propto p(\theta\mid y_1,\ldots,y_{k-1})\,p(y_k\mid\theta), \end{equation*}

so each step is an instance of (2.9)–(2.10) with prior \(\mathrm{N}(\mu_{k-1},\tau_{k-1}^2)\) and single observation \(y_k\) of variance \(\sigma^2\). Claim, by induction on \(k\):

\begin{equation*} \frac{1}{\tau_k^2} = \frac{1}{\tau_0^2}+\frac{k}{\sigma^2}, \qquad \frac{\mu_k}{\tau_k^2} = \frac{\mu_0}{\tau_0^2} + \frac{1}{\sigma^2}\sum_{i=1}^k y_i . \end{equation*}

The case \(k=0\) is the prior. For the step, (2.10) gives

\begin{equation*} \begin{aligned} \frac{1}{\tau_k^2} &= \frac{1}{\tau_{k-1}^2}+\frac{1}{\sigma^2} = \frac{1}{\tau_0^2}+\frac{k-1}{\sigma^2}+\frac{1}{\sigma^2}, \\ \frac{\mu_k}{\tau_k^2} &= \frac{\mu_{k-1}}{\tau_{k-1}^2} + \frac{y_k}{\sigma^2} = \frac{\mu_0}{\tau_0^2} + \frac{1}{\sigma^2}\sum_{i=1}^{k-1}y_i + \frac{y_k}{\sigma^2}, \end{aligned} \end{equation*}

which is the claim at \(k\). Setting \(k=n\) and using \(\sum_{i=1}^n y_i = n\bar y\) yields precisely (2.12), and hence (2.11).

\begin{equation*} E\!\left[Z^m(1-Z)^n\right] = \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\cdot \frac{\Gamma(\alpha+m)\Gamma(\beta+n)}{\Gamma(\alpha+\beta+m+n)} . \end{equation*}

Indeed, with the Beta\((\alpha,\beta)\) density of Appendix A,

\begin{equation*} \begin{aligned} E\!\left[Z^m(1-Z)^n\right] &= \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)} \int_0^1 z^{\alpha+m-1}(1-z)^{\beta+n-1}\,dz\\[2pt] &= \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\cdot \frac{\Gamma(\alpha+m)\Gamma(\beta+n)}{\Gamma(\alpha+\beta+m+n)}, \end{aligned} \end{equation*}

the quoted integral applying because \(\alpha+m>0\) and \(\beta+n>0\).

Taking \((m,n)=(1,0)\) and \((2,0)\) and using \(\Gamma(x+1)=x\Gamma(x)\),

\begin{equation*} \begin{aligned} E(Z) &= \frac{\Gamma(\alpha+\beta)\Gamma(\alpha+1)} {\Gamma(\alpha)\Gamma(\alpha+\beta+1)} = \frac{\alpha}{\alpha+\beta},\\[2pt] E(Z^2) &= \frac{\Gamma(\alpha+\beta)\Gamma(\alpha+2)} {\Gamma(\alpha)\Gamma(\alpha+\beta+2)} = \frac{\alpha(\alpha+1)}{(\alpha+\beta)(\alpha+\beta+1)} . \end{aligned} \end{equation*}

Hence

\begin{equation*} \begin{aligned} \operatorname{var}(Z) &= \frac{\alpha(\alpha+1)}{(\alpha+\beta)(\alpha+\beta+1)}

• \frac{\alpha^2}{(\alpha+\beta)^2}\\[2pt] &= \frac{\alpha(\alpha+1)(\alpha+\beta) - \alpha^2(\alpha+\beta+1)} {(\alpha+\beta)^2(\alpha+\beta+1)} = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)} . \end{aligned} \end{equation*}

(a)

\begin{equation*} p(y) = \binom{n}{y}\, \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\cdot \frac{\Gamma(\alpha+y)\,\Gamma(\beta+n-y)}{\Gamma(\alpha+\beta+n)}, \qquad y=0,\dots,n, \end{equation*}

by integrating the joint density over \(\theta\) and reading off the value of the integral from Exercise 2.15 with exponents \(y\) and \(n-y\):

\begin{equation*} \begin{aligned} p(y) &= \int_0^1 \binom{n}{y}\theta^{y}(1-\theta)^{n-y}\, \mathrm{Beta}(\theta\mid\alpha,\beta)\,d\theta\\[2pt] &= \binom{n}{y}\,E\!\left[\theta^{y}(1-\theta)^{n-y}\right]. \end{aligned} \end{equation*}

(b) Assume \(n\ge 2\) (for \(n=1\) constancy only forces \(\alpha=\beta\), which is the one gap in the printed statement). Constancy is equivalent to \(p(y+1)/p(y)=1\) for \(y=0,\dots,n-1\), and

\begin{equation*} \frac{p(y+1)}{p(y)} = \frac{n-y}{y+1}\cdot\frac{\alpha+y}{\beta+n-y-1}, \end{equation*}

since \(\Gamma(\alpha+y+1)=(\alpha+y)\Gamma(\alpha+y)\) and \(\Gamma(\beta+n-y)=(\beta+n-y-1)\Gamma(\beta+n-y-1)\). Taking \(y=0\) and \(y=n-1\) gives the two equations

\begin{equation*} n\alpha = \beta+n-1, \qquad \alpha+n-1 = n\beta . \end{equation*}

Adding them gives \((n+1)\alpha=(n+1)\beta\), so \(\alpha=\beta\); the first equation then reads \(n\alpha=\alpha+n-1\), i.e. \((n-1)\alpha=n-1\), so \(\alpha=\beta=1\).

(a) Write \(\psi=\sigma^2\), so \(\sigma=\psi^{1/2}\) and \(d\sigma/d\psi=\tfrac12\psi^{-1/2}\); by the change-of-variables formula (1.4),

\begin{equation*} p(\psi)=p(\sigma)\left|\frac{d\sigma}{d\psi}\right| \propto \psi^{-1/2}\cdot\tfrac12\psi^{-1/2} \propto \psi^{-1} = \sigma^{-2}. \end{equation*}

(b) The transformation \(\psi\mapsto\sigma=\psi^{1/2}\) is nonlinear, so it multiplies the density by the non-constant Jacobian \(2\sigma\); a level set of \(p(\psi\mid v)\) is therefore not a level set of \(p(\sigma\mid v)\), and an HPD region in one parameterization maps to a non-HPD region in the other.

Concretely, \(v\) has density \(p(v\mid\psi)=(n/\psi)\,\chi^2_n(nv/\psi)\propto\psi^{-n/2}e^{-nv/(2\psi)}\), so with the prior of part (a)

\begin{equation*} p(\psi\mid v)\propto \psi^{-n/2-1}e^{-nv/(2\psi)}, \qquad \psi\mid v \sim \text{Inv-gamma}\!\left(\tfrac n2,\tfrac{nv}{2}\right), \end{equation*}

which is the scaled inverse-\(\chi^2\) posterior \(\mathrm{Inv}\text{-}\chi^2(n,v)\) of Section 2.6 – proper for \(n\ge1,v>0\), despite the improper prior. That density is unimodal (its log-derivative \(-(n/2+1)/\psi+nv/(2\psi^2)\) vanishes only at \(\psi=nv/(n+2)\)), and likewise for \(\sigma\), so each 95% HPD region is a single interval whose endpoints carry equal density. Thus \([\psi_L,\psi_U]\) of content \(0.95\) is HPD for \(\psi\) iff \(p(\psi_L\mid v)=p(\psi_U\mid v)\), while \([\psi_L^{1/2},\psi_U^{1/2}]\) (same content) is HPD for \(\sigma\) iff

\begin{equation*} 2\psi_L^{1/2}\,p(\psi_L\mid v) = 2\psi_U^{1/2}\,p(\psi_U\mid v). \end{equation*}

Both can hold only if \(\psi_L=\psi_U\), impossible for a nondegenerate 95% region. Hence squaring the endpoints of the 95% posterior interval of highest density for \(\sigma\) never returns the 95% HPD interval for \(\sigma^2\).

The likelihood is gamma-shaped in \(\theta\) (the \(y_i\) are independent given \(\theta\), the model being exchangeable in the pairs \((x_i,y_i)\) rather than in the \(y_i\) alone):

\begin{equation*} p(y\mid\theta)=\prod_{i=1}^{n}\frac{(x_i\theta)^{y_i}e^{-x_i\theta}}{y_i!} \;\propto\; \theta^{\sum_i y_i}\,e^{-\left(\sum_i x_i\right)\theta}, \end{equation*}

the discarded factor \(\prod_i x_i^{y_i}/y_i!\) being free of \(\theta\). Multiplying by the \(\mathrm{Gamma}(\alpha,\beta)\) prior density \(p(\theta)\propto\theta^{\alpha-1}e^{-\beta\theta}\),

\begin{equation*} p(\theta\mid y)\;\propto\; \theta^{\,\alpha+\sum_i y_i-1}\, e^{-\left(\beta+\sum_i x_i\right)\theta}, \qquad \theta>0, \end{equation*}

which is the unnormalized \(\mathrm{Gamma}\!\left(\alpha+\sum_i y_i,\ \beta+\sum_i x_i\right)\) density; both parameters are positive (the \(x_i\) are known positive exposures), so the density is proper, the constant of proportionality is determined, and (2.15) follows.

(a) For \(y=(y_1,\dots,y_n)\) i.i.d. \(\mathrm{Expon}(\theta)\),

\begin{equation*} p(y\mid\theta)=\prod_{i=1}^{n}\theta e^{-\theta y_i} =\theta^{n}e^{-\theta\sum_i y_i}, \end{equation*}

so with \(\theta\sim\mathrm{Gamma}(\alpha,\beta)\),

\begin{equation*} p(\theta\mid y)\propto \theta^{\,\alpha+n-1}e^{-\left(\beta+\sum_i y_i\right)\theta}, \qquad \theta\mid y\sim\mathrm{Gamma}\!\left(\alpha+n,\ \beta+\textstyle\sum_i y_i\right), \end{equation*}

again a gamma distribution, so the family is conjugate.

(b) With \(\phi=1/\theta\), so \(\theta=1/\phi\) and \(|d\theta/d\phi|=\phi^{-2}\), formula (1.4) gives

\begin{equation*} \begin{aligned} p(\phi)&=\frac{\beta^{\alpha}}{\Gamma(\alpha)} \left(\frac1\phi\right)^{\alpha-1}e^{-\beta/\phi}\cdot\frac1{\phi^{2}}\\[2pt] &=\frac{\beta^{\alpha}}{\Gamma(\alpha)}\,\phi^{-(\alpha+1)}e^{-\beta/\phi}, \qquad \phi>0, \end{aligned} \end{equation*}

which is the \(\text{Inv-gamma}(\alpha,\beta)\) density of Appendix A.

(c) \(n=96\). The \(\mathrm{Gamma}(a,b)\) distribution has mean \(a/b\) and variance \(a/b^{2}\), hence coefficient of variation

\begin{equation*} \mathrm{CV} = \frac{\sqrt{a}/b}{a/b} = a^{-1/2}, \end{equation*}

free of \(b\). The prior requirement \(\alpha^{-1/2}=0.5\) gives \(\alpha=4\); by (a) the posterior shape after \(n\) bulbs is \(\alpha+n\), whatever the observed lifetimes, so

\begin{equation*} (\alpha+n)^{-1/2}=0.1 \;\Longrightarrow\; \alpha+n=100 \;\Longrightarrow\; n=100-4=96 . \end{equation*}

(d) \(n=96\) again. By (b), \(\phi\sim\text{Inv-gamma}(a,b)\) has mean \(b/(a-1)\) and variance \(b^{2}/\{(a-1)^{2}(a-2)\}\) for \(a>2\), so

\begin{equation*} \mathrm{CV} = \frac{b/\{(a-1)\sqrt{a-2}\}}{b/(a-1)} = (a-2)^{-1/2}. \end{equation*}

Now the prior condition \((\alpha-2)^{-1/2}=0.5\) gives \(\alpha=6\), and the posterior condition \((\alpha+n-2)^{-1/2}=0.1\) gives \(\alpha+n=102\), so

\begin{equation*} n = 102-6 = 96 . \end{equation*}

The prior is a different one, but the answer to (c) is unchanged.

(a) \(\theta\mid y\ge 100 \sim \mathrm{Gamma}(\alpha,\ \beta+100)\). The censored observation contributes the survival function \(\Pr(y\ge100\mid\theta)=e^{-100\theta}\) as its likelihood, so

\begin{equation*} p(\theta\mid y\ge100)\propto \theta^{\alpha-1}e^{-\beta\theta}\cdot e^{-100\theta} =\theta^{\alpha-1}e^{-(\beta+100)\theta}, \end{equation*}

whence

\begin{equation*} E(\theta\mid y\ge100)=\frac{\alpha}{\beta+100}, \qquad \operatorname{var}(\theta\mid y\ge100)=\frac{\alpha}{(\beta+100)^{2}} . \end{equation*}

(b) \(\theta\mid y=100\sim\mathrm{Gamma}(\alpha+1,\ \beta+100)\), by Exercise 2.19(a) with \(n=1\) and \(\sum y_i=100\), so

\begin{equation*} E(\theta\mid y=100)=\frac{\alpha+1}{\beta+100}, \qquad \operatorname{var}(\theta\mid y=100)=\frac{\alpha+1}{(\beta+100)^{2}} . \end{equation*}

(c) The exact observation adds the density factor \(\theta e^{-100\theta}\) where the censored one added only \(e^{-100\theta}\); the extra \(\theta\) raises the shape from \(\alpha\) to \(\alpha+1\) while leaving the rate at \(\beta+100\), and a gamma variance is proportional to its shape. So

\begin{equation*} \operatorname{var}(\theta\mid y=100)-\operatorname{var}(\theta\mid y\ge100) =\frac{1}{(\beta+100)^{2}}>0 . \end{equation*}

There is no contradiction with (2.8), for two reasons.

(i) Identity (2.8) is a statement about an average over the data, not about one dataset: it says \(\operatorname{var}(\theta)=E[\operatorname{var}(\theta\mid y)]+\operatorname{var}(E(\theta\mid y))\), so \(E[\operatorname{var}(\theta\mid y)]\le\operatorname{var}(\theta)\), while individual values of \(y\) may well raise the posterior variance.

(ii) Averaged in the right place, (2.8) holds exactly here. Take \(p(\theta\mid y\ge100)\) as the prior and the exact \(y\) as the new datum; writing \(r=\beta+100\), the predictive density is \(p(y\mid y\ge100)=\alpha r^{\alpha}/(r+y-100)^{\alpha+1}\) for \(y\ge100\), and since \(\operatorname{var}(\theta\mid y)=(\alpha+1)/(\beta+y)^{2}\) and \(E(\theta\mid y)=(\alpha+1)/(\beta+y)\) are both decreasing in \(y\), \(y=100\) is the least favourable point of that range. Integrating,

\begin{equation*} \begin{aligned} E\!\left[\operatorname{var}(\theta\mid y)\mid y\ge100\right] &= \frac{\alpha(\alpha+1)}{(\alpha+2)r^{2}},\\[2pt] \operatorname{var}\!\left(E(\theta\mid y)\mid y\ge100\right) &= \frac{\alpha(\alpha+1)^{2}}{(\alpha+2)r^{2}}-\frac{\alpha^{2}}{r^{2}} = \frac{\alpha}{(\alpha+2)r^{2}}, \end{aligned} \end{equation*}

whose sum is \(\alpha/r^{2}=\operatorname{var}(\theta\mid y\ge100)\), exactly as (2.8) requires.

The model is \(y_j\mid\theta_j\sim\mathrm{Bin}(n_j,\theta_j)\) with the \(\theta_j\) exchangeable across states, \(\theta_j\sim\mathrm{Beta}(\alpha,\beta)\) independently, giving the shrinkage estimate

\begin{equation*} E(\theta_j\mid y_j) = \frac{\alpha+y_j}{\alpha+\beta+n_j}, \end{equation*}

the beta-binomial analogue of the kidney-cancer formula \(E(\theta_j\mid y_j)=(20+y_j)/(430{,}000+10n_j)\) of Section 2.7.

Totals by state. Dropping the 1,787 respondents with missing or ‘dk-refused’ ideology leaves \(n_j\) respondents in state \(j\), of whom \(y_j\) answered ‘very liberal.’ Alaska does not appear in the file at all; deleting Hawaii and the District of Columbia leaves \(J=48\) states, \(\sum_j n_j = 29{,}353\) and \(\sum_j y_j = 1{,}463\), a pooled proportion of \(0.0498\). The state sample sizes are very unequal – from \(n=29\) (Wyoming) through a median of \(472\) to \(n=2{,}688\) (California) – which is exactly the situation in which shrinkage matters.

Prior. Estimate \((\alpha,\beta)\) by matching the first two moments of the raw proportions \(p_j=y_j/n_j\), correcting the second for binomial noise, as in Section 2.7. Write \(\mu=\alpha/(\alpha+\beta)\) and \(K=\alpha+\beta\). Then \(E(p_j)=\mu\) and

\begin{equation*} \operatorname{var}(p_j) = \operatorname{var}(\theta_j) + E\!\left[\frac{\theta_j(1-\theta_j)}{n_j}\right], \end{equation*}

by (2.8). The observed values are \(\widehat{\operatorname{var}}(p_j)=3.493\times10^{-4}\) and a mean within-state sampling variance \(\overline{p_j(1-p_j)/n_j}=1.700\times10^{-4}\), leaving \(\operatorname{var}(\theta_j)\approx 1.794\times10^{-4}\). With \(\mu=0.0498\) and \(\operatorname{var}(\theta_j)=\mu(1-\mu)/(K+1)\) from Exercise 2.15,

\begin{equation*} K = \frac{\mu(1-\mu)}{\operatorname{var}(\theta_j)}-1 = 263.0, \qquad \alpha = \mu K = 13.1, \qquad \beta = (1-\mu)K = 249.9 . \end{equation*}

So the prior carries about as much information as a poll of 263 people. (Maximizing the beta-binomial marginal likelihood \(\prod_j p(y_j\mid\alpha,\beta)\) from Exercise 2.16(a) instead gives \(\hat\alpha=16.29\), \(\hat\beta=316.0\), i.e. \(K=332\) and \(\mu=0.0490\); no posterior mean below moves by as much as \(0.004\), so the conclusions do not turn on the fitting method.)

Estimates. Shrinkage is severe for the small states and negligible for the large ones:

The raw proportions span \([0.0063,\,0.1014]\) with standard deviation \(0.0187\); the posterior means span only \([0.0330,\,0.0753]\) with standard deviation \(0.0099\). Both extremes of the raw scale are small-sample artifacts – New Hampshire’s \(1/158\) and Idaho’s \(14/138\) – and both are pulled most of the way back to the national \(5\%\).

Graphs. The four panels (Figure bda3-ch02-very-liberal-by-state) plot each state as its two-letter abbreviation. Panel (a), raw proportion against Obama’s 2008 share, is a shapeless cloud of correlation \(0.18\); panel (b), posterior mean against Obama share, tightens to correlation \(0.21\) on a vertical scale less than half as tall. Panels (c) and (d) put \(n_j\) (log scale) on the horizontal axis and make the point of the exercise visible: in (c) the spread of \(p_j\) narrows like \(n_j^{-1/2}\), the outlying rates all belonging to states with a few hundred respondents, exactly as in Figure 2.8 for the kidney-cancer rates; in (d) the posterior means show no such funnel, because the small-\(n_j\) states are the ones nearly replaced by the prior mean \(\alpha/(\alpha+\beta)=0.0498\), precisely the flattening seen in Figure 2.9a.

Take, respectively, \(\theta\sim\mathrm{U}(-1,1)\), \(\theta\sim\mathrm{N}(0.03,\,0.02^{2})\), and \(\theta\sim\mathrm{N}(0,\,0.2^{2})\).

Each student contributes \(\hat\theta_i=\hat p_i^{\text{final}}-\hat p_i^{\text{base}}\), a difference of two independent binomial proportions out of \(100\) shots, so at a baseline near \(p=0.65\)

\begin{equation*} \begin{aligned} \mathrm{sd}(\hat\theta_i)&=\sqrt{\tfrac{2p(1-p)}{100}}\approx 0.067,\\ \mathrm{sd}(\bar{\hat\theta})&=\mathrm{sd}(\hat\theta_i)/\sqrt{100}\approx 0.0067, \end{aligned} \end{equation*}

making the likelihood for \(\theta\) nearly normal with scale about \(0.007\) (a floor: between-student variation in true improvement only widens it). Each prior below is judged against that number.

(a) Noninformative: \(p(\theta)\propto 1\) on \([-1,1]\), flat over the whole logically possible range for a difference of two probabilities; since the likelihood is confined to a window of width \(\approx 0.03\) deep inside that interval, the improper version \(p(\theta)\propto 1\) on \(\mathbb{R}\) gives the same posterior to several decimal places.

(b) Subjective: \(\theta\sim\mathrm{N}(0.03,\,0.02^{2})\), a \(95\%\) prior interval of \((-0.009,\,0.069)\). About \(1500\) practice shots over a month is genuine but modest training, so I expect a gain of roughly three percentage points, would be astonished by ten (that moves a typical \(65\%\) shooter to \(75\%\)), and leave a little mass below zero for fatigue or a baseline inflated by chance. Prior scale \(0.02\) is three times the likelihood scale, so this is mildly informative: the posterior standard deviation drops to

\begin{equation*} \Big(0.02^{-2}+0.0067^{-2}\Big)^{-1/2}\approx 0.0064, \end{equation*}

with weight \(0.02^{-2}/(0.02^{-2}+0.0067^{-2})\approx 0.10\) on the prior mean.

(c) Weakly informative: \(\theta\sim\mathrm{N}(0,\,0.2^{2})\) (or \(t_4\) at \(0\) with scale \(0.1\) for heavier tails), a \(95\%\) prior interval of \((-0.39,\,0.39)\). Centered at the null value \(\theta=0\) and wide enough that any reasonable dataset dominates it — the posterior standard deviation moves by \(0.06\%\) (Check!) — yet proper, and still ruling out the absurd, since a student starting near \(0.65\) cannot improve by \(0.8\).

(a) \(\alpha\mid y\sim\mathrm{Beta}(a_1+y_1,\,a_2+y_2)\), writing \(\mathrm{Dirichlet}(a_1,\dots,a_J)\) for the prior.

By Section 3.4 the Dirichlet is conjugate to the multinomial, so \(\theta\mid y\sim\mathrm{Dirichlet}(a_1+y_1,\dots,a_J+y_J)\). Use the gamma construction of the Dirichlet (Appendix A): let \(x_1,\dots,x_J\) be independent with \(x_j\sim\mathrm{Gamma}(a_j+y_j,1)\) (all shape parameters positive, as required), and set \(\theta_j=x_j/\sum_k x_k\). Then

\begin{equation*} \alpha=\frac{\theta_1}{\theta_1+\theta_2}=\frac{x_1}{x_1+x_2}, \end{equation*}

and the ratio of a \(\mathrm{Gamma}(a_1+y_1,1)\) variate to its sum with an independent \(\mathrm{Gamma}(a_2+y_2,1)\) variate is \(\mathrm{Beta}(a_1+y_1,a_2+y_2)\) (Appendix A). Since \(\alpha\) is a function of \(x_1,x_2\) alone, that is its marginal posterior – free of \(y_3,\dots,y_J\) and of \(a_3,\dots,a_J\).

(b) The same computation with \(y=0\) gives the induced prior \(\alpha\sim\mathrm{Beta}(a_1,a_2)\). Treating \(y_1\mid\alpha\sim\mathrm{Bin}(y_1+y_2,\alpha)\) contributes the likelihood factor \(\alpha^{y_1}(1-\alpha)^{y_2}\), so

\begin{equation*} \begin{aligned} p(\alpha\mid y_1,y_2)&\propto \alpha^{a_1-1}(1-\alpha)^{a_2-1}\cdot\alpha^{y_1}(1-\alpha)^{y_2}\\ &=\alpha^{a_1+y_1-1}(1-\alpha)^{a_2+y_2-1}, \end{aligned} \end{equation*}

that is, \(\mathrm{Beta}(a_1+y_1,a_2+y_2)\) – the distribution found in (a).

\(\Pr(\alpha_2>\alpha_1\mid y)=0.19\): the data give no evidence of a shift toward Bush, and in fact mildly favour a shift away.

Put independent uniform (that is, \(\mathrm{Dirichlet}(1,1,1)\)) priors on the two survey probability vectors. By Section 3.4 the posteriors are independent \(\mathrm{Dirichlet}(295,308,39)\) and \(\mathrm{Dirichlet}(289,333,20)\), and by Exercise 3.1 the induced marginals of the two Bush-versus-Dukakis proportions are independent with

\begin{equation*} \alpha_1\mid y\sim\mathrm{Beta}(295,308),\qquad \alpha_2\mid y\sim\mathrm{Beta}(289,333). \end{equation*}

Their posterior means are \(295/603=0.489\) and \(289/622=0.465\).

Drawing \(2\times10^5\) independent pairs from these two beta distributions and forming \(\alpha_2-\alpha_1\) gives

\begin{equation*} \begin{aligned} \mathrm{E}(\alpha_2-\alpha_1\mid y)&=-0.025,\\ \mathrm{sd}(\alpha_2-\alpha_1\mid y)&=0.028, \end{aligned} \end{equation*}

(Monte Carlo standard error \(0.0001\) on each), with central 95% posterior interval \([-0.080,\,0.031]\) and

\begin{equation*} \Pr(\alpha_2-\alpha_1>0\mid y)=0.19. \end{equation*}

The histogram is essentially normal, centred just below zero, with nearly a fifth of its mass to the right of \(0\).

(a) \(\mu_c\mid y\sim t_{31}(1.013,\,0.24^2/32)\) and \(\mu_t\mid y\sim t_{35}(1.173,\,0.20^2/36)\), independently.

The uniform prior on \((\mu_c,\mu_t,\log\sigma_c,\log\sigma_t)\) factors across the two groups and is the noninformative prior \(p(\mu,\sigma^2)\propto\sigma^{-2}\) of Section 3.2 within each group; the two likelihoods also factor, so the two posteriors are independent and each is the standard one derived there,

\begin{equation*} \mu\mid y\sim t_{n-1}\!\left(\bar y,\ s^2/n\right). \end{equation*}

Numerically the scales are \(s_c/\sqrt{n_c}=0.24/\sqrt{32}=0.04243\) and \(s_t/\sqrt{n_t}=0.20/6=0.03333\).

(b) \(\mu_t-\mu_c\) is the difference of these two independent \(t\) variables; it has no closed form (this is the Behrens–Fisher problem), so simulate. Its mean and variance are available exactly, since \(t_\nu\) has variance \(\nu/(\nu-2)\) times its squared scale:

\begin{equation*} \begin{aligned} \mathrm{E}(\mu_t-\mu_c\mid y)&=1.173-1.013=0.160,\\ \mathrm{var}(\mu_t-\mu_c\mid y)&=\tfrac{0.20^2}{36}\cdot\tfrac{35}{33}+\tfrac{0.24^2}{32}\cdot\tfrac{31}{29}, \end{aligned} \end{equation*}

giving \(\mathrm{sd}(\mu_t-\mu_c\mid y)=0.0557\). Drawing \(2\times10^5\) independent pairs from the two \(t\) distributions reproduces these (\(0.1602\) and \(0.0557\)) and gives the central 95% posterior interval

\begin{equation*} \mu_t-\mu_c\in[0.050,\ 0.269], \end{equation*}

with \(\Pr(\mu_t>\mu_c\mid y)=0.997\). The histogram is unimodal and symmetric about \(0.16\), with slightly heavier tails than a normal and essentially all of its mass positive.

(a) Take independent uniform priors, \(p_0,p_1\sim\mathrm{Beta}(1,1)\). By Section 2.1 the posteriors are independent,

\begin{equation*} p_0\mid y\sim\mathrm{Beta}(40,636),\qquad p_1\mid y\sim\mathrm{Beta}(23,659), \end{equation*}

and \(2\times10^5\) independent draws from each beta distribution are posterior simulations of \((p_0,p_1)\).

(b) Transform each draw to \(\rho=\dfrac{p_1/(1-p_1)}{p_0/(1-p_0)}\). The simulations give

\begin{equation*} \begin{aligned} \text{median}(\rho\mid y)&=0.55,\qquad \mathrm{E}(\rho\mid y)=0.57,\\ \text{95\% interval}&=[0.32,\ 0.93],\\ \Pr(\rho<1\mid y)&=0.988 . \end{aligned} \end{equation*}

On the log scale the posterior is close to normal, \(\mathrm{E}(\log\rho\mid y)=-0.60\) with \(\mathrm{sd}=0.27\). Beta-blockers reduce the odds of death by roughly 45%, and the posterior gives odds of about 80 to 1 that the effect is beneficial.

(c) Negligible. Repeating (a)–(b) with the Jeffreys prior \(\mathrm{Beta}(\tfrac12,\tfrac12)\) and with the Haldane prior \(\mathrm{Beta}(0,0)\) (proper posterior here, since no cell count is \(0\)) gives

\begin{equation*} \begin{aligned} \mathrm{Beta}(\tfrac12,\tfrac12):&\quad \text{median }0.55,\ [0.32,\,0.92],\ \Pr(\rho<1\mid y)=0.989,\\ \mathrm{Beta}(0,0):&\quad \text{median }0.54,\ [0.31,\,0.92],\ \Pr(\rho<1\mid y)=0.989 . \end{aligned} \end{equation*}

Each prior contributes at most one pseudo-observation per cell against 39 and 22 deaths, so it moves the posterior median of \(\rho\) by under 2% and the interval endpoints by about \(0.01\). The choice would matter only if a cell count were near zero.

(a) With \(n=5\), \(\bar y=10.4\) and \(s^2=1.3\), the noninformative-prior results (3.5) and (3.3) of Section 3.2 give

\begin{equation*} \sigma^2\mid y\sim\text{Inv-}\chi^2(4,\,1.3),\qquad \mu\mid\sigma^2,y\sim\mathrm{N}(10.4,\ \sigma^2/5), \end{equation*}

so that \(\mu\mid y\sim t_4(10.4,\ 0.26)\).

(b) The data are \(y_i=\mathrm{round}(z_i)\), so observing \(y_i\) is exactly the event \(z_i\in(y_i-\tfrac12,\,y_i+\tfrac12)\). Hence the correct likelihood is the probability of that event, not a density:

\begin{equation*} p(y\mid\mu,\sigma^2)=\prod_{i=1}^{5}\left[ \Phi\!\left(\frac{y_i+\frac12-\mu}{\sigma}\right)- \Phi\!\left(\frac{y_i-\frac12-\mu}{\sigma}\right)\right], \end{equation*}

and with \(p(\mu,\sigma^2)\propto\sigma^{-2}\),

\begin{equation*} p(\mu,\sigma^2\mid y)\propto\sigma^{-2}\prod_{i=1}^{5}\left[ \Phi\!\left(\frac{y_i+\frac12-\mu}{\sigma}\right)- \Phi\!\left(\frac{y_i-\frac12-\mu}{\sigma}\right)\right]. \end{equation*}

This has no conjugate form; evaluate it on a grid in \((\mu,\log\sigma)\).

(c) They agree about \(\mu\) and disagree about \(\sigma\): rounding inflates the naive variance estimate. Normalizing both densities on a grid covering \(\mu\in[-20,41]\), \(\sigma\in[0.05,400]\):

The posterior of \(\mu\) is unchanged because rounding is symmetric about each \(y_i\); the posterior of \(\sigma^2\) shrinks by \(2.429/2.599=0.935\), almost exactly Sheppard’s correction factor \((s^2-\tfrac1{12})/s^2=1.217/1.3=0.936\), the naive \(s^2\) having absorbed the variance \(\tfrac1{12}\) of the rounding error. Contour plots of the two densities in \((\mu,\sigma)\) are nested teardrop-shaped curves, symmetric about \(\mu=10.4\) and flaring outwards in \(\mu\) as \(\sigma\) grows; the two sets very nearly coincide, with the rounded-likelihood contours displaced downwards in \(\sigma\) by a few percent.

(d) \(\mathrm{E}\!\left[(z_1-z_2)^2\mid y\right]=0.159\).

Given \((\mu,\sigma)\) and \(y\), the \(z_i\) are independent, \(z_i\) being \(\mathrm{N}(\mu,\sigma^2)\) truncated to \((y_i-\tfrac12,\,y_i+\tfrac12)\); simulate by the inverse-cdf method. Since \(y_1=y_2=10\), \(z_1\) and \(z_2\) are conditionally i.i.d.\ on \((9.5,10.5)\), so

\begin{equation*} \mathrm{E}\!\left[(z_1-z_2)^2\mid\mu,\sigma\right]=2\,\mathrm{var}(z_1\mid\mu,\sigma), \end{equation*}

and averaging this truncated-normal variance over the grid posterior of (b) gives \(0.1592\) without simulation error; \(2\times10^5\) raw draws of \((z_1-z_2)^2\) reproduce it as \(0.159\), Monte Carlo standard error \(0.0003\).

(a) It is uniform in \((\log\lambda,\theta)\): \(\lambda=\mathrm{E}(y_i)\) is a positive scale parameter, for which Section 2.8 prescribes the flat prior on the log scale, and \(\theta\) is a probability on the bounded range \([0,1]\), for which the flat prior is the natural noninformative choice. The distribution is improper, since \(\int_0^\infty\lambda^{-1}\,d\lambda\) diverges at both ends.

For the transformation, first change variables from \(\lambda\) to \(\mu=\lambda/\theta\) at fixed \(\theta\); the Jacobian is \(|\partial\lambda/\partial\mu|=\theta\), so

\begin{equation*} p(\mu,\theta)\propto(\mu\theta)^{-1}\cdot\theta=\mu^{-1}. \end{equation*}

Now average the Poisson prior for \(N\) over \(\mu\):

\begin{equation*} \begin{aligned} p(N,\theta)&\propto\int_0^\infty\frac{e^{-\mu}\mu^{N}}{N!}\,\mu^{-1}\,d\mu\\ &=\frac{\Gamma(N)}{N!}=\frac1N,\qquad N=1,2,\dots \end{aligned} \end{equation*}

So \(p(N,\theta)\propto N^{-1}\) on \(N\ge1\), uniform in \(\theta\), and again improper since \(\sum_{N\ge1}N^{-1}=\infty\).

(b) \(\Pr(N>100\mid y)=0.96\).

With \(n=5\), \(\sum y_i=315\) and \(\max y_i=72\), the posterior is

\begin{equation*} p(N,\theta\mid y)\propto\frac1N\prod_{i=1}^{5}\binom{N}{y_i}\,\theta^{315}(1-\theta)^{5N-315}, \end{equation*}

supported on \(N\ge72\). Integrating \(\theta\) out with the beta integral gives the exact marginal

\begin{equation*} p(N\mid y)\propto\frac1N\left[\prod_{i=1}^{5}\binom{N}{y_i}\right]B(316,\,5N-314), \end{equation*}

and \(\theta\mid N,y\sim\mathrm{Beta}(316,\,5N-314)\), so simulation is exact: draw \(N\) from the tabulated marginal, then \(\theta\) from its beta conditional.

This marginal is proper but extremely heavy-tailed. As \(N\to\infty\), \(\prod_i\binom{N}{y_i}\sim N^{315}/\prod_i y_i!\) and \(B(316,5N-314)\sim\Gamma(316)(5N)^{-316}\), so

\begin{equation*} p(N\mid y)\sim c\,N^{-2}, \end{equation*}

with \(c=119.8\) after normalizing. Hence \(\mathrm{E}(N\mid y)=\infty\) and only quantiles are meaningful:

with posterior mode \(N=122\) and \(\Pr(N>100\mid y)=0.958\). The corresponding draws of \(\theta\) have median \(0.27\) and 95% interval \([0.013,\,0.66]\).

The scatterplot of \((N,\theta)\) on a log-\(N\) axis is a narrow hyperbolic ridge running from \((N,\theta)\approx(80,0.8)\) down to \((N,\theta)\approx(3000,0.02)\): \(\log N\) and \(\log\theta\) have posterior correlation \(-0.999\). What the data determine is the product \(\lambda=N\theta\) – posterior median \(63.1\), 95% interval \([57.3,\,69.2]\) – and almost nothing about where along the ridge the truth lies.

(c) Because a fixed \(\mu\) would determine the answer: \(\mathrm{Poisson}(\mu)\) has relative standard deviation \(\mu^{-1/2}\), so it pins \(N\) to within a few percent, while the data themselves say almost nothing about \(N\) given \(\lambda\) – that is the ridge in (b). The posterior would be an artifact of the chosen \(\mu\). And \(\mu\) is the mean of the unobserved \(N\), precisely the quantity Raftery says one has no basis for describing.

The Poisson likelihood factors as (a Poisson likelihood for the total \(b+v\), involving only \(\tau=\theta_b+\theta_v\)) times (exactly the binomial likelihood of the second model). Reparametrize by \((p,\tau)\), a bijection of \((0,\infty)^2\) onto \((0,1)\times(0,\infty)\) with \(\theta_b=p\tau\) and \(\theta_v=(1-p)\tau\):

\begin{equation*} \begin{aligned} p(b,v\mid\theta_b,\theta_v) &=\frac{e^{-\theta_b}\theta_b^{\,b}}{b!}\cdot\frac{e^{-\theta_v}\theta_v^{\,v}}{v!}\\ &=\frac{e^{-\tau}\tau^{\,b+v}}{b!\,v!}\,p^{b}(1-p)^{v}\\ &=\underbrace{\frac{e^{-\tau}\tau^{\,b+v}}{(b+v)!}}_{\mathrm{Poisson}(b+v\mid\tau)} \cdot\underbrace{\binom{b+v}{b}p^{b}(1-p)^{v}}_{\mathrm{Bin}(b\mid b+v,\,p)}, \end{aligned} \end{equation*}

using \(\binom{b+v}{b}/(b+v)!=1/(b!\,v!)\) on the last line. The first factor is free of \(p\), so as functions of \(p\) at any fixed \(\tau\) – equivalently, conditional on the observed total \(b+v\), which is ancillary for \(p\) – the two likelihoods are proportional.

(a) Take the two normal models

\begin{equation*} y_i \mid \mu_y,\sigma_y^2 \sim \mathrm{N}(\mu_y,\sigma_y^2), \qquad z_i \mid \mu_z,\sigma_z^2 \sim \mathrm{N}(\mu_z,\sigma_z^2), \end{equation*}

independently, so \(\theta_y=(\mu_y,\sigma_y^2)\) and \(\theta_z=(\mu_z,\sigma_z^2)\): the proportions themselves are the data, and the parameters describe block-to-block variation. A single binomial for the counts is untenable here, since it would force all the spread in \(y_i\) to come from the block totals, and the ten bike-route proportions range from \(0.091\) to \(0.462\) on totals in the hundreds. (Exercise 5.13 repairs this with a hierarchical beta-binomial.)

(b) Use the standard noninformative prior of Section 3.2 separately in each stratum,

\begin{equation*} p(\mu_y,\sigma_y^2,\mu_z,\sigma_z^2)\;\propto\;\sigma_y^{-2}\,\sigma_z^{-2}, \end{equation*}

which is uniform on \((\mu_y,\log\sigma_y,\mu_z,\log\sigma_z)\) and factors into independent pieces.

(c) The two strata are a priori independent and involve disjoint data, so the posterior factors and each factor is given by (3.5) and (3.3):

\begin{equation*} \begin{aligned} \sigma_y^2\mid y &\sim \text{Inv-}\chi^2(n_y-1,\;s_y^2), & \mu_y\mid\sigma_y^2,y &\sim \mathrm{N}(\bar y,\;\sigma_y^2/n_y),\\ \sigma_z^2\mid z &\sim \text{Inv-}\chi^2(n_z-1,\;s_z^2), & \mu_z\mid\sigma_z^2,z &\sim \mathrm{N}(\bar z,\;\sigma_z^2/n_z), \end{aligned} \end{equation*}

with \(n_y=10\), \(n_z=8\). The observed proportions are

\begin{equation*} \begin{aligned} y &= (0.2162,\,0.0909,\,0.1724,\,0.1857,\,0.1557,\\ &\qquad 0.2597,\,0.1731,\,0.1318,\,0.1136,\,0.4622),\\ z &= (0.0960,\,0.0526,\,0.1250,\,0.0833,\,0.0415,\\ &\qquad 0.0946,\,0.2368,\,0.0494), \end{aligned} \end{equation*}

giving the sufficient statistics

\begin{equation*} \bar y = 0.1961,\quad s_y = 0.1055,\qquad \bar z = 0.0974,\quad s_z = 0.0631 . \end{equation*}

Marginally \(\mu_y\mid y\sim t_9(\bar y, s_y^2/10)\) and \(\mu_z\mid z\sim t_7(\bar z, s_z^2/8)\), with \(95\%\) central intervals \([0.121,\,0.272]\) and \([0.045,\,0.150]\). Drawing \(\sigma^2\) from its scaled inverse-\(\chi^2\) and then \(\mu\) from its normal gives 1000 independent draws of \((\mu_y,\mu_z)\).

(d) Differencing the draws estimates the posterior of the estimand \(\mu_y-\mu_z\):

\begin{equation*} \begin{aligned} E(\mu_y-\mu_z\mid y,z) &= 0.099, &\mathrm{sd} &= 0.046,\\ 95\%\ \text{interval} &= [\,0.007,\;0.190\,], &\Pr(\mu_y>\mu_z\mid y,z) &= 0.98 . \end{aligned} \end{equation*}

These are computed from \(10^5\) draws; with the 1000 the exercise asks for, the Monte Carlo error on the mean is \(0.046/\sqrt{1000}=0.0015\) and the interval endpoints and tail probability are good only to about \(\pm 0.01\). The histogram (figure bda3-ch03-bike-lane-difference) is unimodal and slightly right-skewed about \(0.10\), the skew inherited from the outlying block \(55/64\), which inflates \(s_y\).

The parameters are

\begin{equation*} \begin{aligned} \mu_n &= \frac{\kappa_0}{\kappa_0+n}\,\mu_0 + \frac{n}{\kappa_0+n}\,\bar y,\\ \kappa_n &= \kappa_0+n, \qquad \nu_n = \nu_0+n,\\ \nu_n\sigma_n^2 &= \nu_0\sigma_0^2 + (n-1)s^2

• \frac{\kappa_0 n}{\kappa_0+n}(\bar y-\mu_0)^2 . \end{aligned} \end{equation*}

Multiplying the prior density (3.6) by the likelihood (3.2) gives, as in (3.7),

\begin{equation*} p(\mu,\sigma^2\mid y) \propto \sigma^{-1}(\sigma^2)^{-(\nu_0/2+1)}(\sigma^2)^{-n/2}\,e^{-Q/(2\sigma^2)}, \end{equation*}

\begin{equation*} Q = \nu_0\sigma_0^2 + \kappa_0(\mu-\mu_0)^2 + (n-1)s^2 + n(\bar y-\mu)^2, \end{equation*}

where \(\bar y\) and \(s^2\) are the sufficient statistics of (3.2). Everything rests on the identity

\begin{equation*} \begin{aligned} \kappa_0(\mu-\mu_0)^2 + n(\mu-\bar y)^2 &= (\kappa_0+n)\mu^2 - 2(\kappa_0\mu_0+n\bar y)\mu\\ &\qquad + \kappa_0\mu_0^2 + n\bar y^2\\ &= \kappa_n(\mu-\mu_n)^2\\ &\qquad + \kappa_0\mu_0^2 + n\bar y^2 - \kappa_n\mu_n^2, \end{aligned} \end{equation*}

with \(\kappa_n=\kappa_0+n\) and \(\mu_n=(\kappa_0\mu_0+n\bar y)/\kappa_n\) read off as the coefficient ratio. The leftover constant simplifies:

\begin{equation*} \begin{aligned} \kappa_0\mu_0^2 + n\bar y^2 - \frac{(\kappa_0\mu_0+n\bar y)^2}{\kappa_0+n} &= \frac{\kappa_0 n(\mu_0^2 - 2\mu_0\bar y + \bar y^2)}{\kappa_0+n}\\ &= \frac{\kappa_0 n}{\kappa_0+n}(\bar y-\mu_0)^2 . \end{aligned} \end{equation*}

(Check! Expand the numerator of the first line over the common denominator \(\kappa_0+n\).) Hence

\begin{equation*} Q = \nu_n\sigma_n^2 + \kappa_n(\mu-\mu_n)^2 \end{equation*}

with \(\nu_n\sigma_n^2\) as displayed above, and

\begin{equation*} p(\mu,\sigma^2\mid y)\propto \sigma^{-1}(\sigma^2)^{-(\nu_n/2+1)} \exp\!\Big[-\tfrac{1}{2\sigma^2}\big(\nu_n\sigma_n^2+\kappa_n(\mu-\mu_n)^2\big)\Big], \end{equation*}

since \(\nu_0/2 + 1 + n/2 = \nu_n/2 + 1\). Comparing with (3.6) identifies this as \(\text{N-Inv-}\chi^2(\mu_n,\sigma_n^2/\kappa_n;\ \nu_n,\sigma_n^2)\).

The quantity is a ratio of two independent \(\chi^2\) variables, each over its own degrees of freedom. The prior factors across \(j\) and the two samples are independent given the parameters, so the posterior factors too:

\begin{equation*} p(\mu_1,\sigma_1^2,\mu_2,\sigma_2^2\mid y) = \prod_{j=1}^{2} p(\mu_j,\sigma_j^2\mid y_j), \end{equation*}

each factor being the noninformative-prior normal posterior of Section 3.2. Its marginal for the variance is (3.5):

\begin{equation*} \sigma_j^2\mid y \sim \text{Inv-}\chi^2(n_j-1,\ s_j^2), \qquad j=1,2, \end{equation*}

independently, where \(s_j^2\) is the sample variance of group \(j\) (and \(n_j\ge 2\), so the degrees of freedom are positive). By the definition of the scaled inverse-\(\chi^2\) distribution (Appendix A), \(X\sim\text{Inv-}\chi^2(\nu,s^2)\) means exactly \(\nu s^2/X\sim\chi^2_\nu\); hence

\begin{equation*} u_j := \frac{(n_j-1)s_j^2}{\sigma_j^2}\ \Big|\ y \ \sim\ \chi^2_{n_j-1}, \end{equation*}

and \(u_1,u_2\) are posterior independent. Now simply rearrange the quantity of interest:

\begin{equation*} \frac{s_1^2/s_2^2}{\sigma_1^2/\sigma_2^2} = \frac{s_1^2/\sigma_1^2}{s_2^2/\sigma_2^2} = \frac{u_1/(n_1-1)}{u_2/(n_2-1)} . \end{equation*}

The right-hand side is a ratio of two independent \(\chi^2\) variables each divided by its own degrees of freedom, which is the definition of the \(F\) distribution (Appendix A). Therefore

\begin{equation*} \frac{s_1^2/s_2^2}{\sigma_1^2/\sigma_2^2}\ \Big|\ y \ \sim\ F_{n_1-1,\,n_2-1}. \end{equation*}

(a) The prior is the bivariate normal

\begin{equation*} \begin{pmatrix}\alpha\\beta\end{pmatrix} \sim \mathrm{N}\!\left( \begin{pmatrix}0\\10\end{pmatrix},\ \begin{pmatrix}4 & 10\\ 10 & 100\end{pmatrix} \right), \end{equation*}

since \(\mathrm{cov}(\alpha,\beta)=0.5\cdot 2\cdot 10=10\), so by (3.15) the unnormalized log posterior is

\begin{equation*} \begin{aligned} \log p(\alpha,\beta\mid y) &= \sum_{i=1}^{4}\Big[y_i(\alpha+\beta x_i)\\ &\qquad - n_i\log\!\big(1+e^{\alpha+\beta x_i}\big)\Big]\\ &\quad -\tfrac12 \begin{pmatrix}\alpha\\beta-10\end{pmatrix}^{\!T}\!\! \begin{pmatrix}4&10\\10&100\end{pmatrix}^{-1}\!\! \begin{pmatrix}\alpha\\beta-10\end{pmatrix} +\ \mathrm{const}. \end{aligned} \end{equation*}

Evaluating this on a \(600\times 600\) grid over \([-5,10]\times[-10,40]\) (the same range as Figure 3.3), subtracting the maximum before exponentiating, and normalizing the grid to total probability 1 gives a posterior mode at \((\hat\alpha,\hat\beta)=(0.71,\,7.95)\) and moments

\begin{equation*} \begin{aligned} E(\alpha\mid y)&=0.98, & \mathrm{sd}(\alpha\mid y)&=0.90,\\ E(\beta\mid y)&=10.48, & \mathrm{sd}(\beta\mid y)&=4.60, \end{aligned} \qquad \mathrm{corr}(\alpha,\beta\mid y)=0.60 . \end{equation*}

Drawing 1000 values by the marginal-then-conditional grid method of Section 3.7 (draw \(\alpha^s\) from the numerically summed \(p(\alpha\mid y)\), then \(\beta^s\) from \(p(\beta\mid\alpha^s,y)\), then jitter by a uniform of grid width) gives

\begin{equation*} \begin{aligned} \alpha:&\quad \text{mean } 0.94,\ \ 95\%\ [-0.56,\ 2.81],\\ \beta:&\quad \text{mean } 10.27,\ \ 95\%\ [3.53,\ 20.42], \end{aligned} \end{equation*}

so the Monte Carlo error on those means is \(0.90/\sqrt{1000}=0.03\) and \(4.60/\sqrt{1000}=0.15\). All 1000 draws have \(\beta>0\); summing the grid gives \(\Pr(\beta>0\mid y)=0.99999\). The LD50 \(=-\alpha/\beta\), conditional on \(\beta>0\) as in Section 3.7, has posterior median \(-0.093\) and \(95\%\) interval \([-0.246,\ 0.119]\) on the log-dose scale (grid values; the 1000 draws reproduce them to \(\pm 0.01\)).

(b) Compare the three sets of moments. The prior has \((E\alpha,E\beta)=(0,10)\) with sds \((2,10)\); the likelihood alone, that is the uniform-prior posterior of Section 3.7, has

\begin{equation*} \begin{aligned} E(\alpha\mid y)&=1.31, & \mathrm{sd}&=1.10,\\ E(\beta\mid y)&=11.61, & \mathrm{sd}&=5.71, \end{aligned} \end{equation*}

and the new posterior sits strictly between the two in location, \(0 < 0.98 < 1.31\) and \(10 < 10.48 < 11.61\), and is tighter than either in scale, \(0.90 < \min(2,1.10)\) and \(4.60 < \min(10,5.71)\). The contour plot keeps the banana shape of Figure 3.3a — the prior is too diffuse to impose its elliptical contours — but is pulled down and to the left and is visibly narrower, and the scatterplot of the 1000 draws is correspondingly a slightly compressed version of Figure 3.3b. (Figure bda3-ch03-bioassay-normal-prior.)

(c) The prior is weak relative to the data for \(\beta\) (prior sd 10 against a likelihood sd of 5.7) and only comparable for \(\alpha\) (prior sd 2 against 1.1), so it changes nothing qualitative: the compound is still judged harmful with posterior probability above \(0.999\), and the LD50 is still about \(-0.09\) on the log-dose scale, a dose of roughly \(e^{-0.09}=0.92\) g/ml. What the prior buys is precision: the posterior sds fall by \(18\%\) for \(\alpha\) and \(20\%\) for \(\beta\), and the LD50 interval narrows from \([-0.276,\,0.104]\) to \([-0.246,\,0.119]\). Since the prior encodes a genuine expectation from related bioassays — a steep positive slope near 10 and an intercept near 0, meaning a compound roughly 50-50 lethal at unit dose — the gain is legitimate; but with only 20 animals the analyst should say plainly that the shift in the LD50 median from \(-0.112\) to \(-0.093\) is prior-driven, not data-driven.

(a) Take \(p(\alpha,\beta)\propto 1\) on the region

\begin{equation*} R=\{(\alpha,\beta):\ \alpha+\beta t>0 \text{ for } t=1976,\dots,1985\}, \end{equation*}

which is forced on us, since the Poisson mean must be positive at every observed year. Three candidates are natural. The flat density on \(R\) is invariant under re-origining time, \(t\mapsto t-c\), which sends \((\alpha,\beta)\mapsto(\alpha+c\beta,\beta)\) with unit Jacobian, and this is the only reparametrization the problem really invites; it is improper but gives a proper posterior by (d). A second candidate is the Jeffreys prior

\begin{equation*} p(\alpha,\beta)\propto \Big|\textstyle\sum_t \frac{1}{\alpha+\beta t} \begin{pmatrix}1\\ t\end{pmatrix} \begin{pmatrix}1\\ t\end{pmatrix}^{\!T}\Big|^{1/2}, \end{equation*}

which is invariant but depends on the design and pulls mass toward the boundary of \(R\). A third is flat in \((\log\alpha,\log\beta)\), but \(\beta\) is expected to be negative here, so a log scale is inadmissible. We use the flat prior on \(R\).

(b) A realistic informative prior would say: worldwide fatal accidents run at a few dozen per year and are slowly declining. In the re-origined parameters \(\gamma=\alpha+1980.5\beta\) (the expected count at the center of the data) and \(\beta\), take

\begin{equation*} \gamma\sim\mathrm{N}(25,\,5^2),\qquad \beta\sim\mathrm{N}(-0.5,\,1^2), \end{equation*}

independent, whose contours are axis-aligned ellipses centered at \((25,-0.5)\) in \((\gamma,\beta)\), five times as wide in \(\gamma\) as tall in \(\beta\). Put aside.

(c) By (3.15) with the flat prior,

\begin{equation*} \begin{aligned} p(\alpha,\beta\mid y) &\propto \prod_{t=1976}^{1985}(\alpha+\beta t)^{y_t}e^{-(\alpha+\beta t)}\\ &= \exp\!\Big[-\big(10\alpha + \beta\textstyle\sum_t t\big)\Big] \prod_t (\alpha+\beta t)^{y_t}, \end{aligned} \end{equation*}

on \(R\) and zero off it, with \(\sum_t t = 19805\). There is no reduction: the minimal sufficient statistic is the full vector \(y=(y_{1976},\dots,y_{1985})\) of ten counts. The terms \(y_t\log(\alpha+\beta t)\) cannot be collected into finitely many data-dependent functions because the identity link makes this a curved, non-exponential family in \((\alpha,\beta)\). (Under a log link, \(\log\mu_t=\alpha+\beta t\), the sufficient statistics would collapse to \(\sum_t y_t\) and \(\sum_t t\,y_t\).)

(d) Change variables to the endpoint means

\begin{equation*} u=\alpha+1976\beta,\qquad v=\alpha+1985\beta, \end{equation*}

so \(d\alpha\,d\beta = du\,dv/9\) and, writing \(t_j=1975+j\) and \(w_j=(j-1)/9\) for \(j=1,\dots,10\),

\begin{equation*} \mu_j:=\alpha+\beta t_j=(1-w_j)u+w_jv . \end{equation*}

Each \(\mu_j\) is a convex combination of \(u\) and \(v\), so \(R=\{u>0,\ v>0\}\) and \(\mu_j\le u+v\). Also \(\sum_j w_j = 45/9 = 5\), hence \(\sum_j\mu_j = 5(u+v)\). With \(Y=\sum_t y_t = 238\),

\begin{equation*} \begin{aligned} \int_R \prod_j \mu_j^{y_j}e^{-\sum_j\mu_j}\,d\alpha\,d\beta &\le \frac19\int_0^\infty\!\!\int_0^\infty (u+v)^{Y}e^{-5(u+v)}du\,dv\\ &= \frac19\int_0^\infty s^{Y+1}e^{-5s}\,ds\\ &= \frac{\Gamma(240)}{9\cdot 5^{240}}<\infty, \end{aligned} \end{equation*}

the middle step by substituting \(s=u+v\) and integrating over \(u\in(0,s)\), which contributes the extra factor \(s\). The posterior is therefore proper.

(e) Least squares of \(y_t\) on \(t\), computed with the centered predictor \(x_t=t-1980.5\) (so \(\sum x_t^2=82.5\)), gives

\begin{equation*} \begin{aligned} \hat\gamma &= \bar y = 23.80 &&(\mathrm{se}\ 1.27),\\ \hat\beta &= -0.921 &&(\mathrm{se}\ 0.443), \end{aligned} \end{equation*}

with residual standard deviation \(s=4.02\) on \(8\) degrees of freedom. Transforming back, \(\hat\alpha=\hat\gamma-1980.5\hat\beta=1848\) with standard error \(877\) — the huge value and its huge uncertainty are artifacts of extrapolating the line back to \(t=0\), and \((\hat\alpha,\hat\beta)\) have correlation \(-1.000\) to three decimals.

(f) Evaluate the log posterior of (c) on a \(900\times 900\) grid in \((\gamma,\beta)\in[10,40]\times[-3,2]\), zeroing the cells outside \(R\), subtract the maximum, exponentiate and normalize. The mode is at \((\gamma,\beta)=(23.82,-0.948)\) and

\begin{equation*} \begin{aligned} E(\gamma\mid y)&=24.00, &\mathrm{sd}(\gamma\mid y)&=1.55,\\ E(\beta\mid y)&=-0.940, &\mathrm{sd}(\beta\mid y)&=0.540 . \end{aligned} \end{equation*}

The contours are close to elliptical and nearly axis-aligned in \((\gamma,\beta)\), which is exactly why the centered parameterization is the right one to draw. Sampling 1000 draws by the marginal-then-conditional grid method of Section 3.7 and mapping back by \(\alpha=\gamma-1980.5\beta\) gives

\begin{equation*} \beta:\ \text{mean } -0.94,\quad 95\%\ [-1.99,\ 0.12], \end{equation*}

matching the grid to within the Monte Carlo error \(0.54/\sqrt{1000}=0.017\) on the mean. Summing the grid, \(\Pr(\beta<0\mid y)=0.96\): the evidence for a declining trend is suggestive but not decisive. (Figure bda3-ch03-poisson-regression.)

(g) For each draw form \(\mu_{1986}=\alpha+1986\beta=\gamma+5.5\beta\). The histogram is unimodal and nearly symmetric with

\begin{equation*} E(\mu_{1986}\mid y)=18.8,\quad \mathrm{sd}=3.19,\quad 95\%\ [12.9,\ 25.4]. \end{equation*}

(h) Draw \(\tilde y^s\sim\mathrm{Poisson}(\mu_{1986}^s)\) for each draw of \(\mu_{1986}\). The predictive distribution has mean \(18.8\) and standard deviation \(5.4\), and its \(2.5\%\) and \(97.5\%\) quantiles give

\begin{equation*} \tilde y_{1986}\in[9,\ 30]. \end{equation*}

Discreteness, and the Monte Carlo error of 1000 draws, leave each endpoint uncertain by about one count.

(i) They agree. The informative prior of (b) put \(\gamma\) at \(25\pm 5\) and \(\beta\) at \(-0.5\pm 1\); the data-driven posterior of (f) puts them at \(24.0\pm 1.55\) and \(-0.940\pm 0.540\). Both prior means lie well inside the corresponding posterior intervals, and the prior is three to four times wider on each coordinate, so it would have contributed almost nothing had it been used. The one place the analyses part company is emphasis: the prior asserted a declining trend outright, whereas the data leave a \(4\%\) posterior probability that the trend is flat or rising.

Complete the square on the quadratic form in \(\mu\). From (3.11) and the normal prior,

\begin{equation*} \begin{aligned} &p(\mu\mid y,\Sigma)\\ &\propto \exp\!\Big[-\tfrac12\Big((\mu-\mu_0)^T\Lambda_0^{-1}(\mu-\mu_0)\\ &\qquad\qquad + \textstyle\sum_{i=1}^n (y_i-\mu)^T\Sigma^{-1}(y_i-\mu)\Big)\Big]. \end{aligned} \end{equation*}

Split the data term about \(\bar y=\frac1n\sum_i y_i\), the cross terms vanishing because \(\sum_i(y_i-\bar y)=0\):

\begin{equation*} \begin{aligned} \sum_{i=1}^n (y_i-\mu)^T\Sigma^{-1}(y_i-\mu) &= \sum_{i=1}^n (y_i-\bar y)^T\Sigma^{-1}(y_i-\bar y)\\ &\quad + n(\mu-\bar y)^T\Sigma^{-1}(\mu-\bar y), \end{aligned} \end{equation*}

and the first sum is free of \(\mu\), so it goes into the proportionality constant. What remains is

\begin{equation*} \begin{aligned} Q(\mu)&=(\mu-\mu_0)^T\Lambda_0^{-1}(\mu-\mu_0)\\ &\quad + n(\mu-\bar y)^T\Sigma^{-1}(\mu-\bar y)\\ &= \mu^T\big(\Lambda_0^{-1}+n\Sigma^{-1}\big)\mu\\ &\quad -2\mu^T\big(\Lambda_0^{-1}\mu_0+n\Sigma^{-1}\bar y\big) + c, \end{aligned} \end{equation*}

where \(c=\mu_0^T\Lambda_0^{-1}\mu_0+n\bar y^T\Sigma^{-1}\bar y\) is constant in \(\mu\); the two cross terms combined into one because \(\Lambda_0^{-1}\) and \(\Sigma^{-1}\) are symmetric, so \(\mu^TA\mu_0=\mu_0^TA\mu\) for \(A\) symmetric. Set

\begin{equation*} \Lambda_n^{-1}=\Lambda_0^{-1}+n\Sigma^{-1}, \qquad \mu_n=\Lambda_n\big(\Lambda_0^{-1}\mu_0+n\Sigma^{-1}\bar y\big). \end{equation*}

This is legitimate: \(\Lambda_0\) and \(\Sigma\) are positive definite variance matrices, hence so are their inverses and the sum \(\Lambda_n^{-1}\), which is therefore invertible. Then

\begin{equation*} \begin{aligned} (\mu-\mu_n)^T\Lambda_n^{-1}(\mu-\mu_n) &= \mu^T\Lambda_n^{-1}\mu - 2\mu^T\Lambda_n^{-1}\mu_n\\ &\quad + \mu_n^T\Lambda_n^{-1}\mu_n, \end{aligned} \end{equation*}

and \(\Lambda_n^{-1}\mu_n=\Lambda_0^{-1}\mu_0+n\Sigma^{-1}\bar y\) by construction, so \(Q(\mu)\) and this expression agree up to a constant. Hence

\begin{equation*} p(\mu\mid y,\Sigma)\propto \exp\!\Big[-\tfrac12(\mu-\mu_n)^T\Lambda_n^{-1}(\mu-\mu_n)\Big] = \mathrm{N}(\mu\mid\mu_n,\Lambda_n), \end{equation*}

which is (3.12).

The likelihood is bounded by \(e^{-\varphi}\) for a positively homogeneous convex \(\varphi\) that vanishes only at the origin, and such an exponential is integrable on the plane.

Write \(u_i=\alpha+\beta x_i\) and \(g(u)=\mathrm{logit}^{-1}(u)=(1+e^{-u})^{-1}\). Since \(\log(1+e^{-u})\ge\max(0,-u)\) and \(\log(1+e^{u})\ge\max(0,u)\),

\begin{equation*} \begin{aligned} \log g(u) &= -\log(1+e^{-u}) \le -\max(0,-u),\\ \log\big(1-g(u)\big) &= -\log(1+e^{u}) \le -\max(0,u). \end{aligned} \end{equation*}

Multiplying by the nonnegative weights \(y_i\) and \(n_i-y_i\) and summing,

\begin{equation*} L(\alpha,\beta):=\prod_{i=1}^4 g(u_i)^{y_i}\big(1-g(u_i)\big)^{n_i-y_i} \le e^{-\varphi(\alpha,\beta)}, \end{equation*}

\begin{equation*} \varphi(\alpha,\beta)=\sum_{i=1}^{4} \Big[y_i\,(u_i)^{-} + (n_i-y_i)\,(u_i)^{+}\Big], \end{equation*}

where \((u)^{+}=\max(0,u)\) and \((u)^{-}=\max(0,-u)\).

Two properties of \(\varphi\). It is nonnegative, and since each \(u_i=\alpha+\beta x_i\) is linear (no intercept) while \((\cdot)^{\pm}\) are positively homogeneous, \(\varphi(c\alpha,c\beta)=c\,\varphi(\alpha,\beta)\) for every \(c>0\). Second, \(\varphi(\alpha,\beta)=0\) forces, term by term:

\begin{equation*} \begin{aligned} \text{(i) } i=1\ (y_1=0,\ n_1-y_1=5):&\quad u_1\le 0,\\ \text{(ii) } i=2\ (y_2=1,\ n_2-y_2=4):&\quad u_2=0,\\ \text{(iii) } i=3\ (y_3=3,\ n_3-y_3=2):&\quad u_3=0,\\ \text{(iv) } i=4\ (y_4=5,\ n_4-y_4=0):&\quad u_4\ge 0. \end{aligned} \end{equation*}

Leaves (ii) and (iii) alone give \(\alpha-0.30\beta=0\) and \(\alpha-0.05\beta=0\); the coefficient matrix has determinant \(-0.05+0.30=0.25\neq 0\), so \((\alpha,\beta)=(0,0)\). Hence \(\varphi>0\) everywhere on the unit circle \(S\), and \(\varphi\) is continuous, so by compactness

\begin{equation*} c:=\min_{(\alpha,\beta)\in S}\varphi(\alpha,\beta)>0 \end{equation*}

(numerically \(c=0.2497\)). Homogeneity then upgrades this to a global bound, \(\varphi(v)\ge c\,\|v\|\) for all \(v=(\alpha,\beta)\in\mathbb{R}^2\). Therefore, in polar coordinates,

\begin{equation*} \begin{aligned} \int_{\mathbb{R}^2} L(\alpha,\beta)\,d\alpha\,d\beta &\le \int_{\mathbb{R}^2} e^{-c\|v\|}\,dv\\ &= 2\pi\int_0^\infty r e^{-cr}\,dr = \frac{2\pi}{c^2}<\infty, \end{aligned} \end{equation*}

so the posterior density (3.15) is proper.

(a) Only two factors of the joint density contain \(y_t\). For any \(n > t+1\) the model’s defining conditionals telescope into

\begin{equation*} p(y_1, \ldots, y_n) \;=\; p(y_1) \prod_{s=2}^{n} p(y_s \mid y_{s-1}), \end{equation*}

so, treating every \(y_s\) with \(s \ne t\) as fixed and dropping all factors free of \(y_t\),

\begin{equation*} \begin{aligned} p(y_t \mid y_{-t}) &\propto p(y_t \mid y_{t-1})\, p(y_{t+1} \mid y_t)\\ &\propto \exp\Big(-\tfrac12 (y_t - 0.8 y_{t-1})^2\Big)\\ &\qquad\times \exp\Big(-\tfrac12 (y_{t+1} - 0.8 y_t)^2\Big). \end{aligned} \end{equation*}

The right-hand side involves the conditioning variables only through \(y_{t-1}\) and \(y_{t+1}\), and the normalizing constant is its integral over \(y_t\); being free of \(n\), the result holds for every finite block and hence for the whole series, so \(p(y_t \mid y_{-t}) = p(y_t \mid y_{t-1}, y_{t+1})\).

(b) Complete the square in \(y_t\). The exponent above is

\begin{equation*} -\tfrac12\Big[(1 + 0.8^2)\,y_t^2 - 2(0.8)(y_{t-1} + y_{t+1})\,y_t\Big]

• \text{const}, \end{equation*}

a quadratic with precision \(1 + 0.64 = 1.64\) and linear coefficient \(0.8(y_{t-1} + y_{t+1})\), so

\begin{equation*} (y_t \mid y_{t-1}, y_{t+1}) \sim \mathrm{N}\!\left(\frac{0.8}{1.64}\,(y_{t-1} + y_{t+1}),\; \frac{1}{1.64}\right), \end{equation*}

that is, mean \(\tfrac{20}{41}(y_{t-1} + y_{t+1}) = 0.4878(y_{t-1} + y_{t+1})\) and variance \(\tfrac{25}{41} = 0.6098\).

(a) On \(0\le\theta\le 1\) the uniform prior is a constant, so \(\log p(\theta\mid y)=\text{const}-\sum_{i=1}^{5}\log\!\left(1+(y_i-\theta)^2\right)\) and

\begin{equation*} \frac{d}{d\theta}\log p(\theta\mid y) =\sum_{i=1}^{5}\frac{2(y_i-\theta)}{1+(y_i-\theta)^2}, \end{equation*}

\begin{equation*} \frac{d^2}{d\theta^2}\log p(\theta\mid y) =\sum_{i=1}^{5} \frac{2\left[(y_i-\theta)^2-1\right]}{\left[1+(y_i-\theta)^2\right]^2}, \end{equation*}

the second line by differentiating \(2u/(1+u^2)\) in \(u=y_i-\theta\) and multiplying by \(du/d\theta=-1\).

(b) Newton-Raphson, \(\theta^{t+1}=\theta^{t}-L^{\prime}(\theta^{t})/L^{\prime\prime}(\theta^{t})\) with \(L=\log p(\cdot\mid y)\), started at the sample median \(\theta^{0}=0\):

so the likelihood equation has the unique root \(\hat\theta=-0.137649\) (a scan of \(L^{\prime}\) over \([-6,6]\) shows only this one sign change, so the Cauchy likelihood is unimodal for these data).

This root lies outside the prior support \([0,1]\), on which \(L^{\prime}<0\) throughout, so the posterior mode as printed is the boundary point \(\theta=0\); both values are carried into (c).

(c) With \(I(\theta)=-L^{\prime\prime}(\theta)\),

\begin{equation*} I(\hat\theta)=1.374890,\qquad p(\theta\mid y)\approx \mathrm{N}(-0.1376,\;0.7273),\quad \text{sd}=0.8528, \end{equation*}

and at the boundary mode, \(I(0)=1.323551\), giving \(\mathrm{N}(0,\,0.7555)\), sd \(=0.8692\). Both are renormalized to \([0,1]\) for comparison. The exact density is obtained as in Exercise 2.11: evaluate the unnormalized \(\exp\{L(\theta)\}\) on a grid of \(20001\) points in \([0,1]\) and normalize by the trapezoidal sum. Densities (all normalized on \([0,1]\)):

Exact posterior mean and sd on \([0,1]\): \(0.4534\) and \(0.2845\); the truncated normal at \(\hat\theta\) gives \(0.4310\) and \(0.2772\), the one at the boundary mode \(0.4476\) and \(0.2795\). Against the exact density the maximum absolute errors are \(0.169\) and \(0.094\) (about \(13\%\) and \(8\%\) of the peak \(1.252\)): with \(n=5\) both reproduce the gentle downward slope but exaggerate it.

\begin{equation*} I(\alpha,\beta)= \begin{pmatrix} \sum_i n_i\theta_i(1-\theta_i) & \sum_i n_i x_i\theta_i(1-\theta_i)\\[2pt] \sum_i n_i x_i\theta_i(1-\theta_i) & \sum_i n_i x_i^2\theta_i(1-\theta_i) \end{pmatrix} = X^{T}WX , \end{equation*}

with \(X\) the \(4\times 2\) matrix of rows \((1,x_i)\) and \(W=\mathrm{Diag}\{n_i\theta_i(1-\theta_i)\}\), where \(\theta_i=\mathrm{logit}^{-1}(\alpha+\beta x_i)\).

Writing \(\eta_i=\alpha+\beta x_i\), the log posterior under the uniform prior (3.15) is (the binomial form presumes the \(n_i\) animals within a dose group are exchangeable, and the groups independent)

\begin{equation*} \log p(\alpha,\beta\mid y)=\text{const} +\sum_{i=1}^{k}\Bigl[y_i\eta_i-n_i\log\bigl(1+e^{\eta_i}\bigr)\Bigr], \end{equation*}

and since \(d\theta_i/d\eta_i=\theta_i(1-\theta_i)\),

\begin{equation*} \begin{aligned} \frac{\partial}{\partial\alpha}\log p &=\sum_i (y_i-n_i\theta_i),\\ \frac{\partial}{\partial\beta}\log p &=\sum_i x_i(y_i-n_i\theta_i),\\ \frac{\partial^2}{\partial\alpha^2}\log p &=-\sum_i n_i\theta_i(1-\theta_i),\\ \frac{\partial^2}{\partial\alpha\,\partial\beta}\log p &=-\sum_i n_i x_i\theta_i(1-\theta_i), \end{aligned} \end{equation*}

and \(\partial^2\log p/\partial\beta^2=-\sum_i n_i x_i^2\theta_i(1-\theta_i)\). The \(y_i\) have dropped out (the logit is the canonical link), so the observed information equals the expected information, and \(I\) is positive definite whenever the \(x_i\) are not all equal. The normal approximation (4.2) is therefore

\begin{equation*} p(\alpha,\beta\mid y)\approx \mathrm{N}\!\left(\begin{pmatrix}\hat\alpha\\ \hat\beta\end{pmatrix},\; \left(X^{T}\hat WX\right)^{-1}\right), \qquad \hat W=W\big|_{(\hat\alpha,\hat\beta)} . \end{equation*}

Numerically, the mode (equivalently the maximum likelihood estimate) is \((\hat\alpha,\hat\beta)=(0.8466,\,7.7487)\), giving

so that

\begin{equation*} I(\hat\alpha,\hat\beta) =\begin{pmatrix}1.9648 & -0.2934\\ -0.2934 & 0.08594\end{pmatrix}, \quad I^{-1}=\begin{pmatrix}1.0385 & 3.5459\\ 3.5459 & 23.743\end{pmatrix}, \end{equation*}

that is, posterior standard deviations \(1.02\) for \(\alpha\) and \(4.87\) for \(\beta\) with correlation \(0.714\) – the standard errors \(1.0\) and \(4.9\) quoted in Section 3.7.

\begin{equation*} \mathrm{LD50}\mid y \;\approx\; \mathrm{N}\!\left(-\frac{\hat\alpha}{\hat\beta},\; \nabla g^{T} I(\hat\alpha,\hat\beta)^{-1}\nabla g\right) =\mathrm{N}(-0.1093,\;0.09546^2), \end{equation*}

so the asymptotic posterior median is \(-0.1093\) and the asymptotic posterior standard deviation is \(0.0955\).

Write \(g(\alpha,\beta)=-\alpha/\beta\) and expand about the joint posterior mode \((\hat\alpha,\hat\beta)=(0.8466,\,7.7487)\) of Exercise 4.2:

\begin{equation*} g(\alpha,\beta)=-\frac{\hat\alpha}{\hat\beta} -\frac{1}{\hat\beta}(\alpha-\hat\alpha) +\frac{\hat\alpha}{\hat\beta^{2}}(\beta-\hat\beta)+O\!\left(\|\cdot\|^{2}\right), \end{equation*}

a linear function of \((\alpha,\beta)\) with gradient \(\nabla g=(-1/\hat\beta,\;\hat\alpha/\hat\beta^{2})=(-0.129053,\;0.014099)\). Under the normal approximation (4.2), a linear function of a normal vector is exactly normal, whence the display above; its median equals its mean \(g(\hat\alpha,\hat\beta)=-0.10925\), and with \(I^{-1}\) from Exercise 4.2,

\begin{equation*} \begin{aligned} \nabla g^{T}I^{-1}\nabla g &=1.0385(0.129053)^2-2(3.5459)(0.129053)(0.014099)\\ &\qquad +23.743(0.014099)^2\\ &=0.017296-0.012903+0.004720=0.009113, \end{aligned} \end{equation*}

so sd \(=\sqrt{0.009113}=0.09546\) and the asymptotic central \(95\%\) interval is \(-0.1093\pm1.96(0.0955)=(-0.296,\,0.078)\).

Against Figure 4.2: drawing \(2\times10^{5}\) values from the same \(\mathrm{N}(\hat\theta,I(\hat\theta)^{-1})\) and forming \(-\alpha/\beta\) on the \(94.5\%\) with \(\beta>0\) gives, averaged over twelve replicate runs, median \(-0.1118\) and central \(95\%\) interval \((-0.395,\,0.453)\), with between-run standard errors \(0.0002\), \(0.002\) and \(0.009\) respectively; so the delta-method centre agrees with the histogram but its interval, of width \(0.374\), is less than half the histogram’s \(0.848\). The histogram’s spread is not comparable to \(0.0955\) at all: since \(\hat\beta/\mathrm{sd}(\beta)=7.75/4.87=1.59\), the normal approximation puts mass arbitrarily close to \(\beta=0\), the simulated LD50 ranged over several thousand and its sample standard deviation does not converge. The delta method removes that tail by truncating at first order, and its interval is the closer of the two to the exact posterior of Figure 3.4 (grid computation: median \(-0.112\), central \(95\%\) interval \((-0.276,\,0.103)\), \(\Pr(\beta>0)>0.999\)).

Because the posterior concentrates: all but a vanishing fraction of the posterior mass sits in a neighborhood of \(\theta_0\) of width \(O(n^{-1/2})\), and on such a shrinking neighborhood a smooth \(f\) is linear to within \(o(n^{-1/2})\) – and a linear transformation of a normal distribution is exactly normal.

Quantitatively, let \(f\) be twice differentiable with \(f^{\prime}(\theta_0)\neq 0\) – exactly the condition under which the transformed problem still meets the regularity conditions of Section 4.2, since \(J_\phi=J/(f^{\prime})^2\) must stay finite and positive; a one-to-one \(f\) with vanishing derivative, such as \(\theta\mapsto\theta^{3}\) at \(\theta_0=0\), is excluded. Taylor expansion about \(\theta_0\) gives, for some \(\tilde\theta\) between \(\theta\) and \(\theta_0\),

\begin{equation*} \begin{aligned} \sqrt{n}\bigl(\phi-f(\theta_0)\bigr) &=f^{\prime}(\theta_0)\,\sqrt{n}(\theta-\theta_0)\\ &\quad+\tfrac{1}{2}f^{\prime\prime}(\tilde\theta)\,n^{-1/2} \bigl[\sqrt{n}(\theta-\theta_0)\bigr]^{2}. \end{aligned} \end{equation*}

By Section 4.2, \(\sqrt{n}(\theta-\theta_0)\mid y\to\mathrm{N}\bigl(0,[J(\theta_0)]^{-1}\bigr)\) in distribution, so the bracket is \(O_p(1)\) and the whole second term is \(O_p(n^{-1/2})\to 0\); Slutsky’s theorem then gives

\begin{equation*} \sqrt{n}\bigl(\phi-f(\theta_0)\bigr)\mid y \;\to\; \mathrm{N}\!\left(0,\;\frac{[f^{\prime}(\theta_0)]^{2}}{J(\theta_0)}\right), \end{equation*}

which is exactly the normal limit predicted by the information for \(\phi\), \(J_\phi(\phi_0)=J(\theta_0)/[f^{\prime}(\theta_0)]^{2}\).

Neither posterior is ever exactly normal at finite \(n\); the curvature of \(f\) distorts the \(\theta\)-normal only on scales of order \(1\), to which the posterior assigns probability tending to \(0\).

(a) They do not exist. Since the \(\mathrm{N}(4,1)\) density is strictly positive at \(0\), \(\mathrm{E}\,|1/x|=\infty\), so \(y/x\) has no finite mean and no finite variance; the simulation exhibits exactly this. Drawing \(10^{5}\), \(10^{6}\) and \(10^{7}\) pairs under three seeds gave sample standard deviations

\begin{equation*} 0.75,\;4.65,\;8.63;\qquad 5.20,\;3.43,\;2.73;\qquad 0.73,\;1.14,\;34.2, \end{equation*}

which refuse to settle, with \(\max|y/x|\) growing from \(90\) to \(10^{5}\) as \(n\) grows – the signature of an undefined second moment. What is stable is the centre of the distribution: across all runs the sample median was \(0.750\), the central \(95\%\) interval was \((-0.238,\;2.21)\), and the standard deviation of the central \(95\%\) of the draws was \(0.501\). (The sample mean hovered near \(0.81\), but this too is not a convergent quantity.) So the honest simulation answer is: median \(0.750\), spread about \(0.5\) in the bulk, moments infinite.

(b) With only means and standard deviations specified, linearize \(g(x,y)=y/x\) about \((\mu_x,\mu_y)=(4,3)\):

\begin{equation*} \nabla g=\left(-\frac{\mu_y}{\mu_x^{2}},\;\frac{1}{\mu_x}\right) =\left(-\tfrac{3}{16},\;\tfrac14\right), \end{equation*}

so, using independence,

\begin{equation*} \begin{aligned} \mathrm{E}\!\left(\frac{y}{x}\right)&\approx\frac{\mu_y}{\mu_x}=\frac34=0.75,\\ \mathrm{var}\!\left(\frac{y}{x}\right) &\approx\left(\frac{\mu_y}{\mu_x^{2}}\right)^{2}\sigma_x^{2} +\left(\frac{1}{\mu_x}\right)^{2}\sigma_y^{2}\\ &=\left(\tfrac{3}{16}\right)^{2}(1)+\left(\tfrac14\right)^{2}(4)\\ &=0.035156+0.25=0.285156, \end{aligned} \end{equation*}

giving sd \(=\sqrt{0.285156}=0.534\). Equivalently \(\mathrm{var}(y/x)\approx(\mu_y/\mu_x)^2\left[\sigma_y^2/\mu_y^2+\sigma_x^2/\mu_x^2\right]\): relative variances add. Both agree with the stable features of (a), median \(0.750\) and bulk spread \(0.501\).

(c) That \(\sigma_x/\mu_x\) be small – here \(0.25\) – so that \(x\) is effectively never near \(0\) and \(1/x\) is well approximated by its tangent line over the effective range of \(x\); that \(x\) and \(y\) be independent, or else a \(2\,\mathrm{cov}(x,y)\,\partial_x g\,\partial_y g\) term must be added; and that the third and higher moments of \(x\) be small enough that the neglected quadratic term \(\mu_y(x-\mu_x)^2/\mu_x^3\) is negligible. None of this makes the exact moments finite when \(x\) is normal, as (a) shows; the approximation describes the bulk of the distribution, not its tails.

(a) Complete the square, writing \(\mu=\mathrm{E}(\theta\mid y)\):

\begin{equation*} \mathrm{E}\bigl[(\theta-a)^2\mid y\bigr] =\mathrm{var}(\theta\mid y)+\bigl(\mu-a\bigr)^{2}, \end{equation*}

a strictly convex function of \(a\) whose unique minimum is at \(a=\mu\). (Assume \(\mathrm{E}(\theta^{2}\mid y)<\infty\); otherwise the expected loss is \(+\infty\) for every \(a\) and no comparison is possible.)

(b) Let \(m\) be any posterior median, so \(\Pr(\theta\le m)\ge\tfrac12\) and \(\Pr(\theta\ge m)\ge\tfrac12\) (here and in (c), \(\Pr\) is posterior probability given \(y\)), and assume \(\mathrm{E}(|\theta|\mid y)<\infty\). For \(a>m\) the pointwise inequality

\begin{equation*} |\theta-a|-|\theta-m| \;\ge\;(a-m)\bigl[\mathbf{1}\{\theta\le m\}-\mathbf{1}\{\theta> m\}\bigr] \end{equation*}

holds – with equality when \(\theta\le m\), and by the triangle inequality \(|\theta-a|\ge|\theta-m|-(a-m)\) otherwise. Taking posterior expectations,

\begin{equation*} \mathrm{E}\bigl[|\theta-a|\mid y\bigr]-\mathrm{E}\bigl[|\theta-m|\mid y\bigr] \ge(a-m)\bigl[\Pr(\theta\le m)-\Pr(\theta>m)\bigr]\ge 0 . \end{equation*}

The case \(a<m\) is the mirror image, using \(\Pr(\theta\ge m)\ge\tfrac12\). Hence \(m\) minimizes the posterior expected loss.

(c) Put \(q=k_0/(k_0+k_1)\in[0,1]\) and let \(m\) be any \(q\)-quantile, that is

\begin{equation*} \Pr(\theta<m)\;\le\;q\;\le\;\Pr(\theta\le m). \end{equation*}

Fix \(a\) and set \(D(\theta)=L(\theta,a)-L(\theta,m)\).

(i) \(a>m\). The three regions give

\begin{equation*} D(\theta)=\begin{cases} k_1(a-m), & \theta\le m,\\ k_1(a-m)-(k_0+k_1)(\theta-m), & m<\theta<a,\\ -k_0(a-m), & \theta\ge a, \end{cases} \end{equation*}

(the middle line by writing \(k_1(a-\theta)-k_0(\theta-m)\) as \(k_1(a-m)-(k_0+k_1)(\theta-m)\)), and on \(m<\theta<a\) that line is decreasing in \(\theta\), hence \(\ge-k_0(a-m)\). So \(D(\theta)\ge (a-m)\bigl[k_1\mathbf{1}\{\theta\le m\}-k_0\mathbf{1}\{\theta>m\}\bigr]\), and

\begin{equation*} \begin{aligned} \mathrm{E}\bigl[D(\theta)\mid y\bigr] &\ge(a-m)\bigl[k_1\Pr(\theta\le m)-k_0\Pr(\theta>m)\bigr]\\ &=(a-m)\bigl[(k_0+k_1)\Pr(\theta\le m)-k_0\bigr]\;\ge\;0, \end{aligned} \end{equation*}

the last step because \(\Pr(\theta\le m)\ge q\).

(ii) \(a<m\). Symmetrically \(D(\theta)=k_0(m-a)\) for \(\theta\ge m\), \(D(\theta)=-k_1(m-a)\) for \(\theta<a\), and \(D(\theta)=k_0(\theta-a)-k_1(m-\theta)\) is increasing on \(a\le\theta<m\), hence \(\ge-k_1(m-a)\) there. Therefore

\begin{equation*} \begin{aligned} \mathrm{E}\bigl[D(\theta)\mid y\bigr] &\ge(m-a)\bigl[k_0\Pr(\theta\ge m)-k_1\Pr(\theta<m)\bigr]\\ &=(m-a)\bigl[k_0-(k_0+k_1)\Pr(\theta<m)\bigr]\;\ge\;0, \end{aligned} \end{equation*}

now because \(\Pr(\theta<m)\le q\).

If \(\hat\theta(y)=\mathrm{E}(\theta\mid y)\) were unbiased then \(\mathrm{E}\bigl[(\theta-\hat\theta(y))^{2}\bigr]=0\), so \(\theta=\hat\theta(y)\) with probability \(1\) and the data determine the parameter exactly.

Because the prior is proper, \(p(\theta,y)=p(\theta)p(y\mid\theta)\) is a genuine joint probability distribution and every expectation below is taken over it; assume \(\mathrm{E}(\theta^{2})<\infty\) so that all the second moments exist. Suppose \(\mathrm{E}\bigl(\hat\theta(y)\mid\theta\bigr)=\theta\) for all \(\theta\). Evaluate the cross-moment \(\mathrm{E}\bigl[\theta\,\hat\theta(y)\bigr]\) by the tower property in the two possible orders.

Conditioning on \(\theta\) (so that \(\theta\) is fixed and unbiasedness applies):

\begin{equation*} \mathrm{E}\bigl[\theta\,\hat\theta(y)\bigr] =\mathrm{E}\Bigl[\theta\,\mathrm{E}\bigl(\hat\theta(y)\mid\theta\bigr)\Bigr] =\mathrm{E}\bigl[\theta\cdot\theta\bigr]=\mathrm{E}(\theta^{2}). \end{equation*}

Conditioning on \(y\) (so that \(\hat\theta(y)\) is fixed, being \(\sigma(y)\)-measurable, and the definition of the posterior mean applies):

\begin{equation*} \mathrm{E}\bigl[\theta\,\hat\theta(y)\bigr] =\mathrm{E}\Bigl[\hat\theta(y)\,\mathrm{E}\bigl(\theta\mid y\bigr)\Bigr] =\mathrm{E}\bigl[\hat\theta(y)^{2}\bigr]. \end{equation*}

Hence \(\mathrm{E}(\theta^{2})=\mathrm{E}(\hat\theta^{2})=\mathrm{E}(\theta\hat\theta)\), and therefore

\begin{equation*} \mathrm{E}\bigl[(\theta-\hat\theta(y))^{2}\bigr] =\mathrm{E}(\theta^{2})-2\,\mathrm{E}(\theta\hat\theta)+\mathrm{E}(\hat\theta^{2}) =0 . \end{equation*}

A nonnegative random variable with zero expectation is zero almost surely, so \(\theta=\mathrm{E}(\theta\mid y)\) for almost every \((\theta,y)\); equivalently \(\mathrm{E}\bigl[\mathrm{var}(\theta\mid y)\bigr]=0\), so the posterior variance vanishes for almost every \(y\). That is the degenerate case: the data reveal \(\theta\) with certainty (for instance \(p(y\mid\theta)\) a point mass at an invertible function of \(\theta\)), and only then can a posterior mean be unbiased.

The two conditionals are mirror images, each with slope \(\tfrac12\) in its own conditioning variable, and that symmetry is the whole content of regression to the mean. With \(\mu=160\) and \(\sigma^2\) the common variance, the joint distribution is normal with correlation \(\rho=\tfrac12\), and the bivariate normal conditioning formula of Appendix A gives

\begin{equation*} \begin{aligned} \theta\mid y &\sim \mathrm{N}\!\left(\mu + \tfrac12 (y-\mu),\; \tfrac34\sigma^2\right), \\ y\mid\theta &\sim \mathrm{N}\!\left(\mu + \tfrac12 (\theta-\mu),\; \tfrac34\sigma^2\right). \end{aligned} \end{equation*}

The first line is (4.5). Nothing shrinks in the population: \(\operatorname{Var}(\theta)=\operatorname{Var}(y)=\sigma^2\), and the joint density is symmetric about the \(45^\circ\) line.

Sampling bias of the posterior mean. Taking expectations in (4.5) over \(p(y\mid\theta)\),

\begin{equation*} E\big(E(\theta\mid y)\;\big|\;\theta\big) = \mu + \tfrac12\big(E(y\mid\theta)-\mu\big) = \mu + \tfrac14(\theta-\mu), \end{equation*}

so that

\begin{equation*} \operatorname{bias}(\theta) = \mu + \tfrac14(\theta-\mu) - \theta = -\tfrac34(\theta-\mu), \end{equation*}

shrinkage towards the grand mean by three quarters of the distance.

The unbiased alternative. Since \(E(y\mid\theta)=\mu+\tfrac12(\theta-\mu)\), inverting the map gives \(\hat\theta = \mu + 2(y-\mu)\), and indeed

\begin{equation*} E(\hat\theta\mid\theta) = \mu + 2\big(\tfrac12(\theta-\mu)\big) = \theta, \qquad \operatorname{Var}(\hat\theta\mid\theta) = 4\cdot\tfrac34\sigma^2 = 3\sigma^2 . \end{equation*}

Mean squared error, conditional on \(\theta\). For the posterior mean, \(\operatorname{Var}(E(\theta\mid y)\mid\theta)=\tfrac14\cdot\tfrac34\sigma^2\), so

\begin{equation*} \begin{aligned} \mathrm{MSE}_{(4.5)}(\theta) &= \tfrac{3}{16}\sigma^2 + \tfrac{9}{16}(\theta-\mu)^2, \\ \mathrm{MSE}_{\hat\theta}(\theta) &= 3\sigma^2 . \end{aligned} \end{equation*}

The unbiased estimate has the smaller mean squared error only when

\begin{equation*} \tfrac{9}{16}(\theta-\mu)^2 > 3\sigma^2 - \tfrac{3}{16}\sigma^2 = \tfrac{45}{16}\sigma^2, \quad\text{i.e.}\quad |\theta-\mu| > \sqrt5\,\sigma . \end{equation*}

So the unbiased estimate wins only for \(|\theta-\mu|>\sqrt5\,\sigma\), that is on about \(2.5\%\) of the population; averaged over \(p(\theta)\) the comparison is \(\tfrac34\sigma^2\) against \(3\sigma^2\), a factor of exactly \(4\). Concretely, with \(\sigma=7.5\) cm and a mother of height \(y=175\), \(\theta\mid y\sim\mathrm{N}(167.5,6.50^2)\) gives the \(95\%\) interval \([154.8,180.2]\), while \(\hat\theta=190\) cm lies outside it and above the \(99.99\)th percentile of \(p(\theta)\).

Sketch. The joint density is concentric ellipses centred at \((160,160)\) with major axis along the \(45^\circ\) line. Three lines pass through the centre: \(E(\theta\mid y)=160+0.5(y-160)\), of slope \(\tfrac12\), which bisects every vertical chord of every ellipse; the mirror line \(E(y\mid\theta)\), which on these axes reads \(\theta = 160+2(y-160)\), of slope \(2\) — this is \(\hat\theta\), and it bisects every horizontal chord; and the \(45^\circ\) line between them. The vertical slices at \(y=150,160,175\) are, after normalising, three normal curves of common standard deviation \(0.866\sigma\) centred at \(155,160,167.5\): each centre is pulled half way back to \(160\).

For large \(\sigma\) estimate (1) degenerates into a coin flip between the two endpoints, and a coin flip between \(0\) and \(1\) costs exactly \(\tfrac14\) more than the constant \(\tfrac12\) at every \(\theta\); estimate (2) degenerates into that constant. The two estimates are

\begin{equation*} \hat\theta_1(y) = \operatorname{median}(0,y,1), \qquad \hat\theta_2(y) = \frac{\int_0^1 u\,e^{-(y-u)^2/2\sigma^2}\,du} {\int_0^1 e^{-(y-u)^2/2\sigma^2}\,du}. \end{equation*}

Lower bound for estimate (1). Discarding the event \(\{0\le y\le 1\}\), on which the loss is nonnegative, and using \(\Pr(y<0)=\Phi(-\theta/\sigma)\ge\Phi(-1/\sigma)\) and \(\Pr(y>1)=\Phi((\theta-1)/\sigma)\ge\Phi(-1/\sigma)\) for \(\theta\in[0,1]\), write \(p_\sigma=\Phi(-1/\sigma)\); then

\begin{equation*} \begin{aligned} \mathrm{MSE}_1(\theta) &\ \ge\ \theta^2\Pr(y<0) + (1-\theta)^2\Pr(y>1) \\ &\ \ge\ p_\sigma\big[\theta^2+(1-\theta)^2\big] \ =\ p_\sigma\Big[2\big(\theta-\tfrac12\big)^2 + \tfrac12\Big]. \end{aligned} \end{equation*}

Upper bound for estimate (2). Cancel the factor \(e^{-y^2/2\sigma^2}\) from numerator and denominator: with \(w(u)=\exp\big(yu/\sigma^2 - u^2/2\sigma^2\big)\),

\begin{equation*} \hat\theta_2(y) = \frac{\int_0^1 u\,w(u)\,du}{\int_0^1 w(u)\,du}, \qquad e^{-\lambda}\le w(u)\le e^{\lambda}\ \ \text{on }[0,1], \end{equation*}

where \(\lambda = (|y|+\tfrac12)/\sigma^2\). Hence \(\tfrac12 e^{-2\lambda}\le\hat\theta_2\le\tfrac12 e^{2\lambda}\), so, together with the trivial bound \(|\hat\theta_2-\tfrac12|\le\tfrac12\),

\begin{equation*} \big|\hat\theta_2 - \tfrac12\big| \ \le\ \tfrac12\min\big(1,\ e^{2\lambda}-1\big). \end{equation*}

On \(\{|y-\theta|\le\sigma^{3/2}\}\) we have \(\lambda\le\lambda_\sigma:=\sigma^{-1/2}+\tfrac32\sigma^{-2}\), and the complement has probability \(2\Phi(-\sigma^{1/2})\) whatever \(\theta\) is, so

\begin{equation*} E\big|\hat\theta_2-\tfrac12\big| \ \le\ \tfrac12\big(e^{2\lambda_\sigma}-1\big) + \Phi(-\sigma^{1/2}) \ =:\ \tfrac23\,\varepsilon_\sigma , \end{equation*}

uniformly in \(\theta\in[0,1]\), and \(\varepsilon_\sigma\to0\) as \(\sigma\to\infty\). Expanding about \(\tfrac12\) and using \(|2(\tfrac12-\theta)|\le1\) and \((\hat\theta_2-\tfrac12)^2\le\tfrac12|\hat\theta_2-\tfrac12|\),

\begin{equation*} \mathrm{MSE}_2(\theta) = \big(\theta-\tfrac12\big)^2 + 2\big(\tfrac12-\theta\big)E\big[\hat\theta_2-\tfrac12\big] + E\big[(\hat\theta_2-\tfrac12)^2\big] \ \le\ \big(\theta-\tfrac12\big)^2 + \varepsilon_\sigma . \end{equation*}

Comparison. Subtracting, and using \(0\le(\theta-\tfrac12)^2\le\tfrac14\) with \(2p_\sigma-1\le0\),

\begin{equation*} \begin{aligned} \mathrm{MSE}_1(\theta)-\mathrm{MSE}_2(\theta) &\ \ge\ (2p_\sigma-1)\big(\theta-\tfrac12\big)^2 + \tfrac{p_\sigma}{2} - \varepsilon_\sigma \\ &\ \ge\ \tfrac{2p_\sigma-1}{4} + \tfrac{p_\sigma}{2} - \varepsilon_\sigma \ =\ p_\sigma - \tfrac14 - \varepsilon_\sigma . \end{aligned} \end{equation*}

Since \(p_\sigma\to\tfrac12\) and \(\varepsilon_\sigma\to0\), the right side tends to \(\tfrac14\) and in particular is positive for all \(\sigma\) large enough — with the bounds as written, already for \(\sigma\ge100\), where \(p_\sigma-\tfrac14 = 0.246\) and \(\varepsilon_\sigma\le0.167\). The bound is uniform in \(\theta\), which is what was asked.

The limits are exact: \(\hat\theta_1\to\) a fair coin flip on \(\{0,1\}\) and \(\hat\theta_2\to\tfrac12\), so

\begin{equation*} \mathrm{MSE}_1(\theta)\to\big(\theta-\tfrac12\big)^2+\tfrac14, \qquad \mathrm{MSE}_2(\theta)\to\big(\theta-\tfrac12\big)^2, \end{equation*}

a gap of exactly \(\tfrac14\) at every \(\theta\). The constant \(100\) is an artifact of the bounding: numerical quadrature of the two mean squared errors (the minimum over \(\theta\) is attained at an endpoint of \([0,1]\)) puts the crossover at \(\sigma=0.753\),

(a) Take the estimator to be the maximum likelihood estimate, \((\hat\alpha,\hat\beta) = \arg\max \sum_i [\,y_i(\alpha+\beta x_i) - n_i\log(1+e^{\alpha+\beta x_i})\,]\), computed by Newton–Raphson. Because the prior in Section 3.7 is uniform, this is exactly the posterior mode quoted on page 86. For Table 3.1,

\begin{equation*} (\hat\alpha,\hat\beta) = (0.847,\ 7.749), \qquad \widehat{\mathrm{LD}50} = -\hat\alpha/\hat\beta = -0.109, \end{equation*}

with fitted death probabilities \((0.003,\,0.186,\,0.613,\,0.998)\) at the four doses.

The estimator needs a caveat that will dominate everything below: the maximum is attained at a finite point only if the data are not (quasi-)separated, i.e. only if there is no index \(j\) with \(y_i=0\) for all \(i<j\) and \(y_i=n_i\) for all \(i>j\). Table 3.1 is not separated; many datasets from this design are.

(b) The observed information at \((\hat\alpha,\hat\beta)\), with \(w_i = n_i\hat\theta_i(1-\hat\theta_i)\), is

\begin{equation*} \begin{aligned} J &= \begin{pmatrix} \sum w_i & \sum w_i x_i \\ \sum w_i x_i & \sum w_i x_i^2\end{pmatrix} = \begin{pmatrix} 1.965 & -0.293 \\ -0.293 & 0.0859 \end{pmatrix}, \\ J^{-1} &= \begin{pmatrix} 1.039 & 3.546 \\ 3.546 & 23.74 \end{pmatrix}, \end{aligned} \end{equation*}

so the asymptotic standard errors are \(\mathrm{se}(\hat\alpha)=1.019\), \(\mathrm{se}(\hat\beta)=4.873\), correlation \(0.714\), and the asymptotic bias is \(O(n^{-1})\), nominally zero.

That is the textbook answer; it is badly wrong here, and the parametric bootstrap says why. Plugging \(\hat\theta=(0.003,0.186,0.613,0.998)\) into the binomial model and enumerating all \(6^4=1296\) possible datasets exactly (no Monte Carlo error is needed at this size),

\begin{equation*} \Pr\big(\text{the MLE is infinite}\big) = 0.404 . \end{equation*}

Under the fitted model, two replications in five of this experiment produce a separated dataset and no finite estimate at all, so the bias and variance of \((\hat\alpha,\hat\beta)\) are undefined. Conditioning on the \(59.6\%\) of datasets with a finite MLE,

\begin{equation*} \begin{aligned} E(\hat\alpha) &= 0.930, &\quad \operatorname{bias} &= +0.083, &\quad \mathrm{sd} &= 0.777, \\ E(\hat\beta) &= 7.217, &\quad \operatorname{bias} &= -0.531, &\quad \mathrm{sd} &= 2.240 . \end{aligned} \end{equation*}

The conditional standard deviations sit well below the asymptotic standard errors (\(0.78\) against \(1.02\), and \(2.24\) against \(4.87\)), because conditioning on a finite estimate removes exactly the replications that would have supplied the large values. For \(\mathrm{LD}50\) the delta method gives, with \(g = (-1/\hat\beta,\ \hat\alpha/\hat\beta^2) = (-0.129,\ 0.0141)\),

\begin{equation*} \mathrm{se}(\widehat{\mathrm{LD}50}) = \sqrt{g^{T}J^{-1}g} = 0.0955, \end{equation*}

in agreement with the \(0.096\) obtained by numerical differentiation of the exact marginal posterior of \(\mathrm{LD}50\). Its exact conditional sampling bias is \(E(\widehat{\mathrm{LD}50})-\widehat{\mathrm{LD}50} = -0.1214+0.1093 = -0.012\). (Its conditional variance does not exist for practical purposes — a few very improbable datasets give \(\hat\beta\) near \(0\) and make the second moment arbitrarily large — but its quantiles, used in (e), are perfectly well behaved.)

(c) Using \(\hat\theta - \widehat{\operatorname{bias}} \pm 1.96\,\mathrm{se}\) with the asymptotic standard errors and the bias estimates from (b):

\begin{equation*} \begin{aligned} \alpha:&\quad 0.764 \pm 1.96(1.019) = [-1.23,\ 2.76], \\ \beta:&\quad 8.280 \pm 1.96(4.873) = [-1.27,\ 17.83], \\ \mathrm{LD}50:&\quad -0.097 \pm 1.96(0.0955) = [-0.284,\ 0.090]. \end{aligned} \end{equation*}

Without the bias correction these are \([-1.15,2.84]\), \([-1.80,17.30]\) and \([-0.296,0.078]\); the correction is immaterial next to the standard errors.

(d) Yes, decisively, and for two related reasons. First, the interval for \(\beta\) contains \(0\), which is the same artifact visible in Figure 4.1: the normal approximation assigns probability \(\Phi(-7.749/4.873)=0.056\) to \(\beta\le0\), whereas the exact posterior of Figure 3.3 assigns \(4\times10^{-6}\) (the book reports \(\Pr(\beta>0)>0.999\) on page 87). Since \(\mathrm{LD}50 = -\alpha/\beta\) has a pole at \(\beta=0\), no interval built from a normal approximation that straddles \(\beta = 0\) can be trusted, and the delta-method variance — a first-order expansion of a function with a nearby singularity — is not a reliable scale. Second, and worse, the \(40\%\) separation probability found in (b) means the nominal \(95\%\) procedure fails outright on two replications in five. Enumerating the exact sampling distribution and counting, the actual coverage of the intervals in (c) is

\begin{equation*} \Pr(\text{covers }\alpha) = 0.587,\quad \Pr(\text{covers }\beta) = 0.573,\quad \Pr(\text{covers }\mathrm{LD}50) = 0.593, \end{equation*}

counting a separated replication (where no finite interval is produced) as a failure. Conditionally on a finite MLE the coverages are \(0.986\), \(0.962\) and \(0.996\) — the procedure over-covers when it works at all, and the \(95\%\) label describes neither situation. With \(n=20\) animals and four design points, the asymptotics of Section 4.2 simply have not engaged.

(e) Parametric bootstrap: resample \(y_i^{*}\sim\mathrm{Bin}(5,\hat\theta_i)\) and refit. Again this is done by exact enumeration of the 1296 outcomes, so the reported percentiles are the exact bootstrap percentiles. Discarding the separated replicates,

\begin{equation*} \alpha\in[-0.48,\ 1.95],\quad \beta\in[3.57,\ 11.16],\quad \mathrm{LD}50\in[-0.312,\ 0.086]. \end{equation*}

The nonparametric bootstrap, resampling the 20 animals with replacement (\(20{,}000\) replications, \(43.8\%\) separated and discarded, Monte Carlo standard error about \(0.003\) on the \(\mathrm{LD}50\) endpoints), gives the very similar \(\alpha\in[-0.55,2.31]\), \(\beta\in[4.00,12.04]\), \(\mathrm{LD}50\in[-0.287,0.104]\). All three intervals are conditional on the \(56\%\)–\(60\%\) event that the replicate admits a finite estimate. (For a quasi-separated replicate, one whose middle group sits at the separating dose, \(-\hat\alpha/\hat\beta\) still has a finite limit, namely that dose; for a completely separated one it does not.)

Jackknife, deleting one animal at a time from the 20: this breaks down for \(\alpha\) and \(\beta\), since deleting the single death at \(x=-0.30\) yields \(y=(0,0,3,5)\) with \(n=(5,4,5,5)\), which is separated at \(x=-0.05\); that pseudo-value is infinite, so the jackknife standard errors for \(\alpha\) and \(\beta\) do not exist. For \(\mathrm{LD}50\) the pseudo-values stay finite (the separated one contributes \(-0.05\)) and the jackknife gives \(\mathrm{se}=0.113\) with bias estimate \(+0.027\), hence

\begin{equation*} \mathrm{LD}50 \in -0.109 \pm 1.96(0.113) = [-0.331,\ 0.113]. \end{equation*}

(f) Collecting the four intervals for \(\mathrm{LD}50\):

The first four agree remarkably well: all are centred near \(-0.11\) and all have width close to \(0.4\). The odd one out is the interval derived from the normal approximation to \(p(\alpha,\beta|y)\), which is more than twice as wide, because the \(5.6\%\) of its draws with \(\beta\) near or below zero send \(-\alpha/\beta\) off towards \(\pm\infty\); the book records a range of \([-12.4,\ 5.4]\) for the 950 draws behind Figure 4.2a, and Figure 4.2b only hides the tails by truncation.

I prefer the exact posterior interval of Figure 3.4. It requires no asymptotics, it is exactly the quantity asked about — a statement of where \(\mathrm{LD}50\) lies given these 20 animals — and it is the only one of the five that is well defined for every dataset this design can produce. The delta-method and bootstrap intervals happen to land in the right place for this dataset, but (d) shows their calibration is an accident of these particular data: on two replications in five they do not exist, and their unconditional coverage is \(59\%\), not \(95\%\). The jackknife is the weakest of the four, being both the widest and the most fragile.

Yes as a posterior mode, under a spike-and-slab prior: a lump of prior mass at \(\theta=0\) mixed with a flat density on \(\theta>0\). Never as a posterior mean, for any prior.

Write \(\tau = \sigma/\sqrt{n}\), so that \(\bar y\) is sufficient with \(\bar y\mid\theta\sim\mathrm{N}(\theta,\tau^2)\), and take

\begin{equation*} p(\theta) = (1-\pi)\,\mathrm{N}(\theta\mid 0,\epsilon^2) + \pi\,\mathrm{U}(\theta\mid 0,M), \end{equation*}

with \(\epsilon\) tiny and \(M\) large. The unnormalised posterior is \(q(\theta)= p(\theta)\,\exp\!\big(-(\bar y-\theta)^2/2\tau^2\big)\), and it has at most two local maxima, one contributed by each prior component.

(i) \(\bar y < 0\). On \((0,M)\) the likelihood is strictly decreasing, so the slab branch has no interior maximum and the mode is the spike, \(\hat\theta=0\).

(ii) \(\bar y > 0\). The slab branch peaks at \(\theta=\bar y\) (for \(\bar y<M\)), where \(q(\bar y) = \pi/M\); as \(\epsilon\to0\) the spike branch peaks at \(\theta\approx0\), where \(q(0) \approx \big[(1-\pi)/(\sqrt{2\pi}\epsilon)\big]e^{-\bar y^2/2\tau^2}\). Comparing,

\begin{equation*} \frac{q(\bar y)}{q(0)} = \frac{\pi}{M}\cdot\frac{\sqrt{2\pi}\,\epsilon}{1-\pi}\;e^{\bar y^{2}/2\tau^{2}} \ >\ 1 \iff \frac{\bar y^{2}}{\tau^{2}} \ >\ 2\log K, \quad K := \frac{(1-\pi)M}{\sqrt{2\pi}\,\pi\,\epsilon}. \end{equation*}

Choosing the prior constants so that \(2\log K = 1.96^2 = 3.8416\), i.e. \(K = e^{1.9208} = 6.83\), makes the posterior mode equal to \(\bar y\) exactly when \(\bar y \ge 1.96\tau = 1.96\sigma/\sqrt n\), and \(0\) otherwise. This is the stated estimator.

The threshold \(\tau\sqrt{2\log K}\) carries the factor \(\sigma/\sqrt n\) by itself, so a single fixed prior reproduces the procedure at every sample size; and the one-sidedness of the printed rule is exactly the one-sidedness of the slab, since a uniform component on \((-M,M)\) gives instead \(\hat\theta = \bar y\,\mathbf{1}\{|\bar y|\ge1.96\tau\}\).

It is not a posterior mean under any prior \(G\), however. Bayes’ rule (1.1) with the normal kernel gives

\begin{equation*} E(\theta\mid\bar y) = \frac{\int \theta\,e^{\theta\bar y/\tau^{2}-\theta^{2}/2\tau^{2}}\,dG(\theta)} {\int e^{\theta\bar y/\tau^{2}-\theta^{2}/2\tau^{2}}\,dG(\theta)}, \end{equation*}

a ratio of two-sided Laplace transforms of finite measures, hence real-analytic — in particular continuous — in \(\bar y\) on the interior of its domain of convergence. The estimator jumps from \(0\) to \(1.96\tau\) at \(\bar y = 1.96\tau\), so no prior makes it a posterior mean.

Yes, as a posterior mode under the same spike-and-slab prior on \(\theta\) as in 4.11, now multiplied by the standard noninformative prior \(p(\sigma^2)\propto\sigma^{-2}\); the implied cutoff is not exactly \(1.96\) for finite \(n\), but converges to it.

Keep the prior

\begin{equation*} p(\theta,\sigma^{2}) \;\propto\; \Big[(1-\pi)\,\mathrm{N}(\theta\mid0,\epsilon^{2}) + \pi\,\mathrm{U}(\theta\mid0,M)\Big]\,\sigma^{-2}, \end{equation*}

\(\epsilon\) tiny, \(M\) large. Marginalising \(\sigma^2\) is exactly the gamma-integral calculation of Section 3.2 that produces the marginal posterior of \(\mu\) under \(p(\mu,\log\sigma)\propto1\), and it is unaffected by the factor \(p(\theta)\), which does not involve \(\sigma^2\):

\begin{equation*} p(\theta\mid y)\;\propto\;p(\theta) \left[1+\frac{n(\bar y-\theta)^{2}}{(n-1)s^{2}}\right]^{-n/2}, \end{equation*}

i.e. the prior times a \(t_{n-1}(\bar y, s^2/n)\) kernel. As in 4.11 there are two candidate modes, and for \(\bar y<0\) the slab branch is monotone decreasing on \((0,M)\) so the mode is \(0\). For \(\bar y>0\), writing \(t = \sqrt n\,\bar y/s\) and \(K = (1-\pi)M/(\sqrt{2\pi}\,\pi\,\epsilon)\) as before,

\begin{equation*} \frac{p(\bar y\mid y)}{p(0\mid y)} = \frac{1}{K}\left[1+\frac{t^{2}}{n-1}\right]^{n/2} > 1 \iff t^{2} \;>\; c_n^{2} := (n-1)\big(K^{2/n}-1\big). \end{equation*}

So the estimator is again a hard threshold on the usual \(t\) statistic, exactly the form of the stated rule. Calibrating with the same \(K = e^{1.9208}=6.83\) that worked in 4.11,

\begin{equation*} c_n^2 = (n-1)\big(e^{3.8416/n}-1\big) \;\longrightarrow\; 3.8416 = 1.96^{2} \quad (n\to\infty), \end{equation*}

since \(n(e^{a/n}-1)\to a\). The finite-\(n\) cutoffs, alongside the classical \(t_{n-1,0.975}\) they are imitating:

The implied cutoff exceeds \(1.96\) and decreases to it, tracking \(t_{n-1,0.975}\) from below. So the single \(n\)-free prior with \(K=6.83\) reproduces the stated rule only in the limit, but to within the same order of accuracy with which \(1.96\) itself approximates \(t_{n-1,0.975}\); an exact match at a given \(n\) costs an \(n\)-dependent prior, \(K = (1+3.8416/(n-1))^{n/2}\).

It is still not a posterior mean. For fixed \(s\) the \(t\) kernel is bounded by \(1\) and continuous in \(\bar y\), so whenever \(\int|\theta|\,dG(\theta)<\infty\) (which is what makes the posterior mean exist at all) dominated convergence makes \(E(\theta\mid\bar y,s)\) continuous in \(\bar y\), while \(\hat\theta\) jumps from \(0\) to \(1.96\,s/\sqrt n\).

The trilemma is false in each of its three arms, because it assumes that the prior distribution is the only thing Bayesian inference contributes and that the likelihood is the only thing it shares with everyone else. The Bayesian contribution is the posterior distribution, and that is worth having no matter how weak the prior is.

(a) Weak or uniform prior, so why bother? Because the answer is a distribution rather than an estimate and a standard error, and in small or awkward problems those are not the same thing. The bioassay analysis of Section 3.7 uses \(p(\alpha,\beta)\propto1\) — the prior contributes literally nothing — and yet the Bayesian and non-Bayesian analyses of the same likelihood differ sharply. As computed in 4.10, under the fitted model the maximum likelihood estimate is infinite with probability \(0.40\), the nominal \(95\%\) asymptotic confidence intervals have actual coverage \(59\%\), and \(\mathrm{LD}50 = -\alpha/\beta\) has a pole at \(\beta=0\) that no delta-method standard error can represent. The posterior histogram of Figure 3.4 has none of these problems and required no asymptotics. Similarly in Section 2.4 the posterior for the placenta previa sex ratio is reported as an interval on the probability scale directly, with no transformation or continuity correction. The prior did no work in either case; Bayes did.

Two further points about arm (a). A uniform prior is a substantive choice, not an abstention: it is not invariant to reparameterisation, which is exactly the motivation for Jeffreys’ prior in Section 2.8, and in Chapter 5 the flat hyperprior \(p(\log\tau)\propto1\) yields an improper posterior. And “weak” is relative to the likelihood: the same prior is noninformative for \(n=1000\) and influential for \(n=5\).

(b) Strong prior, so why collect data? Because a strong prior is not an infinitely strong one, and precisions add. For the normal model, (2.12) gives

\begin{equation*} \frac{1}{\tau_n^{2}} = \frac{1}{\tau_0^{2}} + \frac{n}{\sigma^{2}}, \end{equation*}

so the data contribution \(n/\sigma^2\) grows without bound while the prior contribution \(1/\tau_0^2\) stays fixed; this is the statement on page 88 that the likelihood dominates the prior. The placenta previa example is the illustration: with \(980\) births, every informative prior in Table 2.1 — prior sample sizes \(\alpha+\beta\) from \(2\) up to \(200\) — yields the same posterior median to within \(0.001\) and essentially the same \(95\%\) interval. If a prior really is so strong that \(n\) new observations cannot move it, that is a discovery worth making, and it is made by writing the prior down and doing the arithmetic. Furthermore, data do something a prior cannot: they check the model. The posterior predictive checks of Chapter 6 can reject the sampling model outright, whatever the prior on its parameters.

(c) Something in between, so Bayesianism ducks the issue. This inverts the situation. The “issue” is how much weight partial prior information should get, and the Bayesian answer is a number, stated in advance, with its influence on the conclusion measurable by the sensitivity analysis described in Section 6.1 and carried out in Section 6.5. The alternative is not an analysis free of prior information; it is one in which the same information enters through the choice of model, the choice of test statistic, which covariates are retained, which outliers are excluded, and when sampling stopped — none of which appear in the reported standard error. Section 4.5 makes the sharper version of this point: insisting on unbiasedness does not avoid prior information, it merely discards it, as in the mother-and-daughter example worked in 4.8, where the unbiased estimate assigns a daughter a height above the 99.99th percentile of its own population distribution.

The arm (c) objection also ignores the case that occupies most of this book. In a hierarchical model (Chapter 5) the “in between” prior is not a subjective compromise at all: its parameters are estimated from the same data, by pooling information across units. The answer to “where did the prior come from?” is then “from the other 70 experiments in the table.”

What survives of Ehrenberg’s criticism is a reporting standard, not an objection: state the prior, state the likelihood, and show how much the conclusion would change under reasonable alternatives to each.

Exactly backwards: the premise is right and the conclusion reverses it. Bayesian methods are the only ones that require the preconception to be written down as a probability distribution, published with the analysis, and combined with the data by a rule fixed in advance. That is the definition of disciplining a preconception, not of encouraging one.

The prior is auditable; the alternatives are not. An analyst who dismisses a surprising result in a Bayesian report must exhibit the prior under which it is unsurprising, and must have committed to that prior before seeing the data for the report to be honest. A non-Bayesian analysis of the same data offers a much larger and wholly unreported set of levers for the same purpose: which model to fit, which test statistic to use, which covariates to keep, which observations are “outliers,” when to stop sampling, how many comparisons were actually made. None of these appear in the standard error, and all of them respond to preconceptions. The prior is the one place where the subjectivity is labelled.

The influence of a preconception is bounded, and the bound is computable. For the normal model with prior \(\mathrm{N}(\mu_0,\tau_0^2)\), (2.12) gives the posterior mean as the precision-weighted average

\begin{equation*} E(\theta\mid y) = \frac{\tfrac{1}{\tau_0^{2}}\mu_0 + \tfrac{n}{\sigma^{2}}\bar y} {\tfrac{1}{\tau_0^{2}} + \tfrac{n}{\sigma^{2}}}, \end{equation*}

whose weight on the prior is \(O(1/n)\). To sustain a preconception against accumulating data an analyst would need \(\tau_0^{-2}\) to grow with \(n\), which a stated prior cannot do; and since the asymptotic results of Section 4.2 make the posterior converge to \(\theta_0\) for any prior that is positive in a neighbourhood of \(\theta_0\), the only way to be immovable is to assign a surprising conclusion prior probability exactly zero — which is visible on the page.

Bayesian analysis comes with its own anti-self-deception machinery. The posterior predictive checks of Chapter 6 confront the fitted model with the data and can discredit it regardless of what the prior says, and the sensitivity analysis of Sections 6.1 and 6.5 reports how much of the conclusion is prior and how much is likelihood. The classical apparatus has nothing corresponding to the second of these.

Finally, the “undisciplined” behaviour is sometimes the correct behaviour. Discounting a surprising estimate is right when the surprise is more probably noise than signal. That is the content of the regression example of 4.8, where the estimate that refuses to shrink towards the population mean predicts a \(190\) cm daughter, and of the hierarchical shrinkage of Chapter 5, where an extreme rate from a small sample is pulled towards the population of similar experiments and thereby made more accurate. A method that takes every surprising point estimate at face value is not more objective; it is just less calibrated.

The legitimate residue of the complaint is that a strong prior can be used to bury an inconvenient likelihood, and that reporting only a posterior mean conceals this. The remedy is the reporting standard — prior, likelihood, and sensitivity, shown separately — not the abandonment of the only framework in which the preconception has to be declared.

(a) Average the conditional coverage over the marginal of \(y\). Let \(S(y)\) be any set built from \(y\) with \(\Pr(\omega \in S(y) \mid y) = 1/2\) for every \(y\), the defining property of a 50 percent posterior interval. Since the probability is taken under the joint density \(p(\omega, y) = p(\omega) p(y \mid \omega)\), the tower property over the sigma-algebra generated by \(y\) gives

\begin{equation*} \begin{aligned} \Pr\big(\omega \in S(y)\big) &= \mathrm{E}\Big[ \mathrm{E}\big[ 1_{\{\omega \in S(y)\}} \,\big|\, y \big] \Big] \\ &= \mathrm{E}\big[ \Pr(\omega \in S(y) \mid y) \big] = \mathrm{E}\big[ \tfrac12 \big] = \tfrac12 . \end{aligned} \end{equation*}

(b) \(0.535\). By (2.10), with prior \(\mathrm{N}(\mu_0, \tau_0^2) = \mathrm{N}(0, 4)\) and a single observation of variance \(\sigma^2 = 1\), the posterior precision is \(\tau_0^{-2} + \sigma^{-2} = \tfrac14 + 1 = \tfrac54\) and the posterior mean is \((\tfrac14 \cdot 0 + 1 \cdot y)/\tfrac54\), so

\begin{equation*} \omega \mid y \sim \mathrm{N}\!\left( \tfrac{4}{5} y, \ \tfrac{4}{5} \right), \end{equation*}

and the central 50 percent posterior interval is \(\tfrac45 y \pm c\) with

\begin{equation*} c = z_{0.75}\sqrt{4/5} = 0.67449 \times 0.89443 = 0.60328 . \end{equation*}

Now fix the true value \(\omega_0\) and let \(y \sim \mathrm{N}(\omega_0, 1)\). The miss \(\omega_0 - \tfrac45 y\) is normal with mean \(\omega_0 - \tfrac45\omega_0 = \omega_0/5\) and standard deviation \(4/5\), so the coverage is

\begin{equation*} \begin{aligned} C(\omega_0) &= \Pr\!\left( \left| \omega_0 - \tfrac45 y \right| \le c \right) \\ &= \Phi\!\left( \frac{c - \omega_0/5}{4/5} \right)

• \Phi\!\left( \frac{-c - \omega_0/5}{4/5} \right). \end{aligned} \end{equation*}

At \(\omega_0 = 1\) the two arguments are \((0.60328 - 0.2)/0.8 = 0.50410\) and \(-1.00410\), giving

\begin{equation*} C(1) = 0.69291 - 0.15766 = 0.53524 . \end{equation*}

(c) The displayed \(C(\omega_0)\) is the curve to plot: even in \(\omega_0\), peaking at the prior mean with \(C(0) = 2\Phi(0.75410) - 1 = 0.549\) and decaying to \(0\) in both tails. Values:

Coverage exceeds the nominal \(1/2\) for \(|\omega_0| < 1.908\), where shrinkage toward \(0\) pulls the interval toward the truth, crosses \(1/2\) at \(|\omega_0| = 1.908\), and falls below it thereafter.

Exchangeable in all three; independent only in (a); safe to treat as independent in (a) and (c). Code black as \(1\), white as \(0\).

(a) The joint distribution is \(p(y_1,y_2)=\tfrac14\) on \(\{0,1\}^2\), symmetric in its arguments and a product of two \(\mathrm{Bernoulli}(\tfrac12)\) marginals. So (i) yes, (ii) yes, (iii) yes.

(b) The second draw is the ball the first draw left behind, so

\begin{equation*} p(y_1,y_2)=\tfrac12 \quad\text{on } (0,1) \text{ and } (1,0), \qquad p(y_1,y_2)=0 \text{ otherwise.} \end{equation*}

This is symmetric under swapping \(y_1,y_2\), so (i) yes. But \(y_2=1-y_1\) with probability one, whereas \(p(y_1)p(y_2)=\tfrac14\) everywhere; equivalently

\begin{equation*} \operatorname{cov}(y_1,y_2)=E(y_1y_2)-E(y_1)E(y_2)=0-\tfrac14=-\tfrac14, \end{equation*}

the extreme negative value for \(\mathrm{Bernoulli}(\tfrac12)\) variables. So (ii) no, and (iii) emphatically no: the observations are as dependent as two binary variables can be.

(c) With \(N=10^6\) of each colour, every ordered pair of draws is equally likely, so \(p(y_1,y_2)\) depends only on the multiset \(\{y_1,y_2\}\) and (i) holds. For (ii),

\begin{equation*} p(y_2=1\mid y_1=1)=\frac{N-1}{2N-1}=\frac{999{,}999}{1{,}999{,}999}\neq\tfrac12 , \end{equation*}

so no. But the dependence is numerically nil:

\begin{equation*} \operatorname{corr}(y_1,y_2)=-\frac{1}{2N-1}=-5.0\times10^{-7}, \end{equation*}

so for any practical inference (iii) yes – acting as if the draws were independent \(\mathrm{Bernoulli}(\tfrac12)\) perturbs no probability by more than about \(10^{-6}\).

Exchangeable in all three; independent in none; and in none of the three may we act as if they were independent – though in (a) and (c) they are independent conditional on the unknown composition. Write \(\theta\) for the unknown proportion of black balls, with prior \(p(\theta)\), and code black as \(1\).

(a) Given \(\theta\) the draws are i.i.d. \(\mathrm{Bernoulli}(\theta)\), so

\begin{equation*} p(y_1,y_2)=\int \theta^{y_1+y_2}(1-\theta)^{2-y_1-y_2}\,p(\theta)\,d\theta , \end{equation*}

which depends on \((y_1,y_2)\) only through \(y_1+y_2\): exchangeable, (i) yes. This is exactly the mixture-of-i.i.d. form (5.2), and by Exercise 5.5,

\begin{equation*} \operatorname{cov}(y_1,y_2)=\operatorname{var}\!\big(E(y_1\mid\theta)\big)=\operatorname{var}(\theta)>0 \end{equation*}

unless \(p(\theta)\) is a point mass. So (ii) no. For (iii): \(y_1\) is informative about \(\theta\) and hence about \(y_2\), by an amount \(\operatorname{var}(\theta)\) that is not small when we are genuinely ignorant of the composition (for \(\theta\sim\mathrm{U}(0,1)\), \(\operatorname{cov}=1/12\) and \(\operatorname{corr}=1/3\)). We may act as if they are conditionally independent given \(\theta\), which is what the hierarchical model does, but not as if they are marginally independent.

(b) Given the number \(n\theta\) of black balls the two draws are an exchangeable sample without replacement, so \(p(y_1,y_2\mid\theta)\) is symmetric, and averaging over \(p(\theta)\) preserves symmetry: (i) yes. Now two dependencies act at once,

\begin{equation*} \operatorname{cov}(y_1,y_2)=\underbrace{E\big[\operatorname{cov}(y_1,y_2\mid\theta)\big]}_{=\,-E[\theta(1-\theta)]/(n-1)} +\underbrace{\operatorname{var}\big(E(y_1\mid\theta)\big)}_{=\,\operatorname{var}(\theta)} , \end{equation*}

the negative finite-population term and the positive learning-about-\(\theta\) term. Both are generally nonzero, so (ii) no and (iii) no.

(c) Exchangeable for the same reason, (i) yes. “Many balls of each color’’ kills the first term above – it is \(O(1/n)\) – but leaves \(\operatorname{var}(\theta)\) untouched, so (ii) no, and (iii) no: unlike Exercise 5.1(c), where \(\theta=\tfrac12\) was known, here the residual dependence is the whole of our uncertainty about the composition. What we may do is treat the draws as i.i.d. \(\mathrm{Bernoulli}(\theta)\) given \(\theta\), ignoring the without-replacement correction.

Under the hierarchical model no pairwise comparison is sharper than about \(0.73\) and no school is best with probability above \(0.25\); under separate estimation school A is best with probability \(0.55\) and beats school E with probability \(0.95\).

(a) Simulation follows Section 5.4. The hyperprior \(p(\mu,\tau)\propto 1\) is improper, but Section 5.4 records that uniform \(\tau\) (unlike uniform \(\log\tau\)) leaves \(p(\tau\mid y)\) integrable, so the posterior below is proper. Evaluate the marginal posterior (5.21) for \(\tau\) on a grid, \(\tau\in(0,40]\) in steps of \(0.01\), where with \(V_j=\sigma_j^2+\tau^2\),

\begin{equation*} \begin{aligned} \hat\mu(\tau)&=\frac{\sum_j y_j/V_j}{\sum_j 1/V_j}, \qquad V_\mu(\tau)=\Big(\sum_j 1/V_j\Big)^{-1},\\ p(\tau\mid y)&\propto V_\mu(\tau)^{1/2}\prod_{j=1}^{8}V_j^{-1/2} \exp\!\Big(-\frac{(y_j-\hat\mu)^2}{2V_j}\Big), \end{aligned} \end{equation*}

then draw \(\tau\) from the grid, \(\mu\mid\tau,y\sim\mathrm{N}(\hat\mu,V_\mu)\) by (5.20), and finally, independently for each \(j\), by (5.17),

\begin{equation*} \begin{aligned} \theta_j\mid\mu,\tau,y&\sim\mathrm{N}(\hat\theta_j,V_j^{\ast}),\\ \hat\theta_j&=\frac{y_j/\sigma_j^2+\mu/\tau^2}{1/\sigma_j^2+1/\tau^2},\qquad V_j^{\ast}=\Big(\frac1{\sigma_j^2}+\frac1{\tau^2}\Big)^{-1}. \end{aligned} \end{equation*}

With \(S=200{,}000\) draws the posterior quantiles reproduce Table 5.3 (posterior means \(11.4\), \(7.9\), \(6.2\), \(7.7\), \(5.1\), \(6.2\), \(10.7\), \(8.5\); posterior median of \(\tau\) equal to \(5.2\)). Counting draws gives (i), with Monte Carlo standard error at most \(0.001\):

and for (ii), \(\Pr(\theta_j>\theta_k\mid y)\) with \(j\) indexing rows:

Every entry lies in \([0.27,0.73]\): the data do not permit a confident ranking of any pair.

(b) With \(\tau=\infty\) the prior on \(\theta\) is flat and the eight analyses separate: \(\theta_j\mid y\sim\mathrm{N}(y_j,\sigma_j^2)\) independently. Then \(\theta_j-\theta_k\sim\mathrm{N}(y_j-y_k,\sigma_j^2+\sigma_k^2)\), so (ii) is available in closed form,

\begin{equation*} \Pr(\theta_j>\theta_k\mid y)=\Phi\!\left(\frac{y_j-y_k}{\sqrt{\sigma_j^2+\sigma_k^2}}\right), \end{equation*}

giving (rows \(j\)):

The “best of eight’’ probabilities (i) are an eight-dimensional orthant probability; simulating \(200{,}000\) independent draws of \(\theta\) gives (Monte Carlo standard error at most \(0.001\))

(c) The \(\tau=\infty\) probabilities are far more extreme. Shrinkage towards a common \(\mu\) (posterior median \(\tau=5.2\), small relative to the \(\sigma_j\)) pulls the differences \(\theta_j-\theta_k\) towards zero while their posterior spread shrinks much less, so under (a) no pairwise probability leaves \([0.27,0.73]\) and every school’s chance of being best lies between \(0.05\) and \(0.25\), near the \(1/8\) of complete ignorance; under (b) school A is best with probability \(0.55\) and beats school E with probability \(0.95\). A pair can even reverse: \(\Pr(\theta_C>\theta_E\mid y)=0.55\) in (a) but \(0.46\) in (b), because the \(\sigma_j\) differ and shrinkage is stronger for the noisier school C. The hierarchical model, by letting the data report that \(\tau\) is small, supplies the multiplicity correction that separate estimation would need to add by hand.

(d) With \(\tau=0\) all eight parameters coincide, \(\theta_1=\dots=\theta_8=\mu\), so \(\Pr(\theta_j>\theta_k\mid y)=0\) for every \(j\neq k\) and the probability that any given school is strictly the best is \(0\). Breaking the ties at random – the usual convention – these become \(\Pr(\theta_j>\theta_k)=\tfrac12\) for all \(j\neq k\) and \(\Pr(j \text{ best})=\tfrac18\) for all \(j\): the data are given no weight at all in ranking the schools.

Yes to (a); (b) follows from Exercise 5.5 because the covariances are negative; (c) because the family is not the set of marginals of any one infinite exchangeable sequence, and its limit is i.i.d. anyway.

(a) Yes. Let \(S\) be the set of indices drawn from \(\mathrm{N}(1,1)\), uniform over the \(\binom{2J}{J}\) subsets of size \(J\). Then

\begin{equation*} p(\theta)=\binom{2J}{J}^{-1}\sum_{|S|=J}\ \prod_{j\in S}\mathrm{N}(\theta_j\mid 1,1) \prod_{j\notin S}\mathrm{N}(\theta_j\mid -1,1), \end{equation*}

and a permutation \(\pi\) of \((1,\ldots,2J)\) merely relabels the summand \(S\) as \(\pi^{-1}(S)\), permuting the terms of a sum over all subsets of size \(J\). So \(p\) is invariant to permutations: exchangeable.

(b) Let \(z_j=1\) if \(j\in S\) and \(z_j=0\) otherwise, so \(E(\theta_j\mid z)=2z_j-1\) and \(E(\theta_j)=0\) by symmetry. The \(z_j\) are a simple random sample of \(J\) ones among \(2J\) positions, so for \(i\neq j\),

\begin{equation*} \Pr(z_i=z_j)=2\cdot\frac{J(J-1)}{2J(2J-1)}=\frac{J-1}{2J-1}, \end{equation*}

and, using \(\operatorname{cov}(\theta_i,\theta_j)=E[\operatorname{cov}(\theta_i,\theta_j\mid z)]+\operatorname{cov}(E(\theta_i\mid z),E(\theta_j\mid z))\) with the first term zero (\(\theta\) are independent given \(z\)),

\begin{equation*} \begin{aligned} \operatorname{cov}(\theta_i,\theta_j) &=E\big[(2z_i-1)(2z_j-1)\big]\\ &=\Pr(z_i=z_j)-\Pr(z_i\neq z_j)\\ &=\frac{J-1}{2J-1}-\frac{J}{2J-1}=-\frac{1}{2J-1}<0 . \end{aligned} \end{equation*}

By Exercise 5.5 every mixture of i.i.d. components has nonnegative covariances, so this distribution is not of that form.

(c) Because there is no single infinite exchangeable sequence here to which de Finetti’s theorem could be applied. The construction gives, for each \(J\), a distribution on \(2J\) variables, and these are not consistent: the \(2J\)-dimensional law is not the marginal of the \(2(J+1)\)-dimensional one (the first has covariance \(-1/(2J-1)\), the second \(-1/(2J+1)\)). De Finetti’s theorem requires a single distribution on \((\theta_1,\theta_2,\ldots)\) whose finite-dimensional marginals are exchangeable.

Nor does the limit produce a counterexample: for fixed \(k\), the count of the first \(k\) indices in \(S\) is \(\mathrm{Hypergeometric}(2J,J,k)\to\mathrm{Bin}(k,\tfrac12)\), so the \(k\)-dimensional marginal converges to

\begin{equation*} \prod_{j=1}^{k}\Big[\tfrac12\mathrm{N}(\theta_j\mid 1,1)+\tfrac12\mathrm{N}(\theta_j\mid -1,1)\Big], \end{equation*}

i.i.d. as de Finetti requires, with \(\operatorname{cov}(\theta_i,\theta_j)=-1/(2J-1)\to 0\).

\(\operatorname{cov}(\theta_i,\theta_j)=\operatorname{var}\big(E(\theta_i\mid\phi)\big)\ge 0\).

Assume \(E(\theta_j^2)<\infty\), so that all the quantities below exist. Apply (1.9) to \(\theta_i+\theta_j\) and to \(\theta_i-\theta_j\) and subtract; the resulting bilinear form of that identity reads

\begin{equation*} \operatorname{cov}(\theta_i,\theta_j)=E\big[\operatorname{cov}(\theta_i,\theta_j\mid\phi)\big] +\operatorname{cov}\big(E(\theta_i\mid\phi),\,E(\theta_j\mid\phi)\big). \end{equation*}

For \(i\neq j\) the first term vanishes: given \(\phi\) the components are independent by hypothesis, so \(\operatorname{cov}(\theta_i,\theta_j\mid\phi)=0\). In the second term the components are identically distributed given \(\phi\), so \(E(\theta_i\mid\phi)=E(\theta_j\mid\phi)=:m(\phi)\), and the covariance of a random variable with itself is its variance:

\begin{equation*} \operatorname{cov}(\theta_i,\theta_j)=0+\operatorname{cov}\big(m(\phi),m(\phi)\big)=\operatorname{var}\big(m(\phi)\big)\ge 0 . \end{equation*}

For \(i=j\) the statement is \(\operatorname{var}(\theta_i)\ge 0\).

The posterior for \(y_8\) is a three-component mixture, putting weight \(0.22\) on Utah, \(0.44\) on Nevada and \(0.34\) on an ordinary state; and the eight actual data points are not consistent with the prior, which insists on a low outlier that is not there.

(a) Introduce a latent labelling: let \((L,H)\) be an ordered pair of distinct indices drawn uniformly from the \(8\cdot 7=56\) possibilities, \(L\) marking Utah and \(H\) Nevada. Given the labels,

\begin{equation*} \begin{aligned} y_L&\sim\mathrm{N}(4,1^2),\qquad y_H\sim\mathrm{N}(12,3^2),\\ y_j&\stackrel{\text{iid}}{\sim}\mathrm{N}(6.5,1.5^2)\quad (j\neq L,H), \end{aligned} \end{equation*}

all eight independent. (The means are a pre-data guess: a typical Mountain state near \(6.5\) per 1000, Utah well below, Nevada far above, with a generous spread on Nevada.) Marginally

\begin{equation*} \begin{aligned} p(y_1,\ldots,y_8)=\frac{1}{56}\sum_{L\neq H}\ &\mathrm{N}(y_L\mid 4,1)\,\mathrm{N}(y_H\mid 12,9)\\ &\times\prod_{j\neq L,H}\mathrm{N}(y_j\mid 6.5,2.25), \end{aligned} \end{equation*}

and permuting the indices permutes the \(56\) summands among themselves, so the prior is exchangeable.

(b) Condition on the seven observed values \(y_{\text{obs}}\) and partition by the role of index \(8\):

• (i) \(8=L\): then one of the seven observed is Nevada and six are ordinary;
• (ii) \(8=H\): then one of the seven observed is Utah and six are ordinary;
• (iii) \(8\notin\{L,H\}\): both Utah and Nevada are among the seven observed.

Each case contributes its likelihood summed over the admissible label placements; with \(f_L,f_H,f_0\) the three densities and \(P_0=\prod_{i=1}^{7}f_0(y_i)\),

\begin{equation*} \begin{aligned} w_{\text{(i)}}&\propto P_0\sum_{i=1}^{7}\frac{f_H(y_i)}{f_0(y_i)},\\ w_{\text{(ii)}}&\propto P_0\sum_{i=1}^{7}\frac{f_L(y_i)}{f_0(y_i)},\\ w_{\text{(iii)}}&\propto P_0\sum_{i\neq i^{\prime}}\frac{f_L(y_i)}{f_0(y_i)}\frac{f_H(y_{i^{\prime}})}{f_0(y_{i^{\prime}})} . \end{aligned} \end{equation*}

Evaluating at \(y_{\text{obs}}=(5.8,6.6,7.8,5.6,7.0,7.1,5.4)\) and normalizing gives

\begin{equation*} w_{\text{(i)}}=0.222,\qquad w_{\text{(ii)}}=0.440,\qquad w_{\text{(iii)}}=0.338 . \end{equation*}

Since \(y_8\) is independent of the rest given the labels, the posterior is the corresponding mixture of the three prior components:

\begin{equation*} \begin{aligned} p(y_8\mid y_{\text{obs}})=\ &0.222\,\mathrm{N}(4,1^2)+0.440\,\mathrm{N}(12,3^2)\\ &+0.338\,\mathrm{N}(6.5,1.5^2), \end{aligned} \end{equation*}

trimodal with modes at \(4\), \(6.5\) and \(12\), posterior mean \(8.4\) and standard deviation \(4.0\), and \(\Pr(y_8>10\mid y_{\text{obs}})=0.33\). The largest weight falls on “the missing state is Nevada’’ because the seven observed values are all tightly clustered and one of them (\(5.4\), with conditional probability \(0.45\) of being Utah, or \(5.6\) with \(0.30\)) can play Utah comfortably, whereas nothing in the range \(5.4\) to \(7.8\) plays Nevada well.

(c) No. With \(y_8=13.9\) the complete data are \(5.4\), \(5.6\), \(5.8\), \(6.6\), \(7.0\), \(7.1\), \(7.8\), \(13.9\): one clear high outlier and no low one. But the prior of part (a) forces exactly one draw from \(\mathrm{N}(4,1)\), and the smallest observation is \(5.4\). Taking \(T(y)=\min_j y_j\) as a check statistic, the prior predictive probability is

\begin{equation*} \Pr\big(T(y^{\text{rep}})\ge 5.4\big)=0.016 , \end{equation*}

whereas the high end is entirely comfortable, \(\Pr(\max_j y_j^{\text{rep}}\ge 13.9)=0.26\). So the prior did not allow a high outlier without a matching low one: Utah’s actual 1981 rate sits inside the cluster of ordinary states. A prior that would accommodate these data must let the number of outliers at each end be random – an exchangeable i.i.d. mixture, or simply a heavy-tailed i.i.d. distribution centered at \(6.5\).

Mean \(\alpha/\beta\) and variance \((\alpha/\beta)(1+1/\beta)\) in (a); mean \(0\) and variance \((n-1)/(n-3)\) in (b), requiring \(n>2\) and \(n>3\) respectively. Both parts apply (2.7) and (2.8) with the conditioning variable named below.

(a) Condition on \(\theta\), so \(E(y\mid\theta)=\operatorname{var}(y\mid\theta)=\theta\), and use \(E(\theta)=\alpha/\beta\), \(\operatorname{var}(\theta)=\alpha/\beta^2\) for the \(\mathrm{Gamma}(\alpha,\beta)\) prior (Appendix A):

\begin{equation*} \begin{aligned} E(y)&=E\big(E(y\mid\theta)\big)=E(\theta)=\frac{\alpha}{\beta},\\ \operatorname{var}(y)&=E\big(\operatorname{var}(y\mid\theta)\big)+\operatorname{var}\big(E(y\mid\theta)\big)\\ &=E(\theta)+\operatorname{var}(\theta) =\frac{\alpha}{\beta}+\frac{\alpha}{\beta^2} =\frac{\alpha}{\beta}\Big(1+\frac1\beta\Big). \end{aligned} \end{equation*}

In terms of \(p=\beta/(1+\beta)\), so that \(1-p=1/(1+\beta)\) and \(\beta=p/(1-p)\), these read

\begin{equation*} E(y)=\frac{\alpha(1-p)}{p},\qquad \operatorname{var}(y)=\frac{\alpha(1-p)}{p^2}=\frac{E(y)}{p}, \end{equation*}

the standard negative binomial moments.

(b) Write \(t=\sqrt n(\mu-\bar y)/s\) and condition on \(\sigma^2\), treating \(\bar y\) and \(s\) as fixed data. The posterior factors as \(\mu\mid\sigma^2,y\sim\mathrm{N}(\bar y,\sigma^2/n)\) and \(\sigma^2\mid y\sim\text{Inv-}\chi^2(n-1,s^2)\) (equations (3.3) and (3.5)), so

\begin{equation*} t\mid\sigma^2,y\sim\mathrm{N}\!\Big(0,\ \frac{\sigma^2}{s^2}\Big). \end{equation*}

Hence \(E(t\mid\sigma^2)=0\) and \(\operatorname{var}(t\mid\sigma^2)=\sigma^2/s^2\), and by (2.7) and (2.8),

\begin{equation*} \begin{aligned} E(t)&=E\big(E(t\mid\sigma^2)\big)=0,\\ \operatorname{var}(t)&=E\Big(\frac{\sigma^2}{s^2}\Big)+\operatorname{var}(0) =\frac{1}{s^2}\cdot\frac{(n-1)s^2}{(n-1)-2} =\frac{n-1}{n-3}, \end{aligned} \end{equation*}

using \(E(\sigma^2\mid y)=\nu s^2/(\nu-2)\) for \(\sigma^2\sim\text{Inv-}\chi^2(\nu,s^2)\) with \(\nu=n-1\).

The conditions are the hypotheses of those two identities, that the moments being averaged exist: \(E|t|<\infty\) needs \(E(\sigma\mid y)<\infty\), finite for \(\nu>1\), that is \(n>2\); and \(E(\sigma^2\mid y)<\infty\) needs \(\nu>2\), that is \(n>3\).

The posterior is the mixture of the component posteriors with weights reweighted by the component prior predictive densities:

\begin{equation*} p(\omega \mid y) = \sum_{m=1}^{M} \lambda^{\prime}_m \, p_m(\omega \mid y), \qquad \lambda^{\prime}_m = \frac{\lambda_m \, q_m(y)}{\sum_{l} \lambda_l \, q_l(y)}, \end{equation*}

where \(q_m(y) = \int p(y \mid \omega) p_m(\omega) \, d\omega\) is the marginal likelihood under component \(m\). Indeed,

\begin{equation*} \begin{aligned} p(\omega \mid y) &\propto p(y \mid \omega) \sum_{m} \lambda_m p_m(\omega) \\ &= \sum_{m} \lambda_m \, p(y \mid \omega) p_m(\omega) \\ &= \sum_{m} \lambda_m \, q_m(y) \, p_m(\omega \mid y), \end{aligned} \end{equation*}

and each \(p_m(\omega \mid y)\) lies in the same parametric family as \(p_m\) by conjugacy. Hence the posterior is again a finite mixture of members of that family, i.e. the mixture class is closed under sampling. (Each \(q_m(y)\) is finite and positive, so the \(\lambda^{\prime}_m\) are well defined weights summing to 1.)

For the example, take the two-component normal prior

\begin{equation*} p(\omega) = 0.9 \, \mathrm{N}(\omega \mid 1, 0.5^2) + 0.1 \, \mathrm{N}(\omega \mid -1, 0.5^2), \end{equation*}

which is bimodal with modes at \(\pm 1\) (heights \(0.718\) and \(0.080\)). The data enter only through \(\bar{y} \mid \omega \sim \mathrm{N}(\omega, \sigma^2/n)\) with \(\sigma^2/n = 1/10\). By (2.12) each component posterior has precision \(0.5^{-2} + 10 = 14\), so

\begin{equation*} V = \tfrac{1}{14} = 0.07143, \qquad \sqrt{V} = 0.26726, \end{equation*}

and component means

\begin{equation*} \begin{aligned} \mu^{\prime}_1 &= \frac{4(1) + 10(-0.25)}{14} = \tfrac{3}{28} = 0.10714, \\ \mu^{\prime}_2 &= \frac{4(-1) + 10(-0.25)}{14} = -\tfrac{13}{28} = -0.46429 . \end{aligned} \end{equation*}

The marginal likelihoods are \(q_m = \mathrm{N}(\bar{y} \mid \mu_m, \, 0.25 + 0.1)\), so

\begin{equation*} \frac{q_1}{q_2} = \exp\!\left( -\frac{(-1.25)^2 - (0.75)^2}{2(0.35)} \right) = e^{-10/7} = 0.23965 , \end{equation*}

giving

\begin{equation*} \lambda^{\prime}_1 = \frac{0.9 (0.23965)}{0.9(0.23965) + 0.1} = 0.6832, \qquad \lambda^{\prime}_2 = 0.3168 . \end{equation*}

Thus

\begin{equation*} p(\omega \mid y) = 0.683\,\mathrm{N}(\omega \mid 0.107,\, 0.267^2)

• 0.317\,\mathrm{N}(\omega \mid -0.464,\, 0.267^2), \end{equation*}

with mean \(-0.0739\) and standard deviation \(0.377\): the prior odds \(0.9{:}0.1\) become posterior odds \(0.683{:}0.317\), since \(\bar{y} = -0.25\) sits much closer to \(-1\) than to \(1\). Sketch: the prior has two well-separated humps at \(-1\) and \(1\), the right one about nine times taller; the posterior collapses onto \((-1, 1)\), its two component means only \(0.571\) apart against a common standard deviation \(0.267\), so it is unimodal with a single peak at \(\omega = 0.074\) and a pronounced left shoulder near \(-0.46\).

Throughout write \(s = \alpha + \beta\) and \(p = \alpha/(\alpha+\beta)\), so that \(u = \log(\alpha/\beta) = \mathrm{logit}(p)\) and \(v = \log s\). The Jacobian of \((u,v) \mapsto (\alpha, \beta)\) is \(\alpha\beta\), since

\begin{equation*} \left| \frac{\partial(u,v)}{\partial(\alpha,\beta)} \right| = \left| \frac{1}{\alpha}\cdot\frac{1}{s} + \frac{1}{\beta}\cdot\frac{1}{s} \right| = \frac{1}{\alpha\beta}. \end{equation*}

Each factor of (5.8) is the beta-binomial weight

\begin{equation*} L_j(p,s) \;=\; \frac{B(\alpha + y_j,\ \beta + n_j - y_j)}{B(\alpha, \beta)} \;=\; \mathrm{E}_{\omega \sim \mathrm{Beta}(\alpha,\beta)}\!\left[ \omega^{y_j} (1-\omega)^{n_j - y_j} \right] \le 1 . \end{equation*}

(a) The integrand does not decay as \(v \to +\infty\). Population standard deviation \(\approx s^{-1/2} \to 0\) means \(s \to \infty\) at fixed \(p\), and there \(\mathrm{Beta}(\alpha,\beta)\) concentrates at \(p\): writing the factor out,

\begin{equation*} L_j = \frac{\Gamma(\alpha + y_j)}{\Gamma(\alpha)} \cdot \frac{\Gamma(\beta + n_j - y_j)}{\Gamma(\beta)} \cdot \frac{\Gamma(s)}{\Gamma(s + n_j)}, \end{equation*}

with \(\Gamma(\alpha+y_j)/\Gamma(\alpha) = \alpha(\alpha+1)\cdots(\alpha+y_j-1) \sim \alpha^{y_j}\), likewise \(\sim \beta^{n_j - y_j}\), and \(\Gamma(s)/\Gamma(s+n_j) \sim s^{-n_j}\), so

\begin{equation*} L_j(p,s) \longrightarrow p^{y_j}(1-p)^{n_j - y_j} \qquad (s \to \infty, \ p \text{ fixed}). \end{equation*}

With \(p(u,v) \propto 1\) the unnormalized posterior therefore tends to the strictly positive constant \(c(p) = \prod_j p^{y_j}(1-p)^{n_j-y_j}\), and the convergence is uniform on any compact \(u\)-interval \(K\). Choosing \(V\) so large that the integrand exceeds \(\tfrac12 \min_{u \in K} c(p) > 0\) for \(v > V\),

\begin{equation*} \int_{K} \int_{V}^{\infty} p(u,v \mid y)\, dv \, du \; \ge \; \tfrac12 |K| \min_{u \in K} c(p) \int_{V}^{\infty} dv = \infty . \end{equation*}

(b) Put \(t = s^{-1/2}\) and let \((p, t)\) be independent uniform, i.e. \(p(p,t) \propto 1\). Changing variables one coordinate at a time,

\begin{equation*} \frac{dp}{du} = p(1-p) = \frac{\alpha\beta}{s^2}, \qquad \left| \frac{dt}{dv} \right| = \tfrac12 e^{-v/2} \propto s^{-1/2}, \end{equation*}

so

\begin{equation*} p(u,v) \;\propto\; \frac{\alpha\beta}{s^{2}} \cdot s^{-1/2} \;=\; \alpha\beta \,(\alpha+\beta)^{-5/2}, \end{equation*}

which is (5.10). Dividing by the Jacobian \(\alpha\beta\) recovers \(p(\alpha,\beta) \propto (\alpha+\beta)^{-5/2}\), i.e. (5.9).

(c) Work in the coordinates \((p,t) \in (0,1) \times (0,\infty)\), where by (b) the prior is Lebesgue measure and the posterior mass is \(\int_0^1 \!\! \int_0^\infty \prod_j L_j \, dt \, dp\). Split at \(t = 1\).

(i) On \(t \in (0,1)\): \(\prod_j L_j \le 1\) and the region has measure 1, so this part contributes at most 1.

(ii) On \(t > 1\), i.e. \(s < 1\): fix an index \(j^*\) with \(0 < y_{j^*} < n_{j^*}\) and bound all other factors by 1. For \(s \le 1\) we have \(\alpha, \beta \le 1\), hence

\begin{equation*} \begin{aligned} \frac{\Gamma(\alpha + y)}{\Gamma(\alpha)} &= \alpha \prod_{i=1}^{y-1}(\alpha + i) \le \alpha \, y! , \\ \frac{\Gamma(\beta + n - y)}{\Gamma(\beta)} &\le \beta \, (n-y)! , \\ \frac{\Gamma(s)}{\Gamma(s+n)} &= \frac{1}{s(s+1)\cdots(s+n-1)} \le \frac{1}{s \,(n-1)!}, \end{aligned} \end{equation*}

writing \(y = y_{j^*}\), \(n = n_{j^*}\) (the first two products are nonempty precisely because \(0 < y < n\)). Multiplying, and using \(\alpha\beta = s^2 p(1-p) \le s^2\),

\begin{equation*} L_{j^*}(p,s) \;\le\; \frac{y!\,(n-y)!}{(n-1)!}\, \frac{\alpha\beta}{s} \;\le\; C\, s \;=\; C\, t^{-2}, \end{equation*}

uniformly in \(p\). Hence this part contributes at most \(\int_0^1 \int_1^\infty C t^{-2} \, dt \, dp = C < \infty\).

So the posterior is proper. (If instead every \(y_j \in \{0, n_j\}\) then the same expansion gives \(L_j \to (1-p)\) or \(p\) as \(s \to 0\), the integrand does not decay in \(t\), and the integral diverges — so the stated condition is sharp.)

Since \(p(\mu \mid \tau, y)\) is a proper normal density (5.20) and each \(p(\omega_j \mid \mu, \tau, y)\) is a proper normal density (5.17), propriety of the joint posterior is exactly propriety of \(\int_0^\infty p(\tau \mid y)\, d\tau\), so (5.21) decides both parts.

(a) Improper, by a nonintegrable singularity at \(\tau = 0\). Write \(g(\tau)\) for the factor of (5.21) following \(p(\tau)\). Every ingredient of \(g\) is continuous on \([0,\infty)\) and positive at \(\tau = 0\):

\begin{equation*} \begin{aligned} V_\mu^{1/2} &\to \Big( \textstyle\sum_j \sigma_j^{-2} \Big)^{-1/2}, \qquad \prod_j (\sigma_j^2 + \tau^2)^{-1/2} \to \prod_j \sigma_j^{-1}, \\ \exp(\cdot) &\to \exp\!\Big( -\textstyle\sum_j (\bar{y}_{\cdot j} - \hat\mu_0)^2 / (2\sigma_j^2) \Big) > 0 , \end{aligned} \end{equation*}

with \(\hat\mu_0\) the precision-weighted mean at \(\tau = 0\). So \(g(0) = c > 0\) and \(g(\tau) \ge c/2\) on some \((0, \varepsilon)\), whence

\begin{equation*} \int_0^\infty \tau^{-1} g(\tau)\, d\tau \;\ge\; \frac{c}{2} \int_0^{\varepsilon} \frac{d\tau}{\tau} \;=\; \infty . \end{equation*}

(b) With \(p(\tau) \propto 1\) the integrand is \(g(\tau)\), which is continuous on \([0,\infty)\), so only the tail matters. Let \(\sigma_{\max}^2 = \max_j \sigma_j^2\). From \(V_\mu^{-1} = \sum_j (\sigma_j^2+\tau^2)^{-1} \ge J(\sigma_{\max}^2 + \tau^2)^{-1}\) and \(\exp(\cdot) \le 1\) and \((\sigma_j^2+\tau^2)^{-1/2} \le \tau^{-1}\),

\begin{equation*} g(\tau) \;\le\; \frac{(\sigma_{\max}^2 + \tau^2)^{1/2}}{\sqrt{J}} \cdot \tau^{-J} \;\le\; \frac{\sigma_{\max} + \tau}{\sqrt{J}} \, \tau^{-J}, \end{equation*}

and \(\int_1^\infty (\sigma_{\max}\tau^{-J} + \tau^{1-J})\, d\tau < \infty\) exactly when \(J - 1 > 1\), i.e. \(J > 2\). Combined with continuity on \([0,1]\) this gives a finite integral, so the posterior is proper. The bound is tight: the same limits give \(V_\mu^{1/2} \sim \tau/\sqrt{J}\), \(\prod_j(\sigma_j^2+\tau^2)^{-1/2} \sim \tau^{-J}\) and \(\exp(\cdot) \to 1\), so \(g(\tau) \sim \tau^{1-J}/\sqrt{J}\) and the integral diverges for \(J \le 2\).

(c) Do not use the flat prior: by (b) it fails, and \(g(\tau) \sim \tau^{-1}/\sqrt{2}\) means the improper posterior puts all its mass at \(\tau = \infty\), i.e. no pooling at all — so that analysis is just the two schools fitted separately. Two group means give one contrast, hence almost no information with which to separate \(\tau\) from the known \(\sigma_j^2\), and whatever is assumed about \(\tau\) is what gets reported. Two honest options: (i) a proper weakly informative prior, the half-Cauchy \(\tau \sim \mathrm{Cauchy}^{+}(0, A)\) of Section 5.7 with \(A\) at the high end of plausible school-to-school variation (order 25 points for the coaching data), whose \(\tau^{-2}\) tail restores integrability while staying flat for small \(\tau\); (ii) a sensitivity analysis reporting \(\mathrm{E}(\omega_j \mid \tau, y)\) and \(\mathrm{var}(\omega_j \mid \tau, y)\) from Exercise 5.12 as functions of \(\tau\), bracketed by complete pooling at \(\tau = 0\) and no pooling as \(\tau \to \infty\). An inverse-gamma\((\epsilon,\epsilon)\) prior will not do: by Section 5.7 its behavior as \(\tau \to 0\) is arbitrarily sensitive to \(\epsilon\) precisely when \(J\) is small.

(a) Take the hyperprior \(p(\mu, \tau) \propto 1\), which is the uniform-on-\(\tau\) choice of Section 5.4 (uniform on \(\log \tau\) would be improper by Exercise 5.10(a)). The population density for \(\omega_j\) carries the Jacobian \(d\,\mathrm{logit}(\omega_j)/d\omega_j = [\omega_j(1-\omega_j)]^{-1}\), so

\begin{equation*} \begin{aligned} p(\omega, \mu, \tau \mid y) \;\propto\; \tau^{-J} \prod_{j=1}^{J} \omega_j^{\,y_j - 1} (1-\omega_j)^{\,n_j - y_j - 1} \\ \times \exp\!\left( - \frac{\big( \mathrm{logit}(\omega_j) - \mu \big)^2}{2\tau^2} \right), \end{aligned} \end{equation*}

on \(\omega \in (0,1)^J\), \(\mu \in \mathbb{R}\), \(\tau > 0\).

(b) The integral factors over \(j\), and each factor is a logistic-normal integral. Substituting \(\theta_j = \mathrm{logit}(\omega_j)\), so that \(\omega_j^{y_j-1}(1-\omega_j)^{n_j-y_j-1}\, d\omega_j = e^{y_j \theta_j}(1 + e^{\theta_j})^{-n_j} d\theta_j\),

\begin{equation*} p(\mu, \tau \mid y) \;\propto\; \tau^{-J} \prod_{j=1}^{J} I_j(\mu,\tau), \end{equation*}

\begin{equation*} I_j(\mu,\tau) = \int_{-\infty}^{\infty} \frac{e^{y_j \theta}}{(1 + e^{\theta})^{n_j}} \exp\!\left( -\frac{(\theta - \mu)^2}{2\tau^2} \right) d\theta . \end{equation*}

What made (5.7)–(5.8) work in Section 5.3 was that the beta kernel \(\omega^{\alpha-1}(1-\omega)^{\beta-1}\) has exactly the algebraic form of the binomial likelihood, so the integral was a beta function. Here the kernels are of incompatible type: \(I_j\) convolves a Gaussian with the binomial-logit likelihood, and already its smallest instance \(n_j = 1\), \(y_j = 0\) is the logistic-normal integral \(\int (1+e^{\theta})^{-1} \mathrm{N}(\theta \mid \mu, \tau^2)\, d\theta\), for which no expression in elementary or standard special functions is known — which is why the logistic link is replaced by a probit, or approximated by \(\mathrm{logit}^{-1}(\theta) \approx \Phi(\theta/1.702)\), whenever a closed form is wanted. Expanding \((1+e^\theta)^{-n_j}\) in a series and integrating term by term yields only an infinite series in \(\Phi\). (Nonexistence in the Liouville sense is not proved here; the claim asserted is the book’s, that no closed form is available.)

(c) Identity (5.5) reads \(p(\phi \mid y) = p(\omega, \phi \mid y) / p(\omega \mid \phi, y)\), and it is useful only when the denominator is known in normalized form. From (a), the conditional posterior factors as

\begin{equation*} p(\omega_j \mid \mu, \tau, y) = \frac{\omega_j^{\,y_j-1}(1-\omega_j)^{\,n_j-y_j-1} \exp\!\big( -(\mathrm{logit}\,\omega_j - \mu)^2 / (2\tau^2) \big)} {I_j(\mu, \tau)}, \end{equation*}

whose normalizing constant is precisely \(I_j(\mu,\tau)\) — the very integral of part (b). So (5.5) merely relocates the unknown constant from numerator to denominator; it is circular here, whereas in Section 5.3 the denominator was the known beta density (5.7).

Substitute \(\hat\mu\) for \(\mu\) in the mean, and add the shrinkage-weighted variance of \(\hat\mu\):

\begin{equation*} \begin{aligned} \mathrm{E}(\omega_j \mid \tau, y) &= \frac{\frac{1}{\sigma_j^2}\bar{y}_{\cdot j} + \frac{1}{\tau^2}\hat\mu}{\frac{1}{\sigma_j^2} + \frac{1}{\tau^2}}, \\ \mathrm{var}(\omega_j \mid \tau, y) &= V_j + B_j^2 \, V_\mu , \end{aligned} \end{equation*}

where \(B_j = \dfrac{1/\tau^2}{1/\sigma_j^2 + 1/\tau^2} = \dfrac{\sigma_j^2}{\sigma_j^2 + \tau^2}\) is the shrinkage factor.

For the mean, condition on \(\mu\) and use (2.7) with everything also conditioned on \((\tau, y)\):

\begin{equation*} \mathrm{E}(\omega_j \mid \tau, y) = \mathrm{E}\big[ \mathrm{E}(\omega_j \mid \mu, \tau, y) \mid \tau, y \big] = \mathrm{E}(\hat\omega_j \mid \tau, y), \end{equation*}

and \(\hat\omega_j = (1 - B_j)\bar{y}_{\cdot j} + B_j \mu\) is affine in \(\mu\), so the expectation is obtained by replacing \(\mu\) with \(\mathrm{E}(\mu \mid \tau, y) = \hat\mu\).

For the variance, (2.8) in the same conditional form gives

\begin{equation*} \begin{aligned} \mathrm{var}(\omega_j \mid \tau, y) &= \mathrm{E}\big[ \mathrm{var}(\omega_j \mid \mu, \tau, y) \mid \tau, y \big] \\ &\quad + \mathrm{var}\big[ \mathrm{E}(\omega_j \mid \mu, \tau, y) \mid \tau, y \big] \\ &= \mathrm{E}(V_j \mid \tau, y) + \mathrm{var}(\hat\omega_j \mid \tau, y) \\ &= V_j + B_j^2 \, \mathrm{var}(\mu \mid \tau, y) = V_j + B_j^2 V_\mu , \end{aligned} \end{equation*}

since \(V_j\) is a function of \((\sigma_j, \tau)\) only, hence constant given \((\tau, y)\), and \(\hat\omega_j\) is affine in \(\mu\) with slope \(B_j\). Written out in terms of \(\sigma_j\) and \(\tau\) alone,

\begin{equation*} \begin{aligned} \mathrm{E}(\omega_j \mid \tau, y) &= \frac{\tau^2}{\sigma_j^2 + \tau^2}\,\bar{y}_{\cdot j}

• \frac{\sigma_j^2}{\sigma_j^2 + \tau^2}\,\hat\mu , \\ \mathrm{var}(\omega_j \mid \tau, y) &= \frac{\sigma_j^2 \tau^2}{\sigma_j^2 + \tau^2}
• \left( \frac{\sigma_j^2}{\sigma_j^2 + \tau^2} \right)^{\!2} \left( \sum_{k=1}^{J} \frac{1}{\sigma_k^2 + \tau^2} \right)^{\!-1} . \end{aligned} \end{equation*}

(a) The rat-tumor model of Section 5.3 verbatim, with \((y_j, n_j)\) from the table above:

\begin{equation*} \begin{aligned} y_j \mid \omega_j &\sim \mathrm{Bin}(n_j, \omega_j), \quad j = 1,\ldots,10, \text{ independent},\\ \omega_j \mid \alpha, \beta &\sim \mathrm{Beta}(\alpha, \beta), \\ p(\alpha, \beta) &\propto (\alpha + \beta)^{-5/2} \quad \text{(5.9)}, \end{aligned} \end{equation*}

so that, by (5.6),

\begin{equation*} \begin{aligned} p(\omega, \alpha, \beta \mid y) \;\propto\; (\alpha+\beta)^{-5/2} \prod_{j=1}^{10} \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)} \,\omega_j^{\alpha - 1}(1-\omega_j)^{\beta-1} \\ \times \ \omega_j^{\,y_j}(1-\omega_j)^{\,n_j - y_j}. \end{aligned} \end{equation*}

(b) Integrating out \(\omega\) gives (5.8),

\begin{equation*} p(\alpha, \beta \mid y) \;\propto\; (\alpha+\beta)^{-5/2} \prod_{j=1}^{10} \frac{B(\alpha + y_j,\ \beta + n_j - y_j)}{B(\alpha, \beta)}, \end{equation*}

and on the coordinates \((u, v) = (\log(\alpha/\beta), \log(\alpha+\beta))\) the density picks up the Jacobian \(\alpha\beta\), giving (5.10) in the prior factor. Evaluating the log density on a \(400 \times 400\) grid over \(u \in [-3.0, -0.2]\), \(v \in [0.5, 7.0]\) (grid-edge mass below \(10^{-4}\) of the total), the contours are the familiar banana, elongated in \(v\), with posterior mode at \((u, v) = (-1.37, 2.89)\), i.e.

\begin{equation*} (\hat\alpha, \hat\beta) = (3.66,\ 14.42). \end{equation*}

Drawing 10{,}000 pairs \((\alpha, \beta)\) from the grid (with jitter) and then \(\omega_j \mid \alpha, \beta, y \sim \mathrm{Beta}(\alpha + y_j,\, \beta + n_j - y_j)\) by (5.7) gives posterior medians and 95% central intervals

\begin{equation*} \alpha: \ 3.19 \ [1.04,\ 8.77], \qquad \beta: \ 12.5 \ [3.46,\ 36.1], \end{equation*}

with \(\alpha + \beta\) having median \(15.7\) and interval \([4.63, 44.6]\). Repeating with independent seeds moves these hyperparameter quantiles by at most \(0.3\) and every quantity on the probability scale below by at most \(0.001\).

(c) The posterior means shrink the raw proportions toward the population mean \(\approx 0.21\), and the shrinkage is strongest where \(n_j\) is smallest:

The shifts are modest — \(n_j\) runs from 58 to 308, so each \(\omega_j\) is already well determined by its own block — but systematic: block 2 (raw \(.091\), \(n = 99\)) moves up to \(.107\), block 10 (raw \(.462\), \(n = 119\)) moves down to \(.429\), while block 9 (\(n = 308\)) barely moves at all. The intervals are also slightly wider than separate-block binomial intervals, because \(\alpha\) and \(\beta\) are themselves uncertain.

(d) The average underlying proportion is the population mean \(\alpha/(\alpha+\beta)\), whose posterior median is \(0.205\) with 95% central interval

\begin{equation*} \Pr\big( 0.145 \le \tfrac{\alpha}{\alpha+\beta} \le 0.293 \ \big|\ y \big) = 0.95 . \end{equation*}

(e) Draw \(\tilde\omega \mid \alpha, \beta \sim \mathrm{Beta}(\alpha, \beta)\) for a genuinely new block and then \(\tilde y \mid \tilde\omega \sim \mathrm{Bin}(100, \tilde\omega)\). The posterior predictive interval is

\begin{equation*} \Pr(3 \le \tilde y \le 50 \mid y) = 0.95, \qquad \mathrm{E}(\tilde y \mid y) = 20.9, \end{equation*}

with median 19 (the upper endpoint carries Monte Carlo error of about \(\pm 1\) at 10{,}000 draws; the lower endpoint is stable at 3). It is far wider than the interval for any single observed \(\omega_j\) because it must absorb the between-block variation as well: the fitted population distribution has posterior median standard deviation \(0.098\) against a mean of \(0.205\), so blocks genuinely differ a great deal. How much to trust it: the arithmetic is right conditional on the model, but the model’s exchangeability assumption is the weak point. The ten blocks were not a simple random sample of residential bike-route blocks in any well-defined population, they were observed for a single hour on particular days, and traffic composition depends on time of day, weather and the specific street — none of which is in the model. Furthermore \(\tilde y\) is conditioned on exactly 100 vehicles passing, which is itself informative about the kind of block (compare Exercise 5.14, where the totals are modeled rather than conditioned on). I would treat \([3, 50]\) as a lower bound on the real uncertainty.

(f) Yes. Posterior predictive replication of all ten blocks from the fitted population distribution reproduces the observed spread well: with \(T\) ranging over the extremes and the dispersion of the raw proportions,

No \(p\)-value is near 0 or 1, so the two-parameter beta accommodates even block 10’s outlying \(0.462\) without strain — though only by being genuinely dispersed, which is what makes (e) so wide. With \(J = 10\) there is in any case little power to distinguish a beta from other unimodal population distributions on \((0,1)\).

(a) With \(\mathrm{Gamma}(\alpha,\beta)\) in the rate parametrization (density \(\propto \omega^{\alpha-1}e^{-\beta\omega}\)), and the noninformative hyperprior \(p(\alpha,\beta) \propto (\alpha\beta)^{-1}\), i.e. uniform on \((\log\alpha, \log\beta)\):

\begin{equation*} \begin{aligned} p(\omega, \alpha, \beta \mid n) \;\propto\; \frac{1}{\alpha\beta} \prod_{j=1}^{10} \frac{\beta^{\alpha}}{\Gamma(\alpha)} \,\omega_j^{\alpha-1} e^{-\beta\omega_j} \ \frac{\omega_j^{\,n_j} e^{-\omega_j}}{n_j!} . \end{aligned} \end{equation*}

(b) The \(\omega_j\) integrate out to independent negative binomials,

\begin{equation*} p(n_j \mid \alpha, \beta) = \frac{\Gamma(\alpha + n_j)}{\Gamma(\alpha)\, n_j!} \left( \frac{\beta}{\beta+1} \right)^{\!\alpha} \left( \frac{1}{\beta+1} \right)^{\!n_j}, \end{equation*}

so that

\begin{equation*} p(\alpha, \beta \mid n) \;\propto\; \frac{1}{\alpha\beta} \prod_{j=1}^{10} p(n_j \mid \alpha, \beta), \end{equation*}

which on the natural coordinates \((\log(\alpha/\beta), \log\alpha) = (\log\mu, \log\alpha)\) — \(\mu = \alpha/\beta\) being the population mean rate — is flat in the prior factor. Evaluated on a grid the contours are closed and concentrated around \(\log\mu \approx 4.75\) (\(\mu \approx 116\), the observed mean of the \(n_j\)) and \(\log\alpha \approx 1.2\); nothing visible happens at the edges.

(c) Not integrable. Take \(\alpha \to \infty\) with \(\mu = \alpha/\beta\) held fixed. The population distribution \(\mathrm{Gamma}(\alpha, \alpha/\mu)\) has mean \(\mu\) and variance \(\mu^2/\alpha \to 0\), so it degenerates to a point mass at \(\mu\) and

\begin{equation*} \prod_{j=1}^{10} p(n_j \mid \alpha, \beta) \ \longrightarrow \ c(\mu) := \prod_{j=1}^{10} \frac{\mu^{n_j} e^{-\mu}}{n_j!} \;>\; 0 . \end{equation*}

Since the prior is uniform in \(\log\alpha\) (the map \((\log\alpha,\log\beta) \mapsto (\log\mu,\log\alpha)\) has unit Jacobian), integrating the ray gives \(\int_{A}^{\infty} c(\mu)\, d\log\alpha = \infty\) for every fixed \(\mu\). This is exactly the failure of Exercise 5.9(a): a uniform prior on the log of the population dispersion leaves an infinite ridge as the dispersion goes to zero. The empirical check of (b) misses it, and must: the ridge height is \(\max_\mu c(\mu) = c(116) = e^{-188.6} \approx 1.2 \times 10^{-82}\), eighty orders of magnitude below the mode, because the data (\(\bar{n} = 116\), sample variance \(5142\)) are wildly overdispersed relative to Poisson.

(d) Follow the recipe of Exercise 5.9(b): put a uniform prior on the dispersion itself rather than its logarithm. The scale-free dispersion of \(\mathrm{Gamma}(\alpha,\beta)\) is its coefficient of variation \(\alpha^{-1/2}\) (playing the role of \((\alpha+\beta)^{-1/2}\) there), so take

\begin{equation*} p\!\left( \log\!\left( \tfrac{\alpha}{\beta} \right),\, \alpha^{-1/2} \right) \propto 1, \qquad \text{i.e.} \qquad p(\alpha, \beta) \propto \alpha^{-3/2}\beta^{-1}, \end{equation*}

the Jacobian being \(\left| \partial(\log(\alpha/\beta), \alpha^{-1/2}) / \partial(\alpha,\beta) \right| = \tfrac12 \alpha^{-3/2}\beta^{-1}\). The offending direction \(\alpha \to \infty\) now occupies the finite interval \(\alpha^{-1/2} \in (0,\varepsilon)\), so it contributes at most \(\varepsilon\) times a bounded integrand. For the opposite corner, \(\alpha \to 0\) and \(\beta \to 0\), expand each factor: \(\Gamma(\alpha+n)/\Gamma(\alpha) \to \alpha\,(n-1)!\) and \((\beta/(1+\beta))^{\alpha} \le \beta^{\alpha}\), so

\begin{equation*} \prod_{j} p(n_j \mid \alpha, \beta) \ \le \ C\, \alpha^{J} \beta^{\alpha J} \qquad (\alpha \le 1,\ \beta \le 1), \end{equation*}

and with \(J = 10\),

\begin{equation*} \int_0^1\!\!\int_0^1 \alpha^{-3/2}\beta^{-1} C \alpha^{J}\beta^{\alpha J} \, d\beta \, d\alpha = \frac{C}{J}\int_0^1 \alpha^{J - 5/2} \, d\alpha < \infty , \end{equation*}

finite whenever \(J > 3/2\). The remaining directions die fast: as \(\mu \to \infty\) at fixed \(\alpha\) each factor is \(O(\mu^{-\alpha})\), and as \(\mu \to 0\) it is \(O(\mu^{n_j})\). So the posterior is proper. (A uniform prior on the population standard deviation \(\sqrt{\alpha}/\beta\) rather than on the coefficient of variation would not work: its Jacobian gives \(p(\alpha,\beta) \propto \alpha^{-1/2}\beta^{-2}\), and the \(\beta\) integral above then diverges for \(\alpha \le 1/J\).)

Recomputing on a \(400 \times 400\) grid over \(\log(\alpha/\beta) \in [4.0, 5.6]\) and \(\alpha^{-1/2} \in (0, 3]\) (edge mass below \(10^{-4}\)), the contours are closed with posterior mode at

\begin{equation*} \big(\log(\alpha/\beta), \ \alpha^{-1/2}\big) = (4.754,\ 0.466), \qquad (\hat\alpha, \hat\beta) = (4.60,\ 0.0396). \end{equation*}

From \(10{,}000\) grid draws, posterior medians and 95% central intervals:

Across independent seeds these entries move by less than \(0.1\) on the \(\alpha\) scale and less than \(2\) on the \(\mu\) and \(\sqrt{\alpha}/\beta\) scales. The population standard deviation of the rates, median \(62\) against a mean of \(117\), dwarfs the Poisson noise \(\sqrt{116} \approx 11\); the totals are far from a common Poisson rate, which is the whole point of the hierarchy.

(e) Exactly as in Section 5.3: having drawn \((\alpha,\beta)\) from the grid, draw each \(\omega_j\) from its conditional posterior, which is gamma by conjugacy,

\begin{equation*} \omega_j \mid \alpha, \beta, n \ \sim \ \mathrm{Gamma}(\alpha + n_j, \ \beta + 1), \end{equation*}

independently across \(j\). The resulting posterior medians and 95% intervals are

Shrinkage toward \(\mu\) is almost nil — the posterior mean of \(\omega_j\) is \((\alpha + n_j)/(\beta+1)\), and with \(\hat\beta \approx 0.04\) and \(\hat\alpha \approx 3.5\) against \(n_j\) of order 100, each block’s own count dominates. That is the correct answer here: the population distribution is so dispersed that block \(j\)’s rate is learned almost entirely from block \(j\).

Everything is the eight-schools computation of Section 5.5 run on \((y_j,\sigma_j)\), \(J=22\): the model is

\begin{equation*} \begin{aligned} y_j\mid\theta_j &\sim \mathrm{N}(\theta_j,\sigma_j^2),\\ \theta_j\mid\mu,\tau &\sim \mathrm{N}(\mu,\tau^2),\\ p(\mu,\tau) &\propto 1, \end{aligned} \end{equation*}

with \(\sigma_j\) treated as known (justified in Section 5.6: every arm but a handful has \(n>50\)).

(a) By (5.21), with \(\hat\mu(\tau)\) and \(V_\mu(\tau)\) from (5.20),

\begin{equation*} \begin{aligned} V_\mu(\tau)^{-1}&=\sum_j\frac{1}{\sigma_j^2+\tau^2},\\ \hat\mu(\tau)&=V_\mu(\tau)\sum_j\frac{y_j}{\sigma_j^2+\tau^2},\\ p(\tau\mid y)&\propto V_\mu(\tau)^{1/2}\prod_{j=1}^{22}(\sigma_j^2+\tau^2)^{-1/2} \exp\!\left(-\frac{(y_j-\hat\mu(\tau))^2}{2(\sigma_j^2+\tau^2)}\right). \end{aligned} \end{equation*}

Evaluated on a grid of \(\tau\in(0,1)\) this density is unimodal with mode at \(\tau=0.12\) and is negligible beyond \(\tau=0.6\); \(\tau=0\) remains entirely plausible, the density there being \(0.68\) of the modal value (Section 5.6 describes it as about 25% lower). Quantiles, agreeing with Table 5.5 except in the extreme lower tail (Table 5.5 prints \(0.02\) for the 2.5% point; the grid gives \(0.008\)):

(figure bda3-ch05-meta-tau-posterior).

(b) Take \(\hat\theta_j\) and \(V_j=(\sigma_j^{-2}+\tau^{-2})^{-1}\) from (5.17) and average over \(\mu\mid\tau,y\sim\mathrm{N}(\hat\mu(\tau),V_\mu(\tau))\), using \(\mathrm{var}(\theta_j\mid\tau,y)=\mathrm{E}\,\mathrm{var}(\theta_j\mid\mu,\tau,y)+\mathrm{var}\,\mathrm{E}(\theta_j\mid\mu,\tau,y)\):

\begin{equation*} \begin{aligned} \mathrm{E}(\theta_j\mid\tau,y)&=V_j\left(\frac{y_j}{\sigma_j^2}+\frac{\hat\mu(\tau)}{\tau^2}\right),\\ \mathrm{sd}(\theta_j\mid\tau,y)^2&=V_j+\left(\frac{V_j}{\tau^2}\right)^{2}V_\mu(\tau), \end{aligned} \end{equation*}

At \(\tau=0\) all 22 curves start at the pooled estimate \(-0.26\) with common sd \(0.05\), and they fan out as \(\tau\) grows, each \(\mathrm{E}(\theta_j\mid\tau,y)\to y_j\) and \(\mathrm{sd}(\theta_j\mid\tau,y)\to\sigma_j\). Five representative studies:

(figures bda3-ch05-meta-cond-mean, bda3-ch05-meta-cond-sd). The imprecise studies 1 and 18 are still pinned near the mean at \(\tau=0.4\), while the precise studies 7, 10, 14 have all but reached their own \(y_j\) by \(\tau=0.2\).

(c) Drawing \(\tau\) from the grid, then \(\mu\mid\tau,y\), then \(\theta_j\mid\mu,\tau,y\) (20{,}000 draws, Monte Carlo standard error below \(0.006\) on every quantile below) reproduces the posterior quantiles printed in Table 5.4. The posterior medians span only \([-0.35,-0.12]\) while the \(y_j\) span \([-0.74,0.44]\), so the pooling is severe throughout. The amount of pooling is the shrinkage factor

\begin{equation*} B_j=\frac{\sigma_j^2}{\sigma_j^2+\tau^2} \qquad(\text{the weight on }\mu\text{ in }\mathrm{E}(\theta_j\mid\mu,\tau,y)), \end{equation*}

evaluated here at the posterior median \(\tau=0.13\); it is increasing in \(\sigma_j\), and \(\sigma_j\) falls roughly like \(n_j^{-1/2}\), which is the assertion to be verified (the ordering by \(n_j\) is not exact, since \(\sigma_j\) depends on the event rates as well as on \(n_j\): study 18 has \(n=305\) but a larger \(\sigma_j\) than study 2 with \(n=230\)):

The four smallest trials carry \(y_j\) ranging over \(0.8\) on the log-odds scale and yet have posterior medians within \(0.04\) of each other and of \(\mu\): they are pooled \(93\%\)-\(98\%\) of the way to the mean, while the two largest retain about half their own signal. In the scatterplot (bda3-ch05-meta-shrinkage, point size proportional to \(\sqrt{n_j}\)) the small points sit on the horizontal line \(\theta=\mathrm{median}(\mu\mid y)=-0.25\) and the large points lie nearest the 45-degree line.

(d) \(\tilde\theta_j\mid\mu,\tau\sim\mathrm{N}(\mu,\tau^2)\), the new study being assumed exchangeable with the 22 observed ones, drawn alongside each posterior draw gives a symmetric, slightly long-tailed histogram (bda3-ch05-meta-predictive) centred at \(-0.25\) with sd \(0.17\):

matching Table 5.5, and \(\Pr(\tilde\theta_j>0\mid y)=0.064\). (Section 5.6 reports this last probability as just over 10%, which is inconsistent with its own Table 5.5: a median of \(-0.25\) and a 97.5% point of \(0.11\) force a value near \(0.06\).)

(e) The control rate must be supplied; take the pooled control mortality of the 22 trials, \(p_0=0.100\), so \(\mathrm{logit}(p_0)=-2.197\) and, for each draw,

\begin{equation*} p_1=\mathrm{logit}^{-1}\!\left(\mathrm{logit}(p_0)+\tilde\theta_j\right),\qquad \tilde y_0\sim\mathrm{Bin}(100,p_0),\quad \tilde y_1\sim\mathrm{Bin}(100,p_1), \end{equation*}

and \(\tilde y\) is then (5.23) applied to \(\tilde y_0,\tilde y_1\) (with \(0.5\) added to each of the four counts, as recommended in Section 5.6, since \(n=100\) at a \(10\%\) rate produces empty cells in \(0.05\%\) of replications). The histogram (bda3-ch05-meta-newexp) has mean \(-0.25\) and sd \(0.54\):

and \(\Pr(\tilde y<0)=0.64\) against \(\Pr(\tilde\theta_j<0)=0.94\). Decomposing that sd by conditioning on \(\tilde\theta_j\), binomial sampling noise contributes \(0.51\) and the genuine spread of the effect only \(0.19\), and \(\sqrt{0.51^2+0.19^2}=0.54\).

About 850 people in each group, that is 1700 in total.

The new study’s own data supply \(y\mid\theta\sim\mathrm{N}(\theta,\sigma^2)\) and the meta-analysis supplies the prior \(p(\theta\mid y_{1:22})\), which by Exercise 5.15(d) has standard deviation \(s=0.17\) (Table 5.5: \(\tilde\theta_j\) has 95% interval \([-0.58,0.11]\)). Since \(\mathrm{E}(\theta\mid y)\) weights \(y\) and the prior mean by \(\sigma^{-2}\) and \(s^{-2}\), equal weighting is exactly

\begin{equation*} \sigma^2=s^2=0.17^2=0.0289. \end{equation*}

With \(n\) in each arm and death probabilities \(p_0\approx p_1\approx p\), (5.24) gives

\begin{equation*} \sigma^2=\frac{1}{np}+\frac{1}{n(1-p)}+\frac{1}{np}+\frac{1}{n(1-p)} =\frac{2}{n\,p(1-p)}, \end{equation*}

so \(n=2/\bigl(p(1-p)s^2\bigr)\). The 22 trials pool to a mortality rate of \(p=0.089\) (1811 deaths in 20{,}290 patients), whence \(2/(p(1-p))=24.6\) and

\begin{equation*} n=\frac{24.6}{0.0289}=851 \quad\text{per group}. \end{equation*}

(Check: \(\sigma=\sqrt{24.6/851}=0.170\).) Only trials 4, 7, 10 and 14 of Table 5.4 are this large.

Method (2): if one takes the prior for a new study to be \(\mathrm{N}(\mu,\tau^2)\) with \(\mu\) and \(\tau\) fixed at their posterior medians, ignoring hyperparameter uncertainty, the requirement becomes \(\sigma=\tau=0.13\) and \(n=24.6/0.0169=1455\) per group.

(a) \(p(\mu,\sigma)\propto 1\) on \(\mu\in\mathbb{R}\), \(\sigma>0\): the uniform density on \((\mu,\sigma)\), the default of Sections 5.4 and 5.7 for the hierarchical normal model. The posterior is proper: the uniform density on \(\sigma\) has a finite integral near \(\sigma=0\), and its infinite mass at \(\sigma\to\infty\) makes the posterior improper only for \(J=1\) or \(2\) groups (Section 5.7), whereas here \(J=100\). The tempting alternative \(p(\mu,\log\sigma)\propto 1\), that is \(p(\sigma)\propto\sigma^{-1}\), must be avoided, since it gives an improper posterior with an infinite spike at \(\sigma=0\) (Section 5.4).

(b) Independently,

\begin{equation*} \mu\sim\mathrm{N}(0.05,\,0.04^2),\qquad \sigma\sim\mathrm{N}^{+}(0,\,0.05^2), \end{equation*}

the second being a half-normal. The mean: a month of 50 shots a day is 1500 practice shots, enough to move a typical non-varsity college student, who starts around \(60\%\), up by a few percentage points, but the gain is bounded by the \(40\%\) of headroom remaining and some students will get worse, so a prior 95% interval \((-0.03,0.13)\) for \(\mu\) is honest. The scale: the person-to-person spread of a real training gain should be comparable to its mean, and it is in any case hard to learn about, since each \(\theta_i\) is measured by a difference of two binomial proportions whose own standard deviation is \(\sqrt{2(0.6)(0.4)/100}=0.07\); the half-normal above has median \(0.034\) and 95th percentile \(0.098\), which covers that range and puts vanishing mass on values of \(\sigma\) the design could never resolve.

(c) Independently,

\begin{equation*} \mu\sim\mathrm{N}(0,\,0.2^2),\qquad \sigma\sim\text{half-Cauchy}(0,\,0.2), \end{equation*}

the half-Cauchy family of Section 5.7 with scale \(A\) set large relative to the plausible range. These are proper and encode only what the measurement scale itself guarantees, that \(\theta_i\) is a difference of two probabilities and so lies in \([-1,1]\): the prior 95% interval for \(\mu\) is \((-0.39,0.39)\), essentially the whole range physically available, and the prior for \(\sigma\) has median \(0.2\), so it neither favours \(\sigma=0\) nor, having a gentle tail rather than a half-normal’s, prevents the likelihood from pulling \(\sigma\) upward if the data ask for it.

(a) Yes, it fits: the posterior predictive \(p\)-value for the largest observed effect is \(0.39\).

The identical-schools model is \(y_j \mid \theta \sim \mathrm{N}(\theta, \sigma_j^2)\) independently, \(j = 1, \dots, 8\), with the \(\sigma_j\) of Table 5.2 known and \(p(\theta) \propto 1\) (proper posterior, since \(\sum_j \sigma_j^{-2} > 0\)). By (2.11)-(2.12),

\begin{equation*} \theta \mid y \sim \mathrm{N}(\hat{\theta}, V), \quad \hat{\theta} = \frac{\sum_j y_j/\sigma_j^2}{\sum_j 1/\sigma_j^2} = 7.69, \quad V = \Big(\sum_j \sigma_j^{-2}\Big)^{-1} = 16.6, \end{equation*}

the pooled estimate \(7.7\) with standard error \(4.1\) of page 119. Replications \(y^{\mathrm{rep}}\) are drawn from (6.1), here \(\int \prod_j \mathrm{N}(y_j^{\mathrm{rep}} \mid \theta, \sigma_j^2) p(\theta \mid y)\, d\theta\); the aspect of the data examined on page 120 is the order-statistic vector \(T(y) = (y_{(1)} \ge \cdots \ge y_{(8)})\), with scalar summary \(T_{\max}(y) = \max_j y_j = 28\) carrying the question actually asked there (“would it be possible to have one school’s observed effect be 28 just by chance?”). With \(S = 200{,}000\) draws \(\theta^s \sim \mathrm{N}(7.69, 4.07^2)\), \(y_j^{\mathrm{rep}\,s} \sim \mathrm{N}(\theta^s, \sigma_j^2)\):

(the first row reproduces the book’s \((26,19,14,10,6,2,-3,-9)\), which was computed from the cruder reference distribution \(y_j \sim \mathrm{N}(8, 13^2)\)). Every observed order statistic lies well inside one predictive standard deviation of its expectation – their individual upper-tail probabilities \(\Pr(y^{\mathrm{rep}}_{(k)} \ge y_{(k)} \mid y)\) run from \(0.22\) to \(0.59\) – and

\begin{equation*} p_B = \Pr\big(T_{\max}(y^{\mathrm{rep}}) \ge 28 \mid y\big) = 0.39 . \end{equation*}

Companion quantities agree: \(\Pr(\min_j y_j^{\mathrm{rep}} \le -3 \mid y) = 0.79\) and \(\Pr(\mathrm{range}(y^{\mathrm{rep}}) \ge 31 \mid y) = 0.67\). (Monte Carlo standard error \(\sqrt{p(1-p)/S} \le 0.002\) throughout.) The identical-schools model fits this aspect of the data.

(b) Because the test has essentially no power against the alternatives that matter. The observed spread of the \(y_j\) is generated almost entirely by the known sampling errors \(\sigma_j \in [9,18]\), which swamp any plausible between-school variation. Simulating eight schools with \(\theta_j \sim \mathrm{N}(8, \tau^2)\) and \(y_j \mid \theta_j \sim \mathrm{N}(\theta_j, \sigma_j^2)\) gives

so the data would look almost the same under \(\tau = 0\) and under \(\tau = 10\); passing the check is no evidence for \(\tau = 0\). But the practical questions all turn on \(\tau\): the pooled model forces \(\theta_A = \theta_C\), hence asserts \(\Pr(\theta_A > \theta_C \mid y) = 0\) against the hierarchical model’s \(0.70\) (page 123), hands every school the identical interval \([-0.3, 15.7]\), and predicts a new school’s effect with standard error \(4.1\) rather than \(\sqrt{\tau^2 + 4.1^2}\). As page 120 notes, it implies “the probability is \(\tfrac{1}{2}\) that the true effect in A is less than \(7.7\)”, which no one reading Table 5.2 believes.

(a) Two discrepancy measures, both functions of data and parameter. Writing \(E_t = \theta\) (models (a), (c)) or \(E_t = \theta x_t\) (models (b), (d)) for the fitted rate in year \(t\), \(t = 1, \dots, 10\), and \(\bar{t} = 5.5\), take

\begin{equation*} \begin{aligned} T_1(y, \theta) &= \sum_{t=1}^{10} \frac{(y_t - E_t)^2}{E_t}, \\ T_2(y, \theta) &= \frac{\sum_t (t - \bar{t})\,(y_t - E_t)/\sqrt{E_t}} {\sum_t (t - \bar{t})^2}. \end{aligned} \end{equation*}

\(T_1\) is the \(\chi^2\) discrepancy of (6.7); under independent Poisson sampling it has mean \(10\) regardless of \(\theta\), so it detects the overdispersion (or underdispersion) produced by any failure of the independent-Poisson assumption. \(T_2\) is the least-squares slope of the standardized residuals on time, so its posterior predictive distribution is centred at \(0\) under the model and it detects a monotone drift in the rate.

The exposures, obtained as \(x_t = (\text{deaths}_t)/(\text{death rate}_t)\) in units of \(10^{11}\) passenger miles, are

(b) With the Jeffreys prior \(p(\theta) \propto \theta^{-1/2}\) of Exercise 2.12, the posterior is \(\theta \mid y \sim \mathrm{Gamma}(\sum_t y_t + \tfrac12,\ \sum_t x_t)\), and \(S = 40{,}000\) draws \((\theta^s, y^{\mathrm{rep}\,s})\) give

Here \(p_1 = \Pr(T_1(y^{\mathrm{rep}},\theta) \ge T_1(y,\theta) \mid y)\) and, since the substantive alternative is a declining rate, \(p_2 = \Pr(T_2(y^{\mathrm{rep}},\theta) \le T_2(y,\theta) \mid y)\); the Monte Carlo standard error is at most \(0.003\). (For model (b), \(\hat\theta = 4.17\) accidents per \(10^{11}\) miles; for (d), \(121\) deaths per \(10^{11}\) miles.) The graphical display is the realized-versus-replicated scatterplot of Figure 6.4: plot the \(S\) pairs \((T(y,\theta^s), T(y^{\mathrm{rep}\,s},\theta^s))\) with the \(45^{\circ}\) line, one panel per model, \(T_1\) on a \(\log_{10}\) scale because for (c) and (d) the realized values lie two orders of magnitude to the right of the cloud.

Reading the table: the accident counts are consistent with a constant-rate Poisson process (\(p_1 = 0.50\)) but show a marginal downward drift (\(p_2 = 0.044\)); once exposure is included the drift becomes overwhelming (\(p_2 < 0.001\)) and drags the dispersion with it (\(p_1 = 0.004\)), because exposure grew by \(84\%\) while accidents fell. The death counts are grossly overdispersed under both models: \(T_1 = 830\) and \(1026\) against a predictive mean of \(10\), a \(p\)-value indistinguishable from \(0\).

(c) Yes. Exercise 2.13(e) argues on general principles that the Poisson model is most plausible for fatal accidents, since accidents occur as rare independent events in a large exposure, and least plausible for passenger deaths, since deaths arrive in clumps – one accident kills from \(0\) to several hundred people at once, so deaths are a compound rather than a simple Poisson process. The checks confirm exactly this ordering: model (a) passes the dispersion check and the other three fail, with the two death models failing by a factor of \(100\). The checks add one thing the general argument did not: the accident rate per passenger mile is clearly declining over the decade, so no constant-rate model – with or without exposure – is adequate.

(a) A Poisson log-linear trend in the rate per passenger mile,

\begin{equation*} \begin{aligned} y_t \mid \alpha, \beta &\sim \mathrm{Poisson}\big(x_t \exp(\alpha + \beta (t - \bar{t}))\big), \\ t &= 1, \dots, 10, \quad \bar{t} = 5.5, \end{aligned} \end{equation*}

with \(y_t\) the fatal accidents, \(x_t\) the exposure in \(10^{11}\) passenger miles, and \(p(\alpha, \beta) \propto 1\) (proper posterior: the log-likelihood is strictly concave in \((\alpha,\beta)\) with \(\to -\infty\) in every direction, since the \(x_t\) are positive and the \(t\) are not all equal). This is the minimal repair of the two defects Exercise 6.2 exposed in model (b): exposure belongs in the model (traffic grew \(84\%\) over the decade), and the rate per mile is not constant – aviation safety improved steadily through the late 1970s and early 1980s, so the rate should decay smoothly rather than jump. A multiplicative trend keeps the rate positive and matches the form the substantive claim takes, “safety improves by a few percent a year”. Passenger deaths are not modelled here: their overdispersion is compound, not a trend.

(b) The posterior was computed on a \(401 \times 401\) grid over \((\alpha, \beta) \in [0.5, 2.5] \times [-0.35, 0.05]\) and \(20{,}000\) draws taken from it:

\begin{equation*} \begin{aligned} \alpha \mid y &: \ \mathrm{E} = 1.432, \ \mathrm{sd} = 0.066, \\ \beta \mid y &: \ \mathrm{E} = -0.1045, \ \mathrm{sd} = 0.0231, \\ &\quad\ \ 95\% \text{ interval } [-0.150,\, -0.060]. \end{aligned} \end{equation*}

So the baseline rate at mid-decade is \(e^{1.432} = 4.19\) fatal accidents per \(10^{11}\) passenger miles, falling by a factor \(e^{\beta} = 0.901\) per year, \(95\%\) interval \([0.860, 0.942]\) – about a \(10\%\) annual improvement, with \(\Pr(\beta < 0 \mid y) > 0.9999\).

(c) \(\mathrm{E}[y_{1986} \mid y] = 17.1\), with \(95\%\) posterior predictive interval \([8, 28]\), obtained from \(y_{1986} \sim \mathrm{Poisson}(8.0 \cdot e^{\alpha + 6.5\beta})\). Compared with the models of Exercise 2.13:

The exposure-only model (b) is the worst forecaster of the three: it projects the 1986 traffic increase forward while holding the rate fixed at its decade average, so it predicts \(33\) accidents. The trend model does the opposite, extrapolating the \(10\%\) annual decline one year past the data. The realized value was \(22\), for which the trend model gives \(\Pr(y_{1986} \ge 22 \mid y) = 0.18\) – inside the interval, but in its upper tail.

(d) Yes, decisively. The same two discrepancies give

\begin{equation*} \begin{aligned} T_1: \quad &\mathrm{E}[T_1 \mid y] = 7.5 \ \ (\text{predictive mean } 10.0), && p_1 = 0.68, \\ T_2: \quad &\mathrm{E}[T_2 \mid y] = 0.006, && p_2 = 0.52, \end{aligned} \end{equation*}

against \(p_1 = 0.004\), \(p_2 < 0.001\) for model (b). The trend has absorbed the systematic drift, and with it the excess dispersion that the drift had been creating: ten counts now scatter about their fitted rates exactly as independent Poisson variables should.

The analysis chosen is Rubin (1981), “Estimation in parallel randomized experiments”, Journal of Educational Statistics 6, 377-401 – the hierarchical normal analysis of the eight SAT-V coaching experiments reproduced in Section 5.5. The fitted model is

\begin{equation*} y_j \mid \theta_j \sim \mathrm{N}(\theta_j, \sigma_j^2), \quad \theta_j \mid \mu, \tau \sim \mathrm{N}(\mu, \tau^2), \quad p(\mu, \tau) \propto 1, \end{equation*}

with \((y_j, \sigma_j)\) as in Table 5.2 of Exercise 6.1; the \(\theta_j\) are taken exchangeable, which is what licenses the common population distribution and is defensible here because nothing beyond \(y_j\) distinguishes the eight programs (Section 5.5). The uniform prior on \(\tau\) yields a proper posterior (Section 5.4). Draws are taken by the factorization of Section 5.4: a grid draw of \(\tau\) from (5.21), then \(\mu \mid \tau, y\) from (5.20), then \(\theta_j \mid \mu, \tau, y\) from (5.17). Posterior median \(\tau = 5.3\); \(\mathrm{E}[\theta_A \mid y] = 11.4\) with \(\mathrm{sd}\ 8.4\).

(a) The model fits: no test quantity is in a tail. Replications \(y^{\mathrm{rep}}_j \sim \mathrm{N}(\theta_j^s, \sigma_j^2)\) from \(20{,}000\) posterior draws give

with \(p_B = \Pr(T(y^{\mathrm{rep}}) \ge T(y) \mid y)\), Monte Carlo standard error \(0.003\). The only mild signal is that the observed data are slightly less dispersed than typical replications, which is what one expects when the posterior for \(\tau\) has substantial mass away from \(0\) while the data themselves are consistent with \(\tau = 0\). The check has little power here, exactly as in Exercise 6.1(b), because \(\tau\) is a free parameter devoted precisely to matching the dispersion of the \(y_j\).

(b) The inferences that matter are moderately sensitive to the prior on \(\tau\) and insensitive to the shape of the population distribution.

The first three rows are the plausible alternatives and they agree: \(\mathrm{E}[\theta_A \mid y]\) moves only from \(11.4\) to \(9.4\), and \(\Pr(\theta_A > 28.4 \mid y)\) – the separate-analysis point estimate – stays between \(0.01\) and \(0.04\), so Rubin’s central conclusion (school A’s effect is nothing like \(28\), and the eight schools cannot be reliably ranked) survives. Replacing the normal population distribution by a \(t_4\), fitted by Gibbs sampling with the normal scale-mixture representation of the \(t\), changes nothing to within Monte Carlo error, because with one observation per group there is no information with which to identify the tail behaviour of \(p(\theta_j \mid \mu, \tau)\).

The fourth row is the cautionary one, and it is the row Figure 5.9 warns about: \(\text{Inv-gamma}(0.001, 0.001)\) on \(\tau^2\) is proper, but it is so sharply peaked near zero that the posterior median falls to \(0.6\) and the analysis slides towards complete pooling – the answer Section 5.5 rejects on substantive grounds. (A uniform density on \(\log \tau\) is worse still: Section 5.4 notes it gives an improper posterior, so it cannot be used at all.) With only \(J = 8\) groups the data carry little information about \(\tau\), so any prior peaked at \(0\) dominates; the uniform prior on \(\tau\) is the defensible default and the published analysis is a fair representative of the reasonable answers.

Agreed, and it says that a model check must report a magnitude, not a verdict.

Savage’s premise is right: a sharp null such as \(\theta_1 = \theta_2\) is a measure-zero subset of the parameter space and is essentially never true of two real treatments, two real populations, or two real schools. Its rejection is then a statement about \(n\), not about nature – for any fixed \(\theta_1 \ne \theta_2\) the power tends to \(1\), so rejection is guaranteed by enough data. The Bayesian response (Section 6.1, and Exercises 3.2-3.4) is not to test \(\theta_1 = \theta_2\) at all but to report the posterior distribution of \(\theta_1 - \theta_2\): the issue is not whether the difference is exactly zero but how large it is and how well it is determined. The same holds for models, which are never exactly true, so

\begin{equation*} \Pr\big(T(y^{\mathrm{rep}}) \ge T(y) \mid y\big) \longrightarrow 0 \quad \text{as } n \to \infty \end{equation*}

for any test quantity \(T\) sensitive to a discrepancy the model genuinely has, however small. A posterior predictive \(p\)-value is not an estimate of anything; it is a statement about the resolution of the current dataset.

That does not condemn the model checking of this chapter, because the enterprise is not hypothesis testing. Three differences do the work.

(i) The checks are not conducted with an accept/reject decision at the end. The chapter’s own instruction (Section 6.3) is that “we prefer to look at the magnitude of the discrepancy as well as its \(p\)-value”, and the graphical displays – Figures 6.3-6.5, the realized-versus-replicated scatterplots – are the primary output, the \(p\)-value a numerical caption. The speed-of-light check reports that \(\min(y_i) = -44\) is far outside its replicate distribution, and the interesting content is the size of the gap, not that it is significant.

(ii) The null being examined is the fitted model in its entirety, which is being used, not merely entertained. A discrepancy is worth acting on when it distorts the inferences one intends to draw, which is a question of effect size relative to purpose. Exercise 6.1 is the complement of Savage’s point: there the identical-schools model passes the check at \(p_B = 0.39\) and is nevertheless unacceptable, because the check had no power against the alternatives that matter. A \(p\)-value near \(\tfrac12\) is therefore no more a contribution to science than a \(p\)-value near \(0\); the choice of \(T\) is what carries the scientific content.

(iii) Rejection here is constructive rather than terminal. There is no “accept the null”; there is only the loop of Section 6.1 – find where the replications fail to look like the data, expand the model in the direction indicated, refit, and check again, as in Exercises 6.2 and 6.3, where the failed checks on the constant-rate airline models name the missing ingredient (exposure and a declining rate per mile) rather than merely declaring the models false.

(a) Because the stopping rule contributes a factor that does not involve \(\theta\). Under the new protocol the sample size \(n\) is random, the final observation is necessarily a \(0\), and the data are determined by the positions of the \(7\) ones among the first \(19\) places. The sampling density is

\begin{equation*} p(y \mid \theta) = \theta^{7}(1-\theta)^{13} \quad \text{for any admissible sequence } y, \end{equation*}

identical to the fixed-\(n\) density, since in both cases the probability of one particular ordered sequence with \(7\) ones and \(13\) zeros is \(\theta^7(1-\theta)^{13}\). The two protocols differ only in which sequences are admissible, that is, in the support of \(p(y \mid \theta)\) – and that support does not depend on \(\theta\). Hence the likelihood function of \(\theta\) is the same up to a constant, and by the likelihood principle (Section 8.1, and the definition of ignorability in Section 8.2) the posterior is \(\mathrm{Beta}(8,14)\) under both. What changes is not \(p(\theta \mid y)\) but the reference set \(\{y^{\mathrm{rep}}\}\) used for the check.

(b) \(\Pr(T(y^{\mathrm{rep}}) \le 3 \mid y) = 0.070\), against \(0.028\) under the fixed-\(n\) protocol. Replications were drawn \(2 \times 10^5\) times by \(\theta^s \sim \mathrm{Beta}(8,14)\), then the number of ones \(k^s \sim \mathrm{Neg\text{-}bin}(13, 1 - \theta^s)\) preceding the thirteenth zero, then a uniformly random arrangement of those \(k^s\) ones among the first \(12 + k^s\) positions with a \(0\) in the last. Display the histogram of \(T(y^{\mathrm{rep}\,s})\) on integer bins with a vertical line at \(T(y) = 3\), beside Figure 6.5 on the same axes; numerically,

with Monte Carlo standard error at most \(0.001\), and with the new predictive distribution of \(n\) having mean \(21.0\), standard deviation \(5.3\), and \(95\%\) interval \([14, 34]\) (only \(8.6\%\) of replicate datasets have \(n\) exactly \(20\)).

Three differences from Figure 6.5. First, the distribution is much wider – standard deviation \(4.1\) against \(2.6\) – because \(n\) is now random and \(T\) grows roughly linearly in \(n\); the extra variance in the reference set is precisely the information the fixed-\(n\) protocol was supplying for free. Second, it is visibly bimodal in parity: \(T\) must be even when the sequence begins with \(0\) and odd when it begins with \(1\), and since \(\mathrm{E}[\theta \mid y] = 8/22 = 0.36\) the even values carry about two-thirds of the mass, producing the alternating comb

whereas under the fixed-\(n\) protocol both endpoints are free and no parity constraint arises. Third, and as a consequence of the first two, the check is less decisive: the observed \(T(y) = 3\), an odd value falling in a trough of the comb, now sits at the \(7\)th percentile rather than the \(3\)rd. The evidence against the independence assumption is in the same direction and of the same kind, but the wider reference set costs the test some power.

(a) \(p_B = 0.13\). Since \(|\bar{y}| = 5.1 \ll A\), the truncation is irrelevant and the posterior is \(\theta \mid y \sim \mathrm{N}(\bar{y}, 1/n) = \mathrm{N}(5.1, 0.1^2)\) by (2.11)-(2.12). The posterior predictive distribution is therefore the exchangeable normal

\begin{equation*} \begin{aligned} y^{\mathrm{rep}} \mid y &\sim \mathrm{N}\big(5.1\,\mathbf{1},\ \Sigma\big), \\ \Sigma &= I_{100} + \tfrac{1}{100}\mathbf{1}\mathbf{1}^{T}, \end{aligned} \end{equation*}

that is, \(y_i^{\mathrm{rep}} = \theta + \varepsilon_i\) with \(\theta \sim \mathrm{N}(5.1, 0.01)\) and \(\varepsilon_i\) i.i.d. \(\mathrm{N}(0,1)\): each margin is \(\mathrm{N}(5.1, 1.01)\) with pairwise correlation \(1/101\). Because \(\theta\) is pinned near \(5.1\), all hundred draws are positive with overwhelming probability and \(T = \max_i y_i^{\mathrm{rep}}\), so

\begin{equation*} \begin{aligned} p_B &= \Pr\big(T(y^{\mathrm{rep}}) \ge 8.1 \mid y\big) \\ &= \int \Big(1 - \Phi(8.1 - \theta)^{100}\Big)\, \mathrm{N}(\theta \mid 5.1, 0.01)\, d\theta \;=\; 0.131, \end{aligned} \end{equation*}

confirmed by \(2 \times 10^5\) simulations, which give \(0.1309\) (Monte Carlo standard error \(0.0008\)). The predictive distribution of \(T\) has mean \(7.61\), standard deviation \(0.44\), and \(95\%\) interval \([6.87, 8.59]\); its histogram is a single mound over roughly \([6.4, 9.2]\) with the observed \(8.1\) marked by a vertical line well inside the right shoulder. The observed maximum is entirely ordinary.

(b) \(p \approx 0.9999\). The prior predictive distribution is \(y_i^{\mathrm{rep}} = \theta + \varepsilon_i\) with \(\theta \sim \mathrm{U}[-A, A]\), \(A = 10^5\), and \(\varepsilon_i\) i.i.d. \(\mathrm{N}(0,1)\) – marginally each \(y^{\mathrm{rep}}_i\) is very nearly uniform on \([-A, A]\), and the hundred components are locked together, all lying within about \(\pm 3\) of the common \(\theta\). Hence

\begin{equation*} T(y^{\mathrm{rep}}) = \max_i |\theta + \varepsilon_i| \approx |\theta|, \end{equation*}

so \(T(y^{\mathrm{rep}})\) is approximately uniform on \([0, A]\) – a flat slab stretching from \(0\) to \(100{,}000\), with only an \(8 \times 10^{-5}\) sliver of its mass below \(8.1\) – and

\begin{equation*} \Pr\big(T(y^{\mathrm{rep}}) \ge 8.1\big) \approx 1 - \frac{8.1}{10^5} = 0.99992, \end{equation*}

simulation giving \(0.99991\). By the convention of Section 6.3 the \(p\)-value is \(0.9999\), which is as extreme as \(0.0001\): the observed statistic sits in the far left tail of its prior predictive reference distribution.

(c) Yes, and it is the point of the example: the two checks are testing different things. The prior predictive check evaluates the joint model \(p(\theta)p(y \mid \theta)\), prior included, and here it detects nothing about the normal sampling model whatever – it detects that the prior is absurd as a description of where \(\theta\) actually lives. The reference distribution of \(T\) is spread over \(10^5\) units, so every dataset with a plausible mean lands in its extreme lower tail, and the \(p\)-value is driven towards \(1\) by increasing \(A\) alone, without touching the data: it equals \(1 - T(y)/A\), a function of the prior’s range and nothing else, and has no limit as a probability statement when \(A \to \infty\), the prior predictive distribution then being improper.

The posterior predictive check, by contrast, conditions on \(y\) through \(\theta\), so the location of the reference distribution is set by the data rather than by \(A\), and the comparison is between the observed \(\max_i |y_i|\) and what the \(\mathrm{N}(\theta, 1)\) sampling model predicts for it – the question actually asked. Its verdict, \(p_B = 0.13\), correctly reports that a maximum absolute value of \(8.1\) among \(100\) observations centred at \(5.1\) with unit variance is unremarkable, so prior predictive checks are informative only when the prior is itself a substantive part of the model being tested.

(a) Marginalizing the fresh school effects out of the population distribution,

\begin{equation*} p(y^{\mathrm{rep}} \mid y) = \int\!\!\int \prod_{j=1}^{8} N\!\left(y_j^{\mathrm{rep}} \mid \mu, \tau^2+\sigma_j^2\right) p(\mu,\tau \mid y)\, d\mu\, d\tau , \end{equation*}

since the inner replication draws \(\theta_j^{\mathrm{rep}} \mid \mu,\tau \sim N(\mu,\tau^2)\) and then \(y_j^{\mathrm{rep}} \mid \theta_j^{\mathrm{rep}} \sim N(\theta_j^{\mathrm{rep}},\sigma_j^2)\), which convolve to \(N(\mu,\tau^2+\sigma_j^2)\) independently across \(j\) — the same marginalization that produces (5.18).

The corresponding experiment is a study of eight new coaching programs, drawn from the same population of schools that produced the original eight, and evaluated with the same designs (hence the same known standard errors \(\sigma_j\)). Under the reference set of Section 6.5 the same eight programs are re-evaluated; here only the population \((\mu,\tau)\) is held fixed.

(b) Less extreme, because the reference distribution is strictly more dispersed: conditional on the same \((\mu,\tau)\), \(y_j^{\mathrm{rep}}\) has variance \(\tau^2+\sigma_j^2\) here against \(V_j+\sigma_j^2\) in Section 6.5, where \(V_j = (\sigma_j^{-2}+\tau^{-2})^{-1} < \tau^2\) by (5.17). So every test statistic has a wider predictive distribution and the observed value sits nearer its middle. The effect should be slight, since \(E(\tau \mid y) \approx 6.5\) is small beside the \(\sigma_j\), which run from 9 to 18.

(c) Drawing \(200\,000\) posterior samples of \((\theta,\mu,\tau)\) (grid on \(\tau\) from (5.21), then \(\mu\) from (5.20) and \(\theta\) from (5.17), as in Section 5.4; the improper \(p(\tau) \propto 1\) does give a proper posterior here, which is why Section 5.4 adopts it rather than \(p(\log\tau) \propto 1\)) and replicating under each reference set gives \(p = \Pr(T(y^{\mathrm{rep}}) \ge T(y) \mid y)\), with Monte Carlo standard error at most \(0.002\):

The third column reproduces Figure 6.12, whose p-values from only 200 draws are \(0.54\), \(0.19\), \(0.50\), \(0.78\). The predictive standard deviations do widen as predicted, for instance \(\mathrm{sd}(\max_j y_j^{\mathrm{rep}})\) rises from \(10.9\) to \(12.4\) and \(\mathrm{sd}(\mathrm{mean}(y_j^{\mathrm{rep}}))\) from \(6.2\) to \(7.5\), and the p-values for \(\max\), \(\min\) and \(\mathrm{mean}\) move toward \(1/2\) as speculated. The \(\mathrm{sd}\) p-value instead edges from \(0.78\) to \(0.79\): widening the reference set also shifts \(E(\mathrm{sd}(y^{\mathrm{rep}}))\) up from \(13.8\) to \(14.5\), and here the shift beats the spread. In neither reference set does any of the four statistics indicate misfit.

The model passes every check of the within-group binomial fit and of the shape of the population of rates, but fails decisively on one scientifically pointed quantity: the tumor rate rises with the size of the experiment, which an exchangeable \(\mathrm{Beta}(\alpha,\beta)\) prior on the \(\theta_j\) forbids.

Fitting as in Section 5.3 (grid on \((\log(\alpha/\beta), \log(\alpha+\beta))\) over the region of Figure 5.3, then \(\theta_j \mid \alpha,\beta,y \sim \mathrm{Beta}(\alpha+y_j, \beta+n_j-y_j)\) by (5.7); the improper hyperprior (5.9) is the one choice in Section 5.3 shown to leave the posterior integrable, so truncating to the grid loses nothing) gives \(20\,000\) draws with

\begin{equation*} E(\alpha \mid y) = 2.14, \quad E(\beta \mid y) = 12.71, \quad E\!\left(\tfrac{\alpha}{\alpha+\beta}\,\Big|\,y\right) = 0.145 , \end{equation*}

matching the book’s approximate population rate of \(0.14\) on page 128. Two reference sets are useful:

• replication (i), the standard one of (6.1): \(y_j^{\mathrm{rep}} \sim \mathrm{Bin}(n_j,\theta_j)\) with the posterior \(\theta_j\) held fixed, which checks the binomial sampling model;
• replication (ii), 71 new experiments: \(\theta_j^{\mathrm{rep}} \sim \mathrm{Beta}(\alpha,\beta)\) redrawn, then \(y_j^{\mathrm{rep}} \sim \mathrm{Bin}(n_j,\theta_j^{\mathrm{rep}})\), which checks the population model and the exchangeability of the \(\theta_j\) across groups.

With \(p = \Pr(T(y^{\mathrm{rep}}) \ge T(y) \mid y)\) and \(p_j = y_j/n_j\) (Monte Carlo standard error at most \(0.004\)):

The first three are unremarkable. The count of tumor-free groups, the scientifically salient “how often does a control group show nothing” quantity, is 14 against a predictive mean of \(10.3\) and 95% interval \([4,17]\) under (i): high, but within the reference distribution. The largest observed rate, \(9/24 = 0.375\), is if anything smaller than the model expects (\(E = 0.47\)), so there is no outlying group. The chi-square discrepancy

\begin{equation*} T(y,\theta) = \sum_{j=1}^{71} \frac{(y_j - n_j\theta_j)^2}{n_j \theta_j (1-\theta_j)} \end{equation*}

gives \(\Pr(T(y^{\mathrm{rep}},\theta) \ge T(y,\theta) \mid y) = 0.50\): no overdispersion beyond what the beta population already supplies.

The last two rows are the failure. Under (ii) the replicated correlation between rate and group size is centered at \(0.00\) with 95% interval \([-0.20, 0.23]\), and only 3 replications in 1000 reach the observed \(0.335\); the rank correlation is worse, \(1\) in \(1000\). (Under (i) the check has little power: the \(\theta_j\) are conditioned on the very data whose correlation is at issue, so the reference distribution inherits it, centering at \(0.22\).) Substantively, the larger experiments in Tarone’s table are systematically the ones with higher tumor incidence, so \(\theta_j\) and \(n_j\) are dependent and the groups are not exchangeable; \(n_j\) should enter the model, for example through a regression of \(\mathrm{logit}(\theta_j)\) on \(\log n_j\) as in the hierarchical regression of Chapter 15.

The apparent decrease is not real: the sharpest summary of it, the ratio of residual standard deviations for large and small spreads, has posterior predictive p-value \(0.13\), and the replicated scatterplots are indistinguishable from the data.

The model of Section 1.6 has no free parameters, so \(p(y^{\mathrm{rep}} \mid y) = \prod_{i=1}^{672} N(y_i^{\mathrm{rep}} \mid x_i, 14^2)\) and the posterior predictive reference distribution coincides with the prior predictive one; the p-values below are exact tail probabilities of a fully specified null. The observed residuals satisfy \(\overline{y-x} = 0.07\) and \(\mathrm{sd}(y-x) = 13.86\), confirming the \(N(0,14^2)\) calibration overall.

(a) Five replications \(y_i^{\mathrm{rep}} = x_i + 14 z_i\), \(z_i \sim N(0,1)\), plotted with the jitter of Figures 1.1 and 1.2a alongside the real data, show the same funnel: the apparent narrowing at large \(x\) is present in the replications too, because point spreads above \(10\) occur in only 34 of the 672 games and extreme residuals are simply rarer where the data are thin. (Figures bda3-ch06-football-outcome and bda3-ch06-football-resid.)

(b) Take

\begin{equation*} T(x,y) = \frac{\mathrm{sd}\{y_i-x_i : x_i \ge 6.5\}}{\mathrm{sd}\{y_i-x_i : x_i \le 3.5\}} , \end{equation*}

which splits the games into 177 large-spread and 327 small-spread games and centers at 1 under constant variance. Observed, \(T(x,y) = 0.926\); over \(10\,000\) replications \(T(x,y^{\mathrm{rep}})\) has mean \(1.001\), standard deviation \(0.067\) and central 95% interval \([0.874, 1.136]\), so

\begin{equation*} p = \Pr\!\left(T(x,y^{\mathrm{rep}}) \le T(x,y) \,\middle|\, y\right) = 0.13 \end{equation*}

with Monte Carlo standard error \(0.003\) (the lower tail, since the alternative of interest is variance decreasing in \(x\)). A second summary, the correlation \(T_2(x,y) = \mathrm{corr}(|y_i-x_i|, x_i) = -0.030\) against a reference distribution of mean \(0.000\) and standard deviation \(0.039\), gives \(p = 0.22\). Binning confirms the picture, with observed and 95% predictive standard deviations of \(y-x\):

Every observed value lies inside its interval.

\begin{equation*} \mathrm{AIC}=7.96,\quad \mathrm{DIC}=7.91,\quad \mathrm{WAIC}=7.81, \end{equation*}

and LOO cross-validation gives \(\mathrm{lppd}_{\mathrm{loo-cv}}=-4.89\); the four data points are the \(n=4\) prediction units.

AIC. The maximum likelihood estimate is \((\hat\alpha,\hat\beta)_{\mathrm{mle}}=(0.85,7.75)\) with

\begin{equation*} \log p(y\mid\hat\theta_{\mathrm{mle}})=\sum_{i=1}^{4}\log\mathrm{Bin}(y_i\mid n_i,\theta_i)=-1.98 . \end{equation*}

Two parameters are estimated, so by (7.6)

\begin{equation*} \widehat{\mathrm{elpd}}_{\mathrm{AIC}}=-1.98-2=-3.98, \qquad \mathrm{AIC}=7.96 . \end{equation*}

DIC. Posterior summaries come from the Section 3.7 grid posterior on \([-5,10]\times[-10,40]\). The posterior mean is \(\hat\theta_{\mathrm{Bayes}}=(1.31,11.61)\), and

\begin{equation*} \begin{aligned} \log p(y\mid\hat\theta_{\mathrm{Bayes}}) &= -2.23,\\ E_{\mathrm{post}}\bigl(\log p(y\mid\theta)\bigr) &= -3.09 . \end{aligned} \end{equation*}

Hence by (7.8)–(7.9)

\begin{equation*} p_{\mathrm{DIC}}=2\bigl(-2.23-(-3.09)\bigr)=1.73, \end{equation*}

\begin{equation*} \widehat{\mathrm{elpd}}_{\mathrm{DIC}}=-2.23-1.73=-3.95,\qquad \mathrm{DIC}=7.91 . \end{equation*}

The alternative (7.10) gives \(p_{\mathrm{DIC\,alt}}=2\,\mathrm{var}_{\mathrm{post}}(\log p(y\mid\theta))=2.41\); the posterior of \((\alpha,\beta)\) is skewed, so the mean sits off the mode and the two effective-parameter counts straddle \(k=2\).

WAIC. Pointwise, with \(\mathrm{lppd}_i=\log E_{\mathrm{post}}p(y_i\mid\theta)\) and \(V_i=\mathrm{var}_{\mathrm{post}}\log p(y_i\mid\theta)\),

so by (7.5), (7.11), (7.13)

\begin{equation*} \begin{aligned} \mathrm{lppd} &= -2.74, &\quad p_{\mathrm{WAIC}\,1} &= 0.71,\\ p_{\mathrm{WAIC}\,2} &= 1.17, &\quad \widehat{\mathrm{elppd}}_{\mathrm{WAIC}} &= -3.90, \end{aligned} \end{equation*}

and \(\mathrm{WAIC}=-2(-3.90)=7.81\). The two extreme doses are predicted almost perfectly and contribute essentially no effective parameters; all the fitting happens at \(x_2,x_3\).

Cross-validation. By (7.14), evaluating each \(p_{\mathrm{post}(-i)}(y_i)\) on the Section 3.7 grid,

\begin{equation*} \mathrm{lppd}_{\mathrm{loo-cv}}=-4.89,\qquad p_{\mathrm{loo-cv}}=\mathrm{lppd}-\mathrm{lppd}_{\mathrm{loo-cv}}=2.16 . \end{equation*}

With \(n=4\) the bias correction is not negligible: \(\mathrm{lppd}_{-i}=-3.27\), so \(b=\mathrm{lppd}-\mathrm{lppd}_{-i}=0.53\) and \(\mathrm{lppd}_{c\,\mathrm{loo-cv}}=-4.36\).

The grid bound is not cosmetic. Under the untruncated flat prior the leave-one-out posteriors omitting \(i=2\) or \(i=3\) are improper: deleting \(x_2=-0.30\) leaves responses that are separated except at the single dose \(x_3\), and along the ray \(\beta\to\infty\) with \(\alpha+\beta x_3\) held fixed the likelihood tends to the positive constant \(\binom{5}{3}\theta^3(1-\theta)^2\), so \(\int\!\!\int p(y_{-2}\mid\alpha,\beta)\,d\alpha\,d\beta\) diverges linearly in \(\beta\). (Numerically, doubling the integration box doubles the mass.) The numbers above therefore use the bounded grid of Section 3.7, i.e. the uniform prior truncated to \([-5,10]\times[-10,40]\), under which all four leave-one-out posteriors are proper; \(\mathrm{lppd}_{\mathrm{loo-cv}}\) for \(i=2,3\) would drift downward without that truncation. AIC, DIC and WAIC, which condition on all four points, are unaffected.

Both halves reduce to the same two facts: \(p_{\mathrm{DIC}}=k\), and the in-sample log predictive density overshoots \(\mathrm{elpd}_{\hat\theta}\) by exactly \(k\).

The exact normal case. Take \(y\sim \mathrm{N}(X\beta,\sigma^2 I_n)\) with \(\sigma^2\) known, \(X\) of full column rank \(k\), and \(p(\beta)\propto 1\); then \(\beta\mid y\sim\mathrm{N}(\hat\beta,\sigma^2(X^TX)^{-1})\) with \(\hat\beta=(X^TX)^{-1}X^Ty=\hat\beta_{\mathrm{Bayes}}\). The log-likelihood is exactly quadratic,

\begin{equation*} \log p(y\mid\beta)=\log p(y\mid\hat\beta) -\tfrac{1}{2\sigma^{2}}(\beta-\hat\beta)^T X^TX(\beta-\hat\beta), \end{equation*}

so, since \(E_{\mathrm{post}}(\beta-\hat\beta)(\beta-\hat\beta)^T=\sigma^2(X^TX)^{-1}\),

\begin{equation*} \begin{aligned} E_{\mathrm{post}}\log p(y\mid\beta) &=\log p(y\mid\hat\beta)-\tfrac{1}{2\sigma^{2}}\,\mathrm{tr}\bigl(X^TX\cdot\sigma^2(X^TX)^{-1}\bigr)\\ &=\log p(y\mid\hat\beta)-\tfrac{k}{2}, \end{aligned} \end{equation*}

whence \(p_{\mathrm{DIC}}=k\) exactly and \(\widehat{\mathrm{elpd}}_{\mathrm{DIC}}=\log p(y\mid\hat\beta)-k\).

Now take expectations over \(y\sim\mathrm{N}(X\beta_0,\sigma^2I)\). Since \(\|y-X\hat\beta\|^2/\sigma^2\sim\chi^2_{n-k}\),

\begin{equation*} E_y\log p(y\mid\hat\beta)=-\tfrac{n}{2}\log(2\pi\sigma^2)-\tfrac{n-k}{2}. \end{equation*}

For an independent replicate \(\tilde y\sim\mathrm{N}(X\beta_0,\sigma^2 I)\), using \(\tilde y\perp\hat\beta\) and \(E\|X(\hat\beta-\beta_0)\|^2=\sigma^2k\),

\begin{equation*} \begin{aligned} \mathrm{elpd}_{\hat\theta} &=-\tfrac{n}{2}\log(2\pi\sigma^2)-\tfrac{1}{2\sigma^{2}}E\|\tilde y-X\hat\beta\|^{2}\\ &=-\tfrac{n}{2}\log(2\pi\sigma^2)-\tfrac{n}{2}-\tfrac{k}{2}. \end{aligned} \end{equation*}

Subtracting, \(E_y\log p(y\mid\hat\beta)-\mathrm{elpd}_{\hat\theta}=k\), i.e.

\begin{equation*} E_y\bigl(\widehat{\mathrm{elpd}}_{\mathrm{DIC}}\bigr)=\mathrm{elpd}_{\hat\theta}. \end{equation*}

The asymptotic case. Under the Chapter 4 regularity conditions the posterior concentrates, \(\theta\mid y\approx\mathrm{N}(\hat\theta,J^{-1})\) with \(J=-\nabla^2\log p(y\mid\theta)|_{\hat\theta}\) the observed information, \(\hat\theta_{\mathrm{Bayes}}=\hat\theta_{\mathrm{mle}}+O_p(n^{-1})\), and the log-likelihood is quadratic to the same order:

\begin{equation*} \log p(y\mid\theta)=\log p(y\mid\hat\theta)-\tfrac12(\theta-\hat\theta)^TJ(\theta-\hat\theta)+o_p(1). \end{equation*}

Averaging over the posterior gives \(E_{\mathrm{post}}\log p(y\mid\theta)=\log p(y\mid\hat\theta)-\tfrac12\mathrm{tr}(JJ^{-1})+o_p(1)\), so

\begin{equation*} p_{\mathrm{DIC}}=k+o_p(1), \end{equation*}

the \(\chi^2_k\)-based drop of \(k/2\) noted on p. 171.

For the bias, let \(\theta_0\) be the true (or Kullback–Leibler optimal) value and \(I=E_y(J)=n\,I_1\). Expanding in \(\theta\) about \(\hat\theta\) and about \(\theta_0\),

\begin{equation*} \begin{aligned} \log p(y\mid\hat\theta)-\log p(y\mid\theta_0) &=\tfrac12(\hat\theta-\theta_0)^TJ(\hat\theta-\theta_0)+o_p(1),\\ E_{\tilde y}\log p(\tilde y\mid\theta_0)-E_{\tilde y}\log p(\tilde y\mid\hat\theta) &=\tfrac12(\hat\theta-\theta_0)^TI(\hat\theta-\theta_0)+o_p(1), \end{aligned} \end{equation*}

the second line because \(\theta_0\) maximizes \(\theta\mapsto E_{\tilde y}\log p(\tilde y\mid\theta)\) with curvature \(I\) there. Asymptotic normality gives \(\hat\theta-\theta_0\sim\mathrm{N}(0,I^{-1})\) and \(J=I+o_p(n)\), so both quadratic forms are \(\chi^2_k\) in the limit and each has expectation \(k/2\). Adding,

\begin{equation*} \begin{aligned} E_y\log p(y\mid\hat\theta)-\mathrm{elpd}_{\hat\theta} &=\tfrac{k}{2}+\tfrac{k}{2}+o(1)\\ &=k+o(1), \end{aligned} \end{equation*}

since \(E_y\log p(y\mid\theta_0)=E_{\tilde y}\log p(\tilde y\mid\theta_0)\) (\(y\) and \(\tilde y\) have the same distribution). Combining with \(p_{\mathrm{DIC}}=k+o_p(1)\) and \(\log p(y\mid\hat\theta_{\mathrm{Bayes}})=\log p(y\mid\hat\theta_{\mathrm{mle}})+o_p(1)\),

\begin{equation*} E_y\bigl(\widehat{\mathrm{elpd}}_{\mathrm{DIC}}\bigr)=\mathrm{elpd}_{\hat\theta}+o(1). \end{equation*}

The analogue of Table 7.1 for these 22 studies, with \(y_j\) the \(n=22\) prediction units:

Blanks are undefined exactly as in Table 7.1: AIC is defined relative to a maximum likelihood estimate and a fixed parameter count \(k\), neither of which is appropriate for a hierarchical model whose 22 \(\theta_j\) are constrained by an estimated \(\tau\); LOO-CV needs a predictive density for a held-out \(y_j\), and under no pooling \(\theta_j\) has a flat prior and no other data bear on it.

How the entries are obtained. Under no pooling \(\hat\theta_j=y_j\) and \(\theta_j\mid y\sim\mathrm{N}(y_j,\sigma_j^2)\), so

\begin{equation*} -2\log p(y\mid\hat\theta)=\sum_{j=1}^{22}\log(2\pi\sigma_j^2)=-11.9, \end{equation*}

\(p_{\mathrm{DIC}}=22.0\) (exactly \(k\), as in Exercise 7.2), and

\begin{equation*} p_{\mathrm{WAIC}\,2}=\sum_{j}\mathrm{var}_{\mathrm{post}} \log \mathrm{N}(y_j\mid\theta_j,\sigma_j^2)=\tfrac{22}{2}=11, \end{equation*}

since \((y_j-\theta_j)^2/\sigma_j^2\sim\chi^2_1\) has variance 2 and enters with a factor \(-\tfrac12\).

Under complete pooling \(\theta_j\equiv\mu\) with \(\mu\mid y\sim\mathrm{N}(-0.260,0.0503^2)\), giving \(-2\,\mathrm{lpd}=11.4\), \(p_{\mathrm{DIC}}=1.0\), \(p_{\mathrm{WAIC}\,2}=1.32\). LOO is available in closed form: \(y_j\mid y_{-j}\sim\mathrm{N}(\hat\mu_{-j},\sigma_j^2+V_{-j})\) with \(\hat\mu_{-j},V_{-j}\) the precision-weighted mean and variance from (5.20) applied to \(y_{-j}\), and summing gives \(-2\,\mathrm{lppd}_{\mathrm{loo-cv}}=13.85\) against \(\mathrm{WAIC}=13.83\) — for a one-parameter model with \(n=22\) the two agree to the second digit, as they should.

For the hierarchical model we simulate from the Section 5.4 factorization: \(p(\tau\mid y)\) on a grid, then \(\mu\mid\tau,y\) and \(\theta_j\mid\mu,\tau,y\) normal. This reproduces the Section 5.6 inference (\(E(\tau\mid y)=0.13\), \(E(\mu\mid y)=-0.25\)). With \(S=4\times10^{5}\) draws (Monte Carlo error under \(0.02\) on each entry),

\begin{equation*} \begin{aligned} -2\log p(y\mid\hat\theta_{\mathrm{Bayes}}) &= 1.68, &\quad p_{\mathrm{DIC}} &= 6.01,\\ -2\,\mathrm{lppd} &= 3.94, &\quad p_{\mathrm{WAIC}\,2} &= 4.76 . \end{aligned} \end{equation*}

Both effective sizes fall between 1 and 22 as they must. For LOO, the held-out study’s effect is a fresh draw \(\theta_j\mid\mu,\tau\sim\mathrm{N}(\mu,\tau^2)\), so \(\theta_j\) and \(\mu\) integrate out analytically and only \(\tau\) needs the grid:

\begin{equation*} \begin{aligned} p(y_j\mid y_{-j})=\int \mathrm{N}\bigl(y_j\mid\hat\mu_{-j}(\tau),\; \sigma_j^2+\tau^2+V_{-j}(\tau)\bigr)\\ \times\,p(\tau\mid y_{-j})\,d\tau, \end{aligned} \end{equation*}

refitting \(p(\tau\mid y_{-j})\) for each \(j\). Summing the 22 terms gives \(-2\,\mathrm{lppd}_{\mathrm{loo-cv}}=14.5\) and \(p_{\mathrm{loo-cv}}=5.3\).

The no-pooling model has by far the best raw fit and by far the worst corrected score, while the hierarchical and complete-pooling models are indistinguishable on all three criteria, as they must be with \(\hat\tau\approx0.13\) small relative to the typical \(\sigma_j\approx0.3\).

(a) With \(w_j=\sigma_j^{-2}\), \(W=\sum_j w_j\), \(\bar y=W^{-1}\sum_j w_j y_j\) and \(S=\sum_j w_j(y_j-\bar y)^2\),

\begin{equation*} \frac{p(H_2\mid y)}{p(H_1\mid y)}=\frac{p(H_2)}{p(H_1)}\,B, \end{equation*}

\begin{equation*} \begin{aligned} B&=\frac{p(y\mid H_2)}{p(y\mid H_1)} =\frac{\prod_{j}\bigl(1+A^2/\sigma_j^2\bigr)^{1/2}}{(1+WA^2)^{1/2}}\\ &\qquad\times\exp\Bigl\{-\tfrac{S}{2}-\frac{\bar y^{2}}{2(A^{2}+W^{-1})} +\sum_{j}\frac{y_j^{2}}{2(\sigma_j^{2}+A^{2})}\Bigr\}. \end{aligned} \end{equation*}

Both marginal likelihoods are elementary normal integrals. Under \(H_1\) the \(\theta_j\) are independent, so the \(y_j\) are marginally independent:

\begin{equation*} p(y\mid H_1)=\prod_{j=1}^{J}\mathrm{N}(y_j\mid 0,\sigma_j^2+A^2). \end{equation*}

Under \(H_2\), complete the square in \(\theta\) using \(\sum_j w_j(y_j-\theta)^2=S+W(\theta-\bar y)^2\):

\begin{equation*} \begin{aligned} p(y\mid H_2)&=\prod_j(2\pi\sigma_j^2)^{-1/2}e^{-S/2} \int e^{-\frac{W}{2}(\theta-\bar y)^2}\mathrm{N}(\theta\mid0,A^2)\,d\theta\\ &=\prod_j(2\pi\sigma_j^2)^{-1/2}e^{-S/2}\Bigl(\tfrac{2\pi}{W}\Bigr)^{1/2} \mathrm{N}\bigl(\bar y\mid 0,W^{-1}+A^2\bigr), \end{aligned} \end{equation*}

the last step because \(\int\mathrm{N}(\theta\mid\bar y,W^{-1})\mathrm{N}(\theta\mid0,A^2)d\theta=\mathrm{N}(\bar y\mid 0,W^{-1}+A^2)\). Dividing, the factors \((2\pi/W)^{1/2}(2\pi(W^{-1}+A^2))^{-1/2}=(1+WA^2)^{-1/2}\) and \(\prod_j(2\pi\sigma_j^2)^{-1/2}\big/\prod_j(2\pi(\sigma_j^2+A^2))^{-1/2}=\prod_j(1+A^2/\sigma_j^2)^{1/2}\) give the display. (Checked against numerical integration of \(p(y\mid H_2)\) for the eight-schools data of Table 5.2: \(\log B\) agrees to six decimals, giving \(B=63.0\) at \(A=30\) and \(B=8.24\times10^{11}\) at \(A=1000\).)

(b) \(B\to\infty\) like \(A^{J-1}\): as \(A\to\infty\),

\begin{equation*} (1+WA^2)^{-1/2}\sim W^{-1/2}A^{-1},\qquad \prod_j\bigl(1+A^2/\sigma_j^2\bigr)^{1/2}\sim \frac{A^{J}}{\prod_j\sigma_j}, \end{equation*}

and both terms in the exponent involving \(A\) vanish, so

\begin{equation*} B\;\sim\;\frac{A^{J-1}}{W^{1/2}\prod_j\sigma_j}\,e^{-S/2}\;\longrightarrow\;\infty \qquad(J\ge 2). \end{equation*}

So with noninformative priors taken as the limit of proper ones, \(p(H_2\mid y)\to1\) whatever the data: the complete-pooling model wins by fiat, because \(H_1\) pays the \(A^{-1}\) price of a diffuse prior \(J\) times and \(H_2\) pays it once. This is the sensitivity to \(A^2\) asserted on p. 184.

(c) \(\log B=Jc-\tfrac12\log J+O(1)\), so \(B\to\infty\) or \(0\) exponentially in \(J\) according to the sign of

\begin{equation*} c=\tfrac12\log\Bigl(1+\frac{A^{2}}{\sigma^{2}}\Bigr)-\frac{v}{2\sigma^{2}} +\frac{v+\bar y^{2}}{2(\sigma^{2}+A^{2})}, \end{equation*}

where \(\bar y=\frac1J\sum_j y_j\) and \(v=\frac1J\sum_j(y_j-\bar y)^2\) are the fixed sample mean and variance. Indeed with \(\sigma_j\equiv\sigma\) we have \(W=J/\sigma^2\), \(S=Jv/\sigma^2\) and \(\sum_j y_j^2=J(v+\bar y^2)\), so

\begin{equation*} \begin{aligned} \log B&=-\tfrac12\log\bigl(1+JA^{2}/\sigma^{2}\bigr) +\tfrac{J}{2}\log\bigl(1+A^{2}/\sigma^{2}\bigr)\\ &\quad-\frac{Jv}{2\sigma^{2}}-\frac{\bar y^{2}}{2(A^{2}+\sigma^{2}/J)} +\frac{J(v+\bar y^{2})}{2(\sigma^{2}+A^{2})}, \end{aligned} \end{equation*}

and collecting the terms proportional to \(J\) gives \(c\), the rest being \(-\tfrac12\log J+O(1)\).

The sign is decided by whether the observed spread matches what \(H_1\) predicts. If \(v=\sigma^2+A^2\) and \(\bar y=0\) — exactly the marginal variance under \(H_1\) — then, writing \(r=A^2/\sigma^2\),

\begin{equation*} c=\tfrac12\log(1+r)-\tfrac{1+r}{2}+\tfrac12=\tfrac12\bigl(\log(1+r)-r\bigr)<0, \end{equation*}

so \(H_1\) wins exponentially; if the \(y_j\) are less dispersed than that, \(c>0\) and \(H_2\) wins exponentially. For the eight-schools numbers (\(\bar y=8.8\), \(v^{1/2}=9.8\), \(\sigma\approx12.6\)) and \(A=20\) one gets \(c=+0.48\), hence \(B=9.4\) at \(J=8\) but \(B=3.7\times10^{15}\) at \(J=80\): with \(\sigma_j\equiv\sigma\) the Bayes factor depends on the data only through \((\bar y,v)\), which are being held fixed, so the whole movement is the \(J\) in the exponent.

(a) Changing the units of \(y\) tilts the posterior of \(\phi\): for any \(c>0\),

\begin{equation*} p(\phi\mid cy)\;\propto\;c^{\phi}\,p(\phi\mid y). \end{equation*}

To see this, integrate out \((\mu,\log\sigma)\) first. Including the Jacobian \(\prod_i \mathrm{d}y_i^{(\phi)}/\mathrm{d}y_i=\prod_i y_i^{\phi-1}\),

\begin{equation*} p(y\mid\mu,\sigma,\phi)=\prod_{i}\mathrm{N}\bigl(y_i^{(\phi)}\mid\mu,\sigma^2\bigr)\prod_i y_i^{\phi-1}, \end{equation*}

and with \(p(\mu,\log\sigma)\propto1\) the standard normal integral (as in Section 3.2) gives

\begin{equation*} p(\phi\mid y)\;\propto\;p(\phi)\;\Bigl(\prod_i y_i\Bigr)^{\phi-1} \Bigl[\sum_i\bigl(y_i^{(\phi)}-\overline{y^{(\phi)}}\bigr)^2\Bigr]^{-(n-1)/2}. \end{equation*}

Now \((cy_i)^{(\phi)}=c^{\phi}y_i^{(\phi)}+(c^{\phi}-1)/\phi\), an affine map, so the bracket is multiplied by \(c^{2\phi}\) and the bracket to the power \(-(n-1)/2\) by \(c^{-(n-1)\phi}\), while the Jacobian factor is multiplied by \(c^{n(\phi-1)}\). The product of the two is \(c^{\phi-n}\), which up to the \(\phi\)-free constant \(c^{-n}\) is the claimed tilt.

Numerical example. Take \(n=10\) observations

\begin{equation*} y=(1.2,\;2.4,\;3.1,\;4.0,\;5.5,\;6.2,\;8.0,\;9.7,\;12.4,\;18.0), \end{equation*}

with \(p(\phi)\propto1\). The posterior mode of \(\phi\) is

The same measurements, reported in different units, are said to need a log transformation, a square-root transformation, no transformation, or a square. Worse, the tilt destroys propriety: as \(\phi\to\infty\) the log posterior is asymptotically linear with slope \(\sum_i\log(cy_i)-(n-1)\log(c\,y_{\max})=-9.15+\log c\), so for \(c>e^{9.15}\approx 9.4\times10^{3}\) — a change from grams to milligrams and a little — \(p(\phi\mid cy)\) is not integrable at all. A prior that is flat in \((\mu,\log\sigma)\) cannot be right, because \(\mu\) and \(\sigma\) live on the \(y^{(\phi)}\) scale, whose units change with \(\phi\).

(b) The factor \(\dot y^{\,1-\phi}\) is exactly the compensating Jacobian. With it,

\begin{equation*} \begin{aligned} p(\phi\mid y)&\propto p(\phi)\,\dot y^{\,1-\phi}\,\dot y^{\,n(\phi-1)} \Bigl[\sum_i\bigl(y_i^{(\phi)}-\overline{y^{(\phi)}}\bigr)^2\Bigr]^{-(n-1)/2}\\ &= p(\phi)\Bigl[\frac{\sum_i(y_i^{(\phi)}-\overline{y^{(\phi)}})^2}{\dot y^{\,2(\phi-1)}}\Bigr]^{-(n-1)/2}, \end{aligned} \end{equation*}

using \(\prod_i y_i^{\phi-1}=\dot y^{\,n(\phi-1)}\). Under \(y\to cy\) the numerator of the bracket picks up \(c^{2\phi}\) and the denominator \(c^{2(\phi-1)}\), so the whole bracket picks up \(c^{2}\), independent of \(\phi\); hence

\begin{equation*} p(\phi\mid cy)=p(\phi\mid y)\quad\text{for every }c>0 . \end{equation*}

Equivalently, the model in (b) is the flat-prior model applied to the normalized transformation \(z_i(\phi)=y_i^{(\phi)}/\dot y^{\,\phi-1}\), whose units are those of \(y\) for every \(\phi\). In the example of (a) the posterior mode is \(\hat\phi=+0.21\) for \(c=0.1,1,10,100\) alike.

(c) From the display in (b),

\begin{equation*} p(\phi\mid y)\;\propto\;p(\phi)\;\dot y^{\,(n-1)(\phi-1)} \Bigl[\sum_{i=1}^{n}\bigl(y_i^{(\phi)}-\overline{y^{(\phi)}}\bigr)^{2}\Bigr]^{-(n-1)/2}, \end{equation*}

the omitted constant \((2\pi)^{-(n-1)/2}n^{-1/2}2^{(n-3)/2}\Gamma(\tfrac{n-1}{2})\) depending only on \(n\). The conditional posterior completing the model is the usual one: \(\sigma^2\mid\phi,y\sim\mathrm{Inv}\text{-}\chi^2(n-1,s^2_{(\phi)})\) and \(\mu\mid\sigma,\phi,y\sim\mathrm{N}(\overline{y^{(\phi)}},\sigma^2/n)\), with \(s^2_{(\phi)}\) the sample variance of the \(y_i^{(\phi)}\).

(d) The dependence is on \(\dot y\) only, and it is a change of units, not a change of belief. Because \(\mu\) and \(\sigma\) are parameters of the transformed data, a prior on them can only be stated once a scale for \(y^{(\phi)}\) is fixed, and that scale varies with \(\phi\); \(\dot y\) supplies one that is common to all \(\phi\). Formally the rule is not a Bayesian prior — it violates the likelihood principle, two data sets with proportional likelihoods in \(\phi\) but different geometric means get different answers, and the “prior” cannot be stated before seeing \(y\) — but it is an excellent approximation to the honest procedure, which is to put a genuine prior on \((\mu,\sigma)\) in fixed physical units of \(y\) and let it be vague on the scale of the observed data. Since \(\dot y\) is a consistent estimate of that scale, using it changes the answer by \(O(n^{-1/2})\) relative to the honest prior, while restoring exact invariance.

(e) The transformation (7.19) has bounded range: for \(\phi>0\), \(y_i^{(\phi)}>-1/\phi\), and for \(\phi<0\), \(y_i^{(\phi)}<1/|\phi|\); only \(\phi=0\) gives the whole line. So the normal model is at best an approximation and the correct likelihood carries a truncation factor, for \(\phi>0\),

\begin{equation*} p(y\mid\mu,\sigma,\phi)=\Bigl[\Phi\Bigl(\tfrac{\mu+1/\phi}{\sigma}\Bigr)\Bigr]^{-n} \prod_i\mathrm{N}\bigl(y_i^{(\phi)}\mid\mu,\sigma^2\bigr)\prod_i y_i^{\phi-1}, \end{equation*}

which is \(\ge1\) and equals 1 to high accuracy whenever \(\mu+1/\phi\gg\sigma\). Three consequences. First, the uncorrected analysis is safe exactly when the fitted transformation works well — a good normal fit means few standard deviations of room below the boundary — and is misleading precisely in the regions of \((\mu,\sigma,\phi)\) where the normal approximation is poor, which are the regions a model check should be flagging. Second, the uncorrected likelihood is not a density in \(y\), so posterior predictive draws \(\tilde y^{(\phi)}<-1/\phi\) occur and correspond to no real \(\tilde y\); predictive inference must either truncate or admit that \(\tilde y\) is undefined with some probability. Third, and the reason this is more than a technicality, no \(\phi\ne0\) model can be literally true for all-positive data with support down to 0, so the family should be read as a device for finding a good approximate scale, with \(\phi\) reported as an interval rather than selected and then conditioned on.

Deleting the asterisked values leaves \(n=12\), \(10\), \(11\) basement measurements:

(a) Blue Earth alone gives posterior mode \(\hat\phi=0.59\) with 95% interval \([-0.37,\,1.81]\) — the data cannot distinguish a log transformation from none at all. With \(C=1\) group, Exercise 7.5(c) gives

\begin{equation*} p(\phi\mid y)\propto \dot y^{\,(n-1)(\phi-1)} \Bigl[\textstyle\sum_{i}\bigl(y_i^{(\phi)}-\overline{y^{(\phi)}}\bigr)^2\Bigr]^{-(n-1)/2}, \end{equation*}

evaluated on a grid for \(\phi\) with \(p(\phi)\propto1\); here \(\dot y=6.42\). Posterior summaries: median \(0.63\), quartiles \([0.28,1.01]\), \(\Pr(\phi<0\mid y)=0.11\). The posterior density at \(\phi=0\) (lognormal) is \(0.51\) of its maximum and at \(\phi=1\) (normal) is \(0.75\) of its maximum, so with \(n=12\) mildly skewed observations neither special case is contradicted.

(b) With \(C=3\) counties, \(N=33\) observations, common \(\phi\) and free \((\mu_c,\log\sigma_c)\), the invariance argument of Exercise 7.5(b) forces the prior \(p(\{\mu_c\},\{\log\sigma_c\},\phi)\propto\dot y^{\,C(1-\phi)}p(\phi)\), with \(\dot y\) the geometric mean of all \(N\) observations, and integrating out the six nuisance parameters gives

\begin{equation*} p(\phi\mid y)\;\propto\;p(\phi)\,\dot y^{\,(N-C)(\phi-1)} \prod_{c=1}^{3}\Bigl[\sum_{i\in c}\bigl(y_i^{(\phi)}-\overline{y^{(\phi)}_c}\bigr)^{2}\Bigr]^{-(n_c-1)/2}. \end{equation*}

(Each group contributes \(c^{-(n_c-1)\phi}\) to the scaling and the Jacobian contributes \(c^{N(\phi-1)}\), so the compensating exponent is \(C(1-\phi)\), not \(1-\phi\); check the case \(C=1\).) The result is \(\hat\phi=-0.03\), 95% interval \([-0.51,\,0.44]\): pooling the three counties pins \(\phi\) down to roughly a factor of two in the exponent and puts it essentially at the logarithm. Given \(\phi\), \(\sigma_c^2\mid\phi,y\sim\mathrm{Inv}\text{-}\chi^2(n_c-1,s^2_{c(\phi)})\) and \(\mu_c\mid\sigma_c,\phi,y\sim\mathrm{N}(\overline{y^{(\phi)}_c},\sigma_c^2/n_c)\); back-transforming \(\mu_c\) gives the posterior median radon level in each county,

(Posterior median \(\sigma_c\): \(0.55\), \(0.92\), \(0.78\); since \(\hat\phi\approx0\) these read as standard deviations on the log scale.)

The disagreement among counties fitted separately is worth recording, because it is what the common-\(\phi\) assumption is buying: \(\hat\phi=0.59\) (Blue Earth), \(0.42\) (Clay), \(-0.84\) (Goodhue). Goodhue is pulled down by its single value of \(43.5\).

(c) Simulate \(\phi\) from the grid posterior, then \((\sigma_c^2,\mu_c)\) from the conditionals above, then \(y^{\mathrm{rep}}_c\) of the same size as \(y_c\), and back-transform. With \(S=20{,}000\) replications, \(\Pr\bigl(T(y^{\mathrm{rep}})\ge T(y)\mid y\bigr)\) is

Only Goodhue fails, and it fails on skewness (\(p=0.02\)): after the common transformation its 11 values are more right-skewed than replications allow, because \(43.5\) sits far above the rest. The maximum alone does not flag it (\(p=0.18\)), since the fitted \(\sigma_3\) is inflated by that very point — the familiar effect of checking an outlier with a statistic that the outlier itself calibrates.

One further check comes free. In \(5.8\%\) of the simulated replicate data sets (\(0.6\%\), \(3.8\%\), \(2.2\%\) by county) at least one value of \(y^{\mathrm{rep},(\phi)}\) fell outside the range of the transformation (\(y^{(\phi)}>1/|\phi|\) for the negative \(\phi\) draws, which are \(56\%\) of the posterior), so it corresponds to no real measurement. That is the implicit truncation of Exercise 7.5(e) showing up at a rate too large to ignore.

(d) Yes, for these data. The lognormal is \(\phi=0\), which is the posterior median of the common-\(\phi\) fit (\(\hat\phi=-0.03\)), lies well inside the 95% interval \([-0.51,0.44]\), and has posterior density \(0.99\) of the maximum; nothing is lost by setting \(\phi=0\) and much is gained — the parameters \((\mu_c,\sigma_c)\) become interpretable as the mean and standard deviation of \(\log\) radon and are comparable across counties and across data sets, the truncation problem of part (c) disappears entirely since the lognormal has support \((0,\infty)\), and the three-county comparison becomes a standard normal-theory problem. The estimated transformation should be reported as the justification for the lognormal, not suppressed: the width of the interval for \(\phi\) is a statement that these 33 observations could not have detected a moderate departure from lognormality, and the Goodhue skewness check says the one real defect — a heavy upper tail in one county — is not repaired by any choice of \(\phi\) and calls instead for a longer-tailed model or for a hierarchical treatment of the counties as in Section 9.4.

Take

\begin{equation*} p(\mu,\log\sigma,\nu)\;\propto\;\nu^{-2},\qquad \nu\in[1,\infty), \end{equation*}

that is, uniform on \(\nu^{-1}\in(0,1]\) with \(\mu,\log\sigma\) flat given \(\nu\). It is proper in \(\nu\) (total mass \(\int_1^\infty\nu^{-2}d\nu=1\)) and improper only in the location and scale, where impropriety is harmless. Each of the four difficulties of Exercise 7.5 is either absent or is what dictates the choice.

Invariance under change of units (7.5a,b). Absent. For every fixed \(\nu\), \(t_\nu(\mu,\sigma^2)\) is a location–scale family, so under \(y\to a+cy\) the likelihood is exactly reproduced by \((\mu,\sigma,\nu)\to(a+c\mu,\,c\sigma,\,\nu)\), and \(p(\mu,\log\sigma)\propto1\) is invariant under that map. There is no Jacobian mismatch between different values of the expansion parameter — \(\mu\) and \(\sigma\) mean the same thing for every \(\nu\), whereas in (7.19) they refer to the \(y^{(\phi)}\) scale, whose units depend on \(\phi\). Hence no data-dependent factor \(\dot y^{\,1-\phi}\) is needed and none should be used.

Propriety (the real constraint). As \(\nu\to\infty\), \(t_\nu(\mu,\sigma^2)\to\mathrm{N}(\mu,\sigma^2)\), so the integrated likelihood

\begin{equation*} p(y\mid\nu)=\iint\prod_i t_\nu(y_i\mid\mu,\sigma^2)\,d\mu\,\frac{d\sigma}{\sigma} \end{equation*}

converges to \(p_{\mathrm{N}}(y)>0\) rather than to \(0\) (proper for \(n\ge2\) with at least two distinct \(y_i\), as for the normal). Therefore

\begin{equation*} \int_1^\infty p(y\mid\nu)\,d\nu=\infty , \end{equation*}

and a flat prior on \(\nu\in[1,\infty)\) gives an improper posterior — the exact analogue of the divergent tail found in Exercise 7.5(a), and the reason “uniform on \(\nu\)” is not an option. Any admissible \(p(\nu)\) must be integrable at \(\infty\); \(\nu^{-2}\) is.

Which “uniform” (7.5’s parameterization problem). Choose the parameterization in which the likelihood is regular at the limit point. The \(t_\nu\) density is smooth in \(\nu^{-1}\) at \(\nu^{-1}=0\), with

\begin{equation*} p(y\mid\nu)=p_{\mathrm{N}}(y)\Bigl(1+\frac{\kappa(y)}{\nu}+O(\nu^{-2})\Bigr), \end{equation*}

so \(\nu^{-1}\) is the parameter on whose scale the data carry approximately constant information near the normal, and to leading order it matches the excess kurtosis \(6/(\nu-4)\) that the expansion is actually buying. A numerical illustration on \(n=20\) observations

\begin{equation*} \begin{aligned} y=(&1.15,-0.48,2.31,0.02,-1.77,0.63,-0.21,4.90,\\ &0.35,-0.94,0.77,-2.55,1.02,0.18,-0.33,0.44,\\ &-1.21,0.91,-0.07,2.02) \end{aligned} \end{equation*}

gives \(\log p(y\mid\nu)-\log p_{\mathrm{N}}(y)=1.47,\;0.82,\;0.47,\;0.103,\;0.0209,\;0.00525\) at \(\nu=3,10,20,100,500,2000\): exactly the \(\kappa/\nu\) behaviour, with \(\kappa=10.3\). Under the proposed prior the posterior of \(\nu^{-1}\) is well behaved, with median \(0.45\) and 95% interval \([0.06,0.95]\), i.e. \(\nu\in[1.1,17.2]\); under a flat prior on \(\nu\) no such summary exists.

Data dependence (7.5d). None: \(\nu^{-2}\) involves no function of \(y\). The device forced on Box and Cox by the units problem is unnecessary here precisely because of the invariance noted above, so the prior can be stated, and criticized, before the data arrive.

Truncation (7.5e). None: \(t_\nu(\mu,\sigma^2)\) has support \(\mathbb{R}\) for every \(\nu\ge1\), so unlike (7.19) the model never places probability on impossible values, the likelihood needs no normalizing correction, and every posterior predictive draw is a legitimate observation. If the data are all-positive, that is a reason to model \(\log y\) with the \(t\), not a reason to truncate.

To admit the normal itself, extend to \(\nu^{-1}\in[0,1]\), which this prior does continuously: in \(\nu^{-1}\) the null model is a boundary point at finite distance rather than at \(\nu=\infty\).

All six are false; the three concepts live in three different places. Randomization is a property of \(p(I\mid x,y,\phi)\), ignorability is a property of the pair \(\left(p(y\mid x,\omega),\,p(I\mid x,y,\phi)\right)\) through (8.2) versus (8.3), and exchangeability is a property of \(p(y\mid x)\) alone. Nothing about one forces anything about another.

(a) False. Randomization makes \(p(I)\) invariant under permutation of the unit labels; it says nothing about \(p(y)\). In the Latin square of Table 8.4 the treatments are randomized, yet the 25 observed yields are not exchangeable: the plot coordinates \(x\) distinguish the units, and the whole point of the analysis on p. 216 is to model \(y\) given \(x\). Worse, even with no covariates the observed values \(y_{\text{obs}\,i}\) are draws under different treatments, so they are exchangeable only after conditioning on the treatment received.

Kernel of truth: under complete randomization with no recorded covariates, the units are exchangeable given the treatment indicator, and randomization is what licenses treating the treatment-group labels as carrying no information beyond the treatment itself.

(b) False. Exchangeability is a judgement about \(y\), not a consequence of data collection. The 71 rat tumor experiments of Section 5.1 and the eight coaching studies of Section 5.5 are modelled exchangeably at the study level, and no randomization whatsoever selected those studies; Simon Newcomb’s speed-of-light measurements (Section 3.2) are modelled as exchangeable with no randomization at all.

Kernel of truth: randomization is a sufficient device for exchangeability. When the units genuinely carry distinguishing information and we do not wish to model it, randomizing over the labels is the one construction that makes the symmetric model defensible (p. 218).

(c) False. Ignorability requires missing at random and distinct parameters, and randomization delivers only the first. The book’s own counterexample is on p. 220: an experimenter who uses complete randomization when he believes the treatment effects are large and randomized blocks when he suspects they are small. The choice of design is then a function of his prior for \(\omega\), so \(p(\phi\mid x,\omega)\neq p(\phi\mid x)\), the parameters are not distinct, and the design is not ignorable even though every assignment was randomized. Exercise 8.3 is the same phenomenon in the milk-production experiment.

Kernel of truth: randomization guarantees \(p(I\mid x,y,\phi)=p(I\mid x,\phi)\), which is free of \(y_{\text{mis}}\); missing at random is the hard condition, and distinct parameters holds in almost every practical randomized experiment.

(d) False. Any design whose inclusion pattern depends only on fully observed covariates is ignorable. Auditing every tax return with declared income above $1 million (p. 202) is deterministic and ignorable, because declared income is a fully observed \(x\). The systematic field layout ABABABBABABA on twelve plots (p. 219) is deterministic and ignorable given plot location. The milk-production design of Section 8.4 is neither randomized in any documented way nor known, and it is still ignorable given the three pre-treatment covariates.

Kernel of truth: the italicized claim on p. 218 is nearly this statement with the covariates removed. If there are no fully observed \(x\) to condition on, then apart from degenerate designs the only \(p(I_1,\dots,I_n\mid\phi)\) that is both ignorable and invariant to permutation of the indexes is a randomization. Randomization also buys ignorability without our having to know, record, or model which covariates drove the assignment.

(e) False. Stratified sampling is the standing counterexample. The CBS survey of Table 8.2 is ignorable given the sixteen stratum indicators, yet the 1447 observed responses are plainly not exchangeable: the response distributions differ by stratum, and the analysis of Section 8.3 models sixteen distinct parameter vectors \((\omega_{1j},\omega_{2j},\omega_{3j})\). Likewise the audited tax returns are systematically the largest ones in the population.

Kernel of truth: a strongly ignorable design is implicitly randomized over every variable not in \(x\) (p. 220), so an ignorable design given \(x\) does deliver exchangeability of the units conditional on \(x\): units \(i\) and \(j\) with \(x_i=x_j\) have \(p(I_i\mid x)=p(I_j\mid x)\) and may be modelled exchangeably. Exchangeability returns within cells, not across them.

(f) False. Censoring (Section 8.7) is the clean case: the underlying complete data \(y_1,\dots,y_N\) may be modelled as independent and identically distributed given \(\omega\), hence fully exchangeable, while \(p(I\mid y,\phi)\) depends on \(y_{\text{mis}}\) and the design is nonignorable. The exchangeable model for \(y\) is not merely appropriate there, it is the model one actually fits; what the nonignorability forbids is dropping the factor \(p(I\mid y,\phi)\) from the likelihood.

Kernel of truth: ignorability is exactly what transfers an exchangeable model for the complete data \(y\) to a usable exchangeable model for the observed data \(y_{\text{obs}}\). Under a nonignorable design the observed values are a selected, hence distorted, subset, and applying \(p(y\mid\omega)\) directly to \(y_{\text{obs}}\) as in (8.3) is wrong.

Take the eight-schools SAT coaching analysis of Section 5.5. Two design issues are relevant, at the two levels of the hierarchy, and only the second changes the analysis.

Level one, within schools: the students were randomly assigned to coaching and control within each school, so the assignment is strongly ignorable and known, with \(p(I\mid x,y,\phi)=p(I\mid x)\), \(x\) being the school indicator. This is exactly the completely randomized experiment of Section 8.4 carried out eight times in parallel, so the school-level estimates \(y_j\) and their standard errors \(\sigma_j\) may be taken at face value and the posterior computed from \(p(\omega\mid x,y_{\text{obs}})\propto p(\omega)\,p(y_{\text{obs}}\mid x,\omega)\). Chapter 8 also sharpens what \(\theta_j\) means: it is a superpopulation parameter, whereas the finite-population causal effect \(\bar y_j^{A}-\bar y_j^{B}\) for the students actually in school \(j\) is the estimand of (8.17), and for large \(n_j\) the two coincide with \(\theta_j\) to order \(n_j^{-1/2}\).

Level two, selection of the schools: here the design is not ignorable and Section 5.5 tacitly assumes it is. The eight schools are the ones that ran a coaching program, evaluated it, and reported the result; nothing randomized their entry into the dataset. If the probability of a study reaching the analyst rises with the estimated effect, then \(p(I_j\mid\theta_j,y_j,\phi)\) depends on \(y_j\), which is the censoring structure of Section 8.7 and is nonignorable and unknown. This is the one place where Chapter 8 says the Section 5.5 posterior is wrong rather than merely incomplete: an exchangeable \(\theta_j\sim\mathrm{N}(\mu,\tau^2)\) fitted to a selected upper tail overstates \(\mu\), and, because selection truncates from below, understates \(\tau\).

Three concrete changes.

(i) Use the available covariates rather than the minimally adequate summary (p. 217). The sample sizes \(n_j\), equivalently the \(\sigma_j\), are fully observed and were chosen by the investigators, so the principle that everything used in the design belongs in the model applies. Replace the second level by

\begin{equation*} \theta_j \mid \alpha,\beta,\tau \sim \mathrm{N}\!\left(\alpha+\beta\,(\log n_j-\overline{\log n}),\ \tau^2\right), \end{equation*}

which is the exact analogue of including \(M_k\) in the school-level model (8.10) for the Melbourne cluster sample, and check whether \(\beta\) is distinguishable from zero. The model of Section 5.5 is the special case \(\beta=0\).

(ii) Run the selection mechanism as a sensitivity analysis rather than assuming it away: attach a weight \(\Pr(I_j=1\mid y_j,\phi)=\mathrm{logit}^{-1}(\phi_0+\phi_1 y_j/\sigma_j)\) and trace the posterior for \(\mu\) as \(\phi_1\) ranges over plausible values. With eight studies, \(\tau\) already poorly determined, and a posterior mean effect of about 8 points, even mild selection moves the estimate appreciably, and reporting that range is more honest than reporting the ignorable answer alone.

(iii) Widen the posterior predictive checks of Section 6.5: because the within-school assignment was randomized, replications may redraw \(I^{\text{rep}}\) as well as \(y^{\text{rep}}\) (p. 219).

(a) Because the re-randomization rule is the experimenter’s judgement, and that judgement was formed from his beliefs about \(\omega\). He balanced on lactation number, age, and initial weight, and on nothing else, precisely because he believed those three variables predict milk fat; had he believed the coefficients of \(x\) in \(p(y\mid x,\omega)\) were zero, one randomization would have been enough and no balance criterion would have been applied. The unstated rule, that is, the implicit weighting of the three covariates in the balance criterion and the number of randomizations tried before stopping, therefore carries information about the magnitudes of those coefficients, so

\begin{equation*} p(\phi\mid x,\omega)\ \neq\ p(\phi\mid x), \end{equation*}

which is the negation of the distinct-parameters condition on p. 202. The p. 220 mechanism is available too: an experimenter searches harder for balance when he expects the treatment effect to be small relative to covariate-induced variation, so the effort \(\phi\) encodes his prior for the treatment effect itself. Note that missing at random is untouched: the assignment used only the fully observed \(x\), never the unrecorded outcomes, so the failure of ignorability here is entirely a failure of distinctness.

(b) The effect is a mild extra prior factor on \(\omega\), and it is negligible here. Writing \(\phi^{\text{obs}}\) for what the design reveals, the nonignorable posterior is proportional to

\begin{equation*} \begin{aligned} p(\omega\mid x,y_{\text{obs}},I)\ \propto\ &\ p(\omega\mid x)\ p(y_{\text{obs}}\mid x,\omega)\\ &\times \int p(\phi\mid x,\omega)\,p(I\mid x,y_{\text{obs}},\phi)\,d\phi, \end{aligned} \end{equation*}

and the ignorable analysis simply drops the last line. That line is a smooth, diffuse function of \(\omega\): it says roughly that the coefficients on lactation number, age, and weight are not all negligible, and, under the p. 220 variant, that the treatment effect was not expected to be enormous.

Direction. The balance criterion is symmetric in the sign of the treatment effect, so nothing shifts the location of the posterior for the treatment contrasts in a determinate direction. What the factor does favour is larger \(|\beta_x|\), which would slightly reduce the posterior for the residual variance \(\sigma^2\) and hence sharpen, not shift, the treatment-effect posterior. Under the reading in which he searched harder for balance because he expected a small effect, the factor is a weak shrinkage of the treatment contrast toward zero.

Magnitude. Negligible, for two independent reasons. First, the extra factor is vague compared with a likelihood based on fifty cows; it is at most worth a few pseudo-observations, whereas the data determine the regression coefficients to within a standard error that the factor cannot meaningfully compete with. Second, and more decisively, the re-randomization achieved balance, which is the situation of Figure 8.2a: with the covariate distributions matched across the four diets, the estimated treatment effect is insensitive to the form of \(p(y\mid x,\omega)\), and a fortiori to a perturbation of the prior on its coefficients. This is why the text can say (p. 218) that reasonable violations of distinctness should not have large effects on the inferences for this problem. The conclusion fails only in the case the same page warns about: if the assignment rule had depended on beliefs about the treatment efficacy strongly enough to make \(\phi\) nearly determine \(\omega\), the extra factor would no longer be vague and would have to be modelled.

(a) Index the potential outcomes of plot \(i\) by the treatments on the triple \((i-1,i,i+1)\), which restores the stability assumption of p. 200 by enlarging \(y\) until \(I\) no longer changes it. Let \(T=(T_1,\dots,T_{20})\in\{A,B\}^{20}\) be the assignment, with \(\sum_i \mathbb{1}(T_i=B)=10\), and set

\begin{equation*} y \;=\; \big(y_{i,(a,b,c)}\big),\qquad i=1,\dots,20,\ (a,b,c)\in\{A,B\}^3, \end{equation*}

where \(y_{i,(a,b,c)}\) is the yield of plot \(i\) when plot \(i-1\) gets \(a\), plot \(i\) gets \(b\), and plot \(i+1\) gets \(c\). This is a \(20\times 8\) array, larger than \(20\times 4\) as required. The edge plots have no outside neighbour, so row 1 depends only on \((b,c)\) and row 20 only on \((a,b)\); those two rows carry 4 distinct values each, and \(y\) has \(18\cdot 8+2\cdot 4=152\) distinct entries.

The indicator array is \(I=(I_{i,(a,b,c)})\) of the same \(20\times 8\) shape,

\begin{equation*} I_{i,(a,b,c)} \;=\; \mathbb{1}\big\{(T_{i-1},T_i,T_{i+1})=(a,b,c)\big\}, \end{equation*}

with exactly one \(1\) per row, so \(y_{\text{obs}}\) consists of 20 numbers and \(y_{\text{mis}}\) of the remaining 132. Note \(I\) and \(T\) determine each other: the middle index of the observed cell in row \(i\) is \(T_i\). The inclusion model is

\begin{equation*} p(I\mid y,\phi)\;=\;p(I)\;=\;\binom{20}{10}^{-1}=\frac{1}{184756} \end{equation*}

on the \(184756\) arrays generated by a balanced \(T\), and \(0\) elsewhere.

(b) Yes, and in the strongest sense: \(p(I\mid y,\phi)=p(I)\) is exactly (8.4), so the design is known and strongly ignorable, with no covariates needed and constant propensity score \(\tfrac12\). What the interference cost us is not ignorability but the size of \(y_{\text{mis}}\): only 20 of 152 potential values are seen, against 20 of 40 in the no-interference \(20\times 2\) layout. Had we kept the naive structure, the difficulty would not have been a nonignorable design; the notation itself would have been incoherent, since \(y\) would depend on \(I\).

(c) With interference, a treatment label alone no longer names a state of the field, so the estimand must compare two whole assignment vectors, and the natural pair is the all-\(B\) field against the all-\(A\) field:

\begin{equation*} \bar y_B-\bar y_A \;=\; \frac{1}{20}\sum_{i=1}^{20}\Big(y_{i,(B,B,B)}-y_{i,(A,A,A)}\Big), \end{equation*}

reading \(y_{1,(A,A,A)}\) as \(y_{1,(\cdot,A,A)}\) and \(y_{20,(A,A,A)}\) as \(y_{20,(A,A,\cdot)}\). Of the 40 quantities in this sum, only those few plots whose realized triple happened to be \(AAA\) or \(BBB\) are observed; the rest are imputed from the model, which is why (d) matters more here than in a stability-satisfying experiment.

(d) Write \(z_a=\mathbb{1}(a=B)\) and model an own-plot effect \(\beta\) and a per-neighbour leaching effect \(\gamma\) additively on top of a plot fertility term:

\begin{equation*} y_{i,(a,b,c)} \;=\; \mu+\delta_i+\beta z_b+\gamma\,(z_a+z_c)+\epsilon_{i,(a,b,c)}, \end{equation*}

with the edge rows omitting the absent neighbour’s \(\gamma z\) term. For the spatial fertility of a linear array take a first-order autoregression,

\begin{equation*} \delta_1\sim\mathrm{N}\!\left(0,\tfrac{\tau^2}{1-\rho^2}\right),\qquad \delta_i=\rho\,\delta_{i-1}+\eta_i,\quad \eta_i\sim\mathrm{N}(0,\tau^2), \end{equation*}

and let the 8-vector \(\epsilon_i=(\epsilon_{i,(a,b,c)})\) be multivariate normal with mean \(0\) and exchangeable covariance

\begin{equation*} \mathrm{Cov}(\epsilon_{i,u},\epsilon_{i,v})=\sigma^2\big(\lambda+(1-\lambda)\mathbb{1}_{u=v}\big), \end{equation*}

independent across \(i\). Complete with \(p(\mu,\beta,\gamma,\rho,\log\sigma,\log\tau)\propto 1\) on \(|\rho|<1\) and \(\lambda\sim\mathrm{U}(0,1)\).

Two features of this model earn it its place. The design supplies plots with zero, one, and two treated neighbours, so \(\beta\) and \(\gamma\) are separately identified from the 20 observations. And the estimand of (c) reduces under the model to

\begin{equation*} \bar y_B-\bar y_A \;=\; \beta+1.9\,\gamma+\frac{1}{20}\sum_{i=1}^{20}\big(\epsilon_{i,(B,B,B)}-\epsilon_{i,(A,A,A)}\big), \end{equation*}

the coefficient \(1.9=(18\cdot 2+2\cdot 1)/20\) coming from the two edge plots, with \(\mu\) and every \(\delta_i\) cancelling. The within-row correlation \(\lambda\) never enters the likelihood, exactly as \(\omega_{AB}\) does not on p. 215; it must be given a prior, it affects only the finite-population estimand through the residual term above, and its influence vanishes as the number of plots grows.

(a) The units are the \(N=20\) runs, not the five blocks. Label a run by the pair \(i=(j,r)\) with block \(j=1,\dots,5\) and \(r=1,\dots,4\), and let \(y_{ik}\) be the yield run \(i\) would give under treatment \(k\in\{A,B,C,D\}\); then \(y\) is a \(20\times 4\) array of potential outcomes and \(I\) is a \(20\times 4\) array of zeros and ones with exactly one \(1\) in each row. The fully observed covariate is the block label, \(x=(j(i))_{i=1}^{20}\), equivalently a \(20\times 5\) matrix of stratum indicators. Since Table 8.6 reports yields by treatment rather than by run, we may without loss number the runs within each block so that run \((j,r)\) received treatment \(r\), making \(I\) the identity pattern within every block:

and so on through unit \((5,4)\). There are \(20\) observed measurements and \(20\cdot 4-20=60\) unobserved ones, so three quarters of the potential data are missing, the usual state of affairs in an experiment (Table 8.1).

(b) Within each block the four treatments are allotted to the four runs by a uniform random permutation, independently across blocks:

\begin{equation*} p(I\mid x,y,\phi)=p(I\mid x)=\left(\frac{1}{4!}\right)^{5}=\frac{1}{7962624} \end{equation*}

on the set of \(I\) that place exactly one \(1\) in each row and exactly one of each treatment in each block, and \(0\) elsewhere.

Ignorable: yes. The expression is free of \(y\) outright, so missing at random holds, and there is no parameter \(\phi\) at all, so distinct parameters holds vacuously. Known: yes, the display above is the complete specification. Strongly ignorable: yes, \(p(I\mid x,y,\phi)=p(I\mid x)\) with \(x\) fully observed.

Propensity scores: no, they are not an adequate summary. Here

\begin{equation*} \pi_{ik}=\Pr(I_{ik}=1\mid x)=\tfrac14\quad\text{for every }i\text{ and }k, \end{equation*}

a constant, so conditioning on \(\pi\) is conditioning on nothing and discards the block labels entirely. The design is not strongly ignorable given \(\pi\) alone: \(p(I\mid x)\) above is supported on the block-balanced arrays and is not a function of \(\pi\). These same propensity scores would arise from complete randomization and from independent assignment with probability \(\tfrac14\) per run, which is exactly the phenomenon noted on p. 204, and it is why \(\pi\) supports neither the precision of part (c) nor a correct posterior predictive replication.

(c) Condition on the block indicators and nothing less. On the yield scale,

\begin{equation*} y_{ik}\mid\omega \;\sim\; \mathrm{N}\!\left(\mu+\beta_{j(i)}+\tau_k,\ \sigma^2\right), \end{equation*}

independently across runs \(i\), with the block effects exchangeable,

\begin{equation*} \beta_j\mid\tau_\beta \;\sim\; \mathrm{N}(0,\tau_\beta^2),\qquad j=1,\dots,5, \end{equation*}

treatment effects constrained by \(\sum_{k}\tau_k=0\), and \(p(\mu,\tau,\log\sigma,\tau_\beta)\propto 1\). Because \(x\) enters \(p(y\mid x,\omega)\), part (b) makes the design ignorable and the posterior is simply \(p(\omega\mid x,y_{\text{obs}})\propto p(\omega)\,p(y_{\text{obs}}\mid x,\omega)\), with

\begin{equation*} p(y_{\text{obs}}\mid x,\omega)=\prod_{i=1}^{20}\prod_{k}\left[p(y_{ik}\mid x,\omega)\right]^{I_{ik}} . \end{equation*}

For finite-population estimands one further needs the joint law of the four potential outcomes in a row; take \((y_{iA},\dots,y_{iD})\) multivariate normal with the above means and exchangeable correlation \(\lambda\), which does not appear in the likelihood and must be assigned a prior. This is the model taken up again in Exercise 15.2.

(d) Averaging the mean over the block distribution \(\beta_j\sim\mathrm{N}(0,\tau_\beta^2)\),

\begin{equation*} \theta_k \;=\; \mathrm{E}\big(y_{ik}\mid\omega\big) \;=\; \mu+\tau_k,\qquad k=A,B,C,D, \end{equation*}

since \(\mathrm{E}(\beta_j)=0\); the pairwise comparisons are \(\theta_k-\theta_{k^{\prime}}=\tau_k-\tau_{k^{\prime}}\). Had the question asked instead for the average over the five block conditions actually run, the estimand would be \(\mu+\bar\beta+\tau_k\) with \(\bar\beta=\tfrac15\sum_{j=1}^5\beta_j\).

(a) Because the design enters the posterior only through a factor that is the same under both mechanisms. Under complete randomization with five runs per treatment,

\begin{equation*} p(I\mid x,y,\phi)=p(I)=\binom{20}{5,5,5,5}^{-1}=\frac{1}{11732745024}, \end{equation*}

which, like the randomized-blocks \(p(I\mid x)\) of Exercise 8.5(b), is free of \(y_{\text{mis}}\) and carries no parameter, so both designs are ignorable and in (8.2) the factor integrates out, leaving (8.3):

\begin{equation*} p(\omega\mid x,y_{\text{obs}},I)=p(\omega\mid x,y_{\text{obs}})\propto p(\omega\mid x)\,p(y_{\text{obs}}\mid x,\omega). \end{equation*}

The realized \((x,y_{\text{obs}},I)\) are by hypothesis identical, and the prior and complete-data model are unchanged, so the posterior is identical number for number. The block labels stay in the model regardless: they are a fully observed covariate, and the principle of p. 217 is to use all available information and not merely the minimally adequate summary, which under complete randomization would not have included \(x\) at all. This is precisely the Latin-square argument of p. 216.

(b) The check changes because the replication redraws \(I^{\text{rep}}\) from \(p(I\mid x,\phi)\), and that distribution is design-specific even though the posterior is not. Under randomized blocks, every \(I^{\text{rep}}\) is again a within-block permutation, so each replicated dataset has each treatment exactly once per block, each treatment mean is again an average of five observations one per block, and any test statistic measuring treatment-by-block balance is degenerate at its observed value. Under complete randomization, \(I^{\text{rep}}\) scatters five runs per treatment over all twenty, and balance is destroyed almost surely: the probability that a completely randomized replicate reproduces the balanced pattern is

\begin{equation*} \frac{(4!)^5}{20!/(5!)^4}=\frac{7962624}{11732745024}=6.8\times 10^{-4}, \end{equation*}

and the probability that some treatment is missing from some block entirely is \(0.999\). Three consequences. Test statistics must be defined for unbalanced tables, since \(T(y^{\text{rep}},I^{\text{rep}})\) will almost never be computable on a balanced layout. The reference distribution for treatment contrasts is wider, because the variance factor for \(\tau_k-\tau_{k^{\prime}}\) in the additive model averages \(0.551\,\sigma^2\) over completely randomized assignments against a fixed \(0.400\,\sigma^2\) under blocking, so the check is less stringent. And a new check becomes available: taking \(T\) to be a measure of treatment-by-block imbalance, the observed value sits below essentially every replicate, at the \(0.07\)th percentile, which would correctly signal that the data did not arise from complete randomization.

(c) Because it buys precision and robustness at no cost. Precision: balance makes the treatment contrasts orthogonal to the block effects, giving the smallest achievable variance factor \(2/5=0.400\) for \(\tau_k-\tau_{k^{\prime}}\); over completely randomized assignments the same factor has median \(0.507\), mean \(0.551\), and exceeds \(1.39\) in the worst one percent (Monte Carlo over \(10^6\) assignments; the reported digits are stable). With the residual mean square from Table 8.6 equal to \(18.8\) on 12 degrees of freedom, so \(\hat\sigma=4.34\), this is a posterior standard deviation of \(2.74\) for a treatment contrast under blocking against a typical \(3.09\) and an occasional \(5.11\) under complete randomization. The gain is real here because the blocks are genuinely different: the block mean square is \(66.0\) against a residual mean square of \(18.8\), and a model that dropped the blocks would raise the residual standard deviation from \(4.34\) to \(5.53\).

Robustness: balance is what makes the answer insensitive to the form of the block model, exactly as in Figure 8.2a. Under an unbalanced draw the treatment means must be corrected for block effects through the assumed additive structure, and since with probability \(0.999\) some treatment is absent from some block, at least one cell of the correction would rest entirely on additivity rather than on data.

(d) Block the twenty runs into ten blocks of two, say consecutive pairs of runs sharing a fermenter charge. Each block now holds two runs, hence at most two of the four treatments, so a block supplies a direct within-block comparison for at most one of the six treatment pairs, and the remaining contrasts are recovered only by chaining blocks through the additive model. Three difficulties follow, each an instance of p. 220, item 2. The parameter count rises: one intercept, nine block contrasts, and three treatment contrasts, leaving \(20-13=7\) residual degrees of freedom instead of 12, so \(\sigma\) is poorly determined and the posterior for the treatment contrasts becomes sensitive to the prior on \(\tau_\beta\), which is now doing the work of holding ten weakly identified block effects together. Additivity becomes uncheckable: with only 7 residual degrees of freedom and at most one treatment pair per block, no block-by-treatment interaction can be estimated or even diagnosed, yet every between-block comparison assumes it away. And identification can fail outright: if the realized pairing happens to put only \(\{A,B\}\) and only \(\{C,D\}\) in blocks, the design is disconnected and \(\tau_A-\tau_C\) has a flat likelihood, the extreme case in which the observed-data likelihood provides no information to separate treatment from block. In the limit of blocks of size one, every treatment effect is confounded with its block effect and the experiment says nothing at all.

(a) The answer is

\begin{equation*} \bar y\mid y_{\text{obs}} \;\sim\; t_{n-1}\!\left(\bar y_{\text{obs}},\ \left(\tfrac1n-\tfrac1N\right)s_{\text{obs}}^2\right), \end{equation*}

the result quoted on p. 206. Take \(y_i\mid\mu,\sigma^2\sim\mathrm{N}(\mu,\sigma^2)\) independently for \(i=1,\dots,N\) with \(p(\mu,\sigma^2)\propto\sigma^{-2}\). Simple random sampling satisfies \(p(I\mid y,\phi)=p(I)\), so the design is ignorable and \(p(\mu,\sigma^2\mid y_{\text{obs}},I)=p(\mu,\sigma^2\mid y_{\text{obs}})\), which by Section 3.2 is

\begin{equation*} \sigma^2\mid y_{\text{obs}}\sim\text{Inv-}\chi^2\!\left(n-1,\,s^2_{\text{obs}}\right),\qquad \mu\mid\sigma^2,y_{\text{obs}}\sim\mathrm{N}\!\left(\bar y_{\text{obs}},\tfrac{\sigma^2}{n}\right). \end{equation*}

Conditional on \((\mu,\sigma^2)\) the \(N-n\) unsampled values are independent of the sampled ones, so exactly

\begin{equation*} \bar y_{\text{mis}}\mid\mu,\sigma^2,y_{\text{obs}}\;\sim\;\mathrm{N}\!\left(\mu,\ \tfrac{\sigma^2}{N-n}\right), \end{equation*}

and averaging this normal over the normal posterior for \(\mu\) adds the variances:

\begin{equation*} \bar y_{\text{mis}}\mid\sigma^2,y_{\text{obs}}\;\sim\;\mathrm{N}\!\left(\bar y_{\text{obs}},\ \sigma^2\!\left(\tfrac1n+\tfrac1{N-n}\right)\right). \end{equation*}

Now \(\bar y=\tfrac nN\bar y_{\text{obs}}+\tfrac{N-n}{N}\bar y_{\text{mis}}\) by (8.5) is an affine function of \(\bar y_{\text{mis}}\), hence normal given \(\sigma^2\), with mean \(\tfrac nN\bar y_{\text{obs}}+\tfrac{N-n}{N}\bar y_{\text{obs}}=\bar y_{\text{obs}}\) and variance scaled by \(\left(\tfrac{N-n}{N}\right)^2\):

\begin{equation*} \begin{aligned} \left(\frac{N-n}{N}\right)^{2}\left(\frac1n+\frac1{N-n}\right) &=\frac{(N-n)^2}{N^2}\cdot\frac{N}{n(N-n)}\\ &=\frac{N-n}{Nn}\;=\;\frac1n-\frac1N, \end{aligned} \end{equation*}

so that

\begin{equation*} \bar y\mid\sigma^2,y_{\text{obs}}\;\sim\;\mathrm{N}\!\left(\bar y_{\text{obs}},\ \left(\tfrac1n-\tfrac1N\right)\sigma^2\right). \end{equation*}

Finally mix over \(\sigma^2\). A normal with variance proportional to \(\sigma^2\), mixed over \(\sigma^2\sim\text{Inv-}\chi^2(n-1,s^2_{\text{obs}})\), is Student-\(t\) with \(n-1\) degrees of freedom and the same scale factor applied to \(s^2_{\text{obs}}\) (Appendix A; the integral is the one performed in Section 3.2 to obtain \(\mu\mid y\sim t_{n-1}(\bar y,s^2/n)\), the special case \(N\to\infty\) here). This gives the stated result.

(b) The asymptotic form does not need the normal model; it needs only two central limit theorems. Write \(\mu=\mu(\omega)=\mathrm{E}(y_i\mid\omega)\) and \(\sigma^2=\sigma^2(\omega)=\mathrm{var}(y_i\mid\omega)\) for a general exchangeable model \(p(y\mid\omega)=\int\prod_i p(y_i\mid\omega)p(\omega)d\omega\), and let \(n\to\infty\) and \(N-n\to\infty\) with \(N/n\) fixed.

Averaging \(N-n\) conditionally independent draws,

\begin{equation*} p(\bar y_{\text{mis}}\mid\omega)\;\approx\;\mathrm{N}\!\left(\bar y_{\text{mis}}\,\Big|\,\mu,\ \tfrac{\sigma^2}{N-n}\right). \end{equation*}

By the large-sample normality of Section 4.2 applied to the ignorable posterior \(p(\omega\mid y_{\text{obs}})\),

\begin{equation*} \mathrm{E}(\mu\mid y_{\text{obs}})\approx\bar y_{\text{obs}},\quad \mathrm{var}(\mu\mid y_{\text{obs}})\approx\tfrac{1}{n}s^2_{\text{obs}},\quad \mathrm{E}(\sigma^2\mid y_{\text{obs}})\approx s^2_{\text{obs}}, \end{equation*}

with \(\mathrm{var}(\sigma^2\mid y_{\text{obs}})=O(n^{-1})\), so the uncertainty in \(\sigma^2\) contributes only at lower order. Hence

\begin{equation*} p(\bar y_{\text{mis}}\mid y_{\text{obs}})\approx\int p(\bar y_{\text{mis}}\mid\mu,\sigma)\,p(\mu,\sigma\mid y_{\text{obs}})\,d\mu\,d\sigma \end{equation*}

is a normal mixture of normals with approximately normal mixing distribution, hence itself approximately normal. Its mean and variance follow from the conditional variance identity:

\begin{equation*} \begin{aligned} \mathrm{E}(\bar y_{\text{mis}}\mid y_{\text{obs}})&=\mathrm{E}(\mu\mid y_{\text{obs}})\;\approx\;\bar y_{\text{obs}},\\ \mathrm{var}(\bar y_{\text{mis}}\mid y_{\text{obs}})&=\mathrm{var}(\mu\mid y_{\text{obs}})+\mathrm{E}\!\left(\tfrac{\sigma^2}{N-n}\,\Big|\,y_{\text{obs}}\right)\\ &\approx\ \frac{s^2_{\text{obs}}}{n}+\frac{s^2_{\text{obs}}}{N-n}\;=\;\frac{N}{n(N-n)}\,s^2_{\text{obs}} . \end{aligned} \end{equation*}

Substituting into (8.5), \(\bar y\) is affine in \(\bar y_{\text{mis}}\) and therefore approximately normal, with mean \(\bar y_{\text{obs}}\) and variance

\begin{equation*} \left(\frac{N-n}{N}\right)^{2}\cdot\frac{N}{n(N-n)}\,s^2_{\text{obs}} =\frac{N-n}{Nn}\,s^2_{\text{obs}} =\left(\frac1n-\frac1N\right)s^2_{\text{obs}}, \end{equation*}

which is (8.6).

Method (2): under the normal model of part (a) the derivation is immediate, since \(t_{n-1}\) converges to the normal as \(n\to\infty\) with the location and scale of (a) unchanged.

(a) The finite-population contrast collapses to the superpopulation contrast \(\mu_A-\mu_B\), whose posterior distribution is normal with the stated moments.

Write \(m = n/2\) for the number of units per arm. Given \(\theta\) and \(y_{\mathrm{obs}}\), the \(N-m\) unobserved values of \(y_i^A\) are independent draws from their superpopulation distribution, so

\begin{equation*} \bar y^A \,|\, \theta, y_{\mathrm{obs}} \sim \mathrm{N}\!\left(\tfrac{m}{N}\bar y^A_{\mathrm{obs}}

• \tfrac{N-m}{N}\mu_A,\ \tfrac{N-m}{N^{2}}\sigma_A^{2}\right), \end{equation*}

and likewise for \(B\). Measured against the \(O(\sigma_A m^{-1/2})\) posterior uncertainty in \(\mu_A\), the displacement \(\tfrac{m}{N}(\bar y^A_{\mathrm{obs}}-\mu_A)\) is smaller by a factor \(m/N\) and the added standard deviation \(\sigma_A\sqrt{N-m}/N\) by a factor \(\sqrt{m/N}\); both vanish as \(N/n\to\infty\), leaving \(\bar y^A-\bar y^B = \mu_A-\mu_B\) to that order.

The treatment assignment is ignorable and the likelihood factors into a part involving \(\theta_A\) and a part involving \(\theta_B\) (page 215), so the noninformative-prior normal-theory result of Section 3.2 applies separately in each arm: with \(p(\mu_A,\mu_B,\log\sigma_A,\log\sigma_B)\propto 1\),

\begin{equation*} \begin{aligned} \mu_A \,|\, y_{\mathrm{obs}} &\sim t_{m-1}\!\left(\bar y^A_{\mathrm{obs}},\, s^{2A}_{\mathrm{obs}}/m\right),\\ \mu_B \,|\, y_{\mathrm{obs}} &\sim t_{m-1}\!\left(\bar y^B_{\mathrm{obs}},\, s^{2B}_{\mathrm{obs}}/m\right), \end{aligned} \end{equation*}

independently. For large \(m\) the \(t\) densities approach normals (Section 4.4), so

\begin{equation*} (\mu_A-\mu_B)\,|\,y_{\mathrm{obs}} \approx \mathrm{N}\!\left(\bar y^A_{\mathrm{obs}}-\bar y^B_{\mathrm{obs}},\; \frac{s^{2A}_{\mathrm{obs}}+s^{2B}_{\mathrm{obs}}}{m}\right), \end{equation*}

and \(1/m = 2/n\) gives (8.17).

(b) Only the missing half of each sampled pair and the \(N-n\) unsampled pairs need to be imputed. Let \(\mathcal{A}\) and \(\mathcal{B}\) be the two treatment groups and \(d_i = y_i^A - y_i^B\). Under the bivariate normal model the conditional imputations are, for \(i\in\mathcal{A}\),

\begin{equation*} y_i^B \,|\, y_i^A,\theta \sim \mathrm{N}\!\left(\mu_B + \rho\tfrac{\sigma_B}{\sigma_A}(y_i^A-\mu_A),\; \sigma_B^2(1-\rho^2)\right), \end{equation*}

and symmetrically for \(i\in\mathcal{B}\). Therefore

\begin{equation*} \begin{aligned} \mathrm{E}[d_i\,|\,\theta,y_{\mathrm{obs}}] &= \mu_A-\mu_B+\left(1-\rho\tfrac{\sigma_B}{\sigma_A}\right)(y_i^A-\mu_A), \quad i\in\mathcal{A},\\ \mathrm{E}[d_i\,|\,\theta,y_{\mathrm{obs}}] &= \mu_A-\mu_B+\left(\rho\tfrac{\sigma_A}{\sigma_B}-1\right)(y_i^B-\mu_B), \quad i\in\mathcal{B}, \end{aligned} \end{equation*}

with conditional variances \(\sigma_B^2(1-\rho^2)\) and \(\sigma_A^2(1-\rho^2)\) respectively, while for the \(N-n\) unsampled units \(d_i\) has mean \(\mu_A-\mu_B\) and variance \(\sigma_A^2+\sigma_B^2-2\rho\sigma_A\sigma_B\). Averaging over \(i\),

\begin{equation*} \begin{aligned} \mathrm{E}\!\left[\bar y^A - \bar y^B \,\middle|\, \theta, y_{\mathrm{obs}}\right] &= \mu_A-\mu_B \\ &\quad + \tfrac{m}{N}\left(1-\rho\tfrac{\sigma_B}{\sigma_A}\right) (\bar y^A_{\mathrm{obs}}-\mu_A)\\ &\quad + \tfrac{m}{N}\left(\rho\tfrac{\sigma_A}{\sigma_B}-1\right) (\bar y^B_{\mathrm{obs}}-\mu_B), \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} \mathrm{Var}\!\left[\bar y^A - \bar y^B \,\middle|\, \theta, y_{\mathrm{obs}}\right] = \frac{1}{N^2}\Big[&m(1-\rho^2)(\sigma_A^2+\sigma_B^2)\\ &+ (N-n)\big(\sigma_A^2+\sigma_B^2-2\rho\sigma_A\sigma_B\big)\Big] \;=:\; V_1 . \end{aligned} \end{equation*}

Now average over \(\theta\). Taking \(\sigma_A,\sigma_B\) as estimated by \(s^A_{\mathrm{obs}},s^B_{\mathrm{obs}}\) and writing \(\mu_A = \bar y^A_{\mathrm{obs}} + e_A\), \(\mu_B = \bar y^B_{\mathrm{obs}} + e_B\) with \(e_A,e_B\) independent \(\mathrm{N}(0,\sigma_A^2/m)\) and \(\mathrm{N}(0,\sigma_B^2/m)\), the conditional mean above equals \((\bar y^A_{\mathrm{obs}}-\bar y^B_{\mathrm{obs}}) + a\,e_A - b\,e_B\) with

\begin{equation*} a = 1-\frac{m}{N}\left(1-\rho\frac{\sigma_B}{\sigma_A}\right), \qquad b = 1-\frac{m}{N}\left(1-\rho\frac{\sigma_A}{\sigma_B}\right). \end{equation*}

Hence the posterior distribution is normal with

\begin{equation*} \begin{aligned} \mathrm{E}\!\left[\bar y^A-\bar y^B\,\middle|\,y_{\mathrm{obs}}\right] &= \bar y^A_{\mathrm{obs}}-\bar y^B_{\mathrm{obs}},\\ \mathrm{Var}\!\left[\bar y^A-\bar y^B\,\middle|\,y_{\mathrm{obs}}\right] &= \frac{a^2\sigma_A^2+b^2\sigma_B^2}{m} + V_1 . \end{aligned} \end{equation*}

(Check: for \(\rho=1\), \(\sigma_A=\sigma_B=\sigma\) one gets \(a=b=1\), \(V_1=0\), and the variance is \(2\sigma^2/m = (2/n)(\sigma_A^2+\sigma_B^2)\), as it must be since then \(d_i \equiv \mu_A-\mu_B\) for every unit.)

(c) \(\rho\) enters only through \(a\), \(b\) and \(V_1\), all of whose \(\rho\)-dependent pieces carry a factor \(m/N\) or \(1/N\); and \(\rho\) is not identified by the data, because no unit ever has both \(y_i^A\) and \(y_i^B\) observed, so its posterior distribution equals its prior distribution. In practice this means that a finite-population causal inference in which the experiment is a substantial fraction of the population is sensitive to an assumption the data cannot check, and must be reported as a sensitivity analysis over \(\rho\in[-1,1]\) (the extreme \(\rho=1\), additive unit-level effects, gives the smallest posterior variance). In the limit of large \(N/n\) we have \(a,b\to 1\) and \(V_1\to 0\), and the estimand itself converges to the superpopulation contrast \(\mu_A-\mu_B\), a function of the two marginal distributions only; the joint distribution, and hence \(\rho\), drops out and (8.17) is recovered.

(a) Both designs are ignorable given the cluster indicators and the cluster sizes, so the analysis is entirely a modeling problem: put a parameter at every level of clustering, then average the posterior predictive distribution over the unobserved units.

Use the notation of Section 8.3: units \(i\) in clusters \(j = 1,\dots,K\), with \(N_j\) units in cluster \(j\), \(N = \sum_j N_j\), and the estimand \(\bar Y = \frac{1}{N}\sum_{j=1}^{K} N_j \bar y_{\cdot j}\).

(i) One-stage. A simple random sample of \(J\) clusters is drawn and every unit in each sampled cluster is measured. Then \(y_{\mathrm{mis}}\) consists of whole clusters: \(\bar y_{\cdot j}\) is known exactly for the \(J\) sampled clusters and completely unknown for the other \(K-J\). Writing \(\bar y_{\cdot j}\,|\,\theta_j\) and \(\theta_j \sim \mathrm{N}(\alpha,\tau^2)\) as in (8.10), the posterior draws of \((\alpha,\tau)\) from the \(J\) observed cluster means are pushed through \(\theta_j\,|\,\alpha,\tau\) for the unsampled \(j\), and \(\bar Y\) is reassembled with the known weights \(N_j\).

(ii) Two-stage. A sample of \(n_j\) of the \(N_j\) units is drawn within each sampled cluster, so there are now two kinds of missing data: unsampled units inside sampled clusters and entire unsampled clusters. This is exactly the two-level structure (8.9)–(8.10): \(\bar y_{\cdot j}\,|\,\theta_j \sim \mathrm{N}(\theta_j,\sigma_j^2)\) within clusters and \(\theta_j \sim \mathrm{N}(\alpha,\tau^2)\) across clusters. Within a sampled cluster the unsampled units are imputed from \(p(y_{ij}|\theta_j)\) with \(\theta_j\) drawn from its posterior distribution — partially pooled toward \(\alpha\); the unsampled clusters are imputed as in (i).

The hierarchical model is what makes both steps possible. In case (i) it is the only source of information about the \(K-J\) unsampled clusters: an exchangeable population distribution for \(\theta_j\) is what licenses extrapolation to them, and \(\tau\) — the model’s version of the design effect — is estimated from the observed spread of the cluster means. In case (ii) it additionally supplies the partial pooling that stabilizes the \(\theta_j\) when the \(n_j\) are small. Fitting each cluster separately (\(\tau = \infty\)) leaves unsampled clusters undefined; pooling completely (\(\tau = 0\)) is the i.i.d. model and understates the posterior uncertainty about \(\bar Y\).

(b) The measures of size enter as covariates in the model, not as weights in the estimate. Let \(M_j\) be the known measure of size for cluster \(j\), with sampling probability proportional to \(M_j\). This design is ignorable only conditional on \(M\): the sampled clusters are a biased (size-biased) selection, so \(p(\theta_j)\) and \(p(\theta_j\,|\,\text{sampled})\) differ unless \(\theta_j\) is modeled given \(M_j\). Following the principle that all information used in the design must be included in the analysis, model the cluster parameter conditionally — as in (8.10),

\begin{equation*} \theta_j \sim \mathrm{N}(\alpha + \beta M_j,\ \tau^2), \end{equation*}

or with a more flexible function of \(M_j\) if the data warrant it. The design is then ignorable for this model (Section 8.2), and the likelihood is just the product over sampled clusters.

The \(M_j\) then enter the finite-population step a second time: because \(M_j\) is known for the unsampled clusters as well, each unsampled \(\theta_j\) is drawn from \(\mathrm{N}(\alpha+\beta M_j,\tau^2)\) using that cluster’s own \(M_j\), and \(\bar Y\) is assembled with the known \(N_j\). When only a small fraction of clusters is sampled, this collapses to the approximation (8.11),

\begin{equation*} \bar Y \approx \frac{1}{N}\sum_{k=1}^{K} N_k(\alpha+\beta M_k), \end{equation*}

which depends on the hyperparameters and on the population distribution of \((M_k,N_k)\), but not on the individual \(\theta_k\). So where the classical probability-proportional-to-size estimator uses the \(M_j\) as inverse-probability weights \(1/\pi_j\) on the observations, the Bayesian analysis uses them as regression predictors, and the weighting happens implicitly when the predictive draws for the unsampled clusters are summed with weights \(N_j\).

(a) Exchangeable because the lost labels leave our information symmetric in the \(n\) sampled units; not i.i.d. because the units are still clustered, and clustering induces a positive correlation that no relabeling destroys.

Formally, let \(c = (c_1,\dots,c_n)\) be the (unknown) cluster memberships and \(\theta\) the cluster parameters. The complete model is \(p(y|c,\theta) = \prod_i p(y_i|\theta_{c_i})\), a product given \(c\). Having lost \(c\) we must average over it,

\begin{equation*} p(y_1,\dots,y_n) = \int \sum_{c} p( c)\, \prod_{i=1}^{n} p(y_i|\theta_{c_i})\; p(\theta)\,d\theta , \end{equation*}

and since the design makes \(p( c)\) invariant under permutations of the sample labels, this mixture is invariant under permutations of \((y_1,\dots,y_n)\): the model is exchangeable, and it is a mixture of i.i.d. models rather than an i.i.d. model, which is de Finetti’s representation (Section 5.2). That the mixture is nondegenerate shows up in the correlation: with a one-way variance-components model \(y_i = \mu + \alpha_{c_i} + \epsilon_i\), \(\mathrm{Var}(\alpha)=\tau^2\), \(\mathrm{Var}(\epsilon)=\sigma^2\), and intraclass correlation \(\rho_I = \tau^2/(\tau^2+\sigma^2)\),

\begin{equation*} \mathrm{Corr}(y_i,y_{i^{\prime}}) = \rho_I\,\Pr(c_i = c_{i^{\prime}}) = \rho_I\,\frac{b-1}{n-1} \;>\; 0 \qquad (i \ne i^{\prime}), \end{equation*}

the same for every pair, which is what exchangeability demands and what independence forbids. Ignoring this and fitting an i.i.d. model understates the posterior variance of the population mean.

(b) Only when \((b-1)\rho_I \approx 0\). With \(n = ab\) and the model of part (a),

\begin{equation*} \mathrm{Var}(\bar y) = \frac{\tau^2}{a} + \frac{\sigma^2}{ab} = \frac{\tau^2+\sigma^2}{n}\big[1 + (b-1)\rho_I\big], \end{equation*}

so the design effect relative to a simple random sample of the same size \(n\) is \(1+(b-1)\rho_I\), and this is the entire content of the question. It is free of \(a\) and of \(A\), and free of \(B\) except through \(\rho_I\). Hence the limits are:

• (i) \(b = 1\): one unit per sampled cluster. No two sampled units share a cluster, the design effect is \(1\), and if in addition \(a/A \to 0\) the without-replacement selection of clusters is negligible and the \(a\) observations may be treated as i.i.d. draws. (For \(a/A\) not small, the usual finite-population correction \(1-a/A\) applies, as for a simple random sample of units.)
• (ii) \(B = 1\), equivalently \(\tau^2 \to 0\), equivalently \(\rho_I \to 0\): the clusters carry no information, and the design is a simple random sample whatever \(b\) is.
• (iii) \(a = A\) and \(b = B\): a census, and no inference is required.

Shrinking the sampling fractions is not among them: \(b/B\to 0\) or \(a/A\to 0\) leaves \(1+(b-1)\rho_I\) untouched.

(a) The posterior density is

\begin{equation*} p(N\,|\,y) \;\propto\; p(N)\, \frac{\binom{N-100}{70}}{\binom{N}{90}}, \qquad N \ge 170 . \end{equation*}

The sampling rule was inverse (stop at the 20th tagged fish), so the likelihood is the negative hypergeometric probability that the first 89 draws yield 19 tagged and 70 untagged and the 90th draw is tagged:

\begin{equation*} p(y\,|\,N) = \frac{\binom{100}{19}\binom{N-100}{70}}{\binom{N}{89}} \cdot \frac{81}{N-89}. \end{equation*}

Since \(\binom{N}{89}(N-89) = 90\binom{N}{90}\), this equals \(\tfrac{81}{90}\binom{100}{19}\binom{N-100}{70}/\binom{N}{90}\), and the factor in front is free of \(N\). The stopping rule is thus ignorable: the likelihood is proportional in \(N\) to the ordinary hypergeometric likelihood \(\binom{100}{20}\binom{N-100}{70}/\binom{N}{90}\) one would write for a fixed catch of 90.

(b) Take \(p(N)\propto 1\) on \(N\ge 170\) (or \(p(N)\propto 1/N\), the scale-invariant choice); both are improper, so properness of the posterior distribution must be checked. As \(N\to\infty\),

\begin{equation*} \frac{\binom{N-100}{70}}{\binom{N}{90}} \;\sim\; \frac{N^{70}/70!}{N^{90}/90!} \;=\; \frac{90!}{70!}\,N^{-20}, \end{equation*}

and \(\sum_N N^{-20} < \infty\); the posterior distribution is proper under either prior (indeed under any prior with \(p(N) = O(N^{18})\)). The tail decays like \(N^{-20}\) because 20 tagged fish were recaptured; had he recaptured only one, the tail would be \(N^{-1}\) and the posterior distribution improper under a flat prior.

Summing the unnormalized density numerically from \(N=170\): under \(p(N)\propto 1\) the mode is 449 (the classical Lincoln–Petersen value \(100\cdot 90/20 = 450\)), the mean 489.5, the standard deviation 95.9, the central 95% interval \([345,\,717]\); under \(p(N)\propto 1/N\), mode 436, mean 472.7, interval \([337,\,683]\) — the two noninformative priors agree to well within a posterior standard deviation.

(c) Averaging over the posterior distribution of \(N\). The 90 fish of the second catch are held, leaving \(N-90\) fish in the pond of which \(100-20 = 80\) are tagged, so

\begin{equation*} \Pr(\text{next fish tagged}\,|\,y) = \sum_{N=170}^{\infty} p(N\,|\,y)\,\frac{80}{N-90} \;=\; 0.211 \end{equation*}

under \(p(N)\propto 1\) (0.220 under \(p(N)\propto 1/N\)). If instead the second catch was thrown back, replace \(80/(N-90)\) by \(100/N\), giving 0.212.

(d) Let \(z\in\{0,1,\dots,5\}\) be the number of the five ambiguous fish that are truly tagged, so the catch of 90 contains \(t = 15+z\) tagged fish. Model the reading mechanism as independent per fish: a tagged fish is unreadable with probability \(\phi\), an untagged fish is (mis)recorded as ambiguous with probability \(\psi\). Then \(z\) of the \(t\) tagged fish and \(5-z\) of the \(90-t = 75-z\) untagged fish were ambiguous, and the joint posterior density is

\begin{equation*} \begin{aligned} p(N,\phi,\psi,z\,|\,y) \;\propto\; &\;p(N)\,p(\phi)\,p(\psi)\, \frac{\binom{100}{15+z}\binom{N-100}{75-z}}{\binom{N}{90}}\\ &\times \binom{15+z}{z}\phi^{z}(1-\phi)^{15}\\ &\times \binom{75-z}{5-z}\psi^{5-z}(1-\psi)^{70}, \end{aligned} \end{equation*}

for \(z = 0,\dots,5\) on the support \(N-100 \ge 75-z\), that is \(N \ge 175-z\); the catch size 90 is treated as fixed here, since the field count of 20 tagged that triggered the stopping rule is itself now uncertain. Here \(z\) is the missing datum and \((\phi,\psi)\) the parameters of the missingness mechanism; they are only weakly identified by these data and need genuinely informative Beta prior distributions — an untagged fish is rarely mistaken for a tagged one, so \(p(\psi)\) should be concentrated near 0. Summing out the six values of \(z\) gives \(p(N,\phi,\psi\,|\,y)\).

(a) Two ingredients: a logistic model for preference given lines, and a multinomial (Dirichlet) model for the population distribution of lines.

(i) For a household with \(L\in\{1,2,3,4\}\) lines let \(\theta_L = (\theta_{1L},\theta_{2L},\theta_{3L})\) be the probabilities of Bush, Dukakis and no opinion/other. Reparameterize as in (8.8),

\begin{equation*} \alpha_{1L} = \frac{\theta_{1L}}{\theta_{1L}+\theta_{2L}}, \qquad \alpha_{2L} = 1-\theta_{3L}, \end{equation*}

so that \(\alpha_{1L}\) is the Bush share among those expressing a preference and \(\alpha_{2L}\) the probability of expressing a preference. The \(L=3,4\) columns have 5 and 3 respondents, so a separate free pair per column is hopeless; put the logits linear in \(L\),

\begin{equation*} \begin{aligned} \mathrm{logit}\,\alpha_{1L} &= \beta_1 + \gamma_1 (L-1),\\ \mathrm{logit}\,\alpha_{2L} &= \beta_2 + \gamma_2 (L-1), \end{aligned} \end{equation*}

with \(p(\beta_1,\gamma_1,\beta_2,\gamma_2)\propto 1\). Recovering \(\theta_{1L}=\alpha_{2L}\alpha_{1L}\), \(\theta_{2L}=\alpha_{2L}(1-\alpha_{1L})\), \(\theta_{3L}=1-\alpha_{2L}\).

(ii) Let \(\pi = (\pi_1,\pi_2,\pi_3,\pi_4)\) be the proportions of households (in the telephone population) with \(L\) lines, with \(\pi \sim \mathrm{Dirichlet}(1,1,1,1)\). Because a household is reached with probability proportional to \(L\), the probability that a respondent household has \(L\) lines is

\begin{equation*} q_L = \frac{L\,\pi_L}{\sum_{m=1}^{4} m\,\pi_m}, \end{equation*}

and this reweighting is the whole content of the unequal selection probabilities here. The design is ignorable conditional on \(L\) (Section 8.3, unequal probabilities of selection) because \(L\) is fully observed in the sample and \(\pi\), the population distribution of \(L\), is a parameter of the model.

(b) Two assumptions, both unavoidable and both restricting the estimand.

• Zero-line households. They have selection probability \(0\cdot\pi_0 = 0\) and contribute nothing to the likelihood, so \(\pi_0\) is not identified. The estimand is therefore redefined as the mean preference among telephone households, and \(\pi\) is a distribution over \(L\ge 1\). Extending to all households requires an untestable assumption, e.g. that nontelephone households vote like one-line households.
• Missing \(L\). The 17 households in the ‘?’ column are assumed missing at random given preference: a household of type \((j,L)\) fails to report \(L\) with a probability \(r_j\) depending on \(j\) but not on \(L\). Its contribution to the likelihood is then \(\sum_{L} q_L \theta_{jL}\) times a factor free of \((\pi,\beta,\gamma)\), so the mechanism is ignorable (Section 8.2). This matters: the ‘?’ households are 1.1% of Bush respondents and 7% of no-opinion respondents, so the missingness is clearly related to \(j\), and assuming it unrelated to \(L\) given \(j\) is the substantive assumption.

(c) With \(n_{jL}\) the observed counts and \(m_j = (7,3,7)\) the ‘?’ counts,

\begin{equation*} \begin{aligned} p(\pi,\beta,\gamma\,|\,y) \;\propto\;\; &\prod_{j=1}^{3}\prod_{L=1}^{4}\big(q_L\,\theta_{jL}\big)^{n_{jL}} \;\prod_{j=1}^{3}\Big(\sum_{L=1}^{4} q_L\,\theta_{jL}\Big)^{m_j}, \end{aligned} \end{equation*}

where \(q_L = L\pi_L/\sum_m m\pi_m\) and \(\theta_{jL}\) is the function of \((\beta,\gamma)\) above, times the Dirichlet\((1,1,1,1)\) density for \(\pi\). This is a 7-dimensional posterior distribution (\(\pi\) has 3 free coordinates).

(d) Random-walk Metropolis (Section 11.2) on \((\log \pi_L/\pi_1,\ \beta_1,\gamma_1,\beta_2,\gamma_2)\), \(4\times 10^5\) iterations with acceptance rate 0.12, first quarter discarded, thinned to 1000 draws. The fitted line effects are \(\gamma_1 = 0.30 \pm 0.21\) (Bush share rises weakly with number of lines) and \(\gamma_2 = 2.5 \pm 1.3\) (multi-line households almost always express a preference: only 1 of the 74 respondents with \(L\ge 2\) had no opinion). The implied Bush shares \(\theta_{1L}\) have posterior means \(0.516,\ 0.627,\ 0.694,\ 0.744\) for \(L=1,2,3,4\). The population line distribution is

\begin{equation*} \mathrm{E}[\pi\,|\,y] = (0.968,\ 0.030,\ 0.0018,\ 0.0009), \end{equation*}

against a raw respondent distribution of \((0.935,\ 0.058,\ 0.0044,\ 0.0026)\): dividing by \(L\) and renormalizing is exactly the design correction.

(e) The household-level estimands are

\begin{equation*} \Theta_j = \sum_{L=1}^{4}\pi_L\,\theta_{jL}, \qquad \mathrm{E}[\Theta\,|\,y] = (0.520,\ 0.396,\ 0.084), \end{equation*}

with posterior standard deviations \((0.015,\ 0.014,\ 0.009)\). For the contrast,

\begin{equation*} (\Theta_{\mathrm{Bush}}-\Theta_{\mathrm{Dukakis}})\,|\,y:\quad \text{mean } 0.124,\ \text{sd } 0.028,\ \text{95\% interval } [0.071,\ 0.183], \end{equation*}

and \(\Pr(\Theta_{\mathrm{Bush}} > \Theta_{\mathrm{Dukakis}}\,|\,y) > 0.999\); 1000 thinned draws locate the posterior mean to about \(\pm 0.002\), so every figure quoted above is good to its last digit. The histogram is a symmetric unimodal distribution centered at 0.124; compare the unweighted Dirichlet analysis of these same 1162 respondents, which gives \(0.130 \pm 0.028\). Weighting by \(1/L\) therefore shifts the Bush lead down by about 0.006 — one-fifth of a posterior standard deviation — because multi-line households lean Bush and are over-sampled. Figure 3.2, built from the 1447 respondents of Section 3.4 (\(y_1 = 727\), \(y_2 = 583\), \(y_3 = 137\)) rather than the 1162 tabulated here, is centered at \(144/1450 = 0.099\) with a similar spread; the gap between 0.099 and 0.130 is a difference of sample, not of method, and is five times the unequal-probability correction. The phone-line weighting is thus a second-order adjustment, because 96.8% of households have a single line.

(f) The model fits. Take \(T(y,\theta) = \sum_{\text{cells}} (y - \mathrm{E}\,y)^2/\mathrm{E}\,y\) over the \(3\times 4\) table of the 1145 fully classified respondents (the ‘?’ column is uninformative here, since its three expectations are fit exactly by the free \(r_j\)), and for each of the 1000 draws generate \(y^{\mathrm{rep}}\) from the multinomial posterior predictive distribution:

\begin{equation*} \Pr\big(T(y^{\mathrm{rep}},\theta) \ge T(y,\theta)\,\big|\,y\big) = 0.44 \quad (\text{Monte Carlo s.e. } 0.016), \end{equation*}

with a median observed discrepancy of 9.1 on 12 cells. No evidence of misfit: in particular the linear-logit restriction on \(\alpha_{1L}\) and \(\alpha_{2L}\) is not contradicted by the sparse \(L=3,4\) columns.

(g) Nothing inside the model matters; the one assumption that does lies outside it.

Refitting with (i) free \((\alpha_{1L},\alpha_{2L})\) in each column in place of the linear logits, (ii) a \(\mathrm{Dirichlet}(\tfrac14,\dots,\tfrac14)\) prior distribution on \(\pi\), (iii) all 17 ‘?’ households assigned to \(L=1\), and (iv) all assigned to \(L=2\), the posterior mean of \(\Theta_{\mathrm{Bush}}-\Theta_{\mathrm{Dukakis}}\) stays within \(0.124\)–\(0.125\) with standard deviation \(0.027\)–\(0.029\). Weakening the proportionality itself — selection probability proportional to \(L^{1/2}\) or to \(L^{2}\) rather than to \(L\) — gives 0.128 and 0.122. All of this is within half a posterior standard deviation, for the same reason throughout: the \(L\ge 2\) cells hold 74 of 1145 respondents.

The binding assumption is the one identified in (b): the estimand covers telephone households only, and no reweighting of these data can reach the rest. If a fraction \(w\) of households have no telephone and their Bush–Dukakis gap differs by \(\Delta\), the all-household estimand shifts by \(w\Delta\) — for \(w = 0.07\) and \(\Delta = 0.1\) that is 0.007, larger than every within-model sensitivity above and comparable to the entire phone-line correction.

Bush leads Dukakis by \(0.156 \pm 0.030\) among individuals, against \(0.124 \pm 0.028\) among households: weighting toward large households moves the lead up by about one posterior standard deviation, because larger households lean Bush.

(a) Models. Let \(\pi_{SL}\) be the population proportion of telephone households with \(S\) adults and \(L\) lines, \(S = 1,\dots,8\), \(L = 1,\dots,4\), and let \(\theta_{jSL}\) be the preference probabilities within such a household, parameterized as in (8.8) and Exercise 8.12 with

\begin{equation*} \begin{aligned} \mathrm{logit}\,\alpha_{1SL} &= \beta_1+\gamma_1(L-1)+\delta_1(S-2),\\ \mathrm{logit}\,\alpha_{2SL} &= \beta_2+\gamma_2(L-1)+\delta_2(S-2). \end{aligned} \end{equation*}

The key simplification is in the selection probabilities. A household with \(S\) adults contains \(S\) individuals; each is reached with probability proportional to \(L/S\); so the expected number of respondents from cell \((S,L)\) is proportional to

\begin{equation*} \underbrace{S\,\pi_{SL}}_{\text{individuals}} \times \underbrace{\frac{L}{S}}_{\text{per-person prob.}} = L\,\pi_{SL}, \end{equation*}

and the sample cell probabilities are \(q_{SL} = L\pi_{SL}/\sum_{S^{\prime}L^{\prime}} L^{\prime}\pi_{S^{\prime}L^{\prime}}\) — exactly the same reweighting as in 8.12. The \(1/S\) factor cancels the \(S\) individuals per household, which is just the statement that one adult per household is interviewed. Prior: \(q \sim \mathrm{Dirichlet}(1/32,\dots,1/32)\) on the 32 cells (so that the many empty cells are not inflated by prior pseudo-counts), with \(\pi_{SL}\propto q_{SL}/L\), and \(p(\beta,\gamma,\delta)\propto 1\).

Where \(S\) does enter is the estimand. The individual-level mean preferences are

\begin{equation*} \Theta_j^{\mathrm{ind}} = \frac{\sum_{S,L} S\,\pi_{SL}\,\theta_{jSL}}{\sum_{S,L} S\,\pi_{SL}}, \end{equation*}

versus \(\Theta_j^{\mathrm{hh}} = \sum_{S,L}\pi_{SL}\theta_{jSL}\) in 8.12. So relative to a naive analysis of the respondents, each respondent carries weight proportional to \(S/L\): \(1/L\) to undo the multiple-line oversampling, \(S\) to represent the other adults in the household.

(b) Assumptions. As in 8.12, households with \(L=0\) are excluded and the estimand is confined to adults in telephone households. Additionally: (i) the responding adult’s preference is taken as a draw from \(\theta_{jSL}\), i.e. adults within a household are exchangeable given \((S,L)\) — if spouses correlate politically, and they do, this understates the posterior variance and, more seriously, makes \(\theta_{jSL}\) the distribution of the selected adult rather than of a random adult, which is only the same thing if which adult answers the phone is unrelated to preference; (ii) the 17 ‘?’ entries for \(L\) and the 2 ‘?’ entries for \(S\) are missing at random given the other two variables and are imputed within the model; (iii) the printed ‘?’ row for adults lumps nonresponse together with every household size above 8 (Table 8.8 caption), and those 2 respondents are imputed within \(S\le 8\), so the population is taken to contain no household with more than 8 adults.

(c) Joint posterior. With \(n_{jSL}\) the fully observed counts, \(m^{L}_{jS}\) the counts missing \(L\), and \(m^{S}_{jL}\) the counts missing \(S\),

\begin{equation*} \begin{aligned} p(q,\beta,\gamma,\delta\,|\,y)\;\propto\;\; & p(q)\prod_{j,S,L}\big(q_{SL}\theta_{jSL}\big)^{n_{jSL}}\\ &\times \prod_{j,S}\Big(\sum_{L} q_{SL}\theta_{jSL}\Big)^{m^{L}_{jS}} \;\prod_{j,L}\Big(\sum_{S} q_{SL}\theta_{jSL}\Big)^{m^{S}_{jL}}. \end{aligned} \end{equation*}

(d) Simulation. Gibbs sampler with data augmentation (Section 11.1): impute the missing \(L\) and \(S\) from their multinomial conditionals; draw \(q\) from its conjugate Dirichlet conditional given the completed counts \(c_{SL}\); update \((\beta,\gamma,\delta)\) by Metropolis. \(1.2\times 10^5\) iterations, acceptance rate 0.25, first sixth discarded, thinned to 1000 draws. The fitted coefficients are

\begin{equation*} \begin{aligned} \gamma_1 &= 0.25 \pm 0.22, &\quad \delta_1 &= 0.19 \pm 0.08,\\ \gamma_2 &= 2.2 \pm 1.2, &\quad \delta_2 &= 0.31 \pm 0.15. \end{aligned} \end{equation*}

\(\delta_1 > 0\) is the driver: Bush support rises with household size, visible already in the raw margins (Bush share among respondents is 0.434 in one-adult households, 0.550 with two adults, 0.574 with three, 0.622 with four). The posterior mean number of adults per telephone household is 1.97.

(e) Estimands.

\begin{equation*} \mathrm{E}\big[\Theta^{\mathrm{ind}}\,\big|\,y\big] = (0.540,\ 0.384,\ 0.077), \end{equation*}

with posterior standard deviations \((0.016,\ 0.015,\ 0.008)\), and

\begin{equation*} (\Theta^{\mathrm{ind}}_{\mathrm{Bush}}-\Theta^{\mathrm{ind}}_{\mathrm{Dukakis}})\,|\,y:\ \text{mean } 0.156,\ \text{sd } 0.030,\ 95\%\ [0.099,\ 0.212], \end{equation*}

with \(\Pr(\text{Bush ahead}\,|\,y) > 0.999\) and a Monte Carlo error of about \(\pm 0.002\) on the posterior means. Evaluating the household estimand from the same fitted model gives \(0.124 \pm 0.028\), reproducing 8.12 and confirming that the two analyses differ only in the estimand, not in the likelihood.

Explanation of the difference. The entire \(+0.032\) gap between the individual and household estimands comes from the factor \(S\) in the numerator of \(\Theta^{\mathrm{ind}}\) together with \(\delta_1 = 0.19 > 0\): individual weighting shifts mass from one-adult households, which favored Dukakis (0.458 to 0.434), to two-or-more-adult households, which favored Bush (0.550 to 0.377 at \(S=2\)), and there are about two adults per household so the shift is substantial. The \(1/L\) correction, worth \(-0.006\), is already inside the household figure 0.124 and is an order of magnitude smaller — because 97% of households have one line but only 26% have one adult. A third, minor effect: the no-opinion rate falls from 0.108 among one-adult households to 0.035–0.073 among larger ones, so the individual-level residual category is also smaller (0.077 against 0.084). The posterior standard deviation grows from 0.028 to 0.030, the price of the unequal weights.

(f) Model checking. Check the two margins the model actually parameterizes, with \(T(y,\theta) = \sum_{\text{cells}}(y-\mathrm{E}\,y)^2/\mathrm{E}\,y\):

\begin{equation*} \begin{aligned} \text{preference}\times\text{adults (24 cells):}&\quad p_B = 0.33,\ \ T(y,\theta)\ \text{median } 24.7,\\ \text{preference}\times\text{lines (12 cells):}&\quad p_B = 0.38,\ \ T(y,\theta)\ \text{median } 10.3, \end{aligned} \end{equation*}

where \(p_B = \Pr(T(y^{\mathrm{rep}},\theta)\ge T(y,\theta)\,|\,y)\) (Monte Carlo s.e. 0.015). Both margins fit. On the full \(8\times 3\times 4\) table the same statistic gives \(p_B = 0.24\), with median \(T(y,\theta) = 59.3\) against a replicate median of 40.7, but there the statistic is dominated by cells whose expectations are a small fraction of one count, so it is unstable and has little power against the thing most likely to be wrong — linearity of the logits in \(S\) and \(L\).

(g) Sensitivity. Unlike 8.12, one modeling choice here moves the answer: the prior distribution on the 32 cells of \(q\). With \(\mathrm{Dirichlet}(1,\dots,1)\) in place of \(\mathrm{Dirichlet}(\tfrac{1}{32},\dots,\tfrac{1}{32})\) the individual-level lead rises from \(0.156\pm 0.030\) to \(0.169\pm 0.031\) — a shift of nearly half a posterior standard deviation, because a pseudo-count of 1 in each empty large-\(S\) cell is amplified by the factor \(S\) in \(\Theta^{\mathrm{ind}}\) and large households lean Bush. Adding a quadratic term in \(S\) to both logits gives \(0.158 \pm 0.031\), and extending the range of \(S\) to 10 gives \(0.155 \pm 0.030\): those are immaterial. The individual-versus-household gap of \(+0.032\) is therefore robust in sign and order of magnitude but only good to about \(\pm 0.015\), and the telephone-household restriction of 8.12(g) still bounds everything from outside.

(a) \([3.77\times 10^{11},\ 3.96\times 10^{11}]\) passenger miles, with median \(3.86\times 10^{11}\).

Measure \(x\) in units of \(10^8\) passenger miles, so that the death rate is \(\rho = y/x\) with \(y = 734\) known exactly. The rounded value \(r = 0.19\) is a deterministic function of \(x\), so the likelihood is an indicator:

\begin{equation*} p(r\,|\,x) = \mathbf{1}\big\{0.185 \le 734/x < 0.195\big\} = \mathbf{1}\big\{3764.1 < x \le 3967.6\big\}. \end{equation*}

Take \(p(x)\propto 1/x\), the scale-invariant prior distribution on a positive quantity of unknown magnitude (equivalently \(p(\rho)\propto 1/\rho\): with \(x = y/\rho\) the Jacobian is \(|dx/d\rho| = y/\rho^2\), and \((1/x)(y/\rho^2) = 1/\rho\)). It is improper, but the indicator truncates it to a bounded interval, so the posterior distribution is proper — it is simply the prior restricted to the rounding window, \(\log x \sim \mathrm{U}[\log 3764.1,\ \log 3967.6]\), giving the 95% central interval \([3769,\ 3962]\) in units of \(10^8\) miles and median 3864. The prior matters not at all here: a flat \(p(x)\propto 1\) gives \([3769,\ 3963]\).

(b) The same computation year by year. Intervals in units of \(10^8\) passenger miles:

The relative width is essentially \(0.01/r_t\): rounding to two decimals is a \(\pm 2.6\%\) statement when \(r=0.19\) but a \(\pm 17\%\) statement when \(r = 0.03\). The 1984 interval is nearly useless on its own, and the naive point estimate \(223/0.03 = 7433\) is badly placed — it sits above the 1985 figure, which is implausible for a growing series.

(c) Borrow strength through a smoothness prior on the trajectory. Airline traffic is a very smooth series, so penalize second differences of \(\log x_t\):

\begin{equation*} \log x_t - 2\log x_{t-1} + \log x_{t-2} \sim \mathrm{N}(0,\sigma^2), \quad t = 3,\dots,10, \end{equation*}

with \(p(\log x_1,\log x_2,\log\sigma)\propto 1\), multiplied by the ten rounding indicators \(\mathbf{1}\{L_t \le \log x_t \le H_t\}\) from part (a). This is a locally linear (random-walk-on-the-growth-rate) model; it says the growth rate changes slowly, not that growth is constant.

The flat prior distribution on \(\log\sigma\) is improper and would ordinarily put an unnormalizable spike at \(\sigma = 0\), where all eight second differences vanish. Here it does not: linear programming shows no quadratic \(a + bt + ct^2\) lies inside all ten windows simultaneously (the largest achievable slack is \(-0.016\) on the log scale), so the second differences cannot all be driven to zero, \(\sigma\) is bounded away from 0, and the posterior distribution is proper.

Gibbs sampler: each \(\log x_t\) has a truncated normal full conditional (normal from the Gaussian quadratic form, truncated to its rounding window), and \(\sigma^2\) has the usual scaled inverse-\(\chi^2\) conditional given the second differences. \(2\times 10^5\) iterations, first tenth discarded, thinned to 9000 draws; \(\mathrm{E}[\sigma\,|\,y] = 0.042\), and independent runs reproduce the interval endpoints below to within about 10 in these units.

The years with tight rounding windows are unchanged — the constraint already says everything — while 1984 loses three-quarters of its interval width and its median falls from 7539 to 6643, now correctly interpolating between 1983 and 1985. 1981 loses 44%. A cruder joint model, \(\log x_t = \alpha+\beta(t-1980.5)+\epsilon_t\) with i.i.d. \(\epsilon_t\), would achieve almost nothing: the least-squares residual standard deviation about a straight line through the ten window midpoints is 0.068 on the log scale, about the median width of the rounding windows themselves (0.074), so the trend line carries no usable information about individual years. The smoothness prior works because it constrains curvature, not level.

(d) Replace the plug-in exposures \(\hat x_t = y_t/r_t\) of Exercise 2.13 by the posterior draws \(x_t^{(\ell)}\) from (c), and average. Concretely, 2.13 asks for a Poisson model with rate \(\theta\) and exposure proportional to passenger miles; write the joint posterior distribution of everything,

\begin{equation*} p(\theta, x \,|\, y, r) \;\propto\; p(\theta)\,p(x)\,\prod_{t=1}^{10} \mathrm{Poisson}(y_t \,|\, \theta x_t)\, \mathbf{1}\{\text{round}(y_t/x_t) = r_t\}, \end{equation*}

and fit it by adding a \(\theta\) step to the sampler of (c) — or, more simply, by multiple imputation: for each draw \(x^{(\ell)}\), compute \(p(\theta\,|\,y, x^{(\ell)})\), which is \(\mathrm{Gamma}(\alpha+\sum_t y_t,\ \beta+\sum_t x^{(\ell)}_t)\) under a conjugate prior, draw a \(\theta\), then draw the 1986 predictive count from \(\mathrm{Poisson}(\theta\cdot 8000)\). The mixture over \(\ell\) is the correct predictive distribution.

Two things this buys. First, honest uncertainty: the exposures are known only to within a few percent, which for the individual-year fits of 2.13(a),(c) is of the same order as the Poisson noise \(\sqrt{734}/734 = 3.7\%\), so ignoring it understates the posterior variance. Second, and more important, it removes a bias: the naive \(\hat x_{1984} = 7433\) is 12% above the joint-model estimate 6643, and any 2.13 analysis using it correspondingly understates the 1984 death rate. The joint model also removes the circularity in 2.13 of using \(y_t\) both as the Poisson outcome and, through \(y_t/r_t\), as the exposure — here the exposure is a parameter informed by the rounding constraint and by the smoothness of the series, not a transformation of the outcome.

(a) The order of entry into the study: record for each subject \(i\) the time (or rank) \(t_i\) at which he or she arrived. The assignment distribution is not invariant to permutations of the unit indexes, so the unit labels alone do not determine the assignment probabilities: with \(n = 3\) and treatments \(A, B\),

\begin{equation*} \begin{aligned} \Pr(ABA) &= \tfrac{1}{2}\cdot p\cdot\tfrac{1}{2} = \tfrac{p}{4},\\ \Pr(AAB) &= \tfrac{1}{2}\cdot(1-p)\cdot p , \end{aligned} \end{equation*}

which are unequal unless \(p = \frac{1}{2}\): the same multiset of assignments has different probabilities depending on the arrival order.

Given \(x = t\), write \(I = (I_1,\dots,I_n)\) for the assignment vector and \(D_{i-1} = \#\{j < i : I_j = A\} - \#\{j < i: I_j = B\}\). Then

\begin{equation*} p(I \mid x, y, \phi) \;=\; \prod_{i=1}^{n} q(I_i \mid D_{i-1}), \qquad q(A\mid d) = \begin{cases} \tfrac{1}{2}, & d = 0,\\ 1-p, & d > 0,\\ p, & d < 0, \end{cases} \end{equation*}

which is free of \(y\) and involves no unknown parameter (\(p\) is fixed and known). The data are thus missing at random with \(\phi\) trivially distinct from \(\omega\), so (8.3) equals (8.2) and the design is ignorable – indeed strongly ignorable and known, \(p(I \mid x, y, \phi) = p(I \mid x)\). The propensity scores will not do as a substitute for \(t\): \(\Pr(I_i = A \mid x) = \frac{1}{2}\) for every \(i\) by the \(A \leftrightarrow B\) symmetry, the same scores as complete randomization (Section 8.2).

(b) Because the design is ignorable given \(t\), model the data directly and never model \(I\): work with \(p(\omega \mid x, y_{\mathrm{obs}}) \propto p(\omega \mid x)\,p(y_{\mathrm{obs}} \mid x, \omega)\), which is (8.3). Let \(T_i = 1\) if subject \(i\) received treatment \(A\); fit, for example,

\begin{equation*} y_i \mid \omega \sim \mathrm{N}\!\left(\alpha + \theta T_i + f(t_i),\ \sigma^2\right), \end{equation*}

with \(f\) a low-order spline, random-walk, or period-effect term in entry time, plus any genuine baseline covariates and, if warranted, a \(\theta \times t\) interaction. Superpopulation inference is the posterior for \(\theta\); for the finite-population estimand, draw \(\omega\) from its posterior, impute the missing potential outcomes \(y_i^{\mathrm{mis}}\) from \(p(y_{\mathrm{mis}} \mid x, y_{\mathrm{obs}}, \omega)\), and report the posterior distribution of \(\frac{1}{n}\sum_i (y_i^A - y_i^B)\). Since the design is known, posterior predictive checks can replicate \(I\) as well as \(y\): draw a fresh biased-coin sequence, draw \(y^{\mathrm{rep}}\), and check residual drift and serial correlation against the arrival order.

(c) The model for the dependence of \(y\) on entry time, \(f(t)\), and in particular on any component of it at the frequency at which the design alternates. Smooth drift is handled well: an omitted trend of slope \(\beta\) biases \(\theta\) by \(\beta(\bar{t}_A - \bar{t}_B)\), and the design keeps \(|D_i|\) small and hence keeps \(\bar{t}_A - \bar{t}_B\) smaller than complete randomization does, so if \(t\) is dropped altogether the posterior mean of \(\theta\) stays reasonable while its posterior standard deviation comes out too large. High-frequency structure is the danger: coding assignments as \(\pm 1\), the lag-one autocorrelation is \(-\frac{1}{2}\cdot\frac{n}{n-1} = -0.505\) at \(p = 1\), \(n = 100\), and tends to \(0\) as \(p \downarrow \frac{1}{2}\), so as \(p \to 1\) the treatment indicator becomes a deterministic alternation within successive pairs and \(\theta\) is aliased with any “first versus second subject of a pair” effect (time of day, position in a clinic session). The design is also sensitive to the premise that the coin depends only on past assignments: dependence on past outcomes costs strong ignorability (Section 8.2), while clinician adjustment based on the current subject’s unrecorded characteristics costs ignorability outright (Section 8.5, complication 1), as would choosing \(p\) itself in light of beliefs about the treatment effect (complication 3, \(\phi\) and \(\omega\) not distinct).

(d) All three designs are ignorable as specified, so for a fixed model and fixed data the posterior distribution of \(\omega\) and of the finite-population estimands is the same for all of them (Section 8.5); the comparison is entirely about expected precision, sensitivity to modeling assumptions, and the practical complications.

(i) Independent randomization, \(p = \frac{1}{2}\), is ignorable and known with no covariates, \(p(I \mid x,y,\phi) = p(I)\) as in (8.4), which is the simplest possible situation: units are exchangeable, entry order need not be recorded, and replicated assignment vectors for posterior predictive checking are unconstrained. Its cost is imbalance. With \(n = 100\) the expected imbalance is \(\mathrm{E}|n_A - n_B| = 7.96\) and \(\Pr(|n_A - n_B| \le 2) = 0.24\), against \(1.33\) and \(0.88\) at \(p = \frac{2}{3}\) (exact, from the birth-death chain for \(|D_i|\)). Unequal and time-imbalanced groups make inference about \(\theta\) lean much harder on the assumed form of \(f(t)\) and of any covariate adjustment.

(ii) The \(p = 1\) design is randomized blocks of size two: exactly \(n_A = n_B\), and within-pair contrasts that are exactly balanced against arbitrary drift, hence the highest precision under a model with pair effects. But every even-numbered assignment is determined by the odd-numbered one, so the investigator knows the next treatment before the next patient is enrolled and can select or defer patients accordingly – dependence on unrecorded covariates, and therefore a nonignorable mechanism whose selection effects must be modelled, “at best a difficult enterprise with heightened sensitivity to model assumptions” (Section 8.5). It also fixes the aliasing of (c) at its worst and admits only \(2^{n/2}\) assignment vectors, which thins the set of hypothetical replications available for model checking. (The 2nd-edition solutions manual says this design has only two possible assignment vectors; since the pairs are randomized independently there are \(2^{n/2}\).)

The biased coin with \(\frac{1}{2} < p < 1\) buys nearly all of the balance of (ii) while keeping every assignment genuinely random: no assignment probability is ever \(0\) or \(1\), so no treatment can be predicted with certainty in advance, cheating is impeded, and the replicate distribution of \(I\) stays rich enough for posterior predictive checks. Its price is precisely (a): the units are no longer exchangeable, entry order must be recorded and modelled, and the “no covariates” simplicity of (i) is lost.

(a) Yes, yes, and yes. The assignment \(I\) was completely randomized, so

\begin{equation*} p(I \mid X, Y, \phi) = p(I), \end{equation*}

which is exactly form (8.4): ignorable and known, with no unknown \(\phi\) and no dependence on covariates or outcomes. It is a fortiori strongly ignorable, since \(p(I \mid X, Y, \phi) = p(I) = p(I \mid X)\).

The exposure is a different matter: \(U_{\mathrm{obs},i} = I_i C_i(1)\) depends on the unobserved stratum \(C_i(1)\), so it is not a covariate, and one stratifies on \((C(1), C(0))\) instead (Section 8.6).

(b) The two assignment arms have

\begin{equation*} \begin{aligned} n_0 &= 74 + 11514 = 11588, & \bar{y}_0 &= \tfrac{11514}{11588} = 0.99361,\\ n_1 &= 2419 + 9675 = 12094, & \bar{y}_1 &= \tfrac{12048}{12094} = 0.99620, \end{aligned} \end{equation*}

so the intention-to-treat effect \(\overline{Y^1} - \overline{Y^0}\) of (8.18) is estimated by

\begin{equation*} \bar{y}_1 - \bar{y}_0 = 0.0025824, \end{equation*}

an increase in survival of \(2.58\) per \(1000\) infants assigned the supplement. Randomization makes this unbiased for the population ITT, and the binomial standard error is

\begin{equation*} \sqrt{\frac{\bar{y}_0(1-\bar{y}_0)}{n_0} + \frac{\bar{y}_1(1-\bar{y}_1)}{n_1}} = 0.00093, \end{equation*}

so the ITT is about \(2.8\) standard errors from zero.

(c) The proportion of compliers is estimated from the randomized treatment arm, where compliance status is observed:

\begin{equation*} \hat{p}_c = \frac{9675}{12094} = 0.79998. \end{equation*}

The exclusion restriction (no effect of assignment on never-takers, \(\mathrm{NACE} = 0\)) then gives the instrumental variables estimate (8.19),

\begin{equation*} \widehat{\mathrm{CACE}} = \frac{\bar{y}_1 - \bar{y}_0}{\hat{p}_c} = \frac{0.0025824}{0.79998} = 0.0032280, \end{equation*}

an increase in survival of \(3.23\) per \(1000\) compliers, with standard error roughly \(0.00093/0.80 = 0.00116\) (treating \(\hat{p}_c\) as fixed).

(d) With compliance known for every unit, the likelihood in \(\omega = (p_c, \theta_{c0}, \theta_{c1}, \theta_{n0}, \theta_{n1})\) is

\begin{equation*} \begin{aligned} L(\omega) &= p_c^{\,N_{c0}+N_{c1}}\,(1-p_c)^{\,N_{n0}+N_{n1}}\\ &\quad \times \prod_{s \in \{c,n\}} \prod_{z \in \{0,1\}} \theta_{sz}^{\,M_{sz}}\,(1-\theta_{sz})^{\,N_{sz}-M_{sz}}, \end{aligned} \end{equation*}

a binomial for \(p_c\) times four independent binomials, conjugate under independent beta priors. Here the units are exchangeable, the stratum \(C_i \in \{c, n\}\) (complier, never-taker) has \(\Pr(C_i = c) = p_c\), and \(Y_i \mid C_i = s, I_i = z \sim \mathrm{Bernoulli}(\theta_{sz})\); \(N_{sz}\) counts the units in stratum \(s\) assigned \(z\) and \(M_{sz}\) the survivors among them. The assignment factor drops out because the design is ignorable and known by (a). The estimands are \(\mathrm{CACE} = \theta_{c1} - \theta_{c0}\) and \(\mathrm{NACE} = \theta_{n1} - \theta_{n0}\), and the exclusion restriction is the constraint \(\theta_{n1} = \theta_{n0}\).

Table 8.5 supplies only \((N_{n1}, M_{n1}) = (2419, 2385)\) and \((N_{c1}, M_{c1}) = (9675, 9663)\): in the control arm \(C_i\) is missing, so the cells \((N_{c0}, M_{c0})\) and \((N_{n0}, M_{n0})\) are unobserved subject to \(N_{c0}+N_{n0} = 11588\), \(M_{c0}+M_{n0} = 11514\), and the observed-data likelihood is the two-component mixture obtained by summing the display over \(C_i \in \{c,n\}\) for each control unit.

(a) Analysis (ii). The treatment was applied to dishes, not to cultures, so the dish is part of the design and the analysis must condition on it (Section 8.8); analysis (i) does not, and its standard errors are correct only if the between-dish variance is exactly zero, which the researcher’s physical argument makes plausible but cannot establish.

Write \(y_{tij} = \mu_t + \alpha_{ti} + \epsilon_{tij}\) for treatment \(t\), dish \(i = 1,\dots,5\), culture \(j = 1,\dots,6\), with \(\operatorname{var}(\alpha_{ti}) = \tau^2\), \(\operatorname{var}(\epsilon_{tij}) = \sigma^2\) and intraclass correlation \(\rho = \tau^2/(\tau^2+\sigma^2)\). Both analyses estimate \(\bar y_{2\cdot\cdot} - \bar y_{1\cdot\cdot}\), whose variance is \(\tfrac15(\tau^2 + \sigma^2/6)\), while the 30-point sample variance has \(E[s^2] = \sigma^2 + \tfrac{24}{29}\tau^2\) (the sum of squares splits as \(25\sigma^2\) within dishes plus \(24\tau^2 + 4\sigma^2\) between). So (i)’s nominal variance is too small by

\begin{equation*} \begin{aligned} \frac{\sigma^2 + 6\tau^2}{\sigma^2 + \tfrac{24}{29}\tau^2} &= 1.26,\ 1.53,\ 2.07 \\ &\quad\text{at } \rho = 0.05,\ 0.10,\ 0.20, \end{aligned} \end{equation*}

i.e. standard errors understated by 12%, 24%, 44%. Against that, if \(\tau = 0\) the only price of (ii) is degrees of freedom, \(8\) instead of \(58\), widening the expected interval by

\begin{equation*} \begin{aligned} \frac{t_{.975,8}\,c_8}{t_{.975,58}\,c_{58}} &= \frac{2.235}{1.993} = 1.12, \\ c_\nu &= \sqrt{2/\nu}\,\Gamma(\tfrac{\nu+1}{2})/\Gamma(\tfrac{\nu}{2}). \end{aligned} \end{equation*}

The hoped-for gain from \(n = 30\) is thus at most 12%, because with equal dish sizes the dish means \(\bar y_{ti\cdot}\) are sufficient for \(\mu_t\) given \((\tau,\sigma)\): the within-dish spread informs \(\sigma^2\), not \(\theta\).

(b) A hierarchical normal model with the dish as the grouping level, the five dishes within a treatment being exchangeable (Section 5.2), which is what licenses a common \(\tau\). For \(t = 1,2\), \(i = 1,\dots,5\), \(j = 1,\dots,6\),

\begin{equation*} \begin{aligned} y_{tij} \mid \alpha_{ti}, \sigma &\sim \mathrm{N}(\alpha_{ti},\, \sigma^2), \\ \alpha_{ti} \mid \mu_t, \tau &\sim \mathrm{N}(\mu_t,\, \tau^2), \\ \mu_1 &= \mu - \tfrac{\theta}{2}, \quad \mu_2 = \mu + \tfrac{\theta}{2}, \end{aligned} \end{equation*}

so \(\theta\) is the treatment effect. Take \(p(\mu,\theta,\log\sigma) \propto 1\) as in Section 5.4 (the 50 within-dish degrees of freedom pin \(\sigma\) down) and a half-Cauchy\((0,A)\) prior on \(\tau\) with \(A\) on the scale of the outcome (Section 5.7), which with only ten dishes is safer than the uniform prior, though that too gives a proper posterior here. Because dishes were randomized to treatments, the assignment is ignorable given this model and no further design factor is needed (Section 8.4).

Marginalizing over \(\alpha\), the dish means are independent with

\begin{equation*} \begin{aligned} \bar y_{ti\cdot} \mid \mu_t, \tau, \sigma &\sim \mathrm{N}\Big(\mu_t,\ \tau^2 + \tfrac{\sigma^2}{6}\Big), \\ \theta \mid y, \tau, \sigma &\sim \mathrm{N}\Big(\bar y_{2\cdot\cdot} - \bar y_{1\cdot\cdot},\ \tfrac{2}{5}\big(\tau^2 + \tfrac{\sigma^2}{6}\big)\Big), \end{aligned} \end{equation*}

a variance equal to analysis (i)’s at \(\tau = 0\) and to analysis (ii)’s for \(\tau \gg \sigma\), so the posterior averages over \(\tau\) rather than forcing the choice.

Manufacture \(n^{*}=\mu+\sigma\,\Phi^{-1}(1/3)=10{,}000-5000(0.4307)\approx 7850\) widgets, for an expected net profit of \(\mu-3\sigma\varphi(z_{1/3})\approx 4546\) dollars, i.e. $4546.

Let \(D\sim\mathrm{N}(\mu,\sigma^{2})\) with \(\mu=10{,}000\), \(\sigma=5000\) be the market, and let \(n\) be the number manufactured. All \(n\) are paid for, and only \(\min(n,D)\) are sold, so the net profit is

\begin{equation*} g(n,D)=3\min(n,D)-2n . \end{equation*}

This is a utility function linear in dollars (BDA3 Section 9.1), so the decision is to maximize \(\mathrm{E}[g(n,D)]=3\,\mathrm{E}[\min(n,D)]-2n\) over \(n\).

Since \(\tfrac{\partial}{\partial n}\min(n,D)=1\{D>n\}\) and the integrand is bounded by \(1\), differentiation under the integral gives

\begin{equation*} \frac{d}{dn}\mathrm{E}[g(n,D)]=3\,\Pr(D>n)-2 , \end{equation*}

which is strictly decreasing in \(n\): the expected profit is concave and the unique maximizer solves

\begin{equation*} \Pr(D>n^{*})=\tfrac{2}{3}, \qquad\text{i.e.}\qquad \Phi\!\left(\frac{n^{*}-\mu}{\sigma}\right)=\tfrac{1}{3}. \end{equation*}

(The critical fractile \(1/3\) is the ratio of the $1 profit forgone on an unmet sale to the $1 forgone plus the $2 sunk on an unsold widget.) With \(z_{1/3}=\Phi^{-1}(1/3)=-0.43073\),

\begin{equation*} n^{*}=10{,}000+5000(-0.43073)=7846.4 , \end{equation*}

so about \(7850\) widgets.

For the value attained, write \(\mathrm{E}[\min(n,D)]=n-\mathrm{E}[(n-D)^{+}]\) and use the normal partial-expectation identity \(\mathrm{E}[(n-D)^{+}]=(n-\mu)\Phi(z)+\sigma\varphi(z)\) with \(z=(n-\mu)/\sigma\) (Check!). At \(n=n^{*}\), where \(\Phi(z)=1/3\) and \(n^{*}-\mu=\sigma z\),

\begin{equation*} \begin{aligned} \mathrm{E}[g(n^{*},D)]&=n^{*}-3\bigl[(n^{*}-\mu)\Phi(z)+\sigma\varphi(z)\bigr]\\ &=\mu+\sigma z-\sigma z-3\sigma\varphi(z_{1/3})\\ &=\mu-3\sigma\varphi(z_{1/3})\\ &=10{,}000-3(5000)(0.36360)=4546.0\ \text{dollars}. \end{aligned} \end{equation*}

(a) Forest \(A\); the dog is still lost with probability \(0.65\).

Write \(L_{d}\) for the event that the dog is still lost at the end of day \(d\) and carry the joint probabilities \(\pi_{d}(F)=\Pr(\text{dog in }F,\,L_{d})\) rather than the posteriors, the normalization cancelling in every comparison: \(\pi_{0}(A)=0.7\), \(\pi_{0}(B)=0.3\), with per-day success \(q_{A}=0.5\), \(q_{B}=0.8\). Searching \(F\) finds the dog with probability \(\pi(F)q_{F}\), so on day 1

\begin{equation*} \pi_{0}(A)q_{A}=0.7(0.5)=0.35 \quad\text{versus}\quad \pi_{0}(B)q_{B}=0.3(0.8)=0.24 , \end{equation*}

and Oscar searches \(A\). Then \(\Pr(L_{1})=1-0.35=0.65\), with \(\pi_{1}(A)=0.7(0.5)=0.35\) and \(\pi_{1}(B)=0.3\) unchanged.

(b) Forest \(B\); \(\Pr(L_{2})=0.41\).

By Bayes’ rule the posterior after the failed search of \(A\) is

\begin{equation*} \Pr(A\mid L_{1})=\frac{0.35}{0.65}=\frac{7}{13}=0.538, \qquad \Pr(B\mid L_{1})=\frac{0.30}{0.65}=\frac{6}{13}=0.462 , \end{equation*}

so the day-2 success probabilities are \(\tfrac{7}{13}(0.5)=0.269\) for \(A\) against \(\tfrac{6}{13}(0.8)=0.369\) for \(B\): search \(B\). Unconditionally, \(\pi_{2}(A)=0.35\) and \(\pi_{2}(B)=0.3(0.2)=0.06\), so

\begin{equation*} \Pr(L_{2})=0.35+0.06=0.41 . \end{equation*}

(c) Forest \(A\); \(\Pr(L_{3})=0.235\).

Now \(\Pr(A\mid L_{2})=0.35/0.41=0.854\) and \(\Pr(B\mid L_{2})=0.06/0.41=0.146\), giving \(0.854(0.5)=0.427\) for \(A\) against \(0.146(0.8)=0.117\) for \(B\): search \(A\). Then \(\pi_{3}(A)=0.175\), \(\pi_{3}(B)=0.06\), and

\begin{equation*} \Pr(L_{3})=0.175+0.06=0.235 . \end{equation*}

(d) The policy \((A,B,A)\) of parts (a)-(c) induces the day-of-discovery distribution obtained by differencing \(1,\Pr(L_{1}),\Pr(L_{2}),\Pr(L_{3})\):

i. Hence

\begin{equation*} \begin{aligned} \mathrm{E}[\text{payoff}]&=-1(0.350)-2(0.240)-3(0.175)-10(0.235)\\ &=-0.350-0.480-0.525-2.350=-3.705 . \end{aligned} \end{equation*}

Searching stops only on success, so a policy here is just a sequence of forests, and enumerating all \(2^{3}=8\) of them confirms that the myopic path \(ABA\) is optimal for this payoff as well; the nearest rivals are \(AAB\) at \(-3.770\) and \(BAA\) at \(-3.815\).

ii. Told the dog is in \(A\), Oscar searches \(A\) all three days and each day succeeds independently with probability \(0.5\), so the day of discovery is geometric truncated at 3:

\begin{equation*} \begin{aligned} \mathrm{E}[\text{payoff}\mid A]&=-1(0.5)-2(0.25)-3(0.125)-10(0.125)\\ &=-2.625 . \end{aligned} \end{equation*}

iii. Told the dog is in \(B\), he searches \(B\) throughout with per-day success \(0.8\):

\begin{equation*} \begin{aligned} \mathrm{E}[\text{payoff}\mid B]&=-1(0.8)-2(0.16)-3(0.032)-10(0.008)\\ &=-1.296 . \end{aligned} \end{equation*}

iv. Before the search, the informant’s report is \(A\) with probability \(0.7\) and \(B\) with probability \(0.3\), so the expected payoff under perfect information is

\begin{equation*} 0.7(-2.625)+0.3(-1.296)=-2.2263 , \end{equation*}

and the expected value of the information is

\begin{equation*} -2.2263-(-3.705)=1.4787 . \end{equation*}

Oscar should be willing to pay up to \(1.48\) payoff units for the report.

(a) The decision is the size of a follow-up bioassay: run \(m\) additional rats at each of the four doses of Table 3.1, \(m\in\{0,2,4,\dots,40\}\), at a setup cost of $2000 (waived if \(m=0\)) plus $50 per rat. The first three steps of BDA3 Section 9.1 then read as follows, the fourth being part (b).

Decisions. \(d=m\), with \(N=4m\) the number of new animals.

Outcomes. The model of Section 3.7: \(y_{i}\sim\mathrm{Bin}(n_{i},\theta_{i})\) with \(\mathrm{logit}\,\theta_{i}=\alpha+\beta x_{i}\) and a uniform prior on \((\alpha,\beta)\), the animals being exchangeable within a dose group and the new ones exchangeable with the old. The new counts \(\tilde y_{i}\sim\mathrm{Bin}(m,\theta_{i})\) have as their predictive distribution the posterior predictive \(p(\tilde y\mid y)\) from the current \(20\) rats.

Utility. The scientific goal of Section 3.7 is the LD50, \(\lambda=-\alpha/\beta\) (defined, as there, conditional on \(\beta>0\), which carries posterior probability \(0.99999\) on the grid below). A final point estimate \(\hat\lambda\) incurs a dollar loss \(K(\hat\lambda-\lambda)^{2}\) with \(K=10^{6}\) dollars per squared log-dose unit, so with the optimal estimate (the posterior mean) the expected terminal loss is \(K\,\mathrm{var}(\lambda\mid y,\tilde y)\). Total utility:

\begin{equation*} U(m,\tilde y)=-K\,\mathrm{var}(\lambda\mid y,\tilde y)-\bigl(2000+50N\bigr)1\{m>0\} . \end{equation*}

(b) Averaging over the predictive distribution of \(\tilde y\) gives the preposterior expected utility

\begin{equation*} \begin{aligned} \mathrm{E}[U(m)]=-K\,\mathrm{E}_{\tilde y\mid y} \bigl[\mathrm{var}(\lambda\mid y,\tilde y)\bigr] -\bigl(2000+50N\bigr)1\{m>0\} . \end{aligned} \end{equation*}

The inner posterior variances are computed on the Section 3.7 grid, \(\alpha\in[-5,10]\), \(\beta\in[-10,40]\), \(301\times401\) points, and the outer expectation by \(2500\) draws \((\alpha,\beta)\sim p(\alpha,\beta\mid y)\) with \(\tilde y\) simulated from each; the current posterior has \(\mathrm{E}[\lambda\mid y]=-0.107\) and \(\mathrm{var}(\lambda\mid y)=0.00896\) (posterior sd \(0.095\)), matching Figure 3.4.

(Monte Carlo standard errors on the variances are at most \(0.00017\), that is $170 on the utility scale.) The maximum is at \(m^{*}=10\), a follow-up of \(N=40\) rats, with \(\mathrm{E}[U]=-7185\) dollars against \(-8964\) for doing nothing; the utility is flat across \(N=32\) to \(48\), whose entries sit within one simulation standard error of each other, so what is certified is a follow-up of forty-odd rats worth about $1800 more than doing nothing.

The shape of the table is explained by the one-line approximation

\begin{equation*} \mathrm{E}\bigl[\mathrm{var}(\lambda\mid y,\tilde y)\bigr]\;\approx\; \mathrm{var}(\lambda\mid y)\,\frac{n_{0}}{n_{0}+N}, \qquad n_{0}=20 , \end{equation*}

accurate to within 15% across the whole table – the new rats buy precision much as if they were prior sample size. Maximizing \(-Kv_{0}n_{0}/(n_{0}+N)-c_{0}-c_{1}N\) over \(N\) gives the closed form

\begin{equation*} N^{*}=\sqrt{\frac{K v_{0} n_{0}}{c_{1}}}-n_{0} =\sqrt{\frac{10^{6}(0.00896)(20)}{50}}-20=39.9 , \end{equation*}

in agreement with the tabulated \(N^{*}=40\).

(c) Sensitivity. The qualitative conclusion depends almost entirely on \(K\), the dollar value placed on precision, and the entry cost \(c_{0}\); the optimal size scales as \(K^{1/2}\) by the formula above. Entries are the optimal \(N^{*}\) and the net gain in dollars over no experiment, the base case being \(c_{0}=2000\), \(c_{1}=50\):

Halving \(K\) to $500{,}000 reverses the decision – the $2000 setup cost then exceeds the value of any attainable variance reduction, even though a 24-rat follow-up would still cut the LD50 variance by more than half. Raising \(c_{0}\) to $5000 likewise kills the experiment at \(K=10^{6}\) while leaving \(N^{*}\) unchanged at \(K=2\times10^{6}\): a fixed cost affects whether to experiment, never how large. The marginal cost, by contrast, leaves the decision to experiment intact at \(K=10^{6}\) but not its scale, \(N^{*}\) running from \(64\) down to \(24\) as \(c_{1}\) goes from $25 to $100.

The allocation matters less than the size. Holding \(N=40\) fixed, two doses bracketing the current LD50 estimate, 20 rats each at \(-0.30\) and \(-0.05\), cut \(\mathrm{E}[\mathrm{var}(\lambda\mid y,\tilde y)]\) by 13% relative to 10 at each of the four doses of Table 3.1 – worth $400 at \(K=10^{6}\), less than is gained by choosing \(m\) well. Putting all 40 at the single dose \(-0.107\) is no better than the original design, a single dose being unable to identify the slope, and 20 each at the extreme doses \(-0.86\) and \(0.73\) more than doubles the residual variance: those doses are nearly saturated and so say little about where the curve crosses one-half.

The most consequential assumption is the shape of the terminal loss, not any of its constants. Since \(\lambda=-\alpha/\beta\) has Cauchy-like tails as \(\beta\to 0\), \(\mathrm{var}(\lambda\mid\cdot)\) is driven by the rare simulated datasets still consistent with a flat dose-response curve, and quadratic loss pays heavily to remove them. Replacing it by a loss linear in the posterior sd, calibrated to agree at \(m=0\) (that is, \(K^{\prime}=94{,}679\) dollars per log-dose unit of sd), leaves the best interior design essentially where it was, \(N=32\) rats with \(\mathrm{E}[\mathrm{sd}(\lambda\mid y,\tilde y)]\approx 0.057\), but brings its expected utility to within a few tens of dollars of the \(-8964\) of doing nothing – a gap smaller than the simulation error. The $1800 margin is a feature of the quadratic loss, and the recommendation to run the follow-up should be reported as resting on it.

\(n \approx 714\).

(a) For \(n\) independent draws from a density \(f\) with distribution function \(F\), the sample \(p\)-quantile \(\hat q_p\) has the standard asymptotic normal limit

\begin{equation*} \mathrm{sd}(\hat q_p) \;\approx\; \frac{1}{f(q_p)}\sqrt{\frac{p(1-p)}{n}}, \qquad q_p=F^{-1}(p). \end{equation*}

Write \(\sigma=\mathrm{sd}(\theta|y)\). Normality gives \(q_{0.025}=\mu-1.96\sigma\) and \(f(q_{0.025})=\phi(1.96)/\sigma\) with \(\phi(1.96)=0.05845\), and by symmetry the same at \(p=0.975\). Hence

\begin{equation*} \begin{aligned} \mathrm{sd}(\hat q_{0.025}) &\approx \frac{\sigma}{\phi(1.96)}\sqrt{\frac{(0.025)(0.975)}{n}}\\[2pt] &= \frac{0.15613}{0.05845}\,\frac{\sigma}{\sqrt n} \;=\; \frac{2.671\,\sigma}{\sqrt n}. \end{aligned} \end{equation*}

Setting \(2.671/\sqrt n = 0.1\) gives

\begin{equation*} n \;=\; (26.71)^2 \;=\; 713.6, \end{equation*}

so about \(n=714\) draws (in practice one would round up to 1000).

(b) Taking \(\mu=0,\sigma=1\) without loss of generality and repeating the experiment 20,000 times at each \(n\), the Monte Carlo standard deviations of the two estimated quantiles are

against the predicted \(2.671/\sqrt n=0.134,\,0.100,\,0.084\). The asymptotic formula is very slightly conservative at these sample sizes; at \(n=714\) it predicts 0.100 against the observed 0.098.

(a) \(\mathrm{sd}(\bar\theta_1)=\sigma/\sqrt S=4/\sqrt{100}=0.4\).

(b) \(4/\sqrt S=0.1\) gives \(S=1600\).

(c) By the quantile formula derived in Exercise 10.1, at \(p=0.025\) and \(p=0.975\) for a normal posterior,

\begin{equation*} \mathrm{sd}(\hat q)\;\approx\;\frac{2.671\,\sigma}{\sqrt S} \;=\;\frac{2.671\cdot 4}{\sqrt{100}}\;=\;1.07 , \end{equation*}

the same for both endpoints by symmetry: the nominal interval \([0.2,\,15.8]\) is pinned down only to about \(\pm 1\) at each end.

(d) \(2.671\cdot 4/\sqrt S = 0.1\) gives \(S=(106.9)^2\approx 11{,}400\), i.e. of order \(10^4\) draws — seven times more than the 1600 needed for the mean.

(e) Table 5.3 gives school A a 95% interval of \([-2,31]\), so \(\sigma_A\approx 33/(2\cdot 1.96)=8.4\). With \(S=200\),

\begin{equation*} \mathrm{sd}(\hat q_{0.025})\approx\mathrm{sd}(\hat q_{0.975}) \approx\frac{2.671\cdot 8.4}{\sqrt{200}}\;=\;1.6 . \end{equation*}

(The posterior for \(\theta_1\) is right-skewed, so 1.6 understates the noise at the upper end somewhat and overstates it at the lower end.)

(f) Because 1.6 is negligible next to the posterior standard deviation 8.4 that the interval is reporting. Section 10.5 confirms it: \(S=10{,}000\) draws give the 95% interval \([-2,31]\) and the 50% interval \([6,15]\), against \([-2,31]\) and \([7,16]\) from 200 draws.

Beta-binomial conjugacy (Section 2.4) makes the posteriors independent betas: with priors \(p_i\sim\mathrm{Beta}(\alpha_i,\beta_i)\),

\begin{equation*} p_1|y\sim\mathrm{Beta}(\alpha_1+6,\;\beta_1+4),\qquad p_2|y\sim\mathrm{Beta}(\alpha_2+10,\;\beta_2+10), \end{equation*}

and \(p_1\perp p_2\) given \(y\). (The Haldane choice \(\alpha_i=\beta_i=0\) is an improper prior; the posterior is still proper here because \(0<y_i<n_i\) for both groups.) The \(\mathrm{E}(d|y)\) column below is exact, \(\frac{\alpha_1+6}{\alpha_1+\beta_1+10}-\frac{\alpha_2+10}{\alpha_2+\beta_2+20}\).

(a) Drawing \(S=200{,}000\) independent pairs \((p_1^s,p_2^s)\) from these two betas and forming \(d^s=p_1^s-p_2^s\) gives, for four common prior choices (used for both \(p_1\) and \(p_2\)):

The simulation standard error on each probability is \(\sqrt{(0.69)(0.31)/S}=0.001\), and on the interval endpoints (Exercise 10.1) about \(2.671\,\mathrm{sd}(d|y)/\sqrt S\approx 0.001\).

(b) Conditioning on \(p_1\) and using independence,

\begin{equation*} \Pr(p_1>p_2|y)=\int_0^1 \mathrm{Beta}(p\,|\,\alpha_1{+}6,\beta_1{+}4)\; I_{p}(\alpha_2{+}10,\beta_2{+}10)\,dp , \end{equation*}

where \(I_p(a,b)\) is the incomplete beta (the \(\mathrm{Beta}(a,b)\) cdf). Adaptive quadrature on \([0,1]\) evaluates this to an absolute error below \(10^{-9}\):

\begin{equation*} \begin{aligned} \mathrm{Beta}(0,0):&\;0.70725, &\quad \mathrm{Beta}(0.5,0.5):&\;0.69606,\\ \mathrm{Beta}(1,1):&\;0.68638, &\quad \mathrm{Beta}(2,2):&\;0.67044,\\ \mathrm{Beta}(10,10):&\;0.61051. \end{aligned} \end{equation*}

Each agrees with the corresponding simulation value to within one simulation standard error, and the answer is insensitive to the prior over the whole weakly informative range (0.67 to 0.71), only the 20-pseudo-observation \(\mathrm{Beta}(10,10)\) pulling it appreciably toward \(1/2\).

(a) Run one round of the algorithm: draw \(\theta\sim g/c\) and, independently, \(U\sim\mathrm{U}(0,1)\), and accept iff \(U\le p(\theta|y)/(Mg(\theta))\) (a legitimate rule since the bound \(M\) makes the right side lie in \([0,1]\)). Then for any set \(A\),

\begin{equation*} \begin{aligned} \Pr(\theta\in A,\ \text{accept}) &=\int_A \frac{g(\theta)}{c}\cdot\frac{p(\theta|y)}{Mg(\theta)}\,d\theta\\[2pt] &=\frac{1}{cM}\int_A p(\theta|y)\,d\theta . \end{aligned} \end{equation*}

Taking \(A\) to be the whole parameter space gives \(\Pr(\text{accept})=1/(cM)\), which is positive, so the algorithm terminates with probability 1 (the number of rounds is geometric with mean \(cM\)). Dividing,

\begin{equation*} \Pr(\theta\in A\mid \text{accept})=\int_A p(\theta|y)\,d\theta , \end{equation*}

and since the rounds are i.i.d. this is exactly the law of the first accepted draw. Hence the output has density \(p(\theta|y)\).

(b) Because the number in step 2 must be a probability. (The exercise writes \(q(\theta)\) for the approximating density that Section 10.3 calls \(g(\theta)\); \(q\) elsewhere in the chapter denotes an unnormalized target.) If \(p(\theta|y)/g(\theta)\) is unbounded then no finite \(M\) exists and the algorithm is undefined; if one nevertheless uses a finite \(M\) smaller than the supremum, then the acceptance rule effectively becomes \(\min\{p(\theta|y)/(Mg(\theta)),1\}\) and the display in (a) gives accepted draws with density

\begin{equation*} \propto\ \min\{p(\theta|y),\,Mg(\theta)\}\ \neq\ p(\theta|y), \end{equation*}

which caps the target on the region \(\{p>Mg\}\) — exactly where \(g\)’s tails are too light — and so silently under-represents it there.

(a) Drawing \((\alpha,\beta)\) from the prior gave \(\alpha=-0.933\), \(\beta=1.325\); the \(x_j\), \(n_j\) and resulting \(y_j\) are

The unnormalized log posterior is

\begin{equation*} \begin{aligned} \log q(\alpha,\beta|y) ={}&\sum_{j=1}^{10}\Big[y_j\eta_j-n_j\log\big(1+e^{\eta_j}\big)\Big]\\ &+\log t_4(\alpha;0,2^2)+\log t_4(\beta;0,1), \end{aligned} \qquad \eta_j=\alpha+\beta x_j . \end{equation*}

(b) Take \(g=\) the prior itself. Then the importance ratio is exactly the likelihood,

\begin{equation*} \frac{q(\alpha,\beta|y)}{g(\alpha,\beta)}=\prod_j p(y_j|\alpha,\beta) \ \le\ M:=e^{\ell_{\max}}, \end{equation*}

a bound available in closed form from the maximized log likelihood, here \(\ell_{\max}=-35.533\) at the MLE \((\hat\alpha,\hat\beta)=(-1.441,\,2.191)\). So: draw \(\alpha\sim t_4(0,4)\) and \(\beta\sim t_4(0,1)\), accept with probability \(\exp(\ell(\alpha,\beta)-\ell_{\max})\). This is a legitimate rejection sampler by Exercise 10.4(a), and it needs no tuning; its acceptance rate is \(p(y)/e^{\ell_{\max}}\), observed to be \(2.87\%\) (about 35 proposals per accepted draw). The 1000 accepted draws give

A reference run of the same sampler with 14,359 accepted draws gives \(\mathrm{E}(\alpha|y)=-0.920\), \(\mathrm{E}(\beta|y)=1.090\), \(\mathrm{sd}=0.475\) and \(0.854\), so the 1000-draw answers are correct to within their Monte Carlo errors of \(0.47/\sqrt{1000}=0.015\) and \(0.027\).

(c) Maximizing \(\log q\) numerically and inverting minus the Hessian there (the normal approximation of Section 4.1) gives

\begin{equation*} \hat\theta=\begin{pmatrix}-0.829\\ \phantom{-}0.911\end{pmatrix}, \qquad V=\big[-\nabla^2\log q(\hat\theta|y)\big]^{-1} =\begin{pmatrix}0.200 & -0.281\\ -0.281 & 0.628\end{pmatrix}, \end{equation*}

i.e. marginal standard deviations 0.447 and 0.792 with correlation \(-0.79\) — the usual strong negative dependence between intercept and slope when the \(x_j\) are not centered.

(d) Drawing \(S=1000\) values from \(t_4(\hat\theta,V)\) and weighting by \(w^s=q(\theta^s|y)/g(\theta^s)\) as in (10.3),

\begin{equation*} \hat{\mathrm{E}}(\alpha|y)=-0.908,\qquad \hat{\mathrm{E}}(\beta|y)=1.078, \end{equation*}

against the reference values \(-0.920\) and \(1.090\); the weighted standard deviations are 0.487 and 0.880. Across ten independent repetitions of this 1000-draw run the two estimates have Monte Carlo standard deviations 0.017 and 0.031, so both agree with the reference.

(e) The largest normalized weight is only 0.0018, and (10.4) gives

\begin{equation*} S_{\mathrm{eff}}=\frac{1}{\sum_s(\tilde w^s)^2}=898 \end{equation*}

out of \(S=1000\) (the ten repetitions range from 882 to 903): the \(t_4\) approximation covers the posterior so well that importance sampling costs almost nothing in efficiency, in contrast to the \(2.87\%\) acceptance rate of the prior-based rejection sampler in (b).

The proposal is \(g=t_3(0,4/3)\) and the weights are well behaved: \(S_{\mathrm{eff}}/S=87\%\). Take \(p(\theta|y)=\mathrm{N}(0,1)\) without loss of generality; matching curvature at the common mode \(0\), for \(t_\nu(0,s^2)\),

\begin{equation*} \frac{d^2}{d\theta^2}\log t_\nu\Big|_{0} =-\frac{\nu+1}{\nu s^2} \;\overset{!}{=}\;-1 \quad\Longrightarrow\quad s^2=\frac{\nu+1}{\nu}=\frac43 \end{equation*}

for \(\nu=3\). So \(g=t_3(0,4/3)\), which has \(\mathrm{sd}=\sqrt{3s^2}=2\) — twice the target’s, and with polynomial tails that dominate the normal’s everywhere far out. The importance ratios are

\begin{equation*} w(\theta)=\frac{\mathrm{N}(\theta\,|\,0,1)}{t_3(\theta\,|\,0,4/3)} . \end{equation*}

(a) With \(S=100\) the log ratios spread over \([-63.5,\,0.23]\), but the histogram is sharply concentrated at the top of that range with a long left tail: the extreme values are the very small weights produced by the occasional draw from the \(t_3\) tail, where the normal density is negligible. (Figure bda3-ch10-is-logweights-normal-t3.) The weights are bounded above by \(\sup_\theta w(\theta)=1.253\) (attained at \(\theta=0\), since the normal tail vanishes faster than the \(t_3\) tail), and

\begin{equation*} \mathrm{E}_g[w^2]=\int \frac{p(\theta)^2}{g(\theta)}\,d\theta = 1.1493<\infty , \end{equation*}

so no single draw can dominate; here the largest normalized weight is 0.012.

(b) With that sample, \(\hat{\mathrm{E}}(\theta|y)=-0.072\) and \(\hat{\mathrm{var}}(\theta|y)=1.042\), against the true values \(0\) and \(1\). Over 1000 independent replications of the \(S=100\) experiment, the mean estimate is \(-0.007\) (sd 0.095) and \(0.992\) (sd 0.114): both estimators are essentially unbiased with the \(O(S^{-1/2})\) error one expects.

(c) With \(S=10{,}000\): \(\hat{\mathrm{E}}(\theta|y)=0.005\) and \(\hat{\mathrm{var}}(\theta|y)=1.020\), errors down by the expected factor of ten. The log-ratio histogram has the same shape with a longer left tail (minimum \(-518\)), which is irrelevant — as Section 10.4 notes, small importance ratios have little influence on (10.3).

(d) The largest normalized weight in the \(S=10{,}000\) sample is \(1.1\times10^{-4}\), and

\begin{equation*} S_{\mathrm{eff}}=\frac{1}{\sum_{s}(\tilde w^s)^2}=8704, \end{equation*}

i.e. 87% efficiency, matching the limit \(S/\mathrm{E}_g[w^2]=10{,}000/1.1493=8701\) predicted by the finite weight variance found in (a).

The estimates are too low because half of a \(t_3\)’s variance sits in the tails \(|\theta|>4\), where the matched normal proposal essentially never puts a draw; quantified at the end.

Matching curvature at \(0\) as in Exercise 10.6, now in the other direction: \(-1/\sigma^2=-(\nu+1)/\nu=-4/3\), so \(g=\mathrm{N}(0,3/4)\) and

\begin{equation*} w(\theta)\;\propto\;\Big(1+\tfrac{\theta^2}{3}\Big)^{-2}e^{2\theta^2/3}, \end{equation*}

which is unbounded: it grows like \(e^{2\theta^2/3}/\theta^4\). The true values are \(\mathrm{E}(\theta|y)=0\) and \(\mathrm{var}(\theta|y)=\nu/(\nu-2)=3\).

(a) With \(S=100\) the log ratios run over \([-0.23,\,2.65]\), now with a long right tail — the dangerous direction; the minimum \(\log w(0)=-0.23\) is attained at the mode. (Figure bda3-ch10-is-logweights-t3-normal.) The largest normalized weight is 0.13: a single draw carries an eighth of the estimate.

(b) \(\hat{\mathrm{E}}(\theta|y)=0.115\), \(\hat{\mathrm{var}}(\theta|y)=2.19\) against the true 0 and 3.

(c) With \(S=10{,}000\): \(\hat{\mathrm{E}}(\theta|y)=-0.105\), \(\hat{\mathrm{var}}(\theta|y)=1.58\). Over 1000 replications the variance estimate averages 1.26 at \(S=100\) and 1.66 at \(S=10{,}000\), with medians 1.11 and 1.50, and 97–98% of replications fall below the true value 3 at both sample sizes.

(d) Largest normalized weight \(0.012\) and

\begin{equation*} S_{\mathrm{eff}}=\frac{1}{\sum_s(\tilde w^s)^2}=2529 \end{equation*}

out of 10,000. The number is worth little: across replications it swings between about 700 and 6000, and \(S_{\mathrm{eff}}\) is only meaningful when the weights have finite variance (Section 10.4), which here they do not —

\begin{equation*} \mathrm{E}_g[w^2]=\int \frac{p(\theta)^2}{g(\theta)}\,d\theta=\infty , \end{equation*}

since \(p^2/g\) diverges like \(e^{2\theta^2/3}/\theta^8\).

Why the systematic underestimate: split \(\mathrm{var}(\theta|y)=\int\theta^2 p(\theta)\,d\theta=3\) at \(|\theta|=4\),

\begin{equation*} \int_{|\theta|>4}\theta^2\,t_3(\theta)\,d\theta = 1.477 \qquad(49\%\ \text{of the total}), \end{equation*}

while

\begin{equation*} \Pr_g(|\theta|>4)=2\,\Phi\!\left(\frac{-4}{\sqrt{3/4}}\right)=3.9\times10^{-6}, \end{equation*}

so even \(S=10{,}000\) draws from \(g\) yield an expected \(0.04\) draws there. In almost every run that region contributes nothing and the estimate recovers only the central half of the variance; in the rare run that does land a draw there, the enormous weight \(w\) makes the estimate leap upward. The estimator is consistent — \(\mathrm{E}_g[w\theta^2]=3<\infty\), so the ratio (10.3) converges to 3 — but its sampling distribution is so right-skewed that almost all realizations, and the replication mean itself, sit far below 3. Increasing \(S\) barely helps: with \(\mathrm{E}_g[w^2]=\infty\) there is no \(\sqrt S\) central limit theorem, and a hundredfold increase in \(S\) moved the mean estimate only from 1.26 to 1.66.

Importance resampling recovers most of the skew that the normal approximation misses: the \(k=1000\) resampled draws have mean \((\alpha,\beta) = (1.22,10.62)\) without replacement and \((1.30,11.43)\) with, against the exact posterior mean \((1.31,11.64)\) and the starting distribution’s \((0.85,7.75)\).

The starting density is the normal approximation (4.2), \(g = \mathrm{N}(\hat\theta,[I(\hat\theta)]^{-1})\), at the mode of the Section 3.7 posterior:

\begin{equation*} \hat\theta = \begin{pmatrix} 0.847 \\ 7.749 \end{pmatrix}, \qquad [I(\hat\theta)]^{-1} = \begin{pmatrix} 1.039 & 3.547 \\ 3.547 & 23.752 \end{pmatrix}, \end{equation*}

i.e. standard deviations \(1.02\) and \(4.87\) with correlation \(0.71\), reproducing the \((0.8,7.7)\) with standard errors \(1.0\) and \(4.9\) of page 76. With \(p(\cdot \mid y)\) the unnormalized posterior, put

\begin{equation*} w(\theta^s) = \frac{p(\theta^s \mid y)}{g(\theta^s)}, \qquad \tilde w(\theta^s) = w(\theta^s) \Big/ \sum_{s^{\prime}=1}^{S} w(\theta^{s^{\prime}}), \end{equation*}

draw \(S = 10{,}000\) values from \(g\), and resample \(k = 1000\) of them with probabilities \(\tilde w\) by the three-step recipe of Section 10.4. The standard of comparison is the exact posterior on a fine grid over \([-5,12] \times [-10,60]\); every simulation summary below is a mean over 500 independent replications of the whole procedure, with Monte Carlo standard error under \(0.1\) on each entry of the table in (a).

(a) The resample matches Figure 3.3b in location, in the positive \((\alpha,\beta)\) ridge, and in the hard lower boundary at \(\beta \approx 0\):

Two discrepancies remain. First, \(\Pr(\beta > 0)\) is repaired exactly: the \(5.6\%\) of proposal draws with \(\beta \le 0\) carry total normalized weight \(4 \times 10^{-6}\), so none is ever resampled, and the LD50 becomes well behaved (central \(95\%\) interval \([-0.28,0.11]\) against the exact \([-0.28,0.10]\), versus the range \([-12.4,5.4]\) spanned by the \(950\) draws with \(\beta>0\) from the normal approximation, page 87). Second, the extreme upper-right plume is thinner than in Figure 3.3b: \(g\) supplies almost no draws with \(\beta > 30\), and resampling can only reweight the points it is given, never create new ones. Hence \(\mathrm{sd}(\beta) = 4.62\) against the exact \(5.77\), and the \(97.5\%\) point of \(\beta\) falls short by \(4.7\).

(b) The log ratios are unimodal and extremely left-skewed, with a short but consequential right tail. Centring each replication at its maximum, the averaged percentiles of \(\log w\) are

The interquartile range is only \(0.38\) on the log scale, so the bulk of the ratios is nearly constant — \(g\) is a good approximation over the core. The huge left tail is harmless (page 266: small ratios have little influence on (10.2)); what matters is that the largest weight is typically \(95\) times the mean and carries about \(1\%\) of the total, the ten largest \(3.7\%\) and the hundred largest \(10\%\). By (10.4),

\begin{equation*} S_{\mathrm{eff}} = \frac{1}{\sum_{s=1}^{S} (\tilde w(\theta^s))^2}, \end{equation*}

with median \(2730\) over the replications but a \(5\%\)–\(95\%\) range of \([280,4490]\): the estimate is itself very noisy, exactly the caveat issued on page 266, because whether one draw lands far out in the skewed plume changes \(\max_s \tilde w\) by an order of magnitude. A generalized Pareto fit to the largest \(3\sqrt{S} = 300\) ratios gives shape \(\hat k \approx 0.64\) on average, exceeding \(0.7\) in \(27\%\) of replications — a variance finite but only just, since \(g\) has lighter tails than the posterior in the upper-right direction. As \(k = 1000\) sits below the typical \(S_{\mathrm{eff}}\), the resample is usable but not comfortably so.

(c) With replacement the resample is distributionally better and cosmetically worse. Better, because it is an honest draw from the discrete distribution \(\{\theta^s,\tilde w(\theta^s)\}\): its means \((1.30,11.43)\) reproduce the importance-sampling estimate (10.2) to two decimals, and \(\mathrm{sd}(\beta) = 5.33\) and the \(97.5\%\) point \(23.6\) are markedly closer to the exact \(5.77\) and \(25.4\). Worse, because the 1000 draws contain on average only \(890\) distinct values, with median largest multiplicity \(10\) and above \(56\) in the worst \(5\%\) of replications.

The mechanism is the exclusion in step 2 of Section 10.4: without replacement every draw is capped at one copy, whereas its fair share is \(k\tilde w(\theta^s)\). On average \(24\) draws have \(k\tilde w(\theta^s) > 1\), carrying \(7\%\) of the total weight, and these are precisely the points in the upper-right plume; truncating them pulls the resample back toward \(g\), whence the systematically low \(\mathrm{E}(\beta)\), \(\mathrm{sd}(\beta)\) and upper quantile in column three above. The footnote on page 266 grants without-replacement only “a more desirable intermediate approximation somewhere between the starting and target densities,” which is a virtue only if one wants \(k\) distinct points; for posterior summaries, with replacement wins, as the P.S. on that page now recommends.

The chain satisfies detailed balance with respect to \(p(\omega|y)\), and detailed balance implies stationarity.

From \(\omega^{t-1}=\omega_a\) one draws \(\omega^{*}\sim J_t(\cdot|\omega_a)\) and accepts it with probability \(\min(1,r)\), \(r\) being the ratio (11.2); so for \(\omega_b\neq\omega_a\) the transition density is

\begin{equation*} K_t(\omega_b|\omega_a)=J_t(\omega_b|\omega_a)\,\min\bigl(1,r(\omega_b,\omega_a)\bigr), \qquad r(\omega_b,\omega_a)=\frac{p(\omega_b|y)J_t(\omega_a|\omega_b)} {p(\omega_a|y)J_t(\omega_b|\omega_a)}, \end{equation*}

the remaining mass sitting as an atom at \(\omega_a\). (\(r\) is always defined: the jump can occur only if \(p(\omega_a|y)\) and \(J_t(\omega_b|\omega_a)\) are both nonzero.)

Label any two points, as on page 280, so that \(p(\omega_b|y)J_t(\omega_a|\omega_b)\ge p(\omega_a|y)J_t(\omega_b|\omega_a)\), i.e. \(r(\omega_b,\omega_a)\ge 1\ge r(\omega_a,\omega_b)=1/r(\omega_b,\omega_a)\). Then

\begin{equation*} \begin{aligned} p(\omega_a|y)K_t(\omega_b|\omega_a) &=p(\omega_a|y)J_t(\omega_b|\omega_a),\\ p(\omega_b|y)K_t(\omega_a|\omega_b) &=p(\omega_b|y)J_t(\omega_a|\omega_b)\,r(\omega_a,\omega_b)\\ &=p(\omega_a|y)J_t(\omega_b|\omega_a), \end{aligned} \end{equation*}

and the labelling was arbitrary, so \(p(\omega_a|y)K_t(\omega_b|\omega_a)=p(\omega_b|y)K_t(\omega_a|\omega_b)\) for every pair, trivially so on the diagonal. Integrating over \(\omega_a\),

\begin{equation*} \begin{aligned} \int p(\omega_a|y)K_t(\omega_b|\omega_a)\,d\omega_a &=p(\omega_b|y)\int K_t(\omega_a|\omega_b)\,d\omega_a\\ &=p(\omega_b|y), \end{aligned} \end{equation*}

since \(K_t(\cdot|\omega_b)\) is a probability measure once the rejection atom is counted. Hence \(\omega^{t-1}\sim p(\omega|y)\) implies \(\omega^{t}\sim p(\omega|y)\). It is the unique stationary distribution, and the chain converges to it, provided the chain is irreducible, aperiodic and not transient (page 279) — irreducibility being exactly the requirement that \(J_t\) give positive probability of eventually reaching any state from any other.

A random-walk Metropolis chain in \((\alpha,\beta)\) with normal jumps of scale \((1,4)\) reproduces the grid simulations of Section 3.7: posterior medians \(\alpha=1.26\), \(\beta=10.7\), and \(\mathrm{LD}50\) median \(-0.115\) with 95 percent interval \([-0.28,0.09]\).

Log-density. With the uniform prior, the unnormalized log posterior is, from (3.19)-(3.20),

\begin{equation*} \begin{aligned} \log p(\alpha,\beta|y) &\doteq\sum_{i=1}^{4}\Bigl[y_i\log\theta_i+(n_i-y_i)\log(1-\theta_i)\Bigr]\\ &=\sum_{i=1}^{4}\Bigl[y_i\eta_i-n_i\log\bigl(1+e^{\eta_i}\bigr)\Bigr], \qquad \eta_i=\alpha+\beta x_i, \end{aligned} \end{equation*}

the second line following from \(\log\theta_i=\eta_i-\log(1+e^{\eta_i})\) and \(\log(1-\theta_i)=-\log(1+e^{\eta_i})\). Everything is evaluated on the log scale and \(\log(1+e^{\eta})\) is computed as \(\max(\eta,0)+\log(1+e^{-|\eta|})\), so no overflow occurs for the large \(\beta\) values that the posterior reaches (page 261).

Jumping rule. Symmetric bivariate normal steps centred at the current draw,

\begin{equation*} J\bigl((\alpha^{*},\beta^{*})\,\big|\,(\alpha^{t-1},\beta^{t-1})\bigr) =\mathrm{N}\!\left(\begin{pmatrix}\alpha^{t-1}\\ \beta^{t-1}\end{pmatrix}, \begin{pmatrix}1^2 & 0\\ 0& 4^2\end{pmatrix}\right), \end{equation*}

the scales being crude posterior standard deviations read off the contour plot of Figure 3.3. Being symmetric, the ratio (11.2) reduces to (11.1),

\begin{equation*} \log r=\log p(\alpha^{*},\beta^{*}|y)-\log p(\alpha^{t-1},\beta^{t-1}|y), \end{equation*}

and \(\omega^{*}\) is accepted when \(\log u<\log r\) for \(u\sim\mathrm{U}(0,1)\).

Starting points. Five overdispersed starts, chosen to bracket the contour plot of Figure 3.3 in all directions:

\begin{equation*} (\alpha,\beta)\in\{(-3,0),\,(3,0),\,(0,30),\,(0,-5),\,(5,30)\}. \end{equation*}

Run. Five chains of 10,000 iterations, first half discarded as warm-up, then each remaining half split in two, so \(m=10\), \(n=2500\). The average acceptance rate was 0.52. Convergence diagnostics (11.4) and (11.8):

\begin{equation*} \widehat{R}_\alpha=1.005,\quad \hat n_{\mathrm{eff},\alpha}=885; \qquad \widehat{R}_\beta=1.004,\quad \hat n_{\mathrm{eff},\beta}=766, \end{equation*}

well inside the thresholds \(\widehat R<1.1\) and \(\hat n_{\mathrm{eff}}>5m=50\) of page 287.

Results (25,000 saved draws):

These match the 1000 grid draws of Figure 3.3b and the LD50 histogram of Figure 3.4 to within simulation error; the posterior correlation of \(\alpha\) and \(\beta\) is 0.65, the ridge visible in Figure 3.3a. All 25,000 draws had \(\beta>0\), as on page 77, so the LD50 summary is meaningful.

The three models differ chiefly on the seventh machine: separate says nothing at all, pooled says \(\theta_7=\mu\) with posterior 95 percent interval \([86,100]\), and hierarchical gives the honest answer \([52,134]\).

Data: \(J=6\) machines, \(n_j=5\) each, \(n=30\). Group means \(\bar y_{\cdot j}=(76.0,\,106.2,\,87.8,\,111.6,\,90.0,\,86.0)\), grand mean \(\bar y_{\cdot\cdot}=92.93\).

(A) Separate model. \(y_{ij}\sim\mathrm{N}(\theta_j,\sigma^2)\) with \(p(\theta_1,\dots,\theta_6,\log\sigma)\propto 1\). This is Section 11.6 with \(\tau=\infty\), so (11.9)-(11.11) degenerate and the Gibbs steps are

\begin{equation*} \theta_j\,|\,\sigma,y\sim\mathrm{N}\bigl(\bar y_{\cdot j},\;\sigma^2/n_j\bigr), \qquad \sigma^2\,|\,\theta,y\sim\mathrm{Inv-}\chi^2\bigl(n,\hat\sigma^2\bigr), \end{equation*}

with \(\hat\sigma^2\) as in (11.15). The seventh machine has no data and an improper flat prior, so its posterior is that same improper flat distribution: no inference is possible.

(B) Pooled model. \(y_{ij}\sim\mathrm{N}(\mu,\sigma^2)\) with \(p(\mu,\log\sigma)\propto 1\), i.e. \(\theta_1=\dots=\theta_7=\mu\). Gibbs steps

\begin{equation*} \mu\,|\,\sigma,y\sim\mathrm{N}\bigl(\bar y_{\cdot\cdot},\sigma^2/n\bigr), \qquad \sigma^2\,|\,\mu,y\sim\mathrm{Inv-}\chi^2\bigl(n,\;\tilde\sigma^2\bigr), \quad \tilde\sigma^2=\tfrac1n\sum_{i,j}(y_{ij}-\mu)^2 . \end{equation*}

(C) Hierarchical model. Exactly Section 11.6: \(y_{ij}\sim\mathrm{N}(\theta_j,\sigma^2)\), \(\theta_j\sim\mathrm{N}(\mu,\tau^2)\), \(p(\mu,\log\sigma,\tau)\propto 1\). The Gibbs sampler cycles (11.9)-(11.11) for the \(\theta_j\), (11.12)-(11.13) for \(\mu\), (11.14)-(11.15) for \(\sigma^2\), and (11.16)-(11.17) for \(\tau^2\). The seventh machine is exchangeable with the six observed ones, so \(\theta_7|\mu,\tau\sim\mathrm{N}(\mu,\tau^2)\) is drawn alongside each Gibbs iteration.

Computation. Five chains of 4000 iterations for each model, first half discarded as warm-up, starting the \(\theta_j\) at randomly chosen data points from group \(j\) and \(\mu\) at their average (page 289); \(\widehat R\le 1.01\) for every estimand, including \(\log\tau\). Posterior quantiles from the 10,000 saved draws:

Hyperparameters under the hierarchical model: \(\mu\) has median 92.9 with 95 percent interval \([77.1,108.9]\), \(\sigma\) median 14.7 \([11.2,20.4]\), \(\tau\) median 14.0 \([3.6,39.6]\).

(i) The separate model uses only machine 6’s five measurements and centres \(\theta_6\) at \(\bar y_{\cdot 6}=86.0\); the pooled model forces \(\theta_6=\mu=92.9\) and is far too confident; the hierarchical estimate 87.7 sits between them, shrunk about one quarter of the way toward \(\mu\), in line with the shrinkage factor \((\sigma^2/n_j)/(\tau^2+\sigma^2/n_j)\), which is \(0.18\) at the median \((\sigma,\tau)=(14.7,14.0)\) and larger when averaged over the small-\(\tau\) part of the posterior.

(ii) The predictive intervals for \(\tilde y_6\) are all roughly \(\pm 32\) wide because they are dominated by \(\sigma\approx 15\); the pooled one is both shifted upward and inflated, since pooling six genuinely different machines forces \(\sigma\) up to a median of 18.2.

(iii) Only the hierarchical model answers the seventh-machine question: its interval \([52,134]\) is more than four times wider than the pooled interval, because it must account for both the uncertainty in \(\mu\) and the genuine machine-to-machine spread \(\tau\).

Give each machine its own \(\sigma_j^2\) drawn from \(\mathrm{Inv-}\chi^2(\nu,\sigma_0^2)\) with \(\nu=4\) fixed; the conditionals for \(\theta\), \(\mu\), \(\tau^2\) and the \(\sigma_j^2\) all stay conjugate, and \(\sigma_0\) is drawn on a grid.

Model.

\begin{equation*} \begin{aligned} y_{ij}\,|\,\theta_j,\sigma_j^2&\sim\mathrm{N}(\theta_j,\sigma_j^2), && i=1,\dots,n_j,\ j=1,\dots,J,\\ \theta_j\,|\,\mu,\tau^2&\sim\mathrm{N}(\mu,\tau^2),\\ \sigma_j^2\,|\,\sigma_0^2&\sim\mathrm{Inv-}\chi^2(\nu,\sigma_0^2), &&\nu=4\text{ fixed}, \end{aligned} \end{equation*}

with \(p(\mu,\tau,\sigma_0)\propto 1\) (uniform on \(\tau\) and on \(\sigma_0\), following Section 11.6 and Chapter 5; a uniform prior on \(\log\tau\) would make the posterior improper).

Gibbs steps. Relative to Exercise 11.3(C), (ii) is unchanged and the rest are:

(i) \(\theta_j\), as in (11.9)-(11.11) but with \(\sigma_j\) in place of \(\sigma\):

\begin{equation*} \theta_j\,|\,\cdot\sim\mathrm{N}(\hat\theta_j,V_{\theta_j}),\quad V_{\theta_j}=\Bigl(\tfrac1{\tau^2}+\tfrac{n_j}{\sigma_j^2}\Bigr)^{-1},\quad \hat\theta_j=V_{\theta_j}\Bigl(\tfrac{\mu}{\tau^2} +\tfrac{n_j\bar y_{\cdot j}}{\sigma_j^2}\Bigr). \end{equation*}

(ii) \(\mu\,|\,\cdot\sim\mathrm{N}(\bar\theta,\tau^2/J)\) and \(\tau^2\,|\,\cdot\sim\mathrm{Inv-}\chi^2(J-1,\hat\tau^2)\), exactly (11.12)-(11.13) and (11.16)-(11.17).

(iii) \(\sigma_j^2\) is a normal variance with known mean \(\theta_j\) and a conjugate \(\mathrm{Inv-}\chi^2(\nu,\sigma_0^2)\) prior, so by the conjugacy of Section 2.6,

\begin{equation*} \sigma_j^2\,|\,\cdot\sim\mathrm{Inv-}\chi^2\!\left(\nu+n_j,\; \frac{\nu\sigma_0^2+\sum_{i}(y_{ij}-\theta_j)^2}{\nu+n_j}\right). \end{equation*}

(iv) \(\sigma_0\). Collecting the \(J\) \(\mathrm{Inv-}\chi^2(\nu,\sigma_0^2)\) densities, each proportional to \((\sigma_0^2)^{\nu/2}\exp(-\nu\sigma_0^2/2\sigma_j^2)\),

\begin{equation*} p(\sigma_0\,|\,\theta,\sigma^2,\mu,\tau,y) \;\propto\;\sigma_0^{\,J\nu} \exp\!\left(-\frac{\nu\sigma_0^2}{2}\sum_{j=1}^{J}\sigma_j^{-2}\right), \qquad \sigma_0>0 . \end{equation*}

This is evaluated on a grid \(\sigma_0\in(0.5,80]\) of 4000 points, normalized, and sampled by inverse-CDF as instructed — though with a uniform prior on \(\sigma_0\) the change of variable \(u=\sigma_0^2\) turns the display into \(u^{(J\nu+1)/2-1}e^{-au}\) with \(a=\tfrac{\nu}{2}\sum_j\sigma_j^{-2}\), so the conditional is in fact the simple form \(\sigma_0^2\,|\,\cdot\sim\mathrm{Gamma}\bigl(\tfrac{J\nu+1}{2},a\bigr)\), and grid and exact draws agree in every quantile.

Predictions. For a new measurement from machine 6, draw \(\tilde y_6\sim\mathrm{N}(\theta_6,\sigma_6^2)\). For the unobserved seventh machine, draw \(\theta_7\sim\mathrm{N}(\mu,\tau^2)\) and \(\sigma_7^2\sim\mathrm{Inv-}\chi^2(\nu,\sigma_0^2)\) at each iteration.

Results. Five chains of 8000 iterations, first half discarded; \(\widehat R\le 1.001\) throughout. Posterior quantiles from 20,000 draws:

Posterior median \(\sigma_j\) by machine: \(19.7,\,15.2,\,15.4,\,13.7,\,14.4,\,18.8\), against raw within-machine standard deviations \(19.6,\,11.8,\,12.8,\,7.6,\,10.8,\,19.2\) — substantial shrinkage toward \(\sigma_0\), as \(\nu=4\) is comparable to \(n_j-1=4\).

Machine 6 is one of the two noisy machines, so its own variance is now estimated at \(\sigma_6\approx 18.8\) rather than the pooled \(\sigma\approx 14.7\) of Exercise 11.3, and its predictive interval for \(\tilde y_6\) widens from \([56,120]\) to \([44,135]\); \(\theta_7\) is essentially unchanged, since \(\mu\) and \(\tau\) barely move.

(a) \(\widehat{\mathrm{var}}^{+}\) is exactly the average of \(\tfrac12(\psi_{ij}-\psi_{i^{\prime}j^{\prime}})^{2}\) over all \(m(m-1)n^{2}\) ordered pairs of draws lying in different sequences. (The printed \(\phi\) in both parts is a typo for \(\psi\).)

Write \(Q=\sum_{i,j}\psi_{ij}^{2}\) and \(P=\sum_j \bar\psi_{\cdot j}^{2}\). Since \(\sum_{i,j}(\psi_{ij}-\bar\psi_{\cdot j})^2=Q-nP\) and \(\sum_j(\bar\psi_{\cdot j}-\bar\psi_{\cdot\cdot})^2=P-m\bar\psi_{\cdot\cdot}^2\),

\begin{equation*} \begin{aligned} \widehat{\mathrm{var}}^{+} &=\frac{1}{mn}\bigl(Q-nP\bigr)+\frac{1}{m-1}\bigl(P-m\bar\psi_{\cdot\cdot}^{2}\bigr)\\ &=\frac{Q}{mn}+\frac{P}{m(m-1)}-\frac{m}{m-1}\,\bar\psi_{\cdot\cdot}^{2}. \end{aligned} \end{equation*}

Now let

\begin{equation*} D=\frac{1}{m(m-1)n^{2}}\sum_{j\neq j^{\prime}}\sum_{i=1}^{n}\sum_{i^{\prime}=1}^{n} \tfrac12\bigl(\psi_{ij}-\psi_{i^{\prime}j^{\prime}}\bigr)^{2}. \end{equation*}

Expanding \(\tfrac12(a-b)^2=\tfrac12a^2+\tfrac12b^2-ab\), the two square terms each contribute \(\tfrac12(m-1)n\,Q\), and the cross term is

\begin{equation*} \sum_{j\neq j^{\prime}}\bigl(n\bar\psi_{\cdot j}\bigr)\bigl(n\bar\psi_{\cdot j^{\prime}}\bigr) =n^{2}\Bigl[\bigl(\textstyle\sum_j\bar\psi_{\cdot j}\bigr)^{2}-P\Bigr] =n^{2}\bigl[m^{2}\bar\psi_{\cdot\cdot}^{2}-P\bigr], \end{equation*}

so that

\begin{equation*} D=\frac{(m-1)nQ-n^{2}m^{2}\bar\psi_{\cdot\cdot}^{2}+n^{2}P}{m(m-1)n^{2}} =\frac{Q}{mn}+\frac{P}{m(m-1)}-\frac{m}{m-1}\bar\psi_{\cdot\cdot}^{2}. \end{equation*}

Hence \(\widehat{\mathrm{var}}^{+}(\psi|y)=D\) identically, for every realization. (Check!)

If the starting distribution is the target, then every iterate is marginally distributed as \(p(\psi|y)\) (this is Exercise 11.1: the target is stationary, so stationarity propagates from \(t=0\)), and draws from different sequences are independent because the \(m\) sequences are run independently. So for \(j\neq j^{\prime}\),

\begin{equation*} \mathrm{E}\bigl[\tfrac12(\psi_{ij}-\psi_{i^{\prime}j^{\prime}})^{2}\bigr] =\tfrac12\bigl[\mathrm{var}(\psi_{ij})+\mathrm{var}(\psi_{i^{\prime}j^{\prime}})\bigr] =\mathrm{var}(\psi|y), \end{equation*}

the cross term vanishing since \(\mathrm{E}\,\psi_{ij}=\mathrm{E}\,\psi_{i^{\prime}j^{\prime}}\) and the two are independent. Averaging, \(\mathrm{E}\,\widehat{\mathrm{var}}^{+}(\psi|y)=\mathrm{E}\,D=\mathrm{var}(\psi|y)\).

(b) \(\widehat{\mathrm{var}}^{+}\to\mathrm{var}(\psi|y)\), from any starting distribution, provided (i) each sequence is an irreducible, aperiodic, Harris recurrent Markov chain with stationary distribution \(p(\omega|y)\), and (ii) \(\mathrm{E}(\psi^{2}|y)<\infty\).

These are exactly the hypotheses of the ergodic theorem applied to \(\psi\) and to \(\psi^{2}\) along each chain, so for every \(j\),

\begin{equation*} \bar\psi_{\cdot j}\to\mathrm{E}(\psi|y),\qquad s_j^{2}\to\mathrm{var}(\psi|y)\quad\text{a.s.}; \end{equation*}

Harris recurrence is what makes these limits independent of the starting distribution. Hence, splitting \(\widehat{\mathrm{var}}^{+}=\frac{n-1}{n}W+\frac{B}{n}\),

\begin{equation*} \frac{n-1}{n}W\to\mathrm{var}(\psi|y),\qquad \frac{B}{n}=\frac{1}{m-1}\sum_{j=1}^{m} \bigl(\bar\psi_{\cdot j}-\bar\psi_{\cdot\cdot}\bigr)^{2}\to 0 , \end{equation*}

the second because all \(m\) chain means share the one limit; no central limit theorem is needed for this, only the ergodic theorem. Adding, \(\widehat{\mathrm{var}}^{+}\to\mathrm{var}(\psi|y)\) and \(\widehat R\to 1\) in (11.4).

Both conditions bind. If the chain is reducible — each sequence trapped in a different mode — the \(\bar\psi_{\cdot j}\) converge to different limits, \(B\) grows linearly in \(n\), and \(B/n\) tends to a positive constant, so \(\widehat{\mathrm{var}}^{+}\) overestimates forever; that is the failure Figure 11.3a exposes. If \(\mathrm{E}(\psi^{2}|y)=\infty\), neither \(W\) nor \(B/n\) need settle.

(a) Sum the covariance matrix of one sequence and count the lags.

Assume the \(m\) sequences are independent and stationary with the target as marginal, so \(\mathrm{var}(\psi_{ij})=\mathrm{var}(\psi|y)\) and \(\mathrm{cov}(\psi_{ij},\psi_{i+t,j})=\rho_t\,\mathrm{var}(\psi|y)\) for all \(i\). Independence of the sequences gives

\begin{equation*} \mathrm{var}(\bar\psi_{\cdot\cdot}) =\mathrm{var}\Bigl(\tfrac1m\sum_{j=1}^{m}\bar\psi_{\cdot j}\Bigr) =\tfrac1m\,\mathrm{var}(\bar\psi_{\cdot 1}). \end{equation*}

Within one sequence, collecting the \(n\) diagonal terms and the \(2(n-t)\) terms at each lag \(t\),

\begin{equation*} \begin{aligned} \mathrm{var}(\bar\psi_{\cdot 1}) &=\frac{1}{n^{2}}\Bigl[n\,\mathrm{var}(\psi|y) +2\sum_{t=1}^{n-1}(n-t)\,\rho_t\,\mathrm{var}(\psi|y)\Bigr]\\ &=\frac{\mathrm{var}(\psi|y)}{n} \Bigl[1+2\sum_{t=1}^{n-1}\Bigl(1-\frac{t}{n}\Bigr)\rho_t\Bigr]. \end{aligned} \end{equation*}

Therefore

\begin{equation*} mn\,\mathrm{var}(\bar\psi_{\cdot\cdot}) =n\,\mathrm{var}(\bar\psi_{\cdot 1}) =\Bigl[1+2\sum_{t=1}^{n-1}\Bigl(1-\frac{t}{n}\Bigr)\rho_t\Bigr]\mathrm{var}(\psi|y). \end{equation*}

If \(\sum_{t\ge1}|\rho_t|<\infty\) then \(|(1-t/n)\rho_t|\le|\rho_t|\) and \((1-t/n)\rho_t\to\rho_t\) for each \(t\), so dominated convergence over the counting measure on \(t\) gives the limit (11.5). (Summability is needed: with \(\rho_t\equiv\rho>0\) the bracket grows like \(n\).)

(b) Yes on both counts, once the chains have converged: \(\hat n_{\mathrm{eff}}\) settles into a fixed proportion of the saved draws and thereafter grows linearly.

Take the bioassay Metropolis simulation of Exercise 11.2 — five chains, normal jumps of scale \((1,4)\), the same five overdispersed starting points — and run it to 20,000 iterations per chain. At each checkpoint \(N\), discard the first \(N/2\) iterations as warm-up, split the remainder in two (\(m=10\) chains of \(n=N/4\)), and evaluate \(\widehat R\) from (11.4) and \(\hat n_{\mathrm{eff}}\) from (11.7)-(11.8).

Two regimes are visible. Before convergence (\(N\le 1000\), \(\widehat R\) still above 1.05 for \(\beta\)) the estimate is erratic and even decreases — at \(N=700\) it reads 125 for \(\alpha\), at \(N=1000\) only 83 — because \(\widehat{\mathrm{var}}^{+}\) in (11.7) is then badly inflated by the starting values, the estimated \(\rho_t\) are noisy, and the truncation lag \(T\) in (11.8) is unstable. Nothing about \(\hat n_{\mathrm{eff}}\) should be trusted while \(\widehat R>1.1\).

After convergence the ratio \(\hat n_{\mathrm{eff}}/(mn)\) stabilises at about 0.05 for \(\alpha\) and 0.035 for \(\beta\) (integrated autocorrelation times roughly 20 and 30), and \(\hat n_{\mathrm{eff}}\) then grows essentially linearly in \(N\): \(741\to 976\to 1607\to 2495\) as \(N\) goes \(6000\to 10000\to 14000\to 20000\). That is what (11.5) predicts, since \(\mathrm{var}(\bar\psi_{\cdot\cdot})\) decays like \(1/(mn)\) with \(1+2\sum\rho_t\) fixed. The growth is not monotone, though: \(\hat n_{\mathrm{eff}}\) is itself a noisy estimate, because the truncation point \(T\) in (11.8) and the tail estimates \(\hat\rho_t\) both fluctuate, and repeat runs show single-checkpoint dips of 20 to 30 percent even at \(\widehat R=1.002\).

Posterior median of (8.7): 0.099 under the nonhierarchical model, 0.102 under the hierarchical model, with posterior standard deviations 0.0241 and 0.0247 — the hierarchical distribution shifted slightly up and slightly wider, as Figure 8.1 shows.

Counts. Table 8.2 reports only proportions, so the cell counts are reconstructed as \(n_j=\mathrm{round}(1447\,n_j/n)\) and \(y_{kj}=\mathrm{round}(n_j\,y_{kj}/n_j)\), adjusted by at most one in the largest cell so that each row sums to \(n_j\). This gives \(n_j\) ranging from 22 (South, I) to 182 (South, IV), totalling 1442, and raw \(\alpha_{1j}=y_{1j}/(y_{1j}+y_{2j})\) ranging from 0.33 to 0.68, against the 0.33 to 0.67 quoted on page 209 — so the reconstruction is right to within its rounding.

(a) Nonhierarchical model. Dirichlet-multinomial conjugacy (Section 3.4) gives independent exact posteriors, no MCMC needed:

\begin{equation*} (\theta_{1j},\theta_{2j},\theta_{3j})\,|\,y \sim\mathrm{Dirichlet}(y_{1j}+1,\;y_{2j}+1,\;y_{3j}+1), \qquad j=1,\dots,16 . \end{equation*}

Drawing 20,000 independent sets of 16 such vectors and forming (8.7) with \(N_j/N=n_j/n\):

\begin{equation*} \sum_{j}\frac{N_j}{N}(\theta_{1j}-\theta_{2j})\;:\quad \text{median }0.099,\ \ \mathrm{sd}\ 0.0241,\ \ 95\%\ [0.051,\,0.147]. \end{equation*}

The median sits just below the raw weighted difference \(0.1017\) because the Dirichlet\((1,1,1)\) prior adds one voter to each of the three categories in each of the 16 strata, pulling each stratum toward equal shares (page 209). Page 208 quotes 0.097 for this median; the 0.002 gap is the residue of reconstructing integer counts from two-decimal proportions, which is the accuracy limit for everything below.

(b) Hierarchical model. Write \(\theta_{1j}=\alpha_{1j}\alpha_{2j}\), \(\theta_{2j}=(1-\alpha_{1j})\alpha_{2j}\), \(\theta_{3j}=1-\alpha_{2j}\) with \(\alpha_{kj}=\mathrm{logit}^{-1}(\beta_{kj})\). The prior is placed directly on \(\beta\), so no Jacobian is needed in the likelihood, and

\begin{equation*} \begin{aligned} \log p(\beta,\mu,\tau,\rho|y)\doteq &\sum_{j=1}^{16}\Bigl[y_{1j}\log(\alpha_{1j}\alpha_{2j}) +y_{2j}\log\bigl((1-\alpha_{1j})\alpha_{2j}\bigr)\\ &\qquad\qquad+y_{3j}\log(1-\alpha_{2j})\Bigr]\\ &-16\log(\tau_1\tau_2)-8\log(1-\rho^{2}) -\tfrac12\sum_{j=1}^{16} Q_j, \end{aligned} \end{equation*}

where, with \(d_{kj}=\beta_{kj}-\mu_k\),

\begin{equation*} Q_j=\frac{1}{1-\rho^{2}}\left( \frac{d_{1j}^{2}}{\tau_1^{2}} -\frac{2\rho\,d_{1j}d_{2j}}{\tau_1\tau_2} +\frac{d_{2j}^{2}}{\tau_2^{2}}\right). \end{equation*}

Algorithm (Metropolis as a building block inside a Gibbs structure, Section 11.3). One sweep:

(i) for \(j=1,\dots,16\) and \(k=1,2\), a one-dimensional symmetric normal Metropolis jump on \(\beta_{kj}\), accepted by (11.1) on the log scale;

(ii) \(\mu\) by Gibbs — with a uniform prior, \(\mu\,|\,\beta,\tau,\rho,y\sim\mathrm{N}\bigl(\bar\beta,\Sigma/16\bigr)\), where \(\Sigma\) is the \(2\times2\) hyperparameter covariance;

(iii) five rounds of one-dimensional Metropolis jumps on \((\log\tau_1,\log\tau_2,\mathrm{arctanh}\,\rho)\); jumping on the unconstrained scale requires the Jacobian terms \(+\log\tau_1+\log\tau_2+\log(1-\rho^{2})\) in the target, which restores the intended uniform prior on \((\tau_1,\tau_2,\rho)\) itself.

The hyperparameters are updated five times per sweep because the funnel geometry near \(\tau_k=0\) is the bottleneck: with one update per sweep the chains move between the small-\(\tau\) and large-\(\tau\) regions too slowly and \(\widehat R\) for \(\log\tau_1\) stays above 1.1.

Starting distribution and run. Each chain starts at a draw from part (a): sample the 16 Dirichlet posteriors, map to \((\alpha_{1j},\alpha_{2j})\) by (8.8), set \(\beta_{kj}=\mathrm{logit}(\alpha_{kj})\), and take \(\mu\), \(\tau\), \(\rho\) as the empirical mean, standard deviations and correlation of those 16 points. Six chains, 4000 adaptive warm-up sweeps (proposal scales tuned to 0.44 acceptance, then frozen) followed by 20,000 saved sweeps. All \(\widehat R\le 1.03\) (\(\tau_1,\tau_2\) monitored on the log scale).

Results.

The posterior medians of the \(\alpha_{1j}\) run from 0.51 to 0.58, heavy shrinkage from the raw range 0.33-0.68, with \(\mathrm{logit}^{-1}(\mu_1)=0.55\), \(\mathrm{logit}^{-1}(\mu_2)=0.91\) and a negative median for \(\rho\) with enormous spread. For the finite-population estimand (8.7),

\begin{equation*} \text{median }0.102,\quad \mathrm{sd}\ 0.0247,\quad 95\%\ [0.054,\,0.151], \end{equation*}

against \(0.099\) and \(0.0241\) in part (a): higher median and slightly more variability, the two features the text attributes to Figure 8.1b. The mechanism is the one described on page 210 — strata 1 and 5, where support for Bush is lowest, have among the smallest samples (\(n_1=n_5=46\)) and so are pulled hardest toward the grand mean.

Discrepancy flag. Every hyperparameter above comes out smaller than in Table 8.3: \(\tau_1\) median 0.15 against 0.23, \(\tau_2\) 0.14 against 0.28, \(\rho\) \(-0.23\) against \(-0.44\), and correspondingly more shrinkage in the \(\alpha_{1j}\) (stratum 1 median 0.51 against 0.48, range 0.51-0.58 against 0.48-0.59); the estimand median is 0.102 against the 0.11 quoted on page 210. The mathematics favors the smaller values. Indeed the printed 0.11 is inconsistent with Table 8.3 itself: substituting that table’s own \(\alpha_{1j}\) medians and the observed \(\alpha_{2j}\) into (8.7) gives 0.099. Replacing the multinomial likelihood by its normal approximation \(\hat\beta_{1j}\sim\mathrm{N}(\beta_{1j},\,y_{1j}^{-1}+y_{2j}^{-1})\) turns the \(\beta_1\) component into the model of Section 5.4, whose marginal posterior for \(\tau_1\) is evaluated exactly on a grid by (5.21) with no MCMC at all; that grid gives quantiles \((0.01,0.07,0.135,0.21,0.37)\), matching the Metropolis output. The moment estimate \(\hat\tau_1^{2}=\mathrm{var}(\hat\beta_{1j})-\overline{v_j}=0.120-0.071\), i.e. \(\hat\tau_1=0.22\), is the likelihood-based value that Table 8.3 reports as a median; the posterior median falls below it because with only 16 strata the likelihood for \(\tau_1\) stays essentially flat down to 0, and a sampler that fails to reach small \(\tau\) — the funnel bottleneck noted above — reproduces Table 8.3 instead.

The adaptive phase converges on the jumping kernel

\begin{equation*} J(\theta^*\mid\theta^{t-1})=\mathrm{N}\!\left(\theta^{t-1},\; \begin{pmatrix} 2.54 & 7.18\\ 7.18 & 57.01\end{pmatrix}\right), \end{equation*}

which accepts \(38\%\) of jumps and reproduces the Chapter 3 grid posterior to within simulation error.

The log posterior, computed on the log scale as page 261 demands, is

\begin{equation*} \log p(\alpha,\beta\mid y) = \sum_{j=1}^{4}\Bigl[y_j\eta_j - n_j\log\bigl(1+e^{\eta_j}\bigr)\Bigr]

• \text{const},\qquad \eta_j=\alpha+\beta x_j . \end{equation*}

Adaptive phase, step 1. The posterior mode is \((\hat\alpha,\hat\beta)=(0.847,\,7.749)\) and the inverse of the negative Hessian there is

\begin{equation*} \hat\Sigma=\begin{pmatrix}1.039 & 3.546\\ 3.546 & 23.74\end{pmatrix}, \end{equation*}

so the initial jumping rule is \(\mathrm{N}(\theta^{t-1}, c^2\hat\Sigma)\) with \(c=2.4/\sqrt{2}=1.70\). Five chains are started at \((\hat\alpha,\hat\beta)\) plus independent normal noise of standard deviation \((2,5)\), which is overdispersed relative to the posterior.

Step 2. Eight adaptation rounds of 500 iterations each; after each round \(\hat\Sigma\) is replaced by the sample covariance of the second half of the pooled draws and \(c\) is multiplied by \(\exp\{1.5(\hat r - 0.35)\}\), where \(\hat r\) is the round’s acceptance rate and \(0.35\) is the target for \(d=2\) (interpolating between the \(0.44\) and \(0.23\) of Section 12.2):