Solutions to Casella & Berger’s Statistical Inference

(a) The ordered four-tuples of outcomes,

\begin{equation*} S=\{H,T\}^4=\{(x_1,x_2,x_3,x_4):x_i\in\{H,T\}\}, \end{equation*}

a set of \(2^4=16\) points.

(b) \(S=\{0,1,2,\dots,N\}\), where \(N\) is the number of leaves on the plant. If \(N\) is not known in advance one takes \(S=\{0,1,2,\dots\}\).

(c) \(S=[0,\infty)\); the endpoint \(0\) covers a bulb dead on installation, and no upper bound on a lifetime is imposed.

(d) \(S=(0,\infty)\), since a weight is a positive real number.

(e) With \(n\) components in the shipment, the proportion defective is a multiple of \(1/n\):

\begin{equation*} S=\Bigl\{0,\tfrac1n,\tfrac2n,\dots,\tfrac{n-1}{n},1\Bigr\}. \end{equation*}

If \(n\) is unknown, or the shipment is regarded as arbitrarily large, take \(S=[0,1]\).

Throughout, \(A\setminus B:=\{x\in S: x\in A\ \wedge\ x\notin B\}\).

(a) For \(x\in A\) the conditions \(x\notin B\) and \(x\notin A\cap B\) coincide, so

\begin{equation*} \begin{aligned} x\in A\setminus B &\iff x\in A \ \wedge\ x\notin B\\ &\iff x\in A \ \wedge\ x\notin A\cap B \iff x\in A\setminus(A\cap B), \end{aligned} \end{equation*}

and the middle line is verbatim the definition of \(A\cap B^{c}\).

(b) \(A\cup A^{c}=S\), so by the Distributive Law (Theorem 1.1.4c),

\begin{equation*} (B\cap A)\cup(B\cap A^{c})=B\cap(A\cup A^{c})=B\cap S=B. \end{equation*}

(c) Interchange the roles of \(A\) and \(B\) in the second equality of (a).

(d) Distribute over \(A\cup A^{c}=S\) and absorb:

\begin{equation*} \begin{aligned} A\cup(B\cap A^{c}) &=(A\cup B)\cap(A\cup A^{c})\\ &=(A\cup B)\cap S=A\cup B, \end{aligned} \end{equation*}

using the second Distributive Law of Theorem 1.1.4c.

Every part is inherited from the corresponding property of \(\vee\), \(\wedge\), \(\neg\) acting on the defining membership statements; the proof of the Distributive Law on pp. 3–4 is the template, and as there each chain below is a string of equivalences, so both containments hold at once.

(a) Since \(\vee\) and \(\wedge\) are commutative,

\begin{equation*} \begin{aligned} x\in A\cup B &\iff (x\in A)\vee(x\in B)\\ &\iff (x\in B)\vee(x\in A)\iff x\in B\cup A, \end{aligned} \end{equation*}

and the same chain with \(\vee\) replaced by \(\wedge\) gives \(A\cap B=B\cap A\).

(b) Since \(\vee\) is associative,

\begin{equation*} \begin{aligned} x\in A\cup(B\cup C) &\iff (x\in A)\vee\bigl[(x\in B)\vee(x\in C)\bigr]\\ &\iff \bigl[(x\in A)\vee(x\in B)\bigr]\vee(x\in C)\\ &\iff x\in(A\cup B)\cup C . \end{aligned} \end{equation*}

Replacing \(\vee\) by the (likewise associative) \(\wedge\) gives \(A\cap(B\cap C)=(A\cap B)\cap C\).

(c) These are the two De Morgan equivalences of logic, \(\neg(p\vee q)\iff(\neg p)\wedge(\neg q)\) and \(\neg(p\wedge q)\iff(\neg p)\vee(\neg q)\), read off membership:

\begin{equation*} \begin{aligned} x\in (A\cup B)^{c} &\iff \neg\bigl[(x\in A)\vee(x\in B)\bigr]\\ &\iff (x\notin A)\wedge(x\notin B)\iff x\in A^{c}\cap B^{c}, \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} x\in (A\cap B)^{c} &\iff \neg\bigl[(x\in A)\wedge(x\in B)\bigr]\\ &\iff (x\notin A)\vee(x\notin B)\iff x\in A^{c}\cup B^{c}. \end{aligned} \end{equation*}

(a) The event is \(A\cup B\), so by Theorem 1.2.9b

\begin{equation*} P(A\cup B)=P(A)+P(B)-P(A\cap B). \end{equation*}

(b) The event is the symmetric difference \((A\cap B^{c})\cup(A^{c}\cap B)\), a disjoint union, so Axiom 3 and Theorem 1.2.9a give

\begin{equation*} \begin{aligned} P(A\cap B^{c})+P(A^{c}\cap B) &=\bigl[P(A)-P(A\cap B)\bigr]+\bigl[P(B)-P(A\cap B)\bigr]\\ &=P(A)+P(B)-2P(A\cap B). \end{aligned} \end{equation*}

(c) At least one of \(A\), \(B\) is again the event \(A\cup B\), so the answer is that of (a):

\begin{equation*} P(A)+P(B)-P(A\cap B). \end{equation*}

(d) At most one of \(A\), \(B\) excludes only the outcome that both occur, i.e. the event is \((A\cap B)^{c}\), and by Theorem 1.2.8c

\begin{equation*} P\bigl((A\cap B)^{c}\bigr)=1-P(A\cap B). \end{equation*}

(a) \(A\cap B\cap C\) is the event that a U.S. birth results in twins, that the twins are identical, and that both are female: a birth of identical female twins. (The condition \(C\) is redundant, since \(A\subset C\) and \(B\subset C\).)

(b) \(P(A\cap B\cap C)=1/540\). Chain the given rates by the multiplication rule (Definition 1.3.2 rearranged, \(P(D\cap E)=P(E)P(D\mid E)\)), whose three factors are the twin-birth rate \(P( C)=1/90\), the identical fraction among twins \(P(B\mid C)=1/3\), and \(P(A\mid B\cap C)=1/2\) because identical twins are same-sex with each sex equally likely:

\begin{equation*} \begin{aligned} P(A\cap B\cap C)&=P( C)\,P(B\mid C)\,P(A\mid B\cap C)\\ &=\frac{1}{90}\cdot\frac{1}{3}\cdot\frac{1}{2} =\frac{1}{540}\approx 0.00185 . \end{aligned} \end{equation*}

No. By independence,

\begin{equation*} \begin{aligned} p_0&=(1-u)(1-w),\\ p_1&=u(1-w)+w(1-u),\\ p_2&=uw, \end{aligned} \end{equation*}

and the arithmetic–geometric mean inequality applied to the two nonnegative terms of \(p_1\) gives

\begin{equation*} p_1\ \ge\ 2\sqrt{u(1-w)\cdot w(1-u)}\ =\ 2\sqrt{p_0p_2}. \end{equation*}

The three events partition the sample space, so \(p_0+p_1+p_2=1\); if they were all equal each would equal \(1/3\), and the display would read \(\tfrac13\ge 2\sqrt{\tfrac13\cdot\tfrac13}=\tfrac23\), which is false. Hence no \(u,w\in[0,1]\) work.

Method (2): \(p_0=p_2\) forces \(1-u-w+uw=uw\), i.e. \(w=1-u\). Writing \(t=u(1-u)\), this makes

\begin{equation*} p_1=u^2+(1-u)^2=1-2t,\qquad p_2=t, \end{equation*}

so \(p_1=p_2\) demands \(t=\tfrac13\); but \(t=u(1-u)\le\tfrac14\) for every real \(u\).

(a) The wall is hit with probability \(1\), so the proportionality constant is \(1/A\) and \(P(\text{region }R)=\operatorname{area}( R)/A\). The region scoring \(i\) points, \(1\le i\le5\), is the annulus of radii \((5-i)r/5<(6-i)r/5\), of area \(\pi r^2\bigl[(6-i)^2-(5-i)^2\bigr]/25\), whence

\begin{equation*} \begin{aligned} P(\text{scoring }i\text{ points}) &=\frac{\pi r^{2}}{A}\cdot\frac{(6-i)^2-(5-i)^2}{5^{2}}, \quad i=1,\dots,5,\\ P(\text{scoring }0\text{ points})&=1-\frac{\pi r^{2}}{A}, \end{aligned} \end{equation*}

the last line being the complement of the board (Theorem 1.2.8c), since a miss scores nothing. These six numbers are nonnegative (the board lies on the wall, so \(\pi r^2\le A\)) and sum to \(1\), because the bracketed terms telescope:

\begin{equation*} \sum_{i=1}^{5}\bigl[(6-i)^2-(5-i)^2\bigr]=5^{2}-0^{2}=25 . \end{equation*}

Assigning these masses to the six disjoint outcomes defines a probability function by Theorem 1.2.6.

(b) Let \(H=\{\text{board is hit}\}=\bigcup_{i=1}^{5}\{\text{score }i\}\), a disjoint union, so by Axiom 3 and the telescoping sum above \(P(H)=\pi r^{2}/A>0\). Since \(\{\text{score }i\}\subset H\) for \(i\ge1\), Definition 1.3.2 gives

\begin{equation*} \begin{aligned} P(\text{score }i\mid H) &=\frac{P(\{\text{score }i\}\cap H)}{P(H)}\\ &=\frac{(\pi r^{2}/A)\bigl[(6-i)^2-(5-i)^2\bigr]/5^{2}}{\pi r^{2}/A}\\ &=\frac{(6-i)^2-(5-i)^2}{5^{2}},\qquad i=1,\dots,5, \end{aligned} \end{equation*}

which is Example 1.2.7’s distribution; the factor \(\pi r^{2}/A\) cancels, and \(P(\text{score }0\mid H)=0\).

(a) \(P(\text{scoring } i \text{ points}) = \dfrac{11-2i}{25}\), \(i = 1,\dots,5\). Indeed, the region scoring \(i\) points is the annulus between radii \((5-i)r/5\) and \((6-i)r/5\), so proportionality to area gives

\begin{equation*} \begin{aligned} P(\text{scoring } i) &= \frac{\pi\big((6-i)r/5\big)^2 - \pi\big((5-i)r/5\big)^2}{\pi r^2}\\ &= \frac{(6-i)^2-(5-i)^2}{5^2} = \frac{11-2i}{25}, \end{aligned} \end{equation*}

independent of \(\pi\) and \(r\).

(b) \(P(\text{scoring } i+1) - P(\text{scoring } i) = -\tfrac{2}{25} < 0\) for \(i=1,\dots,4\).

(c) Apply Theorem 1.2.6 with the finite sample space \(S=\{1,2,3,4,5\}\) and \(p_i = (11-2i)/25\). Its two hypotheses hold: \(p_i \ge \tfrac{1}{25} > 0\), and

\begin{equation*} \sum_{i=1}^{5} \frac{11-2i}{25} = \frac{55 - 2\cdot 15}{25} = \frac{25}{25} = 1 . \end{equation*}

Hence \(P(A) = \sum_{\{i\,:\,i \in A\}} p_i\) is a probability function on any sigma algebra of subsets of \(S\), i.e. it satisfies the Kolmogorov Axioms of Definition 1.2.4.

(a) Both sides consist of exactly the points lying in no \(A_\alpha\):

\begin{equation*} \begin{aligned} x \in \Big(\bigcup_{\alpha\in\Gamma} A_\alpha\Big)^c &\iff \neg\big(\exists\,\alpha\in\Gamma:\ x \in A_\alpha\big)\\ &\iff \forall\,\alpha\in\Gamma:\ x \notin A_\alpha\\ &\iff \forall\,\alpha\in\Gamma:\ x \in A_\alpha^c\\ &\iff x \in \bigcap_{\alpha\in\Gamma} A_\alpha^c . \end{aligned} \end{equation*}

The second equivalence is the quantifier duality \(\neg\exists = \forall\neg\), valid for \(\Gamma\) of any cardinality; countability was never used.

(b) Apply (a) to the collection \(\{A_\alpha^c : \alpha\in\Gamma\}\) and use \((A^c)^c = A\):

\begin{equation*} \Big(\bigcup_\alpha A_\alpha^c\Big)^c = \bigcap_\alpha \big(A_\alpha^c\big)^c = \bigcap_\alpha A_\alpha . \end{equation*}

Complementing both ends gives \(\bigcup_\alpha A_\alpha^c = \big(\bigcap_\alpha A_\alpha\big)^c\).

Formulation: for any sets \(A_1,\dots,A_n\),

\begin{equation*} \Big(\bigcup_{i=1}^n A_i\Big)^c = \bigcap_{i=1}^n A_i^c, \qquad \Big(\bigcap_{i=1}^n A_i\Big)^c = \bigcup_{i=1}^n A_i^c . \end{equation*}

This is Exercise 1.9 with the index set \(\Gamma = \{1,2,\dots,n\}\), which is a legitimate (indeed finite) choice there, so both identities are already proved.

Method (2): induction from the two-set law, Theorem 1.1.4(c). The case \(n=2\) is that theorem. Assuming the first identity for \(n-1\) sets,

\begin{equation*} \begin{aligned} \Big(\bigcup_{i=1}^{n} A_i\Big)^c &= \Big(\Big(\bigcup_{i=1}^{n-1} A_i\Big) \cup A_n\Big)^c\\ &= \Big(\bigcup_{i=1}^{n-1} A_i\Big)^c \cap A_n^c = \Big(\bigcap_{i=1}^{n-1} A_i^c\Big) \cap A_n^c = \bigcap_{i=1}^{n} A_i^c, \end{aligned} \end{equation*}

the second equality by Theorem 1.1.4(c) and the third by the induction hypothesis. The intersection identity follows by the same induction, or by substituting \(A_i^c\) for \(A_i\) and complementing as in Exercise 1.9(b).

Each part is a check of the three conditions of Definition 1.2.1.

(a) \(\mathcal{B} = \{\emptyset, S\}\). (i) \(\emptyset \in \mathcal{B}\). (ii) \(\emptyset^c = S \in \mathcal{B}\) and \(S^c = \emptyset \in \mathcal{B}\). (iii) If \(A_1, A_2, \dots \in \mathcal{B}\), then \(\bigcup_{i=1}^\infty A_i = S\) if some \(A_i = S\), and \(= \emptyset\) otherwise; either way it is in \(\mathcal{B}\).

(b) \(\mathcal{B}\) is the power set of \(S\), so every condition reduces to the observation that the set produced is again a subset of \(S\): (i) \(\emptyset \subset S\); (ii) \(A^c = S \setminus A \subset S\); (iii) \(\bigcup_{i=1}^\infty A_i \subset S\) whenever every \(A_i \subset S\).

(c) Let \(\mathcal{B}_1, \mathcal{B}_2\) be sigma algebras of subsets of \(S\) and put \(\mathcal{B} = \mathcal{B}_1 \cap \mathcal{B}_2\), the collection of sets belonging to both. (i) \(\emptyset \in \mathcal{B}_1\) and \(\emptyset \in \mathcal{B}_2\), so \(\emptyset \in \mathcal{B}\). (ii) If \(A \in \mathcal{B}\) then \(A \in \mathcal{B}_j\) for \(j=1,2\), hence \(A^c \in \mathcal{B}_j\) for \(j=1,2\), hence \(A^c \in \mathcal{B}\). (iii) If \(A_1, A_2, \dots \in \mathcal{B}\) then for each \(j=1,2\) all the \(A_i\) lie in \(\mathcal{B}_j\), so \(\bigcup_{i=1}^\infty A_i \in \mathcal{B}_j\); therefore \(\bigcup_{i=1}^\infty A_i \in \mathcal{B}\).

(a) Pad with empty sets. First, \(P(\emptyset)=0\): taking \(A_i = \emptyset\) for every \(i\) (a pairwise disjoint sequence, since \(\emptyset \cap \emptyset = \emptyset\)) Axiom 3 gives

\begin{equation*} P(\emptyset) = \sum_{i=1}^{\infty} P(\emptyset), \end{equation*}

and the right side is \(+\infty\) unless \(P(\emptyset)=0\), whereas the left side is a real number. Now let \(A, B \in \mathcal{B}\) be disjoint and set \(A_1 = A\), \(A_2 = B\), \(A_i = \emptyset\) for \(i \ge 3\). These are pairwise disjoint with union \(A \cup B\), so

\begin{equation*} \begin{aligned} P(A\cup B) &= \sum_{i=1}^{\infty} P(A_i)\\ &= P(A) + P(B) + \sum_{i=3}^{\infty} 0 = P(A) + P(B). \end{aligned} \end{equation*}

(b) Let \(A_1, A_2, \dots \in \mathcal{B}\) be pairwise disjoint and put \(A = \bigcup_{i=1}^{\infty} A_i\). Finite additivity extends by induction to any \(n\) pairwise disjoint sets, since \(\bigcup_{i=1}^{n} A_i = \big(\bigcup_{i=1}^{n-1} A_i\big) \cup A_n\) and these two sets are disjoint.

Let \(B_n = \bigcup_{i=n+1}^{\infty} A_i\), which lies in \(\mathcal{B}\) by Definition 1.2.1(c). For each \(n\) the \(n+1\) sets \(A_1,\dots,A_n,B_n\) are pairwise disjoint with union \(A\), so finite additivity gives

\begin{equation*} P(A) = \sum_{i=1}^{n} P(A_i) + P(B_n). \end{equation*}

The \(B_n\) are nested downward, \(B_1 \supset B_2 \supset \cdots\), and \(\bigcap_{n=1}^{\infty} B_n = \emptyset\): if \(x \in B_n\) for every \(n\), then \(x \in A_{i_0}\) for some \(i_0\), while \(x \in B_{i_0}\) forces \(x \in A_i\) for some \(i > i_0\), contradicting disjointness. Thus \(B_n \downarrow \emptyset\), so the Axiom of Continuity gives \(P(B_n) \to 0\) and

\begin{equation*} \begin{aligned} P(A) &= \lim_{n\to\infty}\left(\sum_{i=1}^{n} P(A_i) + P(B_n)\right)\\ &= \sum_{i=1}^{\infty} P(A_i). \end{aligned} \end{equation*}

No. By Theorem 1.2.8(c), \(P(B) = 1 - P(B^c) = \tfrac{3}{4}\). If \(A\) and \(B\) were disjoint, finite additivity (Exercise 1.12(a)) would give

\begin{equation*} P(A \cup B) = P(A) + P(B) = \tfrac{1}{3} + \tfrac{3}{4} = \tfrac{13}{12} > 1, \end{equation*}

contradicting \(P(A\cup B) \le 1\) (Theorem 1.2.8(b)).

There are \(2^n\): specifying a subset of \(S = \{s_1,\dots,s_n\}\) is a job of \(n\) tasks, the \(i\)th being the in-or-out decision for \(s_i\) and having \(2\) possible outcomes, so the Fundamental Theorem of Counting (Theorem 1.2.14) gives

\begin{equation*} \underbrace{2 \times 2 \times \cdots \times 2}_{n \text{ factors}} = 2^n . \end{equation*}

The tasks do indeed determine the subset uniquely: \(A \mapsto \big(I_A(s_1),\dots,I_A(s_n)\big)\) and \((x_1,\dots,x_n) \mapsto \{s_i : x_i = 1\}\) are mutually inverse (Check!), so the subsets of \(S\) are in bijection with \(\{0,1\}^n\).

Method (2): count by size. There are \(\binom{n}{k}\) subsets of size \(k\), and these classes are disjoint and exhaust all subsets, so the total is

\begin{equation*} \sum_{k=0}^{n} \binom{n}{k} = \sum_{k=0}^{n} \binom{n}{k} 1^k 1^{n-k} = (1+1)^n = 2^n \end{equation*}

by the Binomial Theorem (Theorem 3.2.2).

Bundle the first \(k\) tasks into one composite task and apply the case \(k = 2\).

Induct on \(k\), the base case \(k = 2\) being the counting argument in the proof of Theorem 1.2.14. Assume the theorem for some \(k \ge 2\) and let a job consist of \(k+1\) tasks, doable in \(n_1,\dots,n_{k+1}\) ways. Let \(T\) be the single composite task of doing tasks \(1\) through \(k\); a way of doing \(T\) is exactly a choice of a way for each of those \(k\) tasks, so by the induction hypothesis \(T\) can be done in \(n_1 \times \cdots \times n_k\) ways. The job is now the two tasks \(T\) and task \(k+1\), so the case \(k = 2\) gives

\begin{equation*} (n_1 \times \cdots \times n_k)\times n_{k+1} = n_1 \times \cdots \times n_{k+1} \end{equation*}

ways, which is the assertion for \(k+1\).

(a) \(26^3 = 17{,}576\). A set of initials is an ordered string of three letters (two given names, then the surname), each of the \(26\) letters available for each position with repetition allowed: the ordered-with-replacement entry \(n^r\) of Table 1.2.1 with \(n = 26\), \(r = 3\).

(b) \(26^2 + 26^3 = 18{,}252\), since the one-given-name strings (length \(2\)) and the two-given-name strings (length \(3\)) are disjoint collections.

(c) \(26^2 + 26^3 + 26^4 = 475{,}228\), by the same disjoint decomposition into lengths \(2\), \(3\) and \(4\).

\begin{equation*} \binom{n + 2 - 1}{2} = \binom{n+1}{2} = \frac{n(n+1)}{2}. \end{equation*}

A piece is an unordered selection of size \(r = 2\) from the \(n\) numbers, with replacement permitted since doubles \((i,i)\) are legal pieces; that is the unordered-with-replacement entry \(\binom{n+r-1}{r}\) of Table 1.2.1.

Method (2): count directly. There are \(\binom{n}{2}\) pieces with two distinct numbers and \(n\) doubles, and

\begin{equation*} \binom{n}{2} + n = \frac{n(n-1)}{2} + n = \frac{n(n+1)}{2}. \end{equation*}

\begin{equation*} P(\text{exactly one cell empty}) = \frac{\binom{n}{2} n!}{n^n}. \end{equation*}

The balls are distinguishable and each of the \(n^n\) placements (ordered with replacement, Table 1.2.1) is equally likely. With \(n\) balls in \(n\) cells, exactly one empty cell forces the occupancy pattern to be one cell with \(2\) balls, one cell with \(0\), and the remaining \(n-2\) cells with \(1\) each. Build such a placement by Theorem 1.2.14:

(i) choose the empty cell: \(n\) ways;

(ii) choose the doubly occupied cell: \(n-1\) ways;

(iii) choose the two balls that share it: \(\binom{n}{2}\) ways;

(iv) distribute the remaining \(n-2\) balls one apiece among the remaining \(n-2\) cells: \((n-2)!\) ways.

Hence the count of favorable placements is

\begin{equation*} \begin{aligned} n(n-1)\binom{n}{2}(n-2)! &= \binom{n}{2}\, n(n-1)(n-2)! \\ &= \binom{n}{2}\, n!, \end{aligned} \end{equation*}

and dividing by \(n^n\) gives the stated probability.

(a) \(\binom{3+4-1}{4} = \binom{6}{4} = 15\), by (b) with \(n = 3\), \(r = 4\).

(b) Because the order of differentiation is immaterial, an \(r\)th partial derivative of \(f(x_1,\dots,x_n)\) is determined by nothing but how many times each variable is differentiated, i.e. by the vector of nonnegative integers \((r_1,\dots,r_n)\) with \(r_1 + \cdots + r_n = r\), the derivative being

\begin{equation*} \frac{\partial^{\,r}}{\partial x_1^{r_1}\cdots\, \partial x_n^{r_n}} f . \end{equation*}

Such a vector is exactly an unordered sample of size \(r\) drawn with replacement from the \(n\) variables (\(x_i\) drawn \(r_i\) times), and by Table 1.2.1 the number of these is \(\binom{n+r-1}{r}\).

\begin{equation*} P = \frac{1}{7^{12}}\sum_{k=0}^{7}(-1)^k \binom{7}{k}(7-k)^{12} = 0.2285 . \end{equation*}

Model the week as an ordered-with-replacement assignment of each of the \(12\) calls to one of the \(7\) days, so the sample space has \(7^{12}\) equally likely points (Table 1.2.1). Let \(A_i\) be the event that day \(i\) receives no call. If a fixed set of \(k\) days all go without a call, then all \(12\) calls land among the other \(7-k\) days, so

\begin{equation*} P\!\left(A_{i_1}\cap \cdots \cap A_{i_k}\right) = \frac{(7-k)^{12}}{7^{12}}, \end{equation*}

a value depending only on \(k\). Inclusion-exclusion over the \(\binom{7}{k}\) choices of the \(k\) days gives

\begin{equation*} P\!\left(\bigcup_{i=1}^{7} A_i\right) = \sum_{k=1}^{7}(-1)^{k+1}\binom{7}{k}\frac{(7-k)^{12}}{7^{12}}, \end{equation*}

and the event of interest is the complement, whence the displayed alternating sum. The terms of the numerator are

summing to \(3{,}162{,}075{,}840\), and

\begin{equation*} \frac{3{,}162{,}075{,}840}{13{,}841{,}287{,}201} = 0.22845 . \end{equation*}

\begin{equation*} P(\text{no matching pair}) = \frac{\binom{n}{2r}\, 2^{2r}}{\binom{2n}{2r}} . \end{equation*}

The closet holds \(2n\) individual shoes, and a random sample of \(2r\) of them is one of the \(\binom{2n}{2r}\) unordered-without-replacement selections (Table 1.2.1), all equally likely. The sample contains no matching pair exactly when its \(2r\) shoes come from \(2r\) distinct pairs, one shoe from each; such samples are built by Theorem 1.2.14 in two tasks:

(i) choose which \(2r\) of the \(n\) pairs are represented: \(\binom{n}{2r}\) ways (possible since \(2r < n\));

(ii) choose left or right from each represented pair: \(2^{2r}\) ways.

Dividing the product by \(\binom{2n}{2r}\) gives the stated probability.

(a) \(.167\times 10^{-8}\); (b) \(\binom{336}{30}\big/\binom{366}{30}=.0687\).

The first 180 days form an unordered sample drawn without replacement, so all \(\binom{366}{180}\) subsets are equally likely. Evenly distributed means \(180/12=15\) days from each month, and such a subset is built by choosing \(15\) days from each month in turn, so by the Fundamental Theorem of Counting (Theorem 1.2.14) the favourable subsets number \(\prod_i\binom{d_i}{15}\):

\begin{equation*} P_a=\frac{\prod_{i=1}^{12}\binom{d_i}{15}}{\binom{366}{180}}, \qquad (d_1,\dots,d_{12})=(31,29,31,30,31,30,31,31,30,31,30,31). \end{equation*}

Grouping the seven \(31\)-day months, the four \(30\)-day months and February,

\begin{equation*} P_a=\frac{\binom{31}{15}^{7}\binom{30}{15}^{4}\binom{29}{15}}{\binom{366}{180}} =.167\times 10^{-8}. \end{equation*}

(b) September contributes \(30\) days, leaving \(366-30=336\) admissible days. The first \(30\) draws are an equally likely unordered \(30\)-subset of the \(366\), and the favourable ones are exactly the \(30\)-subsets of those \(336\):

\begin{equation*} P_b=\frac{\binom{336}{30}}{\binom{366}{30}}=.0687 . \end{equation*}

\(\left(\tfrac14\right)^{n}\binom{2n}{n}\).

Let \(X\) and \(Y\) be the two head counts. The tosses are independent, and each person’s \(2^{n}\) outcome strings are equally likely with exactly \(\binom{n}{k}\) of them carrying \(k\) heads, so \(P(X=k)=P(Y=k)=\binom{n}{k}2^{-n}\). Decomposing on the common value,

\begin{equation*} \begin{aligned} P(X=Y)&=\sum_{k=0}^{n}P(X=k)P(Y=k)\\ &=\left(\tfrac12\right)^{2n}\sum_{k=0}^{n}\binom{n}{k}^{2}. \end{aligned} \end{equation*}

Since \(\binom{n}{k}=\binom{n}{n-k}\), the sum is \(\sum_{k}\binom{n}{k}\binom{n}{n-k}\), which is the coefficient of \(x^{n}\) in \((1+x)^{n}(1+x)^{n}=(1+x)^{2n}\), namely \(\binom{2n}{n}\) (Vandermonde). Hence

\begin{equation*} P(X=Y)=\left(\tfrac14\right)^{n}\binom{2n}{n}. \end{equation*}

(a) \(2/3\); (b) \(\dfrac{p}{1-(1-p)^{2}}=\dfrac{1}{2-p}\); (c) immediate from (b).

Do (b) first. The tosses are independent, so \(P(E_i)=(1-p)^{i-1}p\), and the \(E_i\) are disjoint. Player A makes the odd-numbered tosses, so \(\{\text{A wins}\}=\bigcup_{i=1}^{\infty}E_{2i-1}\) and countable additivity (Definition 1.2.4) gives

\begin{equation*} \begin{aligned} P(\text{A wins})&=\sum_{i=1}^{\infty}p\,(1-p)^{2i-2}\\ &=p\sum_{j=0}^{\infty}\big[(1-p)^{2}\big]^{j}\\ &=\frac{p}{1-(1-p)^{2}}=\frac{1}{2-p}, \end{aligned} \end{equation*}

the geometric series converging because \((1-p)^{2}<1\) for \(0<p\le 1\).

(a) Put \(p=\tfrac12\): \(P(\text{A wins})=1/(2-\tfrac12)=2/3\).

(c) For \(0<p<1\) we have \(2-p<2\), hence

\begin{equation*} P(\text{A wins})=\frac{1}{2-p}>\frac{1}{2}. \end{equation*}

\(1/3\).

Order the children by birth and take the equally likely sample space

\begin{equation*} S=\{BB,\;BG,\;GB,\;GG\}, \end{equation*}

each outcome having probability \(\tfrac14\) (boys and girls equally likely, sexes of the two children independent). With \(A=\{BB\}\) and \(B=\{\text{at least one boy}\}=\{BB,BG,GB\}\) we have \(A\cap B=A\), so by Definition 1.3.2

\begin{equation*} P(A\mid B)=\frac{P(A\cap B)}{P(B)}=\frac{1/4}{3/4}=\frac13 . \end{equation*}

\((5/6)^{5}=.4019\).

Let \(N\) be the number of the cast on which the first \(6\) appears. The event \(\{N>5\}\) is exactly the event that each of the first five casts fails to show a \(6\), and the casts are independent with failure probability \(\tfrac56\) each, so

\begin{equation*} P(N>5)=\left(\frac56\right)^{5}=\frac{3125}{7776}=.4019 . \end{equation*}

All three are the Binomial Theorem \(\sum_{k=0}^{m}\binom{m}{k}x^{k}=(1+x)^{m}\) evaluated at \(x=\pm 1\), after the absorption identity

\begin{equation*} k\binom{n}{k}=k\cdot\frac{n!}{k!(n-k)!} =n\cdot\frac{(n-1)!}{(k-1)!\,(n-k)!}=n\binom{n-1}{k-1}, \qquad 1\le k\le n. \end{equation*}

(a) Take \(m=n\), \(x=-1\):

\begin{equation*} \sum_{k=0}^{n}(-1)^{k}\binom{n}{k}=(1-1)^{n}=0, \end{equation*}

valid since \(n\ge 2\ge 1\).

(b) Absorb, reindex \(j=k-1\), then take \(m=n-1\), \(x=1\):

\begin{equation*} \begin{aligned} \sum_{k=1}^{n}k\binom{n}{k} &=n\sum_{k=1}^{n}\binom{n-1}{k-1}\\ &=n\sum_{j=0}^{n-1}\binom{n-1}{j}=n2^{n-1}. \end{aligned} \end{equation*}

(c) The same two moves with \(x=-1\); note \((-1)^{k+1}=(-1)^{j}\) when \(j=k-1\):

\begin{equation*} \begin{aligned} \sum_{k=1}^{n}(-1)^{k+1}k\binom{n}{k} &=n\sum_{j=0}^{n-1}(-1)^{j}\binom{n-1}{j}\\ &=n(1-1)^{n-1}=0, \end{aligned} \end{equation*}

where \(n-1\ge 1\) is exactly what the hypothesis \(n\ge 2\) supplies (for \(n=1\) the sum equals \(1\)).

Put \(d_n=\log\!\big(n!\,/\,n^{n+(1/2)}e^{-n}\big)\) and show \(d_n\) decreases to a finite limit; the required limit is then \(e^{\lim d_n}\).

Explicitly \(d_n=\log n!-\left(n+\tfrac12\right)\log n+n\); this is \(\log n!\) measured against the average of the hint’s two integrals \(n\log n-n\) and \((n+1)\log(n+1)-n\), up to a bounded term.

Monotonicity. Telescoping one step and writing \(t=1/(2n+1)\), so that \((n+1)/n=(1+t)/(1-t)\) and \(n+\tfrac12=1/(2t)\),

\begin{equation*} \begin{aligned} d_n-d_{n+1} &=\left(n+\tfrac12\right)\log\frac{n+1}{n}-1\\ &=\frac{1}{2t}\log\frac{1+t}{1-t}-1\\ &=\frac{1}{2t}\cdot 2\left(t+\frac{t^{3}}{3}+\frac{t^{5}}{5}+\cdots\right)-1\\ &=\frac{t^{2}}{3}+\frac{t^{4}}{5}+\cdots\;>\;0, \end{aligned} \end{equation*}

the series being the standard expansion of \(\log\frac{1+t}{1-t}\), legitimate since \(0<t<1\). So \((d_n)\) is strictly decreasing.

Lower bound. Enlarging every coefficient to \(\tfrac13\) and summing the resulting geometric series,

\begin{equation*} \begin{aligned} 0<d_n-d_{n+1}&<\frac13\cdot\frac{t^{2}}{1-t^{2}} =\frac{1}{3\big[(2n+1)^{2}-1\big]}\\ &=\frac{1}{12n(n+1)} =\frac{1}{12}\left(\frac1n-\frac1{n+1}\right). \end{aligned} \end{equation*}

Hence \(d_n-\frac{1}{12n}<d_{n+1}-\frac{1}{12(n+1)}\), so \(d_n-\frac{1}{12n}\) is increasing and therefore

\begin{equation*} d_n>d_n-\frac{1}{12n}\ \ge\ d_1-\frac1{12}=1-\frac1{12} \quad\text{for all }n\ge 1 . \end{equation*}

A decreasing sequence bounded below converges, say \(d_n\downarrow c\) with \(c\ge \tfrac{11}{12}\) finite. Since \(\exp\) is continuous,

\begin{equation*} \frac{n!}{n^{n+(1/2)}e^{-n}}=e^{d_n}\longrightarrow e^{c}, \end{equation*}

a finite positive constant.

(a) Six and twelve ordered samples respectively, matching \(4!/(2!\,2!)=6\) and \(4!/(1!\,2!\,1!)=12\) from part (c):

\begin{equation*} \begin{aligned} \{4,4,12,12\}:\quad &(4,4,12,12),\ (4,12,4,12),\ (4,12,12,4),\\ &(12,4,4,12),\ (12,4,12,4),\ (12,12,4,4); \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} \{2,9,9,12\}:\quad &(2,9,9,12),\ (2,9,12,9),\ (2,12,9,9),\\ &(9,2,9,12),\ (9,2,12,9),\ (9,9,2,12),\\ &(9,9,12,2),\ (9,12,2,9),\ (9,12,9,2),\\ &(12,2,9,9),\ (12,9,2,9),\ (12,9,9,2). \end{aligned} \end{equation*}

(b) \(5/1296\approx .0039\). The sample has size \(k=6\) with multiplicities \(1,2,1,2\) for \(2,7,8,14\), so by (c) it is made up of

\begin{equation*} \frac{6!}{1!\,2!\,1!\,2!}=\frac{720}{4}=180 \end{equation*}

ordered samples, out of \(6^6=46{,}656\) equally likely ordered draws:

\begin{equation*} P\bigl(\{2,7,7,8,14,14\}\bigr)=\frac{180}{46656}=\frac{5}{1296}. \end{equation*}

(c) Treat the \(k\) drawn numbers as distinguishable; counting method 1 of Section 1.2.3 orders them in \(k!\) ways. Two of those orderings give the same ordered sample exactly when one comes from the other by permuting the \(k_i\) copies of the \(i\)th number among themselves, so the map onto ordered samples is exactly \(k_1!\,k_2!\cdots k_m!\)-to-one. Dividing out that double counting, as for the two 4s in Example 1.2.20, leaves

\begin{equation*} \frac{k!}{k_1!\,k_2!\cdots k_m!}. \end{equation*}

(d) The sum counts the ordered \(m\)-tuples \((k_1,\ldots,k_m)\) of nonnegative integers with \(k_1+\cdots+k_m=k\); since an unordered with-replacement sample of size \(k\) from \(m\) numbers is determined by, and determines, its multiplicity vector, this is also the number of distinct bootstrap samples. Encode such a tuple as a row of \(k\) stars and \(m-1\) bars,

\begin{equation*} \begin{aligned} \underbrace{\star\cdots\star}_{k_1}\ \mid\ \underbrace{\star\cdots\star}_{k_2} \ \mid\ \cdots\ \mid\ \underbrace{\star\cdots\star}_{k_m}, \end{aligned} \end{equation*}

a bijection onto the arrangements of \(k+m-1\) symbols of which \(k\) are stars. Choosing the star positions gives

\begin{equation*} \sum_{k_1,\ldots,k_m} I_{\{k_1+\cdots+k_m=k\}}=\binom{k+m-1}{k}, \end{equation*}

which is the unordered-with-replacement count \(\binom{n+r-1}{r}\) of Section 1.2.3 with \(n=m\), \(r=k\).

Each of the \(\binom{11}{6}=462\) unordered samples (Exercise 1.29(d)) gets the multinomial weight of Exercise 1.29(c),

\begin{equation*} P\bigl(\{k_1,\ldots,k_6\}\bigr)=\frac{1}{6^6}\cdot\frac{6!}{k_1!\,k_2!\cdots k_6!}, \end{equation*}

since all \(6^6=46{,}656\) ordered samples are equally likely; this is the multinomial weighting of Figure 1.2.2. Collecting the ordered samples by their average \(\bar x=\tfrac16\sum_{j=1}^{6}x_j\) gives \(85\) distinct values in \([1,20]\), and grouping into unit-width cells gives the histogram

The modal cell \([8,9)\) holds the population mean \(\bar x=52/6=26/3\), whose probability \(2520/46656=35/648\) is the largest of any single average.

(a) The winner is the outcome \(\{x_1,x_2,\ldots,x_n\}\) that contains each population value exactly once; its average is \(\bar x=(x_1+\cdots+x_n)/n\) and its probability is \(n!/n^n\).

Label the population items \(1,\ldots,n\), so that a sample of size \(n\) drawn with replacement is one of \(n^n\) equally likely ordered \(n\)-tuples of labels. An unordered outcome is described by its multiplicity vector \((k_1,\ldots,k_n)\) with \(k_i\ge 0\), \(\sum_i k_i=n\), and by Exercise 1.29(c) it consists of \(n!/(k_1!\cdots k_n!)\) ordered tuples, so

\begin{equation*} P(k_1,\ldots,k_n)=\frac{1}{n^n}\,\frac{n!}{k_1!\,k_2!\cdots k_n!}. \end{equation*}

Maximizing \(P\) is maximizing the multinomial coefficient \(M(k)=n!/\prod_i k_i!\). Suppose some \(k_i\ge 2\). Since \(\sum_i k_i=n\) over \(n\) coordinates, some \(k_j=0\). Let \(k^{\prime}\) move one unit from \(i\) to \(j\); then

\begin{equation*} \frac{M(k^{\prime})}{M(k)} =\frac{k_i!\,k_j!}{(k_i-1)!\,(k_j+1)!} =\frac{k_i}{1}=k_i\ \ge 2>1 . \end{equation*}

So no vector with a coordinate \(\ge 2\) can be maximal, and as the feasible set is finite the maximum is attained only at \(k_1=\cdots=k_n=1\), i.e. at the outcome \(\{x_1,\ldots,x_n\}\), with

\begin{equation*} P(1,1,\ldots,1)=\frac{1}{n^n}\,\frac{n!}{1!\cdots 1!}=\frac{n!}{n^n}. \end{equation*}

(Other outcomes may share the average \(\bar x\), as in Example 1.2.20; the claim is that none is more probable than this one.)

(b) By Exercise 1.28, \(n!\approx\sqrt{2\pi}\,n^{n+(1/2)}e^{-n}\), so

\begin{equation*} \frac{n!}{n^n}\approx\frac{\sqrt{2\pi}\,n^{n+(1/2)}e^{-n}}{n^n} =\sqrt{2\pi n}\,e^{-n}=\frac{\sqrt{2n\pi}}{e^{n}} . \end{equation*}

(c) The \(n\) draws are independent and each avoids \(x_i\) with probability \((n-1)/n\), so

\begin{equation*} P(x_i\text{ missing})=\left(\frac{n-1}{n}\right)^{\!n}=\left(1-\frac1n\right)^{\!n}, \end{equation*}

and since \(n\log\left(1-\tfrac1n\right)=n\left(-\tfrac1n-\tfrac{1}{2n^{2}}-\cdots\right)\to-1\), this tends to \(e^{-1}\approx .368\).

\(P(\text{best candidate hired}\mid\text{hire occurs on trial }i)=\dfrac{i}{N}\).

By rule (b), all that “a candidate was hired on the \(i\)th trial” tells us about the ratings is

\begin{equation*} H_i=\{\text{candidate }i\text{ is better than candidates }1,\ldots,i-1\}, \end{equation*}

i.e. candidate \(i\) is the best of the first \(i\) seen. Write \(B=\{\text{candidate }i\text{ is the best of all }N\}\). The interview order is a uniformly random permutation of the \(N\) candidates, so by exchangeability the best of the first \(i\) is equally likely to occupy any one of those \(i\) positions, and the best of all \(N\) is equally likely to occupy any one of the \(N\) positions:

\begin{equation*} P(H_i)=\frac1i,\qquad P(B)=\frac1N . \end{equation*}

Since \(B\subset H_i\) (the overall best is in particular better than candidates \(1,\ldots,i-1\)), Definition 1.3.2 gives

\begin{equation*} P(B\mid H_i)=\frac{P(B\cap H_i)}{P(H_i)}=\frac{P(B)}{P(H_i)} =\frac{1/N}{1/i}=\frac{i}{N}. \end{equation*}

\(P(M\mid C)=\dfrac{20}{21}\approx .952\).

With \(M,F\) the partition into males and females, \(P(M)=P(F)=\tfrac12\), and \(C\) the event of color-blindness, we are given \(P(C\mid M)=.05\) and \(P(C\mid F)=.0025\). Bayes’ Rule (Theorem 1.3.5) gives

\begin{equation*} \begin{aligned} P(M\mid C) &=\frac{P(C\mid M)P(M)}{P(C\mid M)P(M)+P(C\mid F)P(F)}\\[2pt] &=\frac{(.05)(.5)}{(.05)(.5)+(.0025)(.5)}\\[2pt] &=\frac{.05}{.0525}=\frac{20}{21}\approx .952 . \end{aligned} \end{equation*}

(a) \(P(B)=\dfrac{19}{30}\approx .633\). Litters \(L_1,L_2\) partition the sample space with \(P(L_1)=P(L_2)=\tfrac12\), and sampling uniformly within a litter gives \(P(B\mid L_1)=\tfrac23\) (two brown of three) and \(P(B\mid L_2)=\tfrac35\) (three brown of five). By the law of total probability (the denominator of Theorem 1.3.5),

\begin{equation*} \begin{aligned} P(B)&=\tfrac12\cdot\tfrac23+\tfrac12\cdot\tfrac35\\[2pt] &=\tfrac13+\tfrac{3}{10}=\frac{19}{30}. \end{aligned} \end{equation*}

(b) \(P(L_1\mid B)=\dfrac{10}{19}\approx .526\):

\begin{equation*} P(L_1\mid B)=\frac{P(B\mid L_1)P(L_1)}{P(B)} =\frac{\tfrac12\cdot\tfrac23}{19/30}=\frac{1/3}{19/30}=\frac{10}{19}. \end{equation*}

Each of the three axioms of Definition 1.2.4 is inherited from \(P\) after division by the constant \(P(B)>0\). Throughout, \(P(A\mid B)=P(A\cap B)/P(B)\) by Definition 1.3.2, which is well defined on the same sigma algebra \(\mathcal{B}\) since \(A\cap B\in\mathcal{B}\) whenever \(A,B\in\mathcal{B}\).

(i) Nonnegativity: for any \(A\in\mathcal{B}\), \(P(A\cap B)\ge 0\) by Axiom 1 for \(P\), and \(P(B)>0\), so

\begin{equation*} P(A\mid B)=\frac{P(A\cap B)}{P(B)}\ \ge\ 0 . \end{equation*}

(ii) Normalization: \(S\cap B=B\), so

\begin{equation*} P(S\mid B)=\frac{P(S\cap B)}{P(B)}=\frac{P(B)}{P(B)}=1 . \end{equation*}

(iii) Countable additivity: let \(A_1,A_2,\ldots\in\mathcal{B}\) be pairwise disjoint. Then the sets \(A_i\cap B\) are also pairwise disjoint, since

\begin{equation*} (A_i\cap B)\cap(A_j\cap B)\subset A_i\cap A_j=\emptyset \quad (i\ne j), \end{equation*}

and \(\left(\bigcup_{i=1}^{\infty}A_i\right)\cap B=\bigcup_{i=1}^{\infty}(A_i\cap B)\) by the distributive law of Theorem 1.1.4, extended to a countable union. Hence, applying Axiom 3 for \(P\) to the disjoint sets \(A_i\cap B\),

\begin{equation*} \begin{aligned} P\!\left(\bigcup_{i=1}^{\infty}A_i\,\Big|\,B\right) &=\frac{P\!\left(\bigcup_{i=1}^{\infty}(A_i\cap B)\right)}{P(B)}\\[2pt] &=\frac{\sum_{i=1}^{\infty}P(A_i\cap B)}{P(B)}\\[2pt] &=\sum_{i=1}^{\infty}\frac{P(A_i\cap B)}{P(B)} =\sum_{i=1}^{\infty}P(A_i\mid B), \end{aligned} \end{equation*}

dividing the convergent nonnegative series termwise by the constant \(P(B)\). So \(P(\cdot\mid B)\) is a probability function on \((S,\mathcal{B})\).

\(P(\text{at least two hits}) = 6095609/9765625 = .6242\), and the conditional probability is \(6095609/8717049 = .6993\).

Let \(X\) be the number of hits. The shots are independent with hit probability \(\tfrac15\), so by Definition 1.3.7 the probability of any specified pattern of \(j\) hits and \(10-j\) misses is \((\tfrac15)^j(\tfrac45)^{10-j}\); there is one pattern with \(j=0\) and ten with \(j=1\). Hence

\begin{equation*} \begin{aligned} P(X \ge 2) &= 1 - \left(\tfrac45\right)^{10} - 2\left(\tfrac45\right)^{9}\\ &= \frac{5^{10}-4^{10}-10\cdot 4^{9}}{5^{10}} = \frac{6095609}{9765625} = .6242 . \end{aligned} \end{equation*}

Since \(\{X\ge 2\} \subset \{X \ge 1\}\), Exercise 1.38(b) gives the conditional probability as a ratio, with \(P(X\ge1) = 1-(4/5)^{10} = 8717049/9765625\):

\begin{equation*} P(X\ge 2 \mid X \ge 1) = \frac{P(X\ge 2)}{P(X\ge 1)} = \frac{6095609}{8717049} = .6993 . \end{equation*}

(a) \(P(A\mid\mathcal{W}) = \dfrac{\gamma}{1+\gamma}\), which is \(<\tfrac13\) for \(\gamma<\tfrac12\), \(=\tfrac13\) at \(\gamma=\tfrac12\), and \(>\tfrac13\) for \(\gamma>\tfrac12\).

With \(P(A)=P(B)=P( C)=\tfrac13\), the table gives the joint probabilities of “pardoned” and “warden says B dies”:

\begin{equation*} \begin{aligned} P(A\cap\mathcal{W}) &= \tfrac13\gamma, \qquad P(B\cap\mathcal{W}) = 0,\\ P(C\cap\mathcal{W}) &= \tfrac13\cdot 1 = \tfrac13 , \end{aligned} \end{equation*}

(if B is pardoned the warden must name C, and if C is pardoned he must name B). Since \(A,B,C\) partition the sample space,

\begin{equation*} P(\mathcal{W}) = \tfrac13\gamma + 0 + \tfrac13 = \frac{1+\gamma}{3}, \end{equation*}

so by Definition 1.3.2,

\begin{equation*} P(A\mid\mathcal{W}) = \frac{\gamma/3}{(1+\gamma)/3} = \frac{\gamma}{1+\gamma}. \end{equation*}

The map \(\gamma \mapsto \gamma/(1+\gamma)\) is increasing on \([0,1]\), and since \(1+\gamma>0\),

\begin{equation*} \begin{aligned} \frac{\gamma}{1+\gamma} < \frac13 &\iff 3\gamma < 1+\gamma \iff \gamma < \tfrac12,\\ \frac{\gamma}{1+\gamma} = \frac13 &\iff \gamma = \tfrac12,\\ \frac{\gamma}{1+\gamma} > \frac13 &\iff \gamma > \tfrac12 . \end{aligned} \end{equation*}

(b) After the swap A is pardoned exactly when C is pardoned, so A’s survival probability is \(P(C\mid\mathcal{W}) = \tfrac23\). Indeed, with \(\gamma=\tfrac12\) part (a) gives \(P(\mathcal{W}) = \tfrac12\) and \(P(C\cap\mathcal{W}) = \tfrac13\), so

\begin{equation*} P(C\mid\mathcal{W}) = \frac{1/3}{1/2} = \frac23 , \end{equation*}

against \(P(A\mid\mathcal{W}) = \tfrac13\) from (1.3.2) of Example 1.3.4 had he not swapped.

(a) \(P(B)=1\) forces \(P(B^c)=0\) by Theorem 1.2.8c, so \(P(A\cap B^c) \le P(B^c) = 0\) by Theorem 1.2.9c. Hence by Theorem 1.2.9a,

\begin{equation*} P(A\cap B) = P(A) - P(A\cap B^c) = P(A), \end{equation*}

and \(P(A\mid B) = P(A\cap B)/P(B) = P(A)/1 = P(A)\).

(b) \(A\subset B\) gives \(A\cap B = A\), so Definition 1.3.2 applied twice yields

\begin{equation*} P(B\mid A) = \frac{P(A\cap B)}{P(A)} = \frac{P(A)}{P(A)} = 1, \qquad P(A\mid B) = \frac{P(A\cap B)}{P(B)} = \frac{P(A)}{P(B)} . \end{equation*}

(c) \(A \subset A\cup B\), so \(A \cap (A\cup B) = A\), while \(A\cap B = \emptyset\) makes \(P(A\cup B) = P(A)+P(B)\) by Theorem 1.2.9b (or Axiom 3). Therefore

\begin{equation*} P(A\mid A\cup B) = \frac{P(A\cap(A\cup B))}{P(A\cup B)} = \frac{P(A)}{P(A)+P(B)} . \end{equation*}

(d) Apply (1.3.3) twice, first with conditioning event \(B\cap C\) and then with conditioning event \(C\):

\begin{equation*} \begin{aligned} P(A\cap B\cap C) &= P\bigl(A \mid B\cap C\bigr)\,P(B\cap C)\\ &= P\bigl(A\mid B\cap C\bigr)\,P(B\mid C)\,P( C). \end{aligned} \end{equation*}

(Both steps are legitimate since \(P(B\cap C)>0\) forces \(P( C)>0\).)

(a) Mutual exclusivity gives \(A\cap B = \emptyset\), so \(P(A\cap B) = 0\) by Theorem 1.2.8a, whereas

\begin{equation*} P(A)P(B) > 0 \end{equation*}

because both factors are strictly positive. Thus \(P(A\cap B) \ne P(A)P(B)\) and (1.3.8) fails: \(A\) and \(B\) are not independent.

(b) This is the contrapositive of (a). Explicitly, if \(A\) and \(B\) are independent then \(P(A\cap B) = P(A)P(B) > 0\), so \(A\cap B \ne \emptyset\) and the events are not mutually exclusive.

(b) Independence (1.3.8) is symmetric in the two events, so part (a) of the theorem, applied to the ordered pair \((B,A)\), already says that \(B\) and \(A^c\) are independent.

(c) By (b) the pair \((A^c,B)\) is independent, so part (a) applied to it gives independence of \(A^c\) and \(B^c\).

Method (2): direct computation. For (b), by Theorem 1.2.9a, independence of \(A\) and \(B\), and Theorem 1.2.8c,

\begin{equation*} \begin{aligned} P(A^c\cap B) &= P(B) - P(A\cap B) = P(B) - P(A)P(B)\\ &= P(B)\bigl(1-P(A)\bigr) = P(A^c)P(B) . \end{aligned} \end{equation*}

For (c), by DeMorgan’s Law, Theorem 1.2.8c and Theorem 1.2.9b,

\begin{equation*} \begin{aligned} P(A^c\cap B^c) &= P\bigl((A\cup B)^c\bigr) = 1 - P(A\cup B)\\ &= 1 - P(A) - P(B) + P(A)P(B)\\ &= \bigl(1-P(A)\bigr)\bigl(1-P(B)\bigr) = P(A^c)P(B^c). \end{aligned} \end{equation*}

(a) \(P(\text{dash sent} \mid \text{dash received}) = 32/41\).

The proportion 3:4 gives the prior \(P(\text{dot sent}) = \tfrac37\), \(P(\text{dash sent}) = \tfrac47\), and the stated error rates give the transmission channel

\begin{equation*} \begin{aligned} P(\text{dash rec}\mid\text{dot sent}) &= \tfrac14, & P(\text{dot rec}\mid\text{dot sent}) &= \tfrac34,\\ P(\text{dot rec}\mid\text{dash sent}) &= \tfrac13, & P(\text{dash rec}\mid\text{dash sent}) &= \tfrac23 . \end{aligned} \end{equation*}

Since \(\{\text{dot sent},\text{dash sent}\}\) partitions the sample space, Bayes’ Rule (Theorem 1.3.5) applies:

\begin{equation*} \begin{aligned} P(\text{dash rec}) &= \tfrac23\cdot\tfrac47 + \tfrac14\cdot\tfrac37 = \tfrac{8}{21} + \tfrac{3}{28} = \tfrac{41}{84},\\[2pt] P(\text{dash sent}\mid\text{dash rec}) &= \frac{(2/3)(4/7)}{41/84} = \frac{32/84}{41/84} = \frac{32}{41}. \end{aligned} \end{equation*}

(b) The posterior on the four messages dot-dot, dot-dash, dash-dot, dash-dash is

\begin{equation*} \frac{729}{1849},\ \frac{432}{1849},\ \frac{432}{1849},\ \frac{256}{1849}. \end{equation*}

Write \(a = P(\text{dot sent},\,\text{dot rec}) = \tfrac34\cdot\tfrac37 = \tfrac{27}{84}\) and \(b = P(\text{dash sent},\,\text{dot rec}) = \tfrac13\cdot\tfrac47 = \tfrac{16}{84}\), so that \(P(\text{dot rec}) = a+b = \tfrac{43}{84}\). By independence of the two symbols, the joint probability of sending \((s_1,s_2)\) and receiving dot-dot factors, so Bayes’ Rule gives, for each message \((s_1,s_2)\),

\begin{equation*} P\bigl((s_1,s_2)\text{ sent}\mid\text{dot-dot rec}\bigr) = \frac{c_{s_1}c_{s_2}}{(a+b)^2}, \qquad c_{\text{dot}} = a,\ c_{\text{dash}} = b . \end{equation*}

With \(84^2\) cancelling throughout, the numerators are \(27^2, 27\cdot16, 16\cdot27, 16^2\) over \(43^2 = 1849\):

(a) A sample point lies in \(\cup_{i=1}^n A_i\) if and only if it lies in exactly \(k\) of the \(A_i\) for some \(k\) with \(1\le k\le n\), so

\begin{equation*} \bigcup_{i=1}^n A_i = \bigcup_{k=1}^n E_k , \end{equation*}

and the \(E_k\) are pairwise disjoint (a point lies in exactly one of the counts \(1,\ldots,n\)). Axiom 3 of Definition 1.2.4 gives \(P(\cup_{i=1}^n A_i) = \sum_{k=1}^n P(E_k)\).

(b) Fix the \(k\)-subset and count. Under the stated normalization the points of \(E_k\) lie in \(A_1,\ldots,A_k\) and in no other \(A_i\), so for any index set \(J \subset \{1,\ldots,n\}\),

\begin{equation*} E_k \cap \bigcap_{i\in J} A_i = \begin{cases} E_k, & J \subset \{1,\ldots,k\},\\[2pt] \emptyset, & \text{otherwise.} \end{cases} \end{equation*}

The sum \(P_j\) runs over the \(\binom{n}{j}\) index sets \(J\) of size \(j\), and \(E_k\) contributes \(P(E_k)\) to the term for \(J\) exactly when \(J\subset\{1,\ldots,k\}\), which happens for \(\binom{k}{j}\) of them. Hence \(P(E_k)\) appears \(\binom{k}{1}=k\) times in \(P_1\), \(\binom{k}{2}\) times in \(P_2\), \(\binom{k}{3}\) times in \(P_3\), and \(\binom{k}{j}\) times in \(P_j\) (so not at all once \(j>k\)).

(This is the force of the without-loss-of-generality: in general \(E_k\) splits into disjoint pieces, one for each \(k\)-subset \(S\) of the indices, the display above applies to each piece with \(\{1,\ldots,k\}\) replaced by \(S\), and the counts \(\binom{k}{j}\) depend only on \(|S|=k\).)

(c) Since \(\binom{k}{0}=1\) and \(\binom{k}{1}=k\), Exercise 1.27(a) with \(n=k\) (legitimate for \(k\ge 2\)),

\begin{equation*} \sum_{j=0}^{k} (-1)^j \binom{k}{j} = 0 , \end{equation*}

rearranges after moving the \(j=0\) term across as

\begin{equation*} \sum_{j=1}^{k} (-1)^{j+1}\binom{k}{j} = k - \binom{k}{2} + \binom{k}{3} - \cdots \pm \binom{k}{k} = 1 . \end{equation*}

For \(k=1\) the left side is the single term \(\binom{1}{1}=1\).

(d) By (b), each \(P_j = \sum_{k=j}^{n} \binom{k}{j} P(E_k)\), so interchanging the two finite sums and applying (c),

\begin{equation*} \begin{aligned} \sum_{j=1}^{n} (-1)^{j+1} P_j &= \sum_{j=1}^{n} (-1)^{j+1} \sum_{k=1}^{n} \binom{k}{j} P(E_k)\\ &= \sum_{k=1}^{n} P(E_k) \sum_{j=1}^{k} (-1)^{j+1}\binom{k}{j}\\ &= \sum_{k=1}^{n} P(E_k) \cdot 1 , \end{aligned} \end{equation*}

where \(\binom{k}{j}=0\) for \(j>k\) removes the extra terms. Combining with (a),

\begin{equation*} P\Bigl(\bigcup_{i=1}^n A_i\Bigr) = \sum_{k=1}^n P(E_k) = P_1 - P_2 + P_3 - \cdots \pm P_n . \end{equation*}

(a) Boole’s Inequality (Theorem 1.2.11b) is the first bound of the display, \(P(\cup_{i=1}^n A_i)\le P_1=\sum_{i=1}^n P(A_i)\), which is the case \(m=1\) of the error formula proved in (b). Bonferroni’s Inequality follows by applying Boole to the complements: since \(\cup_i A_i^c = (\cap_i A_i)^c\),

\begin{equation*} \begin{aligned} 1-P\Big(\bigcap_{i=1}^n A_i\Big) &=P\Big(\bigcup_{i=1}^n A_i^c\Big)\\ &\le\sum_{i=1}^n\big(1-P(A_i)\big), \end{aligned} \end{equation*}

that is, \(P(\cap_{i=1}^n A_i)\ge\sum_{i=1}^n P(A_i)-(n-1)\), which is (1.2.10); the case \(n=2\) is (1.2.9).

(b) Let \(E_k\) be the set of sample points lying in exactly \(k\) of \(A_1,\dots,A_n\). Exercise 1.42 supplies the two facts \(P(\cup_{i=1}^n A_i)=\sum_{k\ge1}P(E_k)\) and \(P_j=\sum_{k\ge j}\binom{k}{j}P(E_k)\). Write \(S_m=\sum_{j=1}^m(-1)^{j+1}P_j\) for the \(m\)-term bound. Interchanging the (finite) sums and using \(\sum_{j=0}^m(-1)^j\binom{k}{j}=(-1)^m\binom{k-1}{m}\) (Exercise 1.27),

\begin{equation*} \begin{aligned} S_m &= \sum_{k\ge1}P(E_k)\sum_{j=1}^m(-1)^{j+1}\binom{k}{j}\\ &= \sum_{k\ge1}P(E_k)\Big[1-(-1)^m\binom{k-1}{m}\Big], \end{aligned} \end{equation*}

whence the exact error of the \(m\)-term bound,

\begin{equation*} S_m-P\Big(\bigcup_{i=1}^n A_i\Big)=(-1)^{m+1}\sum_{k>m}\binom{k-1}{m}P(E_k). \end{equation*}

Odd \(m\) therefore give upper bounds and even \(m\) lower bounds (this is the displayed sequence), with error magnitude \(D_m=\sum_{k>m}\binom{k-1}{m}P(E_k)\). Since \(\binom{k-1}{m+1}\le\binom{k-1}{m}\) exactly when \(k\le 2m+2\), and \(k\le n\) always, \(D_{m+1}\le D_m\) for every \(m\ge (n-2)/2\); and \(D_n=0\), so the bounds do improve and close to equality, the identity itself being the case \(m=n\).

The printed ordering \(P_i\ge P_j\) for \(i\le j\) is, however, false as stated: with \(n=4\) and \(A_1=A_2=A_3=A_4=A\), \(P(A)=1\), one has \(P_1=4<6=P_2\). Since \(\binom{k}{j+1}\le\binom{k}{j}\) exactly when \(k\le 2j+1\), what is true is that \(P_j\ge P_{j+1}\) whenever \(P(E_k)=0\) for all \(k>2j+1\); and that ordering is precisely what makes the bounds improve at every step, because

\begin{equation*} S_{m+2}-S_m=(-1)^m\big(P_{m+1}-P_{m+2}\big). \end{equation*}

Part (c) is the extreme failure of the criterion.

(c) With \(A_i=A\) for every \(i\) we have \(P_j=\binom{n}{j}P(A)\), so by the computation in (b) with \(P(E_n)=P(A)\),

\begin{equation*} S_m=P(A)\sum_{j=1}^m(-1)^{j+1}\binom{n}{j}=P(A)\Big[1-(-1)^m\binom{n-1}{m}\Big]. \end{equation*}

The \(m\)-term bound thus misses the true value \(P(\cup_i A_i)=P(A)\) by \(P(A)\binom{n-1}{m}\), an error that increases with \(m\) up to \(m=\lfloor (n-1)/2\rfloor\) before decreasing, and is zero only at \(m=n\). For \(n=10\) and \(P(A)=.1\) the successive bounds are

\begin{equation*} S_1=1.0,\quad S_2=-3.5,\quad S_3=8.5,\quad S_4=-12.5,\ \dots,\quad S_{10}=.1, \end{equation*}

every intermediate one vacuous: with perfectly correlated events no partial expansion improves on the trivial bounds, and all \(n\) terms are needed.

\(P(X\ge 10)=.0139\).

Let \(C_i\) be the event that question \(i\) is answered correctly. Guessing uniformly among the 4 answers gives \(P(C_i)=\tfrac14\), and the \(C_i\) are mutually independent (Definition 1.3.12), so \(X=\#\{i: C_i \text{ occurs}\}\) has the binomial form: a fixed pattern with \(x\) correct and \(20-x\) incorrect has probability \((\tfrac14)^x(\tfrac34)^{20-x}\) by independence, and there are \(\binom{20}{x}\) such patterns. Hence

\begin{equation*} P(X\ge10)=\sum_{x=10}^{20}\binom{20}{x}\Big(\frac14\Big)^{x}\Big(\frac34\Big)^{20-x} =\frac{1}{4^{20}}\sum_{x=10}^{20}\binom{20}{x}3^{20-x}. \end{equation*}

The sum is \(15{,}244{,}087{,}642\) and \(4^{20}=1{,}099{,}511{,}627{,}776\), so

\begin{equation*} P(X\ge10)=\frac{15{,}244{,}087{,}642}{1{,}099{,}511{,}627{,}776}=.01386. \end{equation*}

Write \(X^{-1}(A)=\{s\in S: X(s)\in A\}\), so that (1.4.2) reads \(P_X(A)=P\big(X^{-1}(A)\big)\); each of the three axioms of Definition 1.2.4 for \(P_X\) is the corresponding axiom for \(P\) transported through \(X^{-1}\).

(1) \(P_X(A)=P\big(X^{-1}(A)\big)\ge 0\), since \(P\) satisfies the first axiom.

(2) \(X(s)\in\mathcal X\) for every \(s\in S\), so \(X^{-1}(\mathcal X)=S\) and \(P_X(\mathcal X)=P(S)=1\).

(3) Let \(A_1,A_2,\dots\subset\mathcal X\) be pairwise disjoint. Their preimages are pairwise disjoint, since \(s\in X^{-1}(A_i)\cap X^{-1}(A_j)\) would force \(X(s)\in A_i\cap A_j=\emptyset\), and preimages respect unions:

\begin{equation*} X^{-1}\Big(\bigcup_{i=1}^{\infty}A_i\Big)=\bigcup_{i=1}^{\infty}X^{-1}(A_i), \end{equation*}

both sides being \(\{s: X(s)\in A_i \text{ for some } i\}\). Countable additivity of \(P\) therefore gives

\begin{equation*} P_X\Big(\bigcup_{i=1}^{\infty}A_i\Big) =P\Big(\bigcup_{i=1}^{\infty}X^{-1}(A_i)\Big) =\sum_{i=1}^{\infty}P\big(X^{-1}(A_i)\big) =\sum_{i=1}^{\infty}P_X(A_i). \end{equation*}

Here randomly means the \(7^7=823{,}543\) assignments of the seven (distinguishable) balls to the seven cells are equally likely, so every probability below is a count divided by \(7^7\). Only \(x=0,1,2\) are possible, since three cells with three balls each would need nine balls.

\(x=2\): pick the two triply occupied cells, \(\binom{7}{2}=21\) ways; pick their balls, \(\binom{7}{3}\binom{4}{3}=35\cdot4=140\) ways; drop the last ball in any of the remaining five cells, 5 ways. Thus

\begin{equation*} P(X_3=2)=\frac{21\cdot140\cdot 5}{7^7}=\frac{14{,}700}{823{,}543}=\frac{300}{16807}. \end{equation*}

\(x=1\): pick the triple cell, 7 ways, and its three balls, \(\binom{7}{3}=35\) ways; the remaining four balls go into the other six cells with no cell receiving exactly three. Of the \(6^4=1296\) placements, those with a triple are counted as \(6\cdot\binom{4}{3}\cdot 5=120\) (cell, its three balls, the stray ball elsewhere), and no placement of four balls has two triples, so \(1296-120=1176\) are admissible. Thus

\begin{equation*} P(X_3=1)=\frac{7\cdot 35\cdot 1176}{7^7}=\frac{288{,}120}{823{,}543}=\frac{120}{343}. \end{equation*}

\(x=0\): by complementation,

\begin{equation*} \begin{aligned} P(X_3=0)&=1-\frac{288{,}120+14{,}700}{823{,}543}\\ &=\frac{520{,}723}{823{,}543}=\frac{10627}{16807}. \end{aligned} \end{equation*}

In (a)-(d) the function is differentiable with \(F^{\prime}>0\) on the interior of its domain, which delivers conditions (b) and (c) of Theorem 1.5.3 at once, so only the limits of (a) are at issue; (e) is not continuous and is treated separately.

(a) \(\lim_{x\to-\infty}\tan^{-1}x=-\pi/2\) and \(\lim_{x\to\infty}\tan^{-1}x=\pi/2\), so \(F\) runs from \(\frac12-\frac12=0\) to \(\frac12+\frac12=1\), and

\begin{equation*} F^{\prime}(x)=\frac{1}{\pi(1+x^2)}>0 . \end{equation*}

(b) As \(x\to-\infty\), \(e^{-x}\to\infty\) and \(F(x)\to0\); as \(x\to\infty\), \(e^{-x}\to0\) and \(F(x)\to1\). Moreover

\begin{equation*} F^{\prime}(x)=\frac{e^{-x}}{\left(1+e^{-x}\right)^{2}}>0 . \end{equation*}

(This is the logistic cdf (1.5.5) of Example 1.5.5.)

(c) As \(x\to-\infty\), \(e^{-x}\to\infty\), so \(e^{-e^{-x}}\to 0\); as \(x\to\infty\), \(e^{-x}\to0\), so \(e^{-e^{-x}}\to e^{0}=1\). And

\begin{equation*} F^{\prime}(x)=e^{-e^{-x}}e^{-x}>0 . \end{equation*}

(d) Extend \(F\) to the whole line by \(F(x)=0\) for \(x\le0\), the only extension that is a cdf, since \(1-e^{-x}\to0\) as \(x\downarrow0\) makes the extension continuous there. Then \(\lim_{x\to-\infty}F(x)=0\), \(\lim_{x\to\infty}\left(1-e^{-x}\right)=1\), and \(F^{\prime}(x)=e^{-x}>0\) on \((0,\infty)\) while \(F\) is constant on \((-\infty,0]\); a continuous function that is nondecreasing on each of two abutting intervals is nondecreasing.

(e) Both pieces are \((1-\epsilon)\) times the increasing function of part (b), shifted by a constant, so \(F_Y\) is increasing on \((-\infty,0)\) and on \([0,\infty)\). Its limits are

\begin{equation*} \lim_{y\to-\infty}F_Y(y)=0,\qquad \lim_{y\to\infty}F_Y(y)=\epsilon+(1-\epsilon)=1 . \end{equation*}

At the only possible break, \(y=0\),

\begin{equation*} \lim_{y\uparrow 0}F_Y(y)=\frac{1-\epsilon}{2} \;<\;\epsilon+\frac{1-\epsilon}{2}=F_Y(0)=\lim_{y\downarrow 0}F_Y(y), \end{equation*}

since \(F_Y\) is given by its second branch on all of \([0,\infty)\). So \(F_Y\) is nondecreasing across \(0\) as well, and right-continuous everywhere, with a jump of height \(\epsilon\) at the origin.

Let \(F(x)=F_X(x)=P(X\le x)\) for a random variable \(X\) on \((S,P)\) (Definition 1.5.1); all three properties are properties of \(P\) along monotone sequences of events. Those we need first: if \(A_1\subset A_2\subset\cdots\) with \(A=\cup_n A_n\), then writing \(A\) as the disjoint union \(A_1\cup\bigcup_{n\ge2}(A_n\cap A_{n-1}^c)\) and applying the third Kolmogorov Axiom gives \(P(A_n)\to P(A)\); taking complements gives the decreasing version, \(A_1\supset A_2\supset\cdots\) with \(A=\cap_n A_n\) implies \(P(A_n)\to P(A)\).

(b) If \(x<y\) then \(\{X\le x\}\subset\{X\le y\}\), so \(F(x)\le F(y)\) by Theorem 1.2.9c. Monotonicity is proved first because it is what turns each sequential limit below into a genuine limit.

(a) The events \(\{X\le -n\}\) decrease, and their intersection is empty: \(X(s)\) is a real number for every \(s\in S\), so \(X(s)>-n\) for all large \(n\). Hence \(F(-n)\to P(\emptyset)=0\). Likewise \(\{X\le n\}\) increase to \(S\), so \(F(n)\to P(S)=1\). Since \(F\) is monotone and bounded, the limits along these sequences are the limits as \(x\to-\infty\) and \(x\to\infty\).

(c) Fix \(x_0\). The events \(\{X\le x_0+\frac1n\}\) decrease with

\begin{equation*} \bigcap_{n=1}^{\infty}\Big\{X\le x_0+\tfrac1n\Big\}=\{X\le x_0\}, \end{equation*}

because \(X(s)\le x_0+\frac1n\) for all \(n\) forces \(X(s)\le x_0\). Therefore \(F(x_0+\frac1n)\to F(x_0)\), and monotonicity of \(F\) upgrades this to \(\lim_{x\downarrow x_0}F(x)=F(x_0)\).

Complementation turns each hypothesis directly into its conclusion. Since \(\{X>t\}=\{X\le t\}^{c}\), Theorem 1.2.8c gives \(P(X>t)=1-F_X(t)\) and likewise \(P(Y>t)=1-F_Y(t)\). Hence for every \(t\), the hypothesis \(F_X(t)\le F_Y(t)\) yields

\begin{equation*} P(X>t)=1-F_X(t)\;\ge\;1-F_Y(t)=P(Y>t), \end{equation*}

and at a \(t\) with \(F_X(t)<F_Y(t)\) – one exists by the definition of stochastically greater – the same line with strict inequalities gives \(P(X>t)>P(Y>t)\).

Induction on \(n\); the entire content is the one-line step

\begin{equation*} \frac{1-t^{n}}{1-t} + t^{n} = \frac{1-t^{n+1}}{1-t}. \end{equation*}

For \(n=1\) both sides equal \(1\). Assuming the formula for \(n\),

\begin{equation*} \begin{aligned} \sum_{k=1}^{n+1} t^{k-1} &= \frac{1-t^{n}}{1-t} + t^{n} \\ &= \frac{1-t^{n}+t^{n}(1-t)}{1-t} \\ &= \frac{1-t^{n+1}}{1-t}, \end{aligned} \end{equation*}

which is the formula for \(n+1\).

Method (2): telescoping, with no induction. For \(t \neq 1\),

\begin{equation*} (1-t)\sum_{k=1}^{n} t^{k-1} = \sum_{k=1}^{n} t^{k-1} - \sum_{k=1}^{n} t^{k} = 1 - t^{n}, \end{equation*}

and divide by \(1-t\).

\(X\) is hypergeometric:

\begin{equation*} f_X(x) = P(X=x) = \frac{\binom{5}{x}\binom{25}{4-x}}{\binom{30}{4}}, \qquad x = 0,1,2,3,4 , \end{equation*}

since the \(\binom{30}{4} = 27{,}405\) equally likely samples of size 4 split according to how many of the 5 defectives they contain. (The five counts below sum to \(27{,}405\), so \(\sum_x f_X(x)=1\).)

The cdf is the step function

\begin{equation*} F_X(x) = \begin{cases} 0 & x < 0 \\ 0.4616 & 0 \le x < 1 \\ 0.8812 & 1 \le x < 2 \\ 0.9907 & 2 \le x < 3 \\ 0.9998 & 3 \le x < 4 \\ 1 & x \ge 4 . \end{cases} \end{equation*}

The plot is a right-continuous staircase, flat between integers, with jumps at \(x=0,1,2,3,4\) of heights \(0.4616\), \(0.4196\), \(0.1095\), \(0.0091\), \(0.0002\).

\(g\) satisfies the two conditions of Theorem 1.6.5: it is nonnegative, and it integrates to 1.

Nonnegativity: \(f(x) \ge 0\) and \(1-F(x_0) > 0\) by the standing assumption \(F(x_0)<1\), so \(g(x) \ge 0\) for every \(x\).

Total mass: by (1.6.3), \(\int_{x_0}^{\infty} f(x)\,dx = 1 - F(x_0)\), hence

\begin{equation*} \int_{-\infty}^{\infty} g(x)\,dx = \frac{1}{1-F(x_0)}\int_{x_0}^{\infty} f(x)\,dx = \frac{1-F(x_0)}{1-F(x_0)} = 1 . \end{equation*}

(a) The three conditions of Theorem 1.5.3 hold for

\begin{equation*} F_Y(y) = \begin{cases} 0 & y < 1 \\ 1 - y^{-2} & y \ge 1 , \end{cases} \end{equation*}

namely: \(\lim_{y \to -\infty} F_Y(y) = 0\) and \(\lim_{y \to \infty} F_Y(y) = 1 - \lim_{y \to \infty} y^{-2} = 1\); \(F_Y\) is nondecreasing, being constant on \((-\infty,1)\) and having \(\frac{d}{dy}(1-y^{-2}) = 2y^{-3} > 0\) on \((1,\infty)\); and \(F_Y\) is continuous everywhere (at the join, \(F_Y(1) = 1 - 1 = 0\)), hence in particular right-continuous.

(b) Differentiating, by (1.6.3),

\begin{equation*} f_Y(y) = \frac{d}{dy}\left(1 - \frac{1}{y^{2}}\right) = \frac{2}{y^{3}}, \qquad 1 < y < \infty , \end{equation*}

and \(f_Y(y)=0\) otherwise.

(c) \(Z = 10(Y-1)\) is increasing in \(Y\), so for \(z \ge 0\),

\begin{equation*} \begin{aligned} F_Z(z) &= P\big(10(Y-1) \le z\big) \\ &= P\!\left(Y \le 1 + \tfrac{z}{10}\right) \\ &= 1 - \frac{1}{\left(1+\frac{z}{10}\right)^{2}} = 1 - \frac{100}{(z+10)^{2}} , \end{aligned} \end{equation*}

and \(F_Z(z) = 0\) for \(z < 0\) (since \(Y \ge 1\) gives \(Z \ge 0\)).

(a) \(c = 1\); (b) \(c = \tfrac{1}{2}\). In each case the integrand is already nonnegative for \(c>0\), so condition (a) of Theorem 1.6.5 is automatic and only \(\int f = 1\) is at issue.

(a)

\begin{equation*} \int_{0}^{\pi/2} c \sin x \, dx = c \big[-\cos x\big]_{0}^{\pi/2} = c(0+1) = c , \end{equation*}

so \(c = 1\).

(b) The integrand is even, so

\begin{equation*} \int_{-\infty}^{\infty} c e^{-|x|} \, dx = 2c \int_{0}^{\infty} e^{-x} \, dx = 2c , \end{equation*}

so \(c = \tfrac{1}{2}\).

\begin{equation*} F_V(v) = \begin{cases} 0 & v < 5 \\ 1 - e^{-2} & 5 \le v < 6 \\ 1 - e^{-v/3} & v \ge 6 . \end{cases} \end{equation*}

\(V\) is mixed: it puts an atom at \(5\) and is otherwise supported on \([6,\infty)\), since \(V = 2T \ge 6\) on the event \(\{T \ge 3\}\). From \(f_T\), \(P(T > t) = e^{-t/1.5}\) for \(t>0\), so

\begin{equation*} P(V = 5) = P(T < 3) = 1 - e^{-3/1.5} = 1 - e^{-2} \end{equation*}

and nothing is contributed between \(5\) and \(6\), giving the middle line. For \(v \ge 6\),

\begin{equation*} \begin{aligned} F_V(v) &= P(T < 3) + P\big(3 \le T \le \tfrac{v}{2}\big) \\ &= \big(1 - e^{-2}\big) + \big(e^{-2} - e^{-v/3}\big) \\ &= 1 - e^{-v/3}, \end{aligned} \end{equation*}

using \(e^{-(v/2)/1.5} = e^{-v/3}\).

Every \(g\) here is monotone on the support, so Theorem 2.1.5 applies directly:

\begin{equation*} f_Y(y) = f_X\bigl(g^{-1}(y)\bigr)\left|\frac{d}{dy}g^{-1}(y)\right| , \qquad y \in \mathcal{Y} . \end{equation*}

(a) \(g(x) = x^3\) is increasing on \(\mathcal{X} = (0,1)\), so \(\mathcal{Y} = (0,1)\), \(g^{-1}(y) = y^{1/3}\) and \(\tfrac{d}{dy}g^{-1}(y) = \tfrac13 y^{-2/3}\):

\begin{equation*} \begin{aligned} f_Y(y) &= 42\,y^{5/3}\bigl(1 - y^{1/3}\bigr)\cdot \tfrac13 y^{-2/3}\\ &= 14\bigl(y - y^{4/3}\bigr), \qquad 0 < y < 1 . \end{aligned} \end{equation*}

Then \(\int_0^1 14(y - y^{4/3})\,dy = 14\bigl(\tfrac12 - \tfrac37\bigr) = 14\cdot\tfrac{1}{14} = 1\).

(b) \(g(x) = 4x+3\) is increasing on \(\mathcal{X} = (0,\infty)\), so \(\mathcal{Y} = (3,\infty)\), \(g^{-1}(y) = (y-3)/4\) and \(\tfrac{d}{dy}g^{-1}(y) = \tfrac14\):

\begin{equation*} f_Y(y) = \tfrac{7}{4}\,e^{-7(y-3)/4}, \qquad 3 < y < \infty , \end{equation*}

an exponential shifted to start at 3; \(\int_3^\infty \tfrac74 e^{-7(y-3)/4}\,dy = \tfrac74\cdot\tfrac47 = 1\).

(c) \(g(x) = x^2\) is increasing on \(\mathcal{X} = (0,1)\) (the negative branch is not in the support, so the two-to-one difficulty of Example 2.1.7 does not arise), so \(\mathcal{Y} = (0,1)\), \(g^{-1}(y) = \sqrt{y}\) and \(\tfrac{d}{dy}g^{-1}(y) = 1/(2\sqrt{y})\):

\begin{equation*} \begin{aligned} f_Y(y) &= 30\,y\bigl(1-\sqrt{y}\bigr)^2 \cdot \frac{1}{2\sqrt{y}}\\ &= 15\sqrt{y}\,\bigl(1-\sqrt{y}\bigr)^2, \qquad 0 < y < 1 . \end{aligned} \end{equation*}

Substituting \(u = \sqrt{y}\), \(dy = 2u\,du\),

\begin{equation*} \int_0^1 15\sqrt{y}\,(1-\sqrt{y})^2\,dy = 30\int_0^1 u^2(1-u)^2\,du = 30\cdot\frac{2!\,2!}{5!} = \frac{30\cdot 4}{120} = 1 , \end{equation*}

the beta integral of Section 3.3 with \(\alpha = \beta = 3\).

Each \(g\) is monotone on the support, so Theorem 2.1.5 gives the answer at once.

(a) \(g^{-1}(y) = \sqrt{y}\) on \(\mathcal{Y} = (0,1)\), so

\begin{equation*} f_Y(y) = 1\cdot\frac{1}{2\sqrt{y}} = \frac{1}{2\sqrt{y}}, \qquad 0 < y < 1 . \end{equation*}

(b) \(g(x) = -\log x\) is decreasing on \((0,1)\) with \(\mathcal{Y} = (0,\infty)\), \(g^{-1}(y) = e^{-y}\) and \(|\tfrac{d}{dy}g^{-1}(y)| = e^{-y}\) (as in Example 2.1.4):

\begin{equation*} \begin{aligned} f_Y(y) &= \frac{(n+m+1)!}{n!\,m!}\,e^{-ny}\bigl(1-e^{-y}\bigr)^m\,e^{-y}\\ &= \frac{(n+m+1)!}{n!\,m!}\,e^{-(n+1)y}\bigl(1-e^{-y}\bigr)^m, \qquad 0 < y < \infty . \end{aligned} \end{equation*}

(c) \(g(x) = e^x\) is increasing on \((0,\infty)\) with \(\mathcal{Y} = (1,\infty)\), \(g^{-1}(y) = \log y\) and \(\tfrac{d}{dy}g^{-1}(y) = 1/y\):

\begin{equation*} f_Y(y) = \frac{\log y}{\sigma^2 y}\, \exp\!\left(-\frac{(\log y)^2}{2\sigma^2}\right), \qquad 1 < y < \infty . \end{equation*}

\begin{equation*} f_Y(y) = \frac13\left(\frac23\right)^{y/(1-y)}, \qquad y = 0,\ \tfrac12,\ \tfrac23,\ \tfrac34,\ \ldots,\ \tfrac{x}{x+1},\ \ldots \end{equation*}

Indeed \(g(x) = x/(x+1)\) is strictly increasing on \(\mathcal{X} = \{0,1,2,\ldots\}\), hence one-to-one, and

\begin{equation*} y = \frac{x}{x+1} \iff x = \frac{y}{1-y}, \end{equation*}

so \(g^{-1}(y) = y/(1-y)\) is single-valued and \(\mathcal{Y} = \{x/(x+1) : x = 0,1,2,\ldots\}\), a subset of \([0,1)\) increasing to 1. By the discrete transformation formula following (2.1.2), \(f_Y(y) = \sum_{x \in g^{-1}(y)} f_X(x) = f_X\!\left(\frac{y}{1-y}\right)\), which is the display above.

The two branches are the single formula \(f(x) = \tfrac12\lambda e^{-\lambda|x|}\) on \(-\infty < x < \infty\), the double exponential pdf.

(a) \(f \ge 0\) everywhere since \(\lambda > 0\), and by symmetry of \(e^{-\lambda|x|}\),

\begin{equation*} \int_{-\infty}^{\infty} f(x)\,dx = 2\int_{0}^{\infty} \tfrac12\lambda e^{-\lambda x}\,dx = \Bigl[-e^{-\lambda x}\Bigr]_{0}^{\infty} = 1 , \end{equation*}

so the two conditions of Theorem 1.6.5 hold and \(f\) is a pdf.

(b) For \(t < 0\) only the left branch is involved:

\begin{equation*} P(X < t) = \int_{-\infty}^{t} \tfrac12\lambda e^{\lambda x}\,dx = \Bigl[\tfrac12 e^{\lambda x}\Bigr]_{-\infty}^{t} = \tfrac12 e^{\lambda t} . \end{equation*}

For \(t \ge 0\) split at 0, where the display just obtained gives \(P(X < 0) = \tfrac12\):

\begin{equation*} \begin{aligned} P(X < t) &= \tfrac12 + \int_{0}^{t} \tfrac12\lambda e^{-\lambda x}\,dx\\ &= \tfrac12 + \tfrac12\bigl(1 - e^{-\lambda t}\bigr) = 1 - \tfrac12 e^{-\lambda t} . \end{aligned} \end{equation*}

(c) \(P(|X| < t) = 0\) for \(t \le 0\). For \(t > 0\), symmetry gives

\begin{equation*} P(|X| < t) = P(-t < X < t) = 2\int_{0}^{t}\tfrac12\lambda e^{-\lambda x}\,dx = 1 - e^{-\lambda t} . \end{equation*}

\begin{equation*} f_Y(y) = \frac{1}{\pi\sqrt{y(1-y)}}, \qquad 0 < y < 1, \end{equation*}

the beta\((\tfrac12,\tfrac12)\) (arcsine) pdf.

Method (1), Theorem 2.1.8. The function \(g(x) = \sin^2 x\) is monotone on each quarter-period, so take

\begin{equation*} \begin{aligned} A_0 &= \{\pi/2,\ \pi,\ 3\pi/2\}, &&P(X \in A_0) = 0,\\ A_1 &= (0,\pi/2), & g_1^{-1}(y) &= \arcsin\sqrt{y},\\ A_2 &= (\pi/2,\pi), & g_2^{-1}(y) &= \pi - \arcsin\sqrt{y},\\ A_3 &= (\pi,3\pi/2), & g_3^{-1}(y) &= \pi + \arcsin\sqrt{y},\\ A_4 &= (3\pi/2,2\pi), & g_4^{-1}(y) &= 2\pi - \arcsin\sqrt{y}. \end{aligned} \end{equation*}

These four sets and \(A_0\) partition \(\mathcal{X} = (0,2\pi)\), each \(g_i\) carries \(A_i\) onto the common \(\mathcal{Y} = (0,1)\), so hypothesis (iii) holds, and by the chain rule

\begin{equation*} \frac{d}{dy}\arcsin\sqrt{y} = \frac{1}{\sqrt{1-y}}\cdot\frac{1}{2\sqrt{y}} = \frac{1}{2\sqrt{y(1-y)}}, \end{equation*}

which is continuous on \((0,1)\), giving (iv). Hence \(|\tfrac{d}{dy}g_i^{-1}(y)| = 1/(2\sqrt{y(1-y)})\) for all four \(i\), and since \(f_X \equiv 1/(2\pi)\) on each \(A_i\),

\begin{equation*} f_Y(y) = 4\cdot\frac{1}{2\pi}\cdot\frac{1}{2\sqrt{y(1-y)}} = \frac{1}{\pi\sqrt{y(1-y)}}, \qquad 0 < y < 1 . \end{equation*}

Method (2), differentiating (2.1.6). The two solutions of \(\sin^2 x = y\) in \((0,\pi)\) are \(x_1 = \arcsin\sqrt{y}\) and \(x_2 = \pi - \arcsin\sqrt{y}\), so by uniformity

\begin{equation*} P(X \le x_1) = \frac{\arcsin\sqrt{y}}{2\pi}, \qquad P(x_2 \le X \le \pi) = \frac{\pi - x_2}{2\pi} = \frac{\arcsin\sqrt{y}}{2\pi}, \end{equation*}

and (2.1.6) becomes

\begin{equation*} F_Y(y) = \frac{2}{\pi}\arcsin\sqrt{y}, \qquad 0 < y < 1 . \end{equation*}

Differentiating with the derivative computed above,

\begin{equation*} f_Y(y) = \frac{2}{\pi}\cdot\frac{1}{2\sqrt{y(1-y)}} = \frac{1}{\pi\sqrt{y(1-y)}}, \end{equation*}

the same answer.

Each \(g\) is two-piece, so Theorem 2.1.8 applies with \(A_0 = \{0\}\) and \(A_1\), \(A_2\) the negative and positive parts of \(\mathcal{X}\).

(a) Here \(g_1(x) = -x^3\) on \(A_1 = (-\infty,0)\) and \(g_2(x) = x^3\) on \(A_2 = (0,\infty)\), both with common image \(\mathcal{Y} = (0,\infty)\); the inverses are \(g_1^{-1}(y) = -y^{1/3}\), \(g_2^{-1}(y) = y^{1/3}\), each with \(|\tfrac{d}{dy}g_i^{-1}(y)| = \tfrac13 y^{-2/3}\), continuous on \((0,\infty)\). Since \(f_X(\pm y^{1/3}) = \tfrac12 e^{-y^{1/3}}\),

\begin{equation*} f_Y(y) = 2\cdot\tfrac12 e^{-y^{1/3}}\cdot\tfrac13 y^{-2/3} = \tfrac13 y^{-2/3}e^{-y^{1/3}}, \qquad 0 < y < \infty . \end{equation*}

With \(u = y^{1/3}\), \(du = \tfrac13 y^{-2/3}dy\), we get \(\int_0^\infty f_Y(y)\,dy = \int_0^\infty e^{-u}\,du = 1\).

(b) \(g(x) = 1-x^2\) is increasing on \(A_1 = (-1,0)\) and decreasing on \(A_2 = (0,1)\), with common image \(\mathcal{Y} = (0,1)\), inverses \(g_1^{-1}(y) = -\sqrt{1-y}\) and \(g_2^{-1}(y) = \sqrt{1-y}\), and \(|\tfrac{d}{dy}g_i^{-1}(y)| = 1/(2\sqrt{1-y})\). Writing \(a = \sqrt{1-y}\) and using \((1-a)^2 + (1+a)^2 = 2 + 2a^2 = 4 - 2y\),

\begin{equation*} \begin{aligned} f_Y(y) &= \frac{1}{2\sqrt{1-y}}\Bigl[\tfrac38(1-a)^2 + \tfrac38(1+a)^2\Bigr]\\ &= \frac{3}{8}\cdot\frac{4-2y}{2\sqrt{1-y}} = \frac{3(2-y)}{8\sqrt{1-y}}, \qquad 0 < y < 1 . \end{aligned} \end{equation*}

Substituting \(u = 1-y\),

\begin{equation*} \int_0^1 \frac{3(2-y)}{8\sqrt{1-y}}\,dy = \frac38\int_0^1 \frac{1+u}{\sqrt{u}}\,du = \frac38\Bigl(2 + \tfrac23\Bigr) = 1 . \end{equation*}

(c) Now \(g_1(x) = 1-x^2\) on \(A_1 = (-1,0)\) and \(g_2(x) = 1-x\) on \(A_2 = (0,1)\), each monotone with common image \(\mathcal{Y} = (0,1)\); the inverses are \(g_1^{-1}(y) = -\sqrt{1-y}\) with \(|\tfrac{d}{dy}g_1^{-1}(y)| = 1/(2\sqrt{1-y})\), and \(g_2^{-1}(y) = 1-y\) with \(|\tfrac{d}{dy}g_2^{-1}(y)| = 1\). Hence

\begin{equation*} \begin{aligned} f_Y(y) &= \tfrac38\bigl(1-\sqrt{1-y}\bigr)^2\cdot\frac{1}{2\sqrt{1-y}}

• \tfrac38(2-y)^2\\ &= \frac{3\bigl(1-\sqrt{1-y}\bigr)^2}{16\sqrt{1-y}} + \frac{3(2-y)^2}{8}, \qquad 0 < y < 1 . \end{aligned} \end{equation*}

For the total mass, the second term gives \(\tfrac38\int_0^1 (2-y)^2dy = \tfrac38\cdot\tfrac73 = \tfrac78\), while with \(u = \sqrt{1-y}\), \(dy = -2u\,du\), the first gives

\begin{equation*} \frac{3}{16}\int_0^1 \frac{(1-u)^2}{u}\,2u\,du = \frac38\int_0^1 (1-u)^2\,du = \frac38\cdot\frac13 = \frac18 , \end{equation*}

and \(\tfrac78 + \tfrac18 = 1\).

\begin{equation*} f_Y(y) = \begin{cases} \dfrac{2}{9\sqrt{y}} & 0 < y < 1,\\[2ex] \dfrac{\sqrt{y}+1}{9\sqrt{y}} & 1 < y < 4 . \end{cases} \end{equation*}

(a) Theorem 2.1.8 fails at hypothesis (iii): the sets on which \(g(x) = x^2\) is monotone are \((-1,0)\), \((0,1)\) and \((1,2)\), whose images \((0,1)\), \((0,1)\) and \((1,4)\) are not all the same \(\mathcal{Y}\). So work from the cdf, as in Example 2.1.7. Here

\begin{equation*} F_X(x) = \int_{-1}^{x}\tfrac29(t+1)\,dt = \frac{(x+1)^2}{9}, \qquad -1 \le x \le 2, \end{equation*}

and \(\mathcal{Y} = [0,4)\), with two cases.

(i) \(0 < y < 1\): both roots \(\pm\sqrt{y}\) lie in \(\mathcal{X}\), so

\begin{equation*} \begin{aligned} F_Y(y) &= P\bigl(-\sqrt{y} \le X \le \sqrt{y}\bigr)\\ &= \frac{(\sqrt{y}+1)^2 - (1-\sqrt{y})^2}{9} = \frac{4\sqrt{y}}{9} . \end{aligned} \end{equation*}

(ii) \(1 \le y < 4\): only \(+\sqrt{y}\) lies in \(\mathcal{X}\), so

\begin{equation*} F_Y(y) = P\bigl(X \le \sqrt{y}\bigr) = \frac{(\sqrt{y}+1)^2}{9} . \end{equation*}

Differentiating each piece gives the displayed \(f_Y\).

(b) Extension. Only containment is needed, not equality: let \(A_0, \ldots, A_k\) be disjoint sets with \(P(X \notin \bigcup_{i \ge 0} A_i) = 0\) and \(P(X \in A_0) = 0\), let \(g_i = g\) be monotone on \(A_i \cap \mathcal{X}\) with \(g_i^{-1}\) continuously differentiable on \(\mathcal{Y}_i = g_i(A_i \cap \mathcal{X})\), and drop hypothesis (iii), asking only \(\bigcup_i \mathcal{Y}_i = \mathcal{Y}\). For \(B \subset \mathcal{Y}\), disjointness and the two null sets give

\begin{equation*} \begin{aligned} P(Y \in B) &= \sum_{i=1}^{k} P\bigl(X \in A_i \cap \mathcal{X},\ g_i(X) \in B\bigr)\\ &= \sum_{i=1}^{k} \int_{A_i \cap \mathcal{X}\, \cap\, g_i^{-1}(B)} f_X(x)\,dx\\ &= \sum_{i=1}^{k} \int_{B \cap \mathcal{Y}_i} f_X\bigl(g_i^{-1}(y)\bigr)\left|\frac{d}{dy}g_i^{-1}(y)\right| dy , \end{aligned} \end{equation*}

the last step by the change of variable \(x = g_i^{-1}(y)\) on each piece, legitimate because \(g_i\) is monotone (hence one-to-one) there with continuously differentiable inverse. As \(B\) is arbitrary,

\begin{equation*} f_Y(y) = \sum_{i\,:\, y \in \mathcal{Y}_i} f_X\bigl(g_i^{-1}(y)\bigr)\left|\frac{d}{dy}g_i^{-1}(y)\right| , \end{equation*}

the formula of Theorem 2.1.8 read with the convention \(f_X(g_i^{-1}(y)) = 0\) whenever \(g_i^{-1}(y) \notin \mathcal{X}\).

Now take \(A_0 = \emptyset\), \(A_1 = (-1,1)\), \(A_2 = (1,2)\): the omitted points \(-1, 1, 2\) carry no probability, so \(A_0\) may indeed be empty, and \(\mathcal{Y}_1 = [0,1)\), \(\mathcal{Y}_2 = (1,4)\) now need not agree. Since \(x^2\) is two-to-one on \(A_1\), split it at the null point 0 into \(A_{1a} = (-1,0)\) and \(A_{1b} = (0,1)\) (formula (2.1.11) of Example 2.1.7 applied inside \(A_1\)), with

\begin{equation*} g_{1a}^{-1}(y) = -\sqrt{y}, \quad g_{1b}^{-1}(y) = \sqrt{y}, \quad g_2^{-1}(y) = \sqrt{y}, \end{equation*}

and \(|\tfrac{d}{dy}g_i^{-1}(y)| = 1/(2\sqrt{y})\) in all three cases. For \(0 < y < 1\) only the two \(A_1\) terms are present:

\begin{equation*} f_Y(y) = \frac{1}{2\sqrt{y}} \Bigl[\tfrac29(\sqrt{y}+1) + \tfrac29(-\sqrt{y}+1)\Bigr] = \frac{1}{2\sqrt{y}}\cdot\frac49 = \frac{2}{9\sqrt{y}} . \end{equation*}

For \(1 < y < 4\) only the \(A_2\) term is present:

\begin{equation*} f_Y(y) = \frac{1}{2\sqrt{y}}\cdot\tfrac29(\sqrt{y}+1) = \frac{\sqrt{y}+1}{9\sqrt{y}} , \end{equation*}

agreeing with part (a).

Each function satisfies the three conditions of Theorem 1.5.3 (limits \(0\) and \(1\), nondecreasing, right-continuous), and (2.1.13) reads \(F_X^{-1}(y)=\inf\{x:F_X(x)\ge y\}\).

(a) \(F_X\) is \(0\) on \((-\infty,0)\) and rises continuously from \(0\) to \(1\) on \([0,\infty)\) with \(F_X^{\prime}(x)=e^{-x}>0\) there; it is continuous at \(0\) since \(1-e^{0}=0\). So it is a continuous, strictly increasing cdf on its range, and (2.1.13) reduces to (2.1.12): solving \(1-e^{-x}=y\),

\begin{equation*} F_X^{-1}(y)=-\log(1-y),\qquad 0<y<1 . \end{equation*}

(b) On \((-\infty,0)\), \(e^{x}/2\) increases from \(0\) to \(1/2\); the value at \(x=0\) is \(1/2\), so \(F_X\) is continuous there; on \([1,\infty)\), \(1-e^{1-x}/2\) starts at \(1-e^{0}/2=1/2\) and increases to \(1\). Hence \(F_X\) is continuous and nondecreasing with the correct limits, so it is a cdf — but it is flat at height \(1/2\) on \([0,1]\), which is exactly where (2.1.13) is needed. For \(0<y<1/2\), \(e^{x}/2=y\) gives \(x=\log(2y)\); for \(y=1/2\) the set \(\{x:F_X(x)\ge 1/2\}=[0,\infty)\) has infimum \(0=\log(2\cdot\tfrac12)\); for \(1/2<y<1\), \(1-e^{1-x}/2=y\) gives \(x=1-\log(2(1-y))\). Thus

\begin{equation*} F_X^{-1}(y)=\begin{cases} \log(2y) & 0<y\le 1/2\\ 1-\log\bigl(2(1-y)\bigr) & 1/2<y<1 . \end{cases} \end{equation*}

(c) Each piece is increasing, the limits are \(0\) and \(1\), and at \(x=0\) the left limit is \(1/4\) while \(F_X(0)=3/4\): \(F_X\) is right-continuous with a jump of size \(1/2\) at \(0\), so it is a cdf of a variable with an atom \(P(X=0)=1/2\). Now apply (2.1.13) directly. For \(0<y\le 1/4\) the equation \(e^{x}/4=y\) is solvable with \(x=\log(4y)\le 0\). For \(1/4<y\le 3/4\) every \(x<0\) has \(F_X(x)<1/4<y\) while \(F_X(0)=3/4\ge y\), so the infimum is \(0\). For \(3/4<y<1\), \(1-e^{-x}/4=y\) gives \(x=-\log(4(1-y))\). Hence

\begin{equation*} F_X^{-1}(y)=\begin{cases} \log(4y) & 0<y\le 1/4\\ 0 & 1/4<y\le 3/4\\ -\log\bigl(4(1-y)\bigr) & 3/4<y<1 . \end{cases} \end{equation*}

Take \(u(x)=F_X(x)=(x-1)^2/4\) on \(1<x<3\), since

\begin{equation*} F_X(x)=\int_1^x \frac{t-1}{2}\,dt=\frac{(x-1)^2}{4}, \qquad 1<x<3, \end{equation*}

which is continuous and strictly increasing from \(0\) to \(1\); the probability integral transform (Theorem 2.1.10) applies, its hypothesis of a continuous cdf being met, and gives \(Y=F_X(X)\sim\text{uniform}(0,1)\).

(a) is (b) complemented: \(P(Y>y)=1-F_Y(y)\) and \(P(U>y)=1-y\), so \(F_Y(y)\le y\) is \(P(Y>y)\ge P(U>y)\) and \(F_Y(y)<y\) is the strict version. Prove (b).

Let \(\mathcal{X}\) be the countable support of \(X\). Along \(\mathcal{X}\) the cdf is strictly increasing: for \(x<x^{\prime}\) both in \(\mathcal{X}\),

\begin{equation*} F_X(x^{\prime})\ge F_X(x)+P(X=x^{\prime})>F_X(x), \end{equation*}

so \(F_X\) is one-to-one on \(\mathcal{X}\) and \(F_X(x)\le F_X(x^{\prime})\) forces \(x\le x^{\prime}\).

Fix \(0<y<1\) and put \(S_y=\{x\in\mathcal{X}:F_X(x)\le y\}\), so that \(F_Y(y)=P(X\in S_y)\). If \(S_y=\emptyset\) then \(F_Y(y)=0\le y\). Otherwise let \(M=\sup_{x\in S_y}F_X(x)\le y\) and choose \(x_n\in S_y\) with \(F_X(x_n)\uparrow M\). Every \(x\in S_y\) has \(F_X(x)\le F_X(x_n)\) for some \(n\), hence \(x\le x_n\); so \(\{X\in S_y\}=\bigcup_n\{X\le x_n\}\), an increasing union, and

\begin{equation*} F_Y(y)=\lim_{n\to\infty}F_X(x_n)=M\le y . \end{equation*}

For the strict inequality take any \(x_0\in\mathcal{X}\) — a jump point of \(F_X\) — and set \(y_0=F_X(x_0)\) and \(j=P(X=x_0)>0\). By the strict increase, \(F_X(x)\le y_0\) holds for exactly the support points \(x\le x_0\), so

\begin{equation*} P(Y\le y_0)=P(X\le x_0)=F_X(x_0)=y_0 , \end{equation*}

the hint’s equality, while \(P(Y=y_0)=P(X=x_0)=j\) by injectivity on \(\mathcal{X}\). Hence \(F_Y(y_0^-)=y_0-j\). Take \(y=y_0-j/2\), which lies in \((0,1)\) since \(y_0\ge j\) and \(j>0\); then

\begin{equation*} F_Y(y)\le F_Y(y_0^-)=y_0-j<y_0-\tfrac{j}{2}=y . \end{equation*}

(The hint’s \(y=y_0+\epsilon\) needs a next support point above \(x_0\), which a discrete \(X\) need not have; taking \(y\) just below \(y_0\) avoids that.)

(a) \(\mathrm{E}X^2=1\). Directly, integrate by parts with \(u=x\) and \(dv=xe^{-x^2/2}\,dx\):

\begin{equation*} \begin{aligned} \mathrm{E}X^2 &=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}x\cdot xe^{-x^2/2}\,dx\\ &=\frac{1}{\sqrt{2\pi}}\Bigl[-xe^{-x^2/2}\Bigr]_{-\infty}^{\infty} +\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}e^{-x^2/2}\,dx\\ &=0+1=1 . \end{aligned} \end{equation*}

Via the square transformation (2.1.11) of Example 2.1.7, worked out for the normal in Example 2.1.9,

\begin{equation*} f_Y(y)=\frac{1}{2\sqrt{y}}\bigl(f_X(\sqrt{y})+f_X(-\sqrt{y})\bigr) =\frac{1}{\sqrt{2\pi}}\,\frac{1}{\sqrt{y}}\,e^{-y/2},\qquad y>0, \end{equation*}

the chi squared pdf with \(1\) degree of freedom. Substituting \(y=2t\),

\begin{equation*} \begin{aligned} \mathrm{E}Y&=\frac{1}{\sqrt{2\pi}}\int_0^\infty y^{1/2}e^{-y/2}\,dy =\frac{2^{3/2}}{\sqrt{2\pi}}\int_0^\infty t^{1/2}e^{-t}\,dt\\ &=\frac{2\sqrt{2}}{\sqrt{2\pi}}\,\Gamma(3/2) =\frac{2\sqrt{2}}{\sqrt{2\pi}}\cdot\frac{\sqrt{\pi}}{2}=1 , \end{aligned} \end{equation*}

agreeing with the direct computation.

(b) \(g(x)=|x|\) is monotone on \(A_1=(-\infty,0)\) and \(A_2=(0,\infty)\), both mapped onto \(\mathcal{Y}=(0,\infty)\) with \(|dg_i^{-1}/dy|=1\), and \(P(X\in A_0)=P(X=0)=0\); Theorem 2.1.8 (its hypotheses (i)–(iv) hold, the inverses being \(y\mapsto -y\) and \(y\mapsto y\)) gives the half-normal pdf

\begin{equation*} f_Y(y)=f_X(y)+f_X(-y)=\sqrt{\frac{2}{\pi}}\,e^{-y^2/2},\qquad 0<y<\infty . \end{equation*}

Then, substituting \(u=y^2/2\),

\begin{equation*} \mathrm{E}Y=\sqrt{\frac{2}{\pi}}\int_0^\infty y\,e^{-y^2/2}\,dy =\sqrt{\frac{2}{\pi}}\;\approx 0.7979, \end{equation*}

while \(\mathrm{E}Y^2=\mathrm{E}X^2=1\) because \(|X|^2=X^2\). Hence

\begin{equation*} \operatorname{Var}Y=1-\frac{2}{\pi}\;\approx 0.3634 . \end{equation*}

\(Y=d\tan X\), a half-Cauchy variable with scale \(d\), and \(\mathrm{E}Y=\infty\) (the mean does not exist).

Assume \(d>0\). Since \(\tan\) is strictly increasing on \((0,\pi/2)\) onto \((0,\infty)\), for \(y>0\)

\begin{equation*} F_Y(y)=P(d\tan X\le y)=P\bigl(X\le \arctan(y/d)\bigr) =\frac{\arctan(y/d)}{\pi/2}, \end{equation*}

using the uniform\((0,\pi/2)\) cdf. Differentiating,

\begin{equation*} f_Y(y)=\frac{2}{\pi}\cdot\frac{1/d}{1+(y/d)^2} =\frac{2d}{\pi\,(d^2+y^2)},\qquad 0<y<\infty , \end{equation*}

which integrates to \(1\) (Check!). For the mean, the integrand behaves like \(2d/(\pi y)\) at infinity:

\begin{equation*} \mathrm{E}Y=\int_0^\infty \frac{2d\,y}{\pi(d^2+y^2)}\,dy =\frac{d}{\pi}\,\log\bigl(d^2+y^2\bigr)\Big|_0^{\infty}=\infty . \end{equation*}

With \(q=1-p\) and \(0<p<1\),

\begin{equation*} P(X=x)=p^{x}q+q^{x}p,\qquad x=1,2,\dots, \qquad \mathrm{E}X=\frac{p}{q}+\frac{q}{p}=\frac{p^2+q^2}{pq}. \end{equation*}

The event \(\{X=x\}\) is the disjoint union of \(\{x\text{ heads then a tail}\}\) and \(\{x\text{ tails then a head}\}\), whose probabilities are \(p^{x}q\) and \(q^{x}p\) by independence, giving the pmf above. It sums to \(1\):

\begin{equation*} \sum_{x=1}^{\infty}\bigl(p^{x}q+q^{x}p\bigr) =q\cdot\frac{p}{1-p}+p\cdot\frac{q}{1-q} =p+q=1 . \end{equation*}

For the mean, use \(\sum_{x\ge1}x r^{x}=r/(1-r)^2\) for \(|r|<1\) twice:

\begin{equation*} \begin{aligned} \mathrm{E}X&=q\sum_{x=1}^{\infty}x\,p^{x}+p\sum_{x=1}^{\infty}x\,q^{x}\\ &=q\cdot\frac{p}{q^{2}}+p\cdot\frac{q}{p^{2}} =\frac{p}{q}+\frac{q}{p}. \end{aligned} \end{equation*}

Both identities are the same interchange of order of integration on the region \(\{(x,t):0\le x<t\}\), legitimate because the integrand is nonnegative (Fubini–Tonelli).

(a) Since \(1-F_X(x)=P(X>x)=\int_x^\infty f(t)\,dt\) for \(x\ge0\),

\begin{equation*} \begin{aligned} \int_0^\infty\bigl[1-F_X(x)\bigr]dx &=\int_0^\infty\!\!\int_x^\infty f(t)\,dt\,dx\\ &=\int_0^\infty\!\!\int_0^t dx\,f(t)\,dt\\ &=\int_0^\infty t\,f(t)\,dt=\mathrm{E}X , \end{aligned} \end{equation*}

the last equality because \(f\) vanishes on \((-\infty,0)\).

(b) Here \(1-F_X(k)=P(X>k)=\sum_{j=k+1}^{\infty}P(X=j)\), so summing over \(k\ge0\) and counting each \(j\) once for every \(k\in\{0,1,\dots,j-1\}\),

\begin{equation*} \begin{aligned} \sum_{k=0}^{\infty}\bigl(1-F_X(k)\bigr) &=\sum_{k=0}^{\infty}\sum_{j=k+1}^{\infty}P(X=j)\\ &=\sum_{j=1}^{\infty}\sum_{k=0}^{j-1}P(X=j)\\ &=\sum_{j=1}^{\infty}j\,P(X=j)=\mathrm{E}X . \end{aligned} \end{equation*}

Part (a) is the integral version of exactly this count: the area under the horizontal slices of the survival function \(P(X>x)\) is the expectation, with \(dx\) replacing the unit steps \(k\to k+1\).

The whole exercise is the pointwise identity

\begin{equation*} X + Y = (X \vee Y) + (X \wedge Y), \end{equation*}

which holds at every sample point \(s\) because \(\{X(s), Y(s)\}\) and \(\{(X\wedge Y)(s), (X \vee Y)(s)\}\) are the same pair of numbers: if \(X(s) \le Y(s)\) the right side is \(Y(s) + X(s)\), and if \(X(s) > Y(s)\) it is \(X(s) + Y(s)\).

Both \(X \vee Y\) and \(X \wedge Y\) have expectations whenever \(X\) and \(Y\) do, since

\begin{equation*} |X \vee Y| \le |X| + |Y| \qquad\text{and}\qquad |X \wedge Y| \le |X| + |Y| . \end{equation*}

So apply \(\mathrm{E}\) and use linearity – Theorem 2.2.5a, which Section 4.1 notes persists with \(X\) replaced by \((X,Y)\), here with \(g_1(x,y) = \max(x,y)\), \(g_2(x,y) = \min(x,y)\):

\begin{equation*} \begin{aligned} \mathrm{E}\,X + \mathrm{E}\,Y &= \mathrm{E}\bigl[(X \vee Y) + (X \wedge Y)\bigr] \\ &= \mathrm{E}(X \vee Y) + \mathrm{E}(X \wedge Y), \end{aligned} \end{equation*}

and move \(\mathrm{E}(X \wedge Y)\) to the other side.

\begin{equation*} \mathrm{E}\,T = \frac{a}{\lambda} + \frac{1-a}{\mu}. \end{equation*}

\(T\) is continuous and nonnegative, so Exercise 2.14(a) applies, and \(1 - F_T(t) = P(T > t)\):

\begin{equation*} \begin{aligned} \mathrm{E}\,T &= \int_0^\infty \bigl[1 - F_T(t)\bigr]\,dt \\ &= \int_0^\infty \Bigl( a e^{-\lambda t} + (1-a) e^{-\mu t} \Bigr)\,dt \\ &= a\,\frac{1}{\lambda} + (1-a)\,\frac{1}{\mu}, \end{aligned} \end{equation*}

the mixture of the two exponential means of Example 2.2.2.

(a) \(m = 2^{-1/3} \approx 0.7937\). The cdf is \(F(x) = \int_0^x 3t^2\,dt = x^3\) on \((0,1)\), so

\begin{equation*} \tfrac{1}{2} = F(m) = m^3 \qquad\Longrightarrow\qquad m = \left(\tfrac{1}{2}\right)^{1/3} . \end{equation*}

(b) \(m = 0\). The Cauchy density of Example 2.2.4 is symmetric about \(0\), \(f(-x) = f(x)\), so

\begin{equation*} \int_{-\infty}^{0} f(x)\,dx = \int_{0}^{\infty} f(x)\,dx = \tfrac{1}{2} . \end{equation*}

Explicitly, \(F(x) = \tfrac{1}{2} + \tfrac{1}{\pi}\arctan x\), and \(F(0) = \tfrac{1}{2}\).

For \(a > m\) the integrand difference \(|x-a| - |x-m|\) is at least \(a-m\) on \(\{x \le m\}\) and at least \(-(a-m)\) off it, and the median splits the mass evenly, so the two halves cancel. (Assume \(\mathrm{E}|X| < \infty\); otherwise \(\mathrm{E}|X-a| = \infty\) for every \(a\).) Leaf by leaf:

(i) \(x \le m\): \(\;|x-a| - |x-m| = (a-x) - (m-x) = a-m\).

(ii) \(m < x < a\): \(\;|x-a| - |x-m| = (a-x) - (x-m) = a+m-2x > -(a-m)\).

(iii) \(x \ge a\): \(\;|x-a| - |x-m| = (x-a) - (x-m) = -(a-m)\).

Comparing with the step function \((a-m)\bigl[I(x \le m) - I(x > m)\bigr]\), Theorem 2.2.5c gives

\begin{equation*} \begin{aligned} \mathrm{E}|X-a| - \mathrm{E}|X-m| &\ge (a-m)P(X \le m) - (a-m)P(X > m) \\ &= (a-m)\bigl[\tfrac{1}{2} - \tfrac{1}{2}\bigr] \;=\; 0, \end{aligned} \end{equation*}

using \(P(X \le m) = P(X > m) = \tfrac{1}{2}\), the median’s defining property for continuous \(X\) (Exercise 2.17). The case \(a < m\) is the mirror image, with \(\{x \ge m\}\) and \(\{x < m\}\) and the constant \(m-a\) (Check!). So \(\mathrm{E}|X-a| \ge \mathrm{E}|X-m|\) for every \(a\), with equality at \(a = m\).

Differentiating under the integral sign,

\begin{equation*} \begin{aligned} \frac{d}{da}\,\mathrm{E}(X-a)^2 &= \frac{d}{da}\int_{-\infty}^{\infty} (x-a)^2 f_X(x)\,dx \\ &= \int_{-\infty}^{\infty} \frac{\partial}{\partial a}(x-a)^2 f_X(x)\,dx \\ &= -2\int_{-\infty}^{\infty} (x-a) f_X(x)\,dx \\ &= -2\,(\mathrm{E}\,X - a), \end{aligned} \end{equation*}

which vanishes if and only if \(a = \mathrm{E}\,X\). The same interchange applied once more gives

\begin{equation*} \frac{d^2}{da^2}\,\mathrm{E}(X-a)^2 = 2\int_{-\infty}^{\infty} f_X(x)\,dx = 2 > 0 \end{equation*}

for every \(a\), so \(\mathrm{E}(X-a)^2\) is strictly convex in \(a\) and its unique stationary point \(a = \mathrm{E}\,X\) is the global minimum – the calculus proof of (2.2.3) promised in Example 2.2.6.

Assumptions.

(i) \(F_X\) is absolutely continuous with \(F_X^{\prime}= f_X\), so that \(\mathrm{E}\,g(X) = \int g(x) f_X(x)\,dx\) for the \(g\) used above.

(ii) \(\mathrm{E}\,X^2 < \infty\), so that \(\mathrm{E}(X-a)^2\) is finite for all \(a\) (and then \(\mathrm{E}|X| < \infty\) too).

(iii) The hypotheses of Corollary 2.4.4 for the interchange in the second line: for \(a\) ranging in a neighborhood \((a_0-\delta, a_0+\delta)\),

\begin{equation*} \left| \frac{\partial}{\partial a}(x-a)^2 f_X(x) \right| \le 2\bigl(|x| + |a_0| + \delta\bigr) f_X(x), \end{equation*}

and the dominating function on the right is integrable by (ii); likewise \(2 f_X\) dominates itself for the second derivative. No continuity or boundedness of \(f_X\) is required.

Two children, on the assumption that births are independent with \(P(\text{daughter}) = p = \tfrac{1}{2}\).

Let \(X\) be the number of children. “Keep going until the first success” is exactly the setup of Example 1.5.4 with a daughter playing the role of a head, so \(X\) has the geometric pmf (1.5.2) and cdf

\begin{equation*} P(X = x) = (1-p)^{x-1} p, \qquad F_X(x) = 1 - (1-p)^{x}, \quad x = 1, 2, \ldots, \end{equation*}

with \(F_X(0) = 0\). Since \(X\) takes values in the nonnegative integers, Exercise 2.14(b) turns the mean into a geometric series with no algebra at all:

\begin{equation*} \begin{aligned} \mathrm{E}\,X &= \sum_{k=0}^{\infty} \bigl[1 - F_X(k)\bigr] \\ &= \sum_{k=0}^{\infty} (1-p)^{k} \\ &= \frac{1}{p}, \end{aligned} \end{equation*}

the \(k=0\) term being \(1 - F_X(0) = 1 = (1-p)^0\). With \(p = \tfrac{1}{2}\), \(\mathrm{E}\,X = 2\).

Change of variable \(y = g(x)\) in \(\mathrm{E}\,Y = \int_{\mathcal{Y}} y f_Y(y)\,dy\), with \(f_Y\) supplied by Theorem 2.1.5.

Continuous case. Let \(\mathcal{X} = (c,d)\) be the support of \(X\) (possibly infinite) and \(\mathcal{Y} = g(\mathcal{X})\). Monotonicity makes \(g^{-1}\) single-valued; assume as Theorem 2.1.5 requires that \(f_X\) is continuous on \(\mathcal{X}\) and \(g^{-1}\) continuously differentiable on \(\mathcal{Y}\). Then

\begin{equation*} f_Y(y) = f_X\bigl(g^{-1}(y)\bigr)\left| \frac{d}{dy} g^{-1}(y) \right|, \qquad y \in \mathcal{Y}. \end{equation*}

If \(g\) is increasing, \(\mathcal{Y} = (g( c), g(d))\) and \(\frac{d}{dy}g^{-1}(y) > 0\); substituting \(x = g^{-1}(y)\), so that \(dx = \frac{d}{dy}g^{-1}(y)\,dy\) and the limits map \(g( c) \mapsto c\), \(g(d) \mapsto d\),

\begin{equation*} \begin{aligned} \mathrm{E}\,Y &= \int_{g( c)}^{g(d)} y\, f_X\bigl(g^{-1}(y)\bigr)\,\frac{d}{dy}g^{-1}(y)\,dy \\ &= \int_{c}^{d} g(x)\, f_X(x)\,dx \;=\; \mathrm{E}\,g(X). \end{aligned} \end{equation*}

If \(g\) is decreasing, \(\mathcal{Y} = (g(d), g( c))\) and the absolute value contributes \(-\frac{d}{dy}g^{-1}(y)\), while the substitution now sends \(g(d) \mapsto d\), \(g( c) \mapsto c\); the two sign reversals cancel:

\begin{equation*} \begin{aligned} \mathrm{E}\,Y &= \int_{g(d)}^{g( c)} y\, f_X\bigl(g^{-1}(y)\bigr) \left(-\frac{d}{dy}g^{-1}(y)\right) dy \\ &= -\int_{d}^{c} g(x)\, f_X(x)\,dx \\ &= \int_{c}^{d} g(x)\, f_X(x)\,dx \;=\; \mathrm{E}\,g(X). \end{aligned} \end{equation*}

Discrete case. Monotone means one-to-one, so the Section 2.1 formula \(f_Y(y) = \sum_{x \in g^{-1}(y)} f_X(x)\) collapses to a single term,

\begin{equation*} f_Y(y) = P\bigl(X = g^{-1}(y)\bigr) = f_X\bigl(g^{-1}(y)\bigr), \qquad y \in \mathcal{Y}, \end{equation*}

and \(x \mapsto g(x)\) is a bijection of \(\mathcal{X}\) onto \(\mathcal{Y}\). Reindexing the sum by \(y = g(x)\),

\begin{equation*} \mathrm{E}\,Y = \sum_{y \in \mathcal{Y}} y\, f_Y(y) = \sum_{x \in \mathcal{X}} g(x)\, f_X(x) = \mathrm{E}\,g(X), \end{equation*}

the reindexing being unconditional since \(\mathrm{E}|g(X)| < \infty\) is exactly Definition 2.2.1’s requirement for either side to exist.

Everything follows from the single gamma evaluation

\begin{equation*} \int_{0}^{\infty} u^{n}e^{-u^{2}}\,du = \tfrac12\,\Gamma\!\left(\tfrac{n+1}{2}\right), \qquad n > -1, \end{equation*}

(substitute \(t = u^{2}\)), which gives \(\sqrt{\pi}/4\), \(1/2\) and \(3\sqrt{\pi}/8\) at \(n = 2,3,4\).

(a) \(f \ge 0\) on \((0,\infty)\), and with \(x = \beta u\),

\begin{equation*} \begin{aligned} \int_{0}^{\infty} f(x)\,dx &= \frac{4}{\beta^{3}\sqrt{\pi}}\int_{0}^{\infty}\beta^{3}u^{2}e^{-u^{2}}\,du\\ &= \frac{4}{\sqrt{\pi}}\cdot\frac{\sqrt{\pi}}{4} = 1 . \end{aligned} \end{equation*}

(b) The same substitution turns each moment into the displayed integral:

\begin{equation*} \begin{aligned} \operatorname{E}X &= \frac{4}{\beta^{3}\sqrt{\pi}}\,\beta^{4}\int_{0}^{\infty}u^{3}e^{-u^{2}}\,du = \frac{4\beta}{\sqrt{\pi}}\cdot\frac12 = \frac{2\beta}{\sqrt{\pi}},\\ \operatorname{E}X^{2} &= \frac{4}{\beta^{3}\sqrt{\pi}}\,\beta^{5}\int_{0}^{\infty}u^{4}e^{-u^{2}}\,du = \frac{4\beta^{2}}{\sqrt{\pi}}\cdot\frac{3\sqrt{\pi}}{8} = \frac{3\beta^{2}}{2}. \end{aligned} \end{equation*}

Hence, by Definition 2.3.2 and (2.3.1),

\begin{equation*} \operatorname{Var}X = \frac{3\beta^{2}}{2} - \frac{4\beta^{2}}{\pi} = \beta^{2}\,\frac{3\pi - 8}{2\pi}. \end{equation*}

(a) \(f_{Y}(y) = \dfrac{1}{2\sqrt{y}}\), \(0 < y < 1\): the square transformation formula (2.1.11) of Example 2.1.7 gives, for \(0 < y < 1\),

\begin{equation*} \begin{aligned} f_{Y}(y) &= \frac{1}{2\sqrt{y}}\Bigl(f_{X}(\sqrt{y}) + f_{X}(-\sqrt{y})\Bigr)\\ &= \frac{1}{2\sqrt{y}}\Bigl(\tfrac12(1+\sqrt{y}) + \tfrac12(1-\sqrt{y})\Bigr) = \frac{1}{2\sqrt{y}} . \end{aligned} \end{equation*}

(b) Directly from \(f_{Y}\),

\begin{equation*} \begin{aligned} \operatorname{E}Y &= \frac12\int_{0}^{1} y^{1/2}\,dy = \frac12\cdot\frac23 = \frac13,\\ \operatorname{E}Y^{2} &= \frac12\int_{0}^{1} y^{3/2}\,dy = \frac12\cdot\frac25 = \frac15, \end{aligned} \end{equation*}

so \(\operatorname{Var}Y = \dfrac15 - \dfrac19 = \dfrac{4}{45}\).

(a) \(\operatorname{E}X = \dfrac{a}{a+1}\) and \(\operatorname{Var}X = \dfrac{a}{(a+2)(a+1)^{2}}\). Indeed \(\operatorname{E}X^{k} = a\int_{0}^{1}x^{a+k-1}dx = \dfrac{a}{a+k}\), so

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \frac{a}{a+2} - \frac{a^{2}}{(a+1)^{2}} = \frac{a\bigl[(a+1)^{2} - a(a+2)\bigr]}{(a+2)(a+1)^{2}}\\ &= \frac{a}{(a+2)(a+1)^{2}} . \end{aligned} \end{equation*}

(b) \(\operatorname{E}X = \dfrac{n+1}{2}\) and \(\operatorname{Var}X = \dfrac{n^{2}-1}{12}\), from \(\sum_{x=1}^{n}x = \tfrac{n(n+1)}{2}\) and \(\sum_{x=1}^{n}x^{2} = \tfrac{n(n+1)(2n+1)}{6}\):

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \frac{(n+1)(2n+1)}{6} - \frac{(n+1)^{2}}{4}\\ &= \frac{(n+1)\bigl[2(2n+1) - 3(n+1)\bigr]}{12} = \frac{(n+1)(n-1)}{12} . \end{aligned} \end{equation*}

(c) \(\operatorname{E}X = 1\) and \(\operatorname{Var}X = \dfrac35\). The pdf is symmetric about \(1\), so \(\operatorname{E}X = 1\) by Exercise 2.26(c), and

\begin{equation*} \operatorname{Var}X = \operatorname{E}(X-1)^{2} = \frac32\int_{0}^{2}(x-1)^{4}\,dx = \frac32\cdot\frac25 = \frac35 . \end{equation*}

(a) The two cdfs agree: substituting \(u = -t\) and using \(f_{X}(-u) = f_{X}(u)\),

\begin{equation*} \begin{aligned} F_{-X}(x) &= P(-X \le x) = P(X \ge -x) = \int_{-x}^{\infty} f_{X}(t)\,dt\\ &= \int_{-\infty}^{x} f_{X}(-u)\,du = \int_{-\infty}^{x} f_{X}(u)\,du = F_{X}(x). \end{aligned} \end{equation*}

(b) \(M_{X}(t) = M_{X}(-t)\) for every \(t\), the two sides finite together. By (a) the variables \(X\) and \(-X\) have one distribution, so they have the same expectation of \(g(x) = e^{tx}\) (Definition 2.2.1):

\begin{equation*} M_{X}(t) = \operatorname{E}e^{tX} = \operatorname{E}e^{t(-X)} = \operatorname{E}e^{(-t)X} = M_{X}(-t). \end{equation*}

(a) \(\mathrm{n}(a,\sigma^{2})\); \(\mathrm{uniform}(a-1,a+1)\); the double exponential \(f(x) = \tfrac12 e^{-|x-a|}\), \(-\infty < x < \infty\). (Check!)

(b) Split the total mass at \(a\) and reflect the left half onto the right half with \(x = a - \epsilon\), then \(x = a + \epsilon\):

\begin{equation*} \begin{aligned} P(X \le a) &= \int_{-\infty}^{a} f(x)\,dx = \int_{0}^{\infty} f(a-\epsilon)\,d\epsilon\\ &= \int_{0}^{\infty} f(a+\epsilon)\,d\epsilon = \int_{a}^{\infty} f(x)\,dx = P(X \ge a). \end{aligned} \end{equation*}

The two ends sum to \(1\), so each equals \(\tfrac12\) and \(a\) satisfies the defining requirement of Exercise 2.17.

(c) The same reflection kills the first central term. Since \(\operatorname{E}X\) exists, \(\operatorname{E}(X-a)\) exists and splits at \(a\):

\begin{equation*} \begin{aligned} \operatorname{E}(X-a) &= \int_{a}^{\infty}(x-a)f(x)\,dx + \int_{-\infty}^{a}(x-a)f(x)\,dx\\ &= \int_{0}^{\infty}\epsilon f(a+\epsilon)\,d\epsilon

• \int_{0}^{\infty}\epsilon f(a-\epsilon)\,d\epsilon = 0, \end{aligned} \end{equation*}

the two integrals being equal by symmetry. Hence \(\operatorname{E}X = a\).

(d) Here \(f(x) = 0\) for \(x < 0\), so no centre can work. Given any candidate \(a\), take \(\epsilon = |a| + 1 > 0\). Then \(a + \epsilon \ge 1\) and \(a - \epsilon \le -1\), so

\begin{equation*} f(a+\epsilon) = e^{-(a+\epsilon)} > 0 = f(a-\epsilon), \end{equation*}

contradicting symmetry about \(a\).

(e) The median \(m\) solves \(\int_{0}^{m} e^{-x}dx = \tfrac12\), i.e. \(1 - e^{-m} = \tfrac12\), so \(m = \log 2 \approx 0.693\), while \(\operatorname{E}X = \int_{0}^{\infty} x e^{-x}dx = 1\). Thus \(m = \log 2 < 1 = \operatorname{E}X\).

The definition says exactly that \(f\) is nondecreasing on \((-\infty,a]\) and nonincreasing on \([a,\infty)\).

(a) \(\mathrm{n}(0,1)\): \(f(x) = (2\pi)^{-1/2}e^{-x^{2}/2}\) is strictly increasing on \((-\infty,0]\) and strictly decreasing on \([0,\infty)\), so \(0\) is a mode and strictness makes it the only one.

(b) \(\mathrm{uniform}(0,1)\): for every \(a \in (0,1)\), \(f\) is nondecreasing on \((-\infty,a]\) (values \(0\) then \(1\)) and nonincreasing on \([a,\infty)\) (values \(1\) then \(0\)). Every point of \((0,1)\) is a mode.

(c) Let \(b\) be a mode of \(f\), symmetric about \(a\). Symmetry reads \(f(x) = f(2a-x)\), and \(x \mapsto 2a-x\) reverses order while preserving the value of \(f\), so \(b^{\prime} := 2a-b\) is a mode too. Since \(a = \tfrac12(b+b^{\prime})\), relabelling if necessary gives \(b \le a \le b^{\prime}\), and then

\begin{equation*} \begin{aligned} {}[a,\infty) &\subseteq [b,\infty), \quad\text{on which } f \text{ is nonincreasing},\\ (-\infty,a] &\subseteq (-\infty,b^{\prime}], \quad\text{on which } f \text{ is nondecreasing}, \end{aligned} \end{equation*}

which is exactly the assertion that \(a\) is a mode.

(d) The mode is \(0\), and it is unique. Here \(f(x) = 0\) for \(x < 0\) and \(f(x) = e^{-x}\) for \(x \ge 0\). For \(0 \le x \le y\),

\begin{equation*} f(0) = 1 \ge e^{-x} \ge e^{-y}, \end{equation*}

and for \(0 \ge x \ge y\) we have \(f(0) = 1 \ge f(x) \ge f(y)\) since \(f\) is \(0\) at every negative argument. Uniqueness: \(f(x) < 1 = f(0)\) for every \(x \ne 0\), so no other point can dominate \(f\).

(a) \(\mu_{3} = 0\), hence \(\alpha_{3} = 0\). By Exercise 2.26(c) the mean is the point of symmetry \(a\), so \(\mu_{3} = \operatorname{E}(X-a)^{3}\); splitting at \(a\) and reflecting each half as in Exercise 2.26(c),

\begin{equation*} \mu_{3} = \int_{0}^{\infty}\epsilon^{3}f(a+\epsilon)\,d\epsilon

• \int_{0}^{\infty}\epsilon^{3}f(a-\epsilon)\,d\epsilon = 0, \end{equation*}

the two integrals being equal because \(f(a+\epsilon) = f(a-\epsilon)\).

(b) \(\alpha_{3} = 2\). For \(X \sim \mathrm{exponential}(1)\), \(\operatorname{E}X^{k} = \int_{0}^{\infty}x^{k}e^{-x}dx = k!\), so with \(\operatorname{E}X = 1\),

\begin{equation*} \begin{aligned} \mu_{2} &= \operatorname{E}X^{2} - 1 = 1,\\ \mu_{3} &= \operatorname{E}X^{3} - 3\operatorname{E}X^{2}

• 3\operatorname{E}X - 1 = 6 - 6 + 3 - 1 = 2, \end{aligned} \end{equation*}

and \(\alpha_{3} = 2/1^{3/2} = 2\).

(c) Each pdf is symmetric about \(0\), so the central moments are the raw moments.

(i) \(\mathrm{n}(0,1)\): \(\mu_{2} = 1\) and \(\mu_{4} = 3\) (the standard normal moments, Section 3.3), so

\begin{equation*} \alpha_{4} = \frac{3}{1^{2}} = 3 . \end{equation*}

(ii) \(\mathrm{uniform}(-1,1)\): \(\mu_{2} = \int_{-1}^{1}\tfrac12 x^{2}dx = \tfrac13\) and \(\mu_{4} = \int_{-1}^{1}\tfrac12 x^{4}dx = \tfrac15\), so

\begin{equation*} \alpha_{4} = \frac{1/5}{(1/3)^{2}} = \frac95 = 1.8 . \end{equation*}

(iii) Double exponential: by evenness \(\mu_{k} = \int_{0}^{\infty}x^{k}e^{-x}dx = k!\) for even \(k\), so \(\mu_{2} = 2\), \(\mu_{4} = 24\) and

\begin{equation*} \alpha_{4} = \frac{24}{2^{2}} = 6 . \end{equation*}

The ordering \(1.8 < 3 < 6\) is the comment asked for: the uniform is flattest, the normal intermediate, and the double exponential the most peaked, having a spike at \(0\) and heavy exponential tails.

(a) \(\mathrm{E}[X(X-1)] = n(n-1)p^2\) for the binomial\((n,p)\) and \(\lambda^2\) for the Poisson\((\lambda)\). In both cases the factor \(x(x-1)\) cancels the first two factors of \(x!\) and leaves the same pmf with its index shifted by two. For the binomial,

\begin{equation*} \begin{aligned} \mathrm{E}[X(X-1)] &= \sum_{x=2}^{n} \frac{n!}{(x-2)!\,(n-x)!}\,p^{x}(1-p)^{n-x}\\ &= n(n-1)p^{2}\sum_{x=2}^{n}\binom{n-2}{x-2}p^{x-2}(1-p)^{(n-2)-(x-2)}\\ &= n(n-1)p^{2}, \end{aligned} \end{equation*}

the last sum being the total mass of a binomial\((n-2,p)\) pmf. For the Poisson,

\begin{equation*} \begin{aligned} \mathrm{E}[X(X-1)] &= \sum_{x=2}^{\infty} x(x-1)\,\frac{e^{-\lambda}\lambda^{x}}{x!} = \lambda^{2}\sum_{x=2}^{\infty}\frac{e^{-\lambda}\lambda^{x-2}}{(x-2)!}\\ &= \lambda^{2}. \end{aligned} \end{equation*}

(b) Since \(X^{2} = X(X-1) + X\), Theorem 2.2.5 gives

\begin{equation*} \operatorname{Var}X = \mathrm{E}[X(X-1)] + \mathrm{E}X - (\mathrm{E}X)^{2}. \end{equation*}

With \(\mathrm{E}X = np\) for the binomial,

\begin{equation*} \operatorname{Var}X = n(n-1)p^{2} + np - n^{2}p^{2} = np(1-p), \end{equation*}

and with \(\mathrm{E}X = \lambda\) for the Poisson,

\begin{equation*} \operatorname{Var}X = \lambda^{2} + \lambda - \lambda^{2} = \lambda . \end{equation*}

(c) \(\operatorname{Var}Y = \dfrac{nab(n+a+b)}{(a+b)^{2}(a+b+1)}\).

First put the pmf in beta form: write out the three binomial coefficients and use \(a/a! = 1/(a-1)!\) together with \((y+a)!/(y+a) = (y+a-1)!\), which is where the two stray factors \(a\) and \(1/(y+a)\) go. This gives

\begin{equation*} \begin{aligned} P(Y=y) &= \binom{n}{y}\, \frac{(a+b-1)!\,(y+a-1)!\,(n-y+b-1)!}{(a-1)!\,(b-1)!\,(n+a+b-1)!}\\ &= \binom{n}{y}\,\frac{B(y+a,\;n-y+b)}{B(a,b)}, \end{aligned} \end{equation*}

where \(B(\alpha,\beta) = \Gamma(\alpha)\Gamma(\beta)/\Gamma(\alpha+\beta)\). That these probabilities sum to \(1\) is the identity

\begin{equation*} \sum_{j=0}^{m}\binom{m}{j}B(j+\alpha,\,m-j+\beta) = B(\alpha,\beta), \qquad m \ge 0, \end{equation*}

for positive integers \(\alpha,\beta\); it is the only tool needed below.

For the first factorial moment, drop the \(y=0\) term and set \(j=y-1\), \(m=n-1\):

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \sum_{y=1}^{n}\frac{n!}{(y-1)!\,(n-y)!}\, \frac{B(y+a,\,n-y+b)}{B(a,b)}\\ &= \frac{n}{B(a,b)}\sum_{j=0}^{m}\binom{m}{j}B\bigl(j+(a+1),\,m-j+b\bigr)\\ &= n\,\frac{B(a+1,b)}{B(a,b)} \;=\; \frac{na}{a+b}. \end{aligned} \end{equation*}

For the second, the factor \(y(y-1)\) again shifts the index by two; with \(j=y-2\), \(m=n-2\),

\begin{equation*} \begin{aligned} \mathrm{E}[Y(Y-1)] &= \sum_{y=2}^{n}\frac{n!}{(y-2)!\,(n-y)!}\, \frac{B(y+a,\,n-y+b)}{B(a,b)}\\ &= \frac{n(n-1)}{B(a,b)}\sum_{j=0}^{m}\binom{m}{j} B\bigl(j+(a+2),\,m-j+b\bigr)\\ &= n(n-1)\,\frac{B(a+2,b)}{B(a,b)} \;=\; \frac{n(n-1)a(a+1)}{(a+b)(a+b+1)} . \end{aligned} \end{equation*}

(The two ratios of beta functions collapse by \(\Gamma(\alpha+1)=\alpha\Gamma(\alpha)\).) Now the identity of part (b):

\begin{equation*} \begin{aligned} \operatorname{Var}Y &= \frac{n(n-1)a(a+1)}{(a+b)(a+b+1)} + \frac{na}{a+b}

• \frac{n^{2}a^{2}}{(a+b)^{2}}\\ &= \frac{nab(n+a+b)}{(a+b)^{2}(a+b+1)} . \end{aligned} \end{equation*}

(Check! Put everything over \((a+b)^{2}(a+b+1)\); the numerator is \(na\bigl[(n-1)(a+1)(a+b) + (a+b)(a+b+1) - na(a+b+1)\bigr]\), whose bracket simplifies to \(b(n+a+b)\).)

(a) \(M_X(t) = \dfrac{e^{ct}-1}{ct}\) for \(t \neq 0\), and \(M_X(0)=1\):

\begin{equation*} M_X(t) = \frac{1}{c}\int_{0}^{c} e^{tx}\,dx = \frac{1}{c}\cdot\frac{e^{ct}-1}{t}. \end{equation*}

It is finite for every \(t\), and \((e^{ct}-1)/(ct) \to 1\) as \(t \to 0\), so \(M_X\) is continuous at \(0\) with the required value \(M_X(0)=1\).

(b) \(M_X(t) = \dfrac{2\bigl(cte^{ct} - e^{ct} + 1\bigr)}{c^{2}t^{2}}\) for \(t \neq 0\), and \(M_X(0)=1\). Integrating by parts,

\begin{equation*} \begin{aligned} M_X(t) &= \frac{2}{c^{2}}\int_{0}^{c} x e^{tx}\,dx = \frac{2}{c^{2}}\left[\frac{xe^{tx}}{t}\right]_{0}^{c}

• \frac{2}{c^{2}t}\int_{0}^{c} e^{tx}\,dx\\ &= \frac{2}{c^{2}}\left(\frac{ce^{ct}}{t} - \frac{e^{ct}-1}{t^{2}}\right). \end{aligned} \end{equation*}

(c) \(M_X(t) = \dfrac{e^{\alpha t}}{1-\beta^{2}t^{2}}\), \(\;|t| < 1/\beta\). Substituting \(y = x-\alpha\) and splitting the absolute value at \(0\),

\begin{equation*} \begin{aligned} M_X(t) &= \frac{e^{\alpha t}}{2\beta}\int_{-\infty}^{\infty} e^{ty}e^{-|y|/\beta}\,dy\\ &= \frac{e^{\alpha t}}{2\beta}\left( \int_{0}^{\infty}e^{-(1/\beta - t)y}\,dy

• \int_{0}^{\infty}e^{-(1/\beta + t)y}\,dy\right)\\ &= \frac{e^{\alpha t}}{2\beta}\left( \frac{1}{1/\beta - t} + \frac{1}{1/\beta + t}\right) = \frac{e^{\alpha t}}{1 - \beta^{2}t^{2}} . \end{aligned} \end{equation*}

Both integrals converge exactly when \(|t| < 1/\beta\), which is the stated neighborhood of \(0\).

(d) \(M_X(t) = \left(\dfrac{p}{1-(1-p)e^{t}}\right)^{r}\), \(\;t < -\log(1-p)\). Write \(q=1-p\) and use the negative binomial series \(\sum_{x=0}^{\infty}\binom{r+x-1}{x}u^{x} = (1-u)^{-r}\), valid for \(|u|<1\):

\begin{equation*} \begin{aligned} M_X(t) &= \sum_{x=0}^{\infty}\binom{r+x-1}{x}p^{r}\bigl(qe^{t}\bigr)^{x}\\ &= \frac{p^{r}}{(1-qe^{t})^{r}} . \end{aligned} \end{equation*}

The series converges precisely when \(qe^{t} < 1\), that is \(t < \log(1/q)\), an interval containing \(0\).

No. Every moment generating function satisfies

\begin{equation*} M_X(0) = \mathrm{E}\,e^{0\cdot X} = \mathrm{E}\,1 = 1, \end{equation*}

whereas the proposed function takes the value \(0/(1-0) = 0\) at \(t=0\). So \(t/(1-t)\) is not the mgf of any random variable.

Method (2): the mgf generates the moments by \(M_X^{(n)}(0) = \mathrm{E}X^{n}\) (Definition 2.3.6 and Theorem 2.3.7 in the form stated there). Expanding in a geometric series for \(|t|<1\),

\begin{equation*} \frac{t}{1-t} = \sum_{n=1}^{\infty} t^{n} = \sum_{n=0}^{\infty}\frac{\mathrm{E}X^{n}}{n!}t^{n} \;\Longrightarrow\; \mathrm{E}X^{n} = n!,\ n\ge 1, \end{equation*}

but the constant term forces \(\mathrm{E}X^{0} = 0\), which is impossible since \(X^{0} \equiv 1\).

Differentiate the logarithm and evaluate at \(0\), where \(M_X(0)=1\) and \(M_X^{(n)}(0)=\mathrm{E}X^{n}\) by Theorem 2.3.7 (legitimate because the mgf is assumed to exist in a neighborhood of \(0\), where it is infinitely differentiable). First,

\begin{equation*} \left.\frac{d}{dt}S(t)\right|_{t=0} = \left.\frac{M_X^{\prime}(t)}{M_X(t)}\right|_{t=0} = \frac{\mathrm{E}X}{1} = \mathrm{E}X . \end{equation*}

Second, by the quotient rule,

\begin{equation*} \begin{aligned} \frac{d^{2}}{dt^{2}}S(t) &= \frac{M_X^{\prime\prime}(t)M_X(t) - \bigl(M_X^{\prime}(t)\bigr)^{2}}{\bigl(M_X(t)\bigr)^{2}},\\[2pt] \left.\frac{d^{2}}{dt^{2}}S(t)\right|_{t=0} &= \frac{\mathrm{E}X^{2}\cdot 1 - (\mathrm{E}X)^{2}}{1^{2}} = \operatorname{Var}X, \end{aligned} \end{equation*}

the last equality being Definition 2.3.2 expanded by the linearity of Theorem 2.2.5(a).

Throughout, \(\mathrm{E}X = S^{\prime}(0)\) and \(\operatorname{Var}X = S^{\prime\prime}(0)\) for \(S(t) = \log M_X(t)\), by Exercise 2.32.

(a) Summing the Poisson series,

\begin{equation*} M_X(t) = \sum_{x=0}^{\infty}e^{tx}\frac{e^{-\lambda}\lambda^{x}}{x!} = e^{-\lambda}\sum_{x=0}^{\infty}\frac{(\lambda e^{t})^{x}}{x!} = e^{-\lambda}e^{\lambda e^{t}} = e^{\lambda(e^{t}-1)}, \end{equation*}

finite for every real \(t\). Here \(S(t) = \lambda(e^{t}-1)\), so \(S^{\prime}(t) = S^{\prime\prime}(t) = \lambda e^{t}\) and

\begin{equation*} \mathrm{E}X = \lambda, \qquad \operatorname{Var}X = \lambda . \end{equation*}

(b) With \(q = 1-p\), the geometric series gives

\begin{equation*} M_X(t) = p\sum_{x=0}^{\infty}\bigl(qe^{t}\bigr)^{x} = \frac{p}{1-qe^{t}}, \end{equation*}

convergent for \(t < \log(1/q)\), a neighborhood of \(0\) since \(q<1\). Here \(S(t) = \log p - \log(1-qe^{t})\), so

\begin{equation*} \begin{aligned} S^{\prime}(t) &= \frac{qe^{t}}{1-qe^{t}},\\ S^{\prime\prime}(t) &= \frac{qe^{t}}{1-qe^{t}} + \frac{q^{2}e^{2t}}{(1-qe^{t})^{2}}, \end{aligned} \end{equation*}

and evaluating at \(t=0\), where \(1-q=p\),

\begin{equation*} \mathrm{E}X = \frac{q}{p} = \frac{1-p}{p}, \qquad \operatorname{Var}X = \frac{q}{p} + \frac{q^{2}}{p^{2}} = \frac{q(p+q)}{p^{2}} = \frac{1-p}{p^{2}} . \end{equation*}

(c) Complete the square in the exponent: with \(A = x-\mu\),

\begin{equation*} tx - \frac{(x-\mu)^{2}}{2\sigma^{2}} = \mu t + \frac{\sigma^{2}t^{2}}{2}

• \frac{\bigl(A-\sigma^{2}t\bigr)^{2}}{2\sigma^{2}} . \end{equation*}

Hence

\begin{equation*} \begin{aligned} M_X(t) &= e^{\mu t + \sigma^{2}t^{2}/2} \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}\,\sigma} e^{-\left(x-(\mu+\sigma^{2}t)\right)^{2}/(2\sigma^{2})}\,dx\\ &= e^{\mu t + \sigma^{2}t^{2}/2}, \end{aligned} \end{equation*}

the remaining integral being that of an \(\mathrm{n}(\mu+\sigma^{2}t,\sigma^{2})\) pdf, hence \(1\), for every \(t\). Now \(S(t) = \mu t + \sigma^{2}t^{2}/2\) is a quadratic, so \(S^{\prime}(t) = \mu + \sigma^{2}t\) and \(S^{\prime\prime}(t) = \sigma^{2}\), giving

\begin{equation*} \mathrm{E}X = \mu, \qquad \operatorname{Var}X = \sigma^{2}. \end{equation*}

Both distributions are symmetric about \(0\), which kills \(r=1,3,5\) on both sides at once; the two even moments then match by arithmetic. Precisely: \(f_X(-x)=f_X(x)\) and \(P(Y=-y)=P(Y=y)\), so for odd \(r\) the integrand \(x^{r}f_X(x)\) and the summand \(y^{r}P(Y=y)\) are odd functions and

\begin{equation*} \mathrm{E}X^{r} = 0 = \mathrm{E}Y^{r}, \qquad r = 1,3,5 \end{equation*}

(the normal integrals converge absolutely, so the cancellation is legitimate).

For the even moments of \(X\), take \(\mu=0\), \(\sigma=1\) in Exercise 2.33(c): \(M_X(t) = e^{t^{2}/2}\). Expanding both this and the defining series of Definition 2.3.6,

\begin{equation*} \sum_{k=0}^{\infty}\frac{t^{2k}}{2^{k}k!} = e^{t^{2}/2} = M_X(t) = \sum_{n=0}^{\infty}\frac{\mathrm{E}X^{n}}{n!}\,t^{n}, \end{equation*}

and matching coefficients of \(t^{2k}\) gives \(\mathrm{E}X^{2k} = (2k)!/(2^{k}k!)\). Hence

\begin{equation*} \mathrm{E}X^{2} = \frac{2!}{2\cdot 1} = 1, \qquad \mathrm{E}X^{4} = \frac{4!}{2^{2}\cdot 2!} = 3 . \end{equation*}

For \(Y\), direct summation over the three atoms:

\begin{equation*} \begin{aligned} \mathrm{E}Y^{2} &= 3\cdot\tfrac{1}{6} + 0\cdot\tfrac{2}{3}

• 3\cdot\tfrac{1}{6} = 1,\\ \mathrm{E}Y^{4} &= 9\cdot\tfrac{1}{6} + 0\cdot\tfrac{2}{3}
• 9\cdot\tfrac{1}{6} = 3 . \end{aligned} \end{equation*}

So \(\mathrm{E}X^{r} = \mathrm{E}Y^{r}\) for \(r=1,2,3,4,5\).

(a) The substitution \(y = \log x\), \(dy = dx/x\), turns the \(r\)th moment into the standard normal mgf evaluated at \(r\):

\begin{equation*} \begin{aligned} \mathrm{E}X_1^{r} &= \int_{0}^{\infty} x^{r}\, \frac{1}{\sqrt{2\pi}\,x}e^{-(\log x)^{2}/2}\,dx\\ &= \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}}e^{ry}e^{-y^{2}/2}\,dy\\ &= M_Z( r) = e^{r^{2}/2}, \end{aligned} \end{equation*}

where \(Z \sim \mathrm{n}(0,1)\) and \(M_Z(t)=e^{t^{2}/2}\) by Exercise 2.33(c) with \(\mu=0,\sigma=1\). Since the normal mgf is finite for every real argument, every moment of \(X_1\) is finite.

(b) The same substitution, followed by the shift \(u = y - r\), makes the integrand odd. With \(y=\log x\),

\begin{equation*} \begin{aligned} I_r &:= \int_{0}^{\infty}x^{r}f_1(x)\sin(2\pi\log x)\,dx\\ &= \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{ry-y^{2}/2}\sin(2\pi y)\,dy\\ &= \frac{e^{r^{2}/2}}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-(y-r)^{2}/2}\sin(2\pi y)\,dy \end{aligned} \end{equation*}

by the completion of the square \(ry - y^{2}/2 = r^{2}/2 - (y-r)^{2}/2\). Now put \(u = y-r\) and use \(\sin(2\pi u + 2\pi r) = \sin(2\pi u)\), which holds because \(r\) is an integer:

\begin{equation*} I_r = \frac{e^{r^{2}/2}}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-u^{2}/2}\sin(2\pi u)\,du = 0, \end{equation*}

the integrand being an odd function of \(u\) and absolutely integrable (it is dominated by \(e^{-u^{2}/2}\)).

Consequently, for every \(r = 0,1,2,\dots\),

\begin{equation*} \begin{aligned} \mathrm{E}X_2^{r} &= \int_{0}^{\infty}x^{r}f_1(x)\bigl[1+\sin(2\pi\log x)\bigr]dx\\ &= \mathrm{E}X_1^{r} + I_r = \mathrm{E}X_1^{r}. \end{aligned} \end{equation*}

The integral is \(+\infty\) for every \(t > 0\), so \(M_X\) is finite on no neighbourhood of \(0\).

Fix \(t > 0\). Every term of the exponential series is a lower bound for \(e^{tx}\) on \(x > 0\), so for each \(n = 0,1,2,\dots\),

\begin{equation*} \begin{aligned} M_X(t) &= \operatorname{E} e^{tX} \ \ge\ \operatorname{E}\!\left[\frac{(tX)^n}{n!}\right]\\ &= \frac{t^n}{n!}\,\operatorname{E} X^n \ =\ \frac{t^n e^{n^2/2}}{n!}, \end{aligned} \end{equation*}

the last step by Exercise 2.35(a), which gives \(\operatorname{E} X^n = e^{n^2/2}\) (and in particular guarantees every moment is finite, so the bound is not vacuous). Since \(n! \le n^n\),

\begin{equation*} \frac{t^n e^{n^2/2}}{n!} \ \ge\ \exp\!\left\{\tfrac{1}{2}n^2 + n\log t - n\log n\right\} \ \xrightarrow[n\to\infty]{}\ \infty, \end{equation*}

because the \(n^2/2\) term dominates both \(n\log t\) and \(n\log n\). The left side \(M_X(t)\) does not depend on \(n\), so \(M_X(t) = \infty\).

For \(t \le 0\) we have \(e^{tx} \le 1\) on \(x > 0\) and hence \(M_X(t) \le 1 < \infty\). Thus \(\{t : M_X(t) < \infty\} = (-\infty,0]\), which contains no open interval about \(0\), and the moment generating function does not exist.

Method (2): substituting \(y = \log x\) turns the integral into

\begin{equation*} M_X(t) = \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}}\,\exp\!\left\{te^{y} - \tfrac{1}{2}y^2\right\} dy , \end{equation*}

and for \(t>0\) the exponent \(te^{y}-\frac12 y^2 \to +\infty\) as \(y \to \infty\) (since \(e^{y} > y^{3}/6\)), so the integrand exceeds \(1\) for all large \(y\) and the integral diverges.

(c) is the computation the rest of the exercise dramatizes:

\begin{equation*} M_1(t) = e^{t^2/2}, \qquad M_2(t) = e^{t^2/2}\left[1 + \tfrac{1}{2}e^{-2\pi^2}\sin(2\pi t)\right]. \end{equation*}

(a) \(f_2\) is \(f_1\) modulated by the period-\(1\) wave \(1+\frac12\sin(2\pi x)\): the curves meet at every \(x \in \frac12\mathbb{Z}\), with \(f_2 = \frac32 f_1\) at \(x = k+\frac14\) and \(f_2 = \frac12 f_1\) at \(x = k-\frac14\). Plotted together on \([-3,3]\), \(f_1\) is the single-humped bell while \(f_2\) ripples, rising to \(0.580\) at \(x=\frac14\) and falling to \(0.193\) at \(x=-\frac14\), against a bell whose own maximum is \(0.399\):

(b) Plotted on the same axes, \(K_1\) and \(K_2\) are one parabola: the vertical gap is

\begin{equation*} K_2(t)-K_1(t) = \log\!\left[1+\tfrac{1}{2}e^{-2\pi^2}\sin(2\pi t)\right], \end{equation*}

whose modulus is at most \(\log(1+\frac12 e^{-2\pi^2}) = 1.3376\times10^{-9}\), uniformly in \(t\).

(c) \(M_1(t)=e^{t^2/2}\) is Exercise 2.33(c) with \(\mu = 0\), \(\sigma = 1\). For \(M_2\), complete the square, \(e^{tx}f_1(x) = e^{t^2/2}f_1(x-t)\), and substitute \(u = x-t\):

\begin{equation*} \begin{aligned} \int_{-\infty}^{\infty} e^{tx}f_1(x)\sin(2\pi x)\,dx &= e^{t^2/2}\!\int_{-\infty}^{\infty}\! f_1(u)\sin\!\big(2\pi(u+t)\big)\,du\\ &= e^{t^2/2}\Big[\sin(2\pi t)\operatorname{E}\cos(2\pi Z)\\ &\qquad\quad + \cos(2\pi t)\operatorname{E}\sin(2\pi Z)\Big]\\ &= e^{t^2/2}e^{-2\pi^2}\sin(2\pi t), \end{aligned} \end{equation*}

where \(Z \sim n(0,1)\): the second expectation vanishes because \(f_1\) is even and \(\sin\) odd, and \(\operatorname{E}\cos(sZ) = e^{-s^2/2}\) is the (real) characteristic function of \(n(0,1)\), evaluated at \(s = 2\pi\). Hence

\begin{equation*} M_2(t) = e^{t^2/2} + \tfrac{1}{2}e^{t^2/2}e^{-2\pi^2}\sin(2\pi t), \end{equation*}

which is the stated formula, and \(K_2 = \log M_2\) is the cgf quoted above. Both mgfs are finite for every \(t\).

They are mathematically different, as Theorem 2.3.11(b) demands: the mgfs being finite on a neighbourhood of \(0\), equal mgfs would force \(f_1 = f_2\). Already the first moments separate them,

\begin{equation*} \operatorname{E}_2 X = M_2^{\prime}(0) = \pi e^{-2\pi^2} \ne 0 = \operatorname{E}_1 X . \end{equation*}

But every difference carries the factor \(\frac12 e^{-2\pi^2} = 1.34\times10^{-9}\), so numerically the two mgfs are the same function: McCullagh’s answer is yes for mathematical purposes but a resounding no for numerical purposes.

(d) They are the pair of Example 2.3.10 pulled back through the logarithm. There \(g_1(y) = (\sqrt{2\pi}\,y)^{-1}e^{-(\log y)^2/2}\) is the lognormal and \(g_2(y) = g_1(y)[1+\sin(2\pi\log y)]\). If \(X\) has pdf \(f_1\) then \(Y = e^X\) has pdf \(g_1\); if \(X\) has pdf \(f_2\) then, by Theorem 2.1.5 with \(y = e^x\),

\begin{equation*} f_Y(y) = f_2(\log y)\frac{1}{y} = g_1(y)\left[1+\tfrac{1}{2}\sin(2\pi\log y)\right], \end{equation*}

which is \(g_2\) with the oscillation amplitude halved. So \(f_1, f_2\) are the Example 2.3.10 pair transported to the log scale (up to that factor \(\frac12\)), and the transport is what restores the mgf: \(g_1\) and \(g_2\) have none at all (Exercise 2.36) and identical moments, whereas \(M_1, M_2\) exist everywhere and do differ.

(a)

\begin{equation*} M_X(t) = \left(\frac{p}{1-(1-p)e^{t}}\right)^{r}, \qquad t < -\log(1-p). \end{equation*}

Indeed, with \(u = (1-p)e^{t}\) the negative binomial series \(\sum_{x\ge0}\binom{r+x-1}{x}u^{x} = (1-u)^{-r}\) gives

\begin{equation*} \begin{aligned} M_X(t) &= \sum_{x=0}^{\infty}e^{tx}\binom{r+x-1}{x}p^{r}(1-p)^{x}\\ &= p^{r}\sum_{x=0}^{\infty}\binom{r+x-1}{x}\big[(1-p)e^{t}\big]^{x} = \frac{p^{r}}{\big(1-(1-p)e^{t}\big)^{r}}, \end{aligned} \end{equation*}

the series converging exactly when \(u<1\), i.e. \(t < -\log(1-p)\).

(b) Fix \(|t| < \frac12\). Since \(-\log(1-p) > p > 2pt\) for all \(0<p<1\), the point \(2pt\) lies in the region of convergence of \(M_X\), so

\begin{equation*} M_Y(t) = \operatorname{E} e^{t(2pX)} = M_X(2pt) = \left(\frac{p}{1-(1-p)e^{2pt}}\right)^{r} = \left(\frac{1}{h(p)}\right)^{r}, \end{equation*}

where \(h(p) = \big[1-(1-p)e^{2pt}\big]/p\). Both numerator and denominator of \(h\) vanish at \(p=0\), and by l’Hopital’s rule in \(p\),

\begin{equation*} \begin{aligned} \lim_{p\to 0} h(p) &= \lim_{p\to0}\frac{\partial}{\partial p}\Big[1-(1-p)e^{2pt}\Big]\\ &= \lim_{p\to0}\Big[e^{2pt} - 2t(1-p)e^{2pt}\Big]\\ &= \lim_{p\to0}e^{2pt}\big(1-2t+2pt\big) \;=\; 1-2t . \end{aligned} \end{equation*}

Because \(1-2t>0\), the map \(v\mapsto v^{-r}\) is continuous there and

\begin{equation*} \lim_{p\to 0} M_Y(t) = \left(\frac{1}{1-2t}\right)^{r}, \qquad |t|<\tfrac12 , \end{equation*}

which is the mgf of a chi squared random variable with \(2r\) degrees of freedom (the \(\chi^2_{\nu}\) mgf is \((1-2t)^{-\nu/2}\) for \(t<\frac12\); here \(\nu = 2r\)).

(a) \(e^{-\lambda x}\). Leibnitz’s Rule (Theorem 2.4.1) with \(a(x)=0\), \(b(x)=x\) and an integrand free of the differentiation variable leaves only the boundary term,

\begin{equation*} \frac{d}{dx}\int_0^{x} e^{-\lambda t}\,dt = e^{-\lambda b(x)}\,b^{\prime}(x) = e^{-\lambda x}; \end{equation*}

no interchange is involved, and \(t \mapsto e^{-\lambda t}\) is continuous, so this is just the Fundamental Theorem of Calculus.

(b) \(-1/\lambda^{2}\), for \(\lambda>0\). Here the range is infinite, so the interchange needs Corollary 2.4.4. Fix \(\lambda>0\) and pick \(\delta_0\) with \(0<\delta_0<\lambda\). For all \(\lambda^{\prime}\) with \(|\lambda^{\prime}-\lambda|\le\delta_0\) and all \(t>0\),

\begin{equation*} \left|\frac{\partial}{\partial\lambda}e^{-\lambda t}\right|_{\lambda=\lambda^{\prime}} = t e^{-\lambda^{\prime} t} \le t e^{-(\lambda-\delta_0)t} =: g(t,\lambda), \end{equation*}

and \(\int_0^{\infty} g(t,\lambda)\,dt = (\lambda-\delta_0)^{-2} < \infty\) because \(\lambda-\delta_0>0\). So (2.4.3) holds and

\begin{equation*} \frac{d}{d\lambda}\int_0^{\infty}e^{-\lambda t}\,dt = -\int_0^{\infty} t e^{-\lambda t}\,dt = -\frac{1}{\lambda^{2}}, \end{equation*}

agreeing with the direct evaluation \(\frac{d}{d\lambda}(1/\lambda) = -1/\lambda^{2}\).

(c) \(-1/t^{2}\), for \(t>0\). The integrand has no \(t\) in it, so Leibnitz’s Rule contributes only the lower-limit term, with \(a(t)=t\), \(b=1\):

\begin{equation*} \frac{d}{dt}\int_t^{1}\frac{dx}{x^{2}} = -\frac{1}{a(t)^{2}}\,a^{\prime}(t) = -\frac{1}{t^{2}} . \end{equation*}

(Directly, \(\int_t^1 x^{-2}dx = t^{-1}-1\). Check!) For \(t\le 0\) the integral diverges and there is nothing to differentiate.

(d) \(1/(1-t)^{2}\), for \(t<1\) (the integral diverges otherwise). Evaluating first,

\begin{equation*} \int_1^{\infty}\frac{dx}{(x-t)^{2}} = \left[\frac{-1}{x-t}\right]_{1}^{\infty} = \frac{1}{1-t}, \end{equation*}

whose derivative is \(1/(1-t)^{2}\). Differentiating under the integral sign gives the same number, and that interchange is what needs justifying. Fix \(t_0<1\) and choose \(\delta_0>0\) with \(t_0+\delta_0<1\). For \(x\ge 1\) and \(|t^{\prime}-t_0|\le\delta_0\),

\begin{equation*} \left|\frac{\partial}{\partial t}\frac{1}{(x-t)^{2}}\right|_{t=t^{\prime}} = \frac{2}{(x-t^{\prime})^{3}} \le \frac{2}{(x-t_0-\delta_0)^{3}} =: g(x,t_0), \end{equation*}

and, since \(1-t_0-\delta_0>0\),

\begin{equation*} \int_1^{\infty} g(x,t_0)\,dx = \frac{1}{(1-t_0-\delta_0)^{2}} < \infty . \end{equation*}

Corollary 2.4.4 therefore applies and

\begin{equation*} \frac{d}{dt}\int_1^{\infty}\frac{dx}{(x-t)^{2}} = \int_1^{\infty}\frac{2\,dx}{(x-t)^{3}} = \frac{1}{(1-t)^{2}}, \end{equation*}

which is indeed \(\frac{d}{dt}\big[(1-t)^{-1}\big]\).

Both sides vanish at \(p=1\) and have the same derivative on \((0,1)\). (Throughout \(0 \le x \le n-1\): at \(x=n\) the right side reads \(0\) times a divergent integral, so the identity is stated only for \(x<n\).)

Write \(L(p)\) and \(R(p)\) for the two sides. For \(R\), Leibnitz’s Rule (Theorem 2.4.1) with \(a=0\), \(b(p)=1-p\) and an integrand free of \(p\) (continuous on \([0,1-p]\), since \(n-x-1\ge 0\)) leaves only the boundary term:

\begin{equation*} \begin{aligned} R^{\prime}(p) &= (n-x)\binom{n}{x}\,b(p)^{\,n-x-1}\big(1-b(p)\big)^{x}\,b^{\prime}(p)\\ &= -\,(n-x)\binom{n}{x}\,p^{x}(1-p)^{\,n-x-1}. \end{aligned} \end{equation*}

For \(L\), differentiate termwise and use \(k\binom{n}{k} = n\binom{n-1}{k-1}\) and \((n-k)\binom{n}{k} = n\binom{n-1}{k}\). Putting

\begin{equation*} a_{j} = \binom{n-1}{j}p^{j}(1-p)^{\,n-1-j}, \qquad a_{-1} := 0, \end{equation*}

the \(k\)th derivative is

\begin{equation*} \begin{aligned} \frac{d}{dp}\binom{n}{k}p^{k}(1-p)^{n-k} &= \binom{n}{k}\Big[k p^{k-1}(1-p)^{n-k}\\ &\qquad\quad - (n-k)p^{k}(1-p)^{\,n-k-1}\Big]\\ &= n\big(a_{k-1}-a_{k}\big), \end{aligned} \end{equation*}

so the sum telescopes:

\begin{equation*} L^{\prime}(p) = n\sum_{k=0}^{x}\big(a_{k-1}-a_{k}\big) = -\,n\,a_{x} = -\,n\binom{n-1}{x}p^{x}(1-p)^{\,n-1-x}. \end{equation*}

Since \(n\binom{n-1}{x} = (n-x)\binom{n}{x}\), we get \(L^{\prime}\equiv R^{\prime}\) on \((0,1)\).

At \(p=1\) every term of \(L\) carries a factor \((1-p)^{n-k}\) with \(n-k \ge n-x \ge 1\), so \(L(1)=0\); and \(R(1)=0\) because the integral is over \([0,0]\). Both functions are continuous on \([0,1]\), so \(L\equiv R\) there.

Method (2) (integration by parts): with \(u=(1-t)^{x}\) and \(dv = (n-x)t^{\,n-x-1}dt\),

\begin{equation*} \begin{aligned} R_{x}(p) &= \binom{n}{x}\Big[t^{\,n-x}(1-t)^{x}\Big]_{0}^{1-p}\\ &\quad + x\binom{n}{x}\int_{0}^{1-p} t^{\,n-x}(1-t)^{x-1}\,dt\\ &= \binom{n}{x}p^{x}(1-p)^{\,n-x} + R_{x-1}(p), \end{aligned} \end{equation*}

the last step because \(x\binom{n}{x} = (n-x+1)\binom{n}{x-1}\). Since \(R_{0}(p) = n\int_{0}^{1-p}t^{\,n-1}dt = (1-p)^{n}\), induction on \(x\) gives \(R_{x} = L\).

\begin{equation*} EX=\frac{N_0+N_1}{2},\qquad \operatorname{Var}X=\frac{(N_1-N_0)(N_1-N_0+2)}{12}. \end{equation*}

Put \(n=N_1-N_0+1\) (the number of mass points) and \(Y=X-N_0+1\). Since \(P(X=x)=1/(N_1-N_0+1)\) on \(\{N_0,\ldots,N_1\}\) (p. 86), \(Y\) has the discrete uniform\((1,n)\) pmf (3.2.1), for which the book computes \(EY=(n+1)/2\) and \(\operatorname{Var}Y=(n+1)(n-1)/12\). Location shifts move the mean and leave the variance alone, so

\begin{equation*} \begin{aligned} EX&=EY+N_0-1=\frac{n+1}{2}+N_0-1\\ &=\frac{N_1-N_0+2}{2}+N_0-1=\frac{N_0+N_1}{2},\\[2pt] \operatorname{Var}X&=\operatorname{Var}Y=\frac{(n+1)(n-1)}{12}\\ &=\frac{(N_1-N_0+2)(N_1-N_0)}{12}. \end{aligned} \end{equation*}

(a) \(K=32\). (b) \(K=51\).

This is Example 3.2.1 (acceptance sampling) with \(N=100\). If the lot holds \(M\) defectives, the number \(X\) of defectives in a sample of \(K\) drawn without replacement is hypergeometric\((100,M,K)\), pmf (3.2.2), and an unacceptable lot is one with \(M\ge 6\). Both acceptance rules have the form \(\{X\le c\}\), and for such a rule the worst unacceptable lot is \(M=6\): couple a lot of \(M\) defectives with one of \(M+1\) by nesting the defective sets, so the sampled defective count can only increase and \(P(X\le c\mid M)\) is nonincreasing in \(M\).

(a) Here \(c=0\), so the requirement is

\begin{equation*} P(X=0\mid M=6)=\frac{\binom{94}{K}}{\binom{100}{K}} =\frac{(100-K)(99-K)\cdots(95-K)}{100\cdot 99\cdots 95}<.10 , \end{equation*}

whose left side is decreasing in \(K\), and

\begin{equation*} \begin{aligned} K=31:&\quad .1006,\\ K=32:&\quad .0918 . \end{aligned} \end{equation*}

Hence \(K=32\) is the smallest sample size that works.

(b) Now \(c=1\), so at \(M=6\) the requirement is

\begin{equation*} P(X\le 1\mid M=6)= \frac{\binom{94}{K}+6\binom{94}{K-1}}{\binom{100}{K}}<.10 , \end{equation*}

and

\begin{equation*} \begin{aligned} K=50:&\quad .1022,\\ K=51:&\quad .0933, \end{aligned} \end{equation*}

so \(K=51\).

\begin{equation*} P(W=4)=p(1-p)^3\bigl[1-(1-p)^3\bigr]. \end{equation*}

Number the seconds after the pedestrian’s arrival \(1,2,3,\ldots\) and let \(C_i\) be the event that a car passes in second \(i\); the \(C_i\) are independent with \(P(C_i)=p\). He starts to cross at the first time \(t\ge 0\) at which seconds \(t+1,t+2,t+3\) are all car-free, and \(W=t\) is his wait. Thus \(\{W=4\}\) is the event that seconds \(5,6,7\) are car-free while \(t=0,1,2,3\) each fail:

(i) \(t=3\) fails \(\iff\) a car passes in one of \(4,5,6\); given \(5,6\) free this is \(C_4\).

(ii) \(t=2\) and \(t=1\) then fail automatically, since second \(4\) lies in \(\{3,4,5\}\) and in \(\{2,3,4\}\).

(iii) \(t=0\) fails \(\iff\) a car passes in one of \(1,2,3\).

Hence

\begin{equation*} \{W=4\}=(C_1\cup C_2\cup C_3)\cap C_4\cap C_5^{c}\cap C_6^{c}\cap C_7^{c}, \end{equation*}

an intersection of events determined by the disjoint blocks of seconds \(\{1,2,3\}\), \(\{4\}\), \(\{5,6,7\}\), so the three probabilities multiply:

\begin{equation*} \begin{aligned} P(W=4)&=\bigl[1-(1-p)^3\bigr]\cdot p\cdot (1-p)^3\\ &=p(1-p)^3-p(1-p)^6 . \end{aligned} \end{equation*}

(a) \(n\). (b) \((n+1)/2\).

(a) Each draw is an independent Bernoulli trial with success probability \(1/n\), and \(T\) is the trial of the first success, so \(T\) is geometric\((1/n)\) (p. 97) and

\begin{equation*} ET=\frac1{1/n}=n . \end{equation*}

(b) Sampling without replacement, for \(k=1,\ldots,n\),

\begin{equation*} \begin{aligned} P(T=k)&=\frac{n-1}{n}\cdot\frac{n-2}{n-1}\cdots \frac{n-k+1}{n-k+2}\cdot\frac{1}{n-k+1}\\ &=\frac1n, \end{aligned} \end{equation*}

the first \(k-1\) factors being the conditional probabilities of successive failures and the last that of success on trial \(k\); the product telescopes. So \(T\) is discrete uniform\((1,n)\) and, by (3.2.1) (or Exercise 3.1 with \(N_0=1\), \(N_1=n\)),

\begin{equation*} ET=\frac{n+1}{2}. \end{equation*}

No: under the hypothesis of equal effectiveness, 85 or more successes has probability \(.128\), which is not at all unusual.

Let \(X\) be the number of cases in which the new drug is effective. If the two drugs are equally effective then \(X\) is binomial\((100,.8)\), and

\begin{equation*} P(X\ge 85)=\sum_{x=85}^{100}\binom{100}{x}(.8)^x(.2)^{100-x} =.1285 . \end{equation*}

Equivalently, by the normal approximation with the continuity correction (Example 3.3.2), with \(EX=100(.8)=80\) and \(\operatorname{Var}X=100(.8)(.2)=16\),

\begin{equation*} \begin{aligned} P(X\ge 85)&\approx P\!\left(Z\ge\frac{84.5-80}{4}\right)\\ &=P(Z\ge 1.125)=.130 . \end{aligned} \end{equation*}

(a) \(X\sim\text{binomial}(2000,\,.01)\).

Treat the insects as independent trials, each “success” being survival of the insecticide, which has probability \(1-.99=.01\); with \(n=2000\) fixed trials this is exactly the binomial model.

(b)

\begin{equation*} P(X<100)=\sum_{x=0}^{99}\binom{2000}{x}(.01)^x(.99)^{2000-x}. \end{equation*}

(c) \(P(X<100)\approx 1\).

Here \(n\) is large and \(p\) small, so the Poisson approximation of Example 3.2.5 applies with \(\lambda=np=2000(.01)=20\):

\begin{equation*} P(X<100)\approx\sum_{x=0}^{99}\frac{e^{-20}20^{x}}{x!}. \end{equation*}

Since \(99\) sits about \(18\) standard deviations (\(\sigma=\sqrt{20}\approx 4.47\)) above the mean, the omitted tail is

\begin{equation*} \sum_{x=100}^{\infty}\frac{e^{-20}20^{x}}{x!} \approx 3.5\times 10^{-37}, \end{equation*}

so the probability is \(1\) to any practical accuracy.

\(\lambda=6.6384\) chips per cookie.

Let \(X\sim\text{Poisson}(\lambda)\), so \(EX=\lambda\). As in Example 3.2.4,

\begin{equation*} \begin{aligned} P(X\ge 2)&=1-P(X=0)-P(X=1)\\ &=1-e^{-\lambda}-\lambda e^{-\lambda}, \end{aligned} \end{equation*}

so the requirement \(P(X\ge 2)>.99\) is

\begin{equation*} g(\lambda):=(1+\lambda)e^{-\lambda}<.01 . \end{equation*}

Since \(g^{\prime}(\lambda)=-\lambda e^{-\lambda}<0\) for \(\lambda>0\), \(g\) decreases strictly from \(g(0)=1\) to \(0\), so the inequality holds exactly for \(\lambda>\lambda^{*}\), where \(g(\lambda^{*})=.01\). Numerically,

\begin{equation*} \begin{aligned} g(6.63)&=.010073,\qquad g(6.64)=.0099857,\\ \lambda^{*}&=6.63835 . \end{aligned} \end{equation*}

The admissible set is open, so \(\lambda^{*}\) is its infimum rather than a minimum, and the mean must exceed \(6.6384\).

\(N = 541\) seats in each theater.

(a) Let \(X\) be the number of the 1,000 customers who pick theater \(A\); independence and indifference make \(X \sim \text{binomial}(1000, \tfrac12)\), and theater \(B\) draws the other \(1000 - X\). Somebody is turned away exactly when \(X > N\) or \(1000 - X > N\). For \(N \ge 500\) these two events are disjoint, and the binomial\((1000,\tfrac12)\) pmf is symmetric about \(500\), so they have the same probability:

\begin{equation*} \begin{aligned} P(\text{turn away}) &= P(X > N) + P(X < 1000-N) \\ &= 2\sum_{x=N+1}^{1000}\binom{1000}{x}\Bigl(\tfrac12\Bigr)^{1000}. \end{aligned} \end{equation*}

Take \(N\) to be the smallest integer with

\begin{equation*} \sum_{x=N+1}^{1000}\binom{1000}{x}\Bigl(\tfrac12\Bigr)^{1000} < .005 . \end{equation*}

(b) Here \(\mathrm{E}X = 1000\cdot\tfrac12 = 500\) and \(\operatorname{Var}X = 1000\cdot\tfrac12\cdot\tfrac12 = 250\), so \(\sigma = \sqrt{250} = 15.81\). With the continuity correction,

\begin{equation*} P(X > N) \approx P\Bigl(Z > \frac{N + .5 - 500}{15.81}\Bigr) < .005 \end{equation*}

iff \((N + .5 - 500)/15.81 > z_{.005} = 2.5758\), i.e.

\begin{equation*} N > 499.5 + 2.5758(15.81) = 540.2 , \end{equation*}

so \(N = 541\).

Not impossible at all: over ten years of American kindergartens such a class is essentially certain to appear.

(a) Index the 60 entering children by their births and let each birth be a “success” if it was a twin birth, with probability \(1/90\); treating the 60 births as independent Bernoulli trials makes \(X\), the number of twin births represented in the class, a binomial\((60,1/90)\) variable, and “five sets of twins” is \(\{X \ge 5\}\). Then

\begin{equation*} \begin{aligned} P(X \ge 5) &= \sum_{x=5}^{60}\binom{60}{x}\Bigl(\tfrac1{90}\Bigr)^{x}\Bigl(\tfrac{89}{90}\Bigr)^{60-x} \\ &= .000557 , \end{aligned} \end{equation*}

the sum being dominated by its first term \(.000500\). About one chance in 1,800 – rare enough to print, but nowhere near impossible.

(b) With \(62 \times 5 = 310\) schools in the state, each an independent trial with success probability \(p = .000557\), the number of schools reporting the event is binomial\((310,p)\) and

\begin{equation*} 1 - (1-p)^{310} = .159 , \end{equation*}

roughly one chance in six that at least one New York school has such a class this year. Already unremarkable.

(c) Over 50 states and 10 years there are \(310 \times 50 \times 10 = 155{,}000\) school-years, so the expected number of such classes is \(155{,}000\,p = 86.3\) and

\begin{equation*} 1 - (1-p)^{155{,}000} = 1 - e^{-86.3} \approx 1 . \end{equation*}

Not a statistical impossibility but a near certainty.

(a) The four tested packets are a simple random sample of size 4 from the \(N+M\) packets, so the probability that all four are cocaine is the hypergeometric probability

\begin{equation*} P(\text{4 cocaine}) = \frac{\binom{N}{4}\binom{M}{0}}{\binom{N+M}{4}} = \frac{\binom{N}{4}}{\binom{N+M}{4}} . \end{equation*}

Given that event, the pool left for the second draw has \(N-4\) cocaine and \(M\) noncocaine packets, a total of \(N+M-4\), and the two sold packets are a simple random sample of size 2 from it, so

\begin{equation*} P(\text{2 noncocaine} \mid \text{4 cocaine}) = \frac{\binom{M}{2}}{\binom{N+M-4}{2}} . \end{equation*}

Multiplying the two gives the stated expression.

(b) With \(N + M = 496\) fixed the denominator \(\binom{496}{4}\binom{492}{2}\) is constant, so we maximize \(g(N) = \binom{N}{4}\binom{496-N}{2}\) over \(4 \le N \le 494\). The ratio of consecutive terms is

\begin{equation*} \frac{g(N+1)}{g(N)} = \frac{N+1}{N-3}\cdot\frac{494-N}{496-N} , \end{equation*}

and \(g(N+1) \ge g(N)\) iff \((N+1)(494-N) \ge (N-3)(496-N)\), i.e. iff

\begin{equation*} 493N + 494 \ge 499N - 1488 \iff N \le \tfrac{1982}{6} = 330.33 . \end{equation*}

So \(g\) increases up through \(N = 331\) and decreases thereafter: the maximum is at \(N = 331\), \(M = 165\), where

\begin{equation*} \frac{\binom{331}{4}\binom{165}{2}}{\binom{496}{4}\binom{492}{2}} = .0221 . \end{equation*}

Write \(a^{(b)} = a(a-1)\cdots(a-b+1)\) for the falling factorial, so that \(\binom{a}{b} = a^{(b)}/b!\). Cancelling the three factorial denominators against \(K!/(x!(K-x)!)\) puts the pmf in the form used throughout:

\begin{equation*} P(X = x \mid N,M,K) = \binom{K}{x}\,\frac{M^{(x)}\,(N-M)^{(K-x)}}{N^{(K)}} . \end{equation*}

(a) Here \(K\) and \(x\) are fixed, so split the ratio into three products of finitely many factors:

\begin{equation*} \begin{aligned} \frac{M^{(x)}}{N^{x}} &= \prod_{j=0}^{x-1}\Bigl(\frac{M}{N}-\frac{j}{N}\Bigr) \to p^{x}, \\ \frac{(N-M)^{(K-x)}}{N^{K-x}} &= \prod_{j=0}^{K-x-1}\Bigl(1-\frac{M}{N}-\frac{j}{N}\Bigr) \to (1-p)^{K-x}, \\ \frac{N^{K}}{N^{(K)}} &= \prod_{j=0}^{K-1}\Bigl(1-\frac{j}{N}\Bigr)^{-1} \to 1 , \end{aligned} \end{equation*}

using \(M/N \to p\) and \(N \to \infty\) in each. Their product is exactly \(M^{(x)}(N-M)^{(K-x)}/N^{(K)}\), so

\begin{equation*} P(X = x \mid N,M,K) \to \binom{K}{x}p^{x}(1-p)^{K-x} . \end{equation*}

(No Stirling needed: the falling factorials are products of boundedly many factors.)

(b) By (a), for each fixed \(K\) the hypergeometric pmf is the binomial\((K, p)\) pmf with \(p = M/N\) in the limit of large \(N, M\); here \(p = M/N \to 0\) while \(Kp = KM/N \to \lambda\), which is precisely the regime of the Poisson approximation to the binomial, (2.3.9), established in Example 2.3.13 (its hypotheses are exactly \(p \to 0\) with \(Kp \to \lambda\), and Theorem 2.3.12 converts the mgf convergence proved there into convergence of the pmfs). Hence

\begin{equation*} \binom{K}{x}\Bigl(\frac{M}{N}\Bigr)^{x}\Bigl(1-\frac{M}{N}\Bigr)^{K-x} \to \frac{e^{-\lambda}\lambda^{x}}{x!}, \end{equation*}

and passing the two limits in this order gives the assertion.

(c) Fix \(x\) and factor the displayed pmf as

\begin{equation*} \begin{aligned} P(X = x \mid N,M,K) &= \frac{1}{x!}\,A\,B\,C, \qquad\text{where} \\ A &= \frac{K^{(x)}M^{(x)}}{N^{x}}, \\ B &= \frac{(N-M)^{(K-x)}}{N^{(K-x)}}, \\ C &= \frac{N^{(K-x)}N^{x}}{N^{(K)}} . \end{aligned} \end{equation*}

Note \(K/N = (KM/N)/M \to 0\) since \(M \to \infty\).

\(A = \prod_{j=0}^{x-1}\frac{KM}{N}\bigl(1-\frac{j}{K}\bigr)\bigl(1-\frac{j}{M}\bigr) \to \lambda^{x}\), as \(x\) is fixed and \(K, M \to \infty\).

\(C = \prod_{j=K-x}^{K-1}\bigl(1-\frac{j}{N}\bigr)^{-1} \to 1\), since each of these \(x\) factors lies between \(1\) and \((1-K/N)^{-1} \to 1\).

\(B = \prod_{j=0}^{K-x-1}\bigl(1-\frac{M}{N-j}\bigr)\) has factors decreasing in \(j\) and positive for large \(N\) (as \((M+K)/N \to 0\)), so

\begin{equation*} \Bigl(1-\frac{M}{N-K}\Bigr)^{K-x} \le B \le \Bigl(1-\frac{M}{N}\Bigr)^{K-x} . \end{equation*}

Put \(\lambda_K = KM/N \to \lambda\) and \(\mu_K = KM/(N-K) = \lambda_K/(1-K/N) \to \lambda\). Both bounds are of the form \((1 - c_K/K)^{K}(1-c_K/K)^{-x}\) with \(c_K \to \lambda\), and \((1-c_K/K)^{-x} \to 1\) because \(c_K/K \to 0\); Lemma 2.3.14 (with \(a_K = -c_K \to -\lambda\)) gives \((1-c_K/K)^{K} \to e^{-\lambda}\). So both bounds tend to \(e^{-\lambda}\) and \(B \to e^{-\lambda}\).

Therefore \(P(X = x \mid N,M,K) \to \lambda^{x}e^{-\lambda}/x!\).

Both sides count the same event in one sequence of trials. Run independent Bernoulli\((p)\) trials \(Z_1, Z_2, \ldots\) and set

\begin{equation*} X = \sum_{i=1}^{n} Z_i, \qquad Y = (\text{number of failures before the }r\text{th success}), \end{equation*}

so \(X \sim \text{binomial}(n,p)\) and \(Y \sim \text{negative binomial}(r,p)\) in the pmf (3.2.10) convention of Section 3.2. The \(r\)th success occurs at trial \(Y + r\), so the first \(n\) trials contain at least \(r\) successes if and only if \(Y + r \le n\); that is,

\begin{equation*} \{X \le r-1\} = \{Y + r > n\} = \{Y > n-r\} . \end{equation*}

Taking probabilities,

\begin{equation*} F_X(r-1) = P(X \le r-1) = P(Y > n-r) = 1 - F_Y(n-r) . \end{equation*}

\(X_T\) is \(X\) conditioned on \(\{X > 0\}\), so \(\mathrm{E}\,g(X_T) = \mathrm{E}\,g(X)/P(X > 0)\) for every \(g\) with \(g(0) = 0\); since \(x\) and \(x^{2}\) both vanish at \(0\), the untruncated first two moments carry over after division by \(P(X > 0)\).

(a) \(P(X > 0) = 1 - e^{-\lambda}\), so

\begin{equation*} P(X_T = x) = \frac{e^{-\lambda}\lambda^{x}}{x!\,(1-e^{-\lambda})}, \qquad x = 1,2,\ldots . \end{equation*}

With \(\mathrm{E}X = \lambda\) and \(\mathrm{E}X^2 = \lambda + \lambda^2\),

\begin{equation*} \mathrm{E}X_T = \frac{\lambda}{1-e^{-\lambda}}, \qquad \mathrm{E}X_T^2 = \frac{\lambda+\lambda^2}{1-e^{-\lambda}}, \end{equation*}

\begin{equation*} \begin{aligned} \operatorname{Var}X_T &= \frac{\lambda+\lambda^{2}}{1-e^{-\lambda}} - \frac{\lambda^{2}}{(1-e^{-\lambda})^{2}} \\ &= \frac{\lambda\bigl[(1+\lambda)(1-e^{-\lambda}) - \lambda\bigr]}{(1-e^{-\lambda})^{2}} . \end{aligned} \end{equation*}

(b) \(P(X > 0) = 1 - P(X = 0) = 1 - p^{r}\), so

\begin{equation*} P(X_T = x) = \frac{\binom{r+x-1}{x}p^{r}(1-p)^{x}}{1-p^{r}}, \qquad x = 1,2,\ldots . \end{equation*}

From Section 3.2, \(\mathrm{E}X = r(1-p)/p\) and \(\operatorname{Var}X = r(1-p)/p^{2}\), whence

\begin{equation*} \mathrm{E}X^{2} = \frac{r(1-p)}{p^{2}} + \frac{r^{2}(1-p)^{2}}{p^{2}} = \frac{r(1-p)\bigl[1 + r(1-p)\bigr]}{p^{2}} . \end{equation*}

Therefore

\begin{equation*} \mathrm{E}X_T = \frac{r(1-p)}{p\,(1-p^{r})}, \qquad \mathrm{E}X_T^{2} = \frac{r(1-p)\bigl[1+r(1-p)\bigr]}{p^{2}(1-p^{r})}, \end{equation*}

\begin{equation*} \begin{aligned} \operatorname{Var}X_T &= \frac{r(1-p)\bigl[1+r(1-p)\bigr]}{p^{2}(1-p^{r})}

• \frac{r^{2}(1-p)^{2}}{p^{2}(1-p^{r})^{2}} \\ &= \frac{r(1-p)\bigl[(1+r(1-p))(1-p^{r}) - r(1-p)\bigr]} {p^{2}(1-p^{r})^{2}} . \end{aligned} \end{equation*}

Everything follows from the Taylor series \(-\log(1-q) = \sum_{x=1}^{\infty} q^{x}/x\), valid for \(|q| < 1\), taken at \(q = 1-p\).

(a) Each term is positive: \((1-p)^{x}/x > 0\) and \(\log p < 0\) for \(0 < p < 1\), so the leading minus sign makes \(P(X = x) > 0\). And

\begin{equation*} \sum_{x=1}^{\infty}\frac{-(1-p)^{x}}{x\log p} = \frac{-1}{\log p}\sum_{x=1}^{\infty}\frac{(1-p)^{x}}{x} = \frac{-1}{\log p}\bigl(-\log p\bigr) = 1 . \end{equation*}

(b) The \(x\) in the denominator cancels, leaving geometric series:

\begin{equation*} \mathrm{E}X = \frac{-1}{\log p}\sum_{x=1}^{\infty}(1-p)^{x} = \frac{-1}{\log p}\cdot\frac{1-p}{p} = \frac{-(1-p)}{p\log p} , \end{equation*}

\begin{equation*} \mathrm{E}X^{2} = \frac{-1}{\log p}\sum_{x=1}^{\infty}x(1-p)^{x} = \frac{-1}{\log p}\cdot\frac{1-p}{p^{2}} = \frac{-(1-p)}{p^{2}\log p} , \end{equation*}

using \(\sum_{x \ge 1} q^{x} = q/(1-q)\) and \(\sum_{x \ge 1} x q^{x} = q/(1-q)^{2}\) at \(q = 1-p\). Hence

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \frac{-(1-p)}{p^{2}\log p} - \frac{(1-p)^{2}}{p^{2}(\log p)^{2}} \\ &= \frac{-(1-p)\bigl[\log p + (1-p)\bigr]}{p^{2}(\log p)^{2}} . \end{aligned} \end{equation*}

With \(\lambda_r = r(1-p_r) \to \lambda\), so that \(1-p_r = \lambda_r/r\), the negative binomial\((r,p)\) mgf is

\begin{equation*} M_r(t) \;=\; \left(\frac{p_r}{1-(1-p_r)e^{t}}\right)^{\!r} \;=\; \frac{\bigl(1-\frac{\lambda_r}{r}\bigr)^{r}}{\bigl(1-\frac{\lambda_r e^{t}}{r}\bigr)^{r}}, \end{equation*}

valid for all \(t\) with \((\lambda_r/r)e^{t}<1\), which holds for every fixed \(t\) once \(r\) is large. Both factors are of the form \((1+a_r/r)^r\) with a convergent sequence \(a_r\), so Lemma 2.3.14 applies to each:

\begin{equation*} \begin{aligned} \left(1-\tfrac{\lambda_r}{r}\right)^{r} &\longrightarrow e^{-\lambda},\\ \left(1-\tfrac{\lambda_r e^{t}}{r}\right)^{r} &\longrightarrow e^{-\lambda e^{t}} \neq 0 . \end{aligned} \end{equation*}

Hence for every \(t \in (-\infty,\infty)\),

\begin{equation*} M_r(t) \;\longrightarrow\; \frac{e^{-\lambda}}{e^{-\lambda e^{t}}} \;=\; e^{\lambda(e^{t}-1)}, \end{equation*}

the mgf of a Poisson\((\lambda)\) random variable. Since the limit mgf is finite in a neighborhood of \(0\), Theorem 2.3.12 converts this into convergence of the distributions.

(a) Integrate \(\Gamma(\alpha+1)=\int_0^{\infty} t^{\alpha}e^{-t}\,dt\) by parts with \(u=t^{\alpha}\), \(dv=e^{-t}dt\):

\begin{equation*} \Gamma(\alpha+1) = \Bigl[-t^{\alpha}e^{-t}\Bigr]_0^{\infty}

• \alpha\int_0^{\infty} t^{\alpha-1}e^{-t}\,dt = \alpha\,\Gamma(\alpha), \end{equation*}

the boundary term vanishing at \(0\) because \(\alpha>0\) and at \(\infty\) because \(e^{-t}\) beats \(t^{\alpha}\).

(b) Substitute \(t=z^{2}/2\), so \(dt=z\,dz\) and \(t^{-1/2}=\sqrt{2}/z\) on \(z>0\):

\begin{equation*} \begin{aligned} \Gamma\!\left(\tfrac12\right) &= \int_0^{\infty} t^{-1/2}e^{-t}\,dt = \int_0^{\infty}\frac{\sqrt2}{z}\,e^{-z^{2}/2}\,z\,dz\\[2pt] &= \sqrt{2}\int_0^{\infty} e^{-z^{2}/2}\,dz = \sqrt{2}\cdot\sqrt{\tfrac{\pi}{2}} = \sqrt{\pi} , \end{aligned} \end{equation*}

the value of the last integral being (3.3.14), which the text obtains by squaring it and converting to polar coordinates.

Recognize the integrand as an unnormalized gamma\((\nu+\alpha,\beta)\) pdf, exactly as (3.3.18) recognizes a beta kernel:

\begin{equation*} \begin{aligned} \mathrm{E}X^{\nu} &= \frac{1}{\Gamma(\alpha)\beta^{\alpha}}\int_0^{\infty} x^{\nu}\,x^{\alpha-1}e^{-x/\beta}\,dx\\[2pt] &= \frac{1}{\Gamma(\alpha)\beta^{\alpha}}\int_0^{\infty} x^{(\nu+\alpha)-1}e^{-x/\beta}\,dx\\[2pt] &= \frac{\Gamma(\nu+\alpha)\,\beta^{\nu+\alpha}}{\Gamma(\alpha)\beta^{\alpha}} = \frac{\beta^{\nu}\,\Gamma(\nu+\alpha)}{\Gamma(\alpha)} . \end{aligned} \end{equation*}

The third equality is (3.3.8) with \(\alpha\) there replaced by \(\nu+\alpha\), legitimate because \(\nu>0\) makes \(\nu+\alpha>0\), so gamma\((\nu+\alpha,\beta)\) is a genuine pdf.

By Theorem 2.3.15 and the negative binomial\((r,p)\) mgf (Table of Common Distributions; here \(Y\) counts failures, pmf (3.2.10)),

\begin{equation*} M_{pY}(t) \;=\; M_Y(pt) \;=\; \left(\frac{p}{\,1-(1-p)e^{pt}\,}\right)^{\!r}. \end{equation*}

Fix \(t<1\) and set \(h(p)=1-(1-p)e^{pt}\), a smooth function of \(p\) with \(h(0)=0\) and

\begin{equation*} h^{\prime}(p) = e^{pt} - (1-p)t\,e^{pt}, \qquad h^{\prime}(0) = 1-t . \end{equation*}

Since \(h^{\prime}(0)=1-t \neq 0\), we get \(p/h(p) = p/(h(p)-h(0)) \to 1/h^{\prime}(0) = 1/(1-t)\) as \(p \to 0\); in particular \(h(p)>0\) for all small \(p>0\), which is exactly the condition \(pt < -\log(1-p)\) under which \(M_Y(pt)\) exists. Therefore, with \(r\) held fixed,

\begin{equation*} M_{pY}(t) \;\longrightarrow\; \left(\frac{1}{1-t}\right)^{\!r}, \qquad t<1, \end{equation*}

which is the gamma\((\alpha,\beta)\) mgf \((1-\beta t)^{-\alpha}\) of Example 2.3.8 at \(\alpha=r\), \(\beta=1\). As this limit is an mgf finite on a neighborhood of \(0\), Theorem 2.3.12 gives \(pY \to\) gamma\((r,1)\) in distribution.

Induct on \(\alpha\), writing \(I_\alpha(x)=\frac{1}{\Gamma(\alpha)}\int_x^{\infty}z^{\alpha-1}e^{-z}\,dz\).

Base case \(\alpha=1\): \(I_1(x)=\int_x^{\infty}e^{-z}dz=e^{-x}\), which is the \(y=0\) term alone.

Induction step: for \(\alpha\ge 2\), integrate by parts with \(u=z^{\alpha-1}\), \(dv=e^{-z}dz\),

\begin{equation*} \begin{aligned} \int_x^{\infty} z^{\alpha-1}e^{-z}\,dz &= \Bigl[-z^{\alpha-1}e^{-z}\Bigr]_x^{\infty}

• (\alpha-1)\int_x^{\infty} z^{\alpha-2}e^{-z}\,dz\\[2pt] &= x^{\alpha-1}e^{-x} + (\alpha-1)\int_x^{\infty} z^{\alpha-2}e^{-z}\,dz . \end{aligned} \end{equation*}

Dividing by \(\Gamma(\alpha)=(\alpha-1)!\) and using \(\Gamma(\alpha)=(\alpha-1)\Gamma(\alpha-1)\) (Exercise 3.16(a)),

\begin{equation*} I_\alpha(x) \;=\; \frac{x^{\alpha-1}e^{-x}}{(\alpha-1)!} \;+\; I_{\alpha-1}(x), \end{equation*}

which with the inductive hypothesis gives \(I_\alpha(x)=\sum_{y=0}^{\alpha-1}\frac{x^{y}e^{-x}}{y!}\).

Probabilistically: the left side is \(P(X>x)\) for \(X\sim\) gamma\((\alpha,1)\) and the right side is \(P(Y\le \alpha-1)\) for \(Y\sim\) Poisson\((x)\), so

\begin{equation*} P(X > x) \;=\; P(Y \le \alpha-1), \qquad\text{equivalently}\qquad P(X\le x) = P(Y\ge\alpha). \end{equation*}

Rescaling \(X\) by \(\beta\) (so \(X\sim\) gamma\((\alpha,\beta)\) and \(Y\sim\) Poisson\((x/\beta)\)) recovers (3.3.9) of Example 3.3.1.

(a) \(\mathrm{E}X=\sqrt{2/\pi}\) and \(\operatorname{Var}X = 1-\dfrac{2}{\pi}\). For the mean, \(\frac{d}{dx}\bigl(-e^{-x^2/2}\bigr)=x e^{-x^2/2}\), so

\begin{equation*} \mathrm{E}X = \frac{2}{\sqrt{2\pi}}\int_0^{\infty} x\,e^{-x^{2}/2}\,dx = \frac{2}{\sqrt{2\pi}}\Bigl[-e^{-x^{2}/2}\Bigr]_0^{\infty} = \frac{2}{\sqrt{2\pi}} = \sqrt{\frac{2}{\pi}} . \end{equation*}

For the second moment, \(f\) is exactly twice the n\((0,1)\) pdf restricted to \(x>0\), and \(x^{2}\) is even, so

\begin{equation*} \mathrm{E}X^{2} = \int_{-\infty}^{\infty} x^{2}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}\,dx = 1, \end{equation*}

the variance of a n\((0,1)\) variable. Hence \(\operatorname{Var}X = 1 - 2/\pi \approx 0.3634\).

(b) Take \(Y=g(X)=X^{2}\), with \(\alpha=\tfrac12\) and \(\beta=2\); that is, \(Y\sim\chi^2_1\). The map \(x\mapsto x^{2}\) is monotone on the support \((0,\infty)\) with inverse \(y^{1/2}\) and \(\frac{d}{dy}y^{1/2}=\frac{1}{2}y^{-1/2}\), so Theorem 2.1.5 gives

\begin{equation*} f_Y(y) = \frac{2}{\sqrt{2\pi}}e^{-y/2}\cdot\frac{1}{2}y^{-1/2} = \frac{1}{\sqrt{2\pi}}\,y^{-1/2}e^{-y/2}, \qquad y>0 . \end{equation*}

Since \(\Gamma(\tfrac12)2^{1/2}=\sqrt{\pi}\sqrt{2}=\sqrt{2\pi}\) by Exercise 3.16(b), this is exactly the gamma\((\tfrac12,2)\) pdf (3.3.6).

The integral is

\begin{equation*} M_X(t) \;=\; \int_{-\infty}^{\infty} e^{tx}\,\frac{1}{\pi}\frac{1}{1+x^{2}}\,dx , \end{equation*}

and it is infinite for every \(t \ne 0\). For \(t>0\), \(e^{tx}/(1+x^{2}) \to \infty\) as \(x\to\infty\), so the integrand does not even tend to \(0\) and the integral diverges; for \(t<0\) the same happens as \(x\to-\infty\). Only \(t=0\) gives a finite value, namely \(1\). So the mgf fails to exist on any neighborhood of \(0\).

Yes, this is expected: \(f\) is the Cauchy pdf (3.3.19) with \(\theta=0\), and the Cauchy has no moments at all — \(\mathrm{E}|X|=\infty\) by (3.3.20). A finite mgf near \(0\) would force all moments to be finite (Theorem 2.3.7 generates them by differentiation), so no mgf can exist here.

(a) The factorial moment collapses the series:

\begin{equation*} \begin{aligned} \mathrm{E}X(X-1) &= \sum_{x=2}^{\infty} x(x-1)\,\frac{e^{-\lambda}\lambda^{x}}{x!} = \lambda^{2}\sum_{x=2}^{\infty}\frac{e^{-\lambda}\lambda^{x-2}}{(x-2)!} = \lambda^{2}. \end{aligned} \end{equation*}

Hence \(\mathrm{E}X^{2} = \lambda^{2} + \mathrm{E}X = \lambda^{2}+\lambda\) and \(\operatorname{Var}X = \lambda^{2}+\lambda-\lambda^{2} = \lambda\).

(b) With \(q = 1-p\) and \(P(X=x) = \binom{r+x-1}{x}p^{r}q^{x}\), \(x=0,1,\dots\), the two identities

\begin{equation*} x\binom{r+x-1}{x} = r\binom{r+x-1}{x-1},\qquad x(x-1)\binom{r+x-1}{x} = r(r+1)\binom{r+x-1}{x-2} \end{equation*}

(both immediate from \(\binom{r+x-1}{x}=\frac{(r+x-1)!}{x!\,(r-1)!}\)) reindex each sum into a negative binomial pmf with \(r\) raised. Putting \(y=x-1\), then \(y=x-2\),

\begin{equation*} \begin{aligned} \mathrm{E}X &= \frac{rq}{p}\sum_{y=0}^{\infty}\binom{(r+1)+y-1}{y}p^{r+1}q^{y} = \frac{rq}{p},\\ \mathrm{E}X(X-1) &= \frac{r(r+1)q^{2}}{p^{2}} \sum_{y=0}^{\infty}\binom{(r+2)+y-1}{y}p^{r+2}q^{y} = \frac{r(r+1)q^{2}}{p^{2}}. \end{aligned} \end{equation*}

Therefore

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \frac{r(r+1)q^{2}}{p^{2}} + \frac{rq}{p} - \frac{r^{2}q^{2}}{p^{2}}\\ &= \frac{rq^{2}+rqp}{p^{2}} = \frac{r(1-p)}{p^{2}}. \end{aligned} \end{equation*}

(c) For any \(n\) with \(\alpha+n>0\), the gamma kernel integrates to \(\Gamma(\alpha+n)\beta^{\alpha+n}\), so

\begin{equation*} \begin{aligned} \mathrm{E}X^{n} &= \frac{1}{\Gamma(\alpha)\beta^{\alpha}} \int_{0}^{\infty}x^{\alpha+n-1}e^{-x/\beta}\,dx = \frac{\Gamma(\alpha+n)}{\Gamma(\alpha)}\beta^{n}. \end{aligned} \end{equation*}

Thus \(\mathrm{E}X = \alpha\beta\), \(\mathrm{E}X^{2} = \alpha(\alpha+1)\beta^{2}\), and \(\operatorname{Var}X = \alpha(\alpha+1)\beta^{2}-\alpha^{2}\beta^{2} = \alpha\beta^{2}\).

(d) The same device with the beta function: \(\mathrm{E}X^{n} = B(\alpha+n,\beta)/B(\alpha,\beta)\), and \(B(a,b) = \Gamma(a)\Gamma(b)/\Gamma(a+b)\) gives

\begin{equation*} \mathrm{E}X = \frac{\alpha}{\alpha+\beta},\qquad \mathrm{E}X^{2} = \frac{\alpha(\alpha+1)}{(\alpha+\beta)(\alpha+\beta+1)} . \end{equation*}

Subtracting,

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \frac{\alpha\big[(\alpha+1)(\alpha+\beta)-\alpha(\alpha+\beta+1)\big]} {(\alpha+\beta)^{2}(\alpha+\beta+1)}\\ &= \frac{\alpha\beta}{(\alpha+\beta)^{2}(\alpha+\beta+1)} . \end{aligned} \end{equation*}

(e) Here \(f(x) = \frac{1}{2\sigma}e^{-|x-\mu|/\sigma}\) is symmetric about \(\mu\), so \(\mathrm{E}X = \mu\) (the integral \(\int z e^{-|z|/\sigma}dz\) converges and is odd). For the variance substitute \(z = x-\mu\) and use \(\int_{0}^{\infty}z^{2}e^{-z/\sigma}dz = 2\sigma^{3}\):

\begin{equation*} \operatorname{Var}X = \frac{1}{2\sigma}\int_{-\infty}^{\infty}z^{2}e^{-|z|/\sigma}\,dz = \frac{1}{\sigma}\int_{0}^{\infty}z^{2}e^{-z/\sigma}\,dz = 2\sigma^{2}. \end{equation*}

(a) \(f \ge 0\) on \((\alpha,\infty)\) and

\begin{equation*} \int_{\alpha}^{\infty}\frac{\beta\alpha^{\beta}}{x^{\beta+1}}\,dx = \beta\alpha^{\beta}\Big[\frac{x^{-\beta}}{-\beta}\Big]_{\alpha}^{\infty} = \alpha^{\beta}\cdot\alpha^{-\beta} = 1 . \end{equation*}

(b) For \(\beta>n\) the integral \(\int_{\alpha}^{\infty}x^{n-\beta-1}dx\) converges and

\begin{equation*} \mathrm{E}X^{n} = \beta\alpha^{\beta}\int_{\alpha}^{\infty}x^{n-\beta-1}\,dx = \beta\alpha^{\beta}\cdot\frac{\alpha^{n-\beta}}{\beta-n} = \frac{\beta\alpha^{n}}{\beta-n}. \end{equation*}

So \(\mathrm{E}X = \beta\alpha/(\beta-1)\) for \(\beta>1\), and for \(\beta>2\),

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \frac{\beta\alpha^{2}}{\beta-2}-\frac{\beta^{2}\alpha^{2}}{(\beta-1)^{2}}\\ &= \frac{\beta\alpha^{2}\big[(\beta-1)^{2}-\beta(\beta-2)\big]}{(\beta-2)(\beta-1)^{2}} = \frac{\beta\alpha^{2}}{(\beta-2)(\beta-1)^{2}}, \end{aligned} \end{equation*}

the bracket being \((\beta^{2}-2\beta+1)-(\beta^{2}-2\beta)=1\).

(c) The second moment is

\begin{equation*} \mathrm{E}X^{2} = \beta\alpha^{\beta}\int_{\alpha}^{\infty}x^{1-\beta}\,dx , \end{equation*}

and \(\int_{\alpha}^{\infty}x^{1-\beta}dx = \infty\) exactly when \(1-\beta \ge -1\), i.e. \(\beta \le 2\) (for \(\beta=2\) the integrand is \(x^{-1}\), whose integral diverges logarithmically). Since the integrand is nonnegative, \(\mathrm{E}X^{2}=\infty\), so \(\operatorname{Var}X\) does not exist.

Every \(g\) below is a differentiable, strictly monotone bijection of the support (increasing except in (e)), so Theorem 2.1.5 gives \(f_Y(y) = f_X(g^{-1}(y))\,|\tfrac{d}{dy}g^{-1}(y)|\); reversing each substitution returns \(\int f_X = 1\), and moments of \(Y\) are moments of \(X\) by Definition 2.2.1, using \(\mathrm{E}X^{n} = \Gamma(\alpha+n)\beta^{n}/\Gamma(\alpha)\) for \(n>-\alpha\) (Exercise 3.22(c)).

(a) \(g^{-1}(y)=y^{\gamma}\), \({g^{-1}}^{\prime}(y)=\gamma y^{\gamma-1}\), so for \(y>0\)

\begin{equation*} f_Y(y) = \frac{1}{\beta}e^{-y^{\gamma}/\beta}\,\gamma y^{\gamma-1} = \frac{\gamma}{\beta}y^{\gamma-1}e^{-y^{\gamma}/\beta}, \end{equation*}

the Weibull\((\gamma,\beta)\) pdf; \(u=y^{\gamma}/\beta\) returns \(\int_{0}^{\infty}e^{-u}du=1\). With \(\alpha=1\) in the gamma moment formula,

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \mathrm{E}X^{1/\gamma} = \beta^{1/\gamma}\Gamma(1+\tfrac1\gamma),\\ \operatorname{Var}Y &= \beta^{2/\gamma}\Big[\Gamma\big(1+\tfrac2\gamma\big)

• \Gamma^{2}\big(1+\tfrac1\gamma\big)\Big]. \end{aligned} \end{equation*}

(b) \(g^{-1}(y)=\beta y^{2}/2\), \({g^{-1}}^{\prime}(y)=\beta y\), so for \(y>0\)

\begin{equation*} f_Y(y) = \frac{1}{\beta}e^{-y^{2}/2}\,\beta y = y\,e^{-y^{2}/2}, \end{equation*}

free of \(\beta\), and \(\int_{0}^{\infty}ye^{-y^{2}/2}dy = [-e^{-y^{2}/2}]_{0}^{\infty}=1\). The integrand below is even, and \(\int_{-\infty}^{\infty}y^{2}e^{-y^{2}/2}dy = \sqrt{2\pi}\) is the variance of an \(\mathrm{n}(0,1)\) times \(\sqrt{2\pi}\), so

\begin{equation*} \mathrm{E}Y = \int_{0}^{\infty}y^{2}e^{-y^{2}/2}dy = \tfrac12\int_{-\infty}^{\infty}y^{2}e^{-y^{2}/2}dy = \sqrt{\tfrac{\pi}{2}}, \end{equation*}

while \(\mathrm{E}Y^{2} = \mathrm{E}(2X/\beta) = 2\) directly. Hence \(\operatorname{Var}Y = 2-\pi/2\).

(c) \(g^{-1}(y)=1/y\), \(|{g^{-1}}^{\prime}(y)| = 1/y^{2}\), so for \(y>0\)

\begin{equation*} f_Y(y) = \frac{1}{\Gamma(a)b^{a}}\Big(\frac1y\Big)^{a-1}e^{-1/(by)}\cdot\frac{1}{y^{2}} = \frac{1}{\Gamma(a)b^{a}}\Big(\frac1y\Big)^{a+1}e^{-1/(by)}, \end{equation*}

the \(\mathrm{IG}(a,b)\) pdf; \(u=1/y\) carries the integral back to the gamma\((a,b)\) pdf, so it is \(1\). Negative moments of the gamma exist for \(n<a\):

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \mathrm{E}X^{-1} = \frac{\Gamma(a-1)b^{-1}}{\Gamma(a)} = \frac{1}{b(a-1)} \quad (a>1),\\ \mathrm{E}Y^{2} &= \mathrm{E}X^{-2} = \frac{\Gamma(a-2)b^{-2}}{\Gamma(a)} = \frac{1}{b^{2}(a-1)(a-2)} \quad (a>2), \end{aligned} \end{equation*}

whence

\begin{equation*} \operatorname{Var}Y = \frac{1}{b^{2}(a-1)}\Big[\frac{1}{a-2}-\frac{1}{a-1}\Big] = \frac{1}{b^{2}(a-1)^{2}(a-2)},\qquad a>2 . \end{equation*}

(d) \(g^{-1}(y)=\beta y^{2}\), \({g^{-1}}^{\prime}(y)=2\beta y\), and \(\Gamma(\tfrac32)=\sqrt{\pi}/2\), so for \(y>0\)

\begin{equation*} f_Y(y) = \frac{(\beta y^{2})^{1/2}e^{-y^{2}}}{\Gamma(\tfrac32)\beta^{3/2}}\cdot 2\beta y = \frac{2y^{2}e^{-y^{2}}}{\Gamma(\tfrac32)} = \frac{4}{\sqrt{\pi}}\,y^{2}e^{-y^{2}}, \end{equation*}

the Maxwell pdf, again free of \(\beta\) (and it integrates to \(1\), reversing the substitution). From the gamma moment formula with \(\alpha=\tfrac32\),

\begin{equation*} \mathrm{E}Y = \beta^{-1/2}\mathrm{E}X^{1/2} = \beta^{-1/2}\frac{\Gamma(2)\beta^{1/2}}{\Gamma(\tfrac32)} = \frac{2}{\sqrt{\pi}}, \qquad \mathrm{E}Y^{2} = \frac{\mathrm{E}X}{\beta} = \frac{3}{2}, \end{equation*}

so \(\operatorname{Var}Y = \tfrac32 - 4/\pi\).

(e) \(g^{-1}(y) = e^{(\alpha-y)/\gamma}\), \(|{g^{-1}}^{\prime}(y)| = \tfrac1\gamma e^{(\alpha-y)/\gamma}\), so for \(-\infty<y<\infty\)

\begin{equation*} f_Y(y) = \frac{1}{\gamma}\,e^{(\alpha-y)/\gamma}\exp\!\big(-e^{(\alpha-y)/\gamma}\big), \end{equation*}

the Gumbel\((\alpha,\gamma)\) pdf, with antiderivative \(\exp(-e^{(\alpha-y)/\gamma})\) running from \(0\) to \(1\). Differentiating \(\Gamma(t)=\int_{0}^{\infty}x^{t-1}e^{-x}dx\) under the integral at \(t=1\) gives \(\mathrm{E}\log X = \Gamma^{\prime}(1) = -\gamma_{E}\) and \(\mathrm{E}(\log X)^{2} = \Gamma^{\prime\prime}(1) = \gamma_{E}^{2}+\pi^{2}/6\), where \(\gamma_E \approx 0.5772\) is Euler’s constant; hence \(\operatorname{Var}(\log X)=\pi^{2}/6\) and

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \alpha + \gamma\gamma_{E},\\ \operatorname{Var}Y &= \gamma^{2}\operatorname{Var}(\log X) = \frac{\gamma^{2}\pi^{2}}{6}. \end{aligned} \end{equation*}

Since \(T\) is continuous, \(P(T\ge t) = 1-F_T(t)\) and \(P(t \le T < t+\delta) = F_T(t+\delta)-F_T(t)\), so for any \(t\) with \(F_T(t)<1\)

\begin{equation*} \begin{aligned} h_T(t) &= \lim_{\delta\to 0}\frac{1}{\delta}\cdot\frac{F_T(t+\delta)-F_T(t)}{1-F_T(t)}\\ &= \frac{1}{1-F_T(t)}\lim_{\delta\to 0}\frac{F_T(t+\delta)-F_T(t)}{\delta}\\ &= \frac{F_T^{\prime}(t)}{1-F_T(t)} = \frac{f_T(t)}{1-F_T(t)}, \end{aligned} \end{equation*}

the last step by (1.6.2), whose hypothesis holds at every continuity point of \(f_T\). The second equality is the chain rule applied to \(\log(1-F_T(t))\):

\begin{equation*} -\frac{d}{dt}\log\big(1-F_T(t)\big) = -\,\frac{-f_T(t)}{1-F_T(t)} = \frac{f_T(t)}{1-F_T(t)} . \end{equation*}

In each case substitute into \(h_T = f_T/(1-F_T)\) from Exercise 3.25.

(a) \(1-F_T(t) = e^{-t/\beta}\) and \(f_T(t) = \tfrac1\beta e^{-t/\beta}\) for \(t>0\), so

\begin{equation*} h_T(t) = \frac{\tfrac1\beta e^{-t/\beta}}{e^{-t/\beta}} = \frac{1}{\beta}. \end{equation*}

(b) From Exercise 3.24(a), \(f_T(t) = \tfrac{\gamma}{\beta}t^{\gamma-1}e^{-t^{\gamma}/\beta}\) for \(t>0\), and \(\int_{t}^{\infty}f_T = e^{-t^{\gamma}/\beta}\) gives \(1-F_T(t) = e^{-t^{\gamma}/\beta}\). Hence

\begin{equation*} h_T(t) = \frac{(\gamma/\beta)t^{\gamma-1}e^{-t^{\gamma}/\beta}}{e^{-t^{\gamma}/\beta}} = \frac{\gamma}{\beta}t^{\gamma-1}. \end{equation*}

(c) Write \(u = e^{-(t-\mu)/\beta}\), so \(F_T(t) = (1+u)^{-1}\) and \(du/dt = -u/\beta\). Then

\begin{equation*} \begin{aligned} f_T(t) &= \frac{d}{dt}(1+u)^{-1} = -\frac{1}{(1+u)^{2}}\cdot\frac{-u}{\beta} = \frac{1}{\beta}\cdot\frac{u}{(1+u)^{2}},\\ 1-F_T(t) &= 1-\frac{1}{1+u} = \frac{u}{1+u}, \end{aligned} \end{equation*}

and therefore

\begin{equation*} h_T(t) = \frac{u\big/\big[\beta(1+u)^{2}\big]}{u/(1+u)} = \frac{1}{\beta}\cdot\frac{1}{1+u} = \frac{1}{\beta}F_T(t). \end{equation*}

Yes for (a) and (c); no for (b) and (d). Throughout, “unimodal with mode \(m\)” is Exercise 2.27’s condition: \(f\) is nondecreasing on \((-\infty,m]\) and nonincreasing on \([m,\infty)\). In particular a unimodal pdf is bounded, by \(f(m)\) — that is what kills (b) and (d).

(a) Every uniform\((a,b)\) pdf is unimodal, with every point of \([a,b]\) a mode. Take \(m=(a+b)/2\): for \(y<x\le m\) the values \(f(y),f(x)\in\{0,(b-a)^{-1}\}\) satisfy \(f(y)\le f(x)\), since \(f\) takes the value \(0\) only to the left of \(a\); symmetrically on \([m,\infty)\).

(b) Not all. Differentiating \(f(x)=x^{\alpha-1}e^{-x/\beta}/(\Gamma(\alpha)\beta^{\alpha})\) on \(x>0\),

\begin{equation*} f^{\prime}(x) = \frac{x^{\alpha-2}e^{-x/\beta}}{\Gamma(\alpha)\beta^{\alpha}} \Big[(\alpha-1)-\frac{x}{\beta}\Big], \end{equation*}

whose bracket is positive for \(x<\beta(\alpha-1)\) and negative for \(x>\beta(\alpha-1)\).

• (i) \(\alpha>1\): unimodal with mode \(m=\beta(\alpha-1)\).
• (ii) \(\alpha=1\): \(f(x)=\beta^{-1}e^{-x/\beta}\) is nonincreasing on \([0,\infty)\) and vanishes to the left, so it is unimodal with \(m=0\).
• (iii) \(\alpha<1\): \(f(x)\to\infty\) as \(x\to0^{+}\), so \(f\) is unbounded and no \(m\) can satisfy \(f(m)\ge f(x)\) for all \(x\). Not unimodal.

(c) All are unimodal, with mode \(m=\mu\): since

\begin{equation*} f^{\prime}(x) = -\frac{x-\mu}{\sigma^{2}}\,f(x), \end{equation*}

\(f^{\prime}>0\) on \((-\infty,\mu)\) and \(f^{\prime}<0\) on \((\mu,\infty)\).

(d) Not all; the beta\((\alpha,\beta)\) pdf is unimodal exactly when \(\alpha\ge1\) and \(\beta\ge1\). On \(0<x<1\),

\begin{equation*} f^{\prime}(x) = \frac{x^{\alpha-2}(1-x)^{\beta-2}}{B(\alpha,\beta)} \big[(\alpha-1)-(\alpha+\beta-2)x\big]. \end{equation*}

• (i) \(\alpha>1,\ \beta>1\): \(\alpha+\beta-2>0\), so the bracket changes sign once, from \(+\) to \(-\); unimodal with \(m=(\alpha-1)/(\alpha+\beta-2)\).
• (ii) \(\alpha=1,\ \beta\ge1\) (or \(\alpha\ge1,\ \beta=1\)): \(f\) is monotone on \((0,1)\) and bounded, hence unimodal with \(m=0\) (resp. \(m=1\)).
• (iii) \(\alpha<1\): \(f(x)\to\infty\) as \(x\to0^{+}\), so \(f\) is unbounded and not unimodal; symmetrically for \(\beta<1\) as \(x\to1^{-}\).

In each case pull the parameter-free factors into \(h\), the \(x\)-free factors into \(c\), and write the remaining \(x\)-and-parameter cross terms as \(\exp(\sum_i w_i(\theta)t_i(x))\) — Definition 3.4.1. The algebraic rewrites are:

\begin{equation*} \begin{aligned} \text{(a) }\sigma\text{ known: }& \frac{e^{-x^{2}/(2\sigma^{2})}}{\sqrt{2\pi}\,\sigma}\cdot e^{-\mu^{2}/(2\sigma^{2})} \exp\Big(\frac{\mu}{\sigma^{2}}\,x\Big),\\ \text{(a) }\mu\text{ known: }& \frac{1}{\sqrt{2\pi}}\cdot\frac{1}{\sigma} \exp\Big(-\frac{1}{2\sigma^{2}}(x-\mu)^{2}\Big),\\ \text{(b) }&\frac{1}{\Gamma(\alpha)\beta^{\alpha}} \exp\Big((\alpha-1)\log x - \frac{1}{\beta}x\Big),\qquad x>0,\\ \text{( c) }&\frac{1}{B(\alpha,\beta)} \exp\big((\alpha-1)\log x + (\beta-1)\log(1-x)\big),\quad 0<x<1,\\ \text{(d) }&\frac{1}{x!}\cdot e^{-\lambda}\exp\big(x\log\lambda\big), \quad x=0,1,2,\dots,\\ \text{(e) }&\binom{r+x-1}{x}\cdot p^{r}\exp\big(x\log(1-p)\big),\qquad x=0,1,2,\dots \end{aligned} \end{equation*}

Reading off \(h,c,w_i,t_i\) (a factor already free of the unknown parameter moves into \(h\) when the other parameter is known) gives the following, with \(k=1\) except in the two “both unknown” rows, where \(k=2\):

Definition 3.4.1’s hypotheses hold in every row: \(h\ge0\) and \(c\ge0\); each \(w_i\) depends only on the parameter and each \(t_i\) only on \(x\); and the support — \(\mathbb{R}\), \((0,\infty)\), \((0,1)\), or \(\{0,1,2,\dots\}\) — is the same for all parameter values, being carried by \(h\) alone.

Only one integral has to be tested in each case, since by the definition following (3.4.7),

\begin{equation*} \mathcal{H}=\Big\{\boldsymbol{\eta}:\int h(x)\exp\Big(\sum_{i=1}^{k}\eta_i t_i(x)\Big)dx<\infty\Big\}, \end{equation*}

with the integral replaced by a sum over \(\{x:h(x)>0\}\) in the discrete cases.

(a) Normal. With \(\sigma\) known, \(h(x)=e^{-x^2/(2\sigma^2)}\), \(t_1(x)=x\) and \(\eta_1=\mu/\sigma^2\); the Gaussian factor dominates \(e^{\eta_1x}\) for every real \(\eta_1\), so

\begin{equation*} \mathcal{H}=(-\infty,\infty). \end{equation*}

With \(\mu\) known, \(h(x)=1\), \(t_1(x)=(x-\mu)^2\) and \(\eta_1=-1/(2\sigma^2)\); \(\int e^{\eta_1(x-\mu)^2}dx<\infty\) if and only if \(\eta_1<0\), so \(\mathcal{H}=(-\infty,0)\). (With both unknown this is Example 3.4.6: \(\mathcal{H}=\{(\eta_1,\eta_2):\eta_1>0,\ -\infty<\eta_2<\infty\}\).)

(b) Gamma. Write \(f(x\mid\alpha,\beta)=h(x)c(\alpha,\beta)\exp\{(\alpha-1)\log x-x/\beta\}\) with \(h(x)=I_{(0,\infty)}(x)\), \(t_1(x)=\log x\), \(t_2(x)=x\), \(\eta_1=\alpha-1\), \(\eta_2=-1/\beta\). Both unknown:

\begin{equation*} \int_0^{\infty}x^{\eta_1}e^{\eta_2x}\,dx<\infty \iff \eta_1>-1\ \text{and}\ \eta_2<0, \end{equation*}

the first condition for integrability at \(0\), the second at \(\infty\); so \(\mathcal{H}=(-1,\infty)\times(-\infty,0)\). With \(\beta\) known the factor \(e^{-x/\beta}\) joins \(h\), leaving \(\mathcal{H}=\{\eta_1:\eta_1>-1\}\); with \(\alpha\) known, \(x^{\alpha-1}\) joins \(h\) and \(\mathcal{H}=\{\eta_2:\eta_2<0\}\).

(c) Beta. Here \(h(x)=I_{(0,1)}(x)\), \(t_1(x)=\log x\), \(t_2(x)=\log(1-x)\), \(\eta_1=\alpha-1\), \(\eta_2=\beta-1\), and

\begin{equation*} \int_0^1x^{\eta_1}(1-x)^{\eta_2}dx<\infty\iff \eta_1>-1,\ \eta_2>-1, \end{equation*}

so with both unknown \(\mathcal{H}=(-1,\infty)\times(-1,\infty)\). With \(\beta\) known, \((1-x)^{\beta-1}\) is absorbed into \(h\) and \(\mathcal{H}=(-1,\infty)\); symmetrically for \(\alpha\) known.

(d) Poisson. \(h(x)=1/x!\) on \(x=0,1,2,\dots\), \(t_1(x)=x\), \(\eta_1=\log\lambda\), and \(\sum_{x\ge0}e^{\eta_1x}/x!=e^{e^{\eta_1}}<\infty\) for every real \(\eta_1\). Hence \(\mathcal{H}=(-\infty,\infty)\).

(e) Negative binomial, \(r\) known. \(h(x)=\binom{r+x-1}{x}\) on \(x=0,1,2,\dots\), \(t_1(x)=x\), \(\eta_1=\log(1-p)\), and

\begin{equation*} \sum_{x\ge0}\binom{r+x-1}{x}e^{\eta_1x}=(1-e^{\eta_1})^{-r}<\infty \iff e^{\eta_1}<1, \end{equation*}

so \(\mathcal{H}=(-\infty,0)\).

(a) \(\operatorname{Var}X=np(1-p)\). From Example 3.4.1, \(w_1(p)=\log\frac{p}{1-p}\), \(t_1(x)=x\), \(c(p)=(1-p)^n\), and

\begin{equation*} \begin{aligned} \frac{d}{dp}w_1(p)&=\frac{1}{p(1-p)}, & \frac{d^2}{dp^2}w_1(p)&=\frac{2p-1}{p^2(1-p)^2},\\ \frac{d}{dp}\log c(p)&=\frac{-n}{1-p}, & \frac{d^2}{dp^2}\log c(p)&=\frac{-n}{(1-p)^2}. \end{aligned} \end{equation*}

Example 3.4.3 has already delivered \(\mathrm{E}X=np\) from (3.4.4), so (3.4.5) reads

\begin{equation*} \operatorname{Var}\Big(\frac{X}{p(1-p)}\Big) =\frac{n}{(1-p)^2}-\frac{2p-1}{p^2(1-p)^2}\,np . \end{equation*}

Multiplying through by \(p^2(1-p)^2\),

\begin{equation*} \operatorname{Var}X=np^2-np(2p-1)=np(1-p). \end{equation*}

(b) \(\mathrm{E}X=\operatorname{Var}X=\lambda\). Writing \(f(x\mid\lambda)=\frac{1}{x!}e^{-\lambda}\exp(x\log\lambda)\) identifies \(h(x)=1/x!\), \(c(\lambda)=e^{-\lambda}\), \(w_1(\lambda)=\log\lambda\), \(t_1(x)=x\), whence \(w_1^{\prime}=1/\lambda\), \(w_1^{\prime\prime}=-1/\lambda^2\), \(\log c(\lambda)=-\lambda\), \((\log c)^{\prime}=-1\), \((\log c)^{\prime\prime}=0\). Then (3.4.4) gives

\begin{equation*} \mathrm{E}\Big(\frac{X}{\lambda}\Big)=1 \quad\Longrightarrow\quad \mathrm{E}X=\lambda, \end{equation*}

and (3.4.5) gives

\begin{equation*} \operatorname{Var}\Big(\frac{X}{\lambda}\Big) =-0-\mathrm{E}\Big(-\frac{X}{\lambda^{2}}\Big)=\frac{\lambda}{\lambda^{2}}=\frac1\lambda, \end{equation*}

so \(\operatorname{Var}X=\lambda^2\cdot\lambda^{-1}=\lambda\).

Fix \(x\) and \(j\) and abbreviate the \(\boldsymbol{\theta}\)-dependent factor of the density by

\begin{equation*} g(\boldsymbol{\theta})=c(\boldsymbol{\theta})\exp\Big(\sum_{i=1}^{k}w_i(\boldsymbol{\theta})t_i(x)\Big), \qquad f(x\mid\boldsymbol{\theta})=h(x)g(\boldsymbol{\theta}), \end{equation*}

so that, by the hint,

\begin{equation*} \begin{aligned} \frac{\partial g/\partial\theta_j}{g} &=\frac{\partial}{\partial\theta_j}\log g\\ &=\underbrace{\frac{\partial}{\partial\theta_j}\log c(\boldsymbol{\theta})}_{=:L} +\underbrace{\sum_{i=1}^{k}\frac{\partial w_i(\boldsymbol{\theta})}{\partial\theta_j}t_i(x)}_{=:A(x)} . \end{aligned} \end{equation*}

Throughout, the derivative passes inside the integral: on the interior of the natural parameter space the integral in (3.4.7) converges uniformly on compact \(\boldsymbol{\eta}\)-neighbourhoods, which is exactly the hypothesis needed in Section 2.4 to interchange \(\partial/\partial\theta_j\) and \(\int\).

(a) Differentiating \(\int h(x)g(\boldsymbol{\theta})dx=1\) with respect to \(\theta_j\) and writing \(\partial g/\partial\theta_j=g\cdot(L+A(x))\),

\begin{equation*} 0=\int h(x)\frac{\partial g}{\partial\theta_j}dx =\int f(x\mid\boldsymbol{\theta})\big(L+A(x)\big)dx =L+\mathrm{E}\,A(X), \end{equation*}

since \(\int f=1\) and \(L\) does not depend on \(x\). That is,

\begin{equation*} \mathrm{E}\Big(\sum_{i=1}^{k}\frac{\partial w_i(\boldsymbol{\theta})}{\partial\theta_j}t_i(X)\Big) =-\frac{\partial}{\partial\theta_j}\log c(\boldsymbol{\theta}), \end{equation*}

which is (3.4.4).

(b) The second hint, in the form \(g^{\prime\prime}/g=(\log g)^{\prime\prime}+((\log g)^{\prime})^2\), gives

\begin{equation*} \frac{\partial^2g/\partial\theta_j^2}{g} =\frac{\partial^{2}}{\partial\theta_j^{2}}\log c(\boldsymbol{\theta}) +\sum_{i=1}^{k}\frac{\partial^{2}w_i(\boldsymbol{\theta})}{\partial\theta_j^{2}}t_i(x) +\big(L+A(x)\big)^{2}. \end{equation*}

Differentiating \(\int h(x)g(\boldsymbol{\theta})dx=1\) twice and multiplying the integrand by \(g/g\) therefore yields

\begin{equation*} \begin{aligned} 0&=\int h(x)\frac{\partial^{2}g}{\partial\theta_j^{2}}dx\\ &=\frac{\partial^{2}}{\partial\theta_j^{2}}\log c(\boldsymbol{\theta}) +\mathrm{E}\Big(\sum_{i=1}^{k}\frac{\partial^{2}w_i(\boldsymbol{\theta})}{\partial\theta_j^{2}}t_i(X)\Big) +\mathrm{E}\big(L+A(X)\big)^{2}. \end{aligned} \end{equation*}

By part (a) \(\mathrm{E}(L+A(X))=0\), so the last term is \(\operatorname{Var}(L+A(X))=\operatorname{Var}A(X)\), \(L\) being constant in \(x\). Rearranging,

\begin{equation*} \operatorname{Var}\Big(\sum_{i=1}^{k}\frac{\partial w_i(\boldsymbol{\theta})}{\partial\theta_j}t_i(X)\Big) =-\frac{\partial^{2}}{\partial\theta_j^{2}}\log c(\boldsymbol{\theta}) -\mathrm{E}\Big(\sum_{i=1}^{k}\frac{\partial^{2}w_i(\boldsymbol{\theta})}{\partial\theta_j^{2}}t_i(X)\Big), \end{equation*}

which is (3.4.5).

(a) The natural parameterization (3.4.7) is (3.4.1) with \(\boldsymbol{\theta}=\boldsymbol{\eta}\), \(c=c^{*}\) and \(w_i(\boldsymbol{\eta})=\eta_i\), so the derivatives appearing in Theorem 3.4.2 collapse:

\begin{equation*} \frac{\partial w_i(\boldsymbol{\eta})}{\partial\eta_j}=\delta_{ij}, \qquad \frac{\partial^{2}w_i(\boldsymbol{\eta})}{\partial\eta_j^{2}}=0 . \end{equation*}

The sum in (3.4.4) is then the single term \(t_j(X)\), giving \(\mathrm{E}(t_j(X))=-\frac{\partial}{\partial\eta_j}\log c^{*}(\boldsymbol{\eta})\), and in (3.4.5) the expectation term vanishes, leaving \(\operatorname{Var}(t_j(X))=-\frac{\partial^{2}}{\partial\eta_j^{2}}\log c^{*}(\boldsymbol{\eta})\).

(b) \(\mathrm{E}X=ab\) and \(\operatorname{Var}X=ab^{2}\). The gamma pdf in the form (3.4.7) has

\begin{equation*} h(x)=I_{(0,\infty)}(x),\quad t_1(x)=\log x,\quad t_2(x)=x, \quad \eta_1=a-1,\quad \eta_2=-\frac1b, \end{equation*}

and since \(\int_0^{\infty}x^{\eta_1}e^{\eta_2x}dx=\Gamma(\eta_1+1)(-\eta_2)^{-(\eta_1+1)}\) on the natural parameter space \(\eta_1>-1\), \(\eta_2<0\) (Exercise 3.29(b)),

\begin{equation*} \log c^{*}(\boldsymbol{\eta})=(\eta_1+1)\log(-\eta_2)-\log\Gamma(\eta_1+1). \end{equation*}

Using \(\partial\log(-\eta_2)/\partial\eta_2=1/\eta_2\) and \(t_2(x)=x\),

\begin{equation*} \begin{aligned} \mathrm{E}X&=-\frac{\partial}{\partial\eta_2}\log c^{*} =-\frac{\eta_1+1}{\eta_2}=-\frac{a}{-1/b}=ab,\\ \operatorname{Var}X&=-\frac{\partial^{2}}{\partial\eta_2^{2}}\log c^{*} =\frac{\eta_1+1}{\eta_2^{2}}=\frac{a}{1/b^{2}}=ab^{2}. \end{aligned} \end{equation*}

All four are curved exponential families in the sense of Definition 3.4.7: \(d=1\) while \(k=2,2,2,3\).

(a) Completing the square in the exponent,

\begin{equation*} f(x\mid\theta)=\frac{1}{\sqrt{2\pi\theta}} \exp\Big(-\frac{x^{2}}{2\theta}+x-\frac{\theta}{2}\Big), \qquad \theta>0, \end{equation*}

which is (3.4.1) with \(h(x)=e^{x}\), \(c(\theta)=(2\pi\theta)^{-1/2}e^{-\theta/2}\), \(w_1(\theta)=1/\theta\), \(t_1(x)=-x^{2}/2\) (and, in the Example 3.4.4 parameterization, \(w_2=\mu/\sigma^{2}=1\), \(t_2(x)=x\)). Taking \(\boldsymbol{\theta}=(\mu,\sigma^{2})\) as in Example 3.4.8, the parameter vector is \((\theta,\theta)\): the curve is the line

\begin{equation*} \sigma^{2}=\mu,\qquad \mu>0, \end{equation*}

i.e. the open \(45^{\circ}\) ray out of the origin in the first quadrant of the \((\mu,\sigma^{2})\) plane, a one-dimensional subset of the full space \(\mathcal{R}\times(0,\infty)\).

(b) Here \(a>0\) and \(\theta\neq0\) (else \(\sigma^{2}=0\)), and the verification is Example 3.4.4 with \(\mu=\theta\), \(\sigma^{2}=a\theta^{2}\):

\begin{equation*} w_1=\frac{1}{a\theta^{2}},\quad t_1(x)=-\frac{x^{2}}{2}, \qquad w_2=\frac{1}{a\theta},\quad t_2(x)=x . \end{equation*}

The parameter vector \((\mu,\sigma^{2})=(\theta,a\theta^{2})\) traces the parabola

\begin{equation*} \sigma^{2}=a\mu^{2},\qquad \mu\neq0, \end{equation*}

a parabola through the origin with the vertex deleted, opening upward and narrowing as \(a\) increases; \(a=1\) is Example 3.4.8.

(c) With \(\beta=1/\alpha\),

\begin{equation*} f(x\mid\alpha)=\frac{\alpha^{\alpha}}{\Gamma(\alpha)} \exp\big((\alpha-1)\log x-\alpha x\big)I_{(0,\infty)}(x), \end{equation*}

so \(h(x)=I_{(0,\infty)}(x)\), \(c(\alpha)=\alpha^{\alpha}/\Gamma(\alpha)\), \(w_1(\alpha)=\alpha-1\), \(t_1(x)=\log x\), \(w_2(\alpha)=-\alpha\), \(t_2(x)=x\). The vector \((\alpha,\beta)\) lies on

\begin{equation*} \alpha\beta=1,\qquad \alpha>0, \end{equation*}

the branch of the rectangular hyperbola \(\beta=1/\alpha\) in the open first quadrant, i.e. exactly the gamma densities with mean \(1\).

(d) Expanding \((x-\theta)^{4}=x^{4}-4\theta x^{3}+6\theta^{2}x^{2}-4\theta^{3}x+\theta^{4}\),

\begin{equation*} f(x\mid\theta)=e^{-x^{4}}\cdot Ce^{-\theta^{4}} \exp\big(4\theta x^{3}-6\theta^{2}x^{2}+4\theta^{3}x\big), \end{equation*}

an exponential family with \(k=3\): \(h(x)=e^{-x^{4}}\), \(c(\theta)=Ce^{-\theta^{4}}\) (with \(C=\big(\int e^{-u^{4}}du\big)^{-1}=2/\Gamma(1/4)\approx0.5516\)), and

\begin{equation*} \begin{aligned} w_1(\theta)&=4\theta, & t_1(x)&=x^{3},\\ w_2(\theta)&=-6\theta^{2}, & t_2(x)&=x^{2},\\ w_3(\theta)&=4\theta^{3}, & t_3(x)&=x . \end{aligned} \end{equation*}

The parameter vector \((w_1,w_2,w_3)=(4\theta,-6\theta^{2},4\theta^{3})\), \(-\infty<\theta<\infty\), is a twisted cubic in \(\mathcal{R}^{3}\): eliminating \(\theta=w_1/4\) gives

\begin{equation*} w_2=-\tfrac{3}{8}w_1^{2},\qquad w_3=\tfrac{1}{16}w_1^{3}. \end{equation*}

To sketch it: the curve lies in the downward parabolic cylinder \(w_2=-3w_1^{2}/8\) and runs along it with \(w_3\) increasing cubically, through the origin at \(\theta=0\).

In each case the underlying family is the two-parameter normal family with \(\boldsymbol{\theta}=(\mu,\sigma^{2})\), so a single free parameter confines \(\boldsymbol{\theta}\) to a curve in the half plane \(\mathcal{R}\times(0,\infty)\); \(n\) is fixed throughout.

(a) \((\mu,\sigma^{2})=(\lambda,\lambda/n)\), \(\lambda>0\), so the curve is the straight ray

\begin{equation*} \sigma^{2}=\frac{\mu}{n},\qquad \mu>0, \end{equation*}

a line of slope \(1/n\) out of the (deleted) origin: the whole ray is covered as \(\lambda\) runs over \((0,\infty)\), and increasing \(n\) flattens it toward the \(\mu\) axis. (For \(n=1\) this is Exercise 3.33(a).)

(b) \((\mu,\sigma^{2})=(p,p(1-p)/n)\), \(0<p<1\), so

\begin{equation*} \sigma^{2}=\frac{\mu(1-\mu)}{n},\qquad 0<\mu<1, \end{equation*}

the arch of a downward parabola with (deleted) zeros at \(\mu=0\) and \(\mu=1\) and maximum height \(1/(4n)\) at \(\mu=1/2\).

(c) Solve the mean for \(p\): \(\mu=r(1-p)/p\) gives \(p=r/(\mu+r)\), and \(\mu\) runs over all of \((0,\infty)\) as \(p\) runs over \((0,1)\). Substituting into the variance,

\begin{equation*} \sigma^{2}=\frac{r(1-p)}{np^{2}}=\frac{1}{np}\cdot\frac{r(1-p)}{p} =\frac{\mu}{np}=\frac{\mu(\mu+r)}{nr}, \end{equation*}

so the curve is the arc of the upward parabola

\begin{equation*} \sigma^{2}=\frac{\mu(\mu+r)}{nr},\qquad \mu>0, \end{equation*}

leaving the (deleted) origin with slope \(1/n\) and bending upward thereafter.

(a) The same ray as in Exercise 3.34(a) with \(n=1\): since \((\mu,\sigma^{2})=(e^{\theta},e^{\theta})\),

\begin{equation*} \sigma^{2}=\mu,\qquad \mu>0, \end{equation*}

the open \(45^{\circ}\) ray in the first quadrant. The map \(\theta\mapsto e^{\theta}\) is a bijection of \((-\infty,\infty)\) onto \((0,\infty)\), so this parameterization sweeps out exactly the set that \(\lambda\mapsto(\lambda,\lambda)\) does; only the coordinate used to march along the ray differs, and in both cases \(d=1<k=2\), a curved family. (For general \(n\), Exercise 3.34(a) gives the ray \(\sigma^{2}=\mu/n\), the same ray with slope \(1/n\) instead of \(1\).)

(b) With \(\mu\) fixed, \(\mathrm{E}X=\alpha\beta=\mu\) confines the parameter vector to

\begin{equation*} \{(\alpha,\beta):\alpha\beta=\mu,\ \alpha>0,\ \beta>0\} =\Big\{\Big(\alpha,\frac{\mu}{\alpha}\Big):\alpha>0\Big\}, \end{equation*}

the branch of the rectangular hyperbola \(\beta=\mu/\alpha\) lying in the open first quadrant, asymptotic to both axes and crossing the line \(\beta=\alpha\) at \((\sqrt{\mu},\sqrt{\mu})\); raising \(\mu\) pushes the branch out along the diagonal. Exercise 3.33(c) is the case \(\mu=1\).

(c) The constraint acts on each coordinate pair separately, so the parameter space is the \(n\)-fold product of the hyperbolas of part (b),

\begin{equation*} \Big\{(\alpha_1,\dots,\alpha_n,\beta_1,\dots,\beta_n)\in(0,\infty)^{2n}: \ \alpha_i\beta_i=\mu,\ i=1,\dots,n\Big\}, \end{equation*}

an \(n\)-dimensional surface in \(\mathcal{R}^{2n}\) smoothly parameterized by \((\alpha_1,\dots,\alpha_n)\in(0,\infty)^{n}\) through \(\beta_i=\mu/\alpha_i\). The joint density of independent \(X_1,\dots,X_n\) is an exponential family with \(k=2n\) terms \(\log x_i\) and \(x_i\), so \(d=n<k=2n\): a curved family.

All three curves are the same symmetric bimodal “camel” shape, shifted to be centred at \(\mu\) and stretched by \(\sigma\); the standard one is

\begin{equation*} f(x) = \frac{63}{4}\,x^{6}\left(1 - x^{2}\right),\qquad -1 < x < 1, \end{equation*}

which vanishes at \(x = 0, \pm 1\) and has its two modes where \(\frac{d}{dx}(x^{6}-x^{8}) = 2x^{5}(3 - 4x^{2}) = 0\), i.e. at \(x = \pm\frac{\sqrt{3}}{2} \approx \pm 0.866\), with common height \(f\!\left(\pm\frac{\sqrt3}{2}\right) = \frac{1701}{1024} \approx 1.661\). (Check that \(\int_{-1}^{1} f = \frac{63}{4}\cdot 2\left(\frac17 - \frac19\right) = 1\).)

The three members are

\begin{equation*} \begin{aligned} \text{(a)}\quad & f(x), && -1 < x < 1,\\ \text{(b)}\quad & f(x-3), && 2 < x < 4,\\ \text{( c)}\quad & \tfrac12 f\!\left(\tfrac{x-3}{2}\right), && -1 < x < 5, \end{aligned} \end{equation*}

with landmarks

On one set of axes: (b) is (a) slid three units to the right, shape and height unchanged; (c) sits at the same centre as (b) but is twice as wide and exactly half as tall.

By Theorem 3.5.6 we may write \(X = \sigma Z + \mu\), where \(Z\) has pdf \(f\); since \(\sigma > 0\),

\begin{equation*} P(X \ge \mu) = P(\sigma Z + \mu \ge \mu) = P(Z \ge 0) = \int_{0}^{\infty} f(z)\,dz . \end{equation*}

Substituting \(z = -u\) and using the symmetry \(f(-u) = f(u)\),

\begin{equation*} \int_{0}^{\infty} f(z)\,dz = \int_{-\infty}^{0} f(-u)\,du = \int_{-\infty}^{0} f(u)\,du = P(Z \le 0). \end{equation*}

These two integrals are equal and sum to \(\int_{-\infty}^{\infty} f = 1\), so each equals \(\tfrac12\); that is,

\begin{equation*} P(X \ge \mu) = P(X \le \mu) = \tfrac12 , \end{equation*}

which is precisely the statement that \(\mu\) is a median of \(X\).

By Theorem 3.5.6, \(X\) has pdf \((1/\sigma) f\big((x-\mu)/\sigma\big)\) if and only if \(X = \sigma Z + \mu\) for a random variable \(Z\) with pdf \(f(z)\). Since \(\sigma > 0\), the map \(t \mapsto \sigma t + \mu\) is strictly increasing, so the events \(\{X > x_\alpha\}\) and \(\{Z > z_\alpha\}\) coincide:

\begin{equation*} \begin{aligned} P(X > x_{\alpha}) &= P\left(\sigma Z + \mu > \sigma z_{\alpha} + \mu\right)\\ &= P\left(Z > z_{\alpha}\right) = \alpha . \end{aligned} \end{equation*}

Method (2): substitute \(z = (x-\mu)/\sigma\) in the integral directly, with \(dx = \sigma\,dz\) and the lower limit \(\sigma z_{\alpha}+\mu\) going to \(z_{\alpha}\):

\begin{equation*} \begin{aligned} P(X > x_{\alpha}) &= \int_{\sigma z_{\alpha} + \mu}^{\infty} \frac{1}{\sigma} f\!\left(\frac{x-\mu}{\sigma}\right) dx\\ &= \int_{z_{\alpha}}^{\infty} f(z)\,dz = \alpha . \end{aligned} \end{equation*}

Write the family in standard form: with \(f(z) = \dfrac{1}{\pi\left(1+z^{2}\right)}\), the standard Cauchy pdf (3.3.19) with \(\theta = 0\), we have \(f(x|\mu,\sigma) = (1/\sigma) f\big((x-\mu)/\sigma\big)\), so by Theorem 3.5.6 \(X = \sigma Z + \mu\) with \(Z\) standard Cauchy.

(a) \(f\) is symmetric about \(0\), since \(f(-z) = f(z)\); hence \(\mu\) is the median by Exercise 3.37.

(b) For the standard variable, using \(\frac{d}{dz}\arctan z = (1+z^{2})^{-1}\) and \(\arctan(\pm\infty) = \pm\pi/2\),

\begin{equation*} \begin{aligned} P(Z \ge 1) &= \frac{1}{\pi}\int_{1}^{\infty} \frac{dz}{1+z^{2}} = \frac{1}{\pi}\Big[\arctan z\Big]_{1}^{\infty}\\ &= \frac{1}{\pi}\left(\frac{\pi}{2} - \frac{\pi}{4}\right) = \frac14 , \end{aligned} \end{equation*}

so \(z_{1/4} = 1\) in the notation of Exercise 3.38. That exercise, applied with \(\alpha = \tfrac14\), gives \(x_{1/4} = \sigma \cdot 1 + \mu = \mu + \sigma\) and

\begin{equation*} P(X \ge \mu + \sigma) = P(Z \ge 1) = \tfrac14 . \end{equation*}

By the symmetry of \(f\) about \(0\), \(P(Z \le -1) = P(Z \ge 1) = \tfrac14\), and therefore

\begin{equation*} P(X \le \mu - \sigma) = P(\sigma Z + \mu \le \mu - \sigma) = P(Z \le -1) = \tfrac14 . \end{equation*}

Take

\begin{equation*} f^{*}(x) = \sigma f(\sigma x + \mu), \qquad -\infty < x < \infty , \end{equation*}

and let the family be \(\left\{\,(1/\tau) f^{*}\big((x-\nu)/\tau\big) : -\infty < \nu < \infty,\ \tau > 0 \,\right\}\).

First, \(f^{*}\) is a pdf and generates the same family as \(f\): writing it as

\begin{equation*} f^{*}(x) = \frac{1}{(1/\sigma)}\, f\!\left(\frac{x - (-\mu/\sigma)}{1/\sigma}\right), \end{equation*}

it is the member of the location-scale family built from \(f\) with location \(-\mu/\sigma\) and scale \(1/\sigma\), hence a pdf by Theorem 3.5.1. The two families coincide, since composing affine maps gives

\begin{equation*} \frac{1}{\tau} f^{*}\!\left(\frac{x-\nu}{\tau}\right) = \frac{1}{\tau/\sigma}\, f\!\left(\frac{x - (\nu - \mu\tau/\sigma)}{\tau/\sigma}\right), \end{equation*}

and \((\nu,\tau) \mapsto (\nu - \mu\tau/\sigma,\ \tau/\sigma)\) maps \(\mathbb{R}\times(0,\infty)\) onto itself; \(f\) itself is the member with \(\nu = \mu\), \(\tau = \sigma\).

Second, \(f^{*}\) has the required moments. If \(X\) has pdf \(f\), then by Theorem 3.5.6 a random variable with pdf \(f^{*}\) is \(Z = (X-\mu)/\sigma\), so

\begin{equation*} \mathrm{E}Z = \frac{\mathrm{E}X - \mu}{\sigma} = 0, \qquad \operatorname{Var} Z = \frac{\operatorname{Var} X}{\sigma^{2}} = 1 , \end{equation*}

both existing because \(\mathrm{E}X = \mu\) and \(\operatorname{Var}X = \sigma^{2}\) were assumed finite. By Theorem 3.5.7 the general member \((1/\tau) f^{*}\big((x-\nu)/\tau\big)\) then has mean \(\nu\) and variance \(\tau^{2}\).

In both parts the cdf is a strictly decreasing function of the parameter at every \(x\) where the density has mass, which is more than the definition requires.

(a) Let \(\mu_{1} > \mu_{2}\) and let \(\Phi\) be the standard normal cdf. The \(\mathrm{n}(\mu,\sigma^{2})\) family is the location-scale family with standard pdf \(\mathrm{n}(0,1)\), so by the standardization \(P(X \le x) = P\big(Z \le (x-\mu)/\sigma\big)\) closing Section 3.5,

\begin{equation*} F(x|\mu_{i}) = \Phi\!\left(\frac{x-\mu_{i}}{\sigma}\right), \qquad i = 1,2 . \end{equation*}

Since \(\sigma > 0\) we have \(\frac{x-\mu_{1}}{\sigma} < \frac{x-\mu_{2}}{\sigma}\) for every \(x\), and \(\Phi\) is strictly increasing on \(\mathbb{R}\) (its density \(\phi\) is everywhere positive), so

\begin{equation*} F(x|\mu_{1}) < F(x|\mu_{2}) \quad \text{for every } x \in \mathbb{R}. \end{equation*}

Hence \(F(x|\mu_{1})\) is stochastically greater than \(F(x|\mu_{2})\).

(b) Let \(\beta_{1} > \beta_{2} > 0\) and fix \(\alpha > 0\). As in the derivation of (3.3.6) from (3.3.5), \(X \sim \mathrm{gamma}(\alpha,\beta)\) is the same as \(X = \beta T\) with \(T \sim \mathrm{gamma}(\alpha,1)\), so for every \(x\)

\begin{equation*} F(x|\beta) = P(\beta T \le x) = F_{T}\!\left(\frac{x}{\beta}\right). \end{equation*}

For \(x \le 0\) both sides are \(0\), since the gamma sample space is \((0,\infty)\). For \(x > 0\) we have \(x/\beta_{1} < x/\beta_{2}\), and \(F_{T}\) is strictly increasing on \((0,\infty)\) because the gamma\((\alpha,1)\) density is positive there; hence

\begin{equation*} F(x|\beta_{1}) = F_{T}\!\left(\frac{x}{\beta_{1}}\right) < F_{T}\!\left(\frac{x}{\beta_{2}}\right) = F(x|\beta_{2}). \end{equation*}

So \(F(x|\beta_{1}) \le F(x|\beta_{2})\) for all \(x\) with strict inequality on \((0,\infty)\), which is the required stochastic ordering.

Both parts are the monotonicity of the standard cdf \(F\) applied to a shifted or rescaled argument; the only work is producing the one point of strict inequality that Exercise 1.49 demands.

(a) Let the standard pdf be \(f\) with cdf \(F\), so that the location family of Definition 3.5.2 has cdfs \(F(x|\mu) = F(x-\mu)\). Take \(\mu_{1} > \mu_{2}\) and put \(d = \mu_{1} - \mu_{2} > 0\). Since \(x - \mu_{1} < x - \mu_{2}\) and \(F\) is nondecreasing,

\begin{equation*} F(x|\mu_{1}) = F(x - \mu_{1}) \le F(x - \mu_{2}) = F(x|\mu_{2}) \qquad \text{for all } x . \end{equation*}

For strictness at some point, suppose instead that \(F(t) = F(t+d)\) for every \(t \in \mathbb{R}\). Iterating gives \(F(t) = F(t + nd)\) for every \(n \ge 1\), and letting \(n \to \infty\),

\begin{equation*} F(t) = \lim_{n\to\infty} F(t+nd) = 1 \qquad \text{for every } t , \end{equation*}

contradicting \(\lim_{t \to -\infty} F(t) = 0\). So there is a \(t\) with \(F(t) < F(t+d)\), and at \(x = t + \mu_{1}\),

\begin{equation*} F(x|\mu_{1}) = F(t) < F(t+d) = F(x|\mu_{2}). \end{equation*}

Hence \(F(x|\mu_{1})\) is stochastically greater than \(F(x|\mu_{2})\).

(b) Now the standard pdf \(f\) vanishes off \([0,\infty)\), the scale family (Definition 3.5.4) has pdfs \((1/\sigma) f(x/\sigma)\), and the corresponding cdfs are \(F(x|\sigma) = F(x/\sigma)\), with \(F(x) = 0\) for \(x \le 0\). Take \(\sigma_{1} > \sigma_{2} > 0\) and put \(r = \sigma_{1}/\sigma_{2} > 1\). For \(x \le 0\), \(F(x|\sigma_{1}) = 0 = F(x|\sigma_{2})\) (this is where the sample space \([0,\infty)\) is used: for \(x < 0\) the inequality \(x/\sigma_{1} > x/\sigma_{2}\) runs the wrong way, and only the vanishing of \(F\) there saves it). For \(x > 0\),

\begin{equation*} \frac{x}{\sigma_{1}} < \frac{x}{\sigma_{2}} \quad\Longrightarrow\quad F(x|\sigma_{1}) = F\!\left(\frac{x}{\sigma_{1}}\right) \le F\!\left(\frac{x}{\sigma_{2}}\right) = F(x|\sigma_{2}). \end{equation*}

For strictness, suppose \(F(t) = F(rt)\) for every \(t > 0\). Iterating, \(F(t) = F(r^{n} t)\) for every \(n \ge 1\), and \(r^{n} t \to \infty\), so \(F(t) = 1\) for every \(t > 0\); but \(F\) is continuous with \(F(0) = 0\), so \(F(t) \to 0\) as \(t \downarrow 0\), a contradiction. Hence some \(t > 0\) has \(F(t) < F(rt)\), and at \(x = \sigma_{1} t > 0\),

\begin{equation*} F(x|\sigma_{1}) = F(t) < F(rt) = F\!\left(\frac{\sigma_{1}t}{\sigma_{2}}\right) = F(x|\sigma_{2}), \end{equation*}

which completes the stochastic ordering.

Both statements are the definition read backwards; the only content is that \(x\mapsto 1/x\) and \(\theta\mapsto 1/\theta\) each reverse order.

(a) Since the sample space is \((0,\infty)\), for \(y>0\) the event \(\{1/X\le y\}\) is the event \(\{X\ge 1/y\}\), so

\begin{equation*} F_Y(y|\theta)=P\!\left(X\ge \tfrac1y\,\Big|\,\theta\right)=1-F_X\!\left(\tfrac1y\,\Big|\,\theta\right), \end{equation*}

taking \(F_X(\cdot|\theta)\) continuous, as in Exercises 3.41–3.42. Fix \(\theta_1>\theta_2\). Stochastic increase gives

\begin{equation*} F_X(x|\theta_1)\le F_X(x|\theta_2)\quad\text{for all }x, \end{equation*}

with strict inequality at some \(x_0>0\). Applying this at \(x=1/y\) and negating,

\begin{equation*} \begin{aligned} F_Y(y|\theta_2)&=1-F_X\!\left(\tfrac1y\big|\theta_2\right)\\ &\le 1-F_X\!\left(\tfrac1y\big|\theta_1\right)=F_Y(y|\theta_1) \end{aligned} \end{equation*}

for every \(y>0\), and strictly at \(y_0=1/x_0\). Thus \(F_Y(\cdot|\theta_2)\) is stochastically greater than \(F_Y(\cdot|\theta_1)\) whenever \(\theta_1>\theta_2\), which is exactly stochastic decrease in \(\theta\).

(b) Write \(G(x|\theta)=F_X\!\left(x\big|\tfrac1\theta\right)\) for \(\theta>0\). If \(\theta_1>\theta_2>0\) then \(\tfrac1{\theta_2}>\tfrac1{\theta_1}\), so stochastic increase of \(F_X\) in its parameter, applied to the pair \(\tfrac1{\theta_2}>\tfrac1{\theta_1}\), gives

\begin{equation*} G(x|\theta_2)=F_X\!\left(x\big|\tfrac1{\theta_2}\right)\le F_X\!\left(x\big|\tfrac1{\theta_1}\right)=G(x|\theta_1) \end{equation*}

for all \(x\), with strict inequality for some \(x\). Hence \(G(\cdot|\theta_2)\) is stochastically greater than \(G(\cdot|\theta_1)\), i.e. \(G\) is stochastically decreasing in \(\theta\).

Both bounds are Chebychev’s Inequality (Theorem 3.6.1), \(P(g(X)\ge r)\le \mathrm{E}g(X)/r\), applied to two different nonnegative \(g\).

Take \(g(x)=x^2\) and \(r=b^2>0\). Since \(\{|X|\ge b\}=\{X^2\ge b^2\}\),

\begin{equation*} P(|X|\ge b)=P(X^2\ge b^2)\le \frac{\mathrm{E}X^2}{b^2}. \end{equation*}

Take instead \(g(x)=|x|\) and \(r=b\) (this is Markov’s Inequality):

\begin{equation*} P(|X|\ge b)\le \frac{\mathrm{E}|X|}{b}. \end{equation*}

Both \(g\) are nonnegative and both expectations are assumed finite, so the hypotheses of Theorem 3.6.1 hold in each case.

For \(f(x)=e^{-x}\), \(x>0\) (exponential with mean \(1\)), \(X>0\) so \(|X|=X\), and \(\mathrm{E}X=1\), \(\mathrm{E}X^2=2\). The two bounds are \(2/b^2\) and \(1/b\), and

\begin{equation*} \frac{2}{b^2}<\frac1b\iff b>2 . \end{equation*}

Hence:

(i) \(b=3\): \(\ \mathrm{E}X^2/b^2=\tfrac29\approx .222\) versus \(\mathrm{E}|X|/b=\tfrac13\approx .333\), so the second-moment bound is better (true value \(e^{-3}\approx .0498\)).

(ii) \(b=\sqrt2\): \(\ \mathrm{E}X^2/b^2=1\) (vacuous) versus \(\mathrm{E}|X|/b=1/\sqrt2\approx .707\), so the Markov bound is better (true value \(e^{-\sqrt2}\approx .243\)).

(a) For \(0<t<h\) the map \(u\mapsto e^{tu}\) is strictly increasing, so \(\{X\ge a\}=\{e^{tX}\ge e^{at}\}\). Apply Chebychev’s Inequality (Theorem 3.6.1) with the nonnegative function \(g(x)=e^{tx}\) and \(r=e^{at}>0\); \(\mathrm{E}g(X)=M_X(t)\) is finite because \(|t|<h\):

\begin{equation*} P(X\ge a)=P\!\left(e^{tX}\ge e^{at}\right)\le \frac{\mathrm{E}e^{tX}}{e^{at}}=e^{-at}M_X(t). \end{equation*}

(b) For \(-h<t<0\) the map \(u\mapsto e^{tu}\) is strictly decreasing, so now \(\{X\le a\}=\{e^{tX}\ge e^{at}\}\), and the identical application of Theorem 3.6.1 gives

\begin{equation*} P(X\le a)=P\!\left(e^{tX}\ge e^{at}\right)\le e^{-at}M_X(t). \end{equation*}

(c) It suffices that, for each \(t\ge 0\),

\begin{equation*} h(t,x)\ge 0\ \ \text{for all }x,\qquad h(t,x)\ge 1\ \ \text{for all }x\ge 0 . \end{equation*}

Indeed, with \(I\) the indicator function (Definition 3.4.5), these two conditions give the pointwise bound \(h(t,x)\ge I_{[0,\infty)}(x)\) for every \(x\), whence

\begin{equation*} \mathrm{E}h(t,X)\ \ge\ \mathrm{E}\,I_{[0,\infty)}(X)\ =\ P(X\ge 0). \end{equation*}

In part (a) with \(a=0\) and \(h(t,x)=e^{tx}\): \(e^{tx}>0\) always, and \(e^{tx}\ge 1\) for \(x\ge 0\) when \(t\ge 0\).

\begin{equation*} P(|X-\mu_X|\ge k\sigma_X)= \begin{cases} \left(1-\dfrac{k}{\sqrt3}\right)^{+} & \text{uniform}(0,1),\\[2ex] e^{-(1+k)}+\left(1-e^{-(1-k)}\right)^{+} & \text{exponential}(\lambda), \end{cases} \end{equation*}

against Chebychev’s bound \(1/k^2\) (Example 3.6.2). Here \(u^{+}=\max(u,0)\).

Uniform\((0,1)\): \(\mu=\tfrac12\), \(\sigma^2=\tfrac1{12}\), so \(k\sigma=k/(2\sqrt3)\). The event \(|X-\tfrac12|<k\sigma\) is the interval \(\left(\tfrac12-\tfrac{k}{2\sqrt3},\ \tfrac12+\tfrac{k}{2\sqrt3}\right)\), whose intersection with \((0,1)\) has length \(\min\!\left(k/\sqrt3,\,1\right)\). Hence

\begin{equation*} P\!\left(|X-\tfrac12|\ge k\sigma\right)=1-\frac{k}{\sqrt3}\ \ (0\le k<\sqrt3), \end{equation*}

and it is \(0\) for \(k\ge\sqrt3\approx 1.732\).

Exponential\((\lambda)\) (pdf \(\lambda^{-1}e^{-x/\lambda}\) on \((0,\infty)\)): \(\mu=\lambda\), \(\sigma=\lambda\), so the deviation event is \(\{X\ge \lambda(1+k)\}\cup\{X\le \lambda(1-k)\}\) and

\begin{equation*} \begin{aligned} P(X\ge\lambda(1+k))&=e^{-(1+k)},\\ P(X\le\lambda(1-k))&=1-e^{-(1-k)}\ \ (k<1), \end{aligned} \end{equation*}

the second event being empty for \(k\ge 1\); \(\lambda\) cancels, being a scale parameter (Definition 3.5.4).

Numerically:

Both exact probabilities sit far below \(1/k^2\), as Section 3.6.1 warns: the uniform is already \(0\) past \(k=\sqrt3\), and the exponential tail decays like \(e^{-k}\) against the bound’s \(k^{-2}\).

The whole proof is the antiderivative

\begin{equation*} \frac{d}{dx}\left(-\frac{e^{-x^2/2}}{x}\right)=\left(1+\frac{1}{x^2}\right)e^{-x^2/2},\qquad x>0, \end{equation*}

which integrates over \((t,\infty)\) to

\begin{equation*} \int_t^\infty\left(1+\frac{1}{x^2}\right)e^{-x^2/2}\,dx=\frac{e^{-t^2/2}}{t}. \end{equation*}

On that range \(x>t\), so \(1+x^{-2}\le 1+t^{-2}\) and therefore

\begin{equation*} \begin{aligned} \frac{e^{-t^2/2}}{t}&=\int_t^\infty\left(1+\frac{1}{x^2}\right)e^{-x^2/2}\,dx\\ &\le\left(1+\frac{1}{t^2}\right)\int_t^\infty e^{-x^2/2}\,dx . \end{aligned} \end{equation*}

Dividing by \(1+t^{-2}=(1+t^2)/t^2\),

\begin{equation*} \int_t^\infty e^{-x^2/2}\,dx\ \ge\ \frac{t^2}{1+t^2}\cdot\frac{e^{-t^2/2}}{t}=\frac{t}{1+t^2}e^{-t^2/2}. \end{equation*}

Multiplying by \(1/\sqrt{2\pi}\) gives \(P(Z\ge t)\ge \frac{1}{\sqrt{2\pi}}\frac{t}{1+t^2}e^{-t^2/2}\), and by symmetry \(P(|Z|\ge t)=2P(Z\ge t)\), so

\begin{equation*} P(|Z|\ge t)\ \ge\ \frac{2}{\sqrt{2\pi}}\,\frac{t}{1+t^2}e^{-t^2/2}=\sqrt{\frac{2}{\pi}}\,\frac{t}{1+t^2}e^{-t^2/2}. \end{equation*}

In each case form the ratio \(P(X=x+1)/P(X=x)\) and cancel factorials.

Binomial \((n,p)\), \(x=0,1,\dots,n-1\):

\begin{equation*} \begin{aligned} \frac{P(X=x+1)}{P(X=x)} &=\frac{\binom{n}{x+1}p^{x+1}(1-p)^{n-x-1}}{\binom{n}{x}p^{x}(1-p)^{n-x}}\\ &=\frac{n-x}{x+1}\cdot\frac{p}{1-p}, \end{aligned} \end{equation*}

using \(\binom{n}{x+1}/\binom{n}{x}=(n-x)/(x+1)\). Hence

\begin{equation*} P(X=x+1)=\frac{(n-x)\,p}{(x+1)(1-p)}\,P(X=x),\qquad P(X=0)=(1-p)^n . \end{equation*}

Negative binomial \((r,p)\) in the form (3.2.10), \(P(X=x)=\binom{r+x-1}{x}p^r(1-p)^x\), \(x=0,1,\dots\):

\begin{equation*} \frac{\binom{r+x}{x+1}}{\binom{r+x-1}{x}}=\frac{(r+x)!\,x!}{(x+1)!\,(r+x-1)!}=\frac{r+x}{x+1}, \end{equation*}

so

\begin{equation*} P(X=x+1)=\frac{(r+x)(1-p)}{x+1}\,P(X=x),\qquad P(X=0)=p^r . \end{equation*}

Hypergeometric \((N,M,K)\), \(P(X=x)=\binom{M}{x}\binom{N-M}{K-x}\big/\binom{N}{K}\):

\begin{equation*} \begin{aligned} \frac{P(X=x+1)}{P(X=x)}&=\frac{\binom{M}{x+1}}{\binom{M}{x}}\cdot\frac{\binom{N-M}{K-x-1}}{\binom{N-M}{K-x}}\\ &=\frac{M-x}{x+1}\cdot\frac{K-x}{N-M-K+x+1}, \end{aligned} \end{equation*}

the second factor from \(\binom{m}{k-1}/\binom{m}{k}=k/(m-k+1)\) with \(m=N-M\), \(k=K-x\). Hence

\begin{equation*} P(X=x+1)=\frac{(M-x)(K-x)}{(x+1)(N-M-K+x+1)}\,P(X=x), \end{equation*}

for \(\max(0,K-N+M)\le x<\min(M,K)\), started at \(P(X=0)=\binom{N-M}{K}\big/\binom{N}{K}\).

Each identity is integration by parts against a pdf, exactly as in the proof of Stein’s Lemma (Lemma 3.6.5); the only new ingredient is the right exact derivative.

(a) With \(f(x|\alpha,\beta)=\frac{1}{\Gamma(\alpha)\beta^{\alpha}}x^{\alpha-1}e^{-x/\beta}\) on \((0,\infty)\),

\begin{equation*} \frac{d}{dx}\big[x\,f(x|\alpha,\beta)\big] =\frac{\alpha x^{\alpha-1}-\beta^{-1}x^{\alpha}}{\Gamma(\alpha)\beta^{\alpha}}e^{-x/\beta} =-\frac{1}{\beta}(x-\alpha\beta)f(x|\alpha,\beta), \end{equation*}

i.e. \((x-\alpha\beta)f(x)=-\beta\,\frac{d}{dx}[xf(x)]\). Therefore

\begin{equation*} \begin{aligned} \mathrm{E}\big(g(X)(X-\alpha\beta)\big)&=-\beta\int_0^\infty g(x)\,\frac{d}{dx}\big[xf(x)\big]\,dx\\ &=-\beta\Big[g(x)\,xf(x)\Big]_0^\infty+\beta\int_0^\infty g^{\prime}(x)\,xf(x)\,dx\\ &=\beta\,\mathrm{E}\big(Xg^{\prime}(X)\big), \end{aligned} \end{equation*}

the boundary term vanishing under the appropriate condition on \(g\): \(g(x)\,x\,f(x|\alpha,\beta)\to 0\) at \(0\) and at \(\infty\), with \(\mathrm{E}|Xg^{\prime}(X)|<\infty\) (the analog of \(\mathrm{E}|g^{\prime}(X)|<\infty\) in Lemma 3.6.5).

(b) With \(f(x|\alpha,\beta)=\frac{1}{B(\alpha,\beta)}x^{\alpha-1}(1-x)^{\beta-1}\) on \((0,1)\), put

\begin{equation*} h(x)=(1-x)f(x|\alpha,\beta)=\frac{x^{\alpha-1}(1-x)^{\beta}}{B(\alpha,\beta)} . \end{equation*}

Differentiating,

\begin{equation*} \begin{aligned} h^{\prime}(x)&=\frac{(\alpha-1)x^{\alpha-2}(1-x)^{\beta}-\beta x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}\\ &=-f(x|\alpha,\beta)\left[\beta-(\alpha-1)\frac{1-x}{x}\right], \end{aligned} \end{equation*}

since \(x^{\alpha-2}(1-x)^{\beta}=x^{\alpha-1}(1-x)^{\beta-1}\cdot\frac{1-x}{x}\). Hence

\begin{equation*} \begin{aligned} \mathrm{E}\left[g(X)\left(\beta-(\alpha-1)\tfrac{1-X}{X}\right)\right] &=-\int_0^1 g(x)h^{\prime}(x)\,dx\\ &=-\Big[g(x)h(x)\Big]_0^1+\int_0^1 g^{\prime}(x)h(x)\,dx\\ &=\mathrm{E}\big((1-X)g^{\prime}(X)\big), \end{aligned} \end{equation*}

the boundary term vanishing under the condition \(g(x)(1-x)f(x|\alpha,\beta)\to 0\) at \(0\) and at \(1\), with \(\mathrm{E}|(1-X)g^{\prime}(X)|<\infty\).

The weight \(x/(r+x-1)\) is absorbed by the binomial coefficient, which lowers both of its arguments by one:

\begin{equation*} \frac{x}{r+x-1}\binom{r+x-1}{x} = \frac{x\,(r+x-2)!}{x!\,(r-1)!} = \binom{r+x-2}{x-1}, \qquad x \ge 1 . \end{equation*}

Hence, using the pmf (3.2.10) and substituting \(y = x-1\),

\begin{equation*} \begin{aligned} \mathrm{E}\!\left(\frac{X}{r+X-1}g(X-1)\right) &= \sum_{x=1}^{\infty} \binom{r+x-2}{x-1} g(x-1)\, p^{r}(1-p)^{x} \\ &= (1-p)\sum_{y=0}^{\infty} \binom{r+y-1}{y} g(y)\, p^{r}(1-p)^{y} \\ &= \mathrm{E}\bigl((1-p)g(X)\bigr). \end{aligned} \end{equation*}

The omitted \(x=0\) term contributes nothing: its weight \(x/(r+x-1)\) is \(0\) when \(r > 1\), and when \(r = 1\) it reads \(0/0\) and is taken to be \(0\), which is meaningful only because \(g(-1)\) is finite by hypothesis. The other hypothesis, \(-\infty < \mathrm{E}\,g(X) < \infty\), makes the last sum finite, and the chain above is a term-by-term re-indexing rather than a rearrangement.

\(\pi/4\), \(1/2\), and \(1\). The square \(S = (-1,1)\times(-1,1)\) has area \(4\), so for any \(A \subset S\) the uniform density \(f = \tfrac14\) gives \(P((X,Y) \in A) = \tfrac14\,\mathrm{area}(A)\).

(a) The disk \(\{x^2+y^2<1\}\) is contained in \(S\) and has area \(\pi\), so

\begin{equation*} P(X^2+Y^2<1) \;=\; \tfrac14 \pi \;=\; \frac{\pi}{4} \approx 0.785 . \end{equation*}

(b) The map \((x,y)\mapsto(-x,-y)\) carries \(S\) onto itself and preserves area, while it carries \(\{2x-y>0\}\) onto \(\{2x-y<0\}\). Hence those two events have equal probability, and since the line \(2x=y\) has area \(0\),

\begin{equation*} P(2X-Y>0) \;=\; \tfrac12\bigl(1 - P(2X=Y)\bigr) \;=\; \frac12 . \end{equation*}

(c) On \(S\) we have \(|x+y| \le |x|+|y| < 2\) at every interior point, so the event contains all of \(S\) except a null set:

\begin{equation*} P(|X+Y|<2) \;=\; 1 . \end{equation*}

All four follow from linearity and positivity of the integral, exactly as in the univariate Theorem 2.2.5, whose hypothesis that \(\mathrm{E}(g_1(X,Y))\) and \(\mathrm{E}(g_2(X,Y))\) exist is what licenses the splitting in (a). Only the continuous case is written; the discrete case replaces \(\iint \cdot\, f(x,y)\,dx\,dy\) by \(\sum_{(x,y)}\cdot\, f(x,y)\).

(a) Since the integrals of \(|g_1|f\) and \(|g_2|f\) are finite, the integral of the sum splits:

\begin{equation*} \begin{aligned} \mathrm{E}(a g_1 + b g_2 + c) &= \iint \bigl(a g_1(x,y) + b g_2(x,y) + c\bigr) f(x,y)\,dx\,dy\\ &= a\!\iint\! g_1 f\,dx\,dy + b\!\iint\! g_2 f\,dx\,dy + c\!\iint\! f\,dx\,dy\\ &= a\mathrm{E}(g_1(X,Y)) + b\mathrm{E}(g_2(X,Y)) + c, \end{aligned} \end{equation*}

the last step using \(\iint f(x,y)\,dx\,dy = 1\).

(b) If \(g_1 \ge 0\) then the integrand \(g_1(x,y) f(x,y) \ge 0\) everywhere, since \(f \ge 0\); an integral of a nonnegative function is nonnegative, so \(\mathrm{E}(g_1(X,Y)) \ge 0\).

(c) Apply (b) to \(g_1 - g_2 \ge 0\) and then (a) with \(a=1\), \(b=-1\), \(c=0\):

\begin{equation*} \begin{aligned} 0 &\;\le\; \mathrm{E}(g_1(X,Y) - g_2(X,Y))\\ &\;=\; \mathrm{E}(g_1(X,Y)) - \mathrm{E}(g_2(X,Y)). \end{aligned} \end{equation*}

(d) A constant function has expectation equal to that constant, by (a) with \(a=b=0\). So apply (c) to the pair \((g_1, a)\) for the lower bound and to the pair \((b, g_1)\) for the upper:

\begin{equation*} a = \mathrm{E}(a) \;\le\; \mathrm{E}(g_1(X,Y)) \;\le\; \mathrm{E}(b) = b . \end{equation*}

Count sample points. Here \(S = \{(i,j) : 1 \le i,j \le 6\}\) with all 36 points equally likely, and Definition 4.1.1 makes \((X,Y)\) the function \(S \to \Re^2\) given by

\begin{equation*} \begin{aligned} X(i,j) &= \begin{cases} 0 & i \in \{1,2\},\\ 1 & i \in \{3,4,5,6\},\end{cases}\\[4pt] Y(i,j) &= \begin{cases} 0 & j \in \{1,3,5\},\\ 1 & j \in \{2,4,6\}.\end{cases} \end{aligned} \end{equation*}

Since \(X\) depends only on \(i\) and \(Y\) only on \(j\), the event \(\{X=x, Y=y\}\) is the product set \(\{i : X = x\}\times\{j : Y = y\}\), whose cardinality is the product of the two cardinalities. The first factor has \(2\) elements when \(x=0\) and \(4\) when \(x=1\); the second has \(3\) elements for either value of \(y\). By Definition 4.1.3, \(f(x,y) = P(X=x,Y=y) = \#\{X=x,Y=y\}/36\):

\begin{equation*} \begin{aligned} f(0,0) = f(0,1) &= \frac{2\cdot 3}{36} = \frac16,\\ f(1,0) = f(1,1) &= \frac{4\cdot 3}{36} = \frac13. \end{aligned} \end{equation*}

No sample point maps to any other \((x,y)\), so \(f(x,y)=0\) elsewhere; these are exactly the values listed in Example 4.1.5.

\(C = 1/4\), \(f_X(x) = (x+1)/4\) on \((0,2)\), and \(f_Z(z) = 9/(8z^2)\) on \((1,9)\).

(a) Integrate over the rectangle:

\begin{equation*} \begin{aligned} 1 &= C\int_0^2\!\!\int_0^1 (x+2y)\,dy\,dx = C\int_0^2 \bigl[xy + y^2\bigr]_{y=0}^{1}\,dx\\ &= C\int_0^2 (x+1)\,dx = C\Bigl[\tfrac{x^2}{2}+x\Bigr]_0^2 = 4C, \end{aligned} \end{equation*}

so \(C = \tfrac14\).

(b) The inner integral just computed is the marginal:

\begin{equation*} f_X(x) = \int_0^1 \tfrac14(x+2y)\,dy = \frac{x+1}{4}, \qquad 0<x<2, \end{equation*}

and \(0\) otherwise. Integrating, \(F_X(x) = \tfrac18 x(x+2)\) for \(0<x<2\).

(c) For \(0<x<2\), \(0<y<1\),

\begin{equation*} \begin{aligned} F(x,y) &= \int_0^x\!\!\int_0^y \tfrac14(s+2t)\,dt\,ds = \tfrac14\int_0^x \bigl(sy+y^2\bigr)\,ds\\ &= \tfrac14\Bigl(\tfrac{x^2 y}{2} + x y^2\Bigr) = \frac{xy(x+2y)}{8}. \end{aligned} \end{equation*}

Off that rectangle the cdf is obtained by capping each argument:

\begin{equation*} F(x,y) = \begin{cases} 0 & x \le 0 \text{ or } y \le 0,\\[2pt] \dfrac{xy(x+2y)}{8} & 0<x<2,\ 0<y<1,\\[6pt] \dfrac{y(1+y)}{2} & x \ge 2,\ 0<y<1,\\[6pt] \dfrac{x(x+2)}{8} & 0<x<2,\ y \ge 1,\\[6pt] 1 & x \ge 2,\ y \ge 1. \end{cases} \end{equation*}

(d) On \(0<x<2\) the map \(x \mapsto 9/(x+1)^2\) is strictly decreasing from \(9\) to \(1\), so \(Z\) takes values in \((1,9)\) and, for such \(z\),

\begin{equation*} Z \le z \iff (X+1)^2 \ge \frac{9}{z} \iff X \ge \frac{3}{\sqrt z} - 1 . \end{equation*}

Hence, with \(u = 3/\sqrt z - 1 \in (0,2)\),

\begin{equation*} \begin{aligned} F_Z(z) &= 1 - F_X(u) = 1 - \tfrac18 u(u+2) = 1 - \tfrac18\Bigl(\tfrac{3}{\sqrt z}-1\Bigr)\Bigl(\tfrac{3}{\sqrt z}+1\Bigr)\\ &= 1 - \tfrac18\Bigl(\tfrac{9}{z}-1\Bigr) = \frac98\Bigl(1 - \frac1z\Bigr), \end{aligned} \end{equation*}

which is \(0\) at \(z=1\) and \(1\) at \(z=9\). (Check!) Differentiating,

\begin{equation*} f_Z(z) = \frac{9}{8z^2}, \qquad 1 < z < 9, \end{equation*}

and \(0\) otherwise.

\(7/20\) and \(1/6\).

(a) On the unit square, \(x > \sqrt y \iff y < x^2\), so integrate \(y\) from \(0\) to \(x^2\):

\begin{equation*} \begin{aligned} P(X>\sqrt Y) &= \int_0^1\!\!\int_0^{x^2}(x+y)\,dy\,dx = \int_0^1 \Bigl[xy + \tfrac{y^2}{2}\Bigr]_{y=0}^{x^2} dx\\ &= \int_0^1 \Bigl(x^3 + \tfrac{x^4}{2}\Bigr) dx = \frac14 + \frac{1}{10} = \frac{7}{20}. \end{aligned} \end{equation*}

(b) For \(0 \le x \le 1\) we have \(x^2 \le x\), and both lie in \([0,1]\), so the inner limits are \(x^2\) and \(x\):

\begin{equation*} \begin{aligned} P(X^2<Y<X) &= \int_0^1\!\!\int_{x^2}^{x} 2x\,dy\,dx = \int_0^1 2x\,(x-x^2)\,dx\\ &= \int_0^1 (2x^2 - 2x^3)\,dx = \frac23 - \frac12 = \frac16 . \end{aligned} \end{equation*}

A mixed distribution: an atom of size \(\tfrac12\) at \(0\), plus density \(1-w\) on \((0,1)\). Measure time in hours after 1 PM, let \(X\) and \(Y\) be A’s and B’s arrival times, independent \(\mathrm{uniform}(0,1)\), and let \(W = \max(Y-X,\,0)\) be A’s waiting time.

The joint pdf is \(f(x,y) = 1\) on the unit square, so probabilities are areas. First the atom:

\begin{equation*} P(W=0) = P(Y \le X) = \tfrac12, \end{equation*}

by symmetry of the square about the diagonal \(y=x\) (the diagonal itself has area \(0\)).

For \(0 \le w < 1\) the event \(\{W > w\} = \{Y > X + w\}\) is the triangle with vertices \((0,w)\), \((0,1)\), \((1-w,1)\), whose legs both have length \(1-w\):

\begin{equation*} P(W>w) = \frac{(1-w)^2}{2}, \end{equation*}

so the cdf is

\begin{equation*} F_W(w) = \begin{cases} 0 & w < 0,\\[2pt] 1 - \dfrac{(1-w)^2}{2} & 0 \le w < 1,\\[6pt] 1 & w \ge 1. \end{cases} \end{equation*}

This jumps from \(0\) to \(\tfrac12\) at \(w=0\) and is absolutely continuous on \((0,1)\) with

\begin{equation*} f_W(w) = \frac{d}{dw}\Bigl(1 - \tfrac12(1-w)^2\Bigr) = 1-w, \qquad 0<w<1, \end{equation*}

and \(0\) elsewhere.

\(1/2\). Measure \(X\) in minutes after 8 AM, so \(X \sim \mathrm{uniform}(0,30)\) and \(Y \sim \mathrm{uniform}(40,50)\); by independence the joint pdf is the product

\begin{equation*} f(x,y) = \frac{1}{30}\cdot\frac{1}{10} = \frac{1}{300}, \qquad 0<x<30,\ 40<y<50 . \end{equation*}

She arrives before 9 AM exactly when \(X+Y < 60\). For each \(y \in (40,50)\) the cut \(60-y\) lies in \((10,20)\), hence strictly inside \((0,30)\), so the \(x\)-section has length \(60-y\):

\begin{equation*} \begin{aligned} P(X+Y<60) &= \frac{1}{300}\int_{40}^{50}\!\!\int_0^{60-y} dx\,dy = \frac{1}{300}\int_{40}^{50}(60-y)\,dy\\ &= \frac{1}{300}\Bigl[60y - \tfrac{y^2}{2}\Bigr]_{40}^{50} = \frac{1750-1600}{300} = \frac{150}{300} = \frac12 . \end{aligned} \end{equation*}

(a) The envelope you open is chosen by a fair coin flip that is made independently of the swami’s choice of \(m\), so with \(I\) the indicator of “you hold the first envelope”, \(P(I=1)=1/2\) independently of \(M\), and

\begin{equation*} P(X=m\mid M=m)=P(I=1)=\tfrac12,\qquad P(X=2m\mid M=m)=P(I=0)=\tfrac12 . \end{equation*}

Bayes’ Rule (Theorem 1.3.5) applied to the two exhaustive hypotheses \(\{M=x\}\) (you hold the small envelope) and \(\{M=x/2\}\) (you hold the large one) gives

\begin{equation*} \begin{aligned} P(M=x\mid X=x) &=\frac{P(X=x\mid M=x)\,\pi(x)} {P(X=x\mid M=x)\,\pi(x)+P(X=x\mid M=x/2)\,\pi(x/2)}\\[2pt] &=\frac{\tfrac12\pi(x)}{\tfrac12\pi(x)+\tfrac12\pi(x/2)} =\frac{\pi(x)}{\pi(x)+\pi(x/2)}, \end{aligned} \end{equation*}

and \(P(M=x/2\mid X=x)=1-P(M=x\mid X=x)=\pi(x/2)/(\pi(x)+\pi(x/2))\). (The Miscellanea calls \(\pi\) a pdf, so these are posterior densities at the two points \(x\) and \(x/2\); the arithmetic is the display above either way.)

(b) Conditional on \(X=x\), a trade yields \(2x\) on \(\{M=x\}\) and \(x/2\) on \(\{M=x/2\}\), so the expected winning from a trade is the quantity displayed in the statement, and it beats the \(x\) in hand exactly when

\begin{equation*} \begin{aligned} 2x\,\pi(x)+\tfrac{x}{2}\pi(x/2) &> x\bigl(\pi(x)+\pi(x/2)\bigr)\\ \iff\quad \pi(x) &> \tfrac12\pi(x/2) \quad\iff\quad \pi(x/2)<2\pi(x). \end{aligned} \end{equation*}

For \(\pi(t)=\lambda e^{-\lambda t}\), \(t>0\), this reads

\begin{equation*} \lambda e^{-\lambda x/2}<2\lambda e^{-\lambda x} \iff e^{\lambda x/2}<2 \iff x<\frac{2\log 2}{\lambda}. \end{equation*}

(c) Under the classical reading the envelopes hold \(m\) and \(2m\) with \(m\) fixed, and \(X\) and \(Y\) are these two amounts as assigned to you and your opponent by the fair flip. If \(X=m\) your opponent necessarily holds the other envelope, so \(Y=2m=2x\) with probability \(1\); if \(X=2m\) then \(Y=m=x/2\) with probability \(1\). Since \(Y=2m\) or \(Y=m\) according as \(I=1\) or \(I=0\),

\begin{equation*} E(Y)=\tfrac12(2m)+\tfrac12(m)=\frac{3m}{2}=E(X). \end{equation*}

Inclusion-exclusion on four lower quadrants, then the assumed factorization: for any \(s<b\) and \(t<d\),

\begin{equation*} \begin{aligned} P(s<X\le b,\,t<Y\le d) &=F_{X,Y}(b,d)-F_{X,Y}(s,d)-F_{X,Y}(b,t)+F_{X,Y}(s,t)\\ &=F_X(b)F_Y(d)-F_X(s)F_Y(d)-F_X(b)F_Y(t)+F_X(s)F_Y(t)\\ &=\bigl(F_X(b)-F_X(s)\bigr)\bigl(F_Y(d)-F_Y(t)\bigr). \end{aligned} \end{equation*}

Now close the rectangle. Take \(s_n\uparrow a\) and \(t_n\uparrow c\) strictly; since \(\bigcap_n\{X>s_n\}=\{X\ge a\}\), the events \(\{s_n<X\le b,\ t_n<Y\le d\}\) decrease to \(\{a\le X\le b,\ c\le Y\le d\}\), so continuity of \(P\) from above (Axiom 3 of Definition 1.2.4, countable additivity) gives

\begin{equation*} \begin{aligned} P(a\le X\le b,\,c\le Y\le d) &=\bigl(F_X(b)-F_X(a^-)\bigr)\bigl(F_Y(d)-F_Y(c^-)\bigr)\\ &=P(a\le X\le b)\,P(c\le Y\le d), \end{aligned} \end{equation*}

the last line by that same limit taken one coordinate at a time.

(a) \(P(X=2,\,Y=3)=0\) but \(P(X=2)\,P(Y=3)=\tfrac12\cdot\tfrac13=\tfrac16\neq 0\), so the joint pmf does not factor and Definition 4.2.5 fails.

Summing down columns and across rows, the marginals are

\begin{equation*} \begin{aligned} f_X(1)&=\tfrac1{12}+\tfrac16+0=\tfrac14, &\quad f_X(2)&=\tfrac16+0+\tfrac13=\tfrac12,\\ f_X(3)&=\tfrac1{12}+\tfrac16+0=\tfrac14, &\quad f_Y(2)&=\tfrac1{12}+\tfrac16+\tfrac1{12}=\tfrac13,\\ f_Y(3)&=\tfrac16+0+\tfrac16=\tfrac13, &\quad f_Y(4)&=0+\tfrac13+0=\tfrac13 . \end{aligned} \end{equation*}

(b) Take \(f_{U,V}(u,v)=f_X(u)f_Y(v)\), which is a pmf with the required marginals and is independent by construction (Lemma 4.2.7):

The columns sum to \(\tfrac14,\tfrac12,\tfrac14\) and the rows to \(\tfrac13\) each. (Check!)

No. The second head cannot arrive before the first, so \(V\ge U+1\) with probability \(1\) and in particular

\begin{equation*} P(U=2,\,V=2)=0, \end{equation*}

while each marginal event has positive probability:

\begin{equation*} P(U=2)=P(\text{TH})=\tfrac14,\qquad P(V=2)=P(\text{HH})=\tfrac14, \end{equation*}

so \(P(U=2)P(V=2)=\tfrac1{16}\neq 0\). The joint pmf therefore does not factor, and Definition 4.2.5 fails.

\(1/4\). Model the stick as \([0,1]\) broken at two independent uniform\((0,1)\) points \(X\) and \(Y\), so the pieces are \(\min(X,Y)\), \(|X-Y|\), \(1-\max(X,Y)\). Three nonnegative lengths summing to \(1\) satisfy all three triangle inequalities exactly when no piece exceeds the sum of the other two, i.e. exactly when every piece is less than \(1/2\).

On the half \(\{X<Y\}\) the pieces are \(X\), \(Y-X\), \(1-Y\), and the three constraints read

\begin{equation*} X<\tfrac12,\qquad Y>\tfrac12,\qquad Y-X<\tfrac12 , \end{equation*}

which cut out of the unit square the triangle with vertices \((0,\tfrac12),(\tfrac12,\tfrac12),(\tfrac12,1)\) — a right triangle with legs \(\tfrac12\), hence area \(\tfrac12\cdot\tfrac12\cdot\tfrac12=\tfrac18\). Since \((X,Y)\) is uniform on the unit square and the region for \(\{X>Y\}\) is its reflection in the diagonal,

\begin{equation*} P(\text{triangle})=\tfrac18+\tfrac18=\tfrac14 . \end{equation*}

(a) Add and subtract \(E(Y\mid X)\) and expand:

\begin{equation*} \begin{aligned} E\bigl(Y-g(X)\bigr)^2 &=E\Bigl[\bigl(Y-E(Y\mid X)\bigr)+\bigl(E(Y\mid X)-g(X)\bigr)\Bigr]^2\\ &=E\bigl(Y-E(Y\mid X)\bigr)^2+E\bigl(E(Y\mid X)-g(X)\bigr)^2\\ &\qquad+2E\Bigl[\bigl(Y-E(Y\mid X)\bigr)\bigl(E(Y\mid X)-g(X)\bigr)\Bigr]. \end{aligned} \end{equation*}

The cross term vanishes. Condition on \(X\) using the tower property (Theorem 4.4.3, whose only hypothesis is that the expectations exist — assume the second moments are finite, else both sides are \(+\infty\)); the factor \(E(Y\mid X)-g(X)\) is a function of \(X\), so it comes out of the inner conditional expectation:

\begin{equation*} \begin{aligned} &E\Bigl[\bigl(Y-E(Y\mid X)\bigr)\bigl(E(Y\mid X)-g(X)\bigr)\Bigr]\\ &\quad=E\Bigl\{\bigl(E(Y\mid X)-g(X)\bigr)\, E\bigl[\,Y-E(Y\mid X)\,\bigm|\,X\,\bigr]\Bigr\} =E\bigl\{(\,\cdot\,)\cdot 0\bigr\}=0, \end{aligned} \end{equation*}

because \(E[Y\mid X]-E[E(Y\mid X)\mid X]=E(Y\mid X)-E(Y\mid X)=0\). Hence

\begin{equation*} E\bigl(Y-g(X)\bigr)^2 =E\bigl(Y-E(Y\mid X)\bigr)^2+E\bigl(E(Y\mid X)-g(X)\bigr)^2 \ \ge\ E\bigl(Y-E(Y\mid X)\bigr)^2, \end{equation*}

with equality attained by \(g(x)=E(Y\mid X=x)\), which is the asserted minimum.

(b) Take \(X\) degenerate, say \(P(X=1)=1\). Every function \(g(X)\) is then a constant \(b\), and the conditional distribution of \(Y\) given \(X=1\) is the marginal (Definition 4.2.1 with \(f_X(1)=1\)), so \(E(Y\mid X)=EY\) and part (a) reads

\begin{equation*} \min_b E(Y-b)^2=E(Y-EY)^2, \end{equation*}

which is (2.2.3) with \(Y\) in the role of the book’s \(X\).

(a) \(P(X^2+Y^2<1)=1-e^{-1/2}\approx 0.3935\). By independence the joint pdf is the circularly symmetric \(f(x,y)=(2\pi)^{-1}e^{-(x^2+y^2)/2}\), so passing to polar coordinates \(x=r\cos\theta\), \(y=r\sin\theta\) (Jacobian \(r\)),

\begin{equation*} \begin{aligned} P(X^2+Y^2<1) &=\int_0^{2\pi}\!\!\int_0^1 \frac{1}{2\pi}e^{-r^2/2}\,r\,dr\,d\theta\\ &=\int_0^1 r\,e^{-r^2/2}\,dr =\Bigl[-e^{-r^2/2}\Bigr]_0^1=1-e^{-1/2}. \end{aligned} \end{equation*}

(b) \(X^2\sim\chi^2_1\) is Example 2.1.9: \(g(x)=x^2\) is monotone on \(A_1=(-\infty,0)\) and \(A_2=(0,\infty)\) with \(g_1^{-1}(t)=-\sqrt t\), \(g_2^{-1}(t)=\sqrt t\), and \(A_0=\{0\}\) has probability \(0\), so Theorem 2.1.8 gives, for \(t>0\),

\begin{equation*} f_{X^2}(t)=\frac{1}{\sqrt{2\pi}}e^{-t/2}\Bigl|\frac{-1}{2\sqrt t}\Bigr| +\frac{1}{\sqrt{2\pi}}e^{-t/2}\Bigl|\frac{1}{2\sqrt t}\Bigr| =\frac{1}{\sqrt{2\pi}}\,t^{-1/2}e^{-t/2}, \end{equation*}

which is the gamma\((\tfrac12,2)\) pdf \(\bigl(\Gamma(\tfrac12)=\sqrt\pi\bigr)\), that is \(\chi^2_1\). Hence

\begin{equation*} P(X^2<1)=P(-1<X<1)=2\Phi(1)-1\approx 0.6827 . \end{equation*}

\(X \mid X+Y = n \sim \text{binomial}(n, \theta/(\theta+\lambda))\). For \(x = 0,1,\dots,n\), independence gives \(P(X = x, X+Y = n) = P(X=x)P(Y=n-x)\), so by Definition 4.2.1 and Theorem 4.3.2,

\begin{equation*} \begin{aligned} P(X = x \mid X+Y = n) &= \frac{\dfrac{e^{-\theta}\theta^{x}}{x!}\cdot\dfrac{e^{-\lambda}\lambda^{n-x}}{(n-x)!}} {\dfrac{e^{-(\theta+\lambda)}(\theta+\lambda)^{n}}{n!}}\\[4pt] &= \binom{n}{x}\frac{\theta^{x}\lambda^{n-x}}{(\theta+\lambda)^{n}}\\[4pt] &= \binom{n}{x}\left(\frac{\theta}{\theta+\lambda}\right)^{\!x} \left(1-\frac{\theta}{\theta+\lambda}\right)^{\!n-x}, \end{aligned} \end{equation*}

the \(\text{binomial}(n,\theta/(\theta+\lambda))\) pmf.

Since \(Y = (X+Y) - X\), the same computation with the roles of \(\theta\) and \(\lambda\) interchanged gives

\begin{equation*} Y \mid X+Y = n \;\sim\; \text{binomial}\!\left(n,\ \frac{\lambda}{\theta+\lambda}\right). \end{equation*}

The convention in (b) forces the version of the geometric supported on \(\{0,1,2,\dots\}\) (with the Section 3.2 support \(\{1,2,\dots\}\) the event \(\{X+Y=0\}\) is empty), so throughout \(P(X=x) = pq^{x}\), \(x=0,1,2,\dots\), with \(q = 1-p\), and likewise for \(Y\).

(a) The joint pmf of \((U,V)\) factors. If \(v \ge 0\) then \(\{U=u,\,V=v\} = \{X=u+v,\ Y=u\}\), and if \(v<0\) then \(\{U=u,\,V=v\} = \{X=u,\ Y=u-v\}\); either way a single pair \((x,y)\) is involved and

\begin{equation*} P(U=u, V=v) \;=\; p^{2}q^{\,2u+|v|} \;=\; \underbrace{\left(p^{2}q^{2u}\right)}_{g(u)}\;\underbrace{q^{|v|}}_{h(v)}, \end{equation*}

for \(u = 0,1,2,\dots\) and \(v \in \mathbb{Z}\). The support is the product set \(\{0,1,2,\dots\}\times\mathbb{Z}\), so by Lemma 4.2.7 \(U\) and \(V\) are independent. Summing out gives the marginals

\begin{equation*} \begin{aligned} P(U=u) &= p^{2}q^{2u}\sum_{v\in\mathbb{Z}} q^{|v|} = p^{2}q^{2u}\,\frac{1+q}{1-q} = (1-q^{2})(q^{2})^{u},\\[3pt] P(V=v) &= q^{|v|}\,p^{2}\sum_{u\ge 0} q^{2u} = \frac{p}{1+q}\,q^{|v|}, \end{aligned} \end{equation*}

so \(U\) is geometric with success probability \(1-q^{2}\) and \(V\) has a two-sided geometric law. (Check that each sums to \(1\).)

(b) Write \(z = a/b\) in lowest terms. For \(0 < z \le 1\) the event \(\{Z=z\}\) is the disjoint union

\begin{equation*} \{Z = z\} = \bigcup_{m\ge 1}\{X = ma,\ Y = m(b-a)\}, \end{equation*}

and each pair there has probability \(p^{2}q^{ma}q^{m(b-a)} = p^{2}q^{mb}\), whence

\begin{equation*} P(Z=z) = \sum_{m\ge 1} p^{2}q^{mb} = \frac{p^{2}q^{b}}{1-q^{b}}, \qquad z = \frac{a}{b}\in(0,1]. \end{equation*}

The value \(z=0\) picks up the convention, since \(\{Z=0\} = \{X=0\}\) (this already contains \(\{X=Y=0\}\)):

\begin{equation*} P(Z=0) = P(X=0) = p = \frac{p^{2}q}{1-q} + p^{2}, \end{equation*}

the first term being what the displayed formula gives at \(b=1\).

(c) With \(T = X+Y\), for \(0 \le x \le t\),

\begin{equation*} P(X=x,\ T=t) = P(X=x)P(Y=t-x) = p^{2}q^{x}q^{\,t-x} = p^{2}q^{t}, \end{equation*}

and \(0\) otherwise. Being free of \(x\) on each line \(T=t\), this says \(T\) is negative binomial with \(P(T=t) = (t+1)p^{2}q^{t}\) and \(X \mid T=t\) is uniform on \(\{0,1,\dots,t\}\).

(a) \(Y\) is geometric with success probability \(p = 1-e^{-1}\). Indeed, for \(i = 0,1,2,\dots\),

\begin{equation*} P(Y = i+1) = P(i \le X < i+1) = e^{-i} - e^{-(i+1)} = \left(1-e^{-1}\right)\left(e^{-1}\right)^{i}, \end{equation*}

so writing \(y = i+1\),

\begin{equation*} P(Y=y) = p(1-p)^{\,y-1},\qquad y = 1,2,\dots,\quad p = 1-e^{-1} \approx 0.6321, \end{equation*}

which is the \(\text{geometric}(p)\) pmf of Section 3.2.

(b) \(X-4 \mid Y \ge 5 \;\sim\; \text{exponential}(1)\). The conditioning event is \(\{Y \ge 5\} = \{X \ge 4\}\), and for \(t > 0\)

\begin{equation*} P(X-4 > t \mid X \ge 4) = \frac{P(X > 4+t)}{P(X \ge 4)} = \frac{e^{-(4+t)}}{e^{-4}} = e^{-t}, \end{equation*}

the exponential survival function; this is the memoryless property of the exponential (Section 3.3).

Pass to polar coordinates. \(f \ge 0\) on the open first quadrant \(A = \{(x,y): x,y>0\}\) because \(g \ge 0\), so only the total integral is at issue. The map

\begin{equation*} x = r\cos\theta,\qquad y = r\sin\theta \end{equation*}

is one-to-one from \(B = (0,\infty)\times(0,\pi/2)\) onto \(A\), with Jacobian

\begin{equation*} J = \begin{vmatrix} \cos\theta & -r\sin\theta\\ \sin\theta & \ \ r\cos\theta\end{vmatrix} = r \;>\;0 . \end{equation*}

Hence, by the change of variables (4.3.2),

\begin{equation*} \begin{aligned} \int_{0}^{\infty}\!\!\int_{0}^{\infty} f(x,y)\,dx\,dy &= \int_{0}^{\pi/2}\!\!\int_{0}^{\infty} \frac{2g( r)}{\pi r}\; r \,dr\,d\theta\\[3pt] &= \frac{2}{\pi}\cdot\frac{\pi}{2}\int_{0}^{\infty} g( r)\,dr\\[3pt] &= 1 . \end{aligned} \end{equation*}

So \(f\) is a pdf on \(A\).

(a) \((X_1-X_2)^{2}/2 \sim \chi^{2}_{1}\), with pdf

\begin{equation*} f_{U}(u) = \frac{1}{\sqrt{2\pi}}\,u^{-1/2}e^{-u/2},\qquad u>0 . \end{equation*}

By Theorem 4.2.14, \(X_1 - X_2 \sim \text{n}(0,2)\), so \(Z = (X_1-X_2)/\sqrt{2} \sim \text{n}(0,1)\), and \(U = Z^{2}\) is \(\chi^{2}_{1}\) by Example 2.1.9.

(b) \(X_1/(X_1+X_2) \sim \text{beta}(\alpha_1,\alpha_2)\) and \(X_2/(X_1+X_2) \sim \text{beta}(\alpha_2,\alpha_1)\). Put

\begin{equation*} U = X_1+X_2,\qquad V = \frac{X_1}{X_1+X_2}, \end{equation*}

a one-to-one map of \(A = (0,\infty)^{2}\) onto \(B = (0,\infty)\times(0,1)\) with inverse \(x_1 = uv\), \(x_2 = u(1-v)\) and

\begin{equation*} J = \begin{vmatrix} v & u\\ 1-v & -u\end{vmatrix} = -u, \qquad |J| = u . \end{equation*}

Since \(f_{X_1,X_2}(x_1,x_2) = x_1^{\alpha_1-1}x_2^{\alpha_2-1}e^{-(x_1+x_2)}/(\Gamma(\alpha_1)\Gamma(\alpha_2))\) on \(A\), formula (4.3.2) gives

\begin{equation*} \begin{aligned} f_{U,V}(u,v) &= \frac{(uv)^{\alpha_1-1}\big(u(1-v)\big)^{\alpha_2-1}e^{-u}}{\Gamma(\alpha_1)\Gamma(\alpha_2)}\; u\\[4pt] &= \underbrace{\frac{u^{\alpha_1+\alpha_2-1}e^{-u}}{\Gamma(\alpha_1+\alpha_2)}}_{\text{gamma}(\alpha_1+\alpha_2,\,1)\ \text{pdf}}\\[16pt] &\qquad\times \underbrace{\frac{\Gamma(\alpha_1+\alpha_2)\,v^{\alpha_1-1}(1-v)^{\alpha_2-1}} {\Gamma(\alpha_1)\Gamma(\alpha_2)}}_{\text{beta}(\alpha_1,\alpha_2)\ \text{pdf}} \end{aligned} \end{equation*}

on the product set \(B\). By Lemma 4.2.7 the two factors are the marginals (and \(U,V\) are independent), so \(V = X_1/(X_1+X_2) \sim \text{beta}(\alpha_1,\alpha_2)\). Interchanging the indices, \(X_2/(X_1+X_2) = 1-V \sim \text{beta}(\alpha_2,\alpha_1)\).

(a) The joint pdf is

\begin{equation*} f_{Y_1,Y_2}(y_1,y_2) = \frac{1}{2\sigma^{2}}e^{-y_1/(2\sigma^{2})}\cdot\frac{1}{\pi\sqrt{1-y_2^{2}}}, \qquad y_1>0,\ -1<y_2<1 . \end{equation*}

The transformation is two-to-one, so use (4.3.6) with \(A_1 = \{x_2>0\}\) and \(A_2 = \{x_2<0\}\) (the line \(x_2=0\) has probability \(0\)). On \(A_\varepsilon\) (\(\varepsilon = \pm 1\)) the inverse is

\begin{equation*} x_1 = y_2\sqrt{y_1},\qquad x_2 = \varepsilon\sqrt{y_1}\sqrt{1-y_2^{2}}, \end{equation*}

mapping \(B = (0,\infty)\times(-1,1)\) onto \(A_\varepsilon\), with

\begin{equation*} \begin{aligned} J_\varepsilon &= \begin{vmatrix} \dfrac{y_2}{2\sqrt{y_1}} & \sqrt{y_1}\\[10pt] \dfrac{\varepsilon\sqrt{1-y_2^{2}}}{2\sqrt{y_1}} & \dfrac{-\varepsilon y_2\sqrt{y_1}}{\sqrt{1-y_2^{2}}} \end{vmatrix}\\[10pt] &= -\frac{\varepsilon}{2}\cdot\frac{y_2^{2}+(1-y_2^{2})}{\sqrt{1-y_2^{2}}} \;=\; \frac{-\varepsilon}{2\sqrt{1-y_2^{2}}} . \end{aligned} \end{equation*}

Both inverses carry \(x_1^{2}+x_2^{2} = y_1\) and both have \(|J_\varepsilon| = \big(2\sqrt{1-y_2^{2}}\big)^{-1}\), so

\begin{equation*} \begin{aligned} f_{Y_1,Y_2}(y_1,y_2) &= \sum_{\varepsilon=\pm1}\frac{1}{2\pi\sigma^{2}}e^{-y_1/(2\sigma^{2})}\, \frac{1}{2\sqrt{1-y_2^{2}}}\\[3pt] &= \frac{1}{2\pi\sigma^{2}}\,\frac{e^{-y_1/(2\sigma^{2})}}{\sqrt{1-y_2^{2}}} \end{aligned} \end{equation*}

on \(B\), which is the displayed factorization.

(b) The support \(B\) is a product set and the pdf factors as \(g(y_1)h(y_2)\) above, so \(Y_1\) and \(Y_2\) are independent by Lemma 4.2.7; moreover each factor is already a pdf, so

\begin{equation*} Y_1 \sim \text{exponential}(2\sigma^{2}) = \sigma^{2}\chi^{2}_{2}, \qquad f_{Y_2}(y_2) = \frac{1}{\pi\sqrt{1-y_2^{2}}},\ \ |y_2|<1 . \end{equation*}

Geometrically, in polar form \(R = \sqrt{Y_1}\) and \(Y_2 = \cos\Theta\) with \(\Theta\) the angle to the \(x_1\)-axis, so squared distance from the origin carries no information about direction. That is the rotational symmetry of the spherical normal: its pdf depends on \((x_1,x_2)\) only through \(x_1^{2}+x_2^{2}\), forcing \(\Theta \sim \text{uniform}(0,2\pi)\) whatever \(R\) is, and \(\cos\Theta\) then has the arcsine pdf above.

\(X\) and \(Y\) are independent \(\text{n}(0,1)\):

\begin{equation*} f_{X,Y}(x,y) = \frac{1}{2\pi}\,e^{-(x^{2}+y^{2})/2},\qquad (x,y)\in\mathbb{R}^{2}. \end{equation*}

Since \(\chi^{2}_{2} = \text{exponential}(2)\), \(f_{R^{2}}(w) = \tfrac12 e^{-w/2}\) for \(w>0\); the map \(r \mapsto r^{2}\) is increasing on \((0,\infty)\), so by Theorem 2.1.5

\begin{equation*} f_{R}( r) = \tfrac12 e^{-r^{2}/2}\cdot 2r = r e^{-r^{2}/2},\qquad r>0 . \end{equation*}

By independence, \(f_{R,\theta}(r,\theta) = r e^{-r^{2}/2}/(2\pi)\) on \(A = (0,\infty)\times(0,2\pi)\). The polar map \((r,\theta)\mapsto(r\cos\theta,\,r\sin\theta)\) is one-to-one from \(A\) onto the plane less the nonnegative \(x\)-axis (a set of measure zero), with inverse \(r = \sqrt{x^{2}+y^{2}}\), and

\begin{equation*} J = \frac{\partial(r,\theta)}{\partial(x,y)} = \frac{1}{r} = \frac{1}{\sqrt{x^{2}+y^{2}}} . \end{equation*}

Hence by (4.3.2),

\begin{equation*} \begin{aligned} f_{X,Y}(x,y) &= \frac{\sqrt{x^{2}+y^{2}}\,e^{-(x^{2}+y^{2})/2}}{2\pi}\cdot\frac{1}{\sqrt{x^{2}+y^{2}}}\\[3pt] &= \frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}\cdot\frac{1}{\sqrt{2\pi}}e^{-y^{2}/2}, \end{aligned} \end{equation*}

a product of two \(\text{n}(0,1)\) pdfs on the product set \(\mathbb{R}^{2}\), so by Lemma 4.2.7 \(X\) and \(Y\) are independent standard normals.

Apply (4.3.2). Since \(a>0\) and \(c>0\), the map \((x,y)\mapsto(ax+b,\,cy+d)\) is one-to-one from \(\mathcal{A}=\{(x,y):f(x,y)>0\}\) onto its image \(\mathcal{B}\), with unique inverse \(x=h_1(u,v)=(u-b)/a\), \(y=h_2(u,v)=(v-d)/c\) and Jacobian

\begin{equation*} J \;=\; \begin{vmatrix} 1/a & 0\\[1mm] 0 & 1/c \end{vmatrix} \;=\;\frac{1}{ac}\;\neq\;0, \end{equation*}

so \(|J| = 1/(ac)\). Hence

\begin{equation*} f_{U,V}(u,v) \;=\; f\!\left(\frac{u-b}{a},\,\frac{v-d}{c}\right)\frac{1}{ac} \end{equation*}

on \(\mathcal{B}\); off \(\mathcal{B}\) both sides vanish, since there \(f\bigl((u-b)/a,(v-d)/c\bigr)=0\).

Both transformations return \(XY \sim \text{beta}(\alpha,\beta+\gamma)\), the answer of Example 4.3.3. Throughout write

\begin{equation*} C \;=\; \frac{\Gamma(\alpha+\beta+\gamma)}{\Gamma(\alpha)\Gamma(\beta)\Gamma(\gamma)}, \end{equation*}

the constant left after the \(\Gamma(\alpha+\beta)\) factors cancel in \(f_{X,Y}\).

(a) \(U=XY\), \(V=Y\). On \(\mathcal{A}=(0,1)^2\) the inverse is \(x = u/v\), \(y = v\), unique and one-to-one onto \(\mathcal{B}=\{(u,v):0<u<v<1\}\) (for fixed \(v=y\), \(u=xy\) ranges over \((0,v)\)). The Jacobian is

\begin{equation*} J=\begin{vmatrix} 1/v & -u/v^2\\ 0 & 1\end{vmatrix}=\frac{1}{v}. \end{equation*}

By (4.3.2),

\begin{equation*} \begin{aligned} f_{U,V}(u,v) &= C\left(\frac{u}{v}\right)^{\alpha-1}\!\left(1-\frac{u}{v}\right)^{\beta-1} v^{\alpha+\beta-1}(1-v)^{\gamma-1}\frac{1}{v}\\ &= C\,u^{\alpha-1}(v-u)^{\beta-1}(1-v)^{\gamma-1}, \end{aligned} \end{equation*}

on \(0<u<v<1\), every power of \(v\) cancelling. (Check!) Integrate out \(v\) with \(v = u+(1-u)t\), \(dv=(1-u)\,dt\):

\begin{equation*} \begin{aligned} \int_u^1 (v-u)^{\beta-1}(1-v)^{\gamma-1}\,dv &= (1-u)^{\beta+\gamma-1}\!\int_0^1 t^{\beta-1}(1-t)^{\gamma-1}dt\\ &= (1-u)^{\beta+\gamma-1}\frac{\Gamma(\beta)\Gamma(\gamma)}{\Gamma(\beta+\gamma)} . \end{aligned} \end{equation*}

Hence

\begin{equation*} f_U(u)=\frac{\Gamma(\alpha+\beta+\gamma)}{\Gamma(\alpha)\Gamma(\beta+\gamma)} u^{\alpha-1}(1-u)^{\beta+\gamma-1},\qquad 0<u<1, \end{equation*}

i.e. \(XY\sim\text{beta}(\alpha,\beta+\gamma)\).

(b) \(U=XY\), \(V=X/Y\). Then \(UV=X^2\) and \(U/V=Y^2\), so with \(X,Y>0\) the inverse is

\begin{equation*} x=\sqrt{uv},\qquad y=\sqrt{u/v}, \end{equation*}

one-to-one from \(\mathcal{B}=\{(u,v):0<u<1,\ u<v<1/u\}\) onto \(\mathcal{A}\) (the constraints \(x<1\), \(y<1\) read \(uv<1\), \(u<v\)). The Jacobian is

\begin{equation*} J=\begin{vmatrix} \tfrac12 v^{1/2}u^{-1/2} & \tfrac12 u^{1/2}v^{-1/2}\\[1mm] \tfrac12 v^{-1/2}u^{-1/2} & -\tfrac12 u^{1/2}v^{-3/2} \end{vmatrix} =-\frac{1}{4v}-\frac{1}{4v}=-\frac{1}{2v}, \end{equation*}

so \(|J| = 1/(2v)\) and, by (4.3.2),

\begin{equation*} \begin{aligned} f_{U,V}(u,v)=\;&C\,(uv)^{\frac{\alpha-1}{2}}\bigl(1-\sqrt{uv}\bigr)^{\beta-1}\\ &\times (u/v)^{\frac{\alpha+\beta-1}{2}}\bigl(1-\sqrt{u/v}\bigr)^{\gamma-1}\frac{1}{2v} \end{aligned} \end{equation*}

on \(\mathcal{B}\). Collecting the pure powers, \((uv)^{\frac{\alpha-1}{2}}(u/v)^{\frac{\alpha+\beta-1}{2}}\frac{1}{2v} = \tfrac12 u^{\alpha+\frac{\beta}{2}-1}v^{-\frac{\beta}{2}-1}\). Now integrate out \(v\) by \(t=\sqrt{u/v}\), i.e. \(v=u/t^2\), \(|dv| = 2u\,t^{-3}dt\), which maps \(v\in(u,1/u)\) onto \(t\in(u,1)\) and gives \(\sqrt{uv}=u/t\). The power factors become

\begin{equation*} \tfrac12 u^{\alpha+\frac{\beta}{2}-1}\bigl(u/t^{2}\bigr)^{-\frac{\beta}{2}-1}\,2u\,t^{-3} = u^{\alpha-1}t^{\beta-1}, \end{equation*}

so that

\begin{equation*} f_U(u)=C\,u^{\alpha-1}\!\int_u^1 t^{\beta-1}\Bigl(1-\frac{u}{t}\Bigr)^{\beta-1}(1-t)^{\gamma-1}dt = C\,u^{\alpha-1}\!\int_u^1 (t-u)^{\beta-1}(1-t)^{\gamma-1}dt . \end{equation*}

This is the integral already evaluated in (a), so again

\begin{equation*} f_U(u)=\frac{\Gamma(\alpha+\beta+\gamma)}{\Gamma(\alpha)\Gamma(\beta+\gamma)} u^{\alpha-1}(1-u)^{\beta+\gamma-1},\qquad 0<u<1 . \end{equation*}

\(Z_1 \sim \text{gamma}(r+s,1)\) and \(Z_2 \sim \text{beta}(r,s)\), independent, because the transformed joint pdf factors.

The inverse of \(z_1 = x+y\), \(z_2 = x/(x+y)\) is \(x = z_1z_2\), \(y = z_1(1-z_2)\), a one-to-one map of \(\mathcal{B} = \{(z_1,z_2): z_1>0,\ 0<z_2<1\}\) onto \(\mathcal{A} = \{(x,y): x>0,\ y>0\}\), with Jacobian

\begin{equation*} J=\begin{vmatrix} z_2 & z_1\\ 1-z_2 & -z_1\end{vmatrix}=-z_1,\qquad |J|=z_1 . \end{equation*}

By independence \(f_{X,Y}(x,y) = x^{r-1}e^{-x}y^{s-1}e^{-y}/\bigl(\Gamma( r)\Gamma(s)\bigr)\) on \(\mathcal{A}\), so (4.3.2) gives, for \((z_1,z_2)\in\mathcal{B}\),

\begin{equation*} \begin{aligned} f_{Z_1,Z_2}(z_1,z_2) &=\frac{(z_1z_2)^{r-1}\bigl(z_1(1-z_2)\bigr)^{s-1}e^{-z_1}}{\Gamma( r)\Gamma(s)}\,z_1\\ &=\underbrace{\frac{z_1^{r+s-1}e^{-z_1}}{\Gamma(r+s)}}_{\text{gamma}(r+s,1)} \cdot \underbrace{\frac{\Gamma(r+s)}{\Gamma( r)\Gamma(s)}z_2^{r-1}(1-z_2)^{s-1}}_{\text{beta}(r,s)} . \end{aligned} \end{equation*}

The set \(\mathcal{B}\) is a rectangle (a cross product), and the pdf is a function of \(z_1\) times a function of \(z_2\) on it, so \(Z_1\) and \(Z_2\) are independent by Lemma 4.2.7. Each factor is already a pdf, so they are the marginals: \(Z_1\sim\text{gamma}(r+s,1)\), \(Z_2\sim\text{beta}(r,s)\).

In both examples the transformation \((X,Z)\mapsto(X,Y)\) is a shear, so \(|J|=1\) and the joint pdf of \((X,Y)\) is the joint pdf of \((X,Z)\) with \(z\) replaced by its expression in \((x,y)\).

For Example 4.5.8, independence gives \(f_{X,Z}(x,z) = 1\cdot 10 = 10\) on \(\mathcal{A} = \{0<x<1,\ 0<z<\tfrac{1}{10}\}\). Take \(U = X\), \(Y = X+Z\); the inverse \(x = u\), \(z = y-u\) is one-to-one onto

\begin{equation*} \mathcal{B}=\Bigl\{(u,y): 0<u<1,\ u<y<u+\tfrac{1}{10}\Bigr\}, \end{equation*}

since \(0<z<\tfrac1{10}\) reads \(u<y<u+\tfrac{1}{10}\). Its Jacobian is

\begin{equation*} J=\begin{vmatrix} 1 & 0\\ -1 & 1\end{vmatrix}=1 . \end{equation*}

So by (4.3.2), \(f_{X,Y}(x,y) = f_{X,Z}(x,\,y-x)\cdot 1 = 10\) on \(0<x<1\), \(x<y<x+\tfrac1{10}\), as claimed.

For Example 4.5.9, now \(f_{X,Z}(x,z) = \tfrac12\cdot 10 = 5\) on \(\mathcal{A} = \{-1<x<1,\ 0<z<\tfrac{1}{10}\}\). Take \(U=X\), \(Y=X^2+Z\); the inverse \(x=u\), \(z = y-u^2\) is one-to-one onto

\begin{equation*} \mathcal{B}=\Bigl\{(u,y): -1<u<1,\ u^2<y<u^2+\tfrac{1}{10}\Bigr\}, \end{equation*}

with

\begin{equation*} J=\begin{vmatrix} 1 & 0\\ -2u & 1\end{vmatrix}=1 . \end{equation*}

Hence \(f_{X,Y}(x,y) = f_{X,Z}(x,\,y-x^2) = 5\) on \(-1<x<1\), \(x^2<y<x^2+\tfrac{1}{10}\), the region of Figure 4.5.2.

(a) \(Z\) and \(W\) have the mixed joint distribution

\begin{equation*} f_{Z,W}(z,1)=\frac{1}{\lambda}e^{-\tau z},\qquad f_{Z,W}(z,0)=\frac{1}{\mu}e^{-\tau z},\qquad z>0, \end{equation*}

where \(\tau=\dfrac{1}{\lambda}+\dfrac{1}{\mu}=\dfrac{\lambda+\mu}{\lambda\mu}\) and \(\text{exponential}(\beta)\) carries the Section 3.3 pdf \(\beta^{-1}e^{-x/\beta}\), \(x>0\).

Indeed \(\{Z>z,\ W=1\}=\{z<X<Y\}\) (ties have probability \(0\)), so by independence

\begin{equation*} \begin{aligned} P(Z>z,\,W=1) &=\int_z^\infty \frac{1}{\lambda}e^{-x/\lambda}\,P(Y>x)\,dx\\ &=\int_z^\infty \frac{1}{\lambda}e^{-x/\lambda}e^{-x/\mu}\,dx =\frac{1}{\lambda\tau}e^{-\tau z}, \end{aligned} \end{equation*}

and differentiating in \(-z\) gives the stated \(f_{Z,W}(z,1)\); interchanging the roles of \(X\) and \(Y\) gives \(f_{Z,W}(z,0)\).

Summing over \(w\) recovers the marginals: with \(z=0\) above,

\begin{equation*} P(W=1)=\frac{1}{\lambda\tau}=\frac{\mu}{\lambda+\mu}, \qquad P(W=0)=\frac{\lambda}{\lambda+\mu}, \end{equation*}

and

\begin{equation*} P(Z>z)=P(X>z)P(Y>z)=e^{-z/\lambda}e^{-z/\mu}=e^{-\tau z}, \end{equation*}

so \(Z\sim\text{exponential}(1/\tau)=\text{exponential}\bigl(\lambda\mu/(\lambda+\mu)\bigr)\) and \(W\sim\text{Bernoulli}\bigl(\mu/(\lambda+\mu)\bigr)\).

(b) The joint distribution found in (a) already factors:

\begin{equation*} f_{Z,W}(z,1)=\frac{1}{\lambda}e^{-\tau z} =\underbrace{\frac{\mu}{\lambda+\mu}}_{P(W=1)}\cdot \underbrace{\tau e^{-\tau z}}_{f_Z(z)}, \end{equation*}

and symmetrically \(f_{Z,W}(z,0)=\frac{\lambda}{\lambda+\mu}\cdot\tau e^{-\tau z}\). Equivalently, in the form of the hint, for \(i=0,1\),

\begin{equation*} P(Z\le z\mid W=i)=\frac{P(Z\le z,\,W=i)}{P(W=i)} =1-e^{-\tau z}=P(Z\le z), \end{equation*}

the conditional cdf of \(Z\) not depending on \(i\). Hence \(Z\) and \(W\) are independent by Definition 4.2.5.

\(U \sim \mathrm{n}(\mu+\gamma,\,2\sigma^2)\) and \(V \sim \mathrm{n}(\mu-\gamma,\,2\sigma^2)\), and they are independent.

Standardize: \(X^{*}=(X-\mu)/\sigma\) and \(Y^{*}=(Y-\gamma)/\sigma\) are independent (Theorem 4.3.5, applied to the functions \(g(x)=(x-\mu)/\sigma\) and \(h(y)=(y-\gamma)/\sigma\) of the independent \(X\) and \(Y\)) standard normal variables. Example 4.3.4 shows that

\begin{equation*} S=X^{*}+Y^{*}\quad\text{and}\quad D=X^{*}-Y^{*} \end{equation*}

are independent, with \(S\sim \mathrm{n}(0,2)\) and \(D\sim \mathrm{n}(0,2)\). Since \(X=\sigma X^{*}+\mu\) and \(Y=\sigma Y^{*}+\gamma\),

\begin{equation*} U=\sigma S+(\mu+\gamma),\qquad V=\sigma D+(\mu-\gamma), \end{equation*}

so \(U\) is a function of \(S\) alone and \(V\) a function of \(D\) alone; \(U\) and \(V\) are independent by Theorem 4.3.5 again. Finally a location-scale change of a normal is normal (Corollary 4.6.10 with \(n=1\)), giving

\begin{equation*} U\sim \mathrm{n}\bigl(\mu+\gamma,\,2\sigma^{2}\bigr), \qquad V\sim \mathrm{n}\bigl(\mu-\gamma,\,2\sigma^{2}\bigr). \end{equation*}

Method (2): directly, \(u=x+y\), \(v=x-y\) inverts to \(x=(u+v)/2\), \(y=(u-v)/2\) on all of \(\mathbb{R}^2\) with \(J=-\tfrac12\), so by (4.3.2)

\begin{equation*} \begin{aligned} f_{U,V}(u,v) &=\frac{1}{2\pi\sigma^{2}}\exp\!\left(-\frac{\bigl(\frac{u+v}{2}-\mu\bigr)^{2} +\bigl(\frac{u-v}{2}-\gamma\bigr)^{2}}{2\sigma^{2}}\right)\frac{1}{2}\\ &=\frac{1}{\sqrt{4\pi\sigma^{2}}}e^{-\frac{(u-\mu-\gamma)^{2}}{4\sigma^{2}}} \cdot\frac{1}{\sqrt{4\pi\sigma^{2}}}e^{-\frac{(v-\mu+\gamma)^{2}}{4\sigma^{2}}}, \end{aligned} \end{equation*}

the cross terms in \(u v\) cancelling exactly because \(\operatorname{Var}X=\operatorname{Var}Y\). Lemma 4.2.7 then gives independence and identifies both marginals.

(a) \(X/(X+Y)\) is Cauchy with location \(\tfrac12\) and scale \(\tfrac12\), i.e. it has pdf

\begin{equation*} f(t)=\frac{1}{\pi}\cdot\frac{1/2}{(1/2)^{2}+(t-1/2)^{2}} =\frac{2}{\pi\bigl(1+(2t-1)^{2}\bigr)},\qquad -\infty<t<\infty . \end{equation*}

Put \(S=X+Y\) and \(D=X-Y\). By Example 4.3.4, \(S\) and \(D\) are independent \(\mathrm{n}(0,2)\) variables, and \(X=(S+D)/2\), so

\begin{equation*} \frac{X}{X+Y}=\frac{(S+D)/2}{S}=\frac{1}{2}+\frac{1}{2}\cdot\frac{D}{S} . \end{equation*}

Now \(D/\sqrt2\) and \(S/\sqrt2\) are independent \(\mathrm{n}(0,1)\), so \(D/S\) is a ratio of independent standard normals and is standard Cauchy by Example 4.3.6. Applying Theorem 2.1.5 to the increasing map \(c\mapsto \tfrac12+\tfrac12 c\) shifts and scales the standard Cauchy pdf \(1/\bigl(\pi(1+c^{2})\bigr)\) into the display above.

(b) \(X/|Y|\) is also standard Cauchy. Let \(U=X/|Y|\) and \(V=|Y|\). As in Example 4.3.6 the map is two-to-one: with \(A_1=\{y>0\}\), \(A_2=\{y<0\}\) and \(A_0=\{y=0\}\) (which has probability \(0\)), the two inverses are

\begin{equation*} (x,y)=(uv,\;v)\quad\text{and}\quad (x,y)=(uv,\;-v), \end{equation*}

each with \(|J_i|=v\), and \(\mathcal{B}=\{(u,v):-\infty<u<\infty,\ v>0\}\). Then (4.3.6) gives

\begin{equation*} f_{U,V}(u,v)=2\cdot\frac{1}{2\pi}e^{-(uv)^{2}/2}e^{-v^{2}/2}\,v =\frac{v}{\pi}e^{-(u^{2}+1)v^{2}/2}, \end{equation*}

which is exactly the joint pdf obtained in Example 4.3.6. Hence, by the integration carried out there,

\begin{equation*} f_U(u)=\frac{1}{\pi(u^{2}+1)},\qquad -\infty<u<\infty . \end{equation*}

(c) Not surprising. Write \(X/Y=(X/|Y|)\,\operatorname{sign}(Y)\): conditionally on \(Y=y\), the variable \(X/y\) equals \(X/|y|\) when \(y>0\) and \(-X/|y|\) when \(y<0\), and \(-X\) has the same distribution as \(X\). So the conditional distribution of \(X/Y\) given \(Y=y\) equals that of \(X/|Y|\) given \(Y=y\) for every \(y\ne0\), and the two ratios have the same distribution. The general theorem is:

If \(X\) and \(Y\) are independent and the distribution of \(X\) is symmetric about \(0\) (that is, \(-X\) and \(X\) have the same distribution), and \(P(Y=0)=0\), then \(X/Y\) and \(X/|Y|\) have the same distribution.

(a) \(X/Y = \cot\theta\), because \(R > 0\) cancels. The function \(\cot\) has period \(\pi\) and is a decreasing bijection from \((0,\pi)\) onto \(\mathbb{R}\) (and likewise from \((\pi,2\pi)\)), so each of the two branches contributes equally and

\begin{equation*} f_{X/Y}(u) = 2\cdot\frac{1}{2\pi}\left|\frac{d}{du}\operatorname{arccot} u\right| = \frac{1}{\pi(1+u^{2})},\qquad -\infty<u<\infty, \end{equation*}

the Cauchy(0,1) pdf (3.3.19).

(b) (Part (b) needs the implicit hypothesis that \(R\) and \(\theta\) are independent; part (a) does not.) Since \(X^{2}+Y^{2}=R^{2}\) with \(R>0\),

\begin{equation*} \frac{2XY}{\sqrt{X^{2}+Y^{2}}} = \frac{2R^{2}\cos\theta\sin\theta}{R} = R\sin 2\theta . \end{equation*}

Put \(\psi = (2\theta - \pi/2) \bmod 2\pi\). Reduction mod \(2\pi\) of any fixed affine image \(a\theta+b\) with integer \(a\neq0\) sends \(\text{U}(0,2\pi)\) to \(\text{U}(0,2\pi)\), and \(\psi\) is a function of \(\theta\) alone, so \(\psi\sim\text{U}(0,2\pi)\) independent of \(R\). Since \(\sin2\theta = \cos\psi\),

\begin{equation*} \frac{2XY}{\sqrt{X^{2}+Y^{2}}} = R\cos\psi \;\stackrel{d}{=}\; R\cos\theta = X . \end{equation*}

Specialization: if \(X,Y\) are independent \(\text{n}(0,\sigma^{2})\), their joint pdf \((2\pi\sigma^{2})^{-1}e^{-(x^{2}+y^{2})/(2\sigma^{2})}\) is rotationally invariant, so in polar form \(R = \sqrt{X^{2}+Y^{2}}\) and \(\theta\) are independent with \(\theta\sim \text{U}(0,2\pi)\). Hence

\begin{equation*} \frac{2XY}{\sqrt{X^{2}+Y^{2}}} \sim \text{n}(0,\sigma^{2}). \end{equation*}

(a) \(\mathrm{E}Y = 1/2\), \(\operatorname{Var}Y = 5/12\), \(\operatorname{Cov}(X,Y) = 1/12\). By Theorem 4.4.3 and Theorem 4.4.7, using \(\mathrm{E}(Y|X)=X\) and \(\operatorname{Var}(Y|X)=X^{2}\),

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \mathrm{E}\big(\mathrm{E}(Y|X)\big) = \mathrm{E}X = \tfrac12,\\ \operatorname{Var}Y &= \mathrm{E}\big(\operatorname{Var}(Y|X)\big) + \operatorname{Var}\big(\mathrm{E}(Y|X)\big)\\ &= \mathrm{E}X^{2} + \operatorname{Var}X = \tfrac13 + \tfrac{1}{12} = \tfrac{5}{12},\\ \mathrm{E}(XY) &= \mathrm{E}\big(X\,\mathrm{E}(Y|X)\big) = \mathrm{E}X^{2} = \tfrac13, \end{aligned} \end{equation*}

so \(\operatorname{Cov}(X,Y) = \tfrac13 - \tfrac12\cdot\tfrac12 = \tfrac{1}{12}\).

(b) On \(0<x<1\) we have \(x>0\), so conditionally on \(X=x\) the variable \(Y/X = Y/x\) is a positive rescaling of an \(\text{n}(x,x^{2})\) variable:

\begin{equation*} \frac{Y}{X}\,\Big|\,X=x \;\sim\; \text{n}\!\left(\frac{x}{x},\frac{x^{2}}{x^{2}}\right) = \text{n}(1,1). \end{equation*}

The conditional distribution of \(Y/X\) given \(X=x\) does not depend on \(x\), hence \(Y/X\) is independent of \(X\) (and \(Y/X \sim \text{n}(1,1)\)).

(a) \(\mathrm{E}Y = n/2\) and \(\operatorname{Var}Y = n/6 + n^{2}/12\). By Theorems 4.4.3 and 4.4.7 with \(\mathrm{E}(Y|X)=nX\), \(\operatorname{Var}(Y|X)=nX(1-X)\),

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \mathrm{E}(nX) = \frac{n}{2},\\ \operatorname{Var}Y &= \mathrm{E}\big(nX(1-X)\big) + \operatorname{Var}(nX)\\ &= n\left(\tfrac12-\tfrac13\right) + \frac{n^{2}}{12} = \frac{n}{6}+\frac{n^{2}}{12}. \end{aligned} \end{equation*}

(b) The joint distribution is the mixed pdf-pmf obtained from \(f(x,y)=P(Y=y|X=x)f_{X}(x)\):

\begin{equation*} f(x,y) = \binom{n}{y}x^{y}(1-x)^{n-y}, \qquad 0<x<1,\; y=0,1,\dots,n . \end{equation*}

(c) \(Y\) is discrete uniform on \(\{0,1,\dots,n\}\): integrating out \(x\) against the beta kernel,

\begin{equation*} P(Y=y) = \binom{n}{y}\!\int_{0}^{1}\!\! x^{y}(1-x)^{n-y}dx = \binom{n}{y}\frac{y!\,(n-y)!}{(n+1)!} = \frac{1}{n+1} \end{equation*}

for \(y=0,1,\dots,n\).

(a) \(Y \sim \text{negative binomial}\big(\alpha,\,p=1/(1+\beta)\big)\), with \(\mathrm{E}Y=\alpha\beta\) and \(\operatorname{Var}Y=\alpha\beta(1+\beta)\). Integrating out \(\lambda\) against a gamma kernel,

\begin{equation*} \begin{aligned} P(Y=y) &= \int_{0}^{\infty}\frac{e^{-\lambda}\lambda^{y}}{y!}\, \frac{\lambda^{\alpha-1}e^{-\lambda/\beta}}{\Gamma(\alpha)\beta^{\alpha}}\,d\lambda\\ &= \frac{1}{y!\,\Gamma(\alpha)\beta^{\alpha}}\, \Gamma(y+\alpha)\left(\frac{\beta}{1+\beta}\right)^{y+\alpha}\\ &= \frac{\Gamma(y+\alpha)}{\Gamma(\alpha)\,y!} \left(\frac{1}{1+\beta}\right)^{\alpha} \left(\frac{\beta}{1+\beta}\right)^{y}, \end{aligned} \end{equation*}

for \(y=0,1,2,\dots\). With \(p=1/(1+\beta)\) this is \(\frac{\Gamma(y+\alpha)}{\Gamma(\alpha)y!}p^{\alpha}(1-p)^{y}\), and for integer \(\alpha\) the ratio of gamma functions is \(\binom{\alpha+y-1}{y}\), giving exactly the negative binomial pmf (3.2.10) with \(r=\alpha\). By Theorems 4.4.3 and 4.4.7,

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \mathrm{E}\Lambda = \alpha\beta,\\ \operatorname{Var}Y &= \mathrm{E}\Lambda + \operatorname{Var}\Lambda = \alpha\beta+\alpha\beta^{2} = \alpha\beta(1+\beta). \end{aligned} \end{equation*}

(b) Collapse the top two stages by the binomial-Poisson computation of Example 4.4.2, which gives \(Y|\Lambda \sim \text{Poisson}(p\Lambda)\). Since \(\Lambda\sim\text{gamma}(\alpha,\beta)\) and \(p>0\) is a scale factor, \(p\Lambda \sim \text{gamma}(\alpha,p\beta)\). Thus the three-stage model is the two-stage model

\begin{equation*} Y|\Lambda^{\prime} \sim \text{Poisson}(\Lambda^{\prime}),\qquad \Lambda^{\prime} \sim \text{gamma}(\alpha,p\beta), \end{equation*}

and part (a) gives the same marginal family,

\begin{equation*} P(Y=y) = \frac{\Gamma(y+\alpha)}{\Gamma(\alpha)\,y!} \left(\frac{1}{1+p\beta}\right)^{\alpha} \left(\frac{p\beta}{1+p\beta}\right)^{y}, \end{equation*}

i.e. negative binomial with \(r=\alpha\) and success probability \(1/(1+p\beta)\) — identical to (a) after the reparametrization \(\beta \mapsto p\beta\).

The mgf of \(H\) is \(\big(p/(1-(1-p)e^{t})\big)^{r}\) with \(r = -\lambda/\log p\), which is the \(\text{negative binomial}(r,p)\) mgf.

First, summing the series \(\sum_{x\ge1}z^{x}/x = -\log(1-z)\) at \(z=(1-p)e^{t}\),

\begin{equation*} M_{X}(t) = \frac{-1}{\log p}\sum_{x=1}^{\infty}\frac{\big((1-p)e^{t}\big)^{x}}{x} = \frac{\log\!\big(1-(1-p)e^{t}\big)}{\log p}, \end{equation*}

finite for \(t < -\log(1-p)\), so all the mgfs below exist in a neighbourhood of \(0\).

The \(X_{i}\) are independent of each other and of \(N\), so Theorem 4.6.7 gives \(\mathrm{E}(e^{tH}|N=n) = \big(M_{X}(t)\big)^{n}\) for every \(n\ge0\), and Theorem 4.4.3 then gives, with the \(\text{Poisson}(\lambda)\) mgf \(\mathrm{E}s^{N} = e^{\lambda(s-1)}\) at \(s = M_{X}(t) > 0\),

\begin{equation*} \begin{aligned} M_{H}(t) &= \mathrm{E}\Big(\big(M_{X}(t)\big)^{N}\Big) = \exp\big\{\lambda\big(M_{X}(t)-1\big)\big\}\\ &= \exp\left\{\frac{\lambda}{\log p}\log\!\big(1-(1-p)e^{t}\big) - \lambda\right\}\\ &= \exp\left\{-r\log\!\big(1-(1-p)e^{t}\big) + r\log p\right\}\\ &= \left(\frac{p}{1-(1-p)e^{t}}\right)^{r}, \end{aligned} \end{equation*}

using \(\lambda = -r\log p\). This is the mgf of the negative binomial pmf (3.2.10) with parameters \(r\) and \(p\), so by Theorem 2.3.11 \(H \sim \text{negative binomial}(r,p)\).

(a) Integrate out \(p\); the integrand is a \(\text{beta}(x+\alpha,\,n-x+\beta)\) kernel:

\begin{equation*} \begin{aligned} P(X=x) &= \int_{0}^{1}\binom{n}{x}p^{x}(1-p)^{n-x} \frac{p^{\alpha-1}(1-p)^{\beta-1}}{B(\alpha,\beta)}\,dp\\ &= \binom{n}{x}\frac{B(x+\alpha,\,n-x+\beta)}{B(\alpha,\beta)}\\ &= \binom{n}{x}\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)} \frac{\Gamma(x+\alpha)\Gamma(n-x+\beta)}{\Gamma(\alpha+\beta+n)}, \end{aligned} \end{equation*}

for \(x=0,1,\dots,n\).

(b) With the pmf (3.2.10), \(P(X=x|p) = \binom{r+x-1}{x}p^{r}(1-p)^{x}\) for \(x=0,1,2,\dots\), the same integration gives the beta-Pascal pmf

\begin{equation*} \begin{aligned} P(X=x) &= \binom{r+x-1}{x}\frac{B(r+\alpha,\,x+\beta)}{B(\alpha,\beta)}\\ &= \frac{\Gamma(r+x)}{\Gamma( r)\,x!}\, \frac{\Gamma(\alpha+\beta)\Gamma(r+\alpha)\Gamma(x+\beta)} {\Gamma(\alpha)\Gamma(\beta)\Gamma(r+\alpha+x+\beta)} . \end{aligned} \end{equation*}

For the moments, use \(\mathrm{E}(X|P) = r(1-P)/P\) and \(\operatorname{Var}(X|P) = r(1-P)/P^{2}\), together with the beta moment formula (valid whenever \(\alpha-k>0\))

\begin{equation*} \mathrm{E}\!\left[\frac{(1-P)^{j}}{P^{k}}\right] = \frac{B(\alpha-k,\beta+j)}{B(\alpha,\beta)} = \frac{\Gamma(\alpha-k)\Gamma(\beta+j)\Gamma(\alpha+\beta)} {\Gamma(\alpha)\Gamma(\beta)\Gamma(\alpha+\beta+j-k)} . \end{equation*}

The three instances needed are

\begin{equation*} \begin{aligned} (j,k)=(1,1):&\quad \frac{\beta}{\alpha-1},\\ (j,k)=(1,2):&\quad \frac{\beta(\alpha+\beta-1)}{(\alpha-1)(\alpha-2)},\\ (j,k)=(2,2):&\quad \frac{\beta(\beta+1)}{(\alpha-1)(\alpha-2)}. \end{aligned} \end{equation*}

(Check!) Hence by Theorem 4.4.3, for \(\alpha>1\),

\begin{equation*} \mathrm{E}X = r\,\mathrm{E}\!\left[\frac{1-P}{P}\right] = \frac{r\beta}{\alpha-1}, \end{equation*}

and by Theorem 4.4.7, for \(\alpha>2\),

\begin{equation*} \begin{aligned} \operatorname{Var}X &= r\,\mathrm{E}\!\left[\frac{1-P}{P^{2}}\right]

• r^{2}\left(\mathrm{E}\!\left[\frac{(1-P)^{2}}{P^{2}}\right]

• \frac{\beta^{2}}{(\alpha-1)^{2}}\right)\\ &= \frac{r\beta(\alpha+\beta-1)}{(\alpha-1)(\alpha-2)}

• r^{2}\,\frac{\beta(\alpha+\beta-1)}{(\alpha-1)^{2}(\alpha-2)}\\ &= \frac{r\beta(\alpha+\beta-1)(\alpha-1+r)}{(\alpha-1)^{2}(\alpha-2)}, \end{aligned} \end{equation*}

where the middle line used \(\beta(\beta+1)(\alpha-1)-\beta^{2}(\alpha-2) = \beta(\alpha+\beta-1)\).

(a) Apply Theorem 4.4.7 with \(\mathrm{E}(X|P) = nP\) and \(\operatorname{Var}(X|P) = nP(1-P)\), then substitute \(\mathrm{E}P^{2} = \operatorname{Var}P + (\mathrm{E}P)^{2}\):

\begin{equation*} \begin{aligned} \operatorname{Var}X &= \mathrm{E}\big(nP(1-P)\big) + \operatorname{Var}(nP)\\ &= n\mathrm{E}P - n\big(\operatorname{Var}P + (\mathrm{E}P)^{2}\big) + n^{2}\operatorname{Var}P\\ &= n\,\mathrm{E}P(1-\mathrm{E}P) + n(n-1)\operatorname{Var}P . \end{aligned} \end{equation*}

Here \(\mathrm{E}P = \alpha/(\alpha+\beta)\) and \(\operatorname{Var}P = \alpha\beta/[(\alpha+\beta)^{2}(\alpha+\beta+1)]\).

(b) Theorem 4.4.7 with \(\mathrm{E}(Y|\Lambda) = \operatorname{Var}(Y|\Lambda) = \Lambda\) gives

\begin{equation*} \operatorname{Var}Y = \mathrm{E}\Lambda + \operatorname{Var}\Lambda = \alpha\beta + \alpha\beta^{2}, \end{equation*}

and with \(\mu = \mathrm{E}\Lambda = \alpha\beta\) we have \(\alpha\beta^{2} = (\alpha\beta)^{2}/\alpha = \mu^{2}/\alpha\), so

\begin{equation*} \operatorname{Var}Y = \mu + \frac{1}{\alpha}\mu^{2}. \end{equation*}

The extra-Poisson variation is \(\operatorname{Var}\Lambda = \mu^{2}/\alpha\), the amount by which the marginal variance exceeds that of a \(\text{Poisson}(\mu)\) variable.

\(Y \sim \text{binomial}\!\left(n, \frac{\alpha}{\alpha+\beta}\right)\) exactly: the pairs \((X_i,P_i)\) are independent across \(i\), and each \(X_i\) is a marginally Bernoulli variable with

\begin{equation*} P(X_i = 1) = \mathrm{E}\big(P(X_i=1 \mid P_i)\big) = \mathrm{E}P_i = \frac{\alpha}{\alpha+\beta}, \end{equation*}

by Theorem 4.4.3 and the beta mean of Section 3.3.

(a) By Theorem 4.4.3,

\begin{equation*} \mathrm{E}Y = \sum_{i=1}^{n}\mathrm{E}\big(\mathrm{E}(X_i \mid P_i)\big) = \sum_{i=1}^{n}\mathrm{E}P_i = \frac{n\alpha}{\alpha+\beta}. \end{equation*}

(b) Write \(\mu = \alpha/(\alpha+\beta)\) and recall \(\operatorname{Var}P_i = \alpha\beta/\big[(\alpha+\beta)^2(\alpha+\beta+1)\big]\). Since \(\mathrm{E}P_i - \mathrm{E}P_i^2 = \mu - \mu^2 - \operatorname{Var}P_i\) and \(\mu-\mu^2 = \alpha\beta/(\alpha+\beta)^2\), the conditional variance identity (Theorem 4.4.7) gives

\begin{equation*} \begin{aligned} \operatorname{Var}X_i &= \mathrm{E}\big(P_i(1-P_i)\big) + \operatorname{Var}(P_i)\\ &= \frac{\alpha\beta}{(\alpha+\beta)^2} - \operatorname{Var}P_i + \operatorname{Var}P_i\\ &= \frac{\alpha\beta}{(\alpha+\beta)^2} = \mu(1-\mu). \end{aligned} \end{equation*}

The \(X_i\) are independent (they are functions of the independent pairs \((X_i,P_i)\)), so \(\operatorname{Var}Y = n\alpha\beta/(\alpha+\beta)^2 = n\mu(1-\mu)\), the \(\text{binomial}(n,\mu)\) variance.

(c) Now \(\mathrm{E}(X_i \mid P_i) = n_iP_i\) and \(\operatorname{Var}(X_i \mid P_i) = n_iP_i(1-P_i)\), so by Theorem 4.4.3,

\begin{equation*} \mathrm{E}Y = \sum_{i=1}^{k} n_i\,\mathrm{E}P_i = \frac{\alpha}{\alpha+\beta}\sum_{i=1}^{k} n_i , \end{equation*}

and by Theorem 4.4.7, using \(\mathrm{E}\big(P_i(1-P_i)\big) = \alpha\beta/\big[(\alpha+\beta)(\alpha+\beta+1)\big]\) (the calculation in (b)),

\begin{equation*} \begin{aligned} \operatorname{Var}X_i &= n_i\frac{\alpha\beta}{(\alpha+\beta)(\alpha+\beta+1)}

• n_i^2\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}\\ &= \frac{n_i\,\alpha\beta\,\big[(\alpha+\beta) + n_i\big]}{(\alpha+\beta)^2(\alpha+\beta+1)} . \end{aligned} \end{equation*}

The \(X_i\) are again independent, so \(\operatorname{Var}Y = \sum_{i=1}^{k}\operatorname{Var}X_i\).

(a) \(S = \sum_{i=1}^n X_i\) is a \(\text{binomial}(n,\tfrac12)\) mixture of shifted beta-binomials. Split each trial into its two mechanisms: let \(U_i \sim \text{Bernoulli}(P)\) record that the person discriminates and \(V_i \sim \text{Bernoulli}(1/2)\) record that a guess would be correct, all independent given \(P\), and set \(X_i = U_i + (1-U_i)V_i\), so that \(P(X_i=1\mid P) = P + \tfrac12(1-P) = \tfrac12(1+P) = c\). The number \(M = \sum_{i=1}^n V_i \sim \text{binomial}(n,\tfrac12)\) is independent of \(P\); the \(M\) trials with \(V_i=1\) are correct outright and the remaining \(n-M\) contribute \(\sum U_i\), so

\begin{equation*} S \mid M = m,\ P = p \ \sim\ m + \text{binomial}(n-m,\,p), \end{equation*}

and averaging over \(P \sim \text{beta}(a,b)\) (Example 4.4.6),

\begin{equation*} S \mid M = m \ \sim\ m + \text{beta-binomial}(n-m,\,a,\,b). \end{equation*}

Explicitly,

\begin{equation*} P(S=k) = \sum_{m=0}^{k} \binom{n}{m}\frac{1}{2^{\,n}}\, \binom{n-m}{k-m}\frac{B(a+k-m,\,b+n-k)}{B(a,b)} . \end{equation*}

That this is the stated model, and not merely a construction matching it in one coordinate, follows from \(\binom{n}{m}\binom{n-m}{k-m} = \binom{n}{k}\binom{k}{m}\):

\begin{equation*} \begin{aligned} \sum_{m=0}^{k}\binom{n}{m}\binom{n-m}{k-m}p^{k-m}(1-p)^{n-k} &= \binom{n}{k}(1-p)^{n-k}\sum_{m=0}^{k}\binom{k}{m}p^{k-m}\\ &= \binom{n}{k}(1+p)^{k}(1-p)^{n-k}, \end{aligned} \end{equation*}

which upon division by \(2^n\) is exactly \(\binom{n}{k}c^k(1-c)^{n-k}\) with \(c=(1+p)/2\).

(b) Write \(\mu = \mathrm{E}P = a/(a+b)\) and \(\sigma^2 = \operatorname{Var}P = ab/\big[(a+b)^2(a+b+1)\big]\). Since \(\mathrm{E}(S\mid P) = n(1+P)/2\), Theorem 4.4.3 gives

\begin{equation*} \mathrm{E}S = \frac{n}{2}(1+\mu) = \frac{n(2a+b)}{2(a+b)} . \end{equation*}

With \(\operatorname{Var}(S\mid P) = n\frac{1+P}{2}\cdot\frac{1-P}{2} = \frac{n}{4}(1-P^2)\), Theorem 4.4.7 gives

\begin{equation*} \begin{aligned} \operatorname{Var}S &= \frac{n}{4}\big(1-\mu^2-\sigma^2\big) + \frac{n^2}{4}\sigma^2\\ &= \frac{n\,b(2a+b)}{4(a+b)^2}

• \frac{n(n-1)\,ab}{4(a+b)^2(a+b+1)} , \end{aligned} \end{equation*}

using \(1-\mu^2 = b(2a+b)/(a+b)^2\).

(a) The hinted substitution collapses the integral to \(\Gamma(1-r)\). Here \(f(x) = x^{r-1}e^{-x/\lambda}/\big(\Gamma( r)\lambda^{r}\big)\), \(x>0\), is the \(\text{gamma}(r,\lambda)\) pdf of (3.3.6), and \(0<r<1\) throughout (a) and (b); at \(r=1\) the gamma is itself exponential and \(p_\lambda\) degenerates to the point mass at \(\nu=\lambda\). The map \(u = x/\nu - x/\lambda\) carries \(\nu \in (0,\lambda)\) decreasingly onto \(u \in (0,\infty)\), with

\begin{equation*} d u = -\frac{x}{\nu^{2}}\,d\nu, \qquad \lambda - \nu = \frac{u\,\nu\lambda}{x}, \qquad e^{-x/\nu} = e^{-u}e^{-x/\lambda}. \end{equation*}

Substituting these into the integrand and reversing the orientation,

\begin{equation*} \begin{aligned} \frac{1}{\nu}e^{-x/\nu}\frac{\nu^{r-1}}{(\lambda-\nu)^{r}}\,d\nu &= e^{-u}e^{-x/\lambda}\,\nu^{r-2} \Big(\frac{u\nu\lambda}{x}\Big)^{-r}\frac{\nu^{2}}{x}\,du\\ &= e^{-x/\lambda}\,x^{r-1}\lambda^{-r}\,u^{-r}e^{-u}\,du , \end{aligned} \end{equation*}

every power of \(\nu\) having cancelled. Hence

\begin{equation*} \begin{aligned} \int_0^{\lambda}\frac{1}{\nu}e^{-x/\nu}p_{\lambda}(\nu)\,d\nu &= \frac{x^{r-1}e^{-x/\lambda}\lambda^{-r}}{\Gamma( r)\Gamma(1-r)} \int_0^{\infty}u^{-r}e^{-u}\,du\\ &= \frac{x^{r-1}e^{-x/\lambda}}{\Gamma( r)\lambda^{r}} = f(x), \end{aligned} \end{equation*}

since \(\int_0^{\infty}u^{-r}e^{-u}du = \Gamma(1-r)\), which is finite precisely because \(r<1\).

(b) \(p_{\lambda} \ge 0\), and with \(\nu = \lambda t\),

\begin{equation*} \int_0^{\lambda}p_{\lambda}(\nu)\,d\nu = \frac{1}{\Gamma( r)\Gamma(1-r)}\int_0^1 t^{r-1}(1-t)^{-r}\,dt = \frac{B(r,1-r)}{\Gamma( r)\Gamma(1-r)} = 1, \end{equation*}

because \(B(r,1-r) = \Gamma( r)\Gamma(1-r)/\Gamma(1)\), the integral converging since both exponents \(r-1\) and \(-r\) exceed \(-1\) exactly when \(0<r<1\).

(c) A scale mixture of exponentials is strictly decreasing, while a \(\text{gamma}(r,\lambda)\) pdf with \(r>1\) increases near the origin. Precisely, suppose \(f(x) = g(x) := \int_0^{\infty}\frac{1}{\nu}e^{-x/\nu}q_{\lambda}(\nu)\,d\nu\) for some pdf \(q_\lambda\). On the one hand,

\begin{equation*} \frac{\partial}{\partial x}\log f(x) = \frac{r-1}{x} - \frac{1}{\lambda} > 0 \qquad \text{for } 0 < x < \lambda(r-1), \end{equation*}

an interval that is nonempty exactly when \(r>1\). On the other hand, differentiation under the integral sign is legitimate on \(x \ge x_0 > 0\) (there \(\nu^{-2}e^{-x/\nu} \le \big(2/(ex_0)\big)^2\), a constant, hence \(q_\lambda\)-integrable), so

\begin{equation*} g^{\prime}(x) = -\int_0^{\infty}\frac{1}{\nu^{2}}e^{-x/\nu}q_{\lambda}(\nu)\,d\nu < 0, \end{equation*}

giving \(\frac{\partial}{\partial x}\log g(x) < 0\) for every \(x>0\). The two statements contradict each other at any \(x \in \big(0,\lambda(r-1)\big)\), so no such \(q_\lambda\) exists when \(r>1\).

Both distributional claims are the computations carried out on pages 182–183 for the last coordinate, and the labelling of cells is arbitrary. Collapsing all cells but \(j\) into one makes \(X_j\) the number of successes in \(m\) independent trials of success probability \(p_j\), so \(X_j \sim \text{binomial}(m,p_j)\) (the marginalizing sum is the one evaluated there by Theorem 4.6.4). Given \(X_j = x_j\), the remaining coordinates are multinomial with \(m-x_j\) trials and cell probabilities \(p_\ell/(1-p_j)\), \(\ell \ne j\); taking the \(i\)-th marginal of that multinomial by the same argument,

\begin{equation*} X_i \mid X_j = x_j \ \sim\ \text{binomial}\!\left(m-x_j,\ \frac{p_i}{1-p_j}\right). \end{equation*}

For the covariance, condition on \(X_j\) (Theorem 4.4.3) and use \(\mathrm{E}X_j = mp_j\), \(\mathrm{E}X_j^2 = mp_j(1-p_j) + m^2p_j^2\):

\begin{equation*} \begin{aligned} \mathrm{E}(X_iX_j) &= \mathrm{E}\big(X_j\,\mathrm{E}(X_i \mid X_j)\big) = \frac{p_i}{1-p_j}\,\mathrm{E}\big(X_j(m-X_j)\big)\\ &= \frac{p_i}{1-p_j}\Big(m^2p_j - mp_j(1-p_j) - m^2p_j^2\Big)\\ &= \frac{p_i}{1-p_j}\,m(m-1)p_j(1-p_j) = m(m-1)p_ip_j . \end{aligned} \end{equation*}

Hence, by Theorem 4.5.3,

\begin{equation*} \operatorname{Cov}(X_i,X_j) = m(m-1)p_ip_j - (mp_i)(mp_j) = -mp_ip_j . \end{equation*}

(a) Integrating \(y\) first over \(0<y<1-x\), with \(y = (1-x)t\) and \(s = a+b+c\) throughout:

\begin{equation*} \begin{aligned} \int_0^{1-x} y^{b-1}(1-x-y)^{c-1}dy &= (1-x)^{b+c-1}\int_0^1 t^{b-1}(1-t)^{c-1}dt\\ &= (1-x)^{b+c-1}B(b,c), \end{aligned} \end{equation*}

so that \(\iint f = C\,B(b,c)B(a,b+c)\), and

\begin{equation*} C^{-1} = \frac{\Gamma(b)\Gamma( c)}{\Gamma(b+c)}\cdot \frac{\Gamma(a)\Gamma(b+c)}{\Gamma(s)} = \frac{\Gamma(a)\Gamma(b)\Gamma( c)}{\Gamma(a+b+c)} . \end{equation*}

(b) The inner integral just computed is the marginal:

\begin{equation*} f_X(x) = C\,B(b,c)\,x^{a-1}(1-x)^{b+c-1} = \frac{\Gamma(s)}{\Gamma(a)\Gamma(b+c)}x^{a-1}(1-x)^{b+c-1}, \end{equation*}

so \(X \sim \text{beta}(a, b+c)\), and by the symmetry of \(f\) in \((a,x) \leftrightarrow (b,y)\), \(Y \sim \text{beta}(b, a+c)\).

(c) Dividing,

\begin{equation*} f(y \mid x) = \frac{y^{b-1}(1-x-y)^{c-1}}{B(b,c)\,(1-x)^{b+c-1}}, \qquad 0 < y < 1-x . \end{equation*}

Put \(T = Y/(1-x)\), a linear map with Jacobian \(1-x\), giving

\begin{equation*} f_T(t) = \frac{(1-x)^{b-1}t^{b-1}(1-x)^{c-1}(1-t)^{c-1}(1-x)} {B(b,c)(1-x)^{b+c-1}} = \frac{t^{b-1}(1-t)^{c-1}}{B(b,c)}, \end{equation*}

for \(0<t<1\): that is, \(Y/(1-x) \sim \text{beta}(b,c)\), free of \(x\).

(d) By (c), \(\mathrm{E}(Y \mid X) = \dfrac{b}{b+c}(1-X)\), so by Theorem 4.4.3 and the \(\text{beta}(a,b+c)\) moments \(\mathrm{E}X = a/s\), \(\mathrm{E}X^2 = a(a+1)/\big[s(s+1)\big]\),

\begin{equation*} \begin{aligned} \mathrm{E}(XY) &= \frac{b}{b+c}\,\mathrm{E}\big(X - X^2\big) = \frac{b}{b+c}\cdot\frac{a(b+c)}{s(s+1)}\\ &= \frac{ab}{(a+b+c)(a+b+c+1)}, \end{aligned} \end{equation*}

using \(\mathrm{E}X - \mathrm{E}X^2 = a\big[(s+1)-(a+1)\big]/\big[s(s+1)\big]\). Since \(\mathrm{E}X\,\mathrm{E}Y = ab/s^2\), Theorem 4.5.3 gives

\begin{equation*} \operatorname{Cov}(X,Y) = ab\left(\frac{1}{s(s+1)} - \frac{1}{s^{2}}\right) = \frac{-ab}{(a+b+c)^{2}(a+b+c+1)} . \end{equation*}

\(\operatorname{Cov}(X,c) = 0\). Taking \(Y \equiv c\) gives \(\mathrm{E}Y = c\), so by Definition 4.5.1,

\begin{equation*} \operatorname{Cov}(X,c) = \mathrm{E}\big[(X-\mathrm{E}X)(c-c)\big] = \mathrm{E}[0] = 0, \end{equation*}

a vanishing covariance being exactly what uncorrelatedness asserts; the ratio \(\rho\) of Definition 4.5.2 is not itself defined here, since \(\operatorname{Var}c = 0\) sits in its denominator.

\begin{equation*} \rho_{XY,\,Y} = \frac{\mu_X\sigma_Y} {\sqrt{\sigma_X^2\sigma_Y^2 + \sigma_X^2\mu_Y^2 + \mu_X^2\sigma_Y^2}} . \end{equation*}

Independence lets every expectation factor (Theorem 4.2.10). For the covariance, by Theorem 4.5.3,

\begin{equation*} \begin{aligned} \operatorname{Cov}(XY,Y) &= \mathrm{E}(XY^2) - \mathrm{E}(XY)\,\mathrm{E}Y = \mu_X\,\mathrm{E}Y^2 - \mu_X\mu_Y^2\\ &= \mu_X\big(\sigma_Y^2 + \mu_Y^2\big) - \mu_X\mu_Y^2 = \mu_X\sigma_Y^2 , \end{aligned} \end{equation*}

and, since \(X^2\) and \(Y^2\) are also independent,

\begin{equation*} \begin{aligned} \operatorname{Var}(XY) &= \mathrm{E}X^2\,\mathrm{E}Y^2 - \mu_X^2\mu_Y^2\\ &= \big(\sigma_X^2+\mu_X^2\big)\big(\sigma_Y^2+\mu_Y^2\big) - \mu_X^2\mu_Y^2\\ &= \sigma_X^2\sigma_Y^2 + \sigma_X^2\mu_Y^2 + \mu_X^2\sigma_Y^2 . \end{aligned} \end{equation*}

Dividing by \(\sqrt{\operatorname{Var}(XY)\operatorname{Var}(Y)}\) cancels one factor of \(\sigma_Y\) and gives the displayed expression (Definition 4.5.2), valid whenever \(0 < \sigma_X^2,\sigma_Y^2 < \infty\) – by independence these second moments are all that \(\operatorname{Var}(XY)\) requires.

\(\mathrm{Cov}(X_1+X_2,X_2+X_3)=\sigma^2\) and \(\mathrm{Cov}(X_1+X_2,X_1-X_2)=0\); neither depends on \(\mu\).

Covariance is bilinear (expand \(\mathrm{E}[(\,\cdot-\mathrm{E}\,\cdot)(\,\cdot-\mathrm{E}\,\cdot)]\) termwise), and \(\mathrm{Cov}(X_i,X_j)=0\) for \(i\neq j\) by hypothesis, while \(\mathrm{Cov}(X_i,X_i)=\mathrm{Var}\,X_i=\sigma^2\). Hence

\begin{equation*} \begin{aligned} \mathrm{Cov}(X_1+X_2,\;X_2+X_3) &=\mathrm{Cov}(X_1,X_2)+\mathrm{Cov}(X_1,X_3)\\ &\quad+\mathrm{Cov}(X_2,X_2)+\mathrm{Cov}(X_2,X_3)\\ &=0+0+\sigma^2+0=\sigma^2, \end{aligned} \end{equation*}

and

\begin{equation*} \begin{aligned} \mathrm{Cov}(X_1+X_2,\;X_1-X_2) &=\mathrm{Var}\,X_1-\mathrm{Cov}(X_1,X_2)\\ &\quad+\mathrm{Cov}(X_2,X_1)-\mathrm{Var}\,X_2\\ &=\sigma^2-0+0-\sigma^2=0 . \end{aligned} \end{equation*}

Write \(\mu_i=\mathrm{E}X_i\) and expand the square. With \(S=\sum_{i=1}^n X_i\) we have \(\mathrm{E}S=\sum_i\mu_i\), so

\begin{equation*} \begin{aligned} \mathrm{Var}\,S &=\mathrm{E}\Big(\sum_{i=1}^{n}(X_i-\mu_i)\Big)^{2} =\mathrm{E}\sum_{i=1}^{n}\sum_{j=1}^{n}(X_i-\mu_i)(X_j-\mu_j)\\ &=\sum_{i=1}^{n}\sum_{j=1}^{n}\mathrm{Cov}(X_i,X_j), \end{aligned} \end{equation*}

the interchange of \(\mathrm{E}\) with the finite double sum being linearity of expectation (Theorem 4.2.2), and the last step Definition 4.5.1. Splitting the double sum into its diagonal and off-diagonal parts and using the symmetry \(\mathrm{Cov}(X_i,X_j)=\mathrm{Cov}(X_j,X_i)\),

\begin{equation*} \sum_{i=1}^{n}\sum_{j=1}^{n}\mathrm{Cov}(X_i,X_j) =\sum_{i=1}^{n}\mathrm{Var}\,X_i +2\!\!\sum_{1\le i<j\le n}\!\!\mathrm{Cov}(X_i,X_j). \end{equation*}

Everything falls out of one completion of the square in the exponent of Definition 4.5.10. Put

\begin{equation*} u=\frac{x-\mu_X}{\sigma_X},\qquad v=\frac{y-\mu_Y}{\sigma_Y}, \end{equation*}

so that the exponent of \(f(x,y)\) is \(-\tfrac{1}{2(1-\rho^2)}(u^2-2\rho uv+v^2)\). Since

\begin{equation*} u^{2}-2\rho uv+v^{2}=(v-\rho u)^{2}+(1-\rho^{2})u^{2}, \end{equation*}

and since \(\sigma_Y(v-\rho u)=y-\big[\mu_Y+\rho(\sigma_Y/\sigma_X)(x-\mu_X)\big]\), the pdf factors as

\begin{equation*} f(x,y)=g(x)\,h(y\mid x), \end{equation*}

where, with \(m(x)=\mu_Y+\rho(\sigma_Y/\sigma_X)(x-\mu_X)\),

\begin{equation*} \begin{aligned} g(x)&=\frac{1}{\sqrt{2\pi}\,\sigma_X}\,e^{-u^{2}/2},\\ h(y\mid x)&=\frac{1}{\sqrt{2\pi}\,\sigma_Y\sqrt{1-\rho^{2}}} \exp\!\left(-\frac{(y-m(x))^{2}}{2\sigma_Y^{2}(1-\rho^{2})}\right). \end{aligned} \end{equation*}

(a) For each fixed \(x\), \(h(\cdot\mid x)\) is the \(\mathrm{n}(m(x),\sigma_Y^2(1-\rho^2))\) pdf, hence integrates to \(1\) in \(y\). So \(f_X(x)=\int f(x,y)\,dy=g(x)\), the \(\mathrm{n}(\mu_X,\sigma_X^2)\) pdf. The exponent is symmetric under interchanging the roles of \((x,\mu_X,\sigma_X)\) and \((y,\mu_Y,\sigma_Y)\), so the same factorization with the roles reversed gives \(f_Y\) the \(\mathrm{n}(\mu_Y,\sigma_Y^2)\) pdf.

(b) By Definition 4.2.3 and part (a), \(f(y\mid x)=f(x,y)/f_X(x)=h(y\mid x)\), which is exactly the claimed normal pdf. (Note \(1-\rho^2>0\) since \(-1<\rho<1\).)

(c) Compute the mgf (for \(a=b=0\) read \(\mathrm{n}(0,0)\) as the point mass at \(0\)), using the tower property \(\mathrm{E}(\mathrm{E}(\cdot\mid X))=\mathrm{E}(\cdot)\) of Theorem 4.4.3. For \(t\in\mathbb{R}\), by (b),

\begin{equation*} \mathrm{E}\!\left(e^{tbY}\mid X\right) =\exp\!\Big(tb\,m(X)+\tfrac{1}{2}t^{2}b^{2}\sigma_Y^{2}(1-\rho^{2})\Big), \end{equation*}

by the \(\mathrm{n}(m(X),\sigma_Y^2(1-\rho^2))\) mgf. Writing \(c=a+b\rho\sigma_Y/\sigma_X\), so that \(taX+tb\,m(X)=tcX+tb\mu_Y-tb\rho(\sigma_Y/\sigma_X)\mu_X\),

\begin{equation*} \begin{aligned} M_{aX+bY}(t)&=\mathrm{E}\!\left[e^{taX}\,\mathrm{E}(e^{tbY}\mid X)\right]\\ &=e^{tb\mu_Y-tb\rho(\sigma_Y/\sigma_X)\mu_X+\frac{1}{2}t^{2}b^{2}\sigma_Y^{2}(1-\rho^{2})}\; \mathrm{E}\,e^{tcX}\\ &=e^{tb\mu_Y-tb\rho(\sigma_Y/\sigma_X)\mu_X+\frac{1}{2}t^{2}b^{2}\sigma_Y^{2}(1-\rho^{2})} e^{tc\mu_X+\frac{1}{2}t^{2}c^{2}\sigma_X^{2}}, \end{aligned} \end{equation*}

the last step by (a) and the normal mgf. The coefficient of \(t\) is

\begin{equation*} b\mu_Y-b\rho\frac{\sigma_Y}{\sigma_X}\mu_X+\Big(a+b\rho\frac{\sigma_Y}{\sigma_X}\Big)\mu_X =a\mu_X+b\mu_Y, \end{equation*}

and the coefficient of \(t^2/2\) is

\begin{equation*} \begin{aligned} b^{2}\sigma_Y^{2}(1-\rho^{2})+c^{2}\sigma_X^{2} &=b^{2}\sigma_Y^{2}-b^{2}\rho^{2}\sigma_Y^{2}\\ &\quad+a^{2}\sigma_X^{2}+2ab\rho\sigma_X\sigma_Y+b^{2}\rho^{2}\sigma_Y^{2}\\ &=a^{2}\sigma_X^{2}+b^{2}\sigma_Y^{2}+2ab\rho\sigma_X\sigma_Y . \end{aligned} \end{equation*}

Thus \(M_{aX+bY}\) is the mgf of the asserted normal distribution, and since mgfs exist in a neighbourhood of \(0\) here, Theorem 2.3.11 identifies the distribution.

(a) Immediate from linearity and \(\mathrm{E}Z_i=0\), \(\mathrm{Var}\,Z_i=1\), \(\mathrm{Cov}(Z_1,Z_2)=0\) (Theorem 4.5.5, since \(Z_1\perp Z_2\)):

\begin{equation*} \begin{aligned} \mathrm{E}X&=a_X\mathrm{E}Z_1+b_X\mathrm{E}Z_2+c_X=c_X,\\ \mathrm{Var}\,X&=a_X^2\mathrm{Var}\,Z_1+b_X^2\mathrm{Var}\,Z_2=a_X^2+b_X^2, \end{aligned} \end{equation*}

by Theorem 4.5.6 (independent case), and likewise for \(Y\). Bilinearity of covariance, plus the vanishing of the cross terms, gives

\begin{equation*} \begin{aligned} \mathrm{Cov}(X,Y)&=a_Xa_Y\mathrm{Var}\,Z_1+b_Xb_Y\mathrm{Var}\,Z_2\\ &\quad+(a_Xb_Y+b_Xa_Y)\mathrm{Cov}(Z_1,Z_2)\\ &=a_Xa_Y+b_Xb_Y . \end{aligned} \end{equation*}

(b) Substitute into (a). Writing \(p=\sqrt{(1+\rho)/2}\) and \(q=\sqrt{(1-\rho)/2}\), so \(p^2+q^2=1\) and \(p^2-q^2=\rho\),

\begin{equation*} \begin{aligned} \mathrm{Var}\,X&=\sigma_X^2(p^2+q^2)=\sigma_X^2,\qquad \mathrm{Var}\,Y=\sigma_Y^2(p^2+q^2)=\sigma_Y^2,\\ \mathrm{Cov}(X,Y)&=\sigma_X\sigma_Y\,p^2+(\sigma_Xq)(-\sigma_Yq) =\sigma_X\sigma_Y(p^2-q^2)=\rho\sigma_X\sigma_Y, \end{aligned} \end{equation*}

and \(\mathrm{E}X=c_X=\mu_X\), \(\mathrm{E}Y=c_Y=\mu_Y\). Hence \(\rho_{XY}=\rho\sigma_X\sigma_Y/(\sigma_X\sigma_Y)=\rho\).

(c) Take \(|\rho|<1\) (Definition 4.5.10 requires it; at \(\rho=\pm1\) the map below is singular and \((X,Y)\) is degenerate). With \(p,q\) as above the transformation

\begin{equation*} \begin{pmatrix}x\\y\end{pmatrix} =\begin{pmatrix}\sigma_Xp&\sigma_Xq\\ \sigma_Yp&-\sigma_Yq\end{pmatrix} \begin{pmatrix}z_1\\z_2\end{pmatrix} +\begin{pmatrix}\mu_X\\ \mu_Y\end{pmatrix} \end{equation*}

is one-to-one onto \(\mathbb{R}^2\) with determinant \(-2pq\,\sigma_X\sigma_Y\), and \(2pq=\sqrt{(1+\rho)(1-\rho)}=\sqrt{1-\rho^2}\). Inverting: with \(u=(x-\mu_X)/\sigma_X\) and \(v=(y-\mu_Y)/\sigma_Y\) we get \(u=pz_1+qz_2\), \(v=pz_1-qz_2\), hence \(z_1=(u+v)/(2p)\), \(z_2=(u-v)/(2q)\) and

\begin{equation*} \begin{aligned} z_1^2+z_2^2&=\frac{(u+v)^2}{2(1+\rho)}+\frac{(u-v)^2}{2(1-\rho)}\\ &=\frac{(1-\rho)(u+v)^2+(1+\rho)(u-v)^2}{2(1-\rho^2)}\\ &=\frac{2u^2+2v^2-4\rho uv}{2(1-\rho^2)} =\frac{u^2-2\rho uv+v^2}{1-\rho^2}. \end{aligned} \end{equation*}

Since \(Z_1\) and \(Z_2\) are independent, \(f_{Z_1,Z_2}(z_1,z_2)=(2\pi)^{-1}e^{-(z_1^2+z_2^2)/2}\) (Definition 4.2.5), and the bivariate change-of-variables formula (4.3.2), whose Jacobian here is the constant \(J=-(2pq\,\sigma_X\sigma_Y)^{-1}\), gives

\begin{equation*} f_{X,Y}(x,y)=\frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}} \exp\!\left(-\frac{u^2-2\rho uv+v^2}{2(1-\rho^2)}\right), \end{equation*}

which is exactly Definition 4.5.10 with parameters \(\mu_X,\mu_Y,\sigma_X^2,\sigma_Y^2,\rho\).

(d) Another solution: \(c_X=\mu_X\), \(c_Y=\mu_Y\) and

\begin{equation*} a_X=\sigma_X,\quad b_X=0,\quad a_Y=\rho\sigma_Y,\quad b_Y=\sigma_Y\sqrt{1-\rho^2}, \end{equation*}

for which \(a_X^2+b_X^2=\sigma_X^2\), \(a_Y^2+b_Y^2=\sigma_Y^2(\rho^2+1-\rho^2)=\sigma_Y^2\), and \(a_Xa_Y+b_Xb_Y=\rho\sigma_X\sigma_Y\). (Check!)

There are infinitely many. The last three equations say precisely that the plane vectors \(\mathbf{a}=(a_X,b_X)\) and \(\mathbf{b}=(a_Y,b_Y)\) have lengths \(\sigma_X,\sigma_Y\) and inner product \(\rho\sigma_X\sigma_Y\), i.e. \(\lVert\mathbf{a}\rVert=\sigma_X\), \(\lVert\mathbf{b}\rVert=\sigma_Y\) and the angle between them has cosine \(\rho\). Writing \(\mathbf{a}=\sigma_X(\cos\alpha,\sin\alpha)\) and \(\mathbf{b}=\sigma_Y(\cos\beta,\sin\beta)\), the constraints reduce to \(\cos(\alpha-\beta)=\rho\), so \(\alpha\in[0,2\pi)\) is free and \(\beta=\alpha\mp\arccos\rho\): a one-parameter family, doubled by the sign (the two branches coinciding only when \(\rho=\pm1\)).

\(Z=\mathrm{sgn}(Y)\,|X|\), which settles both parts.

Indeed \(P(XY=0)=0\), so with probability one: if \(Y>0\) then \(XY>0\) forces \(X>0\) and \(Z=X=|X|\), while \(XY<0\) forces \(X<0\) and \(Z=-X=|X|\); if \(Y<0\) the two cases give \(Z=X=-|X|\) and \(Z=-X=-|X|\) respectively.

(a) \(Z\sim\mathrm{n}(0,1)\). Since \(X\) and \(Y\) are independent, so are \(|X|\) and \(\mathrm{sgn}(Y)\), and \(P(Y>0)=P(Y<0)=\tfrac12\). For \(z>0\),

\begin{equation*} \begin{aligned} P(Z\le z)&=P(Y<0)+P(Y>0)\,P(|X|\le z)\\ &=\tfrac12+\tfrac12\big(2\Phi(z)-1\big)=\Phi(z), \end{aligned} \end{equation*}

and for \(z<0\),

\begin{equation*} P(Z\le z)=P(Y<0)\,P(|X|\ge -z) =\tfrac12\cdot2\big(1-\Phi(-z)\big)=\Phi(z), \end{equation*}

using the symmetry \(P(|X|\le t)=2\Phi(t)-1\) for \(t\ge0\). So \(Z\) has the standard normal cdf.

(b) \(ZY=\mathrm{sgn}(Y)|X|\,Y=|X|\,|Y|\ge0\) with probability one, so

\begin{equation*} P(Z>0,\;Y<0)=0 . \end{equation*}

But every bivariate normal pdf of Definition 4.5.10 is strictly positive on all of \(\mathbb{R}^2\) (its exponent is finite everywhere, and \(-1<\rho<1\) by definition), so it assigns strictly positive probability to the open quadrant \(\{z>0,\,y<0\}\). Hence \((Z,Y)\) cannot be bivariate normal, even though both marginals are \(\mathrm{n}(0,1)\).

(a) Fix \(y\) and read the exponent as a quadratic in \(x\). Collecting terms,

\begin{equation*} \begin{aligned} Ax^2y^2+x^2-2Bxy-2Cx &=(Ay^2+1)x^2-2(By+C)x\\ &=(Ay^2+1)\left(x-\frac{By+C}{Ay^2+1}\right)^{2}+k(y), \end{aligned} \end{equation*}

where \(k(y)\) does not involve \(x\). Hence, for fixed \(y\),

\begin{equation*} f(x\mid y)=\frac{f(x,y)}{f_Y(y)} \propto\exp\left\{-\frac{1}{2}\,\frac{\big(x-\frac{By+C}{Ay^2+1}\big)^{2}}{1/(Ay^2+1)}\right\} \end{equation*}

as a function of \(x\), which (the proportionality constant being whatever makes it integrate to \(1\)) is the

\begin{equation*} \mathrm{n}\!\left(\frac{By+C}{Ay^2+1},\;\frac{1}{Ay^2+1}\right) \end{equation*}

pdf, provided \(Ay^2+1>0\). The exponent is symmetric under \((x,C)\leftrightarrow(y,D)\), so the same computation gives

\begin{equation*} Y\mid X=x\;\sim\;\mathrm{n}\!\left(\frac{Bx+D}{Ax^2+1},\;\frac{1}{Ax^2+1}\right). \end{equation*}

(b) With \(A=1,B=0,C=D=8\) the modes of \(f\) are the minimizers of

\begin{equation*} h(x,y)=x^2y^2+x^2+y^2-16x-16y, \end{equation*}

and there are exactly two of them: \((4-\sqrt{15},\,4+\sqrt{15})\) and \((4+\sqrt{15},\,4-\sqrt{15})\).

Setting the partials to zero,

\begin{equation*} \tfrac{\partial h}{\partial x}=2x(y^2+1)-16=0,\qquad \tfrac{\partial h}{\partial y}=2y(x^2+1)-16=0, \end{equation*}

i.e. \(x(y^2+1)=8\) and \(y(x^2+1)=8\). Subtracting,

\begin{equation*} xy(y-x)-(y-x)=0\;\Longrightarrow\;(y-x)(xy-1)=0 . \end{equation*}

(i) \(xy=1\). Then \(8=x(y^2+1)=xy\cdot y+x=x+y\), so \(x+y=8\), \(xy=1\), and \(x,y\) are the roots \(4\pm\sqrt{15}\) of \(t^2-8t+1=0\). At such a point

\begin{equation*} h=(xy)^2+(x+y)^2-2xy-16(x+y)=1+64-2-128=-65 . \end{equation*}

The Hessian there has \(h_{xx}=2y^2+2>0\) and

\begin{equation*} \det=4(x^2+1)(y^2+1)-16(xy)^2=4\cdot 64-16=240>0, \end{equation*}

using \(x^2y^2+x^2+y^2+1=1+62+1=64\); so both points are strict local minima of \(h\).

(ii) \(y=x\). Then \(x(x^2+1)=8\), i.e. \(x^3+x-8=0\), which has the single real root \(x_0\approx1.8338\) (the cubic is strictly increasing). There \(h=x_0^4+2x_0^2-32x_0\approx-40.65\) and

\begin{equation*} \det=4(x_0^2+1)^2-16x_0^4\approx-104.8<0, \end{equation*}

so this critical point is a saddle, not a mode.

Finally \(h(x,y)\ge x^2+y^2-16x-16y=(x-8)^2+(y-8)^2-128\to\infty\) as \(\lVert(x,y)\rVert\to\infty\), so \(h\) attains a global minimum, which must be one of the three critical points; it is \(-65\), attained at the two points in (i). Thus \(f\propto e^{-h/2}\) has exactly two modes, and the distribution is bimodal.

(a) Integrate out, using that \(g_1,g_2\) are densities:

\begin{equation*} f_X(x)=\int_{-\infty}^{\infty}\!\!f(x,y)\,dy =af_1(x)\!\int\! g_1+(1-a)f_2(x)\!\int\! g_2, \end{equation*}

which is \(af_1(x)+(1-a)f_2(x)\); symmetrically \(f_Y(y)=ag_1(y)+(1-a)g_2(y)\).

(b) One algebraic identity does it. Using (a),

\begin{equation*} \begin{aligned} f(x,y)-f_X(x)f_Y(y) &=af_1g_1+(1-a)f_2g_2\\ &\quad-\big[af_1+(1-a)f_2\big]\big[ag_1+(1-a)g_2\big]\\ &=a(1-a)f_1g_1+a(1-a)f_2g_2\\ &\quad-a(1-a)f_1g_2-a(1-a)f_2g_1\\ &=a(1-a)\,[f_1(x)-f_2(x)]\,[g_1(y)-g_2(y)], \end{aligned} \end{equation*}

where all functions are evaluated at \(x\) or \(y\) as appropriate. By Definition 4.2.5, \(X\) and \(Y\) are independent if and only if \(f(x,y)=f_X(x)f_Y(y)\) for all \((x,y)\); since \(a(1-a)\neq0\), this holds exactly when \([f_1(x)-f_2(x)][g_1(y)-g_2(y)]=0\) for all \((x,y)\).

(c) From (a), \(\mathrm{E}X=a\mu_1+(1-a)\mu_2\) and \(\mathrm{E}Y=a\xi_1+(1-a)\xi_2\), while

\begin{equation*} \mathrm{E}XY=\iint xy\,f(x,y)\,dx\,dy=a\mu_1\xi_1+(1-a)\mu_2\xi_2 . \end{equation*}

Hence, by the identity of (b) with \((f_i,g_i)\) replaced by \((\mu_i,\xi_i)\),

\begin{equation*} \begin{aligned} \mathrm{Cov}(X,Y)&=a\mu_1\xi_1+(1-a)\mu_2\xi_2\\ &\quad-\big[a\mu_1+(1-a)\mu_2\big]\big[a\xi_1+(1-a)\xi_2\big]\\ &=a(1-a)(\mu_1-\mu_2)(\xi_1-\xi_2). \end{aligned} \end{equation*}

So: choose \(f_1\neq f_2\) with \(\mu_1=\mu_2\) (equal means, different densities) and any \(g_1\neq g_2\). Then the covariance vanishes while \([f_1-f_2][g_1-g_2]\not\equiv0\), so by (b) \(X\) and \(Y\) are dependent.

(d) Take \(a=\tfrac12\) throughout, \(g_1=\mathrm{binomial}(10,0.2)\) (mean \(\xi_1=2\)) and \(g_2=\mathrm{binomial}(10,0.8)\) (mean \(\xi_2=8\)).

(i) Independent: \(f_1=f_2=\mathrm{binomial}(10,0.5)\). Then \(f_1-f_2\equiv0\), so (b) gives independence (indeed \(f(x,y)=f_1(x)f_Y(y)\)).

(ii) Correlated: \(f_1=\mathrm{binomial}(10,0.2)\), \(f_2=\mathrm{binomial}(10,0.8)\), so \(\mu_1=2\), \(\mu_2=8\) and

\begin{equation*} \mathrm{Cov}(X,Y)=\tfrac12\cdot\tfrac12(2-8)(2-8)=9\neq0 . \end{equation*}

(iii) Uncorrelated but dependent: \(f_1=\mathrm{binomial}(4,1/2)\) and \(f_2=\mathrm{binomial}(8,1/4)\), which have the same mean \(\mu_1=\mu_2=2\) but are different pmfs (e.g. \(f_1(0)=1/16\), \(f_2(0)=(3/4)^8=6561/65536\)). Then

\begin{equation*} \mathrm{Cov}(X,Y)=\tfrac14(2-2)(2-8)=0, \end{equation*}

while \([f_1(x)-f_2(x)][g_1(y)-g_2(y)]\neq0\) at, say, \((x,y)=(0,0)\), so \(X\) and \(Y\) are dependent by (b).

Here \(\mu_X=\mu_Y=0\) and \(\sigma_X=\sigma_Y=1\) in Definition 4.5.10, so by its property (a) and the conditional distribution derived in Section 4.5,

\begin{equation*} X\sim \mathrm{n}(0,1),\qquad Y\mid X=x\sim \mathrm{n}\bigl(\rho x,\;1-\rho^{2}\bigr), \end{equation*}

and symmetrically \(Y\sim\mathrm{n}(0,1)\). Hence \(\operatorname{Var}X=\operatorname{Var}Y=1\), and by the tower property (Theorem 4.4.3),

\begin{equation*} \operatorname{E}XY=\operatorname{E}\bigl[X\operatorname{E}(Y\mid X)\bigr] =\operatorname{E}\bigl[\rho X^{2}\bigr]=\rho . \end{equation*}

Since \(\operatorname{E}X=\operatorname{E}Y=0\), \(\operatorname{Cov}(X,Y)=\rho\) and \(\operatorname{Corr}(X,Y)=\rho/(1\cdot 1)=\rho\).

For the squares, variance plus mean squared gives \(\operatorname{E}(Y^{2}\mid X=x)=(1-\rho^{2})+\rho^{2}x^{2}\), and the standard normal moments are \(\operatorname{E}X^{2}=1\), \(\operatorname{E}X^{4}=3\):

\begin{equation*} \begin{aligned} \operatorname{E}X^{2}Y^{2} &=\operatorname{E}\bigl[X^{2}\operatorname{E}(Y^{2}\mid X)\bigr]\\ &=\operatorname{E}\bigl[(1-\rho^{2})X^{2}+\rho^{2}X^{4}\bigr]\\ &=(1-\rho^{2})+3\rho^{2}=1+2\rho^{2}. \end{aligned} \end{equation*}

Therefore \(\operatorname{Cov}(X^{2},Y^{2})=(1+2\rho^{2})-(1)(1)=2\rho^{2}\), while

\begin{equation*} \operatorname{Var}X^{2}=\operatorname{E}X^{4}-(\operatorname{E}X^{2})^{2}=3-1=2 =\operatorname{Var}Y^{2}, \end{equation*}

so

\begin{equation*} \operatorname{Corr}(X^{2},Y^{2})=\frac{2\rho^{2}}{\sqrt{2}\sqrt{2}}=\rho^{2}. \end{equation*}

(a) For \(t>0\),

\begin{equation*} P(X/Y\le t)= \begin{cases} t/2, & 0<t\le 1,\\[2pt] 1-\dfrac{1}{2t}, & t>1. \end{cases} \end{equation*}

Indeed \(\{X/Y\le t\}=\{X\le tY\}\) is the part of the unit square above the line \(x=ty\): for \(t\le 1\) that line exits at \((t,1)\), cutting off the triangle \((0,0),(t,1),(0,1)\) of area \(t/2\); for \(t>1\) it exits at \((1,1/t)\) and the complement is the triangle \((0,0),(1,0),(1,1/t)\) of area \(1/(2t)\).

For \(XY\) the answer is \(1\) once \(t\ge 1\), and for \(0<t<1\) the complementary region \(\{xy>t\}\) lies between the hyperbola \(y=t/x\) and \(y=1\) for \(x\in(t,1)\):

\begin{equation*} P(XY>t)=\int_{t}^{1}\Bigl(1-\frac{t}{x}\Bigr)dx=1-t+t\log t , \end{equation*}

so \(P(XY\le t)=t-t\log t\) for \(0<t<1\) (and \(=1\) for \(t\ge 1\)).

(b) Condition on \(Z=z\) and use part (a): \(P(XY\le tz)=tz-tz\log(tz)\) when \(tz\le 1\), and \(=1\) otherwise. The two integrals needed are \(\int_{0}^{1}z\,dz=\tfrac12\) and \(\int_{0}^{1}z\log z\,dz=-\tfrac14\).

(i) \(0<t\le 1\). Then \(tz\le 1\) for every \(z\in(0,1)\), so

\begin{equation*} \begin{aligned} P(XY/Z\le t)&=\int_{0}^{1}\bigl[tz-tz\log t-tz\log z\bigr]dz\\ &=\frac{t}{2}-\frac{t\log t}{2}+\frac{t}{4} =\frac{3t}{4}-\frac{t}{2}\log t . \end{aligned} \end{equation*}

(ii) \(t>1\). Split at \(z=1/t\); on \(z>1/t\) the conditional probability is \(1\). Substituting \(u=tz\) in the first piece,

\begin{equation*} \begin{aligned} P(XY/Z\le t)&=\frac{1}{t}\int_{0}^{1}(u-u\log u)\,du+\Bigl(1-\frac1t\Bigr)\\ &=\frac{1}{t}\Bigl(\frac12+\frac14\Bigr)+1-\frac1t =1-\frac{1}{4t}. \end{aligned} \end{equation*}

The distance \(D\) has the Rayleigh density

\begin{equation*} f_{D}(d)=\frac{d}{2}\,e^{-d^{2}/4},\qquad d>0 , \end{equation*}

equivalently \(D^{2}\sim\text{exponential}(4)\) (mean \(4\) in the C&B scale parametrization).

Indeed, with the two impacts \((X_{1},Y_{1})\) and \((X_{2},Y_{2})\) built from four independent \(\mathrm{n}(0,1)\) variables, Corollary 4.6.10 (linear combinations of independent normals are normal) gives

\begin{equation*} X_{1}-X_{2}\sim\mathrm{n}(0,2),\qquad Y_{1}-Y_{2}\sim\mathrm{n}(0,2), \end{equation*}

and these are independent, being functions of disjoint sets of independent variables (Theorem 4.6.12). Writing \(Z_{1}=(X_{1}-X_{2})/\sqrt{2}\) and \(Z_{2}=(Y_{1}-Y_{2})/\sqrt{2}\), which are independent \(\mathrm{n}(0,1)\),

\begin{equation*} \begin{aligned} D^{2}&=(X_{1}-X_{2})^{2}+(Y_{1}-Y_{2})^{2}\\ &=2\bigl(Z_{1}^{2}+Z_{2}^{2}\bigr)\sim 2\chi^{2}_{2}, \end{aligned} \end{equation*}

by Lemma 5.3.2, parts (a) and (b). Since \(\chi^{2}_{2}=\text{gamma}(1,2)=\text{exponential}(2)\), multiplying by the scale factor \(2\) gives \(D^{2}\sim\text{exponential}(4)\). Hence for \(d>0\)

\begin{equation*} P(D>d)=P(D^{2}>d^{2})=e^{-d^{2}/4}, \end{equation*}

and differentiating \(F_{D}(d)=1-e^{-d^{2}/4}\) yields the stated pdf.

\begin{equation*} P(B^{2}\ge 4AC)=\frac{5}{36}+\frac{\log 2}{6}\approx 0.2544 . \end{equation*}

The roots are real exactly when the discriminant \(B^{2}-4AC\) is nonnegative. If \(X\sim\text{uniform}(0,1)\) then \(P(-\log X>s)=P(X<e^{-s})=e^{-s}\), so \(-\log A\) and \(-\log C\) are independent exponential\((1)\); their sum is gamma (Example 4.6.8),

\begin{equation*} T=-\log(AC)\sim\text{gamma}(2,1),\qquad P(T\ge s)=e^{-s}(1+s),\ s>0 . \end{equation*}

Condition on \(B=b\), \(0<b<1\), and note \(AC\le b^{2}/4\iff T\ge\log(4/b^{2})=:s_{b}>0\). Since \(e^{-s_{b}}=b^{2}/4\),

\begin{equation*} P\bigl(4AC\le b^{2}\bigr)=\frac{b^{2}}{4}\Bigl(1+2\log 2-2\log b\Bigr). \end{equation*}

Integrating over \(b\), with \(\int_{0}^{1}b^{2}db=\tfrac13\) and \(\int_{0}^{1}b^{2}\log b\,db=-\tfrac19\),

\begin{equation*} \begin{aligned} P(B^{2}\ge 4AC) &=\frac14\int_{0}^{1}\bigl[(1+2\log 2)b^{2}-2b^{2}\log b\bigr]db\\ &=\frac14\Bigl[\frac{1+2\log 2}{3}+\frac{2}{9}\Bigr]\\ &=\frac14\Bigl[\frac59+\frac{2\log 2}{3}\Bigr] =\frac{5}{36}+\frac{\log 2}{6}. \end{aligned} \end{equation*}

\begin{equation*} f_{U}(u)=\frac{(-\log u)^{\,n-1}}{(n-1)!},\qquad 0<u<1, \end{equation*}

where \(U=\prod_{i=1}^{n}X_{i}\).

Each \(-\log X_{i}\) is exponential\((1)\), since \(P(-\log X_{i}>s)=P(X_{i}<e^{-s})=e^{-s}\), and these are independent (Theorem 4.6.12), so by Example 4.6.8

\begin{equation*} T=-\log U=\sum_{i=1}^{n}(-\log X_{i})\sim\text{gamma}(n,1),\qquad f_{T}(t)=\frac{t^{\,n-1}e^{-t}}{\Gamma(n)} . \end{equation*}

The map \(u\mapsto t=-\log u\) is a decreasing one-to-one transformation of \((0,1)\) onto \((0,\infty)\) with \(|dt/du|=1/u\), so Theorem 2.1.5 gives, for \(0<u<1\),

\begin{equation*} f_{U}(u)=f_{T}(-\log u)\cdot\frac1u =\frac{(-\log u)^{\,n-1}\,e^{\log u}}{\Gamma(n)}\cdot\frac1u =\frac{(-\log u)^{\,n-1}}{(n-1)!} . \end{equation*}

\begin{equation*} f_{T}(t)=\frac{3}{\lambda}\bigl(1-e^{-t/\lambda}\bigr)^{2}e^{-t/\lambda},\qquad t>0, \end{equation*}

where \(T=\max\{X_{1},X_{2},X_{3}\}\) and exponential\((\lambda)\) carries C&B’s scale parametrization, \(f_{X}(x)=\lambda^{-1}e^{-x/\lambda}\), \(F_{X}(x)=1-e^{-x/\lambda}\).

The system functions past \(t\) unless all three components have failed, so independence gives directly

\begin{equation*} F_{T}(t)=P(X_{1}\le t,\,X_{2}\le t,\,X_{3}\le t)=\bigl[F_{X}(t)\bigr]^{3} =\bigl(1-e^{-t/\lambda}\bigr)^{3}, \end{equation*}

the cdf of the sample maximum; differentiating yields the pdf above.

(a) The pooled test is negative exactly when all \(k\) individuals are negative, so by independence

\begin{equation*} P(\text{pool positive})=1-(1-p)^{k}. \end{equation*}

(b) Write \(X=\sum_{j=1}^{m}X_{j}\), where \(X_{j}\) is the number of tests spent on group \(j\). Each \(X_{j}\) takes the value \(1\) with probability \((1-p)^{k}\) and \(k+1\) with probability \(1-(1-p)^{k}\), so

\begin{equation*} \begin{aligned} \operatorname{E}X_{j}&=(1-p)^{k}+(k+1)\bigl[1-(1-p)^{k}\bigr]\\ &=1+k\bigl[1-(1-p)^{k}\bigr]. \end{aligned} \end{equation*}

By linearity of expectation (Theorem 2.2.5, extended to random vectors in Section 4.2), and since \(m=N/k\),

\begin{equation*} \operatorname{E}X=m\Bigl(1+k\bigl[1-(1-p)^{k}\bigr]\Bigr) =N\Bigl(\frac1k+1-(1-p)^{k}\Bigr). \end{equation*}

(c) Plan (ii). Plan (i) costs \(N\) tests, so (ii) is preferred precisely when

\begin{equation*} \frac1k+1-(1-p)^{k}<1 \quad\Longleftrightarrow\quad (1-p)^{k}>\frac1k \quad\Longleftrightarrow\quad p<1-k^{-1/k}, \end{equation*}

and for any fixed \(k\ge 2\) the right-hand threshold is strictly positive, so the inequality holds for all \(p\) near \(0\). Quantitatively, \((1-p)^{k}=1-kp+O(p^{2})\) gives

\begin{equation*} \operatorname{E}X=N\Bigl(\frac1k+kp\Bigr)+O(Np^{2}), \end{equation*}

which tends to \(N/k\) as \(p\to 0\): pooling cuts the expected workload by roughly the factor \(k\).

(a) Directly from the definition, \(A_1 = \frac{1}{n}\sum_i x_i\) and

\begin{equation*} A_{-1} = \left[\frac{1}{n}\sum_i x_i^{-1}\right]^{-1} = \frac{n}{\sum_i (1/x_i)}, \end{equation*}

the arithmetic and harmonic means. For \(A_0\), apply l’Hopital’s rule to \(\log A_r = r^{-1}\log\left(\frac1n\sum_i x_i^{\,r}\right)\), whose numerator vanishes at \(r = 0\):

\begin{equation*} \begin{aligned} \lim_{r\to 0}\log A_r &= \lim_{r\to 0}\frac{\sum_i x_i^{\,r}\log x_i}{\sum_i x_i^{\,r}}\\ &= \frac{1}{n}\sum_i \log x_i, \end{aligned} \end{equation*}

so \(A_0 = \left(\prod_i x_i\right)^{1/n}\), the geometric mean.

(b)(i) \(A_r = \exp(\log A_r)\) and \(\exp\) is increasing, so \(A_r\) inherits the monotonicity of \(\log A_r\).

(ii) For \(r \neq 0\), differentiating \(\log A_r = r^{-1}\log\left(\frac1n\sum_i x_i^{\,r}\right)\) and using \(\frac{d}{dr}x_i^{\,r} = x_i^{\,r}\log x_i\),

\begin{equation*} \begin{aligned} \frac{d}{dr}\log A_r &= -\frac{1}{r^2}\log\left(\frac1n\sum_i x_i^{\,r}\right)

• \frac{1}{r}\cdot\frac{\sum_i x_i^{\,r}\log x_i}{\sum_i x_i^{\,r}}\\ &= \frac{1}{r^2}\left\{\frac{r\sum_i x_i^{\,r}\log x_i}{\sum_i x_i^{\,r}}

• \log\left(\frac1n\sum_i x_i^{\,r}\right)\right\}. \end{aligned} \end{equation*}

(iii) With \(a_i = x_i^{\,r}/\sum_j x_j^{\,r}\) we have \(\log a_i = r\log x_i - \log\sum_j x_j^{\,r}\), hence \(r\sum_i a_i\log x_i = \sum_i a_i\log a_i + \log\sum_j x_j^{\,r}\), and the braces become

\begin{equation*} \begin{aligned} &\sum_i a_i\log a_i + \log\sum_j x_j^{\,r}

• \log\sum_i x_i^{\,r} + \log n\\ &\qquad = \log(n) - \sum_i a_i\log(1/a_i). \end{aligned} \end{equation*}

Since \(a_i > 0\) and \(\sum_i a_i = 1\), let \(a\) be the random variable taking the value \(a_i\) with probability \(a_i\). Jensen’s Inequality (Theorem 4.7.7) applied to the concave function \(\log\) gives

\begin{equation*} \sum_i a_i\log\left(\frac{1}{a_i}\right) = \mathrm{E}\log\left(\frac1a\right) \le \log\,\mathrm{E}\left(\frac1a\right) = \log\sum_i a_i\frac{1}{a_i} = \log n, \end{equation*}

so the braces are nonnegative and \(\frac{d}{dr}\log A_r \ge 0\) for every \(r \ne 0\). Since \(A_r\) is continuous at \(r = 0\) by part (a), \(A_r\) is nondecreasing on \(-\infty < r < \infty\), and \(A_{-1} \le A_0 \le A_1\) reads

\begin{equation*} \begin{aligned} \frac{n}{\sum_i (1/x_i)} &\le \left(\prod_i x_i\right)^{1/n} \le \frac{1}{n}\sum_i x_i . \end{aligned} \end{equation*}

(a) Condition on \(X\) and use Theorem 4.4.3, noting that \(X\) is a constant given \(X\):

\begin{equation*} \begin{aligned} \mathrm{Cov}(X,\mathrm{E}(Y|X)) &= \mathrm{E}\big[X\,\mathrm{E}(Y|X)\big]

• \mathrm{E}X\,\mathrm{E}\big[\mathrm{E}(Y|X)\big]\\ &= \mathrm{E}\big[\mathrm{E}(XY|X)\big] - \mathrm{E}X\,\mathrm{E}Y\\ &= \mathrm{E}(XY) - \mathrm{E}X\,\mathrm{E}Y = \mathrm{Cov}(X,Y). \end{aligned} \end{equation*}

(Finite variances give \(\mathrm{E}|XY| < \infty\) by the Cauchy-Schwarz Inequality, Theorem 4.7.3, so the expectations exist as Theorem 4.4.3 requires.)

(b) By bilinearity of covariance and part (a),

\begin{equation*} \mathrm{Cov}\big(X, Y - \mathrm{E}(Y|X)\big) = \mathrm{Cov}(X,Y) - \mathrm{Cov}(X,\mathrm{E}(Y|X)) = 0 . \end{equation*}

(c) \(\mathrm{E}[Y - \mathrm{E}(Y|X)] = \mathrm{E}Y - \mathrm{E}Y = 0\), so the variance is the raw second moment, and conditioning on \(X\) turns it into a conditional variance:

\begin{equation*} \begin{aligned} \mathrm{Var}\big(Y - \mathrm{E}(Y|X)\big) &= \mathrm{E}\big[(Y - \mathrm{E}(Y|X))^2\big]\\ &= \mathrm{E}\Big[\mathrm{E}\big((Y - \mathrm{E}(Y|X))^2\,\big|\,X\big)\Big]\\ &= \mathrm{E}\big[\mathrm{Var}(Y|X)\big]. \end{aligned} \end{equation*}

Expand each term with the definition \(\mathrm{Cov}(X,Y|Z) = \mathrm{E}(XY|Z) - \mathrm{E}(X|Z)\mathrm{E}(Y|Z)\) and apply Theorem 4.4.3, whose hypothesis holds because finite variances give \(\mathrm{E}|XY| < \infty\) (Cauchy-Schwarz, Theorem 4.7.3):

\begin{equation*} \begin{aligned} \mathrm{E}\big(\mathrm{Cov}(X,Y|Z)\big) &= \mathrm{E}\big[\mathrm{E}(XY|Z)\big]

• \mathrm{E}\big[\mathrm{E}(X|Z)\mathrm{E}(Y|Z)\big]\\ &= \mathrm{E}(XY) - \mathrm{E}\big[\mathrm{E}(X|Z)\mathrm{E}(Y|Z)\big], \end{aligned} \end{equation*}

while, since \(\mathrm{E}[\mathrm{E}(X|Z)] = \mathrm{E}X\) and likewise for \(Y\),

\begin{equation*} \mathrm{Cov}\big(\mathrm{E}(X|Z),\mathrm{E}(Y|Z)\big) = \mathrm{E}\big[\mathrm{E}(X|Z)\mathrm{E}(Y|Z)\big] - \mathrm{E}X\,\mathrm{E}Y . \end{equation*}

Adding, the cross terms cancel and

\begin{equation*} \begin{aligned} &\mathrm{E}\big(\mathrm{Cov}(X,Y|Z)\big)

• \mathrm{Cov}\big(\mathrm{E}(X|Z),\mathrm{E}(Y|Z)\big)\\ &\qquad = \mathrm{E}(XY) - \mathrm{E}X\,\mathrm{E}Y, \end{aligned} \end{equation*}

which is \(\mathrm{Cov}(X,Y)\).

The three interpretations give \(\mathrm{n}(0,\tfrac12)\), the density \(|y|e^{-y^2}\), and no determinate answer at all.

(1) \((Y, Z_1)\) is a linear map of the independent pair \((X,Y)\), hence bivariate normal (Exercise 4.46), with means \(0\), \(\mathrm{Var}\,Y = 1\), \(\mathrm{Var}\,Z_1 = 2\), and \(\mathrm{Cov}(Y,Z_1) = \mathrm{Var}\,Y - \mathrm{Cov}(Y,X) = 1\), so \(\rho = 1/\sqrt2\). The conditional moments following Definition 4.5.10 give

\begin{equation*} \begin{aligned} \mathrm{E}(Y\,|\,Z_1 = 0) &= 0 + \tfrac{1}{2}(0 - 0) = 0,\\ \mathrm{Var}(Y\,|\,Z_1 = 0) &= 1\cdot\left(1 - \tfrac12\right) = \tfrac12, \end{aligned} \end{equation*}

so \(Y\,|\,Z_1 = 0 \sim \mathrm{n}(0,\tfrac12)\).

(2) Change variables from \((x,y)\) to \((y,z)\) by \(z = y/x\), that is \(x = y/z\), with \(|\partial x/\partial z| = |y|/z^2\) for fixed \(y\):

\begin{equation*} f_{Y,Z_2}(y,z) = \frac{1}{2\pi}\exp\left\{-\frac{y^2}{2}\left(\frac{1}{z^2} + 1\right)\right\} \frac{|y|}{z^2}. \end{equation*}

At \(z = 1\) this is \(\frac{1}{2\pi}|y|e^{-y^2}\), and \(\int_{-\infty}^{\infty}|y|e^{-y^2}\,dy = 1\), so \(f_{Z_2}(1) = 1/(2\pi)\) and

\begin{equation*} f(y\,|\,Z_2 = 1) = |y|e^{-y^2},\qquad -\infty < y < \infty, \end{equation*}

equivalently \(Y^2 \sim \mathrm{exponential}(1)\) with a symmetric random sign; in particular \(\mathrm{Var}(Y|Z_2=1) = 1 \ne \tfrac12\).

(3) \(Z_3\) is discrete with \(P(Z_3 = 1) = P(Y = X) = 0\), so there is no conditional density to normalize and the answer is not determined. In the sub sigma-algebra formulation of Miscellanea 4.9.3, \(\mathcal{G} = \sigma(Z_3) = \{\emptyset, \Omega, \{Y = X\}, \{Y \ne X\}\}\) contains only sets of probability \(0\) or \(1\), so the defining relation

\begin{equation*} \mathrm{E}\big[Y I_G\big] = \mathrm{E}\big[\mathrm{E}(Y|\mathcal{G})\,I_G\big], \qquad G \in \mathcal{G}, \end{equation*}

holds whatever value \(\mathrm{E}(Y|\mathcal{G})\) is assigned on \(\{Y = X\}\); every distribution is a version. The default version, inherited from the almost-trivial \(\mathcal{G}\), is the marginal \(\mathrm{n}(0,1)\) – different again from (1) and (2), which is the paradox.

(a) \(X_1|Z = 0 \sim \mathrm{gamma}(2,1)\), while \(X_1|X_2 = 1 \sim \mathrm{exponential}(1)\).

The second is immediate from independence. For the first, transform \((x_1,x_2) \mapsto (x_1, z)\) with \(z = (x_2-1)/x_1\), so \(x_2 = 1 + zx_1\) and \(|\partial x_2/\partial z| = x_1\):

\begin{equation*} f_{X_1,Z}(x_1,z) = e^{-x_1}e^{-(1+zx_1)}x_1, \qquad x_1 > 0,\ 1 + zx_1 > 0 . \end{equation*}

At \(z = 0\) this is \(e^{-1}x_1e^{-x_1}\), and \(\int_0^\infty x_1e^{-x_1}dx_1 = 1\), so

\begin{equation*} f(x_1|Z = 0) = x_1e^{-x_1},\qquad x_1 > 0, \end{equation*}

the gamma\((2,1)\) pdf. Conditioning on \(Z=0\) tilts the exponential toward larger \(X_1\); conditioning on \(X_2 = 1\) leaves it alone.

(b) Since \(x_1 > 0\), \(B_1 = \{1 - \varepsilon x_1 < x_2 < 1 + \varepsilon x_1\}\) is the wedge opening rightward from the vertex \((0,1)\) between the lines \(x_2 = 1 \pm \varepsilon x_1\); its vertical cross-section at \(x_1\) has length \(2\varepsilon x_1\), growing with \(x_1\). \(B_2\) is the horizontal strip of constant height \(2\varepsilon\) about \(x_2 = 1\). The wedge admits a wider band of \(x_2\) values the larger \(x_1\) is, so \(B_1\) is easier for large \(X_1\) to satisfy and is informative about \(X_1\); \(B_2\) involves \(X_2\) alone and \(X_1\) is independent of \(X_2\), so it says nothing about \(X_1\).

(c) \(B_2\) first: by independence, for every \(\varepsilon\),

\begin{equation*} P(X_1 \le x\,|\,B_2) = P(X_1 \le x) = 1 - e^{-x}, \end{equation*}

the exponential\((1)\) cdf, as in part (a). For \(B_1\), take \(\varepsilon < 1/x\) so that \(1 - \varepsilon t > 0\) on \(0 < t \le x\):

\begin{equation*} \begin{aligned} P\big(B_1 \cap \{X_1 \le x\}\big) &= \int_0^x e^{-t}\left[e^{-(1-\varepsilon t)} - e^{-(1+\varepsilon t)}\right]dt\\ &= 2e^{-1}\int_0^x e^{-t}\sinh(\varepsilon t)\,dt, \end{aligned} \end{equation*}

and, splitting at \(t = 1/\varepsilon\) where the wedge’s lower edge hits \(x_2 = 0\),

\begin{equation*} \begin{aligned} P(B_1) &= 2e^{-1}\int_0^{1/\varepsilon} e^{-t}\sinh(\varepsilon t)\,dt\\ &\quad + \int_{1/\varepsilon}^{\infty} e^{-t} \left[1 - e^{-(1+\varepsilon t)}\right]dt . \end{aligned} \end{equation*}

Divide numerator and denominator by \(\varepsilon\) and let \(\varepsilon \to 0\). The tail term is at most \(e^{-1/\varepsilon} = o(\varepsilon)\); on \(0 < t < 1/\varepsilon\) we have \(\sinh(\varepsilon t)/\varepsilon \le t\cosh(\varepsilon t) \le t\cosh(1)\), so the Dominated Convergence Theorem applies with dominating function \(\cosh(1)\,te^{-t}\), and \(\sinh(\varepsilon t)/\varepsilon \to t\). Hence

\begin{equation*} \begin{aligned} \lim_{\varepsilon \to 0} P(X_1 \le x\,|\,B_1) &= \frac{\int_0^x te^{-t}dt}{\int_0^\infty te^{-t}dt}\\ &= 1 - (1+x)e^{-x}, \end{aligned} \end{equation*}

the gamma\((2,1)\) cdf, again as in part (a).

Put \(\mu = \mathrm{E}X\) and \(h(x) = g(x) - (a + bx)\), so that \(h \ge 0\) with \(h(x) = 0\) only at \(x = \mu\), and, since the line is tangent at \(\mu\),

\begin{equation*} \mathrm{E}h(X) = \mathrm{E}g(X) - (a + b\mu) = \mathrm{E}g(X) - g(\mathrm{E}X). \end{equation*}

Since \(h \ge 0\) we always have \(\mathrm{E}g(X) \ge g(\mathrm{E}X)\), so it suffices to show that a nonnegative \(W = h(X)\) with \(\mathrm{E}W = 0\) has \(P(W = 0) = 1\). Chebychev’s Inequality (Theorem 3.6.1, with the nonnegative \(g(w) = w\) and \(r = 1/n\)) gives \(P(W \ge \frac1n) \le n\,\mathrm{E}W = 0\) for every \(n\), and

\begin{equation*} P(W > 0) = P\left(\bigcup_{n=1}^{\infty}\left\{W \ge \tfrac1n\right\}\right) \le \sum_{n=1}^{\infty} P\left(W \ge \tfrac1n\right) = 0 . \end{equation*}

So \(\mathrm{E}g(X) = g(\mathrm{E}X)\) forces \(P(h(X) = 0) = 1\), that is \(P(X = \mu) = 1\); contrapositively, \(P(X = \mathrm{E}X) < 1\) gives \(\mathrm{E}g(X) > g(\mathrm{E}X)\).

Greater than \(1\) (unless \(X \equiv 1\), in which case it equals \(1\)).

Write \(X = e^{Z}\). The function \(g(z) = e^{z}\) is convex, with \(g^{\prime\prime}(z) = e^{z} > 0\), so its tangent line at \(z = \mathrm{E}Z\) lies strictly below \(g\) everywhere else, and Jensen’s Inequality (Theorem 4.7.7) together with Exercise 4.62 gives

\begin{equation*} \mathrm{E}X = \mathrm{E}e^{Z} \ \ge\ e^{\mathrm{E}Z} = e^{0} = 1, \end{equation*}

with strict inequality unless \(P(Z = \mathrm{E}Z) = 1\), that is unless \(P(X = 1) = 1\).

(a) Square both sides and use \(ab \le |ab| = |a|\,|b|\):

\begin{equation*} \begin{aligned} (a+b)^2 &= a^2 + 2ab + b^2 \\ &\le |a|^2 + 2|a|\,|b| + |b|^2 \\ &= \bigl(|a| + |b|\bigr)^2 . \end{aligned} \end{equation*}

Both sides are nonnegative, so taking square roots (increasing on \([0,\infty)\)) gives \(|a+b| \le |a| + |b|\).

(b) Part (a) with \(a = X(\omega)\), \(b = Y(\omega)\) gives \(|X+Y| \le |X| + |Y|\) at every sample point, so monotonicity of bivariate expectation (Exercise 4.2(c)) and then its linearity (Exercise 4.2(a)) give

\begin{equation*} \begin{aligned} \mathrm{E}|X+Y| &\le \mathrm{E}\bigl(|X| + |Y|\bigr) \\ &= \mathrm{E}|X| + \mathrm{E}|Y| , \end{aligned} \end{equation*}

the right side being finite by hypothesis.

Split where \(h - \nu\) changes sign, \(\nu = \mathrm{E}h(X)\), instead of where \(X - \mathrm{E}X\) does. Since all three expectations exist, linearity (Theorem 2.2.5) gives, for every constant \(c\),

\begin{equation*} \mathrm{E}\bigl[(g(X)-c)(h(X)-\nu)\bigr] = \mathrm{E}(g(X)h(X)) - \nu\,\mathrm{E}g(X) , \end{equation*}

the terms carrying \(c\) summing to \(-c\,\mathrm{E}(h(X)-\nu) = 0\). So it suffices to produce a \(c = g(\mu)\) making \((g(X)-g(\mu))(h(X)-\nu)\) of one fixed sign pointwise.

(a) \(g\) nondecreasing, \(h\) nonincreasing. Put

\begin{equation*} A = \{x : h(x) > \nu\}, \qquad \mu = \sup A \quad (\sup \emptyset = -\infty). \end{equation*}

(i) \(\mu = +\infty\): every \(x\) lies below some point of \(A\), so \(h > \nu\) everywhere and \(\mathrm{E}(h(X)-\nu) > 0\), contradicting the definition of \(\nu\).

(ii) \(\mu = -\infty\): then \(A = \emptyset\), so \(h(X) \le \nu\) with \(\mathrm{E}h(X) = \nu\), forcing \(h(X) = \nu\) a.s. and (a) holds with equality.

(iii) \(\mu\) finite, \(c = g(\mu)\): for \(x < \mu\) some \(x^{\prime} \in A\) exceeds \(x\), so \(h(x) \ge h(x^{\prime}) > \nu\) while \(g(x) \le g(\mu)\); for \(x > \mu\), \(x \notin A\) gives \(h(x) \le \nu\) while \(g(x) \ge g(\mu)\); at \(x = \mu\) the first factor vanishes. In every case

\begin{equation*} \bigl(g(x)-g(\mu)\bigr)\bigl(h(x)-\nu\bigr) \le 0 , \end{equation*}

so taking expectations in the display above,

\begin{equation*} \mathrm{E}(g(X)h(X)) \le (\mathrm{E}g(X))(\mathrm{E}h(X)). \end{equation*}

(b) \(g\) and \(h\) both nondecreasing. Now put

\begin{equation*} A = \{x : h(x) > \nu\}, \qquad \mu = \inf A \quad (\inf \emptyset = +\infty), \end{equation*}

the degenerate cases going exactly as in (i) and (ii): \(\mu = -\infty\) forces \(h > \nu\) everywhere, and \(\mu = +\infty\) gives \(h(X) = \nu\) a.s. and equality. For finite \(\mu\) and \(c = g(\mu)\): if \(x < \mu\) then \(x \notin A\), so \(h(x) \le \nu\) and \(g(x) \le g(\mu)\), both factors \(\le 0\); if \(x > \mu\) some \(x^{\prime} \in A\) is below \(x\), so \(h(x) \ge h(x^{\prime}) > \nu\) and \(g(x) \ge g(\mu)\), both factors \(\ge 0\). Hence

\begin{equation*} \bigl(g(x)-g(\mu)\bigr)\bigl(h(x)-\nu\bigr) \ge 0 \end{equation*}

for all \(x\), and taking expectations gives

\begin{equation*} \mathrm{E}(g(X)h(X)) \ge (\mathrm{E}g(X))(\mathrm{E}h(X)). \end{equation*}

If \(g\) and \(h\) are both nonincreasing, apply this to \(-g\) and \(-h\), both nondecreasing: \((-g)(-h) = gh\) and \((\mathrm{E}(-g(X)))(\mathrm{E}(-h(X))) = (\mathrm{E}g(X))(\mathrm{E}h(X))\).

\(n = 299\).

Sampling with replacement makes the \(n\) indicators iid Bernoulli\((.01)\), so the number \(X\) of color-blind people in the sample is binomial\((n, .01)\) and

\begin{equation*} P(X \ge 1) = 1 - (.99)^n \ge .95 \iff (.99)^n \le .05 . \end{equation*}

Taking logarithms (and dividing by \(\log(.99) < 0\), which reverses the inequality),

\begin{equation*} n \ \ge\ \frac{\log(.05)}{\log(.99)} \;=\; 298.073\ldots , \end{equation*}

so the smallest admissible integer is \(n = 299\) (at \(n = 298\) the probability is only \(.94996\)).

(a) \(P(N = n) = \dfrac{1}{n(n-1)}\) for \(n = 2, 3, \ldots\), where \(N = \min\{j \ge 2 : X_j > X_1\}\).

The event \(\{N > n\}\) says that none of \(X_2, \ldots, X_n\) exceeds \(X_1\), i.e. that \(X_1\) is the largest of \(X_1, \ldots, X_n\). The \(X_i\) are iid and continuous, so ties have probability \(0\) and each of the \(n\) variables is equally likely to be the maximum; hence

\begin{equation*} P(N > n) = P\bigl(X_1 = \max\{X_1,\ldots,X_n\}\bigr) = \frac{1}{n}, \qquad n \ge 1 . \end{equation*}

Therefore, for \(n \ge 2\),

\begin{equation*} P(N = n) = P(N > n-1) - P(N > n) = \frac{1}{n-1} - \frac{1}{n} = \frac{1}{n(n-1)} . \end{equation*}

Method (2): condition on \(X_1 = x\) and substitute \(u = F(x)\) (Theorem 2.1.10),

\begin{equation*} P(N=n) = \int_{-\infty}^{\infty} F(x)^{n-2}\bigl(1-F(x)\bigr) f(x)\,dx = \int_0^1 u^{n-2}(1-u)\,du = \frac{1}{n(n-1)} . \end{equation*}

(b) Since \(N \ge 1\) is integer valued, \(\mathrm{E}N = \sum_{n=1}^{\infty} P(N \ge n) = \sum_{n=1}^{\infty} P(N > n-1)\), and by (a)

\begin{equation*} \mathrm{E}N = 1 + \sum_{n=2}^{\infty} \frac{1}{n-1} = 1 + \sum_{m=1}^{\infty} \frac{1}{m} = \infty , \end{equation*}

the harmonic series diverging.

\(\sum_{i=1}^n Y_i \sim \text{binomial}\bigl(n,\ 1 - F_X(\mu)\bigr)\).

Each \(Y_i\) takes only the values \(0\) and \(1\), with

\begin{equation*} P(Y_i = 1) = P(X_i > \mu) = 1 - F_X(\mu) \equiv p , \end{equation*}

so \(Y_i \sim\) Bernoulli\((p)\), identically for each \(i\). (Continuity of \(F_X\) gives \(P(X_i = \mu) = 0\), so the endpoint convention in the definition of \(Y_i\) is immaterial.) Each \(Y_i = I_{(\mu,\infty)}(X_i)\) is a function of \(X_i\) alone, so the \(Y_i\) are mutually independent by Theorem 4.6.12, and a sum of \(n\) iid Bernoulli\((p)\) variables is binomial\((n,p)\) (Example 5.2.12).

(a) Fix any \(k\) indices \(i_1 < \cdots < i_k\) and any \(x_1,\ldots,x_k \in \{0,1\}\), and put \(t = \sum_{j=1}^k x_j\). Conditionally on \(P = p\) the \(X\)s are iid Bernoulli\((p)\), so averaging the conditional pmf over the uniform\((0,1)\) marginal of \(P\),

\begin{equation*} \begin{aligned} P(X_{i_1} = x_1, \ldots, X_{i_k} = x_k) &= \mathrm{E}\Bigl[\textstyle\prod_{j=1}^{k} P^{x_j}(1-P)^{1-x_j}\Bigr]\\ &= \int_0^1 p^{t}(1-p)^{k-t}\,dp . \end{aligned} \end{equation*}

The integrand is the kernel of a beta\((t+1, k-t+1)\) pdf, so by the beta integral (Section 3.3)

\begin{equation*} \int_0^1 p^{t}(1-p)^{k-t}\,dp = \frac{\Gamma(t+1)\Gamma(k-t+1)}{\Gamma(k+2)} = \frac{t!\,(k-t)!}{(k+1)!} . \end{equation*}

This value does not depend on which \(k\) indices were chosen, and depends on \((x_1,\ldots,x_k)\) only through the sum \(t\), which is permutation invariant. Hence every size-\(k\) subset, in every order, has the same joint distribution: the \(X\)s are exchangeable.

(b) Take \(k = 1\) in (a): \(P(X_i = 1) = \int_0^1 p\,dp = \tfrac12\), so \(\prod_{i=1}^{n} P(X_i = x_i) = 2^{-n}\) for every \((x_1,\ldots,x_n)\). But the joint pmf from (a) with \(k=n\) is \(t!\,(n-t)!/(n+1)!\), which does vary with \(t\). Taking \(x_1 = \cdots = x_n = 1\) (so \(t = n\)),

\begin{equation*} P(X_1 = 1, \ldots, X_n = 1) = \frac{n!\,0!}{(n+1)!} = \frac{1}{n+1} \;\neq\; \frac{1}{2^{n}} = \prod_{i=1}^{n} P(X_i = 1) \end{equation*}

for every \(n \ge 2\), since \(2^{n} > n+1\) there. So the \(X\)s are exchangeable but not independent, hence not iid.

With \(T = X_1 + \cdots + X_n\), the map \(\bar{X} = T/n\) is strictly increasing with inverse \(g^{-1}(x) = nx\) and \(|\tfrac{d}{dx}g^{-1}(x)| = n\), so Theorem 2.1.5 gives the claim directly:

\begin{equation*} F_{\bar{X}}(x) = P\!\left(\frac{T}{n} \le x\right) = P(T \le nx) = F_T(nx), \end{equation*}

\begin{equation*} f_{\bar{X}}(x) = \frac{d}{dx} F_T(nx) = n\, f_T(nx) = n\, f_{X_1+\cdots+X_n}(nx). \end{equation*}

(Theorem 2.1.5 needs only continuity of \(f_T\) and of \(\tfrac{d}{dx}g^{-1}\), both immediate here.) No mgf enters, unlike the route through Theorem 5.2.7, which computes \(M_{\bar X}(t) = [M_X(t/n)]^n\) and so presumes \(M_X\) exists.

Apply (4.3.2) to \(f_{X,Y}(x,y) = f_X(x)f_Y(y)\) with an auxiliary \(W\), then integrate \(w\) out; throughout \(J\) is the determinant of the matrix of partials of \((x,y)\) with respect to \((z,w)\).

(a) \(Z = X - Y\), \(W = X\). The inverse is \(x = w\), \(y = w - z\), with

\begin{equation*} J = \begin{vmatrix} 0 & 1 \\ -1 & 1\end{vmatrix} = 1 , \end{equation*}

so \(f_{Z,W}(z,w) = f_X(w) f_Y(w-z)\) and

\begin{equation*} f_Z(z) = \int_{-\infty}^{\infty} f_X(w)\, f_Y(w - z)\,dw . \end{equation*}

(b) \(Z = XY\), \(W = X\). The inverse is \(x = w\), \(y = z/w\) (defined for \(w \neq 0\), and \(P(X=0)=0\) for a continuous \(X\)), with

\begin{equation*} J = \begin{vmatrix} 0 & 1 \\[2pt] 1/w & -z/w^{2}\end{vmatrix} = -\frac{1}{w}, \qquad |J| = \frac{1}{|w|}, \end{equation*}

so \(f_{Z,W}(z,w) = f_X(w) f_Y(z/w)/|w|\) and

\begin{equation*} f_Z(z) = \int_{-\infty}^{\infty} f_X(w)\, f_Y\!\left(\frac{z}{w}\right) \frac{1}{|w|}\,dw . \end{equation*}

(c) \(Z = X/Y\), \(W = Y\). The inverse is \(x = zw\), \(y = w\), with

\begin{equation*} J = \begin{vmatrix} w & z \\ 0 & 1\end{vmatrix} = w, \qquad |J| = |w| , \end{equation*}

so \(f_{Z,W}(z,w) = f_X(zw) f_Y(w)\,|w|\) and

\begin{equation*} f_Z(z) = \int_{-\infty}^{\infty} f_X(zw)\, f_Y(w)\,|w|\,dw . \end{equation*}

(a) With \(\Delta = \bigl(z^2+(\sigma+\tau)^2\bigr)\bigl(z^2+(\sigma-\tau)^2\bigr)\),

\begin{equation*} \begin{aligned} A &= \frac{2\tau^2 z}{\Delta}, &\qquad B &= \frac{\tau^2\,(z^2+\tau^2-\sigma^2)}{\Delta},\\ C &= \frac{2\sigma^2 z}{\Delta}, &\qquad D &= -\,\frac{\sigma^2\,(3z^2+\sigma^2-\tau^2)}{\Delta}. \end{aligned} \end{equation*}

To get them, abbreviate \(u(w) = 1+(w/\sigma)^2\) and \(v(w) = 1+((z-w)/\tau)^2\). The asserted identity is \(1/(uv) = (Aw+B)/u - (Cw+D)/v\), i.e., after multiplying by \(uv\), the polynomial identity in \(w\)

\begin{equation*} (Aw+B)\Bigl(1+\tfrac{w^2-2zw+z^2}{\tau^2}\Bigr)

• (Cw+D)\Bigl(1+\tfrac{w^2}{\sigma^2}\Bigr) = 1 . \end{equation*}

Matching coefficients of \(w^3, w^2, w^1, w^0\) gives

\begin{equation*} \begin{aligned} w^3:&\quad \tfrac{A}{\tau^2} - \tfrac{C}{\sigma^2} = 0, & w^2:&\quad \tfrac{B-2Az}{\tau^2} - \tfrac{D}{\sigma^2} = 0,\\ w^1:&\quad A\tfrac{\tau^2+z^2}{\tau^2} - \tfrac{2Bz}{\tau^2} - C = 0, & w^0:&\quad B\tfrac{\tau^2+z^2}{\tau^2} - D = 1 . \end{aligned} \end{equation*}

The first gives \(C = A\sigma^2/\tau^2\); substituting into the third gives \(AS = 2Bz\) with \(S = \tau^2+z^2-\sigma^2\). The fourth gives \(D = B(\tau^2+z^2)/\tau^2 - 1\), and feeding both into the second leaves

\begin{equation*} B\,\frac{S^2+4\sigma^2 z^2}{S} = \tau^2, \qquad\text{where}\quad S^2+4\sigma^2z^2 = \Delta , \end{equation*}

the last equality because \((\tau^2+z^2-\sigma^2)^2+4\sigma^2z^2 = z^4+2z^2(\sigma^2+\tau^2)+(\sigma^2-\tau^2)^2\), which is \(\Delta\). (Check!) Hence \(B = \tau^2 S/\Delta\), \(A = 2Bz/S = 2\tau^2 z/\Delta\), \(C = 2\sigma^2 z/\Delta\), and

\begin{equation*} D = \frac{(\tau^2+z^2)S - \Delta}{\Delta} = -\frac{\sigma^2(3z^2+\sigma^2-\tau^2)}{\Delta}, \end{equation*}

as claimed. Note for later that

\begin{equation*} A\sigma^2 = C\tau^2 = \frac{2\sigma^2\tau^2 z}{\Delta} \equiv K . \end{equation*}

(b) Rewrite the two pieces of (a) with their denominators cleared, and split the second numerator about \(w=z\): put

\begin{equation*} D^{\prime} = Cz + D = -\frac{\sigma^2(z^2+\sigma^2-\tau^2)}{\Delta} . \end{equation*}

Then the integrand of (5.2.4), apart from the factor \(1/(\pi^2\sigma\tau)\), equals

\begin{equation*} \frac{\sigma^2(Aw+B)}{\sigma^2+w^2}

• \frac{\tau^2\bigl(C(w-z)+D^{\prime}\bigr)}{\tau^2+(w-z)^2}, \end{equation*}

whose antiderivative, by the two stated integration facts, is

\begin{equation*} \begin{aligned} G(w) = \;& \frac{K}{2}\log\frac{\sigma^2+w^2}{\tau^2+(w-z)^2}

• B\sigma \arctan\frac{w}{\sigma}\\ & - D^{\prime}\tau \arctan\frac{w-z}{\tau} , \end{aligned} \end{equation*}

the two logarithms having been combined using \(A\sigma^2 = C\tau^2 = K\) – the delicate point flagged in the problem, since each diverges separately as \(w \to \pm\infty\) while their difference tends to \(\tfrac{K}{2}\log 1 = 0\). With \(\arctan(\pm\infty) = \pm\pi/2\),

\begin{equation*} \int_{-\infty}^{\infty}\!\! \frac{dw}{u(w)v(w)} = \lim_{w\to\infty} G(w) - \lim_{w\to-\infty} G(w) = \pi\bigl(B\sigma - D^{\prime}\tau\bigr). \end{equation*}

Finally

\begin{equation*} \begin{aligned} B\sigma - D^{\prime}\tau &= \frac{\sigma\tau}{\Delta}\Bigl[\tau(z^2+\tau^2-\sigma^2)

• \sigma(z^2+\sigma^2-\tau^2)\Bigr]\\ &= \frac{\sigma\tau(\sigma+\tau)\bigl(z^2+(\sigma-\tau)^2\bigr)}{\Delta} = \frac{\sigma\tau(\sigma+\tau)}{z^2+(\sigma+\tau)^2}, \end{aligned} \end{equation*}

the middle step being the factorization

\begin{equation*} \begin{aligned} z^2(\sigma+\tau) + (\sigma^3+\tau^3) - \sigma\tau(\sigma+\tau) &= (\sigma+\tau)\bigl[z^2+\sigma^2-\sigma\tau+\tau^2-\sigma\tau\bigr]\\ &= (\sigma+\tau)\bigl(z^2+(\sigma-\tau)^2\bigr), \end{aligned} \end{equation*}

after which the factor \(z^2+(\sigma-\tau)^2\) cancels against \(\Delta\). Therefore

\begin{equation*} f_Z(z) = \frac{1}{\pi^2\sigma\tau}\cdot \frac{\pi\,\sigma\tau(\sigma+\tau)}{z^2+(\sigma+\tau)^2} = \frac{\sigma+\tau}{\pi\bigl(z^2+(\sigma+\tau)^2\bigr)}, \end{equation*}

which is exactly

\begin{equation*} f_Z(z) = \frac{1}{\pi(\sigma+\tau)}\, \frac{1}{1+\bigl(z/(\sigma+\tau)\bigr)^2}, \end{equation*}

i.e. \(Z \sim\) Cauchy\((0,\sigma+\tau)\), verifying (5.2.5).

(a) Insert \(\pm\bar X\) and expand; the cross-term dies because \(\sum_i(X_i-\bar X)=0\):

\begin{equation*} \begin{aligned} \sum_{i=1}^n\sum_{j=1}^n (X_i-X_j)^2 &=\sum_{i,j}\bigl[(X_i-\bar X)-(X_j-\bar X)\bigr]^2\\ &=n\sum_i (X_i-\bar X)^2+n\sum_j (X_j-\bar X)^2\\ &\qquad-2\Bigl(\sum_i (X_i-\bar X)\Bigr)^2\\ &=2n(n-1)S^2 . \end{aligned} \end{equation*}

(b) Both \(S^2\) and the \(\theta_j\), \(j\ge 2\), are unchanged by a location shift, so take \(\theta_1=0\). Put \(T=\sum_i X_i^2\) and \(U=\sum_i X_i\), so \((n-1)S^2=T-U^2/n\). Since the \(X_i\) are iid with mean \(0\), every product whose expectation survives must pair the indices up:

\begin{equation*} \begin{aligned} \mathrm{E}T^2&=n\theta_4+n(n-1)\theta_2^2,\\ \mathrm{E}(TU^2)&=n\theta_4+n(n-1)\theta_2^2,\\ \mathrm{E}U^4&=n\theta_4+3n(n-1)\theta_2^2 \end{aligned} \end{equation*}

(in \(\mathrm{E}(TU^2)\) the terms \(\mathrm{E}X_i^2X_jX_k\) with \(j\ne k\) all vanish, since some index then carries an odd, unpaired power). Hence

\begin{equation*} \begin{aligned} \mathrm{E}\bigl[(n-1)S^2\bigr]^2 &=\mathrm{E}T^2-\tfrac{2}{n}\mathrm{E}(TU^2)+\tfrac{1}{n^2}\mathrm{E}U^4\\ &=\frac{(n-1)^2}{n}\,\theta_4+\frac{(n-1)(n^2-2n+3)}{n}\,\theta_2^2 . \end{aligned} \end{equation*}

Subtracting \(\bigl(\mathrm{E}(n-1)S^2\bigr)^2=(n-1)^2\theta_2^2\) (Theorem 5.2.6c) and dividing by \((n-1)^2\),

\begin{equation*} \operatorname{Var}S^2=\frac{\theta_4}{n}+\frac{(3-n)\theta_2^2}{n(n-1)} =\frac{1}{n}\left(\theta_4-\frac{n-3}{n-1}\theta_2^2\right). \end{equation*}

(c) \(\operatorname{Cov}(\bar X,S^2)=\theta_3/n\), zero if and only if the population third central moment vanishes. Still with \(\theta_1=0\),

\begin{equation*} \begin{aligned} \operatorname{Cov}(\bar X,S^2)&=\mathrm{E}(\bar X S^2) =\frac{1}{n(n-1)}\Bigl[\mathrm{E}(UT)-\tfrac{1}{n}\mathrm{E}U^3\Bigr]\\ &=\frac{1}{n(n-1)}\Bigl[n\theta_3-\theta_3\Bigr]=\frac{\theta_3}{n}, \end{aligned} \end{equation*}

using \(\mathrm{E}(UT)=\sum_i \mathrm{E}X_i^3=n\theta_3\) and \(\mathrm{E}U^3=n\theta_3\) (all mixed terms have an unpaired mean-zero factor). So \(\operatorname{Cov}(\bar X,S^2)=0\) exactly when \(\theta_3=\mathrm{E}(X_i-\theta_1)^3=0\).

Write \(A_n=\sum_{i\le n}a_i^2\), \(B_n=\sum_{i\le n}b_i^2\), \(C_n=\sum_{i\le n}a_ib_i\) and \(L_n=\sum_{i<j\le n}(a_ib_j-a_jb_i)^2\); the claim is \(A_nB_n-C_n^2=L_n\).

For \(n=2\),

\begin{equation*} (a_1^2+a_2^2)(b_1^2+b_2^2)-(a_1b_1+a_2b_2)^2=(a_1b_2-a_2b_1)^2 . \end{equation*}

Assume the identity at \(n\). Then

\begin{equation*} \begin{aligned} A_{n+1}B_{n+1}-C_{n+1}^2 &=(A_n+a_{n+1}^2)(B_n+b_{n+1}^2)-(C_n+a_{n+1}b_{n+1})^2\\ &=\bigl(A_nB_n-C_n^2\bigr)+a_{n+1}^2B_n+b_{n+1}^2A_n-2a_{n+1}b_{n+1}C_n\\ &=L_n+\sum_{i=1}^{n}\bigl(a_ib_{n+1}-a_{n+1}b_i\bigr)^2\;=\;L_{n+1}, \end{aligned} \end{equation*}

which completes the induction.

Now apply it with \(a_i=x_i-\bar x\), \(b_i=y_i-\bar y\). The sample correlation is \(r=C_n/\sqrt{A_nB_n}\), defined when \(A_nB_n>0\), and the identity gives

\begin{equation*} 1-r^2=\frac{A_nB_n-C_n^2}{A_nB_n}=\frac{\sum_{i<j}(a_ib_j-a_jb_i)^2}{A_nB_n}. \end{equation*}

Hence \(r^2=1\) if and only if \(a_ib_j=a_jb_i\) for every pair \(i<j\). Since \(A_n>0\) there is a \(k\) with \(a_k\ne 0\); setting \(c=b_k/a_k\), the vanishing of \(a_kb_i-a_ib_k\) gives \(b_i=ca_i\) for every \(i\), i.e.

\begin{equation*} y_i-\bar y=c\,(x_i-\bar x),\qquad i=1,\ldots,n, \end{equation*}

so all the points \((x_i,y_i)\) lie on the line through \((\bar x,\bar y)\) of slope \(c\). Conversely, if \(y_i=\alpha+\beta x_i\) for all \(i\) then \(b_i=\beta a_i\), every \(a_ib_j-a_jb_i\) vanishes, and \(r^2=1\).

(a) \(\theta_1=\mu\), \(\theta_2=\sigma^2\), \(\theta_3=0\), \(\theta_4=3\sigma^4\). The odd central moment vanishes by symmetry of the normal pdf about \(\mu\), and \(Z=(X_i-\mu)/\sigma\sim\mathrm{n}(0,1)\) has \(\mathrm{E}Z^4=3\,\mathrm{E}Z^2=3\) by the recursion \(\mathrm{E}Z^k=(k-1)\mathrm{E}Z^{k-2}\) (integrate \(z^{k-1}\cdot z\varphi(z)\) by parts, using \(\varphi^{\prime}(z)=-z\varphi(z)\)).

(b) By Exercise 5.8(b),

\begin{equation*} \begin{aligned} \operatorname{Var}S^2&=\frac{1}{n}\left(3\sigma^4-\frac{n-3}{n-1}\sigma^4\right) =\frac{\sigma^4}{n}\cdot\frac{3(n-1)-(n-3)}{n-1}\\ &=\frac{\sigma^4}{n}\cdot\frac{2n}{n-1} =\frac{2\sigma^4}{n-1}. \end{aligned} \end{equation*}

(c) By Theorem 5.3.1c, \((n-1)S^2/\sigma^2\sim\chi^2_{n-1}\), which has variance \(2(n-1)\). Therefore

\begin{equation*} \begin{aligned} \operatorname{Var}S^2 &=\frac{\sigma^4}{(n-1)^2}\operatorname{Var}\Bigl(\tfrac{(n-1)S^2}{\sigma^2}\Bigr)\\ &=\frac{\sigma^4\cdot 2(n-1)}{(n-1)^2}=\frac{2\sigma^4}{n-1}. \end{aligned} \end{equation*}

\(0\le\operatorname{Var}S=\mathrm{E}S^2-(\mathrm{E}S)^2=\sigma^2-(\mathrm{E}S)^2\) by Theorem 5.2.6c, and \(S\ge 0\), so \(\mathrm{E}S\le\sigma\).

Equality forces \(\operatorname{Var}S=0\), i.e. \(S\) — equivalently \(S^2\) — is degenerate. So it suffices to show that \(\sigma^2>0\) makes \(S^2\) non-degenerate. Pick support points \(a<b\) of the population (they exist, since \(\sigma^2>0\) rules out a point mass), and for \(\varepsilon\in(0,(b-a)/2)\) let \(I_a=(a-\varepsilon,a+\varepsilon)\), \(I_b=(b-\varepsilon,b+\varepsilon)\), so that \(p=P(X_1\in I_a)>0\) and \(q=P(X_1\in I_b)>0\). Using the representation of Exercise 5.8(a), on the event

\begin{equation*} A=\{X_1\in I_a,\ldots,X_n\in I_a\},\qquad P(A)=p^n>0, \end{equation*}

every one of the \(n(n-1)\) off-diagonal terms satisfies \((X_i-X_j)^2<4\varepsilon^2\), whence \(S^2<2\varepsilon^2\); while on

\begin{equation*} B=\{X_1\in I_a,\ X_2\in I_b,\ldots,X_n\in I_b\},\qquad P(B)=pq^{\,n-1}>0, \end{equation*}

the \(2(n-1)\) terms involving index \(1\) each exceed \((b-a-2\varepsilon)^2\), whence

\begin{equation*} S^2>\frac{2(n-1)(b-a-2\varepsilon)^2}{2n(n-1)}=\frac{(b-a-2\varepsilon)^2}{n}. \end{equation*}

As \(\varepsilon\downarrow 0\) the right-hand bound tends to \((b-a)^2/n>0\) while \(2\varepsilon^2\to 0\), so fix \(\varepsilon\) small enough that \(2\varepsilon^2<(b-a-2\varepsilon)^2/n\). Then \(S^2\) takes values in two disjoint intervals, each with positive probability, so \(\operatorname{Var}S>0\) and \(\mathrm{E}S<\sigma\).

\begin{equation*} \mathrm{E}Y_1=\sqrt{\frac{2}{\pi n}},\qquad \mathrm{E}Y_2=\sqrt{\frac{2}{\pi}},\qquad \mathrm{E}Y_1\le \mathrm{E}Y_2 . \end{equation*}

For \(Z\sim\mathrm{n}(0,\tau^2)\),

\begin{equation*} \mathrm{E}|Z|=2\int_0^\infty \frac{z}{\sqrt{2\pi}\,\tau}e^{-z^2/(2\tau^2)}\,dz =\tau\sqrt{\frac{2}{\pi}} . \end{equation*}

Apply this twice: \(\bar X\sim\mathrm{n}(0,1/n)\) by Theorem 5.3.1b, giving \(\mathrm{E}Y_1=\mathrm{E}|\bar X|=\sqrt{2/(\pi n)}\); and each \(X_i\sim\mathrm{n}(0,1)\), so \(\mathrm{E}Y_2=\mathrm{E}|X_1|=\sqrt{2/\pi}\).

The inequality \(\mathrm{E}Y_1\le\mathrm{E}Y_2\) is the triangle inequality \(|\sum_i X_i|\le\sum_i|X_i|\) in expectation, with equality only at \(n=1\).

\begin{equation*} g(S^2)=c\,S,\qquad c=\sqrt{\tfrac{n-1}{2}}\;\frac{\Gamma\bigl(\tfrac{n-1}{2}\bigr)}{\Gamma\bigl(\tfrac{n}{2}\bigr)} . \end{equation*}

By Theorem 5.3.1c, \(W=(n-1)S^2/\sigma^2\sim\chi^2_{n-1}\), and for \(W\sim\chi^2_p\) the moment \(\mathrm{E}W^{1/2}\) is obtained by recognizing a gamma kernel:

\begin{equation*} \begin{aligned} \mathrm{E}W^{1/2} &=\int_0^\infty \frac{w^{(p-1)/2}e^{-w/2}}{\Gamma(p/2)2^{p/2}}\,dw\\ &=\frac{\Gamma\bigl(\tfrac{p+1}{2}\bigr)2^{(p+1)/2}}{\Gamma(p/2)2^{p/2}} =\sqrt{2}\,\frac{\Gamma\bigl(\tfrac{p+1}{2}\bigr)}{\Gamma\bigl(\tfrac{p}{2}\bigr)} . \end{aligned} \end{equation*}

With \(p=n-1\) and \(S=\sigma\sqrt{W}/\sqrt{n-1}\),

\begin{equation*} \mathrm{E}S=\sigma\sqrt{\frac{2}{n-1}}\; \frac{\Gamma\bigl(\tfrac{n}{2}\bigr)}{\Gamma\bigl(\tfrac{n-1}{2}\bigr)}, \end{equation*}

so \(c=1/\bigl[\sqrt{2/(n-1)}\,\Gamma(\tfrac n2)/\Gamma(\tfrac{n-1}{2})\bigr]\) is the constant displayed above.

(a) Absorb the scale into the coefficients: with \(\tilde a_{ij}=a_{ij}\sigma_j\) and \(\tilde b_{rj}=b_{rj}\sigma_j\),

\begin{equation*} U_i=\sum_j a_{ij}X_j=\sum_j \tilde a_{ij}Z_j+\alpha_i,\qquad V_r=\sum_j b_{rj}X_j=\sum_j \tilde b_{rj}Z_j+\beta_r, \end{equation*}

where \(\alpha_i=\sum_j a_{ij}\mu_j\) and \(\beta_r=\sum_j b_{rj}\mu_j\) are constants. Covariance is unaffected by additive constants (Definition 4.5.1 centers each argument), so

\begin{equation*} \operatorname{Cov}(U_i,V_r) =\operatorname{Cov}\Bigl(\sum_j \tilde a_{ij}Z_j,\ \sum_j \tilde b_{rj}Z_j\Bigr). \end{equation*}

Hence if the left side is \(0\), the hypothesised standard-normal case applies to the coefficient arrays \(\tilde a,\tilde b\) and yields that \(\sum_j \tilde a_{ij}Z_j\) and \(\sum_j \tilde b_{rj}Z_j\) are independent. Independence is preserved by the shifts \(u\mapsto u+\alpha_i\) and \(v\mapsto v+\beta_r\) (Theorem 4.6.12, applied to the two independent variables), so \(U_i\) and \(V_r\) are independent. The converse direction of the lemma needs no reduction: independent random variables with finite variances have zero covariance (Theorem 4.5.5). Part (b) of the lemma transfers by the same substitution, applied to the two vectors.

(b) Bilinearity of covariance and independence of the \(X_j\) (so \(\operatorname{Cov}(X_j,X_l)=0\) for \(j\ne l\), by Theorem 4.5.5) give

\begin{equation*} \begin{aligned} \operatorname{Cov}\Bigl(\sum_{j}a_{ij}X_j,\ \sum_{l}b_{rl}X_l\Bigr) &=\sum_{j=1}^{n}\sum_{l=1}^{n}a_{ij}b_{rl}\operatorname{Cov}(X_j,X_l)\\ &=\sum_{j=1}^{n}a_{ij}b_{rj}\operatorname{Var}X_j =\sum_{j=1}^{n}a_{ij}b_{rj}\sigma_j^2 . \end{aligned} \end{equation*}

(a) Split off the last term of the sum:

\begin{equation*} \bar X_{n+1}=\frac{1}{n+1}\sum_{i=1}^{n+1}X_i =\frac{X_{n+1}+\sum_{i=1}^{n}X_i}{n+1} =\frac{X_{n+1}+n\bar X_n}{n+1}. \end{equation*}

(b) Use \(\sum_{i=1}^{m}(X_i-\bar X_m)^2=\sum_{i=1}^{m}X_i^2-m\bar X_m^2\) for \(m=n+1\) and \(m=n\) and subtract:

\begin{equation*} nS_{n+1}^2-(n-1)S_n^2 = X_{n+1}^2+n\bar X_n^2-(n+1)\bar X_{n+1}^2 . \end{equation*}

By part (a), \((n+1)\bar X_{n+1}^2=(X_{n+1}+n\bar X_n)^2/(n+1)\), so the right side equals

\begin{equation*} \begin{aligned} &\frac{(n+1)X_{n+1}^2+n(n+1)\bar X_n^2-(X_{n+1}+n\bar X_n)^2}{n+1}\\ &\qquad=\frac{nX_{n+1}^2-2nX_{n+1}\bar X_n+n\bar X_n^2}{n+1}\\ &\qquad=\frac{n}{n+1}\,(X_{n+1}-\bar X_n)^2 . \end{aligned} \end{equation*}

Standardize once: \(Z_i=(X_i-i)/i\) are iid \(n(0,1)\), i.e.

\begin{equation*} Z_1=X_1-1,\qquad Z_2=\frac{X_2-2}{2},\qquad Z_3=\frac{X_3-3}{3}. \end{equation*}

(a) By Lemma 5.3.2, \(Z_1^2+Z_2^2+Z_3^2\sim\chi^2_3\):

\begin{equation*} (X_1-1)^2+\left(\frac{X_2-2}{2}\right)^2+\left(\frac{X_3-3}{3}\right)^2 . \end{equation*}

(b) Abbreviate \(Q=Z_2^2+Z_3^2\), the \(\chi^2_2\) variable of part (a) with its first term dropped, so

\begin{equation*} Q=\left(\frac{X_2-2}{2}\right)^2+\left(\frac{X_3-3}{3}\right)^2 . \end{equation*}

Then \(Z_1\sim n(0,1)\) is independent of \(Q\sim\chi^2_2\), so by Definition 5.3.4

\begin{equation*} T=\frac{Z_1}{\sqrt{Q/2}}=\frac{X_1-1}{\sqrt{Q/2}}\sim t_2 . \end{equation*}

(c) Square the statistic in (b); by Theorem 5.3.8(b) it is \(F_{1,2}\):

\begin{equation*} T^2=\frac{(X_1-1)^2}{Q/2}\sim F_{1,2}. \end{equation*}

Throughout write \(X=(U/p)/(V/q)\) with \(U\sim\chi^2_p\), \(V\sim\chi^2_q\) independent, the reduction noted after Definition 5.3.6.

(a) Transform \(x=qu/(pv)\), \(w=v\), so \(u=(p/q)xw\), \(v=w\), with Jacobian \(|\partial(u,v)/\partial(x,w)|=(p/q)w\). Then

\begin{equation*} \begin{aligned} f_X(x)&=\int_0^\infty \frac{\left(\frac pq xw\right)^{p/2-1}e^{-\frac{p}{2q}xw}\,w^{q/2-1}e^{-w/2}} {\Gamma(\frac p2)\Gamma(\frac q2)2^{(p+q)/2}}\cdot\frac pq w\,dw\\[2pt] &=\frac{(p/q)^{p/2}x^{p/2-1}}{\Gamma(\frac p2)\Gamma(\frac q2)2^{(p+q)/2}} \int_0^\infty w^{\frac{p+q}{2}-1}e^{-\frac w2\left(1+\frac pq x\right)}dw . \end{aligned} \end{equation*}

The integrand is a \(\mathrm{gamma}\!\left(\frac{p+q}{2},\,\frac{2}{1+(p/q)x}\right)\) kernel, so the integral equals \(\Gamma\!\left(\frac{p+q}{2}\right)\left[\frac{2}{1+(p/q)x}\right]^{(p+q)/2}\), giving

\begin{equation*} f_X(x)=\frac{\Gamma\!\left(\frac{p+q}{2}\right)}{\Gamma(\frac p2)\Gamma(\frac q2)} \left(\frac pq\right)^{p/2} \frac{x^{p/2-1}}{[1+(p/q)x]^{(p+q)/2}},\qquad 0<x<\infty, \end{equation*}

which is (5.3.9).

(b) \(\mathrm{E}X=q/(q-2)\) for \(q>2\), and

\begin{equation*} \operatorname{Var}X=\frac{2q^2(p+q-2)}{p(q-2)^2(q-4)}\quad (q>4). \end{equation*}

Indeed, negative moments of \(V\sim\chi^2_q\) are

\begin{equation*} \mathrm{E}V^{-r}=\frac{1}{\Gamma(\frac q2)2^{q/2}}\int_0^\infty v^{\frac q2-r-1}e^{-v/2}dv =\frac{\Gamma(\frac q2-r)}{2^{r}\Gamma(\frac q2)},\qquad q>2r, \end{equation*}

so \(\mathrm{E}V^{-1}=1/(q-2)\) and \(\mathrm{E}V^{-2}=1/[(q-2)(q-4)]\); also \(\mathrm{E}U=p\) and \(\mathrm{E}U^2=p(p+2)\). By independence,

\begin{equation*} \begin{aligned} \mathrm{E}X&=\frac qp\,\mathrm{E}U\,\mathrm{E}V^{-1}=\frac{q}{q-2},\\[2pt] \mathrm{E}X^2&=\frac{q^2}{p^2}\,\mathrm{E}U^2\,\mathrm{E}V^{-2} =\frac{q^2(p+2)}{p(q-2)(q-4)} . \end{aligned} \end{equation*}

Subtracting,

\begin{equation*} \operatorname{Var}X=\frac{q^2\big[(p+2)(q-2)-p(q-4)\big]}{p(q-2)^2(q-4)} =\frac{2q^2(p+q-2)}{p(q-2)^2(q-4)} . \end{equation*}

(c) \(1/X=(V/q)/(U/p)\) is a ratio of independent scaled chi squareds with \(q\) and \(p\) degrees of freedom, so \(1/X\sim F_{q,p}\) by Definition 5.3.6. This is Theorem 5.3.8(a).

(d) Since \((p/q)X=U/V\),

\begin{equation*} \frac{(p/q)X}{1+(p/q)X}=\frac{U/V}{1+U/V}=\frac{U}{U+V}. \end{equation*}

Here \(U/2\sim\mathrm{gamma}(p/2,1)\) and \(V/2\sim\mathrm{gamma}(q/2,1)\) are independent, and \(U/(U+V)=(U/2)\big/[(U/2)+(V/2)]\), so Exercise 4.24 gives \(\mathrm{beta}(p/2,q/2)\), which is Theorem 5.3.8(c).

Write \(X=Z/\sqrt{V/p}\) with \(Z\sim n(0,1)\), \(V\sim\chi^2_p\) independent, as in (5.3.5).

(a) \(\mathrm{E}X=0\) if \(p>1\) and \(\operatorname{Var}X=p/(p-2)\) if \(p>2\), which is (5.3.7). By independence \(\mathrm{E}X=\mathrm{E}Z\cdot p^{1/2}\mathrm{E}V^{-1/2}\), and \(\mathrm{E}V^{-1/2}=\Gamma(\frac{p-1}{2})/[\sqrt2\,\Gamma(\frac p2)]<\infty\) exactly when \(p>1\) (the negative-moment formula of Exercise 5.17(b)); the factor \(\mathrm{E}Z=0\) then gives the mean. Likewise

\begin{equation*} \operatorname{Var}X=\mathrm{E}X^2=\mathrm{E}Z^2\cdot p\,\mathrm{E}V^{-1} =1\cdot\frac{p}{p-2},\qquad p>2 . \end{equation*}

(b) \(Z^2\sim\chi^2_1\) by Lemma 5.3.2(a), and it is independent of \(V\sim\chi^2_p\), so

\begin{equation*} X^2=\frac{Z^2/1}{V/p}\sim F_{1,p} \end{equation*}

by Definition 5.3.6.

(c) From (5.3.6),

\begin{equation*} f(x|p)=\frac{\Gamma\!\left(\frac{p+1}{2}\right)}{\Gamma(\frac p2)\sqrt{p}}\cdot\frac{1}{\sqrt{\pi}} \cdot\frac{1}{(1+x^2/p)^{(p+1)/2}} . \end{equation*}

The last factor tends to \(e^{-x^2/2}\), since \((1+x^2/p)^{(p+1)/2}=\big[(1+x^2/p)^{p}\big]^{1/2}(1+x^2/p)^{1/2}\to(e^{x^2})^{1/2}\cdot1\). For the first factor apply Stirling’s Formula \(\Gamma(a)\approx\sqrt{2\pi}\,a^{a-1/2}e^{-a}\) with \(a=\frac{p+1}{2}\) and \(a=\frac p2\):

\begin{equation*} \begin{aligned} \frac{\Gamma\!\left(\frac{p+1}{2}\right)}{\Gamma(\frac p2)} &\approx\frac{\left(\frac{p+1}{2}\right)^{p/2}e^{-(p+1)/2}}{\left(\frac p2\right)^{(p-1)/2}e^{-p/2}}\\ &=\left(\frac{p+1}{p}\right)^{p/2}\left(\frac p2\right)^{1/2}e^{-1/2} \;\sim\;e^{1/2}e^{-1/2}\sqrt{p/2}=\sqrt{p/2}, \end{aligned} \end{equation*}

using \((1+1/p)^{p/2}\to e^{1/2}\); so \(\Gamma(\frac{p+1}{2})/[\Gamma(\frac p2)\sqrt p]\to1/\sqrt2\). Multiplying the three limits,

\begin{equation*} f(x|p)\longrightarrow\frac{1}{\sqrt2}\cdot\frac{1}{\sqrt\pi}\cdot e^{-x^2/2} =\frac{1}{\sqrt{2\pi}}e^{-x^2/2}. \end{equation*}

(d) \(X_p^2\to\chi^2_1\) in distribution, equivalently \(F_{1,p}\to\chi^2_1\). (The argument uses (b) and (c); (a) only corroborates it, since \(\mathrm{E}X_p^2=p/(p-2)\to1=\mathrm{E}\chi^2_1\).) Write \(X_p\sim t_p\): part (c) gives \(X_p\to Z\sim n(0,1)\) in distribution, and \(\pm\sqrt t\) are continuity points of \(\Phi\), so for every \(t>0\)

\begin{equation*} P(X_p^2\le t)=P(-\sqrt t\le X_p\le\sqrt t) \longrightarrow\Phi(\sqrt t)-\Phi(-\sqrt t)=P(Z^2\le t), \end{equation*}

while \(P(X_p^2\le t)=0=P(Z^2\le t)\) for \(t<0\). Here \(X_p^2\sim F_{1,p}\) by (b) and \(Z^2\sim\chi^2_1\) by Lemma 5.3.2(a).

(e) \(qF_{q,p}\to\chi^2_q\) in distribution as \(p\to\infty\), part (d) being the case \(q=1\). Fix \(W\sim\chi^2_q\) and take \(V_p\sim\chi^2_p\) independent of \(W\); then \(qF_{q,p}\overset{d}{=}W/(V_p/p)\). Since \(V_p\) is a sum of \(p\) iid \(\chi^2_1\) variables of mean 1, the weak law (Theorem 5.5.2) gives \(V_p/p\to1\) in probability, hence \(p/V_p\to1\) in probability (Theorem 5.5.4, \(h(u)=1/u\) continuous at 1). Slutsky’s Theorem 5.5.17 applied to the constant sequence \(W\) then gives \(W\cdot(p/V_p)\to W\) in distribution.

(a) Couple: let \(W\sim\chi^2_q\) and \(Y\sim\chi^2_{p-q}\) be independent, so \(W+Y\sim\chi^2_p\) by Lemma 5.3.2(b) (\(p>q\), and \(p-q\ge1\) since degrees of freedom are integers). Since \(Y>0\) with probability 1, \(W+Y>W\) pointwise, hence for every \(a\)

\begin{equation*} P(\chi^2_p>a)=P(W+Y>a)\ge P(W>a)=P(\chi^2_q>a). \end{equation*}

The gap is exact: for every \(a>0\),

\begin{equation*} P(W+Y>a)-P(W>a)=P(W\le a,\ W+Y>a)\ge P(W\le \tfrac a2)P(Y>a)>0, \end{equation*}

both factors being positive because \(W\) and \(Y\) are independent with support \((0,\infty)\). So the inequality is strict at every \(a>0\).

(b) Fix \(\nu\) and \(k>k^{\prime}\) (integers). Write \(kF_{k,\nu}=\chi^2_k\big/(V/\nu)\) with \(V\sim\chi^2_\nu\) independent of the numerator, and use the same \(V\) for both. Conditioning on \(V\) and applying (a) pointwise, for any \(a>0\) and any \(v>0\),

\begin{equation*} P\!\left(\chi^2_k>\frac{av}{\nu}\right)>P\!\left(\chi^2_{k^{\prime}}>\frac{av}{\nu}\right), \end{equation*}

strictly, since \(av/\nu>0\). Taking expectations over \(V\),

\begin{equation*} P(kF_{k,\nu}>a)=\mathrm{E}\,P\!\left(\chi^2_k>\tfrac{aV}{\nu}\,\Big|\,V\right)

\mathrm{E}\,P\!\left(\chi^2_{k^{\prime}}>\tfrac{aV}{\nu}\,\Big|\,V\right)=P(k^{\prime}F_{k^{\prime},\nu}>a), \end{equation*}

so \(kF_{k,\nu}\) is stochastically increasing in \(k\).

(c) The cutoff points are defined by \(P(F_{k,\nu}>F_{\alpha,k,\nu})=\alpha\), equivalently

\begin{equation*} \begin{aligned} P\big(kF_{k,\nu}>kF_{\alpha,k,\nu}\big)&=\alpha,\\[2pt] P\big((k-1)F_{k-1,\nu}>(k-1)F_{\alpha,k-1,\nu}\big)&=\alpha . \end{aligned} \end{equation*}

Put \(a^{*}=(k-1)F_{\alpha,k-1,\nu}>0\). By (b) with \(k^{\prime}=k-1\),

\begin{equation*} P\big(kF_{k,\nu}>a^{*}\big)>P\big((k-1)F_{k-1,\nu}>a^{*}\big)=\alpha . \end{equation*}

The map \(t\mapsto P(kF_{k,\nu}>t)\) is continuous and strictly decreasing on \((0,\infty)\) (the \(F\) pdf of (5.3.9) is positive there), and it takes the value \(\alpha\) at \(t=kF_{\alpha,k,\nu}\). Since it exceeds \(\alpha\) at \(t=a^{*}\), we must have

\begin{equation*} kF_{\alpha,k,\nu}>a^{*}=(k-1)F_{\alpha,k-1,\nu}. \end{equation*}

(a) Collect the constants and recognize a gamma kernel in \(x\):

\begin{equation*} \begin{aligned} f_{T_\nu}(t) &=\frac{1}{\sqrt{2\pi\nu}\,\Gamma(\frac\nu2)2^{\nu/2}} \int_0^\infty x^{\frac{\nu+1}{2}-1}e^{-\frac x2\left(1+\frac{t^2}{\nu}\right)}dx\\[2pt] &=\frac{\Gamma\!\left(\frac{\nu+1}{2}\right)}{\sqrt{2\pi\nu}\,\Gamma(\frac\nu2)2^{\nu/2}} \left[\frac{2}{1+t^2/\nu}\right]^{(\nu+1)/2}\\[2pt] &=\frac{\Gamma\!\left(\frac{\nu+1}{2}\right)}{\Gamma(\frac\nu2)} \frac{1}{(\nu\pi)^{1/2}}\frac{1}{(1+t^2/\nu)^{(\nu+1)/2}}, \end{aligned} \end{equation*}

using \(2^{(\nu+1)/2}/\big(2^{\nu/2}\sqrt2\big)=1\) and \(\sqrt{2\pi\nu}=\sqrt2\sqrt{\pi\nu}\). This is (5.3.6).

(b) Differentiate both sides in \(t\); the integrand is dominated locally, so differentiation passes inside. The left side gives \(\nu f_{F_{1,\nu}}(\nu t)\) and the right side \(\int_0^\infty y\,f_{\chi^2_1}(ty)f_\nu(y)\,dy\), whence

\begin{equation*} f_{F_{1,\nu}}(\nu t)=\frac1\nu\int_0^\infty y\,f_{\chi^2_1}(ty)\,f_\nu(y)\,dy . \end{equation*}

Evaluate the integral with \(f_{\chi^2_1}(u)=u^{-1/2}e^{-u/2}/\sqrt{2\pi}\):

\begin{equation*} \begin{aligned} \int_0^\infty y\,f_{\chi^2_1}(ty)f_\nu(y)\,dy &=\frac{t^{-1/2}}{\sqrt{2\pi}\,\Gamma(\frac\nu2)2^{\nu/2}} \int_0^\infty y^{\frac{\nu+1}{2}-1}e^{-\frac y2(1+t)}dy\\[2pt] &=\frac{\Gamma\!\left(\frac{\nu+1}{2}\right)}{\Gamma(\frac12)\Gamma(\frac\nu2)} \frac{t^{-1/2}}{(1+t)^{(\nu+1)/2}} , \end{aligned} \end{equation*}

so

\begin{equation*} f_{F_{1,\nu}}(\nu t)=\frac{\Gamma\!\left(\frac{\nu+1}{2}\right)}{\Gamma(\frac12)\Gamma(\frac\nu2)} \cdot\frac{1}{\nu}\cdot\frac{t^{-1/2}}{(1+t)^{(\nu+1)/2}} . \end{equation*}

Setting \(p=1\), \(q=\nu\), \(x=\nu t\) in (5.3.9) gives

\begin{equation*} f_{F_{1,\nu}}(\nu t)=\frac{\Gamma\!\left(\frac{1+\nu}{2}\right)}{\Gamma(\frac12)\Gamma(\frac\nu2)} \left(\frac1\nu\right)^{1/2}\frac{(\nu t)^{-1/2}}{(1+t)^{(1+\nu)/2}}, \end{equation*}

the same expression.

(c) Immediate from Definition 5.3.6 and does not need \(m=1\). Write \(F_{m,\nu}=(\chi^2_m/m)\big/(V/\nu)\) with \(V\sim\chi^2_\nu\) independent of \(\chi^2_m\). Then

\begin{equation*} P\!\left(F_{m,\nu}\le\frac\nu m t\right) =P\!\left(\frac{\chi^2_m/m}{V/\nu}\le\frac{\nu t}{m}\right) =P(\chi^2_m\le tV), \end{equation*}

and conditioning on \(V=y\) (independence) gives

\begin{equation*} P(\chi^2_m\le tV)=\int_0^\infty P(\chi^2_m\le ty)\,f_\nu(y)\,dy . \end{equation*}

Differentiating in \(t\) as in (b) confirms the pdf: the right side yields

\begin{equation*} \int_0^\infty y\,f_{\chi^2_m}(ty)f_\nu(y)\,dy =\frac{\Gamma\!\left(\frac{m+\nu}{2}\right)}{\Gamma(\frac m2)\Gamma(\frac\nu2)} \frac{t^{m/2-1}}{(1+t)^{(m+\nu)/2}}, \end{equation*}

while the left side yields \(\frac{\nu}{m}f_{F_{m,\nu}}\!\left(\frac\nu m t\right)\), which by (5.3.9) is the same.

\(3/4\); for a sample of size \(n\) it is \(1-2^{-n}\).

Let \(m\) be the population median, so \(P(X_i\le m)=\tfrac12\) for a continuous population. The largest order statistic fails to exceed \(m\) exactly when every observation fails to, so by independence

\begin{equation*} P(X_{(n)}>m)=1-P(X_1\le m,\dots,X_n\le m)=1-\left(\tfrac12\right)^{n}, \end{equation*}

which is \(1-(1/2)^2=3/4\) at \(n=2\).

The cdf of \(Z^2\) is \(2\Phi(\sqrt{z}\,)-1\), which is the \(\chi^2_1\) cdf.

Write \(\Phi\) for the \(\mathrm{n}(0,1)\) cdf and fix \(z>0\), \(a=\sqrt{z}\). Since \(Z\le t\) fails exactly when both \(X>t\) and \(Y>t\), independence gives \(P(Z\le t)=1-[1-\Phi(t)]^2\), and \(Z\) is continuous, so

\begin{equation*} \begin{aligned} P(Z^2\le z)&=P(-a\le Z\le a)\\ &=\bigl\{1-[1-\Phi(a)]^2\bigr\}-\bigl\{1-[1-\Phi(-a)]^2\bigr\}\\ &=[\Phi(a)]^2-[1-\Phi(a)]^2, \end{aligned} \end{equation*}

using \(1-\Phi(-a)=\Phi(a)\). Factoring the difference of squares,

\begin{equation*} [\Phi(a)]^2-[1-\Phi(a)]^2=\bigl(2\Phi(a)-1\bigr)\cdot 1=P(|X|\le a)=P(X^2\le z). \end{equation*}

By Example 2.1.9 (or Lemma 5.3.2), \(X^2\sim\chi^2_1\); hence \(Z^2\) and \(X^2\) have the same cdf and \(Z^2\sim\chi^2_1\).

\(Z\) has pdf \(f_Z(z)=e^{1-z}/(e-1)\) on \(0<z<1\): a standard exponential truncated to \((0,1)\).

Conditionally on \(X=x\), \(Z\) is the first order statistic of \(x\) iid uniform\((0,1)\) variables, so \(P(Z>z\mid X=x)=(1-z)^x\) for \(0<z<1\). Averaging over \(X\),

\begin{equation*} \begin{aligned} P(Z>z)&=\sum_{x=1}^{\infty}(1-z)^x\,\frac{c}{x!}\\ &=c\Bigl(\sum_{x=0}^{\infty}\frac{(1-z)^x}{x!}-1\Bigr)\\ &=c\bigl(e^{1-z}-1\bigr),\qquad 0<z<1 . \end{aligned} \end{equation*}

Hence \(F_Z(z)=1-c(e^{1-z}-1)\) on \((0,1)\), with \(F_Z(0)=0\) and \(F_Z(1)=1\) (Check!), and

\begin{equation*} f_Z(z)=\frac{d}{dz}F_Z(z)=c\,e^{1-z}=\frac{e^{1-z}}{e-1},\qquad 0<z<1 . \end{equation*}

The joint pdf of \((X_{(1)}/X_{(n)},X_{(n)})\) factors, so they are independent.

By (5.4.7) with \(i=1\), \(j=n\), using \(F_X(x)=x/\theta\) on \((0,\theta)\),

\begin{equation*} f_{X_{(1)},X_{(n)}}(u,v)=\frac{n(n-1)(v-u)^{n-2}}{\theta^{n}},\qquad 0<u<v<\theta . \end{equation*}

Put \(T=X_{(1)}/X_{(n)}\), \(W=X_{(n)}\), so \(u=tw\), \(v=w\); the transformation is one-to-one from \(\{0<u<v<\theta\}\) onto \(\{0<t<1,\ 0<w<\theta\}\) with Jacobian

\begin{equation*} \left|\det\begin{pmatrix} w & t\\ 0 & 1\end{pmatrix}\right| = w . \end{equation*}

By the bivariate change-of-variables formula (4.3.2),

\begin{equation*} \begin{aligned} f_{T,W}(t,w)&=\frac{n(n-1)(w-tw)^{n-2}}{\theta^{n}}\cdot w\\ &=\underbrace{(n-1)(1-t)^{n-2}}_{0<t<1}\cdot\underbrace{\frac{n\,w^{n-1}}{\theta^{n}}}_{0<w<\theta}. \end{aligned} \end{equation*}

The joint pdf is a product of a function of \(t\) alone and a function of \(w\) alone on a rectangular support, so by Lemma 4.2.7 \(T\) and \(W\) are independent, \(T\sim\text{beta}(1,n-1)\) and \(W=X_{(n)}\) has pdf \(nw^{n-1}/\theta^{n}\).

The joint pdf of the ratios factors completely: \(Y_j=X_{(j)}/X_{(j+1)}\sim\text{beta}(ja,1)\) for \(j=1,\dots,n-1\), and \(Y_n=X_{(n)}\) has pdf \(na\,y^{na-1}/\theta^{na}\) on \((0,\theta)\).

By the joint pdf of all \(n\) order statistics (the display following Theorem 5.4.6),

\begin{equation*} f_{X_{(1)},\dots,X_{(n)}}(x_1,\dots,x_n)=n!\Bigl(\frac{a}{\theta^{a}}\Bigr)^{n}\prod_{i=1}^{n}x_i^{a-1}, \qquad 0<x_1<\cdots<x_n<\theta . \end{equation*}

Set \(Y_j=X_{(j)}/X_{(j+1)}\) for \(j<n\) and \(Y_n=X_{(n)}\); the inverse is \(x_i=\prod_{j\ge i}y_j\), a one-to-one map onto \(\{0<y_j<1\ (j<n),\ 0<y_n<\theta\}\). The matrix \((\partial x_i/\partial y_j)\) is upper triangular (\(\partial x_i/\partial y_j=0\) for \(j<i\)), so

\begin{equation*} |J|=\prod_{i=1}^{n}\frac{\partial x_i}{\partial y_i} =\prod_{i=1}^{n}\ \prod_{j>i}y_j=\prod_{j=1}^{n}y_j^{\,j-1}, \end{equation*}

since \(y_j\) occurs once for each \(i<j\). Likewise \(y_j\) occurs in \(x_i\) for each \(i\le j\), so

\begin{equation*} \prod_{i=1}^{n}x_i^{a-1}=\prod_{j=1}^{n}y_j^{\,j(a-1)} . \end{equation*}

Therefore, by the \(n\)-variate change-of-variables formula (4.6.7),

\begin{equation*} f_{Y_1,\dots,Y_n}(y_1,\dots,y_n)=n!\Bigl(\frac{a}{\theta^{a}}\Bigr)^{n}\prod_{j=1}^{n}y_j^{\,ja-1} \end{equation*}

on the rectangle above. The pdf is a product of functions of the individual \(y_j\) over a rectangular support, so by Theorem 4.6.11 \(Y_1,\dots,Y_n\) are mutually independent, with

\begin{equation*} \begin{aligned} f_{Y_j}(y)&=ja\,y^{ja-1},&&0<y<1,\quad j=1,\dots,n-1,\\ f_{Y_n}(y)&=\frac{na}{\theta^{na}}\,y^{na-1},&&0<y<\theta . \end{aligned} \end{equation*}

That is, \(X_{(j)}/X_{(j+1)}\sim\text{beta}(ja,1)\) for \(j=1,\dots,n-1\), and \(X_{(n)}/\theta\sim\text{beta}(na,1)\).

Fix \(u<v\) throughout, and write \(p=F_X(u)\), \(q=F_X(v)-F_X(u)\), \(r=1-F_X(v)\).

(a) Each \(X_l\) falls into exactly one of the three cells \((-\infty,u]\), \((u,v]\), \((v,\infty)\), with probabilities \(p,q,r\) respectively, and the \(X_l\) are iid. Thus the \(n\) classifications are independent identical three-cell trials, and by Definition 4.6.2 the cell counts \((U,V,n-U-V)\) are multinomial with \(n\) trials and cell probabilities \((p,q,r)\).

(b) Since exactly \(U\) of the sample lie in \((-\infty,u]\) and exactly \(U+V\) lie in \((-\infty,v]\),

\begin{equation*} \{X_{(i)}\le u\}=\{U\ge i\},\qquad \{X_{(j)}\le v\}=\{U+V\ge j\}, \end{equation*}

giving the first line. Split on the value of \(U\): if \(U=k\ge j\) then \(U+V\ge j\) automatically, while for \(i\le k\le j-1\) we need \(V\ge j-k\), and \(V\le n-k\) always. Hence

\begin{equation*} F_{X_{(i)},X_{(j)}}(u,v)=\sum_{k=i}^{j-1}\sum_{m=j-k}^{n-k}P(U=k,V=m)+P(U\ge j), \end{equation*}

and substituting the multinomial probabilities from part (a) gives the displayed triple-product form.

(c) By (4.1.4), \(f_{X_{(i)},X_{(j)}}(u,v)=\partial^2 F_{X_{(i)},X_{(j)}}(u,v)/\partial u\,\partial v\). The term \(P(U\ge j)\) is a function of \(u\) only, so it dies under \(\partial/\partial v\).

Differentiate in \(v\) first. Note \(q+r=1-p\) does not depend on \(v\), and \(\partial q/\partial v=-\partial r/\partial v=f_X(v)\). Writing the inner sum with \(N=n-k\), \(M=j-k\),

\begin{equation*} \frac{n!}{k!m!(n-k-m)!}p^{k}q^{m}r^{n-k-m}=\binom{n}{k}p^{k}\binom{N}{m}q^{m}r^{N-m}, \end{equation*}

and the upper-tail binomial derivative (the same cancellation as in (5.4.6)) gives

\begin{equation*} \frac{\partial}{\partial q}\sum_{m=M}^{N}\binom{N}{m}q^{m}(s-q)^{N-m} =N\binom{N-1}{M-1}q^{M-1}(s-q)^{N-M}, \end{equation*}

valid with \(s=q+r\) held fixed, since all but one term of the telescoping sum cancels. With \(s-q=r=1-F_X(v)\) and \(N-M=n-j\),

\begin{equation*} \begin{aligned} \frac{\partial F}{\partial v}&=f_X(v)\,r^{\,n-j}\sum_{k=i}^{j-1}\binom{n}{k}(n-k)\binom{n-k-1}{j-k-1}p^{k}q^{\,j-k-1}\\ &=f_X(v)\,r^{\,n-j}\sum_{k=i}^{j-1}\frac{n!}{k!\,(j-k-1)!\,(n-j)!}\,p^{k}q^{\,j-1-k}, \end{aligned} \end{equation*}

since \(\binom{n}{k}(n-k)\binom{n-k-1}{j-k-1}=\dfrac{n!}{k!(j-k-1)!(n-j)!}\) (Check!).

Now differentiate in \(u\). Here \(p+q=F_X(v)\) is free of \(u\) and \(\partial p/\partial u=-\partial q/\partial u=f_X(u)\). With \(L=j-1\),

\begin{equation*} \frac{n!}{k!(L-k)!(n-j)!}=\frac{n!}{L!\,(n-j)!}\binom{L}{k}, \end{equation*}

so the sum above is \(\dfrac{n!}{(j-1)!(n-j)!}\sum_{k=i}^{L}\binom{L}{k}p^{k}q^{L-k}\), and the same derivative identity (with total \(p+q=F_X(v)\)) yields

\begin{equation*} \frac{\partial}{\partial p}\sum_{k=i}^{L}\binom{L}{k}p^{k}q^{L-k}=L\binom{L-1}{i-1}p^{\,i-1}q^{\,L-i}. \end{equation*}

Collecting, and using

\begin{equation*} \frac{n!}{(j-1)!(n-j)!}\,(j-1)\binom{j-2}{i-1}=\frac{n!}{(i-1)!\,(j-1-i)!\,(n-j)!}, \end{equation*}

we obtain

\begin{equation*} \begin{aligned} f_{X_{(i)},X_{(j)}}(u,v)=\ &\frac{n!}{(i-1)!(j-1-i)!(n-j)!}f_X(u)f_X(v)[F_X(u)]^{i-1}\\ &\times[F_X(v)-F_X(u)]^{j-1-i}[1-F_X(v)]^{n-j}, \end{aligned} \end{equation*}

for \(u<v\), which is (5.4.7).

(a) For \(i<j\), divide (5.4.7) by the pdf (5.4.4) of \(X_{(j)}\):

\begin{equation*} \begin{aligned} f_{X_{(i)}\mid X_{(j)}}(u\mid v)=\ &\frac{(j-1)!}{(i-1)!\,(j-1-i)!}\,\frac{f_X(u)}{F_X(v)}\\ &\times\left[\frac{F_X(u)}{F_X(v)}\right]^{i-1}\left[\frac{F_X(v)-F_X(u)}{F_X(v)}\right]^{j-1-i}, \end{aligned} \end{equation*}

for \(u<v\); the factors \(f_X(v)\) and \([1-F_X(v)]^{n-j}\) cancel. Comparing with (5.4.4), this is exactly the pdf of the \(i\)th order statistic of a sample of size \(j-1\) from the population \(F_X\) truncated to \((-\infty,v)\), i.e. with cdf \(F_X(x)/F_X(v)\) and pdf \(f_X(x)/F_X(v)\) for \(x<v\).

For \(i>j\) the mirror computation (dividing (5.4.7) with indices \(j<i\) by \(f_{X_{(j)}}\)) gives

\begin{equation*} \begin{aligned} f_{X_{(i)}\mid X_{(j)}}(u\mid v)=\ &\frac{(n-j)!}{(i-j-1)!\,(n-i)!}\,\frac{f_X(u)}{1-F_X(v)}\\ &\times\left[\frac{F_X(u)-F_X(v)}{1-F_X(v)}\right]^{i-j-1}\\ &\times\left[\frac{1-F_X(u)}{1-F_X(v)}\right]^{n-i}, \end{aligned} \end{equation*}

for \(u>v\): the \((i-j)\)th order statistic of a sample of size \(n-j\) from \(F_X\) truncated to \((v,\infty)\).

(b) Dividing the joint pdf of Example 5.4.7 by \(f_R( r)\) in (5.4.8),

\begin{equation*} f_{V\mid R}(v\mid r)=\frac{n(n-1)r^{n-2}/a^{n}}{n(n-1)r^{n-2}(a-r)/a^{n}}=\frac{1}{a-r}, \qquad \frac{r}{2}<v<a-\frac{r}{2}. \end{equation*}

So \(V\mid R=r\) is uniform on the interval \((r/2,\;a-r/2)\), of length \(a-r\).

(a) The pdf. Integrate the joint pdf \(n!\prod_{t=1}^{n} f_X(x_t)\) of all \(n\) order statistics (the display following Theorem 5.4.6) over the unobserved coordinates in each gap; with the conventions \(i_0=0\), \(i_{l+1}=n+1\), \(F_X(x_{i_0})=0\), \(F_X(x_{i_{l+1}})=1\),

\begin{equation*} \begin{aligned} f_{X_{(i_1)},\dots,X_{(i_l)}}(x_{i_1},\dots,x_{i_l}) =\ &n!\prod_{r=1}^{l}f_X(x_{i_r})\\ &\times\prod_{r=1}^{l+1}\frac{\bigl[F_X(x_{i_r})-F_X(x_{i_{r-1}})\bigr]^{\,i_r-i_{r-1}-1}}{(i_r-i_{r-1}-1)!}, \end{aligned} \end{equation*}

for \(x_{i_1}<\cdots<x_{i_l}\), and \(0\) otherwise: \(i_r-i_{r-1}-1\) of the remaining observations must land in the \(r\)th gap \((x_{i_{r-1}},x_{i_r})\). For \(l=2\) this is (5.4.7) and for \(l=1\) it is (5.4.4).

The cdf. Evaluate at arbitrary arguments \(u_1,\dots,u_l\) and set \(y_r=\min_{r^{\prime}\ge r}u_{r^{\prime}}\), so \(y_1\le\cdots\le y_l\) and \(\{X_{(i_r)}\le u_r\ \forall r\}=\{X_{(i_r)}\le y_r\ \forall r\}\) because \(X_{(i_1)}\le\cdots\le X_{(i_l)}\). Let \(N_r\) count the sample values \(\le y_r\). Exactly as in Exercise 5.26(b), \(\{X_{(i_r)}\le y_r\}=\{N_r\ge i_r\}\), and \((N_1,N_2-N_1,\dots,N_l-N_{l-1},n-N_l)\) is multinomial with \(n\) trials and cell probabilities \(F_X(y_1),\,F_X(y_2)-F_X(y_1),\dots,\,1-F_X(y_l)\). Hence, with \(n_0=0\) and \(F_X(y_0)=0\),

\begin{equation*} \begin{aligned} F_{X_{(i_1)},\dots,X_{(i_l)}}(u_1,\dots,u_l)=\ &\sum_{\substack{n_1\le\cdots\le n_l\le n\\ n_r\ge i_r\ \forall r}} \frac{n!}{(n-n_l)!\prod_{r=1}^{l}(n_r-n_{r-1})!}\\ &\times\bigl[1-F_X(y_l)\bigr]^{\,n-n_l}\prod_{r=1}^{l}\bigl[F_X(y_r)-F_X(y_{r-1})\bigr]^{\,n_r-n_{r-1}} . \end{aligned} \end{equation*}

(b) The pdf. Write \(I=\{i_1<\cdots<i_l\}\), \(J=\{j_1<\cdots<j_m\}\), and divide the part (a) pdf for the pooled group \(I\cup J\) by the part (a) pdf for \(J\). Set \(j_0=0\), \(j_{m+1}=n+1\), \(x_{j_0}=-\infty\), \(x_{j_{m+1}}=+\infty\), and let block \(t\) (\(t=0,\dots,m\)) consist of those indices of \(I\) lying strictly between \(j_t\) and \(j_{t+1}\), say \(a_1<\cdots<a_{s_t}\) (put \(a_0=j_t\), \(a_{s_t+1}=j_{t+1}\)). Every factor of the \(J\)-pdf cancels, and what survives is

\begin{equation*} \begin{aligned} f_{I\mid J}=\prod_{t=0}^{m}\Bigl\{&(j_{t+1}-j_t-1)!\prod_{r=1}^{s_t}f_X(x_{a_r})\\ &\times\prod_{r=1}^{s_t+1}\frac{\bigl[F_X(x_{a_r})-F_X(x_{a_{r-1}})\bigr]^{\,a_r-a_{r-1}-1}}{(a_r-a_{r-1}-1)!} \Bigl[F_X(x_{j_{t+1}})-F_X(x_{j_t})\Bigr]^{-(j_{t+1}-j_t-1)}\Bigr\}, \end{aligned} \end{equation*}

supported on \(x_{j_t}<x_{a_1}<\cdots<x_{a_{s_t}}<x_{j_{t+1}}\) for every \(t\).

Comparing with part (a), each block factor is exactly the joint pdf of the order statistics of ranks \(a_1-j_t<\cdots<a_{s_t}-j_t\) from a sample of size \(N_t=j_{t+1}-j_t-1\) drawn from \(F_X\) truncated to \((x_{j_t},x_{j_{t+1}})\), i.e. with cdf

\begin{equation*} G_t(x)=\frac{F_X(x)-F_X(x_{j_t})}{F_X(x_{j_{t+1}})-F_X(x_{j_t})},\qquad x_{j_t}<x<x_{j_{t+1}}, \end{equation*}

the powers of the truncation constant matching because \(s_t+\sum_{r=1}^{s_t+1}(a_r-a_{r-1}-1)=N_t\). So, given \(X_{(j_1)},\dots,X_{(j_m)}\), the blocks are mutually independent (the Markov property of the order statistics).

The cdf. By that independence it is the product over blocks of the part (a) cdf for the truncated population: writing \(u_i\) for the argument attached to \(X_{(i)}\), \(i\in I\),

\begin{equation*} F_{I\mid J}\bigl((u_i)_{i\in I}\bigr)=\prod_{t=0}^{m}F^{(N_t,G_t)}\bigl(u_{a_1},\dots,u_{a_{s_t}}\bigr), \end{equation*}

where \(F^{(N_t,G_t)}\) is the part (a) cdf formula with \(n\) replaced by \(N_t\), \(F_X\) by \(G_t\), and the ranks \(a_r\) by \(a_r-j_t\), and where \(G_t\) is extended by \(0\) below \(x_{j_t}\) and by \(1\) above \(x_{j_{t+1}}\).

Approximately \(P(Z > .8) = .2119\).

Write \(W=\sum_{i=1}^{100}X_i\) for the weight of a box, with \(X_i\) the weight of the \(i\)th booklet. Taking the \(X_i\) iid with \(EX_i=1\) and \(\operatorname{Var}X_i=(.05)^2=.0025\) gives \(EW=100\) and \(\operatorname{sd}(W)=\sqrt{100(.0025)}=.5\), and Theorem 5.5.15 (whose only hypothesis, \(0<\sigma^2<\infty\), holds here) applies to \((W-100)/.5=\sqrt n(\bar X_n-\mu)/\sigma\):

\begin{equation*} \begin{aligned} P(W > 100.4) &= P\!\left(\frac{W-100}{.5} > \frac{100.4-100}{.5}\right)\\ &\approx P(Z > .8) = .2119 . \end{aligned} \end{equation*}

The assumptions, which are worth stating to the manufacturer: (i) the booklets in a box are independent draws – a box filled from one mis-set machine run has positively correlated weights, \(\operatorname{Var}W>.25\), and the tail is understated; (ii) \(1\) and \(.05\) are population moments, not estimates from the box being tested; (iii) \(n=100\) is large enough for the limit in Theorem 5.5.15 to have taken hold at \(.8\) standard deviations. No distributional form is assumed, which is also why the answer can only be approximate: two moments of \(X_i\) are given, not the law of \(X_i\).

\(n=332\).

Pair the two samples and set \(D_i=X_{1i}-X_{2i}\). The \(D_i\) are iid with \(ED_i=0\) and \(\operatorname{Var}D_i=2\sigma^2\in(0,\infty)\), and \(\bar D=\bar X_1-\bar X_2\), so Theorem 5.5.15 applies verbatim:

\begin{equation*} \frac{\bar X_1-\bar X_2}{\sigma\sqrt{2/n}} =\frac{\sqrt n\,(\bar D-0)}{\sigma\sqrt2} \;\longrightarrow\;\mathrm{n}(0,1). \end{equation*}

Hence

\begin{equation*} \begin{aligned} .99 &\approx P\!\left(|\bar X_1-\bar X_2|<\frac{\sigma}{5}\right)\\ &= P\!\left(|Z| < \frac{\sigma/5}{\sigma\sqrt{2/n}}\right) = P\!\left(|Z| < \frac{\sqrt n}{5\sqrt2}\right). \end{aligned} \end{equation*}

The \(.995\) standard normal quantile is \(z_{.005}=2.5758\), so

\begin{equation*} \frac{\sqrt n}{5\sqrt2}=2.5758 \quad\Longrightarrow\quad n=50(2.5758)^2=331.74 . \end{equation*}

The probability is increasing in \(n\), so take \(n=332\). The only distributional input is the CLT, which needs no normality of the population, only \(\sigma^2<\infty\); and \(\sigma\) cancels, because the requirement is stated in units of \(\sigma\).

Chebychev gives \(\pm.9487\); the CLT gives \(\pm.4935\).

By Theorem 5.2.6, \(\operatorname{Var}\bar X=\sigma^2/n=9/100=.09\).

Via Chebychev: Theorem 3.6.1 applied to \(g(x)=(x-\mu)^2\) (its hypothesis is only that \(E(\bar X-\mu)^2\) exists, which it does) gives, for every \(\epsilon>0\),

\begin{equation*} P(|\bar X-\mu|<\epsilon)\;\ge\;1-\frac{\operatorname{Var}\bar X}{\epsilon^2} =1-\frac{.09}{\epsilon^2}. \end{equation*}

Setting the bound equal to \(.90\),

\begin{equation*} \frac{.09}{\epsilon^2}=.10 \quad\Longrightarrow\quad \epsilon=\sqrt{.9}=.9487 , \end{equation*}

so \(P(-.9487<\bar X-\mu<.9487)\ge .90\).

Via the Central Limit Theorem: by Theorem 5.5.15, \((\bar X-\mu)/.3\) is approximately \(\mathrm{n}(0,1)\), so

\begin{equation*} \begin{aligned} .90 &\approx P\!\left(|\bar X-\mu|<\epsilon\right) = P\!\left(|Z|<\frac{\epsilon}{.3}\right)\\ \Longrightarrow\quad \frac{\epsilon}{.3}&=1.645,\qquad \epsilon=.4935 . \end{aligned} \end{equation*}

Comment. Chebychev’s interval is valid for every population of variance \(9\) but is only a bound, so its true coverage usually far exceeds \(.90\) (Example 3.6.3: at \(t=2\) Chebychev gives \(.25\) against the normal value \(.054\)). The CLT interval is half as wide because it uses the limiting shape rather than the second moment alone, at the cost of being an approximation resting on \(n=100\) being large for the unnamed population.

(a) Both follow from Theorem 5.5.4 (if \(X_n\to X\) in probability and \(h\) is continuous, then \(h(X_n)\to h(X)\) in probability); the only thing to check is that each \(h\) is continuous on a set carrying all the \(X_i\) and the limit \(a\).

First, \(a\ge0\): if \(a<0\) then \(|X_i-a|>|a|\) with probability 1 for every \(i\), so \(P(|X_i-a|\ge|a|)=1\) does not tend to \(0\), contradicting Definition 5.5.1 at \(\epsilon=|a|\).

(i) \(h(x)=\sqrt x\) is continuous on \([0,\infty)\supseteq\{x:x>0\}\cup\{a\}\), so

\begin{equation*} Y_i=\sqrt{X_i}\;\longrightarrow\;\sqrt a \quad\text{in probability.} \end{equation*}

(ii) For \(a>0\), \(h(x)=a/x\) is continuous on \((0,\infty)\), which carries every \(X_i\) and the point \(a\), so

\begin{equation*} Y_i^{\prime}=\frac{a}{X_i}\;\longrightarrow\;\frac{a}{a}=1 \quad\text{in probability.} \end{equation*}

(If \(a=0\) then \(Y_i^{\prime}=0\) with probability 1 for every \(i\), and the sequence converges in probability to \(0\) trivially.)

(b) Apply (a) twice. Example 5.5.3 gives \(S_n^2\to\sigma^2\) in probability under its hypothesis \(\operatorname{Var}S_n^2\to0\), and \(P(S_n^2\ge0)=1\), so (a)(i) with \(a=\sigma^2\) gives

\begin{equation*} S_n=\sqrt{S_n^2}\;\longrightarrow\;\sigma\quad\text{in probability} \end{equation*}

(this is Example 5.5.5). Now (a)(ii) applied to \(S_1,S_2,\ldots\) with \(a=\sigma>0\) gives

\begin{equation*} \frac{\sigma}{S_n}\;\longrightarrow\;1\quad\text{in probability,} \end{equation*}

as required. Only continuity of \(x\mapsto\sigma/x\) at the one point \(\sigma>0\) is used, so a discrete population with \(P(S_n=0)>0\) is no obstacle: picking \(\delta\in(0,\sigma]\) with \(|s-\sigma|<\delta\Rightarrow|\sigma/s-1|<\epsilon\) gives

\begin{equation*} P\!\left(\left|\frac{\sigma}{S_n}-1\right|\ge\epsilon\right) \le P(|S_n-\sigma|\ge\delta)\longrightarrow0 , \end{equation*}

the event \(\{S_n=0\}\) sitting inside the right-hand one since \(\sigma\ge\delta\).

The containment

\begin{equation*} \{X_n+Y_n\le c\}\;\subseteq\;\{X_n\le -b\}\cup\{Y_n\le c+b\} \end{equation*}

holds for every real \(b\) – if both \(X_n>-b\) and \(Y_n>c+b\), then \(X_n+Y_n>c\) – so Boole’s Inequality gives

\begin{equation*} P(X_n+Y_n>c)\;\ge\;1-P(X_n\le -b)-P(Y_n\le c+b). \end{equation*}

Fix \(\epsilon>0\). Since \(F_X\) is a cdf, \(\lim_{b\to\infty}F_X(-b)=0\), and \(F_X\) has at most countably many discontinuities, so we may choose \(b>0\) with \(-b\) a continuity point of \(F_X\) and \(F_X(-b)<\epsilon\). For that \(b\): convergence in distribution gives \(P(X_n\le -b)=F_{X_n}(-b)\to F_X(-b)<\epsilon\) (the continuity-point requirement in the definition of convergence in distribution is exactly what the choice of \(b\) secures), and the hypothesis on \(Y_n\) applied at the finite number \(c+b\) gives \(P(Y_n\le c+b)\to0\). Therefore

\begin{equation*} \begin{aligned} \liminf_{n\to\infty}P(X_n+Y_n>c) &\ge 1-\lim_{n\to\infty}P(X_n\le -b)\\ &\qquad -\lim_{n\to\infty}P(Y_n\le c+b)\\ &= 1-F_X(-b)\;>\;1-\epsilon . \end{aligned} \end{equation*}

Since \(\epsilon>0\) was arbitrary and each probability is at most 1, \(\lim_{n\to\infty}P(X_n+Y_n>c)=1\).

Both are immediate from Theorem 5.2.6, which gives \(E\bar X_n=\mu\) and \(\operatorname{Var}\bar X_n=\sigma^2/n\) (its hypothesis, \(\sigma^2<\infty\), is assumed here):

\begin{equation*} \begin{aligned} E\,\frac{\sqrt n(\bar X_n-\mu)}{\sigma} &=\frac{\sqrt n}{\sigma}\bigl(E\bar X_n-\mu\bigr) =\frac{\sqrt n}{\sigma}(\mu-\mu)=0,\\[4pt] \operatorname{Var}\frac{\sqrt n(\bar X_n-\mu)}{\sigma} &=\frac{n}{\sigma^2}\operatorname{Var}(\bar X_n-\mu)\\ &=\frac{n}{\sigma^2}\cdot\frac{\sigma^2}{n}=1, \end{aligned} \end{equation*}

using linearity of expectation (Theorem 2.2.5) and \(\operatorname{Var}(aW+b)=a^2\operatorname{Var}W\) (Theorem 2.3.4) with \(a=\sqrt n/\sigma\), \(b=-\sqrt n\mu/\sigma\).

(a) This is Theorem 5.5.14 verbatim. The \(X_i\) are iid with \(EX_i=1\) and \(\operatorname{Var}X_i=1\) (Section 3.3), and the exponential(1) mgf \(M_X(t)=(1-t)^{-1}\) exists for \(|t|<1\), hence in a neighborhood of \(0\) – the one hypothesis Theorem 5.5.14 adds beyond finite variance. So \(G_n\), the cdf of

\begin{equation*} \frac{\sqrt n(\bar X_n-1)}{\sigma}=\frac{\bar X_n-1}{1/\sqrt n}, \end{equation*}

satisfies \(G_n(x)\to\int_{-\infty}^x(2\pi)^{-1/2}e^{-y^2/2}\,dy=P(Z\le x)\) for every \(x\).

(b) By Example 4.6.8 with \(\alpha_i=\beta=1\), \(\sum_{i=1}^nX_i\sim\text{gamma}(n,1)\), so for \(n\) large enough that \(n+x\sqrt n>0\),

\begin{equation*} \begin{aligned} P\!\left(\frac{\bar X_n-1}{1/\sqrt n}\le x\right) &=P\!\left(\sum_{i=1}^nX_i\le n+x\sqrt n\right)\\ &=\int_0^{\,n+x\sqrt n}\frac{1}{\Gamma(n)}t^{n-1}e^{-t}\,dt . \end{aligned} \end{equation*}

Part (a) says this is approximately \(\int_{-\infty}^x(2\pi)^{-1/2}e^{-y^2/2}\,dy\). Differentiating each side in \(x\) – on the left by the Fundamental Theorem of Calculus with the chain rule, the upper limit \(n+x\sqrt n\) contributing the factor \(\sqrt n\) – gives

\begin{equation*} \frac{\sqrt n}{\Gamma(n)}(x\sqrt n+n)^{n-1}e^{-(x\sqrt n+n)} \;\approx\;\frac{1}{\sqrt{2\pi}}e^{-x^2/2}, \end{equation*}

which is the stated approximation – only suggested, since convergence of cdfs does not by itself force convergence of the densities.

Setting \(x=0\):

\begin{equation*} \frac{\sqrt n}{\Gamma(n)}\,n^{n-1}e^{-n}\approx\frac{1}{\sqrt{2\pi}} \quad\Longleftrightarrow\quad \Gamma(n)\approx\sqrt{2\pi}\,n^{n-\frac12}e^{-n}. \end{equation*}

Since \(n!=n\,\Gamma(n)\),

\begin{equation*} n!\;\approx\;\sqrt{2\pi}\,n^{n+\frac12}e^{-n}, \end{equation*}

which is Stirling’s Formula as stated in Exercise 1.28.

(a) \(\mathrm{E}Y = 2\theta\) and \(\operatorname{Var} Y = 8\theta\).

A \(\chi^2_{2N}\) variable has conditional mean \(2N\) and conditional variance \(4N\), so by Theorems 4.4.3 and 4.4.7,

\begin{equation*} \begin{aligned} \mathrm{E}Y &= \mathrm{E}\big[\mathrm{E}(Y\mid N)\big] = \mathrm{E}(2N) = 2\theta,\\ \operatorname{Var} Y &= \mathrm{E}\big[\operatorname{Var}(Y\mid N)\big]

• \operatorname{Var}\big[\mathrm{E}(Y\mid N)\big]\\ &= \mathrm{E}(4N) + \operatorname{Var}(2N) = 4\theta + 4\theta = 8\theta, \end{aligned} \end{equation*}

using \(\mathrm{E}N = \operatorname{Var} N = \theta\).

(b) Condition, then use the Poisson mgf. For \(t < 1/2\), the \(\chi^2_{2n}\) mgf is \((1-2t)^{-n}\), so with \(e^{s} = (1-2t)^{-1}\),

\begin{equation*} M_Y(t) = \mathrm{E}\big[(1-2t)^{-N}\big] = M_N\big(-\log(1-2t)\big) = \exp\Big\{\theta\Big(\tfrac{1}{1-2t}-1\Big)\Big\}, \end{equation*}

that is, \(M_Y(t) = \exp\{2\theta t/(1-2t)\}\), finite for \(t < 1/2\).

Put \(Z_\theta = (Y-2\theta)/\sqrt{8\theta}\) and, for fixed \(t\), write \(s = t/\sqrt{8\theta}\) (so \(s < 1/2\) once \(\theta\) is large). Then

\begin{equation*} \begin{aligned} \log M_{Z_\theta}(t) &= -\frac{2\theta t}{\sqrt{8\theta}} + \frac{2\theta s}{1-2s} = 2\theta s\Big(\frac{1}{1-2s}-1\Big)\\ &= \frac{4\theta s^2}{1-2s} = \frac{t^2/2}{1 - 2t/\sqrt{8\theta}} \;\longrightarrow\; \frac{t^2}{2} \end{aligned} \end{equation*}

as \(\theta \to \infty\), since \(4\theta s^2 = 4\theta t^2/(8\theta) = t^2/2\). Thus \(M_{Z_\theta}(t) \to e^{t^2/2}\) for every \(t\) in a neighborhood of \(0\), and by Theorem 2.3.12, \(Z_\theta \to \mathrm{n}(0,1)\) in distribution.

The continuity correction is closer to the exact value at every \(v \ge 3\), roughly halving the error; in the extreme lower tail (\(v \le 2\)) it overshoots and the uncorrected version is better.

As noted inside Example 5.5.16 (multiply the negative binomial mgfs, Theorem 4.6.7), \(V_n = \sum_{i=1}^{10} X_i \sim\) negative binomial(\(nr, p\)) \(=\) negative binomial(\(20, .7\)), so the exact answer is

\begin{equation*} P(V_n = v) = \binom{19+v}{v}(.7)^{20}(.3)^{v}, \qquad v = 0,1,2,\ldots \end{equation*}

The CLT centering and scaling of Example 5.5.16 give

\begin{equation*} \mu = \frac{nr(1-p)}{p} = \frac{20(.3)}{.7} = \frac{60}{7} = 8.5714, \qquad \sigma^2 = \frac{nr(1-p)}{p^2} = \frac{20(.3)}{.49} = 12.2449, \end{equation*}

so \(\sigma = 3.4993\). The two approximations differ only in where the normal cdf is evaluated:

\begin{equation*} \begin{aligned} \text{(b)}\quad P(V_n = v) &\approx \Phi\Big(\frac{v-\mu}{\sigma}\Big) - \Phi\Big(\frac{v-1-\mu}{\sigma}\Big),\\ \text{( c)}\quad P(V_n = v) &\approx \Phi\Big(\frac{v+\tfrac12-\mu}{\sigma}\Big)

• \Phi\Big(\frac{v-\tfrac12-\mu}{\sigma}\Big). \end{aligned} \end{equation*}

(a) Apply Exercise 3.45 to \(S_n\). Its hypothesis holds: the \(X_i\) being iid, Theorem 4.6.7 gives \(M_{S_n}(t) = [M_X(t)]^n\), finite on the same interval \((-h,h)\). Part (a) of that exercise then reads

\begin{equation*} P(S_n > a) \le e^{-at} M_{S_n}(t) = e^{-at}[M_X(t)]^n, \qquad 0 < t < h, \end{equation*}

and part (b) gives \(P(S_n \le a) \le e^{-at}[M_X(t)]^n\) for \(-h < t \le 0\).

(b) \(M_X\) is differentiable on \((-h,h)\) with \(M_X(0) = 1\) and \({M_X}^{\prime}(0) = \mathrm{E}X < 0\), so \(M_X\) is strictly decreasing at \(0\): there is \(t_0 \in (0,h)\) with

\begin{equation*} c_0 := M_X(t_0) < 1 . \end{equation*}

Hence for every \(a \ge 0\), by part (a),

\begin{equation*} P(S_n > a) \le e^{-a t_0} c_0^{\,n} \le c_0^{\,n}, \end{equation*}

since \(e^{-at_0} \le 1\). Take \(c = c_0\).

Symmetrically, if \(\mathrm{E}X > 0\) then \({M_X}^{\prime}(0) > 0\) forces \(M_X(t_1) =: c_1 < 1\) for some \(t_1 \in (-h,0)\), and for \(a \le 0\) part (a) gives \(P(S_n \le a) \le e^{-a t_1} c_1^{\,n} \le c_1^{\,n}\).

(c) Put \(\mu = \mathrm{E}X_i\) and fix \(\varepsilon > 0\). The \(Y_i = X_i - \mu - \varepsilon\) are iid with

\begin{equation*} \mathrm{E}Y_i = -\varepsilon < 0, \qquad M_Y(t) = e^{-(\mu+\varepsilon)t} M_X(t) \end{equation*}

finite on \((-h,h)\). Since \(\sum_{i=1}^n Y_i = S_n - n\mu - n\varepsilon\),

\begin{equation*} P(\bar{X}_n - \mu > \varepsilon) = P\Big(\sum_{i=1}^n Y_i > 0\Big) \le c_1^{\,n} \end{equation*}

by part (b) with \(a = 0\), for some \(0 < c_1 < 1\).

(d) The \(Y_i = -X_i + \mu - \varepsilon\) are iid with \(\mathrm{E}Y_i = -\varepsilon < 0\) and mgf \(e^{(\mu - \varepsilon)t}M_X(-t)\), finite on \((-h,h)\); and \(\sum_{i=1}^n Y_i = n\mu - S_n - n\varepsilon\), so part (b) with \(a = 0\) gives

\begin{equation*} P(\bar{X}_n - \mu < -\varepsilon) = P\Big(\sum_{i=1}^n Y_i > 0\Big) \le c_2^{\,n} \end{equation*}

for some \(0 < c_2 < 1\). Adding the two one-sided bounds and setting \(c = \max\{c_1,c_2\} < 1\),

\begin{equation*} P(|\bar{X}_n - \mu| > \varepsilon) \le c_1^{\,n} + c_2^{\,n} \le 2c^{\,n}. \end{equation*}

(a) The hint’s \(\delta\) turns the continuity statement into the event inclusion

\begin{equation*} \{|X_n - X| < \delta\} \subset \{|h(X_n) - h(X)| < \varepsilon\}, \end{equation*}

so \(P(|h(X_n)-h(X)| < \varepsilon) \ge P(|X_n - X| < \delta) \to 1\), which is exactly Definition 5.5.1 for \(h(X_n) \to h(X)\).

The one gap is that continuity gives a \(\delta\) depending on the point \(x\); uniformity is bought with one compact set. Fix \(\varepsilon > 0\) and \(\eta > 0\), and choose \(M\) with \(P(|X| > M) < \eta/2\) (possible, since \(P(|X| > M) \downarrow 0\) as \(M \to \infty\)). On the compact interval \([-M-1, M+1]\), \(h\) is uniformly continuous, so there is \(\delta \in (0,1)\) with \(|h(x) - h(y)| < \varepsilon\) whenever \(|x| \le M\) and \(|x - y| < \delta\). Then

\begin{equation*} \{|h(X_n) - h(X)| \ge \varepsilon\} \subset \{|X_n - X| \ge \delta\} \cup \{|X| > M\}, \end{equation*}

whence

\begin{equation*} \limsup_n P(|h(X_n)-h(X)| \ge \varepsilon) \le \lim_n P(|X_n - X| \ge \delta) + \tfrac{\eta}{2} = \tfrac{\eta}{2} . \end{equation*}

As \(\eta > 0\) was arbitrary, \(P(|h(X_n)-h(X)| \ge \varepsilon) \to 0\).

(b) Take the first member of each block, \(n_k = \tfrac{k(k-1)}{2} + 1\), so that

\begin{equation*} X_{n_k}(s) = s + I_{[0,\,1/k]}(s), \qquad k = 1, 2, \ldots \end{equation*}

(the indices \(1, 2, 4, 7, 11, \ldots\)). For every \(s \in (0,1]\) we have \(1/k < s\) for all \(k > 1/s\), so \(X_{n_k}(s) = s = X(s)\) eventually; the convergence fails only at \(s = 0\), and \(P(\{0\}) = 0\). Hence \(X_{n_k} \to X\) almost surely by Definition 5.5.6.

(a) If \(X \le t-\varepsilon\) and \(|X_n - X| < \varepsilon\), then \(X_n < X + \varepsilon \le t\); hence

\begin{equation*} \{X \le t-\varepsilon\} \subset \{X_n \le t\} \cup \{|X_n - X| \ge \varepsilon\}, \end{equation*}

and Boole’s inequality gives

\begin{equation*} F_X(t-\varepsilon) \le F_{X_n}(t) + P(|X_n - X| \ge \varepsilon), \end{equation*}

that is, \(F_{X_n}(t) \ge F_X(t-\varepsilon) - P(|X_n-X| \ge \varepsilon)\).

(b) If \(X_n \le t\) and \(|X_n - X| < \varepsilon\), then \(X < X_n + \varepsilon \le t+\varepsilon\); hence

\begin{equation*} \{X_n \le t\} \subset \{X \le t+\varepsilon\} \cup \{|X_n - X| \ge \varepsilon\}, \end{equation*}

so \(F_{X_n}(t) \le F_X(t+\varepsilon) + P(|X_n - X| \ge \varepsilon)\).

(c) \(P(|X_n - X| \ge \varepsilon) \to 0\) by Definition 5.5.1, so letting \(n \to \infty\) in (a) and (b) with \(\varepsilon\) held fixed,

\begin{equation*} F_X(t-\varepsilon) \;\le\; \liminf_{n} F_{X_n}(t) \;\le\; \limsup_{n} F_{X_n}(t) \;\le\; F_X(t+\varepsilon). \end{equation*}

Now let \(\varepsilon \downarrow 0\). Since \(X\) is continuous, \(F_X\) is continuous at \(t\), so both outer terms tend to \(F_X(t)\) and the two limits are pinched together:

\begin{equation*} \lim_{n\to\infty} P(X_n \le t) = P(X \le t) \end{equation*}

for every \(t\), which is Definition 5.5.10.

The printed “\(1\) if \(x \ge \mu\)” is a misprint for “\(1\) if \(x > \mu\)” (Theorem 5.5.13 itself has \(x > \mu\); at \(x = \mu\) the claim is false, as \(X_n \equiv \mu + 1/n\) shows), and that is the statement proved below.

(a) (\(\Rightarrow\)) Two inclusions, one for each side of \(\mu\); halving \(|x-\mu|\) converts the book’s weak inequalities into the strict ones that the hypothesis controls.

(i) \(x > \mu\). Put \(\varepsilon = (x-\mu)/2 > 0\). Then \(|X_n - \mu| \le \varepsilon\) forces \(X_n \le \mu + \varepsilon < x\), so

\begin{equation*} P(X_n \le x) \ge P(|X_n - \mu| \le \varepsilon) = 1 - P(|X_n - \mu| > \varepsilon) \longrightarrow 1, \end{equation*}

and since \(P(X_n \le x) \le 1\) always, \(P(X_n \le x) \to 1\).

(ii) \(x < \mu\). Put \(\varepsilon = (\mu-x)/2 > 0\). Then \(X_n \le x\) forces \(\mu - X_n \ge 2\varepsilon > \varepsilon\), so

\begin{equation*} P(X_n \le x) \le P(|X_n - \mu| > \varepsilon) \longrightarrow 0 . \end{equation*}

(b) (\(\Leftarrow\)) Fix \(\varepsilon > 0\) and split the event at \(\mu\):

\begin{equation*} \begin{aligned} P(|X_n - \mu| > \varepsilon) &= P(X_n < \mu - \varepsilon) + P(X_n > \mu + \varepsilon)\\ &\le P\big(X_n \le \mu - \tfrac{\varepsilon}{2}\big)

• \Big[1 - P(X_n \le \mu + \varepsilon)\Big]. \end{aligned} \end{equation*}

The point \(\mu - \varepsilon/2\) lies strictly below \(\mu\), so the first term tends to \(0\); the point \(\mu + \varepsilon\) lies strictly above \(\mu\), so \(P(X_n \le \mu+\varepsilon) \to 1\) and the bracket tends to \(0\). Hence \(P(|X_n - \mu| > \varepsilon) \to 0\) for every \(\varepsilon > 0\).

(a) \(\nu = 1/\beta\). The beta(\(1,\beta\)) cdf is \(F(x) = 1 - (1-x)^{\beta}\) on \((0,1)\), so for \(t > 0\) and \(n\) large enough that \(tn^{-\nu} < 1\),

\begin{equation*} \begin{aligned} P\big(n^{\nu}(1 - X_{(n)}) > t\big) &= P\big(X_{(n)} < 1 - t n^{-\nu}\big) = \big[F(1 - tn^{-\nu})\big]^{n}\\ &= \Big[1 - t^{\beta} n^{-\nu\beta}\Big]^{n}. \end{aligned} \end{equation*}

A nondegenerate limit requires \(\nu\beta = 1\); with \(\nu = 1/\beta\), Lemma 2.3.14 gives

\begin{equation*} \Big[1 - \frac{t^{\beta}}{n}\Big]^{n} \longrightarrow e^{-t^{\beta}}, \end{equation*}

so \(n^{1/\beta}(1 - X_{(n)})\) converges in distribution to the Weibull law with cdf \(1 - e^{-t^{\beta}}\), \(t > 0\).

(b) \(a_n = \log n\). With \(F(x) = 1 - e^{-x}\) on \(x>0\), for each fixed \(t\) and all \(n > e^{-t}\),

\begin{equation*} P\big(X_{(n)} - \log n \le t\big) = \big[F(t + \log n)\big]^{n} = \Big[1 - \frac{e^{-t}}{n}\Big]^{n} \longrightarrow \exp\{-e^{-t}\}, \end{equation*}

again by Lemma 2.3.14. Thus \(X_{(n)} - \log n\) converges in distribution to the Gumbel (extreme value) law with cdf \(e^{-e^{-t}}\), \(-\infty < t < \infty\).

(a) Write \(W_n = \sqrt{n}(Y_n - \mu) \to Z \sim \mathrm{n}(0,\sigma^2)\) in distribution. Then \(Y_n - \mu = n^{-1/2}W_n \to 0\cdot Z = 0\) in distribution, by Slutsky’s Theorem 5.5.17(a) applied to the constant sequence \(n^{-1/2} \to 0\); convergence in distribution to a constant is convergence in probability (Theorem 5.5.13).

Method (2): fix \(\varepsilon > 0\) and \(M > 0\); for \(n \ge (M/\varepsilon)^2\),

\begin{equation*} P(|Y_n - \mu| > \varepsilon) = P(|W_n| > \sqrt{n}\,\varepsilon) \le P(|W_n| > M) \to P(|Z| > M), \end{equation*}

the limit because the \(\mathrm{n}(0,\sigma^2)\) cdf is continuous, so \(\pm M\) are continuity points. Let \(M \to \infty\).

(b) Taylor’s expansion (5.5.11) about \(\theta\) is

\begin{equation*} g(Y_n) = g(\theta) + g^{\prime}(\theta)(Y_n - \theta) + R_n , \end{equation*}

which is the definition of \(R_n\). Put

\begin{equation*} h(y) = \begin{cases} \dfrac{g(y) - g(\theta) - g^{\prime}(\theta)(y-\theta)}{y - \theta}, & y \ne \theta,\\ 0, & y = \theta, \end{cases} \end{equation*}

so that \(R_n = h(Y_n)(Y_n - \theta)\) identically. That \(g^{\prime}(\theta)\) exists is exactly the statement \(\lim_{y\to\theta} h(y) = 0 = h(\theta)\), i.e. \(h\) is continuous at \(\theta\). By part (a), \(Y_n \to \theta\) in probability, so Theorem 5.5.4 (whose proof uses continuity of \(h\) only at the limit point, here the constant \(\theta\)) gives

\begin{equation*} h(Y_n) \to 0 \quad\text{in probability.} \end{equation*}

Now \(\sqrt{n}\,R_n = h(Y_n)\cdot\sqrt{n}(Y_n-\theta)\), where \(\sqrt{n}(Y_n-\theta) \to Z \sim \mathrm{n}(0,\sigma^2)\) by hypothesis, so \(\sqrt{n}R_n \to 0\cdot Z = 0\) in distribution by Theorem 5.5.17(a) and hence in probability by Theorem 5.5.13. Then Theorem 5.5.17(b) applied to

\begin{equation*} \sqrt{n}\bigl[g(Y_n) - g(\theta)\bigr] = g^{\prime}(\theta)\sqrt{n}(Y_n - \theta) + \sqrt{n}R_n \end{equation*}

gives convergence in distribution to \(g^{\prime}(\theta)Z + 0\), and \(g^{\prime}(\theta)Z \sim \mathrm{n}\bigl(0,\sigma^2[g^{\prime}(\theta)]^2\bigr)\), which is (5.5.10).

(a) This is the Central Limit Theorem (Theorem 5.5.14) verbatim: the \(X_i\) are iid with \(\mathrm{E}X_i = p\), \(\operatorname{Var}X_i = p(1-p) > 0\), and mgf \(1-p+pe^t\) finite for all \(t\), so the neighborhood-of-zero mgf hypothesis holds.

(b) Take \(g(y) = y(1-y)\), so \(g^{\prime}(y) = 1 - 2y\) and \(g^{\prime}(p) = 1 - 2p \ne 0\) precisely when \(p \ne 1/2\). With \(\sigma^2 = p(1-p)\), the Delta Method (Theorem 5.5.24) applied to the sequence of part (a) gives

\begin{equation*} \sqrt{n}\bigl[g(Y_n) - g(p)\bigr] \to \mathrm{n}\bigl(0,\ p(1-p)(1-2p)^2\bigr) . \end{equation*}

(c) Here \(g^{\prime}(1/2) = 0\) and \(g^{\prime\prime}(y) \equiv -2 \ne 0\), so the hypotheses of the second-order Delta Method (Theorem 5.5.26) hold with \(\theta = 1/2\) and \(\sigma^2 = \tfrac14\), and (5.5.13) reads

\begin{equation*} n\Bigl[Y_n(1-Y_n) - \tfrac14\Bigr] \to \sigma^2 \frac{g^{\prime\prime}(1/2)}{2}\chi^2_1 = \frac14\cdot(-1)\,\chi^2_1 = -\frac14 \chi^2_1 . \end{equation*}

(The parenthetical equivalent form has a misprinted constant: multiplying by \(-4\) gives \(4n[\tfrac14 - Y_n(1-Y_n)] = 4n(Y_n-\tfrac12)^2 \to \chi^2_1\), not \(2n\).)

The three answers are \(p_2 = .0119\), \(.3096\), and \(.7753\).

For integer \(a\) the gamma survival function is the Poisson tail (integrate by parts \(a-1\) times): if \(X \sim \text{gamma}(a,b)\) then

\begin{equation*} P(X \ge x) = \sum_{k=0}^{a-1} e^{-x/b}\frac{(x/b)^k}{k!} . \end{equation*}

(a) \(x/b = 100/25 = 4\), so

\begin{equation*} p_1 = e^{-4}\Bigl(1 + 4 + 8 + \tfrac{32}{3}\Bigr) = \tfrac{71}{3}e^{-4} = .43347 , \end{equation*}

and with \(c = 300\), \(t = 150\),

\begin{equation*} p_2 = \sum_{k=150}^{300}\binom{300}{k}(.43347)^k(.56653)^{300-k} = .01194 . \end{equation*}

(b) \(x/b = 100/5 = 20\), so \(p_1 = e^{-20}\sum_{k=0}^{19} 20^k/k! = .47026\) (the closed form is a sum of twenty terms, hence the hint), and with \(c = 100\), \(t = 50\),

\begin{equation*} p_2 = \sum_{k=50}^{100}\binom{100}{k}(.47026)^k(.52974)^{100-k} = .30962 . \end{equation*}

(c) \(a = 20.7\) is not an integer, so the tail must be evaluated numerically:

\begin{equation*} p_1 = \frac{1}{\Gamma(20.7)}\int_{20}^{\infty} u^{19.7}e^{-u}\,du = .53270 , \end{equation*}

after the substitution \(u = x/5\). Then, with \(c = 100\) and threshold \(50\),

\begin{equation*} p_2 = \sum_{k=50}^{100}\binom{100}{k}(.53270)^k(.46730)^{100-k} = .77529 . \end{equation*}

With the continuity correction the normal approximation is accurate to three decimals in all three parts; without it the errors are large (a relative error of \(16\%\) in part (a)).

For \(X \sim \text{binomial}(c,p_1)\) put \(\mu = cp_1\) and \(\sigma = \sqrt{cp_1(1-p_1)}\). The uncorrected and corrected approximations to \(P(X \ge t)\), \(t = c/2\), are

\begin{equation*} P\!\left(Z \ge \frac{t-\mu}{\sigma}\right) \qquad\text{and}\qquad P\!\left(Z \ge \frac{t-\tfrac12-\mu}{\sigma}\right), \end{equation*}

the second because \(P(X \ge t) = P(X \ge t - \tfrac12)\) by discreteness. In every part \(\min\{\mu,\ c - \mu\} \ge 46\), comfortably past the rule of thumb.

(a) \(\mu = 300(.43347) = 130.041\), \(\sigma = 8.5832\):

\begin{equation*} \begin{aligned} \text{uncorrected} &: P(Z \ge 2.3253) = .01003,\\ \text{corrected} &: P(Z \ge 2.2671) = .01169,\\ \text{exact} &: .01194 . \end{aligned} \end{equation*}

(b) \(\mu = 100(.47026) = 47.026\), \(\sigma = 4.9911\):

\begin{equation*} \begin{aligned} \text{uncorrected} &: P(Z \ge .5959) = .27562,\\ \text{corrected} &: P(Z \ge .4957) = .31004,\\ \text{exact} &: .30962 . \end{aligned} \end{equation*}

(c) \(\mu = 100(.53270) = 53.270\), \(\sigma = 4.9893\):

\begin{equation*} \begin{aligned} \text{uncorrected} &: P(Z \ge -.6555) = .74392,\\ \text{corrected} &: P(Z \ge -.7557) = .77509,\\ \text{exact} &: .77529 . \end{aligned} \end{equation*}

Everything follows from \(V_j = -\log U_j \sim \text{exponential}(1)\): for \(v > 0\),

\begin{equation*} P(V_j > v) = P(U_j < e^{-v}) = e^{-v}, \end{equation*}

since \(U_j\) is uniform\((0,1)\) and \(e^{-v} \in (0,1)\). The \(V_j\) are independent, being functions of the independent \(U_j\) (Theorem 4.6.12).

First display. \(-2\log U_j = 2V_j \sim \text{exponential}(2) = \text{gamma}(1,2) = \chi^2_2\), and chi squared degrees of freedom add over independent summands (Lemma 5.3.2(b)), so

\begin{equation*} -2\sum_{j=1}^{\nu}\log U_j = \sum_{j=1}^{\nu} 2V_j \sim \chi^2_{2\nu}. \end{equation*}

Second display. \(-\beta\log U_j = \beta V_j \sim \text{gamma}(1,\beta)\), with mgf \((1-\beta t)^{-1}\) for \(t < 1/\beta\). By Theorem 4.6.7 the sum of \(a\) independent such variables has mgf

\begin{equation*} \prod_{j=1}^{a}(1-\beta t)^{-1} = (1-\beta t)^{-a}, \qquad t < 1/\beta , \end{equation*}

which is the gamma\((a,\beta)\) mgf; uniqueness (Theorem 2.3.11) gives \(-\beta\sum_{j=1}^a \log U_j \sim \text{gamma}(a,\beta)\). (This is Example 4.6.8.)

Third display. The minus signs cancel in the ratio, so with

\begin{equation*} S = \sum_{j=1}^{a} V_j \sim \text{gamma}(a,1), \qquad T = \sum_{j=a+1}^{a+b} V_j \sim \text{gamma}(b,1) \end{equation*}

(by the second display with \(\beta = 1\)), which are independent because they are functions of disjoint subsets of the independent \(V_j\) (Theorem 4.6.12),

\begin{equation*} Y = \frac{\sum_{j=1}^{a} V_j}{\sum_{j=1}^{a+b} V_j} = \frac{S}{S+T} \sim \text{beta}(a,b) \end{equation*}

by Exercise 4.24.

Take \(U_1,\ldots,U_{(m+n)/2}\) iid uniform\((0,1)\) and set

\begin{equation*} X = \frac{n\sum_{j=1}^{m/2} \log U_j}{m\sum_{j=m/2+1}^{(m+n)/2} \log U_j} \sim F_{m,n} . \end{equation*}

Indeed, \(m/2\) and \(n/2\) are integers, so the first line of (5.6.5) applies to each block:

\begin{equation*} W_1 = -2\sum_{j=1}^{m/2}\log U_j \sim \chi^2_{m}, \qquad W_2 = -2\!\!\sum_{j=m/2+1}^{(m+n)/2}\!\!\log U_j \sim \chi^2_{n}, \end{equation*}

and \(W_1, W_2\) are independent, being functions of disjoint subsets of the independent \(U_j\) (Theorem 4.6.12). By the reduction on p. 225 (see Exercise 5.17), \(F_{m,n}\) is the distribution of \((W_1/m)/(W_2/n)\) for exactly such a pair, and the factors of \(-2\) cancel:

\begin{equation*} \frac{W_1/m}{W_2/n} = \frac{n}{m}\cdot\frac{-2\sum_{j=1}^{m/2}\log U_j}{-2\sum_{j=m/2+1}^{(m+n)/2}\log U_j} = X . \end{equation*}

Method (2): by Theorem 5.3.8(c), \(B = (m/n)X/[1+(m/n)X] \sim \text{beta}(m/2,n/2)\) when \(X \sim F_{m,n}\), so \(X = (n/m)B/(1-B)\). Generating \(B\) by the third line of (5.6.5) with \(a = m/2\), \(b = n/2\) and substituting reproduces the displayed \(X\) exactly.

(a) Both are exponential\((1)\). For \(v > 0\), since \(e^{-v} \in (0,1)\),

\begin{equation*} P(-\log U > v) = P(U < e^{-v}) = e^{-v}, \end{equation*}

and \(1 - U \sim \text{uniform}(0,1)\) as well (its cdf is \(P(1-U \le u) = P(U \ge 1-u) = u\) on \((0,1)\)), so the same computation gives \(P(-\log(1-U) > v) = e^{-v}\).

(b) For \(-\infty < x < \infty\), the map \(u \mapsto u/(1-u)\) is increasing on \((0,1)\), so

\begin{equation*} \begin{aligned} P(X \le x) &= P\!\left(\frac{U}{1-U} \le e^{x}\right)\\ &= P\!\left(U \le \frac{e^{x}}{1+e^{x}}\right)\\ &= \frac{e^{x}}{1+e^{x}} = \frac{1}{1+e^{-x}} , \end{aligned} \end{equation*}

the third equality because \(e^x/(1+e^x) \in (0,1)\). This is the logistic cdf with \(\mu = 0\), \(\beta = 1\).

(c) Set

\begin{equation*} Y = \mu + \beta\log\frac{U}{1-U} = \mu + \beta X . \end{equation*}

Since \(\beta > 0\), \(P(Y \le y) = P(X \le (y-\mu)/\beta) = \bigl[1+e^{-(y-\mu)/\beta}\bigr]^{-1}\), the logistic\((\mu,\beta)\) cdf. (This is (5.6.6), the Probability Integral Transform: \(F^{-1}(u) = \mu + \beta\log[u/(1-u)]\).)

\((X_1,X_2)\) is the polar point \((R\cos\Theta, R\sin\Theta)\) with \(\Theta = 2\pi U_1 \sim\) uniform\((0,2\pi)\) and \(R = \sqrt{-2\log U_2}\) Rayleigh, and (4.3.2) turns that pair into the standard bivariate normal density.

For \(r>0\),

\begin{equation*} \begin{aligned} P(R > r) &= P\!\left(-2\log U_2 > r^2\right)\\ &= P\!\left(U_2 < e^{-r^2/2}\right) = e^{-r^2/2}, \end{aligned} \end{equation*}

so \(f_R( r) = r e^{-r^2/2}\) on \(r>0\) (i.e. \(R^2 \sim\) exponential\((2) = \chi^2_2\)). Since \(U_1\) and \(U_2\) are independent, so are \(\Theta\) and \(R\) (Theorem 4.3.5), whence

\begin{equation*} f_{R,\Theta}(r,\theta) = \frac{1}{2\pi}\, r e^{-r^2/2}, \qquad r>0,\ 0<\theta<2\pi . \end{equation*}

The map \((r,\theta)\mapsto (r\cos\theta, r\sin\theta)\) carries \((0,\infty)\times(0,2\pi)\) one-to-one onto \(\mathbb{R}^2\) minus the ray \(\{(x_1,0): x_1\ge 0\}\), a set of measure zero. Its inverse is \(r = \sqrt{x_1^2+x_2^2}\), \(\theta = \arctan(x_2/x_1)\) taken in the correct quadrant, with Jacobian

\begin{equation*} J = \begin{vmatrix} \dfrac{\partial r}{\partial x_1} & \dfrac{\partial r}{\partial x_2}\\ \dfrac{\partial \theta}{\partial x_1} & \dfrac{\partial \theta}{\partial x_2} \end{vmatrix} = \frac{1}{\sqrt{x_1^2+x_2^2}} \end{equation*}

(Check!). Substituting into (4.3.2),

\begin{equation*} \begin{aligned} f_{X_1,X_2}(x_1,x_2) &= \frac{1}{2\pi}\sqrt{x_1^2+x_2^2}\; e^{-(x_1^2+x_2^2)/2} \cdot \frac{1}{\sqrt{x_1^2+x_2^2}}\\ &= \frac{1}{\sqrt{2\pi}}e^{-x_1^2/2}\cdot\frac{1}{\sqrt{2\pi}}e^{-x_2^2/2} \end{aligned} \end{equation*}

for all \((x_1,x_2)\in\mathbb{R}^2\). The joint pdf factors into a function of \(x_1\) alone times a function of \(x_2\) alone, so by Lemma 4.2.7 \(X_1\) and \(X_2\) are independent, and each factor is the \(\mathrm{n}(0,1)\) pdf.

(a) \(X\) is exactly the Central Limit Theorem statistic at \(n=12\). With \(U_i\sim\) uniform\((0,1)\) we have \(\mu = EU_i = 1/2\) and \(\sigma^2 = \operatorname{Var}U_i = 1/12\) (Example 5.5.3), so \(\sigma = 1/\sqrt{12}\) and

\begin{equation*} \frac{\sqrt{n}\,(\bar U_n - \mu)}{\sigma}\Bigg|_{n=12} = \frac{\sqrt{12}\left(\bar U_{12} - \tfrac12\right)}{1/\sqrt{12}} = \sum_{i=1}^{12} U_i - 6 = X . \end{equation*}

The uniform mgf exists everywhere, so Theorem 5.5.14 applies and this quantity tends to \(\mathrm{n}(0,1)\); the choice \(n=12\) is made purely so that \(\sigma\sqrt{n} = 1\) and no division is needed.

(b) The tails. \(0\le U_i\le 1\) forces \(-6\le X\le 6\) with probability one, so \(P(|X|>6)=0\), while a genuine \(\mathrm{n}(0,1)\) puts probability \(2\times 10^{-9}\) out there. No simulation built this way will ever produce a normal deviate beyond \(6\sigma\).

(c) Let \(V_i = U_i - \tfrac12 \sim\) uniform\(\left(-\tfrac12,\tfrac12\right)\), so \(X=\sum_{i=1}^{12}V_i\) and

\begin{equation*} M_{V}(t) = \int_{-1/2}^{1/2} e^{tv}\,dv = \frac{e^{t/2}-e^{-t/2}}{t} = \frac{\sinh(t/2)}{t/2}. \end{equation*}

By Theorem 4.2.12 (independence), \(M_X(t) = \left[\sinh(t/2)/(t/2)\right]^{12}\), so the cumulant generating function is

\begin{equation*} \begin{aligned} K_X(t) &= 12\log\frac{\sinh(t/2)}{t/2} = 12\left[\frac{(t/2)^2}{6}-\frac{(t/2)^4}{180}+\cdots\right]\\ &= \frac{t^2}{2}-\frac{t^4}{240}+\frac{t^6}{15120}-\cdots, \end{aligned} \end{equation*}

against \(K(t)=t^2/2\) for the \(\mathrm{n}(0,1)\). Hence \(\kappa_1=0\), \(\kappa_2=1\), \(\kappa_3=0\), and \(\kappa_4 = 4!\left(-\tfrac{1}{240}\right) = -\tfrac{1}{10}\), giving

\begin{equation*} EX=0,\quad EX^2=1,\quad EX^3=0,\quad EX^4 = \kappa_4+3\kappa_2^2 = \frac{29}{10}. \end{equation*}

(The odd moments vanish because \(X\) is symmetric about \(0\).) So the first three moments match the \(\mathrm{n}(0,1)\) exactly and the fourth is \(2.9\) against \(3\): the approximation is slightly light-tailed, a \(3.3\%\) kurtosis deficit.

Method (2): expand \(\left(\sum_i V_i\right)^4\) directly, using \(EV_1^2=1/12\), \(EV_1^4=1/80\) and the multinomial count \(\binom{4}{2,2}\binom{12}{2}=396\):

\begin{equation*} EX^4 = 12\,EV_1^4 + 3\cdot 12\cdot 11\,(EV_1^2)^2 = \frac{12}{80}+\frac{396}{144} = \frac{29}{10}. \end{equation*}

In each case apply the discrete inverse-cdf algorithm (5.6.7): generate \(U\sim\) uniform\((0,1)\) and set \(Y=y_{i+1}\) whenever \(F_Y(y_i) < U \le F_Y(y_{i+1})\), with \(F_Y(y_0)=0\).

(a) \(f_Y(y)=\binom{8}{y}\left(\tfrac23\right)^y\left(\tfrac13\right)^{8-y}\), \(y=0,\dots,8\), giving

so generate \(U\) and set \(Y=\min\{y: U\le F_Y(y)\}\); e.g. \(Y=5\) if \(0.2586<U\le 0.5318\). Scanning outward from the mean \(EY=16/3\) rather than from \(y=0\) cuts the expected number of comparisons (see Exercise 5.55).

Method (2): generate \(U_1,\dots,U_8\) iid uniform\((0,1)\) and set \(Y=\sum_{i=1}^{8}I\!\left(U_i\le \tfrac23\right)\), a sum of \(8\) iid Bernoulli\(\left(\tfrac23\right)\), which is binomial\(\left(8,\tfrac23\right)\) by the very definition of the binomial as a count of successes in independent Bernoulli trials (Section 3.2).

(b) With \(N=10\), \(M=8\), \(K=4\) the support is \(y=2,3,4\) (at most \(2\) of the \(4\) draws can come from the \(N-M=2\) non-special items), and \(f_Y(y)=\binom{8}{y}\binom{2}{4-y}\big/\binom{10}{4}\):

Generate \(U\) and set

\begin{equation*} Y=\begin{cases} 2 & \text{if } U\le 0.1333,\\ 3 & \text{if } 0.1333<U\le 0.6667,\\ 4 & \text{if } 0.6667<U\le 1 . \end{cases} \end{equation*}

Method (2): simulate the urn directly — draw \(4\) of the \(10\) items without replacement (at draw \(j\) pick item \(\lceil (10-j+1)U_j\rceil\) from those remaining) and let \(Y\) count how many of the \(8\) special items were drawn.

(c) \(f_Y(y)=\binom{y+4}{4}\left(\tfrac13\right)^5\left(\tfrac23\right)^y\), \(y=0,1,2,\dots\), with \(EY=r(1-p)/p=10\):

Generate \(U\) and return \(Y=\min\{y: U\le F_Y(y)\}\), computing \(F_Y\) on the fly by the recursion \(f_Y(y+1)=\frac{y+5}{y+1}\cdot\frac23 f_Y(y)\), \(f_Y(0)=(1/3)^5\), and starting the scan near \(y=10\).

Method (2): the support is unbounded, so the direct construction is preferable. Generate \(U_1,\dots,U_5\) iid uniform\((0,1)\) and set

\begin{equation*} Y=\sum_{j=1}^{5}\left\lfloor \frac{\log U_j}{\log(2/3)}\right\rfloor . \end{equation*}

Each summand is geometric on \(\{0,1,2,\dots\}\) with \(p=\tfrac13\), since

\begin{equation*} P\!\left(\left\lfloor \tfrac{\log U}{\log(2/3)}\right\rfloor = g\right) = P\!\left(\left(\tfrac23\right)^{g+1}<U\le \left(\tfrac23\right)^{g}\right) = \left(\tfrac23\right)^{g}\tfrac13, \end{equation*}

and a sum of \(5\) iid such geometrics is negative binomial\(\left(5,\tfrac13\right)\).

All three simulations reproduce their targets to within the sampling error the Weak Law (Theorem 5.5.2) allows. Running the three algorithms of Exercise 5.52 with \(n=1000\) replications each:

The theoretical values are \(EY=np=16/3\), \(\operatorname{Var}Y=np(1-p)=16/9\); \(EY=KM/N=3.2\), \(\operatorname{Var}Y=K\frac{M}{N}\frac{N-M}{N}\frac{N-K}{N-1}=0.4267\); and \(EY=r(1-p)/p=10\), \(\operatorname{Var}Y=r(1-p)/p^2=30\). Each discrepancy is within one standard error: e.g. for the negative binomial, \(\operatorname{sd}(\bar Y)=\sqrt{30/1000}=0.173\) and \(9.750\) is \(1.4\) standard errors from \(10\).

The histograms (observed counts against the expected counts \(1000 f_Y(y)\)) agree equally well.

Binomial\(\left(8,\tfrac23\right)\):

Hypergeometric\((10,8,4)\):

Negative binomial\(\left(5,\tfrac13\right)\):

Pearson statistics \(\sum(\text{obs}-\text{exp})^2/\text{exp}\) (pooling \(y\le 2\) in the binomial) come to \(10.22\) on \(6\) df, \(1.26\) on \(2\) df, and \(6.90\) on \(12\) df, with \(p\)-values \(0.12\), \(0.53\), \(0.86\): the simulated relative frequencies match the pmfs.

(a) The Poisson assumption is untenable: no simulated sample came close. For these \(n=13\) counts, \(\sum x_i = 1840\) and \(\sum x_i^2 = 336{,}878\), so

\begin{equation*} \bar x = \frac{1840}{13} = 141.54, \qquad s^2 = \frac{336{,}878 - 13\,\bar x^{2}}{12} = 6370.60 , \end{equation*}

a variance-to-mean ratio of \(s^2/\bar x = 45.0\) where the Poisson demands \(1\). Following Example 5.6.6, simulate \(M=10{,}000\) samples of size \(13\) from a Poisson\((141.54)\) and compute \(S_i^2\) for each. The resulting distribution is centred exactly where it should be, \(\overline{S^2} = 140.9 \approx \lambda\), with

and

\begin{equation*} P_\lambda\!\left(S^2 \ge 6370.60\right) \approx \frac{1}{M}\sum_{i=1}^{M} I\!\left(S_i^2 \ge 6370.60\right) = \frac{0}{10{,}000} . \end{equation*}

The observed \(s^2\) exceeds the largest of \(10{,}000\) simulated values by more than an order of magnitude, so Poisson counts are rejected.

(b) Matching the negative binomial variance to \(s^2\) at \(\mu = \bar x\) gives

\begin{equation*} \hat r = \frac{\bar x^{2}}{s^{2}-\bar x} = \frac{(141.54)^2}{6370.60-141.54} = 3.22 , \end{equation*}

and the simulation confirms that this and its neighbours are consistent. Setting \(p = r/(r+\bar x)\) so the mean stays at \(\bar x\), draw \(M = 20{,}000\) samples of size \(13\) from negative binomial\((r,p)\) and record \(S^2\):

Reading off the last column, \(s^2 = 6370.60\) falls in neither \(5\%\) tail of the simulated \(S^2\) distribution exactly when

\begin{equation*} r \lesssim 6.3 , \end{equation*}

\(r = 6.3\) being where \(P(S^2\ge s^2)\) drops through \(0.05\); \(r=8\) and beyond are rejected. Below \(r=1\) the tail probability peaks near \(0.93\) (at \(r\approx\tfrac12\)) and never reaches \(0.95\), so \(S^2\) alone never contradicts heavy clumping. Any \(r\) in \((0,6.3]\) is thus consistent, and the moment estimate \(\hat r = 3.22\) sits comfortably inside: there the simulated median of \(S^2\) is \(5627\) and the central \(90\%\) interval \([2231,\,13073]\) covers \(6370.60\).

The number of comparisons is \(N = Y+1\) pointwise: algorithm (5.6.7) tests “\(U\le F_Y(0)\)?”, then “\(U\le F_Y(1)\)?”, and so on, stopping at the first success and returning that index, so returning \(Y=y\) costs exactly \(y+1\) comparisons.

For the route the hint intends, note that the algorithm reaches its \((k+1)\)st comparison only if the first \(k\) failed, that is only if \(U > F_Y(k-1)\), which happens exactly when \(Y\ge k\). Hence \(P(N>k) = P(Y\ge k) = P(Y>k-1)\) for \(k\ge 1\), and \(P(N>0)=1\). Both \(N\) and \(Y\) are nonnegative-integer valued, so Exercise 2.14(b) applies to each:

\begin{equation*} \begin{aligned} EN &= \sum_{k=0}^{\infty} P(N>k) = 1 + \sum_{k=1}^{\infty} P(Y > k-1)\\ &= 1 + \sum_{j=0}^{\infty} P(Y>j) = 1 + EY = E(Y+1), \end{aligned} \end{equation*}

the first and last equalities being that exercise applied to \(N\) and to \(Y\).

(a) Errata: the printed pdf and cdf both drop a factor \(1/\pi\) — as printed \(f_Y\) integrates to \(\pi\) and \(\tan^{-1}\) is negative on \(y<0\), so neither is a legitimate density or cdf; the intended Cauchy\((0,1)\) statement, and the one part (b) needs, is \(f_Y(y)=\frac{1}{\pi}\frac{1}{1+y^2}\) with

\begin{equation*} F_Y(y) = \frac{1}{2} + \frac{1}{\pi}\tan^{-1}(y), \end{equation*}

which follows from \(\frac{d}{dt}\tan^{-1}(t) = \frac{1}{1+t^2}\):

\begin{equation*} \begin{aligned} F_Y(y) &= \int_{-\infty}^{y}\frac{1}{\pi}\frac{1}{1+t^2}\,dt = \frac{1}{\pi}\Big[\tan^{-1}(t)\Big]_{-\infty}^{y}\\ &= \frac{1}{\pi}\left(\tan^{-1}(y)+\frac{\pi}{2}\right) = \frac{1}{2}+\frac{1}{\pi}\tan^{-1}(y). \end{aligned} \end{equation*}

(b) \(X = a + b\tan\!\left(\pi\!\left(U-\tfrac12\right)\right)\) with \(U\sim\) uniform\((0,1)\).

The Cauchy\((a,b)\) pdf is \(f_X(x) = \frac{1}{\pi b}\left[1+\left(\frac{x-a}{b}\right)^2\right]^{-1}\), so \(X = a+bY\) and, by part (a),

\begin{equation*} F_X(x) = \frac{1}{2}+\frac{1}{\pi}\tan^{-1}\!\left(\frac{x-a}{b}\right), \qquad F_X^{-1}(u) = a + b\tan\!\left(\pi\!\left(u-\tfrac12\right)\right), \end{equation*}

this inverse being genuine since \(F_X\) is continuous and strictly increasing on \(\mathbb{R}\). By the Probability Integral Transform (Theorem 2.1.10), \(F_X^{-1}(U)\sim F_X\), which is the displayed recipe.

Method (2): \(\pi U\) is uniform on \((0,\pi)\) and \(\tan\) has period \(\pi\), so \(\tan(\pi U)\) has the same distribution as \(\tan\!\left(\pi(U-\tfrac12)\right)\), and \(X = a+b\tan(\pi U)\) serves equally well.

(a) The shared summand carries all the covariance:

\begin{equation*} \begin{aligned} \operatorname{Cov}(Y_1, Y_2) &= \operatorname{Cov}(X_1 + X_3,\, X_2 + X_3) \\ &= \operatorname{Var}(X_3) = \lambda_3, \end{aligned} \end{equation*}

the three cross terms vanishing by independence (Theorem 4.5.5) and \(\operatorname{Var}(X_3) = \lambda_3\) for a Poisson\((\lambda_3)\) variable.

(b) \(Z_i\) is an indicator, hence Bernoulli with success probability

\begin{equation*} P(Y_i = 0) = P(X_i = 0)P(X_3 = 0) = e^{-\lambda_i}e^{-\lambda_3} = p_i , \end{equation*}

using independence and \(P(X = 0) = e^{-\lambda}\) for \(X \sim\) Poisson\((\lambda)\). For the product, \(Z_1 Z_2 = 1\) exactly when all three Poissons vanish, so

\begin{equation*} \begin{aligned} E(Z_1 Z_2) &= P(X_1 = 0, X_2 = 0, X_3 = 0) \\ &= e^{-(\lambda_1 + \lambda_2 + \lambda_3)} = p_1 p_2 e^{\lambda_3}, \end{aligned} \end{equation*}

since \(p_1 p_2 = e^{-(\lambda_1 + \lambda_2 + 2\lambda_3)}\). Therefore \(\operatorname{Cov}(Z_1, Z_2) = p_1 p_2 e^{\lambda_3} - p_1 p_2 = p_1 p_2(e^{\lambda_3} - 1)\), and dividing by \(\sqrt{\operatorname{Var} Z_1 \operatorname{Var} Z_2} = \sqrt{p_1(1-p_1)}\sqrt{p_2(1-p_2)}\) gives the stated correlation.

(c) \(Z_1 Z_2 \le Z_i\) pointwise, so \(E(Z_1 Z_2) \le \min\{p_1, p_2\}\) and hence

\begin{equation*} \operatorname{Cov}(Z_1, Z_2) \le \min\{p_1, p_2\} - p_1 p_2 . \end{equation*}

(i) If \(p_1 \le p_2\) the bound is \(p_1(1 - p_2)\) and

\begin{equation*} \operatorname{Corr}(Z_1, Z_2) \le \frac{p_1(1-p_2)}{\sqrt{p_1(1-p_1)p_2(1-p_2)}} = \sqrt{\frac{p_1(1-p_2)}{p_2(1-p_1)}} . \end{equation*}

(ii) If \(p_2 \le p_1\) the same computation gives the reciprocal expression \(\sqrt{p_2(1-p_1)/[p_1(1-p_2)]}\).

The two candidate bounds are reciprocals of one another, and \(p_1 \le p_2 \iff p_1(1-p_2) \le p_2(1-p_1)\), so in each case the bound produced is the smaller of the two, i.e. the minimum. Since that minimum is \(\le 1\), with equality only when \(p_1 = p_2\), the attainable correlations are strictly confined inside \([-1,1]\).

(a) Induction on \(k\). For \(k = 1\), \(P(U_1 \le t) = t\). If the claim holds for \(k\), then conditioning on \(U_{k+1}\) and using \(0 \le t \le 1\),

\begin{equation*} \begin{aligned} P(S_{k+1} \le t) &= \int_0^1 P(S_k \le t - u)\,du \\ &= \int_0^t \frac{(t-u)^k}{k!}\,du = \frac{t^{k+1}}{(k+1)!} . \end{aligned} \end{equation*}

(b) Since the \(U_i\) are positive, \(\{S_n < 1\} \subset \{S_{n-1} < 1\}\), and \(\{N = n\} = \{S_{n-1} < 1\} \setminus \{S_n < 1\}\), so

\begin{equation*} P(N = n) = P(S_{n-1} < 1) - P(S_n < 1) = \frac{1}{(n-1)!} - \frac{1}{n!} , \end{equation*}

by part (a) at \(t = 1\) (the \(S_k\) are continuous, so \(<\) and \(\le\) agree). The same identity in tail form reads \(P(N > n) = P(S_n < 1) = 1/n!\) for \(n \ge 0\), whence by Exercise 2.14(b) applied to the nonnegative integer variable \(N\),

\begin{equation*} E(N) = \sum_{n=0}^{\infty} P(N > n) = \sum_{n=0}^{\infty} \frac{1}{n!} = e . \end{equation*}

(c) Repeat independently \(m\) times: draw uniform\((0,1)\) variables one at a time, accumulating the sum, and record the index \(N_j\) at which the running sum first exceeds \(1\). By the WLLN (Theorem 5.5.2), \(\bar{N} = m^{-1}\sum_{j=1}^m N_j \to E(N) = e\) in probability, so \(\bar{N}\) estimates \(e\).

(d) About \(n = 1.2 \times 10^7\) replications. From part (b),

\begin{equation*} \begin{aligned} E[N(N-1)] &= \sum_{n \ge 2} \left( \frac{n}{(n-2)!} - \frac{1}{(n-2)!} \right) \\ &= (e + 2e) - e = 2e , \end{aligned} \end{equation*}

so \(E N^2 = 2e + e = 3e\) and

\begin{equation*} \operatorname{Var}(N) = 3e - e^2 = 0.7658, \qquad \sigma = 0.8751 . \end{equation*}

Getting the first four digits right means \(|\bar{N} - e| < 0.0005\), and by the CLT (Theorem 5.5.14) this holds with probability \(0.95\) when \(1.96\,\sigma/\sqrt{n} \le 0.0005\), i.e.

\begin{equation*} n \ge \left( \frac{1.96 \times 0.8751}{0.0005} \right)^{2} = 1.177 \times 10^{7} . \end{equation*}

The accepted \(V\) has cdf \(F_Y\). A finite \(c \ge \max_y f_Y(y)\) exists exactly when the beta pdf is bounded, i.e. \(a \ge 1\) and \(b \ge 1\); then \(f_Y(v)/c \le 1\), and \(y = 1\) in (5.6.10) gives the per-trial acceptance probability

\begin{equation*} q = P\!\left(U < \tfrac{1}{c}f_Y(V)\right) = \frac{1}{c} > 0 , \end{equation*}

so the pass count \(T = \min\{i : U_i < \frac{1}{c}f_Y(V_i)\}\) at which the algorithm stops is geometric\((q)\), finite with probability one, and \(Y = V_T\). The pairs \((U_i, V_i)\) drawn on successive passes are iid, so \(\{T = i\}\) splits into \(i-1\) rejections and acceptance on the \(i\)th pair, and

\begin{equation*} \begin{aligned} P(Y \le y) &= \sum_{i=1}^{\infty} (1-q)^{i-1} P\!\left(V \le y,\, U < \tfrac{1}{c}f_Y(V)\right) \\ &= \frac{1}{q}\cdot\frac{F_Y(y)}{c} = F_Y(y), \end{aligned} \end{equation*}

the inner probability being \(F_Y(y)/c\) by (5.6.10).

(a) For \(a \le w \le b\) the event \(\{W \le w\}\) is exactly \(\{X \le w,\, Y < f(X)\}\) (since \(d > b \ge w\)), so conditioning on \(X\) and using \(P(Y < f(x)) = f(x)/c\),

\begin{equation*} \begin{aligned} P(W \le w) &= \int_a^w P\big(Y < f(x)\big)\,\frac{dx}{b-a} \\ &= \int_a^w \frac{f(x)}{c}\cdot\frac{dx}{b-a} = \frac{\int_a^w f(t)\,dt}{c(b-a)} . \end{aligned} \end{equation*}

(b) Generate independent pairs \((X, Y)\) until one satisfies \(Y < f(X)\), and output that \(X\); its pdf is \(f\). Indeed, taking \(w = b\) in part (a),

\begin{equation*} P(W \le b) = \frac{1}{c(b-a)} = P\big(Y < f(X)\big), \end{equation*}

so for \(a \le w \le b\),

\begin{equation*} P(W \le w \mid W \ne d) = \frac{\int_a^w f(t)\,dt / [c(b-a)]}{1/[c(b-a)]} = \int_a^w f(t)\,dt , \end{equation*}

which is the cdf belonging to \(f\); independence across pairs turns this conditional law into the law of the first accepted \(X\), exactly as in Exercise 5.59. Geometrically, \((X,Y)\) is uniform on the rectangle \([a,b] \times [0,c]\), whose area is \(c(b-a)\); acceptance means the point lands in the region under the curve \(y = f(x)\), of area \(1\), and a uniform point conditioned to lie in that region has horizontal coordinate with density \(f\).

Write \(\alpha = [a]\), \(\beta = [b]\), \(r = a - \alpha \in (0,1)\) and \(s = b - \beta \in (0,1)\); both \(a, b > 1\) so that \(\alpha, \beta \ge 1\) and the candidate densities exist. Two elementary suprema are used below: \(\sup_{0 \le y \le 1} y^r(1-y)^s = r^r s^s/(r+s)^{r+s}\) at \(y = r/(r+s)\), and \(\sup_{y > 0} y^r e^{-\lambda y} = (r/\lambda)^r e^{-r}\) at \(y = r/\lambda\).

(a) The ratio is a bounded power function:

\begin{equation*} \frac{f_Y(y)}{f_V(y)} = \frac{B(\alpha, \beta)}{B(a,b)}\, y^{a-\alpha}(1-y)^{b-\beta}, \qquad 0 < y < 1 , \end{equation*}

and since \(a - \alpha = r > 0\) and \(b - \beta = s > 0\) both exponents are positive, so the ratio extends continuously to \([0,1]\) with

\begin{equation*} M = \frac{B(\alpha,\beta)}{B(a,b)}\cdot\frac{r^r s^s}{(r+s)^{r+s}} < \infty . \end{equation*}

(b) As printed this is false and the scale must be changed: with the common scale \(b\) the exponential factors cancel and

\begin{equation*} \frac{f_Y(y)}{f_V(y)} = \frac{\Gamma(\alpha)\, b^{\alpha}}{\Gamma(a)\, b^{a}}\, y^{a - \alpha} \xrightarrow[y \to \infty]{} \infty , \end{equation*}

so \(M = \infty\). The intended statement is \(V \sim\) gamma\(([a], \beta_0)\) with a scale \(\beta_0 > b\) (the candidate must have the heavier tail), and then

\begin{equation*} \frac{f_Y(y)}{f_V(y)} = \frac{\Gamma(\alpha)\beta_0^{\alpha}}{\Gamma(a)b^{a}}\, y^{r} e^{-\lambda y}, \qquad \lambda = \frac{1}{b} - \frac{1}{\beta_0} > 0 , \end{equation*}

whence

\begin{equation*} M = \frac{\Gamma(\alpha)\beta_0^{\alpha}}{\Gamma(a)b^{a}} \left(\frac{r}{\lambda}\right)^{\!r} e^{-r} < \infty . \end{equation*}

(c) Replacing \(\alpha\) by \(\alpha + 1\) turns the exponent of \(y\) into \(a - \alpha - 1 = r - 1 \in (-1, 0)\), and in both cases the remaining factor tends to a positive constant as \(y \to 0\):

\begin{equation*} \frac{f_Y(y)}{f_V(y)} \sim \text{const} \cdot y^{\,r-1} \xrightarrow[y \to 0^{+}]{} \infty . \end{equation*}

(In the beta case \(\beta + 1\) fails the same way at \(y \to 1^{-}\), the exponent of \(1-y\) becoming \(s - 1 < 0\).) So \(M = \infty\): the candidate must have both parameters no larger than the target’s.

(d) By the remark following Theorem 5.6.8, the number \(N\) of candidate pairs needed per accepted variate is geometric\((1/M)\), so \(E(N) = M\) and optimizing means minimizing \(M\) over the admissible candidates.

Gamma. Here the shape must be an integer \(\alpha \le a\) (so that \(V\) is a sum of \(\alpha\) iid exponentials, generable directly by (5.6.5)), while the scale \(\beta_0 > b\) is a free real parameter. Substituting \(\lambda = (\beta_0 - b)/(b\beta_0)\) into (b) and using \(\alpha + r = a\),

\begin{equation*} M(\beta_0) = \frac{\Gamma(\alpha) r^{r} e^{-r} b^{\,r-a}}{\Gamma(a)} \cdot \frac{\beta_0^{\,a}}{(\beta_0 - b)^{r}} , \end{equation*}

and \(\frac{d}{d\beta_0}\left[a\log\beta_0 - r\log(\beta_0-b)\right] = 0\) gives

\begin{equation*} \frac{a}{\beta_0} = \frac{r}{\beta_0 - b} \;\Longrightarrow\; \beta_0^{*} = \frac{ab}{a - r} = \frac{ab}{\alpha} , \end{equation*}

a minimum since the derivative changes sign from \(-\) to \(+\). Note \(\alpha\beta_0^{*} = ab\): the optimal candidate has exactly the target’s mean. Back-substituting,

\begin{equation*} M^{*}(\alpha) = \frac{\Gamma(\alpha)}{\Gamma(a)}\cdot\frac{a^{a}}{\alpha^{\alpha}}\, e^{-(a-\alpha)} , \end{equation*}

which is free of \(b\) and decreasing in \(\alpha\), because

\begin{equation*} \frac{M^{*}(\alpha+1)}{M^{*}(\alpha)} = \frac{e}{\left(1 + 1/\alpha\right)^{\alpha+1}} < 1 \end{equation*}

(the sequence \((1+1/\alpha)^{\alpha+1}\) decreases to \(e\) from above). Hence take the largest admissible shape: \(V \sim\) gamma\(\left([a],\, ab/[a]\right)\), with

\begin{equation*} E(N) = \frac{\Gamma([a])\, a^{a}}{\Gamma(a)\, [a]^{[a]}}\, e^{-(a - [a])} . \end{equation*}

For \(a = 2.7\) this is \(\beta_0^{*} = 1.35\,b\) and \(E(N) = 1.174\).

Beta. Both parameters must be integers with \(\alpha \le a\), \(\beta \le b\) by part (c), so the admissible set is the finite grid \(1 \le \alpha \le [a]\), \(1 \le \beta \le [b]\), and one minimizes the closed form of part (a),

\begin{equation*} M(\alpha, \beta) = \frac{B(\alpha,\beta)}{B(a,b)} \cdot \frac{r^{r} s^{s}}{(r+s)^{r+s}}, \quad r = a - \alpha,\; s = b - \beta, \end{equation*}

over that grid; there is no closed-form minimizer, and \(([a], [b])\) is generally not it. For \(a = 2.7\), \(b = 6.3\) the grid of \(M(\alpha,\beta)\) values is

so the optimum is \(V \sim\) beta\((2,5)\) with \(E(N) = 1.18\) – better than the beta\((2,6)\) of Example 5.6.9 (\(E(N) = 1.67\)) and than the uniform \(=\) beta\((1,1)\) of Example 5.6.7 (\(E(N) = c = 2.67\)).

(a) \(M = \sqrt{2\pi}\,e^{-1/2} = 1.520\). With \(f_V(v) = [\pi(1+v^2)]^{-1}\),

\begin{equation*} \frac{f_Y(y)}{f_V(y)} = \sqrt{\frac{\pi}{2}}\,(1 + y^{2})\,e^{-y^{2}/2} , \end{equation*}

and writing \(t = y^{2} \ge 0\), \(\frac{d}{dt}\left[(1+t)e^{-t/2}\right] = \frac{1-t}{2}e^{-t/2}\) vanishes at \(t = 1\), a maximum, with value \(2e^{-1/2}\). Hence \(M = 2\sqrt{\pi/2}\,e^{-1/2} = \sqrt{2\pi}\,e^{-1/2} = 1.520\).

(b) \(M = \sqrt{2/\pi}\,e^{1/2} = 1.316\). With \(f_V(v) = \frac{1}{2}e^{-|v|}\),

\begin{equation*} \frac{f_Y(y)}{f_V(y)} = \sqrt{\frac{2}{\pi}}\; e^{\,|y| - y^{2}/2} , \end{equation*}

and \(|y| - y^{2}/2\) is maximized at \(|y| = 1\) with value \(1/2\), so \(M = \sqrt{2/\pi}\,e^{1/2} = 1.316\).

Both suprema are finite because each candidate has heavier tails than the normal, as Theorem 5.6.8 requires.

(c) The double exponential. Its \(M\) is smaller, so by the remark after Theorem 5.6.8 it needs \(E(N) = 1.32\) candidate pairs per normal variate against \(1.52\) for the Cauchy, a 13% saving; and a signed exponential costs only a logarithm of a uniform, while a Cauchy costs the transcendental \(\tan(\pi(U - 1/2))\) of Exercise 5.56.

\(\lambda = 1\), giving \(M = \sqrt{2/\pi}\,e^{1/2} = 1.316\). The signed exponential has density \(f_V(v) = \frac{1}{2\lambda}e^{-|v|/\lambda}\) on \((-\infty, \infty)\), so

\begin{equation*} \frac{f_Y(y)}{f_V(y)} = \frac{2\lambda}{\sqrt{2\pi}}\; \exp\!\left\{\frac{|y|}{\lambda} - \frac{y^{2}}{2}\right\} , \end{equation*}

whose exponent is maximized at \(|y| = 1/\lambda\) with value \(1/(2\lambda^{2})\). Hence

\begin{equation*} M(\lambda) = \lambda\sqrt{\frac{2}{\pi}}\; e^{1/(2\lambda^{2})}, \qquad \frac{d}{d\lambda}\log M = \frac{1}{\lambda} - \frac{1}{\lambda^{3}} , \end{equation*}

which vanishes only at \(\lambda = 1\) and is negative for \(\lambda < 1\), positive for \(\lambda > 1\), so \(\lambda = 1\) minimizes \(E(N) = M\), at \(M(1) = \sqrt{2/\pi}\,e^{1/2} = 1.316\).

Write \(W_i = \dfrac{f(Y_i)}{g(Y_i)}h(Y_i)\) and \(w_i = \dfrac{f(Y_i)}{g(Y_i)}\); the whole exercise is the single change-of-density identity \(E\big[\phi(Y)f(Y)/g(Y)\big] = E\phi(X)\).

(a) Since \(f\) and \(g\) have the same support, \(g > 0\) wherever \(f > 0\) and

\begin{equation*} \begin{aligned} E W_1 &= \int \frac{f(y)}{g(y)}h(y)\, g(y)\,dy\\ &= \int h(y) f(y)\, dy = Eh(X), \end{aligned} \end{equation*}

and the \(W_i\) are identically distributed, so the average has this same mean.

(b) The \(W_i\) are iid with finite mean \(Eh(X)\) by (a), so the WLLN (Theorem 5.5.2, in the version requiring only a finite mean, noted in the remark following that theorem) gives

\begin{equation*} \frac{1}{m}\sum_{i=1}^{m} W_i \longrightarrow Eh(X) \quad \text{in probability.} \end{equation*}

(c) Divide numerator and denominator by \(m\):

\begin{equation*} \sum_{i=1}^{m}\left(\frac{w_i}{\sum_j w_j}\right)h(Y_i) = \frac{\frac{1}{m}\sum_{i=1}^{m} w_i h(Y_i)}{\frac{1}{m}\sum_{j=1}^{m} w_j}. \end{equation*}

The numerator is exactly the estimator of (b), so it converges in probability to \(Eh(X)\). The denominator is an average of iid variables with

\begin{equation*} E w_1 = \int \frac{f(y)}{g(y)}\, g(y)\, dy = \int f(y)\,dy = 1, \end{equation*}

so it converges in probability to \(1\). By Slutsky’s Theorem (Theorem 5.5.17) the ratio converges in probability to \(Eh(X)/1 = Eh(X)\).

If \(h \equiv c\), the estimator of (c) is

\begin{equation*} \sum_{i=1}^{m}\frac{w_i}{\sum_j w_j}\cdot c = c \end{equation*}

identically: it estimates \(Eh(X) = c\) with zero error. The estimator of (a) is \(c\,\overline{w}\), with

\begin{equation*} \operatorname{Var}\big(c\,\overline{w}\big) = \frac{c^2}{m}\operatorname{Var}(w_1) > 0 \end{equation*}

unless \(f = g\) a.e. So (c) has MSE \(0\) and (a) has strictly positive MSE.

Conditionally on the \(Y\)s, the cdf of \(X^{*}\) is the self-normalized importance-sampling estimator of Exercise 5.64(c) applied to \(h(y) = I(y \le x)\), and therefore converges to \(F_X(x)\).

Write \(w_i = f(Y_i)/g(Y_i)\). Since \(X^{*}\) is supported on \(\{Y_1,\ldots,Y_m\}\) with masses \(q_i\),

\begin{equation*} \widehat{F}_m(x) := P\big(X^{*} \le x \mid Y_1,\ldots,Y_m\big) = \frac{\frac{1}{m}\sum_{i=1}^{m} w_i I(Y_i \le x)}{\frac{1}{m}\sum_{j=1}^{m} w_j}. \end{equation*}

The numerator is an average of iid variables with mean

\begin{equation*} E\left[\frac{f(Y)}{g(Y)}I(Y \le x)\right] = \int_{-\infty}^{x} \frac{f(y)}{g(y)}\, g(y)\, dy = F_X(x), \end{equation*}

and the denominator is an average of iid variables with mean \(\int f = 1\); both are finite, so the WLLN (Theorem 5.5.2) and Slutsky’s Theorem (Theorem 5.5.17) give, for each fixed \(x\),

\begin{equation*} \widehat{F}_m(x) \longrightarrow F_X(x) \quad \text{in probability as } m \to \infty . \end{equation*}

Given the \(Y\)s, \(X_1^{*},\ldots,X_r^{*}\) are drawn independently from this discrete distribution, so for fixed \(x_1,\ldots,x_r\),

\begin{equation*} P\big(X_1^{*} \le x_1, \ldots, X_r^{*} \le x_r\big) = E\left[\prod_{k=1}^{r} \widehat{F}_m(x_k)\right]. \end{equation*}

The integrand lies in \([0,1]\) and converges in probability to \(\prod_k F_X(x_k)\) (Theorem 5.5.4, \(h\) = product), so its expectation converges to \(\prod_{k=1}^{r} F_X(x_k)\) by bounded convergence. The joint cdf of \(X_1^{*},\ldots,X_r^{*}\) therefore tends to that of \(r\) iid observations from \(f\), which for finite \(m\) they are only approximately, being drawn from the common pool \(Y_1,\ldots,Y_m\).

(a) The map \(x \mapsto (x-\mu)/\sigma\) is strictly increasing, so it preserves the ordering: \(Z_{(i)} = (X_{(i)} - \mu)/\sigma\) for every \(i\). Applying it to (5.4.1),

\begin{equation*} \begin{aligned} n \text{ odd}: \quad &\frac{X_{((n+1)/2)} - \mu}{\sigma} = Z_{((n+1)/2)},\\ n \text{ even}: \quad &\frac{\frac{1}{2}\big(X_{(n/2)}+X_{(n/2+1)}\big)-\mu}{\sigma} = \tfrac{1}{2}\big(Z_{(n/2)}+Z_{(n/2+1)}\big), \end{aligned} \end{equation*}

which is exactly the median of the \(Z_i = (X_i-\mu)/\sigma\). Since \(Z_i \sim \mathrm{n}(0,1)\), the distribution of \(M\) is that of \(\mu + \sigma M_Z\) with \(M_Z\) the median of a standard normal sample.

(b) With \(n = 15\) (odd), \(M = X_{(8)}\), so by Theorem 5.4.4 its exact pdf is

\begin{equation*} f_M(m) = \frac{15!}{7!\,7!}\, \Phi(m)^{7}\big(1-\Phi(m)\big)^{7}\varphi(m), \end{equation*}

symmetric about \(0\). A simulation of \(200{,}000\) samples of size \(15\) gave

\begin{equation*} \bar{M} = 0.0012, \qquad s_M^2 = 0.1010, \qquad s_M = 0.3178, \end{equation*}

matching the exact moments obtained by integrating \(f_M\): \(EM = 0\) and \(\operatorname{Var} M = 0.10169\), \(\mathrm{sd}(M) = 0.31890\).

(c) Here \(f = \varphi\), \(\varphi(0) = 1/\sqrt{2\pi}\), so the asymptotic variance is

\begin{equation*} \frac{1}{n}\cdot\frac{1}{4\varphi^2(0)} = \frac{1}{n}\cdot\frac{2\pi}{4} = \frac{\pi}{2n} = \frac{\pi}{30} = 0.10472 , \end{equation*}

i.e. \(M \approx \mathrm{n}(0, 0.10472)\), sd \(0.32360\). The asymptotic variance overstates the exact one by only \(0.10472/0.10169 = 1.030\), i.e. \(3\%\), and the quantiles agree closely: