Both sentences are satisfied by the world \(\omega\) with \(\omega(A) = \mathsf{false}\) and \(\omega(B) = \mathsf{false}\).

(a) Since \(\omega \models \neg A\), both \(\neg A \vee B\) and \(\neg A \vee \neg B\) hold at \(\omega\), hence so does their conjunction; here \(\alpha \Rightarrow \beta\) abbreviates \(\neg\alpha \vee \beta\) (Table 2.2). Distributing and using that \(B \wedge \neg B\) is inconsistent (Table 2.2),

\begin{equation*} \begin{aligned} (A \Rightarrow B) \wedge (A \Rightarrow \neg B) &\;=\; \neg A \vee (B \wedge \neg B) \;=\; \neg A , \end{aligned} \end{equation*}

the form Exercise 2.3(b) reuses; its models are the two worlds with \(A\) false.

(b) At \(\omega\) the antecedent \(A \vee B\) is false, so the implication holds. Writing \(\gamma = A \vee B\), de Morgan turns the sentence into \(\gamma \Rightarrow \neg\gamma = \neg\gamma \vee \neg\gamma = \neg A \wedge \neg B\), whose unique model is \(\omega\).

The truth table confirms both columns; a sentence is satisfied at the worlds carrying “true” in its column.

(a) Valid; (b) not valid, falsified at \(\omega(A) = \mathsf{false}\), \(\omega(B) = \mathsf{false}\); (c) valid. Throughout \(\alpha \Rightarrow \beta\) is \(\neg\alpha \vee \beta\), and validity means \(\mathrm{Mods}(\alpha) = \Omega\) (Section 2.3.2).

(a) is modus ponens: if some \(\omega\) falsified it then \(\omega \models A\) and \(\omega \models \neg A \vee B\) while \(\omega \not\models B\), so both disjuncts of \(\neg A \vee B\) fail at \(\omega\) — impossible.

(b) By distribution and the validity of \(B \vee \neg B\) (Table 2.2),

\begin{equation*} (A \wedge B) \vee (A \wedge \neg B) \;=\; A \wedge (B \vee \neg B) \;=\; A , \end{equation*}

so the sentence fails at exactly the worlds setting \(A\) to false, the exhibited world among them.

(c) Table 2.2 lists the contrapositive \(\neg\beta \Rightarrow \neg\alpha\) as equivalent to \(\alpha \Rightarrow \beta\), so antecedent and consequent have the same truth value at every world and the implication is never false.

The truth table records all three verdicts:

(a) Not equivalent — they disagree at \(\omega(A) = \mathsf{false}\), \(\omega(B) = \mathsf{true}\); (b) and (c) are equivalent. Equivalence means equality of model sets (Section 2.3.3).

(a) At that world \(A \Rightarrow B = \neg A \vee B\) holds via \(\neg A\), while \(B \Rightarrow A = \neg B \vee A\) has both disjuncts false.

(b) By Exercise 2.1(a), \((A \Rightarrow B) \wedge (A \Rightarrow \neg B) = \neg A\), and equivalence-preserving rewriting leaves the model set unchanged.

(c) Both sides reduce to \(A \vee \neg B\). On the left, \(\neg A \Rightarrow \neg B = \neg\neg A \vee \neg B = A \vee \neg B\); on the right, distribution (Table 2.2) with \(\alpha = A \vee \neg B\), \(\beta = C\), \(\gamma = \neg C\) gives

\begin{equation*} \begin{aligned} (A \vee \neg B \vee C) \wedge (A \vee \neg B \vee \neg C) &\;=\; (A \vee \neg B) \vee (C \wedge \neg C) \\ &\;=\; A \vee \neg B . \end{aligned} \end{equation*}

The right sentence therefore does not depend on \(\omega( C)\), which is what makes the comparison over the eight worlds below legitimate.

(a) The implication fails, at \(\omega(A) = \mathsf{false}\), \(\omega(B) = \mathsf{false}\); (b) and (c) hold. Here \(\alpha \models \beta\) means \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\) (Section 2.3.3), and \(\beta\) is the atomic sentence \(A\) in all three parts.

(a) At the exhibited world \(\neg A \vee B\) holds via \(\neg A\) and \(\neg B\) holds, yet \(\omega \not\models A\). Distribution and the inconsistency of \(B \wedge \neg B\) (Table 2.2) give

\begin{equation*} (A \Rightarrow B) \wedge \neg B \;=\; (\neg A \wedge \neg B) \vee (B \wedge \neg B) \;=\; \neg A \wedge \neg B , \end{equation*}

so the first sentence entails \(\neg A\) instead — modus tollens.

(b) \((A \vee \neg B) \wedge B = (A \wedge B) \vee (\neg B \wedge B) = A \wedge B\), and \(\mathrm{Mods}(A \wedge B) = \mathrm{Mods}(A) \cap \mathrm{Mods}(B) \subseteq \mathrm{Mods}(A)\).

(c) \((A \vee B) \wedge (A \vee \neg B) = A \vee (B \wedge \neg B) = A\), so the implication holds, in both directions.

Over the four worlds:

(a) Not mutually exclusive — both hold at \(\omega(A) = \mathsf{true}\), \(\omega(B) = \mathsf{false}\) — but exhaustive; (b) and (c) are both mutually exclusive and exhaustive. By Section 2.3.3 these ask whether \(\mathrm{Mods}(\alpha) \cap \mathrm{Mods}(\beta) = \emptyset\) and whether \(\mathrm{Mods}(\alpha) \cup \mathrm{Mods}(\beta) = \Omega\).

(a) At the exhibited world \(A \vee B\) holds via \(A\) and \(\neg A \vee \neg B\) via \(\neg B\). The first sentence excludes only the world with \(A\) and \(B\) both false, the second only the world with both true, so no world is excluded by both: the pair is exhaustive. (Equivalently, the disjunction of the two has \(A\) and \(\neg A\) among its disjuncts.)

(b) By de Morgan (Table 2.2) the second sentence is \(\neg(A \vee B)\), the negation of the first, and \(\mathrm{Mods}(\neg\gamma) = \overline{\mathrm{Mods}(\gamma)}\); a set and its complement are disjoint and cover \(\Omega\).

(c) Distribution and the inconsistency of \(B \wedge \neg B\) (Table 2.2) give \((\neg A \vee B) \wedge (\neg A \vee \neg B) = \neg A\), so the pair is \(A\) and \(\neg A\), again complementary.

In the table below, mutual exclusivity of a pair means no row carries “true” in both of its columns, exhaustiveness that no row carries “false” in both.

By Section 2.3.3, \(\alpha \models \beta\) iff \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\); by the identities \(\mathrm{Mods}(\alpha \wedge \gamma) = \mathrm{Mods}(\alpha) \cap \mathrm{Mods}(\gamma)\) and \(\mathrm{Mods}(\neg\beta) = \overline{\mathrm{Mods}(\beta)}\) of Section 2.3.1 (complement taken in the set \(\Omega\) of all worlds),

\begin{equation*} \begin{aligned} \mathrm{Mods}(\alpha \wedge \neg\beta) &\;=\; \mathrm{Mods}(\alpha) \cap \overline{\mathrm{Mods}(\beta)} \\ &\;=\; \mathrm{Mods}(\alpha) \setminus \mathrm{Mods}(\beta) . \end{aligned} \end{equation*}

For any sets, \(S \setminus T = \emptyset\) iff \(S \subseteq T\). Hence the left-hand side is empty — that is, \(\alpha \wedge \neg\beta\) is inconsistent in the sense of Section 2.3.2 — exactly when \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\), i.e. exactly when \(\alpha \models \beta\). \(\blacksquare\)

Since \(\alpha \Rightarrow \beta\) abbreviates \(\neg\alpha \vee \beta\) (Section 2.2), the model-set identities of Section 2.3.1 give

\begin{equation*} \begin{aligned} \mathrm{Mods}(\alpha \Rightarrow \beta) &\;=\; \mathrm{Mods}(\neg\alpha) \cup \mathrm{Mods}(\beta) \\ &\;=\; \overline{\mathrm{Mods}(\alpha)} \cup \mathrm{Mods}(\beta) . \end{aligned} \end{equation*}

For subsets \(S, T \subseteq \Omega\) one has \(\overline{S} \cup T = \Omega\) iff \(S \subseteq T\), since the worlds outside \(\overline{S}\) are precisely those in \(S\). Taking \(S = \mathrm{Mods}(\alpha)\) and \(T = \mathrm{Mods}(\beta)\): validity of \(\alpha \Rightarrow \beta\), i.e. \(\mathrm{Mods}(\alpha \Rightarrow \beta) = \Omega\) (Section 2.3.2), holds iff \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\), i.e. iff \(\alpha \models \beta\). \(\blacksquare\)

Method (2): by Exercise 2.6, \(\alpha \models \beta\) iff \(\alpha \wedge \neg\beta\) is inconsistent, and a sentence \(\gamma\) is inconsistent iff \(\neg\gamma\) is valid (complementation again). De Morgan and double negation (Table 2.2) turn \(\neg(\alpha \wedge \neg\beta)\) into \(\neg\alpha \vee \beta = \alpha \Rightarrow \beta\).

Assume \(\alpha \models \beta\), that is, \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\) (Section 2.3.3). Since \(S \subseteq T\) implies \(S \cap T = S\), the identity of Section 2.3.1 gives

\begin{equation*} \begin{aligned} \mathrm{Mods}(\alpha \wedge \beta) &\;=\; \mathrm{Mods}(\alpha) \cap \mathrm{Mods}(\beta) \\ &\;=\; \mathrm{Mods}(\alpha) , \end{aligned} \end{equation*}

and sentences with identical model sets are equivalent. \(\blacksquare\)

Assume \(\alpha \models \beta\), that is, \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\) (Section 2.3.3). Since \(S \subseteq T\) implies \(S \cup T = T\), the identity of Section 2.3.1 gives

\begin{equation*} \begin{aligned} \mathrm{Mods}(\alpha \vee \beta) &\;=\; \mathrm{Mods}(\alpha) \cup \mathrm{Mods}(\beta) \\ &\;=\; \mathrm{Mods}(\beta) , \end{aligned} \end{equation*}

so \(\alpha \vee \beta\) and \(\beta\) are equivalent. \(\blacksquare\)

Method (2): \(\alpha \models \beta\) iff \(\neg\beta \models \neg\alpha\), so Exercise 2.8 applied to that pair gives \(\neg\beta \wedge \neg\alpha \equiv \neg\beta\); negating both sides and using de Morgan (Table 2.2) yields \(\alpha \vee \beta \equiv \beta\).

(a) The single clause \(\neg P \vee \neg Q \vee R\); (b) the four-clause CNF displayed below. Both parts run the procedure of Section 2.7.3: eliminate implications, push negations onto variables, distribute \(\vee\) over \(\wedge\).

(a) Eliminating the inner then the outer implication,

\begin{equation*} P \Rightarrow (Q \Rightarrow R) \;\equiv\; \neg P \vee (\neg Q \vee R) \;\equiv\; \neg P \vee \neg Q \vee R , \end{equation*}

already in NNF, with no conjunction to distribute over.

(b) Eliminating implications, then de Morgan twice on the outer negation and double-negation removal,

\begin{equation*} \begin{aligned} \neg\big((P \Rightarrow Q) \wedge (R \Rightarrow S)\big) &\equiv \neg\big((\neg P \vee Q) \wedge (\neg R \vee S)\big) \\ &\equiv (P \wedge \neg Q) \vee (R \wedge \neg S) . \end{aligned} \end{equation*}

Distributing over each conjunction in turn,

\begin{equation*} \begin{aligned} (P \wedge \neg Q) \vee (R \wedge \neg S) &\equiv \big((P \wedge \neg Q) \vee R\big) \\ &\qquad \wedge \big((P \wedge \neg Q) \vee \neg S\big) \\ &\equiv (P \vee R) \wedge (\neg Q \vee R) \\ &\qquad \wedge (P \vee \neg S) \wedge (\neg Q \vee \neg S). \end{aligned} \end{equation*}

No clause is a tautology or subsumed by another, so no clause can be dropped. The table compares \(\varphi = \neg((P \Rightarrow Q) \wedge (R \Rightarrow S))\) with this CNF \(\psi\) over all sixteen worlds, writing \(I_1 = P \Rightarrow Q\) and \(I_2 = R \Rightarrow S\):

Conjoin onto each or-node edge the valid gadgets \(X \vee \neg X\) for the variables its child is missing; with \(m = |\Gamma|\) edges and \(v\) variables this costs \(O(mv)\) time and yields a circuit of size \(O(mv)\).

Write \(\mathrm{Vars}(N)\) for the variables mentioned in the subcircuit at \(N\), and assume every node is reachable from the root.

1. One reverse-topological sweep computes \(\mathrm{Vars}(N)\) everywhere: \(\emptyset\) at a constant input, \(\{X\}\) at an input labeled \(X\) or \(\neg X\), and \(\bigcup_C \mathrm{Vars}( C)\) at an internal node. Store each set as a length-\(v\) bit vector.

2. Build once, per variable \(X\), an or-node \(S_X\) with fresh children labeled \(X\) and \(\neg X\); thus \(S_X\) is valid and \(\mathrm{Vars}(S_X) = \{X\}\).

3. For each edge \(N \to C\) with \(N\) an or-node, let \(D = \mathrm{Vars}(N) \setminus \mathrm{Vars}( C)\); if \(D \neq \emptyset\), redirect the edge to a fresh and-node

\begin{equation*} A_{N,C} \;=\; \mathrm{and}\Big(C,\; \{\,S_X : X \in D\,\}\Big) . \end{equation*}

The gadget must hang off the edge, not off \(C\) itself: in a DAG the same \(C\) may be a child of two or-nodes with different variable sets.

Equivalence. Each \(S_X\) is valid, so \(A_{N,C}\) is equivalent to \(C\) by \(\alpha \wedge \mathit{true} \equiv \alpha\) (Table 2.2), and replacing a child by an equivalent child preserves the sentence represented by every ancestor, hence by the root.

Variable sets are unchanged, since \(\mathrm{Vars}( C) \subseteq \mathrm{Vars}(N)\) (the latter being the union over the children) gives

\begin{equation*} \mathrm{Vars}(A_{N,C}) = \mathrm{Vars}( C) \cup D = \mathrm{Vars}(N) ; \end{equation*}

so the sets from step 1 stay valid and the or-nodes may be processed independently, in one pass, in any order.

Smoothness holds because every child of an or-node \(N\) now mentions exactly \(\mathrm{Vars}(N)\), and each \(S_X\) has both children mentioning \(\{X\}\). Decomposability is preserved: the children of \(A_{N,C}\) are \(C\) and the \(S_X\) with \(X \in D\), pairwise variable-disjoint since \(D \cap \mathrm{Vars}( C) = \emptyset\) and the gadgets are for distinct variables, while original and-nodes keep their children and their variable sets. Determinism is preserved: equivalent replacements of mutually exclusive children remain mutually exclusive, and \(X, \neg X\) are mutually exclusive.

Complexity. Step 1 performs one set union per edge, \(O(mv)\) bit operations (\(O(mv/w)\) machine words) and \(O(mv)\) bits of working space; step 2 is \(O(v)\); step 3 examines each edge once, computes a set difference in \(O(v)\), and adds at most \(1+v\) edges. Hence \(O(mv)\) time and \(O(mv)\) output size, with \(\sum_{N \to C} |\mathrm{Vars}(N) \setminus \mathrm{Vars}( C)|\) edges actually added — none on an already smooth circuit.

Were some or-node \(N\) not smooth, a child \(C_1\) would omit a variable \(X \in \mathrm{Vars}(N)\); extending a model of \(C_1\) (Step 1) to \(\mathrm{Vars}(N)\) in the two ways \(\omega_1, \omega_2\) that differ only at \(X\) gives two models of \(N\), both satisfying \(N\) through \(C_1\), with \(\#\omega_1 = \#\omega_2 + 1\) — impossible by Step 4 below. Write \(\mathrm{Vars}(N)\) and \(\mathrm{Mods}(N)\) for the variables of the subcircuit at \(N\) and its models (assignments to \(\mathrm{Vars}(N)\)), and \(\#\sigma\) for the number of variables \(\sigma\) sets to true; every node is assumed reachable from the root.

Step 1: every node is satisfiable. Bottom-up: an input is \(\mathit{true}\) or a literal, never \(\mathit{false}\) by hypothesis, and each of these is satisfiable; an or-node is satisfied through any child; and at an and-node decomposability makes the children’s variable sets pairwise disjoint with union \(\mathrm{Vars}(N)\), so models \(\sigma_i\) of the children have disjoint domains and \(\sigma_1 \cup \cdots \cup \sigma_k\) is a well-defined model of \(N\).

Step 2: local models extend coherently. For every node \(N\) and \(\sigma, \sigma^{\prime} \in \mathrm{Mods}(N)\) there are models \(\omega, \omega^{\prime}\) of \(\Gamma\) restricting to \(\sigma, \sigma^{\prime}\) on \(\mathrm{Vars}(N)\) and agreeing with each other outside \(\mathrm{Vars}(N)\). Walk a path \(N = N_0, \ldots, N_r\) to the root making the same choice on both branches at each step: at an or-parent extend both by one and the same assignment of \(\mathrm{Vars}(N_{j+1}) \setminus \mathrm{Vars}(N_j)\), which satisfies \(N_{j+1}\) through the child \(N_j\); at an and-parent adjoin one and the same model of each sibling (Step 1), legitimate because decomposability makes the domains disjoint.

Step 3: all models of \(\Gamma\) have one cardinality \(c\). \(\Gamma\) is satisfiable by Step 1, so \(\Delta\) has a model. If a variable \(X\) lay in exactly one of \(V = \mathrm{Vars}(\Delta)\) and \(\mathrm{Vars}(\Gamma)\), then over the common variable set \(V \cup \mathrm{Vars}(\Gamma)\) the sentence not mentioning \(X\) has a model set closed under flipping \(X\); the two model sets are equal, so \(\mathrm{Mods}(\Delta)\) is closed under that flip too, and flipping changes cardinality by one, contradicting the hypothesis. Hence \(\mathrm{Vars}(\Gamma) = V\) and \(\mathrm{Mods}(\Gamma) = \mathrm{Mods}(\Delta)\).

Step 4: every node inherits the property. For \(\sigma, \sigma^{\prime} \in \mathrm{Mods}(N)\), take \(\omega, \omega^{\prime}\) from Step 2; they agree outside \(\mathrm{Vars}(N)\), so

\begin{equation*} \#\sigma - \#\sigma^{\prime} \;=\; \#\omega - \#\omega^{\prime} \;=\; c - c \;=\; 0 . \end{equation*}

This contradicts \(\#\omega_1 = \#\omega_2 + 1\) above, so no or-node child omits a variable of its parent: any two children of an or-node mention the same variables, i.e. \(\Gamma\) is smooth. \(\blacksquare\)

The assigned integers are minimum cardinalities, and the deletion retains exactly the edges lying on a minimum-cardinality model. Write \(\|\sigma\|\) for the number of variables \(\sigma\) sets to false, \(d(N)\) for the integer at \(N\), and \(\mathrm{Mods}(N)\) for the models of the subcircuit at \(N\) (assignments to \(\mathrm{Vars}(N)\)); arithmetic uses \(\infty + x = \infty\), \(\min(\infty,x) = x\), and \(\min\emptyset = \infty\).

Lemma 1. \(d(N) = \min\{\|\sigma\| : \sigma \in \mathrm{Mods}(N)\}\) at every node. Bottom-up. The inputs read off directly: \(0\) for \(\mathit{true}\) and for a positive literal, \(1\) for a negative literal, \(\infty\) for \(\mathit{false}\) (empty model set). At an or-node, smoothness gives \(\mathrm{Vars}(C_i) = \mathrm{Vars}(N)\) for all \(i\), so \(\mathrm{Mods}(N) = \bigcup_i \mathrm{Mods}(C_i)\) with no extension needed and

\begin{equation*} \min_{\sigma \in \mathrm{Mods}(N)} \|\sigma\| \;=\; \min_i \, d(C_i) \;=\; d(N) ; \end{equation*}

smoothness is essential, since extending a child’s model to \(\mathrm{Vars}(N)\) could add false-valued variables and understate its cost. At an and-node, decomposability makes the \(\mathrm{Vars}(C_i)\) pairwise disjoint with union \(\mathrm{Vars}(N)\), so \(\mathrm{Mods}(N)\) consists of the disjoint unions \(\sigma_1 \cup \cdots \cup \sigma_k\), \(\|\cdot\|\) is additive on them, and minimizing a sum of independent terms minimizes each term:

\begin{equation*} \min_{\sigma \in \mathrm{Mods}(N)} \|\sigma\| \;=\; \sum_i \, d(C_i) \;=\; d(N) \end{equation*}

(both sides \(\infty\) if some child is unsatisfiable).

Lemma 2. At an or-node the deleted children are exactly those with \(d( C) > d(N)\), since \(d(N) = \min_i d(C_i)\), and at least one child survives; by smoothness every child carries variable set \(\mathrm{Vars}(N)\), so the surviving union is still \(\mathrm{Vars}(N)\). And-node edges are never deleted, so \(\mathrm{Vars}(N)\) is unchanged in \(\Gamma_m\), which remains decomposable, deterministic, and smooth (a subset of an or-node’s children preserves pairwise mutual exclusivity and pairwise equality of variable sets).

Main claim. For every node \(N\),

\begin{equation*} \begin{aligned} \mathrm{Mods}_{\Gamma_m}(N) \;=\; \{\, \sigma \in \mathrm{Mods}_{\Gamma}(N) : \|\sigma\| = d(N) \,\}, \end{aligned} \end{equation*}

both sides being sets of assignments to \(\mathrm{Vars}(N)\) by Lemma 2. Bottom-up.

(i) Inputs. Untouched, and each has at most one model, of cardinality \(d(N)\) by the base cases of Lemma 1; for \(\mathit{false}\) both sides are empty.

(ii) Or-nodes. By the inductive hypothesis and smoothness,

\begin{equation*} \begin{aligned} \mathrm{Mods}_{\Gamma_m}(N) &= \bigcup_{i \,:\, d(C_i) = d(N)} \{\sigma \in \mathrm{Mods}_\Gamma(C_i) : \|\sigma\| = d(N)\} \\ &= \bigcup_{i=1}^{k} \{\sigma \in \mathrm{Mods}_\Gamma(C_i) : \|\sigma\| = d(N)\}, \end{aligned} \end{equation*}

because a deleted child has \(d(C_i) > d(N)\) and by Lemma 1 all of its models cost at least \(d(C_i)\), so it contributes nothing; the last union is \(\{\sigma \in \mathrm{Mods}_\Gamma(N) : \|\sigma\| = d(N)\}\).

(iii) And-nodes. All children survive, models of \(N\) are the disjoint unions \(\bigcup_i \sigma_i\) with \(\|\bigcup_i \sigma_i\| = \sum_i \|\sigma_i\|\), and \(d(N) = \sum_i d(C_i)\) with \(\|\sigma_i\| \ge d(C_i)\) termwise (Lemma 1); so the sums agree only if \(\|\sigma_i\| = d(C_i)\) for every \(i\), which is exactly what the inductive hypothesis says \(\mathrm{Mods}_{\Gamma_m}(C_i)\) contains. (Both sides empty if some \(d(C_i) = \infty\).)

At the root, Lemma 1 identifies \(d(\mathrm{root})\) with \(\min_{\tau \in \mathrm{Mods}(\Gamma)} \|\tau\|\), so the models of \(\Gamma_m\) are precisely the minimum-cardinality models of \(\Gamma\). \(\blacksquare\)

(a) \(.5\), \(.25\), \(.4\); (b) the table below; (c) \(.75\) and \(.25\); (d) \(A = \text{true}\) is not independent of \(C = \text{true}\), while \(B = \text{true}\) is.

(a) Equation (3.1) sums the beliefs of the worlds satisfying each event:

\begin{equation*} \begin{aligned} \Pr(A = \text{true}) &= .075 + .050 + .225 + .150 = .5, \\ \Pr(B = \text{true}) &= .075 + .050 + .025 + .100 = .25, \\ \Pr(C = \text{true}) &= .075 + .225 + .025 + .075 = .4 . \end{aligned} \end{equation*}

(b) By (3.11) every world falsifying the evidence gets belief \(0\) and every world satisfying it is rescaled by \(\Pr(C = \text{true}) = .4\), e.g. \(.075/.4 = .1875\):

(c) By (3.12), equivalently by summing the surviving entries of (b),

\begin{equation*} \Pr(A = \text{true} \mid C = \text{true}) = \frac{.075 + .225}{.4} = .75, \end{equation*}

\begin{equation*} \Pr(B = \text{true} \mid C = \text{true}) = \frac{.075 + .025}{.4} = .25 . \end{equation*}

(d) By Definition (3.13), \(\alpha\) is independent of \(\beta\) when \(\Pr(\alpha \mid \beta) = \Pr(\alpha)\). Here \(.75 \ne .5 = \Pr(A = \text{true})\), so \(A = \text{true}\) and \(C = \text{true}\) are not independent, while \(.25 = \Pr(B = \text{true})\), so \(B = \text{true}\) and \(C = \text{true}\) are. The symmetric form (3.14) agrees: \(\Pr(A = \text{true} \wedge C = \text{true}) = .3 \ne .2 = (.5)(.4)\), whereas \(\Pr(B = \text{true} \wedge C = \text{true}) = .1 = (.25)(.4)\).

(a) \(.625\); (b) the table below; (c) \(.8\) and \(.4\); (d) no. Write \(\beta\) for the evidence \(A = \text{true} \vee B = \text{true}\).

(a) The worlds satisfying \(\beta\) are \(\omega_1,\ldots,\omega_6\), so by (3.1)

\begin{equation*} \Pr(\beta) = .075 + .050 + .225 + .150 + .025 + .100 = .625 . \end{equation*}

(b) By (3.11) the two falsifying worlds \(\omega_7, \omega_8\) get belief \(0\) and the rest are divided by \(.625\), e.g. \(.075/.625 = .12\):

(c) A world with \(A\) true already satisfies \(\beta\), so \((A = \text{true}) \wedge \beta \equiv (A = \text{true})\), and likewise for \(B\); hence by (3.12)

\begin{equation*} \Pr(A = \text{true} \mid \beta) = \frac{.5}{.625} = .8, \qquad \Pr(B = \text{true} \mid \beta) = \frac{.25}{.625} = .4 . \end{equation*}

(d) Not conditionally independent. By (3.16) the test is whether \(\Pr(B = \text{true} \wedge C = \text{true} \mid \beta)\) equals the product of the two separate conditionals. The worlds with \(B\) and \(C\) true are \(\omega_1, \omega_5\), both satisfying \(\beta\); those with \(C\) true satisfying \(\beta\) are \(\omega_1, \omega_3, \omega_5\). So

\begin{equation*} \Pr(B = \text{true} \wedge C = \text{true} \mid \beta) = \frac{.1}{.625} = .16 , \end{equation*}

\begin{equation*} \Pr(C = \text{true} \mid \beta) = \frac{.325}{.625} = .52 , \end{equation*}

while \(\Pr(B = \text{true} \mid \beta)\Pr(C = \text{true} \mid \beta) = (.4)(.52) = .208 \ne .16\).

Unbiasedness gives \(\Pr(C_1 = h) = \Pr(C_2 = h) = 1/2\), and tossing two separate coins makes the outcomes independent, so by (3.14) each world carries the product \(1/4\):

(a) \(1/2\). The evidence holds in \(\omega_1, \omega_2\) and \(C_2 = t \wedge C_1 = h\) only in \(\omega_2\), so by Bayes conditioning (3.12),

\begin{equation*} \Pr(C_2 = t \mid C_1 = h) = \frac{1/4}{1/2} = \frac{1}{2}. \end{equation*}

(b) \(1/3\). Put \(\beta : C_1 = h \vee C_2 = h\), holding in \(\omega_1,\omega_2,\omega_3\), and \(\alpha : C_1 = h \wedge C_2 = h\), holding only in \(\omega_1\); since \(\alpha \models \beta\) we have \(\alpha \wedge \beta \equiv \alpha\), so by (3.12)

\begin{equation*} \Pr(\alpha \mid \beta) = \frac{1/4}{3/4} = \frac{1}{3}. \end{equation*}

About \(17.9\%\). Writing \(S\) for “the individual smokes” and \(C\) for “the individual has cancer” under uniform sampling from the population, the data read \(\Pr(S) = .24\), \(\Pr( C) = .05\), \(\Pr(S \mid C) = .86\), and Bayes rule (3.19) supplies the reverse conditional:

\begin{equation*} \Pr(C \mid S) = \frac{\Pr(S \mid C)\,\Pr( C)}{\Pr(S)} = \frac{(.86)(.05)}{.24} = \frac{.043}{.24} \approx .1792 . \end{equation*}

The odds of cancer are multiplied by \(k \approx 4.147\), rising from about \(1{:}19\) to about \(1{:}4.58\). Keeping \(S\) and \(C\) and the data of Exercise 3.4, the quantity asked for is the Bayes factor \(k = O^{\prime}( C)/O( C)\), with \(O(\gamma) = \Pr(\gamma)/\Pr(\neg\gamma)\) the odds (3.23) and \(O^{\prime}\) computed in \(\Pr^{\prime} = \Pr(. \mid S)\). From Exercise 3.4, \(\Pr(C \mid S) = .043/.24\), so \(\Pr(\neg C \mid S) = (.24 - .043)/.24 = .197/.24\) and the common denominator \(\Pr(S)\) cancels:

\begin{equation*} k = \frac{O^{\prime}( C)}{O( C)} = \frac{.043/.197}{.05/.95} = \frac{(.043)(.95)}{(.197)(.05)} = \frac{817}{197} \approx 4.147 . \end{equation*}

Method (2): applying (3.19) to numerator and denominator of \(O^{\prime}( C)\) cancels \(\Pr(S)\) and gives \(k = \Pr(S \mid C)/\Pr(S \mid \neg C)\); case analysis (3.17) yields \(\Pr(S \wedge \neg C) = .24 - .043 = .197\), so \(\Pr(S \mid \neg C) = .197/.95\) and \(k = (.86)(.95)/.197 \approx 4.147\).

Take \(O\) for the sex of the older child and \(Y\) for that of the younger, each with values \(b\) and \(g\); the differing ages make the children distinguishable. Under the usual idealization each child is equally likely a boy or a girl and the two sexes are independent, so by (3.14) every world carries the product \(1/2 \times 1/2\):

The event “at least one child is a boy” is \(O = b \vee Y = b\), holding in \(\omega_1,\omega_2,\omega_3\), of belief \(3/4\).

(a) \(\alpha : O = b\), \(\beta : \top\); since \(\Pr(\top) = 1\), the answer is \(\Pr(O = b) = 1/4 + 1/4 = 1/2\).

(b) \(\alpha : O = b\), \(\beta : Y = b\). The evidence holds in \(\omega_1,\omega_3\) and the conjunction only in \(\omega_1\), so by (3.12) the answer is \((1/4)/(1/2) = 1/2\) — the independence assumption showing itself.

(c) \(\alpha : O = b\), \(\beta : O = b \vee Y = b\). Here \(\alpha \models \beta\), so \(\alpha \wedge \beta \equiv \alpha\) and

\begin{equation*} \Pr(O = b \mid O = b \vee Y = b) = \frac{1/2}{3/4} = \frac{2}{3} . \end{equation*}

(d) \(\alpha : O = b \wedge Y = b\), \(\beta : O = b \vee Y = b\). Again \(\alpha \models \beta\), so

\begin{equation*} \Pr(O = b \wedge Y = b \mid O = b \vee Y = b) = \frac{1/4}{3/4} = \frac13 . \end{equation*}

Bayes conditioning (3.12) applied twice to the same joint belief. Assume \(\Pr(\alpha) \ne 0\) and \(\Pr(\beta) \ne 0\), without which \(\Pr(\beta \mid \alpha)\) and the right-hand side are undefined. Applying (3.12) with evidence \(\alpha\) and multiplying through,

\begin{equation*} \begin{aligned} \Pr(\beta \mid \alpha)\,\Pr(\alpha) &= \Pr(\beta \wedge \alpha) \;=\; \Pr(\alpha \wedge \beta), \end{aligned} \end{equation*}

the second equality because \(\alpha \wedge \beta\) and \(\beta \wedge \alpha\) have the same models and (3.1) reads belief off the model set alone. Substituting that numerator into (3.12) with evidence \(\beta\),

\begin{equation*} \Pr(\alpha \mid \beta) = \frac{\Pr(\alpha \wedge \beta)}{\Pr(\beta)} = \frac{\Pr(\beta \mid \alpha)\,\Pr(\alpha)}{\Pr(\beta)} . \qquad \blacksquare \end{equation*}

\(n = 3\). Let \(D\) be the disease and \(T_i\) the event that test \(i\) reads positive, so \(\Pr(D) = .001\), \(f_p = \Pr(T_i \mid \neg D) = .02\) and \(f_n = \Pr(\neg T_i \mid D) = .05\). “Errors independent” means the readings are mutually independent given \(D\) and given \(\neg D\) — conditional independence (3.16), not unconditional independence — so for \(e_n = T_1 \wedge \cdots \wedge T_n\),

\begin{equation*} \Pr(e_n \mid D) = (.95)^n, \qquad \Pr(e_n \mid \neg D) = (.02)^n . \end{equation*}

Applying (3.12) to numerator and denominator of the odds, each reading contributes the noisy-sensor Bayes factor \(k^{+} = (1-f_n)/f_p = 47.5\) of Section 3.6.4:

\begin{equation*} O(D \mid e_n) = \frac{\Pr(e_n \mid D)\Pr(D)}{\Pr(e_n \mid \neg D)\Pr(\neg D)} = (47.5)^n \cdot \frac{.001}{.999} = \frac{(47.5)^n}{999} . \end{equation*}

A belief exceeds \(95\%\) exactly when its odds exceed \(.95/.05 = 19\), so we need \((47.5)^n > 18981\), i.e. \(n > \ln 18981 / \ln 47.5 \approx 2.552\); since \((47.5)^n\) increases with \(n\), the least such integer is \(n = 3\). Converting back with \(\Pr = O/(1+O)\):

No pair is marginally independent, and the only conditional independence of the three is \(B \perp C \mid A\). Write \(a\) for \(A = \text{true}\), \(\bar a\) for \(A = \text{false}\), similarly for \(B\) and \(C\); by (3.14) a pair of variables is independent only if \(\Pr(x,y) = \Pr(x)\Pr(y)\) at every instantiation, so one violation refutes it.

Summing the worlds by (3.1) gives \(\Pr(a) = \Pr(b) = \Pr( c) = .50\), so independence of a pair would force each joint entry to be \(.25\). It does not:

\begin{equation*} \begin{aligned} \Pr(a,b) &= .27+.18 = .45, \\ \Pr(a,c) &= .27+.03 = .30, \\ \Pr(b,c) &= .27+.02 = .29 . \end{aligned} \end{equation*}

(i) \(B\) and \(C\) are independent given \(A\). Conditioning on \(a\) (of probability \(.50\)), with \(\Pr(b \mid a) = (.27+.18)/.5 = .90\) and \(\Pr(c \mid a) = (.27+.03)/.5 = .60\):

Conditioning on \(\bar a\) (also \(.50\)), with \(\Pr(b \mid \bar a) = (.02+.03)/.5 = .10\) and \(\Pr(c \mid \bar a) = (.02+.18)/.5 = .40\):

Every entry of both tables factorizes, so by (3.16) \(I_{\Pr}(B, A, C)\) holds.

(ii) \(A\) and \(B\) are not independent given \(C\). Conditioning on \(c\), with \(\Pr(a \mid c) = (.27+.03)/.5 = .60\) and \(\Pr(b \mid c) = (.27+.02)/.5 = .58\),

\begin{equation*} \Pr(a,b \mid c) = \frac{.27}{.50} = .54 \qquad\text{but}\qquad (.60)(.58) = .348 . \end{equation*}

(iii) \(A\) and \(C\) are not independent given \(B\). Conditioning on \(b\), with \(\Pr(a \mid b) = (.27+.18)/.5 = .90\) and \(\Pr(c \mid b) = (.27+.02)/.5 = .58\),

\begin{equation*} \Pr(a,c \mid b) = \frac{.27}{.50} = .54 \qquad\text{but}\qquad (.90)(.58) = .522 . \end{equation*}

All four parts follow from monotonicity: if \(\alpha \models \beta\) then \(\Pr(\alpha) \le \Pr(\beta)\). Indeed \(\alpha \models \beta\) says \(\mathrm{Mods}(\alpha) \subseteq \mathrm{Mods}(\beta)\), so splitting the sum (3.1) along that inclusion,

\begin{equation*} \begin{aligned} \Pr(\beta) &\;=\; \Pr(\alpha) + \sum_{\omega \models \beta \wedge \neg\alpha} \Pr(\omega) \;\ge\; \Pr(\alpha), \end{aligned} \end{equation*}

the second term being a sum of nonnegative beliefs. That is (c), with the sharper identity \(\Pr(\beta) - \Pr(\alpha) = \Pr(\beta \wedge \neg\alpha)\).

(a) By (c), \(\Pr(\alpha) \le \Pr(\beta) = 0\), while \(\Pr(\alpha) \ge 0\) by (3.2).

(b) Two applications of (c), to \(\alpha \wedge \beta \models \alpha\) and to \(\alpha \models \alpha \vee \beta\).

(d) Assume \(\Pr(\beta) > 0\), as both conditionals require; \(\beta \models \gamma\) and (c) then give \(\Pr(\gamma) \ge \Pr(\beta) > 0\), so \(\Pr(\alpha \mid \gamma)\) is defined too. From \(\alpha \models \beta\), \(\mathrm{Mods}(\alpha \wedge \beta) = \mathrm{Mods}(\alpha) \cap \mathrm{Mods}(\beta) = \mathrm{Mods}(\alpha)\), i.e. \(\alpha \wedge \beta \equiv \alpha\); transitivity of \(\models\) gives \(\alpha \models \gamma\) and hence \(\alpha \wedge \gamma \equiv \alpha\) as well. So by (3.12),

\begin{equation*} \Pr(\alpha \mid \beta) = \frac{\Pr(\alpha)}{\Pr(\beta)} \;\ge\; \frac{\Pr(\alpha)}{\Pr(\gamma)} = \Pr(\alpha \mid \gamma), \end{equation*}

since \(0 < \Pr(\beta) \le \Pr(\gamma)\) and \(\Pr(\alpha) \ge 0\).

Group the worlds by their \(\mathbf{X}\)- and \(\mathbf{Y}\)-parts, then apply the hypothesis. As \(\mathbf{X} \cap \mathbf{Y} = \emptyset\), every world splits uniquely as \(\omega = (\mathbf{x},\mathbf{y},\mathbf{z})\) with \(\mathbf{z}\) the instantiation of the remaining variables; and since \(\alpha\) mentions only \(\mathbf{X}\), its truth value at \(\omega\) depends on \(\mathbf{x}\) alone, so \(\omega \models \alpha\) iff \(\mathbf{x} \models \alpha\), and likewise \(\omega \models \beta\) iff \(\mathbf{y} \models \beta\). Hence by (3.1), regrouping finite sums of nonnegative terms so that the omitted variables are summed out into marginals,

\begin{equation*} \begin{aligned} \Pr(\alpha) = \sum_{\mathbf{x} \models \alpha} \Pr(\mathbf{x}), \qquad \Pr(\beta) = \sum_{\mathbf{y} \models \beta} \Pr(\mathbf{y}) . \end{aligned} \end{equation*}

The same regrouping, followed by the hypothesis \(\Pr(\mathbf{x},\mathbf{y}) = \Pr(\mathbf{x})\Pr(\mathbf{y})\), factors the double sum:

\begin{equation*} \begin{aligned} \Pr(\alpha \wedge \beta) &= \sum_{\mathbf{x}\models\alpha}\ \sum_{\mathbf{y}\models\beta} \Pr(\mathbf{x})\Pr(\mathbf{y}) \\ &= \Big(\sum_{\mathbf{x}\models\alpha} \Pr(\mathbf{x})\Big) \Big(\sum_{\mathbf{y}\models\beta} \Pr(\mathbf{y})\Big) \;=\; \Pr(\alpha)\,\Pr(\beta), \end{aligned} \end{equation*}

which is the definition (3.14) of independence of the events. \(\blacksquare\)

The identical regrouping over \(k\) pairwise disjoint blocks \(\mathbf{X}_1,\ldots,\mathbf{X}_k\) whose joint distribution factorizes gives \(\Pr(\alpha_1 \wedge \cdots \wedge \alpha_k) = \prod_i \Pr(\alpha_i)\) for sentences \(\alpha_i\) over \(\mathbf{X}_i\); Exercise 3.12 needs that form at its and-nodes.

Replace each literal input by its probability, each and-gate by multiplication and each or-gate by addition, then evaluate the resulting arithmetic circuit bottom-up with the values cached at the nodes; one pass costs \(O(|\Gamma|)\).

The distribution factorizes. Each \(X_{k+1}\) is independent of \(\{X_1,\ldots,X_k\}\), so the event form (3.14) gives \(\Pr(x_1,\ldots,x_{k+1}) = \Pr(x_{k+1})\Pr(x_1,\ldots,x_k)\), and induction on \(k\) yields

\begin{equation*} \Pr(x_1,\ldots,x_n) = \prod_{i=1}^{n} \Pr(x_i) \end{equation*}

for every instantiation: the \(n\) given marginals specify \(\Pr\) completely. Summing this over the variables outside pairwise disjoint blocks \(\mathbf{Y}_1,\ldots,\mathbf{Y}_k\) shows the marginal factorizes too, so the \(k\)-block form of Exercise 3.11 applies: for sentences \(\alpha_j\) over \(\mathbf{Y}_j\),

\begin{equation*} \Pr(\alpha_1 \wedge \cdots \wedge \alpha_k) = \prod_{j=1}^{k} \Pr(\alpha_j) . \tag{\(\dagger\)} \end{equation*}

The algorithm. In topological order (children before parents) store \(v(N) = \Pr(\ell)\) at a leaf labeled with literal \(\ell\), \(v(N) = 1\) at \(\mathit{true}\) and \(0\) at \(\mathit{false}\), \(v(N) = \prod_j v(C_j)\) at an and-node, and \(v(N) = \sum_j v(C_j)\) at an or-node; output \(v(\mathrm{root})\).

Correctness. \(v(N) = \Pr(\alpha_N)\), by induction along that order. Leaves are immediate, the constants by (3.4) and (3.3). At an and-node, decomposability makes the variable sets \(\mathbf{Y}_j\) of the children pairwise disjoint, so \((\dagger)\) applies and \(\Pr(\alpha_N) = \prod_j \Pr(\alpha_{C_j}) = \prod_j v(C_j)\). At an or-node, determinism makes the \(\mathrm{Mods}(\alpha_{C_j})\) pairwise disjoint with union \(\mathrm{Mods}(\alpha_N)\), so the sum (3.1) splits along that disjoint union:

\begin{equation*} \Pr(\alpha_N) = \sum_{j=1}^{k} \sum_{\omega \models \alpha_{C_j}} \Pr(\omega) = \sum_{j=1}^{k} \Pr(\alpha_{C_j}) = \sum_{j=1}^{k} v(C_j) = v(N) . \end{equation*}

At the root this gives \(\Pr(\alpha)\).

Complexity. Caching means each node is visited once and performs one arithmetic operation per child edge, for a total of \(O(|\mathrm{nodes}(\Gamma)| + |\mathrm{edges}(\Gamma)|) = O(|\Gamma|)\); the topological order costs the same and each lookup \(\Pr(x_i)\) is \(O(1)\). (Without caching, a shared subcircuit would be re-evaluated once per parent, costing the size of the unfolded tree.)

(a) The worlds are the \(3 \times 2 = 6\) instantiations of \(U \in \{1,2,3\}\) and \(C \in \{\mathrm{b}, \mathrm{w}\}\) (black, white):

\begin{equation*} \begin{aligned} \omega_1 &= (U{=}1, C{=}\mathrm{b}), &\quad \omega_2 &= (U{=}1, C{=}\mathrm{w}), \\ \omega_3 &= (U{=}2, C{=}\mathrm{b}), &\quad \omega_4 &= (U{=}2, C{=}\mathrm{w}), \\ \omega_5 &= (U{=}3, C{=}\mathrm{b}), &\quad \omega_6 &= (U{=}3, C{=}\mathrm{w}). \end{aligned} \end{equation*}

(b) “At random” means \(\Pr(U{=}1) = \Pr(U{=}2) = \Pr(U{=}3) = 1/3\), and given the urn the ball is drawn uniformly from its contents:

By the chain rule, \(\Pr(u, c) = \Pr(c \mid u)\Pr(u)\), giving the joint distribution:

(The six entries sum to \(18/18 = 1\). Check!)

(c) Summing the black worlds by (3.1),

\begin{equation*} \Pr(C{=}\mathrm{b}) = \tfrac16 + \tfrac19 + \tfrac29 = \tfrac{3+2+4}{18} = \tfrac12 . \end{equation*}

(d) By Bayes conditioning (3.12),

\begin{equation*} \Pr(U{=}2 \mid C{=}\mathrm{b}) = \frac{\Pr(U{=}2, C{=}\mathrm{b})}{\Pr(C{=}\mathrm{b})} = \frac{1/9}{1/2} = \frac{2}{9} \approx .2222 . \end{equation*}

(e) For \(\alpha = (U{=}1) \vee (C{=}\mathrm{w})\), inclusion-exclusion (3.6) with \(\Pr(C{=}\mathrm{w}) = 1 - \Pr(C{=}\mathrm{b}) = 1/2\) by (3.5) gives

\begin{equation*} \Pr(\alpha) = \tfrac13 + \tfrac12 - \tfrac16 = \tfrac{2+3-1}{6} = \tfrac{2}{3} . \end{equation*}

(a) Writing \(U\) for the urn picked and \(C\) for the color drawn, case analysis (3.18) over \(U\) gives

\begin{equation*} \Pr(C{=}\mathrm{w}) = \frac{1}{2}\cdot\frac{n}{n+m} \;+\; \frac{1}{2}\cdot\frac{k-n}{2k-n-m} , \end{equation*}

valid whenever both urns are nonempty, i.e. \((n,m) \ne (0,0)\) and \((n,m) \ne (k,k)\).

(b) Yes: take \(n = 1\), \(m = 0\), a single white ball alone in urn 1 and the rest (2 white, 3 black) in urn 2. Then

\begin{equation*} \Pr(C{=}\mathrm{w}) = \frac12 \cdot \frac{1}{1} + \frac12 \cdot \frac{2}{5} = \frac12 + \frac15 = \frac{7}{10} . \end{equation*}

(c) Yes: use the same rule \(n = 1\), \(m = 0\) for every \(k\), so that urn 2 holds \(k-1\) white and \(k\) black balls. Then

\begin{equation*} p_k := \Pr(C{=}\mathrm{w}) = \frac12 + \frac{k-1}{2(2k-1)} = \frac{3k-2}{4k-2}, \end{equation*}

and dividing numerator and denominator by \(k\),

\begin{equation*} \lim_{k \to \infty} p_k = \lim_{k\to\infty} \frac{3 - 2/k}{4 - 2/k} = \frac{3}{4} . \end{equation*}

(Consistently \(p_3 = 7/10\), the value found in (b).)

Three exhaustive cases, in each of which (3.15) and (3.16) hold or fail together.

(i) \(Pr(\gamma) = 0\). Then (3.16) holds by its second disjunct; and \(\beta \wedge \gamma \models \gamma\), so monotonicity (Exercise 3.10(c)) gives \(Pr(\beta \wedge \gamma) \le Pr(\gamma) = 0\) and (3.15) holds by its second disjunct.

(ii) \(Pr(\gamma) > 0\) but \(Pr(\beta \wedge \gamma) = 0\). Then (3.15) holds by its second disjunct. For (3.16), \(\alpha \wedge \beta \wedge \gamma \models \beta \wedge \gamma\) forces \(Pr(\alpha \wedge \beta \wedge \gamma) = 0\), so

\begin{equation*} Pr(\alpha \wedge \beta \mid \gamma) = 0 = Pr(\alpha \mid \gamma)\,Pr(\beta \mid \gamma), \end{equation*}

the right-hand side because \(Pr(\beta \mid \gamma) = Pr(\beta \wedge \gamma)/Pr(\gamma) = 0\).

(iii) \(Pr(\beta \wedge \gamma) > 0\). Then \(Pr(\gamma) \ge Pr(\beta \wedge \gamma) > 0\), so both second disjuncts fail and every conditional probability below is defined. The chain rule inside \(\gamma\) is the bridge:

\begin{equation*} \begin{aligned} Pr(\alpha \wedge \beta \mid \gamma) &= \frac{Pr(\alpha \mid \beta \wedge \gamma)\, Pr(\beta \wedge \gamma)}{Pr(\gamma)} \\ &= Pr(\alpha \mid \beta \wedge \gamma)\, Pr(\beta \mid \gamma). \end{aligned} \tag{\(\ast\)} \end{equation*}

Substituting \(Pr(\alpha \mid \beta \wedge \gamma) = Pr(\alpha \mid \gamma)\) into \((\ast)\) turns (3.15) into (3.16); conversely, (3.16) and \((\ast)\) give \(Pr(\alpha \mid \beta \wedge \gamma)Pr(\beta \mid \gamma) = Pr(\alpha \mid \gamma)Pr(\beta \mid \gamma)\), and \(Pr(\beta \mid \gamma) = Pr(\beta \wedge \gamma)/Pr(\gamma) > 0\) cancels, leaving (3.15).

Abbreviate the joint by \(a = Pr(x,y)\), \(b = Pr(x,\bar{y})\), \(c = Pr(\bar{x},y)\), \(d = Pr(\bar{x},\bar{y})\), so \(a+b+c+d = 1\) and case analysis on the eliminated variable gives

\begin{equation*} \begin{aligned} Pr(x) &= a + b, & Pr(\bar{x}) &= c + d, \\ Pr(y) &= a + c, & Pr(\bar{y}) &= b + d. \end{aligned} \end{equation*}

Independence \(I_{Pr}(X, \emptyset, Y)\) is, by Equation 3.14, the four equations \(Pr(x^{\prime},y^{\prime}) = Pr(x^{\prime})Pr(y^{\prime})\) over \(x^{\prime} \in \{x,\bar{x}\}\), \(y^{\prime} \in \{y,\bar{y}\}\); the claim is that these hold iff \(ad = bc\).

(\(\Rightarrow\)) Independence gives \(a = Pr(x)Pr(y)\), \(b = Pr(x)Pr(\bar{y})\), \(c = Pr(\bar{x})Pr(y)\), \(d = Pr(\bar{x})Pr(\bar{y})\), whence by commutativity

\begin{equation*} ad = Pr(x)Pr(y)Pr(\bar{x})Pr(\bar{y}) = \bigl(Pr(x)Pr(\bar{y})\bigr)\bigl(Pr(\bar{x})Pr(y)\bigr) = bc . \end{equation*}

(\(\Leftarrow\)) Assume \(ad = bc\). Expanding and substituting \(ad\) for \(bc\),

\begin{equation*} \begin{aligned} Pr(x)\,Pr(y) &= (a+b)(a+c) = a^{2} + ac + ab + bc \\ &= a\,(a + b + c + d) = a = Pr(x, y). \end{aligned} \end{equation*}

The other three cells are the same computation with the same substitution: \((a+b)(b+d) = b(a+c+b+d) = b\), \((c+d)(a+c) = c(a+c+b+d) = c\), and \((c+d)(b+d) = d(a+c+b+d) = d\). (Check!) All four product equations hold, so \(X\) and \(Y\) are independent.

Write \(p = Pr(\alpha)\); Equation 3.23 requires \(Pr(\neg\alpha) = 1 - p \neq 0\), so \(O(\alpha) = p/(1-p)\) is a finite nonnegative number and \(1 + O(\alpha) \neq 0\). Clearing the inner fraction by \(1 - p > 0\),

\begin{equation*} \frac{O(\alpha)}{1 + O(\alpha)} = \frac{p/(1-p)}{\bigl((1-p) + p\bigr)/(1-p)} = \frac{p}{(1-p) + p} = p = Pr(\alpha). \end{equation*}

Rewrite both conditional probabilities in the conditional odds by Bayes rule; assume \(Pr(\beta) > 0\), \(Pr(\alpha) > 0\) and \(Pr(\beta \wedge \neg\alpha) > 0\), which is exactly what makes every quantity in the statement defined and \(O(\alpha)\) positive. Then

\begin{equation*} \begin{aligned} O(\alpha \mid \beta) &= \frac{Pr(\alpha \mid \beta)}{Pr(\neg\alpha \mid \beta)} \\[4pt] &= \frac{Pr(\beta \mid \alpha)\, Pr(\alpha) / Pr(\beta)} {Pr(\beta \mid \neg\alpha)\, Pr(\neg\alpha) / Pr(\beta)} \\[4pt] &= \frac{Pr(\beta \mid \alpha)}{Pr(\beta \mid \neg\alpha)} \cdot \frac{Pr(\alpha)}{Pr(\neg\alpha)} \\[4pt] &= \frac{Pr(\beta \mid \alpha)}{Pr(\beta \mid \neg\alpha)} \cdot O(\alpha), \end{aligned} \end{equation*}

the common factor \(1/Pr(\beta)\) cancelling and the last line using Equation 3.23. Dividing by \(O(\alpha) > 0\) gives the claim.

Assume \(0 < Pr(\beta) < 1\) and \(Pr(\neg\alpha \mid \beta), Pr(\neg\alpha \mid \neg\beta) > 0\), which is exactly what makes the two conditional odds defined; then the second disjunct of (3.13) is unavailable and independence means \(Pr(\alpha \mid \beta) = Pr(\alpha)\).

(\(\Rightarrow\)) From \(Pr(\alpha \wedge \beta) = Pr(\alpha)Pr(\beta)\) (Equation 3.14) and case analysis on \(\beta\),

\begin{equation*} Pr(\alpha \wedge \neg\beta) = Pr(\alpha) - Pr(\alpha)Pr(\beta) = Pr(\alpha)Pr(\neg\beta), \end{equation*}

so dividing by \(Pr(\neg\beta) > 0\) gives \(Pr(\alpha \mid \neg\beta) = Pr(\alpha) = Pr(\alpha \mid \beta)\). Taking complements within each conditional distribution makes the denominators agree as well, whence

\begin{equation*} O(\alpha \mid \beta) = \frac{Pr(\alpha)}{1 - Pr(\alpha)} = O(\alpha \mid \neg\beta). \end{equation*}

(\(\Leftarrow\)) Let \(t\) be the common value of the two odds. Each of \(Pr(\cdot \mid \beta)\) and \(Pr(\cdot \mid \neg\beta)\) is itself a state of belief, so Exercise 3.17 applies inside each and yields \(Pr(\alpha \mid \beta) = t/(1+t) = Pr(\alpha \mid \neg\beta) =: p\). Case analysis on \(\beta\) together with (3.12) then recovers the prior,

\begin{equation*} Pr(\alpha) = Pr(\alpha \mid \beta)Pr(\beta)

• Pr(\alpha \mid \neg\beta)Pr(\neg\beta) = p , \end{equation*}

so \(Pr(\alpha \mid \beta) = p = Pr(\alpha)\), which is (3.13).

Since \(\neg(\alpha \Rightarrow \beta) = \alpha \wedge \neg\beta\), the hypothesis says \(Pr(\alpha \wedge \neg\beta) = 0\), so case analysis on \(\beta\) gives

\begin{equation*} Pr(\alpha) = Pr(\alpha \wedge \beta) + Pr(\alpha \wedge \neg\beta) = Pr(\alpha \wedge \beta), \tag{1} \end{equation*}

and monotonicity (Exercise 3.10(c)) applied to \(\alpha \wedge \beta \models \beta\) gives

\begin{equation*} 0 < Pr(\alpha) = Pr(\alpha \wedge \beta) \le Pr(\beta) < 1 , \tag{2} \end{equation*}

so \(Pr(\beta) > 0\) and \(Pr(\neg\alpha) > 0\) and both conditionings below are legitimate.

(a) Splitting \(\beta\) on \(\alpha\) and using (1), \(Pr(\neg\alpha \wedge \beta) = Pr(\beta) - Pr(\alpha)\), so dividing by \(Pr(\neg\alpha) = 1 - Pr(\alpha) > 0\),

\begin{equation*} Pr(\beta \mid \neg\alpha) = \frac{Pr(\beta) - Pr(\alpha)}{1 - Pr(\alpha)} < Pr(\beta) , \end{equation*}

the inequality because, multiplied through by \(1 - Pr(\alpha) > 0\), it reduces to \(Pr(\alpha)\bigl(1 - Pr(\beta)\bigr) > 0\), which holds by (2).

(b) By (1) and \(Pr(\beta) < 1\),

\begin{equation*} Pr(\alpha \mid \beta) = \frac{Pr(\alpha \wedge \beta)}{Pr(\beta)} = \frac{Pr(\alpha)}{Pr(\beta)} > Pr(\alpha) . \end{equation*}

Both questions have the same answer, \(Pr^{\prime}(Killer = david) = 3/4\).

Write \(\beta\) for \(Killer \in \{david, dick\}\) (“the killer is male”) and \(\alpha\) for \(Killer = david\). From the table \(Pr(\beta) = 2/3 + 1/6 = 5/6\) and \(Pr(\neg\beta) = 1/6\), so \(O(\beta) = 5\) by Equation 3.23; and \(\alpha \models \beta\), so \(Pr(\alpha \wedge \beta) = 2/3\) and \(Pr(\alpha \wedge \neg\beta) = 0\).

Bayes factor \(k = 3\). Equation 3.25 gives directly

\begin{equation*} \begin{aligned} Pr^{\prime}(\alpha) &= \frac{k\,Pr(\alpha \wedge \beta) + Pr(\alpha \wedge \neg\beta)} {k\,Pr(\beta) + Pr(\neg\beta)} \\[4pt] &= \frac{3 \cdot \tfrac{2}{3} + 0}{3 \cdot \tfrac{5}{6} + \tfrac{1}{6}} = \frac{2}{\tfrac{16}{6}} = \frac{3}{4} = 75\% . \end{aligned} \end{equation*}

The same formula, with the same denominator \(16/6\), supplies the rest of the new state of belief:

\(Pr^{\prime}(\beta) = 93.75\%\). This is Jeffrey’s rule (Equation 3.21) with \(q = 15/16\), \(Pr(\alpha \mid \beta) = (2/3)/(5/6) = 4/5\) and \(Pr(\alpha \mid \neg\beta) = 0\) (David is male):

\begin{equation*} Pr^{\prime}(\alpha) = \frac{15}{16}\cdot\frac{4}{5} + \frac{1}{16}\cdot 0 = \frac{3}{4} = 75\% . \end{equation*}

The two agree because \(k = 3\) raises \(O(\beta) = 5\) to \(O^{\prime}(\beta) = 15\), i.e. \(Pr^{\prime}(\beta) = 15/16 = 93.75\%\) by Exercise 3.17, so the two specifications describe the same evidence.

The independence collapses the likelihood of \(\mathbf{x}\) to \(\eta f(u)\), and the common factor \(\eta\) then cancels against the normalizer. Write \(\mathbf{x} = u\mathbf{z}\) with \(\mathbf{z}\) the restriction of \(\mathbf{x}\) to \(\mathbf{Z} = \mathbf{X} \setminus \{U\}\), let \(P = Pr(\mathbf{X})\) be the marginal (which changing \(\eta\) leaves alone, \(\eta\) affecting only the \(s\)-row of \(Pr(S \mid \mathbf{X})\)), and assume \(Pr(s) > 0\). The hypothesis \(I_{Pr}(S, U, \mathbf{Z})\) is Equation 3.15 at every instantiation with \(Pr(u\mathbf{z}) \ne 0\), that is, \(Pr(s \mid u\mathbf{z}) = Pr(s \mid u)\), so \(Pr(s \mid \mathbf{x}) = \eta f(u)\). By Bayes rule (Equation 3.19) with the denominator expanded by case analysis over the instantiations \(\mathbf{x}^\star\) of \(\mathbf{X}\),

\begin{equation*} \begin{aligned} Pr(\mathbf{x} \mid s) &= \frac{Pr(s \mid \mathbf{x}) \, P(\mathbf{x})} {\sum_{\mathbf{x}^\star} Pr(s \mid \mathbf{x}^\star) \, P(\mathbf{x}^\star)} \\ &= \frac{\eta \, f(u) \, P(\mathbf{x})} {\eta \sum_{\mathbf{x}^\star} f(u^\star) \, P(\mathbf{x}^\star)} = \frac{f(u) \, P(\mathbf{x})}{\sum_{u^\star} f(u^\star) \, P(u^\star)} , \end{aligned} \end{equation*}

where \(u^\star\) is the value \(\mathbf{x}^\star\) assigns to \(U\), the last step collects the \(\mathbf{x}^\star\) by that value, and cancelling \(\eta > 0\) is legitimate because the denominator is \(Pr(s) > 0\). The right-hand side mentions only \(P\) and \(f\), so it is one and the same number for every admissible \(\eta\).

Every world satisfying \(\alpha\) satisfies exactly one of \(\alpha \wedge \beta\) and \(\alpha \wedge \neg\beta\), so partitioning \(Pr^{\prime}(\alpha) = \sum_{\omega \models \alpha} Pr^{\prime}(\omega)\) (Equation 3.1) accordingly and substituting Equation 3.20 in each part,

\begin{equation*} \begin{aligned} Pr^{\prime}(\alpha) &= \frac{q}{Pr(\beta)} \sum_{\omega \models \alpha \wedge \beta} Pr(\omega)

• \frac{1-q}{Pr(\neg\beta)} \sum_{\omega \models \alpha \wedge \neg\beta} Pr(\omega) \\ &= \frac{q}{Pr(\beta)} \, Pr(\alpha \wedge \beta)
• \frac{1-q}{Pr(\neg\beta)} \, Pr(\alpha \wedge \neg\beta) \\ &= q\,Pr(\alpha|\beta) + (1-q)\,Pr(\alpha|\neg\beta) , \end{aligned} \end{equation*}

the second line by Equation 3.1 again, applied to \(\alpha \wedge \beta\) and \(\alpha \wedge \neg\beta\), and the third by Bayes conditioning (Equation 3.12), both quotients being defined since \(0 < Pr(\beta) < 1\). This is Equation 3.21.

Equation 3.24. With \(q = Pr^{\prime}(\beta)\) the constraint \(O^{\prime}(\beta) = k\,O(\beta)\) reads \(q/(1-q) = k\,Pr(\beta)/Pr(\neg\beta)\), whose right-hand side is finite because \(Pr(\neg\beta) > 0\); hence \(q \neq 1\) and cross-multiplying is legitimate:

\begin{equation*} q\,Pr(\neg\beta) = k\,Pr(\beta)\,(1-q), \qquad\text{i.e.}\qquad q\,\bigl(k\,Pr(\beta) + Pr(\neg\beta)\bigr) = k\,Pr(\beta) . \end{equation*}

The coefficient is strictly positive (\(k > 0\) and \(0 < Pr(\beta) < 1\)), so dividing by it gives Equation 3.24, and correspondingly

\begin{equation*} 1 - q \;=\; \frac{Pr(\neg\beta)}{k\,Pr(\beta) + Pr(\neg\beta)} . \end{equation*}

Equation 3.25. Abbreviate \(D = k\,Pr(\beta) + Pr(\neg\beta) > 0\). Jeffrey’s rule (Equation 3.21) with these \(q\) and \(1-q\), together with Bayes conditioning (Equation 3.12), gives

\begin{equation*} \begin{aligned} Pr^{\prime}(\alpha) &= q\,Pr(\alpha|\beta) + (1-q)\,Pr(\alpha|\neg\beta) \\ &= \frac{k\,Pr(\beta)}{D} \cdot \frac{Pr(\alpha \wedge \beta)}{Pr(\beta)}

• \frac{Pr(\neg\beta)}{D} \cdot \frac{Pr(\alpha \wedge \neg\beta)}{Pr(\neg\beta)} \\ &= \frac{k\,Pr(\alpha \wedge \beta) + Pr(\alpha \wedge \neg\beta)} {k\,Pr(\beta) + Pr(\neg\beta)} , \end{aligned} \end{equation*}

the cancellations being legitimate because \(Pr(\beta)\) and \(Pr(\neg\beta)\) are nonzero.

For the burglary state of belief of Section 3.6.2,

we have \(Pr(Alarm) = .010094\) and \(Pr(\neg Alarm) = .989906\), so \(\alpha : Burglary\), \(\beta : Alarm\) and \(k = 4\) in 3.25 give

\begin{equation*} Pr^{\prime}(Burglary) = \frac{4(.000095) + .000005}{4(.010094) + .989906} = \frac{.000385}{1.030282} \approx 3.74 \times 10^{-4} . \end{equation*}

(a) By Equation 3.28 the Bayes factor of the reading \(y\) as soft evidence on \(X = 0\) is the likelihood ratio \(f(y|X{=}0)/f(y|X{=}1)\). The identical normalizers \(1/\sqrt{2\pi\sigma^2}\) cancel and \(-(y+1)^2 + (y-1)^2 = -4y\), so

\begin{equation*} k = e^{[-(y+1)^2 + (y-1)^2]/2\sigma^2} = e^{-4y/2\sigma^2} = e^{-2y/\sigma^2} . \end{equation*}

(b) Evaluating \(k = e^{-2y/\sigma^2}\) in each case:

(c) The reading \(y = 0\), and only that reading, is neutral for every \(\sigma\):

\begin{equation*} e^{-2y/\sigma^2} = 1 \iff -\frac{2y}{\sigma^2} = 0 \iff y = 0 , \end{equation*}

since \(\sigma^2 > 0\). For a Bayes factor of \(2\) with \(\sigma = 0.2\), so \(\sigma^2 = 0.04\),

\begin{equation*} e^{-50y} = 2 \iff -50y = \ln 2 \iff y = -\frac{\ln 2}{50} \approx -0.0139 . \end{equation*}

Since \(Y = y\) typically has probability zero, \(Pr(x|y)\) means the limit of conditioning on a shrinking interval, \(Pr(x|y) = \lim_{\Delta \to 0^{+}} Pr(x \mid A_\Delta)\) with \(A_\Delta\) the event \(y \le Y \le y + \Delta\). Assume \(0 < Pr(x) < 1\), that \(f(\cdot|x)\) and \(f(\cdot|\bar{x})\) are continuous at \(y\), and that \(f(y) > 0\) and \(f(y|\bar{x}) > 0\).

Case analysis on \(X\) applied to the event \(Y \le t\) gives \(F(t) = F(t|x)Pr(x) + F(t|\bar{x})Pr(\bar{x})\) for the CDFs, so differentiating (each CDF is the integral of its density, Section 3.7.1),

\begin{equation*} f(t) = f(t|x)\,Pr(x) + f(t|\bar{x})\,Pr(\bar{x}) , \end{equation*}

whence \(f\) is continuous at \(y\) too. Bayes rule (Equation 3.19) applied to the ordinary event \(A_\Delta\) — legitimate for all small \(\Delta > 0\), since \(Pr(A_\Delta) \ge \Delta \cdot \min_{[y,y+\Delta]} f > 0\) by continuity and positivity of \(f\) at \(y\) — gives, after dividing by \(Pr(x) > 0\) and writing each probability as an integral scaled by \(\Delta\),

\begin{equation*} \frac{Pr(x \mid A_\Delta)}{Pr(x)} = \frac{Pr(A_\Delta \mid x)}{Pr(A_\Delta)} = \frac{\frac{1}{\Delta}\int_{y}^{y+\Delta} f(t|x)\,dt} {\frac{1}{\Delta}\int_{y}^{y+\Delta} f(t)\,dt} . \end{equation*}

Each average lies between the minimum and the maximum of a function continuous at \(y\), so it tends to that function’s value at \(y\) as \(\Delta \to 0^{+}\); the quotient therefore tends to \(f(y|x)/f(y)\), and the left side to \(Pr(x|y)/Pr(x)\), which is the hint:

\begin{equation*} \frac{Pr(x|y)}{Pr(x)} = \frac{f(y|x)}{f(y)} . \end{equation*}

The identical argument with \(\bar{x}\) in place of \(x\) gives \(Pr(\bar{x}|y)/Pr(\bar{x}) = f(y|\bar{x})/f(y)\), which is nonzero; dividing the two identities cancels \(f(y)\), and rearranging the four factors on the left,

\begin{equation*} \frac{Pr(x|y)/Pr(x)}{Pr(\bar{x}|y)/Pr(\bar{x})} = \frac{Pr(x|y)\,Pr(\bar{x})}{Pr(\bar{x}|y)\,Pr(x)} = \frac{Pr(x|y)/Pr(\bar{x}|y)}{Pr(x)/Pr(\bar{x})} = \frac{f(y|x)}{f(y|\bar{x})} , \end{equation*}

which is Equation 3.27.

Method (2): apply Bayes rule to \(A_\Delta\) at both values of \(X\) and take the ratio, so that \(Pr(A_\Delta)\) cancels at once:

\begin{equation*} \frac{Pr(x|A_\Delta)}{Pr(\bar{x}|A_\Delta)} = \frac{Pr(A_\Delta|x)\,Pr(x)}{Pr(A_\Delta|\bar{x})\,Pr(\bar{x})} ; \end{equation*}

dividing by \(Pr(x)/Pr(\bar{x})\) and letting \(\Delta \to 0^{+}\) gives 3.27 without the marginal density.

Let \(S\) be the event that the sensor reads positive, so \(f_p = Pr(S|\neg\beta)\) and \(f_n = Pr(\neg S|\beta)\), whence \(Pr(S|\beta) = 1 - f_n\) and \(Pr(\neg S|\neg\beta) = 1 - f_p\) by Equation 3.5; assume \(0 < Pr(\beta) < 1\) and \(0 < f_p < 1\), which is exactly what makes \(O(\beta)\) positive and both Bayes factors finite. Bayes rule (Equation 3.19) applied to numerator and denominator of the posterior odds, with the common factor \(Pr(S)\) cancelling, gives

\begin{equation*} O^{\prime}(\beta) = \frac{Pr(S|\beta)\,Pr(\beta)}{Pr(S|\neg\beta)\,Pr(\neg\beta)} = \frac{1 - f_n}{f_p}\, O(\beta) , \end{equation*}

so \(k^{+} = (1 - f_n)/f_p\); the same computation with \(\neg S\) for \(S\) gives \(k^{-} = f_n/(1 - f_p)\). The two requirements are \(k^{+} = k\) and \(k^{-} = 1/k\) (a decrease by a factor \(k\) divides the odds by \(k\)), that is, clearing the nonzero denominators,

\begin{equation*} 1 - f_n = k\,f_p , \qquad k\,f_n = 1 - f_p . \end{equation*}

Substituting \(f_p = 1 - k f_n\) from the second into the first,

\begin{equation*} 1 - f_n = k\,(1 - k f_n) \iff (k^{2} - 1)\,f_n = k - 1 \iff (k-1)(k+1)\,f_n = k - 1 , \end{equation*}

and \(k > 1\) permits cancelling \(k - 1\), leaving \(f_n = 1/(k+1)\) and hence

\begin{equation*} f_p = 1 - k f_n = 1 - \frac{k}{k+1} = \frac{1}{k+1} = f_n . \end{equation*}

Part (a). By (4.1), \(\mathrm{Markov}(G)\) has one statement \(I(V, \mathrm{Parents}(V), \mathrm{NonDescendants}(V))\) per variable \(V\); reading the descendants off the DAG, the Markovian assumptions are

\begin{equation*} \begin{aligned} I(A, \emptyset, \{B,E\}), \qquad & I(B, \emptyset, \{A,C\}),\\ I(C, A, \{B,D,E\}), \qquad & I(D, AB, \{C,E\}),\\ I(E, B, \{A,C,D,F,G\}), \qquad & I(F, CD, \{A,B,E\}),\\ I(G, F, \{A,B,C,D,E,H\}), \qquad & I(H, EF, \{A,B,C,D,G\}). \end{aligned} \end{equation*}

Part (b). By the chain rule for Bayesian networks (4.2), each variable contributes exactly one factor, its own parameter given the values its parents receive:

\begin{equation*} \Pr(a,b,c,d,e,f,g,h) = \theta_a\,\theta_b\,\theta_{c\mid a}\,\theta_{d\mid ab}\, \theta_{e\mid b}\,\theta_{f\mid cd}\,\theta_{g\mid f}\,\theta_{h\mid ef}. \end{equation*}

Part (c). Both quantities are decided by independencies the DAG guarantees, so the incomplete parametrization is no obstacle.

\(\Pr(A=0,B=0)\). Since \(I(B, \emptyset, \{A,C\})\) lies in \(\mathrm{Markov}(G)\), decomposition (4.4) yields \(I_{\Pr}(B, \emptyset, A)\) (equivalently, \(A, B\) are distinct roots, so Exercise 4.3 and Theorem 4.2 apply), whence

\begin{equation*} \Pr(A=0, B=0) = \theta_{A=0}\,\theta_{B=0} = (.8)(.3) = .24 . \end{equation*}

\(\Pr(E=1 \mid A=1)\). Here \(\mathrm{dsep}_G(A, \emptyset, E)\) holds: with an empty conditioning set a convergent valve is automatically closed, and each path from \(A\) to \(E\) carries one (the neighbours of \(E\) are only \(B\) and \(H\), so every path ends \(B \to E\) or \(H \leftarrow E\)):

\begin{equation*} \begin{aligned} &A \to D \leftarrow B \to E &&\text{closed at } D,\\ &A \to C \to F \leftarrow D \leftarrow B \to E &&\text{closed at } F,\\ &A \to C \to F \to H \leftarrow E &&\text{closed at } H,\\ &A \to D \to F \to H \leftarrow E &&\text{closed at } H, \end{aligned} \end{equation*}

so every path is blocked. By Theorem 4.2, \(I_{\Pr}(A, \emptyset, E)\), whence

\begin{equation*} \Pr(E=1 \mid A=1) = \Pr(E=1) = \sum_b \theta_{E=1 \mid b}\,\theta_b = (.1)(.7) + (.9)(.3) = .34 . \end{equation*}

Part (d). All three are false; one open path each.

(i) \(\mathrm{dsep}(A, \{B,H\}, E)\): the path \(A \to C \to F \to H \leftarrow E\) is sequential at \(C\) and at \(F\), neither in \(\{B,H\}\), and convergent at \(H \in \{B,H\}\), so every valve is open.

(ii) \(\mathrm{dsep}(G, D, E)\): on the path

\begin{equation*} G \leftarrow F \leftarrow C \leftarrow A \to D \leftarrow B \to E \end{equation*}

the valves are sequential at \(F\) and \(C\), divergent at \(A\) and at \(B\), none of them \(D\), and convergent at the evidence node \(D\) — all open.

(iii) \(\mathrm{dsep}(AB, F, GH)\): the path \(B \to E \to H\) joins \(B \in \{A,B\}\) to \(H \in \{G,H\}\) through its single valve, sequential at \(E \notin \{F\}\), hence open.

With an empty conditioning set every convergent valve is closed and every sequential or divergent valve open, so an unblocked path is exactly one that climbs from one endpoint to some node \(W\) and descends to the other. Hence

\begin{equation*} \mathrm{dsep}_G(X, \emptyset, Y) \ \text{fails} \iff \mathrm{Anc}(X) \cap \mathrm{Anc}(Y) \neq \emptyset, \end{equation*}

where \(\mathrm{Anc}(V)\) is \(V\) together with its ancestors. (Left to right, the turning point \(W\) is a common ancestor. Right to left, pick a common ancestor \(W\) minimising the total length of shortest directed paths \(W \to \cdots \to X\) and \(W \to \cdots \to Y\); these share no node but \(W\), since a shared node would be a common ancestor with a smaller total, so splicing them gives a path all of whose valves are sequential or divergent.)

The ancestral sets in this DAG are

\begin{equation*} \begin{aligned} &\mathrm{Anc}(A_1) = \{A_1\}, && \mathrm{Anc}(B_1) = \{B_1\},\\ &\mathrm{Anc}(C_1) = \{C_1\}, && \mathrm{Anc}(A_2) = \{A_1, A_2\},\\ &\mathrm{Anc}(B_2) = \{A_1, B_1, B_2\}, && \mathrm{Anc}(C_2) = \{B_1, C_1, C_2\},\\ &\mathrm{Anc}(A_3) = \{A_1, A_2, A_3\},\\ &\mathrm{Anc}(B_3) = \{A_1, A_2, B_1, B_2, B_3\},\\ &\mathrm{Anc}(C_3) = \{A_1, B_1, B_2, C_1, C_2, C_3\}. \end{aligned} \end{equation*}

Case \(i = 1\). All three hold: \(A_1, B_1, C_1\) are roots, so their ancestral sets are the pairwise disjoint singletons \(\{A_1\}, \{B_1\}, \{C_1\}\) (also immediate from Exercise 4.3).

Case \(i = 2\). \(\mathrm{dsep}_G(A_2, \emptyset, B_2)\) fails: \(A_1\) is a common ancestor, witnessing the unblocked path \(A_2 \leftarrow A_1 \to B_2\), divergent at \(A_1\). Likewise \(\mathrm{dsep}_G(B_2, \emptyset, C_2)\) fails on the common ancestor \(B_1\) and the path \(B_2 \leftarrow B_1 \to C_2\). But \(\mathrm{dsep}_G(A_2, \emptyset, C_2)\) holds, since \(\mathrm{Anc}(A_2) = \{A_1, A_2\}\) and \(\mathrm{Anc}(C_2) = \{B_1, C_1, C_2\}\) are disjoint; explicitly, each of the paths

\begin{equation*} \begin{aligned} &A_2 \leftarrow A_1 \to B_2 \leftarrow B_1 \to C_2 &&\text{closed at } B_2,\\ &A_2 \leftarrow A_1 \to B_2 \to C_3 \leftarrow C_2 &&\text{closed at } C_3,\\ &A_2 \to B_3 \leftarrow B_2 \leftarrow B_1 \to C_2 &&\text{closed at } B_3,\\ &A_2 \to B_3 \leftarrow B_2 \to C_3 \leftarrow C_2 &&\text{closed at } B_3, \end{aligned} \end{equation*}

contains a convergent valve, which is closed given \(\emptyset\).

Case \(i = 3\). All three fail, on the common ancestors \(A_2\), \(A_1\) and \(B_2\) respectively, which give the unblocked paths

\begin{equation*} \begin{aligned} &A_3 \leftarrow A_2 \to B_3, \\ &A_3 \leftarrow A_2 \leftarrow A_1 \to B_2 \to C_3, \\ &B_3 \leftarrow B_2 \to C_3, \end{aligned} \end{equation*}

whose valves are all sequential or divergent, hence open.

Summary. Of the nine statements exactly four hold: all three at \(i = 1\), plus \(\mathrm{dsep}_G(A_2, \emptyset, C_2)\).

Every path between two distinct roots contains a convergent valve, which is closed given \(\emptyset\). Let

\begin{equation*} \pi:\ X = N_0 - N_1 - \cdots - N_k = Y \end{equation*}

be any path between roots \(X \neq Y\), and let \(d_j \in \{\rightarrow, \leftarrow\}\) record the orientation of the \(j\)-th edge as traversed from \(N_0\). Then \(k \ge 2\): \(k \ge 1\) as \(X \neq Y\), and \(k = 1\) would make one root a parent of the other. Moreover \(d_1 = \rightarrow\) and \(d_k = \leftarrow\), since no edge points into a root. Take the smallest \(i\) with \(d_{i+1} = \leftarrow\), well defined because \(d_k = \leftarrow\); then \(d_i = \rightarrow\) by minimality (for \(i = 1\) this is \(d_1\)), so \(\pi\) contains

\begin{equation*} N_{i-1} \to N_i \leftarrow N_{i+1}, \end{equation*}

a convergent valve at \(N_i\), closed by Definition 4.2 because \(\mathbf{Z} = \emptyset\) contains neither \(N_i\) nor any of its descendants. Hence every path is blocked and \(\mathrm{dsep}_G(X, \emptyset, Y)\).

Because \(S\) is a leaf it belongs to no family but its own, so its parameter factors out of the chain rule (4.2) and the rescaling becomes a common factor. Write \(u(\mathbf{x})\) for the value \(\mathbf{x}\) assigns to \(U\) and

\begin{equation*} f(\mathbf{x}) \;=\; \prod_{X \in \mathbf{X}} \theta_{x \mid \mathbf{u}_X} \end{equation*}

for the product of the parameters contributed by the families of \(\mathbf{X}\), none of which mentions \(S\). Then (4.2) gives

\begin{equation*} \Pr(\mathbf{x}, s) = \theta_{s \mid u(\mathbf{x})}\, f(\mathbf{x}), \qquad \Pr^{\prime}(\mathbf{x}, s) = \eta\, \theta_{s \mid u(\mathbf{x})}\, f(\mathbf{x}) = \eta\, \Pr(\mathbf{x}, s), \end{equation*}

with the same constant \(\eta\) for every \(\mathbf{x}\), precisely because \(\eta\) does not depend on \(u\). (Only the \(s\)-row of \(\Theta_{S \mid U}\) is constrained; the other rows may be set to restore normalisation, and the argument never touches them.) Summing over the instantiations \(\mathbf{x}\) gives \(\Pr^{\prime}(s) = \eta\,\Pr(s)\), so assuming \(\Pr(s) > 0\) — needed for \(\Pr(\mathbf{x} \mid s)\) to be defined, and giving \(\Pr^{\prime}(s) > 0\) since \(\eta > 0\) —

\begin{equation*} \Pr^{\prime}(\mathbf{x} \mid s) = \frac{\eta\, \Pr(\mathbf{x}, s)}{\eta\, \Pr(s)} = \Pr(\mathbf{x} \mid s). \end{equation*}

Everything follows from one lemma about prefixes of a topological order. Fix a topological ordering \(X_1, \ldots, X_n\) of \(\mathbf{Z}\) (it exists since \(G\) is acyclic), so the parents \(\mathbf{U}_i\) of \(X_i\) occur among \(X_1, \ldots, X_{i-1}\); let \(\mathbf{u}_i\) be the value an instantiation assigns to \(\mathbf{U}_i\). Each variable contributes exactly one compatible parameter, so (4.2) reads

\begin{equation*} \Pr(x_1, \ldots, x_n) = \prod_{i=1}^{n} \theta_{x_i \mid \mathbf{u}_i}. \qquad (\ast) \end{equation*}

Lemma. For every \(0 \le k \le n\) and every instantiation \(x_1 \cdots x_k\),

\begin{equation*} \Pr(x_1, \ldots, x_k) = \sum_{x_{k+1}, \ldots, x_n} \Pr(x_1, \ldots, x_n) = \prod_{i=1}^{k} \theta_{x_i \mid \mathbf{u}_i}. \qquad (\dagger) \end{equation*}

Downward induction on \(k\): at \(k = n\) this is \((\ast)\), and summing \((\dagger)\) at \(k+1\) over the values of \(X_{k+1}\) factors out \(\prod_{i \le k} \theta_{x_i \mid \mathbf{u}_i}\) — legitimately, since \(\mathbf{U}_{k+1} \subseteq \{X_1, \ldots, X_k\}\) makes \(\mathbf{u}_{k+1}\) a function of \(x_1 \cdots x_k\) alone — and leaves \(\sum_{x} \theta_{x \mid \mathbf{u}_{k+1}} = 1\) by Definition 4.1.

Part (a). Take \(k = 0\): the empty product is \(1\), so \(\sum_{\mathbf{z}} \Pr(\mathbf{z}) = 1\); and \(\Pr(\mathbf{z}) \ge 0\) since every parameter is nonnegative.

Part (c). For \(X = X_i\) with parents \(\mathbf{U}\), applying \((\dagger)\) at \(k = i\) and at \(k = i-1\) gives, for every \(x_1 \cdots x_{i-1}\) assigning \(\mathbf{u}\) to \(\mathbf{U}\),

\begin{equation*} \Pr(x_1, \ldots, x_{i-1}, x) = \Pr(x_1, \ldots, x_{i-1})\, \theta_{x \mid \mathbf{u}}, \end{equation*}

the factor \(\theta_{x \mid \mathbf{u}}\) depending on \(x_1 \cdots x_{i-1}\) only through \(\mathbf{u}\). Summing over the instantiations \(\mathbf{v}\) of \(\mathbf{V} = \{X_1, \ldots, X_{i-1}\} \setminus \mathbf{U}\),

\begin{equation*} \Pr(x, \mathbf{u}) = \theta_{x \mid \mathbf{u}} \sum_{\mathbf{v}} \Pr(\mathbf{v}, \mathbf{u}) = \theta_{x \mid \mathbf{u}}\, \Pr(\mathbf{u}), \end{equation*}

so \(\Pr(x \mid \mathbf{u}) = \theta_{x \mid \mathbf{u}}\) whenever \(\Pr(\mathbf{u}) > 0\) (otherwise the claim is vacuous).

Part (b). The lemma holds for every topological order, so choose one adapted to \(X\). Put \(\mathbf{D} = \{X\} \cup \mathrm{Descendants}(X)\), so that \(\mathbf{Z} \setminus \mathbf{D} = \mathbf{U} \cup \mathbf{N}\) with \(\mathbf{N} = \mathrm{NonDescendants}(X)\); this set is closed under parents, since a parent in \(\mathbf{D}\) would make its child a descendant of \(X\). Hence listing \(\mathbf{Z} \setminus \mathbf{D}\) first, then \(X\), then \(\mathbf{D} \setminus \{X\}\), each block in a topological order of the sub-DAG it induces, is a topological order of \(G\). With it \(X = X_i\) and \(\{X_1, \ldots, X_{i-1}\} = \mathbf{U} \cup \mathbf{N}\), so \((\dagger)\) at \(k = i\) and \(k = i-1\) gives

\begin{equation*} \Pr(x, \mathbf{u}, \mathbf{n}) = \Pr(\mathbf{u}, \mathbf{n})\, \theta_{x \mid \mathbf{u}} . \end{equation*}

When \(\Pr(\mathbf{u}, \mathbf{n}) > 0\), dividing gives \(\Pr(x \mid \mathbf{u}, \mathbf{n}) = \theta_{x \mid \mathbf{u}}\), and \(\Pr(\mathbf{u}) \ge \Pr(\mathbf{u}, \mathbf{n}) > 0\) lets part (c) rewrite the right-hand side as \(\Pr(x \mid \mathbf{u})\); the instantiations with \(\Pr(\mathbf{u}, \mathbf{n}) = 0\) are excused by the definition of \(I_{\Pr}\) in Section 4.4. Hence \(I_{\Pr}(X, \mathbf{U}, \mathbf{N})\) for every \(X\), which is \(\mathrm{Markov}(G)\).

Induct on \(k\) to get \((\mathrm{C}_k) = I(S_1, S_2, \{S_3, \ldots, S_k\})\) for \(3 \le k \le n\); at \(k = n\) this is the independence that \(\mathrm{dsep}_G(S_1, S_2, \{S_3, \ldots, S_n\})\) asserts.

The descendants of \(S_i\) are \(S_{i+1}, \ldots, S_n\) and \(O_i, \ldots, O_n\), so for \(i \ge 2\) the set \(\mathrm{Markov}(G)\) contains \(I(S_i, S_{i-1}, \{S_1, \ldots, S_{i-2}\} \cup \{O_1, \ldots, O_{i-1}\})\), and decomposition (4.4) discards the observation variables, leaving

\begin{equation*} I\bigl(S_i,\ S_{i-1},\ \{S_1, \ldots, S_{i-2}\}\bigr). \qquad (\mathrm{N}_i) \end{equation*}

Base \(k = 3\): \((\mathrm{N}_3)\) is \(I(S_3, S_2, \{S_1\})\), so symmetry (4.3) gives \((\mathrm{C}_3)\).

Step: let \(3 \le k < n\) and assume \((\mathrm{C}_k)\). Weak union (4.7) applied to \((\mathrm{N}_{k+1})\) with \(\mathbf{Z} = \{S_k\}\), \(\mathbf{Y} = \{S_2, \ldots, S_{k-1}\}\), \(\mathbf{W} = \{S_1\}\), followed by symmetry (4.3), yields

\begin{equation*} I\bigl(S_1,\ \{S_2, \ldots, S_k\},\ S_{k+1}\bigr). \end{equation*}

Contraction (4.9) on \((\mathrm{C}_k)\) and this statement, with \(\mathbf{Z} = \{S_2\}\), \(\mathbf{Y} = \{S_3, \ldots, S_k\}\) (so \(\mathbf{Z} \cup \mathbf{Y} = \{S_2, \ldots, S_k\}\)) and \(\mathbf{W} = \{S_{k+1}\}\), gives \((\mathrm{C}_{k+1})\).

Store \(G\), with \(n\) nodes and \(e\) edges, in adjacency-list form with each node holding its child list and its parent list (either is computable from the other in \(O(n+e)\)); the three steps below run in \(O(n+e)\) time and \(O(n)\) auxiliary space, which is linear in the size \(\Theta(n+e)\) of \(G\).

Preprocessing, \(O(n+e)\): boolean arrays \(\mathrm{marked}\) (true exactly on \(\mathbf{X} \cup \mathbf{Y} \cup \mathbf{Z}\)), \(\mathrm{inZ}\), \(\mathrm{inY}\), \(\mathrm{deleted}\) (initially false), and an integer array \(\mathrm{outdeg}[V]\) holding the number of children of \(V\).

Step 1, prune leaves to a fixed point. A node is a leaf of the current graph iff \(\mathrm{outdeg} = 0\). Initialise a work list \(Q\) with every unmarked node of out-degree \(0\); while \(Q\) is nonempty, pop an undeleted \(W\), set \(\mathrm{deleted}[W]\), and for each undeleted parent \(P\) of \(W\) decrement \(\mathrm{outdeg}[P]\), pushing \(P\) if that makes it an unmarked leaf. Since \(\mathrm{outdeg}[V]\) is invariantly the number of undeleted children of \(V\), a node enters \(Q\) exactly when it first becomes an unmarked leaf, so the loop halts precisely at the fixed point Theorem 4.1 demands; out-degrees only decrease, so each node is pushed once and each parent list scanned once, giving \(O(n+e)\).

Step 2, delete the edges outgoing from \(\mathbf{Z}\): no change to the data structure is needed. Declare an edge \(P \to C\) usable iff neither endpoint is deleted and \(\mathrm{inZ}[P] = \mathbf{false}\), and let Step 3 cross usable edges in either direction. The test must be on the tail \(P\) even when the search sits at \(C\), since deleting \(P \to C\) makes it unusable both ways: in \(A \to P \to C\) with \(\mathbf{Z} = \{P\}\), \(\mathbf{X} = \{C\}\), \(\mathbf{Y} = \{A\}\), the graph \(G^{\prime}\) isolates \(C\), so \(\mathrm{dsep}_G(C, P, A)\) holds, whereas a search climbing from \(C\) to its parent \(P\) would report the opposite. Each test is \(O(1)\).

Step 3, undirected connectivity. Breadth-first search on \(G^{\prime}\) ignoring edge directions, seeded with all undeleted nodes of \(\mathbf{X}\): pop \(V\), report that \(\mathrm{dsep}_G(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\) fails if \(\mathrm{inY}[V]\), otherwise enqueue every unvisited undeleted neighbour reachable under the Step 2 rule; report that it holds if the queue empties first. Each node is visited once and each adjacency list scanned once, so this is \(O(n+e)\) time and \(O(n)\) space.

Steps 1 and 2 realise exactly the two pruning rules of Theorem 4.1, and no node of \(\mathbf{X} \cup \mathbf{Y} \cup \mathbf{Z}\) is ever deleted, so the search reaches \(\mathbf{Y}\) iff \(\mathbf{X}\) and \(\mathbf{Y}\) are connected in \(G^{\prime}\), which by Theorem 4.1 is iff d-separation fails.

Symmetry (4.3), weak union (4.7), contraction (4.9) and decomposition (4.4), in that order. Write (1) for \(I(\mathbf{X}, \mathbf{Y}, \mathbf{Z})\) and (2) for \(I(\mathbf{X} \cup \mathbf{Y}, \mathbf{Z}, \mathbf{W})\).

Symmetry turns (2) into \(I(\mathbf{W}, \mathbf{Z}, \mathbf{X} \cup \mathbf{Y})\); weak union, \(I(\mathbf{A}, \mathbf{S}, \mathbf{B} \cup \mathbf{C}) \Rightarrow I(\mathbf{A}, \mathbf{S} \cup \mathbf{B}, \mathbf{C})\) with \(\mathbf{B} = \mathbf{Y}\), \(\mathbf{C} = \mathbf{X}\), gives \(I(\mathbf{W}, \mathbf{Z} \cup \mathbf{Y}, \mathbf{X})\); and symmetry again

\begin{equation*} (3)\quad I(\mathbf{X}, \mathbf{Y} \cup \mathbf{Z}, \mathbf{W}). \end{equation*}

Contraction, \(I(\mathbf{A}, \mathbf{S}, \mathbf{B})\) and \(I(\mathbf{A}, \mathbf{S} \cup \mathbf{B}, \mathbf{C})\) only if \(I(\mathbf{A}, \mathbf{S}, \mathbf{B} \cup \mathbf{C})\), has (1) and (3) as its two premises under \(\mathbf{S} = \mathbf{Y}\), \(\mathbf{B} = \mathbf{Z}\), \(\mathbf{C} = \mathbf{W}\), so it yields \(I(\mathbf{X}, \mathbf{Y}, \mathbf{Z} \cup \mathbf{W})\); decomposition then drops \(\mathbf{Z}\), leaving

\begin{equation*} I(\mathbf{X}, \mathbf{Y}, \mathbf{W}). \end{equation*}

Every set operation is legitimate because the premises force \(\mathbf{X}, \mathbf{Y}, \mathbf{Z}, \mathbf{W}\) to be pairwise disjoint, the three arguments of an independence statement being disjoint.

Two reformulations of the definition do all the work; throughout, \(\mathbf{x}, \mathbf{y}, \mathbf{z}, \mathbf{w}\) range over instantiations and \(\Pr\) of a partial instantiation is the corresponding marginal.

Lemma A (product form). \(I_{\Pr}(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\) iff

\begin{equation*} \Pr(\mathbf{x}, \mathbf{y}, \mathbf{z}) \, \Pr(\mathbf{z}) = \Pr(\mathbf{x}, \mathbf{z}) \, \Pr(\mathbf{y}, \mathbf{z}) \quad \text{for all } \mathbf{x}, \mathbf{y}, \mathbf{z}. \tag{A} \end{equation*}

Indeed, if \(\Pr(\mathbf{y}, \mathbf{z}) = 0\) then \(\Pr(\mathbf{x}, \mathbf{y}, \mathbf{z}) = 0\) too and both sides of (A) vanish; otherwise \(\Pr(\mathbf{z}) > 0\) and (A) is the defining equation multiplied by \(\Pr(\mathbf{y}, \mathbf{z})\Pr(\mathbf{z})\), a reversible step.

Lemma B (factorization). \(I_{\Pr}(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\) iff there are nonnegative \(f, g\) with

\begin{equation*} \Pr(\mathbf{x}, \mathbf{y}, \mathbf{z}) = f(\mathbf{x}, \mathbf{z}) \, g(\mathbf{y}, \mathbf{z}) \quad \text{for all } \mathbf{x}, \mathbf{y}, \mathbf{z}. \tag{B} \end{equation*}

(\(\Rightarrow\)) Take \(f = \Pr(\mathbf{x}, \mathbf{z})\) and \(g = \Pr(\mathbf{y}, \mathbf{z})/\Pr(\mathbf{z})\) when \(\Pr(\mathbf{z}) > 0\), \(g = 0\) otherwise; then (A) gives \(fg = \Pr(\mathbf{x}, \mathbf{y}, \mathbf{z})\) in the first case, and both sides vanish in the second. (\(\Leftarrow\)) With \(F(\mathbf{z}) = \sum_{\mathbf{x}} f\) and \(G(\mathbf{z}) = \sum_{\mathbf{y}} g\), summing (B) gives \(\Pr(\mathbf{x}, \mathbf{z}) = fG\), \(\Pr(\mathbf{y}, \mathbf{z}) = Fg\) and \(\Pr(\mathbf{z}) = FG\), whence

\begin{equation*} \Pr(\mathbf{x}, \mathbf{z}) \Pr(\mathbf{y}, \mathbf{z}) = f g F G = \Pr(\mathbf{x}, \mathbf{y}, \mathbf{z}) \Pr(\mathbf{z}), \end{equation*}

which is (A).

Triviality. For \(\mathbf{Y} = \emptyset\) the single empty instantiation gives \(\Pr(\mathbf{x}, \mathbf{z}) = \Pr(\mathbf{x}, \mathbf{z}) \cdot 1\), which is (B) with \(g \equiv 1\).

Symmetry. Condition (A) is unchanged when \(\mathbf{X}\) and \(\mathbf{Y}\) are interchanged.

Decomposition. By Lemma B the premise is \(\Pr(\mathbf{x}, \mathbf{y}, \mathbf{w}, \mathbf{z}) = f(\mathbf{x}, \mathbf{z}) g(\mathbf{y}, \mathbf{w}, \mathbf{z})\); summing over \(\mathbf{w}\) leaves \(f(\mathbf{x}, \mathbf{z})\, g^{\prime}(\mathbf{y}, \mathbf{z})\), again of the form (B), so \(I_{\Pr}(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\); summing over \(\mathbf{y}\) instead gives \(I_{\Pr}(\mathbf{X}, \mathbf{Z}, \mathbf{W})\).

Weak union. In that same factorization read \(\mathbf{z}\mathbf{y}\) as one instantiation of \(\mathbf{Z} \cup \mathbf{Y}\) and set \(f^{\prime}(\mathbf{x}, \mathbf{z}\mathbf{y}) = f(\mathbf{x}, \mathbf{z})\), \(g^{\prime}(\mathbf{w}, \mathbf{z}\mathbf{y}) = g(\mathbf{y}, \mathbf{w}, \mathbf{z})\): this is (B) for the triple \((\mathbf{X}, \mathbf{Z} \cup \mathbf{Y}, \mathbf{W})\).

Contraction. If \(\Pr(\mathbf{y}, \mathbf{w}, \mathbf{z}) = 0\) the required disjunct holds; otherwise \(\Pr(\mathbf{y}, \mathbf{z}) > 0\) as well, and the second premise (at \(\mathbf{w}\) and \(\mathbf{z}\mathbf{y}\)) followed by the first gives

\begin{equation*} \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}, \mathbf{w}) = \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}) = \Pr(\mathbf{x} \mid \mathbf{z}). \end{equation*}

Intersection, \(\Pr\) strictly positive. Every instantiation now has nonzero probability, so all conditionals below are defined and the two premises read \(\Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{w}, \mathbf{y}) = \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{w})\) and \(\Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}, \mathbf{w}) = \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y})\); their left-hand sides coincide, so

\begin{equation*} \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{w}) = \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}) \quad \text{for all } \mathbf{y}, \mathbf{w}. \tag{C} \end{equation*}

Fixing one \(\mathbf{w}_0\), (C) makes \(\Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}) = \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{w}_0) =: h(\mathbf{x}, \mathbf{z})\) for every \(\mathbf{y}\), so

\begin{equation*} \Pr(\mathbf{x} \mid \mathbf{z}) = \sum_{\mathbf{y}} \Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}) \Pr(\mathbf{y} \mid \mathbf{z}) = h(\mathbf{x}, \mathbf{z}), \end{equation*}

and therefore \(\Pr(\mathbf{x} \mid \mathbf{z}, \mathbf{y}, \mathbf{w}) = h(\mathbf{x}, \mathbf{z}) = \Pr(\mathbf{x} \mid \mathbf{z})\). Strict positivity is what makes (C) available for all pairs \(\mathbf{y}, \mathbf{w}\) simultaneously.

Take \(X\) and \(Y\) to be independent uniform binary inputs to an exclusive-or gate with output \(Z\): the DAG \(X \rightarrow Z \leftarrow Y\) with CPTs

\begin{equation*} \theta_{X=1} = \theta_{Y=1} = \tfrac{1}{2}, \qquad \theta_{Z=1 \mid x, y} = [\, x \oplus y = 1 \,], \end{equation*}

where \([\cdot]\) is \(1\) when the condition holds and \(0\) otherwise. The chain rule (Equation 4.2) induces

Summing the table, \(\Pr(X = 1) = \Pr(Y = 1) = \Pr(Z = 1) = \tfrac{1}{2}\). Conditioning on \(X = 1\) leaves \(Y\) uniform with \(Z = 1\) exactly when \(Y = 0\), and conditioning on \(X = 0\) gives \(Z = Y\), so

\begin{equation*} \Pr(Z = 1 \mid X = 1) = \Pr(Z = 1 \mid X = 0) = \tfrac{1}{2} = \Pr(Z = 1), \end{equation*}

that is, \(I_{\Pr}(Z, \emptyset, X)\); and \(I_{\Pr}(Z, \emptyset, Y)\) by the symmetry of the gate and of the input distribution in \(X\) and \(Y\). But the two inputs together determine the output:

\begin{equation*} \Pr(Z = 1 \mid X = 1, Y = 1) = 0 \ne \tfrac{1}{2} = \Pr(Z = 1), \end{equation*}

so \(I_{\Pr}(Z, \emptyset, XY)\) fails and composition (Equation 4.5) is violated.

Take \(X\), \(Y\), \(Z\) binary and always equal, the common value set by a fair coin:

\begin{equation*} \Pr(X = Y = Z = 1) = \Pr(X = Y = Z = 0) = \tfrac{1}{2}, \end{equation*}

every other instantiation having probability \(0\). By Exercise 4.9 intersection holds for every strictly positive distribution, so a counterexample must encode a logical constraint, and equality is the simplest one.

It is induced by \(X \rightarrow Y \rightarrow Z\) with \(\theta_{X=1} = \tfrac{1}{2}\) and the deterministic CPTs \(\theta_{Y = y \mid X = x} = [\, y = x \,]\), \(\theta_{Z = z \mid Y = y} = [\, z = y \,]\).

(i) \(I_{\Pr}(X, Z, Y)\). Given \(Z = z\), which has probability \(\tfrac{1}{2} > 0\), all mass sits on \(X = Y = z\); so \(\Pr(y,z) > 0\) forces \(y = z\) and then \(\Pr(x \mid z, y) = [\, x = z \,] = \Pr(x \mid z)\), while for \(y \ne z\) we have \(\Pr(y,z) = 0\) and the second disjunct of the definition applies.

(ii) \(I_{\Pr}(X, Y, Z)\). The same argument with \(Y\) and \(Z\) interchanged: \(Y = y\) pins \(X = y\), so \(\Pr(x \mid y,z) = [\, x = y \,] = \Pr(x \mid y)\) whenever \(\Pr(y,z) > 0\).

(iii) \(I_{\Pr}(X, \emptyset, YZ)\) fails, since \(\Pr(Y=1, Z=1) = \tfrac{1}{2} \ne 0\) and

\begin{equation*} \Pr(X = 1 \mid Y = 1, Z = 1) = 1 \ne \tfrac{1}{2} = \Pr(X = 1), \end{equation*}

so neither disjunct holds. Intersection, Equation 4.13, is therefore violated.

Take the two DAGs

\begin{equation*} \begin{aligned} G_1 &: \quad A \rightarrow B, \quad B \rightarrow C, \quad C \rightarrow D, \quad A \rightarrow D,\\ G_2 &: \quad B \rightarrow A, \quad B \rightarrow C, \quad C \rightarrow D, \quad A \rightarrow D. \end{aligned} \end{equation*}

Both have four edges, both are acyclic (topological orders \(A,B,C,D\) and \(B,A,C,D\)), and they differ in the orientation of the edge between \(A\) and \(B\). By Definition 4.2, \(\mathrm{dsep}(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\) holds iff \(\mathrm{dsep}(X, \mathbf{Z}, Y)\) for every \(X \in \mathbf{X}\) and \(Y \in \mathbf{Y}\), so only singleton pairs need checking.

(i) Adjacent pairs. Both DAGs have the skeleton \(A - B - C - D - A\), and a single-edge path carries no valve and is never blocked, so \(\{A,B\}\), \(\{B,C\}\), \(\{C,D\}\), \(\{A,D\}\) are d-separated by no \(\mathbf{Z}\) in either DAG.

(ii) The pair \(\{A, C\}\), where \(\mathbf{Z} \subseteq \{B, D\}\). The two paths are \(A \rightarrow B \rightarrow C\) (sequential at \(B\)) and \(A \rightarrow D \leftarrow C\) in \(G_1\), and \(A \leftarrow B \rightarrow C\) (divergent at \(B\)) with the same \(A \rightarrow D \leftarrow C\) in \(G_2\). Sequential and divergent valves close under the identical condition \(B \in \mathbf{Z}\), and \(D\) is a leaf in both DAGs, so both give:

So in both DAGs \(A\) and \(C\) are d-separated exactly by \(\mathbf{Z} = \{B\}\).

(iii) The pair \(\{B, D\}\), where \(\mathbf{Z} \subseteq \{A, C\}\). The paths are \(B \rightarrow C \rightarrow D\) together with \(B \leftarrow A \rightarrow D\) in \(G_1\) and \(B \rightarrow A \rightarrow D\) in \(G_2\); every valve is again sequential or divergent, hence closed exactly when its node lies in \(\mathbf{Z}\):

Again identical: \(B\) and \(D\) are d-separated exactly by \(\mathbf{Z} = \{A, C\}\). Agreeing on every singleton query, \(G_1\) and \(G_2\) agree on every d-separation query.

Both hold, with blocking as in Definition 4.2.

Observation. If \(\mathbf{S} \subseteq \mathbf{S}^{\prime}\) and a valve of a path is closed given \(\mathbf{S}^{\prime}\) but open given \(\mathbf{S}\), it is sequential or divergent with node in \(\mathbf{S}^{\prime}\setminus\mathbf{S}\): such a valve is closed exactly when its node lies in the conditioning set, while a convergent valve closed given \(\mathbf{S}^{\prime}\) is a fortiori closed given the smaller \(\mathbf{S}\), which contains neither its node nor a descendant.

Intersection. Assume (1) \(\mathrm{dsep}(\mathbf{X}, \mathbf{Z}\cup\mathbf{W}, \mathbf{Y})\) and (2) \(\mathrm{dsep}(\mathbf{X}, \mathbf{Z}\cup\mathbf{Y}, \mathbf{W})\), and suppose the conclusion fails; the premises force \(\mathbf{X},\mathbf{Y},\mathbf{Z},\mathbf{W}\) pairwise disjoint. Let \(\pi\) be a shortest path from \(s \in \mathbf{X}\) to \(t \in \mathbf{Y}\cup\mathbf{W}\) unblocked by \(\mathbf{Z}\).

• (i) \(t \in \mathbf{Y}\): by (1) some valve of \(\pi\) is closed given \(\mathbf{Z}\cup\mathbf{W}\), and it is open given \(\mathbf{Z}\), so by the Observation it is sequential or divergent at an interior node \(V \in \mathbf{W}\).
• (ii) \(t \in \mathbf{W}\): by (2), symmetrically, at an interior node \(V \in \mathbf{Y}\).

Either way the sub-path of \(\pi\) from \(s\) to \(V\) carries the same valve types at the same interior nodes, so it too is unblocked by \(\mathbf{Z}\); it runs from \(\mathbf{X}\) into \(\mathbf{Y}\cup\mathbf{W}\) and is shorter, contradicting the choice of \(\pi\). Hence \(\mathrm{dsep}(\mathbf{X}, \mathbf{Z}, \mathbf{Y}\cup\mathbf{W})\).

Chordality. Write \(An(\mathbf{S})\) for \(\mathbf{S}\) with all ancestors of its members and \(M(\mathbf{S})\) for the moral graph of \(G\) restricted to \(An(\mathbf{S})\). By Exercise 4.24, for disjoint sets \(\mathrm{dsep}_G(\mathbf{X},\mathbf{Z},\mathbf{Y})\) iff \(\mathbf{Z}\) separates \(\mathbf{X}\) from \(\mathbf{Y}\) in \(M(\mathbf{X}\cup\mathbf{Y}\cup\mathbf{Z})\). Monotonicity: for \(\mathbf{S}^{\prime}\subseteq\mathbf{S}\), \(M(\mathbf{S}^{\prime})\) is a subgraph of \(M(\mathbf{S})\), since \(An(\mathbf{S}^{\prime}) \subseteq An(\mathbf{S})\) and a marriage through a common child \(c \in An(\mathbf{S}^{\prime})\) is a marriage through \(c \in An(\mathbf{S})\). (Check!)

Assume (1) \(\mathrm{dsep}(X,\{Z,W\},Y)\) and (2) \(\mathrm{dsep}(W,\{X,Y\},Z)\) for distinct \(X,Y,Z,W\), and for contradiction that (3) \(\mathrm{dsep}(X,Z,Y)\) and (4) \(\mathrm{dsep}(X,W,Y)\) both fail. Put \(A = An(\{X,Y,Z,W\})\), \(M = M(\{X,Y,Z,W\})\), \(M_1 = M(\{X,Y,Z\})\), \(M_2 = M(\{X,Y,W\})\); premises (1) and (2) are read in \(M\), and \(M_1, M_2\) are subgraphs of \(M\) by monotonicity.

1. \(X\) and \(Y\) are non-adjacent in \(M\): an edge would be an \(X\)-\(Y\) path meeting neither \(Z\) nor \(W\), against (1).
2. By (3) and Exercise 4.24 there is an \(X\)-\(Y\) path \(p\) of \(M_1\), hence of \(M\), avoiding \(Z\); by 1 it has an interior node and by (1) it meets \(\{Z,W\}\), so it contains \(W\). Then \(W \in An(\{X,Y,Z\})\setminus\{X,Y,Z\}\), i.e. \(W\) is a proper ancestor of \(X\), \(Y\), or \(Z\). Symmetrically (4) gives an \(X\)-\(Y\) path \(q\) of \(M\) avoiding \(W\) and containing \(Z\), with \(Z\) a proper ancestor of \(X\), \(Y\), or \(W\).
3. In \(M^{-} = M\) minus \(X\) and \(Y\), premise (2) puts \(W\) and \(Z\) in distinct components \(C_W \ne C_Z\) joined by no edge of \(M\), and \(\mathrm{interior}(p) \subseteq C_W\), \(\mathrm{interior}(q) \subseteq C_Z\).
4. Let \(R\) be the last of \(X,Y,Z,W\) in a topological order of \(G\); \(R\) is a proper ancestor of none of the others, which by 2 excludes \(Z\) and \(W\), so \(R \in \{X,Y\}\). All of (1), (3), (4) are symmetric in their outer arguments, (2) mentions \(\{X,Y\}\) as a set, and \(A, M, M_1, M_2\) are unchanged by the swap, so take \(R = Y\).
5. \(Y\) has no child in \(A\): a child \(U \in A\) has \(U \notin An(Y)\) by acyclicity, hence is an ancestor of or equal to one of \(X, Z, W\), making \(Y\) a proper ancestor of one of them against 4.
6. So every neighbour of \(Y\) in \(M\) is a parent of \(Y\) in \(G[A]\): the other two sources of an \(M\)-edge at \(Y\), namely \(Y\) being a parent of the neighbour or the two being co-parents of a node of \(A\), both need \(Y\) to have a child in \(A\).
7. Let \(c\) and \(d\) be the neighbours of \(Y\) on \(p\) and on \(q\); by 1 and 3 they are interior, with \(c \in C_W\) and \(d \in C_Z\), so \(c \ne d\). By 6 both are parents of \(Y\), so moralization puts \(c - d\) in \(M\), joining \(C_W\) to \(C_Z\) against 3.

Hence (3) and (4) cannot both fail: \(\mathrm{dsep}(X, Z, Y)\) or \(\mathrm{dsep}(X, W, Y)\).

Run the procedure of Section 4.6.2, giving \(X_i\) as parents a minimal \(\mathbf{P} \subseteq \{X_1,\dots,X_{i-1}\}\) with \(I_{\Pr}(X_i, \mathbf{P}, \{X_1,\dots,X_{i-1}\}\setminus\mathbf{P})\); since \(G\) is a P-MAP of \(\Pr\), each test is the corresponding \(\mathrm{dsep}_G\) test. Two facts shorten every step: a predecessor adjacent to \(X_i\) lies in every admissible \(\mathbf{P}\) (a single edge carries no valve, Definition 4.2), and \(D\) is a leaf, so a convergent valve at \(D\) is closed exactly when \(D \notin \mathbf{P}\).

(a) Order \(A, D, B, C, E\).

• \(A\): \(\mathbf{P} = \emptyset\).
• \(D\): \(\mathrm{dsep}_G(D, \emptyset, A)\) fails, the sequential valve at \(B\) on \(A \rightarrow B \rightarrow D\) being open, so \(\mathbf{P} = \{A\}\).
• \(B\): adjacent to \(A\) and \(D\), so \(\mathbf{P} = \{A, D\}\).
• \(C\): adjacent to \(A\) and \(D\), and \(\mathrm{dsep}_G(C, \{A,D\}, B)\) fails because the convergent valve at \(D\) on \(C \rightarrow D \leftarrow B\) is open; so \(\mathbf{P} = \{A,B,D\}\).
• \(E\): adjacent to \(C\), and \(\{C\}\) suffices, every path out of \(E\) beginning \(E \leftarrow C\) and so carrying a sequential or divergent valve at \(C\).

\begin{equation*} A \rightarrow D, \quad A \rightarrow B, \quad D \rightarrow B, \quad A \rightarrow C, \quad B \rightarrow C, \quad D \rightarrow C, \quad C \rightarrow E. \end{equation*}

(b) Order \(A, B, C, D, E\), a topological order of \(G\), returns \(G\) itself:

• \(A\): \(\emptyset\); \(B\): adjacent to \(A\), so \(\{A\}\).
• \(C\): adjacent to \(A\), and \(\{A\}\) suffices, since \(C \leftarrow A \rightarrow B\) is closed at the divergent valve \(A\) and \(C \rightarrow D \leftarrow B\) at the convergent valve \(D\).
• \(D\): adjacent to \(B\) and \(C\), and \(\{B,C\}\) suffices, the paths \(D \leftarrow B \leftarrow A\) and \(D \leftarrow C \leftarrow A\) being closed at their sequential valves.
• \(E\): \(\{C\}\) as in (a).

\begin{equation*} A \rightarrow B, \quad A \rightarrow C, \quad B \rightarrow D, \quad C \rightarrow D, \quad C \rightarrow E. \end{equation*}

(c) Order \(E, D, C, B, A\).

• \(E\): \(\emptyset\).
• \(D\): \(\mathrm{dsep}_G(D, \emptyset, E)\) fails at the divergent valve \(C\) of \(D \leftarrow C \rightarrow E\), so \(\mathbf{P} = \{E\}\).
• \(C\): adjacent to \(D\) and \(E\), so \(\mathbf{P} = \{D, E\}\).
• \(B\): adjacent to \(D\), and \(\{C,D\}\) passes, since \(B \leftarrow A \rightarrow C \rightarrow E\) is closed at the sequential valve \(C\) and \(B \rightarrow D \leftarrow C \rightarrow E\) at the divergent valve \(C\) (its valve at \(D\) being open). Both \(\{D\}\) and \(\{D,E\}\) leave \(C\) on the right, where \(B \leftarrow A \rightarrow C\) is unblocked. So \(\mathbf{P} = \{C, D\}\).
• \(A\): adjacent to \(B\) and \(C\), and \(\{B,C\}\) suffices: \(A \rightarrow B \rightarrow D\), \(A \rightarrow C \rightarrow D\) and \(A \rightarrow C \rightarrow E\) are blocked at their sequential valves, and \(A \rightarrow B \rightarrow D \leftarrow C \rightarrow E\) at \(B\).

\begin{equation*} E \rightarrow D, \quad E \rightarrow C, \quad D \rightarrow C, \quad C \rightarrow B, \quad D \rightarrow B, \quad B \rightarrow A, \quad C \rightarrow A. \end{equation*}

The edgeless DAG \(G_\emptyset\) over \(\mathbf{X}\) is a D-MAP of every distribution, and the complete DAG \(G_c\) on any fixed ordering \(X_1,\dots,X_n\) (edges \(X_i \to X_j\) for \(i<j\)) is an I-MAP of every distribution. Both conditions of Section 4.6 are implications, and each graph satisfies its own vacuously.

D-MAP. \(G_\emptyset\) has no path at all between a node of \(\mathbf{X}_1\) and a node of \(\mathbf{Y}\), so the blocking condition of Definition 4.2 is met vacuously and \(\mathrm{dsep}_{G_\emptyset}(\mathbf{X}_1,\mathbf{Z},\mathbf{Y})\) holds for every disjoint triple. The consequent of

\begin{equation*} I_{\Pr}(\mathbf{X}_1,\mathbf{Z},\mathbf{Y}) \ \text{only if}\ \mathrm{dsep}_{G_\emptyset}(\mathbf{X}_1,\mathbf{Z},\mathbf{Y}) \end{equation*}

is thus always true, for every \(\Pr\).

I-MAP. \(G_c\) is acyclic, all edges running forward in the ordering. If \(\mathbf{X}_1,\mathbf{Z},\mathbf{Y}\) are disjoint with \(\mathbf{X}_1\) and \(\mathbf{Y}\) nonempty, pick \(X \in \mathbf{X}_1\) and \(Y \in \mathbf{Y}\); the edge joining them is a path with no valves, hence never blocked, so \(\mathrm{dsep}_{G_c}(\mathbf{X}_1,\mathbf{Z},\mathbf{Y})\) fails. D-separation holds in \(G_c\) only when \(\mathbf{X}_1 = \emptyset\) or \(\mathbf{Y} = \emptyset\), and for those triples \(I_{\Pr}\) holds by triviality. So the antecedent of

\begin{equation*} \mathrm{dsep}_{G_c}(\mathbf{X}_1,\mathbf{Z},\mathbf{Y}) \ \text{only if}\ I_{\Pr}(\mathbf{X}_1,\mathbf{Z},\mathbf{Y}) \end{equation*}

is never nontrivially true, for every \(\Pr\).

(a) \(\{B_1, C_1, C_3, B_2\}\); (b) yes; (c) \(A_3\), whose boundary \(\{A_2\}\) has size one.

By Corollary 1 the set \(\mathrm{mb}(X)\) of parents, children, and spouses of \(X\) (a spouse being another parent of a child of \(X\)) is a Markov blanket for \(X\); under the P-MAP hypothesis it is the Markov boundary, since it sits inside every blanket \(\mathbf{B}\). For \(I_{\Pr}(X,\mathbf{B},\mathbf{X}\setminus\mathbf{B}\setminus\{X\})\) gives \(\mathrm{dsep}_G(X,\mathbf{B},\mathbf{X}\setminus\mathbf{B}\setminus\{X\})\), \(G\) being a D-MAP, and any \(Y \in \mathrm{mb}(X)\setminus\mathbf{B}\) refutes that d-separation:

• \(Y\) a parent or child of \(X\): the edge between them is a valveless path, never blocked;
• \(Y\) a spouse through a child \(Z \in \mathbf{B}\): the only valve of \(X \to Z \leftarrow Y\) is convergent at \(Z\), hence open;
• \(Y\) a spouse through a child \(Z \notin \mathbf{B}\): the edge \(X \to Z\) is itself an unblocked path into the third argument.

(a) \(C_2\) has parents \(B_1, C_1\), sole child \(C_3\), and through \(C_3\) the single spouse \(B_2\):

\begin{equation*} \mathrm{mb}(C_2) = \{B_1, C_1, C_3, B_2\}. \end{equation*}

(b) Yes. The root \(A_1\) has children \(A_2, B_2\) and one spouse \(B_1\) (the other parent of \(B_2\)), so \(\mathrm{mb}(A_1) = \{A_2, B_2, B_1\}\); the leaf \(B_3\) has parents \(A_2, B_2\), so \(\mathrm{mb}(B_3) = \{A_2, B_2\}\) is a blanket of \(B_3\) by Corollary 1. Since \(\{A_2,B_2\} \subseteq \mathrm{mb}(A_1)\) and \(B_3 \notin \mathrm{mb}(A_1)\), it is enough that a superset \(\mathbf{B} \supseteq \mathbf{S}\) of a blanket with \(X \notin \mathbf{B}\) is again a blanket: splitting \(\mathbf{X}\setminus\mathbf{S}\setminus\{X\}\) as \((\mathbf{B}\setminus\mathbf{S}) \cup (\mathbf{X}\setminus\mathbf{B}\setminus\{X\})\), weak union turns \(I_{\Pr}(X,\mathbf{S},\mathbf{X}\setminus\mathbf{S}\setminus\{X\})\) into

\begin{equation*} I_{\Pr}\bigl(X,\ \mathbf{B},\ \mathbf{X}\setminus\mathbf{B}\setminus\{X\}\bigr). \end{equation*}

(c) Tabulating parents, children, and spouses for all nine nodes:

The smallest is \(\mathrm{mb}(A_3) = \{A_2\}\), and it is a boundary: its only proper subset \(\emptyset\) fails, since the edge \(A_2 \to A_3\) is an unblockable path and the D-MAP property then denies \(A_3\) marginal independence of the rest.

Partition \(\mathbf{R} = \mathbf{X}\setminus\{X\}\) into

\begin{equation*} \begin{aligned} \mathbf{S} &= \mathbf{B}_1 \cap \mathbf{B}_2, &\qquad \mathbf{D}_1 &= \mathbf{B}_1 \setminus \mathbf{B}_2, \\ \mathbf{D}_2 &= \mathbf{B}_2 \setminus \mathbf{B}_1, &\qquad \mathbf{O} &= \mathbf{R} \setminus (\mathbf{B}_1 \cup \mathbf{B}_2), \end{aligned} \end{equation*}

so that by Definition 4.3 the hypotheses read \(I_{\Pr}(X, \mathbf{S}\cup\mathbf{D}_1, \mathbf{D}_2\cup\mathbf{O})\) and \(I_{\Pr}(X, \mathbf{S}\cup\mathbf{D}_2, \mathbf{D}_1\cup\mathbf{O})\), while the goal is \(I_{\Pr}(X, \mathbf{S}, \mathbf{D}_1\cup\mathbf{D}_2\cup\mathbf{O})\); the side condition \(X \notin \mathbf{S}\) follows from \(X \notin \mathbf{B}_1\). Weak union (4.7) and the intersection axiom (4.13), the latter available because \(\Pr\) is strictly positive, give

\begin{equation*} \begin{aligned} (1)\quad &I_{\Pr}(X,\ \mathbf{S}\cup\mathbf{D}_1\cup\mathbf{O},\ \mathbf{D}_2), \\ (2)\quad &I_{\Pr}(X,\ \mathbf{S}\cup\mathbf{D}_2\cup\mathbf{O},\ \mathbf{D}_1), \\ (3)\quad &I_{\Pr}(X,\ \mathbf{S}\cup\mathbf{O},\ \mathbf{D}_1\cup\mathbf{D}_2), \\ (4)\quad &I_{\Pr}(X,\ \mathbf{S}\cup\mathbf{D}_1\cup\mathbf{D}_2,\ \mathbf{O}), \end{aligned} \end{equation*}

where (1) and (2) are weak union on the two hypotheses with \(\mathbf{Y} = \mathbf{O}\); (3) is intersection on (1) and (2) with \(\mathbf{Z} = \mathbf{S}\cup\mathbf{O}\), \(\mathbf{W} = \mathbf{D}_1\), \(\mathbf{Y} = \mathbf{D}_2\); and (4) is weak union on the first hypothesis with \(\mathbf{Y} = \mathbf{D}_2\), \(\mathbf{W} = \mathbf{O}\). Intersection once more, on (3) and (4) with \(\mathbf{Z} = \mathbf{S}\), \(\mathbf{W} = \mathbf{O}\), \(\mathbf{Y} = \mathbf{D}_1\cup\mathbf{D}_2\), yields

\begin{equation*} I_{\Pr}(X,\ \mathbf{S},\ \mathbf{D}_1\cup\mathbf{D}_2\cup\mathbf{O}), \end{equation*}

which with \(X \notin \mathbf{S}\) is exactly the statement that \(\mathbf{B}_1\cap\mathbf{B}_2\) is a Markov blanket for \(X\). \(\blacksquare\)

(a) The closure is the twelve statements displayed below; (b) the three orders return \(G_1\) (three edges), a four-edge DAG, and the complete DAG.

(a) From \(I(A,\emptyset,B)\) only symmetry is productive, both outer arguments being singletons and the conditioning set empty. From \(I(AB,C,D)\), symmetry gives \(I(D,C,AB)\), whose right argument is a pair: decomposition then gives \(I(D,C,A)\) and \(I(D,C,B)\), weak union (4.7) gives \(I(D,\{C,B\},A)\) and \(I(D,\{C,A\},B)\), and symmetry doubles each. Contraction (4.9) on \(I(D,C,A)\) and \(I(D,\{C,A\},B)\) returns \(I(D,C,AB)\), as does intersection (4.13) on \(I(D,\{C,A\},B)\) and \(I(D,\{C,B\},A)\), so the derivation has stabilized; modulo the triviality statements \(I(\mathbf{X}_1,\mathbf{Z},\emptyset)\):

\begin{equation*} \begin{aligned} &I(A,\emptyset,B), && I(B,\emptyset,A), \\ &I(AB,C,D), && I(D,C,AB), \\ &I(A,C,D), && I(D,C,A), \\ &I(B,C,D), && I(D,C,B), \\ &I(A,\{B,C\},D), && I(D,\{B,C\},A), \\ &I(B,\{A,C\},D), && I(D,\{A,C\},B). \end{aligned} \end{equation*}

That the list is closed is proved rather than inspected. Let \(G_1\) be the DAG \(A \to C\), \(B \to C\), \(C \to D\); its d-separations over disjoint triples are exactly the twelve statements above, both seeds among them, since \(A\) and \(B\) are non-adjacent roots joined only through the convergent valve at \(C\) and \(C\) blocks every path from \(\{A,B\}\) to \(D\). D-separation obeys every positive graphoid axiom — symmetry, decomposition, weak union, contraction and triviality from Definition 4.2, intersection from Exercise 4.13 — so the set of triples d-separated in \(G_1\) is closed under the axioms and contains the seeds, hence contains the closure, hence equals it. Note that \(C\) never occurs as an outer argument.

(b) By Section 4.6.2, \(\mathbf{P} \subseteq \{X_1,\dots,X_{i-1}\}\) is admissible for \(X_i\) exactly when \(I(X_i,\mathbf{P},\{X_1,\dots,X_{i-1}\}\setminus\mathbf{P})\) lies in the closure, and with \(\Pr\) strictly positive the minimal such \(\mathbf{P}\) is unique.

Order \(A, B, C, D\), giving \(G_1\) itself, hence a P-MAP of the statements in (a):

• \(A\): \(\emptyset\). \(B\): \(\emptyset\), by \(I(B,\emptyset,A)\).
• \(C\): none of \(I(C,\emptyset,AB)\), \(I(C,A,B)\), \(I(C,B,A)\) is in the closure, so \(\{A,B\}\).
• \(D\): \(I(D,C,AB)\) is in the closure and \(\emptyset\), \(\{A\}\), \(\{B\}\) fail, so \(\{C\}\).

Order \(D, C, B, A\), giving \(D \to C\), \(C \to B\), \(C \to A\), \(B \to A\):

• \(D\): \(\emptyset\). \(C\): \(I(C,\emptyset,D)\) absent, so \(\{D\}\).
• \(B\): \(I(B,\emptyset,\{C,D\})\) and \(I(B,D,C)\) absent, \(I(B,C,D)\) present, so \(\{C\}\).
• \(A\): \(I(A,\emptyset,\{B,C,D\})\), \(I(A,B,\{C,D\})\), \(I(A,C,\{B,D\})\), \(I(A,D,\{B,C\})\) all absent, \(I(A,\{B,C\},D)\) present, so \(\{B,C\}\).

This is an I-MAP, the d-separations it asserts being \(\mathrm{dsep}(D,C,A)\), \(\mathrm{dsep}(D,C,B)\), \(\mathrm{dsep}(D,C,AB)\), \(\mathrm{dsep}(D,\{B,C\},A)\) and \(\mathrm{dsep}(D,\{A,C\},B)\), all in the closure; it is not a P-MAP, the edge \(B \to A\) destroying \(I(A,\emptyset,B)\).

Order \(A, D, B, C\), giving the complete DAG \(A \to D\), \(A \to B\), \(A \to C\), \(D \to B\), \(D \to C\), \(B \to C\):

• \(A\): \(\emptyset\). \(D\): \(I(D,\emptyset,A)\) absent, only \(I(D,C,A)\) being present, so \(\{A\}\).
• \(B\): \(I(B,\emptyset,\{A,D\})\) is absent, since composition is not a graphoid axiom and so \(I(B,\emptyset,A)\) with \(I(B,C,D)\) does not combine; \(I(B,A,D)\) and \(I(B,D,A)\) are absent too, so \(\{A,D\}\).
• \(C\): no closure statement has \(C\) as an outer argument, so \(\{A,D,B\}\).

Asserting no independence at all, it is trivially an I-MAP, and minimal by Exercise 4.19: deleting \(A \to C\), for instance, would assert \(\mathrm{dsep}(C,\{D,B\},A)\), absent from the closure.

Delete an arbitrary edge \(U \to X_i\) of \(G\), obtaining \(G^{\prime}\); we exhibit a d-separation that holds in \(G^{\prime}\) while the matching independence fails in \(\Pr\), so that \(G^{\prime}\) is not an I-MAP. Every edge of \(G\) issues from a member of \(\mathbf{P}_i\) into \(X_i\), so \(U \in \mathbf{P}_i\) and in \(G^{\prime}\) the parents of \(X_i\) are \(\mathbf{P}_i^{\prime} = \mathbf{P}_i\setminus\{U\}\), those of every other node unchanged. Put

\begin{equation*} \mathbf{V}_i = \{X_1,\dots,X_{i-1}\}, \qquad \mathbf{Y} = \mathbf{V}_i \setminus \mathbf{P}_i^{\prime}. \end{equation*}

Claim: \(\mathrm{dsep}_{G^{\prime}}(X_i, \mathbf{P}_i^{\prime}, \mathbf{Y})\). Apply Theorem 4.1 with \(\mathbf{X}_1 \cup \mathbf{Z} \cup \mathbf{Y} = \{X_1,\dots,X_i\}\). The algorithm draws edges only from earlier to later variables, so \(X_1,\dots,X_n\) is a topological order of \(G^{\prime}\) and deleting \(X_n, X_{n-1},\dots,X_{i+1}\) in turn prunes leaves lying outside \(\{X_1,\dots,X_i\}\). Deleting next the edges outgoing from \(\mathbf{Z} = \mathbf{P}_i^{\prime}\) isolates \(X_i\): its outgoing edges went with its pruned children, and its incoming edges all issue from \(\mathbf{P}_i^{\prime}\). Theorem 4.1 then gives the claim, which is just the local Markov property, \(\mathbf{Y}\) consisting of non-descendants of \(X_i\).

Were \(G^{\prime}\) an I-MAP of \(\Pr\), the claim would force

\begin{equation*} I_{\Pr}\bigl(X_i,\ \mathbf{P}_i \setminus \{U\},\ \{X_1,\dots,X_{i-1}\}\setminus(\mathbf{P}_i\setminus\{U\})\bigr), \end{equation*}

exhibiting \(\mathbf{P}_i\setminus\{U\}\) as a proper subset of \(\mathbf{P}_i\) meeting the very condition the algorithm minimized over, a contradiction. As the edge was arbitrary, \(G\) is a minimal I-MAP of \(\Pr\). \(\blacksquare\)

Run the ordinary test on the determined set \(\mathrm{det}(\mathbf{Z})\) in place of \(\mathbf{Z}\): for disjoint \(\mathbf{X}, \mathbf{Z}, \mathbf{Y}\), declare \(\mathbf{X}\) and \(\mathbf{Y}\) independent given \(\mathbf{Z}\) whenever

\begin{equation*} \mathrm{dsep}_G\bigl(\mathbf{X}\setminus\mathrm{det}(\mathbf{Z}),\ \mathrm{det}(\mathbf{Z}),\ \mathbf{Y}\setminus\mathrm{det}(\mathbf{Z})\bigr), \end{equation*}

where \(\mathrm{det}(\mathbf{Z})\) is the least set with \(\mathbf{Z} \subseteq \mathrm{det}(\mathbf{Z})\) and \(V \in \mathrm{det}(\mathbf{Z})\) whenever \(V \in \mathbf{W}\) and \(\mathrm{Parents}(V) \subseteq \mathrm{det}(\mathbf{Z})\). The point is that a node of \(\mathbf{W}\) is a function of its parents, exactly one of its parameters being \(1\) per parent instantiation, so knowledge propagates beyond the evidence set, which the ordinary test cannot see. One topological sweep computes the closure, so the test stays linear-time.

Soundness. By Exercise 4.5(c), \(\Pr(v\mid\mathbf{u}) = \theta_{v\mid\mathbf{u}}\), so each \(V \in \mathbf{W}\) with \(\mathbf{U} = \mathrm{Parents}(V)\) has a unique value \(v = f_V(\mathbf{u})\) of parameter \(1\), whence \(\Pr(V \ne f_V(\mathbf{u}), \mathbf{u}) = 0\) and \(\Pr(f_V(\mathbf{u})\mid e) = 1\) for every positive-probability \(e\) implying \(\mathbf{U} = \mathbf{u}\). Enumerate \(\mathbf{D} = \mathrm{det}(\mathbf{Z})\setminus\mathbf{Z}\) as \(V_1,\dots,V_k\) in the order the closure added them, so that \(\mathrm{Parents}(V_j) \subseteq \mathbf{Z}\cup\{V_1,\dots,V_{j-1}\}\), and fix \(\mathbf{z}\) with \(\Pr(\mathbf{z}) > 0\). If \(\Pr(v_1\cdots v_{j-1}\mid\mathbf{z}) = 1\) then \(\mathbf{z}v_1\cdots v_{j-1}\) is a positive-probability event fixing \(\mathrm{Parents}(V_j)\), so \(\Pr(v_j \mid \mathbf{z}v_1\cdots v_{j-1}) = 1\) and the product is again \(1\); at \(j = k\) a unique \(\mathbf{d}\) has \(\Pr(\mathbf{d}\mid\mathbf{z}) = 1\), and hence \(\Pr(\alpha\mid\mathbf{z}) = \Pr(\alpha\mid\mathbf{z},\mathbf{d})\) for every event \(\alpha\). Split \(\mathbf{x}\) into its part \(\mathbf{x}_1\) over \(\mathbf{X}\setminus\mathrm{det}(\mathbf{Z})\) and \(\mathbf{x}_2\) over \(\mathbf{X}\cap\mathrm{det}(\mathbf{Z})\), and likewise \(\mathbf{y}\). If \(\mathbf{x}_2\) or \(\mathbf{y}_2\) contradicts \(\mathbf{z}\mathbf{d}\), both sides of the independence equation vanish; otherwise \(\mathbf{z}\mathbf{d}\) implies them and

\begin{equation*} \begin{aligned} \Pr(\mathbf{x},\mathbf{y}\mid\mathbf{z}) &= \Pr(\mathbf{x}_1,\mathbf{y}_1\mid\mathbf{z}\mathbf{d}) \\ &= \Pr(\mathbf{x}_1\mid\mathbf{z}\mathbf{d})\, \Pr(\mathbf{y}_1\mid\mathbf{z}\mathbf{d}) = \Pr(\mathbf{x}\mid\mathbf{z})\,\Pr(\mathbf{y}\mid\mathbf{z}), \end{aligned} \end{equation*}

the middle equality being Theorem 4.2 applied to the displayed d-separation, since \(\mathbf{z}\mathbf{d}\) is an instantiation of \(\mathrm{det}(\mathbf{Z})\), and the last holding because \(\mathbf{z}\mathbf{d}\) implies \(\mathbf{x}_2\) and \(\mathbf{y}_2\). Hence \(I_{\Pr}(\mathbf{X},\mathbf{Z},\mathbf{Y})\).

Never weaker. Let \(\mathrm{dsep}_G(\mathbf{X},\mathbf{Z},\mathbf{Y})\) hold and let \(p\) run from \(\mathbf{X}\setminus\mathrm{det}(\mathbf{Z})\) to \(\mathbf{Y}\setminus\mathrm{det}(\mathbf{Z})\); being also an \(\mathbf{X}\)-\(\mathbf{Y}\) path, it has a valve closed given \(\mathbf{Z}\).

• (i) Sequential or divergent at \(N \in \mathbf{Z}\): then \(N \in \mathrm{det}(\mathbf{Z})\) and the valve is still closed.
• (ii) Convergent at \(N\), with no descendant of \(N\) in \(\mathbf{Z}\). If \(N \notin \mathrm{det}(\mathbf{Z})\) then no descendant \(M\) of \(N\) is in \(\mathrm{det}(\mathbf{Z})\) either: every node on a directed path from \(N\) to such an \(M\) is a descendant of \(N\), hence outside \(\mathbf{Z}\), hence in \(\mathrm{det}(\mathbf{Z})\setminus\mathbf{Z}\), which drags its parents in and would force \(N \in \mathrm{det}(\mathbf{Z})\). So the valve stays closed. If instead \(N \in \mathrm{det}(\mathbf{Z})\setminus\mathbf{Z}\), all parents of \(N\) lie in \(\mathrm{det}(\mathbf{Z})\), in particular the two neighbours of \(N\) on \(p\); each has an edge into \(N\), so it is not convergent, and it is either an endpoint removed from the modified query or a sequential or divergent valve in \(\mathrm{det}(\mathbf{Z})\), closed.

Either way \(p\) is blocked, so the modified test succeeds whenever the original does.

Strictly stronger. Take \(A \to V \leftarrow B\), \(V \to C\), \(V \to D\) with binary variables, \(\mathbf{W} = \{V\}\) and the deterministic CPT \(V = A \oplus B\), and ask whether \(C\) and \(D\) are independent given \(\{A,B\}\). Plain d-separation says no, the divergent valve \(V\) on \(C \leftarrow V \to D\) being open since \(V \notin \{A,B\}\). The modified test says yes: \(\mathrm{det}(\{A,B\}) = \{A,B,V\}\), both parents of \(V\) being evidence, and deleting the edges out of \(A\), \(B\), \(V\) isolates \(C\) and \(D\), so \(\mathrm{dsep}_G(C,\{A,B,V\},D)\) holds by Theorem 4.1. The independence is real: \(a,b\) pins \(V = a \oplus b\), and \(C\) and \(D\) are independent given \(V\), while marginally \(V\) is an unobserved common cause of both.

Everything follows from one lemma: if \(\mathbf{S}\) is a Markov blanket for \(X\) and \(\mathbf{S} \subseteq \mathbf{C}\) with \(X \notin \mathbf{C}\), then \(\mathbf{C}\) is a Markov blanket too. For splitting the right-hand argument of \(I_{\Pr}(X,\mathbf{S},\mathbf{X}\setminus\mathbf{S}\setminus\{X\})\) disjointly as \((\mathbf{C}\setminus\mathbf{S}) \cup (\mathbf{X}\setminus\mathbf{C}\setminus\{X\})\) and applying weak union (4.7) yields

\begin{equation*} I_{\Pr}\bigl(X,\ \mathbf{C},\ \mathbf{X}\setminus\mathbf{C}\setminus\{X\}\bigr), \end{equation*}

which with \(X \notin \mathbf{C}\) is Definition 4.3 for \(\mathbf{C}\).

Blanket. Write \(\mathbf{B}_t, \mathbf{R}_t\) for the two sets at the start of iteration \(t\), and take as invariant that \(\mathbf{B}_t \cap \mathbf{R}_t = \emptyset\), \(\mathbf{B}_t \cup \mathbf{R}_t \cup \{X\} = \mathbf{X}\), \(X \notin \mathbf{B}_t\) and \(\mathbf{B}_t\) is a Markov blanket for \(X\). It holds initially, by the choice of \(\mathbf{R}\) and the hypothesis on \(\mathbf{B}\). If the Step 2 test succeeds for the chosen \(Y\), then since \(\mathbf{R}_t\cup\{Y\} = \mathbf{X}\setminus(\mathbf{B}_t\setminus\{Y\})\setminus\{X\}\) by the invariant, the test is exactly Definition 4.3 for \(\mathbf{B}_{t+1} = \mathbf{B}_t\setminus\{Y\}\), \(\mathbf{R}_{t+1} = \mathbf{R}_t\cup\{Y\}\), and the set equations survive; if it fails nothing changes. So the set \(\mathbf{B}^{*}\) returned on exit is a Markov blanket. (If \(\mathbf{B}\) empties, \(\emptyset\) is returned and is trivially a boundary.)

Minimality. Suppose some Markov blanket \(\mathbf{S} \subsetneq \mathbf{B}^{*}\) and pick \(Y \in \mathbf{B}^{*}\setminus\mathbf{S}\). Since the loop runs until every variable of \(\mathbf{B}\) has been examined and \(Y\) survives, some iteration \(t\) picked \(Y\) and failed the test. There \(\mathbf{S} \subseteq \mathbf{B}^{*}\setminus\{Y\} \subseteq \mathbf{B}_t\setminus\{Y\}\), the second inclusion because \(\mathbf{B}\) only shrinks, and \(X \notin \mathbf{B}_t\setminus\{Y\}\); so the lemma with \(\mathbf{C} = \mathbf{B}_t\setminus\{Y\}\) gives

\begin{equation*} I_{\Pr}\bigl(X,\ \mathbf{B}_t\setminus\{Y\},\ \mathbf{X}\setminus(\mathbf{B}_t\setminus\{Y\})\setminus\{X\}\bigr) = I_{\Pr}(X,\ \mathbf{B}_t\setminus\{Y\},\ \mathbf{R}_t\cup\{Y\}), \end{equation*}

the identity again by the invariant. But that is the Step 2 test at iteration \(t\), which failed. Hence no proper subset of \(\mathbf{B}^{*}\) is a blanket, and \(\mathbf{B}^{*}\) is a Markov boundary for \(X\). \(\blacksquare\)

Let \(G\) be the complete DAG on the ordering \(X_1, \ldots, X_n\), with \(X_j \to X_i\) for every \(j < i\), so that \(\mathbf{U}_i = \{X_1,\ldots,X_{i-1}\}\) and \(\mathbf{U}_1 = \emptyset\), and take

\begin{equation*} \theta_{x_i \mid \mathbf{u}_i} \;=\; \begin{cases} \Pr(x_i \mid \mathbf{u}_i), & \text{if } \Pr(\mathbf{u}_i) > 0,\\[2pt] 1/|X_i|, & \text{if } \Pr(\mathbf{u}_i) = 0 . \end{cases} \end{equation*}

Every edge increases the index of its head, so \(G\) is acyclic, and the parametrization is legal in the sense of Definition 4.1, each row summing to \(1\): a conditional distribution over \(X_i\) in the first case, uniform in the second (rows over a zero-probability parent instantiation are unconstrained by \(\Pr\)). The parameters compatible with an instantiation \(x_1,\ldots,x_n\) are \(\theta_{x_i \mid x_1,\ldots,x_{i-1}}\), one per family, so the chain rule for Bayesian networks (4.2) gives

\begin{equation*} \Pr{}^{\prime}(x_1, \ldots, x_n) \;=\; \prod_{i=1}^{n} \theta_{x_i \mid x_1, \ldots, x_{i-1}} , \end{equation*}

with \(\Pr(x_1,\ldots,x_0) = 1\) for the empty prefix.

(i) \(\Pr(x_1,\ldots,x_n) > 0\). The prefix events are nested, so every \(\Pr(x_1,\ldots,x_{i-1}) \ge \Pr(x_1,\ldots,x_n) > 0\) and the first branch applies throughout:

\begin{equation*} \begin{aligned} \prod_{i=1}^{n} \theta_{x_i \mid x_1, \ldots, x_{i-1}} &= \prod_{i=1}^{n} \Pr(x_i \mid x_1, \ldots, x_{i-1})\\ &= \prod_{i=1}^{n} \frac{\Pr(x_1, \ldots, x_i)}{\Pr(x_1, \ldots, x_{i-1})} = \Pr(x_1, \ldots, x_n), \end{aligned} \end{equation*}

the product telescoping; this is the chain rule (3.16).

(ii) \(\Pr(x_1,\ldots,x_n) = 0\). Let \(k\) be least with \(\Pr(x_1,\ldots,x_k) = 0\). Then \(\Pr(x_1,\ldots,x_{k-1}) > 0\), so the first branch applies to the \(k\)-th factor, \(\theta_{x_k \mid x_1,\ldots,x_{k-1}} = \Pr(x_1,\ldots,x_k)/\Pr(x_1,\ldots,x_{k-1}) = 0\), and the whole product vanishes.

Hence \(\Pr{}^{\prime} = \Pr\). \(\blacksquare\)

\(G^{\prime}\) is the moral graph of \(G\), so \(B_X\) is the set of parents, children and spouses of \(X\) (a spouse being a node sharing a child with \(X\)):

\begin{equation*} B_X \;=\; \Bigl(\mathrm{pa}(X) \,\cup\, \mathrm{ch}(X) \,\cup \bigcup_{C \in \mathrm{ch}(X)} \mathrm{pa}( C)\Bigr) \setminus \{X\}, \end{equation*}

an edge of \(G^{\prime}\) at \(X\) arising either from step 2, as an edge of \(G\) into or out of \(X\), or from step 1, with \(X\) and the other endpoint distinct parents of a common node. Acyclicity gives \(X \notin B_X\), so \(\{X\}\), \(B_X\), \(Z_X\) are pairwise disjoint and \(\mathrm{dsep}_G(\{X\}, B_X, Z_X)\) is well formed.

Suppose some path \(\alpha : X = W_0 - W_1 - \cdots - W_m = Y\) with \(Y \in Z_X\) is unblocked by \(B_X\), so that every valve on it is open (Definition 4.2).

1. \(W_1\) is adjacent to \(X\), hence a parent or a child of \(X\), hence in \(B_X\); as \(Y \notin B_X\) this gives \(W_1 \ne Y\) and \(m \ge 2\), so \(W_1\) is internal and carries a valve. Being open at a node of the conditioning set, that valve is not sequential or divergent, so it is convergent: \(X \to W_1 \leftarrow W_2\).
2. Path nodes are distinct, so \(W_2 \ne X\) and \(X, W_2\) are distinct parents of \(W_1\); step 1 of the construction therefore puts the edge \(X - W_2\) into \(G^{\prime}\), giving \(W_2 \in B_X\), hence \(W_2 \ne Y\) and \(W_2\) internal. Its edge to \(W_1\) is \(W_2 \to W_1\), outgoing, so its valve is sequential or divergent, and \(W_2 \in B_X\) closes it.

That contradicts \(\alpha\) being unblocked, so every path from \(X\) to a node of \(Z_X\) is blocked by \(B_X\) and \(\mathrm{dsep}_G(\{X\}, B_X, Z_X)\).

We prove both directions in contrapositive form. Write \(\mathrm{An}(\mathbf{S})\) for \(\mathbf{S}\) together with all ancestors of its members; step 1 retains exactly \(\mathbf{A} = \mathrm{An}(\mathbf{X} \cup \mathbf{Y} \cup \mathbf{Z})\), which is ancestrally closed, so a retained node keeps all its parents, step 2 marries exactly the pairs of distinct parents in \(G\) of retained nodes, and step 3 keeps every \(G\)-edge with both endpoints retained. By Definition 4.2 a path is active given \(\mathbf{Z}\) when all its valves are open: a sequential or divergent valve at \(W\) iff \(W \notin \mathbf{Z}\), a convergent one iff \(W \in \mathrm{An}(\mathbf{Z})\). So \(\mathrm{dsep}_G(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\) fails exactly when some \(\mathbf{X}\)-\(\mathbf{Y}\) path is active.

Lemma 1. Every node of an active \(\mathbf{X}\)-\(\mathbf{Y}\) path \(\alpha\) lies in \(\mathbf{A}\), hence survives step 1. The endpoints do. For internal \(W\): if its valve is convergent it is open, so \(W \in \mathrm{An}(\mathbf{Z})\). Otherwise \(W\) has an edge on \(\alpha\) outgoing from it; walk that way. Each node reached is entered along an edge directed into it, so its valve is sequential (and we leave along its outgoing edge) or convergent. Either we reach an endpoint by a directed path from \(W\), giving \(W \in \mathrm{An}(\mathbf{X}\cup\mathbf{Y})\), or we reach by a directed path the first node \(K\) with a convergent valve, which is open, giving \(K \in \mathrm{An}(\mathbf{Z})\) and so \(W \in \mathrm{An}(\mathbf{Z})\). (This is the argument in the book’s proof of Theorem 4.1.) \(\square\)

Lemma 2. Adjacent internal nodes of a path cannot both carry convergent valves, the edge between them having to be directed into each. \(\square\)

Direction 1: d-separation failing in \(G\) means separation failing in \(G^{\prime}\). Let \(\alpha : X = W_0, \ldots, W_m = Y\) be active with \(X \in \mathbf{X}\), \(Y \in \mathbf{Y}\); by Lemma 1 all its nodes are retained. Delete from \(\alpha\) every internal node with a convergent valve, obtaining \(\sigma\). By Lemma 2 the deleted nodes are pairwise non-adjacent on \(\alpha\), so consecutive nodes \(U, V\) of \(\sigma\) are either adjacent on \(\alpha\), whence \(U - V\) is an edge of \(G^{\prime}\) by step 3, or separated by one deleted \(W\) with \(U \to W \leftarrow V\), whence \(U - V\) is an edge of \(G^{\prime}\) by step 2, \(U\) and \(V\) being distinct parents of the retained \(W\). Its nodes being a sublist of those of \(\alpha\), \(\sigma\) is a path of \(G^{\prime}\) from \(X\) to \(Y\), and it avoids \(\mathbf{Z}\): the endpoints lie in \(\mathbf{X} \cup \mathbf{Y}\) and each surviving internal node carries an open sequential or divergent valve.

Direction 2: separation failing in \(G^{\prime}\) means d-separation failing in \(G\). Work with walks (consecutive nodes adjacent, repetitions allowed), valves being defined at internal positions as for paths. Call a walk \(\pi = W_0,\ldots,W_m\) admissible when (i) one endpoint lies in \(\mathbf{X}\) and the other in \(\mathbf{Y}\); (ii) every node lies in \(\mathbf{A}\); (iii) every sequential or divergent valve on \(\pi\) is open. By (iii) a closed valve on an admissible walk is convergent at some \(U \notin \mathrm{An}(\mathbf{Z})\); call it bad.

Step A: from a \(G^{\prime}\)-path to an admissible walk. Let \(\sigma : V_0, \ldots, V_k\) be a \(G^{\prime}\)-path with \(V_0 \in \mathbf{X}\), \(V_k \in \mathbf{Y}\), no \(V_i \in \mathbf{Z}\). Each edge of \(\sigma\) came from step 3, so is an edge of \(G\), or from step 2, so joins distinct parents of some retained \(C\); replace each edge of the second kind by \(V_i \to C \leftarrow V_{i+1}\). The resulting walk \(\pi\) satisfies (i); its nodes are the \(V_i\), which lie in \(\mathbf{A}\) as nodes of \(G^{\prime}\), and the inserted retained \(C\), giving (ii); and each inserted \(C\) carries a convergent valve, so every sequential or divergent valve sits at some \(V_i \notin \mathbf{Z}\), giving (iii).

Step B: an admissible walk with no bad valve exists. Induct on the number \(b\) of bad valves; for \(b > 0\) let \(U = W_i\) carry the leftmost, so \(W_{i-1} \to W_i \leftarrow W_{i+1}\) and the valves at \(1,\ldots,i-1\) are open. Since \(U \notin \mathrm{An}(\mathbf{Z})\) while \(U \in \mathbf{A}\), take a shortest directed path

\begin{equation*} U \to D_1 \to \cdots \to D_t = D , \qquad t \ge 0, \end{equation*}

to a node \(D \in \mathbf{X} \cup \mathbf{Y} \cup \mathbf{Z}\) (\(t = 0\) meaning \(D = U\)). As \(\mathbf{Z} \subseteq \mathrm{An}(\mathbf{Z})\) we get \(D \in \mathbf{X} \cup \mathbf{Y}\), so \(U, D \notin \mathbf{Z}\) by disjointness, minimality puts \(D_1,\ldots,D_{t-1}\) outside \(\mathbf{X} \cup \mathbf{Y} \cup \mathbf{Z}\), and every \(D_l\), an ancestor of \(D\), lies in \(\mathbf{A}\). Consider

\begin{equation*} \begin{aligned} \pi_1 &= W_0, \ldots, W_i, D_1, \ldots, D_t,\\ \pi_2 &= D_t, \ldots, D_1, W_i, W_{i+1}, \ldots, W_m . \end{aligned} \end{equation*}

In \(\pi_1\) the valves at \(W_1,\ldots,W_{i-1}\) are those of \(\pi\), hence open; the valve at \(W_i\) is \(W_{i-1} \to W_i \to D_1\), sequential with \(W_i \notin \mathbf{Z}\); and those at \(D_1,\ldots,D_{t-1}\) are sequential outside \(\mathbf{Z}\). So \(\pi_1\) satisfies (ii), (iii) and has no bad valve. In \(\pi_2\) the valves at \(D_{t-1},\ldots,D_1\) are sequential outside \(\mathbf{Z}\), that at \(W_i\) is \(W_{i+1} \to W_i \to D_1\), sequential and open, and those at \(W_{i+1},\ldots,W_{m-1}\) are the valves of \(\pi\) there; so \(\pi_2\) satisfies (ii), (iii) with exactly \(b-1\) bad valves. (For \(t = 0\), \(W_i\) becomes an endpoint carrying no valve and the checks at \(W_i\) and the \(D_l\) are simply absent.) If \(D\) lies in whichever of \(\mathbf{X}, \mathbf{Y}\) does not contain \(W_0\), then \(\pi_1\) satisfies (i) and we are done; otherwise \(D\) lies in the set not containing \(W_m\), so \(\pi_2\) satisfies (i) and the induction applies. Call an admissible walk with no bad valve active: all its valves are open.

Step C: an active walk yields an active path. Pick an active walk \(\pi = W_0,\ldots,W_m\) of minimum length with one endpoint in \(\mathbf{X}\) and the other in \(\mathbf{Y}\), and suppose \(W_i = W_j\) with \(i < j\). The shorter walk

\begin{equation*} \pi^{*} = W_0, \ldots, W_i, W_{j+1}, \ldots, W_m \end{equation*}

has the same endpoints, and every valve of it other than the one at position \(i\) is a valve of \(\pi\), hence open. If \(i = 0\) or \(j = m\), position \(i\) is an endpoint and no new valve arises. Otherwise let \(e\) join \(W_{i-1}\) to \(W_i\) and \(f\) join \(W_j\) to \(W_{j+1}\).

• (i) \(e\) outgoing from \(W_i\) (or, symmetrically, \(f\) outgoing from \(W_j = W_i\)): then the valve of \(\pi\) at \(i\) (at \(j\)) shares that outgoing edge, so it is sequential or divergent and, being open, gives \(W_i \notin \mathbf{Z}\); the new valve also has an outgoing edge, so it too is sequential or divergent, hence open.
• (ii) \(e\) and \(f\) both into \(W_i\): the new valve is convergent, open iff \(W_i \in \mathrm{An}(\mathbf{Z})\). If the valve of \(\pi\) at \(i\) or at \(j\) is convergent, it is open and gives exactly that. Otherwise both are sequential, \(W_{i-1} \to W_i \to W_{i+1}\) and \(W_{j+1} \to W_j \to W_{j-1}\). Traverse the closed walk \(W_i, W_{i+1}, \ldots, W_j = W_i\): its first edge runs along the traversal and its last against it, so there is a least \(r\) with \(i < r < j\) and \(W_{r+1} \to W_r\). By minimality \(W_i \to W_{i+1} \to \cdots \to W_r\) is directed and the valve of \(\pi\) at \(W_r\) is convergent, hence open, so \(W_r \in \mathrm{An}(\mathbf{Z})\) and \(W_i\), its ancestor, lies in \(\mathrm{An}(\mathbf{Z})\) too.

Either way \(\pi^{*}\) is a shorter active walk with the same endpoints, contradicting minimality. So the minimal active walk is a path: an unblocked \(\mathbf{X}\)-\(\mathbf{Y}\) path, and \(\mathrm{dsep}_G(\mathbf{X}, \mathbf{Z}, \mathbf{Y})\) fails. Chaining Steps A, B, C gives the contrapositive, and with Direction 1,

\begin{equation*} \mathrm{dsep}_G(\mathbf{X}, \mathbf{Z}, \mathbf{Y}) \iff \mathbf{X} \text{ and } \mathbf{Y} \text{ are separated by } \mathbf{Z} \text{ in } G^{\prime} . \end{equation*}

By the definition of \(\mathbf{Z}\),

\begin{equation*} \{X, Y\} \cup \mathbf{Z} \;=\; \mathrm{An}(\{X, Y\}), \qquad \mathbf{Z} \;=\; \mathrm{An}(\{X,Y\}) \setminus \{X, Y\}, \end{equation*}

writing \(\mathrm{An}(\mathbf{S})\) for \(\mathbf{S}\) with all ancestors of its members; the three sets \(\{X\}\), \(\{Y\}\), \(\mathbf{Z}\) are pairwise disjoint, so the statement is well formed, and \(\mathrm{An}(\{X,Y\})\) being ancestrally closed gives \(\mathrm{An}(\mathbf{Z}) \subseteq \{X,Y\} \cup \mathbf{Z}\).

Suppose some path \(\alpha : X = W_0 - W_1 - \cdots - W_m = Y\) is unblocked by \(\mathbf{Z}\). Lemma 1 of Exercise 4.24 places every node of \(\alpha\) in \(\mathrm{An}(\{X,Y\}\cup\mathbf{Z}) = \mathrm{An}(\{X,Y\})\), and path nodes are distinct, so every internal node of \(\alpha\) lies in \(\mathrm{An}(\{X,Y\})\setminus\{X,Y\} = \mathbf{Z}\) and hence carries a convergent valve, a sequential or divergent valve at a node of \(\mathbf{Z}\) being closed. By Lemma 2 of Exercise 4.24 adjacent internal nodes cannot both be convergent, so \(\alpha\) has at most one internal node, and it has at least one since \(X\) and \(Y\) are not joined by an edge. Hence \(m = 2\) and \(\alpha\) is

\begin{equation*} X \to W_1 \leftarrow Y , \end{equation*}

with \(W_1 \in \mathbf{Z}\) an ancestor of \(X\) or of \(Y\); the directed path from \(W_1\) to that node closes a cycle with the edge \(X \to W_1\) or \(Y \to W_1\), against the acyclicity of \(G\). So no path is unblocked and \(\mathrm{dsep}_G(X, \mathbf{Z}, Y)\).

Method (2): in Exercise 4.24 the retained set is \(\mathbf{A} = \{X,Y\}\cup\mathbf{Z}\), so a \(G^{\prime}\)-path from \(X\) to \(Y\) avoiding \(\mathbf{Z}\) has no internal node, i.e. \(X\) and \(Y\) are adjacent in \(G^{\prime}\). That needs \(X\) and \(Y\) adjacent in \(G\), excluded, or both parents of some retained \(C \in \mathbf{Z}\), which is an ancestor of \(X\) or \(Y\) and again closes a directed cycle. So \(\mathbf{Z}\) separates \(X\) from \(Y\) in \(G^{\prime}\), and Exercise 4.24 applies.

Network. Two binary variables: the query variable \(C\) with values \(\text{yes}\) and \(\text{no}\), and the evidence variable \(T\) with values \(+\) and \(-\). Cancer is the only direct cause named, so the structure is the single edge

\begin{equation*} C \longrightarrow T . \end{equation*}

The CPTs are read off the statement, the prior for \(C\) being the prevalence \(c\):

The CPT for \(T\) is the test specification, a false negative being a \(-\) on a cancer patient and a false positive a \(+\) on a healthy one:

Query. The posterior marginal \(\Pr(C = \text{yes} \mid T = +)\), with evidence \(T = +\).

Value. By the chain rule for Bayesian networks \(\Pr(C,T)=\theta_C\,\theta_{T\mid C}\), so

\begin{equation*} \begin{aligned} \Pr(C=\text{yes}\mid T=+) &= \frac{c\,(1-f_n)}{c\,(1-f_n) + (1-c)\,f_p} \\[2pt] &= \frac{(.001)(.60)}{(.001)(.60) + (.999)(.02)} \\[2pt] &= \frac{.0006}{.0006 + .01998} = \frac{.0006}{.02058} = \frac{10}{343} . \end{aligned} \end{equation*}

that is, \(\Pr(C=\text{yes}\mid T=+) \approx 2.92\%\).

Sensitivity analysis. Clearing the denominator, which is positive whenever the evidence has nonzero probability, gives the necessary and sufficient condition

\begin{equation*} \frac{c(1-f_n)}{c(1-f_n)+(1-c)f_p} \ \ge\ \frac{1}{10} \iff 9\,c\,(1-f_n) \ \ge\ (1-c)\,f_p , \end{equation*}

and solving it for one parameter with the other two at their stated values:

• \(f_p\): \(.0054 \ge .999\,f_p\), so \(f_p \le 1/185 \approx .0054\), a fourfold improvement.
• \(c\): \(5.42\,c \ge .02\), so \(c \ge 1/271 \approx .00369\), nearly four times the prevalence.
• \(f_n\): \(.009\,(1-f_n) \ge .01998\), so \(1 - f_n \ge 111/50 = 2.22\), impossible. No value of \(f_n\) works; even \(f_n = 0\) gives only \(.001/(.001+.01998) \approx 4.77\%\).

Network. Let \(T\) have values \(\text{high}\), \(\text{nominal}\), \(\text{low}\) and let \(S_1, S_2, S_3\) be binary with values \(\text{on}\) and \(\text{off}\). The temperature is the only direct cause of each reading and the sensors are independent given it, so the structure is the naive Bayes star

\begin{equation*} S_1 \longleftarrow T \longrightarrow S_2, \qquad T \longrightarrow S_3 , \end{equation*}

that is, \(T\) is a root and each \(S_i\) has \(T\) as its only parent. The CPTs are

and, identically for \(i = 1,2,3\),

(the rows for \(S_i = \text{off}\) are the complements and are omitted as redundant).

For (c) the event “at least one sensor triggered” is a disjunction, the value of no variable of the network, so by the technique of Section 5.3.1 add a binary auxiliary variable \(O\) with parents \(S_1, S_2, S_3\) and the deterministic or-gate CPT

\begin{equation*} \Pr(O=\text{true}\mid s_1,s_2,s_3) = \begin{cases} 1, & \text{if some } s_i = \text{on},\\ 0, & \text{otherwise}, \end{cases} \end{equation*}

turning the disjunctive evidence into the ordinary evidence \(O = \text{true}\).

Write \(\alpha_t = \theta_t\) and \(\sigma_t = \Pr(S_i=\text{on}\mid t)\); the \(S_i\) being independent given \(T\),

\begin{equation*} \Pr(s_1,s_2,s_3) = \sum_{t} \alpha_t \prod_{i} \Pr(s_i \mid t). \end{equation*}

(a) The query is \(\Pr(S_1 = \text{on} \mid S_2=\text{on}, S_3=\text{on})\), and \(S_1\) is not independent of \(S_2, S_3\), the unobserved common parent \(T\) leaving \(S_1 \leftarrow T \rightarrow S_2\) unblocked (Definition 4.2). Two sums are needed:

\begin{equation*} \begin{aligned} \Pr(S_2,S_3 \text{ on}) &= \sum_t \alpha_t \sigma_t^2 \\ &= (.2)(.81) + (.7)(.0025) + (.1)(.0001) \\ &= .162 + .00175 + .00001 \;=\; .16376, \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} \Pr(S_1,S_2,S_3 \text{ on}) &= \sum_t \alpha_t \sigma_t^3 \\ &= (.2)(.729) + (.7)(.000125) + (.1)(.000001) \\ &= .1458 + .0000875 + .0000001 \;=\; .1458876 . \end{aligned} \end{equation*}

Hence

\begin{equation*} \Pr(S_1=\text{on}\mid S_2,S_3 \text{ on}) = \frac{.1458876}{.16376} = \frac{364719}{409400} \approx .8909 . \end{equation*}

Two triggers raise the chance that the third fires from its prior \(\sum_t \alpha_t \sigma_t = .216\) to about \(89.1\%\).

(b) The query is \(\Pr(T = \text{high} \mid S_1,S_2,S_3 \text{ on})\), and the numerator terms are already computed:

\begin{equation*} \Pr(T=\text{high}\mid S_1,S_2,S_3 \text{ on}) = \frac{.1458}{.1458876} = \frac{121500}{121573} \approx .99940 . \end{equation*}

(c) The query is \(\Pr(T=\text{high}\mid O=\text{true})\), and conditional independence of the sensors gives \(\Pr(O = \text{true} \mid t) = 1 - (1-\sigma_t)^3\):

Summing the last column, \(\Pr(O=\text{true}) = .3026076\), so

\begin{equation*} \Pr(T=\text{high}\mid O=\text{true}) = \frac{.1998}{.3026076} = \frac{166500}{252173} \approx .6603 . \end{equation*}

Network. Faults being intermittent, use the dynamic Bayesian network of Figure 5.15(a) over three time slices; slice \(t \in \{1,2,3\}\) holds a full copy of the circuit network,

\begin{equation*} A_t,\ B_t,\ X_t,\ Y_t,\ Z_t,\ C_t,\ D_t,\ E_t , \end{equation*}

with the intra-slice edges listed in the statement, and consecutive slices are linked by the persistence edges \(X_t \to X_{t+1}\), \(Y_t \to Y_{t+1}\), \(Z_t \to Z_{t+1}\). The health CPTs are the prior in slice \(1\) and the persistence model in slices \(2\) and \(3\) (identically for \(Y\) and \(Z\)):

The gate CPTs are the two-valued ones of Section 5.3.5, for example

and analogously \(\Pr(D=\text{high}\mid a,b,Y=\text{ok}) = [\![a \wedge b]\!]\), \(\Pr(D=\text{high}\mid a,b,Y=\text{faulty}) = .5\), and \(\Pr(E=\text{high}\mid c,d,Z=\text{ok}) = [\![c \vee d]\!]\), \(\Pr(E=\text{high}\mid c,d,Z=\text{faulty}) = .5\).

Reduction. Write \(h_t = (x_t,y_t,z_t)\) for the health state in slice \(t\) and \(e_t = (a_t,b_t,\varepsilon_t)\) for the \(t\)-th test vector. The inputs \(A_t, B_t\) are roots d-separated from the health variables, so as in Section 5.3.5 their CPTs cancel and

\begin{equation*} \Pr(h_1,h_2,h_3 \mid e_1,e_2,e_3) \ \propto\ \Pr(h_1)\Pr(h_2\mid h_1)\Pr(h_3\mid h_2)\prod_{t=1}^{3} L_t(h_t), \end{equation*}

where \(L_t(h) = \Pr(\varepsilon_t \mid a_t,b_t,h)\) comes from eliminating \(C_t\) and \(D_t\). Rows below are \((X,Y,Z)\) with \(\text{o} = \text{ok}\), \(\text{f} = \text{faulty}\); columns are the test vectors, \(hhl\) abbreviating \(A = \text{high}, B = \text{high}, E = \text{low}\):

The zero entries check out: under \(hhl\), \(Y\) ok forces \(D = \text{high}\) and then \(Z\) ok forces \(E = \text{high}\) whatever \(C\) is, killing rows (o,o,o) and (f,o,o); under \(lhl\), \(X\) ok gives \(C = \text{high}\) and \(Z\) ok forces \(E = \text{high}\), killing every row with \(X\) and \(Z\) both ok. Summing the \(8^3 = 512\) terms gives normalizing constants \(\Pr(\varepsilon_1,\varepsilon_2,\varepsilon_3\mid a,b) \approx 6.0751\times 10^{-3}\) for (a) and \(\approx 1.3097 \times 10^{-3}\) for (b).

(a) Posterior marginals.

The inverter \(X\) is cleared and the blame falls on \(Y\) (\(\approx 80\%\)) and \(Z\) (\(\approx 21\%\)): only the first vector is abnormal, and the \(Y\) explanation costs nothing on the other two (\(L = 1\) at (o,f,o)) while the \(Z\) explanation pays \(1/2\) on each, the \(4:1\) odds ratio the marginals reflect.

(b) Posterior marginals.

The third vector cannot be explained by a faulty \(Y\) at all, \(X\) ok making \(C\) high and an ok \(Z\) then forcing \(E\) high, so a \(Y\)-based story needs a second fault, whereas one persistent fault in \(Z\) covers both abnormal vectors.

(c) MPE and MAP. MAP is taken over the nine health variables \(X_1,Y_1,Z_1,\dots,X_3,Y_3,Z_3\); MPE is taken over all non-evidence variables, i.e. the health variables together with the internal wires \(C_t, D_t\).

Case (a). The MAP answer is

with \(\Pr(\text{MAP}\mid e) \approx 77.33\%\), a persistent and-gate fault; the runner-up, \(X,Y\) ok and \(Z\) faulty throughout, sits at \(\approx 19.33\%\), a factor \(4\) behind through the \((1/2)^2\) penalty the \(Z\) story pays on the two normal vectors.

The MPE is not a completion of the MAP answer and is not unique: its probability is \(\approx 19.33\%\), with five tied instantiations. One extends the \(Z\)-fault hypothesis,

\begin{equation*} \begin{aligned} &X_t = Y_t = \text{ok},\ Z_t = \text{faulty}\ (t=1,2,3),\\ &(C_1,D_1) = (\text{low},\text{high}),\quad (C_2,D_2) = (C_3,D_3) = (\text{high},\text{low}), \end{aligned} \end{equation*}

and the other four extend the \(Y\)-fault hypothesis,

\begin{equation*} \begin{aligned} &X_t = Z_t = \text{ok},\ Y_t = \text{faulty}\ (t=1,2,3),\\ &(C_1,D_1) = (\text{low},\text{low}),\quad C_2 = C_3 = \text{high}, \end{aligned} \end{equation*}

with \(D_2\) and \(D_3\) free, four equally likely combinations, a faulty \(Y\) making \(D\) a coin flip while \(C = \text{high}\) already forces \(E = \text{high}\). The \(Y\) hypothesis wins the MAP by spreading its mass over four wire completions, each only as likely as the single \(Z\)-fault completion, which is why the \(Z\) hypothesis’s MAP and the MPE coincide at \(19.33\%\).

Case (b). MAP and MPE agree. The MAP answer is

with \(\Pr(\text{MAP}\mid e) \approx 89.68\%\), and the MPE is its unique completion

\begin{equation*} (C_1,D_1) = (\text{low},\text{high}),\qquad (C_2,D_2) = (C_3,D_3) = (\text{high},\text{low}), \end{equation*}

also at \(\approx 89.68\%\) and with no ties. The runner-up is far behind: \(Y\) faulty throughout with \(Z\) also faulty in slice \(3\), at \(\approx 1.81\%\), the double fault the third test vector forces on any \(Y\)-based explanation.

Networks. Let \(T\) have values \(\text{extreme}\) and \(\text{normal}\) with \(\Pr(T=\text{extreme}) = .2\) and let \(S_1, S_2\) be the readings, values \(+\) and \(-\). Each reading has two direct causes, the temperature and the power driving it, and the kits differ only in the number of battery variables.

Kit A (shared battery) has one root \(B\) with \(\Pr(B=\text{on}) = .9\) and structure

\begin{equation*} \begin{aligned} &S_1 \longleftarrow T \longrightarrow S_2,\\ &S_1 \longleftarrow B \longrightarrow S_2 . \end{aligned} \end{equation*}

Kit B (independent batteries) has two roots \(B_1,B_2\), each with \(\Pr(B_i=\text{on}) = .9\), and structure

\begin{equation*} S_1 \longleftarrow T \longrightarrow S_2, \qquad B_1 \to S_1, \qquad B_2 \to S_2 . \end{equation*}

The sensor CPT is the same in both kits, the specifications being identical: a false positive is a \(+\) on a normal temperature, a false negative a \(-\) on an extreme one, and a dead battery forces \(-\):

The queries are the posterior marginals \(\Pr(T=\text{extreme}\mid s_1,s_2)\); for (c) take \(S_1 = +\), \(S_2 = -\), the other ordering giving the same posterior by symmetry.

Kit A. Case on \(B\), the only unobserved root besides \(T\): with \(B\) on the readings are independent given \(T\), with \(B\) off both are \(-\) with probability \(1\).

(a) Both negative. With the battery on a negative costs \(.03\) under an extreme temperature and \(.99\) under a normal one:

\begin{equation*} \begin{aligned} \Pr(-,-,\text{extreme}) &= .2\,[\,.9(.03)^2 + .1\,] = .2(.10081) = .020162,\\ \Pr(-,-,\text{normal}) &= .8\,[\,.9(.99)^2 + .1\,] = .8(.98209) = .785672, \end{aligned} \end{equation*}

so \(\Pr(-,-) = .805834\) and

\begin{equation*} \Pr(T=\text{extreme}\mid -,-) = \frac{.020162}{.805834} = \frac{593}{23701} \approx .0250 . \end{equation*}

(b) Both positive. A \(+\) is impossible with a dead battery, so \(B=\text{off}\) drops out:

\begin{equation*} \begin{aligned} \Pr(+,+,\text{extreme}) &= .2(.9)(.97)^2 = .169362,\\ \Pr(+,+,\text{normal}) &= .8(.9)(.01)^2 = .000072, \end{aligned} \end{equation*}

\begin{equation*} \Pr(T=\text{extreme}\mid +,+) = \frac{.169362}{.169434} = \frac{9409}{9413} \approx .99958 . \end{equation*}

(c) Mixed. The positive reading again forces \(B=\text{on}\):

\begin{equation*} \begin{aligned} \Pr(+,-,\text{extreme}) &= .2(.9)(.97)(.03) = .005238,\\ \Pr(+,-,\text{normal}) &= .8(.9)(.01)(.99) = .007128, \end{aligned} \end{equation*}

\begin{equation*} \Pr(T=\text{extreme}\mid +,-) = \frac{.005238}{.012366} = \frac{97}{229} \approx .4236 . \end{equation*}

Kit B. The roots \(B_1, B_2\) being separate, fold each battery into its own sensor; given \(T\) the readings are independent, with effective per-sensor probabilities

\begin{equation*} \begin{aligned} \Pr(+\mid \text{extreme}) &= .9(.97) = .873, &\Pr(-\mid \text{extreme}) &= .127,\\ \Pr(+\mid \text{normal}) &= .9(.01) = .009, &\Pr(-\mid \text{normal}) &= .991 . \end{aligned} \end{equation*}

(a) Both negative.

\begin{equation*} \begin{aligned} \Pr(-,-,\text{extreme}) &= .2(.127)^2 = .0032258,\\ \Pr(-,-,\text{normal}) &= .8(.991)^2 = .7856648, \end{aligned} \end{equation*}

\begin{equation*} \Pr(T=\text{extreme}\mid -,-) = \frac{.0032258}{.7888906} = \frac{16129}{3944453} \approx .00409 . \end{equation*}

(b) Both positive.

\begin{equation*} \begin{aligned} \Pr(+,+,\text{extreme}) &= .2(.873)^2 = .1524258,\\ \Pr(+,+,\text{normal}) &= .8(.009)^2 = .0000648, \end{aligned} \end{equation*}

\begin{equation*} \Pr(T=\text{extreme}\mid +,+) = \frac{.1524258}{.1524906} = \frac{9409}{9413} \approx .99958 . \end{equation*}

(c) Mixed.

\begin{equation*} \begin{aligned} \Pr(+,-,\text{extreme}) &= .2(.873)(.127) = .0221742,\\ \Pr(+,-,\text{normal}) &= .8(.009)(.991) = .0071352, \end{aligned} \end{equation*}

\begin{equation*} \Pr(T=\text{extreme}\mid +,-) = \frac{.0221742}{.0293094} = \frac{12319}{16283} \approx .7566 . \end{equation*}

Summary.

Case (b) agrees across kits because a positive reading entails a live battery, so both networks reduce to the same naive Bayes model with sensor probabilities \(.97\) and \(.01\), giving \(.2(.97)^2 / [.2(.97)^2 + .8(.01)^2] = 9409/9413\). Under \(-,-\) Kit A keeps about six times as much belief in an extreme temperature, the shared battery supplying one common-cause explanation of both negatives where Kit B would need two failures at \((.1)^2\); under \(+,-\) the comparison reverses, the positive reading proving Kit A’s shared battery live while saying nothing about Kit B’s other battery.

Network. Take five flip variables \(F_1,\dots,F_5\) with values \(h\) and \(t\), where \(F_1,F_2\) are the \(C_1\) flips and \(F_3,F_4,F_5\) the last three, plus an auxiliary variable \(S\) with values \(\text{same}\) and \(\text{different}\) recording the comparison that governs Jack’s decision (the device of Section 5.3.1, turning an event that is the value of no natural variable into one that can be queried). The causal story gives

\begin{equation*} F_1 \to S \leftarrow F_2, \qquad S \to F_3, \quad S \to F_4, \quad S \to F_5 . \end{equation*}

\(F_1\) and \(F_2\) are roots with \(\Pr(F_i = h) = p_1 = .4\); \(S\) is a deterministic child of \(F_1,F_2\) (an equivalence gate),

and each of \(F_3,F_4,F_5\) has \(S\) as its only parent, the value of \(S\) selecting which coin is being flipped:

Query. The posterior marginal \(\Pr(S = \text{same} \mid F_3,F_4,F_5)\), whose prior is

\begin{equation*} \Pr(S=\text{same}) = p_1^2 + (1-p_1)^2 = .16 + .36 = .52 . \end{equation*}

The flips being independent given \(S\), the likelihoods are products of coin parameters; writing \(L_2, L_3\) for the likelihood of the observed triple under \(C_2\) and under \(C_3\),

\begin{equation*} \Pr(S=\text{same}\mid e) = \frac{.52\,L_2}{.52\,L_2 + .48\,L_3}. \end{equation*}

(a) \(t,h,t\).

\begin{equation*} \begin{aligned} L_2 &= (1-p_2)\,p_2\,(1-p_2) = (.4)(.6)(.4) = .096,\\ L_3 &= (1-p_3)\,p_3\,(1-p_3) = (.9)(.1)(.9) = .081 . \end{aligned} \end{equation*}

Hence \(\Pr(e, \text{same}) = (.52)(.096) = .04992\) and \(\Pr(e, \text{different}) = (.48)(.081) = .03888\), so \(\Pr(e) = .0888\) and

\begin{equation*} \Pr(S=\text{same}\mid e) = \frac{.04992}{.0888} = \frac{104}{185} \approx .5622 . \end{equation*}

The evidence is mild: \(L_2/L_3 = 32/27\), so the posterior barely moves from the prior \(.52\).

(b) \(t,t,t\).

\begin{equation*} L_2 = (.4)^3 = .064, \qquad L_3 = (.9)^3 = .729 . \end{equation*}

Hence \(\Pr(e,\text{same}) = (.52)(.064) = .03328\), \(\Pr(e,\text{different}) = (.48)(.729) = .34992\), so \(\Pr(e) = .3832\) and

\begin{equation*} \Pr(S=\text{same}\mid e) = \frac{.03328}{.3832} = \frac{208}{2395} \approx .0868 . \end{equation*}

Here \(L_3/L_2 \approx 11.4\) favours the tails-biased \(C_3\), flipped only when the first two flips differ, so the posterior on “same” drops from \(52\%\) to under \(9\%\).

Network. A dynamic Bayesian network with eight slices: \(S_i\) is the coin in Lisa’s hand at flip \(i\), with values \(\text{fair}\) and \(\text{biased}\), and \(F_i\) the outcome, values \(h\) and \(t\). The structure is the chain

\begin{equation*} S_1 \to S_2 \to \cdots \to S_8, \qquad S_i \to F_i \ \ (i = 1,\dots,8), \end{equation*}

a hidden Markov model. The CPTs encode the three stated facts. Lisa flipped the fair coin initially:

Between consecutive flips she attempts the switch and succeeds \(10\%\) of the time; once the switch has happened it is not undone (she wanted the biased coin):

And the flips are governed by whichever coin is in hand:

Query. The MAP over \(S_1,\dots,S_8\) given the evidence \(F_1,\dots,F_8\). The transition CPT being deterministic in the \(\text{biased}\) row, every state sequence of positive probability is fair up to some point and biased thereafter, so the query reduces to a distribution over the nine hypotheses

\begin{equation*} K = \min\{\, i : S_i = \text{biased} \,\} \in \{2,3,\dots,8\} \cup \{\text{never}\}, \end{equation*}

and \(\Pr(S_8 = \text{biased}\mid e)\) is the probability of a switch at all. The unnormalized weight of \(K = k\) is

\begin{equation*} w_k = (.9)^{k-2}(.1)\cdot (.5)^{k-1}\prod_{i \ge k} \Pr(f_i \mid \text{biased}), \end{equation*}

the factor \((.5)^{k-1}\) being the likelihood of the first \(k-1\) flips under the fair coin; for “never”, \(\Pr(K=\text{never}) = (.9)^{7} = .4782969\) and \(w_{\text{never}} = (.9)^{7}(.5)^{8} = .00186835\).

(a) \(t,t,t,h,h,h,h,h\).

The weights sum to \(\Pr(e) = .011154668\). The MAP answer is \(K = 4\), i.e.

\begin{equation*} S_1 = S_2 = S_3 = \text{fair}, \qquad S_4 = \cdots = S_8 = \text{biased}, \end{equation*}

at \(\approx 29.74\%\), with a switch occurring at all with probability \(1 - .1675 \approx 83.25\%\). The switch lands on the tails-to-heads boundary.

(b) \(t,t,h,h,h,h,h,h\).

The weights sum to \(\Pr(e) = .017544428\). The MAP answer is \(K = 3\),

\begin{equation*} S_1 = S_2 = \text{fair}, \qquad S_3 = \cdots = S_8 = \text{biased}, \end{equation*}

at \(\approx 33.62\%\), with a switch occurring at all with probability \(1 - .1065 \approx 89.35\%\). Again the switch sits at the tails-to-heads boundary, the longer run of heads sharpening both its location and its occurrence.

The system reliability drops to \(\approx 95.42\%\). Network: the only change is to the fans, no longer independent roots. Add a root \(C\) with values \(\text{established}\) and \(\text{not established}\),

and make \(C\) a parent of both fans, giving the edges \(C \to F_1\) and \(C \to F_2\). Every other node and edge of Figure 5.18 is unchanged. The fan CPTs become

which is the statement “given the condition each fan is \(90\%\) reliable; without it, both fans fail”. The fans are now dependent, \(C\) being the only d-separator: \(F_1 \not\perp F_2\) a priori while \(F_1 \perp F_2 \mid C\) (Definition 4.2).

Reliability. The roots \(C, E, P_1, P_2, D\) being independent and all internal nodes deterministic gates, propagate availabilities up the block diagram, eliminating \(C\) before combining the fans:

\begin{equation*} \begin{aligned} \Pr(F_1 \vee F_2) &= \Pr( C)\bigl[1 - (1-.9)^2\bigr] + \Pr(\neg C)\cdot 0 \\ &= (.995)(.99) \;=\; .98505 . \end{aligned} \end{equation*}

Then \(O_1 = E \wedge (F_1 \vee F_2)\), since \(A_1 \vee A_2 = (E \wedge F_1) \vee (E \wedge F_2)\), and \(E\) is independent of \(C, F_1, F_2\):

\begin{equation*} \Pr(O_1 = \text{avail}) = (.99)(.98505) = .9751995 . \end{equation*}

Likewise \(O_2 = O_1 \wedge (P_1 \vee P_2)\) with \(P_1, P_2\) independent roots, so

\begin{equation*} \begin{aligned} \Pr(P_1 \vee P_2) &= 1 - (1-.96)^2 = 1 - .0016 = .9984,\\ \Pr(O_2 = \text{avail}) &= (.9751995)(.9984) = .9736391808 . \end{aligned} \end{equation*}

Finally \(S = O_2 \wedge D\):

\begin{equation*} \Pr(S = \text{avail}) = (.9736391808)(.98) = .954166397184 . \end{equation*}

So the system reliability is \(\approx 95.42\%\), down from the \(\approx 95.90\%\) of Section 5.3.6. The fan block has fallen from \(1 - (.1)^2 = .99\) to \((.995)(.99) = .98505\), and no amount of parallel redundancy recovers it, since \(\Pr(F_1 \vee \cdots \vee F_n) = (.995)[1 - (.1)^n]\) is capped at \(\Pr( C) = .995\).

The network is \(99.96\%\) reliable.

The Bayesian network. The five connections of Figure 5.33 are the only unreliable elements, so introduce one root variable for each,

\begin{equation*} C_{12},\; C_{23},\; C_{34},\; C_{45},\; C_{51}, \end{equation*}

with values \(\mathsf{up}\) and \(\mathsf{down}\) and CPT \(\Pr(C_{ij} = \mathsf{up}) = r_{ij}\); everything else is a deterministic node in the sense of Section 5.4.2, making this a “model from design” as in Section 5.3.6.

Figure 5.33 is cyclic while a Bayesian network must be a DAG, so a single variable “station \(S_i\) is energized” with its neighbours as parents is not available. Unroll the propagation in stages instead, as one unrolls a dynamic system into a DBN (Section 5.3.5); with five stations any station reachable from a source is reachable in at most four hops, so four stages suffice.

For \(k = 0,1,2,3,4\) let \(E_i^k\) be binary, meaning “station \(S_i\) is energized by some conducting route of length at most \(k\)”. Stage \(0\) records the injections, \(I_1\) being tapped onto \(S_5 - S_1\) and \(I_2\) onto \(S_1 - S_2\), so that a conducting tapped connection energizes both its endpoints:

\begin{equation*} \begin{aligned} E_1^0 &\iff C_{51} \vee C_{12}, \\ E_5^0 &\iff C_{51}, \qquad E_2^0 \iff C_{12}, \\ E_3^0 &\iff \bot, \qquad\; E_4^0 \iff \bot . \end{aligned} \end{equation*}

One round of propagation along conducting connections gives the next stage,

\begin{equation*} E_i^{k+1} \iff E_i^{k} \;\vee\; \bigvee_{j \,\sim\, i} \bigl( E_j^{k} \wedge C_{ij} \bigr), \end{equation*}

\(j \sim i\) ranging over the ring neighbours of \(S_i\). So \(E_i^{k+1}\) has parents \(E_i^k\), the two neighbours’ \(E_j^k\) and the two \(C_{ij}\), and every edge runs from stage \(k\) to stage \(k+1\) or from a root into a stage variable: the structure is acyclic. These CPTs are deterministic, written straight from the propositional equivalences in the encoding of Section 5.4.2.

Three more deterministic nodes finish the model. Outlet \(O_1\), tapped onto \(S_4 - S_5\), is fed exactly when that connection conducts and one of its endpoints is energized, and likewise \(O_2\) on \(S_2 - S_3\):

\begin{equation*} \begin{aligned} A_1 &\iff C_{45} \wedge (E_4^4 \vee E_5^4), \\ A_2 &\iff C_{23} \wedge (E_2^4 \vee E_3^4), \\ S &\iff A_1 \vee A_2 . \end{aligned} \end{equation*}

The variable \(S\) says that the network is operational.

The query. The marginal \(\Pr(S = \mathsf{true})\), a probability-of-evidence query with no evidence, exactly as reliability was the marginal of the system node in Section 5.3.6; the MAP over \(C_{12},\dots,C_{51}\) given \(S = \mathsf{false}\) names the connection to blame for an outage.

Reducing the structure function. For this ring \(\Pr(S)\) has a closed form, since

\begin{equation*} S \iff (C_{12} \wedge C_{23}) \;\vee\; (C_{45} \wedge C_{51}), \end{equation*}

making \(S_3 - S_4\) irrelevant. Sufficiency: \(C_{51}\) conducting lets \(I_1\) energize \(S_5\), and then the tap of \(O_1\) on the conducting \(S_4 - S_5\) carries current; symmetrically \(C_{12} \wedge C_{23}\) feeds \(O_2\). Necessity: all electricity originates at the two taps, so the energized stations are those reachable through conducting connections from \(\{S_1,S_5\}\) (available only if \(C_{51}\) conducts) or from \(\{S_1,S_2\}\) (only if \(C_{12}\) conducts). Suppose \(O_1\) is fed, so \(C_{45}\) conducts and one of \(S_4, S_5\) is energized.

• If \(C_{51}\) conducts, \(C_{45} \wedge C_{51}\) holds.
• Otherwise all energy comes through the tap on \(S_1 - S_2\), so \(C_{12}\) conducts and the energized set is what is reachable from \(S_2\) along \(S_2 - S_3 - S_4 - S_5\); reaching \(S_4\) or \(S_5\) then forces \(C_{23}\), giving \(C_{12} \wedge C_{23}\).

The case of \(O_2\) follows by the symmetry of Figure 5.33: \(S_2 \leftrightarrow S_5\), \(S_3 \leftrightarrow S_4\) maps the ring to itself fixing \(S_1\) and \(C_{34}\), exchanges \(I_1\) with \(I_2\) and \(O_1\) with \(O_2\), and swaps \(C_{51}\) with \(C_{12}\) and \(C_{45}\) with \(C_{23}\).

The number. The path sets \(\{C_{12}, C_{23}\}\) and \(\{C_{45}, C_{51}\}\) are disjoint and the connection variables are independent roots, so the two events are independent:

\begin{equation*} \Pr(S) = 1 - (1 - r_{12} r_{23})\,(1 - r_{45} r_{51}). \end{equation*}

With \(r_{ij} = .99\) for every connection, \(r_{12} r_{23} = r_{45} r_{51} = .9801\) and

\begin{equation*} \begin{aligned} \Pr(S) &= 1 - (1 - .9801)^2 = 1 - (.0199)^2 \\ &= 1 - .00039601 = .99960399 . \end{aligned} \end{equation*}

Brute force over the \(2^5\) instantiations of the connection variables in the network above returns exactly \(99960399/100000000\), confirming the reduction.

Replace the or-gate of Figure 5.17 by a threshold gate firing when at least \(k\) of its \(n\) inputs are available, and implement that gate as a saturating counter chain rather than a single CPT of size \(2^{n+1}\). Figure 5.17 is then the case \(k = 1\) and an and-gate the case \(k = n\).

Variables. Let the RBD have blocks \(B_1, \dots, B_m\), and for a block \(C\) let \(\mathrm{pa}( C) = \{D_1, \dots, D_n\}\) be the blocks pointing to \(C\), with threshold \(k\), \(1 \le k \le n\). Each block \(C\) gets two variables:

• a component variable \(C\) with values \(\mathsf{avail}\) and \(\mathsf{unavail}\);
• a subsystem variable \(S_C\) with values \(\mathsf{true}\) and \(\mathsf{false}\), meaning “the subsystem represented by block \(C\) is available”.

The component variables are roots with CPT \(\Pr(C = \mathsf{avail}) = r_C\), as for the roots \(E, F_1, F_2, P_1, P_2, D\) of Figure 5.18.

Threshold gate as a counter chain. For a block \(C\) with parents \(D_1, \dots, D_n\) and threshold \(k\), introduce counting variables

\begin{equation*} N_1^C, N_2^C, \dots, N_n^C, \qquad N_j^C \in \{0, 1, \dots, k\}, \end{equation*}

whose intended meaning is

\begin{equation*} N_j^C \;=\; \min\Bigl(k,\; \bigl|\{\, i \le j : S_{D_i} = \mathsf{true} \,\}\bigr| \Bigr), \end{equation*}

the number of available incoming subsystems among the first \(j\), saturated at \(k\). They are chained: \(N_1^C\) has the single parent \(S_{D_1}\), and \(N_j^C\) for \(j \ge 2\) has parents \(N_{j-1}^C\) and \(S_{D_j}\), with deterministic CPTs

\begin{equation*} \begin{aligned} N_1^C &= \begin{cases} 1, & S_{D_1} = \mathsf{true} \\ 0, & S_{D_1} = \mathsf{false} \end{cases} \\[4pt] N_j^C &= \begin{cases} \min(k,\, N_{j-1}^C + 1), & S_{D_j} = \mathsf{true} \\ N_{j-1}^C, & S_{D_j} = \mathsf{false}. \end{cases} \end{aligned} \end{equation*}

These realize the intended meaning by induction on \(j\): the case \(j = 1\) is immediate, and \(\min(k, c_{j-1}) + 1\) saturates to \(\min(k, c_j)\) because \(\min(k,\cdot)\) is monotone and idempotent above \(k\), while an unavailable parent leaves the count alone. (Check!) Hence at least \(k\) incoming subsystems are available iff \(N_n^C = k\).

Putting a block together. Define \(S_C\) by the deterministic CPT

\begin{equation*} S_C = \mathsf{true} \iff \bigl( C = \mathsf{avail} \bigr) \wedge \bigl( N_n^C = k \bigr), \end{equation*}

so \(S_C\) has parents \(C\) and \(N_n^C\) and a CPT of eight entries. A source block, one with \(n = 0\), carries no threshold condition and gets

\begin{equation*} S_C = \mathsf{true} \iff C = \mathsf{avail}, \end{equation*}

as Figure 5.18 does for the power supply, the only source block of Figure 5.16. Every CPT here except those of the component roots is deterministic, written in the propositional form of Section 5.4.2.

Acyclicity and query. Every edge runs from a component root into its own \(S_C\), from \(S_{D_i}\) into the counter chain of \(C\), or forward along a counter chain, and the RBD is a DAG with \(D_i\) a parent of \(C\), so the construction is a DAG. Its single leaf block \(L\) represents the whole system (Section 5.3.6), so the reliability is

\begin{equation*} \Pr(S_L = \mathsf{true}), \end{equation*}

and the most likely explanation of a failure is the MAP over \(B_1, \dots, B_m\) given \(S_L = \mathsf{false}\), as in Section 5.3.6.

Cost. A single threshold node with parents \(S_{D_1}, \dots, S_{D_n}\) would need \(2^{n+1}\) entries, the blow-up of Section 5.4; the chain uses \(n\) nodes of at most \(2(k+1)^2\) entries each, so \(O(n k^2)\) numbers in all — the micro-model device that turns the noisy-or CPT of (5.2) into the network of Figure 5.30. The extremes check out: at \(k = 1\) the counters are binary and the chain is a cascaded or-gate, recovering Figure 5.17, and at \(k = n\) it is a cascaded and-gate.

(a) The network. A hidden Markov model, the dynamic Bayesian network of Section 5.3.5 with one slice per day. For \(d = 1, \dots, 5\) take

• \(I_d\), with values \(\mathsf{yes}\) and \(\mathsf{no}\): the cow is infected on day \(d\);
• \(T_d\), with values \(+\) and \(-\): the milk test outcome on day \(d\).

The infection state on a day depends only on the previous day’s and the test outcome only on the same day’s, so the edges are

\begin{equation*} I_1 \to I_2 \to I_3 \to I_4 \to I_5, \qquad I_d \to T_d \;\; (d = 1, \dots, 5), \end{equation*}

and nothing else. The CPTs are the same in every slice:

the last table being fixed by the false negative rate \(\Pr(T_d = - \mid I_d = \mathsf{yes}) = 1/1{,}000\) and the false positive rate \(\Pr(T_d = + \mid I_d = \mathsf{no}) = 5/1{,}000\).

The query. The MAP over \(\mathbf{M} = \{I_1, \dots, I_5\}\) given the evidence \(e : T_1 = t_1, \dots, T_5 = t_5\) (Section 5.2.4). Since \(\mathbf{M}\) and \(\mathbf{E} = \{T_1,\dots,T_5\}\) exhaust the network, the remaining set \(\mathbf{Y}\) is empty and MAP coincides exactly with MPE, the degenerate case of Exercise 5.11. As \(\Pr(\mathbf{i}, e)\) factors along the chain,

\begin{equation*} \Pr(i_1,\dots,i_5, e) \;=\; \theta_{i_1}\theta_{t_1 \mid i_1} \prod_{d=2}^{5} \theta_{i_d \mid i_{d-1}} \theta_{t_d \mid i_d}, \end{equation*}

maximize by the left-to-right dynamic program (Viterbi), or enumerate the \(2^5 = 32\) instantiations.

Test vector (1): \(+,+,-,+,+\). The MPE is infection on all five days,

\begin{equation*} I_1 = I_2 = I_3 = I_4 = I_5 = \mathsf{yes}, \end{equation*}

with

\begin{equation*} \begin{aligned} \Pr(\mathbf{i}, e) &= (10^{-4})(.999)^4(.001)(.7)^4 \\ &= 2.39141 \times 10^{-8}, \end{aligned} \end{equation*}

and \(\Pr(e) = 3.197439 \times 10^{-8}\), so \(\Pr(\mathbf{i} \mid e) \approx 74.79\%\). The day-3 negative costs one false negative at \(.001\), against \((.005)^4\) for the competing all-healthy story.

Test vector (2): \(+,-,-,+,+\). The MPE is

\begin{equation*} I_1 = I_2 = I_3 = \mathsf{no}, \qquad I_4 = I_5 = \mathsf{yes}, \end{equation*}

with \(\Pr(\mathbf{i}, e) = 6.912864 \times 10^{-7}\) and \(\Pr(e) = 8.267956 \times 10^{-7}\), hence \(\Pr(\mathbf{i} \mid e) \approx 83.61\%\): the isolated day-1 positive is a false positive, while the two consecutive positives force a new infection from day 4.

Test vector (3): \(+,-,-,-,+\). The MPE is

\begin{equation*} I_1 = I_2 = I_3 = I_4 = I_5 = \mathsf{no}, \end{equation*}

with \(\Pr(\mathbf{i}, e) = 2.460472 \times 10^{-5}\) and \(\Pr(e) = 2.574235 \times 10^{-5}\), hence \(\Pr(\mathbf{i} \mid e) \approx 95.58\%\): two false positives at \((.005)^2 = 2.5 \times 10^{-5}\) are cheaper than an infection that appears, disappears, and produces three false negatives.

(b) Test failures that degrade the next test. The day-\(d\) error rates now depend on whether the day-\((d-1)\) test failed, i.e. disagreed with the true infection state, so the model needs a variable recording that. For \(d = 1, \dots, 4\) add

\begin{equation*} \begin{aligned} &F_d \in \{\mathsf{true}, \mathsf{false}\}, \quad \text{where} \\ &F_d = \mathsf{true} \iff (I_d = \mathsf{yes} \wedge T_d = -) \vee (I_d = \mathsf{no} \wedge T_d = +). \end{aligned} \end{equation*}

So \(F_d\) has parents \(I_d, T_d\) and a deterministic CPT (Section 5.4.2), the new edges being

\begin{equation*} I_d \to F_d, \qquad T_d \to F_d, \qquad F_d \to T_{d+1}, \end{equation*}

so \(T_{d+1}\) has parents \(I_{d+1}\) and \(F_d\) while the chain \(I_1 \to \cdots \to I_5\) is unchanged. The graph stays acyclic, every edge running within slice \(d\) or into slice \(d+1\). The CPT of \(T_1\) is the original emission table; for \(d = 2, \dots, 5\) it is indexed additionally by \(F_{d-1}\), doubling both error rates after a failure:

The query is unchanged, the MAP over \(I_1, \dots, I_5\) given \(T_1, \dots, T_5\), and it can still be read off an MPE: each \(F_d\) is a deterministic function of \(I_d\) and the observed \(T_d\), so every instantiation of the MAP variables and the evidence admits exactly one positive-probability instantiation of \(\mathbf{Y} = \{F_1,\dots,F_4\}\), which is the condition of Exercise 5.11.

The hypothesis makes summing out \(\mathbf{Y}\) the same as maximizing it out: for every \(\mathbf{m}\),

\begin{equation*} \Pr(\mathbf{m}, \mathbf{e}) = \sum_{\mathbf{y}} \Pr(\mathbf{m}, \mathbf{e}, \mathbf{y}) = \max_{\mathbf{y}} \Pr(\mathbf{m}, \mathbf{e}, \mathbf{y}). \tag{\(\ast\)} \end{equation*}

Indeed, if \(\Pr(\mathbf{m}, \mathbf{e}) > 0\) exactly one term of the sum is nonzero, so the sum equals that term, which is the maximum; and if \(\Pr(\mathbf{m}, \mathbf{e}) = 0\) every term vanishes and both sides are \(0\).

Assume \(\Pr(\mathbf{e}) > 0\) (else neither query is defined) and let \(\mathbf{m}^\star \mathbf{y}^\star\) be an MPE given \(\mathbf{e}\), i.e. a maximizer of \(\Pr(\mathbf{m}, \mathbf{y}, \mathbf{e})\) (Section 5.2.3; the conditional and the joint differ by the positive constant \(\Pr(\mathbf{e})\)). For arbitrary \(\mathbf{m}\), apply \((\ast)\), then optimality of the MPE, then \((\ast)\) again:

\begin{equation*} \begin{aligned} \Pr(\mathbf{m}, \mathbf{e}) &= \max_{\mathbf{y}} \Pr(\mathbf{m}, \mathbf{e}, \mathbf{y}) \\ &\le \Pr(\mathbf{m}^\star, \mathbf{e}, \mathbf{y}^\star) \\ &\le \max_{\mathbf{y}} \Pr(\mathbf{m}^\star, \mathbf{e}, \mathbf{y}) = \Pr(\mathbf{m}^\star, \mathbf{e}). \end{aligned} \end{equation*}

So \(\mathbf{m}^\star\) maximizes \(\Pr(\mathbf{m}, \mathbf{e})\), hence also \(\Pr(\mathbf{m} \mid \mathbf{e}) = \Pr(\mathbf{m}, \mathbf{e}) / \Pr(\mathbf{e})\), and is a MAP solution.

The root parameters factor out of numerator and denominator alike and cancel. Write \(\mathbf{X}\) for all variables, \(\mathbf{Z} = \mathbf{X} \setminus (\mathbf{Q} \cup \mathbf{R})\) (disjoint, as \(\mathbf{Q}\) are nonroots), and split the chain rule for Bayesian networks at \(\mathbf{R}\): each \(R \in \mathbf{R}\) is a root, so its factor is \(\theta_r\), depending on \(\mathbf{r}\) alone. With

\begin{equation*} \kappa(\mathbf{r}) = \prod_{R \in \mathbf{R}} \theta_{r}, \qquad f(\mathbf{w}, \mathbf{r}) = \prod_{X \in \mathbf{X} \setminus \mathbf{R}} \theta_{x \mid \mathbf{u}}, \end{equation*}

the latter evaluated at \(\mathbf{w}\mathbf{r}\) and using no parameter of a CPT of \(\mathbf{R}\), we get \(\Pr(\mathbf{w}, \mathbf{r}) = \kappa(\mathbf{r}) f(\mathbf{w}, \mathbf{r})\) for every instantiation \(\mathbf{w}\) of \(\mathbf{X} \setminus \mathbf{R}\). Summing out,

\begin{equation*} \begin{aligned} \Pr(\mathbf{q}, \mathbf{r}) &= \kappa(\mathbf{r}) \sum_{\mathbf{z}} f(\mathbf{q}\mathbf{z}, \mathbf{r}), \\ \Pr(\mathbf{r}) &= \kappa(\mathbf{r}) \sum_{\mathbf{w}} f(\mathbf{w}, \mathbf{r}). \end{aligned} \end{equation*}

Since \(\Pr(\mathbf{r}) \ne 0\), both \(\kappa(\mathbf{r})\) and \(\sum_{\mathbf{w}} f(\mathbf{w}, \mathbf{r})\) are nonzero, so

\begin{equation*} \Pr(\mathbf{q} \mid \mathbf{r}) = \frac{\Pr(\mathbf{q}, \mathbf{r})}{\Pr(\mathbf{r})} = \frac{\sum_{\mathbf{z}} f(\mathbf{q}\mathbf{z}, \mathbf{r})} {\sum_{\mathbf{w}} f(\mathbf{w}, \mathbf{r})} , \end{equation*}

an expression in the parameters \(\theta_{x \mid \mathbf{u}}\), \(X \notin \mathbf{R}\), only. Replacing the CPTs of \(\mathbf{R}\) by any others leaves this expression unchanged, so \(\Pr(\mathbf{q} \mid \mathbf{r})\) is independent of them.

No: the equivalence breaks. It rested on each health variable having a single child, so that it could be bypassed by (5.1) of Section 5.3.4; in Figure 5.15(b) the shared health variable \(X\) has the two children \(C\) and \(C^{\prime}\) (likewise \(Y\), \(Z\)), and the models genuinely differ, because a stuck-at mode forces the same wrong output in both tests whereas a faulty gate with a \(.5/.5\) output CPT reflips independently.

Witness: condition on \(A = A^{\prime} = \mathsf{high}\) and query the inverter’s outputs. Under fault modes \(C = C^{\prime} = \mathsf{high}\) holds exactly when \(X = \mathsf{stuckat1}\) (ok or stuckat0 gives low in both tests), while under ok/faulty it needs \(X = \mathsf{faulty}\) and two high flips:

\begin{equation*} \begin{aligned} \Pr(C = \mathsf{high}, C^{\prime} = \mathsf{high} \mid A = A^{\prime} = \mathsf{high}) &= .005 &&\text{(fault modes)}, \\ &= (.01)(.5)(.5) = .0025 &&\text{(ok/faulty)}. \end{aligned} \end{equation*}

This is a query over \(A, C, A^{\prime}, C^{\prime}\) alone, so the two models are not equivalent with respect to queries over \(A, \dots, E, A^{\prime}, \dots, E^{\prime}\).

Bypassing every health variable turns the two networks into the same network over the circuit variables \(\mathbf{W}\) (primary inputs and wires), which gives the equivalence for every query over \(\mathbf{W}\). Write \(\Pr\), \(\theta\) for the ok/faulty network and \(\Pr^{\prime}\), \(\theta^{\prime}\) for the fault-mode one; the two share the circuit part of the DAG and the primary-input CPTs.

Let \(f\) be the Boolean function of a given component, so that the CPTs of its output are

\begin{equation*} \begin{aligned} \theta_{O = o \mid \mathbf{i},\, \mathsf{ok}} &= \theta^{\prime}_{O = o \mid \mathbf{i},\, \mathsf{ok}} = [\, o = f(\mathbf{i}) \,], \\ \theta^{\prime}_{O = o \mid \mathbf{i},\, \mathsf{stuckat0}} &= [\, o = 0 \,], \\ \theta^{\prime}_{O = o \mid \mathbf{i},\, \mathsf{stuckat1}} &= [\, o = 1 \,], \end{aligned} \end{equation*}

where \([\,\cdot\,]\) is \(1\) when the enclosed statement holds and \(0\) otherwise; the stuck-at CPTs are deterministic because a stuck-at gate ignores its inputs. The exercise’s conditions do not mention \(\mathbf{i}\), so they force \(\theta_{O = o \mid \mathbf{i}, \mathsf{faulty}}\) to be one number \(\theta_{O = o \mid \mathsf{faulty}}\) for all \(\mathbf{i}\).

Each health variable \(H\) is a root whose only child is its own component’s output \(O\), and \(H\) is neither query nor evidence here, so Section 5.3.4 permits bypassing it: delete \(H\) and reset

\begin{equation*} \theta^{\mathrm{new}}_{o \mid \mathbf{i}} = \sum_{h} \theta_{o \mid \mathbf{i}, h} \, \theta_{h} \end{equation*}

by (5.1). The bypass adds no edges and touches no other CPT, so the remaining health variables still have one child each and all of them may be bypassed in turn, in either network; what is left in both cases is the circuit DAG on \(\mathbf{W}\).

The bypassed wire CPTs then coincide. Abbreviate

\begin{equation*} p = \theta_{H = \mathsf{ok}}, \quad s_0 = \theta^{\prime}_{H = \mathsf{stuckat0}}, \quad s_1 = \theta^{\prime}_{H = \mathsf{stuckat1}} . \end{equation*}

Since \(\theta_{H = \mathsf{faulty}} = 1 - p\), \(\theta^{\prime}_{H = \mathsf{ok}} = 1 - s_0 - s_1\), and the stuck-at CPTs are deterministic,

\begin{equation*} \begin{aligned} \theta^{\mathrm{new}}_{o \mid \mathbf{i}} &= [\, o = f(\mathbf{i}) \,]\, p + \theta_{O = o \mid \mathsf{faulty}} (1 - p), \\ {\theta^{\prime}}^{\mathrm{new}}_{o \mid \mathbf{i}} &= [\, o = f(\mathbf{i}) \,]\,\theta^{\prime}_{H = \mathsf{ok}} + [\, o = 0 \,]\, s_0 + [\, o = 1 \,]\, s_1 . \end{aligned} \end{equation*}

The first condition gives \(\theta^{\prime}_{H = \mathsf{ok}} = p\), hence \(1 - p = s_0 + s_1\); the second and third then give \(\theta_{O = o \mid \mathsf{faulty}}(1 - p) = s_o\) for \(o = 0\) and \(o = 1\). (If \(s_0 + s_1 = 0\) those ratios are undefined, but then the component never fails in either model and that identity reads \(0 = 0\).) Substituting,

\begin{equation*} \theta^{\mathrm{new}}_{o \mid \mathbf{i}} = [\, o = f(\mathbf{i}) \,]\, p + s_o = {\theta^{\prime}}^{\mathrm{new}}_{o \mid \mathbf{i}} \end{equation*}

for every \(\mathbf{i}\) and \(o\). So after the bypasses the two networks have the same DAG, the same primary-input CPTs (assumed) and the same wire CPTs: they are the same network. Since bypassing preserves \(\Pr(\mathbf{q}, \mathbf{e})\) over the surviving variables (Section 5.3.4),

\begin{equation*} \Pr(\mathbf{q}, \mathbf{e}) = \Pr^{\prime}(\mathbf{q}, \mathbf{e}) \qquad \text{for all } \mathbf{Q}, \mathbf{E} \subseteq \mathbf{W}, \end{equation*}

for any feedback-free assembly of components, as required.

Both sides factor componentwise, and the identity reduces to one local check per component. Index the components \(c = 1,\dots,n\); component \(c\) has health \(H_c\) (resp. \(H^{\prime}_c\)), inputs \(\mathbf{I}_c\) and output \(O_c\), and \(x\) assigns \(i_c, o_c\) to these wires and \(a\) to each primary input \(A \in \mathbf{R}\). By the chain rule (4.2),

\begin{equation*} \Pr(x, h) \;=\; \pi(x) \prod_{c=1}^{n} \theta_{h_c}\, \theta_{o_c \mid i_c,\, h_c}, \qquad \Pr^{\prime}(x, h^{\prime\prime}) \;=\; \pi(x) \prod_{c=1}^{n} \theta^{\prime}_{h^{\prime\prime}_c}\, \theta^{\prime}_{o_c \mid i_c,\, h^{\prime\prime}_c}, \end{equation*}

with the common primary-input factor \(\pi(x) = \prod_{A \in \mathbf{R}} \theta_a = \prod_{A \in \mathbf{R}} \theta^{\prime}_a\), the two models being assumed to agree there.

Reading off its definition, an instantiation \(h^{\prime\prime}\) of \(\mathbf{H}^{\prime}\) satisfies \(h^{\prime}\) exactly when \(h^{\prime\prime}_c \in \mathcal{V}_c(h)\) for every \(c\), where \(\mathcal{V}_c(h) = \{\text{ok}\}\) if \(h \models H_c = \text{ok}\) and \(\mathcal{V}_c(h) = \{\text{stuckat0}, \text{stuckat1}\}\) if \(h \models H_c = \text{faulty}\). So \(h^{\prime}\) is the disjoint union of the instantiations in the Cartesian product \(\mathcal{V}_1(h) \times \cdots \times \mathcal{V}_n(h)\), and additivity followed by the distributive law gives

\begin{equation*} \Pr^{\prime}(x, h^{\prime}) \;=\; \sum_{h^{\prime\prime} \models h^{\prime}} \Pr^{\prime}(x, h^{\prime\prime}) \;=\; \pi(x) \prod_{c=1}^{n} \Big(\sum_{v \,\in\, \mathcal{V}_c(h)} \theta^{\prime}_{H^{\prime}_c = v}\, \theta^{\prime}_{o_c \mid i_c,\, v}\Big). \end{equation*}

It therefore suffices to prove, componentwise, the local identity

\begin{equation*} \theta_{h_c}\, \theta_{o_c \mid i_c,\, h_c} \;=\; \sum_{v \,\in\, \mathcal{V}_c(h)} \theta^{\prime}_{H^{\prime}_c = v}\, \theta^{\prime}_{o_c \mid i_c,\, v}. \tag{\(\star\)} \end{equation*}

(i) \(h \models H_c = \text{ok}\). The sum is the single term \(\theta^{\prime}_{H^{\prime}_c=\text{ok}}\, \theta^{\prime}_{o_c \mid i_c,\, \text{ok}}\), whose factors are \(\theta_{H_c = \text{ok}}\) by the first condition of Exercise 5.14 and \(\theta_{o_c \mid i_c,\, \text{ok}}\) since both models use the same gate function when healthy.

(ii) \(h \models H_c = \text{faulty}\). Put \(\alpha = \theta^{\prime}_{H^{\prime}_c = \text{stuckat0}}\), \(\beta = \theta^{\prime}_{H^{\prime}_c = \text{stuckat1}}\), so that the first condition gives \(\theta_{H_c = \text{faulty}} = 1 - \theta^{\prime}_{H^{\prime}_c = \text{ok}} = \alpha + \beta\). The stuck-at CPTs are deterministic and input-independent, so exactly one term of the two-term sum survives, namely \(\alpha\) when \(o_c = 0\) and \(\beta\) when \(o_c = 1\); the second and third conditions give the same two values on the left, since \(\theta_{H_c=\text{faulty}}\, \theta_{o_c \mid i_c,\, \text{faulty}}\) is \((\alpha + \beta) \alpha / (\alpha + \beta)\) or \((\alpha + \beta) \beta / (\alpha + \beta)\). (If \(\alpha + \beta = 0\) the ratios are undefined, but then also \(\theta_{H_c=\text{faulty}} = 0\) and both sides of \((\star)\) vanish.) Note this is where the input-independence of \(\theta_{O=0 \mid i, H=\text{faulty}}\) is needed.

Substituting \((\star)\) for every \(c\) yields \(\Pr^{\prime}(x, h^{\prime}) = \Pr(x, h)\) for all \(x\) and \(h\).

The reliability is the prior marginal \(\Pr(R_T = \text{true})\) in the following network, where \(R_X\) means “some directed \(S\)-to-\(X\) path has all links up.” Write \(G = (V,E)\), let \(p_{UX}\) label the edge \(U \to X\), and take the link states mutually independent, as intended.

• For each edge \(U \to X\), a root \(L_{UX}\) with values \(\text{up}\), \(\text{down}\) and \(\theta_{L_{UX} = \text{up}} = p_{UX}\). These are the only stochastic variables.
• \(R_S\), a root with \(\theta_{R_S = \text{true}} = 1\) (the empty path reaches \(S\)).
• For each non-root \(X\) with parents \(U_1, \dots, U_k\) in \(G\), a cascade of deterministic Boolean variables \(O_{X,1}, \dots, O_{X,k}\), and \(R_X := O_{X,k}\):

\begin{equation*} \begin{aligned} O_{X,1} \;&=\; R_{U_1} \wedge (L_{U_1 X} = \text{up}), \\ O_{X,j} \;&=\; O_{X,j-1} \;\vee\; \big( R_{U_j} \wedge (L_{U_j X} = \text{up}) \big), \qquad j = 2, \dots, k. \end{aligned} \end{equation*}

The cascade is the micro-model device of Section 5.4.1: the direct CPT \(R_X = \bigvee_j (R_{U_j} \wedge (L_{U_j X} = \text{up}))\) would have \(2^{2k+1}\) entries, whereas each \(O_{X,j}\) has at most three parents and so at most \(16\). Every node other than \(S\) has a parent (\(S\) is the only root), so no cascade is empty. The network has \(2|E| + 1\) variables and \(O(|E|)\) entries, proportional to \(G\); it is acyclic, since ordering the link roots and \(R_S\) first and then the \(O\)-blocks by a topological order of \(G\), internally by \(j\), is a topological order of it.

Correctness is by induction along that order: the cascade unwinds to \(R_X(\ell) = \bigvee_j (R_{U_j}(\ell) \wedge (\ell_{U_j X} = \text{up}))\) for each joint link state \(\ell\), and an all-up \(S\)-to-\(X\) path is exactly an all-up \(S\)-to-\(U_j\) path followed by an up edge \(U_j \to X\), which by hypothesis holds iff \(R_{U_j}(\ell) = \text{true}\); the base case \(X = S\) is the empty path. Since the \(L_{UX}\) are independent roots and every other variable is a function of \(\ell\),

\begin{equation*} \Pr(R_T = \text{true}) \;=\; \sum_{\ell} [\, R_T(\ell) = \text{true} \,] \prod_{U \to X \,\in\, E} \Pr(\ell_{UX}) \;=\; \mathrm{rel}(S,T). \end{equation*}

Part (a). \(\Pr(E, D = \text{false}) = (.05957, .24093)\) and \(\Pr(E \mid D = \text{false}) \approx (.1982, .8018)\), obtained as follows.

Pruning (Algorithm 8, VE_PR) does nothing here: \(Q \cup \mathbf{E} = \{E, D\}\) contains both leaves \(D\) and \(E\), and the evidence variable \(D\) is childless.

The interaction graph (Definition 6.4) of the five CPTs is

\begin{equation*} G:\quad A - B,\ \ A - C,\ \ B - C,\ \ B - D,\ \ C - D,\ \ C - E , \end{equation*}

with degrees \(2, 3, 4, 2, 1\) for \(A, \dots, E\). Min-degree over the non-query variables, alphabetical tie-break, gives \(\pi = A, B, D, C\):

• \(A\) and \(D\) tie at degree \(2\), take \(A\); \(B, C\) already adjacent, leaving \(B - C,\ B - D,\ C - D,\ C - E\).
• \(B\) and \(D\) tie at degree \(2\), take \(B\); \(C, D\) already adjacent, leaving \(C - D,\ C - E\).
• \(d(D) = 1 < d( C) = 2\), take \(D\), leaving \(C - E\); then \(C\).

The constructed factors are over \(\{B,C\}, \{C,D\}, \{C\}, \{E\}\), so the width is \(2\). Reducing on \(e : D = \text{false}\) (Definition 6.5) touches only \(\Theta_{D \mid BC}\), whose surviving rows are

Eliminating \(A\) from \(\Theta_A, \Theta_{B \mid A}, \Theta_{C \mid A}\) gives \(f_1(B,C) = \sum_{A} \Theta_A\,\Theta_{B \mid A}\,\Theta_{C \mid A}\), e.g. \(f_1(\text{true},\text{true}) = (.6)(.2)(.8) + (.4)(.75)(.1) = .126\):

Eliminating \(B\) from \(f_1\) and \(\Theta_{D \mid BC}^{\,e}\), then \(D\) from the result, then \(C\) against \(\Theta_{E \mid C}\):

\begin{equation*} \begin{aligned} f_2(\text{true}, \text{false}) &= (.126)(.05) + (.394)(.2) = .0851,\\ f_2(\text{false}, \text{false}) &= (.294)(.1) + (.186)(1) = .2154, \qquad f_2(C, \text{true}) = 0,\\ f_3( C) &= \textstyle\sum_{D} f_2(C,D) = (.0851, .2154),\\ f_4(\text{true}) &= (.0851)(.7) + (.2154)(0) = .05957,\\ f_4(\text{false}) &= (.0851)(.3) + (.2154)(1) = .24093 . \end{aligned} \end{equation*}

Thus \(f_4 = \Pr(E, D = \text{false})\), and \(\Pr(D = \text{false}) = .05957 + .24093 = .3005\), so dividing gives \(.05957/.3005 \approx .1982\) and \(.24093/.3005 \approx .8018\).

Part (b). \(\text{pruneNetwork}(\mathcal{N}, \{A\}, e) = A \rightarrow C \rightarrow E\), with CPTs \(\Theta_A\), \(\Theta_{C \mid A}\), \(\Theta_{E \mid C}\).

Node pruning (Theorem 6.4). Delete leaves outside \(Q \cup \mathbf{E} = \{A, E\}\): first \(D\) with \(\Theta_{D \mid BC}\), then the newly exposed leaf \(B\) with \(\Theta_{B \mid A}\). The only leaf left is \(E \in Q \cup \mathbf{E}\) (\(C\) is not a leaf, as \(C \rightarrow E\) remains), so this halts.

Edge pruning (Theorem 6.5). No edge leaves an evidence variable, the sole one being the leaf \(E\), so no edge is deleted and no CPT reduced.

\(\Pr(B, C = \text{true}) = (.1, .45)\), \(\Pr(C = \text{true}) = .55\), and \(\Pr(B \mid C = \text{true}) = (2/11, 9/11) \approx (.1818, .8182)\).

Running VE_PR2 (Algorithm 7), reduction on \(e : C = \text{true}\) (Definition 6.5) touches only \(\Theta_{C \mid B}\), leaving the rows

Eliminating \(\pi = D, C, A\) in turn:

\begin{equation*} \begin{aligned} f_1(B) &= \textstyle\sum_{D} \Theta_{D \mid B} = 1 &&\text{(a CPT sums to \(1\) over its head)},\\ f_2(B) &= \textstyle\sum_{C} \Theta_{C \mid B}^{\,e} = (1.0,\, .5) &&\text{(the likelihood of \(e\) given \(b\))},\\ f_3(\text{true}) &= (.2)(.5) + (.8)(.0) = .1, \qquad f_3(\text{false}) = (.2)(.5) + (.8)(1.0) = .9 . \end{aligned} \end{equation*}

Line 7 returns the product of the surviving factors, all over \(B\):

\begin{equation*} \begin{aligned} \Pr(B = \text{true},\, C = \text{true}) &= (1)(1.0)(.1) = .1,\\ \Pr(B = \text{false},\, C = \text{true}) &= (1)(.5)(.9) = .45, \end{aligned} \end{equation*}

whence \(\Pr(C = \text{true}) = .1 + .45 = .55\) and, by Bayes conditioning, \(.1/.55 = 2/11\) and \(.45/.55 = 9/11\).

\(\Pr(C_2) = (\lambda^2,\, 1 - \lambda^2)\) and \(\Pr(C_n) = (\lambda^n,\, 1 - \lambda^n)\) over \((\text{ok}, \text{faulty})\). The CPTs, completed by the requirement that each row sums to \(1\), are

and, for every \(t > 0\),

There is no evidence, and the query variable is the chain’s only leaf, so nothing is reduced or pruned.

Part (a). Eliminating \(C_0\), then \(C_1\):

\begin{equation*} \begin{aligned} f_1(\text{ok}) &= (1)(\lambda) + (0)(0) = \lambda, &f_1(\text{faulty}) &= (1)(1-\lambda) + (0)(1) = 1 - \lambda,\\ f_2(\text{ok}) &= \lambda \cdot \lambda + (1-\lambda)\cdot 0 = \lambda^2, &f_2(\text{faulty}) &= \lambda(1-\lambda) + (1-\lambda) = 1 - \lambda^2 . \end{aligned} \end{equation*}

Part (b). By induction, eliminating \(C_0, \dots, C_{t-1}\) leaves the single factor \(f_t(\text{ok}) = \lambda^{t}\), \(f_t(\text{faulty}) = 1 - \lambda^{t}\): the case \(t = 1\) is above, and since \(C_t\) occurs only in \(f_t\) and \(\Theta_{C_{t+1} \mid C_t}\),

\begin{equation*} \begin{aligned} f_{t+1}(\text{ok}) &= \lambda^{t}\cdot\lambda + (1-\lambda^{t})\cdot 0 = \lambda^{t+1},\\ f_{t+1}(\text{faulty}) &= \lambda^{t}(1-\lambda) + (1-\lambda^{t})\cdot 1 = 1 - \lambda^{t+1}. \end{aligned} \end{equation*}

At \(t = n\) the order has covered every non-query variable, so \(f_n\) is the marginal \(\Pr(C_n)\).

Bypassing is one step of variable elimination on \(X\), whose output happens to be a legitimate CPT. Write \(\mathbf{Z}\) for the variables of \(\mathcal{N}\) and \(\mathbf{Z}^{\prime} = \mathbf{Z} \setminus \{X\}\) for those of \(\mathcal{N}^{\prime}\).

Well-formedness. \(\mathcal{N}^{\prime}\) is acyclic: all new edges end at \(Y\), so a cycle in \(\mathcal{N}^{\prime}\) could use only one of them, and replacing that \(U \rightarrow Y\) by the path \(U \rightarrow X \rightarrow Y\) of \(\mathcal{N}\) would produce a cycle in \(\mathcal{N}\). The new table is a CPT: its entries are nonnegative and, for every \(\mathbf{u}\mathbf{v}\),

\begin{equation*} \sum_{y} \theta^{\prime}_{y \mid \mathbf{u}\mathbf{v}} = \sum_{x} \theta_{x \mid \mathbf{u}} \sum_{y} \theta_{y \mid x\mathbf{v}} = \sum_{x} \theta_{x \mid \mathbf{u}} \cdot 1 = 1 . \end{equation*}

\(\Pr^{\prime}\) is \(\Pr\) with \(X\) summed out. Since \(Y\) is \(X\)’s only child, every \(W \notin \{X, Y\}\) keeps its family and CPT. Fix \(\mathbf{z}^{\prime}\) and let \(y, \mathbf{u}, \mathbf{v}, \mathbf{p}_W\) be the values it assigns. By the chain rule in \(\mathcal{N}^{\prime}\), then (5.1), then Theorem 6.1 (the product over \(W \ne X, Y\) does not mention \(X\), so it moves inside the sum), then the chain rule in \(\mathcal{N}\):

\begin{equation*} \begin{aligned} \Pr^{\prime}(\mathbf{z}^{\prime}) &= \theta^{\prime}_{y \mid \mathbf{u}\mathbf{v}} \prod_{W \ne X, Y} \theta_{w \mid \mathbf{p}_W} = \Bigl(\sum_{x} \theta_{y \mid x\mathbf{v}}\,\theta_{x \mid \mathbf{u}}\Bigr) \prod_{W \ne X, Y} \theta_{w \mid \mathbf{p}_W}\\ &= \sum_{x} \Bigl( \theta_{x \mid \mathbf{u}}\,\theta_{y \mid x\mathbf{v}} \prod_{W \ne X, Y} \theta_{w \mid \mathbf{p}_W} \Bigr) = \sum_{x} \Pr(\mathbf{z}^{\prime}, x) = \Pr(\mathbf{z}^{\prime}) . \end{aligned} \end{equation*}

Queries. For \(\mathbf{Q} \cup \mathbf{E} \subseteq \mathbf{Z}^{\prime}\), the instantiations \(\mathbf{z}\) of \(\mathbf{Z}\) compatible with \(\mathbf{q}\mathbf{e}\) are exactly the pairs \((\mathbf{z}^{\prime}, x)\) with \(\mathbf{z}^{\prime}\) compatible, so

\begin{equation*} \Pr^{\prime}(\mathbf{q}, \mathbf{e}) = \sum_{\mathbf{z}^{\prime} \sim \mathbf{q}\mathbf{e}} \sum_{x} \Pr(\mathbf{z}^{\prime}, x) = \sum_{\mathbf{z} \sim \mathbf{q}\mathbf{e}} \Pr(\mathbf{z}) = \Pr(\mathbf{q}, \mathbf{e}). \end{equation*}

(a) Width \(1\). (b) Width \(n\).

The interaction graph (Definition 6.4) of \(\Theta_X\) and the \(\Theta_{Y_i \mid X}\) is the star with centre \(X\):

\begin{equation*} G:\quad X - Y_1,\ \ X - Y_2,\ \ \ldots,\ \ X - Y_n, \end{equation*}

with no edge between any \(Y_i, Y_j\). Eliminating a variable builds a factor over its current neighbours, then connects them pairwise and deletes it (Section 6.6).

(a) Each \(Y_i\) has the single neighbour \(X\), so its elimination builds a factor over \(\{X\}\), adds no fill-in edge, and leaves a smaller star; \(X\) is eliminated last from an isolated node, giving a factor over \(\emptyset\). The sizes are \(1, \dots, 1, 0\).

(b) \(X\) has all \(n\) neighbours \(Y_1, \dots, Y_n\), so eliminating it first builds

\begin{equation*} f_1(Y_1, \ldots, Y_n) \;=\; \sum_{X} \Theta_X \prod_{i=1}^{n} \Theta_{Y_i \mid X} \end{equation*}

over \(n\) variables and fills in every pair \(Y_i Y_j\), leaving the complete graph on \(Y_1, \dots, Y_n\). Eliminating from a complete graph keeps it complete, so \(Y_i\) then yields a factor of size \(n - i\); the maximum is \(n\).

The effective treewidth for the query \(\Pr(Y)\) is \(|\mathbf{U}_Y| \le k\), the number of parents of \(Y\), whatever the treewidth of \(\mathcal{N}\) itself.

With \(\mathbf{E} = \emptyset\), pruneEdges deletes nothing, and node pruning (Theorem 6.4) first deletes every leaf \(Y^{\prime} \in \mathbf{Y} \setminus \{Y\}\), then every root of \(\mathbf{X} \setminus \mathbf{U}_Y\), which has just become childless. It halts there: each \(U \in \mathbf{U}_Y\) still has the child \(Y\), and \(Y \in Q\). So \(\text{pruneNetwork}(\mathcal{N}, \{Y\}, \emptyset)\) is \(Y\) with its parents \(\mathbf{U}_Y\).

Its factor \(\Theta_{Y \mid \mathbf{U}_Y}\) mentions all \(|\mathbf{U}_Y| + 1\) surviving variables, so the interaction graph is complete on them, and every elimination order of a complete graph on \(m\) nodes has width \(m - 1\) (the first variable eliminated is adjacent to the other \(m - 1\), and eliminating leaves the graph complete). Hence by Definition 6.6 the effective treewidth is \(|\mathbf{U}_Y|\).

Node pruning iteratively deletes leaves outside \(Q \cup \mathbf{E}\) with their CPTs (Theorem 6.4); edge pruning deletes every edge out of an evidence variable and reduces the child’s CPT at the observed value (Theorem 6.5).

Part (a): \(Q \cup \mathbf{E} = \{A, B, E\}\).

Node pruning.

(\(C\) is never a leaf, as \(C \rightarrow E\) survives.) This leaves \(A \rightarrow B\), \(A \rightarrow C \rightarrow E\) with CPTs \(\Theta_A\), \(\Theta_{B \mid A}\), \(\Theta_{C \mid A}\), \(\Theta_{E \mid C}\).

Edge pruning. The evidence variable \(A\) has the surviving outgoing edges \(A \rightarrow B\) and \(A \rightarrow C\); delete both and set

\begin{equation*} \Theta_B := \Theta_{B \mid A}^{\,A = \text{true}}, \qquad \Theta_C := \Theta_{C \mid A}^{\,A = \text{true}}, \end{equation*}

each a one-variable table. The pruned network is \(A\) and \(B\) isolated together with \(C \rightarrow E\), with CPTs \(\Theta_A\), \(\Theta_B\), \(\Theta_C\), \(\Theta_{E \mid C}\); a further node-pruning round removes nothing.

Part (b): \(Q \cup \mathbf{E} = \{A, B, F\}\).

Node pruning.

This leaves the chain \(A \rightarrow B \rightarrow D \rightarrow F\) with CPTs \(\Theta_A\), \(\Theta_{B \mid A}\), \(\Theta_{D \mid B}\), \(\Theta_{F \mid D}\).

Edge pruning. \(F\) is now childless, so only \(B \rightarrow D\) goes; replace \(\Theta_{D \mid B}\) by \(\Theta_D := \Theta_{D \mid B}^{\,B = \text{true}}\). The pruned network is the two disconnected edges \(A \rightarrow B\) and \(D \rightarrow F\), with CPTs \(\Theta_A\), \(\Theta_{B \mid A}\), \(\Theta_D\), \(\Theta_{F \mid D}\); a further node-pruning round removes nothing, both leaves being in \(\mathbf{E}\).

Width \(2\). The interaction graph of the CPTs (Definition 6.4) is the moral graph, since the CPT of \(X\) is a factor over \(\{X\} \cup \mathbf{U}\) and a factor’s variables form a clique; only \(H\) has two parents, so moralization adds just \(F - G\):

\begin{equation*} A-B,\ A-C,\ B-D,\ C-E,\ D-F,\ E-G,\ F-H,\ G-H,\ F-G. \end{equation*}

Running OrderWidth (Algorithm 4), recording the degree \(d\) of the eliminated variable and connecting its neighbours pairwise:

The fill-in edge simply walks the cut down the two chains – after step 4 the graph is \(E-F, E-G, F-G, F-H, G-H\), so \(E\) needs no new edge and what is left is the triangle \(F, G, H\). Hence \(\max_i d_i = 2\).

Min-degree with alphabetical tie-breaking returns \(\pi = A, B, C, D, E, F, G, H\), of width \(2\).

Algorithm 5 runs on the moral graph, which adds only \(F - G\) (the sole node with two parents is \(H\)):

\begin{equation*} A-B,\ A-C,\ B-D,\ C-E,\ D-F,\ E-G,\ F-H,\ G-H,\ F-G, \end{equation*}

with degrees

Every node but \(F, G\) starts at degree \(2\), so the first pick is \(A\); eliminating it (neighbours \(B, C\), fill-in \(B-C\)) leaves \(B\) at degree \(2\), and the same step repeats along the chain: \(B\) (fill-in \(C-D\)), \(C\) (\(D-E\)), \(D\) (\(E-F\)), each time restoring a degree-\(2\) alphabetically-first candidate while \(F, G\) stay at degree \(3\). Then \(E\) has neighbours \(F, G\) with \(F-G\) already present, leaving the triangle \(F, G, H\), from which \(F\), \(G\), \(H\) are eliminated at degrees \(2, 1, 0\). (Check!)

The degrees encountered are \(2,2,2,2,2,2,1,0\).

The treewidth is \(2\).

Upper bound. Exercise 6.8 exhibits the complete order \(A, B, C, D, E, F, G, H\) of width \(2\), and treewidth is the minimum width over complete orders.

Lower bound. Widths are measured on the moral graph, in which every family is a clique; here \(H\) has parents \(F, G\), so \(\{F, G, H\}\) is a triangle (\(F-H\), \(G-H\) from the DAG, \(F-G\) from moralization). Elimination never breaks a clique, since its only structural effects are adding fill-in edges and deleting the eliminated variable. So for any complete order \(\pi\), whichever of \(F, G, H\) comes first is still adjacent to the other two when eliminated, giving it degree \(\ge 2\) and \(\pi\) width \(\ge 2\). (This is the case \(k = 2\) of Exercise 6.15.)

Reverse any topological order \(X_1, \dots, X_n\) of the network: \(\pi = X_n, X_{n-1}, \ldots, X_1\), i.e. peel leaves inward toward the roots. Depth-first search produces a topological order in \(O(n + m)\) time, and \(m \le n\) in a tree network, so this is linear in the network structure.

Its width is \(1\). No node has two parents, so moralization adds nothing and the interaction graph \(G_0\) (Definition 6.4) is the undirected version of the DAG, a forest. When OrderWidth (Algorithm 4) reaches \(X_i\), the current graph is \(G_0\) restricted to \(\{X_1, \dots, X_i\}\) with no fill-in ever added: the neighbours of \(X_i\) in \(G_0\) are its unique parent and its children, and every child follows \(X_i\) topologically, hence precedes it in \(\pi\) and is already gone. So \(X_i\) has at most its parent as a neighbour, connecting a set of size \(\le 1\) adds no edge, and deleting \(X_i\) leaves \(G_0\) on \(\{X_1, \dots, X_{i-1}\}\) – the same statement one step down. Hence \(d \le 1\) at every step, with \(d = 1\) at any node having a parent, one of which exists since the network has at least one edge.

Call \(\text{VE\\_PR}(\mathcal N, \{X\}, \mathbf e)\) once per variable \(X = v_k\), supplying on Line 2 the order

\begin{equation*} \pi_X \;=\; v_1, \ldots, v_{k-1}, v_{k+1}, \ldots, v_n \end{equation*}

restricted to the variables surviving the pruning on Line 1; Line 2 admits any order of the variables outside \(\mathbf Q\), so each call is correct and returns \(Pr(X, \mathbf e)\). Total cost \(O(n^2 \exp(w))\) time and \(O(n \exp(w))\) space, where \(w\) is the width of \(\pi\).

The cost follows from two facts about widths, both measured on interaction graphs (Definition 6.4), and both proved by running the elimination twice in parallel on graphs \(G_i\) (for the full order) and \(H_i\) (for the restricted one).

(i) Deleting one variable from an order costs at most \(1\). The runs of \(\pi\) and \(\pi_X\) on the same graph satisfy the invariant that every edge of \(H_i\) avoiding \(X\) is an edge of \(G_i\): the step \(v_i = X\) only adds edges on the \(G\) side, and at a step \(v_i \ne X\) a fill-in edge \(\{Y,Z\}\) of \(H_{i+1}\) has \(Y, Z\) adjacent to \(v_i\) in \(H_i\), hence in \(G_i\) by hypothesis, so the same edge appears in \(G_{i+1}\). Applying the invariant to the pairs \(\{v_i, Y\}\), the neighbours of \(v_i\) in \(H_i\) are among those in \(G_i\) plus possibly \(X\), so \(|N_{H_i}(v_i)| \le |N_{G_i}(v_i)| + 1 \le w + 1\).

(ii) Restricting to a subgraph never hurts. For \(G^{\prime}\) a subgraph of \(G\) on \(S \subseteq V(G)\) and \(\sigma\) the restriction of \(\tau\) to \(S\), the same induction with the invariant \(E(H_i) \subseteq E(G_i)\) shows \(\sigma\) has width at most that of \(\tau\): steps eliminating a variable outside \(S\) only add edges on the \(G\) side, and at a step \(v_i \in S\) the \(H\)-neighbours of \(v_i\) are \(G\)-neighbours, so the recorded degree is no larger and every \(H\)-fill-in is a \(G\)-fill-in.

Since \(\text{pruneNetwork}\) only deletes nodes and edges and shrinks CPTs, the moral graph of the pruned network is such a subgraph, so by (i) and (ii) each call runs on an order of width at most \(w + 1\). With \(|\mathbf Q| = 1\), Theorem 6.2 bounds each call by \(O(n \exp(w+1))\), pruning costing only \(O(n \exp(w))\) more; over \(n\) variables this is \(O(n^2 \exp(w+1)) = O(n^2 \exp(w))\) for bounded variable cardinalities. The \(n\) calls are independent, so they reuse the same \(O(n \exp(w))\) space, retaining only the \(n\) single-variable answers.

Both sides are factors over \(\mathbf Z = \mathbf X \cup \mathbf Y\) (Definition 6.3 for the product, and reduction preserves a factor’s variables), so it suffices to compare their values at each instantiation \(\mathbf z\). Fix \(\mathbf z\) and let \(\mathbf x, \mathbf y\) be its restrictions to \(\mathbf X, \mathbf Y\), so that Definition 6.3 gives \((f_i f_j)(\mathbf z) = f_i(\mathbf x) f_j(\mathbf y)\) and \((f_i^{\mathbf e} f_j^{\mathbf e})(\mathbf z) = f_i^{\mathbf e}(\mathbf x) f_j^{\mathbf e}(\mathbf y)\).

(i) \(\mathbf z \sim \mathbf e\). Any variable shared by \(\mathbf X\) and \(\mathbf e\) is shared by \(\mathbf Z\) and \(\mathbf e\), where \(\mathbf x\) copies \(\mathbf z\), so \(\mathbf x \sim \mathbf e\), and likewise \(\mathbf y \sim \mathbf e\). Definition 6.5 then leaves all three values unreduced:

\begin{equation*} (f_i f_j)^{\mathbf e}(\mathbf z) = f_i(\mathbf x) f_j(\mathbf y) = f_i^{\mathbf e}(\mathbf x) \, f_j^{\mathbf e}(\mathbf y) . \end{equation*}

(ii) \(\mathbf z \not\sim \mathbf e\). The left side is \(0\). Some variable \(V\) of \(\mathbf Z\) is instantiated differently by \(\mathbf z\) and \(\mathbf e\), and \(V\) lies in \(\mathbf X\) or \(\mathbf Y\); in the first case \(\mathbf x\) inherits that value, so \(\mathbf x \not\sim \mathbf e\) and \(f_i^{\mathbf e}(\mathbf x) = 0\), in the second \(f_j^{\mathbf e}(\mathbf y) = 0\). Either way the right side vanishes too.

Following the hint, \(Pr\) and \(Pr^{\prime}\) agree on every row consistent with \(\mathbf e\), and the two marginals sum over the same such rows.

Write \(\mathbf U_X\) for the parents of \(X\) in \(\mathcal N\), split as \(\mathbf U_X^{\mathbf E} = \mathbf U_X \cap \mathbf E\) and \(\mathbf U_X^{-} = \mathbf U_X \setminus \mathbf E\), and let \(\mathbf e_X\) be the restriction of \(\mathbf e\) to \(\mathbf U_X^{\mathbf E}\). Pruning deletes exactly the edges out of evidence variables, so \(\mathcal N^{\prime}\) has the same nodes, \(X\) has parents \(\mathbf U_X^{-}\), and deleting edges preserves acyclicity. Its new CPT is \(\theta^{\prime}_{x \mid \mathbf u^{-}} = \theta_{x \mid \mathbf u^{-} \mathbf e_X}\), a legitimate CPT since \(\mathbf u^{-} \mathbf e_X\) is a full instantiation of \(\mathbf U_X\) and so \(\sum_{x} \theta_{x \mid \mathbf u^{-} \mathbf e_X} = 1\).

Let \(\mathbf z\) be an instantiation of all variables with \(\mathbf z \sim \mathbf e\), i.e. extending \(\mathbf e\), and let \(x, \mathbf u_X, \mathbf u_X^{-}\) be the values it assigns. It gives the evidence parents exactly \(\mathbf e_X\), so \(\mathbf u_X = \mathbf u_X^{-} \mathbf e_X\) and, by the chain rule in each network,

\begin{equation*} Pr^{\prime}(\mathbf z) = \prod_{X \in \mathbf Z} \theta^{\prime}_{x \mid \mathbf u_X^{-}} = \prod_{X \in \mathbf Z} \theta_{x \mid \mathbf u_X} = Pr(\mathbf z) . \end{equation*}

Now fix \(\mathbf q\). If \(\mathbf q \not\sim \mathbf e\) both marginals are \(0\). Otherwise both are sums over the same instantiations \(\mathbf z\) extending \(\mathbf q \mathbf e\),

\begin{equation*} Pr(\mathbf q, \mathbf e) = \sum_{\mathbf z \,\sim\, \mathbf q \mathbf e} Pr(\mathbf z) = \sum_{\mathbf z \,\sim\, \mathbf q \mathbf e} Pr^{\prime}(\mathbf z) = Pr^{\prime}(\mathbf q, \mathbf e) , \end{equation*}

each such \(\mathbf z\) extending \(\mathbf e\) and so covered by the previous display.

Let \(X\) have parents \(U_1,\ldots,U_k\), put \(C = \{X, U_1, \ldots, U_k\}\), and let \(\pi\) be any order over all network variables.

\(C\) is a clique of the initial interaction graph \(G_1\): the CPT \(\Theta_{X|U_1\ldots U_k}\) is a factor over exactly \(C\), and by Definition 6.4 variables sharing a factor are pairwise adjacent. Elimination never destroys an edge between two surviving variables, since it only adds fill-in edges and removes edges incident on the eliminated variable (Section 6.6).

Let \(i\) be least with \(\pi(i) \in C\); it exists since \(\pi\) covers every variable. Steps \(1, \dots, i-1\) all eliminate variables outside \(C\), so the other \(k\) members of \(C\) survive into \(G_i\) and remain adjacent to \(\pi(i)\). Hence \(\pi(i)\) has at least \(k\) neighbours when eliminated, so the \(d\) recorded by OrderWidth at iteration \(i\) is at least \(k\) and

\begin{equation*} w \;=\; \max_j d_j \;\ge\; k . \end{equation*}

\begin{equation*} \Pr(c \mid a_1,\ldots,a_m)#

\frac{\theta_c \prod_{i=1}^{m} \theta_{a_i|c}} {\theta_c \prod_{i=1}^{m} \theta_{a_i|c} \;+\; \theta_{\bar c} \prod_{i=1}^{m} \theta_{a_i|\bar c}} , \end{equation*}

where \(c, \bar c\) are the values of \(C\). Run VE_PR with \(Q = \{C\}\), \(e : A_1 = a_1, \ldots, A_m = a_m\) and order \(\pi = A_1, \ldots, A_m\). Line 3 reduces each CPT (Definition 6.5), leaving \(\Theta_C\) untouched and giving

\begin{equation*} \Theta^{e}_{A_i \mid C}(\alpha, \gamma) = \begin{cases} \theta_{a_i \mid \gamma}, & \alpha = a_i, \\ 0, & \alpha \neq a_i, \end{cases} \qquad \gamma \in \{c, \bar c\}, \end{equation*}

which is legitimate because reduction distributes over the product of CPTs (Theorem 6.3). Each \(A_i\) occurs in the single factor \(\Theta^{e}_{A_i \mid C}\), so its elimination needs no multiplication and yields \(f_i(\gamma) = \theta_{a_i \mid \gamma}\), a factor over \(C\) alone (width \(1\), so the run costs \(O(m)\)). Line 9 multiplies the survivors \(\Theta_C, f_1, \ldots, f_m\):

\begin{equation*} \Pr(C, e)( c) \;=\; \theta_c \prod_{i=1}^{m} \theta_{a_i|c}, \qquad \Pr(C, e)(\bar c) \;=\; \theta_{\bar c} \prod_{i=1}^{m} \theta_{a_i|\bar c}, \end{equation*}

and dividing by \(\Pr(e) = \Pr(C,e)( c) + \Pr(C,e)(\bar c)\) gives the displayed form.

Both parts turn on one fact: with \(X\) set to false, the row \(\theta_{\bar x \mid \mathbf{u}}\) of an OR-like CPT factorizes into unary functions of the parents, and a unary factor on \(U_i\) is exactly what evidence on \(U_i\) contributes. Write \(\mathbf{z}\) for an instantiation of \(\mathcal{N}\)’s variables (the exercise’s \(\mathbf{x}\)), \(\mathbf{u}\) for its restriction to the parents, \(\bar x\) for \(X = \text{false}\), and \(R(\mathbf{z}) = \prod_{V \neq X} \theta_{z_V \mid \mathbf{z}_{\mathrm{Pa}(V)}}\) for the chain-rule product over the other CPTs, which nothing below touches.

(a) Logical OR. Take \(\mathcal{N}^{\prime}\) with the edges \(U_i \to X\) deleted and \(\Theta_{X \mid U_1 \ldots U_n}\) replaced by the root CPT \(\theta^{\prime}_{\bar x} = 1\), \(\theta^{\prime}_{x} = 0\), and take \(e^{\prime} = e \cup \{U_1 = \text{false}, \ldots, U_n = \text{false}\}\); deleting edges preserves acyclicity. Since \(X\) is false iff every parent is,

\begin{equation*} \theta_{\bar x \mid \mathbf{u}} = \prod_{i=1}^{n} \big[\, U_i = \text{false in } \mathbf{u} \,\big], \end{equation*}

with \([\cdot]\) the \(0/1\) indicator. If \(\mathbf{z}\) is inconsistent with \(e\) both sides vanish, \(e^{\prime}\) extending \(e\); otherwise \(z_X = \bar x\) and

\begin{equation*} \Pr(\mathbf{z}, e) = \theta_{\bar x \mid \mathbf{u}} \, R(\mathbf{z}) = R(\mathbf{z}) \prod_{i=1}^{n} \big[\, z_{U_i} = \text{false} \,\big] = \Pr^{\prime}(\mathbf{z}, e^{\prime}), \end{equation*}

since \(\Pr^{\prime}(\mathbf{z}) = R(\mathbf{z})\) and \(\mathbf{z} \sim e^{\prime}\) iff every \(z_{U_i}\) is false. (If \(e\) already sets some \(U_i\) true then \(\Pr(e) = 0\) and the claim is trivial.)

(b) Noisy OR. By (5.2) of Section 5.4.1,

\begin{equation*} \theta_{\bar x \mid \mathbf{u}} = (1-\theta_l) \prod_{i=1}^{n} g_i(z_{U_i}), \qquad g_i(\text{true}) = \theta_{q_i}, \quad g_i(\text{false}) = 1 , \end{equation*}

and the \(g_i\) are no longer \(0/1\), so they are likelihoods rather than hard evidence; realize them as virtual evidence (Section 3.6.4). Let \(\mathcal{N}^{\prime}\) delete the edges \(U_i \to X\), give the now-root \(X\) the CPT \(\theta^{\prime}_{\bar x} = 1 - \theta_l\), \(\theta^{\prime}_{x} = \theta_l\), and attach to each \(U_i\) a fresh binary child \(V_i\) with

\begin{equation*} \theta^{\prime}_{v_i \mid u_i} = \theta_{q_i}, \qquad \theta^{\prime}_{v_i \mid \bar u_i} = 1 , \end{equation*}

the complementary rows being \(1-\theta_{q_i}\) and \(0\); set \(e^{\prime} = e \cup \{V_1 = v_1, \ldots, V_n = v_n\}\). Both new tables are CPTs (rows nonnegative and summing to \(1\)). Since \(e^{\prime}\) fixes every \(V_i\), each \(\mathbf{z}\) extends to exactly one instantiation of \(\mathcal{N}^{\prime}\) consistent with \(e^{\prime}\). For \(\mathbf{z}\) consistent with \(e\), so that \(z_X = \bar x\), the chain rule over the enlarged variable set gives

\begin{equation*} \begin{aligned} \Pr^{\prime}(\mathbf{z}, e^{\prime}) &= \theta^{\prime}_{\bar x} \cdot R(\mathbf{z}) \cdot \prod_{i=1}^{n} \theta^{\prime}_{v_i \mid z_{U_i}} = (1-\theta_l) \, R(\mathbf{z}) \prod_{i=1}^{n} g_i(z_{U_i}) = \Pr(\mathbf{z}, e), \end{aligned} \end{equation*}

because \(\theta^{\prime}_{v_i \mid z_{U_i}}\) is \(\theta_{q_i}\) when \(z_{U_i}\) is true and \(1\) when it is false, i.e. \(g_i(z_{U_i})\); and both sides are \(0\) when \(\mathbf{z} \not\sim e\).

The gap is the off-diagonal mass, which is nonnegative because factors are nonnegative (Definition 6.1). Fix \(\mathbf{u}\) with restrictions \(\mathbf{y}, \mathbf{z}\) to \(\mathbf{Y}, \mathbf{Z}\). By Definitions 6.2 and 6.3 the left side decouples the two sums,

\begin{equation*} \left(\left(\sum_X f_1\right)\left(\sum_X f_2\right)\right)(\mathbf{u}) = \sum_{x} \sum_{x^{\prime}} f_1(x, \mathbf{y}) \, f_2(x^{\prime}, \mathbf{z}), \end{equation*}

whereas \(\left(\sum_X f_1 f_2\right)(\mathbf{u}) = \sum_{x} f_1(x, \mathbf{y}) f_2(x, \mathbf{z})\) is its diagonal, so the difference is \(\sum_{x \neq x^{\prime}} f_1(x, \mathbf{y}) f_2(x^{\prime}, \mathbf{z}) \ge 0\).

Mini-buckets. For nonnegative factors, \(f \ge \hat f\) pointwise implies \(fg \ge \hat f g\) and \(\sum_W f \ge \sum_W \hat f\), so replacing a factor set by one whose product dominates pointwise makes the final result of VE_PR dominate, by induction on the remaining eliminations. Fix a bound \(b\) on the variables allowed in a constructed factor. At iteration \(i\), partition the factors mentioning \(\pi(i)\) into groups \(P_1, \ldots, P_r\) with

\begin{equation*} \Big| \bigcup_{f \in P_j} \mathrm{vars}(f) \Big| \;\le\; b+1 , \end{equation*}

and replace them all by the \(r\) factors \(\hat f_{i,j} = \sum_{\pi(i)} \prod_{f \in P_j} f\) rather than by the exact \(f_i\). Applying the inequality \(r - 1\) times, each application decoupling one more copy of the summation variable,

\begin{equation*} \prod_{j=1}^{r} \hat f_{i,j} \;\ge\; \sum_{\pi(i)} \ \prod_{j=1}^{r} \prod_{f \in P_j} f \;=\; f_i , \end{equation*}

so by monotonicity Line 9 returns \(\widehat{\Pr}(\mathbf{q}, e) \ge \Pr(\mathbf{q}, e)\) for every \(\mathbf{q}\), and \(\mathbf{Q} = \emptyset\) bounds \(\Pr(e)\). Every factor built has at most \(b\) variables, so the cost is \(O(n \exp(b))\) rather than \(O(n \exp(w))\), with \(b \ge w\) recovering exact VE_PR. The guarantee is on the joint \(\Pr(\mathbf{Q}, e)\), not on the conditional: normalizing divides two upper bounds.

A worked instance. Take \(X, Y, Z\) binary with

Summing out \(X\) gives \((\sum_X f_1) = (0.6, 0.8)\) and \((\sum_X f_2) = (0.9, 0.8)\), so

and each gap is the off-diagonal sum, e.g. \(0.2 \cdot 0.6 + 0.4 \cdot 0.3 = 0.24\) at \(y = z = 0\).

Write \(f_A = \Theta_A\), \(f_B = \Theta_B\), \(f_C = \Theta_{C \mid AB}\), \(f_D = \Theta_{D \mid C}\), \(f_E = \Theta_{E \mid C}\), \(f_F = \Theta_{F \mid D}\), \(f_G = \Theta_{G \mid E}\). Summing \(f_A f_B f_C\) over \(B\) once gives the joint over \(A, C\) that all three parts use:

\begin{equation*} \begin{aligned} \Pr(A=t, C=t) &= .6\,(.5\cdot .9 + .5\cdot .1) = .30,\\ \Pr(A=t, C=f) &= .6\,(.5\cdot .1 + .5\cdot .9) = .30,\\ \Pr(A=f, C=t) &= .4\,(.5\cdot .5 + .5\cdot .3) = .16,\\ \Pr(A=f, C=f) &= .4\,(.5\cdot .5 + .5\cdot .7) = .24 , \end{aligned} \end{equation*}

so \(\Pr(C=t) = .46\) and \(\Pr(C=f) = .54\).

Part 1. With \(\mathbf{Q} = \{B, C\}\) and no evidence, node pruning deletes the leaves \(F, G\) and then \(D, E\), leaving \(A \to C \leftarrow B\), so \(\Pr(B, C) = \sum_{A} f_A f_B f_C\):

\begin{equation*} \begin{aligned} \Pr(B=t, C=t) &= .5\,(.6 \cdot .9 + .4 \cdot .5) = .37,\\ \Pr(B=t, C=f) &= .5\,(.6 \cdot .1 + .4 \cdot .5) = .13,\\ \Pr(B=f, C=t) &= .5\,(.6 \cdot .1 + .4 \cdot .3) = .09,\\ \Pr(B=f, C=f) &= .5\,(.6 \cdot .9 + .4 \cdot .7) = .41 . \end{aligned} \end{equation*}

Part 2. With \(\mathbf{Q} = \{C\}\), \(\mathbf{e} : D = \text{true}\), node pruning deletes \(F, G\) and then \(E\), and edge pruning removes nothing (\(D\) is childless), leaving \(A \to C \leftarrow B\) with \(C \to D\). Hence \(\Pr(C, D=t) = \Pr( C)\,\Theta_{D=t \mid C}\):

\begin{equation*} \Pr(C=t, D=t) = .46 \cdot .2 = .092, \qquad \Pr(C=f, D=t) = .54 \cdot .7 = .378 . \end{equation*}

Part 3. With \(\mathbf{Q} = \{A\}\), \(\mathbf{e} : D = E = \text{true}\), pruning deletes only \(F, G\). Put \(g( C) = \Theta_{D=t \mid C}\,\Theta_{E=t \mid C}\), so \(g(t) = .02\) and \(g(f) = .14\); summing out \(C\) against the joint above,

\begin{equation*} \begin{aligned} \Pr(A=t, D=t, E=t) &= .30 \cdot .02 + .30 \cdot .14 = .048,\\ \Pr(A=f, D=t, E=t) &= .16 \cdot .02 + .24 \cdot .14 = .0368 , \end{aligned} \end{equation*}

so \(\Pr(D=t, E=t) = .0848\) and dividing gives \(.048/.0848 = 30/53\), \(.0368/.0848 = 23/53\).

Take \(T\) to be the path

\begin{equation*} 3 - 2 - 1 - 4 - 5 - 6 - 7 - 8 \end{equation*}

with the one-to-one assignment of CPTs to nodes tabulated below; its width is \(2\), the smallest attainable, since \(\Theta_{E \mid BC}\) must sit alone at some node \(i\), forcing \(\{B,C,E\} = \text{vars}(i) \subseteq \mathbf{C}_i\).

This is an elimination tree by Definition 7.2 (\(T\) is a tree and each of the eight CPTs is assigned to exactly one node). Since \(T\) is a path, each edge splits it into an initial and a final segment, so Definition 7.3 gives

\begin{equation*} \begin{aligned} \mathbf{S}_{32} &= \{A\} \cap \{A,B,C,D,E,F,G,H\} = \{A\},\\ \mathbf{S}_{21} &= \{A,B\} \cap \{A,B,C,D,E,F,G,H\} = \{A,B\},\\ \mathbf{S}_{14} &= \{A,B,C\} \cap \{B,C,D,E,F,G,H\} = \{B,C\},\\ \mathbf{S}_{45} &= \{A,B,C,E\} \cap \{B,D,E,F,G,H\} = \{B,E\},\\ \mathbf{S}_{56} &= \{A,B,C,D,E\} \cap \{D,E,F,G,H\} = \{D,E\},\\ \mathbf{S}_{67} &= \{A,B,C,D,E,F\} \cap \{E,F,G,H\} = \{E,F\},\\ \mathbf{S}_{78} &= \{A,B,C,D,E,F,G\} \cap \{F,G,H\} = \{F,G\}. \end{aligned} \end{equation*}

and Definition 7.4, \(\mathbf{C}_i = \text{vars}(i) \cup \bigcup_j \mathbf{S}_{ij}\), gives

\begin{equation*} \begin{aligned} \mathbf{C}_3 &= \{A\} \cup \{A\} = \{A\},\\ \mathbf{C}_2 &= \{A,B\} \cup \{A\} \cup \{A,B\} = \{A,B\},\\ \mathbf{C}_1 &= \{A,C\} \cup \{A,B\} \cup \{B,C\} = \{A,B,C\},\\ \mathbf{C}_4 &= \{B,C,E\} \cup \{B,C\} \cup \{B,E\} = \{B,C,E\},\\ \mathbf{C}_5 &= \{B,D\} \cup \{B,E\} \cup \{D,E\} = \{B,D,E\},\\ \mathbf{C}_6 &= \{D,E,F\} \cup \{D,E\} \cup \{E,F\} = \{D,E,F\},\\ \mathbf{C}_7 &= \{E,G\} \cup \{E,F\} \cup \{F,G\} = \{E,F,G\},\\ \mathbf{C}_8 &= \{F,G,H\} \cup \{F,G\} = \{F,G,H\}. \end{aligned} \end{equation*}

The largest cluster has three variables, so the width is \(3 - 1 = 2\), matching the lower bound above.

Part 1. Width \(2\). With \(\text{vars}(1) = \{A\}\), \(\text{vars}(2) = \{B\}\), \(\text{vars}(3) = \{A,B,C\}\), \(\text{vars}(4) = \{C,D\}\), \(\text{vars}(5) = \{C,E\}\), \(\text{vars}(6) = \{D,F\}\), \(\text{vars}(7) = \{E,G\}\) (each indicator \(\lambda_X\) sits at a node already containing \(X\), so it adds no variable), Definition 7.3 gives

\begin{equation*} \begin{aligned} \mathbf{S}_{13} &= \{A\} \cap \{A,B,C,D,E,F,G\} = \{A\},\\ \mathbf{S}_{23} &= \{B\} \cap \{A,B,C,D,E,F,G\} = \{B\},\\ \mathbf{S}_{34} &= \{A,B,C,E,G\} \cap \{C,D,F\} = \{C\},\\ \mathbf{S}_{35} &= \{A,B,C,D,F\} \cap \{C,E,G\} = \{C\},\\ \mathbf{S}_{46} &= \{A,B,C,D,E,G\} \cap \{D,F\} = \{D\},\\ \mathbf{S}_{57} &= \{A,B,C,D,E,F\} \cap \{E,G\} = \{E\}. \end{aligned} \end{equation*}

and Definition 7.4, \(\mathbf{C}_i = \text{vars}(i) \cup \bigcup_j \mathbf{S}_{ij}\), gives

\begin{equation*} \begin{aligned} \mathbf{C}_1 &= \{A\}, & \mathbf{C}_2 &= \{B\}, & \mathbf{C}_3 &= \{A,B,C\},\\ \mathbf{C}_4 &= \{C,D\}, & \mathbf{C}_5 &= \{C,E\}, & \mathbf{C}_6 &= \{D,F\},\\ \mathbf{C}_7 &= \{E,G\}. & & & & \end{aligned} \end{equation*}

The largest cluster is \(\mathbf{C}_3\), of size three, so the width is \(3 - 1 = 2\).

Part 2. Messages come from Equation 7.2, \(M_{ij} = \text{project}(\phi_i \prod_{k \ne j} M_{ki}, \mathbf{S}_{ij})\), with \(\phi_i\) including node \(i\)’s indicators (a \(\lambda_X\) for \(X\) outside the current evidence is the constant \(1\)).

Query 1: \(\Pr(G = \text{true})\), evidence \(G = \text{true}\), root \(7\). The messages toward \(7\) are

\begin{equation*} \begin{aligned} M_{13}(A) &= f_A \lambda_A = (.6, .4),\\ M_{23}(B) &= f_B \lambda_B = (.5, .5),\\ M_{64}(D) &= \textstyle\sum_F f_F \lambda_F = (1, 1),\\ M_{43}( C) &= \textstyle\sum_D f_D \lambda_D M_{64} = (1,1),\\ M_{35}( C) &= \textstyle\sum_{A,B} f_C \lambda_C M_{13} M_{23} M_{43},\\ M_{57}(E) &= \textstyle\sum_C f_E \lambda_E M_{35}, \end{aligned} \end{equation*}

over \(A\), \(B\), \(D\), \(C\), \(C\), \(E\) respectively, matching the separators of Part 1. Numerically \(M_{35} = \Pr( C)\):

\begin{equation*} \begin{aligned} M_{35}(C=t) &= .6 (.5 \cdot .9 + .5 \cdot .1) + .4 (.5 \cdot .5 + .5 \cdot .3)\\ &= .30 + .16 = .46,\\ M_{35}(C=f) &= .6 (.5 \cdot .1 + .5 \cdot .9) + .4 (.5 \cdot .5 + .5 \cdot .7)\\ &= .30 + .24 = .54, \end{aligned} \end{equation*}

and then

\begin{equation*} \begin{aligned} M_{57}(E=t) &= .46 \cdot .1 + .54 \cdot .2 = .154,\\ M_{57}(E=f) &= .46 \cdot .9 + .54 \cdot .8 = .846 . \end{aligned} \end{equation*}

Query 2: \(\Pr(G, F = \text{true})\), evidence \(F = \text{true}\), root \(7\). The same six messages serve; \(M_{13}, M_{23}\) are reused (Part 3) and the other four recomputed with \(\lambda_F\) the \(F = \text{true}\) indicator and \(\lambda_G \equiv 1\):

\begin{equation*} \begin{aligned} M_{64}(D) &= \Theta_{F=t \mid D} = (.7,\ .6),\\ M_{43}(C=t) &= .2 \cdot .7 + .8 \cdot .6 = .62,\\ M_{43}(C=f) &= .7 \cdot .7 + .3 \cdot .6 = .67,\\ M_{35}(C=t) &= .46 \cdot .62 = .2852,\\ M_{35}(C=f) &= .54 \cdot .67 = .3618,\\ M_{57}(E=t) &= .1 \cdot .2852 + .2 \cdot .3618 = .10088,\\ M_{57}(E=f) &= .9 \cdot .2852 + .8 \cdot .3618 = .54612 . \end{aligned} \end{equation*}

Query 3: evidence \(A = \text{true}, F = \text{true}\), root \(6\). Only \(M_{23}\) survives (Part 3), so the messages toward \(6\) are

\begin{equation*} \begin{aligned} M_{13}(A) &= f_A \lambda_A = (.6,\ 0),\\ M_{75}(E) &= \textstyle\sum_G f_G \lambda_G = (1, 1),\\ M_{53}( C) &= \textstyle\sum_E f_E \lambda_E M_{75} = (1, 1),\\ M_{34}( C) &= \textstyle\sum_{A,B} f_C \lambda_C M_{13} M_{23} M_{53} = \Pr(C, A = t),\\ M_{46}(D) &= \textstyle\sum_C f_D \lambda_D M_{34} . \end{aligned} \end{equation*}

Numerically,

\begin{equation*} \begin{aligned} M_{34}(C=t) &= .6\,(.5 \cdot .9 + .5 \cdot .1) = .30,\\ M_{34}(C=f) &= .6\,(.5 \cdot .1 + .5 \cdot .9) = .30,\\ M_{46}(D=t) &= .2 \cdot .30 + .7 \cdot .30 = .27,\\ M_{46}(D=f) &= .8 \cdot .30 + .3 \cdot .30 = .33 . \end{aligned} \end{equation*}

Part 3. A change of evidence at a node invalidates every message directed away from it (the invalidation rule of Algorithm 12, Figure 7.10).

(i) Query 1 to Query 2: evidence changes at node \(6\) (\(\lambda_F\)) and node \(7\) (\(\lambda_G\)). Away from \(6\) lie \(M_{64}, M_{43}, M_{35}, M_{57}\) (plus \(M_{31}, M_{32}, M_{34}\), never computed); away from \(7\) lie \(M_{75}, M_{53}, M_{31}, M_{32}, M_{34}, M_{46}\), none computed. So exactly \(M_{13}, M_{23}\) survive.

(ii) Query 2 to Query 3: only \(\lambda_A\) changes, at node \(1\). Away from \(1\) lie \(M_{13}, M_{32}, M_{34}, M_{35}, M_{46}, M_{57}\), which invalidates the stored \(M_{13}, M_{35}, M_{57}\). \(M_{23}, M_{64}, M_{43}\) stay valid, but with root \(6\) only \(M_{23}\) points the right way; \(M_{64}\) and \(M_{43}\) point at node \(3\), while Query 3 needs their reverses \(M_{34}, M_{46}\).

Part 4. Line 9 of Algorithm 12 returns \(\Pr(\mathbf{C}_i, \mathbf{e}) = \phi_i \prod_k M_{ki}\) at the root, where all incoming messages are available.

Query 1. At node \(7\), \(f_G \lambda_G M_{57}\) vanishes for \(G = \text{false}\) and otherwise

\begin{equation*} \begin{aligned} \Pr(E=t, G=t) &= .2 \cdot .154 = .0308,\\ \Pr(E=f, G=t) &= .8 \cdot .846 = .6768 . \end{aligned} \end{equation*}

Summing out \(E\),

\begin{equation*} \Pr(G = \text{true}) = .0308 + .6768 = .7076 . \end{equation*}

Query 2. At node \(7\), \(f_G M_{57}\) gives

\begin{equation*} \begin{aligned} \Pr(E=t, G=t, F=t) &= .2 \cdot .10088 = .020176,\\ \Pr(E=f, G=t, F=t) &= .8 \cdot .54612 = .436896,\\ \Pr(E=t, G=f, F=t) &= .8 \cdot .10088 = .080704,\\ \Pr(E=f, G=f, F=t) &= .2 \cdot .54612 = .109224 . \end{aligned} \end{equation*}

Summing out \(E\),

\begin{equation*} \Pr(G = \text{true}, F = \text{true}) = .457072, \qquad \Pr(G = \text{false}, F = \text{true}) = .189928 . \end{equation*}

Query 3. At node \(6\), \(f_F \lambda_F M_{46}\) vanishes for \(F = \text{false}\) and otherwise

\begin{equation*} \begin{aligned} \Pr(D=t, F=t, A=t) &= .7 \cdot .27 = .189,\\ \Pr(D=f, F=t, A=t) &= .6 \cdot .33 = .198 . \end{aligned} \end{equation*}

Summing out \(D\),

\begin{equation*} \Pr(F = \text{true}, A = \text{true}, F = \text{true}) = .387, \qquad \Pr(F = \text{false}, A = \text{true}, F = \text{true}) = 0 . \end{equation*}

The printed third query lists \(F\) both as query variable and as evidence, so its literal answer is degenerate; on the intended reading \(\Pr(F, A = \text{true})\) the same messages apply with \(\lambda_F \equiv 1\) at node \(6\), giving

\begin{equation*} \begin{aligned} \Pr(F = \text{true}, A = \text{true}) &= .7 \cdot .27 + .6 \cdot .33 = .387,\\ \Pr(F = \text{false}, A = \text{true}) &= .3 \cdot .27 + .4 \cdot .33 = .213 . \end{aligned} \end{equation*}

It suffices to prove that at each elimination step \(\text{vars}(\phi^{\ast}_i) = \mathbf{C}_i\) and \(\mathbf{V} = \mathbf{C}_i \setminus \mathbf{S}_{ij}\), since then summing out \(\mathbf{V}\) is summing out \(\text{vars}(\phi^{\ast}_i) \setminus \mathbf{S}_{ij}\), which is Equation 7.1.

Write \(T^{\prime}\) for the current tree and \(\phi^{\ast}_k\) for the current factor at node \(k\), and let \(D_k\) be the neighbors of \(k\) in \(T\) already eliminated; separators, clusters and the sets \(\text{vars}(i,j)\) always refer to the original \(T\). Line 3 defines \(\mathbf{V} = \text{vars}(\phi^{\ast}_i) \setminus \text{vars}(T^{\prime} - i)\). Induct on the number of steps, carrying the invariant

(a) \(\text{vars}(\phi^{\ast}_k) = \text{vars}(k) \cup \bigcup_{m \in D_k} \mathbf{S}_{km}\) for every \(k \in T^{\prime}\),

true initially since \(D_k = \emptyset\) and \(\text{vars}(\phi_k) = \text{vars}(k)\) (Definition 7.2). Let FE2 remove \(i\) with single remaining neighbor \(j\). Since Line 2 removes only leaves of \(T^{\prime}\), \(T^{\prime}\) is always a connected subtree of \(T\), so every neighbor of \(i\) other than \(j\) has been eliminated, i.e. \(D_i\) is all of them; (a) and Exercise 7.5 then give

\begin{equation*} \text{vars}(\phi^{\ast}_i) = \text{vars}(i) \cup \bigcup_{m \ne j} \mathbf{S}_{im} = \mathbf{C}_i . \end{equation*}

It remains to show \(\mathbf{C}_i \cap \text{vars}(T^{\prime} - i) = \mathbf{S}_{ij}\). Note first that \(T^{\prime} - i\) is connected, avoids \(i\), and contains \(j\), so it lies on the \(j\)-side of the edge \(i-j\).

(i) Left to right. Let \(X \in \mathbf{C}_i \cap \text{vars}(T^{\prime}-i)\); then \(X \in \text{vars}(i,j)\) by Exercise 7.7. Also \(X \in \text{vars}(\phi^{\ast}_k)\) for some remaining \(k \ne i\), so by (a) either \(X \in \text{vars}(k)\) or \(X \in \mathbf{S}_{km} \subseteq \text{vars}(m,k)\) for some \(m \in D_k\). Both \(k\) and \(m\) lie on the \(j\)-side (\(m\) is a neighbor of \(k\) other than the uneliminated \(i\)), hence so does the whole \(m\)-side of \(k-m\); thus \(X \in \text{vars}(j,i)\) and \(X \in \mathbf{S}_{ij}\).

(ii) Right to left. Let \(X \in \mathbf{S}_{ij}\), so \(X \in \mathbf{C}_i\) by Definition 7.4 and \(X \in \text{vars}(p)\) for some \(p\) on the \(j\)-side. If \(p \in T^{\prime}\) then \(X \in \text{vars}(\phi^{\ast}_p)\) by (a) and we are done. Otherwise let \(k\) be the first node of \(T^{\prime}\) on the path from \(p\) to \(i\) (it exists as \(j \in T^{\prime}\), and \(k \ne i\) as \(j\) precedes \(i\)), and let \(m\) be its predecessor, so \(m \in D_k\) and \(X \in \text{vars}(p) \subseteq \text{vars}(m,k)\). Since \(X \in \text{vars}(i,j)\) there is \(s\) on the \(i\)-side with \(X \in \text{vars}(s)\), and the path \(s, \dots, i, j, \dots, k, m\) places \(s\) on the \(k\)-side of \(k-m\), so \(X \in \text{vars}(k,m)\). Hence \(X \in \mathbf{S}_{km} \subseteq \text{vars}(\phi^{\ast}_k) \subseteq \text{vars}(T^{\prime}-i)\).

Therefore \(\mathbf{V} = \mathbf{C}_i \setminus \mathbf{S}_{ij}\) and

\begin{equation*} \sum_{\mathbf{V}} \phi^{\ast}_i = \sum_{\text{vars}(\phi^{\ast}_i) \setminus \mathbf{S}_{ij}} \phi^{\ast}_i = \text{project}(\phi^{\ast}_i, \mathbf{S}_{ij}), \end{equation*}

which is Equation 7.1. Line 4 then multiplies this factor over \(\mathbf{S}_{ij} = \mathbf{S}_{ji}\) into \(\phi^{\ast}_j\) and changes nothing else, so (a) holds for \(j\) with the enlarged set \(D_j \cup \{i\}\) and the induction closes.

Both claims follow from two facts about the tree, with \(j\) ranging over the neighbors of \(i\) other than \(k\) and \(\mathbf{S}_{ij} = \text{vars}(i,j) \cap \text{vars}(j,i)\) as in Definition 7.3.

Fact 1. Deleting \(i-k\) leaves the component of \(i\), which is \(\{i\}\) together with the subtrees hanging off \(i\) through its other neighbors, so

\begin{equation*} \text{vars}(i,k) = \text{vars}(i) \cup \bigcup_{j \ne k} \text{vars}(j,i). \end{equation*}

Fact 2. Every node on the \(k\)-side of \(i-k\) reaches \(j\) by a path through \(k\) and then \(i\), so deleting \(i-j\) leaves it in the component of \(i\); hence \(\text{vars}(k,i) \subseteq \text{vars}(i,j)\).

Let \(X \in \mathbf{S}_{ik}\), so \(X \in \text{vars}(i,k)\) and \(X \in \text{vars}(k,i)\). By Fact 1 either \(X \in \text{vars}(i)\), or \(X \in \text{vars}(j,i)\) for some \(j \ne k\), in which case Fact 2 gives \(X \in \text{vars}(k,i) \subseteq \text{vars}(i,j)\) and hence \(X \in \mathbf{S}_{ij}\). Either way \(X\) lies in \(\text{vars}(i) \cup \bigcup_{j \ne k} \mathbf{S}_{ij}\), which is the first claim. (If \(k\) is the only neighbor of \(i\), the second alternative cannot arise and \(\mathbf{S}_{ik} \subseteq \text{vars}(i,k) = \text{vars}(i)\).)

Definition 7.4 then gives

\begin{equation*} \mathbf{C}_i = \Big( \text{vars}(i) \cup \bigcup_{j \ne k} \mathbf{S}_{ij} \Big) \cup \mathbf{S}_{ik} = \text{vars}(i) \cup \bigcup_{j \ne k} \mathbf{S}_{ij}, \end{equation*}

the trailing term being absorbed by the first claim.

Part 1 is an induction on the number of nodes on the \(i\)-side of \(i-j\), with \(\phi_{ij}\) the product of all factors (CPTs and indicators, by Lines 1 to 5 of Algorithm 12) assigned there, so that \(\text{vars}(\phi_{ij}) = \text{vars}(i,j)\). Throughout, a factor pulls out of any summation over variables it does not mention, and summations over distinct variables commute.

Base case: the \(i\)-side is \(\{i\}\), so \(i\) is a leaf, the product in Equation 7.2 is empty and \(\phi_{ij} = \phi_i\).

Inductive step: let \(k_1, \dots, k_n\) be the neighbors of \(i\) other than \(j\). Deleting \(i\) splits the \(i\)-side into \(\{i\}\) and the pairwise disjoint \(k_t\)-sides of the edges \(k_t - i\) (Fact 1 of Exercise 7.5), so

\begin{equation*} \phi_{ij} = \phi_i \prod_{t=1}^{n} \phi_{k_t i}, \qquad \text{vars}(\phi_{ij}) = \text{vars}(i) \cup \bigcup_{t} \text{vars}(k_t, i), \end{equation*}

and each \(k_t\)-side is smaller, so \(M_{k_t i} = \text{project}(\phi_{k_t i}, \mathbf{S}_{k_t i})\) by hypothesis. Put \(\mathbf{U}_t = \text{vars}(k_t, i) \setminus \mathbf{S}_{k_t i}\). A variable of \(\mathbf{U}_t\) lies in \(\text{vars}(k_t,i)\) but not \(\text{vars}(i,k_t)\), hence occurs in no factor outside the \(k_t\)-side; so the \(\mathbf{U}_t\) are pairwise disjoint and disjoint from \(\text{vars}(i)\), from every \(\mathbf{S}_{k_s i}\), and from \(\mathbf{S}_{ij} \subseteq \text{vars}(j,i)\). They are therefore among the variables that \(\text{project}(\phi_{ij}, \mathbf{S}_{ij})\) sums out, and may be summed out first:

\begin{equation*} \begin{aligned} \sum_{\mathbf{U}_1 \cup \cdots \cup \mathbf{U}_n} \phi_i \prod_t \phi_{k_t i} &= \phi_i \prod_t \Big( \sum_{\mathbf{U}_t} \phi_{k_t i} \Big)\\ &= \phi_i \prod_t \text{project}(\phi_{k_t i}, \mathbf{S}_{k_t i}) = \phi_i \prod_t M_{k_t i}, \end{aligned} \end{equation*}

the second equality using \(\text{vars}(\phi_{k_t i}) \setminus \mathbf{S}_{k_t i} = \mathbf{U}_t\). The variables so removed are exactly \(\bigcup_t \mathbf{U}_t\), and by the disjointness above

\begin{equation*} \text{vars}(\phi_{ij}) \setminus \bigcup_t \mathbf{U}_t = \text{vars}(i) \cup \bigcup_t \mathbf{S}_{k_t i} = \text{vars}\Big(\phi_i \prod_t M_{k_t i}\Big), \end{equation*}

which is the cluster \(\mathbf{C}_i \supseteq \mathbf{S}_{ij}\) by Exercise 7.5. Summing out what remains, \(\mathbf{C}_i \setminus \mathbf{S}_{ij}\), and applying Equation 7.2,

\begin{equation*} \text{project}(\phi_{ij}, \mathbf{S}_{ij}) = \text{project}\Big(\phi_i \prod_t M_{k_t i}, \mathbf{S}_{ij}\Big) = M_{ij}. \end{equation*}

Part 2. By Part 1 and \(\mathbf{S}_{ji} = \mathbf{S}_{ij}\), both \(M_{ij}\) and \(M_{ji}\) are factors over \(\mathbf{S}_{ij}\). Write \(\mathbf{A} = \text{vars}(i,j) \setminus \mathbf{S}_{ij}\) and \(\mathbf{B} = \text{vars}(j,i) \setminus \mathbf{S}_{ij}\); Definition 7.3 makes \(\mathbf{A}, \mathbf{B}\) disjoint, with no variable of \(\mathbf{B}\) in \(\phi_{ij}\) and none of \(\mathbf{A}\) in \(\phi_{ji}\), so

\begin{equation*} \begin{aligned} \text{project}(\phi_{ij}\phi_{ji}, \mathbf{S}_{ij}) &= \sum_{\mathbf{A}} \sum_{\mathbf{B}} \phi_{ij}\phi_{ji} = \sum_{\mathbf{A}} \phi_{ij} \sum_{\mathbf{B}} \phi_{ji}\\ &= \Big( \sum_{\mathbf{A}} \phi_{ij} \Big) \Big( \sum_{\mathbf{B}} \phi_{ji} \Big) = M_{ij} M_{ji}. \end{aligned} \end{equation*}

Every node lies on exactly one side of \(i-j\) and every CPT and indicator is assigned to exactly one node (Definition 7.2, Lines 1 to 5 of Algorithm 12), so \(\phi_{ij}\phi_{ji}\) is the product of all factors,

\begin{equation*} \phi_{ij}\phi_{ji} = \prod_{X} \Theta_{X \mid \mathbf{P}_X} \prod_{E \in \mathbf{e}} \lambda_E , \end{equation*}

with \(\mathbf{P}_X\) the parents of \(X\). By the chain rule (Equation 4.2) the first product evaluates at an instantiation \(\mathbf{z}\) of all network variables to \(\Pr(\mathbf{z})\), while the second is \(1\) if \(\mathbf{z} \sim \mathbf{e}\) and \(0\) otherwise. Hence for each instantiation \(\mathbf{s}\) of \(\mathbf{S}_{ij}\),

\begin{equation*} \text{project}(\phi_{ij}\phi_{ji}, \mathbf{S}_{ij})(\mathbf{s}) = \sum_{\mathbf{z} \sim \mathbf{s}} \Pr(\mathbf{z}) \prod_{E} \lambda_E(\mathbf{z}) = \sum_{\mathbf{z} \sim \mathbf{s}, \ \mathbf{z} \sim \mathbf{e}} \Pr(\mathbf{z}) = \Pr(\mathbf{s}, \mathbf{e}). \end{equation*}

Combining the last two displays, \(M_{ij} M_{ji} = \Pr(\mathbf{S}_{ij}, \mathbf{e})\).

One inclusion is immediate from Definition 7.4: the union defining \(\mathbf{C}_i\) has \(j\) among its neighbors and so contains the term \(\mathbf{S}_{ij}\), while the union defining \(\mathbf{C}_j\) contains \(\mathbf{S}_{ji} = \mathbf{S}_{ij}\) (the expression of Definition 7.3 is symmetric). Hence \(\mathbf{S}_{ij} \subseteq \mathbf{C}_i \cap \mathbf{C}_j\).

The reverse inclusion follows from the claim that \(\mathbf{C}_i \subseteq \text{vars}(i,j)\) for every edge \(i-j\). Since \(\mathbf{C}_i = \text{vars}(i) \cup \bigcup_m \mathbf{S}_{im}\), check the three kinds of piece: \(\text{vars}(i) \subseteq \text{vars}(i,j)\) as \(i\) sits on the \(i\)-side; \(\mathbf{S}_{ij} \subseteq \text{vars}(i,j)\) by Definition 7.3; and for a neighbor \(m \ne j\), every node on the \(m\)-side of \(i-m\) reaches \(j\) through \(m\) and then \(i\), so deleting \(i-j\) leaves it in the component of \(i\), giving \(\mathbf{S}_{im} \subseteq \text{vars}(m,i) \subseteq \text{vars}(i,j)\).

Applying the claim at both ends of the edge,

\begin{equation*} \mathbf{C}_i \cap \mathbf{C}_j \subseteq \text{vars}(i,j) \cap \text{vars}(j,i) = \mathbf{S}_{ij}. \end{equation*}

Run FE3 (Algorithm 11) on \((T,\phi)\) from any root \(r\), let \(i_1,\ldots,i_{m-1}\) be the nodes it removes and \(j_k\) the unique remaining neighbour of \(i_k\) at step \(k\), and take

\begin{equation*} \pi \;=\; V_1,\; \ldots,\; V_{m-1},\; C_r, \qquad V_k \;=\; C_{i_k}\setminus S_{i_k j_k}, \end{equation*}

the variables inside each block in arbitrary order. This is well defined: \(i_k\) is eliminated only once all its neighbours but \(j_k\) are gone, so by Exercise 7.5 its current factor is over exactly \(C_{i_k}\), and projecting on \(S_{i_k j_k}\) sums out \(V_k\).

\(\pi\) is a legal order. By Line 3 of FE2 a variable \(X\in V_k\) appears in \(\phi_{i_k}\) but in no factor of the tree remaining after step \(k\), so it lies in no later block and not in \(C_r\); conversely every network variable occurs in some CPT, hence is either summed out at a unique step or survives into \(\phi_r\), i.e. into \(C_r\). So \(\pi\) lists every variable exactly once.

Write \(T_k\) for the tree remaining just before step \(k\). Two observations:

(1) A variable \(X\in V_k\) occurs only in the factor \(\phi_{i_k}\) of \(T_k\), hence in no separator of \(T_k\) (a separator needs factors on both sides of its edge) and, since \(C_l=\mathrm{vars}(l)\cup\bigcup_m S_{lm}\), in no cluster of \(T_k\) other than \(C_{i_k}\).

(2) The clusters and separators of \(T_k\) are the original ones throughout. Passing to \(T_{k+1}\) deletes exactly \(V_k\) from one side of each edge, and by (1) those variables occur nowhere else, so no separator changes; the factor at \(j_k\) gains \(S_{i_k j_k}\), exactly the separator term its cluster loses with the deleted edge, and no other factor changes.

Width. On the interaction graph, eliminating \(X\) connects its current neighbours pairwise and deletes \(X\), and the width is the largest \(\{X\}\) together with those neighbours, minus one. Let \(G_k\) be the interaction graph just before block \(V_k\), and \(H_k\) the graph whose edges are the pairs lying together in some cluster \(C_l\), \(l\in T_k\). Then \(G_k\subseteq H_k\): for \(k=1\) each CPT sits at a node \(i\) with its variables inside \(\mathrm{vars}(i)\subseteq C_i\); and given \(G_k\subseteq H_k\), each \(X\in V_k\) lies by (1) in the single cluster \(C_{i_k}\), so its \(G_k\)-neighbours lie in \(C_{i_k}\setminus\{X\}\), its cluster is contained in \(C_{i_k}\), and the fill-in edges it creates join variables already adjacent in \(H_k\). After the block the surviving cliques are the \(C_l\), \(l\neq i_k\), together with \(C_{i_k}\setminus V_k=S_{i_k j_k}\subseteq C_{j_k}\), which by (2) are the cliques of \(H_{k+1}\).

After step \(m-1\) only \(r\) survives, the remaining variables are \(C_r\), and the graph is a subgraph of that clique. Every variable of \(\pi\) therefore has its cluster inside some \(C_i\), so

\begin{equation*} \mathrm{width}(\pi)\;\le\;\max_i |C_i| - 1 \;=\; w . \end{equation*}

The optimal width is \(2\), attained by the four-node star below and by no tree with three or fewer nodes.

Lower bound \(2\): each factor sits at some node \(i\) with \(\mathrm{vars}(i)\subseteq C_i\), and each has three variables, so some cluster has size \(\ge 3\).

Width \(2\) with four nodes. Take the star \(T\) with a fourth node \(4\) carrying no factor, adjacent to the three factor nodes \(1,2,3\):

\begin{equation*} \phi_1=f(ABE),\quad \phi_2=f(ACD),\quad \phi_3=f(DEF),\quad \phi_4=1, \end{equation*}

with edges \(1\!-\!4\), \(2\!-\!4\), \(3\!-\!4\) (Definition 7.2 requires only that each factor sit at exactly one node, so node \(4\) may carry the empty product, \(\mathrm{vars}(4)=\emptyset\)). The separators are

\begin{equation*} \begin{aligned} S_{14}&=\{A,B,E\}\cap\{A,C,D,E,F\}=\{A,E\},\\ S_{24}&=\{A,C,D\}\cap\{A,B,D,E,F\}=\{A,D\},\\ S_{34}&=\{D,E,F\}\cap\{A,B,C,D,E\}=\{D,E\}, \end{aligned} \end{equation*}

and therefore the clusters are

\begin{equation*} \begin{aligned} C_1&=\{A,B,E\},& C_2&=\{A,C,D\},\\ C_3&=\{D,E,F\},& C_4&=\{A,D,E\}. \end{aligned} \end{equation*}

All four clusters have size \(3\), so the width is \(2\), optimal by the lower bound.

At most three nodes. Two cases.

(i) Some node carries two factors. Its cluster contains the union of their variables, and

\begin{equation*} ABE\cup ACD=ABCDE,\quad ABE\cup DEF=ABDEF,\quad ACD\cup DEF=ACDEF \end{equation*}

each have size \(5\), so the width is at least \(4\).

(ii) Exactly three nodes, one factor each — the one-to-one case. A tree on three nodes is a path, and in each of the three arrangements the middle node absorbs both separators into a cluster of size \(4\), giving width \(3\). Writing \(1,2,3\) for the nodes of \(f(ABE)\), \(f(ACD)\), \(f(DEF)\):

• \(1\!-\!2\!-\!3\): \(S_{12}=\{A,E\}\), \(S_{23}=\{D,E\}\), so \(C_2=\{A,C,D,E\}\).
• \(2\!-\!1\!-\!3\): \(S_{12}=\{A,D\}\), \(S_{13}=\{D,E\}\), so \(C_1=\{A,B,D,E\}\).
• \(1\!-\!3\!-\!2\): \(S_{13}=\{A,E\}\), \(S_{23}=\{A,D\}\), so \(C_3=\{A,D,E,F\}\).

So no elimination tree with three or fewer nodes attains width \(2\), and the optimal tree has more than three nodes.

Let \(G^{\prime}\) be \(G\) with every edge leaving \(\mathbf{C}\) deleted, a polytree by hypothesis, and take \(T\) with one node per variable of \(G\), edges those of the skeleton of \(G^{\prime}\) plus any set joining its components into a single tree, and node \(X\) carrying the original CPT \(\Theta_{X|\mathbf{U}_X}\), where \(\mathbf{U}_X\) are the parents of \(X\) in \(G\). By Definition 7.2 this is an elimination tree (each CPT sits at exactly one node), \(\mathrm{vars}(X)=\{X\}\cup\mathbf{U}_X\) has at most \(k+1\) variables, and its width is at most \(k+|\mathbf{C}|\).

Key observation: an edge \(Y\to Z\) of \(G\) survives in \(G^{\prime}\) unless \(Y\in\mathbf{C}\), so every parent of \(Z\) is either in \(\mathbf{C}\) or a skeleton neighbour of \(Z\), hence a neighbour of \(Z\) in \(T\).

Separators. Let \(X\!-\!Y\) be an edge of \(T\), splitting it into \(T_X\ni X\) and \(T_Y\ni Y\), and take \(Z\in T_X\) with a parent \(W\in\mathbf{U}_Z\setminus\mathbf{C}\). By the observation \(Z\!-\!W\) is an edge of \(T\), and the only edge crossing the cut is \(X\!-\!Y\), so either \(W\in T_X\) or \(Z=X\) and \(W=Y\); hence \(\mathrm{vars}(X,Y)\subseteq T_X\cup\mathbf{C}\cup\{Y\}\), and symmetrically \(\mathrm{vars}(Y,X)\subseteq T_Y\cup\mathbf{C}\cup\{X\}\). If \(X\!-\!Y\) comes from the polytree edge \(X\to Y\) the exception cannot arise, since it would put \(Y\in\mathbf{U}_X\) against the acyclicity of \(G^{\prime}\); then \(\mathrm{vars}(X,Y)\subseteq T_X\cup\mathbf{C}\) and, as \(T_X\cap T_Y=\emptyset\),

\begin{equation*} S_{XY}\;\subseteq\;\bigl(T_X\cup\mathbf{C}\bigr) \cap\bigl(T_Y\cup\mathbf{C}\cup\{X\}\bigr)\;=\;\{X\}\cup\mathbf{C}. \end{equation*}

If \(X\!-\!Y\) is one of the added edges, no skeleton edge crosses the cut and the same computation gives \(S_{XY}\subseteq\mathbf{C}\).

Clusters. A neighbour \(Y\) of \(X\) in \(T\) is of one of three kinds: \(X\to Y\) in \(G^{\prime}\), giving \(S_{XY}\subseteq\{X\}\cup\mathbf{C}\); \(Y\to X\) in \(G^{\prime}\), giving \(S_{XY}\subseteq\{Y\}\cup\mathbf{C}\) by the same argument with the roles swapped, and \(Y\in\mathbf{U}_X\) since \(Y\) is then a parent of \(X\) in \(G^{\prime}\) hence in \(G\); or joined by an added edge, giving \(S_{XY}\subseteq\mathbf{C}\). In every case \(S_{XY}\subseteq\{X\}\cup\mathbf{U}_X\cup\mathbf{C}\), so

\begin{equation*} C_X\;\subseteq\;\{X\}\cup\mathbf{U}_X\cup\mathbf{C}, \qquad \max_X |C_X| - 1 \;\le\; k+|\mathbf{C}| . \end{equation*}

Accumulate the sum one cluster instantiation at a time, so that \(F=\Phi_i\prod_{k\neq j}M_{ki}\) — the factor over \(C_i\) that a naive reading of Equation 7.4 would build first — is never materialised. Let \(f_1,\ldots,f_p\) be the factors at node \(i\) (CPTs and indicators) and \(M_{k_1i},\ldots,M_{k_qi}\) the incoming messages, \(k_s\neq j\); all are already-stored inputs.

1. Allocate \(M_{ij}\) over \(S_{ij}\) and set \(M_{ij}(\mathbf{s})\leftarrow 0\) for every instantiation \(\mathbf{s}\).
2. Enumerate the instantiations \(\mathbf{c}\) of \(C_i\) one at a time, by an odometer over its variables. For each, compute the single number \(v = \prod_{t} f_t(\mathbf{c}_{\mathrm{vars}(f_t)}) \prod_{s} M_{k_si}(\mathbf{c}_{S_{k_si}})\), writing \(\mathbf{c}_{\mathbf{X}}\) for the restriction of \(\mathbf{c}\) to \(\mathbf{X}\), and update \(M_{ij}(\mathbf{s})\leftarrow M_{ij}(\mathbf{s})+v\) at \(\mathbf{s}=\mathbf{c}_{S_{ij}}\).
3. Return \(M_{ij}\).

Correctness: every factor in the product is over a subset of \(C_i\), since \(\mathrm{vars}(f_t)\subseteq\mathrm{vars}(i)\subseteq C_i\) and \(S_{k_si}\subseteq C_i\) by Definition 7.4, so \(\mathbf{c}\) determines each of their values and \(v=F(\mathbf{c})\). Grouping the \(\mathbf{c}\) by their restriction to \(S_{ij}\), the accumulator holds

\begin{equation*} M_{ij}(\mathbf{s})\;=\;\sum_{\mathbf{c}\sim\mathbf{s}} F(\mathbf{c}) \;=\;\Bigl(\sum_{C_i\setminus S_{ij}} F\Bigr)(\mathbf{s}), \end{equation*}

which is Equation 7.4.

Space: beyond its inputs the procedure allocates only the output table, of \(\prod_{X\in S_{ij}}|X|\) entries, and the odometer, running product and indices, \(O(|C_i|)\) numbers — in total \(O(\exp(|S_{ij}|)+|C_i|)\), exponential in the separator alone. Nor are the inputs exponential in \(|C_i|\): each \(f_t\) is a CPT or an evidence indicator, and each \(M_{k_si}\) is stored over a separator.

Part 1. Add the auxiliary factor \(f(H)\equiv 1\) at node \(4\), keeping the printed structure and all other assignments; a factor identically one changes no marginal and only enlarges \(\mathrm{vars}(4)\) to \(\{A,F,H\}\), which is the elimination-tree form of the nonminimal-jointree trick of Section 7.7.2. Since FE3 (Algorithm 11) needs a root \(r\) with \(\mathbf{Q}\subseteq C_r\), it is enough to drive \(H\) into \(C_4\); the separators carry \(A\), \(E\), \(F\) there already.

Only \(S_{46}\) changes: the \(6\)-side of \(4\!-\!6\) has variables \(\{E,F,H\}\) and the \(4\)-side now has \(\{D,F,G\}\cup\{A,C,E\}\cup\{A,F,H\}\cup\{A,B,D\}\), all eight, so

\begin{equation*} S_{46}=\{E,F,H\}. \end{equation*}

The rest are as before: on \(3\!-\!4\) the two sides are \(\{A,C,E,F,H\}\) and \(\{A,B,D,F,G\}\), giving \(S_{34}=AF\); likewise \(S_{13}=DF\), \(S_{35}=AD\), \(S_{24}=AE\). Hence

\begin{equation*} \begin{aligned} C_1&=DFG, & C_2&=ACE, & C_3&=ADF,\\ C_4&=AEFH, & C_5&=ABD, & C_6&=EFH. \end{aligned} \end{equation*}

since \(C_4=\{A,F,H\}\cup\{A,F\}\cup\{A,E\}\cup\{E,F,H\}\). The largest cluster has four variables, so the width is \(3\), and with root \(4\), FE3 returns \(\Pr(AEFH)\).

Part 2. Add \(f(GH)\equiv 1\) at node \(1\), so \(\mathrm{vars}(1)=\{D,F,G,H\}\), all other assignments unchanged. A one-variable factor will not do here: \(G\) occurs only at node \(1\) and \(H\) only at node \(6\), three edges away, so \(H\) has to be carried along the path \(6-4-3-1\). Along it \(H\) now appears on both sides of every edge:

\begin{equation*} \begin{aligned} S_{46}&=\{E,F,H\}\cap\{A,B,C,D,E,F,G,H\}=\{E,F,H\},\\ S_{34}&=\{A,B,D,F,G,H\}\cap\{A,C,E,F,H\}=\{A,F,H\},\\ S_{13}&=\{D,F,G,H\}\cap\{A,B,C,D,E,F,H\}=\{D,F,H\}, \end{aligned} \end{equation*}

while \(S_{35}=AD\) and \(S_{24}=AE\) are untouched. The clusters are

\begin{equation*} \begin{aligned} C_1&=DFGH, & C_2&=ACE, & C_3&=ADFH,\\ C_4&=AEFH, & C_5&=ABD, & C_6&=EFH. \end{aligned} \end{equation*}

for instance \(C_3=S_{13}\cup S_{34}\cup S_{35}=\{D,F,H\}\cup\{A,F,H\}\cup\{A,D\}\). The largest clusters have size \(4\), so the width is again \(3\); with root \(1\), FE3 returns \(\Pr(C_1)=\Pr(DFGH)\), whose projection on \(GH\) is \(\Pr(GH)\).

Both messages are factors over the separator \(S_{XY}\), since Equation 7.2 projects onto it and \(S_{ij}=S_{ji}\); so it is enough to show \(S_{XY}=\{X\}\).

Node \(Z\) carries \(\Theta_{Z|\mathbf{U}_Z}\) and \(\lambda_Z\), so \(\mathrm{vars}(Z)=\{Z\}\cup\mathbf{U}_Z\) is the family of \(Z\). Removing \(X\!-\!Y\) splits the tree into the disjoint \(T_X\ni X\) and \(T_Y\ni Y\), and by Definition 7.3, \(\mathrm{vars}(X,Y)=\bigcup_{Z\in T_X}(\{Z\}\cup\mathbf{U}_Z)\).

The \(X\)-side. Let \(Z\in T_X\) and \(W\in\mathbf{U}_Z\), so that \(W\!-\!Z\) is an edge of the elimination tree. The only edge joining \(T_X\) to \(T_Y\) is \(X\!-\!Y\), so \(W\in T_X\) unless \(Z=X\) and \(W=Y\) — and that exception would give \(Y\to X\), impossible alongside \(X\to Y\) in an acyclic polytree. Hence \(\mathrm{vars}(X,Y)=T_X\), the reverse inclusion holding since \(Z\in\mathrm{vars}(Z)\).

The \(Y\)-side. The same argument permits exactly one exception, \(Z=Y\) with \(W=X\), and it does occur since \(X\to Y\) means \(X\in\mathbf{U}_Y\). Hence \(\mathrm{vars}(Y,X)=T_Y\cup\{X\}\), and

\begin{equation*} S_{XY}\;=\;T_X\cap\bigl(T_Y\cup\{X\}\bigr)\;=\;\{X\}. \end{equation*}

Write \(T_X\ni X\) and \(T_Y\ni Y\) for the two sides of the edge \(X\!-\!Y\), and \(\phi_Z=\Theta_{Z|\mathbf{U}_Z}\lambda_Z\) for the node factors, so that \(\mathbf{e}^{+}_{XY}\) is the part of \(\mathbf{e}\) on \(T_X\) and \(\mathbf{e}^{-}_{XY}\) the part on \(T_Y\). By Exercise 7.13,

\begin{equation*} S_{XY}=\{X\},\qquad \mathrm{vars}(X,Y)=T_X,\qquad \mathrm{vars}(Y,X)=T_Y\cup\{X\}, \end{equation*}

and by Exercise 7.6, \(M_{ij}=\mathrm{project}(\phi_{ij},S_{ij})\) with \(\phi_{ij}=\prod_{Z\in T_i}\phi_Z\). Hence

\begin{equation*} M_{XY}\;=\;\sum_{T_X\setminus\{X\}}\;\prod_{Z\in T_X}\Theta_{Z|\mathbf{U}_Z}\lambda_Z , \end{equation*}

a factor over \(X\). Exercise 7.13 also shows every parent of a node of \(T_X\) lies in \(T_X\), so \(T_X\) is ancestrally closed and

\begin{equation*} \prod_{Z\in T_X}\Theta_{Z|\mathbf{U}_Z}\;=\;\Pr(T_X) : \end{equation*}

sum the chain-rule product \(\prod_{Z}\Theta_{Z|\mathbf{U}_Z}=\Pr\) (Equation 4.2) over the complement in reverse topological order, where each variable summed has no child left in the product, so occurs only in its own CPT, which sums to \(1\). The remaining factor \(\prod_{Z\in T_X}\lambda_Z\) is the indicator of \(\mathbf{e}^{+}_{XY}\), so for each value \(x\) of \(X\),

\begin{equation*} M_{XY}(x)\;=\;\sum_{\substack{\mathbf{t}\sim x\\ \mathbf{t}\sim\mathbf{e}^{+}_{XY}}}\Pr(\mathbf{t}) \;=\;\Pr(x,\mathbf{e}^{+}_{XY}), \end{equation*}

the sum ranging over instantiations \(\mathbf{t}\) of \(T_X\). That is, \(M_{XY}=\Pr(X,\mathbf{e}^{+}_{XY})\).

Backward, Exercise 7.6 with \(i=Y\) gives \(M_{YX}=\sum_{T_Y}\prod_{Z\in T_Y}\Theta_{Z|\mathbf{U}_Z}\lambda_Z\), again a factor over \(X\). Every parent of a node of \(T_Y\) lies in \(T_Y\cup\{X\}\), so \(\prod_{Z\in T_Y}\Theta_{Z|\mathbf{U}_Z}\) is a function of \(T_Y\cup\{X\}\) alone, and multiplying it by \(\Pr(T_X)\) recovers the full chain-rule product:

\begin{equation*} \Pr(T_X,T_Y)\;=\;\Pr(T_X)\prod_{Z\in T_Y}\Theta_{Z|\mathbf{U}_Z}. \end{equation*}

So wherever \(\Pr(\mathbf{t}_X)>0\) that product equals \(\Pr(\mathbf{t}_Y\mid \mathbf{t}_X)\), and as it depends on \(\mathbf{t}_X\) only through \(x\), the common value is \(\Pr(\mathbf{t}_Y\mid x)\) for every \(x\) with \(\Pr(x)>0\). (Where \(\Pr(x)=0\) the conditional is undefined and the product stands in for it; nothing is affected, since then \(M_{XY}(x)=\Pr(x,\mathbf{e}^{+}_{XY})=0=\Pr(x,\mathbf{e})\).) Consequently

\begin{equation*} M_{YX}(x)\;=\;\sum_{\mathbf{t}_Y\sim\mathbf{e}^{-}_{XY}}\Pr(\mathbf{t}_Y\mid x) \;=\;\Pr(\mathbf{e}^{-}_{XY}\mid x), \end{equation*}

i.e. \(M_{YX}=\Pr(\mathbf{e}^{-}_{XY}\mid X)\).

Finally, that same factorisation, with \(\mathbf{e}^{+}_{XY}\) constraining only \(T_X\) and \(\mathbf{e}^{-}_{XY}\) only \(T_Y\), gives for each \(x\)

\begin{equation*} \begin{aligned} \Pr(x,\mathbf{e}) &=\sum_{\substack{\mathbf{t}_X\sim x,\;\mathbf{t}_X\sim\mathbf{e}^{+}_{XY}\\ \mathbf{t}_Y\sim\mathbf{e}^{-}_{XY}}}\Pr(\mathbf{t}_X)\Pr(\mathbf{t}_Y\mid x)\\ &=\Bigl(\sum_{\mathbf{t}_X\sim x,\,\mathbf{e}^{+}_{XY}}\Pr(\mathbf{t}_X)\Bigr) \Bigl(\sum_{\mathbf{t}_Y\sim\mathbf{e}^{-}_{XY}}\Pr(\mathbf{t}_Y\mid x)\Bigr)\\ &=\Pr(x,\mathbf{e}^{+}_{XY})\,\Pr(\mathbf{e}^{-}_{XY}\mid x)\\ &=M_{XY}(x)\,M_{YX}(x). \end{aligned} \end{equation*}

Hence \(M_{XY}M_{YX}=\Pr(X,\mathbf{e})\), which is also the second half of Exercise 7.6 at the separator \(S_{XY}=\{X\}\).

Part 1. Both sides are factors over \(\mathbf{Y}\) — on the left, \(f_1/f_2\) is over \(\mathbf{X}\) by the generalized definition and \(\mathbf{Z} = \mathbf{X}\setminus\mathbf{Y}\) is summed out; on the right, \(\sum_{\mathbf{Z}} f_1\) and \(f_2\) are both over \(\mathbf{Y}\), so their quotient is the ordinary one of Definition 7.7. Fix an instantiation \(\mathbf{y}\). The instantiations summed on the left are exactly the \(\mathbf{x} = \mathbf{y}\mathbf{z}\), all with the same compatible \(\mathbf{y}\), so both the divisor \(f_2(\mathbf{y})\) and the applicable case of the definition are constant across the sum.

(i) \(f_2(\mathbf{y}) \neq 0\). The first case applies at every \(\mathbf{y}\mathbf{z}\), so

\begin{equation*} \begin{aligned} \left( \sum_{\mathbf{Z}} f_1/f_2 \right)(\mathbf{y}) &= \sum_{\mathbf{z}} (f_1/f_2)(\mathbf{y}\mathbf{z}) = \sum_{\mathbf{z}} \frac{f_1(\mathbf{y}\mathbf{z})}{f_2(\mathbf{y})}\\ &= \frac{1}{f_2(\mathbf{y})} \sum_{\mathbf{z}} f_1(\mathbf{y}\mathbf{z}) = \frac{\left( \sum_{\mathbf{Z}} f_1 \right)(\mathbf{y})}{f_2(\mathbf{y})} . \end{aligned} \end{equation*}

the pull-out being legitimate exactly because \(f_2\) mentions no variable of \(\mathbf{Z}\); and since \(f_2(\mathbf{y}) \neq 0\) this is the value of \(( \sum_{\mathbf{Z}} f_1 )/f_2\) at \(\mathbf{y}\) under Definition 7.7.

(ii) \(f_2(\mathbf{y}) = 0\). The second case applies at every \(\mathbf{y}\mathbf{z}\), so

\begin{equation*} \left( \sum_{\mathbf{Z}} f_1/f_2 \right)(\mathbf{y}) = \sum_{\mathbf{z}} 0 = 0 . \end{equation*}

On the other side, the divisor \(f_2\) vanishes at \(\mathbf{y}\), so Definition 7.7 gives \(\left( \left( \sum_{\mathbf{Z}} f_1 \right)/f_2 \right)(\mathbf{y}) = 0\) as well.

The two sides agree at every \(\mathbf{y}\), hence

\begin{equation*} \sum_{\mathbf{X} \setminus \mathbf{Y}} f_1/f_2 = \left( \sum_{\mathbf{X} \setminus \mathbf{Y}} f_1 \right) \Big/ f_2 , \end{equation*}

Part 2. At the moment node \(i\) is ready to send to \(j\), the identity applies with \(\mathbf{X} = \mathbf{C}_i\), \(\mathbf{Y} = S_{ij}\), \(f_1 = \Phi_i\) and \(f_2 = \Phi_{ij}^{old}\), since \(S_{ij} = \mathbf{C}_i \cap \mathbf{C}_j \subseteq \mathbf{C}_i\). Both \(\Phi_{ij}^{new} = \sum_{\mathbf{C}_i \setminus S_{ij}} \Phi_i\) and \(\Phi_{ij}^{old}\) are factors over \(S_{ij}\), so their quotient is the ordinary division of Definition 7.7 and

\begin{equation*} M_{ij} = \left( \sum_{\mathbf{C}_i \setminus S_{ij}} \Phi_i \right) \Big/ \Phi_{ij}^{old} = \sum_{\mathbf{C}_i \setminus S_{ij}} \Phi_i / \Phi_{ij}^{old}, \end{equation*}

the right-hand quotient being the generalized division of this exercise. Writing \(\Phi_{ij}\) for the separator factor before the update, the Hugin step at node \(i\) becomes

\begin{equation*} \begin{aligned} M_{ij} &= \sum_{\mathbf{C}_i \setminus S_{ij}} \frac{\Phi_i}{\Phi_{ij}},\\ \Phi_{ij} &\leftarrow \Phi_{ij} M_{ij},\\ \Phi_j &\leftarrow \Phi_j M_{ij}. \end{aligned} \end{equation*}

that is, divide first and marginalize afterwards in place of the book’s marginalize first and divide afterwards, with the separator now updated by multiplying the message in exactly as the receiving cluster is, so that the auxiliary copy \(\Phi_{ij}^{old}\) disappears.

It is the same algorithm, by the following invariant of the two-pass schedule. Put \(G_{ij} = \Psi_i \prod_{k \neq j} M_{ki}\). Then \(\Phi_{ij}\) is the product of the messages \(j\) has so far sent to \(i\), and

\begin{equation*} \Phi_i = \Phi_{ij} \, G_{ij} . \end{equation*}

Indeed \(\Phi_i\) changes only by \(\Phi_i \leftarrow \Phi_i M_{ki}\) and \(\Phi_{ij}\) only when a message crosses \(i - j\); under the inward/outward schedule exactly one message crosses each edge in each direction and \(i\) has not yet sent to \(j\), so the only change so far is from \(j\)’s message to \(i\), if sent. On the inward pass \(j\) has not sent, so \(\Phi_{ij} = 1\) while \(i\) has heard from all \(k \neq j\), giving \(\Phi_i = G_{ij}\); on the outward pass \(j\) has sent, its message computed against \(\Phi_{ij} = 1\), so \(\Phi_{ij} = M_{ji}\) and \(\Phi_i = \Psi_i \prod_k M_{ki} = M_{ji} G_{ij}\).

Given the invariant the separator update is exact: at \(\mathbf{s}\) with \(\Phi_{ij}(\mathbf{s}) \neq 0\) the division and multiplication cancel, and at \(\Phi_{ij}(\mathbf{s}) = 0\) the left side is \(0\) while the right is \(\sum_{\mathbf{c}_i \sim \mathbf{s}} \Phi_{ij}(\mathbf{s}) G_{ij}(\mathbf{c}_i) = 0\). So \(\Phi_{ij} M_{ij} = \sum_{\mathbf{C}_i \setminus S_{ij}} \Phi_i\) identically, the reformulated step leaves the jointree exactly where the book’s step would, and the guarantees \(\Pr(\mathbf{C}_i, e) = \Phi_i\) and \(\Pr(S_{ij}, e) = \Phi_{ij}\) at the end of the outward pass are untouched.

Part (a) is the table below and the width in part (b) is \(3\).

Compute \(\mathrm{vars}\) bottom-up from the leaf CPTs by \(\mathrm{vars}(T) = \mathrm{vars}(T^l) \cup \mathrm{vars}(T^r)\):

\begin{equation*} \begin{aligned} \mathrm{vars}(13) &= \{A\}, & \mathrm{vars}(12) &= \{A,B,D\},\\ \mathrm{vars}(11) &= \{A,B,D\}, & \mathrm{vars}(10) &= \{B\},\\ \mathrm{vars}(9) &= \{A,B,D\}, & \mathrm{vars}(8) &= \{B,C,E\},\\ \mathrm{vars}(7) &= \{A,B,C,D,E\}, & \mathrm{vars}(6) &= \{C,D,F\},\\ \mathrm{vars}(5) &= \{E,F,G\}, & \mathrm{vars}(4) &= \{A,B,C,D,E,F\},\\ \mathrm{vars}(3) &= \{C\}, & \mathrm{vars}(2) &= \{A,\dots,G\},\\ \mathrm{vars}(1) &= \{A,\dots,G\}. & & \end{aligned} \end{equation*}

Then cutsets top-down by Definition 8.3, \(\mathrm{cutset}(T) = (\mathrm{vars}(T^l) \cap \mathrm{vars}(T^r)) \setminus \mathrm{acutset}(T)\), each node’s a-cutset being the union of its ancestors’ cutsets and empty at the root:

\begin{equation*} \begin{aligned} \mathrm{cutset}(1) &= ABCDEFG \cap C = \{C\},\\ \mathrm{cutset}(2) &= (ABCDEF \cap EFG) \setminus \{C\} = \{E,F\},\\ \mathrm{cutset}(4) &= (CDF \cap ABCDE) \setminus \{C,E,F\} = \{D\},\\ \mathrm{cutset}(7) &= (BCE \cap ABD) \setminus \{C,D,E,F\} = \{B\},\\ \mathrm{cutset}(9) &= (B \cap ABD) \setminus \{B,C,D,E,F\} = \emptyset,\\ \mathrm{cutset}(11) &= (ABD \cap A) \setminus \{B,C,D,E,F\} = \{A\}. \end{aligned} \end{equation*}

Contexts are \(\mathrm{vars}(T) \cap \mathrm{acutset}(T)\) (Definition 8.5), and clusters are \(\mathrm{vars}(T)\) at a leaf and \(\mathrm{cutset}(T) \cup \mathrm{context}(T)\) otherwise (Definition 8.6):

A dash means the node is a leaf and so carries no cutset.

Part (b). The largest clusters are \(\mathrm{cluster}(4) = CDEF\) and \(\mathrm{cluster}(7) = BCDE\), of size \(4\), so the width is \(4 - 1 = 3\). This is optimal: eliminating \(A, G, B, C, D, E, F\) from the moral graph gives clusters \(ABD\), \(EFG\), \(BCDE\), \(CDEF\), \(DEF\), \(EF\), \(F\), so the treewidth is \(3\).

(a) \(309\) calls, (b) \(109\), (c) \(12\) hits, (d) the nonleaf nodes \(2\), \(4\), \(11\) (and the root). Throughout, the cutsets, a-cutsets, contexts and clusters are those of Exercise 8.1, and \(\mathbf{X}^{\#} = 2^{|\mathbf{X}|}\) counts instantiations.

Part (a). Lines 6–7 of RC1 loop over the instantiations of \(\mathrm{cutset}(T)\) compatible with the evidence, one call to each child per instantiation; with empty evidence none is excluded, so

\begin{equation*} \mathrm{calls}(T^l) = \mathrm{calls}(T^r) = \mathrm{calls}(T)\cdot \mathrm{cutset}(T)^{\#}, \end{equation*}

with \(\mathrm{calls}(\text{root}) = 1\), whose solution is \(\mathrm{calls}(T) = \mathrm{acutset}(T)^{\#}\) — the bound of Theorem 8.1, met with equality since the evidence is empty:

Summing,

\begin{equation*} \begin{aligned} 1 + 2\cdot 2 + 2\cdot 8 + 2\cdot 16 + 4\cdot 32 + 2\cdot 64 &= 1 + 4 + 16 + 32 + 128 + 128\\ &= 309 . \end{aligned} \end{equation*}

that is, \(308\) recursive calls beyond the top-level call on the root.

Part (b). RC2 caches at every nonleaf node, so \(T\) executes its loop once per instantiation of \(\mathrm{context}(T)\) and answers all later calls from the cache; with empty evidence every such instantiation is generated, so the loop runs \(\min(\mathrm{calls}(T), \mathrm{context}(T)^{\#})\) times and each child of \(T\) receives

\begin{equation*} \mathrm{cutset}(T)^{\#}\,\mathrm{context}(T)^{\#} = \mathrm{cluster}(T)^{\#} \end{equation*}

calls, the bound of Theorem 8.3 attained everywhere. Reading the parents’ clusters off Exercise 8.1:

summing to

\begin{equation*} 1 + 2(2 + 8 + 16 + 16 + 4 + 8) = 1 + 2\cdot 54 = 109 , \end{equation*}

against \(309\) for RC1.

Part (c). Node \(9\) receives \(16\) calls, and \(\mathrm{context}(9) = \{B,D\}\) has \(4\) instantiations, all occurring under empty evidence. The first call carrying a given instantiation misses and fills that entry, so there are \(4\) misses and

\begin{equation*} 16 - 4 = 12 \end{equation*}

cache hits at node \(9\).

Part (d). By Equation 8.4 the cache at \(T\) is dead when \(\mathrm{context}(T) = \mathrm{cluster}(T^p)\), since then the \(\mathrm{cluster}(T^p)^{\#}\) calls \(T\) receives all carry distinct instantiations of \(\mathrm{context}(T)\) and no entry is looked up after being filled. Checking each nonroot node against its parent’s cluster:

RC2 does not cache at leaves (they are answered by LOOKUP, a tabular CPT serving as its own cache), so the dead caches are those of the nonleaf nodes \(2\), \(4\), \(11\), together with the root, which is called exactly once and so never retrieves its single entry.

Take the dtree with root \(R\) and internal nodes \(L\), \(M\), \(L_1\), \(L_2\), \(M_1\):

• \(R\) has children \(L\) and \(M\)
• \(L\) has children \(L_1\) and \(L_2\)
• \(L_1\) has children the leaves \(A\) and \(ABD\)
• \(L_2\) has children the leaves \(B\) and \(BCE\)
• \(M\) has children the leaf \(EFG\) and the node \(M_1\)
• \(M_1\) has children the leaves \(C\) and \(CDF\)

Every node has \(0\) or \(2\) children and the seven leaves are exactly the seven CPTs, so this is a dtree (Definition 8.2); the leaf \(EFG\) sits at depth \(2\) and the other six at depth \(3\), so the height is \(3\), and the computation below gives width \(3\).

The variable sets are

\begin{equation*} \begin{aligned} \mathrm{vars}(L_1) &= \{A,B,D\}, & \mathrm{vars}(L_2) &= \{B,C,E\},\\ \mathrm{vars}(L) &= \{A,B,C,D,E\}, & \mathrm{vars}(M_1) &= \{C,D,F\},\\ \mathrm{vars}(M) &= \{C,D,E,F,G\}, & \mathrm{vars}( R) &= \{A,\dots,G\}. \end{aligned} \end{equation*}

At the root, \(\mathrm{acutset}( R) = \emptyset\) and

\begin{equation*} \mathrm{cutset}( R) = \mathrm{vars}(L) \cap \mathrm{vars}(M) = \{A,B,C,D,E\} \cap \{C,D,E,F,G\} = \{C,D,E\}, \end{equation*}

so \(\mathrm{cluster}( R) = CDE\) and \(\mathrm{acutset}(L) = \mathrm{acutset}(M) = \{C,D,E\}\). Then

\begin{equation*} \begin{aligned} \mathrm{cutset}(L) &= \bigl(\{A,B,D\} \cap \{B,C,E\}\bigr) \setminus \{C,D,E\} = \{B\},\\ \mathrm{context}(L) &= \{A,B,C,D,E\} \cap \{C,D,E\} = \{C,D,E\}, \end{aligned} \end{equation*}

giving \(\mathrm{cluster}(L) = \{B,C,D,E\}\), and

\begin{equation*} \begin{aligned} \mathrm{cutset}(M) &= \bigl(\{E,F,G\} \cap \{C,D,F\}\bigr) \setminus \{C,D,E\} = \{F\},\\ \mathrm{context}(M) &= \{C,D,E,F,G\} \cap \{C,D,E\} = \{C,D,E\}, \end{aligned} \end{equation*}

giving \(\mathrm{cluster}(M) = \{C,D,E,F\}\). Descending once more, \(\mathrm{acutset}(L_1) = \mathrm{acutset}(L_2) = \{B,C,D,E\}\) and \(\mathrm{acutset}(M_1) = \{C,D,E,F\}\), so

\begin{equation*} \begin{aligned} \mathrm{cutset}(L_1) &= \bigl(\{A\} \cap \{A,B,D\}\bigr) \setminus \{B,C,D,E\} = \{A\},\\ \mathrm{context}(L_1) &= \{A,B,D\} \cap \{B,C,D,E\} = \{B,D\},\\ \mathrm{cutset}(L_2) &= \bigl(\{B\} \cap \{B,C,E\}\bigr) \setminus \{B,C,D,E\} = \emptyset,\\ \mathrm{context}(L_2) &= \{B,C,E\} \cap \{B,C,D,E\} = \{B,C,E\},\\ \mathrm{cutset}(M_1) &= \bigl(\{C\} \cap \{C,D,F\}\bigr) \setminus \{C,D,E,F\} = \emptyset,\\ \mathrm{context}(M_1) &= \{C,D,F\} \cap \{C,D,E,F\} = \{C,D,F\}. \end{aligned} \end{equation*}

In full:

The largest clusters are \(\mathrm{cluster}(L) = BCDE\) and \(\mathrm{cluster}(M) = CDEF\), of size \(4\), so by Definition 8.6 the width is \(4 - 1 = 3\): no worse than the original, and optimal, the treewidth being \(3\) (Exercise 8.1).

RC1 is worse on polytrees; cutset conditioning is worse on chains of diamonds.

The two complexities: with \(n\) nodes, a loop-cutset of size \(s\) (Definition 8.1) and at most \(k\) parents per node, cutset conditioning makes \(O(\exp(s))\) calls to the polytree algorithm at \(O(n\exp(k))\) each, hence \(O(n \exp(k+s))\) time; while RC1 on a balanced dtree of cutset width \(w\) takes \(O(n \exp(w \log n)) = n^{O(w)}\) by Theorem 8.2. Neither dominates: either of \(s\), \(w\) can be small while the other is large.

Class 1: RC1 worse. Take the chain

\begin{equation*} X_1 \to X_2 \to \cdots \to X_n , \end{equation*}

a polytree, so \(s = 0\), with \(k = 1\); cutset conditioning is then a single run of the polytree algorithm, \(\Theta(n)\) time.

RC1 costs \(\Theta(n^2)\). The \(n\) CPTs are \(\{X_1\}\) and \(\{X_{i-1},X_i\}\), and the balanced dtree of Figure 8.7 (right) splits each subchain at a midpoint variable, so every internal cutset is a single variable; a node at depth \(d\) then has \(|\mathrm{acutset}| = d\) and receives \(2^d\) calls (Theorem 8.1), and there are \(2^d\) such nodes. With depth \(\log_2 n\),

\begin{equation*} \sum_{d=0}^{\log_2 n} 2^d \cdot 2^d = \sum_{d=0}^{\log_2 n} 4^d = \Theta(4^{\log_2 n}) = \Theta(n^2). \end{equation*}

No dtree does better. For any dtree over the chain whose internal cutsets all have size \(1\), a leaf at depth \(d_i\) takes \(2^{d_i}\) calls while the Kraft equality \(\sum_i 2^{-d_i} = 1\) holds over the \(n\) leaves, so Cauchy–Schwarz gives

\begin{equation*} \Bigl(\sum_i 2^{d_i}\Bigr)\Bigl(\sum_i 2^{-d_i}\Bigr) \ \ge\ \Bigl(\sum_i 1\Bigr)^2 = n^2 , \end{equation*}

so at least \(n^2\) calls go to leaves alone; and unbalancing buys an occasional empty cutset only by deepening other leaves — in the extreme the right-deep dtree of Figure 8.7 (left) has a-cutset width \(n-1\) and \(\Theta(2^n)\) calls. So on chains RC1 is asymptotically the worse algorithm, because it caches nothing and re-solves each subnetwork once per instantiation of its a-cutset.

Class 2: cutset conditioning worse. Take the networks of Figure 8.4: a chain of \(n\) diamonds on middle variables \(M_0,\dots,M_n\), top variables \(T_1,\dots,T_n\) and bottom variables \(B_1,\dots,B_n\), with edges

\begin{equation*} M_{i-1} \to T_i,\quad T_i \to M_i,\quad M_{i-1} \to B_i,\quad B_i \to M_i \qquad (i = 1,\dots,n), \end{equation*}

so that consecutive diamonds share a middle variable. There are \(3n+1\) nodes and \(k = 2\), only the \(M_i\) with \(i \ge 1\) having two parents.

The \(i\)-th diamond is the edge-disjoint undirected cycle \(M_{i-1}, T_i, M_i, B_i\), and by Definition 8.1 a loop-cutset must break each by deleting outgoing edges. On cycle \(i\) the outgoing edges belong to \(M_{i-1}\), \(T_i\), \(B_i\) (the node \(M_i\) has none), so the cutset contains one of these three for every \(i\); and each of them has all its outgoing edges on cycle \(i\) alone, so no node serves two cycles. Hence \(s \ge n\), and with binary variables cutset conditioning considers at least \(2^n\) cases: \(\Omega(2^n)\) time.

RC1 is polynomial here. Moralizing adds only the edges \(T_i - B_i\), the \(M_i\) being the sole nodes with two parents, and eliminating in the order \(M_0, T_1, B_1, M_1, T_2, \dots\) creates no fill-in: \(M_{i-1}\) leaves \(T_i, B_i\), adjacent through the moral edge, cluster \(\{M_{i-1},T_i,B_i\}\); \(T_i\) leaves \(B_i, M_i\), adjacent through \(B_i \to M_i\), cluster \(\{T_i,B_i,M_i\}\); \(B_i\) leaves only \(M_i\). Every cluster has size \(3\), so the treewidth is \(2\) whatever \(n\), and the construction of Chapter 9 yields a balanced dtree of cutset width \(w \le 3\), for which Theorem 8.2 bounds RC1 by

\begin{equation*} O\bigl(n \exp(w \log n)\bigr) = n^{O(1)} . \end{equation*}

Take a dtree \(T^{\prime}\) of small width for the polytree and transplant its shape, relabelling the leaves with the true families; the result has width at most \(k+s\), and Theorem 8.4 — RC2 on a width-\(w\) dtree costs \(O(n\exp(w))\) time and space — then gives the claim.

Write \(F_X = \{X\} \cup \mathbf{U}_X\) for the family of \(X\) in \(\mathcal{N}\), \(\mathbf{C}\) for a loop-cutset (Definition 8.1) of size \(s\), \(k\) for the maximum number of parents, and \(\mathcal{N}_{\mathbf{C}}\) for the polytree obtained by deleting the edges out of \(\mathbf{C}\), in which \(X\) has family \(F^{\prime}_X = \{X\} \cup (\mathbf{U}_X \setminus \mathbf{C})\), so that

\begin{equation*} F_X \setminus \mathbf{C} \;=\; F^{\prime}_X \setminus \mathbf{C} , \qquad F^{\prime}_X \subseteq F_X , \qquad F_X \subseteq F^{\prime}_X \cup \mathbf{C} . \end{equation*}

Step 1. \(\mathcal{N}_{\mathbf{C}}\) has an elimination order of width \(k^{\prime} := \max_X |F^{\prime}_X| - 1 \le k\): moralize and repeatedly eliminate a leaf \(X\) of the underlying undirected tree, whose neighbours are either its parents \(\mathbf{U}^{\prime}_X\), made pairwise adjacent by moralization, or (if \(X\) is parentless with one child \(Y\)) the uneliminated part of \(F^{\prime}_Y \setminus \{X\}\), again a clique; either way no fill-in and a cluster inside a family. By the polytime construction of Chapter 9 there is then a dtree \(T^{\prime}\) for \(\mathcal{N}_{\mathbf{C}}\) of width \(\le k^{\prime}\), its leaves labelled \(F^{\prime}_X\).

Step 2. Let \(T\) have the same shape as \(T^{\prime}\) with the leaf of \(X\) relabelled by the true family \(F_X\). This is the carefully chosen dtree: a full binary tree whose leaves correspond one-to-one with the CPTs of \(\mathcal{N}\) (Definition 8.2).

Step 3. Fix a tree shape and two labellings of its leaves, by sets \(L_i\) and by \(L_i \setminus \mathbf{C}\), writing \(\mathrm{vars}^-\) and so on for the second. Since \(\mathrm{vars}(T)\) is the union of the leaf labels below \(T\), \(\mathrm{vars}^-(T) = \mathrm{vars}(T) \setminus \mathbf{C}\) at every node, and we claim

\begin{equation*} \mathrm{cutset}^-(T) = \mathrm{cutset}(T)\setminus \mathbf{C}, \qquad \mathrm{acutset}^-(T) = \mathrm{acutset}(T)\setminus \mathbf{C}. \end{equation*}

by induction on depth: both a-cutsets are empty at the root, and given \(\mathrm{acutset}^-(T) = \mathrm{acutset}(T)\setminus \mathbf{C}\), Definition 8.3 gives

\begin{equation*} \begin{aligned} \mathrm{cutset}^-(T) &= \bigl(\mathrm{vars}^-(T^l) \cap \mathrm{vars}^-(T^r)\bigr) \setminus \mathrm{acutset}^-(T)\\ &= \Bigl(\bigl(\mathrm{vars}(T^l) \cap \mathrm{vars}(T^r)\bigr)\setminus \mathbf{C}\Bigr) \setminus \bigl(\mathrm{acutset}(T)\setminus \mathbf{C}\bigr)\\ &= \bigl(\mathrm{vars}(T^l) \cap \mathrm{vars}(T^r)\bigr) \setminus \bigl(\mathbf{C} \cup \mathrm{acutset}(T)\bigr)\\ &= \mathrm{cutset}(T)\setminus \mathbf{C}, \end{aligned} \end{equation*}

the third equality because removing \(\mathbf{C}\) and then \(\mathrm{acutset}(T)\setminus\mathbf{C}\) removes exactly \(\mathbf{C} \cup \mathrm{acutset}(T)\); a child’s a-cutset being its parent’s a-cutset together with its cutset, the induction goes through. Consequently

\begin{equation*} \mathrm{context}^-(T) = \mathrm{vars}^-(T)\cap \mathrm{acutset}^-(T) = \mathrm{context}(T)\setminus \mathbf{C}, \end{equation*}

and therefore \(\mathrm{cluster}^-(T) = \mathrm{cluster}(T)\setminus \mathbf{C}\) at every node by Definition 8.6, leaves included.

Reduce both \(T\) and \(T^{\prime}\) by \(\mathbf{C}\). Their shapes agree and their reduced leaf labels agree, \(F_X \setminus \mathbf{C} = F^{\prime}_X \setminus \mathbf{C}\), so the two reductions are the same labelled tree and

\begin{equation*} \mathrm{cluster}_T(T) \setminus \mathbf{C} \;=\; \mathrm{cluster}_{T^{\prime}}(T)\setminus \mathbf{C} \;\subseteq\; \mathrm{cluster}_{T^{\prime}}(T), \end{equation*}

and so

\begin{equation*} |\mathrm{cluster}_T(T)| \le |\mathrm{cluster}_{T^{\prime}}(T)| + |\mathbf{C}| \le (k^{\prime}+1) + s . \end{equation*}

The width of \(T\) is therefore at most \(k^{\prime} + s \le k + s\).

Step 4. By Theorem 8.4, RC2 on \(T\) takes

\begin{equation*} O\bigl(n\exp(k+s)\bigr) = O\bigl(\exp(s)\cdot n\exp(k)\bigr) \end{equation*}

in both time and space. With the loop-cutset bounded, \(s = O(1)\), this is \(O(n \exp(k))\) — the complexity of the polytree algorithm, and linear in the size of the network, since a network with \(n\) nodes and at most \(k\) parents each has \(O(n\exp(k))\) CPT parameters.

Suppose Line 6 is reached at a leaf dtree node \(T\) carrying \(\Theta_{X|\mathbf{U}}\); two facts force a contradiction. Since RC1 computes \(\Pr(\mathbf{e})\), the query variables are \(\mathbf{Q} = \emptyset\), and pruning under that query (Section 6.9) prunes the edges outgoing from the evidence nodes \(\mathbf{E}\) and repeatedly deletes leaf nodes outside \(\mathbf{Q}\cup\mathbf{E} = \mathbf{E}\), leaving a network \(\mathcal{N}^{\star}\) for which the dtree is built.

Fact 1: every leaf node of \(\mathcal{N}^{\star}\) is in \(\mathbf{E}\), node pruning having run until no such node remains.

Fact 2: the evidence RC1 passes to LOOKUP at \(T\) is \(\mathbf{e}\mathbf{c}\), with \(\mathbf{c}\) an instantiation of \(\mathrm{acutset}(T)\), since Line 7 extends the evidence by the current cutset instantiation at each call and the cutsets along the root-to-\(T\) path are exactly \(\mathrm{acutset}(T)\) (Definition 8.3). So Line 3 succeeds exactly when \(X \in \mathbf{E} \cup \mathrm{acutset}(T)\), and Line 6 is reached only if \(X \notin \mathbf{E}\) and \(X \notin \mathrm{acutset}(T)\).

The latter forces \(X\) into a single dtree leaf. Let \(\mathcal{S}\) be the leaves whose CPT mentions \(X\) and suppose \(|\mathcal{S}| \ge 2\), with deepest common ancestor \(T^{\star}\). Then \(T^{\star}\) is a nonleaf with a member of \(\mathcal{S}\) in each subtree, so \(X \in \mathrm{vars}(T^{\star l}) \cap \mathrm{vars}(T^{\star r})\); and for any proper ancestor \(A\) of \(T^{\star}\) all of \(\mathcal{S}\) lies in one of \(A\)’s subtrees, so \(X\) is missing from the other and \(X \notin \mathrm{cutset}(A)\), i.e. \(X \notin \mathrm{acutset}(T^{\star})\). Definition 8.3 then gives \(X \in \mathrm{cutset}(T^{\star}) \subseteq \mathrm{acutset}(T)\), against Fact 2. Hence \(|\mathcal{S}| = 1\).

But \(X\) occurs in its own CPT and in that of each of its children in \(\mathcal{N}^{\star}\), so \(|\mathcal{S}| = 1\) makes \(X\) a leaf node of \(\mathcal{N}^{\star}\) and thus \(X \in \mathbf{E}\) by Fact 1, contradicting \(X \notin \mathbf{E}\). So Line 6 is never reached.

Stage 1. Line 2 of DT2DG drops the edge directions, removes the dtree root and joins its neighbors. The dtree edges are

\begin{equation*} R - N_2,\quad R - BCD,\quad N_2 - N_3,\quad N_2 - B,\quad N_3 - A,\quad N_3 - ABC . \end{equation*}

so deleting \(R\) and joining \(N_2\) to \(BCD\) leaves \(\mathit{Tree}\) with edges

\begin{equation*} N_2 - BCD,\quad N_2 - N_3,\quad N_2 - B,\quad N_3 - A,\quad N_3 - ABC . \end{equation*}

Writing \(c_1 = N_2\) and \(c_2 = N_3\), this tree has four leaves \(A\), \(ABC\), \(B\), \(BCD\) and two internal nodes of degree three:

\begin{equation*} c_1 \text{ adjacent to } \{BCD,\ B,\ c_2\}, \qquad c_2 \text{ adjacent to } \{A,\ ABC,\ c_1\} . \end{equation*}

Stage 2. For a leaf \(P\) with neighbor \(C\), DT2DG creates a root with children \(P\) and \(\mathrm{ORIENT}(C,P)\), where \(\mathrm{ORIENT}(C,P)\) returns \(C\) if \(C\) is a leaf and otherwise a node with children \(\mathrm{ORIENT}(N_1,C)\), \(\mathrm{ORIENT}(N_2,C)\) for the two neighbors \(N_1,N_2 \ne P\) of \(C\); results are cached under \((C,P)\).

Toward \(A\): here \(C = c_2\), with other neighbors \(ABC\) and \(c_1\), and the neighbors of \(c_1\) other than \(c_2\) are the leaves \(BCD\), \(B\), so

\begin{equation*} R_A = (\,A,\ X_1\,),\qquad X_1 = (\,ABC,\ X_2\,),\qquad X_2 = (\,BCD,\ B\,). \end{equation*}

Toward \(ABC\): again \(C = c_2\), now with other neighbors \(A\) and \(c_1\), and \(\mathrm{ORIENT}(c_1,c_2)\) hits the cache, returning \(X_2\):

\begin{equation*} R_{ABC} = (\,ABC,\ X_3\,),\qquad X_3 = (\,A,\ X_2\,). \end{equation*}

Toward \(B\): here \(C = c_1\), with other neighbors \(BCD\) and \(c_2\), whose remaining neighbors are the leaves \(A\), \(ABC\), so \(\mathrm{ORIENT}(c_2,c_1) = X_5\):

\begin{equation*} R_B = (\,B,\ X_4\,),\qquad X_4 = (\,BCD,\ X_5\,),\qquad X_5 = (\,A,\ ABC\,). \end{equation*}

Toward \(BCD\): here \(C = c_1\), with other neighbors \(B\) and \(c_2\), and \(\mathrm{ORIENT}(c_2,c_1)\) hits the cache, returning \(X_5\):

\begin{equation*} R_{BCD} = (\,BCD,\ X_6\,),\qquad X_6 = (\,B,\ X_5\,). \end{equation*}

The dgraph therefore has four roots \(R_A, R_{ABC}, R_B, R_{BCD}\), six internal nodes \(X_1,\dots,X_6\), and the four shared leaves \(A\), \(ABC\), \(B\), \(BCD\):

The sharing is visible: \(X_2\) has parents \(X_1, X_3\) and \(X_5\) has parents \(X_4, X_6\), while each leaf has three parents. The counts match Theorem 8.6 for \(n = 7\): \((5n-7)/2 = 14\) nodes and \(4(n-2) = 20\) edges.

The root cutsets deliver the family marginals. Computing \(\mathrm{vars}\) bottom-up,

\begin{equation*} \begin{aligned} \mathrm{vars}(X_2) &= \{B,C,D\}, & \mathrm{vars}(X_5) &= \{A,B,C\},\\ \mathrm{vars}(X_1) &= \{A,B,C,D\}, & \mathrm{vars}(X_3) &= \{A,B,C,D\},\\ \mathrm{vars}(X_4) &= \{A,B,C,D\}, & \mathrm{vars}(X_6) &= \{A,B,C\}, \end{aligned} \end{equation*}

so, since each root has an empty a-cutset,

\begin{equation*} \begin{aligned} \mathrm{cutset}(R_A) &= \{A\} \cap \{A,B,C,D\} = \{A\},\\ \mathrm{cutset}(R_{ABC}) &= \{A,B,C\} \cap \{A,B,C,D\} = \{A,B,C\},\\ \mathrm{cutset}(R_B) &= \{B\} \cap \{A,B,C,D\} = \{B\},\\ \mathrm{cutset}(R_{BCD}) &= \{B,C,D\} \cap \{A,B,C\} = \{B,C\}. \end{aligned} \end{equation*}

Since \(A\), \(B\), \(C\) all have children, their roots carry the full families \(\{A\}\), \(\{B\}\), \(\{A,B,C\}\), and running RC there yields \(\Pr(A,\mathbf{e})\), \(\Pr(B,\mathbf{e})\), \(\Pr(A,B,C,\mathbf{e})\); \(D\) is a network leaf, so its root carries only \(\mathbf{U}_D = \{B,C\}\) and RC yields \(\Pr(B,C,\mathbf{e})\), whence \(\Pr(B,C,D,\mathbf{e}) = \Pr(B,C,\mathbf{e})\,\theta_{d|bc}\).

Yes: the dgraph’s height is at most twice the input dtree’s height \(h\), so \(O(\log n)\) is preserved.

Step 1: in the dtree \(T_P\) obtained by orienting \(\mathrm{Tree}\) toward a leaf \(P\), a node created by \(\mathrm{ORIENT}(C,\cdot)\) — say it represents \(C\) — sits at depth \(d_{\mathrm{Tree}}(P,C)\), the root \(R\) having depth \(0\). By induction on depth: the children of \(R\) are \(P\) at depth \(1\) and \(\mathrm{ORIENT}(C,P)\) for \(C\) the unique neighbor of \(P\), at depth \(1 = d_{\mathrm{Tree}}(P,C)\); and if a node representing \(C\) sits at depth \(d = d_{\mathrm{Tree}}(P,C)\), created by \(\mathrm{ORIENT}(C,P^{\prime})\), then \(P^{\prime}\) is the neighbor of \(C\) on the path from \(P\) (the recursion only moves away from \(P\)), so for each of the two other neighbors \(N_i\) that path passes through \(C\) and

\begin{equation*} d_{\mathrm{Tree}}(P,N_i) = d_{\mathrm{Tree}}(P,C)+1 = d+1 . \end{equation*}

Those two neighbors always exist: every nonleaf node of \(\mathrm{Tree}\) has degree \(3\), a nonroot internal dtree node keeping its parent and two children, and each neighbor of the deleted root gaining the other in exchange for it.

Step 2: the leaves of \(T_P\) are the leaves of \(\mathrm{Tree}\) other than \(P\), returned unchanged by ORIENT, together with the copy of \(P\) at depth \(1\). So by Step 1 the height of \(T_P\) is \(\max_L d_{\mathrm{Tree}}(P,L)\) over those leaves, the eccentricity of \(P\), since the farthest node from any node of a tree is a leaf — hence at most \(\mathrm{diam}(\mathrm{Tree})\).

Step 3: \(\mathrm{diam}(\mathrm{Tree}) \le 2h\). In the undirected version of the input dtree the path between \(u\) and \(v\) runs through their deepest common ancestor, so has length at most \(\mathrm{depth}(u)+\mathrm{depth}(v) \le 2h\); and deleting the root \(r\) while joining its two children replaces each subpath \(u^{\prime}-r-v^{\prime}\) by a single edge, leaving other distances unchanged, so distances only decrease.

Hence every dtree in the dgraph has height at most \(2h\), and \(h = O(\log n)\) gives a dgraph of height \(O(\log n)\).

Take the evidence empty, the worst case, since evidence only removes instantiations from the loop on Line 11 of RC. Write

\begin{equation*} \mathbf{W} \;=\; \mathrm{context}(T_1) \cup \bigcup_{i=1}^{n-1} \mathrm{cutset}(T_i), \qquad \mathbf{X} \;=\; \mathbf{W} \setminus \mathrm{context}(T_n), \end{equation*}

so the claimed bound is \(\mathbf{X}^{\#} - 1\).

Part 1: \(\mathbf{W}\) is a disjoint union. The cutsets of distinct nonleaf nodes are pairwise disjoint by Lemma 8.1(d); and \(\mathrm{context}(T_1) \subseteq \mathrm{acutset}(T_1) \subseteq \mathrm{acutset}(T_i)\) (Definition 8.5, and \(T_1\) an ancestor of each \(T_i\)) while Definition 8.3 makes \(\mathrm{cutset}(T_i)\) exclude \(\mathrm{acutset}(T_i)\), so \(\mathrm{cutset}(T_i) \cap \mathrm{context}(T_1) = \emptyset\). Hence

\begin{equation*} \mathbf{W}^{\#} \;=\; \mathrm{context}(T_1)^{\#}\prod_{i=1}^{n-1}\mathrm{cutset}(T_i)^{\#}. \end{equation*}

Part 2: the calls to \(T_n\) are labelled injectively by instantiations of \(\mathbf{W}\). Since \(cf(T_1) = 1\), the first call to \(T_1\) carrying an instantiation \(\mathbf{y}\) of \(\mathrm{context}(T_1)\) executes Lines 9-14 and stores the result while every later call carrying \(\mathbf{y}\) returns on Line 7, so \(T_1\) is expanded at most \(\mathrm{context}(T_1)^{\#}\) times, each expansion calling \(T_2\) once per instantiation of \(\mathrm{cutset}(T_1)\). For \(1 < i \le n-1\), \(cf(T_i) = 0\), so no call to \(T_i\) returns on Line 7 and every one is expanded, again calling \(T_{i+1}\) once per instantiation of \(\mathrm{cutset}(T_i)\). So each call to \(T_n\) is identified by the record

\begin{equation*} (\mathbf{y},\mathbf{c}_1,\mathbf{c}_2,\ldots,\mathbf{c}_{n-1}), \end{equation*}

with \(\mathbf{y}\) the instantiation of \(\mathrm{context}(T_1)\) at the expansion of \(T_1\) that started the chain and \(\mathbf{c}_i\) the instantiation of \(\mathrm{cutset}(T_i)\) chosen on Line 11. By Part 1 such a record is exactly an instantiation of \(\mathbf{W}\), distinct calls receive distinct records, and each record is a subinstantiation of the evidence \(\mathbf{ec}\) passed to \(T_n\) in that call.

Part 3: \(\mathrm{context}(T_n) \subseteq \mathbf{W}\). The ancestors of \(T_n\) are \(T_1,\ldots,T_{n-1}\) together with those of \(T_1\), so

\begin{equation*} \mathrm{acutset}(T_n) = \mathrm{acutset}(T_1) \cup \bigcup_{i=1}^{n-1}\mathrm{cutset}(T_i). \end{equation*}

and \(\mathrm{vars}(T_n) \subseteq \mathrm{vars}(T_1)\), \(T_n\) being a descendant of \(T_1\). Hence

\begin{equation*} \begin{aligned} \mathrm{context}(T_n) &= \mathrm{vars}(T_n) \cap \mathrm{acutset}(T_n)\\ &\subseteq \big(\mathrm{vars}(T_1) \cap \mathrm{acutset}(T_1)\big) \cup \bigcup_{i=1}^{n-1}\mathrm{cutset}(T_i)\\ &= \mathrm{context}(T_1) \cup \bigcup_{i=1}^{n-1}\mathrm{cutset}(T_i) \;=\; \mathbf{W}. \end{aligned} \end{equation*}

Part 4: counting one \(T_n\)-type. Fix an instantiation \(\mathbf{z}\) of \(\mathrm{context}(T_n)\), that is, one entry of \(cache_{T_n}\). By Part 2 the calls of type \(\mathbf{z}\) carry pairwise distinct instantiations of \(\mathbf{W}\), and by Part 3 each agrees with \(\mathbf{z}\) on \(\mathrm{context}(T_n) \subseteq \mathbf{W}\); the number of instantiations of \(\mathbf{W}\) extending a fixed one of \(\mathrm{context}(T_n)\) is

\begin{equation*} \big(\mathbf{W}\setminus\mathrm{context}(T_n)\big)^{\#} = \mathbf{X}^{\#}, \end{equation*}

so \(T_n\) receives at most \(\mathbf{X}^{\#}\) calls of type \(\mathbf{z}\). Since \(cf(T_n) = 1\), the first fills \(cache_{T_n}[\mathbf{z}]\) on Line 14 and each later one retrieves it on Line 7, so the retrievals after filling number at most

\begin{equation*} \mathbf{X}^{\#} - 1 = \Big(\big(\mathrm{context}(T_1) \cup \bigcup_{i=1}^{n-1}\mathrm{cutset}(T_i)\big) \setminus \mathrm{context}(T_n)\Big)^{\#} - 1, \end{equation*}

independently of the entry chosen, so for every entry of \(cache_{T_n}\).

Part 5. Take \(n = 2\), i.e. \(T_1 = T^p\) and \(T_2 = T\) with \(cf(T^p) = cf(T) = 1\), the situation of Equation 8.4. Then

\begin{equation*} \mathbf{W} = \mathrm{context}(T^p) \cup \mathrm{cutset}(T^p) = \mathrm{cluster}(T^p) \end{equation*}

by Definition 8.6, so each entry of \(cache_T\) is retrieved at most

\begin{equation*} \big(\mathrm{cluster}(T^p)\setminus\mathrm{context}(T)\big)^{\#} - 1 \end{equation*}

times. Under the condition of Equation 8.4, \(\mathrm{context}(T) = \mathrm{cluster}(T^p)\), that difference is \(\emptyset\), with \(\emptyset^{\#} = 1\), and the bound is \(0\): the cache at \(T\) is dead.

The dgraph of Figure 8.9 is already a counterexample: its shared node \(M\) satisfies Equation 8.4 at both parents, yet each of its four cache entries is retrieved once. Its network is that of Figure 8.10, with binary \(A,B,C,D\) and edges \(A \to B\), \(A \to C\), \(B \to C\), \(C \to D\), so the four CPTs — the leaves of every dtree in the dgraph — have variables \(A\), \(AB\), \(ABC\), \(CD\); Figure 8.9 is what DT2DG returns on the dtree of Figure 8.10, whose root has children \(N\) (children \(A\), \(AB\)) and \(M_0\) (children \(ABC\), \(CD\)).

Step 1. Deleting that root and joining \(N\) to \(M_0\) leaves the undirected tree \(\mathrm{Tree}\) of Figure 8.11, with edges

\begin{equation*} N - A, \quad N - AB, \quad N - M_0, \quad M_0 - ABC, \quad M_0 - CD . \end{equation*}

Orienting toward the leaf \(A\) gives

\begin{equation*} R_A \to \{\, A,\ Q_A \,\},\qquad Q_A \to \{\, AB,\ M \,\},\qquad M \to \{\, ABC,\ CD \,\}, \end{equation*}

where \(M = \mathrm{ORIENT}(M_0,N)\), and orienting toward the leaf \(AB\) gives

\begin{equation*} R_{AB} \to \{\, AB,\ Q_{AB} \,\},\qquad Q_{AB} \to \{\, A,\ M \,\}, \end{equation*}

where ORIENT is called with \((M_0,N)\) a second time and DT2DG’s cache returns the very same node \(M\). So \(M\) has the two parents \(Q_A\), \(Q_{AB}\) and one cache \(cache_M\) shared by both dtrees.

Step 2. With \(\mathrm{vars}(M) = ABCD\), Definitions 8.3, 8.5 and 8.6 give, top down in the dtree oriented toward \(A\),

\begin{equation*} \begin{aligned} \mathrm{cutset}(R_A) &= \mathrm{vars}(A)\cap\mathrm{vars}(Q_A) = A \cap ABCD = A,\\ \mathrm{context}(R_A) &= \emptyset, \qquad \mathrm{cluster}(R_A) = A,\\ \mathrm{cutset}(Q_A) &= \big(\mathrm{vars}(AB)\cap\mathrm{vars}(M)\big)\setminus A = AB \setminus A = B,\\ \mathrm{context}(Q_A) &= \mathrm{vars}(Q_A)\cap\mathrm{acutset}(Q_A) = ABCD \cap A = A,\\ \mathrm{cluster}(Q_A) &= B \cup A = AB,\\ \mathrm{cutset}(M) &= (ABC \cap CD)\setminus AB = C,\\ \mathrm{context}(M) &= ABCD \cap AB = AB . \end{aligned} \end{equation*}

In the dtree oriented toward \(AB\),

\begin{equation*} \begin{aligned} \mathrm{cutset}(R_{AB}) &= \mathrm{vars}(AB)\cap\mathrm{vars}(Q_{AB}) = AB,\\ \mathrm{context}(R_{AB}) &= \emptyset, \qquad \mathrm{cluster}(R_{AB}) = AB,\\ \mathrm{cutset}(Q_{AB}) &= \big(\mathrm{vars}(A)\cap\mathrm{vars}(M)\big)\setminus AB = A \setminus AB = \emptyset,\\ \mathrm{context}(Q_{AB}) &= ABCD \cap AB = AB,\\ \mathrm{cluster}(Q_{AB}) &= \emptyset \cup AB = AB,\\ \mathrm{context}(M) &= ABCD \cap (AB \cup \emptyset) = AB . \end{aligned} \end{equation*}

The context of \(M\) is \(AB\) in both dtrees, so \(cache_M\) is indexed by instantiations of \(AB\) and has four entries, as the sharing requires.

Step 3: Equation 8.4 holds at both parents, since

\begin{equation*} \mathrm{cluster}(Q_A) = AB = \mathrm{context}(M), \qquad \mathrm{cluster}(Q_{AB}) = AB = \mathrm{context}(M). \end{equation*}

so if the condition were sufficient for a dgraph node, \(cache_M\) could be deleted without penalty.

Step 4: the entries are nevertheless retrieved. Assume full caching, \(cf \equiv 1\), empty evidence, and run RC on the roots in turn, \(R_A\) first.

In \(RC(R_A,\mathrm{true})\), the root is expanded once and Line 11 ranges over the two instantiations \(\mathbf{a}\) of \(\mathrm{cutset}(R_A) = A\), calling \(RC(Q_A,\mathbf{a})\); these carry distinct instantiations of \(\mathrm{context}(Q_A) = A\), so both miss and expand, each looping over the two instantiations \(\mathbf{b}\) of \(\mathrm{cutset}(Q_A) = B\) and calling \(RC(M,\mathbf{ab})\). The four calls carry the four distinct instantiations of \(\mathrm{context}(M) = AB\), so all four miss and fill \(cache_M\) on Line 14 — no retrieval, as Equation 8.4 and Exercise 8.9 on the path \(Q_A, M\) predict.

In \(RC(R_{AB},\mathrm{true})\), the root loops over the four instantiations \(\mathbf{ab}\) of \(\mathrm{cutset}(R_{AB}) = AB\), calling \(RC(Q_{AB},\mathbf{ab})\); these carry the four distinct instantiations of \(\mathrm{context}(Q_{AB}) = AB\), so all miss and expand, and since \(\mathrm{cutset}(Q_{AB}) = \emptyset\) each expansion runs Line 11 once, issuing \(RC(M,\mathbf{ab})\). Those four calls carry instantiations already stored from the first run, so all four are hits on Line 7: every entry of \(cache_M\) is retrieved once after being filled, and Equation 8.4 is not sufficient for a dgraph cache.

Take \(\mathcal{N}_k\) below: a single dense family hung with a long chain, of treewidth exactly \(k\) yet solved by RC in \(O(n)\) time and space. The assumption buys only that a leaf \(T\) with CPT \(\Theta_{X|\mathbf{U}}\) costs \(O(|\mathbf{U}|+1)\) on Line 2 of RC rather than a table of size \(\Theta(\exp(|\mathbf{U}|))\), so leaves need no caches and a family may be arbitrarily large without the network being exponential; it does not reduce the number of recursive calls, and that is what must stay linear.

For \(k = 1,2,3,\ldots\) let \(\mathcal{N}_k\) be the network over binary variables

\begin{equation*} U_1,\ldots,U_k,\; X,\; Y_1,\ldots,Y_m, \qquad m = k\,2^k, \end{equation*}

with edges \(U_i \to X\), \(X \to Y_1\), and \(Y_j \to Y_{j+1}\) for \(j = 1,\ldots,m-1\), so that

\begin{equation*} n \;=\; k+1+m \;=\; k+1+k\,2^{k} \;=\; \Theta(k\,2^{k}). \end{equation*}

Moralizing makes \(\{X,U_1,\ldots,U_k\}\) a clique, forcing treewidth \(\ge k\), while the clusters \(\{X,U_1,\ldots,U_k\}\), \(\{X,Y_1\},\{Y_1,Y_2\},\ldots\) form a jointree of width \(k\); so the treewidth is exactly \(k\) and the class is unbounded.

The leaves of the dtree are the \(k+m+1\) CPTs, namely \(\{U_i\}\), the family \(\{X,U_1,\ldots,U_k\}\), and \(\{Y_1,X\}, \{Y_2,Y_1\},\ldots,\{Y_m,Y_{m-1}\}\). Build

• \(D\): the comb over the family part. Let \(D_0\) be the leaf \(\{X,U_1,\ldots,U_k\}\) and let \(D_i\) have children \(D_{i-1}\) and the leaf \(\{U_i\}\); put \(D = D_k\).
• \(C\): any balanced dtree over the \(m\) chain leaves, as in the right-hand dtree of Figure 8.7.
• the root \(R\) with children \(D\) and \(C\).

Since \(\mathrm{vars}(D) = \{X,U_1,\ldots,U_k\}\) and \(\mathrm{vars}( C) = \{X,Y_1,\ldots,Y_m\}\), Definition 8.3 gives \(\mathrm{cutset}( R) = \{X\}\): conditioning on \(X\) alone separates the dense family from the chain. Inside \(D\), top down,

\begin{equation*} \mathrm{cutset}(D_i) = \{U_i\},\qquad \mathrm{context}(D_i) = \{X,U_{i+1},\ldots,U_k\}, \end{equation*}

while inside \(C\) every cutset is a single chain variable and every cluster has at most three variables, the balanced chain dtree having width \(2\) once \(X\) is counted.

Run RC with full caching at the internal nodes and none at the leaves. Every internal node being cached, Equation 8.5 collapses for a nonroot \(T\) to \(calls(T) = \mathrm{cutset}(T^p)^{\#}\,\mathrm{context}(T^p)^{\#}\), the bound of Theorem 8.3 attained with equality, giving \(2 \cdot 2^{k-i+1}\) calls to \(D_{i-1}\) and so

\begin{equation*} \begin{aligned} \text{calls inside } D &= \textstyle\sum_{i=0}^{k}\Theta(2^{k-i+1}) = \Theta(2^{k}),\\ \text{calls inside } C &= O(m\exp(2)) = O(m), \end{aligned} \end{equation*}

the second by Theorem 8.4 on the width-\(2\) balanced chain dtree, whose subproblems are solved once per value of \(X\). The big leaf \(D_0\) receives \(2^{k+1}\) calls at \(O(k)\) each by the assumed LOOKUP, contributing \(O(k2^{k}) = O(n)\), so

\begin{equation*} \text{time} = O(2^{k}\,k + m) = O(k2^{k}) = O(n). \end{equation*}

For space, the caches hold \(\sum_T \mathrm{context}(T)^{\#} = \Theta(2^{k}) + O(m) = O(n)\) entries, the dtree has \(O(n)\) nodes, and by the assumption the CPTs occupy \(O(k) + O(m) = O(n)\) rather than the \(\Theta(2^{k})\) a tabular \(\Theta_{X|U_1\ldots U_k}\) would need. Hence

\begin{equation*} \text{space} = O(n). \end{equation*}

(a) The idea is unsound: it returns only cache factors extending \(cf_{\mathrm{old}}\), and the optimum for \(M_{\mathrm{new}}\) need not extend the optimum for \(M_{\mathrm{old}}\).

(i) Wrong search space. Let \(\mathrm{context}(T)^{\#} = 8\) and \(\mathrm{context}(T^{\prime})^{\#} = 4\), with caching at \(T\) alone removing \(100\) calls and at \(T^{\prime}\) alone removing \(20\). Under \(M_{\mathrm{old}} = 4\) only \(T^{\prime}\) fits, so \(cf_{\mathrm{old}}\) caches at \(T^{\prime}\); under \(M_{\mathrm{new}} = 8\) the optimum caches at \(T\) alone, but the procedure is locked into \(T^{\prime}\), has \(4\) entries left, cannot afford \(T\), and misses the optimum by a factor of five. (Knapsack behaviour: an optimal small packing need not sit inside an optimal larger one.)

(ii) Ill-posed subproblem. Call reduction is not additive over nodes: by Equation 8.7 the gain from caching at \(T\) is

\begin{equation*} c_1 - c_2 = \mathrm{cutset}(T)^{\#}\big(calls_{cf}(T) - \mathrm{context}(T)^{\#}\big) \big(cpc_{cf}(T^l) + cpc_{cf}(T^r)\big), \end{equation*}

which depends on the whole factor \(cf\) through \(calls_{cf}\) and \(cpc_{cf}\) (Equation 8.5). Indeed caching at an ancestor \(T\) lowers \(calls(T^{\prime})\) at a descendant and can render \(cache_{T^{\prime}}\) dead (Equation 8.4 and Exercise 8.9), so memory already committed becomes waste. A correct algorithm must be free to withdraw a cache; this one is not.

(b) Reuse \(cf_{\mathrm{old}}\) as an incumbent, not as a fixed core. Since \(M_{\mathrm{new}} > M_{\mathrm{old}}\), every old-feasible factor is new-feasible, so \(c^{\ast}_{\mathrm{new}} \le c^{\ast}_{\mathrm{old}}\), and we may rerun OPTIMAL CF with bound \(M_{\mathrm{new}}\) after replacing Lines 1–2 by

\begin{equation*} cf^{o} \leftarrow cf_{\mathrm{old}}, \qquad c^{o} \leftarrow c^{\ast}_{\mathrm{old}} \end{equation*}

in place of \(c^{o} \leftarrow \infty\). Every search node with lower bound \(\ge c^{\ast}_{\mathrm{old}}\) is then pruned immediately on Line 1, where a fresh search must first descend to a complete factor; optimality survives because we prune only completions that cannot beat a factor in hand, and factors not extending \(cf_{\mathrm{old}}\) are still explored.

Three further savings. (i) If \(c^{\ast}_{\mathrm{old}}\) equals \(\mathrm{lower\ bound}(\emptyset)\), the call count under full caching, then \(cf_{\mathrm{old}}\) is optimal for any memory and no search is needed; in general stop as soon as an incumbent attains that value. (ii) Order Line 7 to prefer nodes cached by \(cf_{\mathrm{old}}\), ties by largest \(\mathrm{context}(T)^{\#}\) (Section 8.6.1), so the first dive spends the leftover memory and tightens the incumbent early. (iii) The dgraph, the lower-bound function and the quantities \(calls(\cdot)\), \(cpc(\cdot)\) of Section 8.6.2 do not depend on \(M\), so the incremental update scheme starts from the old values.

Not reusable: the record of subtrees pruned for violating \(M_{\mathrm{old}}\), since such a subtree may hold the new optimum. Prunings from the bound test remain valid and are subsumed by the initialization above.

GREEDY CF selects node \(9\), with score \(42\), allocating it \(\mathrm{context}(9)^{\#} = 4\) cache entries.

Cutsets, contexts and clusters top down by Definitions 8.3, 8.5 and 8.6, using \(\mathrm{vars}(2) = ABCDEFG\), \(\mathrm{vars}(4) = ABCDEF\), \(\mathrm{vars}(7) = ABCDE\), \(\mathrm{vars}(9) = \mathrm{vars}(11) = ABD\):

(For instance \(\mathrm{cutset}(9) = (\mathrm{vars}(10)\cap\mathrm{vars}(11))\setminus\mathrm{acutset}(9) = B \setminus BCDEF = \emptyset\) and \(\mathrm{context}(9) = ABD \cap BCDEF = BD\); the rest is routine. Check!)

GREEDY CF starts from \(cf_1 \equiv 0\), so Equation 8.5 collapses to \(calls(T) = \mathrm{cutset}(T^p)^{\#}\,calls(T^p)\) and Equation 8.6 to \(cpc(T) = 1 + \mathrm{cutset}(T)^{\#}(cpc(T^l)+cpc(T^r))\), with \(cpc = 1\) at leaves. Traversing downward for \(calls\) and upward for \(cpc\):

Consistently, \(c_1 = \sum_T calls(T) = 309 = cpc(1)\), the calls RC1 makes with no evidence.

The internal nodes are \(2, 4, 7, 9, 11\), and Equation 8.7 gives

\begin{equation*} score_{cf_1}(T) = \frac{\mathrm{cutset}(T)^{\#}\big(calls(T)-\mathrm{context}(T)^{\#}\big) \big(cpc(T^l)+cpc(T^r)\big)}{\mathrm{context}(T)^{\#}} . \end{equation*}

Substituting the tables above:

\begin{equation*} \begin{aligned} score(2) &= \frac{4\,(2-2)(37+1)}{2} = 0,\\ score(4) &= \frac{2\,(8-8)(1+17)}{8} = 0,\\ score(7) &= \frac{2\,(16-8)(1+7)}{8} = 16,\\ score(9) &= \frac{1\,(32-4)(1+5)}{4} = \frac{168}{4} = 42,\\ score(11) &= \frac{2\,(32-4)(1+1)}{4} = \frac{112}{4} = 28 . \end{aligned} \end{equation*}

Directly: caching at \(T\) replaces \(calls(T)\) by \(\mathrm{context}(T)^{\#}\) where \(T\) feeds its children, leaving counts above \(T\) unchanged, so recomputing \(c_2 = \sum_S calls(S)\) gives

(At node \(9\), caching turns its \(32\) calls into \(4\) expansions, so \(10, 11\) drop from \(32\) to \(4\) and \(12, 13\) from \(64\) to \(8\), giving \(c_2 = 117 + 4 + 4 + 8 + 8 = 141\).) The direct counts match Equation 8.7 throughout, and the largest score is \(42\) at node \(9\).

The zeros are dead caches in the sense of Equation 8.4, since \(\mathrm{context}(2) = C = \mathrm{cluster}(1)\) and \(\mathrm{context}(4) = CEF = \mathrm{cluster}(2)\), so by Exercise 8.9 neither cache is ever read; deleting such nodes first leaves \(7, 9, 11\) and again selects \(9\).

Write \(V\) for the set of dgraph nodes and, for a cache factor \(cf\), put

\begin{equation*} e_{cf}(S) \;=\; cf(S)\,\mathrm{context}(S)^{\#} + (1-cf(S))\,calls_{cf}(S), \end{equation*}

the number of calls to \(S\) actually expanded (\(cf\) being discrete, this is \(\mathrm{context}(S)^{\#}\) when \(cf(S) = 1\) and \(calls_{cf}(S)\) when \(cf(S) = 0\)). Equation 8.5 then reads

\begin{equation*} calls_{cf}(S) \;=\; \sum_{S^p} \mathrm{cutset}(S^p)^{\#}\,e_{cf}(S^p), \tag{\(\ast\)} \end{equation*}

with \(calls_{cf}(S) = 1\) for a root \(S\), independently of \(cf\). The total number of recursive calls is \(c = \sum_{S\in V} calls_{cf}(S)\).

Set \(\delta(S) = calls_{cf_2}(S) - calls_{cf_1}(S)\) and \(\varepsilon(S) = e_{cf_2}(S) - e_{cf_1}(S)\). Inducting in topological order, \(\delta = 0\) at every root, and if \(S\) is not a proper descendant of \(T\) then no parent of \(S\) is \(T\) or a descendant of it, so \(\delta = 0\) and (as \(cf_1 = cf_2\) off \(T\)) \(\varepsilon = 0\) at every parent, whence \(\delta(S) = 0\) by \((\ast)\). In particular

\begin{equation*} calls_{cf_2}(T) = calls_{cf_1}(T), \end{equation*}

(a DAG node is not a proper descendant of itself), while \(cf_1(T)=0\), \(cf_2(T)=1\) give

\begin{equation*} \varepsilon(T) \;=\; \mathrm{context}(T)^{\#} - calls_{cf_1}(T), \end{equation*}

and \(\varepsilon(S) = (1-cf_1(S))\,\delta(S)\) for \(S \neq T\). Unrolling \((\ast)\) along the DAG (terminating by acyclicity) therefore gives, for every \(S\),

\begin{equation*} \delta(S) \;=\; \varepsilon(T)\sum_{\pi\,:\,T \rightsquigarrow S} w(\pi), \end{equation*}

the sum ranging over all directed paths \(\pi : T = S_0 \to S_1 \to \cdots \to S_r = S\) with \(r \ge 1\), where

\begin{equation*} w(\pi) \;=\; \mathrm{cutset}(S_0)^{\#}\prod_{j=1}^{r-1}\big(1-cf_1(S_j)\big)\,\mathrm{cutset}(S_j)^{\#} . \end{equation*}

each application of \((\ast)\) pushing the perturbation to the children scaled by \(\mathrm{cutset}(\cdot)^{\#}\), while a node with \(cf_1(S_j) = 1\) absorbs it entirely (\(e(S_j) = \mathrm{context}(S_j)^{\#}\) is independent of \(calls(S_j)\)). Paths are the right bookkeeping because a node may be reached from \(T\) by several routes, and by \((\ast)\) calls from different parents add.

Let \(W(S)\) be the sum over all directed paths out of \(S\), the empty path included, of \(\prod_{j=0}^{r-1}(1-cf_1(S_j))\mathrm{cutset}(S_j)^{\#}\) (empty path weight \(1\)). Splitting off the first edge,

\begin{equation*} W(S) \;=\; 1 + \big(1-cf_1(S)\big)\mathrm{cutset}(S)^{\#}\big(W(S^l)+W(S^r)\big), \end{equation*}

with \(W(S) = 1\) at a leaf. As \(cf_1\) is discrete, \(1-cf_1(S)\) is \(0\) when \(cf_1(S)=1\) and \(1\) otherwise, so this recurrence is Equation 8.6 verbatim and \(W = cpc_{cf_1}\) everywhere. Summing the path expansion over all nodes and grouping paths out of \(T\) by whether the first edge goes to \(T^l\) or \(T^r\),

\begin{equation*} \begin{aligned} c_2-c_1 &= \sum_{S\in V}\delta(S) = \varepsilon(T)\!\!\sum_{\pi \text{ starts at } T,\ |\pi|\ge 1}\!\! w(\pi)\\ &= \varepsilon(T)\,\mathrm{cutset}(T)^{\#}\big(W(T^l)+W(T^r)\big)\\ &= \varepsilon(T)\,\mathrm{cutset}(T)^{\#}\big(cpc_{cf_1}(T^l)+cpc_{cf_1}(T^r)\big). \end{aligned} \end{equation*}

Substituting \(\varepsilon(T) = \mathrm{context}(T)^{\#} - calls_{cf_1}(T)\) and changing sign,

\begin{equation*} c_1-c_2 = \mathrm{cutset}(T)^{\#}\big(calls_{cf_1}(T)-\mathrm{context}(T)^{\#}\big) \big(cpc_{cf_1}(T^l)+cpc_{cf_1}(T^r)\big). \end{equation*}

Dividing by the memory \(M = \mathrm{context}(T)^{\#}\) that the greedy method allocates to \(cache_T\) yields Equation 8.7:

\begin{equation*} score_{cf_1}(T) = \frac{c_1-c_2}{\mathrm{context}(T)^{\#}} = \frac{\mathrm{cutset}(T)^{\#}\big(calls_{cf_1}(T)-\mathrm{context}(T)^{\#}\big) \big(cpc_{cf_1}(T^l)+cpc_{cf_1}(T^r)\big)}{\mathrm{context}(T)^{\#}} . \qquad \blacksquare \end{equation*}

Algorithm 23 returns the chain \(ABD - BCDE - CDEF - EFG\), with separators \(BD\), \(CDE\), \(EF\).

Moralizing marries the parents of \(D\), \(E\), \(F\), \(G\), adding \(A - B\), \(B - C\), \(C - D\), \(E - F\), so \(G^m\) has adjacency

By Definition 9.4, \(C_i\) is \(\pi(i)\) with its current neighbours, which are then made a clique. Eliminating \(A\) (neighbours \(B, D\), already adjacent) and \(G\) (neighbours \(E, F\), already adjacent) creates no fill-in; eliminating \(B\) (remaining neighbours \(C, D, E\), with \(D - E\) missing) adds the fill-in \(D - E\), after which \(C, D, E, F\) are pairwise adjacent and the remaining eliminations add nothing. The induced sequence is

\begin{equation*} \begin{aligned} C_1 &= ABD, & C_2 &= EFG, & C_3 &= BCDE, & C_4 &= CDEF,\\ C_5 &= DEF, & C_6 &= EF, & C_7 &= F. \end{aligned} \end{equation*}

so \(\mathrm{width}(\pi, G) = 3\) by Definition 9.5, and every family lies in a cluster as Theorem 9.5 guarantees (\(A, B, ABD \subseteq C_1\); \(C, BCE \subseteq C_3\); \(CDF \subseteq C_4\); \(EFG = C_2\)).

Each of \(C_5 = DEF\), \(C_6 = EF\), \(C_7 = F\) is contained in \(C_4 = CDEF\), which is in each case the largest index below with this property, so Theorem 9.7 deletes them one at a time, moving \(C_4\) into the vacated adjacent position and thus leaving the order unchanged:

\begin{equation*} C_1 = ABD,\quad C_2 = EFG,\quad C_3 = BCDE,\quad C_4 = CDEF . \end{equation*}

None of the four contains another, and the running intersection property of Theorem 9.6 survives:

\begin{equation*} \begin{aligned} C_1 \cap (C_2 \cup C_3 \cup C_4) &= BD \subseteq C_3,\\ C_2 \cap (C_3 \cup C_4) &= EF \subseteq C_4,\\ C_3 \cap C_4 &= CDE \subseteq C_4 . \end{aligned} \end{equation*}

Theorem 9.8 now attaches each \(C_i\), working backwards from \(C_4\), to a later cluster containing \(C_i \cap (C_{i+1} \cup \dots \cup C_4)\): (i) \(CDE \subseteq C_4\), giving \(BCDE - CDEF\); (ii) \(EF\) sits only in \(C_4\) (\(BCDE\) lacks \(F\)), giving \(EFG - CDEF\); (iii) \(BD\) sits only in \(C_3\), giving \(ABD - BCDE\). The jointree is the chain

\begin{equation*} ABD \;\overset{BD}{-\!\!-}\; BCDE \;\overset{CDE}{-\!\!-}\; CDEF \;\overset{EF}{-\!\!-}\; EFG , \end{equation*}

separators written above their edges. Definition 9.13 holds: families lie in clusters as checked, and each variable spans a subpath of the chain (\(D\) in \(ABD, BCDE, CDEF\); \(E\) in \(BCDE, CDEF, EFG\); the rest in one or two adjacent clusters). Width \(3 = \mathrm{width}(\pi, G)\).

The maximum spanning tree is unique, namely the chain \(ABD - BCDE - CDEF - EFG\) of Exercise 9.1.

Algorithm 24 keeps the maximal clusters, weights edge \(C_i - C_j\) of the complete cluster graph by \(|C_i \cap C_j|\), and returns a maximum spanning tree (Theorem 9.9). The clusters are those of Exercise 9.1:

\begin{equation*} \begin{aligned} C_1 &= ABD, & C_2 &= EFG, & C_3 &= BCDE, & C_4 &= CDEF,\\ C_5 &= DEF, & C_6 &= EF, & C_7 &= F. \end{aligned} \end{equation*}

Since \(DEF, EF, F \subset CDEF\) and none of the other four contains another,

\begin{equation*} S = \{\, ABD,\; EFG,\; BCDE,\; CDEF \,\} , \end{equation*}

no reordering by Theorem 9.7 being needed here, as EO2JT2 wants only the set. The six weights \(|C_i \cap C_j|\) are

A spanning tree on four nodes has three edges, so its weight is at most \(3 + 2 + 2 = 7\); the three heaviest edges \(BCDE - CDEF\), \(ABD - BCDE\), \(EFG - CDEF\) form a simple path, hence attain it. Omitting any of the three forces weight at most \(3 + 2 + 1 = 6\), since the remaining weights are \(\{1, 1, 0\}\), so the maximum spanning tree is unique:

\begin{equation*} ABD \;-\; BCDE \;-\; CDEF \;-\; EFG , \end{equation*}

with separators \(BD\), \(CDE\), \(EF\). This is Exercise 9.1’s jointree; by Theorem 9.9 it has the jointree property, of width \(4 - 1 = 3 = \mathrm{width}(\pi, G)\).

Labelling each dtree node by \(\mathrm{cluster}(T)\) gives a jointree, which reduces to the chain \(ABD - BCDE - CDEF - EFG\); by Theorem 9.10 the dtree clusters satisfy the jointree property, and the leaf clusters are exactly the families.

Bottom-up, \(\mathrm{vars}\) is the family at a leaf and \(\mathrm{vars}(T^l) \cup \mathrm{vars}(T^r)\) at a nonleaf:

\begin{equation*} \begin{aligned} \mathrm{vars}(11) &= ABD \cup A = ABD,\\ \mathrm{vars}(9) &= B \cup ABD = ABD,\\ \mathrm{vars}(7) &= BCE \cup ABD = ABCDE,\\ \mathrm{vars}(4) &= CDF \cup ABCDE = ABCDEF,\\ \mathrm{vars}(2) &= ABCDEF \cup EFG = ABCDEFG,\\ \mathrm{vars}(1) &= ABCDEFG \cup C = ABCDEFG . \end{aligned} \end{equation*}

Descending with \(\mathrm{cutset}(T) = (\mathrm{vars}(T^l) \cap \mathrm{vars}(T^r)) \setminus \mathrm{acutset}(T)\) and \(\mathrm{context}(T) = \mathrm{vars}(T) \cap \mathrm{acutset}(T)\), where \(\mathrm{acutset}(T)\) unions the cutsets of the proper ancestors: \(\mathrm{cutset}(1) = C\), \(\mathrm{cutset}(2) = EF \setminus C = EF\), \(\mathrm{cutset}(4) = CD \setminus CEF = D\), \(\mathrm{cutset}(7) = B \setminus CDEF = B\), \(\mathrm{cutset}(9) = B \setminus BCDEF = \emptyset\), \(\mathrm{cutset}(11) = A \setminus BCDEF = A\). With \(\mathrm{cluster}(T) = \mathrm{cutset}(T) \cup \mathrm{context}(T)\) at nonleaves and \(\mathrm{vars}(T)\) at leaves,

a jointree of width \(4 - 1 = 3\). Now apply the Section 9.4 transformations, both of which preserve the jointree properties: remove cluster deletes a \(C_j\) whose single neighbour \(C_i\) contains it, and merge clusters replaces neighbours by \(C_i \cup C_j\), inheriting their neighbours. Removals: leaf \(3\) (\(C\)) into \(1\) (\(C\)); then leaf \(1\) (\(C\)) into \(2\) (\(CEF\)); then \(2\) (\(CEF\)) merges into \(4\) (\(CDEF\)), reattaching \(5\) to \(4\); leaf \(6\) (\(CDF\)) into \(4\); leaf \(8\) (\(BCE\)) into \(7\) (\(BCDE\)); \(9\) (\(BD\)) merges into \(7\), reattaching \(10, 11\); leaf \(10\) (\(B\)) into \(7\); leaves \(12\) (\(ABD\)) and \(13\) (\(A\)) into \(11\) (\(ABD\)).

Surviving are \(4\), \(5\), \(7\), \(11\), with \(5\) and \(7\) on \(4\) and \(11\) on \(7\), and no remaining cluster contains another (\(A \notin BCDE\), \(G \notin CDEF\)):

\begin{equation*} ABD \;\overset{BD}{-\!\!-}\; BCDE \;\overset{CDE}{-\!\!-}\; CDEF \;\overset{EF}{-\!\!-}\; EFG , \end{equation*}

the jointree of Exercises 9.1 and 9.2, still of width \(3\).

\(\pi = A, B, D, G, E, F, C\).

By Definition 9.11, \(X\) is eliminated at the node \(T\) with \(X \in \mathrm{cluster}(T) \setminus \mathrm{context}(T)\), which at a nonleaf is \(\mathrm{cutset}(T)\), and by Theorem 9.3 this node is unique. From Exercise 9.3,

\begin{equation*} \begin{aligned} \mathrm{cutset}(1) &= C, & \mathrm{cutset}(2) &= EF, & \mathrm{cutset}(4) &= D,\\ \mathrm{cutset}(7) &= B, & \mathrm{cutset}(9) &= \emptyset, & \mathrm{cutset}(11) &= A, \end{aligned} \end{equation*}

while at the leaves \(\mathrm{cluster} \setminus \mathrm{context}\) is empty except at node \(5\), where it is \(EFG \setminus EF = G\):

By Definition 9.12, \(X < Y\) when \(X\)’s node is a descendant of \(Y\)’s, and the two are unordered when the nodes coincide. The dtree’s ancestor chains are \(11 \prec 9 \prec 7 \prec 4 \prec 2 \prec 1\) and \(5 \prec 2 \prec 1\), with \(5\) and \(11\) in different subtrees of \(2\), so

\begin{equation*} A < B < D < E, \qquad A < B < D < F, \end{equation*}

\begin{equation*} G < E, \qquad G < F, \qquad E < C, \qquad F < C, \end{equation*}

with \(E, F\) unordered (both at node \(2\)) and \(G\) unordered against \(A\), \(B\), \(D\). Extending greedily, alphabetically first among the available minimal elements: \(A\) (over \(G\)), then \(B\) (over \(G\)), then \(D\) (over \(G\)), then \(G\) forced since \(E, F\) await it, then \(E\) before \(F\) alphabetically, then \(C\):

\begin{equation*} \pi = A,\; B,\; D,\; G,\; E,\; F,\; C . \end{equation*}

Theorem 9.4 predicts \(\mathrm{width}(\pi, G) \le 3\), the Exercise 9.3 dtree width. Applying \(\pi\) to the moral graph (edges \(A - B\), \(A - D\), \(B - C\), \(B - D\), \(B - E\), \(C - D\), \(C - E\), \(C - F\), \(D - F\), \(E - F\), \(E - G\), \(F - G\)) gives

\begin{equation*} \begin{aligned} ABD,\; BCDE,\; CDEF,\; EFG,\; CEF,\; CF,\; C , \end{aligned} \end{equation*}

the only fill-in being \(D - E\) on eliminating \(B\). The largest cluster has four variables, so \(\mathrm{width}(\pi, G) = 3\), matching the dtree width.

Both widths are \(3\).

The order: by Exercise 9.1, moralization adds \(A - B\), \(B - C\), \(C - D\), \(E - F\), and \(\pi\) induces the clusters \(ABD, EFG, BCDE, CDEF, DEF, EF, F\), the largest of size four, so \(\mathrm{width}(\pi, G) = 3\) by Definition 9.5.

Algorithm 25 starts with \(\Sigma = \{ A, ABD, B, BCE, C, CDF, EFG \}\), one single-node dtree per family, and for each \(\pi(i)\) composes all trees mentioning it (a single tree composes to itself, leaving \(\Sigma\) fixed). The trace:

so the algorithm returns \(T_6\). Taking the left-linear \(\mathrm{COMPOSE}\), which may connect its arguments into a binary tree any way, in the order listed:

rooted at \(n_6\): a full binary tree whose seven leaves biject with the families, hence a dtree by Definition 9.14. Bottom-up,

\begin{equation*} \begin{aligned} \mathrm{vars}(n_1) &= ABD, & \mathrm{vars}(n_2) &= ABD, & \mathrm{vars}(n_3) &= ABCDE,\\ \mathrm{vars}(n_4) &= ABCDE, & \mathrm{vars}(n_5) &= ABCDEF, & \mathrm{vars}(n_6) &= ABCDEFG, \end{aligned} \end{equation*}

then cutsets top-down:

\begin{equation*} \begin{aligned} \mathrm{cutset}(n_6) &= (ABCDEF \cap EFG) \setminus \emptyset = EF,\\ \mathrm{cutset}(n_5) &= (ABCDE \cap CDF) \setminus EF = CD,\\ \mathrm{cutset}(n_4) &= (ABCDE \cap C) \setminus CDEF = \emptyset,\\ \mathrm{cutset}(n_3) &= (ABD \cap BCE) \setminus CDEF = B,\\ \mathrm{cutset}(n_2) &= (ABD \cap B) \setminus BCDEF = \emptyset,\\ \mathrm{cutset}(n_1) &= (A \cap ABD) \setminus BCDEF = A . \end{aligned} \end{equation*}

With \(\mathrm{context}(T) = \mathrm{vars}(T) \cap \mathrm{acutset}(T)\) and \(\mathrm{cluster}(T) = \mathrm{cutset}(T) \cup \mathrm{context}(T)\) at nonleaves, \(\mathrm{cluster}(T) = \mathrm{vars}(T)\) at leaves, we get

The largest clusters \(CDEF\) and \(BCDE\) have size four, so the dtree has width \(4 - 1 = 3\), meeting the Theorem 9.14 bound \(\mathrm{width}(\pi, G) = 3\).

Both reduce under the preprocessing rules to a \(K_4\), so both have treewidth \(3\).

Work in the moral graphs \(G_d^m\), \(G_e^m\), widths with respect to a DAG being widths with respect to its moral graph (Definition 9.5). The multi-parent nodes are \(C\) (\(A, B\)), \(I\) (\(G, F\)), \(K\) (\(I, C\)) in \(G_d\), joined by \(E\) (\(C, B\)) and \(G\) (\(E, D\)) in \(G_e\), so moralization adds \(A - B\), \(F - G\), \(C - I\), plus \(D - E\) in \(G_e\) (\(B - C\) is already there):

The Section 9.3.2 invariant is \(\mathrm{treewidth}(G) = \max(\mathrm{treewidth}(G^{\prime}), low)\), the eliminated variables forming an optimal prefix; start at \(low = 1\), each graph having an edge. In both graphs the twig rule takes \(J\) then \(H\) (degree \(1\), \(low = 1\)), and the simplicial rule takes \(A\) (neighbours \(B, C\) with \(B - C\)) and \(K\) (neighbours \(C, I\) with \(C - I\)), raising \(low\) to \(2\) and adding no fill-in.

(i) Figure 6.6(d). What remains on \(B, C, D, E, F, G, I\) is

\begin{equation*} \begin{aligned} B &: C, D & C &: B, E, I & D &: B, F & E &: C, G\\ F &: D, G, I & G &: E, F, I & I &: C, F, G . \end{aligned} \end{equation*}

Here \(B\), \(D\), \(E\) have degree \(2\), hence are almost simplicial (a single neighbour is trivially a clique), so with \(low = 2\) the series rule eliminates each, with fill-ins \(C - D\), \(C - F\), \(C - G\) respectively. This leaves \(K_4\) on \(\{C, F, G, I\}\), of treewidth exactly \(3\) (at least \(3\) by Theorem 9.1, at most \(3\) since any order gives clusters \(4, 3, 2, 1\)). By the invariant,

\begin{equation*} \mathrm{treewidth}(G_d^m) = \max(3, low) = \max(3, 2) = 3 . \end{equation*}

The prefix, completed by the four survivors, is

\begin{equation*} \pi_d = J,\, H,\, A,\, K,\, B,\, D,\, E,\, C,\, F,\, G,\, I, \end{equation*}

whose induced cluster sequence is

\begin{equation*} \begin{aligned} &HJ,\; FH,\; ABC,\; CIK,\; BCD,\; CDF,\\ &CEG,\; CFGI,\; FGI,\; GI,\; I , \end{aligned} \end{equation*}

of maximum size \(4\), so \(\mathrm{width}(\pi_d, G_d) = 3\) and \(\pi_d\) is optimal.

(ii) Figure 6.6(e). After the same four eliminations the remaining \(G^{\prime}\) on \(\{B, C, D, E, F, G, I\}\) is

\begin{equation*} \begin{aligned} B &: C, D, E & C &: B, E, I & D &: B, E, F, G\\ E &: B, C, D, G & F &: D, G, I & G &: D, E, F, I\\ I &: C, F, G . \end{aligned} \end{equation*}

and the rules stall: every degree is \(\ge 3\) and no node is simplicial (\(B\) lacks \(C - D\), \(C\) lacks \(B - I\), and so on), while the almost simplicial rule needs \(low \ge 3\). Raise the bound instead. Since \(J, H, A, K\) were simplicial when eliminated, no fill-in was ever added, so \(G^{\prime}\) is the induced subgraph of \(G_e^m\) on \(\{B, C, D, E, F, G, I\}\), with degrees

\begin{equation*} B \mapsto 3,\quad C \mapsto 3,\quad D \mapsto 4,\quad E \mapsto 4, \end{equation*}

\begin{equation*} F \mapsto 3,\quad G \mapsto 4,\quad I \mapsto 3, \end{equation*}

so \(G^{\prime}\) has degree \(3\) in the sense of Definition 9.7. Degree lower-bounds treewidth (Exercise 9.12) and treewidth is subgraph-monotone, so \(\mathrm{treewidth}(G_e^m) \ge \mathrm{treewidth}(G^{\prime}) \ge 3\) and we may set \(low = 3\), which unlocks the almost simplicial rule at degree \(3\).

\(C\) (neighbours \(B, E, I\)) is almost simplicial: discarding \(I\) leaves the clique \(\{B, E\}\), since \(B - E\) is one of the two extra edges of Figure 6.6(e). Eliminating it adds fill-ins \(B - I\), \(E - I\):

\begin{equation*} \begin{aligned} B &: D, E, I & D &: B, E, F, G & E &: B, D, G, I\\ F &: D, G, I & G &: D, E, F, I & I &: B, E, F, G . \end{aligned} \end{equation*}

Now \(B\) (neighbours \(D, E, I\)) is almost simplicial, \(\{D, E\}\) being a clique; eliminating it adds \(D - I\):

\begin{equation*} \begin{aligned} D &: E, F, G, I & E &: D, G, I & F &: D, G, I\\ G &: D, E, F, I & I &: D, E, F, G . \end{aligned} \end{equation*}

Then \(E\) (neighbours \(D, G, I\), pairwise adjacent) is simplicial of degree \(3\), so it goes with no fill-in and \(low = 3\). The remainder on \(\{D, F, G, I\}\) carries all six edges, again a \(K_4\) of treewidth \(3\), so by the invariant

\begin{equation*} \mathrm{treewidth}(G_e^m) = \max(3, low) = \max(3, 3) = 3 . \end{equation*}

The resulting order is

\begin{equation*} \pi_e = J,\, H,\, A,\, K,\, C,\, B,\, E,\, D,\, F,\, G,\, I, \end{equation*}

with induced cluster sequence

\begin{equation*} \begin{aligned} &HJ,\; FH,\; ABC,\; CIK,\; BCEI,\; BDEI,\\ &DEGI,\; DFGI,\; FGI,\; GI,\; I , \end{aligned} \end{equation*}

whose largest clusters have four variables, so \(\mathrm{width}(\pi_e, G_e) = 3\).

Every leaf of the DAG \(G\) is simplicial in the moral graph \(G^m\), and distinct leaves are nonadjacent there, so the simplicial rule of Section 9.3.2 applies to them one after another; any enumeration \(\tau = L_1, \dots, L_k\) of the leaves is thus an optimal prefix. Widths with respect to \(G\) are widths with respect to \(G^m\) (Definition 9.5), so it suffices to argue in \(G^m\).

Simpliciality: a leaf \(X\) has no children, so the DAG edges at \(X\) are exactly \(U \to X\) for \(U \in \mathrm{Parents}(X)\), and \(X\) is never married as a co-parent; hence

\begin{equation*} \mathrm{Neighbours}_{G^m}(X) = \mathrm{Parents}(X), \end{equation*}

which moralizing the family of \(X\) makes a clique. So \(X\) is simplicial of degree \(d_X = |\mathrm{Parents}(X)|\).

Nonadjacency: for leaves \(X \ne Y\), neither is a parent of the other and neither is a co-parent of anything, so \(X - Y \notin G^m\). Eliminating \(X\) therefore removes no neighbour of \(Y\), and being simplicial it adds no fill-in at all, so \(Y\) is still simplicial with the same neighbourhood. Inductively, after \(L_1, \dots, L_{i-1}\) the node \(L_i\) remains simplicial with neighbourhood \(\mathrm{Parents}(L_i)\), no fill-in has appeared, the cluster induced at step \(i\) is the family \(\{L_i\} \cup \mathrm{Parents}(L_i)\), and the graph left is \(G^m \setminus L\).

The simplicial rule of Section 9.3.2 is thus licensed at each of the \(k\) steps, its invariant giving both optimality of the prefix and

\begin{equation*} \mathrm{treewidth}(G^m) = \max\Bigl(\max_{i} |\mathrm{Parents}(L_i)|,\; \mathrm{treewidth}(G^m \setminus L)\Bigr). \qquad \blacksquare \end{equation*}

Each single-child root is simplicial in the moral graph \(G^m\), so the simplicial rule of Section 9.3.2 applies to them in turn, exactly as for the leaves in Exercise 9.7. Widths with respect to \(G\) are widths with respect to \(G^m\) (Definitions 9.2 and 9.5).

Let \(X\) be a root whose only child is \(C\). Its neighbours in \(G^m\) are its parents (none), its children (just \(C\)) and its spouses, the other parents of \(C\), so

\begin{equation*} N_{G^m}(X) \;=\; \{C\} \cup \bigl(\mathrm{Pa}( C) \setminus \{X\}\bigr), \end{equation*}

a clique: moralization retains every edge \(U \to C\) and marries all parents of \(C\). Hence \(X\) is simplicial.

Let \(X_1, \dots, X_k\) be all such roots, in any order, and put \(G^m_j = G^m - X_1 - \cdots - X_{j-1}\). Inductively, no fill-in has been added, so \(G^m_j\) is the induced subgraph of \(G^m\) on the survivors and

\begin{equation*} N_{G^m_j}(X_j) \;=\; N_{G^m}(X_j) \setminus \{X_1,\ldots,X_{j-1}\}, \end{equation*}

a subset of a \(G^m\)-clique and hence a clique of \(G^m_j\), since deleting nodes destroys no edges among the survivors. So \(X_j\) is simplicial in \(G^m_j\) and its elimination again adds no fill-in.

Applying the simplicial rule \(k\) times therefore yields an optimal elimination order of \(G^m\) beginning \(X_1, \dots, X_k\), and the choice of order among them is immaterial by Theorem 9.2. \(\blacksquare\)

A graph of treewidth \(\le 1\) is a forest, on which one of the two rules always fires and never creates fill-in, so the order they generate has width \(\le 1\) and is optimal.

Cycles force treewidth \(\ge 2\): let \(Z_1 - \cdots - Z_m - Z_1\) be a cycle (\(m \ge 3\)), let \(\pi\) be any order, and let \(i\) be the first step eliminating a cycle node \(X = \pi(i) = Z_t\), with distinct cycle neighbours \(U = Z_{t-1}\), \(W = Z_{t+1}\) still present in \(G_i\). Elimination only adds edges and deletes the eliminated node (Definition 9.3), so \(X - U\) and \(X - W\) survive in \(G_i\) and

\begin{equation*} C_i \;\supseteq\; \{X, U, W\}, \qquad |C_i| \ge 3, \end{equation*}

so \(\mathrm{width}(\pi,G) \ge 2\) (Definition 9.5) and, \(\pi\) being arbitrary, \(\mathrm{treewidth}(G) \ge 2\) (Definition 9.6). Contrapositively, \(\mathrm{treewidth}(G) \le 1\) makes \(G\) a forest.

A nonempty forest has a node of degree \(\le 1\): with no edges every degree is \(0\); otherwise an endpoint \(v\) of a longest path has no neighbour off the path (it would extend it) and none on the path but its predecessor (it would close a cycle), so \(\deg(v) = 1\). One of the two rules therefore always fires, and eliminating a node of degree \(d \le 1\) creates no fill-in (no nonadjacent neighbour pair exists), leaving the forest \(F - X\) and a cluster of size \(d + 1 \le 2\).

Iterating to the empty graph gives an order \(\pi\) with all clusters of size \(\le 2\), so \(\mathrm{width}(\pi, G) \le 1\). If \(G\) has an edge then \(\mathrm{treewidth}(G) \ge 1\) by Theorem 9.1, forcing \(\mathrm{width}(\pi,G) = \mathrm{treewidth}(G) = 1\) since no width falls below the treewidth; if \(G\) has none, every cluster is a singleton and both are \(0\). Either way \(\pi\) is optimal. \(\blacksquare\)

On a graph \(G\) with \(\mathrm{treewidth}(G) \le 2\) the three rules never stall and the order they generate is optimal. Three ingredients.

(i) A nonempty graph has a node of degree at most its treewidth, since the degree of a graph lower-bounds its treewidth (Exercise 9.12); so a nonempty graph of treewidth \(\le 2\) offers a node of degree \(0\), \(1\) or \(2\).

(ii) Eliminating a node \(X\) of degree \(d \le 2\) in \(H\) yields a minor \(H^{\prime}\) (Definition 9.3).

• If \(d \le 1\) there is no nonadjacent pair of neighbors, so no fill-in is added and \(H^{\prime} = H - X\), a subgraph of \(H\).
• If \(d = 2\) with neighbors \(U, W\), then \(H^{\prime}\) is \(H - X\) together with the single fill-in edge \(U - W\) (absent when \(U\) and \(W\) are already adjacent, in which case \(H^{\prime} = H - X\)). But \(H^{\prime}\) is exactly the graph obtained from \(H\) by contracting the edge \(X - U\): contraction replaces \(X\) and \(U\) by a node adjacent to

\begin{equation*} \bigl(N(X) \cup N(U)\bigr) \setminus \{X,U\} \;=\; \{W\} \cup \bigl(N(U)\setminus\{X\}\bigr), \end{equation*}

and renaming that node \(U\) gives precisely \(H - X\) plus the edge \(U-W\).

So \(H^{\prime}\) is a minor of \(H\) (Definition 9.9), and since deleting nodes cannot raise the treewidth (restrict any order: every cluster shrinks) while contraction cannot either (Exercise 9.13),

\begin{equation*} \mathrm{treewidth}(H^{\prime}) \;\le\; \mathrm{treewidth}(H). \end{equation*}

(iii) Minimum degree \(\ge 2\) forces a cycle, hence treewidth \(\ge 2\) by Exercise 9.9: take a longest path, let \(v\) be an endpoint and \(u\) its predecessor; \(v\) has a neighbour \(w \ne u\), which cannot lie off the path (it would extend it), so the path segment from \(w\) to \(v\) closes into a cycle of length \(\ge 3\).

Now run from \(H_1 = G\) with any sound initial \(\mathrm{low}\). At each stage \(H_i\) is a minor of \(G\), so \(\mathrm{treewidth}(H_i) \le 2\) by (ii), and if nonempty it has a node of degree \(\le 2\) by (i).

• If it has a node of degree \(0\) or \(1\), fire the Islet or Twig rule.
• Otherwise the minimum degree is exactly 2, so \(\mathrm{treewidth}(G) \ge \mathrm{treewidth}(H_i) \ge 2\) by (iii) and minor-monotonicity; \(\mathrm{low} \leftarrow 2\) is sound and the Series rule fires.

Either way one node goes, so the run terminates after \(n\) steps with an order \(\pi\) whose every cluster is \(\{X\} \cup N_{H_i}(X)\) for some \(X\) of degree \(\le 2\), of size \(\le 3\); hence

\begin{equation*} \mathrm{width}(\pi, G) \;\le\; 2 \end{equation*}

by Definition 9.5. Two cases match this to the treewidth. (i) \(G\) a forest: every \(H_i\) is then a forest with a node of degree \(\le 1\) (Exercise 9.9), so the Series rule never fires and the run is that of Exercise 9.9, giving \(\mathrm{width}(\pi,G) = \mathrm{treewidth}(G) \in \{0,1\}\). (ii) \(G\) not a forest: it has a cycle, so \(\mathrm{treewidth}(G) \ge 2\) by Exercise 9.9, hence \(= 2\), and as no width falls below the treewidth,

\begin{equation*} 2 \;\le\; \mathrm{width}(\pi,G) \;\le\; 2 . \end{equation*}

Either way \(\pi\) is optimal. \(\blacksquare\)

Every graph in the min-degree run keeps treewidth \(\le 2\), which caps every cluster at size \(3\).

Let \(\pi\) be a min-degree order on \(G\) with \(\mathrm{treewidth}(G) \le 2\), with graph sequence \(G_1 = G, \dots, G_n\) and clusters \(C_i\), and put \(d_i = |C_i| - 1\), the minimum degree of \(G_i\) by the heuristic, that is \(\mathrm{degree}(G_i)\) in the sense of Definition 9.7. Inductively, if \(\mathrm{treewidth}(G_i) \le 2\) then \(d_i \le 2\) by Exercise 9.12, so eliminating \(\pi(i)\) yields a minor of \(G_i\) by Exercise 9.10(ii) and \(\mathrm{treewidth}(G_{i+1}) \le \mathrm{treewidth}(G_i) \le 2\) by minor-monotonicity (Exercise 9.13 for the contraction case). Hence \(|C_i| \le 3\) throughout and, by Definition 9.5,

\begin{equation*} \mathrm{width}(\pi, G) \;=\; \max_{i} |C_i| - 1 \;\le\; 2 . \end{equation*}

Since \(\mathrm{width}(\pi,G) \ge \mathrm{treewidth}(G)\) always (Definition 9.6), only a strict gap need be ruled out. (i) \(\mathrm{treewidth}(G) = 2\): the display forces equality. (ii) \(\mathrm{treewidth}(G) \le 1\): then \(G\) is a forest (Exercise 9.9), whose minimum degree is \(\le 1\), so each elimination adds no fill-in and leaves a forest; inductively \(d_i \le 1\) and \(\mathrm{width}(\pi,G) \le 1\), which equals \(1 = \mathrm{treewidth}(G)\) by Theorem 9.1 if \(G\) has an edge and equals \(0\) otherwise. \(\blacksquare\)

The first cluster of any elimination order already witnesses the bound.

Let \(\pi\) be any elimination order for a nonempty \(G\), with clusters \(C_1, \dots, C_n\) (Definition 9.4). Since \(G_1 = G\) carries no fill-in yet,

\begin{equation*} C_1 \;=\; \{\pi(1)\} \cup N_G(\pi(1)), \qquad |C_1| \;=\; 1 + \deg_G(\pi(1)). \end{equation*}

and the width of \(\pi\) is the largest cluster size minus one (Definition 9.5), hence at least \(|C_1| - 1\):

\begin{equation*} \mathrm{width}(\pi,G) \;\ge\; |C_1| - 1 \;=\; \deg_G(\pi(1)) \;\ge\; \min_{v \in V}\deg_G(v) \;=\; \mathrm{degree}(G), \end{equation*}

the last step because \(\pi(1)\) is one of the nodes minimized over. As \(\pi\) was arbitrary, Definition 9.6 gives

\begin{equation*} \mathrm{treewidth}(G) \;=\; \min_{\pi} \mathrm{width}(\pi,G) \;\ge\; \mathrm{degree}(G), \end{equation*}

the required bound. Equivalently: every nonempty graph has a node of degree at most its treewidth, the form used in Exercises 9.10 and 9.11. \(\blacksquare\)

Rename \(X\) and \(Y\) to \(Z\) inside every cluster of an optimal jointree; the same tree then works for the contracted graph, with no cluster larger.

Let \(G^{\prime}\) be \(G\) with the edge \(X - Y\) contracted, so its nodes are \((V(G) \setminus \{X,Y\}) \cup \{Z\}\), its edges being \(A - B\) for edges of \(G\) avoiding \(X, Y\), and \(Z - A\) whenever \(A \notin \{X,Y\}\) neighbours \(X\) or \(Y\) in \(G\). Let \((T, C)\) be a jointree for \(G\) of minimal width \(w = \mathrm{treewidth}(G)\) and define \(C^{\prime}\) on the same \(T\) by

\begin{equation*} C^{\prime}_i \;=\; \begin{cases} \bigl(C_i \setminus \{X,Y\}\bigr) \cup \{Z\}, & \text{if } C_i \cap \{X,Y\} \ne \emptyset,\\ C_i, & \text{otherwise.} \end{cases} \end{equation*}

The three conditions hold for \((T, C^{\prime})\) over \(G^{\prime}\).

(1) Each \(C^{\prime}_i\) drops \(X, Y\) and possibly adds \(Z\), all of whose elements are nodes of \(G^{\prime}\).

(2) An edge \(A - B\) of \(G^{\prime}\) avoiding \(X, Y\) is an edge of \(G\), so lies in some \(C_i\), and both survive renaming; an edge \(Z - A\) comes from an edge \(X - A\) or \(Y - A\) of \(G\), whose cluster \(C_i\) renames to one containing \(Z\) and \(A\).

(3) For \(A \ne Z\) the renaming touches nothing, so \(\{i : A \in C^{\prime}_i\} = \{i : A \in C_i\}\) is connected. For \(Z\), write

\begin{equation*} T_X = \{\, i : X \in C_i \,\}, \qquad T_Y = \{\, i : Y \in C_i \,\}, \end{equation*}

so that by construction

\begin{equation*} \{\, i : Z \in C^{\prime}_i \,\} \;=\; \{\, i : C_i \cap \{X,Y\} \ne \emptyset \,\} \;=\; T_X \cup T_Y . \end{equation*}

Each of \(T_X\), \(T_Y\) is a connected subtree by the jointree property for \((T,C)\), and they meet: \(X - Y\) being an edge of \(G\), some \(C_k\) holds both, so \(k \in T_X \cap T_Y\). Two connected subtrees sharing a node have connected union – for \(i \in T_X\), \(j \in T_Y\), the \(i\)-to-\(k\) path stays in \(T_X\) and the \(k\)-to-\(j\) path in \(T_Y\), and their concatenation contains the unique \(i\)-to-\(j\) path. So the jointree property holds for \(Z\) too. Finally, for each \(i\),

\begin{equation*} |C^{\prime}_i| \;=\; |C_i| - |C_i \cap \{X,Y\}| + \bigl[\, C_i \cap \{X,Y\} \ne \emptyset \,\bigr] \;\le\; |C_i|, \end{equation*}

the bracketed indicator being at most \(|C_i \cap \{X,Y\}|\) when that intersection is nonempty and both sides vanishing otherwise. So \(\mathrm{width}(T,C^{\prime}) \le w\), and as the treewidth of \(G^{\prime}\) is the width of its best jointree,

\begin{equation*} \mathrm{treewidth}(G^{\prime}) \;\le\; w \;=\; \mathrm{treewidth}(G). \qquad \blacksquare \end{equation*}

The maximum \(M\) returned by the min-degree removal sequence equals \(\mathrm{degeneracy}(G)\). The procedure generates

\begin{equation*} G \;=\; H_1,\ H_2,\ \ldots,\ H_n, \qquad H_{i+1} \;=\; H_i - v_i , \end{equation*}

where \(v_i\) is a node of minimum degree in \(H_i\), and returns

\begin{equation*} M \;=\; \max_{1 \le i \le n} \mathrm{degree}(H_i) \;=\; \max_{1 \le i \le n} \deg_{H_i}(v_i), \end{equation*}

the two agreeing since \(v_i\) has minimum degree in \(H_i\), and Definition 9.8 makes \(\mathrm{degeneracy}(G) = \max_H \mathrm{degree}(H)\) over nonempty subgraphs.

The maximum is attained at an induced subgraph: a nonempty subgraph \(H\) with \(S = V(H)\) sits inside \(G[S]\), so \(\deg_H(v) \le \deg_{G[S]}(v)\) on \(S\) and

\begin{equation*} \mathrm{degree}(H) \;\le\; \mathrm{degree}(G[S]). \end{equation*}

so only the \(G[S]\) matter.

\(M \le \mathrm{degeneracy}(G)\): each \(H_i = G[V(H_i)]\) is a nonempty subgraph of \(G\), so \(\mathrm{degeneracy}(G) \ge \mathrm{degree}(H_i)\) by Definition 9.8, and the maximum over \(i\) is \(M\).

\(M \ge \mathrm{degeneracy}(G)\): fix \(S \ne \emptyset\), put \(\delta = \mathrm{degree}(G[S])\), and let

\begin{equation*} i \;=\; \min\{\, j : v_j \in S \,\} \end{equation*}

be the step removing the first node of \(S\) (every node is removed exactly once). By minimality \(S \subseteq V(H_i)\), so \(G[S]\) is an induced subgraph of \(H_i\) containing \(v_i\) and \(\deg_{G[S]}(v_i) \le \deg_{H_i}(v_i)\), restricting to \(S\) only dropping neighbours. Since \(v_i\) is one of the nodes \(\delta\) minimizes over,

\begin{equation*} \delta \;=\; \mathrm{degree}(G[S]) \;\le\; \deg_{G[S]}(v_i) \;\le\; \deg_{H_i}(v_i) \;=\; \mathrm{degree}(H_i) \;\le\; M . \end{equation*}

the penultimate equality being precisely where the min-degree choice is used: \(v_i\) realizes \(\mathrm{degree}(H_i)\). Maximizing over \(S\) gives \(\mathrm{degeneracy}(G) \le M\), so \(M = \mathrm{degeneracy}(G)\). \(\blacksquare\)

The claim is the second half of a loop invariant: (A) no two \(OL\) nodes share a subgraph; (B) no \(OL\) node shares a subgraph with a \(CL\) node. Both hold initially, \(OL\) being a singleton and \(CL\) empty. Note that a search node’s subgraph depends only on the set of eliminated variables (Theorem 9.2), which is what makes the duplicate tests meaningful, and that eliminating a variable deletes exactly that variable (Definition 9.3), so a child \(\rho\) of \(\tau\) has \(|G_\rho| = |G_\tau| - 1\) and hence

\begin{equation*} G_\rho \ne G_\tau \quad\text{for every child } \rho \text{ of } \tau , \end{equation*}

expansion being entered only for nonempty \(G_\tau\).

Assume (A) and (B) at the start of an iteration whose chosen node \((G_\tau, \tau, w_\tau, b_\tau)\) is expanded (otherwise the algorithm returns).

(A) survives: \(OL\) changes only by the replacement on Lines 18–19, which keeps the count of \(G_\rho\)-nodes at one and touches no other subgraph, and by the insertion on Line 22, reached only when Line 16 found no \(G_\rho\)-node on \(OL\), raising that count from zero to one.

(B) survives insertion into \(OL\): a child inserted on Line 19 has \(OL\) already carrying \(G_\rho\), so (B) puts no \(G_\rho\)-node on \(CL\); a child inserted on Line 22 has passed the Line 21 guard, which checks exactly that. These are the only ways into \(OL\).

(B) survives insertion into \(CL\) on Line 25: the chosen node was the only \(OL\) node with subgraph \(G_\tau\) by (A), and was removed on Line 6, while every node added to \(OL\) during the expansion is a child, with \(G_\rho \ne G_\tau\) by the display. So no \(OL\) node carries \(G_\tau\) when Line 25 runs.

Nodes never leave \(CL\) and no other line touches either list, so (A) and (B) persist. \(\blacksquare\)

Every node the algorithm selects satisfies \(f = \max(w_\sigma, b_\sigma) \le w^\star = \mathrm{treewidth}(G)\), so a selected complete order has \(w_\tau \le w^\star\) and is optimal.

Fix an optimal order \(\pi = \pi(1), \dots, \pi(n)\), and for \(0 \le k \le n\) let \(H_k\) be the subgraph left by the prefix \(\pi(1), \dots, \pi(k)\), so \(H_0 = G\) and \(H_n\) is empty.

Fact 1 (width decomposition). By Definition 9.4 the graph and cluster sequences induced by \(\sigma\rho\) on \(G\) split into those of \(\sigma\) on \(G\) followed by those of \(\rho\) on \(G_\sigma\), so by Definition 9.5

\begin{equation*} \mathrm{width}(\sigma\rho, G) = \max\bigl(w_\sigma,\ \mathrm{width}(\rho, G_\sigma)\bigr) \;\ge\; \max(w_\sigma, b_\sigma), \end{equation*}

using \(\mathrm{width}(\rho, G_\sigma) \ge \mathrm{treewidth}(G_\sigma) \ge b_\sigma\); thus \(f\) is admissible. Fact 2: \(\mathrm{treewidth}(H_k) \le w^\star\), since the tail of \(\pi\) eliminates \(H_k\) within width \(w^\star\).

Call \(N = (G_\sigma, \sigma, w_\sigma, b_\sigma)\) good when \(G_\sigma = H_k\) for some \(k\) and \(w_\sigma \le w^\star\). Then \(\sigma\,\pi(k+1)\cdots\pi(n)\) is a complete order (Theorem 9.2) of width \(\le w^\star\) by Fact 1, and \(f(N) \le w^\star\) by Fact 2.

Fact 3 (persistence). Once \(OL \cup CL\) holds a good node with subgraph \(H_k\), it always does: a node leaves \(OL\) only by selection (going to \(CL\)) or by replacement with a same-subgraph node of strictly smaller width, itself good; \(CL\) never loses nodes.

Induct on the iteration index \(t\) for (I1) the node selected at \(t\) has \(f \le w^\star\), and (I2) \(OL\) holds a good node at the start of \(t\). Initially \(OL = \{(G, \varepsilon, 0, b)\}\), good since \(G = H_0\), giving (I2) at \(t = 1\); and (I2) gives (I1) at once, Line 6 minimizing \(f\) over a list containing a node of \(f \le w^\star\).

Expanding a good node \(N^\star = (H_k, \sigma, w_\sigma, b_\sigma)\) with \(H_k\) nonempty yields a good successor. Eliminating \(\pi(k+1)\) from \(H_k\) gives subgraph \(H_{k+1}\) (Theorem 9.2) and width \(w_\rho = \max(w_\sigma, d)\), where \(\pi(k+1)\) together with its \(d\) neighbours in \(H_k\) is exactly the cluster \(C_{k+1}\) induced by \(\pi\) on \(G\), so \(d = |C_{k+1}| - 1 \le w^\star\) and the child is good. Lines 16–23 then either update the \(OL\) node for \(H_{k+1}\) to width \(\min(w_\rho, w_{\rho^{\prime}}) \le w^\star\), or insert the good child, or discard it because \(CL\) holds an \(H_{k+1}\)-node, which was selected earlier and so has width \(\le f \le w^\star\) by (I1). All three leave a good \(H_{k+1}\)-node in \(OL \cup CL\).

For (I2) at \(t+1\), assuming the algorithm did not return at \(t\), put

\begin{equation*} J = \{\, k : OL \cup CL \text{ contains a good node with subgraph } H_k \,\}, \end{equation*}

at the start of iteration \(t+1\). Then \(0 \in J\) by Fact 3, so \(k = \max J\) exists; let \(M\) be a good \(H_k\)-node in \(OL \cup CL\). Were \(M \in CL\), Exercise 9.15 would put no \(H_k\)-node at all on \(OL\); also \(k < n\), an \(H_n\)-node never reaching \(CL\) since selecting it returns. But \(M\) reached \(CL\) by being selected and expanded while good (width \(\le f \le w^\star\) by (I1)), so by the previous paragraph and Fact 3 a good \(H_{k+1}\)-node survives, contradicting maximality of \(k\). Hence \(M \in OL\), which is (I2).

Now let a selected node have \(\tau\) complete, so \(G_\tau\) is empty and \(b_\tau \le 0 \le w_\tau\), giving \(f = w_\tau \le w^\star\) by (I1). Since \(w_\tau = \mathrm{width}(\tau, G) \ge \mathrm{treewidth}(G) = w^\star\) (Definition 9.6), \(w_\tau = w^\star\) and \(\tau\) is optimal. \(\blacksquare\)

Every factor VE PR builds under the order \(\pi\) returned by JT2EO lives inside a jointree cluster, so \(\mathrm{width}(\pi, G) \le w\), the jointree width.

JT2EO removes leaves one at a time, so the remaining tree is always a subtree of \(T\), and since a path in a subtree is the path in \(T\), the jointree property (Definition 9.13) holds at every stage.

\(\pi\) is an elimination order. Let \(T_X = \{i : X \in C_i\}\), a connected subtree. When JT2EO removes \(i \in T_X\) with sole neighbour \(j\): if some other remaining node \(k\) lies in \(T_X\), the path from \(i\) to \(k\) starts with \(i - j\), so \(X \in C_j\) and \(X\) is not appended; otherwise \(X \notin C_j\) and \(X\) is appended now. So \(X\) is appended exactly once, when the last node of \(T_X\) goes (or with \(C_r\) if that node is the survivor \(r\)). Every variable lies in its own family, hence in some cluster (Definition 9.13), so \(\pi\) lists all variables exactly once.

The width claim reduces to a bound on VE factors. Run VE PR on any network with DAG \(G\) using \(\pi\), with empty query and evidence, so the initial factors are the CPTs. By Definition 6.4, their interaction graph is the moral graph \(G^m\), since \(\mathrm{vars}(f_X)\) is the family of \(X\) and making each family a clique is moralization (Definition 9.1). The two observations after Definition 6.4 say that eliminating \(\pi(i)\) builds a factor over \(\pi(i)\) and its current neighbours, and that the new interaction graph connects those neighbours pairwise and deletes \(\pi(i)\) – exactly graph elimination (Definition 9.3). Inductively the interaction graph before step \(i\) is \(G_i\) and the factor built has variable set \(C_i^{\pi}\), so

\begin{equation*} \mathrm{width}(\pi, G) \;=\; \max_i |C_i^\pi| - 1 \;=\; \bigl(\text{largest number of variables in a constructed factor}\bigr) - 1 , \end{equation*}

and it suffices to confine every constructed factor to a cluster.

Every family lies in some cluster (Definition 9.13), so assign each CPT \(f_X\) to a node \(i\) with \(\mathrm{vars}(f_X) \subseteq C_i\), and run JT2EO and VE PR in lockstep under the invariant

(INV) every factor held at a node \(i\) of the remaining tree has its variables inside \(C_i\),

which holds initially. Let JT2EO remove leaf \(i\) with sole neighbour \(j\), so VE PR eliminates \(V = C_i \setminus (C_i \cap C_j)\) in some order, and let \(X \in V\). No remaining cluster but \(C_i\) contains \(X\), so by (INV) no factor away from \(i\) mentions \(X\), and eliminating \(X\) multiplies exactly the factors at \(i\), building a factor inside \(C_i\) and leaving \(\sum_X \prod f_k\) inside \(C_i \setminus \{X\}\) at node \(i\). The argument repeats down \(V\), and the surviving factor lands inside \(C_i \setminus V = C_i \cap C_j \subseteq C_j\), so moving it to \(j\) restores (INV). When only \(r\) remains, all factors lie inside \(C_r\) and eliminating \(C_r\) builds only subsets of it.

Hence every constructed factor has at most \(|C_i| \le w + 1\) variables, so every \(C_i^\pi\) does, and

\begin{equation*} \mathrm{width}(\pi, G) \;=\; \max_i |C_i^\pi| - 1 \;\le\; w . \qquad \blacksquare \end{equation*}

The witness is \(\pi(i)\) itself, which lies in \(C_i\) but is gone by \(G_j\).

The nodes of \(G_k\) are those of \(G\) other than \(\pi(1), \dots, \pi(k-1)\): eliminating a node adds fill-in edges and deletes that node (Definition 9.3), leaving the node set otherwise untouched, so the claim follows by induction from \(G_1 = G\).

Fix \(i < j\). Then \(\pi(i) \in C_i\) by Definition 9.4, while \(C_j\) consists of nodes of \(G_j\), from which \(\pi(i)\) has already been deleted since \(i \le j-1\). Hence

\begin{equation*} \pi(i) \in C_i \quad\text{and}\quad \pi(i) \notin C_j , \end{equation*}

so \(C_i \not\subseteq C_j\). \(\blacksquare\)

Everything follows from \(\mathrm{acutset}(T) \subseteq U(T)\) and \(\mathrm{vars}(T) \cap U(T) \subseteq \mathrm{acutset}(T)\), where \(U(T) = \mathrm{vars}^\uparrow(T)\).

Note that \(U(T)\) unions \(\mathrm{vars}\) over the leaves outside the subtree of \(T\) (deleting \(T\) leaves exactly the part hanging off its parent), that \(\mathrm{vars}(T)\) unions \(\mathrm{vars}\) over the leaves inside it, and that \(\mathrm{vars}(T) \subseteq \mathrm{vars}(T^p)\).

First inclusion: if \(X \in \mathrm{cutset}(T^\star) \subseteq \mathrm{vars}(T^{\star l}) \cap \mathrm{vars}(T^{\star r})\) for a proper ancestor \(T^\star\), with \(T\) inside the subtree of, say, \(T^{\star l}\), then \(X\) occurs at a leaf under \(T^{\star r}\), which is outside the subtree of \(T\); so \(X \in U(T)\).

Second inclusion: let \(X \in \mathrm{vars}(T) \cap U(T)\), occurring at a leaf \(L\) outside the subtree of \(T\), and let \(T^\star\) be the deepest node whose subtree holds both. Then \(T^\star \ne T\) is a proper ancestor of \(T\), with \(T\) and \(L\) under different children, say \(T^{\star l}\) and \(T^{\star r}\), so upward monotonicity gives \(X \in \mathrm{vars}(T^{\star l}) \cap \mathrm{vars}(T^{\star r})\). Either \(X \in \mathrm{acutset}(T^\star)\), so \(X\) lies in the cutset of an ancestor of \(T\), or \(X \in \mathrm{cutset}(T^\star)\) by definition of cutset; either way \(X \in \mathrm{acutset}(T)\).

Hence \(\mathrm{vars}(T) \cap \mathrm{acutset}(T)\), \(\mathrm{vars}(T) \cap U(T)\) and \(\mathrm{vars}(T) \cap \mathrm{acutset}(T)\) chain into each other, so all coincide:

\begin{equation*} \mathrm{context}(T) = \mathrm{vars}(T) \cap \mathrm{acutset}(T) = \mathrm{vars}(T) \cap U(T). \end{equation*}

For the cutset, take \(T\) nonleaf and abbreviate \(L = \mathrm{vars}(T^l)\), \(R = \mathrm{vars}(T^r)\), \(U = U(T)\), so \(\mathrm{vars}(T) = L \cup R \supseteq L \cap R\). Subtracting a set from a subset of \(\mathrm{vars}(T)\) is subtracting its intersection with \(\mathrm{vars}(T)\), so

\begin{equation*} \begin{aligned} \mathrm{cutset}(T) &= (L \cap R) \setminus \mathrm{acutset}(T) \\ &= (L \cap R) \setminus \bigl(\mathrm{vars}(T) \cap \mathrm{acutset}(T)\bigr) \\ &= (L \cap R) \setminus \bigl(\mathrm{vars}(T) \cap U\bigr) \\ &= (L \cap R) \setminus U , \end{aligned} \end{equation*}

the third equality being the context equation and the last using \(L \cap R \subseteq \mathrm{vars}(T)\). For the cluster, again with \(T\) nonleaf,

\begin{equation*} \mathrm{cluster}(T) = \mathrm{cutset}(T) \cup \mathrm{context}(T) = \bigl((L \cap R) \setminus U\bigr) \cup \bigl((L \cup R) \cap U\bigr), \end{equation*}

so, distributing the second term,

\begin{equation*} \mathrm{cluster}(T) = \bigl((L \cap R) \setminus U\bigr) \cup (L \cap U) \cup (R \cap U). \end{equation*}

which equals \((L \cap R) \cup (L \cap U) \cup (R \cap U)\): \(\subseteq\) is clear, and for \(\supseteq\) only \(L \cap R\) needs covering, an \(X\) there lying in \((L \cap R) \setminus U\) when \(X \notin U\) and in \(L \cap U\) otherwise.

Finally \(\mathrm{cluster}(T) \subseteq \mathrm{vars}(T)\), trivially at a leaf and because all three terms sit inside \(L \cup R\) at a nonleaf. \(\blacksquare\)

Yes. Take \(G\) on \(X, Y, Z, W\) with edges \(X \to Y\), \(X \to Z\), \(Y \to Z\), \(X \to W\), families

\begin{equation*} L_1 = \{X\}, \quad L_2 = \{X, Y\}, \quad L_3 = \{X, Y, Z\}, \quad L_4 = \{X, W\}, \end{equation*}

and \(\pi = X, Y, Z, W\). At \(i = 1\) the variable \(X\) occurs in all four trees, so EO2DT replaces them by \(\mathrm{COMPOSE}(L_1, L_2, L_3, L_4)\), which may be the vine

\begin{equation*} N_1 = (L_2, L_3), \qquad N_2 = (N_1, L_4), \qquad N_3 = (N_2, L_1), \end{equation*}

rooted at \(N_3\); the later steps see a single tree and change nothing. With \(\mathrm{vars}(N_1) = XYZ\) and \(\mathrm{vars}(N_2) = \mathrm{vars}(N_3) = XYZW\), the Section 9.5 definitions give

The two loaded rows: \(\mathrm{acutset}(N_3) = \emptyset\) and \(\mathrm{cutset}(N_3) = XYZW \cap X = X\), so \(X\) is eliminated at \(N_3\); and since \(\mathrm{cutset}(N_2) = (XYZ \cap XW) \setminus X = \emptyset\) gives \(\mathrm{acutset}(N_1) = X\),

\begin{equation*} \mathrm{cutset}(N_1) = (\mathrm{vars}(L_2) \cap \mathrm{vars}(L_3)) \setminus X = (XY \cap XYZ) \setminus X = Y , \end{equation*}

so \(Y\) is eliminated at \(N_1\), each variable at a unique node as Theorem 9.3 requires. (The remaining rows: Check!)

But \(N_1\) is a descendant of \(N_3\), so Definition 9.12 imposes \(Y < X\), while \(\pi\) places \(X\) first. Hence \(\pi\) is inconsistent with the dtree EO2DT built from it.