Sorting: a search engine (or an online shop) must return the matching pages (products) ordered by relevance score (or price), so the \(n\) candidate records must be sorted by key before the first page of results is shown; likewise a bank sorts the day’s cheques by account number before posting them.

Convex hull: to fence the smallest region enclosing a set of \(n\) trees, or to test whether two robots’ footprints can collide, one computes the convex hull of the given points in the plane; the fence posts (or the collision boundary) are exactly the hull’s vertices, and the fence length is the hull’s perimeter.

Memory (space) usage is the main one, alongside several others:

• (i) space: RAM, cache and disk consumed (Section 1.2 names memory as a bounded resource next to time);
• (ii) energy or power draw, decisive on phones, sensors and in data centres;
• (iii) communication: network bandwidth, number of messages, or disk I/O transfers;
• (iv) monetary cost: cloud compute hours, hardware, licences;
• (v) parallel resources: number of processors or machines needed, and how well the work scales across them;
• (vi) latency or worst-case response time (as opposed to throughput), crucial for real-time systems;
• (vii) quality per unit effort: accuracy of an approximate answer, and the programmer time needed to implement and maintain the method.

Take the array (a contiguous block of \(n\) equal-sized slots).

Strengths:

• (i) element \(A[i]\) is reached in \(O(1)\) time by address arithmetic (random access);
• (ii) no per-element pointer overhead, and contiguous storage gives excellent cache locality;
• (iii) if kept sorted, membership is answered by binary search in \(O(\lg n)\) time.

Limitations:

• (i) the size is fixed at allocation, so growth requires reallocating and copying (\(\Theta(n)\) per resize, amortized \(O(1)\) per append with geometric, e.g. doubling, growth as in Section 17.4);
• (ii) inserting or deleting at an arbitrary position shifts up to \(n\) elements, costing \(\Theta(n)\);
• (iii) searching an unsorted array takes \(\Theta(n)\), and keeping it sorted makes each insertion \(\Theta(n)\);
• (iv) a large array needs one contiguous chunk of memory, which may be unavailable even when enough total memory is free.

Both are minimum-weight route problems on the same model; they differ in the route’s constraints and, consequently, in their computational difficulty.

Similar:

• (i) both model the road network as a weighted graph (vertices = intersections or addresses, edge weights = distances);
• (ii) both are optimization problems: among exponentially many candidate routes, find one of minimum total length;
• (iii) both have direct applications in transport, logistics and network routing.

Different:

• (i) a shortest path joins two given vertices and may skip every other vertex, while a traveling-salesman tour must visit every delivery address and return to the depot (a Hamiltonian cycle on the addresses, with road distances as weights);
• (ii) shortest paths are solvable efficiently, e.g. by Dijkstra’s algorithm in polynomial time (Chapter 24), whereas the traveling-salesman problem is NP-complete (Section 34.5.4), with no known polynomial-time algorithm;
• (iii) shortest paths have optimal substructure (every subpath of a shortest path is shortest, Lemma 24.1), which the dynamic-programming and greedy methods exploit; for the traveling-salesman problem one instead settles for approximation algorithms (Section 35.2) that guarantee a tour within a constant factor of optimal when the distances obey the triangle inequality.

Only the best will do: awarding a scholarship to the highest-scoring applicant; a candidate whose score is merely close to the maximum is the wrong winner, so the exact optimum is required.

Approximately best is fine: the delivery company of Section 1.1 routing its trucks (the traveling-salesman problem); a tour a few percent longer than optimal costs little extra fuel, while finding the true optimum is NP-complete and may be infeasible, so a fast approximation algorithm (Chapter 35) is preferred.

A turn-by-turn navigation service (a phone maps app) is such an application: its core deliverable, the route, is the output of algorithms.

• (i) Shortest paths. The road network is a weighted directed graph (intersections as vertices, road segments as edges weighted by expected travel time). Answering a route query is a single-pair shortest-path problem, solved with Dijkstra’s algorithm (Section 24.3) or a goal-directed variant such as A* with a straight-line-distance heuristic; precomputation schemes speed up the repeated queries on a fixed continental graph.
• (ii) Map matching and geometry. Raw GPS fixes are snapped to the nearest road segment, a nearest-neighbour search over a spatial index, and the displayed map is clipped to the viewport with computational-geometry primitives (Chapter 33).
• (iii) Dynamic re-weighting. Live traffic changes edge weights, so routes are recomputed as data arrive; the algorithms must answer within milliseconds on graphs with tens of millions of vertices, which is only possible because their running times are near-linear rather than exponential.
• (iv) Multi-stop trips. Ordering several delivery stops is a travelling-salesman instance (Section 35.2), NP-complete, so the application uses an approximation or heuristic rather than an exact search.

Insertion sort wins exactly for \(2 \le n \le 43\).

For \(n \ge 1\),

\begin{equation*} 8n^2 < 64 n \lg n \iff n < 8\lg n \iff f(n) := n - 8\lg n < 0 . \end{equation*}

Since \(f^{\prime\prime}(x) = 8/(x^2\ln 2) > 0\), \(f\) is convex on \((0,\infty)\), so \(\{x : f(x) < 0\}\) is an interval. Its integer endpoints are fixed by

\begin{equation*} \begin{aligned} f(1) &= 1 > 0, & f(2) &= 2 - 8 = -6 < 0,\\ f(43) &= 43 - 8\lg 43 \approx -0.41 < 0, & f(44) &= 44 - 8\lg 44 \approx 0.32 > 0, \end{aligned} \end{equation*}

so \(f(n) < 0\) for the integers \(n = 2, 3, \ldots, 43\) and for no others.

The smallest such \(n\) is \(n = 15\).

We need \(g(n) := 2^n/n^2 > 100\). For \(n = 1, 2\), \(100n^2 \ge 100 > 4 \ge 2^n\), so the inequality fails. For \(n \ge 3\), \(g\) is increasing, because

\begin{equation*} \frac{g(n+1)}{g(n)} = \frac{2n^2}{(n+1)^2} > 1 \iff \sqrt{2}\,n > n+1 \iff n > \frac{1}{\sqrt 2 - 1} \approx 2.41 . \end{equation*}

Hence for \(3 \le n \le 14\),

\begin{equation*} g(n) \le g(14) = \frac{16384}{196} \approx 83.6 < 100, \end{equation*}

while

\begin{equation*} g(15) = \frac{32768}{225} \approx 145.6 > 100, \end{equation*}

i.e. \(100\cdot 15^2 = 22500 < 32768 = 2^{15}\). By monotonicity the \(100n^2\) algorithm stays faster for every \(n \ge 15\).

The answer is the largest integer \(n\) with \(f(n) \le T\), where \(T\) is \(t\) in microseconds; taking a month as 30 days, a year as 365 days and a century as 100 years,

\begin{equation*} \begin{aligned} T &= 10^6,\ 6\cdot 10^7,\ 3.6\cdot 10^9,\ 8.64\cdot 10^{10}, \\ &\quad 2.592\cdot 10^{12},\ 3.1536\cdot 10^{13},\ 3.1536\cdot 10^{15}. \end{aligned} \end{equation*}

Since every \(f\) is increasing, the inverses give the entries directly:

\begin{equation*} \begin{aligned} n &= 2^{T},\quad T^2,\quad T, \\ n &= \lfloor T^{1/2}\rfloor,\quad \lfloor T^{1/3}\rfloor,\quad \lfloor \lg T\rfloor \end{aligned} \end{equation*}

for \(\lg n\), \(\sqrt{n}\), \(n\), \(n^2\), \(n^3\), \(2^n\) respectively. For \(n\lg n\) and \(n!\) there is no closed-form inverse; the entry is the \(n\) with \(n\lg n \le T < (n+1)\lg(n+1)\) (found by bisection, as \(n\lg n\) is increasing) and the \(n\) with \(n! \le T < (n+1)!\) (e.g. \(9! = 362880 \le 10^6 < 10! = 3628800\)).

The array becomes \(\langle 26, 31, 41, 41, 58, 59\rangle\) after five iterations of the for loop; each row below shows \(A\) at the start of iteration \(j\) (rows (a)-(e)) and the final array (f).

Reverse the comparison on line 5: shift elements right while they are smaller than the key.

NONINCREASING-INSERTION-SORT(A)
1  for j = 2 to A.length
2      key = A[j]
3      // Insert A[j] into the nonincreasing sequence A[1..j-1].
4      i = j - 1
5      while i > 0 and A[i] < key
6          A[i+1] = A[i]
7          i = i - 1
8      A[i+1] = key

Correctness is the Section 2.1 argument with the invariant: at the start of each for-loop iteration, \(A[1\,..\,j-1]\) holds the elements originally in \(A[1\,..\,j-1]\), in nonincreasing order (the while loop stops at the first \(A[i] \ge key\), so placing key at \(A[i+1]\) keeps \(A[1\,..\,j]\) nonincreasing).

Scan \(A[1], A[2], \ldots\) in order and return the first index holding \(v\); return NIL if the scan finishes.

LINEAR-SEARCH(A, v)
1  for i = 1 to A.length
2      if A[i] == v
3          return i
4  return NIL

Loop invariant: at the start of each iteration of the for loop of lines 1-3, \(v \notin A[1\,..\,i-1]\).

Initialization: before the first iteration \(i = 1\), and \(A[1\,..\,0]\) is empty, so it does not contain \(v\).

Maintenance: suppose the invariant holds at the start of the iteration for \(i\). If \(A[i] = v\), line 3 returns \(i\), which is a correct answer. Otherwise \(A[i] \ne v\), and together with \(v \notin A[1\,..\,i-1]\) this gives \(v \notin A[1\,..\,i]\), which is the invariant for the next value \(i+1\).

Termination: the procedure either returns at line 3 an index \(i\) with \(A[i] = v\) (correct), or the loop exits because \(i = A.length + 1 = n+1\); then the invariant says \(v \notin A[1\,..\,n]\), so \(v\) does not appear in \(A\) and returning NIL on line 4 is correct.

Formally (storing the least significant bit first):

Input: two arrays \(A[1\,..\,n]\) and \(B[1\,..\,n]\) with every entry in \(\{0,1\}\), representing \(a = \sum_{k=1}^{n} A[k]\,2^{k-1}\) and \(b = \sum_{k=1}^{n} B[k]\,2^{k-1}\).

Output: an array \(C[1\,..\,n+1]\) with every entry in \(\{0,1\}\) such that \(\sum_{k=1}^{n+1} C[k]\,2^{k-1} = a + b\).

The algorithm is grade-school addition with a carry bit, from the low-order end.

BINARY-ADD(A, B, n)
1  let C[1..n+1] be a new array
2  carry = 0
3  for i = 1 to n
4      s = A[i] + B[i] + carry
5      C[i] = s mod 2
6      carry = floor(s / 2)
7  C[n+1] = carry
8  return C

Loop invariant: at the start of each iteration of the for loop, \(carry \in \{0,1\}\) and

\begin{equation*} \begin{aligned} \sum_{k=1}^{i-1} C[k]\,2^{k-1} + carry \cdot 2^{i-1} &= \sum_{k=1}^{i-1} (A[k]+B[k])\,2^{k-1}. \end{aligned} \end{equation*}

Initialization: \(i = 1\) and \(carry = 0\), so both sides are \(0\). Maintenance: \(s = A[i] + B[i] + carry \in \{0,1,2,3\}\) and \(s = C[i] + 2\,carry^{\prime}\) with \(C[i], carry^{\prime} \in \{0,1\}\) after lines 5-6, so

\begin{equation*} \begin{aligned} \sum_{k=1}^{i} C[k]\,2^{k-1} + carry^{\prime} \cdot 2^{i} &= \sum_{k=1}^{i-1} C[k]\,2^{k-1} + (C[i] + 2\,carry^{\prime})\,2^{i-1} \\ &= \sum_{k=1}^{i-1} C[k]\,2^{k-1} + carry \cdot 2^{i-1} + (A[i]+B[i])\,2^{i-1} \\ &= \sum_{k=1}^{i} (A[k]+B[k])\,2^{k-1}, \end{aligned} \end{equation*}

which is the invariant for \(i+1\). Termination: the loop exits with \(i = n+1\), and line 7 sets \(C[n+1] = carry\), so the invariant becomes \(\sum_{k=1}^{n+1} C[k]\,2^{k-1} = a + b\), with every \(C[k] \in \{0,1\}\).

\(f(n) = n^3/1000 - 100n^2 - 100n + 3 = \Theta(n^3)\). For \(n \ge 1\) we have \(-100n^2 - 100n + 3 < 0\), and for \(n \ge 400000\) we have \(n^3/2000 \ge 200n^2 \ge 100n^2 + 100n\), so

\begin{equation*} \begin{aligned} f(n) &\le \frac{n^3}{1000} && (n \ge 1),\\ f(n) &\ge \frac{n^3}{1000} - \frac{n^3}{2000} = \frac{n^3}{2000} && (n \ge 400000). \end{aligned} \end{equation*}

Hence \(c_1 n^3 \le f(n) \le c_2 n^3\) for all \(n \ge n_0\) with \(c_1 = 1/2000\), \(c_2 = 1/1000\), \(n_0 = 400000\), and so \(f(n) = \Theta(n^3)\).

Selection sort runs in \(\Theta(n^2)\) time in both the best and the worst case. Here is the pseudocode:

SELECTION-SORT(A)
1  n = A.length
2  for i = 1 to n - 1
3      smallest = i
4      for j = i + 1 to n
5          if A[j] < A[smallest]
6              smallest = j
7      exchange A[i] with A[smallest]

Loop invariant: at the start of each iteration of the for loop of lines 2–7, the subarray \(A[1..i-1]\) holds the \(i-1\) smallest elements of \(A\) in sorted order, and \(A[i..n]\) holds the remaining elements.

Initialization: for \(i = 1\) the subarray \(A[1..0]\) is empty.

Maintenance: lines 3–6 leave \(smallest\) as the index of a minimum of \(A[i..n]\) (this is the inner loop’s own invariant: \(A[smallest] = \min A[i..j-1]\)). By the invariant, that minimum is at least every element of \(A[1..i-1]\) and is an \(i\)-th smallest element of \(A\). Line 7 puts it in \(A[i]\), so \(A[1..i]\) holds the \(i\) smallest elements in sorted order. Incrementing \(i\) restores the invariant.

Termination: the loop stops when \(i = n\). Then \(A[1..n-1]\) holds the \(n-1\) smallest elements in sorted order, so \(A[n]\) is a largest element and the whole of \(A[1..n]\) is sorted.

That last step is why only \(n-1\) passes are needed. Once the \(n-1\) smallest elements are in place, the one element left over is forced to be the largest, and it already sits in \(A[n]\). A pass with \(i = n\) would compare nothing and swap \(A[n]\) with itself.

Running time: the inner loop body (lines 5–6) runs \(n - i\) times on pass \(i\) for every input, because the loop bounds do not depend on the data. Only the assignment in line 6 depends on the input, and it costs at most a constant per test. So the total time is

\begin{equation*} \Theta\Bigl(n + \sum_{i=1}^{n-1} (n-i)\Bigr) = \Theta\Bigl(n + \frac{n(n-1)}{2}\Bigr) = \Theta(n^2) \end{equation*}

for every input, so the best and worst cases are both \(\Theta(n^2)\).

On average \((n+1)/2\) elements are checked, in the worst case \(n\), and both running times are \(\Theta(n)\). Assume the target is the element in position \(k\), where \(k\) is uniform on \(\{1,\dots,n\}\) and the elements are distinct (if there are repeats, take \(k\) to be the position of the first occurrence). Then the search checks exactly \(A[1],\dots,A[k]\), which is \(k\) elements, so the expected number checked is

\begin{equation*} \mathrm{E}[k] = \frac{1}{n}\sum_{k=1}^{n} k = \frac{1}{n}\cdot\frac{n(n+1)}{2} = \frac{n+1}{2}. \end{equation*}

In the worst case all \(n\) elements are checked: this happens when \(v = A[n]\) only, or when \(v\) is not in the array at all.

Each check costs a constant amount of time (a loop test, a comparison and an increment), and there is a constant amount of overhead outside the loop. So the running time on an input that checks \(k\) elements is \(a k + b\) for constants \(a > 0\) and \(b\). The average-case time is therefore

\begin{equation*} a\cdot\frac{n+1}{2} + b = \Theta(n), \end{equation*}

and the worst-case time is \(a n + b = \Theta(n)\).

Add a special case at the start. The algorithm first checks whether the input is one particular input (or belongs to a class of inputs) whose answer is precomputed or cheap to get. If it is, the algorithm returns that answer right away; if not, it runs the original algorithm.

For example, a sorting algorithm can first scan \(A\) in \(\Theta(n)\) time to test whether \(A\) is already sorted, and return immediately if so. Its best-case running time drops to \(\Theta(n)\). The scan adds only \(O(n)\) to every other input, so the worst case does not change asymptotically.

Reading bottom-up as in Figure 2.4, the eight singletons are merged in pairs, then the pairs into runs of four, then the two runs into the sorted output:

Each run in a row is the MERGE of two adjacent runs in the row above it.

Compare heads only while both arrays are nonempty, then drain the leftover array:

MERGE(A, p, q, r)
 1  n1 = q - p + 1
 2  n2 = r - q
 3  let L[1..n1] and R[1..n2] be new arrays
 4  for i = 1 to n1
 5      L[i] = A[p + i - 1]
 6  for j = 1 to n2
 7      R[j] = A[q + j]
 8  i = 1
 9  j = 1
10  k = p
11  while i <= n1 and j <= n2
12      if L[i] <= R[j]
13          A[k] = L[i];  i = i + 1
14      else A[k] = R[j];  j = j + 1
15      k = k + 1
16  while i <= n1
17      A[k] = L[i];  i = i + 1;  k = k + 1
18  while j <= n2
19      A[k] = R[j];  j = j + 1;  k = k + 1

While both \(i \le n_1\) and \(j \le n_2\), line 12 makes exactly the comparison of the sentinel version, so the loop invariant of Section 2.3.1 holds through lines 11–15. When one array, say \(R\), is exhausted, the uncopied \(L[i \mathbin{..} n_1]\) is sorted and, by the invariant, no smaller than anything in \(A[p \mathbin{..} k-1]\); lines 16–17 append it, and line 18 does nothing (symmetrically if \(L\) is exhausted). Each loop iteration copies one element, so the running time is still \(\Theta(n)\) with \(n = r-p+1\).

Induct on \(k \ge 1\), where \(n = 2^k\). Base case \(k=1\): \(T(2) = 2 = 2\lg 2\). Inductive step: for \(k > 1\) assume \(T(2^{k-1}) = 2^{k-1}(k-1)\), i.e. \(T(n/2) = (n/2)\lg(n/2)\). Then

\begin{equation*} \begin{aligned} T(n) &= 2T(n/2) + n = 2 \cdot \frac{n}{2}\lg\frac{n}{2} + n \\ &= n(\lg n - 1) + n = n \lg n . \end{aligned} \end{equation*}

Hence \(T(n) = n\lg n\) for every \(n = 2^k\), \(k \ge 1\).

\begin{equation*} T(n) = \begin{cases} \Theta(1) & \text{if } n = 1, \\ T(n-1) + \Theta(n) & \text{if } n > 1. \end{cases} \end{equation*}

The recursive call costs \(T(n-1)\) in the worst case, and inserting \(A[n]\) into the sorted \(A[1 \mathbin{..} n-1]\) is one pass of the while loop of lines 5–7 of INSERTION-SORT, which shifts up to \(n-1\) elements, so it costs \(O(n)\), and \(\Theta(n)\) when \(A[n]\) is smaller than all of \(A[1 \mathbin{..} n-1]\). Both worst cases occur together on a reverse-sorted input: its prefix \(A[1 \mathbin{..} n-1]\) is itself reverse sorted, and \(A[n]\) is its minimum.

Keep a window \(A[low \mathbin{..} high]\) that must contain \(v\) if \(v\) occurs at all, and halve it by probing its midpoint:

BINARY-SEARCH(A, v)
 1  low = 1
 2  high = A.length
 3  while low <= high
 4      mid = floor((low + high) / 2)
 5      if A[mid] == v
 6          return mid
 7      elseif A[mid] < v
 8          low = mid + 1
 9      else high = mid - 1
10  return NIL

Correctness. Loop invariant: at the start of each iteration, if \(v\) occurs in \(A\) then it occurs in \(A[low \mathbin{..} high]\). Initialization: the window is all of \(A\). Maintenance: if \(A[mid] < v\) then, \(A\) being sorted, \(A[low \mathbin{..} mid] < v\), so \(v\) can only lie in \(A[mid+1 \mathbin{..} high]\); symmetrically if \(A[mid] > v\). Termination: a return in line 6 is correct since \(A[mid] = v\); otherwise the loop exits with \(low > high\), the window is empty, and by the invariant \(v \notin A\), so NIL is correct.

Running time. Each iteration costs \(\Theta(1)\). Let \(m = high - low + 1\) be the window size. Since \(mid - low = \lfloor (m-1)/2 \rfloor\) and \(high - mid = \lceil (m-1)/2 \rceil\), an iteration that does not return leaves a window of size at most \(\lceil (m-1)/2 \rceil = \lfloor m/2 \rfloor\). By induction, after \(t\) iterations the size is at most \(\lfloor n/2^t \rfloor\), which is \(0\) once \(t = \lfloor \lg n \rfloor + 1\); so there are at most \(\lfloor \lg n \rfloor + 1\) iterations and the running time is \(O(\lg n)\). Conversely, if \(v\) exceeds every element, line 8 runs every time and the size goes from \(m\) to exactly \(\lfloor m/2 \rfloor\), so exactly \(\lfloor \lg n \rfloor + 1\) iterations are executed. Hence the worst case is \(\Theta(\lg n)\).

No: binary search finds the insertion point with \(O(\lg j)\) comparisons, but inserting the key still requires shifting every element of \(A[1 \mathbin{..} j-1]\) larger than it one place to the right, and that is \(j-1\) moves in the worst case. On a reverse-sorted input every key \(A[j]\) is smaller than all of \(A[1 \mathbin{..} j-1]\), so iteration \(j\) performs \(j-1\) shifts regardless of how the position was found, and the total work is at least

\begin{equation*} \sum_{j=2}^{n} (j-1) = \frac{n(n-1)}{2} = \Theta(n^2). \end{equation*}

Each iteration still costs \(O(j)\), so the worst case stays \(\Theta(n^2)\).

Sort \(S\) with merge sort, then sweep two indices inward from the ends of the sorted array: if the current pair sums to less than \(x\), advance the left index; if more, retreat the right index; stop on equality or when the indices meet.

SUM-EXISTS(S, x)
1  let A[1..n] hold the elements of S
2  MERGE-SORT(A, 1, n)
3  i = 1
4  j = n
5  while i < j
6      if A[i] + A[j] == x
7          return TRUE
8      elseif A[i] + A[j] < x
9          i = i + 1
10     else j = j - 1
11 return FALSE

Correctness rests on the loop invariant: at the start of each iteration of lines 5–10, every pair of indices \(p < q\) with \(A[p] + A[q] = x\) satisfies \(i \le p < q \le j\).

Initialization: \(i = 1\) and \(j = n\), so every pair \(p < q\) lies in \([i, j]\).

Maintenance: if line 7 is not reached, suppose \(A[i] + A[j] < x\). Since \(A\) is sorted, for every \(q\) with \(i < q \le j\),

\begin{equation*} A[i] + A[q] \le A[i] + A[j] < x, \end{equation*}

so no solution pair has \(p = i\), and incrementing \(i\) preserves the invariant. Symmetrically, if \(A[i] + A[j] > x\) then \(A[p] + A[j] \ge A[i] + A[j] > x\) for all \(i \le p < j\), so no solution pair has \(q = j\), and decrementing \(j\) preserves it.

Termination: a TRUE return at line 7 exhibits the pair \(A[i], A[j]\) with \(i < j\). Otherwise the loop ends with \(i \ge j\), so no indices \(p < q\) satisfy \(i \le p < q \le j\); by the invariant there is no solution pair, and FALSE is correct.

Running time: line 2 costs \(\Theta(n \lg n)\) (Section 2.3.2). Each iteration of the loop decreases \(j - i\) by \(1\), and \(j - i = n - 1\) at the start, so there are at most \(n - 1\) iterations of \(O(1)\) work each. The total is \(\Theta(n \lg n) + O(n) = \Theta(n \lg n)\).

(a) By Section 2.2, insertion sort on \(k\) elements takes worst-case time \(ak^2+bk+c=\Theta(k^2)\), so there are constants \(0<c_1\le c_2\) with \(c_1k^2\le T_{\mathrm{ins}}(k)\le c_2k^2\) for all \(k\ge 1\). Hence the \(n/k\) sublists cost

\begin{equation*} \frac{n}{k}\,T_{\mathrm{ins}}(k)\in\Big[\frac{n}{k}c_1k^2,\ \frac{n}{k}c_2k^2\Big]=[c_1nk,\ c_2nk], \end{equation*}

which is \(\Theta(nk)\), the upper bound being attained when every sublist is reverse sorted.

(b) Run merge sort but stop the recursion at subarrays of length \(k\) (assume, as in Section 2.3.2, that \(n/k\) is a power of \(2\)):

COARSE-MERGE-SORT(A, p, r, k)
1  if r - p + 1 <= k
2      INSERTION-SORT(A, p, r)      // sort A[p..r] in place
3  else q = floor((p + r)/2)
4      COARSE-MERGE-SORT(A, p, q, k)
5      COARSE-MERGE-SORT(A, q + 1, r, k)
6      MERGE(A, p, q, r)

Correctness is that of MERGE-SORT (Section 2.3.1), with sorted leaves supplied by insertion sort. For the cost of the merging, use the recursion tree of Figure 2.5 truncated where subproblems reach size \(k\): level \(d\) has \(2^d\) subarrays of size \(n/2^d\), and MERGE on a subarray of length \(m\) takes \(\Theta(m)\) time (it copies all \(m\) elements and does \(m\) iterations of lines 12–17), so every level costs \(\Theta(n)\). The leaves sit at depth \(\lg(n/k)\), so there are \(\lg(n/k)\) levels of merging and

\begin{equation*} T_{\mathrm{merge}}(n)=\lg(n/k)\cdot\Theta(n)=\Theta\big(n\lg(n/k)\big). \end{equation*}

(c) The largest such \(k\) is \(k=\Theta(\lg n)\). If \(k=O(\lg n)\), then, using \(\lg k\le k\),

\begin{equation*} \begin{aligned} nk+n\lg(n/k)&\le O(n\lg n)+n\lg n=O(n\lg n),\\ nk+n\lg(n/k)&=n\lg n+n(k-\lg k)\ge n\lg n, \end{aligned} \end{equation*}

so the running time is \(\Theta(n\lg n)\), the same as merge sort. If \(k\) is not \(O(\lg n)\), then \(nk\) is not \(O(n\lg n)\) and the modified algorithm is asymptotically slower. So \(k=\Theta(\lg n)\) is the largest admissible order of growth.

(d) Choose \(k\) as the largest sublist length on which insertion sort is still faster than merge sort on the actual machine, i.e. the crossover point where \(c_{\mathrm{ins}}k^2\) overtakes \(c_{\mathrm{merge}}k\lg k\); it is found by timing both sorts on inputs of increasing size, and is a small constant (typically tens of elements).

(a) We must also prove that \(A^{\prime}\) is a permutation of the input \(A\), i.e. that \(A^{\prime}\) consists of exactly the original elements. (Here this is immediate: the array is modified only by the exchanges of line 4, and each exchange permutes its entries.)

(b) Invariant: at the start of each iteration of the for loop of lines 2–4, the subarray \(A[j\,..\,n]\) consists of the elements originally in \(A[j\,..\,n]\) before the inner loop began, possibly reordered, \(A[1\,..\,j-1]\) is unchanged, and \(A[j]\) is the smallest element of \(A[j\,..\,n]\).

Initialization: before the first iteration \(j=n\); nothing has moved, and \(A[n\,..\,n]\) has the single element \(A[n]\), which is its own minimum.

Maintenance: by the invariant \(A[j]=\min A[j\,..\,n]\). Lines 3–4 exchange \(A[j]\) and \(A[j-1]\) exactly when \(A[j]<A[j-1]\), so afterwards

\begin{equation*} A[j-1]=\min(A[j-1],A[j])=\min A[j-1\,..\,n], \end{equation*}

and the only change was a swap inside \(A[j-1\,..\,n]\), leaving \(A[1\,..\,j-2]\) unchanged. Decrementing \(j\) re-establishes the invariant.

Termination: the loop ends when \(j=i\). Then \(A[i\,..\,n]\) is a permutation of its contents before the inner loop, \(A[1\,..\,i-1]\) is unchanged, and \(A[i]\) is the smallest element of \(A[i\,..\,n]\).

(c) Invariant: at the start of each iteration of the for loop of lines 1–4, \(A\) is a permutation of the input, \(A[1\,..\,i-1]\) is sorted, and every element of \(A[1\,..\,i-1]\) is \(\le\) every element of \(A[i\,..\,n]\).

Initialization: \(i=1\), so \(A[1\,..\,0]\) is empty and the claims hold vacuously.

Maintenance: by the termination condition of (b), the inner loop leaves \(A[1\,..\,i-1]\) alone, permutes \(A[i\,..\,n]\), and puts \(A[i]=\min A[i\,..\,n]\). Since the permuted block has the same elements, every element of \(A[1\,..\,i-1]\) is still \(\le\) every element of \(A[i\,..\,n]\); in particular \(A[i-1]\le A[i]\), so \(A[1\,..\,i]\) is sorted, and \(A[i]\le A[i+1\,..\,n]\) by minimality. Incrementing \(i\) re-establishes the invariant.

Termination: the loop ends at \(i=n\). Then \(A[1\,..\,n-1]\) is sorted and each of its elements is \(\le A[n]\), so

\begin{equation*} A[1]\le A[2]\le\cdots\le A[n-1]\le A[n], \end{equation*}

which is (2.3); with the permutation clause this proves that BUBBLESORT sorts.

(d) Bubblesort runs in \(\Theta(n^2)\) time in the worst case, and in fact on every input. Each test in line 3 costs \(\Theta(1)\), and that test is executed

\begin{equation*} \sum_{i=1}^{n-1}(n-i)=\frac{n(n-1)}{2} \end{equation*}

times whatever the input, while the loop overhead and line 4 cost at most a constant factor more. Insertion sort has the same \(\Theta(n^2)\) worst case (Section 2.2), but its best case is \(\Theta(n)\) (already sorted input), whereas bubblesort never does better than \(\Theta(n^2)\); on every input bubblesort makes at least as many comparisons as insertion sort, so in practice it is the slower of the two.

(a) \(\Theta(n)\): the loop body runs \(n+1\) times (for \(i = n, n-1, \ldots, 0\)), each time doing one multiplication, one addition and one assignment in constant time, and the loop test runs \(n+2\) times.

(b) The naive algorithm takes \(\Theta(n^2)\) time, a factor of \(n\) slower than Horner’s rule.

NAIVE-POLY-EVAL(a, n, x)
1  y = 0
2  for k = 0 to n
3      t = 1
4      for j = 1 to k
5          t = t * x
6      y = y + a_k * t
7  return y

For each \(k\) the inner loop of lines 4–5 performs \(k\) multiplications, so the total work is

\begin{equation*} \sum_{k=0}^{n} \Theta(k + 1) = \Theta\!\left(\frac{(n+1)(n+2)}{2}\right) = \Theta(n^2). \end{equation*}

Horner’s rule needs only \(n+1\) multiplications in total.

(c) Initialization: before the first iteration \(i = n\), so the sum runs from \(k = 0\) to \(-1\), is empty, and equals \(0 = y\).

Maintenance: suppose the invariant holds at the start of the iteration with index \(i\). Line 3 sets

\begin{equation*} \begin{aligned} y^{\prime} = a_i + x y &= a_i + \sum_{k=0}^{n-i-1} a_{k+i+1} x^{k+1} \\ &= a_i + \sum_{k=1}^{n-i} a_{k+i} x^{k} = \sum_{k=0}^{n-i} a_{k+i} x^{k}, \end{aligned} \end{equation*}

and the next iteration has index \(i^{\prime} = i - 1\), for which \(n - (i^{\prime}+1) = n - i\) and \(a_{k+i^{\prime}+1} = a_{k+i}\). So the invariant holds at the start of the next iteration.

Termination: the loop ends when \(i = -1\). Substituting \(i = -1\) into the invariant gives

\begin{equation*} y = \sum_{k=0}^{n} a_{k} x^k . \end{equation*}

(d) The loop runs exactly \(n+1\) iterations and so terminates, and by (c) the final value of \(y\) is \(\sum_{k=0}^{n} a_k x^k = P(x)\). Hence the fragment correctly evaluates the polynomial with coefficients \(a_0, a_1, \ldots, a_n\) at \(x\).

(a) The inversions are \((1,5), (2,5), (3,4), (3,5), (4,5)\), i.e. the value pairs \(2 > 1\), \(3 > 1\), \(8 > 6\), \(8 > 1\), \(6 > 1\).

(b) The reversed array \(\langle n, n-1, \ldots, 1 \rangle\), with \(\binom{n}{2} = n(n-1)/2\) inversions. Any array has at most \(\binom{n}{2}\) pairs \(i < j\), and in the reversed array \(A[i] = n + 1 - i > n + 1 - j = A[j]\) for every such pair, so all of them are inversions. It is the unique maximizer: if every pair is inverted, \(A\) is strictly decreasing, hence the reversed array.

(c) Insertion sort runs in \(\Theta(n + I)\) time on an input with \(I\) inversions: the while loop of lines 5–7 of INSERTION-SORT (Section 2.1) executes its body exactly \(I\) times in total.

Fix an iteration \(j\) of the outer loop with key \(= A[j]\) (original position \(j\)). By the loop invariant of Section 2.1, \(A[1 \ldots j-1]\) then holds the original elements of positions \(1, \ldots, j-1\) in sorted order. The while loop shifts right exactly those of them that exceed the key, and stops at the first one that does not. So the number of shifts in iteration \(j\) is

\begin{equation*} \#\{\, i < j : A[i] > A[j] \,\} , \end{equation*}

the number of inversions whose second index is \(j\). Summing over \(j = 2, \ldots, n\) counts every inversion once, giving \(I\) shifts in total. Each outer iteration costs \(\Theta(1)\) besides its shifts, and each shift costs \(\Theta(1)\), so the running time is \(\Theta(n) + \Theta(I) = \Theta(n + I)\).

(d) Run merge sort, and in each merge add the number of cross inversions: whenever an element of the right half is copied out, every element still waiting in the left half is larger than it and forms an inversion with it.

COUNT-INVERSIONS(A, p, r)
1  if p >= r
2      return 0
3  q = floor((p + r) / 2)
4  c = COUNT-INVERSIONS(A, p, q) + COUNT-INVERSIONS(A, q + 1, r)
5  return c + MERGE-INVERSIONS(A, p, q, r)

MERGE-INVERSIONS(A, p, q, r)
1  n1 = q - p + 1
2  copy A[p..q] into L[1..n1] and A[q+1..r] into R[1..r-q]
3  L[n1 + 1] = infinity, R[r - q + 1] = infinity
4  i = 1, j = 1, c = 0
5  for k = p to r
6      if L[i] <= R[j]
7          A[k] = L[i], i = i + 1
8      else A[k] = R[j], j = j + 1
9          c = c + n1 - i + 1
10 return c

Correctness, by induction on \(r - p\): COUNT-INVERSIONS\((A, p, r)\) returns the number of inversions of \(A[p \ldots r]\) and leaves it sorted. A subarray of length at most \(1\) has no inversions. Otherwise, each inversion \((s, t)\) of \(A[p \ldots r]\) has either \(s < t \le q\), or \(q < s < t\), or \(s \le q < t\); by the inductive hypothesis line 4 counts the first two kinds and sorts both halves. The number of cross pairs, i.e. pairs \((x, y)\) with \(x\) in the left half, \(y\) in the right half and \(x > y\), depends only on the sets of elements in the two halves, so sorting them does not change it. MERGE-INVERSIONS is MERGE (Section 2.3.1) with line 9 added, so it produces the sorted output. When an element \(y = R[j]\) is copied at line 8, the elements copied before it are smaller and those copied after it are larger (distinct elements, sorted output). Hence the left-half elements larger than \(y\) are exactly the not-yet-copied \(L[i \ldots n_1]\), which number \(n_1 - i + 1\) (the sentinel is never counted, as \(i \le n_1 + 1\)). Summing over all \(y\), line 9 accumulates exactly the cross inversions, which completes the induction.

Running time: MERGE-INVERSIONS does \(\Theta(1)\) extra work per iteration on top of MERGE, so it runs in \(\Theta(n)\) time, and the running time satisfies the merge-sort recurrence \(T(n) = 2T(n/2) + \Theta(n)\), whose solution is \(\Theta(n \lg n)\) (Section 2.3.2).

Take \(c_1 = 1/2\), \(c_2 = 1\). Since \(f\) and \(g\) are asymptotically nonnegative, there is an \(n_0\) with \(f(n) \ge 0\) and \(g(n) \ge 0\) for all \(n \ge n_0\). For such \(n\), the maximum of two nonnegative numbers is at least their average and at most their sum:

\begin{equation*} 0 \le \tfrac12\bigl(f(n) + g(n)\bigr) \le \max(f(n), g(n)) \le f(n) + g(n). \end{equation*}

This is exactly the definition of \(\max(f(n), g(n)) = \Theta(f(n) + g(n))\) with constants \(c_1 = 1/2\), \(c_2 = 1\) and \(n_0\).

Take \(c_1 = 2^{-b}\), \(c_2 = 2^b\) and \(n_0 = \max(1, 2|a|)\). For \(n \ge n_0\) we have \(|a| \le n/2\), so

\begin{equation*} \tfrac{n}{2} \le n - |a| \le n + a \le n + |a| \le \tfrac{3n}{2} \le 2n . \end{equation*}

All these quantities are positive and \(x \mapsto x^b\) is increasing on \((0, \infty)\) because \(b > 0\). Raising to the power \(b\) therefore gives, for all \(n \ge n_0\),

\begin{equation*} 0 \le 2^{-b} n^b \le (n + a)^b \le 2^b n^b , \end{equation*}

which is the definition of \((n + a)^b = \Theta(n^b)\).

Because every running time is at least some function in \(O(n^2)\), so the statement carries no information. \(O\)-notation only gives an upper bound: \(O(n^2)\) contains every asymptotically nonnegative function that eventually lies below \(cn^2\), including the zero function \(f(n) = 0\). Read literally, the claim says \(T(n) \ge f(n)\) for some \(f(n) = O(n^2)\). With \(f = 0\),

\begin{equation*} T(n) \ge 0 = f(n) \quad \text{for all } n , \end{equation*}

which holds for every running time \(T\).

Yes to the first, no to the second.

(i) For all \(n \ge 1\) we have \(0 \le 2^{n+1} = 2 \cdot 2^n\), so \(2^{n+1} = O(2^n)\) with \(c = 2\), \(n_0 = 1\).

(ii) Suppose \(2^{2n} = O(2^n)\), so there are \(c > 0\) and \(n_0\) with \(2^{2n} \le c\, 2^n\) for all \(n \ge n_0\). Dividing by \(2^n > 0\),

\begin{equation*} 2^n \le c \quad \text{for all } n \ge n_0 , \end{equation*}

which is false for any \(n \ge \max(n_0, \log_2 c + 1)\). So \(2^{2n} \ne O(2^n)\).

Both directions just combine or split the constants in the definitions of \(\Theta\), \(O\) and \(\Omega\).

(\(\Rightarrow\)) Suppose \(f(n) = \Theta(g(n))\), so there are \(c_1, c_2, n_0 > 0\) with

\begin{equation*} 0 \le c_1 g(n) \le f(n) \le c_2 g(n) \quad \text{for all } n \ge n_0 . \end{equation*}

Then \(0 \le f(n) \le c_2 g(n)\) for \(n \ge n_0\), which gives \(f(n) = O(g(n))\) with constants \(c_2, n_0\). Also \(0 \le c_1 g(n) \le f(n)\) for \(n \ge n_0\), which gives \(f(n) = \Omega(g(n))\) with constants \(c_1, n_0\).

(\(\Leftarrow\)) Suppose there are \(c_2, n_2 > 0\) with \(0 \le f(n) \le c_2 g(n)\) for all \(n \ge n_2\) (so \(f = O(g)\)), and \(c_1, n_1 > 0\) with \(0 \le c_1 g(n) \le f(n)\) for all \(n \ge n_1\) (so \(f = \Omega(g)\)). Let \(n_0 = \max(n_1, n_2)\). For every \(n \ge n_0\) both chains hold, so

\begin{equation*} 0 \le c_1 g(n) \le f(n) \le c_2 g(n) \quad \text{for all } n \ge n_0 . \end{equation*}

Hence \(f(n) = \Theta(g(n))\).

Sandwich every running time between the best and worst cases. With \(T(x)\) the time on input \(x\), \(W(n) = \max_{|x| = n} T(x)\) and \(B(n) = \min_{|x| = n} T(x)\), we have \(B(n) \le T(x) \le W(n)\) for \(|x| = n\); “the running time is \(\Theta(g(n))\)” means there are \(c_1, c_2, n_0 > 0\) with \(0 \le c_1 g(n) \le T(x) \le c_2 g(n)\) for every input \(x\) of size \(n \ge n_0\).

(\(\Leftarrow\)) Suppose \(W(n) \le c_2 g(n)\) for \(n \ge n_2\) and \(0 \le c_1 g(n) \le B(n)\) for \(n \ge n_1\). Then for every input \(x\) of size \(n \ge \max(n_1, n_2)\),

\begin{equation*} 0 \le c_1 g(n) \le B(n) \le T(x) \le W(n) \le c_2 g(n), \end{equation*}

so the running time is \(\Theta(g(n))\).

(\(\Rightarrow\)) If every input of size \(n \ge n_0\) satisfies \(0 \le c_1 g(n) \le T(x) \le c_2 g(n)\), then so do the particular inputs that achieve \(W(n)\) and \(B(n)\). Hence \(0 \le W(n) \le c_2 g(n)\) and \(0 \le c_1 g(n) \le B(n)\) for \(n \ge n_0\), that is, \(W(n) = O(g(n))\) and \(B(n) = \Omega(g(n))\).

Suppose some \(f(n)\) lay in both sets. Apply the definitions of \(o\) and \(\omega\) (Section 3.1) with the single constant \(c = 1\): there are \(n_0, n_1\) with

\begin{equation*} \begin{aligned} 0 \le f(n) &< g(n) \quad \text{for all } n \ge n_0, \\ 0 \le g(n) &< f(n) \quad \text{for all } n \ge n_1 . \end{aligned} \end{equation*}

For \(n = \max(n_0, n_1)\) both hold, giving \(f(n) < g(n) < f(n)\), a contradiction. Hence \(o(g(n)) \cap \omega(g(n)) = \emptyset\).

Keep the quantifiers of \(O(g(n,m))\) and swap in the inequalities of the one-parameter \(\Omega\) and \(\Theta\) (Section 3.1):

\begin{equation*} \begin{aligned} \Omega(g(n,m)) = \{ f(n,m) :\ & \text{there exist positive constants } c, n_0, m_0 \\ & \text{such that } 0 \le c\, g(n,m) \le f(n,m) \\ & \text{for all } n \ge n_0 \text{ or } m \ge m_0 \} , \end{aligned} \end{equation*}

\begin{equation*} \begin{aligned} \Theta(g(n,m)) = \{ f(n,m) :\ & \text{there exist positive constants } c_1, c_2, n_0, m_0 \\ & \text{such that } 0 \le c_1 g(n,m) \le f(n,m) \le c_2 g(n,m) \\ & \text{for all } n \ge n_0 \text{ or } m \ge m_0 \} . \end{aligned} \end{equation*}

Fix \(m \le n\), so that \(f(m)\le f(n)\) and \(g(m)\le g(n)\).

Sum: adding the two inequalities gives \(f(m)+g(m)\le f(n)+g(n)\).

Composition: \(g(m)\le g(n)\), and applying the monotonically increasing \(f\) to this inequality gives \(f(g(m))\le f(g(n))\).

Product: since \(f(m)\ge 0\) and \(g(n)\ge 0\),

\begin{equation*} f(m)\,g(m) \;\le\; f(m)\,g(n) \;\le\; f(n)\,g(n), \end{equation*}

where the first step multiplies \(g(m)\le g(n)\) by \(f(m)\ge 0\) and the second multiplies \(f(m)\le f(n)\) by \(g(n)\ge 0\). \(\blacksquare\)

Write \(a = b^{\log_b a}\) (the first identity before (3.15)) and use \((x^m)^n = x^{mn} = (x^n)^m\):

\begin{equation*} a^{\log_b c} = \bigl(b^{\log_b a}\bigr)^{\log_b c} = b^{(\log_b a)(\log_b c)} = \bigl(b^{\log_b c}\bigr)^{\log_b a} = c^{\log_b a}. \qquad \blacksquare \end{equation*}

Upper bound: \(n!\le n^n\) (each of the \(n\) factors is at most \(n\)), so \(\lg(n!)\le n\lg n\) for all \(n\ge 1\).

Lower bound: by (3.20), with \(\alpha_n>0\) by (3.21) and \(\sqrt{2\pi n}\ge 1\), we get \(n!\ge (n/e)^n\), hence

\begin{equation*} \lg(n!) \;\ge\; n\lg n - n\lg e \;\ge\; \tfrac12\, n\lg n \quad\text{whenever } \lg n \ge 2\lg e,\ \text{i.e. } n\ge e^2, \end{equation*}

so the bound holds with constant \(\tfrac12\) for all \(n\ge 8\). Together, \(\tfrac12 n\lg n\le \lg(n!)\le n\lg n\) for \(n\ge 8\), which is (3.19).

\(n! = o(n^n)\): every factor \(k/n\) below is at most \(1\), and the first is \(1/n\), so

\begin{equation*} 0 < \frac{n!}{n^n} = \prod_{k=1}^{n}\frac{k}{n} \le \frac1n , \end{equation*}

which is below any given \(c>0\) once \(n>1/c\), so \(n! = o(n^n)\).

\(n! = \omega(2^n)\): for \(n\ge 2\), every factor \(k/2\) with \(k\ge 2\) is at least \(1\), so

\begin{equation*} \frac{n!}{2^n} = \frac12\prod_{k=2}^{n}\frac{k}{2} \ge \frac12\cdot\frac n2 = \frac n4 , \end{equation*}

which exceeds any given \(c>0\) once \(n>4c\), so \(n! = \omega(2^n)\). \(\blacksquare\)

\(\lceil\lg n\rceil!\) is not polynomially bounded; \(\lceil\lg\lg n\rceil!\) is, in fact \(\lceil\lg\lg n\rceil!\le n\) for all \(n\ge 2\).

(i) Suppose \(\lceil\lg n\rceil!\le c\,n^k\) for all \(n\ge n_0\), with constants \(c,k>0\). Take \(n=2^m\), so \(\lceil\lg n\rceil = m\). The \(m-\lceil m/2\rceil+1\ge m/2\) factors \(k^{\prime}\in[\lceil m/2\rceil, m]\) of \(m!\) are each at least \(m/2\), so for \(m\ge 2\), \(m!\ge (m/2)^{m/2}\), and

\begin{equation*} \frac{\lceil\lg n\rceil!}{n^k} = \frac{m!}{2^{km}} \ge \frac{(m/2)^{m/2}}{(2^{2k})^{m/2}} = \Bigl(\frac{m}{2^{2k+1}}\Bigr)^{m/2} \longrightarrow \infty \quad (m\to\infty), \end{equation*}

contradicting \(m!/2^{km}\le c\) for all large \(m\). Hence \(\lceil\lg n\rceil!\ne O(n^k)\) for every \(k\).

(ii) First, \(t!\le 2^{2^{t-1}}\) for every integer \(t\ge 1\), by induction: \(1!=1\le 2\), and if it holds for \(t\) then, since \(t+1\le 2^{2^{t-1}}\) for \(t\ge 1\),

\begin{equation*} (t+1)! = (t+1)\,t! \le 2^{2^{t-1}}\cdot 2^{2^{t-1}} = 2^{2^t}. \end{equation*}

Now let \(n\ge 3\) and \(t=\lceil\lg\lg n\rceil\ge 1\). Then \(\lg\lg n > t-1\), i.e. \(n > 2^{2^{t-1}}\), so

\begin{equation*} \lceil\lg\lg n\rceil! = t! \le 2^{2^{t-1}} < n , \end{equation*}

and for \(n=2\) we have \(\lceil\lg\lg 2\rceil! = 0! = 1\le 2\). Thus \(\lceil\lg\lg n\rceil! = O(n)\), which is polynomially bounded. \(\blacksquare\)

\(\lg^{*}(\lg n)\) is asymptotically larger: \(\lg(\lg^{*} n) = o(\lg^{*}(\lg n))\).

For \(n>1\), \(\lg^{(i)}(\lg n) = \lg^{(i+1)} n\) for every \(i\), so by the definition of \(\lg^{*}\), \(\lg^{*}(\lg n) = \lg^{*} n - 1\). Writing \(x=\lg^{*} n\), which tends to \(\infty\) with \(n\) (it is unbounded and monotonically increasing),

\begin{equation*} \lim_{n\to\infty}\frac{\lg(\lg^{*} n)}{\lg^{*}(\lg n)} = \lim_{x\to\infty}\frac{\lg x}{x-1} = 0, \end{equation*}

the last limit because \(\lg x = o(x)\) (the polylogarithm-versus-polynomial limit after (3.17), with \(a=b=1\)) and \(x-1\ge x/2\) for \(x\ge 2\). By (3.1), \(\lg(\lg^{*} n) = o(\lg^{*}(\lg n))\). \(\blacksquare\)

Squaring directly, with the sign \(\pm\) taken as \(+\) for \(\phi\) and \(-\) for \(\hat\phi\):

\begin{equation*} \Bigl(\frac{1\pm\sqrt5}{2}\Bigr)^2 = \frac{1\pm 2\sqrt5+5}{4} = \frac{3\pm\sqrt5}{2} = \frac{1\pm\sqrt5}{2} + 1 . \end{equation*}

Hence \(\phi^2=\phi+1\) and \(\hat\phi^2=\hat\phi+1\). \(\blacksquare\)

Induct on \(i\), with base cases \(i = 0, 1\): since \(\phi - \hat\phi = \sqrt5\) by (3.24),

\begin{equation*} \frac{\phi^0 - \hat\phi^0}{\sqrt5} = 0 = F_0, \qquad \frac{\phi - \hat\phi}{\sqrt5} = 1 = F_1 . \end{equation*}

For \(i \ge 2\), assume the formula for \(i-1\) and \(i-2\). Both \(x = \phi\) and \(x = \hat\phi\) satisfy \(x^2 = x + 1\) (Exercise 3.2-6), hence \(x^{i-1} + x^{i-2} = x^i\), and by (3.22)

\begin{equation*} \begin{aligned} F_i = F_{i-1} + F_{i-2} &= \frac{\phi^{i-1} - \hat\phi^{i-1}}{\sqrt5} + \frac{\phi^{i-2} - \hat\phi^{i-2}}{\sqrt5} \\ &= \frac{(\phi^{i-1} + \phi^{i-2}) - (\hat\phi^{i-1} + \hat\phi^{i-2})}{\sqrt5} \\ &= \frac{\phi^i - \hat\phi^i}{\sqrt5} . \qquad \blacksquare \end{aligned} \end{equation*}

Write \(k = (k\ln k)/\ln k\) and show \(\ln k = \Theta(\ln n)\). By hypothesis there are constants \(c_1, c_2 > 0\) and \(n_0\) with \(c_1 n \le k\ln k \le c_2 n\) for all \(n \ge n_0\). Since \(k \ln k \ge c_1 n \to \infty\), also \(k \to \infty\) (a bounded \(k \le K\) would give \(k\ln k \le K\ln K\)), so enlarging \(n_0\) we may assume \(k \ge e\) and \(n \ge \max(c_2, 1/c_1^2, 2)\) for \(n \ge n_0\). Then \(0 \le \ln\ln k \le \ln k\), so taking logarithms of the hypothesis,

\begin{equation*} \begin{aligned} \ln k &\le \ln k + \ln\ln k = \ln(k\ln k) \le \ln(c_2 n) \le \ln(n^2) = 2\ln n, \\ 2\ln k &\ge \ln k + \ln\ln k = \ln(k\ln k) \ge \ln(c_1 n) \ge \ln\sqrt n = \tfrac12 \ln n , \end{aligned} \end{equation*}

using \(c_2 \le n\) and \(c_1 \ge n^{-1/2}\). Thus \(\tfrac14 \ln n \le \ln k \le 2\ln n\), and therefore, for all \(n \ge n_0\),

\begin{equation*} \begin{aligned} k &= \frac{k\ln k}{\ln k} \ge \frac{c_1 n}{2\ln n} = \frac{c_1}{2}\cdot\frac{n}{\ln n}, \\ k &= \frac{k\ln k}{\ln k} \le \frac{c_2 n}{\tfrac14\ln n} = 4c_2\cdot\frac{n}{\ln n}. \end{aligned} \end{equation*}

Hence \(k = \Theta(n/\ln n)\) by the definition of \(\Theta\)-notation. \(\blacksquare\)

Everything follows from the sandwich \(\tfrac{a_d}{2} n^d \le p(n) \le (d+1)A\, n^d\) for all \(n \ge n_0\), where \(A = \max_i |a_i|\) and \(n_0 = \max(1,\, 2dA/a_d)\). Indeed, for \(n \ge n_0\) (so \(n \ge 1\) and \(n^i \le n^{d-1}\) for \(i < d\)),

\begin{equation*} \begin{aligned} p(n) &\le \sum_{i=0}^{d} |a_i|\, n^i \le (d+1)A\, n^d, \\ p(n) &\ge a_d n^d - \sum_{i=0}^{d-1} |a_i|\, n^i \ge a_d n^d - dA\, n^{d-1} \\ &= n^{d-1}(a_d n - dA) \ge \tfrac{a_d}{2} n^d , \end{aligned} \end{equation*}

the last step because \(n \ge 2dA/a_d\) gives \(a_d n - dA \ge a_d n/2\). In particular \(p(n) > 0\) for \(n \ge n_0\).

(a) For \(k \ge d\) and \(n \ge n_0 \ge 1\) we have \(n^d \le n^k\), so \(0 \le p(n) \le c\, n^k\) with \(c = (d+1)A\); hence \(p(n) = O(n^k)\).

(b) For \(k \le d\) and \(n \ge n_0\), \(n^k \le n^d\), so \(0 \le c\, n^k \le p(n)\) with \(c = a_d/2 > 0\); hence \(p(n) = \Omega(n^k)\).

(c) With \(k = d\) both (a) and (b) apply, so \(c_1 n^k \le p(n) \le c_2 n^k\) for \(n \ge n_0\) with \(c_1 = a_d/2\), \(c_2 = (d+1)A\); that is, \(p(n) = \Theta(n^k)\) (equivalently, Theorem 3.1).

(d) Let \(k > d\) and let \(c > 0\) be arbitrary. For \(n \ge n_1 = \max\bigl(n_0,\, ((d+1)A/c)^{1/(k-d)} + 1\bigr)\),

\begin{equation*} 0 \le p(n) \le (d+1)A\, n^{d-k} \cdot n^k < c\, n^k , \end{equation*}

since \(n^{k-d} > (d+1)A/c\). As \(c\) was arbitrary, \(p(n) = o(n^k)\).

(e) Let \(k < d\) and let \(c > 0\) be arbitrary. For \(n \ge n_1 = \max\bigl(n_0,\, (2c/a_d)^{1/(d-k)} + 1\bigr)\),

\begin{equation*} p(n) \ge \tfrac{a_d}{2}\, n^{d-k} \cdot n^k > c\, n^k \ge 0 , \end{equation*}

since \(n^{d-k} > 2c/a_d\). As \(c\) was arbitrary, \(p(n) = \omega(n^k)\).

Throughout, \(f = o(g)\) means \(f(n)/g(n) \to 0\) (equation (3.1)), which implies \(f = O(g)\) and rules out \(f = \Omega(g)\) (a bound \(f \ge c^{\prime} g\) with \(c^{\prime} > 0\) contradicts \(f/g \to 0\)), hence also rules out \(\omega\) and \(\Theta\); symmetrically for \(\omega\).

(a) \(\lg^k n = o(n^\epsilon)\), the polylogarithm-versus-polynomial limit derived from (3.10) in Section 3.2 (with \(b = k\), \(a = \epsilon\)).

(b) \(n^k = o(c^n)\) by equation (3.10) (with \(b = k\), \(a = c > 1\)).

(c) Since \(\pi\) is irrational, the values \(n \bmod 2\pi\) are dense in \([0, 2\pi)\), so infinitely many integers \(n\) have \(\sin n > 3/4\) and infinitely many have \(\sin n < 0\). Along the first set,

\begin{equation*} \frac{\sqrt n}{n^{\sin n}} \le n^{1/2 - 3/4} = n^{-1/4} \to 0 , \end{equation*}

so no \(c > 0\) gives \(\sqrt n \ge c\, n^{\sin n}\) for all large \(n\): not \(\Omega\). Along the second set \(n^{\sin n} \le 1\), so the ratio is at least \(\sqrt n \to \infty\): not \(O\). Failing both \(O\) and \(\Omega\), \(A\) is none of the five.

(d) \(2^n / 2^{n/2} = 2^{n/2} \to \infty\), so \(2^n = \omega(2^{n/2})\), hence \(\Omega\) but not \(O\), \(o\), \(\Theta\).

(e) By equation (3.16) with base \(2\), \(n^{\lg c} = c^{\lg n}\) for every \(n > 0\); the ratio is identically \(1\), so \(\Theta\) (thus \(O\) and \(\Omega\)) holds and \(o\), \(\omega\) fail.

(f) \(\lg(n^n) = n \lg n\) and \(\lg(n!) = \Theta(n \lg n)\) by equation (3.19), so \(A = \Theta(B)\) with \(A/B\) bounded between positive constants for large \(n\); hence \(O\), \(\Omega\), \(\Theta\) yes and \(o\), \(\omega\) no.

(a) From fastest to slowest, the 24 classes are (functions in one row are \(\Theta\) of each other, and each row is \(\omega\) of the next):

Within-class identities: \((\lg n)^{\lg n} = 2^{\lg n\lg\lg n} = n^{\lg\lg n}\), \(4^{\lg n} = n^2\), \(2^{\lg n} = n\), \((\sqrt2)^{\lg n} = \sqrt n\), \(n^{1/\lg n} = 2^{\lg n/\lg n} = 2\), \(\lg(n!) = \Theta(n\lg n)\) by equation (3.19), and \(\lg^*(\lg n) = \lg^* n - 1\) for \(n > 1\) straight from the definition of \(\lg^*\). Throughout, if \(\lg a(n) - \lg b(n) \to \infty\) then \(a = \omega(b)\). The strict steps between classes:

\begin{equation*} \begin{aligned} 1/2:&\ \ 2^{2^{n+1}}/2^{2^n} = 2^{2^n} \to \infty,\\ 2/3:&\ \ \lg (n+1)! \le (n+1)\lg(n+1) = o(2^n),\\ 3/4:&\ \ (n+1)!/n! = n+1,\\ 4/5:&\ \ n! \ge (n/e)^n \text{ by (3.20), so } n!/e^n \ge (n/e^2)^n \to \infty,\\ 5/6,\ 6/7,\ 7/8:&\ \ \text{ratios } (e/2)^n/n \to \infty \text{ by (3.10)},\ n,\ (4/3)^n,\\ 8/9:&\ \ \lg (\lg n)^{\lg n} = \lg n\,\lg\lg n = o(n\lg(3/2)),\\ 9/10:&\ \ (\lg n)!/(\lg n)^{\lg n} = \Theta\bigl(\sqrt{\lg n}\,e^{-\lg n}\bigr) \to 0 \text{ by (3.18)},\\ 10/11:&\ \ \lg (\lg n)! = \Theta(\lg n\,\lg\lg n) = \omega(3\lg n) \text{ by (3.19)},\\ 11/12,\ 12/13,\ 13/14:&\ \ \text{ratios } n,\ n/\lg n,\ \lg n,\\ 14/15:&\ \ n/\sqrt n = \sqrt n,\\ 15/16:&\ \ \tfrac12\lg n - \sqrt{2\lg n} \to \infty,\\ 16/17:&\ \ \sqrt{2\lg n} - 2\lg\lg n \to \infty,\\ 17/18,\ 18/19:&\ \ \lg^2 n/\ln n = \Theta(\lg n),\ \ \ln n/\sqrt{\lg n} = \Theta(\sqrt{\lg n}),\\ 19/20:&\ \ \ln\ln n = \Theta(\lg(\lg n)) = o\bigl((\lg n)^{1/2}\bigr), \end{aligned} \end{equation*}

the last by the polylogarithm bound \(\lg^b m = o(m^a)\) of Section 3.2 with \(m = \lg n\). For \(20/21\), put \(k = \lg^* n\) and \(T(0) = 1\), \(T(j+1) = 2^{T(j)}\); minimality of \(k\) gives \(\lg^{(k-1)} n > 1\), i.e. \(n > T(k-1)\), so \(\lg\lg n > T(k-3)\) and, using \(T(j) \ge 2^{j-1}\) for \(j \ge 1\),

\begin{equation*} \frac{2^{\lg^* n}}{\lg\lg n} < \frac{2^k}{T(k-3)} = 2^{\,k - T(k-4)} \le 2^{\,k-2^{k-5}} \to 0 \quad (k \to \infty). \end{equation*}

Finally \(2^{\lg^* n}/\lg^* n = 2^k/k \to \infty\), \(\lg(\lg^* n) = o(\lg^* n)\), and \(\lg(\lg^* n) \to \infty\) while class 24 is the constant \(2\) or \(1\), which gives \(21/22\), \(22/23\), \(23/24\).

(b) Take

\begin{equation*} f(n) = \begin{cases} 2^{2^{n+2}} & n \text{ even},\\ 0 & n \text{ odd}. \end{cases} \end{equation*}

By part (a), each \(g_i \le C_i\, 2^{2^{n+1}}\) for large \(n\), so on even \(n\) we get \(f(n)/g_i(n) \ge 2^{2^{n+1}}/C_i \to \infty\) and \(f \ne O(g_i)\). Every \(g_i\) is positive for large \(n\), so on odd \(n\) we have \(f(n) = 0 < c\,g_i(n)\) for every \(c > 0\) and \(f \ne \Omega(g_i)\).

True: (c), (f), (h). False: (a), (b), (d), (e), (g).

(a) False: \(f(n) = n\), \(g(n) = n^2\) gives \(n = O(n^2)\), but \(n^2 \le cn\) fails for \(n > c\).

(b) False: the same pair has \(\min(f,g) = n\), while \(n + n^2 \le cn\) fails for \(n \ge c\).

(c) True. Pick \(c \ge 1\) and \(n_0\) with \(1 \le f(n) \le c\,g(n)\) and \(\lg g(n) \ge 1\) for \(n \ge n_0\). Then

\begin{equation*} 0 \le \lg f(n) \le \lg c + \lg g(n) \le (\lg c + 1)\lg g(n) \qquad (n \ge n_0). \end{equation*}

(d) False: \(f(n) = 2n\) and \(g(n) = n\) have \(f = O(g)\), but \(2^{2n}/2^n = 2^n\) is unbounded.

(e) False: \(f(n) = 1/n\) is positive, yet \(1/n \le c/n^2\) means \(n \le c\).

(f) True: \(0 \le f(n) \le c\,g(n)\) for \(n \ge n_0\) gives \(0 \le (1/c) f(n) \le g(n)\) for \(n \ge n_0\).

(g) False: \(f(n) = 4^n\) has \(f(n/2) = 2^n\), and \(4^n \le c\,2^n\) means \(2^n \le c\).

(h) True. Let \(h(n) \in o(f(n))\). The definition of \(o\) with constant \(1\) gives \(n_0\) such that \(0 \le h(n) < f(n)\) for \(n \ge n_0\), so

\begin{equation*} f(n) \le f(n) + h(n) \le 2f(n) \qquad (n \ge n_0), \end{equation*}

i.e. \(f(n) + h(n) = \Theta(f(n))\).

(a) If \(f(n) \ne O(g(n))\) then \(f(n) = \overset{\infty}{\Omega}(g(n))\) with \(c = 1\). Pick \(N\) with \(f(n) \ge 0\) and \(g(n) \ge 0\) for \(n \ge N\). Failure of \(f = O(g)\) with constant \(c=1\) means that for every \(n_0 \ge N\) there is \(n \ge n_0\) with \(f(n) < 0\) or \(f(n) > g(n)\); the first is impossible, so

\begin{equation*} f(n) > 1\cdot g(n) \ge 0 \quad \text{for infinitely many } n, \end{equation*}

i.e. \(f(n) = \overset{\infty}{\Omega}(g(n))\). With \(\Omega\) the dichotomy fails: take \(f(n) = n\) and

\begin{equation*} g(n) = \begin{cases} n^2, & n \text{ even},\\ 1, & n \text{ odd}. \end{cases} \end{equation*}

On odd \(n\), \(f(n) = n \le c\cdot 1\) fails for \(n > c\), so \(f \ne O(g)\); on even \(n\), \(n \ge c n^2\) fails for \(n > 1/c\), so \(f \ne \Omega(g)\).

(b) Advantage: by (a) every pair of (asymptotically nonnegative) running times is comparable, and \(\overset{\infty}{\Omega}\) can record a lower bound for a program whose cost is large only on some input sizes (e.g. the \(g\) of part (a) satisfies \(g = \overset{\infty}{\Omega}(n^2)\) but \(g \ne \Omega(n^2)\)), which \(\Omega\) cannot express. Disadvantages: the bound is much weaker, since a program with time \(\overset{\infty}{\Omega}(n^2)\) may run in \(O(n)\) time on all but a sparse set of sizes, so it says nothing about a typical large \(n\); \(O\) and \(\overset{\infty}{\Omega}\) together no longer pin down the growth rate as \(\Theta\) does; and \(\overset{\infty}{\Omega}\) is not transitive. For the last point, take \(f(n) = n\) (\(n\) even), \(1\) (\(n\) odd); \(g(n) = n\); \(h(n) = n^2\) (\(n\) even), \(n\) (\(n\) odd). Then \(f = \overset{\infty}{\Omega}(g)\) (even \(n\), \(c = 1\)) and \(g = \overset{\infty}{\Omega}(h)\) (odd \(n\), \(c=1\)), but \(f \ge c\,h\) needs \(n \ge c n^2\) on even \(n\) or \(1 \ge cn\) on odd \(n\), each true for only finitely many \(n\).

(c) Both directions still hold. Theorem 3.1 now reads: \(f = \Theta(g)\) iff \(f = O^{\prime}(g)\) and \(f = \Omega(g)\). (i) (\(\Rightarrow\)) From \(0 \le c_1 g(n) \le f(n) \le c_2 g(n)\) for \(n \ge n_0\) we get \(f = \Omega(g)\), and \(|f(n)| = f(n) \le c_2 g(n)\), so \(f = O^{\prime}(g)\). (ii) (\(\Leftarrow\)) From \(f = \Omega(g)\), \(0 \le c_1 g(n) \le f(n)\) for \(n \ge n_1\), so \(f(n) = |f(n)|\) there; from \(|f| = O(g)\), \(|f(n)| \le c_2 g(n)\) for \(n \ge n_2\). Hence \(0 \le c_1 g(n) \le f(n) \le c_2 g(n)\) for \(n \ge \max(n_1, n_2)\), i.e. \(f = \Theta(g)\).

(d) Define, with \(\lg^{-k} n = (\lg n)^{-k}\),

\begin{equation*} \begin{aligned} \widetilde{\Omega}(g(n)) &= \{ f(n) : \exists\, c, k > 0,\ n_0 \ge 2,\\ &\qquad 0 \le c\,g(n)\lg^{-k} n \le f(n)\ \forall n \ge n_0 \},\\ \widetilde{\Theta}(g(n)) &= \{ f(n) : \exists\, c_1, c_2, k > 0,\ n_0 \ge 2,\\ &\qquad 0 \le c_1 g(n) \lg^{-k} n \le f(n) \le c_2 g(n) \lg^{k} n\ \forall n \ge n_0 \}. \end{aligned} \end{equation*}

Theorem: \(f(n) = \widetilde{\Theta}(g(n))\) if and only if \(f(n) = \widetilde{O}(g(n))\) and \(f(n) = \widetilde{\Omega}(g(n))\).

(\(\Rightarrow\)) The two halves of the defining inequality of \(\widetilde{\Theta}\) are exactly the inequalities of \(\widetilde{\Omega}\) (constants \(c_1, k, n_0\)) and of \(\widetilde{O}\) (constants \(c_2, k, n_0\); note \(0 \le f(n)\) follows from the left half).

(\(\Leftarrow\)) Suppose \(0 \le f(n) \le c_2 g(n) \lg^{k_2} n\) for \(n \ge n_2\) and \(0 \le c_1 g(n) \lg^{-k_1} n \le f(n)\) for \(n \ge n_1\). Put \(k = \max(k_1, k_2)\) and \(n_0 = \max(n_1, n_2, 2)\). For \(n \ge n_0\) we have \(\lg n \ge 1\) and \(g(n) \ge 0\), so \(\lg^{-k} n \le \lg^{-k_1} n\) and \(\lg^{k_2} n \le \lg^{k} n\), giving

\begin{equation*} \begin{aligned} 0 \le c_1 g(n) \lg^{-k} n &\le c_1 g(n) \lg^{-k_1} n \le f(n)\\ &\le c_2 g(n) \lg^{k_2} n \le c_2 g(n) \lg^{k} n, \end{aligned} \end{equation*}

so \(f(n) = \widetilde{\Theta}(g(n))\).

The answers (exact for the stated range of \(n\); \(f_c^*(n) = 0\) whenever \(n \le c\)):

In (a), (c)-(g) the iterates have closed forms, and \(f_c^*(n)\) is the least integer \(i \ge 0\) satisfying the displayed condition:

\begin{equation*} \begin{aligned} &\text{(a)}\ n - i \le 0 \iff i \ge n,\\ &\text{( c)}\ n/2^i \le 1 \iff i \ge \lg n,\\ &\text{(d)}\ n/2^i \le 2 \iff i \ge \lg n - 1,\\ &\text{(e)}\ n^{1/2^i} \le 2 \iff 2^i \ge \lg n \iff i \ge \lg\lg n,\\ &\text{(g)}\ n^{1/3^i} \le 2 \iff 3^i \ge \lg n \iff i \ge \log_3 \lg n. \end{aligned} \end{equation*}

(b) is the definition of \(\lg^* n\) in Section 3.2. (f) For \(n > 1\), \(n^{1/2^i} > 1\) for every \(i\), so the set is empty and \(f_1^*(n)\) is undefined.

(h) Here \(f\) is increasing for \(n \ge e\), \(f(n) \le 2\) for \(2 \le n \le 4\), and \(f(n) < n\) for \(n > 2\). Let \(x_0 = n\), \(x_{i+1} = x_i/\lg x_i\), \(L = \lg n\), \(y_i = \lg x_i\), so \(y_{i+1} = y_i - \lg y_i\). Lower bound: \(x_i \le n\) gives \(x_{i+1} \ge x_i / L\), so at \(i = f_2^*(n)\),

\begin{equation*} 2 \ge x_i \ge n/L^i \quad\Longrightarrow\quad f_2^*(n) \ge \frac{L - 1}{\lg L}. \end{equation*}

Upper bound: let \(k\) be the number of indices with \(y_i \ge \sqrt{L}\) (an initial segment, as \(y_i\) decreases). Each such step lowers \(y\) by \(\lg y_i \ge \tfrac12 \lg L\), so \(\sqrt{L} \le y_{k-1} \le L - (k-1)\tfrac12\lg L\), i.e. \(k \le 2L/\lg L + 1\). Then \(x_k < 2^{\sqrt L}\), and from any \(m > 4\) one step at least halves (\(\lg m \ge 2\)), while from \(m \in (2,4]\) one step reaches \(\le 2\); hence \(f_2^*(m) \le \lg m + 1\) and

\begin{equation*} f_2^*(n) \le k + f_2^*(x_k) \le \frac{2L}{\lg L} + 1 + \sqrt{L} + 1 = O\!\left(\frac{\lg n}{\lg\lg n}\right). \end{equation*}

Together, \(f_2^*(n) = \Theta(\lg n/\lg\lg n)\).

It returns the one-element subarray holding the largest entry: the triple \((k, k, A[k])\), where \(k\) is the smallest index with \(A[k] = \max_i A[i]\).

Induct on \(n = high - low + 1\). For \(n = 1\) lines 1–2 return \((low, low, A[low])\). For \(n > 1\), by induction the left call returns \((p, p, A[p])\) with \(p\) the leftmost maximum of \(A[low..mid]\), and the right call returns \((q, q, A[q])\) with \(q\) the leftmost maximum of \(A[mid+1..high]\). The crossing subarray is \(A[i..j]\) with \(i \le mid < j\), and since every entry is negative,

\begin{equation*} \text{cross-sum} = \sum_{t=i}^{mid} A[t] + \sum_{t=mid+1}^{j} A[t] < \sum_{t=i}^{mid} A[t] \le A[mid] \le A[p], \end{equation*}

and symmetrically cross-sum \(< A[mid+1] \le A[q]\). So the crossing candidate is never returned; line 7 returns the left answer exactly when \(A[p] \ge A[q]\) and line 10 the right answer otherwise, which is \((k, k, A[k])\) for the leftmost overall maximum \(k\).

Try every left end \(i\), and extend a running sum to the right so that each sum \(A[i..j]\) costs \(O(1)\) given \(A[i..j-1]\).

BRUTE-FORCE-MAXIMUM-SUBARRAY(A, n)
1  best-sum = -infinity
2  for i = 1 to n
3      sum = 0
4      for j = i to n
5          sum = sum + A[j]
6          if sum > best-sum
7              best-sum = sum
8              best-low = i
9              best-high = j
10 return (best-low, best-high, best-sum)

Correctness: the inner loop maintains the invariant that, at the start of the iteration for \(j\), \(sum = \sum_{t=i}^{j-1} A[t]\) and best-sum is the largest sum over all pairs \((i^{\prime}, j^{\prime})\) already visited, attained by \(A[best\text{-}low..best\text{-}high]\); line 5 restores the first part and lines 6–9 the second. Termination after the outer loop means every pair \(1 \le i \le j \le n\) has been visited, so the triple returned is a maximum subarray.

Running time: lines 5–9 cost \(\Theta(1)\) and execute once per pair \(i \le j\), and the outer loop overhead is \(\Theta(n)\), so

\begin{equation*} T(n) = \Theta\Big(\sum_{i=1}^{n} (n - i + 1)\Big) = \Theta\Big(\frac{n(n+1)}{2}\Big) = \Theta(n^2). \end{equation*}

On the machine used here (CPython 3.9, Apple arm64, random integer arrays, best of 15 timings of 300 arrays) the crossover is \(n_0 \approx 60\); with the brute-force base case the crossover against brute force stays at \(n_0\) (below \(n_0\) the hybrid is the brute-force algorithm), but at and beyond \(n_0\) the hybrid is far faster than both, and a smaller cutoff is better still. The code:

def brute(A, lo, hi):
    best = (lo, lo, A[lo])
    for i in range(lo, hi + 1):
        s = 0
        for j in range(i, hi + 1):
            s += A[j]
            if s > best[2]: best = (i, j, s)
    return best

def crossing(A, lo, mid, hi):
    s, ls, ml = 0, A[mid], mid
    for i in range(mid, lo - 1, -1):
        s += A[i]
        if s > ls: ls, ml = s, i
    s, rs, mr = 0, A[mid + 1], mid + 1
    for j in range(mid + 1, hi + 1):
        s += A[j]
        if s > rs: rs, mr = s, j
    return (ml, mr, ls + rs)

def recursive(A, lo, hi, n0=1):     # n0 = 1: the book's algorithm
    if hi - lo + 1 < n0 or hi == lo: return brute(A, lo, hi)
    mid = (lo + hi) // 2
    cands = [recursive(A, lo, mid, n0), recursive(A, mid + 1, hi, n0),
             crossing(A, lo, mid, hi)]
    return max(cands, key=lambda t: t[2])

Measured times in microseconds per call:

The two pure algorithms are within noise of each other for \(56 \le n \le 58\) and the recursive one wins from \(n_0 \approx 60\) on, as expected from \(\Theta(n \lg n)\) against \(\Theta(n^2)\) with a larger constant for the recursive code. The hybrid with cutoff \(n_0\) coincides with brute force for \(n < n_0\) (differences there are timing noise), so the crossover point itself does not change; but at \(n = n_0\) the hybrid already nearly halves the brute-force time, because one split costs about

\begin{equation*} 2c\Big(\frac{n}{2}\Big)^2 + dn = \frac{c n^2}{2} + dn < c n^2 \quad\text{once } n > \frac{2d}{c}, \end{equation*}

where \(cn^2\) is the brute-force cost and \(dn\) the crossing step. Since \(2d/c\) is much smaller than \(n_0\), the best cutoff is well below \(n_0\): with cutoff 12 the hybrid beats brute force from \(n \approx 14\) on and is the fastest of the four at every measured size above 12.

Run any of the algorithms unchanged and, if the sum it returns is negative, return the empty subarray (say \(A[low..low-1]\)) with sum 0 instead; otherwise return its answer.

This is correct because the nonempty optimum \(s^\ast\) satisfies

\begin{equation*} s^\ast < 0 \iff \text{every } A[i] < 0 \iff \text{every nonempty subarray has negative sum}. \end{equation*}

(i) If \(s^\ast < 0\), the empty subarray (sum \(0\)) beats every nonempty one. (ii) If \(s^\ast \ge 0\), the returned subarray is at least as good as the empty one, so it remains optimal. The extra test costs \(O(1)\), so all running times are unchanged.

Equivalently, the change can be built into each algorithm: in BRUTE-FORCE-MAXIMUM-SUBARRAY of Exercise 4.1-2 initialize best-sum \(= 0\), best-low \(= 1\), best-high \(= 0\) (the empty subarray) instead of best-sum \(= -\infty\); in the linear-time algorithm of Exercise 4.1-5 start with best-sum \(= 0\) for the empty subarray and allow the subarray ending at \(j\) to be empty, i.e. replace the current sum by \(\max(0, \text{current sum} + A[j])\).

A maximum subarray ending at \(j+1\) is \(A[j+1]\) alone if the maximum sum of a subarray ending at \(j\) is \(\le 0\), and otherwise is that subarray extended by \(A[j+1]\); keep this current subarray and the best one seen so far.

LINEAR-MAXIMUM-SUBARRAY(A, n)
1  best-low = 1, best-high = 1, best-sum = A[1]
2  cur-low = 1, cur-sum = A[1]
3  for j = 2 to n
4      if cur-sum > 0
5          cur-sum = cur-sum + A[j]
6      else cur-low = j
7           cur-sum = A[j]
8      if cur-sum > best-sum
9          best-low = cur-low, best-high = j
10         best-sum = cur-sum
11 return (best-low, best-high, best-sum)

The constant-time step: every subarray ending at \(j\) is either \(A[j..j]\) or \(A[i..j-1]\) followed by \(A[j]\) with \(i \le j-1\), so

\begin{equation*} \max_{1 \le i \le j} \sum_{t=i}^{j} A[t] = A[j] + \max\Big(0, \max_{1 \le i \le j-1} \sum_{t=i}^{j-1} A[t]\Big), \end{equation*}

which lines 4–7 compute from cur-sum.

Loop invariant: at the start of the iteration for \(j\), \(A[cur\text{-}low..j-1]\) is a maximum subarray among those ending at \(j-1\), with sum cur-sum, and \(A[best\text{-}low..best\text{-}high]\) is a maximum subarray of \(A[1..j-1]\), with sum best-sum.

Initialization: for \(j = 2\) the only subarray of \(A[1..1]\) is \(A[1..1]\), as set in lines 1–2.

Maintenance: by the display, lines 4–7 make \(A[cur\text{-}low..j]\) a maximum subarray ending at \(j\). A maximum subarray of \(A[1..j]\) either ends before \(j\), where the best is \(A[best\text{-}low..best\text{-}high]\), or ends at \(j\), where the best is \(A[cur\text{-}low..j]\); lines 8–10 keep the larger, so the invariant holds for \(j+1\).

Termination: the loop ends with \(j = n+1\), so the triple returned is a maximum subarray of \(A[1..n]\) (for \(n = 1\) the loop is skipped and \(A[1..1]\) is returned).

Running time: each of the \(n-1\) iterations does \(O(1)\) work, so the algorithm runs in \(\Theta(n)\) time and uses \(O(1)\) extra space.

The product is \(\begin{pmatrix} 18 & 14 \\ 62 & 66 \end{pmatrix}\). Here the blocks are scalars: \(A_{11}=1\), \(A_{12}=3\), \(A_{21}=7\), \(A_{22}=5\), \(B_{11}=6\), \(B_{12}=8\), \(B_{21}=4\), \(B_{22}=2\). Step 2 of Section 4.2 gives

\begin{equation*} \begin{aligned} S_1 &= 8-2 = 6, & S_2 &= 1+3 = 4, & S_3 &= 7+5 = 12,\\ S_4 &= 4-6 = -2, & S_5 &= 1+5 = 6, & S_6 &= 6+2 = 8,\\ S_7 &= 3-5 = -2, & S_8 &= 4+2 = 6, & S_9 &= 1-7 = -6,\\ S_{10} &= 6+8 = 14, &&&& \end{aligned} \end{equation*}

step 3 the seven products

\begin{equation*} \begin{aligned} P_1 &= A_{11}S_1 = 6, & P_2 &= S_2B_{22} = 8,\\ P_3 &= S_3B_{11} = 72, & P_4 &= A_{22}S_4 = -10,\\ P_5 &= S_5S_6 = 48, & P_6 &= S_7S_8 = -12,\\ P_7 &= S_9S_{10} = -84, && \end{aligned} \end{equation*}

and step 4

\begin{equation*} \begin{aligned} C_{11} &= P_5+P_4-P_2+P_6 = 48-10-8-12 = 18,\\ C_{12} &= P_1+P_2 = 6+8 = 14,\\ C_{21} &= P_3+P_4 = 72-10 = 62,\\ C_{22} &= P_5+P_1-P_3-P_7 = 48+6-72+84 = 66. \end{aligned} \end{equation*}

Recurse on the four \(n/2\times n/2\) blocks, forming the ten sums \(S_1,\dots,S_{10}\), the seven products \(P_1,\dots,P_7\) and the four output blocks exactly as in Section 4.2 (\(n\) an exact power of 2; partitioning by index calculation as in SQUARE-MATRIX-MULTIPLY-RECURSIVE).

STRASSEN(A, B)
 1  n = A.rows
 2  let C be a new n x n matrix
 3  if n == 1
 4      c11 = a11 * b11
 5  else partition A, B, C into n/2 x n/2 blocks as in (4.9)
 6      P1 = STRASSEN(A11, B12 - B22)
 7      P2 = STRASSEN(A11 + A12, B22)
 8      P3 = STRASSEN(A21 + A22, B11)
 9      P4 = STRASSEN(A22, B21 - B11)
10      P5 = STRASSEN(A11 + A22, B11 + B22)
11      P6 = STRASSEN(A12 - A22, B21 + B22)
12      P7 = STRASSEN(A11 - A21, B11 + B12)
13      C11 = P5 + P4 - P2 + P6
14      C12 = P1 + P2
15      C21 = P3 + P4
16      C22 = P5 + P1 - P3 - P7
17  return C

Correctness is by induction on \(n\): for \(n=1\) line 4 is the scalar product, and for \(n>1\) the inductive hypothesis makes each \(P_i\) the true product of its arguments, whereupon the expansions of \(C_{11},C_{12},C_{21},C_{22}\) in Section 4.2 show that lines 13-16 produce equations (4.11)-(4.14). Lines 6-12 make seven recursive calls on \(n/2\times n/2\) matrices after ten \(\Theta(n^2)\) additions/subtractions, and lines 13-16 do eight more, so the running time satisfies recurrence (4.18), \(T(n)=7T(n/2)+\Theta(n^2)\), whence \(T(n)=\Theta(n^{\lg 7})\) by case 1 of the master theorem (Theorem 4.1; \(n^2=O(n^{\lg 7-\epsilon})\) with \(\epsilon=0.8\)).

Pad with zeros: let \(m=2^{\lceil \lg n\rceil}\), embed \(A\) and \(B\) as the upper-left blocks of \(m\times m\) matrices \(A^{\prime},B^{\prime}\) that are zero elsewhere, compute \(C^{\prime}=A^{\prime}B^{\prime}\) by STRASSEN, and return the upper-left \(n\times n\) block of \(C^{\prime}\). This is correct because

\begin{equation*} A^{\prime}B^{\prime} = \begin{pmatrix} A & 0\\ 0 & 0\end{pmatrix} \begin{pmatrix} B & 0\\ 0 & 0\end{pmatrix} = \begin{pmatrix} AB & 0\\ 0 & 0\end{pmatrix}. \end{equation*}

Since \(\lg n\le \lceil \lg n\rceil<\lg n+1\), we have \(n\le m<2n\). Building \(A^{\prime},B^{\prime}\) and copying out the answer costs \(\Theta(m^2)=\Theta(n^2)\), and by Exercise 4.2-2 (recurrence (4.18) and Theorem 4.1) the call costs \(T(m)=\Theta(m^{\lg 7})\); so for constants \(0<c_1\le c_2\) and all large \(n\),

\begin{equation*} c_1 n^{\lg 7} \le c_1 m^{\lg 7} \le T(m) \le c_2 m^{\lg 7} < c_2 (2n)^{\lg 7} = 7c_2\, n^{\lg 7}. \end{equation*}

Hence the total is \(\Theta(n^2)+\Theta(n^{\lg 7})=\Theta(n^{\lg 7})\).

The largest is \(k=21\), giving running time \(\Theta(n^{\log_3 21})\approx\Theta(n^{2.771})\). Because the \(3\times 3\) scheme never uses commutativity, it remains valid when its entries are \(n/3\times n/3\) blocks; applying it recursively (padding to a power of 3 as in Exercise 4.2-3, which changes \(n\) by a factor less than 3) performs \(k\) recursive block products and a constant number of \(n/3\times n/3\) block additions, so

\begin{equation*} T(n)=kT(n/3)+\Theta(n^2). \end{equation*}

For \(k\ge 10\) we have \(\log_3 k>2\), so case 1 of Theorem 4.1 applies (\(n^2=O(n^{\log_3 k-\epsilon})\) with \(\epsilon=\log_3 k-2>0\)) and \(T(n)=\Theta(n^{\log_3 k})\). This is \(o(n^{\lg 7})\) exactly when

\begin{equation*} \log_3 k<\lg 7 \iff k<3^{\lg 7}=7^{\lg 3}\approx 21.85 . \end{equation*}

So \(k=21\) works, with \(\log_3 21\approx 2.7712<2.8074\approx\lg 7\), while \(k=22\) gives \(\log_3 22\approx 2.8136>\lg 7\) and hence \(\Theta(n^{\log_3 22})\), which is not \(o(n^{\lg 7})\).

The \(70\times 70\) scheme is best, giving \(\Theta(n^{\log_{70}143640})\approx\Theta(n^{2.795123})\), which beats Strassen’s \(\Theta(n^{\lg 7})\approx\Theta(n^{2.807355})\). As in Exercise 4.2-4, a scheme multiplying \(b\times b\) matrices with \(m\) noncommutative multiplications yields \(T(n)=mT(n/b)+\Theta(n^2)\), which is \(\Theta(n^{\log_b m})\) by case 1 of Theorem 4.1 since \(\log_b m>2\) in all three cases. The exponents are

\begin{equation*} \begin{aligned} \log_{68} 132464 &\approx 2.795128,\\ \log_{70} 143640 &\approx 2.795123,\\ \log_{72} 155424 &\approx 2.795147, \end{aligned} \end{equation*}

so the \(70\times 70\) method has the smallest exponent.

The \(kn\times n\) by \(n\times kn\) product takes \(\Theta(k^2 n^{\lg 7})\) time, and the reversed product takes \(\Theta(k n^{\lg 7})\) time. Split each input into \(n\times n\) blocks (padding as in Exercise 4.2-3 if \(n\) is not a power of 2): \(A\) (\(kn\times n\)) into a column of blocks \(A_1,\dots,A_k\) and \(B\) (\(n\times kn\)) into a row of blocks \(B_1,\dots,B_k\).

(i) The product \(AB\) is the \(kn\times kn\) matrix whose \((i,j)\) block is \(A_iB_j\), so it costs \(k^2\) Strassen calls and no additions:

\begin{equation*} k^2\cdot\Theta(n^{\lg 7})=\Theta(k^2 n^{\lg 7}). \end{equation*}

(ii) The product \(BA\) is the \(n\times n\) matrix

\begin{equation*} BA=\sum_{i=1}^{k} B_iA_i , \end{equation*}

which costs \(k\) Strassen calls plus \(k-1\) additions of \(n\times n\) matrices:

\begin{equation*} k\cdot\Theta(n^{\lg 7})+(k-1)\Theta(n^2)=\Theta(k n^{\lg 7}). \end{equation*}

Form the three products

\begin{equation*} P_1 = c(a+b), \qquad P_2 = a(d-c), \qquad P_3 = b(c+d), \end{equation*}

and output real part \(P_1 - P_3\) and imaginary part \(P_1 + P_2\).

COMPLEX-MULTIPLY(a, b, c, d)
1  p1 = c * (a + b)
2  p2 = a * (d - c)
3  p3 = b * (c + d)
4  return (p1 - p3, p1 + p2)

Only lines 1–3 multiply (the rest are additions and subtractions), and

\begin{equation*} \begin{aligned} P_1 - P_3 &= (ac + bc) - (bc + bd) = ac - bd,\\ P_1 + P_2 &= (ac + bc) + (ad - ac) = ad + bc. \end{aligned} \end{equation*}

We prove \(T(n) \le cn^2\) for all \(n \ge 1\), where \(c = \max(1, T(1))\) and \(T(1)\) is the (constant) boundary value.

Base case: \(T(1) \le c = c \cdot 1^2\) by the choice of \(c\).

Inductive step: let \(n \ge 2\) and assume \(T(n-1) \le c(n-1)^2\). Then

\begin{equation*} \begin{aligned} T(n) &\le c(n-1)^2 + n = cn^2 - (2cn - c - n) \\ &= cn^2 - \bigl(c(2n-1) - n\bigr) \le cn^2 , \end{aligned} \end{equation*}

since \(c \ge 1\) gives \(c(2n-1) \ge 2n - 1 \ge n\) for \(n \ge 1\). Hence \(T(n) \le cn^2\) for all \(n \ge 1\), i.e. \(T(n) = O(n^2)\).

We prove \(T(n) \le c \lg n\) for all \(n \ge 2\), with \(c = \max(3, T(2))\). (\(n = 1\) cannot be the base case, as \(c \lg 1 = 0\).)

Base case \(n = 2\): \(T(2) \le c = c \lg 2\).

Inductive step: let \(n \ge 3\), so \(2 \le \lceil n/2 \rceil < n\), and assume the bound for \(\lceil n/2 \rceil\). Since \(\lceil n/2 \rceil \le (n+1)/2\) and \(1 + 1/n \le 3/2\) for \(n \ge 2\),

\begin{equation*} \begin{aligned} \lg \lceil n/2 \rceil &\le \lg n - 1 + \lg(1 + 1/n) \\ &\le \lg n - (1 - \lg(3/2)) \le \lg n - 1/3 , \end{aligned} \end{equation*}

because \(1 - \lg(3/2) \approx 0.415 > 1/3\). Therefore

\begin{equation*} T(n) \le c \lg \lceil n/2 \rceil + 1 \le c \lg n - c/3 + 1 \le c \lg n , \end{equation*}

as \(c \ge 3\). Hence \(T(n) \le c \lg n\) for all \(n \ge 2\), i.e. \(T(n) = O(\lg n)\).

We prove the stronger hypothesis \(T(n) \ge c(n+1)\lg(n+1)\) for all \(n \ge 1\), with \(c = \min(1/2,\ T(1)/2) > 0\) (the boundary value \(T(1)\) is a positive constant).

Base case \(n = 1\): \(c \cdot 2 \lg 2 = 2c \le T(1)\).

Inductive step: let \(n \ge 2\), so \(1 \le \lfloor n/2 \rfloor < n\), and assume the bound for \(\lfloor n/2 \rfloor\). Since \(\lfloor n/2 \rfloor + 1 \ge (n+1)/2 \ge 1\) and \(x \lg x\) is increasing for \(x \ge 1\),

\begin{equation*} \begin{aligned} T(n) &\ge 2c\bigl(\lfloor n/2 \rfloor + 1\bigr)\lg\bigl(\lfloor n/2 \rfloor + 1\bigr) + n \\ &\ge 2c \cdot \tfrac{n+1}{2} \lg \tfrac{n+1}{2} + n \\ &= c(n+1)\lg(n+1) + \bigl(n - c(n+1)\bigr) \ge c(n+1)\lg(n+1) , \end{aligned} \end{equation*}

because \(c \le 1/2\) gives \(c(n+1) \le (n+1)/2 \le n\) for \(n \ge 1\). Hence \(T(n) \ge c(n+1)\lg(n+1) \ge c\, n \lg n\) for all \(n \ge 1\), so \(T(n) = \Omega(n \lg n)\).

Combined with the bound \(T(n) = O(n \lg n)\) proved in the text for recurrence (4.19), Theorem 3.1 gives \(T(n) = \Theta(n \lg n)\).

Use the hypothesis \(T(n) \le n \lg n + n\), which adds the lower-order term \(n\).

Base case \(n = 1\): \(T(1) = 1 = 1 \cdot \lg 1 + 1\).

Inductive step: let \(n \ge 2\), so \(1 \le \lfloor n/2 \rfloor < n\), and assume the bound for \(\lfloor n/2 \rfloor\). Using \(\lfloor n/2 \rfloor \le n/2\) and that \(x \lg x + x\) is increasing for \(x \ge 1\),

\begin{equation*} \begin{aligned} T(n) &\le 2\bigl(\lfloor n/2 \rfloor \lg \lfloor n/2 \rfloor + \lfloor n/2 \rfloor\bigr) + n \\ &\le n \lg(n/2) + n + n = n \lg n - n + 2n = n \lg n + n . \end{aligned} \end{equation*}

Hence \(T(n) \le n \lg n + n\) for all \(n \ge 1\). Since \(n \le n \lg n\) for \(n \ge 2\), this gives \(T(n) \le 2 n \lg n\) for \(n \ge 2\), i.e. \(T(n) = O(n \lg n)\).

Make the \(\Theta\)’s explicit: \(T(1) = a > 0\), and for \(n \ge 2\), \(T(n) = T(\lceil n/2 \rceil) + T(\lfloor n/2 \rfloor) + f(n)\) with \(c_1 n \le f(n) \le c_2 n\) for constants \(0 < c_1 \le c_2\). For \(n \ge 2\) both \(\lceil n/2 \rceil\) and \(\lfloor n/2 \rfloor\) lie in \([1, n)\), and

\begin{equation*} \frac n3 \le \lfloor n/2 \rfloor \le \lceil n/2 \rceil \le \frac{n+1}{2} \le \frac{3n}{4} \qquad (n \ge 2). \end{equation*}

Upper bound: we prove \(T(n) \le d\, n \lg n + a n\) for all \(n \ge 1\), with \(d = c_2 / \lg(4/3)\). Base: \(T(1) = a\). Step (\(n \ge 2\)): both lg’s are at most \(\lg(3n/4) = \lg n - \lg(4/3)\), so

\begin{equation*} \begin{aligned} T(n) &\le d\bigl(\lceil n/2 \rceil + \lfloor n/2 \rfloor\bigr)\bigl(\lg n - \lg\tfrac43\bigr) + a n + c_2 n \\ &= d\, n \lg n + a n + n\bigl(c_2 - d \lg\tfrac43\bigr) = d\, n \lg n + a n . \end{aligned} \end{equation*}

Hence \(T(n) \le (d + a)\, n \lg n\) for \(n \ge 2\), so \(T(n) = O(n \lg n)\).

Lower bound: we prove \(T(n) \ge e\, n \lg n\) for all \(n \ge 1\), with \(e = c_1 / \lg 3\). Base: \(T(1) = a \ge 0\). Step (\(n \ge 2\)): both lg’s are at least \(\lg(n/3) = \lg n - \lg 3\), so

\begin{equation*} \begin{aligned} T(n) &\ge e\bigl(\lceil n/2 \rceil + \lfloor n/2 \rfloor\bigr)(\lg n - \lg 3) + c_1 n \\ &= e\, n \lg n + n(c_1 - e \lg 3) = e\, n \lg n . \end{aligned} \end{equation*}

So \(T(n) = \Omega(n \lg n)\), and by Theorem 3.1, \(T(n) = \Theta(n \lg n)\).

We prove \(T(n) \le c(n - 34)\lg(n - 34)\) for all \(n \ge 36\), for a suitable constant \(c \ge 2\); this gives \(T(n) \le c\, n \lg n\) for \(n \ge 36\).

Set \(m = \lfloor n/2 \rfloor + 17\). Since \(m < n\) only for \(n \ge 35\), the values \(T(1), \dots, T(34)\) are boundary constants, and

\begin{equation*} m - 34 = \lfloor n/2 \rfloor - 17 \le \frac{n - 34}{2} . \end{equation*}

Base cases \(36 \le n \le 67\): a finite set of values, each with \((n-34)\lg(n-34) \ge 2\), so we take

\begin{equation*} c = \max\Bigl(2,\ \max_{36 \le n \le 67} \frac{T(n)}{(n-34)\lg(n-34)}\Bigr) . \end{equation*}

Inductive step: let \(n \ge 68\). Then \(36 \le 51 \le m < n\), so the hypothesis holds at \(m\). Since \(x \lg x\) is increasing for \(x \ge 1\) and \(1 \le m - 34 \le (n-34)/2\),

\begin{equation*} \begin{aligned} T(n) &\le 2c(m - 34)\lg(m - 34) + n \le c(n-34)\lg\frac{n-34}{2} + n \\ &= c(n-34)\lg(n-34) - \bigl(c(n-34) - n\bigr) \le c(n-34)\lg(n-34) , \end{aligned} \end{equation*}

because \(c \ge 2\) and \(n \ge 68\) give \(c(n - 34) \ge 2n - 68 \ge n\). Hence \(T(n) \le c(n-34)\lg(n-34) \le c\, n \lg n\) for all \(n \ge 36\), i.e. \(T(n) = O(n \lg n)\).

The naive hypothesis leaves an uncancelled \(+n\); the strengthened hypothesis \(T(n) \le c n^{p} - 3n\), with \(p = \log_3 4\) (so \(4 \cdot 3^{-p} = 1\)), works.

Naive attempt. Assuming \(T(n/3) \le c (n/3)^p\),

\begin{equation*} T(n) \le 4c\left(\frac{n}{3}\right)^p + n = c n^p + n , \end{equation*}

and \(c n^p + n > c n^p\) for every \(c\) and every \(n > 0\), so the inductive step cannot conclude \(T(n) \le c n^p\): the proof fails.

Strengthened hypothesis. Suppose \(T(m) \le c m^p - d m\) for all \(m < n\). Then

\begin{equation*} \begin{aligned} T(n) &\le 4\left(c\left(\frac{n}{3}\right)^p - \frac{dn}{3}\right) + n = c n^p - \frac{4d}{3} n + n \\ &= c n^p - d n - \left(\frac{d}{3} - 1\right) n \le c n^p - d n \qquad\text{whenever } d \ge 3 . \end{aligned} \end{equation*}

Base case: take \(T(n) = \Theta(1)\), say \(T(n) \le k\), for \(1 \le n < 3\). With \(d = 3\) and \(n \ge 1\) (so \(n^{p-1} \ge 1\)),

\begin{equation*} c n^p - 3n = n\left(c n^{p-1} - 3\right) \ge c - 3 \ge k \qquad\text{if } c \ge k + 3 . \end{equation*}

Hence \(T(n) \le c n^{\log_3 4} - 3n = O(n^{\log_3 4})\) for all \(n \ge 1\).

The naive hypothesis leaves an uncancelled \(+n\); the strengthened hypothesis \(T(n) \le c n^2 - n\) works.

Naive attempt. Assuming \(T(n/2) \le c (n/2)^2\),

\begin{equation*} T(n) \le 4c\left(\frac{n}{2}\right)^2 + n = c n^2 + n , \end{equation*}

and \(c n^2 + n > c n^2\) for every \(c\) and every \(n > 0\), so the inductive step cannot conclude \(T(n) \le c n^2\): the proof fails.

Strengthened hypothesis. Suppose \(T(m) \le c m^2 - d m\) for all \(m < n\). Then

\begin{equation*} \begin{aligned} T(n) &\le 4\left(c\left(\frac{n}{2}\right)^2 - \frac{dn}{2}\right) + n = c n^2 - 2dn + n \\ &= c n^2 - d n - (d - 1) n \le c n^2 - d n \qquad\text{whenever } d \ge 1 . \end{aligned} \end{equation*}

Base case: take \(T(n) = \Theta(1)\), say \(T(n) \le k\), for \(1 \le n < 2\). With \(d = 1\) and \(n \ge 1\),

\begin{equation*} c n^2 - n = n(cn - 1) \ge c - 1 \ge k \qquad\text{if } c \ge k + 1 . \end{equation*}

Hence \(T(n) \le c n^2 - n = O(n^2)\) for all \(n \ge 1\).

\(T(n) = \Theta\left((\lg n)^{\lg 3}\right)\).

Put \(m = \lg n\) and \(S(m) = T(2^m)\); since \(\sqrt{n} = 2^{m/2}\) and \(\log n = \Theta(m)\) (whatever the base of the logarithm),

\begin{equation*} S(m) = 3S(m/2) + \Theta(m) . \end{equation*}

Here \(a = 3\), \(b = 2\), \(m^{\log_b a} = m^{\lg 3}\) with \(\lg 3 \approx 1.585\), and \(f(m) = \Theta(m) = O(m^{\lg 3 - \epsilon})\) for \(\epsilon = \lg 3 - 1 > 0\). Case 1 of the master theorem (Theorem 4.1) gives \(S(m) = \Theta(m^{\lg 3})\), and changing back,

\begin{equation*} T(n) = S(\lg n) = \Theta\left((\lg n)^{\lg 3}\right) . \end{equation*}

\(T(n) = O(n^{\lg 3})\).

Recursion tree: depth \(i\) has \(3^i\) nodes, each of size at most \(n/2^i\) (floors only shrink subproblems), so level \(i\) costs at most \((3/2)^i n\); the tree has about \(\lg n\) levels and \(3^{\lg n} = n^{\lg 3}\) leaves of constant cost. Hence

\begin{equation*} \begin{aligned} T(n) &\le \sum_{i=0}^{\lg n - 1} \left(\tfrac32\right)^i n + \Theta(n^{\lg 3}) = 2n\left(n^{\lg 3 - 1} - 1\right) + \Theta(n^{\lg 3}) \\ &= O(n^{\lg 3}), \end{aligned} \end{equation*}

using \((3/2)^{\lg n} = n^{\lg 3 - 1}\).

Substitution: the bare guess \(T(n) \le c n^{\lg 3}\) gives \(T(n) \le c n^{\lg 3} + n\), so we subtract a lower-order term and prove

\begin{equation*} T(n) \le c\, n^{\lg 3} - 4n \qquad (n \ge 1). \end{equation*}

Base cases: for \(1 \le n \le 5\), take \(c \ge \max_{1 \le n \le 5} (T(n) + 4n)/n^{\lg 3}\), a finite constant. Inductive step, \(n \ge 6\): then \(1 \le \lfloor n/2 \rfloor < n\), so the hypothesis applies to \(\lfloor n/2 \rfloor\), and with \(\lfloor n/2 \rfloor \le n/2\) and \(\lfloor n/2 \rfloor \ge (n-1)/2\),

\begin{equation*} \begin{aligned} T(n) &\le 3c \lfloor n/2 \rfloor^{\lg 3} - 12 \lfloor n/2 \rfloor + n \le 3c\, \frac{n^{\lg 3}}{3} - 6(n-1) + n \\ &= c\, n^{\lg 3} - 4n - (n - 6) \le c\, n^{\lg 3} - 4n . \end{aligned} \end{equation*}

Hence \(T(n) \le c n^{\lg 3}\) for all \(n \ge 1\), i.e. \(T(n) = O(n^{\lg 3})\).

\(T(n) = O(n^2)\) (indeed \(\Theta(n^2)\), since \(T(n) \ge n^2\)).

Recursion tree: a single path whose node at depth \(i\) has size \(n/2^i\) and cost \((n/2^i)^2 = n^2/4^i\), so

\begin{equation*} T(n) \le \sum_{i=0}^{\infty} \frac{n^2}{4^i} + \Theta(1) = \frac43 n^2 + \Theta(1) = O(n^2). \end{equation*}

Substitution: let the recurrence hold for \(n \ge 2\) and let \(T(n) \le B\) for \(1 \le n < 2\). We prove \(T(n) \le c n^2\) for all \(n \ge 1\) with \(c = \max(4/3, B)\). Base: for \(1 \le n < 2\), \(T(n) \le B \le c \le c n^2\). Step, \(n \ge 2\): \(n/2 \ge 1\), so

\begin{equation*} T(n) \le c \left(\frac n2\right)^2 + n^2 = \left(\frac c4 + 1\right) n^2 \le c n^2, \end{equation*}

because \(c/4 + 1 \le c \iff c \ge 4/3\).

\(T(n) = O(n^2)\).

Since \(n/2 + 2 < n\) only for \(n > 4\), and subproblem sizes drift toward the fixed point \(4\), take the recurrence to hold for \(n > 36\), with \(0 < T(n) \le B\) for \(20 < n \le 36\) (a child of \(n > 36\) has size \(n/2 + 2 > 20\), so every recursion lands in this range).

Recursion tree: depth \(i\) has \(4^i\) nodes, all of size \(s_i\) where \(s_0 = n\), \(s_{i+1} = s_i/2 + 2\), i.e. \(s_i = 4 + (n-4)/2^i\). Level \(i\) costs \(4^i s_i = 4^{i+1} + 2^i (n-4)\), and the depth \(k\) is the least \(k\) with \(s_k \le 36\), so \((n-4)/2^{k-1} > 32\), i.e. \(2^k < (n-4)/16 < n\). Hence

\begin{equation*} \begin{aligned} T(n) &\le \sum_{i=0}^{k-1}\left(4^{i+1} + 2^i n\right) + 4^k B \le \tfrac{4}{3}4^{k} + 2^k n + 4^k B \\ &= O(4^k + 2^k n) = O(n^2). \end{aligned} \end{equation*}

Substitution: \(T(n) \le c n^2\) fails (\(4c(n/2+2)^2 = c(n+4)^2\) leaves \(+8cn\)), so with \(c \ge 1\) we prove

\begin{equation*} T(n) \le c n^2 - (8c + 1) n \qquad (n > 20). \end{equation*}

Base, \(20 < n \le 36\): \(c n^2 - (8c+1)n = n\left(c(n - 8) - 1\right) > 20(12c - 1) \ge 220c\), which is \(\ge B\) once \(c \ge \max(1, B/220)\). Step, \(n > 36\): the child size \(m = n/2 + 2\) satisfies \(20 < m < n\), so

\begin{equation*} \begin{aligned} T(n) &\le 4\left[c\left(\tfrac n2 + 2\right)^2 - (8c+1)\left(\tfrac n2 + 2\right)\right] + n \\ &= c n^2 + 8cn + 16c - (8c+1)(2n + 8) + n \\ &= c n^2 - (8c+1) n - (48c + 8) \le c n^2 - (8c+1) n . \end{aligned} \end{equation*}

Hence \(T(n) \le c n^2\) for all \(n > 20\).

\(T(n) = O(2^n)\) (in fact \(T(n) = (T(0)+1)2^n - 1 = \Theta(2^n)\)).

Recursion tree: a complete binary tree of depth \(n\); level \(i\) has \(2^i\) nodes of cost \(1\) for \(i < n\), and there are \(2^n\) leaves of cost \(T(0)\), so

\begin{equation*} T(n) = \sum_{i=0}^{n-1} 2^i + 2^n T(0) = (T(0) + 1)2^n - 1 = O(2^n). \end{equation*}

Substitution: let the recurrence hold for \(n \ge 1\) with \(T(0) = t_0\). The bare guess \(c2^n\) gives \(c2^n + 1\), so we prove \(T(n) \le c 2^n - 1\) with \(c = t_0 + 1\). Base: \(T(0) = t_0 = c - 1\). Step, \(n \ge 1\):

\begin{equation*} T(n) \le 2\left(c 2^{n-1} - 1\right) + 1 = c 2^n - 1 . \end{equation*}

\(T(n) = O(n^{\lg n}) = O(2^{\lg^2 n})\).

Read \(n/2\) as \(\lfloor n/2 \rfloor\), let the recurrence hold for \(n \ge 4\), and let \(T(1), T(2), T(3)\) be positive constants.

Recursion tree: a node of size \(m\) costs \(m \le n\) and has children of sizes \(m - 1\) and \(\lfloor m/2 \rfloor\). A root-to-node path uses some number \(h \le \lg n\) of halving edges, separated into \(h + 1\) runs of decrement edges, each run of length at most \(n\); the node is determined by \(h\) and the run lengths, so

\begin{equation*} \#\text{nodes} \le \sum_{h=0}^{\lg n} (n+1)^{h+1} \le 2(n+1)^{\lg n + 1}, \qquad T(n) \le 2n^{\prime}(n+1)^{\lg n + 1} = n^{O(\lg n)} , \end{equation*}

where \(n^{\prime} = \max(n, T(1), T(2), T(3))\) bounds every node’s cost. This suggests the guess \(f(n) = n^{\lg n}\), which is increasing for \(n \ge 1\).

Substitution: we prove \(T(n) \le c\, n^{\lg n}\) for all \(n \ge 1\), with \(c = \max(4, T(1), T(2), T(3))\); the base cases \(n \le 3\) hold since \(n^{\lg n} \ge 1\). For \(n \ge 4\) put \(k = \lg n \ge 2\). By the mean value theorem and Bernoulli’s inequality, using \((k-1)/n \le 1/2\) for \(n \ge 4\),

\begin{equation*} \begin{aligned} f(n) - f(n-1) &\ge n^k - (n-1)^k \ge k (n-1)^{k-1} \\ &\ge k\, n^{k-1}\left(1 - \tfrac{k-1}{n}\right) \ge \tfrac{k}{2} n^{k-1}, \end{aligned} \end{equation*}

while \(f(\lfloor n/2 \rfloor) \le f(n/2) = (n/2)^{k-1} = 2 n^{k-2}\). Since \(2 \le \lfloor n/2 \rfloor < n - 1 < n\), the hypothesis gives

\begin{equation*} \begin{aligned} c f(n) - T(n) &\ge c\left(f(n) - f(n-1)\right) - c f(n/2) - n \\ &\ge c\, n^{k-2}\left(\tfrac{nk}{2} - 2\right) - n \ge 2c\, n^{k-2} - n \ge 0, \end{aligned} \end{equation*}

using \(nk/2 \ge 4\); the last step holds because \(n^{k-2} \ge n\) when \(n \ge 8\), and \(2c \ge 8 > n\) when \(4 \le n < 8\).

Every level of the recursion tree (Figure 4.6) down to depth \(\log_3 n\) is complete and costs exactly \(cn\), so \(T(n) \ge cn \log_3 n = \Omega(n \lg n)\).

Take \(c > 0\), let the recurrence hold for \(n \ge 1\), and let \(T(n) \ge 0\) for \(n < 1\). A node at depth \(i\) has size \(n (1/3)^a (2/3)^b\) with \(a + b = i\), hence size at least \(n/3^i\); this is the shortest path \(n \to n/3 \to n/9 \to \cdots\). So for \(i \le \log_3 n\) every node at depth \(i\) has size \(\ge 1\) and is internal, and since each internal node splits its size \(m\) into \(m/3 + 2m/3 = m\), the sizes at each such depth still sum to \(n\). Therefore depth \(i\) costs exactly \(cn\) for \(0 \le i \le \lfloor \log_3 n \rfloor\), and dropping all deeper levels,

\begin{equation*} T(n) \ge cn\left(\lfloor \log_3 n \rfloor + 1\right) \ge cn \log_3 n = \frac{c}{\lg 3}\, n \lg n = \Omega(n \lg n). \end{equation*}

\(T(n) = \Theta(n^2)\) (taking \(c > 0\), with \(T(1)\) a positive constant and the recurrence in force for \(n \ge 2\)).

Recursion tree (ignoring floors, \(n\) a power of \(2\)): depth \(i\) has \(4^i\) nodes of cost \(cn/2^i\), so level \(i\) costs \(2^i cn\); there are \(\lg n\) internal levels and \(4^{\lg n} = n^{\lg 4} = n^2\) leaves of cost \(\Theta(1)\).

\begin{equation*} \begin{aligned} T(n) &= \sum_{i=0}^{\lg n - 1} 2^i cn + \Theta(n^2) \\ &= cn(n-1) + \Theta(n^2) = \Theta(n^2). \end{aligned} \end{equation*}

Substitution, upper bound. Claim \(T(n) \le dn^2 - 2cn\) for all \(n \ge 2\), where \(d = \max\{(T(2)+4c)/4,\ (T(3)+6c)/9\}\). The base cases \(n = 2, 3\) hold by the choice of \(d\). For \(n \ge 4\) we have \(\lfloor n/2 \rfloor \ge 2\), so the hypothesis applies to \(\lfloor n/2 \rfloor\); using \((n-1)/2 \le \lfloor n/2 \rfloor \le n/2\),

\begin{equation*} \begin{aligned} T(n) &\le 4\bigl(d\lfloor n/2\rfloor^2 - 2c\lfloor n/2\rfloor\bigr) + cn \\ &\le dn^2 - 4c(n-1) + cn \\ &= dn^2 - 2cn - (cn - 4c) \le dn^2 - 2cn \qquad (n \ge 4). \end{aligned} \end{equation*}

Substitution, lower bound. Claim \(T(n) \ge d^{\prime}n^2\) for all \(n \ge 1\), where \(d^{\prime} = \min\{c/2,\ T(1)\}\). The base case \(n = 1\) holds since \(d^{\prime} \le T(1)\). For \(n \ge 2\),

\begin{equation*} \begin{aligned} T(n) &\ge 4d^{\prime}\lfloor n/2\rfloor^2 + cn \ge d^{\prime}(n-1)^2 + cn \\ &= d^{\prime}n^2 + \bigl(cn - d^{\prime}(2n-1)\bigr) \ge d^{\prime}n^2, \end{aligned} \end{equation*}

since \(d^{\prime}(2n-1) \le (c/2)(2n) = cn\). Hence \(T(n) = \Theta(n^2)\).

\(T(n) = \Theta(n^2)\); in fact \(T(n) = cn^2/(2a) + O(n)\) (taking \(T(n) = \Theta(1)\) for \(0 < n \le a\), so \(T(a)\) is a constant).

For \(n > a\) let \(k = \lceil n/a \rceil - 1 \ge 1\) and \(r = n - ka \in (0, a]\). The recursion tree is a spine: the node at depth \(i\) (\(0 \le i \le k-1\)) has subproblem size \(n - ia\) and cost \(c(n-ia)\), and two children, a leaf \(T(a)\) and the next spine node \(T(n-(i+1)a)\); the spine ends at depth \(k\) in the leaf \(T( r)\). Summing the tree, and writing \(n - ia = r + (k-i)a\),

\begin{equation*} \begin{aligned} T(n) &= c\sum_{i=0}^{k-1}(n - ia) + kT(a) + T( r) \\ &= c\Bigl(kr + \frac{a\,k(k+1)}{2}\Bigr) + kT(a) + T( r). \end{aligned} \end{equation*}

Since \(n/a - 1 \le k \le n/a\), \(0 < r \le a\) and \(T( r) = \Theta(1)\),

\begin{equation*} \frac{c\,n(n-a)}{2a} \le T(n) \le \frac{c\,n(n+a)}{2a} + cn + \frac{n}{a}T(a) + O(1). \end{equation*}

Both sides are \(cn^2/(2a) + O(n)\) (\(a\) and \(c\) are constants), so \(T(n) = \Theta(n^2)\).

\(T(n) = \Theta(n \lg n)\) (taking \(T(n) = \Theta(1)\), positive, for \(n < 1\), and the recurrence for \(n \ge 1\)).

Put \(\beta = \min\{\alpha, 1-\alpha\}\) and \(\gamma = \max\{\alpha, 1-\alpha\}\), so \(0 < \beta \le 1/2 \le \gamma < 1\). As in Figure 4.6, a node at depth \(i\) has size \(\alpha^j(1-\alpha)^{i-j}n \in [\beta^i n, \gamma^i n]\); a node is internal (cost \(c\) times its size) when its size is \(\ge 1\) and a leaf otherwise. The sizes of the nodes at any depth sum to at most \(n\), with equality as long as no leaf lies above that depth.

Upper bound. Internal nodes have \(\gamma^i n \ge 1\), so they occupy depths \(0 \le i \le \log_{1/\gamma} n\), and each depth costs at most \(cn\). Each leaf has size \(\ge \beta\) (its parent has size \(\ge 1\)), and the leaves partition the root’s size \(n\), so there are at most \(n/\beta\) leaves. Hence

\begin{equation*} T(n) \le cn\bigl(\log_{1/\gamma} n + 1\bigr) + \frac{n}{\beta}\,\Theta(1) = O(n \lg n). \end{equation*}

Lower bound. For \(0 \le i \le \log_{1/\beta} n\) every node at depth \(i\) has size \(\ge \beta^i n \ge 1\), so these depths are complete levels of internal nodes, each costing exactly \(cn\). Hence

\begin{equation*} T(n) \ge cn\bigl(\lfloor \log_{1/\beta} n \rfloor + 1\bigr) \ge \frac{c}{\lg(1/\beta)}\, n \lg n = \Omega(n \lg n). \end{equation*}

Therefore \(T(n) = \Theta(n \lg n)\).

(a) \(\Theta(\sqrt n)\), (b) \(\Theta(\sqrt n\lg n)\), (c) \(\Theta(n)\), (d) \(\Theta(n^2)\). In all four, \(a=2\), \(b=4\), so \(n^{\log_b a} = n^{\log_4 2} = n^{1/2}\), and Theorem 4.1 gives:

(a) \(f(n) = 1 = O(n^{1/2-\epsilon})\) with \(\epsilon = 1/2\): case 1, \(T(n) = \Theta(\sqrt n)\).

(b) \(f(n) = \sqrt n = \Theta(n^{1/2})\): case 2, \(T(n) = \Theta(\sqrt n \lg n)\).

(c) \(f(n) = n = \Omega(n^{1/2+\epsilon})\) with \(\epsilon = 1/2\), and regularity holds with \(c = 1/2\): \[ af(n/b) = 2\cdot\frac n4 = \frac12\, n = c f(n). \] Case 3: \(T(n) = \Theta(n)\).

(d) \(f(n) = n^2 = \Omega(n^{1/2+\epsilon})\) with \(\epsilon = 3/2\), and regularity holds with \(c = 1/8\): \[ af(n/b) = 2\left(\frac n4\right)^2 = \frac18\, n^2 = c f(n). \] Case 3: \(T(n) = \Theta(n^2)\).

\(a = 48\). Here \(n^{\log_b a} = n^{\log_4 a}\), and the threshold is \[ \log_4 a = \lg 7 \iff a = 4^{\lg 7} = (2^{\lg 7})^2 = 49 . \] (i) \(a = 48\): \(\log_4 48 \approx 2.79\), so \(f(n) = \Theta(n^2) = O(n^{\log_4 48 - \epsilon})\) with \(\epsilon = \log_4 48 - 2 > 0\); case 1 of Theorem 4.1 gives \(T(n) = \Theta(n^{\log_4 48})\), and \(\log_4 48 < \log_4 49 = \lg 7\), so \(T(n) = o(n^{\lg 7})\).

(ii) \(a \ge 49\): \(\log_4 a \ge \lg 7 > 2\), so again case 1 applies and \(T(n) = \Theta(n^{\log_4 a}) = \Omega(n^{\lg 7})\), which is not asymptotically faster than Strassen.

Case 2 of Theorem 4.1 applies: with \(a = 1 \ge 1\), \(b = 2 > 1\), \[ n^{\log_b a} = n^{\log_2 1} = 1, \qquad f(n) = \Theta(1) = \Theta(n^{\log_b a}), \] so \(T(n) = \Theta(n^{\log_b a}\lg n) = \Theta(\lg n)\).

No; but \(T(n) = O(n^2\lg^2 n)\) (in fact \(\Theta(n^2\lg^2 n)\)). Here \(a = 4\), \(b = 2\), \(n^{\log_b a} = n^2\), and \[ \frac{f(n)}{n^{\log_b a}} = \lg n = \omega(1), \qquad \lg n = o(n^{\epsilon})\ \ (\epsilon > 0) . \] So \(f(n)\) is neither \(O(n^{2-\epsilon})\) (case 1) nor \(\Theta(n^2)\) (case 2) nor \(\Omega(n^{2+\epsilon})\) (case 3): \(f\) is larger than \(n^{\log_b a}\) but not polynomially larger, the gap between cases 2 and 3.

For the bound, take \(n = 2^k\) and sum the recursion tree exactly, as in Lemma 4.2: level \(j\) has \(4^j\) nodes of cost \((n/2^j)^2\lg(n/2^j) = 4^{k-j}(k-j)\), and there are \(4^k = n^2\) leaves, so

\begin{equation*} \begin{aligned} T(n) &= n^2\,T(1) + \sum_{j=0}^{k-1} 4^j\cdot 4^{k-j}(k-j) = n^2\,T(1) + 4^k\sum_{i=1}^{k} i \\ &= n^2\left(T(1) + \frac{\lg n\,(\lg n+1)}{2}\right) = \Theta(n^2\lg^2 n) . \end{aligned} \end{equation*}

Floors and ceilings are handled as in Section 4.6.2 (see also Exercise 4.6-2), so \(T(n) = O(n^2\lg^2 n)\).

Take \(a = 1\), \(b = 2\) and \(f(n) = 4^{\lceil \log_4 n\rceil}\), the least power of 4 that is \(\ge n\). Then \(n^{\log_b a} = n^0 = 1\) and \[ f(n) \ge n = \Omega(n^{\log_b a + \epsilon}) \quad\text{with } \epsilon = 1, \] so the growth condition of case 3 holds. But for every \(n = 4^k\) with \(k \ge 1\), the argument \(n/2 = 2\cdot 4^{k-1}\) is an integer (so floors and ceilings change nothing) lying in \((4^{k-1}, 4^k]\), hence \[ af(n/b) = f(2\cdot 4^{k-1}) = 4^k = f(n) > cf(n) \quad\text{for every } c < 1 , \] for arbitrarily large \(n\), so regularity fails.

\(n_j = \lceil n/b^j \rceil\) for every \(j \ge 0\).

By induction on \(j\): the case \(j = 0\) is \(n_0 = n = \lceil n/b^0 \rceil\) (\(n\) is an integer), and if \(n_{j-1} = \lceil n/b^{j-1} \rceil\) then equation (3.4), \(\lceil \lceil x/a \rceil / b \rceil = \lceil x/(ab) \rceil\) for real \(x \ge 0\) and positive integers \(a, b\), applied with \(x = n\) and \(a = b^{j-1}\) (a positive integer because \(b\) is) gives

\begin{equation*} n_j = \left\lceil \frac{\lceil n/b^{j-1} \rceil}{b} \right\rceil = \left\lceil \frac{n}{b^{j}} \right\rceil . \end{equation*}

Write \(p = \log_b a\), \(n = b^L\), and sum the recursion tree: by Lemma 4.2 (whose hypothesis is that \(f\) is nonnegative, as a driving function of the master recurrence is),

\begin{equation*} T(n) = \Theta(n^{p}) + \sum_{j=0}^{L-1} a^j f(b^{L-j}) . \end{equation*}

By hypothesis there are constants \(c_1, c_2 > 0\) and an integer \(i_0 \ge 1\) with \(c_1 m^p \lg^k m \le f(m) \le c_2 m^p \lg^k m\) for all \(m = b^i\), \(i \ge i_0\). Since \(a^j (b^{L-j})^p = a^j a^{L-j} = n^p\) and \(\lg b^{i} = i \lg b\), splitting the sum at \(i = L - j = i_0\) gives

\begin{equation*} \begin{aligned} \sum_{j=0}^{L-1} a^j f(b^{L-j}) &= \Theta\Big(n^p \lg^k b \sum_{i=i_0}^{L} i^k\Big) + \sum_{i=1}^{i_0-1} a^{L-i} f(b^{i}) \\ &= \Theta\Big(n^p \sum_{i=i_0}^{L} i^k\Big) + O(n^p), \end{aligned} \end{equation*}

because the second sum has \(i_0 - 1\) terms, each at most \(a^L \max_{i < i_0} f(b^i) = O(n^p)\). Finally, for \(L \ge 2 i_0\) (using \(k \ge 0\)),

\begin{equation*} \begin{aligned} \Big(\frac{L}{2}\Big)^{k+1} &\le \sum_{\lceil L/2 \rceil \le i \le L} i^k \\ &\le \sum_{i=i_0}^{L} i^k \le L \cdot L^k = L^{k+1}, \end{aligned} \end{equation*}

so the sum is \(\Theta(L^{k+1}) = \Theta(\lg^{k+1} n)\), as \(L = \lg n / \lg b\). Hence

\begin{equation*} \begin{aligned} T(n) &= \Theta(n^p) + \Theta(n^p \lg^{k+1} n) + O(n^p) \\ &= \Theta(n^{\log_b a} \lg^{k+1} n), \end{aligned} \end{equation*}

the last step because \(\lg^{k+1} n \ge 1\) for \(n \ge 2\).

Take \(\epsilon = \log_b(1/c)\), which is positive because \(b > 1\) and \(0 < c < 1\) (\(c > 0\) since \(af(n/b) > 0\) for large \(n\)); then \(a/c = b^{\log_b a + \epsilon} > 1\).

As in Theorem 4.1, \(n/b\) means \(\lfloor n/b \rfloor\) or \(\lceil n/b \rceil\), so \(f\) lives on the integers. Fix \(N\) with regularity holding and \(f > 0\) on all integers \(m \ge N/b - 1\), and let \(d > 0\) be the minimum of \(f\) over the finitely many integers in \([N/b - 1, N)\). For \(n \ge N\) iterate \(m_0 = n\), \(m_{i+1} = m_i/b\) (rounded), and let \(J\) be the first index with \(m_J < N\); then \(m_J \ge N/b - 1\). Unrolling the nested roundings as in Section 4.6.2 gives \(m_J > n/b^J - b/(b-1)\), so \(b^J > n/K\) with \(K = N + b/(b-1)\). Applying \(f(m_i) \ge (a/c) f(m_{i+1})\) for \(i < J\) (each \(m_i \ge N\)),

\begin{equation*} \begin{aligned} f(n) &\ge \Big(\frac{a}{c}\Big)^{J} f(m_J) \ge d \, (b^J)^{\log_b a + \epsilon} \\ &\ge \frac{d}{K^{\log_b a + \epsilon}} \, n^{\log_b a + \epsilon} , \end{aligned} \end{equation*}

so \(f(n) = \Omega(n^{\log_b a + \epsilon})\).

All bounds are tight: (a) \(\Theta(n^4)\), (b) \(\Theta(n)\), (c) \(\Theta(n^2\lg n)\), (d) \(\Theta(n^2)\), (e) \(\Theta(n^{\lg 7})\), (f) \(\Theta(\sqrt{n}\lg n)\), (g) \(\Theta(n^3)\). Parts (a) to (f) use the master theorem (Theorem 4.1) with \(a \ge 1\), \(b > 1\) in each case.

(a) \(a=b=2\), \(n^{\log_b a} = n\), and \(f(n) = n^4 = \Omega(n^{1+3})\). Regularity: \(2(n/2)^4 = n^4/8 \le \tfrac18 f(n)\). Case 3 gives \(T(n) = \Theta(n^4)\).

(b) \(a=1\), \(b = 10/7\), \(n^{\log_b a} = n^0 = 1\), and \(f(n) = n = \Omega(n^{0+1})\). Regularity: \(7n/10 \le \tfrac{7}{10} f(n)\). Case 3 gives \(T(n) = \Theta(n)\).

(c) \(a=16\), \(b=4\), \(n^{\log_4 16} = n^2\) and \(f(n) = n^2 = \Theta(n^2)\). Case 2 gives \(T(n) = \Theta(n^2 \lg n)\).

(d) \(a=7\), \(b=3\), \(\log_3 7 \approx 1.77 < 2\), so \(f(n) = n^2 = \Omega(n^{\log_3 7 + \epsilon})\) with \(\epsilon = 2 - \log_3 7 > 0\). Regularity: \(7(n/3)^2 = \tfrac79 f(n)\). Case 3 gives \(T(n) = \Theta(n^2)\).

(e) \(a=7\), \(b=2\), \(\lg 7 \approx 2.81 > 2\), so \(f(n) = n^2 = O(n^{\lg 7 - \epsilon})\) with \(\epsilon = \lg 7 - 2 > 0\). Case 1 gives \(T(n) = \Theta(n^{\lg 7})\).

(f) \(a=2\), \(b=4\), \(n^{\log_4 2} = n^{1/2}\) and \(f(n) = \sqrt{n} = \Theta(n^{1/2})\). Case 2 gives \(T(n) = \Theta(\sqrt{n}\lg n)\).

(g) The master theorem does not apply; unroll instead. Let \(k = \lceil (n-2)/2 \rceil\) be the number of steps that bring the argument down to at most \(2\), and let \(c = \max_{m \le 2} T(m)\). Then

\begin{equation*} T(n) = T(n-2k) + \sum_{j=0}^{k-1} (n-2j)^2 . \end{equation*}

Upper bound: there are \(k \le n/2\) terms, each at most \(n^2\), so \(T(n) \le c + n^3/2\). Lower bound: for \(n \ge 8\) we have \(k - 1 \ge n/2 - 2 \ge n/4\), so the sum includes every \(j\) with \(0 \le j \le n/4\). That is at least \(n/4\) terms, and each has \(n - 2j \ge n/2\), so

\begin{equation*} T(n) \ge \frac{n}{4}\Big(\frac{n}{2}\Big)^2 = \frac{n^3}{16}. \end{equation*}

Hence \(T(n) = \Theta(n^3)\).

(a) Binary search costs \(\Theta(\lg N)\), \(\Theta(N\lg N)\) and \(\Theta(N)\) under methods 1, 2 and 3. Each call does \(\Theta(1)\) work, then makes at most one recursive call on a subarray of size at most \(\lfloor n/2 \rfloor\). In the worst case (an unsuccessful search) the recursion goes all the way down. So the recurrences are:

(i) Pointer: \(T(n) = T(n/2) + \Theta(1)\). Case 2 of Theorem 4.1 (\(a=1\), \(b=2\), \(n^{\log_2 1} = 1\)) gives \(T(N) = \Theta(\lg N)\).

(ii) Full copy: \(T(n) = T(n/2) + \Theta(N)\), where \(N\) is a constant throughout the recursion. With \(T(n) \le T(n/2) + cN\) there are \(\lfloor \lg n \rfloor + 1\) levels, so

\begin{equation*} T(N) \le cN(\lg N + 1) + O(1) = O(N \lg N), \end{equation*}

and the same count with a lower constant gives \(\Omega(N\lg N)\). Hence \(T(N) = \Theta(N\lg N)\).

(iii) Subrange copy: copying the half of size about \(n/2\) costs \(\Theta(n)\), so \(T(n) = T(n/2) + \Theta(n)\). Here \(f(n) = n = \Omega(n^{0+1})\) and the regularity condition holds, since \(f(n/2) = \tfrac12 f(n)\). Case 3 of Theorem 4.1 gives \(T(N) = \Theta(N)\).

(b) Merge sort costs \(\Theta(N\lg N)\), \(\Theta(N^2)\) and \(\Theta(N\lg N)\) under methods 1, 2 and 3. A call on \(n > 1\) elements makes two recursive calls on halves and one call to MERGE, which does \(\Theta(n)\) work on top of what it costs to pass the array.

(i) Pointer: \(T(n) = 2T(n/2) + \Theta(n)\). Case 2 of Theorem 4.1 gives \(T(N) = \Theta(N\lg N)\).

(ii) Full copy: each of the three calls costs \(\Theta(N)\) to pass the array, so \(T(n) = 2T(n/2) + \Theta(n) + \Theta(N)\). Take \(n = 2^k\) with \(T(n) = 2T(n/2) + cn + dN\). Level \(i\) of the recursion tree has \(2^i\) nodes, each costing \(c\,n/2^i + dN\), and there are \(n\) leaves. So

\begin{equation*} \begin{aligned} T(n) &= \sum_{i=0}^{k-1} \big(cn + 2^i dN\big) + n\,T(1) \\ &= cn\lg n + dN(n-1) + \Theta(n). \end{aligned} \end{equation*}

Setting \(n = N\) gives \(T(N) = \Theta(N^2)\).

(iii) Subrange copy: passing \(A[p \,.\,.\, q]\) to the two recursive calls and to MERGE costs \(\Theta(n)\) in total. So \(T(n) = 2T(n/2) + \Theta(n)\), the same recurrence as in (i), and \(T(N) = \Theta(N\lg N)\).

All ten bounds are tight: (a) \(\Theta(n^{\log_3 4})\), (b) \(\Theta(n\lg\lg n)\), (c) \(\Theta(n^{5/2})\), (d) \(\Theta(n\lg n)\), (e) \(\Theta(n\lg\lg n)\), (f) \(\Theta(n)\), (g) \(\Theta(\lg n)\), (h) \(\Theta(n\lg n)\), (i) \(\Theta(n/\lg n)\), (j) \(\Theta(n\lg\lg n)\).

(a) Here \(a=4\), \(b=3\), \(n^{\log_3 4} \approx n^{1.262}\). Since \(\lg n = O(n^{0.1})\), \(f(n) = n\lg n = O(n^{\log_3 4 - \epsilon})\) with \(\epsilon = 0.1\), so case 1 of the master theorem (Theorem 4.1) gives \(T(n) = \Theta(n^{\log_3 4})\).

(b) The master theorem does not apply: \(n/\lg n\) is \(o(n)\) but not \(O(n^{1-\epsilon})\). Take \(n = 3^k\). By Lemma 4.2 (with \(f\) only evaluated at \(n/3^j\), \(j \le k-1\), so never at \(1\)),

\begin{equation*} \begin{aligned} T(n) &= \Theta(n) + \sum_{j=0}^{k-1} 3^j \frac{n/3^j}{\lg(n/3^j)} = \Theta(n) + \sum_{j=0}^{k-1} \frac{n}{(k-j)\lg 3} \\ &= \Theta(n) + \frac{n}{\lg 3} H_k = \Theta(n) + \frac{n}{\lg 3}\bigl(\ln k + O(1)\bigr) = \Theta(n\lg\lg n), \end{aligned} \end{equation*}

using \(H_k = \ln k + O(1)\) (equation (A.7)) and \(k = \log_3 n\). For general \(n\), \(T\) is nondecreasing (as it is for any reasonable reading with floors or ceilings), and sandwiching \(3^k \le n < 3^{k+1}\) changes \(n\lg\lg n\) by at most a constant factor.

(c) Here \(n^{\log_2 4} = n^2\) and \(f(n) = n^{5/2} = \Omega(n^{2+\epsilon})\) with \(\epsilon = 1/2\). Regularity: \(4f(n/2) = 4(n/2)^{5/2} = n^{5/2}/\sqrt{2}\), so \(af(n/b) \le c f(n)\) with \(c = 1/\sqrt{2} < 1\). Case 3 of Theorem 4.1 gives \(T(n) = \Theta(n^{5/2})\).

(d) \(T(n) = \Theta(n\lg n)\), by substitution. Fix the base region as \(2 \le n < 12\), where \(C^{\prime} \le T(n) \le C\) for positive constants (any bounded range of arguments has bounded, positive \(T\)); for \(n \ge 12\) the argument \(m = n/3 - 2\) satisfies \(n/6 \le m < n\) and \(m \ge 2\), and arguments drop by at least \(2\) per step, so induction on \(n\) is valid.

Upper bound: claim \(T(n) \le c\,n\lg n\) for \(n \ge 2\) with \(c = \max(C, 1/(2\lg 3))\). Base: \(T(n) \le C \le c\,n\lg n\) as \(n\lg n \ge 2\). Step (\(n \ge 12\), so \(n - 6 \ge 0\) and \(\lg(n/3) \ge 0\)):

\begin{equation*} \begin{aligned} T(n) &\le 3c\,m\lg m + n/2 \le c(n-6)\lg(n/3) + n/2 \\ &\le c\,n\lg n - (c\lg 3 - 1/2)\,n \le c\,n\lg n. \end{aligned} \end{equation*}

Lower bound: claim \(T(n) \ge d\,n\lg n\) for \(n \ge 2\) with \(d = \min\bigl(C^{\prime}/(12\lg 12),\ 1/(2(\lg 6 + 1))\bigr)\). Base: \(T(n) \ge C^{\prime} \ge d\,n\lg n\) for \(n < 12\). Step, using \(m \ge n/6\) and \(\lg(n/6) \le n/6\):

\begin{equation*} \begin{aligned} T(n) &\ge 3d\,m\lg m + n/2 \ge d(n-6)\lg(n/6) + n/2 \\ &= d\,n\lg n - d\,n\lg 6 - 6d\lg(n/6) + n/2 \\ &\ge d\,n\lg n - \bigl(d(\lg 6 + 1) - 1/2\bigr)n \ge d\,n\lg n. \end{aligned} \end{equation*}

(e) The same computation as (b) with \(a = b = 2\) and \(n = 2^k\): by Lemma 4.2,

\begin{equation*} T(n) = \Theta(n) + \sum_{j=0}^{k-1} \frac{n}{k-j} = \Theta(n) + nH_k = \Theta(n\lg\lg n), \end{equation*}

since \(k = \lg n\) and \(H_k = \ln k + O(1)\) by (A.7); monotonicity extends this to all \(n\) as in (b).

(f) \(T(n) = \Theta(n)\). The lower bound \(T(n) \ge n\) is immediate. For the upper bound claim \(T(n) \le c\,n\) for \(n \ge 1\), with \(c \ge 8\) large enough to cover the base cases; then

\begin{equation*} T(n) \le c\frac{n}{2} + c\frac{n}{4} + c\frac{n}{8} + n = \frac{7c}{8}n + n \le c\,n \quad\text{since } c \ge 8. \end{equation*}

(g) Unrolling down to the constant base value \(T(n_0)\),

\begin{equation*} T(n) = T(n_0) + \sum_{k=n_0+1}^{n} \frac{1}{k} = H_n - H_{n_0} + O(1) = \ln n + O(1) = \Theta(\lg n) \end{equation*}

by (A.7).

(h) Unrolling likewise,

\begin{equation*} T(n) = T(n_0) + \sum_{k=n_0+1}^{n} \lg k = \lg(n!) + O(1) = \Theta(n\lg n) \end{equation*}

by equation (3.19).

(i) \(T(n) = \Theta(n/\lg n)\). Take \(T\) constant for \(n < 4\). Unrolling gives \(T(n) = \Theta(1) + \sum_{k \in K} 1/\lg k\) with \(K = \{n, n-2, n-4, \ldots\} \cap [4, n]\), and \((n-4)/2 \le |K| \le n/2\). Each term is at least \(1/\lg n\), so the sum is at least \((n-4)/(2\lg n)\). For the upper bound, split at \(\sqrt{n}\): terms with \(k \le \sqrt{n}\) are at most \(1/2\) each (as \(\lg k \ge 2\)), and terms with \(k > \sqrt{n}\) satisfy \(1/\lg k < 2/\lg n\), so

\begin{equation*} \sum_{k \in K} \frac{1}{\lg k} \le \frac{\sqrt{n}}{2} + \frac{n}{2}\cdot\frac{2}{\lg n} = O\!\left(\frac{n}{\lg n}\right). \end{equation*}

(j) \(T(n) = \Theta(n\lg\lg n)\). Divide by \(n\): \(S(n) = T(n)/n\) satisfies \(S(n) = S(\sqrt{n}) + 1\). Let \(T\) be constant for \(n \le n_0\), where \(n_0 > 1\), and let \(k\) be the least integer with \(n^{1/2^k} \le n_0\), that is,

\begin{equation*} k = \bigl\lceil \lg\lg n - \lg\lg n_0 \bigr\rceil . \end{equation*}

Iterating \(k\) times gives \(S(n) = k + S(m)\) with \(m = n^{1/2^k} \in (\sqrt{n_0}, n_0]\), a fixed bounded interval on which \(T(m)/m\) is bounded above and below by positive constants. Hence \(S(n) = \lg\lg n + \Theta(1)\), so \(T(n) = n\lg\lg n + \Theta(n) = \Theta(n\lg\lg n)\).

The generating function is \(\mathcal{F}(z) = z/(1-z-z^2)\), which gives \(F_i = (\phi^i - \hat\phi^i)/\sqrt5\), the integer nearest to \(\phi^i/\sqrt5\).

(a) Shift indices and use \(F_0 = 0\), \(F_1 = 1\) and \(F_i = F_{i-1} + F_{i-2}\) for \(i \ge 2\) (recurrence (3.22)):

\begin{equation*} \begin{aligned} z + z\mathcal{F}(z) + z^2\mathcal{F}(z) &= z + \sum_{i \ge 1} F_{i-1} z^i + \sum_{i \ge 2} F_{i-2} z^i \\ &= z + F_0 z + \sum_{i \ge 2} (F_{i-1} + F_{i-2}) z^i \\ &= F_0 + F_1 z + \sum_{i \ge 2} F_i z^i = \mathcal{F}(z). \end{aligned} \end{equation*}

(b) By (a), \(\mathcal{F}(z)(1 - z - z^2) = z\). The series \(1 - z - z^2\) has constant term \(1\), so it is invertible in the ring of formal power series, and \(\mathcal{F}(z) = z/(1-z-z^2)\). Since \(\phi\) and \(\hat\phi\) are the roots of \(x^2 = x + 1\), we have \(\phi + \hat\phi = 1\) and \(\phi\hat\phi = -1\), hence

\begin{equation*} (1-\phi z)(1-\hat\phi z) = 1 - (\phi+\hat\phi)z + \phi\hat\phi z^2 = 1 - z - z^2 , \end{equation*}

which is the second equality. For the third, \(\phi - \hat\phi = \sqrt5\) gives

\begin{equation*} \frac{1}{1-\phi z} - \frac{1}{1-\hat\phi z} = \frac{(1-\hat\phi z) - (1-\phi z)}{(1-\phi z)(1-\hat\phi z)} = \frac{\sqrt5\, z}{(1-\phi z)(1-\hat\phi z)} . \end{equation*}

(c) For any constant \(\alpha\), \((1-\alpha z)\sum_{i\ge0}\alpha^i z^i = 1\) (the sum telescopes), so \(1/(1-\alpha z) = \sum_{i \ge 0} \alpha^i z^i\) as formal series. Applying this with \(\alpha = \phi\) and \(\alpha = \hat\phi\) in the last form of (b),

\begin{equation*} \mathcal{F}(z) = \frac{1}{\sqrt5}\left(\sum_{i\ge0}\phi^i z^i - \sum_{i\ge0}\hat\phi^i z^i\right) = \sum_{i \ge 0} \frac{1}{\sqrt5}\bigl(\phi^i - \hat\phi^i\bigr) z^i . \end{equation*}

(d) Two formal power series are equal exactly when their coefficients agree, so comparing the coefficient of \(z^i\) in (c) with the definition of \(\mathcal{F}\) gives \(F_i = (\phi^i - \hat\phi^i)/\sqrt5\). Since \(|\hat\phi| < 1\), for \(i > 0\)

\begin{equation*} \left| F_i - \frac{\phi^i}{\sqrt5} \right| = \frac{|\hat\phi|^i}{\sqrt5} < \frac{1}{\sqrt5} < \frac12 . \end{equation*}

So the integer \(F_i\) lies strictly within \(1/2\) of \(\phi^i/\sqrt5\), which means it is that number rounded to the nearest integer.

(a) Let \(G\), \(B\) be the true good and bad sets, \(|B| \ge n/2\), \(n \ge 2\). The bad chips pick \(G^{\prime} \subseteq B\) with \(G^{\prime} \ne G\) and \(|G^{\prime}| \le n/2\), and every bad chip answers as if \(G^{\prime}\) were the good set; then the professor sees the same answers in the true world and in the world with good set \(G^{\prime}\). Such a \(G^{\prime}\) exists:

(i) if \(G \ne \emptyset\), take any \(G^{\prime} \subseteq B\) with \(|G^{\prime}| = |G| \le n/2 \le |B|\); it is nonempty and disjoint from \(G\), so \(G^{\prime} \ne G\);

(ii) if \(G = \emptyset\), take \(G^{\prime} = \{c\}\) for any chip \(c\); \(1 \le n/2\).

In the real world a chip in \(G\) says “good” exactly about chips of \(G\) (truthfully), and every chip of \(B\) says “good” exactly about chips of \(G^{\prime}\). In the world with good set \(G^{\prime}\), the chips of \(G^{\prime} \subseteq B\) answer by that same rule, which is now truthful, while all other chips (\(G\) and \(B \setminus G^{\prime}\)) are bad there and may give any answers, in particular the answers above. This world also satisfies the hypothesis, since it has \(n - |G^{\prime}| \ge n/2\) bad chips. So every test turns out the same in both worlds, and the professor cannot tell whether the good set is \(G\) or \(G^{\prime} \ne G\).

(b) Split the chips into \(\lfloor n/2 \rfloor\) disjoint pairs, leaving one chip \(x\) over if \(n\) is odd, and test each pair. Throw away every pair in which either chip says “bad”, and keep one chip from each pair in which both say “good”. If \(n\) is odd, keep \(x\) exactly when the number of kept chips is even. The kept set has more good chips than bad ones and at most \(\lceil n/2 \rceil\) chips.

A pair containing a good chip and a bad chip is always thrown away, because the good chip says “bad”. Let \(a\) be the number of good-good pairs and \(b\) the number of bad-bad pairs that both say “good”. Let \(m\) be the number of mixed pairs and \(b^{\prime}\) the number of thrown-away bad-bad pairs, and write \([\cdot]\) for an indicator. The kept chips from pairs are \(a\) good and \(b\) bad, and the majority hypothesis reads

\begin{equation*} 2a + m + [x \text{ good}] > 2b + 2b^{\prime} + m + [x \text{ bad}], \quad\text{so}\quad 2a + [x\text{ good}] > 2b + [x \text{ bad}]. \end{equation*}

(i) \(n\) even (no \(x\)): \(a > b\).

(ii) \(x\) good: \(a \ge b\). If \(a+b\) is odd, then \(a \ne b\), so \(a > b\) and \(x\) is dropped. If \(a+b\) is even, \(x\) is kept and \(a + 1 > b\).

(iii) \(x\) bad: \(a > b\). If \(a+b\) is odd, \(x\) is dropped. If \(a+b\) is even, then \(a \ge b+2\), so \(a > b + 1\) with \(x\) kept.

In every case good chips are a strict majority of the kept set, so the set is nonempty. Its size is at most \(a + b + [n \text{ odd}] \le \lceil n/2 \rceil\).

(c) Repeat (b) until one chip is left. That chip is good, because good chips are a strict majority. Next, test it against each of the other \(n-1\) chips; its reports are truthful, so they identify every good chip. With \(T(n)\) the worst-case number of tests in the first phase,

\begin{equation*} T(n) \le T(\lceil n/2 \rceil) + \lfloor n/2 \rfloor \quad (n \ge 2), \qquad T(1) = 0, \end{equation*}

and \(T(n) \le n - 1\) by induction (\(\lceil n/2 \rceil < n\) for \(n \ge 2\)):

\begin{equation*} T(n) \le \bigl(\lceil n/2 \rceil - 1\bigr) + \lfloor n/2 \rfloor = n - 1 . \end{equation*}

So at most \((n-1) + (n-1) = 2n - 2 = O(n)\) tests are used.

For the matching lower bound with \(n \ge 3\), suppose every chip is good, so every report is “good”. If some chip \(c\) is never tested, the world in which only \(c\) is bad gives the same reports and still has \(n - 1 > n/2\) good chips. Hence every chip must be tested, which takes at least \(\lceil n/2 \rceil\) tests, and the total is \(\Theta(n)\).

(a) The “only if” direction is the special case \(k = i+1\), \(l = j+1\). For “if”, write \(D(r,s) = A[r,s] + A[r+1,s+1] - A[r,s+1] - A[r+1,s]\), which is \(\le 0\) by hypothesis. For \(i < k\) and \(j < l\), the double sum telescopes, first over \(s\) and then over \(r\):

\begin{equation*} \begin{aligned} \sum_{r=i}^{k-1} \sum_{s=j}^{l-1} D(r,s) &= \sum_{r=i}^{k-1} \bigl(A[r,j] + A[r+1,l] - A[r,l] - A[r+1,j]\bigr) \\ &= A[i,j] + A[k,l] - A[i,l] - A[k,j], \end{aligned} \end{equation*}

and the sum is \(\le 0\) because every term is.

(b) Changing \(A[2,3]\) from \(7\) to \(5\) makes the array Monge. The adjacent differences \(D(i,j)\) of part (a) for the given array are

The only violation is \(D(1,2) = 23 + 7 - 22 - 6 = 2\). The entry \(A[2,3]\) appears in \(D(1,2)\) and \(D(2,3)\) with sign \(+\), and in \(D(1,3)\) and \(D(2,2)\) with sign \(-\). Lowering it by \(2\) gives

\begin{equation*} D(1,2) = 0,\quad D(2,3) = -4,\quad D(1,3) = -5,\quad D(2,2) = -3, \end{equation*}

so all \(D(i,j) \le 0\), and by (a) the new array is Monge.

(c) Suppose \(f(i) > f(i+1)\) for some \(i\), and put \(j = f(i+1) < l = f(i)\). Since \(l\) is the leftmost minimum of row \(i\) and \(j < l\), we have \(A[i,j] > A[i,l]\). Since \(j\) is a minimum of row \(i+1\), we have \(A[i+1,l] \ge A[i+1,j]\). Adding the two,

\begin{equation*} A[i,j] + A[i+1,l] > A[i,l] + A[i+1,j], \end{equation*}

which contradicts the Monge inequality for rows \(i < i+1\) and columns \(j < l\). Hence \(f(i) \le f(i+1)\) for every \(i\).

(d) For each odd row \(i\), scan row \(i\) only over columns \(f(i-1)\) through \(f(i+1)\), with the conventions \(f(0) = 1\) and \(f(m+1) = n\), and take the leftmost minimum found there.

The answer is correct because \(f(i-1) \le f(i) \le f(i+1)\) by (c), applied to the Monge array \(A\), so the true leftmost minimum \(f(i)\) lies in the scanned range. Any scanned column left of \(f(i)\) holds an entry larger than the row minimum, so the leftmost minimum of the range is \(f(i)\).

Row \(i\) costs \(f(i+1) - f(i-1) + 1\) comparisons. The ranges for consecutive odd rows share only their endpoints, so the total telescopes:

\begin{equation*} \begin{aligned} \sum_{i \text{ odd}} \bigl(f(i+1) - f(i-1) + 1\bigr) &\le f(m+1) - f(0) + \lceil m/2 \rceil \\ &\le n + m = O(m+n). \end{aligned} \end{equation*}

(e) The recursion passes the even rows as a list of row indices, so \(A^{\prime}\) is never copied; copying it would cost \(\Theta(mn)\). For some constant \(c\),

\begin{equation*} T(m,n) \le T(\lfloor m/2 \rfloor, n) + c(m+n) \quad (m \ge 2), \qquad T(1,n) \le cn . \end{equation*}

Claim: \(T(m,n) \le c\,(2m + n + n\lg m)\). The base case holds since \(T(1,n) \le cn \le c(2 + n)\). For \(m \ge 2\), apply the inductive hypothesis to \(\lfloor m/2 \rfloor \ge 1\):

\begin{equation*} \begin{aligned} T(m,n) &\le c\bigl(2\lfloor m/2 \rfloor + n + n \lg(m/2)\bigr) + c(m+n) \\ &\le c\,(m + n\lg m) + c\,(m + n) \\ &= c\,(2m + n + n \lg m). \end{aligned} \end{equation*}

Hence \(T(m,n) = O(m + n + n \lg m)\), which is \(O(m + n\lg m)\) for \(m \ge 2\) (at \(m = 1\) the book’s bound must be read as \(O(m+n)\)).

The comparison in line 4 defines a strict relation \(x \succ y\) ("\(x\) is better than \(y\)") that is total, asymmetric and transitive, i.e. a total order (Section B.2), and the ranks are read off from it.

Because HIRE-ASSISTANT must work for every interview order, any two distinct candidates \(x, y\) can arrive as candidates 1 and 2, so line 4 must decide between them: exactly one of \(x \succ y\), \(y \succ x\) holds (totality and asymmetry). Moreover, the variable best must always be better than every candidate interviewed so far, so for every set \(S\) of interviewed candidates there is a unique \(m \in S\) with \(m \succ s\) for all \(s \in S \setminus \{m\}\).

Transitivity: let \(a \succ b\) and \(b \succ c\), and interview \(a, b, c\) in some order, with best element \(m\) of \(\{a,b,c\}\). Then \(m \ne b\) (since \(b \succ a\) fails) and \(m \ne c\) (since \(c \succ b\) fails), so \(m = a\) and \(a \succ c\).

Hence \(\succ\) is a strict total order on the \(n\) candidates, and

\begin{equation*} \mathit{rank}(i) = 1 + \bigl|\{\, j : i \succ j \,\}\bigr| \end{equation*}

assigns them the distinct ranks \(1, \dots, n\), with higher rank meaning better qualified.

Draw \(k = \lceil \lg(b-a+1) \rceil\) random bits to form a uniform integer \(r \in \{0, \dots, 2^k - 1\}\), return \(a + r\) if \(r \le b - a\), and otherwise start over (rejection sampling); the expected running time is \(\Theta(\lg(b-a+1))\) (and \(O(1)\) when \(a = b\)).

RANDOM(a, b)
1  n = b - a + 1
2  k = ceiling(lg n)
3  repeat
4      r = 0
5      for i = 1 to k
6          r = 2r + RANDOM(0, 1)
7  until r < n
8  return a + r

Correctness: the \(k\) independent fair bits make each \(r \in \{0, \dots, 2^k-1\}\) occur with probability \(2^{-k}\), and rounds are independent, so for each \(0 \le j < n\)

\begin{equation*} \Pr\{\text{output } a + j\} = \sum_{t \ge 1} \Bigl(1 - \frac{n}{2^k}\Bigr)^{t-1} \frac{1}{2^k} = \frac{1/2^k}{n/2^k} = \frac{1}{n}. \end{equation*}

Running time: each round succeeds with probability \(q = n/2^k\), and \(2^{k-1} < n \le 2^k\) gives \(q > 1/2\). The number of rounds is geometric (Section C.4), with expectation \(1/q = 2^k/n < 2\), and each round costs \(\Theta(k)\), so the expected number of calls to RANDOM\((0,1)\) is

\begin{equation*} k \le k \cdot \frac{2^k}{n} \le 2k, \qquad\text{hence}\qquad \mathrm{E}[T] = \Theta(1 + k) = \Theta\bigl(1 + \lg(b-a+1)\bigr). \end{equation*}

Call BIASED-RANDOM twice; if the two results differ return the first one, otherwise repeat (von Neumann’s trick). The expected running time is \(\Theta\bigl(1/(p(1-p))\bigr)\).

UNBIASED-RANDOM()
1  repeat
2      x = BIASED-RANDOM()
3      y = BIASED-RANDOM()
4  until x != y
5  return x

Correctness: the calls are independent, so in any round

\begin{equation*} \Pr\{x = 1, y = 0\} = p(1-p) = \Pr\{x = 0, y = 1\}, \end{equation*}

and since rounds are independent and the output is \(x\) from the first round with \(x \ne y\), each of the two values is returned with probability \(p(1-p)/\bigl(2p(1-p)\bigr) = 1/2\).

Running time: a round succeeds with probability \(q = 2p(1-p) > 0\), so the number of rounds is geometric with expectation \(1/q\) (Section C.4), each round costing \(\Theta(1)\) (two calls). Hence

\begin{equation*} \mathrm{E}[T] = \Theta\Bigl(\frac{1}{2p(1-p)}\Bigr) = \Theta\Bigl(\frac{1}{p(1-p)}\Bigr), \end{equation*}

with exactly \(1/(p(1-p))\) expected calls to BIASED-RANDOM.

Exactly one hire has probability \(1/n\); exactly \(n\) hires has probability \(1/n!\).

Candidate 1 is always hired, and afterwards a hire happens exactly when a candidate beats everyone seen so far. Hence exactly one hire occurs iff candidate 1 is the best of all \(n\); in a uniform random order the best candidate is equally likely to be in each of the \(n\) positions, so

\begin{equation*} \Pr\{\text{exactly one hire}\} = \frac{1}{n}. \end{equation*}

Exactly \(n\) hires occur iff every candidate beats all earlier ones, i.e. the candidates arrive in strictly increasing order of quality. That is one of the \(n!\) equally likely orderings, so

\begin{equation*} \Pr\{\text{exactly } n \text{ hires}\} = \frac{1}{n!}. \end{equation*}

The probability is \(H_{n-1}/n = \frac{1}{n}\sum_{k=1}^{n-1}\frac{1}{k}\).

Rank the candidates \(1,\dots,n\) with \(n\) the best. Candidate 1 is always hired; the next hire is the first later candidate who outranks candidate 1, and that person is hired last iff they are the overall best (rank \(n\)). So exactly two hires occur iff candidate 1 has rank \(i<n\) and, among the \(n-i\) candidates of rank above \(i\), the one of rank \(n\) appears first. Candidate 1 has rank \(i\) with probability \(1/n\), and given that, the remaining candidates are in uniformly random order, so each of the \(n-i\) better candidates is equally likely to come first among them. Hence

\begin{equation*} \begin{aligned} \Pr\{\text{exactly two hires}\} &= \sum_{i=1}^{n-1} \frac{1}{n}\cdot\frac{1}{n-i} = \frac{1}{n}\sum_{k=1}^{n-1}\frac{1}{k} = \frac{H_{n-1}}{n}. \end{aligned} \end{equation*}

The expected sum is \(7n/2\).

For die \(i\in\{1,\dots,n\}\) and face \(j\in\{1,\dots,6\}\) let \(X_{ij} = \mathrm{I}\{\text{die } i \text{ shows } j\}\), so by Lemma 5.1 \(\mathrm{E}[X_{ij}] = \Pr\{\text{die } i \text{ shows } j\} = 1/6\). The total is \(X = \sum_{i=1}^{n}\sum_{j=1}^{6} j\,X_{ij}\), and linearity of expectation gives

\begin{equation*} \mathrm{E}[X] = \sum_{i=1}^{n}\sum_{j=1}^{6} j\cdot\frac{1}{6} = n\cdot\frac{21}{6} = \frac{7n}{2}. \end{equation*}

The expected number is \(1\), for every \(n\ge 1\).

Let \(X_i = \mathrm{I}\{\text{customer } i \text{ gets their own hat}\}\). Under a uniformly random return order, customer \(i\)’s hat is equally likely to go to any of the \(n\) customers, so by Lemma 5.1 \(\mathrm{E}[X_i] = 1/n\). With \(X = \sum_{i=1}^{n} X_i\) the number of customers who get their own hat, linearity of expectation gives

\begin{equation*} \mathrm{E}[X] = \sum_{i=1}^{n}\mathrm{E}[X_i] = n\cdot\frac{1}{n} = 1. \end{equation*}

The expected number of inversions is \(n(n-1)/4\).

For \(1\le i<j\le n\) let \(X_{ij} = \mathrm{I}\{A[i]>A[j]\}\). In a uniform random permutation, swapping the entries in positions \(i\) and \(j\) is a bijection between permutations with \(A[i]>A[j]\) and those with \(A[i]<A[j]\), so \(\Pr\{A[i]>A[j]\} = 1/2\) and \(\mathrm{E}[X_{ij}] = 1/2\) by Lemma 5.1. The number of inversions is \(X = \sum_{1\le i<j\le n} X_{ij}\), so by linearity of expectation

\begin{equation*} \mathrm{E}[X] = \sum_{1\le i<j\le n}\frac{1}{2} = \binom{n}{2}\cdot\frac{1}{2} = \frac{n(n-1)}{4}. \end{equation*}

Perform the first swap before the loop and start the loop at \(i = 2\) (assume \(n \ge 1\)):

RANDOMIZE-IN-PLACE'(A)
1  n = A.length
2  swap A[1] with A[RANDOM(1, n)]
3  for i = 2 to n
4      swap A[i] with A[RANDOM(i, n)]

The loop invariant is the one of Lemma 5.5, now asserted only for \(i \ge 2\): just prior to the iteration with index \(i\) of the loop of lines 3-4, for each possible \((i-1)\)-permutation of the \(n\) elements, the subarray \(A[1 \,.\,.\, i-1]\) contains this \((i-1)\)-permutation with probability \((n-i+1)!/n!\).

Initialization: before the first iteration \(i = 2\), and \(A[1 \,.\,.\, 1]\) is a nonempty subarray. A 1-permutation is a single element \(x\); line 2 brings \(x\) into \(A[1]\) exactly when RANDOM picks \(x\)’s position, so

\begin{equation*} \Pr\{A[1] = x\} = \frac{1}{n} = \frac{(n-1)!}{n!} = \frac{(n-2+1)!}{n!} . \end{equation*}

Maintenance: word for word as in Lemma 5.5. For an \(i\)-permutation \(\langle x_1, \ldots, x_i \rangle\), let \(E_1\) be the event that \(A[1 \,.\,.\, i-1] = \langle x_1, \ldots, x_{i-1} \rangle\) and \(E_2\) the event that line 4 then puts \(x_i\) into \(A[i]\); by the invariant and the uniform choice among the \(n-i+1\) positions \(A[i \,.\,.\, n]\), equation (C.14) gives

\begin{equation*} \Pr\{E_2 \cap E_1\} = \Pr\{E_2 \mid E_1\} \Pr\{E_1\} = \frac{1}{n-i+1} \cdot \frac{(n-i+1)!}{n!} = \frac{(n-i)!}{n!} . \end{equation*}

Incrementing \(i\) restores the invariant.

Termination: the loop ends with \(i = n+1\), so each \(n\)-permutation occupies \(A[1 \,.\,.\, n]\) with probability \(0!/n! = 1/n!\); the output is a uniform random permutation. (For \(n = 1\) the loop body never runs and the invariant with \(i = 2 = n+1\) already gives probability \(1 = 1/1!\).)

No: it never outputs the identity, but it also cannot output many other non-identity permutations. Take \(n = 3\) and \(A = \langle 1, 2, 3 \rangle\). Iteration \(i = 2\) must swap \(A[2]\) with \(A[3]\), so only the choice at \(i = 1\) is random:

Only the two 3-cycles appear; the three non-identity permutations \(\langle 1,3,2 \rangle\), \(\langle 2,1,3 \rangle\), \(\langle 3,2,1 \rangle\) have probability 0.

No, not for \(n \ge 3\) (for \(n \le 2\) it happens to be uniform). The \(n\) calls to RANDOM produce one of \(n^n\) equally likely sequences, and the sequence determines the output, so each permutation \(\sigma\) appears with probability \(k_\sigma / n^n\) for some integer \(k_\sigma\). Uniformity would force \(k_\sigma = n^n/n!\) to be an integer, i.e. \(n! \mid n^n\). But for \(n \ge 3\),

\begin{equation*} (n-1) \mid n!, \qquad \gcd(n-1, n^n) = 1, \qquad n - 1 \ge 2, \end{equation*}

so \(n-1 \nmid n^n\) and hence \(n! \nmid n^n\). Concretely, for \(n = 3\) the \(27\) sequences give

\begin{equation*} \begin{aligned} &\Pr\{\langle 1,3,2\rangle\} = \Pr\{\langle 2,1,3\rangle\} = \Pr\{\langle 2,3,1\rangle\} = 5/27, \\ &\Pr\{\langle 1,2,3\rangle\} = \Pr\{\langle 3,1,2\rangle\} = \Pr\{\langle 3,2,1\rangle\} = 4/27, \end{aligned} \end{equation*}

not \(1/6\) each.

Each position is hit with probability \(1/n\), but only \(n\) of the \(n!\) permutations can occur. Lines 5-7 send \(A[i]\) to

\begin{equation*} \mathit{dest}(i) = ((i + \mathit{offset} - 1) \bmod n) + 1 , \end{equation*}

and for fixed \(i\) and \(j \in \{1, \ldots, n\}\) the congruence \(i + \mathit{offset} \equiv j \pmod n\) has exactly one solution \(\mathit{offset} \in \{1, \ldots, n\}\). Since \(\mathit{offset}\) is uniform on \(n\) values,

\begin{equation*} \Pr\{A[i] \text{ lands in } B[j]\} = \Pr\{\mathit{offset} \equiv j - i \pmod n\} = \frac{1}{n} . \end{equation*}

On the other hand, the whole output \(B\) is determined by the single value \(\mathit{offset}\), so at most \(n\) distinct permutations (the \(n\) cyclic shifts of \(A\)) are produced, each with probability \(1/n\). For \(n \ge 3\) we have \(n < n!\), so some permutation, e.g. the one swapping \(A[1]\) and \(A[2]\) and fixing the rest (not a cyclic shift, since a nontrivial shift fixes no position), has probability \(0 \ne 1/n!\). Hence the permutation is not uniformly random.

By Boole’s inequality (C.19) over the \(\binom{n}{2}\) pairs. For \(i < j\), conditioning on the value of \(P[i]\), the independent uniform \(P[j]\) equals it with probability \(1/n^3\), so

\begin{equation*} \begin{aligned} \Pr\{\text{some } P[i] = P[j],\ i < j\} &\le \sum_{1 \le i < j \le n} \Pr\{P[i] = P[j]\} = \binom{n}{2} \frac{1}{n^3} \\ &= \frac{n-1}{2n^2} \le \frac{1}{n} , \end{aligned} \end{equation*}

the last step being \(n - 1 \le 2n\). Taking complements, all entries of \(P\) are distinct with probability at least \(1 - 1/n\).

Redraw the whole priority array until its entries are distinct, then sort:

PERMUTE-BY-SORTING'(A)
1  n = A.length
2  let P[1..n] be a new array
3  repeat
4      for i = 1 to n
5          P[i] = RANDOM(1, n^3)
6  until all entries of P are distinct
7  sort A, using P as sort keys
8  return A

Line 6 is checked by sorting a copy of \(P\) and comparing neighbours.

Correctness: each round draws \(P\) uniformly from \(\{1, \ldots, n^3\}^n\), independently of earlier rounds, so the accepted \(P\) is uniformly distributed over the set \(D\) of tuples with distinct entries. The output is determined by the rank vector of \(P\): \(A[i]\) goes to position \(r_i\), where \(P[i]\) is the \(r_i\)th smallest priority. For any permutation \(\pi\) of \(\{1, \ldots, n\}\), the map

\begin{equation*} (p_1, \ldots, p_n) \mapsto (p_{\pi(1)}, \ldots, p_{\pi(n)}) \end{equation*}

is a bijection of \(D\) onto itself that carries rank vector \((r_1, \ldots, r_n)\) to \((r_{\pi(1)}, \ldots, r_{\pi(n)})\). Hence all \(n!\) rank vectors are realised by equally many tuples of \(D\), and each output permutation has probability \(1/n!\).

Running time: by Exercise 5.3-5 a round succeeds with probability \(p \ge 1 - 1/n \ge 1/2\) for \(n \ge 2\), so the number of rounds is geometric with expectation \(1/p \le 2\) (equation (C.32)). Each round costs \(O(n)\) to draw plus \(O(n \lg n)\) to test, so the expected running time is \(O(n \lg n)\), the same as the original procedure.

By induction on \(m\): for all \(0 \le m \le n\), RANDOM-SAMPLE\((m,n)\) makes exactly \(m\) calls to RANDOM and returns each \(m\)-subset \(T \subseteq \{1,\dots,n\}\) with probability \(1/\binom{n}{m}\).

Base case \(m = 0\): no calls, and the only \(0\)-subset \(\emptyset\) is returned with probability \(1 = 1/\binom{n}{0}\).

Inductive step, \(1 \le m \le n\): since \(0 \le m-1 \le n-1\), the hypothesis says line 3 makes \(m-1\) calls and yields \(S^{\prime}\) uniform over the \((m-1)\)-subsets of \(\{1,\dots,n-1\}\); line 4 adds one call, for \(m\) in total, and \(i\) is uniform on \(\{1,\dots,n\}\) independently of \(S^{\prime}\). The output has exactly \(m\) elements: in line 6, \(n \notin S^{\prime}\) because \(S^{\prime} \subseteq \{1,\dots,n-1\}\); in line 7, \(i \notin S^{\prime}\). Fix an \(m\)-subset \(T\) of \(\{1,\dots,n\}\).

(i) \(n \in T\). The output is \(T\) iff \(S^{\prime} = T \setminus \{n\}\) and \(i \in T\) (either \(i \in S^{\prime}\), so line 6 adds \(n\), or \(i = n\), so line 7 adds \(n\)); for any other \(S^{\prime}\) the output omits some element of \(T \setminus\{n\}\) or \(n\) itself. Hence

\begin{equation*} \Pr\{\text{output} = T\} = \frac{1}{\binom{n-1}{m-1}} \cdot \frac{m}{n}. \end{equation*}

(ii) \(n \notin T\). Line 6 would put \(n\) into the output, so the output is \(T\) iff line 7 runs with \(S^{\prime} = T \setminus \{i\}\) for some \(i \in T\). These are \(m\) disjoint events, one per \(j \in T\) (namely \(S^{\prime} = T\setminus\{j\}\), \(i = j\)), each of probability \(\frac{1}{\binom{n-1}{m-1}} \cdot \frac1n\), so again

\begin{equation*} \Pr\{\text{output} = T\} = \frac{m}{n\binom{n-1}{m-1}}. \end{equation*}

In both cases, because \(\binom{n}{m} = \frac{n}{m}\binom{n-1}{m-1}\),

\begin{equation*} \Pr\{\text{output} = T\} = \frac{m}{n\binom{n-1}{m-1}} = \frac{1}{\binom{n}{m}}, \end{equation*}

which completes the induction. \(\blacksquare\)

(i) \(253\) other people; (ii) \(613\) people.

(i) Each of the \(k\) other people misses your birthday independently with probability \(364/365\), so

\begin{equation*} \Pr\{\text{someone matches you}\} = 1 - \left(\tfrac{364}{365}\right)^k \ge \tfrac12 \iff k \ge \frac{\ln 2}{\ln(365/364)} \approx 252.65 , \end{equation*}

and the least such integer is \(k = 253\) (a room of \(254\) counting you).

(ii) The number of July 4 birthdays among \(k\) people has the binomial distribution \(b(j; k, 1/365)\), so

\begin{equation*} \Pr\{\text{at least two on July 4}\} = 1 - \left(\tfrac{364}{365}\right)^k - \frac{k}{365}\left(\tfrac{364}{365}\right)^{k-1} . \end{equation*}

This is increasing in \(k\) and equals \(0.49978\ldots\) at \(k = 612\) and \(0.50064\ldots\) at \(k = 613\), so \(613\) people are needed.

The expected number of tosses is

\begin{equation*} \mathrm{E}[X] = \sum_{k=0}^{b} \prod_{i=0}^{k-1}\left(1 - \frac{i}{b}\right) = \sum_{k=0}^{b} \frac{b!}{(b-k)!\, b^k} = \Theta(\sqrt{b}) . \end{equation*}

Let \(X\) be the number of tosses. We have \(X > k\) exactly when the first \(k\) balls land in distinct bins, which by the computation of \(\Pr\{B_k\}\) following recurrence (5.7) (with \(n = b\)) has probability \(\prod_{i=0}^{k-1}(1 - i/b)\); this vanishes for \(k > b\), so \(X \le b+1\). Equation (C.25) gives \(\mathrm{E}[X] = \sum_{k \ge 1}\Pr\{X \ge k\} = \sum_{k \ge 0}\Pr\{X > k\}\), which is the sum above.

Upper bound: by inequality (3.12), \(\Pr\{X > k\} \le e^{-k(k-1)/2b} \le e^{-(k-1)^2/2b}\) for \(k \ge 1\), so

\begin{equation*} \mathrm{E}[X] \le 1 + \sum_{j \ge 0} e^{-j^2/2b} \le 2 + \int_0^\infty e^{-x^2/2b}\,dx = 2 + \sqrt{\pi b/2} . \end{equation*}

Lower bound: by Boole’s inequality (C.19) over the \(\binom{k}{2}\) pairs, each colliding with probability \(1/b\), \(\Pr\{X > k\} \ge 1 - k(k-1)/2b \ge 1/2\) for \(k \le m = \lfloor\sqrt{b}\rfloor\), so

\begin{equation*} \mathrm{E}[X] \ge \sum_{k=0}^{m} \Pr\{X > k\} \ge \frac{m+1}{2} \ge \frac{\sqrt{b}}{2} . \end{equation*}

Pairwise independence suffices for the indicator-variable analysis (expected number of matching pairs), but the probability analysis needs mutual independence, and its conclusion can fail under pairwise independence alone.

Indicator analysis: \(\mathrm{E}[X_{ij}] = \Pr\{b_i = b_j\} = 1/n\) by equation (5.6), whose derivation uses only the independence of the single pair \(b_i, b_j\); linearity of expectation (C.21) then gives \(\mathrm{E}[X] = \binom{k}{2}/n\) with no further assumption.

Probability analysis: the factor \(\Pr\{A_k \mid B_{k-1}\} = (n-k+1)/n\) in recurrence (5.7) requires \(b_k\) to be uniform given all of \(b_1, \ldots, b_{k-1}\) jointly, i.e. mutual independence. Counterexample with \(n = p\) prime: choose \(a, c\) independently and uniformly from \(\mathbb{Z}_p\) and set \(b_i = a + ci \bmod p\) for \(i = 0, 1, \ldots, p-1\). For \(i \ne j\) the map \((a,c) \mapsto (b_i, b_j)\) is a bijection of \(\mathbb{Z}_p^2\) (as \(i - j\) is invertible mod \(p\)), so each \(b_i\) is uniform and the \(b_i\) are pairwise independent. But the birthdays are all distinct when \(c \ne 0\) and all equal when \(c = 0\), so for every \(2 \le k \le p\),

\begin{equation*} \Pr\{\text{some two of } b_0, \ldots, b_{k-1} \text{ match}\} = \Pr\{c = 0\} = \frac1p , \end{equation*}

which stays at \(1/p\) (far below \(1/2\)) even for \(k\) much larger than \(\sqrt{2p\ln 2}\).

About \(94\) guests by the indicator-variable analysis (expected number of triple matches at least \(1\)); the exact probability of a triple match first exceeds \(1/2\) at \(88\) guests.

Indicator analysis (\(n = 365\) days, independent uniform birthdays): for each triple \(i < j < l\) of the \(k\) guests let \(X_{ijl} = \mathrm{I}\{b_i = b_j = b_l\}\), so by Lemma 5.1 \(\mathrm{E}[X_{ijl}] = \sum_{r=1}^{n} n^{-3} = 1/n^2\), and with \(X = \sum X_{ijl}\), by linearity (C.21),

\begin{equation*} \mathrm{E}[X] = \binom{k}{3}\frac{1}{n^2} \ge 1 \iff k(k-1)(k-2) \ge 6n^2 = 799350 . \end{equation*}

Since \(93\cdot 92\cdot 91 = 778596\) and \(94\cdot 93\cdot 92 = 804264\), the least such \(k\) is \(94\).

Exact count: an assignment with no day used three times has some \(j\) days used twice and \(k - 2j\) days used once; choosing those days and distributing the guests gives

\begin{equation*} \Pr\{\text{no triple}\} = \frac{1}{n^k}\sum_{j} \frac{n!}{j!\,(k-2j)!\,(n-k+j)!}\cdot\frac{k!}{2^j} , \end{equation*}

summed over \(\max(0, k-n) \le j \le \lfloor k/2 \rfloor\). Evaluating, \(\Pr\{\text{triple}\} \approx 0.4995\) at \(k = 87\) and \(\approx 0.5111\) at \(k = 88\), so \(88\) guests make a shared triple birthday more likely than not.

The probability is

\begin{equation*} \frac{n!/(n-k)!}{n^k} = \prod_{i=0}^{k-1}\left(1 - \frac{i}{n}\right) \end{equation*}

for \(k \le n\), and \(0\) for \(k > n\): by Section C.1 there are \(n^k\) equally likely \(k\)-strings, of which \(n(n-1)\cdots(n-k+1) = n!/(n-k)!\) are \(k\)-permutations.

A uniformly random \(k\)-string over the \(n\) days is exactly the birthday vector \(\langle b_1, \ldots, b_k\rangle\) of \(k\) people with independent uniform birthdays, and it is a \(k\)-permutation exactly when the birthdays are distinct, i.e. on the event \(B_k\). So this probability is the value of \(\Pr\{B_k\}\) obtained from recurrence (5.7), and the birthday paradox says it drops to at most \(1/2\) once \(k \ge (1 + \sqrt{1 + 8n\ln 2})/2\) (that is, \(k \ge 23\) for \(n = 365\)).

Both expectations are close to \(n/e\):

\begin{equation*} \mathrm{E}[\text{empty bins}] = n\left(1 - \frac1n\right)^{n}, \qquad \mathrm{E}[\text{singleton bins}] = n\left(1 - \frac1n\right)^{n-1} . \end{equation*}

For bin \(i\) let \(E_i = \mathrm{I}\{\text{bin } i \text{ is empty}\}\) and \(S_i = \mathrm{I}\{\text{bin } i \text{ has exactly one ball}\}\). The number of balls in bin \(i\) has the binomial distribution \(b(j; n, 1/n)\), so by Lemma 5.1 and equation (C.35)

\begin{equation*} \mathrm{E}[E_i] = \left(1 - \frac1n\right)^n , \qquad \mathrm{E}[S_i] = n\cdot\frac1n\left(1 - \frac1n\right)^{n-1} = \left(1 - \frac1n\right)^{n-1} , \end{equation*}

and summing over the \(n\) bins by linearity (C.21) gives the two formulas (both \(\sim n/e\), as \((1 - 1/n)^n \to 1/e\)).

Cut the flips into disjoint blocks of length \(2s\), where \(s = \lfloor \lg n - 2\lg\lg n\rfloor + 1\) is the shortest streak length that is longer than \(\lg n - 2\lg\lg n\), as in Section 5.4.3. We prove the bound for \(n \ge 107\). For \(3 \le n \le 106\) an exact run-length computation also gives a probability below \(1/n\), while \(n = 2\) is a genuine exception (\(3/4 > 1/2\)).

Fix one block of \(2s\) flips. For \(1 \le i \le s+1\), let \(E_i\) be the event that a run of at least \(s\) heads starts at position \(i\) of the block. For \(i = 1\) this means flips \(1,\dots,s\) are heads. For \(i \ge 2\) it means flip \(i-1\) is a tail and flips \(i,\dots,i+s-1\) are heads. The block contains a run of length \(\ge s\) exactly when some \(E_i\) occurs.

The \(E_i\) are pairwise disjoint. If \(i<j\), the run starting at \(i\) covers positions \(i\) to \(i+s-1\), so the tail at position \(j-1\) forces \(j-1 \ge i+s\). The run starting at \(j\) then ends at \(j+s-1 \ge i+2s \ge 2s+1\), which is outside the block. Hence

\begin{equation*} p := \Pr\{\text{block has a run} \ge s\} = \frac{1}{2^{s}} + \frac{s}{2^{s+1}} = \frac{s+2}{2^{s+1}} . \end{equation*}

There are \(m = \lfloor n/(2s)\rfloor \ge n/(2s) - 1\) disjoint blocks, and they are independent. If no streak of length \(\ge s\) occurs anywhere, then no block contains one. Using \(1-p \le e^{-p}\) (inequality (3.12)),

\begin{equation*} \Pr\{\text{no streak} \ge s\} \le (1-p)^m \le e^{-mp}. \end{equation*}

Since \(s \le \lg n - 2\lg\lg n + 1\), we have \(2^{-s} \ge (\lg n)^2/(2n)\). Also \((s+2)/2^{s+1} \le 3/4\) for \(s \ge 1\). Therefore

\begin{equation*} \begin{aligned} mp &\ge \Bigl(\frac{n}{2s} - 1\Bigr)\frac{s+2}{2^{s+1}} \ge \frac{n}{2s}\cdot\frac{s}{2^{s+1}} - \frac{s+2}{2^{s+1}} \ge \frac{n}{4\cdot 2^{s}} - 1 \\ &\ge \frac{(\lg n)^2}{8} - 1 . \end{aligned} \end{equation*}

Write \(x = \lg n\), so that \(\ln n = x\ln 2\). The quadratic \(x^2 - (8\ln 2)x - 8\) is positive for \(x > 6.74\), that is, for \(n \ge 107\). For these \(n\) we get \((\lg n)^2/8 - 1 > \ln n\), and hence

\begin{equation*} \Pr\{\text{no streak} \ge s\} \le e^{-mp} < e^{-\ln n} = \frac{1}{n}. \end{equation*}

(a) Each INCREMENT raises the represented value by exactly \(1\) in expectation, so by induction the expectation after \(n\) operations is \(n\). Let \(C_j\) be the counter after \(j\) operations and \(V_j = n_{C_j}\) the represented value. Conditioned on \(C_j = i\) (no overflow), the value rises by \(n_{i+1} - n_i\) with probability \(1/(n_{i+1} - n_i)\) and otherwise stays put, so

\begin{equation*} \mathrm{E}[V_{j+1} - V_j \mid C_j = i] = (n_{i+1} - n_i) \cdot \frac{1}{n_{i+1} - n_i} = 1 . \end{equation*}

Averaging over \(i\) gives \(\mathrm{E}[V_{j+1}] = \mathrm{E}[V_j] + 1\), and since \(V_0 = n_0 = 0\), induction on \(j\) yields \(\mathrm{E}[V_n] = n\).

(b) The variance is exactly \(99n\). When \(n_i = 100i\), every INCREMENT raises the counter with the same probability \(1/100\), whatever the current state, and independently of the earlier operations. So the counter after \(n\) operations is \(X \sim \mathrm{Binomial}(n, 1/100)\) (sum of \(n\) independent indicator variables, as in Section 5.2), and \(V_n = 100X\). Hence

\begin{equation*} \mathrm{Var}[V_n] = 100^2 \,\mathrm{Var}[X] = 10^4 \cdot n \cdot \frac{1}{100} \cdot \frac{99}{100} = 99n , \end{equation*}

using \(\mathrm{Var}[X] = npq\) for the binomial distribution (equation (C.40)).

(a) Mark checked indices and count the distinct ones; stop when \(x\) is found or the count reaches \(n\).

RANDOM-SEARCH(A, n, x)
1  let checked[1..n] be a new array
2  for i = 1 to n
3      checked[i] = FALSE
4  count = 0
5  while count < n
6      i = RANDOM(1, n)
7      if A[i] == x
8          return i
9      if checked[i] == FALSE
10         checked[i] = TRUE
11         count = count + 1
12 return NIL

At line 5, \(\mathit{count}\) is the number of marked indices, all examined and not holding \(x\) (lines 2-4 start it at \(0\); lines 9-11 add \(1\) exactly when a new index is marked). So NIL is returned only after all \(n\) indices were examined, and the loop ends with probability 1 since each unmarked index is picked with probability \(1/n\) per iteration.

(b) The expected number of picks is \(n\). Each pick independently hits the unique index of \(x\) with probability \(1/n\). The procedure cannot stop through \(\mathit{count}=n\) before that index has been picked, so it stops at the first success. The number of picks is therefore geometric with \(p=1/n\), and by equation (C.32) its mean is \(1/p=n\).

(c) The expected number of picks is \(n/k\). The argument is the same as in (b), now with success probability \(p=k/n\) on each independent pick, and the mean is \(1/p=n/k\).

(d) The expected number of picks is \(nH_n=n(\ln n+O(1))\). This is the coupon-collector setting of Section 5.4.2, with indices as bins and picks as balls. The procedure stops exactly when every index has been picked at least once. When \(i-1\) distinct indices have been seen, the wait for a new one is geometric with \(p=(n-i+1)/n\). Adding these waits,

\begin{equation*} \sum_{i=1}^{n}\frac{n}{n-i+1}=n\sum_{j=1}^{n}\frac1j=nH_n=n(\ln n+O(1)) \end{equation*}

by equation (A.7).

(e) The average case is \((n+1)/2\) comparisons and the worst case is \(n\). Under a uniformly random permutation the position \(P\) of \(x\) is uniform on \(\{1,\dots,n\}\), and the search makes \(P\) comparisons. Hence \(\mathrm E[P]=(n+1)/2=\Theta(n)\). The worst case \(P=n\) gives \(\Theta(n)\).

(f) The average case is \((n+1)/(k+1)\) comparisons and the worst case is \(n-k+1\). The \(k\) copies of \(x\) sit on a uniformly random \(k\)-subset of the positions. Let \(P\) be the smallest of these positions. Then \(P>j\) exactly when all \(k\) copies lie in the last \(n-j\) positions, so

\begin{equation*} \begin{aligned} \mathrm E[P]&=\sum_{j=0}^{n-k}\Pr\{P>j\}=\sum_{j=0}^{n-k}\frac{\binom{n-j}{k}}{\binom nk}\\ &=\frac{\binom{n+1}{k+1}}{\binom nk}=\frac{n+1}{k+1}, \end{aligned} \end{equation*}

by the hockey-stick identity \(\sum_{m=k}^{n}\binom mk=\binom{n+1}{k+1}\). In the worst case all \(k\) copies are at the end, so \(P=n-k+1\).

(g) Both the average case and the worst case are \(n\) comparisons, that is, \(\Theta(n)\). With no copy of \(x\), every input is scanned to the end.

(h) Permuting by RANDOMIZE-IN-PLACE (Section 5.3) costs \(\Theta(n)\) on every input. After that the search runs on a uniformly random permutation (Lemma 5.5), so its expected cost is exactly the average case from (e)-(g).

• Case \(k=0\). The expected and worst-case costs are both \(n+n=\Theta(n)\).
• Case \(k=1\). The expected cost is \(n+(n+1)/2=\Theta(n)\), and the worst case is \(n+n=\Theta(n)\).
• Case \(k\ge1\). The expected cost is \(n+(n+1)/(k+1)=\Theta(n)\), and the worst case is \(n+(n-k+1)=\Theta(n)\).

The permutation step dominates in every case.

(i) Use DETERMINISTIC-SEARCH. Its worst case is at most \(n\) for every \(k\), it needs no extra space, and on random inputs its average \((n+1)/(k+1)\le n/k\) (as \(k\le n\)) beats both alternatives.

• SCRAMBLE-SEARCH achieves the same expected search cost on every input, but it pays \(\Theta(n)\) up front for the shuffle.
• RANDOM-SEARCH needs \(\Theta(n)\) extra space, and its worst case is unbounded. For \(k=0\) its expectation is \(\Theta(n\lg n)\).

A heap of height \(h\) has at least \(2^h\) and at most \(2^{h+1}-1\) elements.

A heap is a complete binary tree filled level by level from the left, so a heap of height \(h\) has every level \(0,1,\dots,h-1\) full, contributing \(1+2+\dots+2^{h-1}=2^h-1\) nodes, and between \(1\) and \(2^h\) nodes on level \(h\). Hence

\begin{equation*} 2^h-1+1 \;\le\; n \;\le\; 2^h-1+2^h , \qquad\text{i.e.}\qquad 2^h \le n \le 2^{h+1}-1 , \end{equation*}

with the minimum attained by a single node on level \(h\) and the maximum by a full level \(h\).

Let \(h\) be the height. By Exercise 6.1-1,

\begin{equation*} 2^h \le n \le 2^{h+1}-1 < 2^{h+1} \quad\Longrightarrow\quad h \le \lg n < h+1 , \end{equation*}

so \(h\) is the integer with \(h\le \lg n<h+1\), namely \(h=\lfloor \lg n\rfloor\).

Let the subtree be rooted at node \(i\) and let \(j\) be any node in it. There is a downward path \(i=i_0,i_1,\dots,i_k=j\) with \(i_{t}=\mathrm{PARENT}(i_{t+1})\) for \(0\le t<k\). Applying the max-heap property \(A[\mathrm{PARENT}(x)]\ge A[x]\) along the path,

\begin{equation*} A[i]=A[i_0]\ge A[i_1]\ge \cdots \ge A[i_k]=A[j] , \end{equation*}

(for \(k=0\) the claim is trivial), so \(A[i]\) is the largest value in the subtree.

The smallest element must be at a leaf, i.e. at one of the indices \(\lfloor n/2\rfloor+1,\dots,n\) (Exercise 6.1-7), and every leaf is possible.

(i) It must be a leaf: if node \(i\) has a child \(c\), then \(A[i]\ge A[c]\) by the max-heap property, and \(A[i]\ne A[c]\) by distinctness, so \(A[c]<A[i]\) and \(A[i]\) is not the minimum.

(ii) Every leaf \(\ell\) is possible: take the values \(1,\dots,n\) with \(A[i]=n+1-i\), which is a max-heap because \(\mathrm{PARENT}(i)<i\); it puts the minimum at the leaf \(n\). For \(\ell<n\), swap the values at \(\ell\) and \(n\), so \(A[\ell]=1\) and \(A[n]=n+1-\ell\). Leaves \(\ell\) and \(n\) have no children, the parent of \(\ell\) still holds a value larger than \(1\), and \(q=\mathrm{PARENT}(n)\) is not a leaf, so \(q\le\lfloor n/2\rfloor<\ell\) and

\begin{equation*} A[q]=n+1-q>n+1-\ell=A[n] . \end{equation*}

All other parent-child pairs are unchanged, so this is a max-heap with its minimum at \(\ell\).

Yes. For every node \(i>1\) we have \(\mathrm{PARENT}(i)=\lfloor i/2\rfloor<i\), so sortedness gives \(A[\mathrm{PARENT}(i)]\le A[i]\), which is exactly the min-heap property.

No. Node \(9\) has parent \(\mathrm{PARENT}(9)=\lfloor 9/2\rfloor=4\), but

\begin{equation*} A[4]=6<7=A[9] , \end{equation*}

which violates the max-heap property.

Node \(i\) is a leaf exactly when \(\mathrm{LEFT}(i) = 2i > n\), and this holds exactly when \(i \ge \lfloor n/2 \rfloor + 1\).

Node \(i\) (\(1 \le i \le n\)) has a child iff its left child exists, since \(\mathrm{RIGHT}(i) = 2i+1 > 2i\), so \(2i > n\) already forces \(2i+1 > n\). Because \(i\) is an integer,

\begin{equation*} \begin{aligned} i \text{ is a leaf} &\iff 2i > n \iff i > n/2 \\ &\iff i > \lfloor n/2 \rfloor \iff i \ge \lfloor n/2 \rfloor + 1, \end{aligned} \end{equation*}

where \(i > n/2 \iff i > \lfloor n/2 \rfloor\) holds because no integer lies strictly between \(\lfloor n/2 \rfloor\) and \(n/2\). So the leaves are precisely the indices \(\lfloor n/2 \rfloor + 1, \ldots, n\).

The value \(3\) sinks from node 3 to node 6 to node 13, giving \(\langle 27, 17, 10, 16, 13, 9, 1, 5, 7, 12, 4, 8, 3, 0\rangle\).

The successive arrays are

\begin{equation*} \begin{aligned} &\text{(a)}\ \langle 27, 17, \mathbf{3}, 16, 13, 10, 1, 5, 7, 12, 4, 8, 9, 0\rangle,\\ &\text{(b)}\ \langle 27, 17, 10, 16, 13, \mathbf{3}, 1, 5, 7, 12, 4, 8, 9, 0\rangle,\\ &\text{( c)}\ \langle 27, 17, 10, 16, 13, 9, 1, 5, 7, 12, 4, 8, \mathbf{3}, 0\rangle, \end{aligned} \end{equation*}

with the node \(i\) of the current call in bold; node 13 is a leaf, so the final call changes nothing.

Reverse every comparison between keys; the running time is identical: \(O(\lg n)\), or \(O(h)\) at a node of height \(h\), with the same \(\Omega(\lg n)\) worst case.

MIN-HEAPIFY(A, i)
1  l = LEFT(i)
2  r = RIGHT(i)
3  if l <= A.heap-size and A[l] < A[i]
4      smallest = l
5  else smallest = i
6  if r <= A.heap-size and A[r] < A[smallest]
7      smallest = r
8  if smallest != i
9      exchange A[i] with A[smallest]
10     MIN-HEAPIFY(A, smallest)

Correctness is the argument of Section 6.2 with \(\le\) and \(\ge\) interchanged. Each call does \(\Theta(1)\) work and recurses on a child subtree of size at most \(2n/3\), so \(T(n) \le T(2n/3) + \Theta(1)\) and \(T(n) = O(\lg n)\) by case 2 of the master theorem (Theorem 4.1); the construction of Exercise 6.2-6 with the order reversed (root holding the largest key, all others equal) forces \(\lfloor \lg n\rfloor + 1\) calls.

Nothing changes: the call returns after \(\Theta(1)\) work without modifying \(A\). Indeed, the test in line 3 fails (since \(A[\mathrm{LEFT}(i)] < A[i]\) whenever the left child exists), so line 5 sets \(\mathit{largest} = i\); line 6 then fails because \(A[\mathrm{RIGHT}(i)] < A[i] = A[\mathit{largest}]\) whenever the right child exists; hence the test \(\mathit{largest} \ne i\) in line 8 is false, and neither the exchange nor the recursive call is executed.

Nothing changes: node \(i\) is a leaf (Exercise 6.1-7), so the call returns after \(\Theta(1)\) work without modifying \(A\). Explicitly, with \(n = A.\mathit{heap\text{-}size}\),

\begin{equation*} l = \mathrm{LEFT}(i) = 2i > n, \qquad r = \mathrm{RIGHT}(i) = 2i + 1 > n, \end{equation*}

so both tests in lines 3 and 6 fail, \(\mathit{largest} = i\), and line 8 does nothing.

The recursive call is a tail call, so replace it by the assignment \(i = \mathit{largest}\) and loop until \(\mathit{largest} = i\).

ITERATIVE-MAX-HEAPIFY(A, i)
1  while TRUE
2      l = LEFT(i)
3      r = RIGHT(i)
4      if l <= A.heap-size and A[l] > A[i]
5          largest = l
6      else largest = i
7      if r <= A.heap-size and A[r] > A[largest]
8          largest = r
9      if largest == i
10         return
11     exchange A[i] with A[largest]
12     i = largest

Let \(i_0\) be the original argument. Loop invariant: at the start of each iteration, every node of the subtree rooted at \(i_0\) satisfies the max-heap property except possibly node \(i\), the subtrees at LEFT\((i)\) and RIGHT\((i)\) are max-heaps, and if \(i \ne i_0\) then \(A[\mathrm{PARENT}(i)]\) is at least every key in the subtree of \(i\).

Initialization: this is exactly the precondition of MAX-HEAPIFY (the last clause is vacuous).

Maintenance: lines 2-8 find the index \(\mathit{largest}\) of the maximum of \(A[i]\) and its existing children. If \(\mathit{largest} \ne i\), after line 11 node \(i\) holds the maximum of its subtree, so \(i\) satisfies the max-heap property and dominates the subtree of \(\mathit{largest}\); that subtree’s children subtrees are untouched max-heaps; so the invariant holds for \(i = \mathit{largest}\).

Termination: each iteration either returns or moves \(i\) one level down, so the loop stops after at most \(h+1\) iterations (\(h\) the height of \(i_0\)). It stops when \(\mathit{largest} = i\), i.e. \(A[i]\) dominates its children; with the invariant, every node of the subtree at \(i_0\) now satisfies the max-heap property.

Each iteration costs \(\Theta(1)\), so the running time is \(O(h) = O(\lg n)\), with \(O(1)\) extra space.

Take \(A[1] = 0\) and \(A[j] = 1\) for \(2 \le j \le n\); then MAX-HEAPIFY\((A,1)\) makes \(\lfloor \lg n\rfloor + 1\) calls. The precondition holds, since the subtrees at nodes 2 and 3 contain only 1’s and so are max-heaps. Claim: for \(k = 0, 1, \dots, \lfloor \lg n \rfloor\), the \((k+1)\)st call is MAX-HEAPIFY\((A, 2^k)\) with the 0 at position \(2^k\). If \(2^k \le n/2\) then LEFT\((2^k) = 2^{k+1} \le n\) and \(A[2^{k+1}] = 1 > 0 = A[2^k]\), so line 4 sets \(\mathit{largest} = 2^{k+1}\); line 6 cannot change it because the right child, if present, holds \(1 \not> 1\); the exchange moves the 0 to \(2^{k+1}\) and line 10 calls MAX-HEAPIFY\((A, 2^{k+1})\). The calls thus follow the leftmost root-to-leaf path

\begin{equation*} 1, 2, 4, \dots, 2^{\lfloor \lg n \rfloor}, \end{equation*}

ending at the leaf \(2^{\lfloor \lg n\rfloor}\) (its left child \(2^{\lfloor \lg n\rfloor+1} > n\)). Every call executes at least lines 1-3, costing at least some constant \(c > 0\), so

\begin{equation*} T(n) \ge c\,(\lfloor \lg n \rfloor + 1) \ge c \lg n = \Omega(\lg n). \end{equation*}

The result is the max-heap \(A = \langle 84, 22, 19, 10, 3, 17, 6, 5, 9 \rangle\). Here \(n = 9\), so the loop runs \(i = \lfloor 9/2 \rfloor = 4, 3, 2, 1\); each row shows the array just before the call MAX-HEAPIFY\((A, i)\), with the last row the final heap.

The individual calls:

• \(i = 4\): \(A[4] = 10 < A[8] = 22\), swap; node 8 is a leaf.
• \(i = 3\): \(A[3] = 17 < A[6] = 19\), swap; node 6 is a leaf.
• \(i = 2\): \(A[2] = 3 < A[5] = 84\) (the larger child, since \(A[4] = 22\)), swap; node 5 is a leaf.
• \(i = 1\): \(5\) is swapped with \(A[2] = 84\), then with \(A[4] = 22\), then with \(A[8] = 10\) (the larger of \(10, 9\)), ending at leaf 8.

Because MAX-HEAPIFY\((A, i)\) is only correct when the subtrees rooted at LEFT\((i)\) and RIGHT\((i)\) are already max-heaps (Section 6.2), and only the decreasing order guarantees this: the children \(2i, 2i+1\) have larger indices, so they have already been processed (this is exactly the maintenance step of the loop invariant in Section 6.3). In increasing order a node is heapified before its subtrees are, and a large key deep in the tree can never climb above the nodes already handled. For example, with \(A = \langle 1, 2, 3, 4 \rangle\) the increasing loop does

\begin{equation*} \langle 1, 2, 3, 4 \rangle \xrightarrow{\ i=1\ } \langle 3, 2, 1, 4 \rangle \xrightarrow{\ i=2\ } \langle 3, 4, 1, 2 \rangle, \end{equation*}

which is not a max-heap since \(A[2] = 4 > A[1] = 3\).

Induct on \(h\), stripping off the leaves. Let \(N_h(n)\) be the number of nodes of height \(h\) in the \(n\)-element heap (its shape depends only on \(n\)).

Base \(h = 0\): the nodes of height 0 are the leaves, which by Exercise 6.1-7 are exactly \(\lfloor n/2 \rfloor + 1, \dots, n\), so \(N_0(n) = n - \lfloor n/2 \rfloor = \lceil n/2 \rceil\).

Step \(h \ge 1\): deleting the leaves leaves the nodes \(1, \dots, \lfloor n/2 \rfloor\), i.e. the heap on \(\lfloor n/2 \rfloor\) elements (the parent/child relations among indices are unchanged). Every non-leaf \(x\) has height in the pruned tree exactly \(h(x) - 1\): a longest downward path from \(x\) ends at a leaf, and dropping that leaf gives a path of length \(h(x) - 1\) in the pruned tree; conversely any downward path in the pruned tree ends at an original non-leaf, so it extends by one child to an original path, whence its length is at most \(h(x) - 1\). Hence the nodes of height \(h\) in the original heap are precisely the nodes of height \(h-1\) in the pruned one, and by the inductive hypothesis

\begin{equation*} \begin{aligned} N_h(n) = N_{h-1}(\lfloor n/2 \rfloor) &\le \left\lceil \frac{\lfloor n/2 \rfloor}{2^{h}} \right\rceil \le \left\lceil \frac{n/2}{2^{h}} \right\rceil = \left\lceil \frac{n}{2^{h+1}} \right\rceil, \end{aligned} \end{equation*}

using that \(\lceil \cdot \rceil\) is nondecreasing. \(\blacksquare\)

HEAPSORT outputs \(\langle 2, 4, 5, 7, 8, 13, 17, 20, 25 \rangle\). Line 1 (BUILD-MAX-HEAP) turns \(A\) into the max-heap \(\langle 25, 13, 20, 8, 7, 17, 2, 5, 4 \rangle\). The table below lists the array after each call of MAX-HEAPIFY in line 5, together with the value of \(i\) in that iteration. The heap is \(A[1 \,..\, i-1]\), and the elements already placed in their final positions are \(A[i \,..\, 9]\).

As in Figure 6.4, each row’s tree is the level-order reading of its heap column; e.g. in row (b) the root 20 has children 13 and 17, 13 has children 8 and 7, 17 has children 4 and 2, and 8 has left child 5.

The invariant holds before every iteration. When the loop ends we have \(i = 1\), and at that point the invariant says \(A\) is sorted.

Initialization. Before the first iteration, \(i = n\). Line 1 has made \(A[1 \,..\, n]\) a max-heap with \(A.\mathit{heap\text{-}size} = n\) (by the correctness of BUILD-MAX-HEAP, Section 6.3). This heap trivially holds the \(n\) smallest elements. The subarray \(A[n+1 \,..\, n]\) is empty, so it trivially holds the \(0\) largest elements in sorted order.

Maintenance. At the start of an iteration with index \(i\), the root \(A[1]\) is the largest element of the max-heap \(A[1 \,..\, i]\) (Section 6.1). So \(A[1]\) is the \((n-i+1)\)st largest element of \(A[1 \,..\, n]\), and it is no larger than any element of \(A[i+1 \,..\, n]\). Line 3 swaps it into position \(i\). After the swap, \(A[i \,..\, n]\) holds the \(n-i+1\) largest elements in sorted order, and \(A[1 \,..\, i-1]\) holds the \(i-1\) smallest elements. Line 4 sets \(A.\mathit{heap\text{-}size} = i - 1\). In the heap \(A[1 \,..\, i-1]\), the subtrees rooted at \(\mathrm{LEFT}(1)\) and \(\mathrm{RIGHT}(1)\) are still max-heaps, because the swap changed only \(A[1]\) inside this heap. That is exactly the precondition of MAX-HEAPIFY, so line 5 makes \(A[1 \,..\, i-1]\) a max-heap (Section 6.2). MAX-HEAPIFY only permutes \(A[1 \,..\, i-1]\), so the set of elements it holds does not change. Decrementing \(i\) therefore restores the invariant.

Termination. The loop exits with \(i = 1\). By the invariant, \(A[2 \,..\, n]\) holds the \(n - 1\) largest elements in sorted order, and \(A[1]\) holds the smallest. Hence \(A[1 \,..\, n]\) is sorted.

Both cases take \(\Theta(n \lg n)\) time, assuming the keys are distinct (that is, the order is strict). The upper bound holds for every input: line 1 costs \(O(n)\) and each of the \(n-1\) calls of MAX-HEAPIFY costs \(O(\lg n)\), so HEAPSORT runs in \(O(n \lg n)\) time (Section 6.4). The lower bound is Exercise 6.4-5: on any input with distinct elements, HEAPSORT takes \(\Omega(n \lg n)\) time, and that includes both sorted orders. (With repeated keys the answer can drop: if all \(n\) keys are equal, every MAX-HEAPIFY call returns at once and the time is \(\Theta(n)\).)

HEAPSORT is a comparison sort. It learns about keys only by comparing them in MAX-HEAPIFY (lines 3 and 6), and it otherwise only moves them. By Theorem 8.1, every comparison sort makes \(\Omega(n \lg n)\) comparisons in the worst case. Each comparison takes \(\Theta(1)\) time, so the worst-case running time of HEAPSORT is \(\Omega(n \lg n)\).

Method (2): By Exercise 6.4-5, even the best case on \(n\) distinct keys takes \(\Omega(n \lg n)\) time. The worst case is at least as slow.

Look only at the first \(m = \lfloor n/2 \rfloor\) iterations of the loop, during which the \(m\) largest elements are removed. In these iterations the MAX-HEAPIFY calls already make \(\Omega(n \lg n)\) exchanges.

Let \(S\) be the set of the \(m\) largest elements. Call position \(p\) “red” when \(A[p] \in S\), and let \(R\) be the set of red positions inside the current heap. Write \(d(p) = \lfloor \lg p \rfloor\) for the depth of position \(p\).

(1) Keys are distinct, so in a max-heap every parent is larger than its child. A parent of an element of \(S\) is therefore in \(S\), which means \(R\) is closed under taking parents. This holds right after line 1, and again after every line 5. (2) MAX-HEAPIFY swaps a parent with a child only when the child is larger. So if the parent is in \(S\), the child is in \(S\) too. An element of \(S\) never trades places with an element outside \(S\) while moving down.

Consider one of the first \(m\) iterations, with index \(i\). Then \(R \ne \emptyset\), so the root holds the maximum, which is in \(S\). Line 3 removes the maximum and moves \(x = A[i]\) to the root. There are two cases.

(i) \(x \notin S\), so \(i \notin R\). MAX-HEAPIFY moves \(x\) down a path. While \(x\) is above a red child, the larger child is red, so \(x\) moves through red positions \(1 = c_0, c_1, \ldots, c_k\). Here \(c_k\) is the first position on the path with no red child. Each red element on the path moves up one step. Afterwards the red set is \(R^{\prime} = R \setminus \{c_k\}\), and the call made at least \(k = d(c_k)\) exchanges.

(ii) \(x \in S\), so \(i \in R\). By fact 2, \(x\) moves down only through red positions. Afterwards \(R^{\prime} = R \setminus \{i\}\), at a cost of at least \(0\).

Every iteration removes exactly one position from \(R\). After \(m\) iterations \(R = \emptyset\), so every position of the initial red set \(R_0\) is removed exactly once. Case (ii) removes the position \(i\), which is the heap size at that time. In the first \(m\) iterations that position lies in \(J = \{n-m+1, \ldots, n\}\). So every \(p \in R_1 = R_0 \setminus J\) is removed by case (i), which costs at least \(d(p)\) exchanges. The number of exchanges is therefore at least \(\sum_{p \in R_1} d(p)\).

Next we show \(|R_1| \ge m/3\). Every \(p \in J\) satisfies \(p > n/2\), so \(\mathrm{PARENT}(p) \le n/2 \le n - m\). By fact 1, each \(p \in R_0 \cap J\) has its parent in \(R_1\). Each node has at most two children, so \(|R_0 \cap J| \le 2|R_1|\), and \(m = |R_1| + |R_0 \cap J| \le 3|R_1|\).

Finally, at most \(2^d\) positions have depth \(d\). So if \(k = |R_1|\), the \(j\)th smallest depth in \(R_1\) is at least \(\lfloor \lg j \rfloor\). Hence

\begin{equation*} \begin{aligned} \sum_{p \in R_1} d(p) &\ge \sum_{j=1}^{k} \lfloor \lg j \rfloor \ge \sum_{j=\lceil k/2 \rceil}^{k} (\lg j - 1) \ge \frac{k}{2}\,(\lg k - 2) . \end{aligned} \end{equation*}

Since \(k \ge m/3 \ge (n-1)/6\), this is \(\Omega(n \lg n)\). Each exchange costs \(\Theta(1)\) time, so every run of HEAPSORT on \(n\) distinct keys takes \(\Omega(n \lg n)\) time.

The procedure returns \(15\) and leaves the heap \(\langle 13, 12, 9, 5, 6, 8, 7, 4, 0, 1, 2 \rangle\). Lines 3–5 save \(max = 15\), copy the last element \(A[12] = 1\) into \(A[1]\) and shrink the heap to size \(11\); then MAX-HEAPIFY\((A, 1)\) sifts the \(1\) down, each time exchanging it with its larger child:

At node \(1\) the children are \(13, 9\); at node \(2\) they are \(5, 12\); at node \(5\) they are \(6, 2\); node \(10\) is a leaf (\(2 \cdot 10 > 11\)), so MAX-HEAPIFY stops and line 7 returns \(15\).

The result is \(A = \langle 15, 13, 10, 5, 12, 9, 7, 4, 0, 6, 2, 1, 8 \rangle\). Line 1 grows the heap to size \(13\), line 2 sets \(A[13] = -\infty\), and HEAP-INCREASE-KEY\((A, 13, 10)\) sets \(A[13] = 10\) and bubbles it up past every smaller parent:

Now \(i = 3\) and \(A[\mathrm{PARENT}(3)] = A[1] = 15 \ge 10\), so the while loop of lines 4–6 terminates.

Mirror the four max-heap procedures of Section 6.5, reversing every key comparison, replacing \(-\infty\) by \(+\infty\), and calling MIN-HEAPIFY (Exercise 6.2-2) in place of MAX-HEAPIFY:

HEAP-MINIMUM(A)
1  return A[1]
HEAP-EXTRACT-MIN(A)
1  if A.heap-size < 1
2      error "heap underflow"
3  min = A[1]
4  A[1] = A[A.heap-size]
5  A.heap-size = A.heap-size - 1
6  MIN-HEAPIFY(A, 1)
7  return min
HEAP-DECREASE-KEY(A, i, key)
1  if key > A[i]
2      error "new key is larger than current key"
3  A[i] = key
4  while i > 1 and A[PARENT(i)] > A[i]
5      exchange A[i] with A[PARENT(i)]
6      i = PARENT(i)
MIN-HEAP-INSERT(A, key)
1  A.heap-size = A.heap-size + 1
2  A[A.heap-size] = +infinity
3  HEAP-DECREASE-KEY(A, A.heap-size, key)

Correctness: \(A\) is a min-heap iff the negated array \(-A\) is a max-heap, and each procedure above, run on \(A\), performs exactly the moves of HEAP-MAXIMUM, HEAP-EXTRACT-MAX, HEAP-INCREASE-KEY\((-A, i, -key)\) and MAX-HEAP-INSERT\((-A, -key)\) run on \(-A\) (every comparison \(x < y\) becomes \(-x > -y\), and \(-\infty\) becomes \(+\infty\)). Hence their correctness is the book’s correctness argument for the max versions (Exercise 6.5-5 for the key-change loop), transported through negation.

It guarantees that HEAP-INCREASE-KEY’s preconditions hold for the new node. Without line 2, \(A[A.heap\text{-}size]\) holds whatever stale value was left in the array, and:

(i) If that stale value exceeds \(key\), line 1 of HEAP-INCREASE-KEY fires and the insertion aborts with “new key is smaller than current key”; since \(key \ge -\infty\) for every key, setting \(-\infty\) makes the check pass always.

(ii) HEAP-INCREASE-KEY is correct only when called on a max-heap (the hypothesis of Exercise 6.5-5). With the new leaf equal to \(-\infty\), its parent satisfies \(A[\mathrm{PARENT}(n)] \ge -\infty = A[n]\), so \(A[1..n]\) is a genuine max-heap before the key is raised, and the call is a legitimate INCREASE-KEY from \(-\infty\) to \(key\), to which Exercise 6.5-5 applies verbatim.

The invariant holds initially, is preserved by each exchange, and at exit the single possible violation is absent, so \(A\) is a max-heap. Write \(l, r\) for the children of \(i\), \(p = \mathrm{PARENT}(i)\), and \(A_0\) for the array on entry (a max-heap).

Initialization. Line 3 changes only \(A[i]\), raising it to \(key \ge A_0[i]\). Every edge not incident to \(i\) is untouched; the edges from \(i\) to its children still hold, since \(A[i] = key \ge A_0[i] \ge A[l], A[r]\); so the only possible violation is on the edge \((p, i)\). Moreover \(A[p] = A_0[p] \ge A_0[i] \ge A[l], A[r]\), which is the first clause.

Maintenance. The body runs only when \(i > 1\) and \(A[p] < A[i]\). Let \(s\) be the sibling of \(i\) (if any), and let \(a = A[i]\), \(b = A[p]\) before line 5, so \(b < a\). After the exchange \(A[p] = a\) and \(A[i] = b\), and:

\begin{equation*} \begin{aligned} A[i] &= b \ge A[l],\ A[r] &&\text{(first clause of the invariant)},\\ A[p] &= a > b = A[i],\\ A[p] &= a > b \ge A[s] &&\text{(edge } (p,s) \text{ was not the violation)}. \end{aligned} \end{equation*}

So every edge below \(p\) now holds, and the only edge that can fail is \((\mathrm{PARENT}(p), p)\). Line 6 sets \(i = p\). For the first clause at the new \(i\) (when \(p > 1\)), the children of \(p\) now hold \(b\) and \(A[s]\), and \(A[\mathrm{PARENT}(p)] \ge b \ge A[s]\) because the edge \((\mathrm{PARENT}(p), p)\) held before the exchange and \(\mathrm{PARENT}(p)\) was not modified.

Termination. Each iteration halves \(i\), so the loop ends after at most \(\lfloor \lg n \rfloor\) iterations, either with \(i = 1\) (the root has no parent, so there is no violation) or with \(A[\mathrm{PARENT}(i)] \ge A[i]\) (the one possible violation is absent). In both cases the invariant says \(A[1..A.heap\text{-}size]\) is a max-heap; and its multiset of keys is that of \(A_0\) with \(A_0[i]\) replaced by \(key\), since line 5 only permutes entries.

Do not write \(key\) into the array until its final position is known: treat \(A[i]\) as a hole, compare each parent against \(key\) itself, and shift smaller parents down one level into the hole, exactly as INSERTION-SORT shifts larger elements right (lines 5–7 of INSERTION-SORT, Section 2.1).

HEAP-INCREASE-KEY(A, i, key)
1  if key < A[i]
2      error "new key is smaller than current key"
3  while i > 1 and A[PARENT(i)] < key
4      A[i] = A[PARENT(i)]
5      i = PARENT(i)
6  A[i] = key

Correctness: by induction on the number of iterations, the array of this version equals the array of the book’s version at the same point except that position \(i\) holds a stale copy instead of \(key\) (the book’s version always has \(A[i] = key\) at the loop test). Hence both versions evaluate the same test \(A[\mathrm{PARENT}(i)] < key\); an iteration of the book’s version moves \(A[\mathrm{PARENT}(i)]\) to \(i\) and \(key\) to \(\mathrm{PARENT}(i)\), while line 4 here moves \(A[\mathrm{PARENT}(i)]\) to \(i\) and leaves the hole at \(\mathrm{PARENT}(i)\), preserving the correspondence. At exit line 6 fills the hole with \(key\), so the final array is identical to the book’s, which is a max-heap by Exercise 6.5-5. The loop body now performs one array assignment (line 4) per iteration plus a single assignment on line 6.

Stamp each element with an insertion counter and use the stamp as its key: decreasing stamps give a queue, increasing stamps give a stack. We use a max-priority queue \(Q\) together with a counter \(c\), initially \(0\).

Queue: ENQUEUE\((x)\) sets \(c = c+1\) and calls MAX-HEAP-INSERT\((Q, x)\) with key \(-c\). DEQUEUE calls HEAP-EXTRACT-MAX\((Q)\). The keys of the stored elements are distinct, and an element inserted earlier has a larger key. So the maximum key belongs to the oldest element still present, which is exactly FIFO order.

Stack: PUSH\((x)\) sets \(c = c+1\) and calls MAX-HEAP-INSERT\((Q, x)\) with key \(c\). POP calls HEAP-EXTRACT-MAX\((Q)\). Now the maximum key belongs to the newest element still present, which is LIFO order.

Each operation is one priority-queue operation plus \(O(1)\) extra work, so it runs in \(O(\lg n)\) time on an \(n\)-element heap (Section 6.5).

Move the last element into node \(i\), shrink the heap, and then sift that element up or down, depending on whether it is larger or smaller than the item it replaced.

HEAP-DELETE(A, i)
1  key = A[A.heap-size]
2  A.heap-size = A.heap-size - 1
3  if i > A.heap-size            // node i was the last node
4      return
5  if key > A[i]
6      HEAP-INCREASE-KEY(A, i, key)
7  else A[i] = key
8      MAX-HEAPIFY(A, i)

Correctness. Deleting the last leaf leaves a max-heap \(A[1 \,..\, n-1]\) in which \(A[i]\) still holds the item being deleted. Lines 5-8 then change the key at node \(i\) to key. (i) If key \(> A[i]\), this is exactly what HEAP-INCREASE-KEY does, and it is correct by Exercise 6.5-5. (ii) If key \(\le A[i]\), every ancestor of \(i\) is \(\ge\) the old \(A[i] \ge\) key, so the only possible violation is at \(i\) against its children. The subtrees rooted at LEFT\((i)\) and RIGHT\((i)\) are max-heaps, which is the precondition of MAX-HEAPIFY, so line 8 restores the heap. In either case the remaining items are exactly the original items minus the deleted one.

Running time. Lines 1-5 take \(O(1)\) time, and HEAP-INCREASE-KEY and MAX-HEAPIFY each take \(O(\lg n)\) time (Section 6.5 and Section 6.2), so HEAP-DELETE runs in \(O(\lg n)\) time.

Keep a min-heap that holds the current front element of each list, keyed by its value. Repeatedly extract the minimum, append it to the output, and insert the next element of the list it came from. We assume empty lists are discarded first, so \(k \le n\).

K-WAY-MERGE(L_1, ..., L_k)
1  let H be an empty min-heap and B an empty list
2  for j = 1 to k
3      append (head of L_j, j) to H, keyed by the value
4  BUILD-MIN-HEAP(H)
5  while H is nonempty
6      (x, j) = HEAP-EXTRACT-MIN(H)
7      append x to B
8      remove x from the front of L_j
9      if L_j is nonempty
10         MIN-HEAP-INSERT(H, (head of L_j, j))
11 return B

Loop invariant: at the start of each iteration of the while loop, \(B\) is the sorted list of the elements already removed from the lists, every one of them is \(\le\) every element still in a list, and \(H\) holds exactly the head of each nonempty list.

Initialization: \(B\) is empty, and lines 2-4 put every head into \(H\) as a min-heap.

Maintenance: each list is sorted, so its head is its minimum, and the minimum \(x\) of \(H\) is therefore the minimum of all remaining elements. Appending \(x\) keeps \(B\) sorted and keeps \(B \le\) the remaining elements. Lines 8-10 replace the head of \(L_j\) by its new head (if any), so the statement about \(H\) holds again.

Termination: when \(H\) is empty, every list is empty, so \(B\) is the sorted merge of all \(n\) elements.

Running time: lines 2-4 take \(O(k)\) time. Each of the \(n\) elements is extracted once and inserted at most once, and the heap never holds more than \(k\) items, so each heap operation costs \(O(\lg k)\) time. The total is \(O(k + n \lg k) = O(n \lg k)\), using \(k \le n\) and \(k \ge 2\) (for \(k = 1\) the lone list is returned as is).

(a) No. Take \(A = \langle 1, 2, 3 \rangle\).

BUILD-MAX-HEAP calls MAX-HEAPIFY\((A,1)\) only; the larger child of the root is \(A[3] = 3\), so it swaps \(A[1]\) and \(A[3]\) and returns \(\langle 3, 2, 1 \rangle\).

BUILD-MAX-HEAP’ starts with the heap \(\langle 1 \rangle\); inserting \(2\) places it at node \(2\), where it exceeds its parent, giving \(\langle 2, 1 \rangle\); inserting \(3\) places it at node \(3\), whose parent holds \(2 < 3\), giving \(\langle 3, 1, 2 \rangle\). The two heaps differ.

(b) Upper bound: the heap never has more than \(n\) elements, and MAX-HEAP-INSERT on a heap of at most \(n\) elements runs in \(O(\lg n)\) time (Section 6.5), so the \(n-1\) calls take \(O(n \lg n)\).

Lower bound: let \(A = \langle 1, 2, \ldots, n \rangle\) be sorted increasingly. When \(A[i] = i\) is inserted at node \(i\), it is larger than every key already in the heap, so the while loop of HEAP-INCREASE-KEY never finds \(A[\mathrm{PARENT}(i)] \ge A[i]\) and moves the key all the way to the root, performing one iteration per ancestor, i.e. \(\lfloor \lg i \rfloor\) iterations (the depth of node \(i\)). Hence for \(n \ge 4\) the total work is at least

\begin{equation*} \begin{aligned} \sum_{i=2}^{n} \lfloor \lg i \rfloor &\ge \sum_{i=\lceil n/2 \rceil}^{n} \lfloor \lg (n/2) \rfloor \ge \frac{n}{2}\left(\lg \frac{n}{2} - 1\right) \\ &= \frac{n}{2}(\lg n - 2) = \Omega(n \lg n). \end{aligned} \end{equation*}

(a) Store the nodes level by level, left to right, in \(A[1 \ldots n]\) with the root in \(A[1]\); then node \(i\) has children \(d(i-1)+2, \ldots, di+1\) and parent \(\lfloor (i-2)/d \rfloor + 1\):

\begin{equation*} \begin{aligned} \mathrm{CHILD}(i,j) &= d(i-1) + j + 1, \qquad j = 1, \ldots, d, \\ \mathrm{PARENT}(i) &= \lfloor (i-2)/d \rfloor + 1 . \end{aligned} \end{equation*}

Indeed, the nodes preceding the children of \(i\) are the root and the \(d(i-1)\) children of nodes \(1, \ldots, i-1\), which accounts for the first formula; for \(c = d(i-1)+j+1\) with \(1 \le j \le d\) we get \(\lfloor (c-2)/d \rfloor = i - 1 + \lfloor (j-1)/d \rfloor = i-1\), which gives the second.

(b) The height is \(h = \lceil \log_d (n(d-1)+1) \rceil - 1 = \Theta(\lg n / \lg d)\). A heap of height \(h\) has all levels \(0, \ldots, h-1\) full and at least one node at depth \(h\), so

\begin{equation*} \begin{aligned} 1 + d + \cdots + d^{h-1} &< n \le 1 + d + \cdots + d^{h}, \\ \text{i.e.}\quad d^{h} &< n(d-1) + 1 \le d^{h+1}, \end{aligned} \end{equation*}

which is the stated formula; since \(n \le n(d-1)+1 \le dn\), it is \(\log_d n + O(1)\).

(c) Exactly as HEAP-EXTRACT-MAX (Section 6.5), but sifting down with a \(d\)-way MAX-HEAPIFY that compares \(A[i]\) against all of its children:

D-ARY-MAX-HEAPIFY(A, i)
1  largest = i
2  for j = 1 to d
3      c = CHILD(i, j)
4      if c <= A.heap-size and A[c] > A[largest]
5          largest = c
6  if largest != i
7      exchange A[i] with A[largest]
8      D-ARY-MAX-HEAPIFY(A, largest)

D-ARY-EXTRACT-MAX(A)
1  if A.heap-size < 1
2      error "heap underflow"
3  max = A[1]
4  A[1] = A[A.heap-size]
5  A.heap-size = A.heap-size - 1
6  D-ARY-MAX-HEAPIFY(A, 1)
7  return max

Correctness is the argument for MAX-HEAPIFY with “two children” replaced by “\(d\) children”: if the subtrees of all children of \(i\) are max-heaps, then after lines 1–5 \(A[\mathit{largest}]\) is the maximum of \(A[i]\) and its children, so after the exchange node \(i\) satisfies the heap property and only the subtree rooted at \(\mathit{largest}\) can be violated, and only at its root, which the recursive call fixes by induction on height. In D-ARY-EXTRACT-MAX the root’s subtrees are untouched max-heaps after line 5, so line 6 restores the heap and the returned \(A[1]\) was the maximum. Each call of D-ARY-MAX-HEAPIFY does \(\Theta(d)\) work and then recurses one level lower, so by (b) the running time is

\begin{equation*} O(d \cdot h) = O(d \log_d n) = O\!\left(\frac{d \lg n}{\lg d}\right). \end{equation*}

(d) Exactly MAX-HEAP-INSERT (Section 6.5): increment \(A.\mathit{heap\text{-}size}\), set the new last key to \(-\infty\), and call D-ARY-INCREASE-KEY from (e) on it. Correctness is that of MAX-HEAP-INSERT and of (e); the cost is \(O(1)\) plus that of (e), namely \(O(\log_d n) = O(\lg n / \lg d)\).

(e) Sift up, as in HEAP-INCREASE-KEY, using the PARENT of (a):

D-ARY-INCREASE-KEY(A, i, k)
1  if k < A[i]
2      error "new key is smaller than current key"
3  A[i] = k
4  while i > 1 and A[PARENT(i)] < A[i]
5      exchange A[i] with A[PARENT(i)]
6      i = PARENT(i)

Correctness is the loop invariant of Exercise 6.5-5 with “LEFT\((i)\) and RIGHT\((i)\)” replaced by “every child of \(i\)”: the only possible violation is between \(A[i]\) and \(A[\mathrm{PARENT}(i)]\), an exchange moves it one level up (the old parent’s key is at least each of \(i\)’s children, and the new key exceeds the old parent’s), and the loop stops at the root or when that violation is gone. Only one comparison per level is needed, so the running time is \(O(h) = O(\log_d n) = O(\lg n / \lg d)\).

(a) Fill the rows in sorted order:

Every row and every column is nondecreasing. (Check!)

(b) \(Y[1,1]\) is the minimum and \(Y[m,n]\) the maximum entry: for every cell \((i,j)\),

\begin{equation*} Y[1,1] \le Y[1,j] \le Y[i,j] \le Y[i,n] \le Y[m,n], \end{equation*}

using row \(1\), column \(j\), row \(i\), column \(n\) in turn. So \(Y[1,1] = \infty\) forces every entry to be \(\infty\) (empty), and \(Y[m,n] < \infty\) forces every entry to be finite (full).

(c) Remove \(Y[1,1]\), put \(\infty\) there, and sift that entry down/right as in MAX-HEAPIFY: swap it with the smaller of its lower and right neighbours while that neighbour is smaller.

EXTRACT-MIN(Y)
1  x = Y[1,1]
2  Y[1,1] = infinity
3  YOUNGIFY(Y, 1, 1)
4  return x

YOUNGIFY(Y, i, j)
1  (s, t) = (i, j)
2  if i < m and Y[i+1, j] < Y[s, t]
3      (s, t) = (i+1, j)
4  if j < n and Y[i, j+1] < Y[s, t]
5      (s, t) = (i, j+1)
6  if (s, t) != (i, j)
7      exchange Y[i, j] with Y[s, t]
8      YOUNGIFY(Y, s, t)

Correctness. Invariant at each call YOUNGIFY\((Y,i,j)\): every row/column ordering holds except possibly \(Y[i,j] \le Y[i+1,j]\) and \(Y[i,j] \le Y[i,j+1]\); moreover \(Y[i-1,j] \le Y[i+1,j]\) and \(Y[i,j-1] \le Y[i,j+1]\) (whenever these cells exist).

Initialization: at \((1,1)\) only the two excepted orderings involve the changed cell, and there is no row \(0\) or column \(0\).

Maintenance: say \(b = Y[i+1,j]\) is chosen, so \(b \le Y[i,j+1]\) and \(b < x = Y[i,j]\). After the swap, \(b\) sits at \((i,j)\) with

\begin{equation*} Y[i-1,j] \le b, \quad Y[i,j-1] \le Y[i+1,j-1] \le b, \quad b \le Y[i,j+1], \quad b < x, \end{equation*}

(the first by the invariant, the second by column \(j-1\) then row \(i+1\)), and \(x\) at \((i+1,j)\) satisfies \(Y[i+1,j-1] \le b < x\). The new extra conditions are \(b = Y[i,j] \le Y[i+2,j]\) (column \(j\)) and \(Y[i+1,j-1] \le b \le Y[i+1,j+1]\) (row \(i+1\)). The case where the right neighbour is chosen is symmetric (Check!).

Termination: the recursion stops when \(x\) is \(\le\) both existing neighbours, so the excepted orderings hold too and \(Y\) is a Young tableau. Since \(Y[1,1]\) was the minimum by (b), EXTRACT-MIN returns the minimum and removes exactly one finite entry.

Running time. A call at \((i,j)\) works on the subtableau of rows \(i..m\), columns \(j..n\), of size \((m-i+1) \times (n-j+1)\); it does \(\Theta(1)\) work and recurses on a subtableau with one fewer row or one fewer column. With \(p\) the sum of the dimensions,

\begin{equation*} T(2) = \Theta(1), \qquad T(p) \le T(p-1) + c \quad (p > 2), \end{equation*}

so by induction \(T(p) \le T(2) + c(p-2) = O(p) = O(m+n)\).

(d) By (b), a nonfull tableau has \(Y[m,n] = \infty\). Put the new key there and sift it up/left, swapping with the larger of its upper and left neighbours while that neighbour is larger.

INSERT(Y, k)
1  (i, j) = (m, n)
2  Y[m, n] = k
3  repeat
4      (s, t) = (i, j)
5      if i > 1 and Y[i-1, j] > Y[s, t]
6          (s, t) = (i-1, j)
7      if j > 1 and Y[i, j-1] > Y[s, t]
8          (s, t) = (i, j-1)
9      if (s, t) == (i, j)
10         return
11     exchange Y[i, j] with Y[s, t]
12     (i, j) = (s, t)

This is YOUNGIFY applied to the tableau rotated by \(180^\circ\) with the order reversed (which turns a Young tableau into a Young tableau and maps \((m,n)\) to \((1,1)\)), so its correctness follows from (c). Replacing \(Y[m,n]=\infty\) by \(k\) only can break orderings with the upper and left neighbours, which is exactly the excepted situation. Each iteration costs \(\Theta(1)\) and decreases \(i + j\) by \(1\), from \(m+n\) down to at least \(2\), so it runs in \(O(m+n)\) time.

(e) Start from the empty \(n \times n\) tableau (all \(\infty\)), INSERT the \(n^2\) numbers, then call EXTRACT-MIN \(n^2\) times and output the values in order. Before each insertion fewer than \(n^2\) cells are finite, so the tableau is nonfull; each extraction returns the current minimum by (c), so the output is sorted. The cost is

\begin{equation*} n^2 \cdot O(2n) + n^2 \cdot O(2n) = O(n^3). \end{equation*}

(f) Start at the top-right corner and discard a row or a column at each comparison.

YOUNG-SEARCH(Y, x)
1  i = 1
2  j = n
3  while i <= m and j >= 1
4      if Y[i, j] == x
5          return TRUE
6      elseif Y[i, j] > x
7          j = j - 1
8      else i = i + 1
9  return FALSE

Invariant: if \(x\) is stored in \(Y\), it lies in rows \(i..m\) and columns \(1..j\). Initially this is the whole tableau. If \(Y[i,j] > x\), every entry of column \(j\) in rows \(i..m\) is \(\ge Y[i,j] > x\), so column \(j\) can be dropped; if \(Y[i,j] < x\), every entry of row \(i\) in columns \(1..j\) is \(\le Y[i,j] < x\), so row \(i\) can be dropped. On exit the remaining region is empty, so FALSE is correct, and TRUE is returned only on an actual match. Each iteration takes \(\Theta(1)\) time and increases \(i\) or decreases \(j\), so there are at most \(m + n\) iterations: \(O(m+n)\) time.

PARTITION\((A, 1, 12)\) uses pivot \(x = A[12] = 11\) and returns \(q = 8\), leaving \(A = \langle 9, 5, 8, 7, 4, 2, 6, 11, 21, 13, 19, 12 \rangle\). The table shows the array after each iteration of the loop of lines 3–6 (the value \(j\) is the index just examined; \(i = 0\) means \(i = p - 1\)), so that \(A[1 \,..\, i] \le 11\) and \(A[i+1 \,..\, j] > 11\) in every row.

It returns \(q = r\): every \(A[j]\) satisfies \(A[j] \le x\), so \(i\) is incremented in each of the \(r - p\) iterations, ending at \(i = r - 1\), and line 8 returns \(i + 1 = r\).

To fix this, send elements equal to the pivot to the low side only on every other index:

PARTITION'(A, p, r)
1  x = A[r]
2  i = p - 1
3  for j = p to r - 1
4      if A[j] < x or (A[j] == x and (j - p) mod 2 == 1)
5          i = i + 1
6          exchange A[i] with A[j]
7  exchange A[i + 1] with A[r]
8  return i + 1

Correctness: the argument for the loop invariant of Section 7.1 goes through verbatim with condition 2 weakened to \(A[k] \ge x\) for \(i + 1 \le k \le j - 1\), because an element passed over by line 4 satisfies \(A[j] \ge x\). Hence on return \(A[p \,..\, q-1] \le A[q] = x \le A[q+1 \,..\, r]\), which is all QUICKSORT needs. The running time is unchanged, \(\Theta(n)\) (Exercise 7.1-3).

All-equal case: the test in line 4 succeeds exactly for the \(j \in \{p, \ldots, r-1\}\) with \(j - p\) odd, i.e. \(j - p \in \{1, 3, \ldots\} \cap \{0, \ldots, r-p-1\}\), which has \(\lfloor (r-p)/2 \rfloor\) elements. So

\begin{equation*} q = p - 1 + \left\lfloor \tfrac{r-p}{2} \right\rfloor + 1 = \left\lfloor \tfrac{2p + r - p}{2} \right\rfloor = \left\lfloor \tfrac{p+r}{2} \right\rfloor . \end{equation*}

Lines 1, 2, 7 and 8 run once, and the for loop of lines 3–6 runs once for each \(j = p, \ldots, r - 1\), i.e. \(n - 1\) times, where \(n = r - p + 1\). Each iteration performs exactly one comparison (line 4) and at most one increment and one exchange (lines 5–6), so it costs between constants \(c_1 > 0\) and \(c_2\). With \(c_0\) the cost of lines 1, 2, 7, 8,

\begin{equation*} c_0 + c_1 (n - 1) \;\le\; T(n) \;\le\; c_0 + c_2 (n - 1), \end{equation*}

hence \(T(n) = \Theta(n)\).

Reverse the comparison in line 4 of PARTITION, testing \(A[j] \ge x\) instead of \(A[j] \le x\); QUICKSORT itself is unchanged.

With this change the loop invariant of Section 7.1 holds with every inequality reversed (the proof is word for word the same): at the start of each iteration \(A[k] \ge x\) for \(p \le k \le i\), \(A[k] < x\) for \(i + 1 \le k \le j - 1\), and \(A[r] = x\). So on return

\begin{equation*} A[p \,..\, q-1] \;\ge\; A[q] = x \;>\; A[q+1 \,..\, r] . \end{equation*}

By induction on \(r - p\), the two recursive calls leave \(A[p \,..\, q-1]\) and \(A[q+1 \,..\, r]\) in nonincreasing order, and together with the display the whole of \(A[p \,..\, r]\) is then nonincreasing.

Write the recurrence as \(T(n) = T(n-1) + f(n)\) for \(n \ge 1\), with \(T(0) = d \ge 0\) a constant and \(c_1 n \le f(n) \le c_2 n\) for all \(n \ge 1\), where \(0 < c_1 \le c_2\) (the \(\Theta(n)\) bounds hold for \(n \ge n_0\); the finitely many positive partitioning costs below \(n_0\) are absorbed by shrinking \(c_1\) and enlarging \(c_2\)).

Upper bound. Inductive hypothesis: \(T(m) \le c_2 m^2 + d\) for all \(m < n\). Base case: \(T(0) = d = c_2 \cdot 0^2 + d\). For \(n \ge 1\),

\begin{equation*} \begin{aligned} T(n) &\le c_2 (n-1)^2 + d + c_2 n \\ &= c_2 n^2 + d - c_2 (n-1) \le c_2 n^2 + d . \end{aligned} \end{equation*}

Hence \(T(n) \le (c_2 + d) n^2\) for all \(n \ge 1\), so \(T(n) = O(n^2)\).

Lower bound. Inductive hypothesis: \(T(m) \ge (c_1/2) m^2\) for all \(m < n\). Base case: \(T(0) = d \ge 0\). For \(n \ge 1\),

\begin{equation*} \begin{aligned} T(n) &\ge \tfrac{c_1}{2}(n-1)^2 + c_1 n \\ &= \tfrac{c_1}{2} n^2 + \tfrac{c_1}{2} \ge \tfrac{c_1}{2} n^2 , \end{aligned} \end{equation*}

so \(T(n) = \Omega(n^2)\). Together, \(T(n) = \Theta(n^2)\).

\(\Theta(n^2)\). In PARTITION\((A,p,r)\) the test \(A[j] \le x\) in line 4 succeeds for every \(j = p, \ldots, r-1\), so \(i\) is incremented \(r-p\) times and the procedure returns \(q = r\). Every call on a subarray of size \(m\) therefore costs \(\Theta(m)\) and leaves subproblems of sizes \(m-1\) (again all equal) and \(0\), giving

\begin{equation*} T(n) = T(n-1) + T(0) + \Theta(n) = T(n-1) + \Theta(n), \end{equation*}

which is \(\Theta(n^2)\) by Exercise 7.2-1.

Every call of PARTITION has an empty side, so the subarray sizes are \(n, n-1, \ldots, 1\) and the total cost is \(\Theta(n^2)\).

Let \(a_1 > a_2 > \cdots > a_m\) occupy \(A[p\,..\,r]\), \(m \ge 2\). The pivot \(a_m\) is the minimum, so line 4 never succeeds, \(i = p-1\), and line 7 swaps \(A[p]\) with \(A[r]\): \(q = p\), the low side is empty, and the high side is \(A[p+1\,..\,r] = \langle a_2, \ldots, a_{m-1}, a_1 \rangle\). In the recursive call on it the pivot \(a_1\) is the maximum, so (as in Exercise 7.2-2) every test succeeds, each swap is of \(A[j]\) with itself, and \(q = r\): the high side is empty and the low side is \(\langle a_2, \ldots, a_{m-1} \rangle\), again decreasing. By induction every call is on a decreasing array or on a decreasing array followed by its maximum, and every split is \((m-1) : 0\). Since PARTITION on \(m\) elements costs \(\Theta(m)\), the running time satisfies \(T(n) = T(n-1) + \Theta(n)\), which is \(\Theta(n^2)\) by Exercise 7.2-1.

On almost-sorted input INSERTION-SORT runs in nearly linear time, while QUICKSORT (with its last-element pivot) makes nearly maximally unbalanced splits and runs in nearly quadratic time.

Make “almost sorted” precise: the \(n\) check numbers are distinct and each check has at most \(k\) larger-numbered checks cashed before it.

INSERTION-SORT. Each element is in at most \(k\) inversions with elements to its left, so the number of inversions is \(I \le kn\), and by Problem 2-4(c) the running time is \(\Theta(n + I) = O((k+1) n)\).

QUICKSORT. Two facts about a call PARTITION\((A,p,r)\) on a subarray that is a subsequence of the input, with pivot \(x = A[r]\):

(i) the elements greater than \(x\) all precede \(x\), so there are at most \(k\) of them and the high side has size at most \(k\);

(ii) the elements smaller than \(x\) are moved into \(A[p\,..\,q-1]\) in the order in which \(j\) meets them, so the low side is again a subsequence of the input.

Following the chain of low sides from the root, the subarray sizes \(s_0 = n, s_1, s_2, \ldots\) satisfy \(s_{t+1} \ge s_t - (k+1)\), so \(s_t \ge n - t(k+1) \ge 1\) for \(t \le T = \lfloor (n-1)/(k+1) \rfloor\), and a PARTITION on \(s_t\) elements makes \(s_t - 1\) comparisons. The summands below decrease linearly from \(n-1\) to a nonnegative value, so their average is at least \((n-1)/2\):

\begin{equation*} \begin{aligned} \sum_{t=0}^{T} (s_t - 1) &\ge \sum_{t=0}^{T} \bigl(n - 1 - t(k+1)\bigr) \\ &\ge \frac{(T+1)(n-1)}{2} \ge \frac{(n-1)^2}{2(k+1)} = \Omega\Bigl(\frac{n^2}{k+1}\Bigr). \end{aligned} \end{equation*}

So for \(k = o(\sqrt{n})\), in particular \(k\) constant (checks cashed with reasonable dispatch), INSERTION-SORT takes \(O(kn + n) = o(n^{3/2})\) time against QUICKSORT’s \(\Omega(n^2/(k+1))\).

A node at depth \(d\) has size between \(\alpha^d n\) and \((1-\alpha)^d n\), so leaf depths lie between \(\log_{1/\alpha} n = -\lg n/\lg\alpha\) and \(\log_{1/(1-\alpha)} n = -\lg n/\lg(1-\alpha)\), and both extremes are attained.

Each child of a node of size \(s\) has size \(\alpha s\) or \((1-\alpha) s\), and \(\alpha \le 1-\alpha\), so by induction on \(d\) a node at depth \(d\) has size \(s\) with \(\alpha^d n \le s \le (1-\alpha)^d n\). A leaf has size about \(1\), so its depth \(d\) satisfies (recall \(\lg\alpha \le \lg(1-\alpha) < 0\))

\begin{equation*} \begin{aligned} \alpha^d n \le 1 &\iff d \ge -\lg n / \lg \alpha , \\ (1-\alpha)^d n \ge 1 &\iff d \le -\lg n / \lg(1-\alpha) . \end{aligned} \end{equation*}

The path that always takes the \(\alpha\) side has size \(\alpha^d n\) at depth \(d\) and reaches size \(1\) at depth \(-\lg n/\lg\alpha\); the path that always takes the \(1-\alpha\) side reaches size \(1\) at depth \(-\lg n/\lg(1-\alpha)\). These are therefore the minimum and maximum leaf depths.

The pivot’s rank is uniform on \(\{1, \ldots, n\}\), and the split beats \(1-\alpha\) to \(\alpha\) exactly for the roughly \((1-2\alpha)n\) middle ranks, so the probability is \(1 - 2\alpha + O(1/n)\).

Under the Section 7.2 assumption that all orderings of \(n\) distinct elements are equally likely, the pivot \(x = A[r]\) is equally likely to have each rank \(k \in \{1, \ldots, n\}\). If \(x\) has rank \(k\), PARTITION returns sides of sizes \(k-1\) and \(n-k\), and the split is more balanced than \(1-\alpha\) to \(\alpha\) when both sides exceed \(\alpha n\), i.e. when

\begin{equation*} \alpha n + 1 < k < (1-\alpha) n . \end{equation*}

This open interval has length \(L = (1-2\alpha) n - 1\), so it contains \(c\) integers with \(L - 1 \le c \le L + 1\). The probability is \(c/n\), hence

\begin{equation*} 1 - 2\alpha - \frac{2}{n} \le \frac{c}{n} \le 1 - 2\alpha , \end{equation*}

which is \(1 - 2\alpha\) up to round-off of order \(1/n\).

Because for a randomized algorithm the worst case is no longer caused by the input but by an unlucky run of random choices, and that happens with negligible probability on every input; the expected time is what describes how the algorithm actually behaves.

For RANDOMIZED-QUICKSORT the worst-case time is still \(\Theta(n^2)\) (Section 7.4.1), but it occurs only when the random pivots keep producing very unbalanced splits, an event of tiny probability that no input can force, since the pivot is chosen independently of the input order. The informative quantity is the expected running time over the random choices, maximized over inputs; for RANDOMIZED-QUICKSORT it is \(O(n \lg n)\) on every input (Section 7.4.2).

In both cases the answer is \(\Theta(n)\): exactly \(n-1\) calls in the worst case and \(\lfloor n/2 \rfloor\) in the best case.

Each invocation of RANDOMIZED-QUICKSORT on a subarray with \(p<r\), meaning at least 2 elements, calls RANDOMIZED-PARTITION once, and that procedure calls RANDOM once. Invocations on 0 or 1 elements call nothing. So if \(C(n)\) is the number of calls on \(n\) elements and the pivot ends up at rank \(q+1\), then

\begin{equation*} C(0)=C(1)=0, \qquad C(n) = 1 + C(q) + C(n-1-q) \quad (n \ge 2,\ 0 \le q \le n-1). \end{equation*}

Worst case: \(C(n) \le n-1\) for \(n \ge 1\), by induction. If \(q=0\) (and symmetrically if \(q=n-1\)), then \(C(n) \le 1 + (n-2) = n-1\). If \(1 \le q \le n-2\), then \(C(n) \le 1 + (q-1) + (n-q-2) = n-2\). Equality holds when the pivot is always the largest element, since then \(C(n) = 1 + C(n-1)\).

Best case: \(C(n) \ge \lfloor n/2 \rfloor\), again by induction. Using \(\lfloor a/2 \rfloor + \lfloor b/2 \rfloor \ge (a+b-2)/2\),

\begin{equation*} C(n) \ge 1 + \left\lfloor \tfrac{q}{2} \right\rfloor + \left\lfloor \tfrac{n-1-q}{2} \right\rfloor \ge 1 + \frac{n-3}{2} = \frac{n-1}{2}, \end{equation*}

and since \(C(n)\) is an integer, \(C(n) \ge \lceil (n-1)/2 \rceil = \lfloor n/2 \rfloor\). Equality holds when the pivot is always the second-smallest element, since then \(C(n) = 1 + C(1) + C(n-2) = 1 + C(n-2)\).

We prove \(T(n) \ge c n^2\) for all \(n \ge 0\) with \(c = a/2\), where \(a > 0\) is a constant such that the \(\Theta(n)\) term is at least \(an\) for every \(n \ge 1\) (it is a positive running-time term, so such \(a\) exists), and \(T \ge 0\).

Base case: \(T(0) \ge 0 = c \cdot 0^2\).

Inductive step: let \(n \ge 1\) and assume \(T(m) \ge c m^2\) for all \(m < n\). The maximum is at least its \(q = 0\) term, so

\begin{equation*} \begin{aligned} T(n) &\ge T(0) + T(n-1) + an \ge c(n-1)^2 + an \\ &= cn^2 + \bigl(an - c(2n-1)\bigr) \\ &= cn^2 + \tfrac{a}{2} \ge cn^2 , \end{aligned} \end{equation*}

using \(c = a/2\) in the last line. Hence \(T(n) \ge (a/2) n^2\) for all \(n\), i.e. \(T(n) = \Omega(n^2)\).

The best-case time \(B(n)\) satisfies \(B(n) \ge \min_{0 \le q \le n-1}\bigl(B(q) + B(n-q-1)\bigr) + an\) for \(n \ge 1\) and some constant \(a > 0\). The reason is that whatever split PARTITION produces, the run costs at least \(an\) for the partition plus at least the best-case cost of each of the two recursive calls. We show \(B(n) \ge c\, n \lg n\) for all \(n \ge 0\) with \(c = a/3\) and the convention \(0 \lg 0 = 0\).

Base cases: \(B(0), B(1) \ge 0 = c\,n\lg n\).

Inductive step, \(n \ge 2\): the function \(x \lg x\) is convex on \([0,\infty)\), so for every \(q\), by Jensen’s inequality with midpoint \((n-1)/2\),

\begin{equation*} \begin{aligned} q\lg q + (n-q-1)\lg(n-q-1) &\ge (n-1)\lg\tfrac{n-1}{2} \\ &= (n-1)\lg(n-1) - (n-1) . \end{aligned} \end{equation*}

By the inductive hypothesis, then,

\begin{equation*} B(n) \ge c(n-1)\lg(n-1) - c(n-1) + an . \end{equation*}

Also \(n\lg n - (n-1)\lg(n-1) = \lg n + (n-1)\lg\frac{n}{n-1} \le \lg n + \lg e\), because \((n-1)\ln\bigl(1 + \frac{1}{n-1}\bigr) \le 1\). Therefore

\begin{equation*} \begin{aligned} B(n) - c\,n\lg n &\ge an - c\bigl(\lg n + \lg e + n - 1\bigr) \\ &\ge an - 3cn = 0 , \end{aligned} \end{equation*}

since \(\lg n \le n - 1\) for \(n \ge 2\) and \(\lg e < 2\). Hence \(B(n) \ge (a/3)\, n \lg n = \Omega(n \lg n)\).

Write \(f(q) = q^2 + (n-1-q)^2\). Then

\begin{equation*} \begin{aligned} f(q) &= \bigl(q + (n-1-q)\bigr)^2 - 2q(n-1-q) \\ &= (n-1)^2 - 2q(n-1-q) . \end{aligned} \end{equation*}

For \(0 \le q \le n-1\) both \(q\) and \(n-1-q\) are nonnegative, so \(f(q) \le (n-1)^2\). Equality holds exactly when \(q(n-1-q) = 0\), that is, when \(q = 0\) or \(q = n-1\), and \(f(0) = f(n-1) = (n-1)^2\). So the maximum over the range is \((n-1)^2\), attained at the two endpoints.

In fact \(\mathrm{E}[X] = 2(n+1)H_n - 4n \ge 2(n+1)\ln(n+1) - 4n = \Omega(n\lg n)\), where \(X\) is the number of comparisons and the elements are distinct, as assumed in Section 7.4.2. Each comparison in line 4 of PARTITION is one iteration of its for loop and costs at least one unit, so the running time is at least \(X\). It therefore suffices to bound \(\mathrm{E}[X]\) from below. By equations (7.2) and (7.3), substituting \(m = n - i + 1\),

\begin{equation*} \begin{aligned} \mathrm{E}[X] &= \sum_{i=1}^{n-1}\sum_{k=1}^{n-i}\frac{2}{k+1} = 2\sum_{m=2}^{n}\bigl(H_m - 1\bigr) \\ &= 2\Bigl(\sum_{m=1}^{n} H_m - n\Bigr) = 2(n+1)H_n - 4n , \end{aligned} \end{equation*}

using \(\sum_{m=1}^{n} H_m = \sum_{k=1}^{n}\frac{n-k+1}{k} = (n+1)H_n - n\). By the integral approximation (A.12) (page 1154), \(H_n \ge \ln(n+1)\), so

\begin{equation*} \mathrm{E}[X] \ge 2(n+1)\ln(n+1) - 4n \ge n\ln n \end{equation*}

as soon as \(\ln(n+1) \ge 4\), i.e. for \(n \ge 54\). Hence the expected running time is \(\Omega(n \lg n)\).

The truncated quicksort phase costs \(O(n + n\lg(n/k))\) in expectation and the insertion-sort phase costs \(O(nk)\) in the worst case. Their sum is \(O(nk + n\lg(n/k))\) for \(1 \le k \le n\).

Quicksort phase. There are at most \(n\) partitioning calls, since each fixes a pivot, and each other call is a leaf of the recursion tree, so there are at most \(n+1\) of those. The \(O(1)\) overhead per call therefore totals \(O(n)\). By Lemma 7.1 the remaining cost is proportional to the number of comparisons \(X\), and \(X = \sum_x N_x\), where \(N_x\) is the number of partitioning calls (all on \(\ge k\) elements) in which \(x\) is a non-pivot. Call a partitioning call on \(m\) elements good if the pivot rank \(r\) satisfies \(m/4 \le r \le 3m/4 + 1\). Then both sides have at most \(3m/4\) elements. At least \(\lfloor m/2 + 1\rfloor \ge m/2\) ranks qualify (Check!), and the pivot is chosen afresh and uniformly, so every call is good with probability at least \(1/2\) whatever happened before. Along the chain of calls containing \(x\), sizes never increase. After \(g\) good calls the size is at most \(n(3/4)^g\), so at most \(g^* = 1 + \log_{4/3}(n/k)\) of the calls counted by \(N_x\) are good. The calls between consecutive good ones form waiting times dominated by geometric variables with mean at most \(2\), so

\begin{equation*} \mathrm{E}[N_x] \le 2g^* = O\bigl(1 + \lg(n/k)\bigr) , \qquad \mathrm{E}[X] = O\bigl(n + n\lg(n/k)\bigr) . \end{equation*}

Insertion-sort phase. When quicksort returns, every pivot sits in its final position. The rest of the array is a sequence of untouched blocks of fewer than \(k\) elements each, and every element of a block is smaller than every element of each later block. Inversions therefore occur only inside blocks, so their number \(I\) satisfies \(I < nk\), since each element is inverted with fewer than \(k\) others. Each iteration of the inner while loop of INSERTION-SORT moves a key past exactly one larger key and removes exactly one inversion, so the phase runs in \(\Theta(n + I) = O(nk)\).

Choice of \(k\). In theory, \(nk + n\lg(n/k) = O(n\lg n)\) exactly when \(k = O(\lg n)\), and larger \(k\) makes the \(nk\) term dominate. With constants, the cost is \(c_i nk + c_q n\lg(n/k)\), which is minimised at

\begin{equation*} k = \frac{c_q}{c_i \ln 2} . \end{equation*}

Here \(c_i\) and \(c_q\) are the per-step costs of insertion sort and quicksort. This is a constant independent of \(n\). In practice those constants are unknown, so \(k\) is chosen by timing the hybrid for a range of small \(k\) on the target machine and taking the fastest. Tuned implementations typically end up with \(k\) somewhere around \(5\) to \(20\).

For \(0 < \alpha \le 1/2\) the probability is approximately \(1 - 6\alpha^2 + 4\alpha^3\). For \(\alpha > 1/2\), apply the same formula with \(\alpha\) replaced by \(1 - \alpha\), since the \(\alpha\)-to-\((1-\alpha)\) and \((1-\alpha)\)-to-\(\alpha\) conditions coincide.

For large \(n\), treat the three sampled elements as independent, each with a relative rank uniform on \([0,1]\). The \(O(1/n)\) effect of sampling without replacement and of rounding is ignored. The split is worse than \(\alpha\)-to-\((1-\alpha)\) exactly when the median has relative rank below \(\alpha\) or above \(1-\alpha\). The median is below \(\alpha\) exactly when at least two of the three samples are below \(\alpha\):

\begin{equation*} \Pr\{\text{median} < \alpha\} = 3\alpha^2(1-\alpha) + \alpha^3 = 3\alpha^2 - 2\alpha^3 . \end{equation*}

By symmetry, the median is above \(1-\alpha\) with the same probability. For \(\alpha \le 1/2\) the two events are disjoint, so

\begin{equation*} \begin{aligned} \Pr\{\text{at worst } \alpha\text{-to-}(1-\alpha)\} &\approx 1 - 2(3\alpha^2 - 2\alpha^3) \\ &= 1 - 6\alpha^2 + 4\alpha^3 . \end{aligned} \end{equation*}

(a) With \(p = 1\), \(r = 12\) and pivot \(x = 13\), the while loop runs three times and returns \(j = 9\):

So \(A[1\,..\,9] = \langle 6,2,9,5,12,8,7,4,11\rangle\) (all \(\le 13\)) and \(A[10\,..\,12] = \langle 19, 13, 21\rangle\) (all \(\ge 13\)).

For (b)-(d), write \(i_k, j_k\) for the values of \(i, j\) at the test in line 11 during iteration \(k\), and let \(n = r - p + 1 \ge 2\). We prove that every iteration \(k\) that does not return ends with

\begin{equation*} p \le i_k < j_k \le r, \qquad A[p\,..\,i_k] \le x, \qquad A[j_k\,..\,r] \ge x \end{equation*}

(entrywise), and that every scan stays inside \(A[p\,..\,r]\).

(b) Iteration 1: the \(j\)-scan starts at \(r\) and moves down; it must stop at an index \(\ge p\), because \(A[p] = x \le x\). The \(i\)-scan stops at once at \(i_1 = p\), because \(A[p] = x \ge x\). If \(i_1 < j_1\), the swap puts an element \(\le x\) (the old \(A[j_1]\)) in \(A[p]\) and \(x\) in \(A[j_1]\); every element the \(j\)-scan passed over was \(> x\). Hence the invariant holds after iteration 1. Iteration \(k+1\), assuming the invariant after iteration \(k\): the \(j\)-scan starts at \(j_k - 1\) and moves down; it stops at index \(i_k\) at the latest, since \(A[i_k] \le x\), so \(p \le i_k \le j_{k+1} < j_k\). The \(i\)-scan starts at \(i_k + 1\) and stops at index \(j_k\) at the latest, since \(A[j_k] \ge x\), so \(i_k < i_{k+1} \le j_k \le r\). Every element that the scans passed over satisfies

\begin{equation*} A[j_{k+1}+1\,..\,j_k - 1] > x, \qquad A[i_k + 1\,..\,i_{k+1} - 1] < x . \end{equation*}

If \(i_{k+1} < j_{k+1}\), the swap leaves \(A[i_{k+1}] \le x\) and \(A[j_{k+1}] \ge x\), so the invariant holds again. Therefore all accesses lie in \(A[p\,..\,r]\). Since \(j\) strictly decreases and never drops below \(p\), the loop stops after finitely many iterations.

(c) By (b) the returned value satisfies \(j \ge p\). If the return happens in iteration 1, then \(j_1 \le i_1 = p < r\). If it happens in iteration \(k+1 \ge 2\), then \(j_{k+1} < j_k \le r\). Either way \(p \le j < r\).

(d) Let the return happen in iteration \(m\), with \(i_m \ge j_m\). Combining the invariant from iteration \(m-1\) (empty for \(m=1\)) with the scanned regions of iteration \(m\) gives

\begin{equation*} A[p\,..\,i_m - 1] \le x, \qquad A[j_m] \le x, \qquad A[j_m + 1\,..\,r] \ge x . \end{equation*}

Since \(j_m - 1 \le i_m - 1\), every element of \(A[p\,..\,j_m]\) is \(\le x \le\) every element of \(A[j_m + 1\,..\,r]\).

(e) The recursion keeps the pivot inside the left part:

QUICKSORT(A, p, r)
1  if p < r
2      q = HOARE-PARTITION(A, p, r)
3      QUICKSORT(A, p, q)
4      QUICKSORT(A, q + 1, r)

By (c) both subarrays have size at most \(r - p\), so the recursion terminates. By induction on \(r - p\) both recursive calls sort their parts, and by (d) the concatenation of \(A[p\,..\,q]\) and \(A[q+1\,..\,r]\) is then sorted.

(a) \(\Theta(n^2)\), whatever the random choices are. If all values equal \(x\), the test \(A[j] \le x\) in PARTITION always succeeds, so the call returns \(q = r\). Every split therefore has sizes \(n-1\) and \(0\), which is the worst case of Section 7.2:

\begin{equation*} T(n) = T(n-1) + T(0) + \Theta(n) = \Theta(n^2). \end{equation*}

(b) Keep three regions behind the scan index \(j\) (values \(< x\), then \(= x\), then \(> x\)), followed by the unscanned part.

PARTITION'(A, p, r)
 1  x = A[r]
 2  q = p
 3  t = p - 1
 4  for j = p to r - 1
 5      if A[j] < x
 6          y = A[j]
 7          A[j] = A[t + 1]
 8          A[t + 1] = A[q]
 9          A[q] = y
10          q = q + 1
11          t = t + 1
12      elseif A[j] == x
13          exchange A[t + 1] with A[j]
14          t = t + 1
15  exchange A[t + 1] with A[r]
16  return q, t + 1

Loop invariant: at the start of each iteration of the for loop of lines 4-14,

\begin{equation*} A[p\,..\,q-1] < x, \qquad A[q\,..\,t] = x, \qquad A[t+1\,..\,j-1] > x, \qquad A[r] = x . \end{equation*}

Initialization: when \(j = p\), the first three regions are empty and \(A[r] = x\) by line 1.

Maintenance: there are three cases. (i) \(A[j] > x\): only \(j\) advances, so the \(>\) region grows by one element. (ii) \(A[j] = x\): the swap in line 13 puts \(x\) at \(t+1\) and moves the first element of the \(>\) region (or \(A[j]\) itself, if that region is empty) to \(j\); then \(t\) advances. (iii) \(A[j] < x\): lines 6-9 rotate three entries. The value \(y < x\) goes to position \(q\). The old \(A[q]\) goes to \(t+1\); it is an \(x\) when the \(=\) region is nonempty, and otherwise \(q = t+1\) and this step does nothing. The old \(A[t+1]\) goes to \(j\); it is a \(>\) element when the \(>\) region is nonempty, and otherwise \(t+1 = j\). Then \(q\) and \(t\) both advance. In all three cases the four regions keep their properties with \(j+1\) in place of \(j\).

Termination: at \(j = r\), line 15 puts the pivot at \(t+1\) and moves a \(>\) element (or the pivot itself) to \(r\). So with \(t+1\) returned as the new \(t\), we get \(A[p\,..\,q-1] < x\), \(A[q\,..\,t] = x\) and \(A[t+1\,..\,r] > x\). Also \(p \le q \le t \le r\), since \(q \le t_{\text{old}} + 1\).

The loop runs \(r - p\) times and each iteration does \(O(1)\) work, so the running time is \(\Theta(r-p)\).

(c) The new procedures are:

RANDOMIZED-PARTITION'(A, p, r)
1  i = RANDOM(p, r)
2  exchange A[r] with A[i]
3  return PARTITION'(A, p, r)

QUICKSORT'(A, p, r)
1  if p < r
2      (q, t) = RANDOMIZED-PARTITION'(A, p, r)
3      QUICKSORT'(A, p, q - 1)
4      QUICKSORT'(A, t + 1, r)

The block \(A[q\,..\,t]\) is already in its final sorted position, so QUICKSORT’ never recurses on it.

(d) Replace “the first pivot chosen from \(Z_{ij}\)” by “the first pivot whose value lies in \([z_i, z_j]\)”. After that, the bound \(\Pr\{z_i \text{ is compared to } z_j\} \le 2/(j-i+1)\) still holds, and the rest of Section 7.4.2 goes through unchanged.

Let \(z_1 \le z_2 \le \cdots \le z_n\) be the elements in sorted order, with ties broken arbitrarily but fixed. Let \(S_{ij}\) be the set of elements whose values lie in \([z_i, z_j]\); then \(S_{ij} \supseteq Z_{ij} = \{z_i, \dots, z_j\}\).

The analogue of Lemma 7.1 still holds. Each call of PARTITION’ makes at most two key comparisons per element against the pivot. Each call removes its pivot block \(A[q\,..\,t]\) for good, so there are at most \(n\) calls. Hence the running time is \(O(n + X)\), where \(X\) counts pairs (element, pivot) that are compared. As before, a pair is compared only when one of the two is the current pivot, and at most once, because the pivot is then frozen.

Now fix \(i < j\). (i) A pivot whose value is outside \([z_i, z_j]\) sends all of \(S_{ij}\) to the same side. So until the first pivot with value in \([z_i, z_j]\) is chosen, all of \(S_{ij}\) stays in one subarray, and that pivot is uniform over \(S_{ij}\). (ii) If that pivot is \(z_i\) or \(z_j\), the two may be compared. (iii) If it is any other element \(y\), then neither \(z_i\) nor \(z_j\) is the pivot, so they are not compared now. If \(z_i < y < z_j\), they are separated for good. If \(y\) equals \(z_i\) or \(z_j\) in value, the element equal to \(y\) joins the frozen block and is never compared again.

Therefore

\begin{equation*} \begin{aligned} \Pr\{z_i \text{ is compared to } z_j\} &\le \frac{2}{|S_{ij}|} \le \frac{2}{j-i+1}, \\ E[X] &\le \sum_{i=1}^{n-1} \sum_{j=i+1}^{n} \frac{2}{j-i+1} = O(n \lg n), \end{aligned} \end{equation*}

which replaces equation (7.3) and then gives equation (7.4) exactly as before. So QUICKSORT’ runs in \(O(n \lg n)\) expected time on any input, with or without repeated values.

(a) \(\mathrm{E}[X_i] = 1/n\). RANDOMIZED-PARTITION takes the pivot to be \(A[i]\) with \(i = \text{RANDOM}(p,r)\), uniform over the \(n = r-p+1\) positions, so each element (in particular the \(i\)th smallest) is the pivot with probability \(1/n\), and by Lemma 5.1 \(\mathrm{E}[X_i] = \Pr\{X_i = 1\} = 1/n\).

(b) Exactly one \(X_q\) equals \(1\). If the pivot is the \(q\)th smallest element, PARTITION costs \(\Theta(n)\) and leaves subarrays of sizes \(q-1\) and \(n-q\) to be sorted recursively, so \(T(n) = T(q-1)+T(n-q)+\Theta(n)\) on that event. Hence the identity \(T(n) = \sum_{q=1}^n X_q\bigl(T(q-1)+T(n-q)+\Theta(n)\bigr)\) holds for every outcome, and taking expectations gives (7.5).

(c) The pivot choice at the top level is independent of the random choices made inside the recursive calls, and for distinct elements the distribution of a call’s running time depends only on its size. So \(X_q\) is independent of \(T(q-1)\) and \(T(n-q)\), and \(\sum_q X_q = 1\). By linearity, part (a), and the fact that \(\sum_q T(q-1)\) and \(\sum_q T(n-q)\) run over the same sizes \(0,\ldots,n-1\),

\begin{equation*} \begin{aligned} \mathrm{E}[T(n)] &= \sum_{q=1}^{n}\mathrm{E}[X_q]\bigl(\mathrm{E}[T(q-1)]+\mathrm{E}[T(n-q)]\bigr) + \Theta(n)\\ &= \frac{2}{n}\sum_{q=0}^{n-1}\mathrm{E}[T(q)] + \Theta(n) = \frac{2}{n}\sum_{q=2}^{n-1}\mathrm{E}[T(q)] + \Theta(n), \end{aligned} \end{equation*}

where the last step absorbs \(\frac2n(\mathrm{E}[T(0)]+\mathrm{E}[T(1)]) = O(1/n)\) into \(\Theta(n)\).

(d) Let \(n \ge 2\) and \(m = \lceil n/2\rceil - 1\). For \(k \le m\) we have \(k < n/2\), so \(\lg k < \lg n - 1\); for \(k \ge m+1\), \(\lg k < \lg n\). Since \(m \ge n/2 - 1 \ge 0\), \(\sum_{k=2}^{m} k \ge \frac{m(m+1)}2 - 1 \ge \frac12(\frac n2 - 1)\frac n2 - 1\). Therefore

\begin{equation*} \begin{aligned} \sum_{k=2}^{n-1} k\lg k &\le (\lg n - 1)\sum_{k=2}^{m} k + \lg n\sum_{k=m+1}^{n-1} k = \lg n\sum_{k=2}^{n-1} k - \sum_{k=2}^{m} k\\ &\le \lg n\Bigl(\frac{n(n-1)}{2} - 1\Bigr) - \frac{n^2}{8} + \frac n4 + 1\\ &= \frac12 n^2\lg n - \frac18 n^2 - \Bigl(\frac n2\lg n + \lg n - \frac n4 - 1\Bigr) \le \frac12 n^2\lg n - \frac18 n^2, \end{aligned} \end{equation*}

because the bracket is \(2 - \frac32 > 0\) at \(n = 2\) and is increasing in \(n\) (its derivative is \(\frac12\lg n + \frac{1}{2\ln 2} + \frac{1}{n\ln 2} - \frac14 > 0\)).

(e) Write the \(\Theta(n)\) term of (7.6) as \(f(n)\) with \(c^{\prime}n \le f(n) \le cn\) for all \(n \ge 2\) (finitely many small \(n\) only adjust the constants).

Upper bound. Claim \(\mathrm{E}[T(n)] \le an\lg n\) for all \(n \ge 2\), with \(a = 4c\). Base \(n = 2\): the sum in (7.6) is empty, so \(\mathrm{E}[T(2)] \le 2c \le 2a = a\cdot 2\lg 2\). For \(n \ge 3\), assume the claim for \(2 \le q < n\); then by (7.7)

\begin{equation*} \mathrm{E}[T(n)] \le \frac{2a}{n}\sum_{q=2}^{n-1} q\lg q + cn \le \frac{2a}{n}\Bigl(\frac12 n^2\lg n - \frac18 n^2\Bigr) + cn = an\lg n - \Bigl(\frac a4 - c\Bigr)n \le an\lg n . \end{equation*}

Lower bound. Claim \(\mathrm{E}[T(n)] \ge bn\lg n\) for \(n \ge 2\), with \(b = c^{\prime}/3\). Base: \(\mathrm{E}[T(2)] \ge 2c^{\prime} \ge 2b\). For the step, \(x\lg x\) is increasing on \([1,\infty)\), so the sum dominates the integral, and \(\int_1^n x\lg x\,dx = \frac{n^2}{2}\lg n - \frac{n^2-1}{4\ln 2}\):

\begin{equation*} \begin{aligned} \sum_{q=2}^{n-1} q\lg q &\ge \int_1^{n-1} x\lg x\,dx \ge \int_1^{n} x\lg x\,dx - n\lg n\\ &\ge \frac{n^2}{2}\lg n - \frac{n^2}{4\ln 2} - n\lg n,\\ \mathrm{E}[T(n)] &\ge \frac{2b}{n}\sum_{q=2}^{n-1} q\lg q + c^{\prime}n \ge bn\lg n + \Bigl(c^{\prime} - \frac{b}{2\ln 2} - 2b\Bigr)n \ge bn\lg n, \end{aligned} \end{equation*}

using \(\lg n \le n\) and \(\frac{1}{2\ln 2} + 2 < 3\).

So \(\mathrm{E}[T(n)] = \Theta(n\lg n)\).

(a) By strong induction on \(n = r - p + 1\), TAIL-RECURSIVE-QUICKSORT\((A,p,r)\) sorts \(A[p_0..r]\) (writing \(p_0\) for the initial \(p\)) and touches nothing outside it; \(n \le 1\) is trivial. For the while loop use the invariant: \(A[p_0..p-1]\) is sorted and each of its elements is \(\le\) each element of \(A[p..r]\).

• Initialization: \(A[p_0..p_0-1]\) is empty.
• Maintenance: PARTITION permutes only \(A[p..r]\) and returns \(q\) with \(A[p..q-1] \le A[q] \le A[q+1..r]\) elementwise; the call in line 4 is on fewer than \(n\) elements, so by induction it sorts \(A[p..q-1]\) in place. Hence \(A[p_0..q]\) is sorted and is \(\le\) every element of \(A[q+1..r]\), which is the invariant for the new \(p = q+1\).
• Termination: \(p\) grows by at least \(1\) per iteration (\(q \ge p\)), so the loop stops with \(p \ge r\); then \(A[p..r]\) has at most one element and the invariant says \(A[p_0..r]\) is sorted.

(b) Run it on an already sorted array of distinct elements. Then PARTITION always returns \(q = r\), so the call on \(A[1..n]\) calls TAIL-RECURSIVE-QUICKSORT\((A,1,n-1)\) before it can finish, that call calls \((A,1,n-2)\), and so on down to \((A,1,1)\). That puts \(n\) records on the stack at once, so the depth is \(\Theta(n)\) (it is \(O(n)\) since each nested call is on a strictly shorter subarray).

(c) Recurse on the smaller side of the partition and loop on the larger:

MODIFIED-TAIL-RECURSIVE-QUICKSORT(A, p, r)
1  while p < r
2      q = RANDOMIZED-PARTITION(A, p, r)
3      if q - p < r - q
4          MODIFIED-TAIL-RECURSIVE-QUICKSORT(A, p, q - 1)
5          p = q + 1
6      else MODIFIED-TAIL-RECURSIVE-QUICKSORT(A, q + 1, r)
7          r = q - 1

Correctness: use the invariant of part (a) together with its mirror image (\(A[r+1..r_0]\) is sorted and each of its elements is \(\ge\) each element of \(A[p..r]\)). Lines 6-7 maintain the mirror half just as lines 4-5 maintain the original half.

Stack depth: inside a call whose current subarray has \(m = r-p+1\) elements, the two sides have sizes summing to \(m - 1\), so the recursive call gets at most \((m-1)/2 < m/2\) elements. Also \(m\) never exceeds the size of that call’s own subarray. So along any chain of nested calls with sizes \(s_0 = n, s_1, \ldots, s_d\) we have \(s_j < n/2^j\). A call recurses only if its size is at least \(2\), so \(2 \le s_{d-1} < n/2^{d-1}\), which gives \(d < \lg n\) and depth \(d + 1 \le \lg n + 1 = O(\lg n)\). The bound is attained up to a constant: for \(n = 2^k - 1\), if every pivot is the median, the sizes go \(2^k - 1, 2^{k-1} - 1, \ldots, 1\), which is \(k = \Theta(\lg n)\) nested calls. So the worst-case depth is \(\Theta(\lg n)\).

Running time: each partition of \(A[p..r]\) hands one side to a recursive call and the other side to the next loop iteration, so the partitions performed are exactly those of RANDOMIZED-QUICKSORT with the same random choices, plus \(O(1)\) extra work per partition. By Section 7.4.2 (or Problem 7-3) the expected running time stays \(O(n\lg n)\).

(a) \(p_i = \dfrac{6(i-1)(n-i)}{n(n-1)(n-2)}\): the 3-element sample is a uniformly random 3-subset of the \(\binom{n}{3}\) possible ones, and its median is \(A^{\prime}[i]\) exactly when it consists of \(A^{\prime}[i]\), one of the \(i-1\) smaller elements and one of the \(n-i\) larger ones, so

\begin{equation*} p_i = \frac{(i-1)(n-i)}{\binom{n}{3}} = \frac{6(i-1)(n-i)}{n(n-1)(n-2)} . \end{equation*}

(b) The limiting ratio is \(3/2\). The ordinary method picks the median with probability \(1/n\). With \(m = \lfloor (n+1)/2 \rfloor\), part (a) gives

\begin{equation*} \begin{aligned} n \text{ odd}: &\quad p_m = \frac{3(n-1)}{2n(n-2)}, \qquad p_m - \frac1n = \frac{n+1}{2n(n-2)},\\ n \text{ even}: &\quad p_m = \frac{3}{2(n-1)}, \qquad\quad\; p_m - \frac1n = \frac{n+2}{2n(n-1)}, \end{aligned} \end{equation*}

so the increase is about \(1/(2n)\), and in both cases \(p_m/(1/n) = n p_m \to 3/2\).

(c) The probability of a good split rises from about \(1/3\) to about \(13/27\), an increase of \(4/27\) (a factor \(13/9\)). Indeed \(p_i = \frac{6}{n}\cdot\frac{i}{n}\left(1-\frac{i}{n}\right) + O(1/n^2)\) uniformly in \(i\), and the sum over the \(n/3 + O(1)\) indices \(n/3 \le i \le 2n/3\) is a Riemann sum, so

\begin{equation*} \sum_{n/3 \le i \le 2n/3} p_i \;\to\; \int_{1/3}^{2/3} 6t(1-t)\,dt = \Bigl[3t^2 - 2t^3\Bigr]_{1/3}^{2/3} = \frac{13}{27}, \end{equation*}

whereas the ordinary method gives \(\#\{i : n/3 \le i \le 2n/3\}/n \to 1/3 = 9/27\).

(d) Every execution of quicksort, whatever pivots it picks, performs \(\Omega(n\lg n)\) comparisons, and median-of-3 still runs in \(O(n\lg n)\) expected time; so it can only change the constant.

Lower bound. A call on \(m\) elements makes at least \(m-1\) comparisons in PARTITION, so the minimum \(L(n)\) of the total number of comparisons over all pivot choices satisfies \(L(0)=L(1)=0\) and \(L(n) = n-1+\min_{1\le q\le n}\bigl(L(q-1)+L(n-q)\bigr)\). We claim \(L(n) \ge (n+1)\lg(n+1) - 2n\), true for \(n \le 1\). Inductively, with \(u = q\), \(v = n-q+1\) (so \(u+v = n+1\)) and convexity of \(t \lg t\),

\begin{equation*} \begin{aligned} L(n) &\ge n-1 + \min_q \bigl(u\lg u + v\lg v\bigr) - 2(n-1)\\ &\ge n-1 + (n+1)\lg\tfrac{n+1}{2} - 2(n-1) = (n+1)\lg(n+1) - 2n . \end{aligned} \end{equation*}

Upper bound. Choosing the median of 3 costs \(O(1)\) extra per call, so \(T(n) = T(q-1) + T(n-q) + \Theta(n)\) with \(\Pr\{q = i\} = p_i\). Suppose \(E[T(k)] \le ak\lg k + b\) for \(k<n\), and let \(g(q) = a\bigl((q-1)\lg(q-1) + (n-q)\lg(n-q)\bigr)\). This \(g\) is convex and symmetric about \((n+1)/2\), hence a nondecreasing function of \(d(q) = |q - (n+1)/2|\). The ratio \(p_q/(1/n) = 6(q-1)(n-q)/((n-1)(n-2))\) is a decreasing function of \(d(q)\), so for every tail set \(S_t = \{q : d(q) \ge t\}\) we get \(\sum_{q\in S_t} p_q \le |S_t|/n\) (if some ratio on \(S_t\) exceeds 1, all ratios off \(S_t\) do, and the complement inequality gives the same conclusion). Summing over the level sets of \(d\),

\begin{equation*} \begin{aligned} E[T(n)] &\le \sum_q p_q\, g(q) + 2b + cn \le \frac1n\sum_{q=1}^{n} g(q) + 2b + cn\\ &= \frac{2a}{n}\sum_{k=2}^{n-1} k\lg k + 2b + cn , \end{aligned} \end{equation*}

and by (7.7) this is at most \(an\lg n - an/4 + 2b + cn \le an\lg n + b\) once \(a \ge 4c + 2b\), exactly as in Section 7.4.2. Hence \(E[T(n)] = \Theta(n\lg n)\) with median-of-3 as well, and only the constant can differ.

(a) Pick a random pivot interval and use its left endpoint \(x\) as a pivot point; partition three ways into intervals lying entirely left of \(x\) (\(b_k < x\)), intervals containing \(x\), and intervals lying entirely right of \(x\) (\(a_k > x\)); recurse on the outer two groups only. The middle group needs no further work, since \(c = x\) serves for all of them.

FUZZY-SORT(A, p, r)
 1  if p < r
 2      x = A[RANDOM(p, r)].a
 3      lt = p;  gt = r;  k = p
 4      while k <= gt
 5          if A[k].b < x
 6              exchange A[k] with A[lt]
 7              lt = lt + 1;  k = k + 1
 8          elseif A[k].a > x
 9              exchange A[k] with A[gt]
10              gt = gt - 1
11          else k = k + 1
12      FUZZY-SORT(A, p, lt - 1)
13      FUZZY-SORT(A, gt + 1, r)

Partition. The loop of lines 4-11 maintains the invariant: every interval in \(A[p..lt-1]\) has \(b < x\), every interval in \(A[lt..k-1]\) contains \(x\), and every interval in \(A[gt+1..r]\) has \(a > x\). Initially all three ranges are empty. Each case of lines 5-11 puts \(A[k]\) into the right range (in line 6 the interval swapped from position \(lt\) contains \(x\) and moves to \(k\), behind the advancing \(k\)), so the invariant is maintained. The loop stops at \(k = gt+1\), when the three ranges cover \(A[p..r]\). The pivot interval contains its own left endpoint \(x\), so the middle range is nonempty and both recursive calls are on fewer elements.

Correctness, by induction on \(r-p+1\). The recursive calls fuzzy-sort the outer ranges with points \(c\) in their intervals. On the left \(c \le b_k < x\), on the right \(c \ge a_k > x\), and \(c = x\) works for every middle interval, so the sequence of points is nondecreasing over the whole of \(A[p..r]\).

(b) A call on \(m\) intervals takes \(\Theta(m)\) time, since each loop iteration costs \(O(1)\) and decreases \(gt - k\) by one. Order the intervals by \(a_i\), breaking ties arbitrarily, and let the pivot have rank \(q\), which is uniform on \(\{1, \ldots, n\}\). Every left interval has \(a_k \le b_k < x\) and every right interval has \(a_k > x\), so

\begin{equation*} |\text{left}| \le q-1, \qquad |\text{right}| \le n-q . \end{equation*}

General upper bound. Suppose \(E[T(m)] \le am\lg m + b\) for every input of size \(m < n\). Since \(t\lg t\) is nondecreasing,

\begin{equation*} \begin{aligned} E[T(n)] &\le \frac1n\sum_{q=1}^{n} a\bigl((q-1)\lg(q-1) + (n-q)\lg(n-q)\bigr)\\ &\quad + 2b + cn\\ &= \frac{2a}{n}\sum_{k=2}^{n-1} k\lg k + 2b + cn , \end{aligned} \end{equation*}

and by (7.7) this is at most \(an\lg n - an/4 + 2b + cn \le an\lg n + b\) for \(a \ge 4c + 2b\), as in Section 7.4.2. So \(E[T(n)] = O(n\lg n)\).

Lower bound in general. For pairwise disjoint intervals (for example \([i, i]\), \(i = 1, \ldots, n\)) the middle group is only the pivot, and every call on \(m\) intervals costs at least \(m-1\). The argument of Problem 7-5(d) then applies to every run and gives at least \((n+1)\lg(n+1) - 2n\) steps, so the worst-case expected time is \(\Theta(n\lg n)\).

All intervals overlap. Suppose \(y \in [a_i, b_i]\) for all \(i\). Then \(x = a_{\text{pivot}} \le y \le b_k\) for every \(k\), so the left group is empty, and the right group again has the common point \(y\). Hence \(T(n) \le T(|\text{right}|) + cn\) with \(|\text{right}| \le n - q\). Suppose \(E[T(m)] \le c(2m+1)\) for all such inputs of size \(m < n\) (with \(c\) at least the cost of a call on at most one interval). Then

\begin{equation*} E[T(n)] \le \frac1n\sum_{q=1}^{n} c\bigl(2(n-q)+1\bigr) + cn = cn + cn \le c(2n+1) , \end{equation*}

and the first partition alone costs \(\Omega(n)\), so \(E[T(n)] = \Theta(n)\). The algorithm never checks for this case: the overlap simply empties the left group and stops the recursion on the middle group.

The smallest possible depth is \(n-1\).

Lower bound. Fix a reachable leaf, reached by an input \(\langle a_1,\dots,a_n\rangle\) of distinct elements, and let \(G\) be the graph on \(\{1,\dots,n\}\) with an edge \(\{i,j\}\) for each comparison \(i:j\) on the root-to-leaf path. Suppose \(G\) is disconnected, and let \(C\) be one connected component, so \(\emptyset \ne C \ne \{1,\dots,n\}\). For a constant \(M\) larger than \(\max_i a_i - \min_i a_i\) define

\begin{equation*} a_i^{\pm} = \begin{cases} a_i \pm M, & i \in C,\\ a_i, & i \notin C. \end{cases} \end{equation*}

Every comparison on the path joins two indices of the same component, so its outcome is the same for \(a\), \(a^{+}\) and \(a^{-}\); hence \(a^{+}\) and \(a^{-}\) both follow the path to the same leaf. But in \(a^{+}\) every element indexed by \(C\) exceeds every other element, while in \(a^{-}\) every one is smaller, so the two inputs have different sorted orders, and a single leaf cannot report both correctly. Therefore \(G\) is connected, and a connected graph on \(n\) vertices has at least \(n-1\) edges, so the path has at least \(n-1\) comparisons.

Achievability. Insertion sort (Section 2.1) on an already sorted input makes exactly one comparison per iteration of its outer loop, \(j=2,\dots,n\): the test \(A[i] > key\) fails at once. That is \(n-1\) comparisons, so its decision tree has a leaf at depth \(n-1\).

\(\lg(n!) = \sum_{k=1}^{n}\lg k = \Theta(n\lg n)\); in fact \(\tfrac14 n\lg n \le \lg(n!) \le n\lg n\) for all \(n \ge 4\).

Upper bound: bound each term by the largest one,

\begin{equation*} \sum_{k=1}^{n}\lg k \le \sum_{k=1}^{n}\lg n = n\lg n . \end{equation*}

Lower bound: split the sum (as in Section A.2), drop the terms with \(k < \lceil n/2\rceil\), and bound each remaining term by the smallest one. There are \(n-\lceil n/2\rceil+1 \ge n/2\) remaining terms, each at least \(\lg(n/2)\), so for \(n \ge 4\) (where \(\lg n - 1 \ge \tfrac12\lg n\))

\begin{equation*} \begin{aligned} \sum_{k=1}^{n}\lg k &\ge \sum_{k=\lceil n/2\rceil}^{n}\lg k \ge \frac n2 \lg\frac n2 \\ &= \frac n2(\lg n - 1) \ge \frac n4 \lg n . \end{aligned} \end{equation*}

Hence \(\lg(n!) = \Theta(n\lg n)\).

No comparison sort can do it for any of the three fractions, since \(\lg(n!/2)\), \(\lg(n!/n)\) and \(\lg(n!/2^n)\) are all \(\Theta(n\lg n)\).

Suppose a comparison sort makes at most \(cn\) comparisons on \(m\) of the \(n!\) input permutations. Distinct permutations need distinct sorted outputs, so these \(m\) inputs reach \(m\) distinct leaves of the decision tree, all at depth at most \(cn\). Cutting the tree off at depth \(cn\) leaves a binary tree of height at most \(cn\) in which all these leaves are still leaves. Such a tree has at most \(2^{cn}\) leaves (the bound in the proof of Theorem 8.1), so \(m \le 2^{cn}\), that is, \(\lg m \le cn\). By Exercise 8.1-2, \(\lg(n!) \ge \tfrac14 n\lg n\) for \(n\ge 4\), and \(\lg n \le n\), so

\begin{equation*} \begin{aligned} \lg\frac{n!}{2} &= \lg(n!) - 1, \\ \lg\frac{n!}{n} &= \lg(n!) - \lg n \ge \lg(n!) - n, \\ \lg\frac{n!}{2^n} &= \lg(n!) - n \ge \frac n4\lg n - n . \end{aligned} \end{equation*}

All three are therefore at least \(\tfrac14 n\lg n - n\), which exceeds \(cn\) as soon as \(\lg n > 4(c+1)\). So for every constant \(c\), once \(n\) is large enough, no comparison sort makes at most \(cn\) comparisons (and so runs in time at most \(cn\)) on a \(1/2\), a \(1/n\), or even a \(1/2^n\) fraction of the inputs.

Any decision tree for this problem has at least \((k!)^{n/k}\) reachable leaves, so its height is at least \((n/k)\lg(k!) \ge \tfrac14 n\lg k\).

Fix the block values: say block \(b\) (\(b = 1,\dots,n/k\)) holds the values \((b-1)k+1,\dots,bk\) in some order. Each block can be in any of \(k!\) orders independently, which gives \((k!)^{n/k}\) valid inputs. Any two of them differ inside some block, so they need different output permutations, and therefore they reach different leaves of the decision tree. A binary tree of height \(h\) has at most \(2^h\) leaves, so, as in the proof of Theorem 8.1,

\begin{equation*} \begin{aligned} 2^h &\ge (k!)^{n/k}, \\ h &\ge \frac nk \lg(k!) \ge \frac nk \cdot \frac k4 \lg k = \frac n4 \lg k . \end{aligned} \end{equation*}

Here \(\lg(k!) \ge \tfrac14 k\lg k\) holds for \(k \ge 4\) by Exercise 8.1-2, and also for \(k=1,2,3\) (the values are \(0 \ge 0\), \(1 \ge 0.5\), and \(\lg 6 \ge \tfrac34\lg 3\)). So the worst-case number of comparisons is \(\Omega(n\lg k)\).

The output is \(B = \langle 0, 0, 1, 1, 2, 2, 3, 3, 4, 6, 6 \rangle\). After line 5 (counts of each value) and after line 8 (running sums):

The loop of lines 10–12 then runs for \(j = 11, 10, \ldots, 1\); each row shows the array \(B[1..11]\) (a dot marks an unfilled slot) and \(C[0..6]\) after that iteration.

Equal keys are placed by the downward loop of lines 10–12 at strictly decreasing positions, so the later one in \(A\) lands later in \(B\). Fix a value \(v\) and indices \(i < j\) with \(A[i] = A[j] = v\). Let \(p_m\) denote the value of \(C[v]\) at the start of iteration \(m\) of that loop; iteration \(m\) writes \(A[m]\) to \(B[p_m]\) when \(A[m] = v\). Within the loop, \(C[v]\) changes only in line 12 of an iteration with \(A[m] = v\), where it drops by \(1\); hence \(p_m\) is nonincreasing as \(m\) runs downward, and it strictly drops across iteration \(j\). Because \(j > i\), iteration \(j\) is executed before iteration \(i\), so

\begin{equation*} \text{position of } A[i] = p_i \le p_j - 1 < p_j = \text{position of } A[j]. \end{equation*}

Thus whenever \(i < j\) and \(A[i] = A[j]\), element \(A[i]\) precedes \(A[j]\) in \(B\), which is stability.

It still sorts, but it is not stable: elements with equal keys come out in reverse input order. Write \(L_v = \#\{i : A[i] < v\}\) and \(E_v = \#\{i : A[i] = v\}\), so after line 8, \(C[v] = L_v + E_v\). Loop invariant for the modified loop: at the start of the iteration for \(j\),

\begin{equation*} C[v] = L_v + \#\{\, i \ge j : A[i] = v \,\} \quad \text{for every } v, \end{equation*}

and the elements \(A[1..j-1]\) with value \(v\) occupy exactly the slots \(B[C[v]+1 \,..\, L_v + E_v]\).

Initialization (\(j = 1\)): \(\#\{i \ge 1 : A[i] = v\} = E_v\), which matches line 8, and no slots are filled.

Maintenance: with \(v = A[j]\), line 11 writes \(A[j]\) into slot \(C[v] = L_v + \#\{i \ge j : A[i] = v\}\), which lies in \((L_v, L_v + E_v]\) and is the slot just below those already filled with \(v\)’s; line 12 decrements \(C[v]\), which equals \(L_v + \#\{i \ge j+1 : A[i] = v\}\). All other \(C[u]\) are untouched.

Termination (\(j = n + 1\)): \(C[v] = L_v\), so for every \(v\) the block \(B[L_v + 1 \,..\, L_v + E_v]\) holds exactly the \(E_v\) copies of \(v\). These blocks are disjoint, in increasing order of \(v\), and cover \(B[1..n]\), so \(B\) is the sorted permutation of \(A\).

Not stable: each copy of \(v\) is placed one slot below the previous copy of \(v\), so if \(i < j\) and \(A[i] = A[j]\) then \(A[i]\) lands after \(A[j]\) in \(B\). For instance, \(A = \langle 1_a, 1_b \rangle\) with \(k = 1\) gives \(C = [0, 2]\), then \(B[2] = 1_a\) and \(B[1] = 1_b\), i.e. \(B = \langle 1_b, 1_a \rangle\).

Build the prefix-count array \(C\) of COUNTING-SORT (lines 1–8), so that \(C[i]\) is the number of inputs \(\le i\); a query is then answered by \(C[b] - C[a-1]\), with the ends clipped to \([0, k]\).

PREPROCESS(A, k)
 1  let C[0..k] be a new array
 2  for i = 0 to k
 3      C[i] = 0
 4  for j = 1 to A.length
 5      C[A[j]] = C[A[j]] + 1
 6  for i = 1 to k
 7      C[i] = C[i] + C[i - 1]
 8  return C

RANGE-COUNT(C, k, a, b)
 1  a = max(a, 0)
 2  b = min(b, k)
 3  if a > b
 4      return 0
 5  if a == 0
 6      return C[b]
 7  return C[b] - C[a - 1]

Correctness: after line 7 of PREPROCESS, \(C[i] = \#\{j : A[j] \le i\}\) for \(0 \le i \le k\), exactly as after line 8 of COUNTING-SORT. Since every input lies in \([0, k]\), clipping the query to \([a^{\prime}, b^{\prime}] = [\max(a,0), \min(b,k)]\) does not change the answer; if \(a^{\prime} > b^{\prime}\) the range contains no input. Otherwise, for integer inputs,

\begin{equation*} \#\{j : a^{\prime} \le A[j] \le b^{\prime}\} = \#\{j : A[j] \le b^{\prime}\} - \#\{j : A[j] \le a^{\prime} - 1\} = C[b^{\prime}] - C[a^{\prime}-1], \end{equation*}

where the subtracted term is \(0\) when \(a^{\prime} = 0\).

Running time: PREPROCESS consists of two loops over \(k + 1\) entries and one loop over \(n\) elements, each with constant work per iteration, hence \(\Theta(n + k)\); RANGE-COUNT does a constant number of comparisons and array lookups, hence \(O(1)\).

Three stable passes, on the third, second and then first letter, give the columns below; the last column is the sorted list.

Insertion sort and merge sort (as written in the book) are stable; heapsort and quicksort are not. Write \(x_a, x_b\) for two copies of the key \(x\), with \(x_a\) first in the input.

(i) INSERTION-SORT: the inner loop shifts \(A[i]\) only while \(A[i] > key\) (strict), so a key never moves past an equal key that was before it.

(ii) MERGE-SORT: on a tie MERGE takes \(L[i]\) because its test is \(L[i] \le R[j]\), and every element of \(L\) precedes every element of \(R\) in the input; by induction on the recursion each merge preserves the input order of equal keys.

(iii) HEAPSORT on \(\langle 1_a, 1_b\rangle\): BUILD-MAX-HEAP changes nothing (MAX-HEAPIFY swaps only on a strict \(>\)), then the loop exchanges \(A[1]\) and \(A[2]\), giving \(\langle 1_b, 1_a\rangle\).

(iv) QUICKSORT on \(\langle 2_a, 2_b, 1\rangle\): PARTITION with pivot \(1\) finds no element \(\le 1\) and swaps \(A[1]\) with \(A[3]\), giving \(\langle 1, 2_b, 2_a\rangle\); the recursive call on \(\langle 2_b, 2_a\rangle\) (pivot \(2_a\), and \(2_b \le 2_a\)) leaves it unchanged, so the output is \(\langle 1, 2_b, 2_a\rangle\).

Scheme: replace each key \(A[i]\) by the pair \((A[i], i)\) and compare pairs lexicographically. All pairs are distinct, so the output order is forced, and among equal keys it is increasing in original index, i.e. stable. Each comparison now costs at most two key comparisons, so the running time grows only by a constant factor (same asymptotic time); the extra space is \(\Theta(n)\) for the \(n\) stored indices (each of \(\lceil \lg n\rceil\) bits).

Claim: after iteration \(i\) of the loop of RADIX-SORT, \(A\) is sorted by the number formed from digits \(1, \dots, i\) (the \(i\) lowest-order digits); with \(i = d\) this is the full key. Write \(a_{[i]}\) for the number formed by the \(i\) lowest digits of \(a\), so \(a_{[i]} < b_{[i]}\) iff \(a_i < b_i\), or \(a_i = b_i\) and \(a_{[i-1]} < b_{[i-1]}\).

Base, \(i = 0\): every array is sorted on the empty key (\(a_{[0]} = 0\) for all \(a\)).

Step: assume the claim after iteration \(i-1\), and let \(a\) end iteration \(i\) before \(b\). We show \(a_{[i]} \le b_{[i]}\).

(i) \(a_i \ne b_i\): the pass sorts on digit \(i\), so \(a_i < b_i\), hence \(a_{[i]} < b_{[i]}\).

(ii) \(a_i = b_i\): because the pass is stable, \(a\) was before \(b\) already after iteration \(i-1\), so by the hypothesis \(a_{[i-1]} \le b_{[i-1]}\), hence \(a_{[i]} \le b_{[i]}\).

So after iteration \(i\) the array is sorted on digits \(1, \dots, i\), and after iteration \(d\) it is sorted. Stability is used exactly in case (ii): without it, the pass on digit \(i\) could reverse two keys with equal digit \(i\) that the earlier passes had put in the correct order (e.g. \(\langle 12, 11\rangle\) sorted on digit 1 gives \(\langle 11, 12\rangle\), and an unstable pass on digit 2 may return \(\langle 12, 11\rangle\)).

Write each integer in base \(n\) (for \(n \ge 2\); \(n = 1\) is trivial) and radix sort with counting sort on the 3 base-\(n\) digits. Every \(x\) with \(0 \le x \le n^3 - 1\) has a unique representation

\begin{equation*} x = x_3 n^2 + x_2 n + x_1, \qquad 0 \le x_1, x_2, x_3 \le n - 1, \end{equation*}

with \(x_1 = x \bmod n\), \(x_2 = \lfloor x/n\rfloor \bmod n\), \(x_3 = \lfloor x/n^2\rfloor\), each computable in \(O(1)\) time, so all digits are extracted in \(O(n)\). Comparing such numbers is comparing \((x_3, x_2, x_1)\) lexicographically, so RADIX-SORT with \(d = 3\) applies; COUNTING-SORT is stable and runs in \(\Theta(n + k)\) on digits in \(0, \dots, k-1\) with \(k = n\). By Lemma 8.3 (with \(d = 3\), \(k = n\)) the sort is correct and takes

\begin{equation*} \Theta(d(n + k)) = \Theta(3(n + n)) = \Theta(n) \end{equation*}

time, i.e. \(O(n)\).

In the worst case it takes \((10^d - 1)/9\) passes, and \(9d + 1\) piles are held at once. The worst case is a deck large enough that every bin at every level holds at least two cards, so no bin can be skipped.

Passes: one pass on digit \(d\) (most significant) splits the deck into \(10\) bins; each bin is sorted on the next digit and splits into \(10\) more, and so on. Level \(j\) (\(j = 1, \dots, d\)) therefore needs \(10^{j-1}\) passes, giving

\begin{equation*} \sum_{j=1}^{d} 10^{j-1} = \frac{10^d - 1}{9}. \end{equation*}

Piles: the operator works depth first, sorting one bin fully before starting its siblings. While a pass at level \(j\) is being done, the \(9\) siblings of each pile on the current path at levels \(1, \dots, j-1\) are set aside unsorted. The maximum occurs during a pass on the last (least significant) digit, level \(d\). Then there are \(9\) waiting piles at each of the \(d - 1\) shallower levels plus the \(10\) bins that pass produces (finished cards form one combined output deck, not counted):

\begin{equation*} 9(d-1) + 10 = 9d + 1 . \end{equation*}

Here \(n = 10\), so line 6 sends \(A[i]\) to bucket \(B[\lfloor 10A[i] \rfloor]\); bucket \(i\) holds the values in \([i/10, (i+1)/10)\). After the loop of lines 5-6 (elements appended in input order) and the insertion sorts of lines 7-8, the buckets are:

Only \(B[7]\) is changed by its insertion sort. Line 9 concatenates \(B[0], \ldots, B[9]\), giving

\begin{equation*} \langle .13, .16, .20, .39, .42, .53, .64, .71, .79, .89 \rangle . \end{equation*}

The worst case is when every element lands in one bucket and insertion sort must sort all \(n\) of them; the fix is to sort each bucket in line 8 with an \(O(n \lg n)\) worst-case sort such as merge sort (or heapsort) instead of insertion sort.

Worst case of the original. All lines except line 8 take \(\Theta(n)\) time, and insertion sort on \(n_i\) elements takes \(O(n_i^2)\) time (Section 2.2), so

\begin{equation*} T(n) = \Theta(n) + \sum_{i=0}^{n-1} O(n_i^2) \le \Theta(n) + O\Bigl(\bigl(\textstyle\sum_i n_i\bigr)^2\Bigr) = O(n^2). \end{equation*}

The bound is attained: take \(A[i] = (n-i)/n^2\) for \(i = 1, \ldots, n\). Every value is less than \(1/n\), so all go into \(B[0]\), in strictly decreasing order; insertion sort on a reverse-sorted list of length \(n\) makes \(n(n-1)/2\) element shifts, i.e. \(\Theta(n^2)\) time.

The modified algorithm. Merge sort on \(n_i\) elements costs at most \(c(1 + n_i \lg n_i)\) for a constant \(c\). Since \(\sum_i n_i = n\) and \(n_i \lg n_i \le n_i \lg n\),

\begin{equation*} T(n) \le \Theta(n) + c \sum_{i=0}^{n-1} (1 + n_i \lg n) = \Theta(n) + cn + cn \lg n = O(n \lg n) \end{equation*}

in the worst case. For the average case, \(1 + n_i \lg n_i \le 1 + n_i^2\), so the analysis leading to equation (8.1) still applies with the same \(O(n_i^2)\) per-bucket bound, and equation (8.2) gives

\begin{equation*} \mathrm{E}[T(n)] \le \Theta(n) + c\sum_{i=0}^{n-1}\bigl(1 + \mathrm{E}[n_i^2]\bigr) = \Theta(n) + cn\Bigl(3 - \frac1n\Bigr) = \Theta(n). \end{equation*}

\(\mathrm{E}[X^2] = 3/2\) and \(\mathrm{E}^2[X] = 1\). Indeed \(\Pr\{X=0\} = \Pr\{X=2\} = 1/4\) and \(\Pr\{X=1\} = 1/2\), so

\begin{equation*} \begin{aligned} \mathrm{E}[X^2] &= 0^2 \cdot \tfrac14 + 1^2 \cdot \tfrac12 + 2^2 \cdot \tfrac14 = \tfrac32, \\ \mathrm{E}^2[X] &= \bigl(0 \cdot \tfrac14 + 1 \cdot \tfrac12 + 2 \cdot \tfrac14\bigr)^2 = 1^2 = 1 . \end{aligned} \end{equation*}

Run BUCKET-SORT on the keys \(u_i = d_i^2 = x_i^2 + y_i^2\), which are uniform on \((0,1]\). Bucket \(j\) is then the ring \(\sqrt{j/n} < d \le \sqrt{(j+1)/n}\) (up to endpoints), and all \(n\) rings have the same area \(\pi/n\). A point has \(d_i \le r\) exactly when it lies in the disk of radius \(r\), of area \(\pi r^2\) out of \(\pi\), so for \(0 \le t \le 1\)

\begin{equation*} \Pr\{u_i \le t\} = \Pr\{d_i \le \sqrt t\} = \frac{\pi t}{\pi} = t . \end{equation*}

DISK-SORT(p, n)
1  let B[0 .. n-1] be a new array of empty lists
2  for i = 1 to n
3      u = p[i].x * p[i].x + p[i].y * p[i].y
4      insert p[i] into list B[min(n-1, floor(n * u))]
5  for j = 0 to n-1
6      sort list B[j] by key x*x + y*y with insertion sort
7  concatenate the lists B[0], B[1], ..., B[n-1] in order

Correctness. Since \(d \mapsto d^2\) is strictly increasing on \([0,1]\), sorting by \(u_i\) is sorting by \(d_i\), and no square root is needed. If \(u_i \le u_k\) then \(\min(n-1, \lfloor n u_i \rfloor) \le \min(n-1, \lfloor n u_k \rfloor)\), so the argument following BUCKET-SORT in Section 8.4 applies verbatim: equal buckets are ordered by line 6, distinct buckets by line 7. (The \(\min\) only moves the key \(u = 1\) into the last bucket \(B[n-1]\), which covers \([(n-1)/n, 1]\).)

Running time. Lines 1-5 and 7 take \(\Theta(n)\). The \(u_i\) are independent and uniform on \((0,1]\), so each point falls in bucket \(j\) with probability \(\Pr\{j/n \le u_i < (j+1)/n\} = 1/n\) (the single point \(u_i = 1\) has probability \(0\)), independently of the other points. These are exactly the hypotheses under which equation (8.2) is proved, so \(\mathrm{E}[n_j^2] = 2 - 1/n\), and by (8.1)

\begin{equation*} \mathrm{E}[T(n)] = \Theta(n) + \sum_{j=0}^{n-1} O\bigl(2 - 1/n\bigr) = \Theta(n). \end{equation*}

Bucket the numbers by \(P(X_i)\), which is uniform on \([0,1]\): put \(X_i\) into bucket \(\min(n-1, \lfloor n P(X_i) \rfloor)\), insertion-sort each bucket by the values \(X_i\) themselves, and concatenate.

DIST-SORT(X, n)
1  let B[0 .. n-1] be a new array of empty lists
2  for i = 1 to n
3      insert X[i] into list B[min(n-1, floor(n * P(X[i])))]
4  for j = 0 to n-1
5      sort list B[j] with insertion sort
6  concatenate the lists B[0], B[1], ..., B[n-1] in order

Correctness. \(P\) is nondecreasing, so \(X_i \le X_k\) implies \(\min(n-1, \lfloor nP(X_i) \rfloor) \le \min(n-1, \lfloor nP(X_k) \rfloor)\); the argument following BUCKET-SORT in Section 8.4 then applies: equal buckets are ordered by line 5 (which compares the \(X\)’s, so ties in \(P\) do no harm), distinct buckets by line 6.

Uniformity of \(U_i = P(X_i)\). Fix \(u \in (0,1)\). The set \(\{x : P(x) = u\}\) is nonempty (intermediate value theorem, as \(P\) is continuous with limits \(0\) and \(1\)), closed, and bounded above (as \(P(x) \to 1\)), so it has a largest element \(x_u\). Monotonicity gives \(P(x) \le u\) for \(x \le x_u\) and \(P(x) > u\) for \(x > x_u\), hence

\begin{equation*} \Pr\{U_i \le u\} = \Pr\{X_i \le x_u\} = P(x_u) = u . \end{equation*}

So \(U_1, \ldots, U_n\) are independent and uniform on \([0,1]\), and \(X_i\) falls in bucket \(j\) with probability \(\Pr\{j/n \le U_i < (j+1)/n\} = 1/n\), independently of the other \(X_k\) (\(U_i = 1\) has probability \(0\)).

Running time. Lines 1-4 and 6 take \(\Theta(n)\), using \(O(1)\) per evaluation of \(P\). The bucket sizes \(n_j\) have exactly the distribution analysed in Section 8.4, so equation (8.2) gives \(\mathrm{E}[n_j^2] = 2 - 1/n\), and by (8.1)

\begin{equation*} \mathrm{E}[T(n)] = \Theta(n) + \sum_{j=0}^{n-1} O\bigl(2 - 1/n\bigr) = \Theta(n). \end{equation*}

(a) Each of the \(n!\) input orderings reaches its own leaf, and no other leaf is reachable. Since \(A\) is deterministic, a given input follows a single root-to-leaf path. A leaf outputs one fixed permutation, and on distinct keys only one input ordering is sorted by it, so two different orderings never reach the same leaf. Hence the \(n!\) orderings reach \(n!\) distinct leaves, each with probability \(1/n!\). Every other leaf is reached by no input and gets probability \(0\).

(b) Since \(k > 1\), the root is a comparison node and every leaf of \(T\) lies in \(LT\) or \(RT\). Its depth in \(T\) is one more than its depth in that subtree. Summing over all \(k\) leaves gives

\begin{equation*} \begin{aligned} D(T) &= \sum_{\ell \in LT} (\mathrm{depth}_{LT}(\ell) + 1) \\ &\quad + \sum_{\ell \in RT} (\mathrm{depth}_{RT}(\ell) + 1) \\ &= D(LT) + D(RT) + k . \end{aligned} \end{equation*}

(c) Put \(d(1) = 0\) (a lone leaf). \((\le)\): for any \(1 \le i \le k-1\), hang optimal trees with \(i\) and \(k-i\) leaves under a new comparison root. By (b) the result has \(D = d(i) + d(k-i) + k\), so \(d(k)\) is at most the minimum. \((\ge)\): take \(T\) with \(k\) leaves and \(D(T) = d(k)\). Both subtrees are nonempty, so \(1 \le i_0 \le k-1\), and by (b)

\begin{equation*} \begin{aligned} d(k) &= D(LT) + D(RT) + k \ge d(i_0) + d(k-i_0) + k \\ &\ge \min_{1 \le i \le k-1}\{d(i) + d(k-i) + k\}. \end{aligned} \end{equation*}

(d) Let \(f(x) = x \lg x + (k-x)\lg(k-x)\) on \([1, k-1]\). Then

\begin{equation*} f^{\prime}(x) = \lg x - \lg(k-x) = \lg\frac{x}{k-x}, \end{equation*}

which is negative for \(x < k/2\) and positive for \(x > k/2\). So \(f\) attains its minimum \(f(k/2) = k\lg(k/2) = k \lg k - k\) at \(x = k/2\). We now show \(d(k) \ge k \lg k\) for all \(k \ge 1\) by strong induction. The base case is \(d(1) = 0 = 1 \lg 1\). If \(d(m) \ge m \lg m\) for all \(m < k\), then by (c)

\begin{equation*} \begin{aligned} d(k) &\ge \min_{1 \le i \le k-1} \{ i \lg i + (k-i)\lg(k-i) \} + k \\ &\ge (k\lg k - k) + k = k \lg k . \end{aligned} \end{equation*}

Hence \(d(k) = \Omega(k \lg k)\).

(e) Repeatedly delete a probability-\(0\) leaf and splice its sibling’s subtree into the parent’s place. No remaining leaf gets deeper, and we end with a binary decision tree \(T^{\prime}\) whose leaves are exactly the \(n!\) reachable leaves of (a). By (d),

\begin{equation*} \begin{aligned} D(T_A) &\ge \sum_{\ell \text{ reachable}} \mathrm{depth}_{T_A}(\ell) \\ &\ge D(T^{\prime}) \ge d(n!) \ge n! \lg(n!) , \end{aligned} \end{equation*}

so \(D(T_A) = \Omega(n! \lg(n!))\). On the input that reaches leaf \(\ell\), \(A\) makes \(\mathrm{depth}(\ell)\) comparisons. By (a), the average number of comparisons is therefore

\begin{equation*} \frac{1}{n!} \sum_{\ell \text{ reachable}} \mathrm{depth}_{T_A}(\ell) \ge \lg(n!) = \Theta(n \lg n) \end{equation*}

by equation (3.19). So the average-case time is \(\Omega(n \lg n)\).

(f) Fix one randomization node of \(B\) and keep only its cheapest child. Repeating this until no randomization nodes remain gives \(A\). To justify each step, model \(B\)’s randomness by drawing, in advance and independently of the input \(\pi\), a uniform label \(X_u \in \{1, \dots, r_u\}\) for every randomization node \(u\). An execution visits each node at most once, and at \(u\) it follows child \(X_u\), so this reproduces \(B\) exactly. The number of comparisons is then a function \(N(\pi, X)\). Pick a randomization node \(v\) with \(r\) children. Let \(B_j\) be the algorithm whose tree has \(v\) replaced by its \(j\)th child’s subtree, i.e. \(B\) with \(X_v\) fixed to \(j\). Because \(X_v\) is independent of \(\pi\) and of the other labels,

\begin{equation*} \begin{aligned} \mathrm{E}[N_B] &= \frac{1}{r} \sum_{j=1}^{r} \mathrm{E}[N \mid X_v = j] \\ &= \frac{1}{r} \sum_{j=1}^{r} \mathrm{E}[N_{B_j}] \ge \min_j \mathrm{E}[N_{B_j}] . \end{aligned} \end{equation*}

Every root-to-leaf path of \(B_j\) is a path of \(B\), so \(B_j\) still sorts correctly, and it has fewer randomization nodes. Replace \(B\) by a cheapest \(B_j\) and repeat; the tree is finite, so this stops. The result is a tree with only comparison nodes, i.e. a deterministic comparison sort \(A\), with \(\mathrm{E}[N_A] \le \mathrm{E}[N_B]\), the expectation taken over uniformly random inputs (and over \(B\)’s choices).

(a) Count the \(0\)-keys, then copy the records into a fresh array in one left-to-right pass: \(0\)-records fill \(B[1..z]\) and \(1\)-records fill \(B[z+1..n]\). This is counting sort with \(k = 1\).

STABLE-BIT-SORT(A, n)
1  let B[1..n] be a new array
2  z = 0
3  for j = 1 to n
4      if A[j].key == 0
5          z = z + 1
6  p = 1
7  q = z + 1
8  for j = 1 to n
9      if A[j].key == 0
10         B[p] = A[j]
11         p = p + 1
12     else B[q] = A[j]
13         q = q + 1
14 return B

Loop invariant of lines 8-13: \(B[1..p-1]\) holds the \(0\)-records of \(A[1..j-1]\) in their original order, and \(B[z+1..q-1]\) holds the \(1\)-records of \(A[1..j-1]\) in their original order. Initialization: both ranges are empty. Maintenance: \(A[j]\) goes directly after the records of its own key that came earlier. Termination: at \(j = n+1\) the two ranges hold all \(z\) zeros and all \(n - z\) ones, in input order. So \(B\) is sorted and the sort is stable. There are two passes of \(O(1)\) work per element, so the time is \(\Theta(n)\).

(b) Run the partition loop of Section 7.1 with \(0\) in the role of “\(\le\) pivot”.

BIT-PARTITION(A, n)
1  i = 0
2  for j = 1 to n
3      if A[j].key == 0
4          i = i + 1
5          exchange A[i] with A[j]

Loop invariant: \(A[1..i]\) has only \(0\)-keys and \(A[i+1..j-1]\) has only \(1\)-keys. Initialization: both ranges are empty. Maintenance: a \(1\)-key just extends the second range. For a \(0\)-key, the swap moves the first \(1\)-record (if any) to position \(j\), and the \(0\)-record joins the first range. Termination: at \(j = n+1\) the zeros precede the ones. There is one pass with \(O(1)\) work and two extra variables, so the time is \(\Theta(n)\) and the extra space is \(O(1)\). The algorithm is not stable: on \(\langle 1_a, 1_b, 0 \rangle\) it outputs \(\langle 0, 1_b, 1_a \rangle\).

(c) INSERTION-SORT from Section 2.1 works. It is in place, needing only the variables \(i\), \(j\) and \(key\). It is stable: its inner loop shifts \(A[i]\) only while \(A[i].key > key.key\), so the record being inserted stops right after every earlier record with an equal key. Its correctness is the loop invariant of Section 2.1. The running time is \(O(n^2)\); this is \(\Theta(n^2)\) on \(n/2\) ones followed by \(n/2\) zeros, since each zero passes \(n/2\) ones.

(d) Yes: use the algorithm of (a). It is stable and sorts on one bit in \(\Theta(n)\) time. By Lemma 8.3, with \(d = b\) digits each taking \(k = 2\) values, RADIX-SORT then sorts in \(\Theta(b(n + 2)) = \Theta(bn)\) time. Lemma 8.3 needs a stable per-digit sort, so (b) does not work. On the keys \(\langle 10, 11, 01 \rangle\) (binary), the pass on the low bit leaves the array unchanged, and the pass on the high bit swaps the first and last records, giving the unsorted \(\langle 01, 11, 10 \rangle\). Algorithm (c) is stable, so it gives a correct radix sort, but each pass costs \(\Theta(n^2)\) in the worst case, for \(O(bn^2)\) in total.

(e) Compute the counting-sort boundaries, then cycle records into their blocks: block \(v\) is \(A[C[v-1]+1 \,..\, C[v]]\), and \(P[v]\) is the first slot of block \(v\) not yet filled correctly.

IN-PLACE-COUNTING-SORT(A, n, k)
1  let C[0..k] and P[1..k] be new arrays
2  for v = 0 to k
3      C[v] = 0
4  for j = 1 to n
5      C[A[j].key] = C[A[j].key] + 1
6  for v = 1 to k
7      C[v] = C[v] + C[v-1]
8      P[v] = C[v-1] + 1
9  for v = 1 to k
10     while P[v] <= C[v]
11         w = A[P[v]].key
12         if w == v
13             P[v] = P[v] + 1
14         else exchange A[P[v]] with A[P[w]]
15             P[w] = P[w] + 1

After line 8, \(C[v]\) is the number of keys \(\le v\), so block \(v\) has exactly as many slots as there are key-\(v\) records. Invariant of lines 9-15: for every \(u\), the slots \(C[u-1]+1, \dots, P[u]-1\) hold key-\(u\) records (“placed”), and every block \(u < v\) is full (\(P[u] = C[u] + 1\)). Consequently the number of unplaced key-\(u\) records equals the number of free slots, \(C[u] - P[u] + 1\).

Maintenance: let \(w = A[P[v]].key\) in line 11. All keys \(< v\) are already placed, so \(w \ge v\). If \(w = v\), line 13 extends block \(v\)’s placed prefix. If \(w > v\), that record is an unplaced key-\(w\) record, so \(P[w] \le C[w]\). The swap puts it in slot \(P[w]\), and line 15 extends block \(w\)’s placed prefix. Only unplaced slots are touched, so placed records never move.

Termination: the while loop for \(v\) stops when block \(v\) is full. After \(v = k\) every block is full, so \(A\) is sorted.

Running time: each iteration of lines 11-15 increments some \(P[\cdot]\). The values \(P[u]\) only increase, from \(C[u-1]+1\) to at most \(C[u]+1\), so there are at most \(\sum_u (C[u] - C[u-1]) = n\) iterations in total. Together with \(O(n + k)\) for lines 1-9, the time is \(O(n + k)\). Extra storage is the two arrays \(C\) and \(P\), which is \(O(k)\), and records are moved only by in-place exchanges.

The algorithm is not stable: for keys \(\langle 2_a, 1_b, 1_c \rangle\), the first iteration swaps \(2_a\) with \(1_c\), and the output is \(\langle 1_c, 1_b, 2_a \rangle\).

(a) Group the integers by number of digits with one counting sort, radix-sort each group separately, and concatenate the groups in increasing digit count; negative integers are handled by the same procedure on their absolute values. Integers are written in decimal without leading zeros, so \(m \le n\) integers are present (each has at least one digit), and \(d(x)\), the digit count of \(x\), is found by scanning \(x\) in \(O(d(x))\) time.

SORT-NONNEG(A)                    // A holds m integers >= 0, n digits in all
1  for each x in A
2      compute d(x), the number of digits of x
3  COUNTING-SORT A on the key d(x), keys in 1..n,
       giving consecutive groups G[1], G[2], ..., G[n]
4  for d = 1 to n
5      if G[d] is nonempty
6          RADIX-SORT(G[d], d)    // base 10, d digits
7  return G[1], G[2], ..., G[n] concatenated

SORT-INTEGERS(A)
1  N = absolute values of the negative elements of A
2  P = the nonnegative elements of A
3  N = SORT-NONNEG(N);  P = SORT-NONNEG(P)
4  return (negations of N in reverse order), then P

Correctness: an integer \(x \ge 0\) with \(d\) digits satisfies \(10^{d-1} \le x < 10^{d}\) when \(d \ge 2\), and \(0 \le x < 10\) when \(d=1\), so every element of \(G_d\) is smaller than every element of \(G_{d^{\prime}}\) for \(d < d^{\prime}\); within a group all keys have exactly \(d\) digits, and RADIX-SORT sorts it correctly by Lemma 8.3. Hence line 7 returns \(A\) sorted. Negation reverses order, which gives SORT-INTEGERS.

Running time: lines 1-2 cost \(O(n)\), and line 3 costs \(\Theta(m+n) = O(n)\) (counting sort with \(k=n\)). If \(G_d\) has \(m_d\) elements, line 6 costs \(\Theta(d(m_d+10))\) by Lemma 8.3, and \(m_d \ge 1\) for every group that is sorted, so

\begin{equation*} \sum_{d:\, m_d \ge 1} d\,(m_d + 10) \le \sum_{d} 11\, d\, m_d = 11 n . \end{equation*}

The loop overhead of lines 4-5 is \(O(n)\), so the whole sort is \(O(n)\).

(b) Sort most-significant-character first: distribute the strings into one bucket per character according to their first character, put the strings that have no further characters ahead of all buckets, and recurse on each bucket with the next character position. The alphabet \(\Sigma\) has constant size \(k\) (for example \(k=26\)) and every string is nonempty, so there are \(m \le n\) strings.

STRING-SORT(S, j)       // all strings in S agree in positions 1..j
1  if |S| <= 1
2      return S
3  E = list of the strings in S of length exactly j
4  for each character c in the alphabet
5      B[c] = empty list
6  for each s in S of length > j, in order
7      append s to B[s[j+1]]
8  R = E
9  for each character c in the alphabet, in increasing order
10     R = R followed by STRING-SORT(B[c], j+1)
11 return R

The call is STRING-SORT(S, 0).

Correctness, by induction on the total length of the strings in \(S\): let \(s, t \in S\) (all agreeing in positions \(1..j\)) lie in different parts of \(R\). If \(s \in E\) and \(t \notin E\), then \(s\) is a proper prefix of \(t\), so \(s < t\) in alphabetical order, and \(E\) precedes the buckets. If \(s \in B[c]\) and \(t \in B[c^{\prime}]\) with \(c < c^{\prime}\), then \(s\) and \(t\) first differ at position \(j+1\), where \(s[j+1] = c < c^{\prime} = t[j+1]\), so \(s < t\), and \(B[c]\) precedes \(B[c^{\prime}]\). Strings of \(E\) are all equal, and within one bucket the recursive call sorts correctly by the inductive hypothesis (its strings agree in positions \(1..j+1\)). So \(R\) is sorted.

Running time: a call with \(|S| \ge 2\) costs \(O(|S| + k)\), including the \(k\) constant-time child calls on empty or singleton buckets. Every such call is labelled by the common prefix of length \(j\) of its strings, distinct calls have distinct labels, and each label is either the empty prefix or a prefix \(s[1..j]\), \(1 \le j \le |s|\), of some string; hence there are at most \(n+1\) such calls. A string \(s\) occurs in calls with \(j = 0, 1, \ldots, |s|\) only, so

\begin{equation*} \sum_{\text{calls}} |S| \le \sum_{s} (|s| + 1) \le 2n, \qquad \text{total time} = O\bigl(2n + k(n+1)\bigr) = O(n) \end{equation*}

for a fixed alphabet.

(a) Take the red jugs one at a time and compare each with the still-unmatched blue jugs until its partner is found; when only one unmatched blue jug is left to try, it is the partner without a comparison.

NAIVE-MATCH(R, B)          // R = red jugs r_1..r_n, B = blue jugs
1  U = B                   // unmatched blue jugs
2  for i = 1 to n
3      for each jug y in U, in order
4          if y is the last jug of U not yet tried against r_i
5              pair r_i with y; remove y from U; break
6          if COMPARE(r_i, y) reports equal
7              pair r_i with y; remove y from U; break

Correctness: by the loop invariant that, at the start of iteration \(i\), \(U\) consists exactly of the partners of \(r_i, \ldots, r_n\). (Initialization: \(U = B\). Maintenance: the partner of \(r_i\) is in \(U\), every tried jug that is not equal is not the partner, so the jug reached in line 4 or 6 is the partner; removing it restores the invariant for \(i+1\). Termination: all \(n\) reds are paired with their partners.)

Comparisons: in iteration \(i\), \(|U| = n-i+1\), and at most \(|U|-1 = n-i\) comparisons are made, so the total is at most \(\sum_{i=1}^{n}(n-i) = n(n-1)/2\); when each red’s partner is always the last jug of \(U\) in order, exactly this many are made. The worst case is therefore \(n(n-1)/2 = \Theta(n^2)\).

(b) Model any algorithm by a decision tree in the sense of Section 8.1, except that each internal node compares a red jug with a blue jug and has three children (less, equal, greater), and each leaf is labelled by a pairing. Number the red jugs \(r_1, \ldots, r_n\) and the blue jugs \(b_1, \ldots, b_n\); a pairing is a permutation \(\pi\) with \(r_i\) paired to \(b_{\pi(i)}\). Every one of the \(n!\) permutations occurs as the correct answer for some input (give \(r_i\) capacity \(i\) and \(b_{\pi(i)}\) capacity \(i\)), and inputs with different answers must reach different leaves, so the tree has at least \(n!\) leaves. A ternary tree of height \(h\) has at most \(3^h\) leaves, hence

\begin{equation*} 3^h \ge n! \;\Rightarrow\; h \ge \log_3 n! = \frac{\lg n!}{\lg 3} = \Omega(n \lg n) \end{equation*}

by equation (3.19). The worst-case number of comparisons is the height \(h\).

(c) Use randomized quicksort with a red pivot: choose a red jug \(r\) uniformly at random, compare it with every blue jug to split the blues into smaller ones, larger ones, and its partner \(b\); then compare \(b\) with every other red jug to split the reds the same way; recurse on the two sides.

MATCH-JUGS(R, B)       // R, B: red and blue jugs with equal capacity sets
1  if R is empty
2      return
3  r = a jug of R chosen uniformly at random
4  for each jug y in B
5      COMPARE(r, y): put y in B_less, in B_more, or set b = y
6  pair r with b
7  for each jug x in R other than r
8      COMPARE(x, b): put x in R_less or in R_more
9  MATCH-JUGS(R_less, B_less)
10 MATCH-JUGS(R_more, B_more)

Correctness, by induction on \(|R|\): \(R_{\text{less}}\) and \(B_{\text{less}}\) are the red and blue jugs smaller than \(r\), so they have the same set of capacities, and likewise for the larger sides; \(b\) is the partner of \(r\), and the recursive calls pair the rest correctly by the inductive hypothesis.

Expected comparisons: let the capacities be \(z_1 < z_2 < \cdots < z_n\), write \(Z_{ij} = \{z_i, \ldots, z_j\}\), and call a capacity a pivot when its red jug is chosen in line 3. Comparisons in a call are only between the pivot red jug and blue jugs, or between the pivot’s partner and red jugs, and both jugs of the pivot leave the recursion afterwards, so each red-blue pair is compared at most once. The matched pair of \(z_i\) is compared exactly once. For \(i<j\), the red jug of \(z_i\) and the blue jug of \(z_j\) (or the blue of \(z_i\) and the red of \(z_j\)) stay in a common subproblem until the first pivot from \(Z_{ij}\) is chosen; they are compared exactly when that pivot is \(z_i\) or \(z_j\), and are separated if it lies strictly between. Since the pivot is uniform over the capacities of its subproblem, which contains \(Z_{ij}\), each of the \(j-i+1\) elements of \(Z_{ij}\) is equally likely to be that first pivot, exactly as in Section 7.4.2. With \(X\) the number of comparisons,

\begin{equation*} \begin{aligned} E[X] &= n + 2\sum_{i=1}^{n-1}\sum_{j=i+1}^{n} \frac{2}{j-i+1} \le n + 4\sum_{i=1}^{n-1}\sum_{k=1}^{n} \frac{1}{k} \\ &= n + 4(n-1)H_n = O(n \lg n) \end{aligned} \end{equation*}

by equation (A.7).

Worst case: a call on \(k\) pairs makes \(k + (k-1) = 2k-1\) comparisons. Every red-blue pair is compared at most once, so there are never more than \(n^2\) comparisons; if every pivot is the smallest jug of its subproblem, the sizes are \(n, n-1, \ldots, 1\) and

\begin{equation*} \sum_{k=1}^{n} (2k-1) = n^2, \end{equation*}

so the worst case is exactly \(n^2 = \Theta(n^2)\).

(a) It means the array is sorted: with \(k = 1\) the condition reads \(A[i] \le A[i+1]\) for \(i = 1, \ldots, n-1\).

(b) Take \(1, 6, 2, 7, 3, 8, 4, 9, 5, 10\). The odd positions hold \(1,2,3,4,5\) and the even positions hold \(6,7,8,9,10\), so \(A[i] \le A[i+2]\) for every \(i\), and this is 2-sorted by part (c). It is not sorted, since \(A[2] = 6 > 2 = A[3]\).

(c) Multiply both sides by \(k > 0\) and cancel the common terms \(A[i+1], \ldots, A[i+k-1]\):

\begin{equation*} \begin{aligned} \frac{1}{k}\sum_{j=i}^{i+k-1} A[j] \le \frac{1}{k}\sum_{j=i+1}^{i+k} A[j] &\iff \sum_{j=i+1}^{i+k} A[j] - \sum_{j=i}^{i+k-1} A[j] \ge 0 \\ &\iff A[i+k] - A[i] \ge 0 . \end{aligned} \end{equation*}

So the \(i\)-th condition holds exactly when \(A[i] \le A[i+k]\), and the array is \(k\)-sorted iff this holds for every \(i = 1, \ldots, n-k\).

(d) Split the positions into the \(k\) residue classes \(r, r+k, r+2k, \ldots\) (\(r = 1, \ldots, k\)) and merge sort each class in place within its own positions (take \(1 \le k \le n/2\); for \(k \ge n\) every array is vacuously \(k\)-sorted, and for \(n/2 < k < n\) it suffices to swap \(A[i], A[i+k]\) when out of order for \(i \le n-k\), costing \(O(n-k) = O(n \lg(n/k))\) since \(\lg(n/k) = -\lg(1 - (n-k)/n) \ge (n-k)/n\)).

K-SORT(A, n, k)
1  for r = 1 to k
2      m = 0
3      for i = r to n by k          // copy class r into B
4          m = m + 1
5          B[m] = A[i]
6      MERGE-SORT(B, 1, m)
7      m = 0
8      for i = r to n by k          // write it back in order
9          m = m + 1
10         A[i] = B[m]

Correctness: afterwards each class \(A[r], A[r+k], A[r+2k], \ldots\) is in nondecreasing order. For any \(i \le n-k\), the positions \(i\) and \(i+k\) are consecutive members of the same class, so \(A[i] \le A[i+k]\), and \(A\) is \(k\)-sorted by part (c).

Running time: each class has \(m_r \le \lceil n/k \rceil \le 2n/k\) elements and \(\sum_r m_r = n\). The copy loops cost \(\Theta(n)\) in total, and merge sort costs at most \(c\, m_r \lg m_r\) for some constant \(c\), so the total is

\begin{equation*} \Theta(n) + \sum_{r=1}^{k} c\, m_r \lg \frac{2n}{k} = \Theta(n) + c\,n\Bigl(1 + \lg\frac{n}{k}\Bigr) = O\Bigl(n \lg \frac{n}{k}\Bigr), \end{equation*}

using \(\lg(n/k) \ge 1\).

(e) By part (c), each of the \(k\) lists \(L_r = \langle A[r], A[r+k], A[r+2k], \ldots \rangle\) (\(r = 1, \ldots, k\)) is sorted. Extracting them takes \(O(n)\) time. They hold \(n\) elements altogether, so the min-heap merge of Exercise 6.5-9 combines them into one sorted list in \(O(n \lg k)\) time. The total is \(O(n) + O(n \lg k) = O(n \lg k)\) (for \(k = 1\) the array is already sorted by part (a)).

(f) Let \(T(n)\) be the worst-case number of comparisons made by a comparison-based \(k\)-sorting algorithm. Running it and then the algorithm of part (e), which also uses only comparisons, gives a comparison sort. Its worst case is at most \(T(n) + c\,n \lg k\) comparisons, and by Theorem 8.1 it is at least \(c^{\prime} n \lg n\) for large \(n\). So

\begin{equation*} T(n) \ge c^{\prime} n \lg n - c\,(\lg k)\, n = \Omega(n \lg n), \end{equation*}

because \(\lg k\) is a constant when \(k\) is constant. The time is at least the number of comparisons, so \(k\)-sorting takes \(\Omega(n \lg n)\) time.

(a) There are \(\binom{2n}{n}\) ways (for distinct numbers): choose which \(n\) numbers go into the first list. Each list must then be in increasing order, so the choice fixes both lists.

(b) Fix the values \(1, \ldots, 2n\), and for each \(n\)-subset \(S\) let the input be \(S\) in sorted order as the first list and the complement in sorted order as the second. A leaf of the decision tree fixes the permutation that maps input positions to output ranks. That permutation determines \(S\) as the set of ranks given to the first list’s positions, so the \(\binom{2n}{n}\) inputs reach distinct leaves. A binary tree of height \(h\) has at most \(2^h\) leaves. The term \(\binom{2n}{n}\) is the largest of the \(2n+1\) terms of \(\sum_j \binom{2n}{j} = 4^n\), so

\begin{equation*} 2^h \ge \binom{2n}{n} \ge \frac{4^n}{2n+1} \quad\Longrightarrow\quad h \ge 2n - \lg(2n+1) = 2n - o(n). \end{equation*}

The worst-case number of comparisons is \(h\), so it is at least \(2n - o(n)\).

(c) Assume the elements are distinct. Suppose \(x\) in list \(A\) and \(y\) in list \(B\) are consecutive in sorted order, with \(x < y\), and that on this input the algorithm never compares \(x\) with \(y\). Build a second input by exchanging \(x\) and \(y\): put \(y\) in \(x\)’s slot in \(A\) and \(x\) in \(y\)’s slot in \(B\). Both lists stay sorted, because no element lies strictly between \(x\) and \(y\). Every other element \(z\) satisfies \(z < x \iff z < y\), so every comparison the algorithm makes has the same outcome on both inputs. The algorithm therefore follows the same path to the same leaf and applies the same rearrangement to both inputs. But \(x\) and \(y\) must end up in different relative orders of their input slots: on the first input \(A\)’s slot goes first, and on the second \(B\)’s slot does. So the algorithm is wrong on one of the two inputs, a contradiction.

(d) Take \(A = \langle 1, 3, 5, \ldots, 2n-1 \rangle\) and \(B = \langle 2, 4, \ldots, 2n \rangle\). For each \(i = 1, \ldots, 2n-1\), the elements \(i\) and \(i+1\) are consecutive in sorted order and lie in different lists, so by part (c) any correct merging algorithm compares them on this input. These \(2n-1\) pairs \(\{i, i+1\}\) are distinct, and one comparison tests only one pair, so the algorithm makes at least \(2n-1\) comparisons on this input. Hence its worst case is at least \(2n-1\) comparisons.

Throughout (a)-(b) the values are distinct (this is what makes “the location where \(A[p]\) belongs” well defined). Let \(C\) be the output of X on \(A\), and let \(k\) be the rank of \(A[p]\), so exactly \(k-1\) values are smaller than \(A[p]\) and \(A[p]\) belongs at position \(k\).

(a) By the minimality of \(A[p]\), every value smaller than \(A[p]\) is put in its correct place, so these \(k-1\) values fill positions \(1,\dots,k-1\) of \(C\). Position \(k\) holds \(A[q]\), which is not \(A[p]\) (that value is misplaced) and not smaller than \(A[p]\) (those values sit in positions \(1,\dots,k-1\)). Hence \(A[q]>A[p]\), so \(B[q]=1\), while \(B[p]=0\) because \(A[p]\le A[p]\).

(b) Let \(f(x)=0\) for \(x\le A[p]\) and \(f(x)=1\) for \(x>A[p]\), so that \(B=f(A)\) entrywise. Since \(f\) is nondecreasing,

\begin{equation*} f(\min(a,b))=\min(f(a),f(b)),\qquad f(\max(a,b))=\max(f(a),f(b)), \end{equation*}

so a compare-exchange on positions \((i,j)\) applied to \(f(\text{array})\) produces \(f\) of the result of the same compare-exchange applied to the array. X is oblivious, so it performs the same sequence of index pairs on \(B\) as on \(A\); by induction on the number of operations performed, the array X holds when run on \(B\) is always \(f\) of the array it holds when run on \(A\), and its output on \(B\) is \(f( C)\). Now \(C[k]=A[q]\) gives \(f(C[k])=1\), while \(A[p]\) ends at some position \(m\neq k\) with \(m\notin\{1,\dots,k-1\}\), so \(m>k\) and \(f(C[m])=f(A[p])=0\). A 1 precedes a 0 in \(f( C)\), so X fails to sort the 0-1 array \(B\). (With repeated values, pick \(i<j\) with \(C[i]>C[j]\) and use the threshold \(f(x)=[x>C[j]]\) instead: the same simulation gives \(f(C[i])=1>0=f(C[j])\).)

(c) The sorted order of a column is uniquely determined by its multiset of values, so each odd step may be replaced, without changing any output, by INSERTION-SORT applied to each column, which is a fixed sequence of compare-exchanges on that column’s cells (upper cell first). Each even step is a permutation of the cells that depends only on \(r\) and \(s\), so instead of moving data we can relabel: after steps 2, 4, 6 we keep track of which memory cell plays the role of each array cell, and this correspondence is fixed in advance. Thus the whole algorithm is a sequence of compare-exchanges on prespecified pairs of cells, independent of the data. (A relabelled compare-exchange may send its minimum to the larger memory index; parts (a) and (b) never used \(i<j\), only that each operation puts the min and the max into prespecified cells, so the 0-1 lemma still applies.)

The following counting fact drives (d), (g) and (h). Write a 0-1 sequence \(w_1,\dots,w_{rs}\) into the array in row-major order, so \(w_t\) lands in column \(c(t)\) with \(c(t)\equiv t\pmod s\), and let \(Z_c\) be the number of 0s in column \(c\). Each residue class mod \(s\) meets an interval of \(m\) consecutive integers in either \(\lfloor m/s\rfloor\) or \(\lfloor m/s\rfloor+1\) points, so if the 0s of \(w\) form at most \(k\) intervals of positions, then

\begin{equation*} \max_c Z_c-\min_c Z_c\le k. \qquad (\ast) \end{equation*}

(d) After step 1 every column is sorted, so in column-major order the array is \(0^{z_1}1^{r-z_1}\,0^{z_2}1^{r-z_2}\cdots 0^{z_s}1^{r-z_s}\), and its 0s form at most \(s\) intervals. Step 2 writes exactly this column-major sequence into the array in row-major order (column \(j\) fills rows \((j-1)r/s+1,\dots,jr/s\) in order), so by \((\ast)\) the column counts satisfy \(\max Z_c-\min Z_c\le s\). Step 3 sorts each column, after which column \(c\) holds 0s exactly in rows \(1,\dots,Z_c\). Hence rows \(1,\dots,\min Z_c\) are clean 0s, rows below \(\max Z_c\) are clean 1s, and at most \(\max Z_c-\min Z_c\le s\) rows lie in between.

(e) Step 2 maps the \(t\)-th cell in column-major order to the \(t\)-th cell in row-major order, so its inverse, step 4, maps the \(t\)-th cell in row-major order to the \(t\)-th cell in column-major order. Thus the column-major reading after step 4 equals the row-major reading after step 3, which by (d) is a clean area of 0s, then at most \(s\) dirty rows of \(s\) entries each, so at most \(s^2\) elements, then a clean area of 1s.

(f) Let \(x_1,\dots,x_n\) be the column-major reading after step 5. Step 6 keeps this reading (skipping empty cells), and the columns of the shifted array are the windows

\begin{equation*} \begin{aligned} W_0&=[1,\,r/2],\qquad W_s=[(s-1)r+r/2+1,\,n],\\ W_j&=[(j-1)r+r/2+1,\,jr+r/2]\quad(1\le j\le s-1), \end{aligned} \end{equation*}

i.e. the bottom half of column \(j\) followed by the top half of column \(j+1\). So steps 6-8 together sort each window \(W_j\) of the column-major sequence. By (e), after step 4 the reading is \(0\cdots0\,D\,1\cdots1\) with \(|D|\le s^2\le r/2\) (using \(r\ge 2s^2\)), so \(D\) meets at most two consecutive columns. (i) \(D\) lies inside one column \(j\). Step 5 sorts column \(j\), the other columns are clean, so the sequence is already sorted, and sorting windows of a sorted sequence changes nothing. (ii) \(D\) meets column \(j\) in \(\ell_1\ge1\) cells and column \(j+1\) in \(\ell_2\ge1\) cells, \(\ell_1+\ell_2\le r/2\). Column \(j\) has at most \(\ell_1\) ones, so after step 5 they lie in its last \(\ell_1\le r/2\) cells, its bottom half; column \(j+1\) has at most \(\ell_2\) zeros, so after step 5 they lie in its top half. Hence after step 5 every position before \(W_j\) holds 0, every position after \(W_j\) holds 1, and step 7 sorts \(W_j\), so the output is sorted. Either way the 0-1 input is sorted. By (c) columnsort is an oblivious compare-exchange algorithm, so by the 0-1 sorting lemma ((a)-(b)) it sorts every input.

(g) When \(s\nmid r\), step 2 still means writing the column-major sequence into the array in row-major order, and after step 1 that sequence is again \(0^{z_1}1^{r-z_1}\cdots 0^{z_s}1^{r-z_s}\), with 0s in at most \(s\) intervals. So the argument of (d) applies verbatim: by \((\ast)\) and step 3 there are at most \(s\le 2s-1\) dirty rows between the clean 0 rows and the clean 1 rows. Step 4 still maps row-major to column-major order, so the dirty area after step 4 has at most \((2s-1)s\) elements, and the proof of (f) goes through whenever \((2s-1)s\le r/2\) with \(r\) even. So the \(2s-1\) bound needs \(r\ge 2s(2s-1)=4s^2-2s\), while the sharper count \(s\) gives at most \(s^2\) dirty elements, so \(r\ge 2s^2\) (with \(r\) even) is already enough.

(h) Change step 1 so that odd-numbered columns are sorted in increasing order and even-numbered columns in decreasing order. After step 1 the column-major sequence is then

\begin{equation*} 0^{z_1}1^{r-z_1}\;1^{r-z_2}0^{z_2}\;0^{z_3}1^{r-z_3}\cdots, \end{equation*}

in which each column’s 0s are still one interval of positions, so its 0s form at most \(s\) intervals. Writing it row-major in step 2 and sorting columns in step 3 gives, by \((\ast)\), at most \(s\) dirty rows, exactly as in (d). Then (e) gives a dirty area of at most \(s^2\) elements after step 4, and (f) gives a sorted 0-1 output whenever \(r\) is even and \(r\ge 2s^2\); neither argument uses \(s\mid r\). Sorting a column in decreasing order is still a fixed sequence of compare-exchanges (lower cell first), so (c) still holds, and by the 0-1 sorting lemma the modified columnsort sorts every input.

Run a knockout tournament for the minimum in rounds, recording whom each element beats; the second smallest is then the smallest of the at most \(\lceil \lg n \rceil\) elements beaten directly by the minimum.

SECOND-SMALLEST(A, n)                  // n >= 2
 1  S = <1, 2, ..., n>                  // survivors (indices)
 2  for i = 1 to n
 3      beaten[i] = empty list
 4  while |S| > 1
 5      T = empty list
 6      for each consecutive pair (x, y) of S
 7          if A[x] < A[y]
 8              append y to beaten[x]; append x to T
 9          else append x to beaten[y]; append y to T
10      if |S| is odd
11          append the last element of S to T
12      S = T
13  m = the unique element of S
14  return MINIMUM of the values A[j], j in beaten[m]

Every match eliminates its loser, so the final survivor \(m\) is the minimum (take the elements distinct). Let \(s\) be the second smallest. Since \(s \ne m\) it was eliminated in some match, and the only element smaller than \(s\) is \(m\); hence \(s\) lost to \(m\), i.e. \(s \in \mathit{beaten}[m]\). Every element of \(\mathit{beaten}[m]\) is at least \(s\), so line 14 returns \(A[s]\).

Each match eliminates exactly one element and only \(m\) survives, so lines 4-12 make exactly \(n-1\) comparisons. If \(|S| = k\) before a round, then \(\lceil k/2 \rceil\) survive it, so by equation (3.4) after \(r\) rounds

\begin{equation*} |S| = \lceil \cdots \lceil \lceil n/2 \rceil /2 \rceil \cdots /2 \rceil = \lceil n/2^r \rceil , \end{equation*}

which equals \(1\) first at \(r = \lceil \lg n \rceil\). The minimum plays at most one match per round, so \(|\mathit{beaten}[m]| \le \lceil \lg n \rceil\), and MINIMUM on that list uses at most \(\lceil \lg n \rceil - 1\) comparisons. In total

\begin{equation*} (n-1) + (\lceil \lg n \rceil - 1) = n + \lceil \lg n \rceil - 2 . \end{equation*}

An adversary can answer so that only comparisons between two never-compared elements make two units of progress; there are at most \(\lfloor n/2 \rfloor\) such comparisons, which forces \(2n - 2 - \lfloor n/2 \rfloor = \lceil 3n/2 \rceil - 2\) comparisons.

Potential. At any moment let

\begin{equation*} \Phi = \#\{x : x \text{ has never lost}\} + \#\{x : x \text{ has never won}\}, \end{equation*}

so \(\Phi = 2n\) initially. When the algorithm stops, at most one element has never lost: if \(x \ne y\) both had never lost, raising either one’s value above all others is consistent with every answer given (it only won its comparisons), so either could be the maximum and the algorithm cannot know which. Symmetrically at most one element has never won, so finally \(\Phi \le 2\), and the comparisons must lower \(\Phi\) by at least \(2n-2\). A comparison in which \(w\) beats \(l\) lowers \(\Phi\) by

\begin{equation*} \delta = [\,w \text{ had never won}\,] + [\,l \text{ had never lost}\,] \le 2 . \end{equation*}

Adversary. It keeps an actual assignment of distinct values and answers by it, with one exception: if the value-winner \(w\) has never won, the value-loser \(l\) has never lost, and the two are not both uncompared, it first raises \(l\) above every current value (consistent, since \(l\) has only won so far) and declares \(l\) the winner. Then: (i) if \(w, l\) are both uncompared, \(\delta \le 2\); (ii) if the honest answer has \(\delta = 2\) and we are not in (i), then \(w\) has lost or \(l\) has won, so after the flip \(\delta = [\,l \text{ had never won}\,] + [\,w \text{ had never lost}\,] \le 1\); (iii) otherwise \(\delta \le 1\) directly. All answers remain consistent with the current values, and the values change only by raising never-lost elements, so the adversary’s final input realizes every answer.

Count. Let \(a\) be the number of comparisons of type (i) and \(C\) the total. Each such comparison uses two never-compared elements and makes both compared, so \(a \le \lfloor n/2 \rfloor\), and

\begin{equation*} \begin{aligned} 2n - 2 &\le 2a + (C - a) = C + a,\\ C &\ge 2n - 2 - \lfloor n/2 \rfloor = \lceil 3n/2 \rceil - 2, \end{aligned} \end{equation*}

the last equality because \(2n - \lfloor n/2 \rfloor = \lceil 3n/2 \rceil\) (check \(n\) even and odd).

Every call RANDOMIZED-SELECT\((A,p,r,i)\) satisfies \(1 \le i \le n\) with \(n = r-p+1 \ge 1\); this holds for the top-level call (we only ask for an order statistic that exists), and it is inherited by each recursive call, which is therefore on a nonempty subarray. Indeed, suppose a call with \(1 \le i \le n\) reaches line 3, so \(p<r\), and let \(k = q-p+1 \in \{1,\dots,n\}\).

(i) Line 8 runs only when \(i<k\). Then \(k \ge i+1 \ge 2\), so \(A[p \,..\, q-1]\) has length \(k-1 \ge 1\), and the new rank satisfies \(1 \le i \le k-1\).

(ii) Line 9 runs only when \(i>k\). Then \(k < i \le n\), so \(A[q+1 \,..\, r]\) has length \(n-k \ge 1\), and the new rank satisfies \(1 \le i-k \le n-k\).

By induction on the depth of recursion every call has \(1 \le i \le r-p+1\), so no call is ever made on a \(0\)-length array.

They are functions of disjoint, mutually independent sets of random choices, hence independent. Fix \(k\) and put \(m = \max(k-1,n-k)\), a constant. Let \(R_0\) be the random index drawn by RANDOMIZED-PARTITION in the top-level call on \(n\) elements, and let \(R_1, R_2, \dots\) be the draws made in all later calls. Then:

(i) \(X_k = \mathrm{I}\{R_0 \text{ selects the element of rank } k\}\) is a function of \(R_0\) alone.

(ii) \(T(m)\) is the running time of RANDOMIZED-SELECT on an \(m\)-element subarray of distinct elements; it is a random variable determined only by the pivot draws made inside that call and its descendants, i.e. by \(R_1, R_2, \dots\), and its distribution depends on \(m\) (and the target rank) but not on how the \(m\) elements happen to be arranged, because each draw picks a uniformly random element regardless of arrangement.

The random-number generator produces independent draws, so \(R_0\) is independent of \((R_1,R_2,\dots)\), and functions of independent random variables are independent (Section C.3). Hence \(X_k\) and \(T(m)\) are independent, which is exactly what equation (C.24) needs:

\begin{equation*} \mathrm{E}[X_k \cdot T(\max(k-1,n-k))] = \mathrm{E}[X_k]\cdot \mathrm{E}[T(\max(k-1,n-k))] . \end{equation*}

Both recursive calls are tail calls, so replace them by updating \(p\), \(r\), \(i\) and looping.

ITERATIVE-RANDOMIZED-SELECT(A, p, r, i)
1  while p < r
2      q = RANDOMIZED-PARTITION(A, p, r)
3      k = q - p + 1
4      if i == k
5          return A[q]
6      elseif i < k
7          r = q - 1
8      else p = q + 1
9          i = i - k
10 return A[p]

Let \(x\) be the \(i_0\)th smallest element of the original \(A[p_0 \,..\, r_0]\), with \(1 \le i_0 \le r_0-p_0+1\).

Loop invariant: at the start of each iteration of line 1, \(1 \le i \le r-p+1\) and \(x\) is the \(i\)th smallest element of \(A[p \,..\, r]\).

Initialization: true with \(p=p_0\), \(r=r_0\), \(i=i_0\).

Maintenance: after line 2, \(A[p \,..\, q-1] \le A[q] \le A[q+1 \,..\, r]\) elementwise and \(A[q]\) is the \(k\)th smallest of \(A[p \,..\, r]\). If \(i=k\), line 5 returns \(A[q]=x\). If \(i<k\), \(x\) lies in \(A[p \,..\, q-1]\) and is its \(i\)th smallest; if \(i>k\), the \(k\) elements of \(A[p \,..\, q]\) are all below \(x\), so \(x\) is the \((i-k)\)th smallest of \(A[q+1 \,..\, r]\). The bounds \(1 \le i \le r-p+1\) survive exactly as in cases (i) and (ii) of Exercise 9.2-1.

Termination: each iteration that does not return removes at least the pivot, so \(r-p\) strictly decreases and the loop ends. At exit \(p \ge r\) and \(1 \le i \le r-p+1\), so \(p=r\), \(i=1\), and \(A[p]=x\) is returned.

Iterations correspond one-to-one with the calls of RANDOMIZED-SELECT under the same random draws, so the running time is unchanged and the extra space is \(O(1)\).

Partition every time around the largest remaining element: the pivots are \(9,8,7,6,5,4,3,2,1\) in that order. With \(i=1\), a pivot of rank \(k\) in an \(m\)-element subarray ends the search only if \(k=1\), and otherwise leaves a subarray of \(k-1 \le m-1\) elements; choosing \(k=m\) each time shrinks the subarray by only one element, so we get the maximum possible number of partitions, \(9\), each as large as possible. Running RANDOMIZED-PARTITION (pivot swapped into \(A[r]\), then PARTITION) with these choices gives:

The tenth call has \(p=r\) and returns \(0\) at line 2. Partitioning \(m\) elements costs \(m-1\) comparisons, so the total is

\begin{equation*} \sum_{m=2}^{10} (m-1) = 45 = \binom{10}{2}, \end{equation*}

the \(\Theta(n^2)\) worst case. It is the maximum: if \(f(m)\) is the worst comparison count for the minimum of \(m\) elements, then \(f(1)=0\) and \(f(m) = (m-1) + \max_{2 \le k \le m} f(k-1) = (m-1)+f(m-1)\) since \(f\) is nondecreasing, giving \(f(10)=45\).

Groups of 7 still give linear time; groups of 3 give a \(\Theta(n\lg n)\) recurrence.

Groups of 7. Of the \(\lceil n/7\rceil\) group medians, at least half are \(\ge x\); discounting \(x\)’s own group and the possibly short last group, each of these groups contributes 4 elements greater than \(x\), so the number of elements greater than \(x\) is at least

\begin{equation*} 4\left(\left\lceil \tfrac12 \lceil n/7\rceil\right\rceil - 2\right) \ge \frac{2n}{7} - 8 , \end{equation*}

and symmetrically for elements less than \(x\). Step 5 therefore recurses on at most \(5n/7 + 8\) elements, and

\begin{equation*} T(n) \le T(\lceil n/7\rceil) + T(5n/7 + 8) + O(n) . \end{equation*}

Assume \(T(m)\le cm\) for \(m<n\) and let \(an\) bound the \(O(n)\) term, exactly as in the text’s analysis:

\begin{equation*} \begin{aligned} T(n) &\le c(n/7 + 1) + c(5n/7 + 8) + an \\ &= cn + (-cn/7 + 9c + an) , \end{aligned} \end{equation*}

which is at most \(cn\) iff \(c \ge 7a\, n/(n-63)\) for \(n > 63\). For \(n \ge 126\) we have \(n/(n-63)\le 2\), so \(c \ge 14a\) works (and \(c\) large enough covers the \(O(1)\) inputs with \(n<126\)). Hence \(T(n) = O(n)\).

Groups of 3. Now at least half of the \(\lceil n/3\rceil\) groups (less two) contribute 2 elements greater than \(x\), giving only

\begin{equation*} 2\left(\left\lceil \tfrac12 \lceil n/3\rceil\right\rceil - 2\right) \ge \frac{n}{3} - 4 \end{equation*}

elements on each side, and this is essentially attained: if in every group \(\{a<b<c\}\) whose median \(b\) is above \(x\) we also have \(a > x\), and in every group with \(b<x\) we have \(c>x\), then the low side consists only of the \(\approx n/3\) elements \(a,b\) of the low groups, so for \(i\) near \(n\) step 5 recurses on about \(2n/3\) elements. The analysis therefore yields only

\begin{equation*} T(n) = T(\lceil n/3\rceil) + T(2n/3 + O(1)) + \Theta(n) . \end{equation*}

The two subproblem sizes add up to \(n + O(1)\), so the substitution step yields \(cn + \Theta( c) + an > cn\) and can never close. This is the recurrence \(T(n)=T(n/3)+T(2n/3)+\Theta(n)\) of Figure 4.6 (Section 4.4): each of the first \(\log_3 n\) levels of its recursion tree is full and costs \(\Theta(n)\), so \(T(n) = \Theta(n\lg n)\), which is not linear.

It suffices that \(3n/10 - 6 \ge \lceil n/4\rceil\) for \(n\ge 140\), since the analysis in Section 9.3 shows that at least \(3n/10-6\) elements are greater than \(x\) and at least \(3n/10-6\) are less than \(x\). Using \(\lceil n/4\rceil \le n/4 + 3/4\),

\begin{equation*} \begin{aligned} \frac{3n}{10} - 6 - \left\lceil \frac n4 \right\rceil &\ge \frac{3n}{10} - 6 - \frac n4 - \frac34 \\ &= \frac{n}{20} - \frac{27}{4} \ge 7 - \frac{27}{4} > 0 \end{aligned} \end{equation*}

for \(n \ge 140\). Hence each side of \(x\) holds at least \(\lceil n/4\rceil\) elements.

Use SELECT to find the (lower) median in linear time and partition around it, so both sides have at most \(n/2\) elements.

BEST-QUICKSORT(A, p, r)
1  if p < r
2      i = floor((r - p + 2) / 2)      // lower median rank
3      x = SELECT(A, p, r, i)
4      q = PARTITION-AROUND(A, p, r, x)
5      BEST-QUICKSORT(A, p, q - 1)
6      BEST-QUICKSORT(A, q + 1, r)

Here PARTITION-AROUND is the modified PARTITION of Section 9.3 that takes the pivot value as a parameter and returns its final index. Correctness is that of QUICKSORT (Section 7.1), which holds for any choice of pivot: after line 4 everything in \(A[p..q-1]\) is below \(A[q]\) and everything in \(A[q+1..r]\) is above it, and the recursive calls sort these subarrays in place.

Running time. With \(n = r-p+1\) distinct elements, \(x\) has rank \(\lceil n/2\rceil\), so the subarrays have \(\lceil n/2\rceil - 1 \le n/2\) and \(\lfloor n/2\rfloor\) elements. Lines 3 and 4 take \(\Theta(n)\) in the worst case (SELECT is worst-case linear, Section 9.3), so

\begin{equation*} T(n) \le 2T(\lfloor n/2\rfloor) + \Theta(n) . \end{equation*}

This is case 2 of the master theorem (Theorem 4.1) with \(a=b=2\): \(f(n) = \Theta(n) = \Theta(n^{\log_2 2})\), so \(T(n) = O(n\lg n)\).

Record each comparison outcome as a directed edge \(u \to v\) meaning \(u > v\); the smaller elements are exactly those reachable from the answer \(x\), and the larger ones exactly those that reach \(x\), so a graph search (no comparisons) lists them.

By transitivity, everything reachable from \(x\) is \(< x\) and everything reaching \(x\) is \(> x\). It remains to show that every \(y \ne x\) is one or the other (elements are distinct, as throughout Chapter 9; fix the algorithm’s random choices if it is randomized, so it is deterministic). Suppose \(y < x\) but \(x\) does not reach \(y\). Let

\begin{equation*} U = \{y\} \cup \{z : z \text{ reaches } y\}, \end{equation*}

so \(x \notin U\). Form a new input by adding to every element of \(U\) a constant \(M\) larger than the spread of all values. Every comparison the algorithm made has the same outcome on the new input:

(i) both operands in \(U\), or both outside \(U\): unchanged;

(ii) \(u \in U\), \(v \notin U\), and originally \(u > v\): still \(u + M > v\);

(iii) \(u \in U\), \(v\notin U\), and originally \(u < v\): impossible, since then \(v \to u\) and \(u\) reaches \(y\) (or is \(y\)), so \(v\) reaches \(y\) and \(v \in U\).

Since the algorithm’s behaviour depends only on comparison outcomes, it performs the same comparisons on the new input and again outputs \(x\). But \(x\) is unchanged and \(y\) has moved above it, while no element outside \(U\) moved, so \(x\) now has rank at most \(i-1\), contradicting correctness. The case \(y > x\) with \(y\) not reaching \(x\) is symmetric (subtract \(M\) from \(y\) and everything reachable from \(y\)). Hence the \(i-1\) elements reachable from \(x\) are the smaller ones and the \(n-i\) elements reaching \(x\) are the larger ones, found by DFS (Section 22.3) on the recorded comparisons.

Partition around the black-box median and recurse into the one side that contains the \(i\)th smallest element; each step halves the problem.

SELECT-BB(A, p, r, i)
1  if p == r
2      return A[p]
3  x = MEDIAN(A, p, r)                // black box, lower median
4  q = PARTITION-AROUND(A, p, r, x)
5  k = q - p + 1
6  if i == k
7      return A[q]
8  elseif i < k
9      return SELECT-BB(A, p, q - 1, i)
10 else return SELECT-BB(A, q + 1, r, i - k)

Correctness is that of RANDOMIZED-SELECT (Section 9.2): after line 4, \(A[q]\) is the \(k\)th smallest of \(A[p..r]\), the \(i\)th smallest lies in \(A[p..q-1]\) (with the same rank) if \(i<k\), and in \(A[q+1..r]\) with rank \(i-k\) if \(i>k\); induction on \(n = r-p+1\) finishes it.

Running time. For \(n\ge2\), \(x\) has rank \(k = \lceil n/2\rceil\), so the recursive call has at most \(\max(k-1, n-k) = \lfloor n/2\rfloor\) elements. With \(cn\) bounding lines 3 to 5 and \(c \ge T(1)\),

\begin{equation*} T(n) \le T(\lfloor n/2\rfloor) + cn \le 2c\lfloor n/2\rfloor + cn \le 2cn \end{equation*}

by induction on the hypothesis \(T(m) \le 2cm\) for \(m<n\) (base \(T(1)\le c \le 2c\)). So \(T(n) = O(n)\).

Select the middle quantile with SELECT, partition around it, and recurse on each side with the quantiles that fall there; the number of quantiles halves at each level, so there are \(O(\lg k)\) levels of \(O(n)\) work each.

Take \(2\le k\le n\) and target ranks \(R[j] = \lceil jn/k\rceil\) for \(j = 1,\dots,k-1\): the blocks of ranks \((R[j-1], R[j]]\) (with \(R[0]=0\), \(R[k]=n\)) have sizes \(\lfloor n/k\rfloor\) or \(\lceil n/k\rceil\), and the \(R[j]\) are strictly increasing since \(n/k\ge1\). Call MULTI-SELECT(A, 1, n, R, 1, k-1).

MULTI-SELECT(A, p, r, R, a, b)
1  if a > b
2      return
3  m = floor((a + b) / 2)
4  x = SELECT(A, p, r, R[m] - p + 1)
5  PARTITION-AROUND(A, p, r, x)      // x lands at index R[m]
6  MULTI-SELECT(A, p, R[m] - 1, R, a, m - 1)
7  print x
8  MULTI-SELECT(A, R[m] + 1, r, R, m + 1, b)

Correctness. Invariant for every call: \(A[p..r]\) holds exactly the elements of global ranks \(p,\dots,r\), and \(p \le R[a] \le \dots \le R[b] \le r\). It holds initially. Within a call, the element of global rank \(R[m]\) is the \((R[m]-p+1)\)st smallest of \(A[p..r]\), so line 4 finds it and line 5 places it at index \(R[m]\) with the ranks \(p..R[m]-1\) to its left and \(R[m]+1..r\) to its right; the ranks \(R[a..m-1]\) and \(R[m+1..b]\) lie in those ranges, so both recursive calls satisfy the invariant. By induction each call prints the elements of ranks \(R[a..b]\) in increasing order (left part, then \(x\), then right part), so the top call lists all \(k-1\) quantiles in order.

Running time. A call with \(t = b-a+1\) ranks makes recursive calls with \(m-a \le \lfloor t/2\rfloor\) and \(b-m \le \lfloor t/2\rfloor\) ranks, so calls with at least one rank occur only at depths \(0,1,\dots,\lfloor\lg(k-1)\rfloor\). Lines 4 and 5 cost \(O(r-p+1)\) (SELECT is worst-case linear), and the subarrays of the calls at one depth are disjoint, so each depth costs \(O(n)\) plus \(O(1)\) per call; there are at most \(2(k-1)+1 = O(n)\) calls in all. Hence

\begin{equation*} T(n,k) = O\big(n(\lfloor \lg(k-1)\rfloor + 1)\big) + O(n) = O(n\lg k) . \end{equation*}

Find the median \(m\) with SELECT. Give each \(s \in S\) the key \((|s-m|, s)\) and use SELECT again to find the \(k\)th smallest key. The output is every element whose key is at most that one, with keys compared lexicographically. Since the elements of \(S\) are distinct, the keys are distinct too, so the tie-break by value means SELECT never has to handle equal keys.

K-CLOSEST-TO-MEDIAN(S, n, k)
1  m = SELECT(S, ceil(n/2))
2  let P[1..n] and C be new arrays; c = 0
3  for i = 1 to n
4      P[i] = (|S[i] - m|, S[i])
5  t = SELECT(P, k)            // lexicographic order on pairs
6  for i = 1 to n
7      if P[i] <= t            // lexicographic order on pairs
8          c = c + 1
9          C[c] = S[i]
10 return C[1..c]

Correctness: the keys are distinct, so exactly \(k\) of them are \(\le t = (d, x)\), and line 10 returns exactly \(k\) elements. Every returned \(s\) has \(|s-m| \le d\). Every element left out has a key \(> t\), so its distance is \(\ge d\). So no element outside \(C\) is closer to \(m\) than any element inside \(C\), which means \(C\) is a set of \(k\) closest elements.

Running time: the two SELECT calls take \(O(n)\) worst-case time each (Section 9.3), because comparing two pairs takes \(O(1)\) time. The two loops take \(O(n)\), so the total is \(O(n)\).

Binary search for \(i^*\), the number of elements of \(X\) among the \(n\) smallest elements. The lower median of the \(2n\) elements is the \(n\)th smallest. Use the sentinels \(X[0] = Y[0] = -\infty\) and \(X[n+1] = Y[n+1] = +\infty\), and let \(i^*\) be the largest \(i \in \{0, \dots, n\}\) with

\begin{equation*} P(i):\quad X[i] \le Y[n-i+1]. \end{equation*}

As \(i\) grows, \(X[i]\) does not decrease and \(Y[n-i+1]\) does not increase, so \(P\) is monotone: it is true at \(0\), and once it fails it stays false. Let \(A = X[1 \,..\, i^*] \cup Y[1 \,..\, n-i^*]\), so \(|A| = n\). We have \(X[i^*] \le Y[n-i^*+1]\) by \(P(i^*)\), and \(Y[n-i^*] < X[i^*+1]\) because \(P(i^*+1)\) fails (or \(X[n+1] = +\infty\) when \(i^* = n\)). Together with the sortedness of each array, this gives \(\max A \le\) every remaining element. So the \(n\)th smallest element is

\begin{equation*} \max A = \max(X[i^*],\, Y[n-i^*]). \end{equation*}

TWO-ARRAY-MEDIAN(X, Y, n)
1  lo = 0
2  hi = n
3  while lo < hi
4      mid = ceil((lo + hi) / 2)
5      if X[mid] <= Y[n - mid + 1]
6          lo = mid
7      else hi = mid - 1
8  if lo == 0
9      return Y[n]
10 elseif lo == n
11     return X[n]
12 else return max(X[lo], Y[n - lo])

Loop invariant: \(P(lo)\) holds, and \(P(i)\) fails for every \(i > hi\). Initialization: \(P(0)\) holds because \(X[0] = -\infty\), and no \(i \in \{0,\dots,n\}\) exceeds \(n\). Maintenance: \(1 \le mid \le n\), so both indices in line 5 are real array positions. If \(P(mid)\) holds then line 6 keeps the first half of the invariant. If \(P(mid)\) fails then by monotonicity \(P\) fails for all \(i \ge mid\), so line 7 keeps the second half. Termination: \(lo = hi = i^*\). Lines 8–12 then return \(\max(X[i^*], Y[n-i^*])\) with the sentinels written out.

Running time: each iteration shrinks \(hi - lo\) to at most \(\lfloor (hi-lo)/2 \rfloor\), so there are at most \(\lceil \lg(n+1) \rceil\) iterations of \(O(1)\) work each. The total is \(O(\lg n)\).

Put the pipeline on the line \(y = \) a median of the wells’ \(y\)-coordinates (for example the lower median) and ignore the \(x\)-coordinates. With SELECT this takes \(O(n)\) time.

A pipeline at height \(y\) has total spur length \(f(y) = \sum_{i=1}^n |y_i - y|\), because the \(x\)-coordinates do not affect any spur. Sort the coordinates as \(y_{(1)} \le \dots \le y_{(n)}\). For \(1 \le j \le \lfloor n/2 \rfloor\) the triangle inequality gives

\begin{equation*} |y_{(j)} - y| + |y_{(n+1-j)} - y| \ge y_{(n+1-j)} - y_{(j)}, \end{equation*}

with equality exactly when \(y_{(j)} \le y \le y_{(n+1-j)}\). If \(n\) is odd, the unpaired term \(|y_{((n+1)/2)} - y|\) is \(\ge 0\), and it equals \(0\) at \(y = y_{((n+1)/2)}\). Summing over the pairs,

\begin{equation*} f(y) \ge \sum_{j=1}^{\lfloor n/2 \rfloor} \bigl(y_{(n+1-j)} - y_{(j)}\bigr) \quad \text{for all } y. \end{equation*}

The lower median \(y_{(\lfloor (n+1)/2 \rfloor)}\) lies in every interval \([y_{(j)}, y_{(n+1-j)}]\) with \(j \le \lfloor n/2 \rfloor\), and when \(n\) is odd it equals the middle value. So it attains the bound and is optimal. (When \(n\) is even, every \(y\) between the lower and upper medians is optimal.)

Algorithm: find the lower median \(y^*\) of \(y_1, \dots, y_n\) by calling SELECT with \(i = \lfloor (n+1)/2 \rfloor\), and build the pipeline along \(y = y^*\). SELECT runs in \(O(n)\) worst-case time (Section 9.3), so the whole algorithm is linear. If some wells share a \(y\)-value, select on the pairs \((y_i, i)\) instead, which are distinct.

(a) \(\Theta(n\lg n)\). Sort with merge sort or heapsort (\(\Theta(n\lg n)\) worst case) and read off the last \(i\) entries in \(\Theta(i)\) time:

\begin{equation*} T(n,i)=\Theta(n\lg n)+\Theta(i)=\Theta(n\lg n)\qquad(i\le n). \end{equation*}

No comparison sort does better in the worst case (Theorem 8.1), so this is the best possible for this method.

(b) \(O(n+i\lg n)\). BUILD-MAX-HEAP takes \(O(n)\) (Section 6.3), and each of the \(i\) calls to HEAP-EXTRACT-MAX takes \(O(\lg n)\) on a heap of at most \(n\) elements (Section 6.5); the extracted keys come out in decreasing order:

\begin{equation*} T(n,i)=O(n)+i\cdot O(\lg n)=O(n+i\lg n). \end{equation*}

(c) \(O(n+i\lg i)\). Find the \(i\)th largest number, i.e. the \((n-i+1)\)st order statistic, with SELECT in \(O(n)\) worst-case time (Section 9.3); partition around it with the deterministic PARTITION of Section 7.1 (pivot swapped to the end) in \(\Theta(n)\), which places the \(i\) largest numbers in \(A[n-i+1 \,..\, n]\); then merge sort that subarray in \(\Theta(i\lg i)\):

\begin{equation*} T(n,i)=O(n)+\Theta(n)+\Theta(i\lg i)=O(n+i\lg i). \end{equation*}

The weighted median is the sorted element \(y_k\) with \(L_k<\tfrac12\le L_{k+1}\), where \(y_1<\cdots<y_n\) carry weights \(v_1,\ldots,v_n\) and \(L_k=\sum_{j<k}v_j\) (so \(0=L_1<\cdots<L_{n+1}=1\)): the defining conditions read \(L_k<\tfrac12\) and \(1-L_{k+1}\le\tfrac12\), i.e.

\begin{equation*} L_k<\tfrac12\le L_{k+1}, \end{equation*}

and since \(L\) is strictly increasing there is exactly one such \(k\): the weighted median exists and is unique.

(a) With \(v_j=1/n\) we have \(L_k=(k-1)/n\), and the lower median is \(y_k\) with \(k=\lfloor (n+1)/2\rfloor\). Then \(n/2\le k\le (n+1)/2\), so

\begin{equation*} L_k=\frac{k-1}{n}\le\frac{n-1}{2n}<\frac12, \qquad L_{k+1}=\frac kn\ge\frac12 , \end{equation*}

and \(y_k\) is the weighted median.

(b) Sort the pairs \((x_i,w_i)\) by \(x_i\) with merge sort, then scan, keeping the running sum \(s=L_k\), and return the first \(y_k\) with \(s+v_k\ge\tfrac12\). That \(y_k\) is the least \(k\) with \(L_{k+1}\ge\frac12\), so \(L_k<\frac12\) as well, and it is the weighted median by the characterization above. Time: \(\Theta(n\lg n)\) for sorting plus \(\Theta(n)\) for the scan, \(O(n\lg n)\) in all.

(c) Generalize to finding, for a target \(0<t\le\sum_{a\in A}w_a\), the element of \(A\) with \(L_k<t\le L_{k+1}\) (prefix sums within \(A\)); the weighted median is the call with \(t=\tfrac12\). Split around the lower median, decide which side holds the target, and recurse on that side only:

WEIGHTED-SELECT(A, t)
1  if |A| == 1
2      return the element of A
3  m = SELECT(A, floor((|A|+1)/2))      // lower median by key
4  L = elements of A with key < m.key
5  H = elements of A with key > m.key
6  WL = total weight of L
7  if WL < t <= WL + m.w
8      return m
9  elseif t <= WL
10     return WEIGHTED-SELECT(L, t)
11 else return WEIGHTED-SELECT(H, t - WL - m.w)

Correctness by induction on \(|A|\). In \(A\) sorted, \(m\) sits just after \(L\), so its prefix sums are \(L_k=W_L\), \(L_{k+1}=W_L+w_m\) and line 8 is right. If \(t\le W_L\), the target lies in \(L\) and prefix sums inside \(L\) are those of \(A\); \(W_L\ge t>0\) makes \(L\) nonempty. If \(t>W_L+w_m\), the target lies in \(H\), whose prefix sums are those of \(A\) minus \(W_L+w_m\), so the shifted target \(t-W_L-w_m\in(0,W_H]\) is valid and \(H\ne\emptyset\). For \(|A|=1\) the hypothesis \(0<t\le w\) makes the single element the answer.

Running time: SELECT is \(O(n)\) worst case (Section 9.3), lines 4 to 6 are \(\Theta(n)\), and \(|L|=\lfloor (n+1)/2\rfloor-1\le n/2\), \(|H|=n-\lfloor(n+1)/2\rfloor\le n/2\), so

\begin{equation*} T(n)\le T(n/2)+\Theta(n)=\Theta(n) \end{equation*}

by case 3 of the master theorem (\(a=1\), \(b=2\), \(f(n)=\Theta(n)=\Omega(n^{0+\epsilon})\), regularity \(f(n/2)=\tfrac12 f(n)\)). Every element must be examined, so the bound is \(\Theta(n)\).

(d) Let \(f(p)=\sum_i w_i|p-x_i|\) and \(x_k\) the weighted median, \(W_{<}=\sum_{x_i<x_k}w_i<\frac12\), \(W_{>}=\sum_{x_i>x_k}w_i\le\frac12\). For \(q>x_k\), each term with \(x_i\le x_k\) grows by exactly \(q-x_k\) and each other term shrinks by at most \(q-x_k\) (triangle inequality), so

\begin{equation*} f(q)-f(x_k)\ge (q-x_k)\bigl((1-W_{>})-W_{>}\bigr)=(q-x_k)(1-2W_{>})\ge 0 . \end{equation*}

Symmetrically, for \(q<x_k\),

\begin{equation*} f(q)-f(x_k)\ge (x_k-q)\bigl((1-W_{<})-W_{<}\bigr)=(x_k-q)(1-2W_{<})> 0 . \end{equation*}

Hence \(x_k\) minimizes \(f\). (If the weights do not sum to 1, divide them by their total; this scales \(f\) and does not move its minimizer.)

(e) Take \(p=(x^*,y^*)\) with \(x^*\) the weighted median of the \(x\)-coordinates and \(y^*\) the weighted median of the \(y\)-coordinates, both with weights \(w_i\). Indeed the objective separates,

\begin{equation*} \sum_i w_i\,d(p,p_i)=\sum_i w_i|x-x_i|+\sum_i w_i|y-y_i| , \end{equation*}

the first sum depends only on \(x\) and the second only on \(y\), and by (d) each is minimized by the corresponding weighted median. (Repeated coordinate values are merged into one element carrying the sum of their weights; the argument of (d) never used distinctness beyond that.) By (c) the optimum is found in \(\Theta(n)\) time.

(a) Pair up, keep the winners, recurse, then finish on the at most \(2i\) survivors with SELECT. Take \(T\) nondecreasing (replace \(T(n)\) by \(\max_{m \le n} T(m)\), still \(\Theta(n)\)), and note that SELECT also yields, at no extra cost, the set of elements below its answer (the low sides it discards plus the low side of its last PARTITION), so the procedure returns the \(i\) smallest as a set.

SMALL-SELECT(A, n, i)      // returns the i smallest elements of A
1  if i >= n/2
2      return the i smallest of A, found by SELECT(A, i)
3  S = empty; for t = 1 to floor(n/2)
4      compare A[2t-1] with A[2t]
5      put the smaller in S; record the larger as its partner
6  if n is odd
7      put A[n] in S                    // unpaired, no partner
8  L = SMALL-SELECT(S, ceil(n/2), i)   // i smallest of S
9  C = L together with the partners of the elements of L
10 return the i smallest of C, found by SELECT(C, i)

Since \(i < n/2\) in line 8, \(i \le \lceil n/2 \rceil = |S|\), so the recursive call is legal. Correctness: let \(x\) be one of the \(i\) smallest of \(A\), so fewer than \(i\) elements of \(A\) are below \(x\). (i) If \(x \in S\), fewer than \(i\) elements of \(S\) lie below \(x\), so \(x \in L\). (ii) If \(x \notin S\), \(x\) lost its pair to some \(y < x\) with \(y \in S\); fewer than \(i\) elements lie below \(y\), so \(y \in L\) and \(x\), its partner, is in \(C\). Hence \(C\) contains the \(i\) smallest of \(A\), and the \(i\) smallest of \(C\) are the \(i\) smallest of \(A\). Comparisons: \(\lfloor n/2\rfloor\) in lines 3-7, \(U_i(\lceil n/2\rceil)\) in line 8, and at most \(T(|C|) \le T(2i)\) in line 10, since \(|C| \le 2|L| = 2i\). Total \(U_i(n)\).

(b) Let \(n_0 = n\), \(n_{t+1} = \lceil n_t/2 \rceil\), and let \(m \ge 1\) be the first index with \(i \ge n_m/2\). Unrolling, and using \(\lfloor n_t/2 \rfloor = n_t - n_{t+1}\) (telescoping),

\begin{equation*} \begin{aligned} U_i(n) &= \sum_{t=0}^{m-1}\bigl(n_t - n_{t+1}\bigr) + m\,T(2i) + T(n_m)\\ &= n - n_m + m\,T(2i) + T(n_m) \le n + (m+1)\,T(2i), \end{aligned} \end{equation*}

since \(n_m \le 2i\). As \(\lceil x/2 \rceil \le x/2 + 1/2\), induction gives \(n_t < n/2^t + 1\). Minimality of \(m\) gives \(n_{m-1} > 2i\), so \(n/2^{m-1} + 1 > 2i\), i.e.

\begin{equation*} 2^{m-1} < \frac{n}{2i-1} \le \frac{n}{i}, \qquad m < 1 + \lg(n/i). \end{equation*}

Because \(i < n/2\) we have \(\lg(n/i) > 1\), so \(U_i(n) \le n + (2 + \lg(n/i))T(2i) \le n + 3\,T(2i)\lg(n/i) = n + O(T(2i)\lg(n/i))\).

(c) For constant \(i\), \(T(2i)\) is a constant and \(\lg(n/i) \le \lg n\), so (b) gives \(U_i(n) = n + O(\lg n)\).

(d) If \(k = 2\) then \(i = n/2\), so \(U_i(n) = T(n) \le n + T(2n/k)\lg k\). If \(k > 2\) then \(i < n/2\) and (b) with \(2i = 2n/k\), \(n/i = k\) gives \(U_i(n) = n + O(T(2n/k)\lg k)\).

(a) \(\mathrm{E}[X_{ijk}] = 2/(\max(j,k) - \min(i,k) + 1)\), that is,

\begin{equation*} \mathrm{E}[X_{ijk}] = \begin{cases} 2/(j-i+1) & \text{if } i \le k \le j,\\ 2/(j-k+1) & \text{if } k < i,\\ 2/(k-i+1) & \text{if } j < k. \end{cases} \end{equation*}

Let \(Z = \{z_{\min(i,k)}, \ldots, z_{\max(j,k)}\}\), which has at least two elements. As long as no pivot from \(Z\) has been chosen, all of \(Z\) lies in the current subarray: the subarray contains \(z_k \in Z\), and a pivot outside \(Z\) is below or above all of \(Z\), so the side kept (the one holding \(z_k\)) keeps all of \(Z\). In particular the subarray has at least two elements, so a pivot is drawn and, as in Section 7.4.2, comparisons only involve the pivot. Consider the first pivot \(x \in Z\): (i) \(x \in \{z_i, z_j\}\): PARTITION compares \(x\) with the other one, so \(X_{ijk} = 1\). (ii) \(z_i < x < z_j\): they go to opposite sides and are never compared. (iii) \(x\) outside \([z_i, z_j]\) (then \(x\) lies between \(z_k\) and the pair, or \(x = z_k\)): either the algorithm returns, or \(z_i, z_j\) fall on the side not containing \(z_k\) and are discarded; either way \(X_{ijk} = 0\). Each element of the subarray is equally likely to be the pivot, so the first pivot from \(Z\) is uniform on \(Z\) and \(\mathrm{E}[X_{ijk}] = \Pr\{X_{ijk}=1\} = 2/|Z|\).

(b) Each pair is compared at most once (Section 7.4.2), so \(X_k = \sum_{i<j} X_{ijk}\). Split by the cases of (a). For \(i \le k \le j\) we add the nonnegative term \(i = j = k\); for \(k < i < j\), a fixed \(j\) has \(j-k-1\) choices of \(i\); for \(i < j < k\), a fixed \(i\) has \(k-i-1\) choices of \(j\):

\begin{equation*} \begin{aligned} \mathrm{E}[X_k] &\le \sum_{i=1}^{k}\sum_{j=k}^{n}\frac{2}{j-i+1} + \sum_{j=k+1}^{n}\sum_{i=k+1}^{j-1}\frac{2}{j-k+1} + \sum_{i=1}^{k-2}\sum_{j=i+1}^{k-1}\frac{2}{k-i+1}\\ &= 2\left(\sum_{i=1}^{k}\sum_{j=k}^{n}\frac{1}{j-i+1} + \sum_{j=k+1}^{n}\frac{j-k-1}{j-k+1} + \sum_{i=1}^{k-2}\frac{k-i-1}{k-i+1}\right). \end{aligned} \end{equation*}

(c) In the double sum put \(d = j - i + 1 \in \{1, \ldots, n\}\); for fixed \(d\) the pairs with \(i \le k \le j\) have \(k - d + 1 \le i \le k\), so there are at most \(d\) of them. Every fraction in the last two sums is below \(1\). Hence

\begin{equation*} \mathrm{E}[X_k] \le 2\left(\sum_{d=1}^{n} d\cdot\frac{1}{d} + (n-k) + \max(k-2,0)\right) \le 4n. \end{equation*}

(d) Each call of RANDOMIZED-SELECT on a subarray of \(m \ge 2\) elements runs RANDOMIZED-PARTITION in \(\Theta(m)\) time with \(m - 1 \ge 1\) comparisons, plus \(O(1)\) other work, and calls with \(m = 1\) cost \(O(1)\) and end the recursion. So, as in Lemma 7.1, the number of calls is at most \(X_k + 1\) and the running time is \(O(X_k + 1)\). By (c) the expected running time is \(O(\mathrm{E}[X_k] + 1) = O(n)\).

The final stack is \(\langle 4,1\rangle\) with \(S.top=2\), the two pops having returned \(3\) and then \(8\). The stack is \(S[1\,..\,S.top]\); cells above \(S.top\) keep stale values (shown in the array column) but are not part of the stack. Starting from \(S.top=0\) (stack listed bottom first; “-” marks a cell never written):

Grow stack 1 upward from the left end and stack 2 downward from the right end: stack 1 occupies \(A[1\,..\,top_1]\) (top at \(A[top_1]\)) and stack 2 occupies \(A[top_2\,..\,n]\) (top at \(A[top_2]\)), initially \(top_1=0\), \(top_2=n+1\).

PUSH1(A, x)                      PUSH2(A, x)
1  if top1 + 1 == top2           1  if top1 + 1 == top2
2      error "overflow"          2      error "overflow"
3  top1 = top1 + 1               3  top2 = top2 - 1
4  A[top1] = x                   4  A[top2] = x

POP1(A)                          POP2(A)
1  if top1 == 0                  1  if top2 == n + 1
2      error "underflow"         2      error "underflow"
3  top1 = top1 - 1               3  top2 = top2 + 1
4  return A[top1 + 1]            4  return A[top2 - 1]

Invariant: stack 1 is \(A[1\,..\,top_1]\) bottom to top, stack 2 is \(A[n], A[n-1],\dots,A[top_2]\) bottom to top, and \(0\le top_1<top_2\le n+1\). It holds initially (both empty), and each procedure moves only its own pointer by one in the direction that keeps the stated layout, so it is maintained. The free cells are exactly \(A[top_1+1\,..\,top_2-1]\), so

\begin{equation*} \#\text{free} = top_2-top_1-1 = n-\bigl(top_1+(n+1-top_2)\bigr) = n-(\text{total size}). \end{equation*}

Hence the overflow test \(top_1+1=top_2\) fires exactly when the two stacks together hold \(n\) elements, as required. Each procedure does a constant number of comparisons and assignments, so runs in \(O(1)\) time.

The final queue is \(\langle 3,8\rangle\) with \(Q.head=3\), \(Q.tail=5\), the two dequeues having returned \(4\) and then \(1\). The queue is \(Q[Q.head\,..\,Q.tail-1]\) (no wrap-around occurs here); other cells hold stale values. Starting from \(Q.head=Q.tail=1\) ("-" marks a cell never written):

Test the book’s emptiness and fullness conditions before touching the array: the queue is empty iff \(Q.head=Q.tail\), and full iff \(Q.head=Q.tail+1\) or (\(Q.head=1\) and \(Q.tail=Q.length\)) (Section 10.1, pp. 234-235).

ENQUEUE(Q, x)
1  if Q.head == Q.tail + 1 or (Q.head == 1 and Q.tail == Q.length)
2      error "overflow"
3  Q[Q.tail] = x
4  if Q.tail == Q.length
5      Q.tail = 1
6  else Q.tail = Q.tail + 1

DEQUEUE(Q)
1  if Q.head == Q.tail
2      error "underflow"
3  x = Q[Q.head]
4  if Q.head == Q.length
5      Q.head = 1
6  else Q.head = Q.head + 1
7  return x

The queue occupies \(Q.head, Q.head+1,\dots,Q.tail-1\) cyclically, so it holds \((Q.tail-Q.head)\bmod n\) elements, \(n=Q.length\). Line 1 of \(\text{DEQUEUE}\) is true exactly when this count is \(0\), and line 1 of \(\text{ENQUEUE}\) is true exactly when \(Q.tail+1\equiv Q.head \pmod n\), i.e. when the count is \(n-1\), the capacity of this representation (one more insertion would make \(Q.head=Q.tail\), indistinguishable from empty). When neither error fires, lines 3-6 (resp. 3-7) are the original procedures, which are correct. Each procedure still does \(O(1)\) work.

Use the circular-array queue layout of Section 10.1: the deque, front to back, is \(D[D.head], D[D.head+1],\dots,D[D.tail-1]\) (indices mod \(n=D.length\), wrapping \(n\to 1\)), initially \(D.head=D.tail=1\). Inserting at the back and deleting from the front are \(\text{ENQUEUE}\) and \(\text{DEQUEUE}\) with the checks of Exercise 10.1-4; the other two ends move the pointers the opposite way.

FULL(D) = (D.head == D.tail + 1) or (D.head == 1 and D.tail == D.length)

TAIL-INSERT(D, x)   same as checked ENQUEUE(D, x) of Exercise 10.1-4
HEAD-DELETE(D)      same as checked DEQUEUE(D) of Exercise 10.1-4

HEAD-INSERT(D, x)
1  if FULL(D)
2      error "overflow"
3  if D.head == 1
4      D.head = D.length
5  else D.head = D.head - 1
6  D[D.head] = x

TAIL-DELETE(D)
1  if D.head == D.tail
2      error "underflow"
3  if D.tail == 1
4      D.tail = D.length
5  else D.tail = D.tail - 1
6  return D[D.tail]

Invariant: the deque’s contents in order are the cyclic block \(D.head,\dots,D.tail-1\), of size \((D.tail-D.head)\bmod n\le n-1\). \(\text{HEAD-INSERT}\) steps \(D.head\) back one cell cyclically and writes \(x\) there, so \(x\) becomes the new first element; \(\text{TAIL-DELETE}\) steps \(D.tail\) back one cell, so the old last element \(D[D.tail-1]\) is returned and leaves the block; the other two are the queue operations, already correct. As in Exercise 10.1-4, the guards fire exactly when the size is \(n-1\) (insertions) or \(0\) (deletions), so the invariant is preserved. Each procedure performs a constant number of tests and assignments: \(O(1)\) time.

Keep an inbox stack \(I\) and an outbox stack \(O\): enqueue pushes onto \(I\); dequeue pops \(O\), first refilling \(O\) by moving all of \(I\) onto it when \(O\) is empty. Enqueue is \(\Theta(1)\), dequeue is \(\Theta(k)\) in the worst case (\(k=|I|\)), and any sequence of \(m\) operations costs \(O(m)\), i.e. \(O(1)\) amortized.

ENQUEUE(Q, x)
1  PUSH(I, x)

DEQUEUE(Q)
1  if STACK-EMPTY(O)
2      while not STACK-EMPTY(I)
3          PUSH(O, POP(I))
4  if STACK-EMPTY(O)
5      error "underflow"
6  return POP(O)

Invariant: the queue, head to tail, is the contents of \(O\) from top to bottom followed by the contents of \(I\) from bottom to top. Initially both stacks are empty. \(\text{ENQUEUE}\) appends \(x\) at the top of \(I\), i.e. at the tail. In \(\text{DEQUEUE}\), if \(O\) is nonempty its top is the head; if \(O\) is empty, lines 2-3 reverse \(I\) onto \(O\), so the bottom of \(I\) (the oldest element) becomes the top of \(O\) and the invariant still holds with \(I\) empty; then line 6 removes the head. Underflow is reported exactly when both stacks are empty, i.e. the queue is empty.

Running time (each stack operation is \(O(1)\) by Section 10.1). \(\text{ENQUEUE}\) is \(\Theta(1)\). A \(\text{DEQUEUE}\) that finds \(O\) empty costs \(\Theta(1+|I|)\); e.g. after \(n\) enqueues the first dequeue does \(2n+1\) stack operations, so the worst case is \(\Theta(n)\). But every element is moved from \(I\) to \(O\) at most once, at the cost of one pop and one push. Hence a sequence of \(m\) operations, \(e\) of them enqueues and \(d\) dequeues, starting from an empty queue, performs at most

\begin{equation*} e + 2e + d \le 3m \end{equation*}

stack operations (plus \(O(1)\) tests per operation), so its total time is \(O(m)\) and the amortized cost per operation is \(O(1)\) (aggregate analysis, Section 17.1).

Keep every element of the stack in queue \(Q_1\) in push order (bottom at the head, top at the tail) with \(Q_2\) empty between operations; PUSH is a single ENQUEUE onto \(Q_1\), and POP moves the first \(n-1\) elements of \(Q_1\) into \(Q_2\), dequeues the last one (the top) as the answer, and swaps the roles of the two queues. PUSH costs \(\Theta(1)\) and POP costs \(\Theta(n)\) on a stack of \(n\) elements.

PUSH(S, x)
1  ENQUEUE(S.q1, x)
2  S.size = S.size + 1

POP(S)
1  if S.size == 0
2      error "underflow"
3  for i = 1 to S.size - 1
4      ENQUEUE(S.q2, DEQUEUE(S.q1))
5  x = DEQUEUE(S.q1)
6  exchange S.q1 with S.q2
7  S.size = S.size - 1
8  return x

Between operations, \(Q_2\) is empty and \(Q_1\) holds the stack’s elements \(x_1, \dots, x_n\) from head to tail in the order they were pushed, with \(n = S.size\). PUSH appends \(x_{n+1}\) at the tail of \(Q_1\), preserving it. For POP, the loop of lines 3–4 has the invariant: at the start of iteration \(i\), \(Q_2\) holds \(x_1, \dots, x_{i-1}\) and \(Q_1\) holds \(x_i, \dots, x_n\), both in order.

• Initialization (\(i=1\)): \(Q_2\) is empty and \(Q_1 = x_1, \dots, x_n\).
• Maintenance: DEQUEUE removes \(x_i\), the head of \(Q_1\), and ENQUEUE puts it at the tail of \(Q_2\) behind \(x_{i-1}\).
• Termination (\(i = n\)): \(Q_1\) holds only \(x_n\) and \(Q_2 = x_1, \dots, x_{n-1}\).

So line 5 returns \(x_n\), the most recently pushed element (LIFO), and after the exchange \(Q_1 = x_1, \dots, x_{n-1}\) and \(Q_2\) is empty, restoring the invariant with \(n-1\).

Running time. Each ENQUEUE and DEQUEUE takes \(O(1)\) (Section 10.1). PUSH does one of them, so it takes \(\Theta(1)\). POP does \(n-1\) loop iterations of two queue operations each, plus one more DEQUEUE:

\begin{equation*} T_{\text{POP}}(n) = 2(n-1) + 1 + O(1) = \Theta(n). \end{equation*}

The bound holds amortized too, since alternating PUSH and POP on a stack of size \(n\) costs \(\Theta(n)\) per pair.

Method (2): make POP cheap instead. PUSH\((S,x)\) enqueues \(x\) into the empty \(Q_2\), moves all \(n\) elements of \(Q_1\) into \(Q_2\) behind it, and swaps the queues, so the top is always at the head of \(Q_1\); POP is then one DEQUEUE. This gives PUSH \(\Theta(n)\) and POP \(\Theta(1)\).

INSERT, yes; DELETE, no: on a singly linked list it takes \(\Theta(n)\) time in the worst case, even when we are handed a pointer to \(x\).

For INSERT, splice \(x\) in at the head: set \(x.next = L.head\), then \(L.head = x\). That is two pointer assignments, so \(O(1)\) (a dynamic set has no required order, so the front is as good a place as any).

For DELETE, removing \(x\) means changing the \(next\) field of the node before \(x\) (or \(L.head\) if \(x\) is first). Nodes store no \(prev\) field, so the only way to reach that predecessor is to walk from \(L.head\) along \(next\) pointers:

SLIST-DELETE(L, x)
1  if L.head == x
2      L.head = x.next
3  else y = L.head
4      while y.next != x
5          y = y.next
6      y.next = x.next

The loop invariant is that \(y\) is a node strictly before \(x\). It holds at the start (\(y = L.head \ne x\)). Each iteration keeps it, since the loop body runs only when \(y.next \ne x\). The loop ends when \(y.next = x\), so line 6 splices out exactly \(x\). When \(x\) is the tail of an \(n\)-element list the loop runs \(n-2\) times, and no method can do better: the predecessor can be reached only through the \(n-2\) nodes before it, following \(next\) pointers from the head. So the worst case is \(\Theta(n)\).

Keep the top of the stack at the head of the list: PUSH inserts at the head and POP removes the head.

PUSH(L, x)
1  x.next = L.head
2  L.head = x

POP(L)
1  if L.head == NIL
2      error "underflow"
3  x = L.head
4  L.head = x.next
5  return x

Invariant: reading the list from \(L.head\) gives the stack’s elements from the most recently pushed one to the least recent, and \(L.head = \mathrm{NIL}\) exactly when the stack is empty. PUSH places the new element in front of all the others. POP removes and returns the front element, which is the most recently pushed one still present, so the order is LIFO. Neither procedure has a loop, and each does a constant number of pointer operations, so both take \(O(1)\) time.

Store a tail pointer \(L.tail\) in addition to \(L.head\). ENQUEUE appends at the tail and DEQUEUE removes the head. This is the right way round: deleting at the tail would need the tail’s predecessor, which costs \(\Theta(n)\) by Exercise 10.2-1.

ENQUEUE(L, x)
1  x.next = NIL
2  if L.head == NIL
3      L.head = x
4  else L.tail.next = x
5  L.tail = x

DEQUEUE(L)
1  if L.head == NIL
2      error "underflow"
3  x = L.head
4  L.head = x.next
5  return x

Invariant: reading the list from \(L.head\) gives the queue’s elements in order of arrival, the queue is empty exactly when \(L.head = \mathrm{NIL}\), and when it is nonempty \(L.tail\) points to the last node. ENQUEUE on an empty queue makes \(x\) both head and tail. Otherwise it links \(x\) after the current tail and makes \(x\) the new tail. Either way \(x\) ends up last and the invariant holds. DEQUEUE removes the element that arrived earliest, so the order is FIFO. If that empties the queue, \(L.tail\) is left stale, which is harmless: every operation tests \(L.head\) first, and the next ENQUEUE resets \(L.tail\). Each procedure does a constant amount of work, so both take \(O(1)\) time.

Store the search key in the sentinel before the loop starts. The scan is then guaranteed to stop at \(L.nil\) at the latest, so the test \(x \ne L.nil\) is unnecessary.

LIST-SEARCH''(L, k)
1  L.nil.key = k
2  x = L.nil.next
3  while x.key != k
4      x = x.next
5  return x