Single-Variable Calculus

the calculus of one real variable, assembled in logical order: limits, then continuity and its two workhorse theorems (ivt, evt), then the derivative and the mean value theorem, taylor’s theorem with an honest error bound, and finally the riemann integral and both parts of the fundamental theorem. everything downstream — multivariable calculus, differential equations, every convergence argument in machine learning — leans on the theorems here. proofs are given where they are short and instructive; the deferred foundations live in real analysis (Courant, Richard, 1996).

limits#

the \(\varepsilon\)–\(\delta\) definition#

let \(f\) be defined on an open interval around \(a\) (except possibly at \(a\) itself). then

\begin{equation} \lim_{x \to a} f(x) = L \quad \Longleftrightarrow \quad \forall \varepsilon > 0 \;\; \exists \delta > 0 : \quad 0 < |x - a| < \delta \implies |f(x) - L| < \varepsilon. \end{equation}

read it as a game: the adversary names a tolerance \(\varepsilon\) around \(L\); you must produce a window \(\delta\) around \(a\) inside which \(f\) never leaves the tolerance. you win if you have a winning move for every \(\varepsilon\). 𐃏

a worked proof#

claim. \(\lim_{x \to 2} x^2 = 4\).

proof. let \(\varepsilon > 0\). factor the error:

\begin{equation} |x^2 - 4| = |x - 2|\,|x + 2|. \end{equation}

• tame the nuisance factor: first insist \(|x - 2| < 1\), so \(1 < x < 3\) and hence \(|x + 2| < 5\).
• choose delta: set \(\delta = \min\{1, \varepsilon/5\}\).
• verify: if \(0 < |x - 2| < \delta\) then \(|x^2 - 4| = |x-2|\,|x+2| < \tfrac{\varepsilon}{5} \cdot 5 = \varepsilon\). \(\blacksquare\)

the pattern generalises: bound the ugly factor on a preliminary window, then shrink \(\delta\) to kill the rest. the limit laws (sum, product, quotient with nonzero denominator, squeeze) are all proved by this template and thereafter used freely.

one-sided limits \(x \to a^{+}\), \(x \to a^{-}\) restrict the window to one side; \(\lim_{x \to \infty}\) replaces the \(\delta\)-window with a threshold \(x > N\). a two-sided limit exists iff both one-sided limits exist and agree.

continuity#

\(f\) is continuous at \(a\) if \(\lim_{x \to a} f(x) = f(a)\) — the limit exists, the value exists, and they agree. continuous on \([a,b]\) means continuous at every interior point with one-sided continuity at the ends. two theorems make continuity on a closed bounded interval powerful:

• intermediate value theorem (ivt). if \(f\) is continuous on \([a,b]\) and \(y\) lies strictly between \(f(a)\) and \(f(b)\), then \(f( c) = y\) for some \(c \in (a,b)\). proof idea: say \(f(a) < y < f(b)\), and let \(c = \sup\{x \in [a,b] : f(x) < y\}\). continuity at \(c\) forbids both \(f( c) < y\) (points just right of \(c\) would also lie below \(y\), contradicting that \(c\) is an upper bound) and \(f( c) > y\) (a whole window left of \(c\) would lie above \(y\), contradicting that \(c\) is the least upper bound). so \(f( c) = y\). the proof is really a statement about the completeness of \(\mathbb{R}\) — over the rationals it is false. 𐃏
• extreme value theorem (evt). a continuous \(f\) on \([a,b]\) is bounded and attains its maximum and minimum. the proof needs compactness (bolzano–weierstrass) and is developed properly in real analysis; both hypotheses bite — \(1/x\) on \((0,1]\) is continuous and unbounded, \(x\) on \((0,1)\) attains neither extremum.

differentiation#

definition and first consequences#

\begin{equation} f’(a) = \lim_{h \to 0} \frac{f(a+h) - f(a)}{h}, \end{equation}

when the limit exists. equivalently: \(f(a+h) = f(a) + f’(a)\,h + o(h)\) — the derivative is the coefficient of the best linear approximation, which is the formulation that survives into higher dimensions.

differentiable implies continuous: \(f(a+h) - f(a) = \frac{f(a+h)-f(a)}{h} \cdot h \to f’(a) \cdot 0 = 0\). the converse fails: \(|x|\) at \(0\), and weierstrass constructed functions continuous everywhere yet differentiable nowhere.

the rules, with the two proofs worth knowing#

product rule. add and subtract the mixed term:

\begin{align*} \frac{f(x+h)g(x+h) - f(x)g(x)}{h} &= f(x+h)\,\frac{g(x+h) - g(x)}{h} + g(x)\,\frac{f(x+h) - f(x)}{h} \\ &\to f(x)g’(x) + g(x)f’(x), \end{align*}

using continuity of \(f\) at \(x\) to send the first factor to \(f(x)\). \(\blacksquare\)

chain rule (sketch). the naive proof multiplies and divides by \(f(x+h) - f(x)\), which may vanish for \(h\) arbitrarily close to \(0\). the repair: define

\begin{equation} \varphi(k) = \begin{cases} \dfrac{g(f(x) + k) - g(f(x))}{k}, & k \neq 0, \\[1ex] g’(f(x)), & k = 0, \end{cases} \end{equation}

which is continuous at \(k = 0\) precisely because \(g\) is differentiable at \(f(x)\). with \(k(h) = f(x+h) - f(x)\), the identity

\begin{equation} \frac{g(f(x+h)) - g(f(x))}{h} = \varphi(k(h)) \cdot \frac{f(x+h) - f(x)}{h} \end{equation}

holds for all small \(h\) — including those where \(k(h) = 0\), when both sides vanish — and letting \(h \to 0\) gives \(g’(f(x))\,f’(x)\). \(\blacksquare\)

the mean value theorem#

• fermat (interior extremum). if \(f\) has a local extremum at an interior point \(c\) and \(f’( c)\) exists, then \(f’( c) = 0\): the one-sided difference quotients have opposite signs, and their common limit must be both \(\geq 0\) and \(\leq 0\).

• rolle’s theorem. if \(f\) is continuous on \([a,b]\), differentiable on \((a,b)\), and \(f(a) = f(b)\), then \(f’( c) = 0\) for some \(c \in (a,b)\). proof: by evt, \(f\) attains a max and a min on \([a,b]\). if both occur at endpoints, max equals min, \(f\) is constant, and any interior \(c\) works. otherwise some extremum is interior and fermat finishes. \(\blacksquare\)

• mean value theorem (mvt). same hypotheses minus \(f(a) = f(b)\): there is \(c \in (a,b)\) with

  \begin{equation} f’( c) = \frac{f(b) - f(a)}{b - a}. \end{equation}

  proof: tilt the picture until rolle applies —

  \begin{equation} g(x) = f(x) - f(a) - \frac{f(b) - f(a)}{b - a}\,(x - a) \end{equation}

  satisfies \(g(a) = g(b) = 0\), so \(g’( c) = 0\) somewhere, which is the claim. \(\blacksquare\)

the mvt is the bridge from local information (\(f’\)) to global conclusions: \(f’ \equiv 0\) on an interval forces \(f\) constant, \(f’ > 0\) forces strict increase, and the cauchy (two-function) variant powers both l’hopital and the taylor remainder below.

taylor’s theorem#

statement with lagrange remainder#

let \(f\) be \(n+1\) times differentiable on an open interval containing \(a\) and \(x\). then

\begin{equation} f(x) = \underbrace{\sum_{k=0}^{n} \frac{f^{(k)}(a)}{k!}\,(x - a)^k}_{T_n(x)} \; + \; \underbrace{\frac{f^{(n+1)}(\xi)}{(n+1)!}\,(x - a)^{n+1}}_{R_n(x)} \end{equation}

for some \(\xi\) strictly between \(a\) and \(x\). the remainder has exactly the shape of the next term, with its derivative evaluated at an unknown intermediate point — the mvt is the case \(n = 0\). 𐃏

a worked error bound#

approximate \(e^x\) on \([0,1]\) by its degree-4 taylor polynomial at \(0\), \(T_4(x) = 1 + x + \tfrac{x^2}{2} + \tfrac{x^3}{6} + \tfrac{x^4}{24}\). since \(e^{\xi} \leq e\) on the interval,

\begin{equation} |R_4(x)| = \frac{e^{\xi}}{5!}\,x^5 \leq \frac{e}{120}\,x^5 \leq \frac{e}{120} \approx 0.0227 \qquad (0 \leq \xi \leq x \leq 1). \end{equation}

the true worst error is \(e - T_4(1) \approx 0.00995\), so the bound is honest and conservative by a factor of about \(2.3\) — the price of replacing \(e^{\xi}\) by its worst case. the numeric check at the bottom of the page confirms both numbers.

to force error below \(10^{-8}\) on \([0,1]\), demand \(\frac{e}{(n+1)!} < 10^{-8}\): \(n = 11\) suffices, since \(12! \approx 4.8 \times 10^{8}\) gives \(e/12! \approx 5.7 \times 10^{-9}\). factorials beat exponentials — this is why the taylor series of \(e^x\), \(\sin x\), \(\cos x\) converge everywhere.

l’hopital’s rule#

if \(f, g\) are differentiable near \(a\) (not necessarily at \(a\)), \(g’(x) \neq 0\) near \(a\), and

1. \(\lim_{x \to a} f(x) = \lim_{x \to a} g(x) = 0\) (or both tend to \(\pm\infty\)), and
2. \(\lim_{x \to a} \dfrac{f’(x)}{g’(x)}\) exists (or is \(\pm\infty\)),

then

\begin{equation} \lim_{x \to a} \frac{f(x)}{g(x)} = \lim_{x \to a} \frac{f’(x)}{g’(x)}. \end{equation}

the version at \(a = \pm\infty\) holds too. both conditions bite: without an indeterminate form the rule is simply false, and if \(f’/g’\) oscillates without a limit the rule is silent — the original limit may still exist (try \(f(x) = x + \sin x\), \(g(x) = x\) as \(x \to \infty\): the quotient tends to \(1\) but \(f’/g’ = 1 + \cos x\) has no limit).

example. \(\lim_{x \to 0} \frac{e^x - 1 - x}{x^2} = \lim_{x \to 0} \frac{e^x - 1}{2x} = \lim_{x \to 0} \frac{e^x}{2} = \frac{1}{2}\), applying the rule twice; each intermediate quotient is again \(0/0\).

a circularity warning. using l’hopital on \(\lim_{x \to 0} \frac{\sin x}{x}\) is a logical crime: computing \((\sin x)’\) requires that very limit. taylor expansions settle such limits faster and without circularity — expand, cancel, read off the answer.

sequences and series#

a sequence \(a_n \to L\) if for every \(\varepsilon > 0\) there is \(N\) with \(|a_n - L| < \varepsilon\) for all \(n > N\) — the same game with the \(\delta\)-window replaced by a threshold. two facts carry most of the weight: monotone bounded sequences converge (completeness again), and a sequence converges iff it is cauchy.

a series \(\sum a_n\) converges if its partial sums do. the standard kit:

• geometric: \(\sum_{n=0}^{\infty} r^n = \frac{1}{1-r}\) for \(|r| < 1\), divergent otherwise.
• harmonic and p-series: \(\sum \frac{1}{n}\) diverges (group terms into blocks of doubling length; each block exceeds \(\tfrac12\)); \(\sum \frac{1}{n^p}\) converges iff \(p > 1\).
• ratio test: if \(|a_{n+1}/a_n| \to \rho\), the series converges absolutely for \(\rho < 1\), diverges for \(\rho > 1\), and says nothing at \(\rho = 1\).
• power series: \(\sum c_n (x-a)^n\) converges for \(|x - a| < R\) (the radius of convergence, \(1/R = \limsup |c_n|^{1/n}\)) and may do anything at the endpoints. inside the radius it may be differentiated and integrated term by term — taylor series are power series, and the remainder analysis above says when they actually equal their function.

integration#

the riemann integral#

partition \([a,b]\) as \(a = x_0 < x_1 < \dots < x_n = b\), pick tags \(x_i^* \in [x_{i-1}, x_i]\), and form the riemann sum

\begin{equation} S = \sum_{i=1}^{n} f(x_i^*)\,\Delta x_i, \qquad \Delta x_i = x_i - x_{i-1}. \end{equation}

\(f\) is riemann integrable with \(\int_a^b f(x)\,dx = I\) if the sums converge to \(I\) as the mesh \(\max_i \Delta x_i \to 0\), regardless of how the tags are chosen. equivalently (darboux): the infimum of upper sums equals the supremum of lower sums. every continuous function on \([a,b]\) is integrable — continuity on a closed bounded interval is uniform, so upper and lower sums squeeze together — and so is every bounded function with finitely many discontinuities. 𐃏

the fundamental theorem of calculus#

let \(f\) be continuous on \([a,b]\).

part 1. \(F(x) = \int_a^x f(t)\,dt\) is differentiable with \(F’(x) = f(x)\).

proof sketch: for \(h > 0\),

\begin{equation} \frac{F(x+h) - F(x)}{h} = \frac{1}{h} \int_x^{x+h} f(t)\,dt, \end{equation}

and this average of \(f\) over \([x, x+h]\) is squeezed between \(\min_{[x,x+h]} f\) and \(\max_{[x,x+h]} f\), both of which tend to \(f(x)\) by continuity. the same argument runs for \(h < 0\). \(\blacksquare\)

part 2. if \(\Phi’ = f\) on \([a,b]\), then \(\int_a^b f(x)\,dx = \Phi(b) - \Phi(a)\).

proof sketch: take any partition and telescope:

\begin{equation} \Phi(b) - \Phi(a) = \sum_{i=1}^{n} \big(\Phi(x_i) - \Phi(x_{i-1})\big) = \sum_{i=1}^{n} f(c_i)\,\Delta x_i, \end{equation}

using the mvt on each subinterval to produce tags \(c_i \in (x_{i-1}, x_i)\). the right side is a riemann sum for \(\int_a^b f\); refine the mesh and it converges to the integral, while the left side never moves. \(\blacksquare\)

note the division of labour: part 1 needs continuity of \(f\), part 2 needs an antiderivative to exist — and the mvt is the engine inside part 2. together they say differentiation and integration are mutually inverse (Courant, Richard and Robbins, Herbert, 1941).

techniques#

every technique is a differentiation rule read backwards.

• substitution (chain rule reversed): with \(u = g(x)\),

  \begin{equation} \int f(g(x))\,g’(x)\,dx = \int f(u)\,du. \end{equation}

  worked: \(\int x e^{x^2}\,dx\) with \(u = x^2\), \(du = 2x\,dx\) gives \(\tfrac12 \int e^u\,du = \tfrac12 e^{x^2} + C\). check by differentiating — the chain rule regenerates the integrand.

• integration by parts (product rule reversed):

  \begin{equation} \int u\,dv = uv - \int v\,du. \end{equation}

  worked: \(\int \ln x\,dx\) with \(u = \ln x\), \(dv = dx\): \(x \ln x - \int x \cdot \tfrac{1}{x}\,dx = x \ln x - x + C\). differentiating the log and integrating the trivial factor is the whole art of choosing \(u\).

• partial fractions for rational integrands: factor the denominator, split into simple pieces, integrate each to logs and arctangents. (over \(\mathbb{C}\) this story becomes residues — see complex analysis.)

improper integrals#

two ways an integral can be improper: an infinite interval, or an unbounded integrand. both are defined as limits of proper integrals:

\begin{equation} \int_1^{\infty} \frac{dx}{x^p} = \begin{cases} \frac{1}{p-1}, & p > 1, \\ \text{divergent}, & p \leq 1, \end{cases} \qquad\qquad \int_0^1 \frac{dx}{x^p} \;\text{ converges iff }\; p < 1. \end{equation}

the boundary case \(p = 1\) fails at both ends — the logarithm diverges, slowly, in both directions. 𐃏 for integrands without elementary antiderivatives, comparison decides convergence: \(0 \leq f \leq g\) with \(\int g\) convergent forces \(\int f\) convergent. e.g. \(\int_0^{\infty} e^{-x^2}\,dx\) converges by comparison with \(e^{-x}\) on \([1,\infty)\); its exact value \(\tfrac{\sqrt{\pi}}{2}\) needs the polar-coordinates trick on the multivariable page.

numerics: checking the taylor bound#

the degree-4 bound from the taylor section, verified on a grid of \(10^5\) points:

import numpy as np
from math import e, factorial

# degree-4 taylor polynomial of exp about 0
def p4(x):
    return sum(x**k / factorial(k) for k in range(5))

x = np.linspace(0, 1, 100001)
actual = np.abs(np.exp(x) - p4(x))
bound = e * x**5 / factorial(5)        # lagrange: |R_4| <= e^c x^5/5! <= e x^5/5!

print(f"max actual error on [0,1]: {actual.max():.6f}  (at x = {x[actual.argmax()]:.2f})")
print(f"lagrange bound at x = 1  : {e / factorial(5):.6f}")
print(f"bound holds everywhere   : {bool(np.all(actual <= bound + 1e-15))}")
print(f"tightness ratio at x = 1 : {actual[-1] / bound[-1]:.3f}")

max actual error on [0,1]: 0.009948  (at x = 1.00)
lagrange bound at x = 1  : 0.022652
bound holds everywhere   : True
tightness ratio at x = 1 : 0.439

the bound holds pointwise across the whole interval, the worst error sits at the far endpoint exactly as the \(x^5\) shape predicts, and the tightness ratio quantifies the slack from replacing \(e^{\xi}\) with \(e\).

see also#

• calculus — the parent map
• multivariable calculus — the same theorems with jacobians
• differential equations — where the antiderivatives go to work
• real analysis — the deferred proofs: completeness, compactness, uniform continuity
• complex analysis — integration upgraded to contours

References#

Courant, Richard (1996). Differential and Integral Calculus, Springer.

Courant, Richard and Robbins, Herbert (1941). What is Mathematics?, Oxford University Press.