Taylor's theorem

In calculus, Taylor's theorem gives an approximation of a \(k\)-times differentiable function around a given point by a polynomial of degree \(k\), called the \(k\)-th-order Taylor polynomial. For a smooth function, the Taylor polynomial is the truncation at the order \(k\) of the Taylor series of the function. The first-order Taylor polynomial is the linear approximation of the function, and the second-order Taylor polynomial is often referred to as the quadratic approximation. There are several versions of Taylor's theorem, some giving explicit estimates of the approximation error of the function by its Taylor polynomial.

Taylor's theorem is named after Brook Taylor, who stated a version of it in 1715, although an earlier version of the result was already mentioned in 1671 by James Gregory.

Taylor's theorem is taught in introductory-level calculus courses and is one of the central elementary tools in mathematical analysis. It gives simple arithmetic formulas to accurately compute values of many transcendental functions such as the exponential function and trigonometric functions. It is the starting point of the study of analytic functions, and is fundamental in various areas of mathematics, as well as in numerical analysis and mathematical physics. Taylor's theorem also generalizes to multivariate and vector valued functions. It provided the mathematical basis for some landmark early computing machines: Charles Babbage's difference engine calculated sines, cosines, logarithms, and other transcendental functions by numerically integrating the first 7 terms of their Taylor series.

If a real-valued function \(f(x)\) is differentiable at the point \(x=a\), then it has a linear approximation near this point. This means that there exists a function h₁(x) such that

\[f(x) = f(a) + f'(a)(x - a) + h_1(x)(x - a), \quad \lim_{x \to a} h_1(x) = 0.\]

Here

\[P_1(x) = f(a) + f'(a)(x - a)\]

is the linear approximation of \(f(x)\) for x near the point a, whose graph \(y=P_1(x)\) is the tangent line to the graph \(y=f(x)\) at x = a. The error in the approximation is: \[R_1(x) = f(x) - P_1(x) = h_1(x)(x - a).\]

As x tends to a, this error goes to zero much faster than \((x-a)\), making \(f(x)\approx P_1(x)\) a useful approximation.

For a better approximation to \(f(x)\), we can fit a quadratic polynomial instead of a linear function:

1. Estimate the error for a polynomial P_k(x) of degree k estimating \(f(x)\) on a given interval (a, r, a + r). (Given the interval and degree, we find the error.)
2. Find the smallest degree k for which the polynomial P_k(x) approximates \(f(x)\) to within a given error tolerance on a given interval (a − r, a + r) . (Given the interval and error tolerance, we find the degree.)
3. Find the largest interval (a − r, a + r) on which P_k(x) approximates \(f(x)\) to within a given error tolerance. (Given the degree and error tolerance, we find the interval.)

Condensed: the full section is in Wikipedia.

The precise statement of the most basic version of Taylor's theorem is as follows:

Taylor's theorem, Let \(k \geq 1\) be an integer and let the function \(f : \R \to \R\) be \(k\) times differentiable at the point \(a \in \R\). Then there exists a function \(h_k : \R \to \R\) such that

\[f(x) = \sum_{i=0}^k \frac{f^{(i)}(a)}{i!} (x-a)^i + h_k(x)(x-a)^k,\]

and

\[\lim_{x\to a} h_k(x) = 0.\]

This is called the Peano form of the remainder.

The polynomial appearing in Taylor's theorem is the \(\boldsymbol{k}\)-th order Taylor polynomial

\[P_k(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \cdots + \frac{f^{(k)}(a)}{k!}(x-a)^k\]

of the function \(f\) at the point \(a\). The Taylor polynomial is the unique "asymptotic best fit" polynomial in the sense that if there exists a function \(h_k : \R \to \R\) and a \(k\)-th order polynomial p such that

\[f(x) = p(x) + h_k(x)(x-a)^k, \quad \lim_{x\to a} h_k(x) = 0 ,\]

then \(p = P_k\). Taylor's theorem describes the asymptotic behavior of the remainder term

\[R_k(x) = f(x) - P_k(x),\]

Condensed: the full section is in Wikipedia.

Under stronger regularity assumptions on f there are several precise formulas for the remainder term R_k of the Taylor polynomial, the most common ones being the following.

Mean-value forms of the remainder, Let f : R → R be k + 1 times differentiable on the open interval between \(a\) and \(x\) with f continuous on the closed interval between \(a\) and \(x\). Then

\[R_k(x) = \frac{f^{(k+1)}(\xi_L)}{(k+1)!} (x-a)^{k+1}\]

for some real number \(\xi_L\) between \(a\) and \(x\). This is the Lagrange form of the remainder.

Similarly,

\[R_k(x) = \frac{f^{(k+1)}(\xi_C)}{k!}(x-\xi_C)^k(x-a)\]

for some real number \(\xi_C\) between \(a\) and \(x\). This is the Cauchy form of the remainder.

Both can be thought of as specific cases of the following result: Consider \(p>0\)

\[R_k(x) = \frac{f^{(k+1)}(\xi_S)}{k!}(x-\xi_S)^{k+1-p}\,\frac{(x-a)^p}{p}\] for some real number \(\xi_S\) between \(a\) and \(x\). This is the Schlömilch form of the remainder (sometimes called the Schlömilch-Roche). The choice \(p=k+1\) is the Lagrange form, whilst the choice \(p=1\) is the Cauchy form.

These refinements of Taylor's theorem are usually proved using the mean value theorem, whence the name. Additionally, notice that this is precisely the mean value theorem when \(k=0\). Also other similar expressions can be found. For example, if G(t) is continuous on the closed interval and differentiable with a non-vanishing derivative on the open interval between \(a\) and \(x\), then

\[R_k(x) = \frac{f^{(k+1)}(\xi)}{k!}(x-\xi)^k \frac{G(x)-G(a)}{G'(\xi)}\]

for some number \(\xi\) between \(a\) and \(x\). This version covers the Lagrange and Cauchy forms of the remainder as special cases, and is proved below using Cauchy's mean value theorem. The Lagrange form is obtained by taking \(G(t)=(x-t)^{k+1}\) and the Cauchy form is obtained by taking \(G(t)=t-a\).

The statement for the integral form of the remainder is more advanced than the previous ones, and requires understanding of Lebesgue integration theory for the full generality. However, it holds also in the sense of Riemann integral provided the (k + 1)th derivative of f is continuous on the closed interval [a,x].

Integral form of the remainder, Let \(f^{(k)}\) be absolutely continuous on the closed interval between \(a\) and \(x\). Then

\[R_k(x) = \int_a^x \frac{f^{(k+1)} (t)}{k!} (x - t)^k \, dt.\]

Due to the absolute continuity of f on the closed interval between \(a\) and \(x\), its derivative f exists as an L-function, and the result can be proven by a formal calculation using the fundamental theorem of calculus and integration by parts.

It is often useful in practice to be able to estimate the remainder term appearing in the Taylor approximation, rather than having an exact formula for it. Suppose that f is (k + 1)-times continuously differentiable in an interval I containing a. Suppose that there are real constants q and Q such that

\[q\le f^{(k+1)}(x)\le Q\]

throughout I. Then the remainder term satisfies the inequality

\[q\frac{(x-a)^{k+1}}{(k+1)!}\le R_k(x)\le Q\frac{(x-a)^{k+1}}{(k+1)!},\]

if x > a, and a similar estimate if x < a. This is a simple consequence of the Lagrange form of the remainder. In particular, if

\[|f^{(k+1)}(x)|\le M\]

on an interval I = (a − r,a + r) with some \(r > 0\) , then

Condensed: the full section is in Wikipedia.

Suppose that we wish to find the approximate value of the function \(f(x)=e^x\) on the interval \([-1,1]\) while ensuring that the error in the approximation is no more than 10. In this example we pretend that we only know the following properties of the exponential function:

From these properties it follows that \(f^{(k)}(x)=e^x\) for all \(k\), and in particular, \(f^{(k)}(0)=1\). Hence the \(k\)-th order Taylor polynomial of \(f\) at \(0\) and its remainder term in the Lagrange form are given by

\[P_k(x) = 1+x+\frac{x^2}{2!}+\cdots+\frac{x^k}{k!}, \qquad R_k(x)=\frac{e^\xi}{(k+1)!}x^{k+1},\]

where \(\xi\) is some number between 0 and x. Since e is increasing by (★), we can simply use \(e^x \leq 1\) for \(x \in [-1,0]\) to estimate the remainder on the subinterval \([-1,0]\). To obtain an upper bound for the remainder on \([0,1]\), we use the property e^\xi

\[e^x = 1 + x + \frac{e^\xi}{2}x^2 < 1 + x + \frac{e^x}{2}x^2, \qquad 0 < x\leq 1\]

using the second order Taylor expansion. Then we solve for e to deduce that

\[e^x \leq \frac{1+x}{1-\frac{x^2}{2}} = 2\frac{1+x}{2-x^2} \leq 4, \qquad 0 \leq x\leq 1\]

Condensed: the full section is in Wikipedia.

Let I ⊂ R be an open interval. By definition, a function f : I → R is real analytic if it is locally defined by a convergent power series. This means that for every a ∈ I there exists some r > 0 and a sequence of coefficients c_k ∈ R such that (a − r, a + r) ⊂ I and

\[f(x) = \sum_{k=0}^\infty c_k(x-a)^k = c_0 + c_1(x-a) + c_2(x-a)^2 + \cdots, \qquad |x-a|

In general, the radius of convergence of a power series can be computed from the Cauchy-Hadamard formula

\[\frac{1}{R} = \limsup_{k\to\infty}|c_k|^\frac{1}{k}.\]

This result is based on comparison with a geometric series, and the same method shows that if the power series based on a converges for some b ∈ R, it must converge uniformly on the closed interval \([a-r_b,a+r_b]\), where \(r_b=\left\vert b-a \right\vert\). Here only the convergence of the power series is considered, and it might well be that (a − R,a + R) extends beyond the domain I of f.

The Taylor polynomials of the real analytic function f at a are simply the finite truncations

\[P_k(x) = \sum_{j=0}^k c_j(x-a)^j, \qquad c_j = \frac{f^{(j)}(a)}{j!}\]

Condensed: the full section is in Wikipedia.

The Taylor series of f will converge in some interval in which all its derivatives are bounded and do not grow too fast as k goes to infinity. (However, even if the Taylor series converges, it might not converge to f, as explained below; f is then said to be non-analytic.)

One might think of the Taylor series

\[f(x) \approx \sum_{k=0}^\infty c_k(x-a)^k = c_0 + c_1(x-a) + c_2(x-a)^2 + \cdots\]

of an infinitely many times differentiable function f : R → R as its "infinite order Taylor polynomial" at a. Now the estimates for the remainder imply that if, for any r, the derivatives of f are bounded over (a − r, a + r), then for any order k and for any r > 0 there exists a constant M_k,r > 0 such that

for every x ∈ (a − r,a + r). Sometimes the constants M_k,r can be chosen in such way that M_k,r is bounded above, for fixed r and all k. Then the Taylor series of f converges uniformly to some analytic function

\[\begin{aligned} & T_f:(a-r,a+r)\to\R \\ & T_f(x) = \sum_{k=0}^\infty \frac{f^{(k)}(a)}{k!} \left(x-a\right)^k \end{aligned}\]

(One also gets convergence even if M_k,r is not bounded above as long as it grows slowly enough.)

• The Taylor series of f converges uniformly to the zero function T_f(x) = 0, which is analytic with all coefficients equal to zero.
• The function f is unequal to this Taylor series, and hence non-analytic.
• For any order k ∈ N and radius r > 0 there exists M_k,r > 0 satisfying the remainder bound (★★) above.

Condensed: the full section is in Wikipedia.

Taylor's theorem generalizes to functions f : C → C which are complex differentiable in an open subset U ⊂ C of the complex plane. However, its usefulness is dwarfed by other general theorems in complex analysis. Namely, stronger versions of related results can be deduced for complex differentiable functions f : U → C using Cauchy's integral formula as follows.

Let r > 0 such that the closed disk B(z, r) ∪ S(z, r) is contained in U. Then Cauchy's integral formula with a positive parametrization γ(t) = z + re of the circle S(z, r) with \(t \in [0,2 \pi]\) gives

\[f(z) = \frac{1}{2\pi i}\int_\gamma \frac{f(w)}{w-z}\,dw, \quad f'(z) = \frac{1}{2\pi i}\int_\gamma \frac{f(w)}{(w-z)^2} \, dw, \quad \ldots, \quad f^{(k)}(z) = \frac{k!}{2\pi i}\int_\gamma \frac{f(w)}{(w-z)^{k+1}} \, dw.\]

Here all the integrands are continuous on the circle S(z, r), which justifies differentiation under the integral sign. In particular, if f is once complex differentiable on the open set U, then it is actually infinitely many times complex differentiable on U. One also obtains Cauchy's estimate

\[|f^{(k)}(z)| \leq \frac{k!}{2\pi}\int_\gamma \frac{M_r}{|w-z|^{k+1}} \, dw = \frac{k!M_r}{r^k}, \quad M_r = \max_{|w-c|=r}|f(w)|\]

for any z ∈ U and r > 0 such that B(z, r) ∪ S(c, r) ⊂ U. The estimate implies that the complex Taylor series

\[T_f(z) = \sum_{k=0}^\infty \frac{f^{(k)}(c)}{k!}(z-c)^k\]

Condensed: the full section is in Wikipedia.

The function

\[\begin{aligned} & f : \R \to \R \\ & f(x) = \frac{1}{1+x^2} \end{aligned}\]

is real analytic, that is, locally determined by its Taylor series. This function was plotted above to illustrate the fact that some elementary functions cannot be approximated by Taylor polynomials in neighborhoods of the center of expansion which are too large. This kind of behavior is easily understood in the framework of complex analysis. Namely, the function f extends into a meromorphic function

\[\begin{aligned} & f:\Complex \cup \{\infty\} \to \Complex \cup \{\infty\} \\ & f(z) = \frac{1}{1+z^2} \end{aligned}\]

on the compactified complex plane. It has simple poles at \(z=i\) and \(z=-i\), and it is analytic elsewhere. Now its Taylor series centered at z₀ converges on any disc B(z₀, r) with r < |z − z₀|, where the same Taylor series converges at z ∈ C. Therefore, Taylor series of f centered at 0 converges on B(0, 1) and it does not converge for any z ∈ C with |z| > 1 due to the poles at i and −i. For the same reason the Taylor series of f centered at 1 converges on \(B(1, \sqrt{2})\) and does not converge for any z ∈ C with \(\left\vert z-1 \right\vert>\sqrt{2}\).

A function f: R → R is differentiable at a ∈ R if and only if there exists a linear functional L : R → R and a function h : R → R such that

\[f(\boldsymbol{x}) = f(\boldsymbol{a}) + L(\boldsymbol{x}-\boldsymbol{a}) + h(\boldsymbol{x})\lVert\boldsymbol{x}-\boldsymbol{a}\rVert, \qquad \lim_{\boldsymbol{x}\to\boldsymbol{a}} h(\boldsymbol{x})=0.\]

If this is the case, then \(L = df(\boldsymbol{a})\) is the (uniquely defined) differential of f at the point a. Furthermore, then the partial derivatives of f exist at a and the differential of f at a is given by

\[df( \boldsymbol{a} )( \boldsymbol{v} ) = \frac{\partial f}{\partial x_1}(\boldsymbol{a}) v_1 + \cdots + \frac{\partial f}{\partial x_n}(\boldsymbol{a}) v_n.\]

Introduce the multi-index notation

\[|\alpha| = \alpha_1+\cdots+\alpha_n, \quad \alpha!=\alpha_1!\cdots\alpha_n!, \quad \boldsymbol{x}^\alpha=x_1^{\alpha_1}\cdots x_n^{\alpha_n}\]

for α ∈ N and x ∈ R. If all the \(k\)-th order partial derivatives of f : R → R are continuous at a ∈ R, then by Clairaut's theorem, one can change the order of mixed derivatives at a, so the short-hand notation

Condensed: the full section is in Wikipedia.

Using notations of the preceding section, one has the following theorem.

Multivariate version of Taylor's theorem, Let f : R → R be a k-times continuously differentiable function at the point a ∈ R. Then there exist functions h_α : R → R, where \(|\alpha|=k,\) such that

\[\begin{aligned} & f(\boldsymbol{x}) = \sum_{|\alpha|\leq k} \frac{D^\alpha f(\boldsymbol{a})}{\alpha!} (\boldsymbol{x}-\boldsymbol{a})^\alpha + \sum_{|\alpha|=k} h_\alpha(\boldsymbol{x})(\boldsymbol{x}-\boldsymbol{a})^\alpha, \\ & \mbox{and}\quad \lim_{\boldsymbol{x}\to \boldsymbol{a}}h_\alpha(\boldsymbol{x})=0. \end{aligned}\]

If the function f : R → R is k + 1 times continuously differentiable in a closed ball \(B = \{ \mathbf{y} \in \R^n : \left\|\mathbf{a}-\mathbf{y}\right\| \leq r\}\) for some \(r > 0\), then one can derive an exact formula for the remainder in terms of (k+1)-th order partial derivatives of f in this neighborhood. Namely,

\[\begin{aligned} & f( \boldsymbol{x} ) = \sum_{|\alpha|\leq k} \frac{D^\alpha f(\boldsymbol{a})}{\alpha!} (\boldsymbol{x}-\boldsymbol{a})^\alpha + \sum_{|\beta|=k+1} R_\beta(\boldsymbol{x})(\boldsymbol{x}-\boldsymbol{a})^\beta, \\ & R_\beta( \boldsymbol{x} ) = \frac{|\beta|}{\beta!} \int_0^1 (1-t)^{|\beta|-1}D^\beta f \big(\boldsymbol{a}+t( \boldsymbol{x}-\boldsymbol{a} )\big) \, dt. \end{aligned}\]

In this case, due to the continuity of (k+1)-th order partial derivatives in the compact set B, one immediately obtains the uniform estimates

\[\left|R_\beta(\boldsymbol{x})\right| \leq \frac{1}{\beta!} \max_{|\alpha|=|\beta|} \max_{\boldsymbol{y}\in B} |D^\alpha f(\boldsymbol{y})|, \qquad \boldsymbol{x}\in B.\]

For example, the third-order Taylor polynomial of a smooth function \(f:\mathbb R^2\to\mathbb R\) is, denoting \(\boldsymbol{x}-\boldsymbol{a}=\boldsymbol{v}\),

\[\begin{aligned} P_3(\boldsymbol{x}) = f ( \boldsymbol{a} ) + {} &\frac{\partial f}{\partial x_1}( \boldsymbol{a} ) v_1 + \frac{\partial f}{\partial x_2}( \boldsymbol{a} ) v_2 + \frac{\partial^2 f}{\partial x_1^2}( \boldsymbol{a} ) \frac {v_1^2}{2!} + \frac{\partial^2 f}{\partial x_1 \partial x_2}( \boldsymbol{a} ) v_1 v_2 + \frac{\partial^2 f}{\partial x_2^2}( \boldsymbol{a} ) \frac{v_2^2}{2!} \\ & + \frac{\partial^3 f}{\partial x_1^3}( \boldsymbol{a} ) \frac{v_1^3}{3!} + \frac{\partial^3 f}{\partial x_1^2 \partial x_2}( \boldsymbol{a} ) \frac{v_1^2 v_2}{2!} + \frac{\partial^3 f}{\partial x_1 \partial x_2^2}( \boldsymbol{a} ) \frac{v_1 v_2^2}{2!} + \frac{\partial^3 f}{\partial x_2^3}( \boldsymbol{a} ) \frac{v_2^3}{3!} \end{aligned}\]

Let

\[h_k(x) = \begin{cases} \frac{f(x) - P(x)}{(x-a)^k} & x\not=a\\ 0&x=a \end{cases}\]

where, as in the statement of Taylor's theorem,

\[P(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \cdots + \frac{f^{(k)}(a)}{k!}(x-a)^k.\]

It is sufficient to show that

\[\lim_{x\to a} h_k(x) =0.\]

The proof here is based on repeated application of L'Hôpital's rule. Note that, for each \(j=0,1,...,k-1\), \(f^{(j)}(a)=P^{(j)}(a)\). Hence each of the first \(k-1\) derivatives of the numerator in \(h_k(x)\) vanishes at \(x=a\), and the same is true of the denominator. Also, since the condition that the function \(f\) be \(k\) times differentiable at a point requires differentiability up to order \(k-1\) in a neighborhood of said point (this is true, because differentiability requires a function to be defined in a whole neighborhood of a point), the numerator and its \(k-2\) derivatives are differentiable in a neighborhood of \(a\). Clearly, the denominator also satisfies said condition, and additionally, doesn't vanish unless \(x=a\), therefore all conditions necessary for L'Hôpital's rule are fulfilled, and its use is justified. So

Condensed: the full section is in Wikipedia.