Notes for Calculus

Appendix: Taylor's Theorem

Author: Kenneth, S.K. Cheng

Notes for Calculus
- Appendix: Taylor's Theorem
  - Author: Kenneth, S.K. Cheng
- Table of Contents

Introduction

Recall that we made the following approximation

\Delta f(x_0) \approx df(x_0)

replacing the increment

\Delta f(x_0) = f(x_0 + \Delta x) - f(x_0)

of a function differentiable at $x_0$ by its differential

df(x_0) = f'(x_0) \Delta x

at x_0. We now estimate the error committed in making the approximation, and then develop a series of sharper approximations involving "higher-order terms," i.e., terms proportional to $(\Delta x)^2$ , $(\Delta x)^3$ , etc. In particular, this will allow us to handle the case $df(x_0)=0$ , where the approximation above breaks down.

We begin by refining the mean value theorem:

Theorem: Let $f$ be a function with a finite second derivative $f''$ in an interval containing the points $x_0$ and $x_0+\Delta x$ . Then the increment of $f$ at $x_0$ can be written in the form

\Delta f(x_0) = f(x_0 + \Delta x) - f(x_0) = \Delta x f'(x_0) + \frac{1}{2} (\Delta x)^2 f''(x_0 + \theta \Delta x)

where $\theta$ is a number between 0 and 1. Equaivalently, if $f$ has finite second derivative in a neightborhood of $x_0$ , then the value of $f$ at any point $x$ of the neighborhood is given by

f(x) = f(x_0) + (x - x_0) f'(x_0) + \frac{1}{2} (x - x_0)^2 f''(\xi),

where $\xi$ is a number between $x_0$ and $x$ .

Proof: Let

\varphi(x) = f(x_0 + x) - f(x_0)-f'(x_0)x, \quad g(x)=x^2.

Then it follows from Cauchy's theorem applied to the functions $\varphi$ and $g$ in the interval $[0,\Delta x]$ or $[\Delta x, 0]$ , depending on the sign of $\Delta x$ , that

\frac{\varphi'(c)}{2c} = \frac{\varphi(\Delta x) - \varphi(0)}{g(\Delta x) -g(0)} = \frac{\varphi(\Delta x)}{g(\Delta x)},

where $c$ lies between $0$ and $\Delta x$ . But, by the mean value theorem,

\frac{\varphi'(c)}{c} = \frac{f'(x_0+c)-f'(x_0)}{c} = f''(x_0 + \theta_1 c),

where $0<\theta_1<1$ . Hence

\varphi(\Delta x) = \frac{1}{2} \Delta x^2 f''(x_0 + \theta_1 \Delta x),

i.e.,

f(x_0+\Delta x)-f(x_0) = \Delta x f'(x_0)+\frac{1}{2} (\Delta x)^2 f''(x_0 + \theta_1 c).

The numbers $\theta_1$ and $\frac{c}{\Delta x}$ are both positive and lie in the interval $(0,1)$ , and thence the same is true of

\theta = \frac{\theta_1 c}{\Delta x}.

In other words, $\theta_1 c = \theta \Delta x$ , where $\theta$ lies between 0 and 1. The second part of the theorem follows by setting $x=x_0+\Delta x$ in the first part.

Taylor's Theorem

Theorem (Taylor's Theorem): Let $f$ be a function with a finite $(n+1)$st derivative in an interval containing the points $x_0$ and $x_0+\Delta x$ . Then the increment of $f$ at $x_0$ can be written in the form

\Delta f(x_0) = f(x_0 + \Delta x) - f(x_0) = \Delta x f'(x_0) + \frac{1}{2} (\Delta x)^2 f''(x_0) + \cdots + \frac{1}{n!} (\Delta x)^n f^{(n)}(x_0) + \frac{1}{(n+1)!} (\Delta x)^{n+1} f^{(n+1)}(x_0 + \theta \Delta x)

where $\theta$ is a number between 0 and 1. Equivalently, if $f$ has finite $(n+1)$st derivative in a neighborhood of $x_0$ , then the value of $f$ at any point $x$ of the neighborhood is given by Taylor's formula

f(x) = f(x_0) + (x - x_0) f'(x_0) + \frac{1}{2} (x - x_0)^2 f''(x_0) + \cdots + \frac{1}{n!} (x - x_0)^n f^{(n)}(x_0) + \frac{1}{(n+1)!} (x - x_0)^{n+1} f^{(n+1)}(\xi),

where usually the last term is written as

R_n(x) = \frac{1}{(n+1)!} (x - x_0)^{n+1} f^{(n+1)}(\xi),

and is called the remainder of the approximation. The variable $\xi$ lies between $x_0$ and $x$ .

Proof: The proof is quite constructive. It is enough to show that the Remainder $R_n(x)$ holds if we set

R_n(x) = f(x) - \left[ f(x_0) + (x - x_0) f'(x_0) + \frac{1}{2} (x - x_0)^2 f''(x_0) + \cdots + \frac{1}{n!} (x - x_0)^n f^{(n)}(x_0) \right]

Differentiating $R_n(x)$ $n$ times with respect to $x$ gives

\begin{aligned} R_n'(x) &= f'(x) - f'(x_0) - (x - x_0) f''(x_0) - \cdots - \frac{1}{(n-1)!} (x - x_0)^{n-1} f^{(n)}(x_0) \\ R_n''(x) &= f''(x) - f''(x_0) - (x - x_0) f^{(3)}(x_0) - \cdots - \frac{1}{(n-2)!} (x - x_0)^{n-2} f^{(n)}(x_0) \\ &\vdots \\ R_n^{(n)}(x) &= f^{(n)}(x) - f^{(n)}(x_0) \end{aligned}

Therefore, it implies that

R_n(x_0) = R_n'(x_0) = \cdots = R_n^{(n)}(x_0) = 0.

Applyin Cauchy's theorem to the function $R_n(x)$ and $(x-x_0)^{n+1}$ in the interval $[x_0, x]$ or $[x, x_0]$ gives

\frac{R_n(x)}{(x-x_0)^{n+1}} = \frac{R_n(x) - R_n(x_0)}{(x-x_0)^{n+1} - 0} = \frac{R_n'(\xi_1)}{(n+1)!},

where $\xi_1$ lies between $x_0$ and $x$ . Applying Cauchy's theorem to the function $R_n'(x)$ and $(n+1)(\xi_1-x_0)^n$ in the interval $[x_0, \xi_1]$ or $[\xi_1, x_0]$ gives

\frac{R_n'(x)}{(n+1)(\xi_1-x_0)^n} = \frac{R_n'(\xi_1) - R_n'(x_0)}{(n+1)(\xi_1-x_0)^n - 0} = \frac{R_n''(\xi_2)}{(n+1)n(\xi_2-x_0)^{n-1}},

where $\xi_2$ lies between $x_0$ and $\xi_1$ . Repeating this process $n$ times gives

\frac{R_n(x)}{(x-x_0)^{n+1}} = \frac{R_n^{(n)}(\xi)}{(n+1)!(\xi_n-x_0)} = \frac{R^{(n)}(\xi_n)-R^{(n)}(x_0)}{(n+1)!(\xi_n-x_0)},

where $\xi_n$ lies between $x_0$ and $x$ . Finally by applying the mean value theorem, obtaining

\frac{R_n(x)}{(x-x_0)^{n+1}} =\frac{(\xi_n-x_0)R^{(n+1)}(\xi)}{(n+1)!(\xi_n-x_0)}=\frac{R^{(n+1)}(\xi)}{(n+1)!},

where $\xi$ lies between $x_0$ and $\xi_n$ and hence between $x_0$ and $x$ . But

R^{(n+1)}(x) = f^{(n+1)}(x),

and hence

R_n(x) = \frac{1}{(n+1)!} (x - x_0)^{n+1} f^{(n+1)}(\xi).

which completes the proof.

Example: Find the Taylor series of the function $f(x) = e^x$ at $x=0$ .

Solution: We have

\begin{aligned} f'(x) &= e^x, \\ f''(x) &= e^x, \\ f'''(x) &= e^x, \\ &\vdots \\ f^{(n)}(x) &= e^x. \end{aligned}

Therefore, the Taylor series of $f(x)$ at $x=0$ is

\begin{aligned} f(x) &= f(0) + x f'(0) + \frac{1}{2} x^2 f''(0) + \cdots + \frac{1}{n!} x^n f^{(n)}(0) + \frac{1}{(n+1)!} x^{n+1} f^{(n+1)}(\xi) \\ &= 1 + x + \frac{1}{2} x^2 + \cdots + \frac{1}{n!} x^n + \frac{1}{(n+1)!} x^{n+1} e^{\xi} \\ &= \sum_{k=0}^n \frac{1}{k!} x^k + \frac{1}{(n+1)!} x^{n+1} e^{\xi}. \end{aligned}

where $\xi$ lies between 0 and $x$ . The remainder terms vanish as $n\to\infty$ and the series converges to $e^x$ for all $x$ .

To illustrate the idea, one may think of the following figure:

Example2 — Figure 1: The Taylor's Series for Exponential Function

It reveals that more terms in the series give a better approximation to the function.

Useful Formulas for Taylor's Series

There some useful formulas for Taylor's series that are worth mentioning:

Taylor's Series for $e^x$ : The Taylor series for $e^x$ at $x=0$ is

e^x = \sum_{k=0}^{\infty} \frac{1}{k!} x^k.

Taylor's Series for $\sin x$ : The Taylor series for $\sin x$ at $x=0$ is

\sin x = \sum_{k=0}^{\infty} \frac{(-1)^k}{(2k+1)!} x^{2k+1}.

Taylor's Series for $\cos x$ : The Taylor series for $\cos x$ at $x=0$ is

\cos x = \sum_{k=0}^{\infty} \frac{(-1)^k}{(2k)!} x^{2k}.

Taylor's Series for $\ln(1+x)$ : The Taylor series for $\ln(1+x)$ at $x=0$ is

\ln(1+x) = \sum_{k=1}^{\infty} \frac{(-1)^{k-1}}{k} x^k.

Taylor's Series for $\frac{1}{1-x}$ : The Taylor series for $\frac{1}{1-x}$ at $x=0$ is

\frac{1}{1-x} = \sum_{k=0}^{\infty} x^k.

Further Applications of Taylor's Theorem

Taylor's theorem is a powerful tool in approximating functions. It is used in many areas of mathematics, physics, and engineering. Here are some applications of Taylor's theorem:

Error Analysis: Taylor's theorem is used to estimate the error in numerical methods. For example, in numerical analysis, the error in approximating a function by a polynomial can be estimated using Taylor's theorem.
Physics: Taylor's theorem is used in physics to approximate the behavior of physical systems. For example, in classical mechanics, Taylor's theorem is used to approximate the motion of a particle.