next_inactive up previous


AMTH142 Lecture 6

Numerical Approximations


Contents


Reference: For the remainder of the unit the best reference is -- Michael Heath, Scientific Computing: An Introductory Survey, 2nd edition, McGraw Hill, 2002, IBSN: 0-07-112229-X. The books's home page: http://www.cse.uiuc.edu/heath/scicomp/index.html includes lecture notes based on the book.


Reading: Heath §1.1 and §1.2

Computational Errors

Computational Problems

Finding a computational solution to a problem usually involves a number of steps:
  1. Formulation of a mathematical model of the problem.
  2. Development of a numerical method to solve the mathematical problem.
  3. Implementation of the numerical method, for example in Scilab.
  4. Running the numerical method to obtain the result.
  5. Analysis and interpretation of the result, repeating previous steps if necessary.

Sources of Error

The first source of error occurs before the computation begins. These are the errors that arise from the mathematical model iself by simplifying or omitting features of the problem. These may be referred to as modelling errors. The next source of error are data errors:
  1. Empirical Measurements. Many computations have as input experimental data which, by their nature, have limited accuracy.
  2. Previous Computations. Often the input data to to a computation are produced by previous computations whose results were only approximate.
The third sort of error is computational error:
  1. Truncation Error. In order to turn a mathematical problem into a computational problem it is usually necessary to make a number of mathematical approximations, for example using a finite number of terms to approximate an infinite series.
  2. Rounding Error. Numerical calculations are limited in accuracy by the finite precision of computer arithmetic.

Example

Computing the area of the surface of the earth by the formula

$\displaystyle A = 4 \pi r^2 $

involves the following approximations:
  1. The earth is modelled by a sphere which is an approximation to its true shape.
  2. The value for the radius of the earth ( $ \approx 6370$   km) is based on empirical measurements.
  3. The value used for $ \pi$ has to be an approximation to its true value.
  4. The actual calculation will involve rounding error.

Absolute and Relative Error

These are defined by:

Absolute error$\displaystyle \quad =$    Approximate value$\displaystyle -$   True value (1)
Relative error$\displaystyle \quad =$ $\displaystyle \quad \frac{\text{Absolute error}}{\text{True value}}$ (2)

Often the relative error is represented as a percentage. A useful way to think of relative error is via the expression

Approximate value$\displaystyle \quad =$   True value$\displaystyle \times (1 +$   Relative error$\displaystyle )$ (3)

Data Error and Computational Error

Consider the problem of computing the value of some function $ f$ at a point $ x$. Suppose we have to work with an approximation $ \hat{x}$ to the data $ x$ and with an approximation $ \hat{f}$ to the function $ f$. Then
Total error$\displaystyle \quad =$ $\displaystyle \hat{f}(\hat{x}) - f(x)$    
$\displaystyle =$ $\displaystyle (\hat{f}(\hat{x}) - f(\hat{x}))$ $\displaystyle + \quad (f(\hat{x}) - f(x))$  
$\displaystyle =$ Computational error $\displaystyle +$   Propagated data error  

The first term on the left is the difference between the exact and approximate function values for the same input and can therefore be considered as computational error in evaluating the function $ f$. The second term is the difference in the values of the exact function $ f$ due to errors in $ x$ and can therefore be considered as propagated data error. Note that the propagated data error does not depend on the how we choose to approximate the function $ f(x)$, in other words the propagated data error does not depend on method used to solve the problem.

Example

Consider the following very crude approximation:

$\displaystyle \sin(\frac{\pi}{8}) \approx \sin(\frac{3}{8}) \approx \frac{3}{8} = 0.37500 $

We have used two approximations here:

$\displaystyle \pi$ $\displaystyle \approx 3$    
$\displaystyle \sin x$ $\displaystyle \approx x$    

Using the notation above, we have:

$\displaystyle x$ $\displaystyle = \frac{\pi}{8}$ $\displaystyle f(x)$ $\displaystyle = \sin(x)$    
$\displaystyle \hat{x}$ $\displaystyle = \frac{3}{8}$ $\displaystyle \hat{f}(x)$ $\displaystyle = x$    

The correct value is

$\displaystyle \sin(\frac{\pi}{8}) = 0.38268 $

and the total error is

$\displaystyle \hat{f}(\hat{x}) - f(x) = \frac{3}{8} - \sin(\frac{\pi}{8}) = 0.37500 - 0.38268 = -0.00768 $

The propogated data error is

$\displaystyle f(\hat{x}) - f(x) = \sin (\frac{3}{8}) - \sin (\frac{\pi}{8}) = -0.01641 $

and the computational error is

$\displaystyle \hat{f}(\hat{x}) - f(\hat{x}) = \frac{3}{8} - \sin(\frac{3}{8}) = 0.00873 $

with the sum of these

$\displaystyle -0.01641 + 0.00873 = -0.00768 $

equal to the total error.

Taylor Series

Taylor series are a very common and important type of approximation. Taylor's theorem states:

$\displaystyle f(x+h)$ $\displaystyle = \sum_{n=0}^{\infty} \frac{1}{n!} f^{(n)}(x) h^n$    
  $\displaystyle = f(x) + f'(x) h + \frac{1}{2} f''(x) h^2 + \frac{1}{6} f'''(x) h^3 + \frac{1}{24} f^{(4)}(x) h^4 + \dots$    

Notes:
  1. Taylor's theorem is only true, in general, for ``small'' values of $ h$. The infinite series may diverge for large $ h$.
  2. There are some technical conditions on the function $ f(x)$ in order for Taylor's theorem to hold, e.g. the function $ f(x)$ must be infinitely differentiable. These are usually satisfied in practice.
  3. The main use of Taylor's theorem is to approximate a function by the first few terms of its Taylor series, i.e.

    $\displaystyle f(x+h) \approx f(x) + f'(x) h + \frac{1}{2} f''(x) h^2 + \dots + \frac{1}{n!} f^{(n)} h^n $

  4. For fixed number of terms in a truncated Taylor series, the accuracy of the approximation decreases as $ h$ increases. The accuracy generally increases as the number of terms is increased.

Examples

Here are a few examples of Taylor series:
  1. $ f(x) = \sin x $ at $ x = 0$: We need to evaluate the derivatives of $ \sin x$ at $ x = 0$:

    $\displaystyle f(0)$ $\displaystyle = \sin(0) = 0$ $\displaystyle f^{(4)}(0)$ $\displaystyle = \sin(0) = 0$    
    $\displaystyle f'(0)$ $\displaystyle = \cos(0) = 1$ $\displaystyle f^{(5)}(0)$ $\displaystyle = \cos(0) = 1$    
    $\displaystyle f''(0)$ $\displaystyle = - \sin(0) = 0$ $\displaystyle f^{(6)}(0)$ $\displaystyle = - \sin(0) = 0$    
    $\displaystyle f'''(0)$ $\displaystyle = - \cos(0) = -1$ $\displaystyle f^{(7)}(0)$ $\displaystyle = - \cos(0) = -1$    

    to give the approximation

    $\displaystyle \sin (h) \approx h - \frac{1}{6} h^3 + \frac{1}{120} h^5 - \frac{1}{5040} h^7 + \dots $

  2. $ f(x) = \cos x $ at $ x = 0$: A similar calculation gives

    $\displaystyle \cos (h) \approx 1 - \frac{1}{2} h^2 + \frac{1}{24} h^4 - \frac{1}{720} h^6 + \dots $

  3. $ f(x) = e^x$ at $ x = 0$: All the derivatives of $ f(x) = e^x$ at $ x = 0$ are equal to 1, so

    $\displaystyle e^h \approx 1 + h +\frac{1}{2} h^2 + \frac{1}{6} h^3 + \frac{1}{24} h^4 + \dots $

Approximation of Functions

The most direct application is to approximate functions by their truncated Taylor series. The following example examines the approximations to $ \sin x$ by its Taylor series given above:

Example:

We will use Scilab to compare the approximation of $ \sin x$ by truncated Taylor series containing terms up to $ h^3$, $ h^5$ and $ h^7$ respectively over the interval $ [-\pi,\pi]$: 
-->h = linspace(-%pi, %pi, 1000);         
 
-->y = sin(h);                            
 
-->y3 = h - h.^3/6;                       
 
-->y5 = h - h.^3/6 + h.^5/120;            
 
-->y7 = h - h.^3/6 + h.^5/120 - h.^7/5040;
Note the use of the .^ operator. (Make sure you understand what it does and why it is needed here, see §2.4 of Practical 2.) We can compare the approximations graphically (do not clear the graphics window between successive plots): 
-->plot2d(h, y3)
 
-->plot2d(h, y5)
 
-->plot2d(h, y7)
 
-->plot2d(h, y)
We can also examine the error in the best approximation: 
-->plot2d(h,y-y7)

Propagated Data Error

Taylor series approximations are also useful as a theoretical tool. The simple approximation

$\displaystyle f(x+h) \approx f(x) + f'(x) h $

is particularly common. This amounts to approximating $ f(x+h)$ by the tangent to $ f(x)$ at $ x$. Earlier in this lecture we came across the propagated data error

Propagated data error$\displaystyle \quad = \quad f(\hat{x}) - f(x) $

where $ \hat{x}$ is an approximation to $ x$. Let $ \hat{x} = x + h$, then

Propagated data error$\displaystyle \quad$ $\displaystyle = f(\hat{x}) - f(x)$    
  $\displaystyle = f(x+h) - f(x)$    
  $\displaystyle \approx (f(x) + f'(x)h) - f(x)$    
  $\displaystyle = f'(x) h$    

This gives a useful estimate for the propagated data error for small $ h$.

Example:

Returning to an earlier example, we calculated the propagated data error in $ \sin(\frac{\pi}{8})$ due to the approximation $ \frac{\pi}{8} \approx \frac{3}{8}$ as $ -0.01641$. Here we have $ f(x) = \sin x $, $ f'(x) = \cos x$ and $ h = \frac{3}{8} - \frac{\pi}{8} = -0.01770$. Using our approximation for the propagated data error gives:

Propagated data error$\displaystyle \quad$ $\displaystyle \approx f'(x) h$    
  $\displaystyle = \cos(\frac{\pi}{8}) h$    
  $\displaystyle = -0.01635$    

About this document ...

Error in the 7th order Taylor series approximation.

This document was generated using the LaTeX2HTML translator Version 2K.1beta (1.61)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -split +0 lecture.tex

The translation was initiated by Applied Mathematics 142 on 2002-08-13

next_inactive up previous
Applied Mathematics 142 2002-08-13