INTRODUCTION TO NUMERICAL ANALYSIS

Dr. Gabriel Obed Fosu

Department of Computer Science & Information Technology
Google Scholar: [Link]
ResearchGate ID: [Link]

Dr. Gabby Introduction To Numerical Analysis 1 / 40

Lecture Outline

1 Definition
2 Significant digits
3 Accuracy and Precision
4 Rounding and Chopping
5 Absolute and Relative Errors
6 Review of Calculus
7 Taylor and Maclaurin Series

Dr. Gabby Introduction To Numerical Analysis 2 / 40

 Definition

Definition

Definition
Numerical analysis and method is the area of mathematics and computer science
that creates, analyses, and implements algorithms for solving numerically the
problems of continuous mathematics.

1 Such problems originate generally from real-world applications of algebra,

geometry, and calculus, and they involve variables which vary continuously.
2 The overall goal of the field is the design and analysis of techniques to give
approximate but not exact solutions to problems that do may/may not have
analytical solutions.

Dr. Gabby Introduction To Numerical Analysis 4 / 40

 Definition

Definition

1 When an analytical solution does not exist, numerical techniques are

employed to solving hard problems.
2 There are times that an analytical solution to a problem may exist but might
be cumbersome and time wasting to find such solution. In such instances, it
is better to resort to numerical solutions.

Most numerical solutions are iterative.

This implies that a sequence of approximate solution is obtained by repeating a
given procedure. These solutions are implemented using computer programs.

Dr. Gabby Introduction To Numerical Analysis 5 / 40

 Definition

Example
Analytically we can find the roots of
x2 − 1 = 0
as
x = ±1

Since the highest degree of x is two, then we have two roots.

Example
Find all the five roots of the problem
x5 − 1 = 0

It will be very difficult to find analytical (exact) solution to this problem, hence it
would require that we find the approximate solution to the polynomial function
using numerical methods.
Dr. Gabby Introduction To Numerical Analysis 6 / 40
 Definition

Method vs. Algorithm vs. Implementation

1 Method: a general (mathematical) framework describing the solution process

2 Algorithm: a detailed description of executing the method
3 Implementation: a particular instantiation of the algorithm

We use a pseudocode to describe the algorithms. This pseudocode specifies the

form of the input to be supplied and the form of the desired output.

Not all numerical procedures give satisfactory output for arbitrarily chosen input.
As a consequence, a stopping technique independent of the numerical technique
is incorporated into each algorithm to avoid infinite loops.

Dr. Gabby Introduction To Numerical Analysis 7 / 40

 Definition

Numerical focus
People who employ numerical methods for solving problems have the following concerns:
Accuracy
Very few of our computations will yield the exact answer to a problem, so we will have to
understand how much error is made, and how to control (or even diminish) that error. That
is the difference between the approximate and exact solutions.

Efficiency
How fast and cheap (memory) can we compute a solution? Does the algorithm take
an excessive amount of computer time? This might seem to be an odd question to
concern ourselves with. The numerical analyst prefers algorithms that has faster rate
of convergence.

Completeness
Is the solution unique or do other solutions exist excluding your obtained approximate
solution.
Dr. Gabby Introduction To Numerical Analysis 8 / 40
 Definition

Numerical focus

Stability
1 Does the method produce similar results for similar data?
2 We are interested in choosing methods that will produce dependably accurate
results for a wide range of problems.
3 One criterion we will impose on an algorithm whenever possible is that small
changes in the initial data produce correspondingly small changes in the final
results. An algorithm that satisfies this property is called stable;
4 otherwise the method is unstable, and unstable methods tend to produce
unreliable results.
5 Some algorithms are stable only for certain choices of initial data. These are
called conditionally stable.

Dr. Gabby Introduction To Numerical Analysis 9 / 40

 Significant digits

Significant Digits

Definition
There are digits beginning with the leftmost non-zero digit and ending with the
rightmost correct digit, including final zeros that are exact.
1 All non-zero digits are considered significant. For example 91 has two
significant figures, likewise 123.45 has five significant figures.
2 Zeros appearing anywhere between two non-zero digits are significant.
Example 101.1203 has seven significant figures.
3 Leading zeros are not significant. Example 0.00053 has two significant
figures.
4 Trailing zeros in a number containing a decimal point are significant. Example
12.2300 has six significant figures.
5 Again 0.0001200 has four significant figures (the zeros before 1 are not
significant).
Dr. Gabby Introduction To Numerical Analysis 11 / 40
 Accuracy and Precision

Accuracy and Precision

Definition
Accurate to n decimal places means that you can trust n digits to the right of the
decimal place.
Accurate to n significant digits means that you can trust a total of n digits as being
meaningful beginning with the leftmost nonzero digit.

Example
12.356 has three decimal places but five significant figures.

Dr. Gabby Introduction To Numerical Analysis 13 / 40

 Rounding and Chopping

Rounding and Chopping

Definition
We say that a number x is chopped to n digits or figures when all digits that follow
the nt h digit are discarded and none of the remaining n digits are changed.
Conversely, x is rounded to n digits or figures when x is replaced by an n -digit
number that approximates x with minimum error.

Example
The results of rounding some three-decimal numbers to two digits are
0.217 ≈ 0.22, 0.365 ≈ 0.36, 0.475 ≈ 0.48, 0.592 ≈ 0.59, (1)
while chopping them gives
0.217 ≈ 0.21, 0.365 ≈ 0.36, 0.475 ≈ 0.47, 0.592 ≈ 0.59. (2)

Dr. Gabby Introduction To Numerical Analysis 15 / 40

 Absolute and Relative Errors

Absolute and Relative Errors

Suppose that α and β are two numbers, of which one is regarded as an

approximation to the other. The error of β as an approximation to α is
α−β (3)
that is, the error equals the exact value minus the approximate value.

Definition
The absolute error of β as an approximation to α is
|α − β| (4)
The relative error of β as an approximation to α is
|α − β|
(5)
|α|

Dr. Gabby Introduction To Numerical Analysis 17 / 40

 Absolute and Relative Errors

Generally
absolute error = |exact value - approximate value| (6)
and
|exact value - approximate value|
relative error = (7)
|exact value|

For practical reasons, the relative error is usually more meaningful than the
absolute error.
Example
If x = 0.00347 is rounded to x̃ = 0.0035, what is its number of significant digits. Again
find the absolute error, and relative error. Interpret the results.

Dr. Gabby Introduction To Numerical Analysis 18 / 40

 Absolute and Relative Errors

Solution
x̃ = 0.0035 has two significant digits.

absolute error = |exact value - approximate value| (8)

= |0.00347 − 0.0035| (9)
= | − 0.00003| (10)
= 0.00003 (11)

|exact value - approximate value| 0.00003

relative error = = = 0.008646 (12)
|exact value| |0.00347|
Clearly, the relative error is a better indication of the number of significant digits
than the absolute error

Dr. Gabby Introduction To Numerical Analysis 19 / 40

 Review of Calculus

Differentiable Functions

Definition
Assume that f (x) is defined on an open interval containing x 0 . Then f is said to
be differentiable at x 0 if
f (x) − f (x 0 )
lim (13)
x→x 0 x − x0
exists. When this limit exists, it is denoted by f ′ (x 0 ) called the derivative of f at x 0 .

An equivalent way to express this limit is to use the h -increment notation:

f (x 0 + h) − f (x 0 )
lim = f ′ (x 0 ) (14)
h→0 h

Dr. Gabby Introduction To Numerical Analysis 21 / 40

 Taylor and Maclaurin Series

Taylor Series

1 Most students will have encountered infinite series (particularly Taylor series)
in their study of calculus.
2 Consequently, this section is particularly important for numerical analysis, and
deserves careful study.
3 Once students are well grounded with a basic understanding of Taylor series
we can proceed to study the fundamentals of numerical methods with better
comprehension.

Definition
Taylor series is a representation of a function as an infinite sum of terms that are
calculated from the values of the function’s derivatives at a single point.

Dr. Gabby Introduction To Numerical Analysis 23 / 40

 Taylor and Maclaurin Series

Some examples include

∞ xn x2 x3 x4
ex =
P
1 = 1+x + + + +...
n=0 n! 2! 3! 4!
∞ (−1)n x 2n
P x2 x4 x6
2 cos x = = 1− + − +...
n=0 (2n)! 2! 4! 6!
∞ (−1)n x 2n+1
P x3 x5 x7
3 sin x = =x− + − +...
n=0 (2n + 1)! 3! 5! 7!
∞ x 2n
P x2 x4 x6
4 cosh x = = 1+ + + +...
n=0 (2n)! 2! 4! 6!
∞
P x 2n+1 x3 x5 x7
5 sinh x = =x+ + + +...
n=0 (2n + 1)! 3! 5! 7!
1 ∞
xn = 1 + x + x2 + x3 + · · · ,
P
6 = |x| < 1
1 − x n=0
∞ xn x2 x3 x4
(−1)n−1
P
7 ln(1 + x) = =x− + − +··· , −1 < x <≤ 1
n=1 n 2 3 4
Dr. Gabby Introduction To Numerical Analysis 24 / 40
 Taylor and Maclaurin Series

Example
Use five terms of the Taylor series to approximate ln(1.1).

Solution
Taking x = 0.1, then the first five terms of the series for l n(1 + x) gives us
0.01 0.001 0.0001 0.00001
ln(1.1) ≈ 0.1 − + − + = 0.095310333 . . .
2 3 4 5
Compute ln(1.1) directly on your calculator and compare your answer.
This value is correct to six decimal places of accuracy.

Dr. Gabby Introduction To Numerical Analysis 25 / 40

 Taylor and Maclaurin Series

Taylor Series

Taylor series of f at the point c

Generally, the Taylor series for a function f about the point c is given as:

f ′′ (c) f ′′′ (c)

f (x) ≈ f (c) + f ′ (c)(x − c) + (x − c)2 + (x − c)3 + · · · (15)
2! 3!
∞ f (n) (c)
(x − c)n
X
≈ (16)
n=0 n!

f ′ , f ′′ , · · · f (n) are the derivatives of the function f . Again the values of these
successive derivatives should exist at the point c .

Dr. Gabby Introduction To Numerical Analysis 26 / 40

 Taylor and Maclaurin Series

Maclaurin Series

In the special case where c = 0, then we obtain the Maclaurin series given by:

f ′′ (0) 2 f ′′′ (0) 3

f (x) ≈ f (0) + f ′ (0)x + x + x +··· (17)
2! 3!
∞ f (n) (0)
xn
X
≈ (18)
n=0 n!

Example
What is the Taylor series of the function
f (x) = 3x 5 − 2x 4 + 15x 3 + 13x 2 − 12x − 5
at the point c = 2?

Dr. Gabby Introduction To Numerical Analysis 27 / 40

 Taylor and Maclaurin Series

Solution
To compute the coefficients in the series, we need the numerical values of f (n) (2)
for n ≥ 0. Here are the details of the computation:

f (x) = 3x 5 − 2x 4 + 15x 3 + 13x 2 − 12x − 5 f (2) = 207 (19)

′ 4 3 2 ′
f (x) = 15x − 8x + 45x + 26x − 12 f (2) = 396 (20)
′′ 3 2 ′′
f (x) = 60x − 24x + 90x + 26 f (2) = 590 (21)
f ′′′ (x) = 180x 2 − 48x + 90 f ′′′ (2) = 714 (22)
f (4) (x) = 360x − 48 f (4) (2) = 672 (23)
(5) (5)
f (x) = 360 f (2) = 360 (24)
(6) (6)
f (x) = 0 f (2) = 0 (25)

Dr. Gabby Introduction To Numerical Analysis 28 / 40

 Taylor and Maclaurin Series

Therefore, we have
f ′′ (c) f ′′′ (c)
f (x) ≈ f (c) + f ′ (c)(x − c) + (x − c)2 + (x − c)3 + (26)
2! 3!
f (4) (c) f (5) (c) f (6) (c)
(x − c)4 + (x − c)5 + (x − c)6
4! 5! 6!
(27)
′′ ′′′
f (2) f (2)
≈ f (2) + f ′ (2)(x − 2) + (x − 2)2 + (x − 2)3 +
2! 3!
f (4) (2) f (5) (2) f (6) (2)
(x − 2)4 + (x − 2)5 + (x − 2)6
4! 5! 6!
(28)
2 3 4 5
≈ 207 + 396(x − 2) + 295(x − 2) + 119(x − 2) + 28(x − 2) + 3(x − 2)

Dr. Gabby Introduction To Numerical Analysis 29 / 40

 Taylor and Maclaurin Series

Example
Find the Taylor series and the Maclaurin series of the following function using the
first four terms of the series.

f (x) = e 2x , where c = 1

Solution
i. For the Taylor series we obtain

f (x) = e 2x f (1) = e 2 (29)

f ′ (x) = 2e 2x f ′ (1) = 2e 2 (30)
′′ 2x ′′ 2
f (x) = 4e f (1) = 4e (31)
′′′ 2x ′′′ 2
f (x) = 8e f (1) = 8e (32)

Dr. Gabby Introduction To Numerical Analysis 30 / 40

 Taylor and Maclaurin Series

f ′′ (c) f ′′′ (c)

f (x) ≈ f (c) + f ′ (c)(x − c) + (x − c)2 + (x − c)3 (33)
2! 3!
f ′′ (1) 2 f ′′′ (1)
′
≈ f (1) + f (1)(x − 1) + (x − 1) + (x − 1)3 (34)
2! 3!
2 2
4e 8e
= e 2 + 2e 2 (x − 1) + (x − 1)2 + (x − 1)3 (35)
· 2 6 ¸
2 2 4 3 2
= e 1 + 2x − 2 + 2x − 4x + 2 + (x − 3x + 3x − 1) (36)
3
µ ¶
7 4
= e 2 + 2x − 2x 2 + x 3 (37)
3 3

Dr. Gabby Introduction To Numerical Analysis 31 / 40

 Taylor and Maclaurin Series

ii. The Maclaurin series is obtained with c = 0. Thus,

f (x) = e 2x f (0) = e 0 = 1 (38)
′ 2x ′ 0
f (x) = 2e f (0) = 2e = 2 (39)
′′ 2x ′′ 0
f (x) = 4e f (0) = 4e = 4 (40)
f ′′′ (x) = 8e 2x f ′′′ (0) = 8e 0 = 8 (41)

f ′′ (c) 2 f ′′′ (c)

′
f (x) ≈ f (c) + f (c)(x − c) + (x − c) + (x − c)3 (42)
2! 3!
′′ ′′′
f (0) f (0)
≈ f (0) + f ′ (0)x + x2 + x3 (43)
2! 3!
4
= 1 + 2x + 2x 2 + x 3 (44)
3
Dr. Gabby Introduction To Numerical Analysis 32 / 40
 Taylor and Maclaurin Series

Taylor’s Theorem for f (x)

Definition
If the function f possesses continuous derivatives of order 0, 1, 2, · · · , (n + 1) in a
closed interval I = [a, b], then for any c and x in I ,
n f (k) (x)
(x − c)k + E n+1
X
f (x) = (45)
k=0 k!
where the error term E n+1 can be given in the form
f n+1 (ϵ)
E n+1 = (x − c)n+1 (46)
(n + 1)!
for some ϵ between c and x and depends on both.

In practical computations with Taylor series, it is usually necessary to truncate the

series because it is not possible to carry out an infinite number of additions.

Dr. Gabby Introduction To Numerical Analysis 33 / 40

 Taylor and Maclaurin Series

Example
Derive the formal Taylor series for f (x) = ln(1 + x) at c = 0

f (x) = l n(1 + x) =⇒ f (0) = 0 (47)

f ′ (x) = (1 + x)−1 =⇒ f ′ (0) = 1 (48)
′′ −2 ′′
f (x) = −(1 + x) =⇒ f (0) = −1 (49)
f ′′′ (x) = 2(1 + x)−3 =⇒ f ′′′ (0) = 2 (50)
f (4) (x) = −6(1 + x)−4 =⇒ f (4) (0) = −6 (51)
.. ..
. . (52)
(k) k−1 −k (k) k−1
f (x) = (−1) (k − 1)!(1 + x) =⇒ f (0) = (−1) (k − 1)!

Dr. Gabby Introduction To Numerical Analysis 34 / 40

 Taylor and Maclaurin Series

Hence by Taylor’s Theorem, we obtain

n (k − 1)! k (−1)n n!(1 + ϵ)−n−1 n+1

(−1)k−1
X
ln(1 + x) = x + x (53)
k=1 k! (n + 1)!
n x k (−1)n
(−1)k−1 (1 + ϵ)x n+1
X
= + (54)
k=1 k n +1

Dr. Gabby Introduction To Numerical Analysis 35 / 40

 Taylor and Maclaurin Series

Taylor’s Theorem for f (x + h)

Definition
If the function f possesses continuous derivatives of order 0, 1, 2, · · · , (n + 1) in a
closed interval I = [a, b], then for any x in I ,
n f (k) (x)
h k + E n+1
X
f (x + h) = (55)
k=0 k!

where h is any value such that x + h is in I and where

f n+1 (ϵ) n+1

E n+1 = h (56)
(n + 1)!

for some ϵ between x and x + h .

Dr. Gabby Introduction To Numerical Analysis 36 / 40
 Taylor and Maclaurin Series

Big O notation
1 The error term E n+1 depends on h in two ways:
1 First, h n+1 is explicitly present;
2 Second, the point ϵ generally depends on h.
2 So as h converges to zero, E n+1 converges to zero with essentially the same
rapidity with which h n+1 converges to zero. quite rapid.
3 To express this qualitative fact, we write

E n+1 = O (h n+1 ) (57)

as h → 0. This is called big O notation, and it is shorthand for the inequality

|E n+1 | ≤ C |h|n+1 (58)

where C is a constant.
Dr. Gabby Introduction To Numerical Analysis 37 / 40
 Taylor and Maclaurin Series

Derived Equations from Equation(55)

n=0
f (x + h) = f (x) + f ′ (ϵ1 )h (59)
= f (x) + O (h) (60)
n=1
1 ′′
f (x + h) = f (x) + f ′ (x)h + f (ϵ2 )h 2 (61)
2!
= f (x) + f ′ (x)h + O (h 2 ) (62)
n=2
1 ′′ 1
f (x + h) = f (x) + f ′ (x)h +f (x)h 2 + f ′′′ (ϵ3 )h 3 (63)
2! 3!
1 ′′
= f (x) + f (x)h + f (x)h + O (h 3 )
′ 2
(64)
2!
Dr. Gabby Introduction To Numerical Analysis 38 / 40
 Taylor and Maclaurin Series

Exercise
Find the Taylor series and the Maclaurin series of the following function using the
first four terms of the series.
1

f (x) = sin(2x), where c = π

2

f (x) = cosh(3x), where c = 2

Dr. Gabby Introduction To Numerical Analysis 39 / 40

 END OF LECTURE
THANK YOU

Dr. Gabby Introduction To Numerical Analysis 40 / 40