Numerical Differentiation

Forward Difference Approximation of the First Derivative

The forward Taylor series expansion at 𝑥𝑖+1 is

1
f (xi+1) = f (xi ) + f (xi )h+ f (xi )h2 +.... (1)
2!
Now let us truncate the series after the first derivative term we have

f (xi )  f (xi+1) − f (xi ) +O(h) (2)
h
where h is called the step size that is, the length of the interval over which the approximation is
made, 𝑥𝑖+1 − 𝑥𝑖 . It is termed a forward difference because it utilizes data at i and i + 1 to
estimate the derivative. This forward difference is but one of many that can be developed from
the Taylor series to approximate derivatives numerically.

Backward Difference Approximation of the First Derivative

The Taylor series can be expanded backward to calculate a previous value

on the basis of a present value, as in
1
f (xi−1) = f (xi ) − f (xi )h+ f (xi )h2 −.... (3)
2!
Truncating this equation after the first derivative and rearranging yields

f (xi )  f (xi ) − f (xi−1) +Oh

() (4)
h
where the error is O(h).
Centered Difference Approximation of the First Derivative

A third way to approximate the first derivative is to subtract Eq. (3) from the forward Taylor
series expansion Eq.(1) to yield

2
f (xi+1) = f (xi−1) +2 f (xi )h+ f (3) (xi )h3 +....
3!
which can be solved for

f (xi )  f (xi+1) − f (xi−1) −O(h2) (5)

2h
Equation (5) is a centered finite difference representation of the first derivative.
Example: Use forward and backward difference approximations of O(h) and a centered
difference approximation of O(ℎ2 ) to estimate the first derivative of
f (x) = −0.1𝑥 4 − 0.15 𝑥 3 − 0.5 𝑥 2 − 0.25 x + 1.2 at x = 0.5 using a step size h = 0.5

Solution. For h = 0.5, the function can be employed to determine x 0 0.5 1

These values can be used to compute the forward difference [Eq. (2)], F(x) 1.2 0.925 0.2

f (0.5)  0.2− 0.925 =-1.45

0.5
the backward difference [Eq. (4)], f (0.5)  0.925 − 1.2 =-0.55
0.5
and the centered difference [Eq.(5)],
f (0.5)  0.2− 1.2 =-1.0
2(0.5)
Finite-Difference Approximations of Higher Derivatives

Besides first derivatives, the Taylor series expansion can be used to derive numerical
estimates of higher derivatives. To do this, we write a forward Taylor series expansion for f
(𝑥𝑖+2 ) in terms of f (𝑥𝑖 ):
1
f (xi+2) = f (xi ) + f (xi )(2h) + f (xi )(2h)2 +.... (6)
2!
Equation (1) can be multiplied by 2 and subtracted from Eq. (6) to give

f (xi+2) −2f (xi+1) =−f (xi ) + f (xi )h +.... 2

which can be solved for

f (xi ) = f (xi+2 ) −2 f (xi+1) + f (xi ) +O(h) (7)

2
h
This relationship is called the second forward finite difference. Similar manipulations can be
employed to derive a backward version

f (xi ) = f (xi ) −2 f (xi−1) + f (xi−2 ) +O(h) (7)

2
h
A centered difference approximation for the second derivative can be derived by adding
Eqs. (1) and (3) and rearranging the result to give

f (xi ) = f (xi+1) −2 f (xi ) + f (xi−1) +O(h )

2
(8)
2
h
Example: Use forward and backward difference approximations of O(h) and a centered
difference approximation of O(ℎ2 ) to estimate the first and the second derivative of
f (x) = 𝑒 5𝑥+1 at x = 0.5 using a step size h = 0.1

Solution. For h = 0.1, the function can be employed to determine

x 0.3 0.4 0.5 0.6 0.7

f (x) = 𝑒 5𝑥+1 12.1825 20.0855 33.1155 54.5982 90.0171

f (xi )  f (xi+1) − f (xi )

h
f (0.5)  f (0.6) − f (0.5) = 54.5982−33.1155 = 214.827
0.1 0.1
f (xi )  f (xi ) − f (xi−1)
h
f (0.5)  f (0.5) − f (0.4) = 33.1155−20.0855 =130.3
0.1 0.1
f (xi )  f (xi+1) − f (xi−1)
2h
f (0.5)  f (0.6) − f (0.4) = 54.5982−20.0855 =127.5635
2(0.1) 2(0.1)
 
f (xi ) = f ( xi+ 2 ) − 2 f ( x i+1) + f ( x i )
h2

f (0.5) = f (0.7) − 2 f (0.6) + f (0.5) = 90.0171− 2(54.5982) +33.1155 =1393.62
2 2
(0.1) (0.1)
f (xi ) = f ( xi ) − 2 f (x i−1 ) + f (x i− 2 )
h2

f (0.5) = f (0.5) − 2 f (0.4) + f (0.3) = 33.1155− 2(20.0855) +12. 1825 =512.7
2 2
(0.1) (0.1)
f (xi ) = f ( xi+1 ) − 2 f ( x i ) + f (x i −1 )
h2
f (0.5) = f (0.6) − 2 f (0.5) + f (0.4) = 54.5982 −2(33.1155) + 20.0855 =845.27
2 2
(0.1) (0.1)
Exercises: Compute forward , backward difference approximations, and central
approximation for the first and second derivative for the following functions

1] y = cos x at x = ; h= 
4 12
2] y = e3x+2 at x = 0.5 ; h = 0.1
3] y =x2 cos x at x = 0.4 ; h = 0.5
4] y = tan( 3x ) at x =3 ; h = 0.5
5] y = sin(0.5 x)/ x at x =1 ; h = 0.2
6] y = xln(x +1) at x =1 ; h = 0.2
7] y = tan(x)/ x at x = h= 
;
4 12
8] y = ex sin(x) at x = ; h=
3 6
9] y =x2 e4x+1 at x =1 ; h= 0.2
 Newton’s forward and backward for differentiation

Another way to find the derivative, we can use Newton’s forward difference
polynomials

1 1
P(x) = f (x0) + f (x0)s +  f (x0) s(s −1) +...+ n f (x0) s(s −1)....(s −(n−1)) (1)
2
2! n!

Where 𝑥 = 𝑥0 + s h or using Newton’s backward difference

P(x) = f (xn) +f (xn)(s) + 1 2 f (xn)(s)(s +1)+...+ 1 2 f (xn)(s)(s +1)(s +2)...(s +n−1) (2)
2! n!
Where 𝑥 = 𝑥𝑛 + s h
 df = df dx = f  h
ds dx ds
f =  1 df 1 1 1 1
= [ f (x0) +  f (x0)(2s −1) +  f (x0)(3s −6s +2) + 4 f (x0)(4s3 −18s2 +22s −6) ]
2 3 2
h ds h 2! 3! 4!
df  = df  dx = f  h
ds dx ds
1 df  1 1 1
 f = = 2 [ f (x0)+  f (x0)(6s −6) + 4 f (x0)(12s2 −36s +22) + ]
2 3
h ds h 3! 4!
similarly we can prove
 1 1
f = 3 [ f (x0)+ 4 f (x0)(24s −36) + ]
3
h 4!

Similarly by using Newton’s backward we can prove

 df = df dx = f  h
ds dx ds
 f = 1 df = 1[ f (xn) + 1 2 f (xn)(2s +1) + 1 3 f (xn)(3s2 +6s +2) + 1 4 f (xn)(4s3 +18s2 +22s +6) ]
h ds h 2! 3! 4!
df  = df  dx = f  h
ds dx ds

 f = 1 df = 12 [2 f (xn)+ 1 3 f (xn)(6s +6) + 1 4 f (xn)(12s2 +36s +22) + ]
h ds h 3! 4!
similarly we can prove
f = 13 [3 f (xn)+ 1 4 f (xn)(24s +36) + ]
h 4!

Example: Find f’’ (1.5) &f’’’(1.5) and f’’(8)& f’’’(8) using the following
table
x 1 3 5 7 9

F(x) 4 5 8 11 16
Solution: since h=2, x=1.5, 𝑥0 =1 , x= 𝑥0 +s h, then 1.5=1+s(2) , hence s=0.25

5 2

3 -2

8 0 4

3 2

11 2

16
  1 1 1
f = 2 [ f (x0)+  f (x0)(6s −6) + 4 f (x0)(12s2 −36s +22) + ]
2 3
h 3! 4!
 f (1.5) = 1[2−2(0.25−1) + 1 (4)(12(0.25)2 −36(0.25) +22]
4 4!
f  = 13 [3 f (x0)+ 1 4 f (x0)(24s −36) + ]
h 4!
f (1.5) = 1[−2+ 1 (4)(24(0.25) −36)]
8 4!

To compute f’’(8)&f’’’(8) by using Newtons backward. Since h=2, x=8, 𝑥4 =1 , x= 𝑥4 +s h,

then 8=9+s(2) , hence s= - 0.5
 f (8) = 12 [2 f (xn)+ 1 3 f (xn)(6s +6) + 1 4 f (xn)(12s2 +36s +22) + ]
h 3! 4!
f (8)=1[2+ 1 (2)(6(−0.5) +6) + 1 4(12(−0.5)2 +36(−0.5) +22)]
4 3! 4!
f = 13 [3 f (xn)+ 1 4 f (xn)(24s +36) + ]
h 4!
f(8) = 1[2+ 1 (4)(24(−0.5) +36)]
8 4!
Exercises: Using Newton’s method to evaluate the following
1] Find f’’(0.05) and f’’’(0.75)
x 0 0.2 0.4 0.6 0.8
f(x) 1 1.2214 1.4918 1.8221 2.2255

2] Find f’(1.03)& 𝑓′′′(1.23) x 1 1.05 1.1 1.15 1.2

f(x) 1 1.0247 1.0488 1.0724 1.0954

x 2 2.2 2.4 2.6 2.8 3

3] Find f’’’(2.1) & f′′(2.9)
f(x) 1.3863 1.4351 1.4816 1.5261 1.5686 1.6094