1-3 Curve fittings using least square method

1. Introduction

This method is used to determine the best curve fitting to data obtained from measurements. In
this chapter we explain how to apply the least square fitting method to data that remain in linear
relation.

2. The basic theory of least square method

Let’s assume that there are data sets, (x1, y1 ), (x 2 , y 2 ),....(x n , yn ) , obtained in a certain experiment.
The data may be represented graphically as shown in Figure 1. We assume that x is a linear function
of y. In such a case, the best-fitting curve for the data will be expressed by the equation y = ax + b
and it can be plotted as shown in Figure 2.

Figure 1 Distribution of measured data Figure 2 The best fitting

For any point representing the set of data (xi, yi), we may calculate the difference ∆y i between
y and yi as ∆yi = yi − (ax i + b ) . In the equation, y is the value of the fitted linear function that
corresponds to the argument xi (see Figure 2). According to the linear least square curve fitting
method, if the linear equation y = ax + b is to be the best fitted curve for the data, the deviation
between the ∆y i should be minimized. Let S be the sum of all the deviation squared, i.e.
S = ∑ (∆yi ) ; the relation of this data to S can be written as follows:
2

S = ∑ (∆yi ) = ∑ {yi − (ax i + b )}

2 2
(1)
i i

Equation (1) can be written as:

S = ∑ (yi ) − 2a ∑ yi x i + a 2 ∑ x i2 + 2ab ∑ x i − 2b ∑ yi + nb 2
2
(2)
i i i i i
 (x1, y1 ), (x 2 , y 2 ),, (x n , yn ) are measured values already known; S is a function with two
variables, a and b. For S to have the minimum value, all of its partial derivatives must be equal to
zero, i.e. ∂S = 0 and ∂S = 0 . Differentiating S with respect to a and b, we obtain:
∂a ∂b

∂S
= −2∑ yi xi + 2a ∑ xi2 + 2b∑ xi = 0 (3)
∂a i i i

∂S
= −2∑ yi + 2a ∑ x i + 2nb = 0 (4)
∂b i i

We want both of the above equations to have a value of zero (the necessary condition for a

minimum). Solving the above equations, we get:

a ∑ x i2 + b ∑ x i = ∑ x i yi (5)

a ∑ x i + nb = ∑ yi (6)

where

∑x i = x 1 + x 2 + ... + x n

∑y i = y1 + y 2 + ... + yn

∑x = x1 + x 2 + ... + x n
2 2 2 2
i

∑x y i i = x1y1 + x 2y 2 + ... + x n yn

Retrieving from the above equations a and b we get:

n ∑ x i yi − (∑ x i )(∑ yi )
a= (7)
n ∑ x i2 − (∑ x1 )2

(∑ yi )(∑ x i2 ) − (∑ x i )(∑ x i yi )
b= (8)
n ∑ x i2 − (∑ x1 )2

The calculated values of a and b give us the best-fitted linear function y = ax + b to the set of
experimental data.
3. Example: step by step fitting using the least square method

Table. 1 Calculation process using

experimental data
xi yi xiyi xi2

1 1.0 18.1 18.1 1.0

2 2.1 21.2 44.5 4.4

3 2.9 29.3 85.0 8.4

4 4.2 30.5 128.1 17.6

5 5.1 34.3 174.9 26.0

Σ 15.3 133.4 450.6 57.5

Figure 3 The plot representing the dependence of y on

x variable

In Figure 3, a set of experimental data is shown. The measured values of the quantities x and y
are shown in the inserted table. The values are plotted on the figure as filled dots. The blue line
represents the best fitted function determined with the use of the least square method. Below, we
shall explain step by step how the function was determined.
It is a good method to construct a table just like Table 1. The second and third columns contain
the measured values of x and y quantities. The next, fourth column, is filled with the product of x
and y, and the last column contains the squared value of x. These numbers, more specifically their
sums, are necessary for calculating a and b. The sums: ∑ x i , ∑ yi , ∑ x i yi , ∑ x i2 are given in the
last row of the table. The same expressions appear in the Equations (7) and (8) so it is very useful to
have them in the table.
In the next step we simply substitute the sums that appear in the Equations (7) and (8) with the
values from the last row of the table. The number n in the equations is the number of measurements –
in the example, it is 5.

5(450.6) − (15.3)(133.4) 212.08

a= = = 3.98
5(57.5) − (15.3)2 53.26

(133.4)(57.5) − (15.3)(450.6) 772.01

b= = = 14.5
5(57.5) − (15.3)2 53.26
 Therefore, the best fitted linear equation is y = 3.98x + 14.5 .

4. An error analysis

The concept of “best fit” requires a definition of some measure of an error between the data and
the line. The choice for this method is to look at each value of xi, and calculate the square of the
difference between the data values yi and y determined by evaluating the above equation at that xi,
n
i.e. S = ∑ (ax i + b − yi )2 . The process of squaring guarantees a positive number so that we can
i =1

sum the errors at all points to obtain an overall measure of error.

To determine the error that obviously depends on both a and b, we first calculate the deviation
di = ax i + b − yi using slope a and interception b determined earlier. Next, it should be checked
whether the next condition ∑ di = 0 is satisfied. If so, calculate (di )2 and ∑ (di )2 .
i i

For determining the errors with what a and b are charged, we define:

α 2
=
∑ (d i
2
)
(9)
n −2

and

∆ ≡ n∑ ( xi ) − (∑ xi ) 2 ..
2
(10)

Then, the errors ∆a and ∆b (standard deviations) that we are looking for can be defined as follows:

n
∆a = α (11)
∆

∆b = α
∑ (x ) i
2

(12)
∆

Below, we present a step-by-step calculation of the errors. In the example, we use the same set of
data as in the above example for explaining the best fitting method. Just like in the procedure of
determining the values of a and b, it is good to build a table filled in with the numbers that are
necessary to calculate the errors. It is very easy to analyze Table 2.
 [Table 2] An error calculation process of the coefficients of linear equation

xi yi xiyi xi 2 di = ax i + b − yi di2

1 1.0 18.1 18.1 1.0 +0.377 0.142

2 2.1 21.2 44.5 4.4 +1.66 2.75
3 2.9 29.3 85.0 8.4 -3.26 10.6
4 4.2 30.5 128.1 17.6 +0.720 0.518
5 5.1 34.3 174.9 26.0 +0.503 0.253
Σ 15.3 133.4 450.6 57.5 0.000 14.3

Substituting the result from the last row of the table to Equations (9), (10), (11), and (12) we obtain:

α 2
=
∑ (d i
2
)
=
14.3 14.3
= = 4.76  α = 2.18
n −2 5−2 3

∆ ≡ n ∑ (x i ) − (∑ x i )2 = 5(57.5) − (15.3)2 = 53.3

2

n 5
∆a = α = (2.18) = 0.668
∆ 53.3

∆b = α
∑ n (x ) i
2

= (2.18)
57.5
= 2.27
∆ 53.3

We can now state that (in our example):

a = 3.98 ± 0.668 and b = 14.5 ± 2.27

which may be also written as:

y = (3.98 ± 0.668)x + (14.5 ± 2.27 )

Finally, after adjusting the significant figure, it can be rewritten as the following:

y = (4.0 ± 0.67 )x + (15 ± 2.3)

5. Problem 1

Table 3 contains the result of certain measurements. We expect that the quantity x depends linearly
on the quantity y. Calculate a linear equation for the best fit to these measured values and draw a
straight line corresponding to the derived equation. Also, calculate the errors with what the slope m
and the intercept b are charged.
 Table 3. Problem 1 - data

xi yi xiyi xi 2 di = ax i + b − yi di2

1 2 5.2

2 4 10.3

3 6 14.9

4 8 20.1