Linear Models

 Linear models are a class of equations that describe relationships

between, two quantities exhibiting a constant rate of change.
 Formula (or) Equation: 𝑦 = 𝑚𝑥 + 𝑐.
Here 𝑦 is the dependent variable, 𝑥 is the independent
variable, 𝑚 is the constant rate of change and 𝑐 is a constant
(or) initial value.
 Linear models particularly Linear Regression is a cornerstone of
Economists to model relationship between variables, quantify
impacts and make prediction.
 A sample Regression function is an equation that represent a
relationship between the variables 𝑌 and 𝑋 using only a sample of
the total data.
The Regression Model:
Consider the Linear Regression Model:
𝑦𝑖 = 𝛽0 + 𝛽1 𝑥𝑖 + 𝜖𝑖 , 𝑖 = 1, 2, 3, … , 𝑛.
𝑦1 ∈1
1 𝑥1
𝑦2 ∈2
1 𝑥2
𝑦3 ∈3
1 𝑥3
. 𝛽0 .
In matrix notation, = . . +
. 𝛽1 .
. .
. .
. .
𝑦𝑛 ∈𝑛
1 𝑥𝑛
𝑦1 ∈1
1 𝑥1
𝑦2 ∈2
1 𝑥2
𝑦3 ∈3
1 𝑥3
. 𝛽 .
In short, 𝑌 = 𝛽𝑋 + 𝜀, assigning ,𝑋 = . . , 𝛽 = 0 and 𝜀 =
. 𝛽1 .
. .
. .
. .
𝑦𝑛 ∈𝑛
1 𝑥𝑛
The Ordinary Least Square Estimate of 𝛽 (OLS of 𝛽):
The sample Regression equation is 𝑦𝑖 = 𝛽0 + 𝛽1 𝑥𝑖 + 𝜖𝑖 , 𝑖 = 1, 2,3 … , 𝑛.

𝑦1 ∈1
1 𝑥1
𝑦2 ∈2
1 𝑥2
𝑦3 ∈3
1 𝑥3
. 𝛽0 .
In matrix notation, .
= . . + or 𝑌 = 𝛽 𝑋 + 𝜀 .
𝛽1 .
. .
. .
. .
𝑦𝑛 ∈𝑛
1 𝑥𝑛
The OLS estimate of 𝛽 is obtained by minimising
2
𝜀𝑖 2 = 𝑦𝑖 − 𝛽0 − 𝛽1 𝑥𝑖 .
The normal equations are
𝑛𝛽0 + 𝑥𝑖 𝛽1 = 𝑦𝑖 ; 𝑥𝑖 𝛽0 + 𝑥𝑖 2 𝛽1 = 𝑥𝑖 𝑦𝑖 .
The normal equations can be written in matrix form as

1 𝑥1 ′ 1 𝑥1 1 𝑥1 ′ 𝑦1
1 𝑥2 1 𝑥2 1 𝑥2 𝑦2
1 𝑥3 1 𝑥3 1 𝑥3 𝑦3
𝛽0 .
. . . . = . .
.
𝛽1
. . . . . . .
. . . . . . 𝑦𝑛
1 𝑥𝑛 1 𝑥𝑛 1 𝑥𝑛
In matrix form, 𝑋 ′ 𝑋𝛽 = 𝑋 ′ Y ⇒ 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑋 𝛽 = 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑌
⇒ 𝐼𝛽 = 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑌 ⇒ 𝛽 = 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑌 . This is the fundamental
theory using matrix notation.
In 𝑌 = 𝛽𝑋 + 𝜀 , 𝑋 is called the Design matrix, 𝑌, the Response matrix, 𝜀
is the error vector and 𝛽 is the vector of parameters.
Problems
1 2 3 𝛽0
1. Given 𝑋 = 1 3 , 𝑌 = 6 with 𝛽 = , show that
𝛽1
1 4 7
1.3333
𝛽= .
1.0000
We know that 𝛽 = 𝑋 ′ 𝑋 −1 𝑋 ′ 𝑌
1 56 −12 16 1 32 1.3333
= = = .
24 −12 3 72 24 24 1.0000
1 1 1
1 2 3 𝛽0
2. 𝑋 = ,𝑌 = &𝛽 = .
1 3 3 𝛽1
1 4 5

Answer: 𝛽0 = 0; 𝛽1 = 1.2, the line of best fit is 𝑦 = 1.2𝑥.

3. Given the diameter (X) in inches and price (Y) in Dollars of a
product, what will be the price of a 20-inch product?

The matrix 𝑋 ′ 𝑋 and 𝑋 ′ 𝑌 can be constructed using the following

standard matrices.
𝑛 𝑛
𝑛 𝑖=1 𝑥𝑖 𝑖=1 𝑦𝑖
𝑋′𝑋 = 𝑛 𝑛 2 and 𝑋 ′ 𝑌 = 𝑛
𝑖=1 𝑥𝑖 𝑖=1 𝑥 𝑖 𝑖=1 𝑥𝑖 𝑦𝑖
3 30 39
In this problem, 𝑋 ′ 𝑋 = and 𝑋 ′ 𝑌 =
30 308 402
′ −1 ′
1 308 −30 39 1 −48 −2
(𝑋 𝑋 ) 𝑋 𝑌 = = =
24 −30 3 402 24 36 1.5
The linear model is 𝑦 = −2 + 1.5𝑥. When 𝑥 = 20, 𝑦 = 28.
4. The relationship between number of widgets in a package and the
length of the package in inches, is given in the table.
No. of Widgets Length in the package
3 9.00
4 9.25
5 9.50
6 9.75
Find the line of best fit.
Cramer’s Rule:
Consider a system of simultaneous equations in three variables.
𝑎1 𝑥 + 𝑏1 𝑦 + 𝑐1 𝑧 = 𝑑1 ; 𝑎2 𝑥 + 𝑏2 𝑦 + 𝑐2 𝑧 = 𝑑2 ; 𝑎3 𝑥 + 𝑏3 𝑦 + 𝑐3 𝑧 = 𝑑3
𝑎1 𝑏1 𝑐1 𝑥 𝑑1
In matrix form, 𝑎2 𝑏2 𝑐2 𝑦 = 𝑑2 .
𝑎3 𝑏3 𝑐3 𝑧 𝑑3
𝑎1 𝑏1 𝑐1 𝑑1 𝑏1 𝑐1 𝑎1 𝑑1 𝑐1
Let ∆= 𝑎2 𝑏2 𝑐2 ; ∆𝑥 = 𝑑2 𝑏2 𝑐2 ; ∆𝑦 = 𝑎2 𝑑2 𝑐2 &
𝑎3 𝑏3 𝑐3 𝑑3 𝑏3 𝑐3 𝑎3 𝑑3 𝑐3
 𝑎1 𝑏1 𝑑1
∆𝑧 = 𝑎2 𝑏2 𝑑2
𝑎3 𝑏3 𝑑3
∆𝑥 ∆𝑦 ∆𝑧
The solution is given by 𝑥 = ;𝑦 = ; z= .
∆ ∆ ∆
Problems
Solve the following equations using Cramer’s rule.
𝑖 2𝑥 + 3𝑦 = 7; 3𝑥 + 5𝑦 = 9
2 3 7 3 2 7
∆= = 1; ∆𝑥 = = 8; ∆𝑦 = = −3
3 5 9 5 3 9
∆𝑥 8 ∆𝑦 −3
𝑥= = = 8; 𝑦 = = = −3.
∆ 1 ∆ 1
𝑖𝑖 5𝑥 + 3𝑦 = 17; 3𝑥 + 7𝑦 = 31
𝑖𝑖𝑖 2𝑥 + 𝑦 − 𝑧 = 3; 𝑥 + 𝑦 + 𝑧 = 1; 𝑥 − 2𝑦 − 3𝑧 = 4
𝑖𝑣 𝑥 + 𝑦 + 𝑧 = 6; 2𝑥 + 3𝑦 − 𝑧 = 5; 6𝑥 − 2𝑦 − 3𝑧 = −7