Chapter 3
The Simple Linear Regression Model: Specification and Estimation
3.1 An Economic Model
The simple regression function
E ( y | x) = y| x = 1 + 2 x
(3.1.1)
E ( y | x) dE ( y | x)
=
x
dx
(3.1.2)
Slope of regression line
2 =
denotes change in
Slide 3.1
Undergraduate Econometrics, 2nd Edition Chapter 3
3.2 An Econometric Model
Assumptions of the Simple Linear Regression Model-I
The average value of y, for each value of x, is given by the linear regression
E ( y ) = 1 + 2 x
For each value of x, the values of y are distributed about their mean value,
following probability distributions that all have the same variance,
var( y ) = 2
Slide 3.2
Undergraduate Econometrics, 2nd Edition Chapter 3
The values of y are all uncorrelated, and have zero covariance, implying that
there is no linear association among them.
cov( yi , y j ) = 0
This assumption can be made stronger by assuming that the values of y are all
statistically independent.
The variable x is not random and must take at least two different values
(optional) The values of y are normally distributed about their mean for each
value of x,
y ~ N [(1 + 2 x), 2 ]
Slide 3.3
Undergraduate Econometrics, 2nd Edition Chapter 3
3.2.1
Introducing the Error Term
The random error term is
e = y E ( y ) = y 1 2 x
(3.2.1)
y = 1 + 2 x + e
(3.2.2)
Rearranging gives
y is dependent variable; x is independent or explanatory variable
Slide 3.4
Undergraduate Econometrics, 2nd Edition Chapter 3
Assumptions of the Simple Linear Regression Model-II
SR1
y = 1 + 2 x + e
SR2.
E (e) = 0 E ( y ) = 1 + 2 x
SR3.
var(e) = 2 = var( y )
SR4.
cov(ei , e j ) = cov( yi , y j ) = 0
SR5. The variable x is not random and must take at least two different values.
SR6. (optional) The values of e are normally distributed about their mean
e ~ N (0, 2 )
Slide 3.5
Undergraduate Econometrics, 2nd Edition Chapter 3
3.3 Estimating the Parameters for the Expenditure Relationship
3.3.1
The Least Squares Principle
The fitted regression line is
yt = b1 + b2 xt
(3.3.1)
The least squares residual
et = yt yt = yt b1 b2 xt
(3.3.2)
Any other fitted line
y t* = b1* + b2* xt
(3.3.3)
Least squares line has smaller sum of squared residuals
e = ( y
2
t
y t ) 2 et*2 = ( yt y t* ) 2
Slide 3.6
Undergraduate Econometrics, 2nd Edition Chapter 3
Least squares estimates are obtained by minimizing the sum of squares function
T
S (1 , 2 ) = ( yt 1 2 xt ) 2
(3.3.4)
t =1
Math: Obtain partial derivatives
S
= 2T 1 2 yt + 2 xt 2
1
(3.3.5)
S
= 2 xt22 2 xt yt + 2 xt 1
2
Set derivatives to zero
2( yt Tb1 xt b2 ) = 0
(3.3.6)
2( xt yt xt b1 xt2b2 ) = 0
Slide 3.7
Undergraduate Econometrics, 2nd Edition Chapter 3
Rearranging equation 3.3.6 leads to two equations usually known as the normal
equations,
Tb1 + xt b2 = yt
(3.3.7a)
xb +x b = x y
2
t 2
t 1
(3.3.7b)
Formulas for least squares estimates
b2 =
T xt yt xt yt
T x ( xt )
(3.3.8a)
b1 = y b2 x
(3.3.8b)
2
t
Since these formulas work for any values of the sample data, they are the least squares
estimators.
Slide 3.8
Undergraduate Econometrics, 2nd Edition Chapter 3
3.3.2
Estimates for the Food Expenditure Function
b2 =
T xt yt xt yt
T x ( xt )
2
t
(40)(3834936.497) (27920)(5212.520)
(40)(21020623.02) (27920) 2
(3.3.9a)
= 0.1283
b1 = y b2 x = 130.313 (0.1282886)(698.0) = 40.7676
(3.3.9b)
A convenient way to report the values for b1 and b2 is to write out the estimated or fitted
regression line:
yt = 40.7676 + 0.1283xt
(3.3.10)
Slide 3.9
Undergraduate Econometrics, 2nd Edition Chapter 3
3.3.3 Interpreting the Estimates
The value b2 = 0.1283 is an estimate of 2, the amount by which weekly expenditure
on food increases when weekly income increases by $1. Thus, we estimate that if
income goes up by $100, weekly expenditure on food will increase by approximately
$12.83.
Strictly speaking, the intercept estimate b1 = 40.7676 is an estimate of the weekly
amount spent on food for a family with zero income
Slide 3.10
Undergraduate Econometrics, 2nd Edition Chapter 3
3.3.3a
Elasticities
The income elasticity of demand is a useful way to characterize the responsiveness of
consumer expenditure to changes in income. From microeconomic principles the
elasticity of any variable y with respect to another variable x is
=
percentage change in y y / y y x
=
=
percentage change in x x / x x y
(3.3.11)
In the linear economic model given by equation 3.1.1 we have shown that
2 =
E ( y )
x
(3.3.12)
The elasticity of average expenditure with respect to income is
=
E ( y ) / E ( y ) E ( y ) x
x
=
= 2
x / x
x E ( y )
E ( y)
(3.3.13)
Slide 3.11
Undergraduate Econometrics, 2nd Edition Chapter 3
A frequently used alternative is to report the elasticity at the point of the means
( x , y ) = (698.00,130.31) since that is a representative point on the regression line.
= b2
3.3.3b
698.00
x
= 0.1283
= 0.687
130.31
y
(3.3.14)
Prediction
Suppose that we wanted to predict weekly food expenditure for a household with a
weekly income of $750. This prediction is carried out by substituting x = 750 into our
estimated equation to obtain
yt = 40.7676 + 0.1283xt = 40.7676 + 0.1283(750) = $130.98
(3.3.15)
We predict that a household with a weekly income of $750 will spend $130.98 per week
on food.
Slide 3.12
Undergraduate Econometrics, 2nd Edition Chapter 3
3.3.3c
Examining Computer Output
Dependent Variable: FOODEXP
Method: Least Squares
Sample: 1 40
Included observations: 40
Variable
Coefficient
Std. Error
t-Statistic
Prob.
40.76756
22.13865
1.841465
0.0734
INCOME
0.128289
0.030539
4.200777
0.0002
R-squared
0.317118
Mean dependent var
130.3130
Adjusted R-squared
0.299148
S.D. dependent var
45.15857
S.E. of regression
37.80536
Akaike info criterion
10.15149
Sum squared resid
54311.33
Schwarz criterion
10.23593
F-statistic
17.64653
Prob(F-statistic)
0.000155
Log likelihood
Durbin-Watson stat
-201.0297
2.370373
Figure 3.10 EViews Regression Output
Slide 3.13
Undergraduate Econometrics, 2nd Edition Chapter 3
Dependent Variable: FOODEXP
Analysis of Variance
Source
DF
Sum of
Squares
Mean
Square
F Value
Prob>F
Model
Error
C Total
1
38
39
25221.22299
54311.33145
79532.55444
25221.22299
1429.24556
17.647
0.0002
Root MSE
Dep Mean
C.V.
37.80536
130.31300
29.01120
R-square
Adj R-sq
0.3171
0.2991
Parameter Estimates
Variable
DF
Parameter
Estimate
Standard
Error
T for H0:
Parameter=0
Prob > |T|
INTERCEP
INCOME
1
1
40.767556
0.128289
22.13865442
0.03053925
1.841
4.201
0.0734
0.0002
Figure 3.11 SAS Regression Output
Slide 3.14
Undergraduate Econometrics, 2nd Edition Chapter 3
3.3.4
Other Economic Models
The log-log model ln( y ) = 1 + 2 ln( x)
The derivative of ln(y) with respect to x is
d [ln( y )] 1 dy
=
dx
y dx
The derivative of 1 + 2 ln( x) with respect to x is
d [1 + 2 ln( x)] 1
= 2
dx
x
Setting these two pieces equal to one another, and solving for 2 gives
2 =
dy x
=
dx y
(3.3.16)
Slide 3.15
Undergraduate Econometrics, 2nd Edition Chapter 3
