Econometrı́a I
Regresión lineal simple
UPB
Semestre II - 2022
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 1 / 17
Introducción
Regresión lineal simple
Ecuación lineal
y = a + bx
Regresión lineal simple
y = β0 + β1 x + u
y , variable dependiente (variable explicada)
x, variable Independiente (variable explicativa, covariate)
β0 + β1 , constantes
u, error
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 2 / 17
Introducción
Supuestos, u
Mean of (random) error is 0
Variable explicativa y error no pueden estar correlacionados
No afecta a la pendiente, E (u) = α0
Zero condicional mean, E (u|x)
Cov (x, u) = 0
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 3 / 17
OLS
Ordinary least squares (OLS)
y = β0 + β1 x + u
Estimar parámetros para una muestra
(xi , yi ), i = 1, ..., n
yi = β0 + β1 xi + ui
E (y |x)
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 4 / 17
OLS
Regression line
Sample data points
Associated error terms
(xi , yi ), i = 1, ..., n
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 5 / 17
OLS
Derivación estimadores OLS
E (u) = 0
Cov (x, u) = 0
Math
1 Pn
n i=1 (x) = x̄
n
xi (xi − x̄) = Pni=1 (xi − x̄)2
P P
Pi=1
n n
i=1 xi (yi − ȳ ) = i=1 (xi − x̄)(yi − ȳ )
Moment restriction
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 6 / 17
OLS
Slope
Pn
(x − x̄)(yi − ȳ )
βˆ1 = Pn i
i=1
2
i=1 (xi − x̄)
Estimate the sample covariance between x and y divided by the
sample variance of x
The correlation determines the slope
OLS fits a line through the sample points such that the sum of
squared residuals is as small as possible, hence the term least squares
Estimate of the error, û = yi − yˆi
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 7 / 17
OLS
Properties of OLS
Residual error is different from the error
OLS residuals add up to 0
Sample covariance (correlation) between regressor and residual is 0
Sample average is the same as the sample average of the fitted values
Sample covariance of fitted values and residuals is zero
The OLS regression line always goes through the mean of the sample
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 8 / 17
OLS
Analysis of residuals
yi = yˆi + ûi
(yi − ȳ )2
P
Total sum of squares (SST),
Explained sum of squares (SSE), (ŷi − ȳ )2
P
P 2
Residual sum of squares (SSR), ûi
Proof:
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 9 / 17
OLS
Goodness-of-Fit
SST = SSE + SSR
R 2 is the fraction of the total variation in yi that is explained by xi
0 ≤ R2 ≤ 1
R2 = 0
R2 = 1
Having a high R 2 is neither necessary nor sufficient to infer causality.
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 10 / 17
OLS
Units of measurement and functional form
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 11 / 17
OLS
Unbiasedness assumptions
Required assumptions
Linear parameters
Random sampling
Sample variation
Zero conditional mean
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 12 / 17
OLS
Unbiasedness of OLS estimators
Pn Pn
Pni=1 (x i − x̄)(yi − ȳ ) = i=1 (xi − x̄)yi
Pi=1 (xi − x̄) = 0 P
n n 2
i=1 xi (xi − x̄) = i=1 (xi − x̄)
OLS unbiased estimators proof
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 13 / 17
OLS
Variance of OLS estimator
Now we know that the sampling distribution of our estimate is
centered around the true parameter
Variance under an additional assumption
Homoskedasticity
Var (u|x) = σ 2
Var (u|x) = E (u 2 |x) − [E (u|x)]2 = σ 2
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 14 / 17
OLS
Variance of OLS estimator
Homoskedasticity
Heteroskedastic
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 15 / 17
OLS
Variance
1 2X 1 2X 2
Var (βˆ1 ) = ( ) di Var (ui ) = σ 2 ( ) di
SSTx SSTx
1 2 σ2
= σ2( ) SSTx = = Var (βˆ1 )
SSTx SSTx
The larger the error variance, σ 2 , the larger the variance of the slope
estimate
The larger the variability in the xi , the smaller the variance of the
slope estimate
A larger sample size should decrease the variance of the slope estimate
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 16 / 17
OLS
Estimating variance error
ûi = yi − βˆ0 − βˆ1 xi
An unbiased estimator
P
ûi 2 SSR
σˆ2 = =
(n − 2) n−2
Standard error q
σ̂ = σˆ2
Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 17 / 17
