6/7/2024
Multiple Regression Analysis - Inference
Gujarati (4th edition) – Chapter 8
Multiple Regression Analysis – Hypothesis testing
• The general form of the multiple regression model
• Yi = β1 + β2X2i + β3X3i + …………….. + βkXki + µi
• A multiple regression model with two explanatory variables
• Yi = β1 + β2X2i + β3X3i + µi
• The estimated equation is (by OLS)
• To test the hypothesis about the population parameters we assume µi is
normally distributed with zero mean and fixed variance, σ2.
• This ensures the estimators (𝛽 , 𝛽 , and 𝛽 ) are normally distributed with
their respective means and variances.
1
6/7/2024
Multiple Regression Analysis – Hypothesis testing
• The following variables follow the t distributions with (n-3) df.
• The t distribution can be used to construct confidence intervals as well as test
statistical hypotheses. CI for β2
• 𝛽 −𝑡 / se(𝛽 ) ≤ β2 ≤ 𝛽 + 𝑡 / se(𝛽 )
Multiple Regression Analysis – Hypothesis testing
• Hypothesis testing in multiple regression:
• Testing hypotheses about an individual partial regression coefficient
• Testing the overall significance of the estimated multiple regression model,
• that is, finding out if all the partial slope coefficients are simultaneously equal to zero
• Testing that two or more coefficients are equal to one another
• Testing that the partial regression coefficients satisfy certain restrictions
• Testing the stability of the estimated regression model over time or in different cross-
sectional units
• Testing the functional form of regression models
•
2
6/7/2024
8.2 EXAMPLE 8.1: CHILD MORTALITY EXAMPLE REVISITED
Multiple Regression Analysis – Hypothesis testing
• Hypothesis testing about individual partial regression coefficients
• With the assumption that the error term is normally distributed, we can use t-test to
test a hypothesis about any individual partial regression coefficient.
• H0: β2 = 0
• H1: β2 ≠ 0
• If the absolute value of tcalculated > tcritical at the chosen level of significance, we
may reject the null hypothesis; otherwise, we may not reject it.
• We can also do a one-tail test.
• Since a priori Child mortality and PGNP are negatively related:
• H0: β2 < 0 (A right tail test)
• H1: β2 ≥ 0 (if tcal>tcritical, reject the null).
3
6/7/2024
8.4 Hypothesis testing about individual regression coefficients
8.5 Testing the overall significance of the sample regression
4
6/7/2024
Testing the overall significance of the sample regression
5.9 Regression analysis and analysis of variance (ANOVA)
• TSS = ESS + RSS
• TSS = Total sum of squares
• ESS = Explained (or regression) sum of squares
• RSS = Residual (or error) sum of squares
• A study of these components of TSS is known as the analysis of variance
(ANOVA) from the regression viewpoint.
5
6/7/2024
ANOVA Table: A General K-1 variable case
Sources of variation SS df MSS
Due to regression ESS (=∑ 𝑦 ) K-1 ESS/(K-1)
Due to residuals RSS ( = ∑ 𝜇̂ ) n-K RSS/(n – K)
Total TSS ( = ∑ 𝒚𝟐𝒊 ) n-1
Here, SS means sum of squares, df is degrees of freedom, and MSS is Mean
sum of squares. K is the number of parameters to be estimated and n is the
sample size
The Analysis of Variance Approach to Testing the Overall Significance three variable
Regression: The F Test
6
6/7/2024
The Analysis of Variance Approach to Testing the Overall Significance three variable Regression:
The F Test
The Analysis of Variance Approach to Testing the Overall Significance
three variable Regression: The F Test
7
6/7/2024
5.9 Regression analysis and analysis of variance – two variable case
5.9 Regression analysis and analysis of variance
8
6/7/2024
Testing overall significance – General F-test
• The general form of the multiple regression model
Yi = β1 + β2X2i + β3X3i + …………….. + βkXki + µi (8.5.9)
• Testing overall significance
• H0: β2 = β2 = …….. = βk = 0 (all slope coefficients are simultaneously zero)
• Ha = Not all slope coefficient are simultaneously zero
• The test statistics follow a F distribution as follows with (K-1) df in the numerator
and (n-k) df in the denominator.
( )
If Fcalculated is greater than Fcritical at a given significance
• 𝐹= level, then the Null hypothesis is rejected at that
( ) significance level
Testing the Overall Significance of a Multiple Regression: The F Test
9
6/7/2024
An Important Relationship between R2 and F
where use is made of the definition R2 =
ESS/TSS. The equation on the left
shows how F and R2 are related. These
two vary directly. When R2 = 0, F is zero
ipso facto. The larger the R2, the greater
the F value. In the limit, when R2 = 1, F
is infinite. Thus the F test, which is a
measure of the overall significance of
the estimated regression, is also a test
of significance of R2. In other words,
testing the null hypothesis (8.5.9) is
equivalent to testing the null hypothesis
that (the population) R2 is zero.
8.6 Testing The Equality of Two Regression Coefficients
10
6/7/2024
8.6 Testing The Equality Of Two Regression Coefficients (cont.)
8.6 Testing The Equality of Two Regression Coefficients (cont.)
11
6/7/2024
8.6 Testing The Equality Of Two Regression Coefficients
8.7 Restricted Least Squares: Testing Linear Equality Restrictions
12
6/7/2024
8.7 Restricted Least Squares: Testing Linear Equality Restrictions
8.7 Restricted Least Squares: Testing Linear Equality Restrictions (cont.)
13
