MULTIPLE LINEAR
REGRESSION
NURULJANNAH BT NOR AZMI
INTRODUCTION
Multiple linear regression is used
to estimate the relationship
between two or more independent
variables and one dependent
variable.
Dependent (outcome) : numerical
Independent (predictor) : 2 or more
numerical variables
INTRODUCTION
If independent variables are
combination of numerical and
categorical or categorical only -
General Linear Regression
Dependent (outcome) : numerical
Independent (predictor) : 2 or more
combination of numerical and
categorical or categorical only
SIMPLE LINEAR REGRESSION - ONLY ONE INDEPENDENT VARIABLE
Independent variable (x) Dependent variable (y)
Mother's height Length of baby
MULTIPLE LINEAR REGRESSION - MORE THAN ONE INDEPENDENT VARIABLES
Independent variables (x) Dependent variable (y)
Mother's height
Mother's weight Length of baby
Age
You can use multiple linear
regression when you want to
know:
When to 1. How strong the relationship is
between two or more independent
apply variables and one dependent variable
Multiple (e.g. how mother's height, weight and
age affect length of baby).
Linear 2. The value of the dependent variable at
a certain value of the independent
Regression? variables (e.g. the expected length of
baby at certain levels of mother's
height, weight and age)
Multiple Linear Regression Model
Y= 0 + 1 X1 + 2 X2 + 3 X 3 + ........ n Xn
Y = outcome
0 = intercept
1 ........ n = regression coefficient for independent variable
X 1 ........ X n = independent variable
STEPS IN MULTIPLE LINEAR REGRESSION
1 Descriptive statistics
2 Simple linear regression (Univariable analysis)
3 Multiple linear regression (Multivariable analysis)
4 Checking multicollinearity & interaction (Preliminary final model)
5 Checking assumptions (final model)
6 Interpretation & presentation
EXAMPLE
Open dataset:
birthweight.sav
This dataset contains information on new born babies and their
parents admitted in Hospital Kuala Lumpur. A researcher is
interested to determine the factors that are associated with the
length of baby.
EXAMPLE
RQ: What are the factors that associated with the length of baby?
Length of baby (DV) Factors (IV) List down all the variables
Mother's age
Mother's height
Mother's weight
Identify the types of
Numerical Numerical variables
Identify the right
Multiple Linear Regression statistical analysis
STEP 1: DESCRIPTIVE STATISTICS
1.Data exploration and cleaning.
2.For categorical data, run the data by using Frequencies in SPSS.
3.For numerical data, run the data by using Descriptives/Explore in
SPSS.
Run frequencies for categorical data
Go to: Analyze > Descriptive statistics > Frequencies
Enter
categorical
variables
Run descriptive for numerical data
Go to: Analyze > Descriptive statistics > Descriptives/Explore
Enter numerical
variables
STEP 2: SIMPLE LINEAR REGRESSION (UNIVARIABLE ANALYSIS)
1.Do Simple Linear Regression analysis for each independent
variable:
Mother's age
Mother's height
Mother's weight
2.At the end, choose variables with p-value < 0.25 and/or clinically
important.
Go to: Analyze > Regression > Linear
Length of baby vs Mother's age
There is a significant relationship between mother's age and the length of baby.
Length of baby vs Mother's height
There is a significant relationship between mother's height and the length
of baby.
Length of baby vs Mother's weight
There is a significant relationship between mother's pre-pregnancy weight
and the length of baby.
Table 1: Associated factors of the length of baby by Simple Linear
Regression
STEP 3: MULTIPLE LINEAR REGRESSION
(MULTIVARIABLE ANALYSIS)
1.Variables selection can be done by using following methods:
Forward
Backward
Stepwise
2. Perform all the methods and select the model with all
variables significant as the preliminary main effect model.
Go to: Analyze > Regression > Linear
METHOD: FORWARD (Automatically enters the IMPORTANT independent
variable into the model)
Enter all the
selected
variables
Select Forward
METHOD: FORWARD
Mother's age and height are significant.
METHOD: BACKWARD (Automatically removes the UNIMPORTANT
independent variable out of the model)
Select Backward
METHOD: BACKWARD
Mother's age and height are significant.
METHOD: STEPWISE (The procedure adds or removes independent variables
one at a time using the variable’s statistical significance)
Select Stepwise
METHOD: STEPWISE
Mother's age and height are significant.
During this step, mother's age and height were found to be
significant in all methods.
Run the model once again using 'Enter' method by using the
chosen variables.
This will be the preliminary main effect model.
STEP 4: CHECKING MULTICOLLINEARITY
1.Multicollinearity occurs when independent variables in a
regression model are correlated.
2. This correlation is a problem because independent variables
should be independent.
3.If the degree of correlation between variables is high enough,
it can cause problems when you fit the model and interpret the
results.
4. There is a high chance of getting inaccurate p-values and wide
confidence interval of regression coefficient.
STEP 4: CHECKING MULTICOLLINEARITY
5. Multicollinearity can be checked by using Variance Inflation
Factor (VIF).
6. If VIF is more than 10, then there is a multicollinearity
amongst independent variables.
Go to: Analyze > Regression > Linear
The values of VIF for both variables are less than 10. There is no
multicollinearity problem in this model.
STEP 4: CHECKING INTERACTION
1.An interaction effect occurs when the effect of one variable
depends on the value of another variable.
2.The interaction terms need to be biologically meaningful.
3.The interaction term needs to be computed in SPSS and then
added to the model as an independent variable. If you have more
than one interaction term, add to the model one by one.
4.If the interaction term is statistically significant, include the
term in the model.
Go to: Transform > Compute variable
age*height
mage*mheight
Go to: Analyze > Regression > Linear
add interaction term
The interaction age_height is not statistically significant
(p=0.909).
STEP 5: CHECKING ASSUMPTIONS
Assumptions How to check?
1.Independent observation Done during design stage
Scatter plot between residuals and
2.Overall linearity
predicted values (XP - YR)
3.Homoscedasticity Scatter plot between residuals and
(Equal variances) predicted values (XP - YR)
Scatter plot residual vs each independent
4.Linearity of each independent variable
variable (XI - YR)
5.Residuals should be approximately Histogram with overlaid normal curve of
normally distributed residuals
Checking assumption: Overall linearity & Homoscedasticity
Go to: Analyze > Regression > Linear
Go to: Graph > Legacy Dialogs > Scatter/Dot
XP - YR
Double click the plot and click
Linearity:
If there is a peculiar shape of concavity or convexity, then assumption is NOT
MET.
Since there is no peculiar shape, linearity assumption is MET.
Homoscedasticity (Equal variance):
If there is a peculiar shape of divergence or convergence or fan-shape, then
assumption is NOT MET.
Since there is no peculiar shape, homoscedasticity assumption is MET.
Example of non-linear
relationship
Checking assumption: Linearity of each independent variable
Go to: Graph > Legacy Dialogs > Scatter/Dot
Mother's age vs Residual
XI - YR
Double click the plot and click
There is no peculiar shape, linearity assumption is MET.
Mother's height vs Residual
XI - YR
Double click the plot and click
There is no peculiar shape, linearity assumption is MET.
Checking assumption: Normality distribution of residuals
Go to: Graphs > Legacy Dialogs > Histogram
Residuals are normally distributed. Assumption
is met.
STEP 6: INTERPRETATION AND PRESENTATION
Run the final model. All the assumptions were checked and MET.
STEP 6: INTERPRETATION AND PRESENTATION
Table 2: Factors associated with the length of baby in HKL (n=42)
STEP 6: INTERPRETATION AND PRESENTATION
There is a significant linear negative relationship between mother's age
and the length of baby. For every one-year increase in the mother's age,
the baby's length is 0.28 cm lower. (adjusted b = -0.28; 95% CI
-0.37,-0.19; p<0.001)
There is a significant linear positive relationship between mother's height
and the length of baby. For every 1 cm increase in the mother's height,
the baby's length increases by 0.14 cm. (adjusted b = 0.14; 95% CI
0.05,0.24; p=0.004)
62.1% of the variation in the length of baby is explained by mother's age
and height according to the multiple linear regression model (R2 = 0.621).
MDM NURULJANNAH
BT NOR AZMI
EMAIL: [email protected]
