INFERENTIAL ANALYSES II
(RELATIONSHIPS)
Dr. Abdul Rahman Mahmoud Fata Nahhas
KOP – IIUM
Final Year Research Project
SEM 1
2023-24
CORRELATION ANALYSIS
Introduction
Correlation measures the strength and direction of a
relationship that exists between two variables
Partial correlation: Three or more variables are included,
& correlation between two variables is explored, while the
effect of others is removed
E.G. Correlation between blood pressure and amount of
salt intake after adjustment for the effect of a third variable;
such as amount of fluid intake
Introduction
Example (positive correlation)
Typically, in the summer as the temperature
increases people are thirstier, consuming
more water
Introduction
Water
Temperature Consumption
For seven (C) (Liters)
random summer days, 25 1
a person recorded the
temperature and his
29 1.3
water consumption, 35 1.7
during a three-hour 37 1.9
period spent outside 39 2
41 2.3
44 3.1
Introduction
3.5
3
Water Consumption (L)
2.5
1.5
0.5
0
20 25 30 35 40 45 50
Temperature (C)
Introduction
Correlation treats all variables equally
Correlation does not take into consideration
whether a variable has been classified as a
dependent or independent variable
Introduction
For instance, you might want to find out whether
basketball performance is correlated to a
person's height
Thus, you’ll plot a graph of performance against
height and calculate the correlation coefficient r
If - let’s say - r = 0.72, hence, we can conclude
that as height increases so does basketball
performance
Types of correlation
Two main types of Correlation Analysis
Pearson product-moment Spearman's Rank-Order
correlation (Parametric) Correlation (Non-Parametric)
REQUIRES DOES NOT REQUIRE
A normally distributed Pearson correlation
data assumptions
A linear relationship
between the two
variables in question
No heteroscedasticity
Pearson product-moment correlation
A parametric measure of the strength and direction of a
linear relationship that exists between two continuous
variables
Denoted by the symbol r
Attempts to draw a line of best fit through the data of two
variables
The Pearson correlation coefficient, r, indicates how far
away all these data points are to this line of best fit (i.e.,
how well the data points fit this new line of best fit)
Spearman Rank-order Correlation
A nonparametric measure of the strength and direction of
relationship that exists between two variables measured on at
least an ordinal scale
Denoted by the symbol rs (or the Greek letter ρ, pronounced
rho)
Used for either ordinal variables or for continuous data that
has failed the assumptions necessary for conducting the
Pearson's product-moment correlation
Detecting a linear relationship
How can you detect a linear relationship
between tested variables?
Simply by plotting the variables on a graph
(a scatterplot, for example) and visually
inspecting the graph's shape and observe the data
points and their location compared to the line of
best fit
Detecting a linear relationship
3.5
2.5
2 Linear relationship
1.5
0.5
0
5 10 15 20 25 30 35 40 45 50 55
Detecting a linear relationship
3.5
2.5
2 Linear relationship
1.5
0.5
0
0 10 20 30 40 50 60
Detecting a linear relationship
1.2
0.8
0.6 Non-linear relationship
0.4
0.2
0
10 12 14 16 18 20 22
Detecting a linear relationship
1.8
1.6
1.4
1.2
1
Curvilinear relationship
0.8
0.6
0.4
0.2
0
5 10 15 20 25 30
Correlation Coefficient
With the help of Correlation Coefficient, we can
determine:
1. The DIRECTION of the relation →
Positive or Negative
2. The STRENGTH of the relation among the
variables
Direction of Correlation
3.5
Positive
3
Correlation
Water Consumption (L)
2.5
1.5
0.5
0
20 25 30 35 40 45 50
Temperature (C)
Direction of Correlation
6
Negative
5 Correlation
4
Stress Score
0
15 20 25 30 35 40 45 50
Work Performance Score
Strength of Correlation
Strength of Coefficient
Correlation
Positive Negative
Small 0.1 to 0.29 - 0.1 to - 0.29
Medium 0.3 to 0.49 - 0.3 to - 0.49
Large 0.5 to 1 - 0.5 to - 1
Strength of Correlation
If r (or rs) equals zero, then there is NO
RELATIONSHIP between the two variables
r = 1 → perfect positive linear relationship
r = -1 → perfect negative linear relationship
Strength of Correlation
Achieving a value of +1 or -1 means that all your
data points are included on the line of best fit
4.5 4
4
There are no data3.5points that show
3.5
any variation away 3 from this line
3
2.5
2.5
2
2
r = -1 r = +1
1.5
1.5
1
1
0.5 0.5
0 0
5 10 15 20 25 30 35 40 45 50 5 10 15 20 25 30 35 40 45 50 55
REGRESSION ANALYSIS
Definition
A predictive statistical method that investigate
the strength of the relationship between TWO
SETS of variables
It studies the dependence of one or more
variables (dependent variables) on one or more
variables (independent or predictor variables)
Regression Main Purposes
Regression PRIMARILY used to:
1. Estimate (describe) the relationship that exists between
the dependent variable(s) and the explanatory variable(s)
2. Determine the strength of impact of each of the predictor
variables on the dependent variable(s), controlling the
effects of all other predictor variables
3. Predict the value of dependent variable(s) for a given value
of the predictor variable(s)
Regression Equation
Can be obtained from all types of regression analysis
Once known, regression equation is used to predict
values of dependent variables, given the values of
independent (predictor) variables
E.g., if we knew a person's weight, we can then
predict their blood pressure using regression
equation
Regression Equation
E.g., using the simple linear regression model, an equation
obtained can be as the following:
Y = β0 + β1 *X + e
Typically Y is referred to the dependent variable, &
X as the independent variable
β0 is the intercept of the estimated line i.e. the value of Y
when X = 0
β1 is the gradient of the estimated line [slope of the line] i.e.
the amount by which Y change with one unit change of X
e is the error term or disturbance in the relationship,
represents factors other than X that affect Y
Types of Regression
Regression analysis is generally classified into two types
Simple Multiple
Regression involves only Regression involves more
two variables, one of than two variables,
which is dependent MAINLY, one of which is
variable and the other is dependent variable and the
explanatory (independent) others are explanatory
variable (independent) variables
The associated model will The associated model will
be a simple regression be a multiple regression
model model
Types of Regression
Type of dependent variables
Continuous Categorical
Number of Predictor Variables
1 Simple Linear Simple Logistic
>1 Multiple Linear Multiple Logistic
Linear Regression
Linear Regression establishes a relationship
between dependent (Continuous) variable
(Y) and one or more independent (predictor)
variables (X) using a best fit straight
line (also known as regression line)
Linear Regression
E.g., Predicting patients measured blood glucose level (in
mg/dl) based on dose of insulin infusion (in IU) … SIMPLE
LINEAR REGRESSION
Presume a sample of 20 DM patients for whom insulin infusion was
administered
We can plot the values on a graph, with insulin dose on the X axis
and blood glucose on the Y axis
If there were a perfect linear relationship between insulin dose and
blood glucose, then all 20 points on the graph would fit on a straight
line (But, this is never the case [unless your data are rigged])
Linear Regression
E.g., Predicting patients measured blood glucose level (in
mg/dl) based on dose of insulin infusion (in IU) … SIMPLE
LINEAR REGRESSION
If there is a (non-perfect) linear relationship between insulin dose
and blood glucose (presumably a negative), then we would get a
cluster of points on the graph which slopes downward
In other words, as insulin dose is increased; blood glucose level
declines…
Linear Regression
6
Y = β0 + β1 * X + e
5
4
BG = - 7.15 + .095 * Insulin dose
Glucose level
0
10 20 30 40 50 60 70 80
Inslin dose
Linear Regression
MULTIPLE LINEAR REGRESSION is the same idea as
simple linear regression, except that we have several
independent variables predicting the dependent variable
To continue with the previous example, assume that we now
want to predict a patient’s BG from insulin dose and gender
as well. In other word, we need to see if gender has also an
impact on the measured BG
In this case independent variables (Predictors) are Insulin
dose & Gender; while dependent variable is BG
Linear Regression
Multiple regression tells us the predictive
value of the overall model; all predictor
variables…
In our example, then, the regression would
tell us how will Insulin dose and Gender
predict a patient’s BG
Linear Regression
DETERMINES THE STRENGTH OF IMPACT OF EACH OF THE PREDICTOR
VARIABLE ON THE DEPENDENT VARIABLE(S), CONTROLLING THE EFFECTS
OF ALL OTHER EXPLANATORY VARIABLES
Multiple regression ALSO tells us how well each
predictor variable predicts the dependent variable,
controlling for each of the other predictor variables…
In our example, then, the regression would tell us how
will Insulin dose predicts a patient’s BG, while
controlling for Gender, as well as how will Gender
predict a patient’s BG, while controlling for Insulin
dose
Linear Regression
Assumptions
1. Number of cases: When doing regression, the cases-to-
Independent Variables (IVs) ratio should ideally be 20:1;
that is 20 cases for every IV in the model. The lowest your
ratio should be is 5:1 (i.e., 5 cases for every IV
in the model)
2. Normality: the scores for each variable should be
normally distributed
3. Linearity: There must be linear relationship between
independent and dependent variables
Linear Regression
Assumptions
4. Absence of Multicollinearity: Multicollinearity exists when the
independent variables are highly correlated (r=.9 and above)
5. Absence of Singularity: Singularity occurs when one
independent variable is actually a combination of other
independent variables (e.g. when both subscale scores and the
total score of a scale are included)
6. Outliers: Linear regression is very sensitive to outliers (very
high or very low value on a particular item). Outliers can terribly
affect the regression line and eventually the forecasted values
Logistic Regression
Used to find the probability of event of
Success and event of Failure
Used when the dependent variable is binary
(0/ 1, True/ False, Yes/ No) in nature
Logistic Regression
E.g., Predicting if a group of people having depression
or no (depression Yes/No) based, for instance, on
place of residence (Urban/Rural)… SIMPLE
LOGISTIC REGRESSION
Presume a sample of 50 persons whom depression
was assessed by a psychologist
Person’s place of residence was reported
Logistic Regression
E.g., Predicting if a group of people having depression or no
(depression Yes/No) based for instance on place of residence
(Urban/Rural)… SIMPLE LOGISTIC REGRESSION
On a graph, we can plot the result of depression
assessment (Y/N) on the Y axis and the results of
reported place of residence (U/R) on the X axis
From the graph, we can infer if depression is more
likely to be present among urban or rural persons
Logistic Regression
Assumptions
Number of cases: When doing regression, the cases-to-
Independent Variables (IVs) ratio should ideally be 20:1; that is
20 cases for every IV in the model. The lowest your ratio
should be is 5:1 (i.e., 5 cases for every IV
in the model)
Normality: Logistic regression doesn’t require that data to be
normally distributed (non-parametric test)
Linearity: Logistic regression doesn’t require linear
relationship between dependent and independent variables
Logistic Regression
Assumptions
Absence of Multicollinearity: Multicollinearity exists when the
independent variables are highly correlated (r=.9 and above)
Absence of Singularity: Singularity occurs when one
independent variable is actually a combination of other
independent variables (e.g. when both subscale scores and the
total score of a scale are included)
Outliers: Logistic regression is sensitive to outliers (very high or
very low value on a particular item). Outliers can terribly affect
the regression line and eventually the forecasted values
THANK
YOU!
