Business Analytics
Fall 2021
Week-04
Making Numerical Predictions
Simple Linear Regression
Question: How much? or How many?
uses regression algorithms
Regression algorithms make numerical predictions,
such as:
What will be next month car sales?
What will be next week’s temp?
They help answer any question that asks for a
number.
Predict the values of a dependent (response) variable
based on values of at least one independent
(explanatory) variable
Scatterplot
Examine possible relationships between two numerical
variables
Types of Regression Models
Positive Linear Relationship Relationship NOT Linear
Negative Linear Relationship No Relationship
Chap 13-4
Simple Linear Regression Model
Simple: One (independent) variable
Linear: Relationship between Variables is
Described by a Linear Function
The Change of One Variable Causes the Other
Variable to Change
A Dependence of One Variable on the Other
Introduction to Linear Regression
(cont.)
Any straight line can be represented by an
equation of the form Y = a + bX, where ‘b’ and
‘a’ are constants.
The value of ‘b’ is called the slope constant and
determines the direction and degree to which the
line is tilted.
The value of ‘a’ is called the Y-intercept and
determines the point where the line crosses the
Y-axis.
6
Introduction to Linear Regression
(cont.)
How well a set of data points fits a straight line can be
measured by calculating the distance between the data points
and the line.
The total error between data points and the line is obtained by
squaring each distance & then summing the squared values.
Regression equation: Minimize the sum of squared errors.
Error = Actual – Predicted
8
Regression Equation
Calculating a and b
Simple Linear Regression:
Example 1
• Determine / predict salary column values (Dependent
Variables) based on years of experience.
Example Solution
Finding the regression equation
X Y X2 XY
2 15 n=9 (no of data points)
3 28
5 42
13 64
8 50
16 90
11 58
1 8
9 54
Avg(X) Avg(Y) Sum(X2) Sum(XY)
Example Solution
Finding the regression equation
X Y X2 XY
2 15 4 30
n=9
3 28 9 84
5 42 25 210
13 64 169 832
8 50 64 400
b = 4.80
16 90 256 1440
11 58 121 638 a = 45.44 – (4.80 x 7.56)
1 8 1 8 a = 9.18
9 54 81 486
Y = 9.18 + 4.80 X
Sal = 9.18 + 4.80 (Exp)
Avg(X) = 7.56 Sum(X2) = 730
Avg(Y) = 45.44 Sum(XY) = 4128
Interpretation of Results: Example
Interpreting the slope
Y = 9.18 + 4.80 X
Sal = 9.18 + 4.80 (Exp)
The slope of 4.80 means that for each increase of
one unit in X, we predict the average of Y to
increase by an estimated 4.80 units.
The equation estimates that for each increase of 1
year in experience, the expected salary is
predicted to increase by $4800 (4.80 x 1000$).
Making predictions
Sal = 9.18 + 4.80 (Exp)
What is the predicted salary for someone with
an experience of
6 years
14 years
Example Solution
Predicted Values & Residuals (errors)
X Y Pred Y Residual Y = 9.18 + 4.80 X
2 15 18.78 -3.78
3 28 23.58 4.42
5 42 33.18 8.82
13 64 71.58 -7.58
8 50 47.58 2.42
16 90 85.97 4.03
11 58 61.98 -3.98
1 8 13.98 -5.98
9 54 52.38 1.62
What is the predicted salary for experience of 10
years? What is the prediction error?
Evaluating Regression Results
The ability of the regression equation to accurately
predict the Y values is measured by first computing
the proportion of the Y-score variability that
is predicted by the regression equation and the
proportion that is not predicted.
Measures of Variation:
The Sum of Squares
(continued)
_
Y
X
Xi
Measures of Variation:
The Sum of Squares
(continued)
Y
SSE =(Yi - Yi )2
_
SST = (Yi - Y)2
_
SSR = (Yi - Y)2
_
Y
X
Xi
Measures of Variation:
The Sum of Squares
(continued)
SST = Total Sum of Squares
Measures the variation of the Yi values around
their mean, Y
SSR = Regression Sum of Squares
Explained variation attributable to the relationship
between X and Y
SSE = Error Sum of Squares
Variation attributable to factors other than the
relationship between X and Y
Measures of Variation:
The Sum of Squares
SST = SSR + SSE
Total
= Explained + Unexplained
Sample
Variability Variability
Variability
Measures of Variation
The Sum of Squares: Example
Excel Output for Salary example
SSR
SSE SST
The Coefficient of Determination
SSR Regression Sum of Squares
r
2
SST Total Sum of Squares
Measures the proportion of variation in Y that
is explained by the independent variable X in
the regression model
Excel Output: Salary example
R-sq = SSR/SST
= 4980.907/5226.222
= 0.953
The Coefficient of Determination
r2
Recall:
The coefficient of determination r2 is the
proportion of variability in the response variable
“explained” by the regression.
It’s another way of saying, “By introducing this
other variable, how much better is my estimate
than it would be if I simply used the average to
make my estimate.”
The Coefficient of Determination
r2
Example: In a study of bone density as a function
of body weight, an ‘r’ of .6 is noted.
Interpretation 1 (full credit): “About 36% of the
variability in bone density is explained by the
linear regression of bone density on body weight.”
Interpretation 2 (less credit): “About 36% of
variability in bone density is accounted for by body
weight.”
Inferences about the Slope:
F Test
F Test for a Population Slope
Is there a linear relationship between Y and X ?
A significant F-ratio (p-value < 0.05) indicates that
the equation predicts a significant portion of the
variability in the Y scores (more than would be
expected by chance alone).
Chap 13-26
Important concepts
Calculating ‘a’ and ‘b’ using formula, & writing the
regression equation.
Interpreting the slope (with + or – sign).
Using the equation, making a prediction &
calculating prediction errors (residuals)
From regression output: R-sq, SSR, SSE, SST, F-
statistic and its significance, Slope and intercept
values.
Calculating & Interpreting the r-squared value
The Simple Linear Regression
• Example 2: Car Odometer Price
A car dealer wants to find 1 37388 14636
the relationship between 2 44758 14122
the odometer reading and 3 45833 14016
the selling price of used 4 30862 15590
cars. 5 31705 15568
A random sample is 6 34010 14718
selected, and the data . . .
Independent Dependent
recorded. . . x
variable . y
variable
Find the regression line. . . .
Interpret the slope
28
Scatterplot
Car Odometer Price
1 37388 14636
2 44758 14122
3 45833 14016
4 30862 15590
5 31705 15568
6 34010 14718
. . .
. . .
. . .
29
6.30
Excel Output of car price example
Note: Can get R2 from ANOVA table too.
SS Regression = SSR = 2052339
SS Total = SST = 2318134
R2 = SSR/SST = 2052339 / 231814 = 0.8853
Homework Problem: Predict annual sales based upon
Store area. Use both calculator & MS Excel.
QUESTIONS
Data for 7 Stores:
1. What is the correlation
Annual coefficient? SSR, SST, SSE?
Store Square Sales
2. What is the regression equation?
Feet ($000)
3. Interpret the slope.
1 1,726 3,681
4. Interpret R-sq value.
2 1,542 3,395
5. Interpret the F-statistic.
3 2,816 6,653
6. What’s the prediction error for
4 5,555 9,543
Store 3 and 6?
5 1,292 3,318
7. Consider a store with 2000 square
6 2,208 5,563 feet area. Predict Annual Sales for
7 1,313 3,760 this case…
