Modelling Relationships with
Regression
Debanjan Mitra
Indian Institute of Management Udaipur
November 3, 2020
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Data: Advertising Expenditure
Data Source: James, Witten, Hastie, and Tibshirani (2017)
Data on:
Sales (in thousands on units) for a particular product
Advertising budget (in thousands of dollars) for TV, radio, and
newspaper meda
Data from 200 markets
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Data: Advertising Expenditure
Table: Advertising Expenditure in Different Media and Sales
Serial No. TV Radio Newspaper Sales
1 230.1 37.8 69.2 22.1
2 44.5 39.3 45.1 10.4
3 17.2 45.9 69.3 9.3
4 151.5 41.3 58.5 18.5
5 180.8 10.8 58.4 12.9
. . . . .
. . . . .
. . . . .
. . . . .
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Questions of Interest
Is there a relationship between advertising budget in TV and sales?
How strong is the relationship between advertising budget in TV and
sales?
Which media contribute to sales?
Can we estimate the effect of each medium on sales?
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Questions of Interest
Which media generate the biggest boost in sales?
How much increase in sales is associated with a given increase in TV
expenditure?
How can we predict future sales based on advertising budget in
different media?
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Our Goal
To understand the relationship of sales and advertising expenditure
in different media
And in particular, to predict sales, based on advretising expenditure
in all or some of the media
Note that, here Sales is the response, and advertising expenditure in
TV, radio, and newpaper media are predictors
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Exploration Begins With a Scatterplot
Scatterplot Matrix
0 10 20 30 40 50 5 10 15 20 25
250
TV
0 100
40
Radio
20
0
80
Newspaper
40
0
25
15
Sales
5
0 50 150 250 0 20 40 60 80
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
A More Detailed Look
TV Radio Newspaper Sales
0.004
0.003
Corr: Corr: Corr:
TV
0.002
0.055 0.057 0.782***
0.001
0.000
50
40
Corr: Corr:
Radio
30
20 0.354*** 0.576***
10
0
Newspaper
90
Corr:
60
0.228**
30
0
20
Sales
10
Debanjan Mitra 0 100 200 300 0 10 20 30 40 50 0 30 60 90Indian Institute10 20
of Management Udaipur
Modelling Relationships with Regression
Our Aprroach to Exploration
We shall fit
Simple regression models with TV, radio, and newspaper advertising
expenditures as individual predictors
A multiple regression model with all three media expenditures as
predictors
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Predictor: TV
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Predictor: Radio
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Predictor: Newspaper
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Predictors: TV, Radio, and Newspaper
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Significance of a Predictor
How to know whether a pedictor is indeed useful or not?
In the Spreadsheet output, we have a quantity called p-value
The p-value indicates whether a predictor is significant for the
regression model or not
If the p-value is small, the predictor is useful
If the p-value is large, the predictor is not useful
We compare the p-value with a cut-off value (usually 0.05), and
decide whether it is small or large
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Example: Significance of a Predictor
In the combined model, the p-values corresponding to TV, Radio,
and Newspaper advertising expenditures are 0, 0, and 0.86,
respectively
Useful or significant predictors: TV, Radio advertising expenditures
Insignificant predictor: Newspaper advertising expenditure!
A curious observation:
From the individual models, the p-values for TV, Radio, and
Newspaper advertising expenditures are 0, 0, and 0.001, respectively
This implies, individually, all predictors are significant!
Conflict? What is happening really?
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Newspaper: Significant or Not?
TV Radio Newspaper Sales
0.004
0.003
Corr: Corr: Corr:
TV
0.002
0.055 0.057 0.782***
0.001
0.000
50
40
Corr: Corr:
Radio
30
20 0.354*** 0.576***
10
0
Newspaper
90
Corr:
60
0.228**
30
0
20
Sales
10
Debanjan Mitra 0 100 200 300 0 10 20 30 40 50 0 30 60 90Indian Institute10 20
of Management Udaipur
Modelling Relationships with Regression
Newspaper: Significant or Not?
The correlation between radio and newspaper is 0.35
Implication: tendency to spend more on newspaper in markets where
more is spent on radio
Explanation:
Newspaper advertising has no direct impact on sales
Radio advertising does increase sales
Then, in markets where we spend more on radio our sales will tend
to be higher
We also spend more on newspaper in those same markets
Thus, in a simple linear regression with sales and newspaper,
newspaper seems to be signficant
Lurking variable: Radio!
This explanation is supported by the results of the individual and full
regression!
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Is The Regression Model Useful?
Is there a relationship between advertising budget and sales? How
strong is the relationship?
Coefficient of Determination, R 2 , is 89.7%
Implication: This model explains about 90% of the total variation in
sales
Useful, for sure! And quite a strong relationshipt between predictors
and response!
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Which media contribute to sales?
Which media contribute to sales?
The p-values of TV and Radio are close to zero (tiny)
Newspaper has a high p-value
TV and Radio significant, and Newspaper is not
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
Can we estimate the effect of each medium
on sales?
How much increase in sales is associated with a given increase in
expenditures in these media?
The coefficient of TV is 0.046
The coefficient of Radio is 0.189
Radio has a larger effect!
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
How can we predict sales using this model?
Suppose advertising expenditure in TV, Radio, and Newspaper are
214.7, 24, 4 thousand dollars, respectively
The predicted sales from the model would be
Predicted Sales = 2.939 + 0.046 ∗ 214.7 + 0.189 ∗ 24 − 0.001 ∗ 4
= 17.2851
How close is the prediction?
“Standard Error” in the regression output gives an idea about
closeness of prediction
The actual sales in each market deviate from the predicted sales by
1686 units, on average
Debanjan Mitra Indian Institute of Management Udaipur
Modelling Relationships with Regression
