Linear Regression
• Managerial decisions are often based on the relationship between two or more variables
• Example: After considering the relationship between advertising expenditures and sales,
a marketing manager might attempt to predict sales for a given level of advertising
expenditures
• Sometimes a manager will rely on intuition to judge how two variables are related
• If data can be obtained, a statistical procedure called regression analysis can be used to develop
an equation showing how the variables are related
• Dependent variable or response: Variable being predicted
• Independent variables or predictor variables: Variables being used to predict the value of the
dependent variable
• Linear regression: A regression analysis involving one independent variable and one dependent
variable
• In statistical notation:
y = dependent variable
x = independent variable
• Simple linear regression: A regression analysis for which any one unit change in the
independent variable, x, is assumed to result in the same change in the dependent variable, y
• Multiple linear regression: Regression analysis involving two or more independent variables
REGRESSION MODEL
• The equation that describes how y is related to x and an error term
• Simple Linear Regression Model:
y = β0 + β1x + ε
• Parameters: The characteristics of the population, β0 and β1
• Random variable: Error term, ε
• The error term accounts for the variability in y that cannot be explained by the
linear relationship between x and y
Simple Linear Regression Model
• Estimated simple linear regression equation:
^y = b0 + b1x
• ^y = Point estimator of E(y|x)
• b0 = Estimated y-intercept
• b1 = Estimated slope
• The graph of the estimated simple linear regression equation is called the estimated
regression line
LEAST SQUARES METHOD
• Least squares method: A procedure for using sample data to find the estimated regression
equation
• Determine the values of b0 and b1
• Interpretation of b0 and b1:
• The slope b1 is the estimated change in the mean of the dependent variable y that is
associated with a one unit increase in the independent variable x
• The y-intercept b0 is the estimated value of the dependent variable y when the
independent variable x is equal to 0
• ith residual: The error made using the regression model to estimate the mean value of the
dependent variable for the ith observation
• Experimental region: The range of values of the independent variables in the data used to
estimate the model
• The regression model is valid only over this region
• Extrapolation: Prediction of the value of the dependent variable outside the experimental
region
• It is risky
• Because we have no empirical evidence that the relationship we have found holds true for values
of x outside of the range of values of x in the data used to estimate the relationship,
extrapolation is risky and should be avoided if possible.
• For Butler Trucking, this means that any prediction outside the travel time for a driving
distance less than 50 miles or greater than 100 miles is not a reliable estimate, and so
for this model the estimate of β0 is meaningless.
• However, if the experimental region for a regression problem includes zero, the y-
intercept will have a meaningful interpretation.
Assessing the Fit of the Simple Linear Regression Model
Sums of Squares
• Sum of squares due to error: The value of SSE is a measure of the error in using the estimated
regression equation to predict the values of the dependent variable in the sample
Coefficient of Determination
• The ratio SSR/SST used to evaluate the goodness of fit for the estimated regression equation
SSR
• r2 =
SST
• Take values between zero and one
• Interpreted as the percentage of the total sum of squares that can be explained by using
the estimated regression equation
• Square of the correlation between the y i and ^y i
• Referred to as the simple coefficient of determination in simple regression
MULTIPLE REGRESSION MODEL
• Multiple regression model
y = β0 + β1x1 + β2x2 + ∙ ∙ ∙ + βqxq + ε
• y = dependent variable
• x1, x2, . . . , xq = independent variables
• β0, β1, β2, . . . , βq = parameters
• ε = error term (accounts for the variability in y that cannot be explained by the linear
effect of the q independent variables)
