Regression Analysis – Detailed Notes
Regression Analysis is a powerful statistical tool used to study the relationship
between a dependent variable and one or more independent variables. It helps in
predicting the value of the dependent variable based on known values of
independent variables. It is widely used in business, economics, and management
research for forecasting, decision-making, and performance evaluation.
Types of Regression Analysis
1. Simple Linear Regression: Examines the relationship between one dependent
variable (Y) and one independent variable (X).
2. Multiple Linear Regression: Analyzes the effect of two or more independent
variables on a dependent variable.
3. Non-linear Regression: Used when the relationship between variables is not
linear (e.g., exponential, logarithmic models).
4. Logistic Regression: Used when the dependent variable is categorical or
binary (e.g., yes/no outcomes).
Mathematical Model of Regression
For Simple Linear Regression:
Y = a + bX + e
where,
- Y = Dependent variable
- X = Independent variable
- a = Intercept (value of Y when X = 0)
- b = Regression coefficient (slope, indicates change in Y for one-unit change in X)
- e = Error term (difference between actual and predicted values)
Interpretation of Regression Coefficients
- Intercept (a): Represents the expected value of Y when X = 0.
- Slope (b): Measures the rate of change in Y for every one-unit change in X.
If b > 0: Positive relationship (X increases → Y increases).
If b < 0: Negative relationship (X increases → Y decreases).
Method of Least Squares
The least squares method estimates the regression coefficients by minimizing the
sum of the squared differences between the observed and predicted values of Y.
Formula for slope (b):
b = Σ[(X - X■)(Y - ■)] / Σ[(X - X■)²]
Intercept (a):
a = ■ - bX■
Goodness of Fit – Coefficient of Determination (R²)
R² measures how well the regression line fits the data.
R² = Explained Variation / Total Variation
Values:
- R² close to 1: Strong relationship.
- R² close to 0: Weak relationship.
Assumptions of Linear Regression
1. Linearity: Relationship between X and Y is linear.
2. Independence: Observations are independent.
3. Homoscedasticity: Constant variance of errors.
4. Normality: Errors are normally distributed.
5. No Multicollinearity: Independent variables are not highly correlated.
Applications of Regression Analysis in Business and Management
- Sales forecasting and demand estimation.
- Cost analysis and budgeting.
- Financial performance prediction.
- Market trend and consumer behavior analysis.
- Risk assessment and investment decisions.
Limitations of Regression Analysis
- Assumes linear relationships (not suitable for complex non-linear behavior).
- Sensitive to outliers.
- Results depend on data accuracy and assumptions.
- Cannot establish causation, only correlation.
