Chapter 3 Multiple Linear Regression Model The linear model
Yiru Xu
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
We consider the problem of regression when study variable depends on more than one explanatory or independent variables, called as multiple linear regression model. This model generalizes the simple linear regression in two ways. It allows the mean function () E y to depend on more than one explanatory variables and to have shapes other than straight lines, although it does not allow for arbitrary shapes.
Key takeaways
- So a simple linear regression model can be expressed as 0 1 income education .
- These assumptions are used to study the statistical properties of estimator of regression coefficients.
- In such a case, the matrix ' W W is in the form of correlation matrix, i.e., The regression coefficients obtained after such scaling, viz., or usually called standardized regression coefficients.
- The multiple linear regression model can also be expressed in the deviation form.
- It is relatively easy to define a joint confidence region for in multiple regression model.
Related papers
Multiple Linear Regression
it also appears as one of the default data sets in Minitab software). The response variable is y (that is, "quality") and we wish to find the "best" regression equation that relates quality to the other five parameters.
Multi-regression
The following sections have been adapted from Field (2009) Chapter 7. These sections have been edited down considerably and I suggest (especially if you're confused) that you read this Chapter in its entirety. You will also need to read this chapter to help you interpret the output. If you're having problems there is plenty of support available: you can (1) email or see your seminar tutor (2) post a message on the course bulletin board or (3) drop into my office hour.
Regression Models 1.1 Introduction
Regression models form the core of the discipline of econometrics. Although econometricians routinely estimate a wide variety of statistical models, using many different types of data, the vast majority of these are either regression models or close relatives of them. In this chapter, we introduce the concept of a regression model, discuss several varieties of them, and introduce the estimation method that is most commonly used with regression models, namely, least squares. This estimation method is derived by using the method of moments, which is a very general principle of estimation that has many applications in econometrics. The most elementary type of regression model is the simple linear regression model, which can be expressed by the following equation: y t = β 1 + β 2 X t + u t. (1.01) The subscript t is used to index the observations of a sample. The total number of observations, also called the sample size, will be denoted by n. Thus, for a sample of size n, the subscript t runs from 1 to n. Each observation comprises an observation on a dependent variable, written as y t for observation t, and an observation on a single explanatory variable, or independent variable, written as X t. The relation (1.01) links the observations on the dependent and the explanatory variables for each observation in terms of two unknown parameters, β 1 and β 2 , and an unobserved error term, u t. Thus, of the five quantities that appear in (1.01), two, y t and X t , are observed, and three, β 1 , β 2 , and u t , are not. Three of them, y t , X t , and u t , are specific to observation t, while the other two, the parameters, are common to all n observations. Here is a simple example of how a regression model like (1.01) could arise in economics. Suppose that the index t is a time index, as the notation suggests. Each value of t could represent a year, for instance. Then y t could be household consumption as measured in year t, and X t could be measured disposable income of households in the same year. In that case, (1.01) would represent what in elementary macroeconomics is called a consumption function.
3 Multiple Regression Analysis: Estimation 3.1 Motivation for Multiple Regression The Model with Two Independent Variables
I n Chapter 2, we learned how to use simple regression analysis to explain a dependent variable, y, as a function of a single independent variable, x. The primary drawback in using simple regression analysis for empirical work is that it is very difficult to draw ceteris paribus conclusions about how x affects y: the key assumption, SLR.4-that all other factors affecting y are uncorrelated with x-is often unrealistic. Multiple regression analysis is more amenable to ceteris paribus analysis because it allows us to explicitly control for many other factors that simultaneously affect the dependent variable. This is important both for testing economic theories and for evaluating policy effects when we must rely on nonexperimental data. Because multiple regression models can accommodate many explanatory variables that may be correlated, we can hope to infer causality in cases where simple regression analysis would be misleading. Naturally, if we add more factors to our model that are useful for explaining y, then more of the variation in y can be explained. Thus, multiple regression analysis can be used to build better models for predicting the dependent variable. An additional advantage of multiple regression analysis is that it can incorporate fairly general functional form relationships. In the simple regression model, only one function of a single explanatory variable can appear in the equation. As we will see, the multiple regression model allows for much more flexibility. Section 3.1 formally introduces the multiple regression model and further discusses the advantages of multiple regression over simple regression. In Section 3.2, we demonstrate how to estimate the parameters in the multiple regression model using the method of ordinary least squares. In Sections 3.3, 3.4, and 3.5, we describe various statistical properties of the OLS estimators, including unbiasedness and efficiency. The multiple regression model is still the most widely used vehicle for empirical analysis in economics and other social sciences. Likewise, the method of ordinary least squares is popularly used for estimating the parameters of the multiple regression model. We begin with some simple examples to show how multiple regression analysis can be used to solve problems that cannot be solved by simple regression. 89782_03_c03_p073-122.qxd 5/26/05 11:46 AM Page 73
Multiple Regression Analysis : Estimation I
2004
In Chapter 2, we learned how to use simple regression analysis to explain a dependent variable, y, as a function of a single independent variable, x. The primary drawback in using simple regression analysis for empirical work is that it is very difficult to draw ceteris paribus conclusions about how x affects y: the key assumption, SLR.4—that all other factors affecting y are uncorrelated with x—is often unrealistic. Multiple regression analysis is more amenable to ceteris paribus analysis because it allows us to explicitly control for many other factors that simultaneously affect the dependent variable. This is important both for testing economic theories and for evaluating policy effects when we must rely on nonexperimental data. Because multiple regression models can accommodate many explanatory variables that may be correlated, we can hope to infer causality in cases where simple regression analysis would be misleading. Naturally, if we add more factors to our model that are use...
BASIC IDEAS OF LINEAR REGRESSION: THE TWO-VARIABLE MODEL
QUESTIONS 2.1. (a) It states how the population mean value of the dependent variable is related to one or more explanatory variables. (b) It is the sample counterpart of the PRF. (c) It tells how the individual Y are related to the explanatory variables and the stochastic error term, u, in the population as a whole.
Related papers
Linear Regression
Once we've acquired data with multiple variables, one very important question is how the variables are related. For example, we could ask for the relationship between people's weights and heights, or study time and test scores, or two animal populations. Regression is a set of techniques for estimating relationships, and we'll focus on them for the next two chapters.
Multiple Regression: A Leisurely Primer
2001
Multiple regression is a useful statistical technique when the researcher is considering situations in which variables of interest are theorized to be multiply caused. It may also be useful in those situations in which the researchers is interested in studies of predictability of phenomena of interest. This paper provides an introduction to regression analysis, focusing on five major questions a novice user might ask. The presentation is set in the framework of the general linear model and builds on correlational theory. More advanced topics are introduced briefly with suggested references for the reader who might wish to pursue the subject. (Contains 11 references.) (Author/SLD) Reproductions supplied by EDRS are the best that can be made from the original document.
Regression Analysis
Bootstrap Methods, 2021
Assume we measure the insulin level Y 1 , . . . , Y n of n persons. Every person has a different weight X 1 , . . . , X n . Can we somehow explain the insulin level using the weights? This is the general context of regression analysis. There are different reasons why such a question might be of interest. For instance, a scientist could be interested in understanding the mechanics behind insulin level, i.e., which factor influences the insulin level and how? Other scientists may only be interested in predicting the insulin level. One common way to achieve this is to find a way to express the conditional expectation of Y given X . Call the function m(X ) = E(Y |X ) the regression function. This chapter is dedicated to methods that estimate parametric forms m(X, ϑ) under various assumptions. We start with the classical linear models that assume that m(X, ϑ) = ϑ X is linear in X while Y follows a normal distribution first under independence assumptions and later under certain correlation assumptions. Afterward, we allow other distributions for Y like the negative-binomial distribution which lead to the classical generalized linear models. The chapter concludes with semi-parametric models, i.e., we do not explicitly assume a distribution for Y but the regression function m(X, ϑ) still depends on some (multi-dimensional) parameter ϑ. Beside bootstrapping in the classical manner, that is sampling with replacement, other options are available. Therefore, after investigating the estimators (asymptotic) distribution we present resampling techniques that can be used to bootstrap the distribution. Of course, this allows again to estimate confidence intervals or to derive other statistics, but these results will also be used (in the next chapter) to construct goodness-of-fit statistics for the regression function itself. Usually, visual techniques are used to assess if the model fits the data well. The next chapter provides a more rigorous approach to that leveraging the results from this chapter.
Regression analysis and dependence
Metrika, 2005
In this paper we study the relationship between regression analysis and a multivariate dependency measure. If the general regression model Y ¼ f ðX i 1 ; X i 2 ;. .. ; X i m Þ holds for some function f , where 1 i 1 < i 2 < Á Á Á i m k, and X 1 ;. .. ; X k is a set of possible explanatory random variables for Y. Then there exists a dependency relation between the random variable Y and the random vector ðX i 1 ; X i 2 ;. .. ; X i m Þ. Using the dependency statistic d j Y ;ðX i 1 ;...;X im Þ , defined below, we can detect such dependency even if the function f is not linear. We present several examples with real and simulated data to illustrate this assertion. We also present a way to select the appropriate subset X i 1 ;. .. ; X i m among the random variables X 1 ; X 2 ;. .. ; X k , which better explain Y .
Linear Models With MATLAB
This book has been prepared for the beginners to help them understand basic to advanced functionality of MATLAB. After completing this chapter 1 (Which included an explanation of the Matlab language) you will find yourself at a moderate level of expertise in using MATLAB from where you can take yourself to next levels. On other side, in spite of the availability of highly innovative tools in statistics, the main tool of the applied statistician remains the linear model. The linear model involves the simplest and seemingly most restrictive statistical properties: independence, normality, constancy of variance, and linearity. However, the model and the statistical methods associated with it are surprisingly versatile and robust. More importantly, mastery of the linear model is a prerequisite to work with advanced statistical tools because most advanced tools are generalizations of the linear model. The linear model is thus central to the training of any statistician, applied or theoretical. This book develops the basic theory of linear models for regression, analysis-of variance, and analysis–of–covariance. Applications are illustrated by examples and problems using real data. This combination of theory and applications will prepare the reader to further explore the literature and to more correctly interpret the output from a linear models computer package and MATLAB. This introductory linear models book is designed primarily for a one- semester course for advanced undergraduates or MS students. It includes more material than can be covered in one semester so as to give an instructor a choice of topics and to serve as a reference book for researchers who wish to gain a better understanding of regression and analysis-of-variance. The book would also serve well as a text for PhD classes in which the instructor is looking for a one-semester introduction, and it would be a good supplementary text or reference for a more advanced PhD class for which the students need to review the basics on their own. Our overriding objective in the preparation of this book has been clarity of exposition. We hope that students, instructors, researchers, and practitioners will find these linear models' text more comfortable than most. In the final stages of development, we asked students for
Multiple Linear Regression Model Used in Economic Analyses
Romanian Statistical Review Supplement, 2014
The multiple regression is a tool that offers the possibility to analyze the correlations between more than two variables, situation which account for most cases in macroeconomic studies. The best known method of estimation for multiple regression is the method of least squares. As in the two-variable regression, we choose the regression function of sample and minimize the sum of squared residual values. Another method that allows us to take into account the number of variables factor when determining the validity of harmonization is given by the Akaike information criterion.
Summer Seminar: Philosophy of Statistics Lecture Notes 10: Linear Regression Model: a brief introduction
2019
The Linear Regression (LR) model is arguably the most widely used statistical model in empirical modeling across many disciplines. It provides the exemplar for all regression models as well as several other statistical models referred to as ‘regression-like’ models, some of which will be discussed briefly in this chapter. The primary objective is to discuss the LR model and its associated statistical inference procedures. Special attention is paid to the model assumptions and how they relate to the sampling distributions of the statistics of interest. The main lesson of this chapter is that when any of the probabilistic assumptions of the LR model are invalid for data z0:={( ) =1 } inferences based on it will be unreliable. The unreliability of inference will often stem from inconsistent estimators and sizeable discrepancies between actual and nominal error probabilities induced by statistical misspecification.
The ABC of linear regression analysis: What every author and editor should know
Regression analysis is a widely used statistical technique to build a model from a set of data on two or more variables. Linear regression is based on linear correlation, and assumes that change in one variable is accompanied by a proportional change in another variable. Simple linear regression, or bivariate regression, is used for predicting the value of one variable from another variable (predictor); however, multiple linear regression, which enables us to analyse more than one predictor or variable, is more commonly used. This paper explains both simple and multiple linear regressions illustrated with an example of analysis and also discusses some common errors in presenting the results of regression, including inappropriate titles, causal language, inappropriate conclusions, and misinterpretation.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.