Scribd
Upload
369 views5 pages

Formulas Linear Regression PDF

This document provides a summary of formulas for simple linear regression including: - The slope formula which calculates the slope (b) as a function of the differences between the mean of the x and y values. - Formulas for calculating the variance, intercept, estimated mean and variance at different x values, total sum of squares, regression sum of squares, error sum of squares, R-squared, mean square error, and degrees of freedom. - An example of applying the formulas to a dataset with 5 data points to calculate the slope, intercept, variance, residuals and more.

Uploaded by

NatRib
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
369 views5 pages

Formulas Linear Regression PDF

This document provides a summary of formulas for simple linear regression including: - The slope formula which calculates the slope (b) as a function of the differences between the mean of the x and y values. - Formulas for calculating the variance, intercept, estimated mean and variance at different x values, total sum of squares, regression sum of squares, error sum of squares, R-squared, mean square error, and degrees of freedom. - An example of applying the formulas to a dataset with 5 data points to calculate the slope, intercept, variance, residuals and more.

Uploaded by

NatRib
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Summary formula sheet for simple linear

regression

Slope b = ! (Yi -Y)(Xi -X) / ! (Xi -X)2


_ _ _

Variance 5 2 / ! (Xi -X)2


_
_ _
Intercept a= Y - b X _
2
Variance of a [ n + !
1 X _
2
] 5 2
(Xi -X)

Estimated mean at X_ 0 a + b X0
(X0 -X)_2
Variance [ n + !
1
2
] 5 2
(Xi -X)

Estimated individual at _X0 a + b X0


(X0 -X)_2
Variance [1 + n + !
1
] 5 2
(Xi -X)2

Total SS = ! (Yi -Y)2


_

[ ! (Yi -Y)(Xi -X)] / ! (Xi -X)2


Regression
_ SS =_ _
2

Error SS = Total SS - Regression SS


R2 = Regression SS/ Total SS = "proportion
explained"

MSE = error mean square = estimate of 5 2


= Error SS/ df
df= degrees of freedom = n-2 for simple
linear.
Example

Data points (x1 ,y1 ), (x2 ,y2 ), ...., (xn ,yn )


_ (1,5), (2,7),_ (3,9), (4,6), (5,8)
x = 15/5=3, y = 7
Corrected sum of squares for x:
!(xi - x)2 = Sxx = (1-3)2 +...+(5-3)2 = 10
n _
i=1
Corrected sum of squares for y:
!(yi -y)2 = Syy = (5-7)2 +...+(8-7)2 = 10
n _
i=1
Corrected sum of cross products = Sxy =
!(xi - x)(yi -y) =
n _ _
i=1
(-2)(-2)+(-1)(0)+...+(2)(1) = 5 =
!xi yi -n x y = 110-5(3)(7)
n __
i=1

Slope: b = Sxy /Sxx = 5/10 = 0.5


Intercept:
_ _
y - b x = 7-0.5(3) = 5.5
^
y=5.5 + 0.5x

y 5 7 9 6 8
y^ 6 6.5 7 7.5 8
r=y- y^ -1 0.5 2 -1.5 0

!ri2 = "Error sum of squares" =


n

i=1
SSE = 1+0.25+4+2.25=7.5
SSE is also Syy - S2xy /Sxx = Syy - b2 Sxx =
10-52 /10

Variance of b:

ÈMSE/Sxx is called "standard error" of b.


MSE/Sxx œ 2.5/10 = 0.25.
t = b/È0.25 = 1 which is not an
Task: test H0 : true slope is 0

unusual t.
data a; input x y @@; cards;
15 27 39 46 58
;
proc reg; model Y =X / p;
run;
Dependent Variable: y

Analysis of Variance

Sum of Mean
Source DF Squares Square F Value Pr > F
Model 1 2.50000 2.50000 1.00 0.3910
Error 3 7.50000 2.50000
Corr Total 4 10.00000

Root MSE 1.58114 R-Square 0.2500


Dependent Mean 7.00000 Adj R-Sq 0.0000
Coeff Var 22.58770

Parameter Estimates

Parameter Standard
Variable DF Estimate Error t Value Pr > |t|
Intercept 1 5.50000 1.65831 3.32 0.0452
x 1 0.50000 0.50000 1.00 0.3910

Output Statistics
Dep Var Predicted
Obs y Value Residual

1 5.0000 6.0000 -1.0000


2 7.0000 6.5000 0.5000
3 9.0000 7.0000 2.0000
4 6.0000 7.5000 -1.5000
5 8.0000 8.0000 0

You might also like