UNIVERSITY OF CALIFORNIA, LOS ANGELES
Department of Economics

Economics 143 (Cameron) - Applied Regression Analysis

Classroom Handout #6:
Derivation of Means and Variances of the OLS Estimators for Slope and Intercept


Each sample you might have drawn for estimating and ordinary least squares (OLS) regression would have produced different point estimates for the slope and the intercept of the regression line. Over all possible samples, there will be distributions for each of these estimators: intercept b1 and slope b2.

NOTE: if you would like more details concerning these derivations, an annotated version of this handout is also available

What are the E(b1) and E(b2)? (i.e. Are these estimators unbiased for B1 and B2?)

1 E(b2) = E [ S (Xi - )(Yi - ) / S (Xi - )2 ]              let c = S (Xi - )2

2            = E [ S (Xi - )Yi / c - S (Xi - ) / c ]

3           = E [ S (Xi - )Yi / c ]

4           = { S (Xi - ) E[Yi] } / c

5           = { S (Xi - ) E[B1 + B2 Xi + ui] } / c

6           = S (Xi - ) { E[B1] + E[B2Xi] + E[ui] } / c

7           = B2 { S (Xi - )Xi / c }

8           = B2 { S (Xi - )2 / S (Xi - )2 } = B2 ...thus b2 unbiased

9  E(b1) = E [ - b2] = E [B1 + B2 + - b2 ]

10           = E[B1] + E[B2 ] + E[] - E[b2 ]

11           = B1 + B2 + 0 - E[b2] = B1 + B2 - B2 = B1 ...thus b1 unbiased.

What are Var(b1) and Var(b2)? (i.e. How "noisy" are the estimators for B1 and B2?)

12 Var(b2) = Var [ S (Xi - )Yi / S (Xi - )2 ] = Var [ S (Xi - )Yi / c ]

13           = Var [ ((X1 - )/c) Y1 + ((X2 - )/c) Y2 + ... + ((Xn - )/c) Yn ]

14           = ((X1 - )/c)2 Var(Y1) + ((X2 - )/c)2 Var(Y2) + ... + ((Xn - )/c)2 Var(Yn)

15           = S [ (Xi - )2/c2 ] Var(Yi) = S [ (Xi - )2/c2 ] s 2

16           = s 2 S [ (Xi - )2/c2 ] = s 2 S [ (Xi - )2/ (S (Xi - )2)2]

17           = s 2 / S xi2  .... s.e.(b2) = s /Ö (S xi2 )

18 Var(b1) = Var [ - b2 ] = Var [ B1 + B2 + - b2 ]

19           = Var(B1) + Var(B2) + Var() + Var(b2)

20           = 0 + 0 + Var (S ui/n) + 2 Var(b2)

21           = s 2/n + 2 ( s 2 / S xi2  )

22           = s 2 [ (1/n) + 2/S (Xi - )2 ], or can be alternatively expressed as

23           = s 2 [ S Xi2  / nS xi2  ] .... s.e.(b1) = s Ö (S Xi2  / nS xi2 )

For s 2, use sample variance with modified degrees of freedom (2 estimated parameters required before ei can be ascertained)

24      s2 = S e /(n-2) = (1/(n-2)) S (Yi - b1 - b2Xi)2

COURSE OUTLINE LECTURE OUTLINES PROBLEM SETS PROBLEM SOLUTIONS COMPUTER LABS
SHAZAM EXAMPLES DATA SETS ONLINE QUIZZES GRAPHICS HANDOUTS
Updated: January 26, 1998
Prepared by: Trudy Ann Cameron