UCLA; Economics 143; Cameron; Computing Lab Session #4

UNIVERSITY OF CALIFORNIA, LOS ANGELES Department of Economics
Economics 143 (Cameron) - Applied Regression Analysis
Computing Lab Session #4: Multiple Regression and Omitted Variables Bias

Goals for this lab:

Reinforce interpretation of simple and multiple regression output:
- meaning of intercept; meaning(s) of slope(s)
- point estimates
- standard errors
- t-ratios (and P-values--you do not need the tables in the text)
- zero hypotheses
- use of TEST command (for other hypotheses)
- R-squared (adjusted R-squared)
Reinforce the connections between (a) verbal formulation of hypotheses, (b) corresponding regression model and statistical hypotheses, (c) translation back into verbal interpretation of results
Reinforce issue of potential omitted variables bias in ANY regression; explore simulation program that illustrates how omitted variable bias in slopes on included variables depends on correlations between included and omitted variables.

Tasks:

1. Omitted Variables Bias Example from Problem Set #4: Concept: Finding an apparent relationship in a simple regression that actually isn't there, as a consequence of omitted variables bias. File needed: colds.sha.

a.) Try a naive regression model that treats COLDS as a function of VISITS to places of worship: OLS COLDS VISITS. If your atheist brother-in-law asserts that more-religious people are light-weight hypochondriacs who are always getting sick, what would this regression imply with respect to his assertion? Using the basic regression results, test the null hypothesis that VISITS have no effect on COLDS. (Also try an appropriate version of the TEST command to verify these results.) BE SURE YOU KNOW HOW TO DO THIS AND HOW TO INTERPRET THE RESULTS. b.) You wonder if any other factors besides religiosity affect health status. AGE is a variable you have handy. You control for AGE before examining the effect of VISITS on COLDS by running the regression OLS COLDS VISITS AGE. (Note that the order of the explanatory variables doesn't matter; the results for each variable will be unaffected.) Repeat the test of the hypothesis that VISITS have no effect on COLDS. What happens now? Test the hypothesis that AGE has no effect on COLDS. c.) Examine a plot of the relationship between the two explanatory variables: VISITS and AGE. Use PLOT VISITS AGE / GNU (Why do you not use the LINEONLY or LINE option?) Explain what accounts for the conflicting implications of the simple regression and the multiple regression. Which model is more "right"? Why? d.) Compare the "fit" of the simple and multiple regression models. Which model does a better job of explaining the observed variation across people in the number of colds? What about the fact that one model has an unfair advantage in that it uses more regressors? How do you control for this unfair advantage?

2. Omitted Variables Bias Example from Problem Set #4: Concept: Finding NO effect of one variable on another (or an effect that is counter to what you might expect) when there actually is an effect, but it is obscured by omitted variables. File needed: study.sha.

a.) Repeat the steps for the colds.sha example using the study.sha program and data. Here you are interested in knowing whether hours of studying (STUDY) has a statistically significant effect on midterm grades (MIDTERM). If you fail to control for "ability" approximated by prior GPA, you find some counter-intuitive results; an effect you might think should be there is not statistically significant--and the sign is even "wrong." When you do control for ability, something more reasonable appears. b.) Be sure to examine the relationship between study hours and GPA for a clue to the source of the bias in the simple regression. Ensure that you can explain in words what accounts for the difference in results between the simple and the multiple regressions in this case. c.) For your preferred specification, test the hypothesis that study time has no effect on midterm grade. What do you conclude, based on this specification? Now test the hypothesis that an extra hour of study time, on average, will produce a 5-point-higher midterm score. What does your model imply? d.) Compare the "fit" of the simple and multiple regression models. Which model does a better job of explaining the observed variation across people in midterm scores? What about the fact that one model has an unfair advantage in that it uses more regressors? How do you control for this unfair advantage? e.) Contemplate the interpretation of the intercept in this model. Is it meaningful? Why or why not?

3. Suppose the "true" data-generating process (DGP) which we call the population regression function (PRF) is of the form:

Y_i = B₁ + B₂ X_i + B₃ Z_i + u_i

Suppose that we draw a random sample from this population, forming a sample of size n with observations on Y_i, X_i and Z_i. Perhaps we are most interested in the size and sign of the coefficient B₂. If we regress Y on both X and Z, and the usual "maintained hypotheses" for regression are met, then OLS is appropriate and it will provide "best linear unbiased estimates" (B.L.U.E.) of the true populations parameters B₁, B₂, and B₃.

However, it is possible, since we do not really know the form of the underlying PRF, that we might estimate a "misspecified" model of the following form:

Y_i = b₁ + b₂ X_i + e_i.

What are the properties of the estimated parameters under these circumstances? They may no longer be unbiased. We have omitted the Z_i variable when it really ought to be there, since Z_i truly does help explain Y_i.

Some discussion:

1. If both X and Z have a positive effect on Y (i.e. B₂, B₃ > zero) and Z_i is uncorrelated with X_i, then omission of Z_i from the regression model will NOT bias b₂. However, the error terms will be larger, since they are now capturing (B₃ Z_i + u_i). Thus, the estimate of s ₂ (s²) is larger than it needs to be, meaning that the standard errors of the regression parameters are all larger than they need to be. Consequently, the associated t-test statistics will be smaller than necessary and we will be less likely to be able to reject specific hypotheses about the sizes of any of the underlying parameters.

2. If both X and Z have a positive effect on Y (i.e. B₂, B₃ > zero) and Z_i is positively correlated with X_i, then the omission of Z_i from the regression model WILL bias b₂. This is because larger values of X_i are proxying for larger values of Z_i (and smaller values of X_i are proxying for smaller values of Z_i. Thus, the estimated parameter b₂ is not telling us how Y_i changes when X_i changes, ceteris paribus. Instead, it is telling us how Y_i changes when BOTH X_i and Z_i change. Whatever the sign of the coefficient b₂, the parameter estimate will be larger in absolute value than it ought to be. The more correlated is Z_i with X_i, the greater will be the bias. That portion of the variability in Z_i that does not coincide with the variability in X_i (i.e. that part which remains "orthogonal" to the variation in X_i) will be absorbed by the intercept and the error term, leading to a potential for the error term again to be inflated. This will again make for smaller t-test statistics and more difficulty in rejecting hypotheses about the parameters.

3. If both X and Z have a positive effect on Y (i.e. B₂, B₃ > zero) and Z_i is negatively correlated with X_i, then the omission of Z_i from the regression model WILL bias b₂. The bias may be sufficiently large in some cases that the apparent sign of b₂ is actually reversed. The problem is that when Z_i is left out, larger values of X_i (which should increase Y_i) are associated with smaller values of Z_i (which should decrease Y_i). If the effect of smaller Z_i values dominates the effect of larger X_i values, Y_i will appear to be made smaller by larger values of X_i and the estimated value of b₂ will be negative.

There are several more cases to consider....

In general:

The direction and size of the bias in b₂ as an estimator for the true B₂ depends on:

a.) the actual signs and sizes of B₂ and B₃
b.) the degree of correlation (positive or negative) between X_i and Z_i.

Some key points to remember are:

a.) if the omitted variable(s) is uncorrelated with any of the included variable(s), then the point estimates of the coefficients on the included variables will be unbiased.

b.) despite the lack of bias with exclusion of uncorrelated relevant variables, the efficiency of the estimation can be compromised (i.e. the standard errors on the slope coefficients for included variables will be inflated).

c.) omission of correlated relevant variables is often the explanation for the appearance of unexpected signs (or sizes) for coefficients on included variables.

The simulation program:

To illustrate some of these points, we have the simulation program contained in omitvar.sha. Here is what is going on in the default version of the program:

1. We are exploring the consequences of omitting a relevant explanatory variable as the correlation between the excluded (Z) explanatory variable and the included (X) explanatory variable changes from "close to -1," to zero, to "close to +1."

2. For each of almost 100 different degrees of correlation, we generate a set of data on X and Z. As defaults, the mean of X is set to 3, the mean of Z is set to 5, and the marginal standard deviation of X is 3 and that of Z is 2.

3. We use a common statistical method involving so-called "Cholesky factorization" of matrices to convert a pair of independent N(0,1) random variables into a pair of correlated random variables each with a N(0,1) marginal distributions. Then we change the location and scale of these correlated normal random variables to the default values mentioned above. (You are not responsible for this matrix algebra--it is simply a means to an end.)

4. For these correlated data on X and Z, we specify a population regression function of the form:

Y_i = B₁ + B₂ X_i + B₃ Z_i + u_i.

As default values, we have set B₁ = 1, B₂ = 2 and B₃ = 3 (all positive, in this case). We determine a "random" value of Y_i to go with each (X_i,Z_i) pair by plugging the X_i and Z_i values into this formula, and then tacking on a random draw from a N(0,s ²) distribution, where the default value of s is set to 3 in this illustration.

5. Now that we have this sample of "data" from a known population regression function, we use it in two regression models:

Model I. Y_i = b₁ + b₂ X_i + b₃ Z_i + e_i Model II. Y_i = b₁* + b₂* X_i + e_i

We then save the estimated parameters and their associated t-test statistics from the first model and from the second model (which may suffer from omitted variables bias).

6. The final task is to observe how certain key regression quantities change as we consider data sets with differing levels of correlation between X_i and Z_i. We can plot:

a.) b₂ from Model I and b₂* from Model II as functions of the error correlation between X and Z (called CORR_X_Z in the program).

b.) b₁ from Model I and b₁* from Model II as function of the error correlation between X and Z.

c.) t-ratios on b₂ and b₃ from Model I alone as a function of the correlation between X and Z (this illustrates the consequences of multicollinearity between regressors in one model).

d.) t-ratios on b₂ from Model I and b₂* from Model II as functions of the error correlation between X and Z.

4. (Time permitting) Exploring how to get fancy laser-printed plots that show both raw data and fitted values: GNUPLOT option on SHAZAM PLOT command with subsequent editing of the gnuplot command file.

a.) Consider a plot of the relationship between VISITS and AGE from the colds.sha example (as an illustration of the technique). Append to the end of your own version of colds.sha the following additional code:

ols visits age / predict=vhat
plot visits vhat age / gnu lineonly
plot visits vhat age / gnu lineonly commfile=cold.gnu &
datafile=cold.dat

The first plot command lets you see a screen version of the graphics plot. It will have a straight line and a wiggly line. We are going to erase the wiggly line and leave just its "dots." Editing a gnuplot plotting program requires the second version of the plot command and subsequent editing of the appropriate hidden file indicated in the cold.gnu output file generated by this code. Note (as before) that the names you specify for the commfile= and datafile= options cannot be more than 8 characters in total, including the "." and the extension, so if you are going to use informative extensions like those above (.gnu and .dat), the first part of the filenames should be no more than 4 characters long. Make a note of the name you selected for the commfile. Now type STOP and exit SHAZAM. b.) Now select an editor (perhaps Notepad), because we are going to peek into the cold.gnu file. Note the name of the real gnuplot program that this file points to, and open this generically named hidden file. This is where you can change the title for the plot, and change the names of the variables if you like. In particular, you want to find the line of code for the VISITS variable (not VHAT) and delete the w lines part. When you are happy with your modifications, save and exit. Be sure that the file name under which the gnuplot program is save is what you intended. c.) Now select GNUPLOT for Windows and open the .gnu file you want processed to create a plot for the laser printer. This will be the COLDS.GNU file (or the genericaly named hidden filename), if you have been proceeding as above. d.) As before, you can print the plot on the laser printer by right-clicking on the plot after it appears on the screen, ensuring that the print options are to your liking, and then sending the print task to the printer.

Update date: 9:42 PM 10/26/98; Prepared by: Trudy Ann Cameron; Site Index