Determine if we can get an unbiased estimate of the treatment
effect using a treatment that people have to opt into. The treatment for this homework is
actually answering and listening to the entire call. That is, not all people selected to receive
a phone call actually answered the phone or listened to the message until the end. The point
of this homework is to see if you can get the “right” estimate (similar in magnitude to the
experimental estimate from the first homework) in a non-experimental setting.
Use the datasets on Canvas for Assignment 2. The datasets include only people the
researchers tried to call (i.e., the dataset only includes those with treatreal=1 from the
data you used in Assignment 1). The variable contact is equal to 1 for people who actually
received and listened to the entire call and 0 otherwise. We can think of people who listened
to the call as opting into the treatment in the sense that they were available to receive the call
and did not just hang up. Those who did not finish listening to the call are the comparison
group. The variables in the datasets are the same as those in the first assignment.
- Create a Table 1 that lets you compare the means of the sample characteristics in the
treatment and comparison group. Please create a clean table that includes columns
with the means of each group, the difference between the two groups, and the p-value
of the difference. The table should be comprehensible on its own. - Does Table 1 suggest that the control group will provide a good counterfactual for the
treatment group’s voting potential outcomes (can we just compare the fraction voting
2
in each group)? Why or why not? Explain. - How does Table 1 differ from the table you created to test for balance of covariates
between the treatment and control groups in Assignment 1? Why do you think they
differ? Explain. - Now you will examine if regression analysis generates reliable and correct estimates of
the causal effect of receiving a voting phone call after adjusting for differences between
the two groups you documented in Table 1. Create a carefully labeled Table 2 where
each column corresponds to a regression. The first column contains the parameter
estimates (and their standard errors) for the regression vote02 = B0 + B1contact + u.
In each subsequent column add one more covariate to the regression. - What effect does adding covariates have on your estimate of the treatment effect?
What does this tell you about the relationship between the covariates, the outcome,
and treatment (listening to the call)? - How do the results in Table 2 differ from the regression results from the first homework?
Carefully examine both the point estimates and their standard errors for the treatment
effect (as well as how the estimates vary as covariates are added). Why do these two
tables differ? - Did adding covariates to the regression in Table 2 reduce the bias (i.e., cause the
estimates to become more similar to the correct causal estimate)? If yes, why do you
think it did? Which variable reduced the bias the most? Why do you think this is the
case? - Did adding covariates to the regression eliminate all bias in the estimates? Why or
why not? Explain your answer. - Please attach all of the code you wrote to generate your results.