Homework

Week 2 - Causality I

Intergroup Contact and Exclusionary Attitudes

This week’s homework uses data based on: Enos, R. D. 2014. “Causal Effect of Intergroup Contact on
Exclusionary Attitudes.” Proceedings of the National Academy of Sciences 111(10): 3699–3704. Enos
conducted a randomized field experiment assessing the extent to which individuals living in suburban
communities around Boston, Massachusetts, were affected by exposure to demographic change.
Subjects in the experiment were individuals riding on the commuter train line and were overwhelmingly
white. Every morning, multiple trains pass through various stations in suburban communities that were
used for this study. For pairs of trains leaving the same station at roughly the same time, one was
randomly assigned to receive the treatment and one was designated as a control. By doing so all the
benefits of randomization apply for this dataset.
The treatment in this experiment was the presence of two native Spanish-speaking ‘confederates’ (a term
used in experiments to indicate that these individuals worked for the researcher, unbeknownst to the
subjects) on the platform each morning prior to the train’s arrival. The presence of these confederates,
who would appear as Hispanic foreigners to the subjects, was intended to simulate the kind of demographic
change anticipated for the United States in coming years. For those individuals in the control group, no
such confederates were present on the platform. The treatment was administered for 10 days. Participants
were asked questions related to immigration policy both before the experiment started and after the
experiment had ended. The names and descriptions of variables in the data set boston.csv are:

Name Description
age Age of individual at time of experiment
male Sex of individual, male (1) or female (0)
income Income group in dollars (not exact income)
white Indicator variable for whether individual identifies as white (1) or not (0)
college Indicator variable for whether individual attended college (1) or not (0)
usborn Indicator variable for whether individual is born in the US (1) or not (0)
treatment Indicator variable for whether an individual was treated (1) or not (0)
ideology Self-placement on ideology spectrum from Very Liberal (1) through
Moderate (3) to Very Conservative (5)
numberim.pre Policy opinion on question about increasing the number immigrants
allowed in the country from Increased (1) to Decreased (5)
numberim.post Same question as above, asked later
remain.pre Policy opinion on question about allowing the children of undocumented
immigrants to remain in the country from Allow (1) to Not Allow (5)
remain.post Same question as above, asked later

1
Name Description
english.pre Policy opinion on question about passing a law establishing English as
the official language from Not Favor (1) to Favor (5)
english.post Same question as above, asked later

You can download this data from Moodle (in the "Homework materials and activities" section) or
here. Once you have done so, store it in the data subfolder you created earlier. Then start a new R script
which you should save as homework2.R. You can then load the data into R with the following code:

## Load data

boston <- read.csv("data/boston.csv")

As last week, please submit one R-script per group. You may want to indicate the names of your homework
group members at the top (after a #).

Question 1

The benefit of randomly assigning individuals to the treatment or control groups is that the two groups
should be similar, on average, in terms of their other characteristics, or “covariates”. This is referred to
as “covariate balance.”

Use the mean function to determine whether the treatment and control groups
are balanced with respect to the age (age) and income (income) variables. Also,
compare the proportion of males (male) in the treatment and control groups.
Interpret these numbers.
Hint: to calculate the proportion of observations with a given attribute on a
binary variable, you can just use mean(data_frame_name$variable_name).

Answer 1

## Mean age for treatment and control units

mean_age_treated <- mean(boston$age[boston$treatment == 1])
mean_age_control <- mean(boston$age[boston$treatment == 0])
mean_age_treated - mean_age_control

[1] -3.912299

## Mean income levels for treatment and control units

mean_income_treated <- mean(boston$income[boston$treatment == 1])
mean_income_control <- mean(boston$income[boston$treatment == 0])
mean_income_treated - mean_income_control

[1] -15972.59

2
 ## Proportion "male" for treatment and control units
prop_male_treated <- mean(boston$male[boston$treatment == 1])
prop_male_control <- mean(boston$male[boston$treatment == 0])
prop_male_treated - prop_male_control

[1] -0.06096257

Despite the randomization of treatment assignment, there are some differences in the average characteris-
tics of treatment and control units. For example, the average age of treated individuals is 40.4, where is
is 44.3 for control units. Similarly, while 53% of treated individuals are male, 59% of control individuals
are male. Most notably, the average income of the treated group is approximately $16000 lower than it
is for the control group.
Overall, while the treatment and control groups are relatively well balanced, there remain some potentially
problematic confounding differences between these groups. This is an example of the point made in
lecture: although randomized experiments provide unbiased estimates on average, any given instance
of randomization may not create perfect balance across all covariates. That is, you might be unlucky!
That is why it is often important to run replication studies of randomized experiments to ensure that the
results we obtain are not simply because we were lucky/unlucky in any particular randomization of the
treatment.

Question 2

Individuals in the experiment were asked “Do you think the number of immigrants from Mexico who are
permitted to come to the United States to live should be increased, left the same, or decreased?” The
response to this after the experiment is in the variable numberim.post. The variable is coded on a 1 – 5
scale. Responses with values of 1 are inclusionary (‘pro-immigration’) and responses with values of 5 are
exclusionary (‘anti-immigration’).

Calculate the mean value of this variable for the treatment and control groups.
What is the difference in means? What does the result suggest about the effects
of intergroup contact on exclusionary attitudes?

Answer 2

## Calculate the mean in each group (note that na.rm = T is required here)
treat_mean <- mean(boston$numberim.post[boston$treatment == 1], na.rm = T)
control_mean <- mean(boston$numberim.post[boston$treatment == 0], na.rm = T)

## Calculate the difference in means

treat_mean - control_mean

[1] 0.3945701

3
The difference in means suggests that the treatment group reported, on average, 0.39 points higher on the
5 point scale than the control group. As higher values of the outcome variable suggest more exclusionary
attitudes, this suggests that contact with the Spanish speaking confederates increases exclusionary atti-
tudes, at least in this experiment. Because the responses of individuals in the treatment group were more
exclusionary than the control group, we would conclude on the basis of this experiment that exposure to
potential demographic changes cause increases in exclusionary attitudes.

Question 3

Does having attended college influence the effect of being exposed to ‘outsiders’ on exclusionary attitudes?
Another way to ask the same question is this: is there evidence of a differential impact of treatment,
conditional on attending college versus not attending college?

Calculate the difference in means between treatment and control observations

amongst those who attended college and those who did not. Interpret your results.
Hint: You may want to subset the data using more than one logical con-
dition here. For example, if I wanted to subset the data to include
only the observations which were treated and went to college, I could use
boston$numberim.post[boston$treatment == 1 & boston$college == 1].

Answer 3

## First calculate the mean outcome for treatment and control

## observations *who went to college*
treat_college_mean <- mean(boston$numberim.post[boston$treatment == 1
& boston$college == 1], na.rm = TRUE)

control_college_mean <- mean(boston$numberim.post[boston$treatment == 0

& boston$college == 1], na.rm = TRUE)

## Now calculate the mean outcome for treatment and control

## observations *who did not go to college*
treat_nocollege_mean <- mean(boston$numberim.post[boston$treatment == 1
& boston$college == 0], na.rm = TRUE)

control_nocollege_mean <- mean(boston$numberim.post[boston$treatment == 0

& boston$college == 0], na.rm = TRUE)

## Difference in means for college observations

diff_college <- treat_college_mean - control_college_mean
diff_college

[1] 0.4929467

4
 ## Difference in means for non-college observations
diff_nocollege <- treat_nocollege_mean - control_nocollege_mean
diff_nocollege

[1] -0.4285714

The average treatment effect (using the numberim.post variable) among those with a college education is
an increase in exclusionary attitudes of about 0.49 points. Among those without a college education, there
is a decrease in exclusionary attitudes of about .43 points. Both of these effects are on a 5 point scale.
At face value, this suggests that the effects of “outgroup” contact on exclusionary attitudes might differ
according to education. However, given what we find in the next answer, there seems to be a sample size
problem with the non-college educated respondents.

Question 4

Calculate the number of observations used to calculate each of the mean outcome
values you used in the answer for question 3. What does this suggest about the
reliability of the conclusions you drew from that analysis?

Answer 4

table("treatment"=boston$treatment,"college"=boston$college)

college
treatment 0 1
0 7 61
1 8 47

Using the table function reveals that some of the averages calculated above are based on a very small
number of observations. In particular, the vast majority of the data in our sample is of individuals
who have college degrees. Only 15 individuals are not college educated. Accordingly, we might worry
that the averages we have calculated above for non-college individuals may capture idiosyncrasies of
these individuals rather than anything general about the broader population of non-college educated
individuals.
How many observations is “enough”? The short answer is: it depends. The long answer: we will cover
this extensively in future weeks!