Course syllabus: Causal Inference in Sociology: Counterfactual ...

Magnus Bygren/Martin Hällsten
March 13, 2013
Course syllabus: Causal Inference in Sociology: Counterfactual Models
1. General information
The course consists of 7.5 ECTS credit and is at advanced level in Sociology
2. Popular description
Correlation is not causation, this has since long been known to analysts in the social
sciences. The ultimate method to obtain causal estimates is to conduct an experiment with
treatment and control groups. In the social sciences, experiments may be used in some
contexts, but often this alternative is not applicable or even inappropriate because the
experimental situation in itself is too synthetic and deviates from normal life. Analysts thus
have to work with observational data, which often miss information crucial for making causal
interpretations of statistical associations. However, under some circumstances and subject
to specific assumptions, one can interpret estimated associations as casual with substantially
higher confidence. This course deals with methods that can be used under such
circumstances and subject to the specific assumptions. The course offers practical skills in
implementing these methods and the theoretical skills needed to understand and value
evidence from them.
3. Decision
[Not yet]
4. Entrance qualifications
Course applicants should have completed the advanced level courses Quantitative Methods
in the Social Sciences (Course code SO7030) and/or Introduction to Quantitative Data
Management and Statistics (Course code SO7365), or the equivalent. Equivalent knowledge
must be verified to the course manager at the time of the application. Admitted students
can expect only limited practical help with the computer-based assignments in the course.
NB: We use the statistical software Stata for all instructions and assignments. The support
we are able to offer is dedicated to learning new routines, not managing basic programming.
5. Course contents
The course deals with counterfactual models in quantitative social science. By counterfactual
models we mean models that include attempts to handle endogeneity, selection and/or
reverse causation through an advanced research design. The models can be described as
special cases of an overarching counterfactual framework which is inspired by the lingua of
experimental research. The course deals with the following methods:
1. Experiments, the ideal-typical reference
2. Instrumental variables (the Wald estimator and 2SLS)
3. Difference in differences (DD) and its variants, and fixed effects methods (FE)
4. Regression discontinuity (classical and fuzzy)
Each of these methods is built on assumptions in order to indentify the supposedly causal

effect. These assumptions are critical to the interpretation of quantitative estimates, and not
empirically testable. A great deal of the course revolves around the different assumptions
invoked when using counterfactual models.
6. Expected learning outcomes
The course has two different expected learning outcomes. After the course, the student is
expected to be able to
a. Conduct analyses based on a counterfactual design, i.e., use, and interpret output
from, counterfactual models
b. Describe and explain the counterfactual designs work, including:
‒ Understand the motivation for and the theoretical underpinnings of common
counterfactual designs
‒ Critically examine and discuss internal and external validity in relation to
common counterfactual models
‒ Discuss merits and limitations of counterfactual designs for specific research
topics
7. Course organization and instruction
The course is provided at half time during 10 weeks. The course is organized around 4
mandatory assignments, with introductory and thematic lectures and voluntary computer
laboratories as supplementary learning resources.
7.1. Course assignments
The course consists of four mandatory written assignments. Three computer assignments
deal with methods 2-4 listed under Course contents, where students perform estimation on
a provided dataset and write a short report. The assignment report may contain a maximum
of three tables/graphs and 1500 words, and are to be handed in by teams with two students
in each team. Each report should begin with a general description of the counterfactual
method – if necessary using formal language – and explain the inner workings of it using
relevant references to the methodological literature. The composition of the student teams
are changed for each assignment and decided by the teacher. The reports are presented
orally at a seminar as a part of the formative assessment.
The assignments are organized around peer review: for the computer assignments, teams
are paired and shall provide constructive comments and, if applicable, also practical help
during the work with the assignments.
Assignments 1-3 are graded according to the following scale: Pass (P) Fail (F). In order to
receive at least grade E on the entire course, assignments 1-3 all have to be graded P.
The fourth assignment is to individually write a critical examination of published work in a
research field of the student’s choice, present a research design employing one or more
methods covered by the course, describe its theoretical underpinnings, have a realistic plan
to collect/find real data using this design, and discuss strengths and weaknesses of such a
research endeavor in relation to its potential contribution to the accumulated knowledge in
the research field (maximum 3000 words). Because this assignment is more demanding, the

amount of time and the number of seminars devoted to reviewing is doubled. This individual
assignment is also presented orally at two regular seminars, and two student peer-review
seminars as part of the formative assessment. As for the computer assignments, students
are organized in pairs and shall provide constructive comments during the work with it.
Assignment 4 is graded according to the scale A-F (cf. grading criteria below), and forms the
main basis for the individual grades on the course.
In sum, each assignment involves giving peer-review during the work with the assignment,
writing a report, conducting an oral presentation at a seminar, and contributing to a critical
discussion on the seminars.
All assignments are to be uploaded to the Mondo course site no later than one week after
the final seminar. If a student fails to meet this deadline or submits at least one assignment
with substantial errors, s/he may (re)submit a (revised) assignment no later than six months
after the final seminar. At this point in time the course teachers will assess the grading of the
student’s revised assignments. If a student fails yet again on this occasion, assessment of
revised assignments will only take place the next time the course is given.
7.2. Instruction
Introductory meetings and seminars for course assignments
Instructions for the assignments (1) through (4) are given at the lectures, where
student pairs and peer-review pairs are organized. These student pairs will change across
assignments. The assignments are additionally presented at seminars where the course
instructors give feedback.
Lectures
The course offers four lectures:
1. Lecture 1
a. Introduction to counterfactual models. Counterfactual vs. parametric
methods for causal inference. Experiments. The focus is to learn the
overarching concepts in counterfactual models in mathematical lingua, issues
of external and internal validity, homogeneous or heterogeneous effects, the
standard threats to causal inferences and to give a background of the limits of
standard statistical techniques, in particular the realism of the conditional
independence assumption. Key concepts covered: average treatment effect
(ATE), the local average treatment effect (LATE), the marginal treatment
effect (MTE), the quantile treatment effect (QTE), the stable unit treatment
value assumption (SUTVA)
b. Orientation on the course structure.
2. Lecture 2: Difference in differences and fixed effects methods
3. Lecture 3: Instrumental variables
4. Lecture 4: Regression discontinuity
5. Lecture 5
a. Counterfactual models from a theory of science perspective. The first part of
this lecture reviews the history of counterfactual thought in the theory of

science, and discusses some of the epistemological implications of this
tradition. The second part reviews and discusses the critique it has been
subject to.
b. Seminar: Explanation and causal inference in the social sciences (student-pairs
prepare questions based on thematic issues of AJS and JEL)
Computer laboratories
We offer five different computer laboratories with a focus in practical skills needed in order
to conduct assignments (1) to (3). The topics of the laboratories are:
1. Difference in differences (DD) and fixed effects regressions
2. Instrumental variables
3. Regression discontinuity
The laboratories are semi-interactive, with a mix of teacher provided instruction and work
on assignments.
Course schedule
Calendar
Week Date Time Room Content Teacher
14
15
Wed April 3 12-17 B397 Lecture 1, Lecture 2 & assignment 1
prep
Wed April 10 13-15 B389 Reserved time for student peer
review Assignment 1
MH+MB
-
16 Thur April 18 13-16 B389 Lecture 3 & assignment 2 prep MB
16 Wed April 17 13-15 B397 Seminar on Assignment 1 MH
17
Wed April 24 13-15 B397, Reserved time for student peer
review Assignment 2
-
Tue April 30 13-15 B397 Seminar on Assignment 2 MB
18 Thur May 2 13-16 B389 Lecture 4 & Assignment 3 prep MH
19
Wed May 8 13-15 B389 Reserved time for student peer
review Assignment 3
-
Wed May 15 13-15 B389 Seminar on Assignment 3 MH
20 Thur May 16 13-16 B389 Lecture 5 & assignment 4 prep MB
21 Wed May 22 13-15 B389 Seminar on Assignment 4 MH+MB
22
Wed May 29 13-15 B389 Reserved time for student peer
review Assignment 4
-
23 Tue June 4 13-16 B389 Seminar on Assignment 4 MH+MB

8. Examination
The grading criteria are hierarchical (to receive a higher valued grade, all requirements of the
lower levels need to be fulfilled for a higher grade), and are qualitatively different.
Grade
A
B
C
D
E
Fx
F
Expected learning outcome (a): Describe and explain counterfactual
research designs
The student can independently identify opportunities for unexploited
counterfactual research designs for a specific research topic, and discuss
and weight merits and limitations of these.
The student can independently discuss issues of internal and external
validity of counterfactual designs for a specific research topic.
The student can provide a clear account of the theoretical underpinnings of
counterfactual designs for a specific research topic.
The student can give a clear account of the problems involved in the
estimation of causal effects using observational data and motivate the use
of counterfactual research designs for a specific research topic.
The student can use, and interpret output from, counterfactual research
designs.
The student makes errors in describing, or explaining, counterfactual
research designs. The student makes errors in the use, or the interpretation
of output from, counterfactual research designs.
The student cannot describe and explain counterfactual research designs.
The student cannot use, or interpret, output from counterfactual research
designs. The course has to be taken in its entirety at a later point in time.
9. Literature
There is one course book that is required reading. In addition to the book, articles some of
which are listed below will have to be read in advance of lectures/seminars. The teachers
will give reading instructions during the course.
Course book (required reading)
Angrist, Joshua D., and Jörn-Steffen Pischke. 2009. Mostly harmless econometrics: an
empiricist's companion. Princeton: Princeton University Press.
Articles (more specific reading instructions will be announced during the course)
Overview/background
Gangl, M. 2010. “Causal Inference in Sociological Research” Annual Review of Sociology
36:21-47.
Freeman, David A. “Statistical Models and Shoe Leather” Sociological Methodology, 21:291-
313.

Morgan, Stephen L., and Christofer Winship. 2007. Counterfactuals and causal inference:
Methods and principles for social research. New York, NY: Cambridge University
Press.
Thematic issues of scientific journals
“Symposium: Con out of Economics”, Journal of Economic Literature, Vol 24(2): 3-94.
“Moving to Opportunity: A Symposium”, American Journal of Sociology, Vol 114(1): 107-231.
Experiments
Shelley J. Correll, Stephen Benard, and In Paik. 2007. “Getting a Job: Is There a Motherhood
Penalty? American Journal of Sociology, 112:1297-1338.
Fuegen, K., Biernat, M., Haines, E. & Deaux, K. 2004. ”Mothers and Fathers in the Workplace:
How Gender and Parental Status Influence Judgments of Job-Related Competence.”
Journal of Social Issues 60:737-754.
Pager, Devah, Western, Bruce, and Bart Bonikowski, 2009. Discrimination in a Low-Wage
Labor Market: A Field Experiment. American Sociological Review, 74:777-799.
Fershtman, Chaim, Gneezy, Uri. 2001. Discrimination in a Segmented Society: An Experimental
Approach The Quarterly Journal of Economics 116 (1): 351-377.
DD, DDD
Goldin, C. & Rouse, C. Orcestrating Impartiality: the impact of “blind” auditions on female
musicians. American Economic Review, 90:715-741.
Ruhm, Christopher J. 1998. “The Economic Consequences of Parental Leave Mandates:
Lessons from Europe.” Quarterly Journal of Economics 113:285-317.
Fixed effects
Bygren, M. and Gähler, M. 2012. ”Family Formation and Men’s and Women’s Attainment of
Workplace Authority.” Social Forces, doi: 10.1093/sf/sor008.
Isacsson, Gunnar. 2004. "Estimating the economic return to educational levels using data on
twins." Journal of Applied Econometrics 19:99-119.
—. 2007. "Twin Data vs. Longitudinal Data to Control for Unobserved Variables in Earnings
Functions – Which Are the Differences?*." Oxford Bulletin of Economics and
Statistics 69:339-362.
Nordström Skans, Oskar. 2004. "Scarring effects of the first labour market experience: A
sibling based analysis." IFAU Working Paper 2004:14.
Matching
Morgan, Stephen L. and DJ Harding. Matching Estimators of Causal Effects. Prospects and
Pitfalls in Theory and Practice.” Sociological Methods and Research, 35:3-60.
Stuart, E.A. (2010). “Matching Methods for Causal Inference: A review and a look forward.”
Statistical Science 25: 1-21.
Instrumental variables

Becker, Sasha O. and Ludger Woessman. 2009. “Was Weber Wrong? A Human Capital
Theory of Protestant Economic History” Quarterly Journal of Economics, 124:531-
596.
Angrist, Joshua D., and Alan B. Krueger. 1991. "Does Compulsory School Attendance Affect
Schooling and Earnings?" Quarterly Journal of Economics 106:979-1014.
Bound, John, and David A. Jaeger. 1996. "On the validity of season of birth as an instrument
in wage equations: a comment on Angrist and Krueger's' Does compulsory school
attendance affect schooling and earnings?'." NBER Working paper No. 5835.
Plug, Erik J. S. 2001. "Season of birth, schooling and earnings." Journal of Economic
Psychology 22:641-660.
Regression discontinuity
Cook, Thomas D. 2008. “‘Waiting for Life to Arrive’: A history of the regression-discontinuity
design in Psychology, Statistics and Economics” Journal of Econometrics, 142:636-
654.
Öckert, Björn. 2010. "What's the value of an acceptance letter? Using admissions data to
estimate the return to college." Economics of Education Review 29:504-516.
Folke, Olle. 2011. Shades of Brown and Green: Party Effects in Proportional Election
Systems.
http://www.columbia.edu/~of2152/documents/Shades_of_Brown_and_Green.pdf
Ekberg, John, Rickard Eriksson, and Guido Friebel. 2005. "Parental Leave-A Policy
Evaluation of the Swedish 'Daddy-Month' Reform." IZA Discussion Paper 1617.
Reference literature
Quasi/natural experiments
Kirk, David S. 2009. “A Natural Experiment on Residential Change and Recidivism: Lessons
from Hurricane Katrina.” American Sociological Review, 74:484-505
Angrist, Joshua D., and Alan B. Krueger. 2001. "Instrumental Variables and the Search for
Identification: From Supply and Demand to Natural Experiments." The Journal of
Economic Perspectives 15:69-85.
Meghir, Costas, and Mårten Palme. 2005. "Educational Reform, Ability, and Family
Background." The American Economic Review 95:414-424.
Card, David. 1990. "The Impact of the Mariel Boatlift on the Miami Labor Market."
Industrial and Labor Relations Review 43:245-257.
Cruz, Luiz M., and Marcelo J. Moreira. 2005. "On the Validity of Econometric Techniques
with Weak Instruments: Inference on Returns to Education Using Compulsory
School Attendance Laws." J. Human Resources XL:393-410.
Holmlund, Helena, Mikael Lindahl, and Erik Plug. 2011. "The Causal Effect of Parents'
Schooling on Children's Schooling: A Comparison of Estimation Methods." Journal of
Economic Literature 49:615-51.
Almond, Douglas, Lena Edlund, and Mårten Palme. 2009. "Chernobyl's Subclinical Legacy:
Prenatal Exposure to Radioactive Fallout and School Outcomes in Sweden."
Quarterly Journal of Economics 124:1729-1772.
Black, Sandra E., Paul J. Devereux, and Kjell G. Salvanes. 2005. "The More the Merrier? The
Effect of Family Size and Birth Order on Children's Education." Quarterly Journal of

Economics 120:669-700.
—. 2005. "Why the Apple Doesn't Fall Far: Understanding Intergenerational Transmission of
Human Capital." The American Economic Review 95:437-449.
Methodological articles
Rubin, Donald B. 1977. "Assignment to Treatment Group on the Basis of a Covariate."
Journal of Educational and Behavioral Statistics 2:1-26.
Holland, Paul W. 1986. "Statistics and Causal Inference." Journal of the American Statistical
Association 81:945-960.
Murray, Michael P. 2006. "Avoiding Invalid Instruments and Coping with Weak Instruments."
The Journal of Economic Perspectives 20:111-132.
Angrist, Joshua D., Guido W. Imbens, and Donald B. Rubin. 1996. "Identification of Causal
Effects Using Instrumental Variables." Journal of the American Statistical Association
91:444-455.
Rosenbaum, Paul R., and Donald B. Rubin. 1983. "The central role of the propensity score in
observational studies for causal effects." Biometrika 70:41-55.
Other
Wooldridge, Jeffrey M. 2002. Econometric Analysis of Cross Section and Panel Data: MIT
Press.
Pearl, Judea. 2000. Causality: Models, reasoning, and inference. Cambridge: Cambridge
University Press.
Black, Dan A., and Jeffrey A. Smith. 2004. "How robust is the evidence on the effects of
college quality? Evidence from matching." Journal of Econometrics 121:99-124.
Morgan, Stephen L. 2001. "Counterfactuals, Causal Effect Heterogeneity, and the Catholic
School Effect on Learning." Sociology of Education 74:341-374.
Brand, Jennie E., and Yu Xie. 2010. "Who Benefits Most from College? Evidence for
Negative Selection in Heterogeneous Economic Returns to Higher Education."
American Sociological Review 75:273-302.
Dale, Stacy, and Alan B. Krueger. 2011. "Estimating the Return to College Selectivity over
the Career Using Administrative Earning Data." IZA Discussion Paper 5533.
Dale, Stacy Berg, and Alan B. Krueger. 2002. "Estimating the Payoff to Attending A More
Selective College: An Application of Selection on Observables and
Unobservables*." Quarterly Journal of Economics 117:1491-1527.
Baum, Christopher F, Mark E Schaffer, and Steven Stillman. 2007. "IVREG2: Stata module
for extended instrumental variables/2SLS, GMM and AC/HAC, LIML, and k-class
regression." Boston College Department of Economics, Statistical Software
Components S425401 http://ideas.repec.org/c/boc/bocode/s425401.html.
Becker, S. O., and A. Ichino. 2002. "Estimation of average treatment effects based on
propensity scores." Stata Journal 2:358-377. http://www.statajournal.
com/sjpdf.html?articlenum=st0026