The Welfare Effects of Coordinated School Assignment ... - iSites
The Welfare Effects of Coordinated School Assignment ... - iSites
The Welfare Effects of Coordinated School Assignment ... - iSites
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>The</strong> <strong>Welfare</strong> <strong>Effects</strong> <strong>of</strong> <strong>Coordinated</strong> <strong>School</strong> <strong>Assignment</strong>:<br />
Evidence from the NYC High <strong>School</strong> Match ∗<br />
Atila Abdulkadiroğlu Nikhil Agarwal Parag A. Pathak †<br />
November 2013<br />
Abstract<br />
Centralized and coordinated school assignment systems are a growing part <strong>of</strong> recent education<br />
reforms. This paper estimates school demand using student rank order lists submitted<br />
in New York City’s high school assignment system launched in Fall 2003 to study the effects<br />
<strong>of</strong> coordinating admissions in a single-<strong>of</strong>fer mechanism based on the deferred acceptance<br />
algorithm. Compared to the previous mechanism with multiple <strong>of</strong>fers, a limited number<br />
<strong>of</strong> choices, and three rounds <strong>of</strong> <strong>of</strong>fer processing, there is a 7.2 percentage point increase in<br />
matriculation at assigned school in the new mechanism. Student preferences trade <strong>of</strong>f proximity<br />
and school quality, but are substantially heterogeneous. Uncoordinated <strong>of</strong>fers leave<br />
more than a third unassigned after the first round, <strong>of</strong> which the majority eventually obtain<br />
over-the-counter assignments close to home. <strong>The</strong> new mechanism assigns students to schools<br />
on average 0.69 miles further away. Even though all else equal students prefer nearby schools,<br />
we estimate that the average student welfare gain from the improved school match is equal<br />
to eight times the disutility from increased travel. Students from across all demographic<br />
groups, boroughs, and baseline achievement categories obtain a more preferred assignment<br />
on average, though the benefit is larger for student groups more likely to be unassigned after<br />
the old mechanism’s first round. <strong>The</strong>se facts imply that allocative changes involving matching<br />
mechanisms need not be zero-sum. However, the gains from coordinating assignment are<br />
larger than from relaxing constraints imposed by the new mechanism’s assignment algorithm.<br />
JEL: C78, D50, D61, I21<br />
Keywords: <strong>School</strong> Choice, Matching <strong>The</strong>ory, Deferred Acceptance, Congestion<br />
∗ We thank Neil Dorosin, Jesse Margolis, and Elizabeth Sciabarra for their expertise and for facilitating access to<br />
the data used in this study. A special thanks to Alvin E. Roth for his collaboration, which made this study possible.<br />
Jesse Margolis provided extremely helpful detailed comments. We also thank Josh Angrist, Jerry Hausman, and<br />
Ariel Pakes and seminar participants at the NBER Market Design conference for input. We received excellent<br />
research assistance from Weiwei Hu, Danielle Wedde, and a hardworking team <strong>of</strong> MIT undergraduates. Pathak<br />
acknowledges support from the National Science Foundation (grant SES-1056325). Abdulkadiroğlu and Pathak<br />
are board members <strong>of</strong> the Institute for Innovation in Public <strong>School</strong> Choice, a non-pr<strong>of</strong>it that provides technical<br />
assistance to districts on enrollment issues.<br />
† Abdulkadiroğlu: Department <strong>of</strong> Economics, Duke University, email: atila.abdulkadiroglu@duke.edu. Agarwal:<br />
Cowles Foundation, Yale University and Department <strong>of</strong> Economics, MIT email: agarwaln@mit.edu. Pathak:<br />
Department <strong>of</strong> Economics, MIT and NBER, email: ppathak@mit.edu.
1 Introduction<br />
Whether transferring to an out-<strong>of</strong>-zone public school, applying to a charter, magnet or selective<br />
exam school, or using a voucher to attend a private school, a growing number <strong>of</strong> families can now<br />
opt out <strong>of</strong> attending their neighborhood school. As school choice options have proliferated, so<br />
too has the way in which choices are expressed and students are assigned. Ad hoc decentralized<br />
assignment systems where students applied separately to schools with uncoordinated admissions<br />
<strong>of</strong>fers have been replaced with centralized, coordinated single-<strong>of</strong>fer assignment mechanisms in<br />
a number <strong>of</strong> cities including Denver, London, New Orleans, and New York City. 1 In these<br />
cities, student demand is matched with the school seat supply using algorithms inspired by the<br />
theory <strong>of</strong> matching markets (Gale and Shapley 1962, Shapley and Scarf 1974, Abdulkadiroğlu<br />
and Sönmez 2003). Changes in assignment protocols are inevitably controversial, however, since<br />
reallocating school seats may result in some students assigned to high quality schools while others<br />
are assigned to less desirable alternatives.<br />
In this paper, we exploit a large-scale policy change in New York City to study the effects<br />
<strong>of</strong> moving from an uncoordinated assignment system to a coordinated single-<strong>of</strong>fer system on the<br />
allocation <strong>of</strong> students to schools. Prior to 2003, roughly 80,000 aspiring high school students<br />
applied to five out <strong>of</strong> more than 600 school programs; they could receive multiple <strong>of</strong>fers and<br />
be placed on wait lists. Students in turn were allowed to accept only one school and one wait<br />
list <strong>of</strong>fer, and the cycle <strong>of</strong> <strong>of</strong>fers and acceptances repeated two more times. However, a total <strong>of</strong><br />
three rounds <strong>of</strong> <strong>of</strong>fer processing proved insufficient to allocate all students to schools, and more<br />
than 25,000 applicants were assigned to schools not on their original application list through<br />
an administrative “over-the-counter” process. Students initially expressed their preferences on a<br />
common application, but admissions <strong>of</strong>fers were not coordinated across schools, so we refer to<br />
the old mechanism as an uncoordinated mechanism.<br />
In Fall 2003, the system was replaced by a single-<strong>of</strong>fer assignment system, based on the<br />
student-proposing deferred acceptance algorithm for the main round. Applicants were allowed<br />
to rank up to 12 programs for enrollment in 2004-05 and a supplementary round assigned those<br />
unassigned in the main round. Since most students received one <strong>of</strong>fer through the central <strong>of</strong>fice,<br />
we refer to this new mechanism as a coordinated mechanism. <strong>The</strong> Department <strong>of</strong> Education<br />
(DOE) adopted the new mechanism to provide a greater voice to student’s choices, to utilize<br />
school places more efficiently, and to reduce gaming involved to obtain a school seat (Kerr 2003).<br />
Parents were also provided with additional information on school options through an expanded<br />
set <strong>of</strong> parent workshops and high school fairs. At the conclusion <strong>of</strong> the first year, 82% <strong>of</strong> students<br />
were assigned in the main round and one-third fewer participants were processed in the over-thecounter<br />
round.<br />
This large-scale policy change provides a unique opportunity to investigate the consequences<br />
<strong>of</strong> centralized and coordinated school assignment. Changes were advertised widely beginning<br />
only in September 2003, and students submitted their preferences in November 2003. This brief<br />
timeline limits the scope for participants to react by moving in response to the new mechanism.<br />
1 A number <strong>of</strong> U.S. cities are in the process <strong>of</strong> adopting unified enrollment systems using similar matching<br />
algorithms, including Chicago, Newark, Philadelphia, and Washington DC (Darville 2013).<br />
1
Moreover, rich micro-level data from both systems allow us to surmount many <strong>of</strong> the usual challenges<br />
associated with evaluating allocative aspects <strong>of</strong> assignment mechanisms. A key strength <strong>of</strong><br />
our study is that we observe submitted preferences, round-by-round placements, and subsequent<br />
school enrollment <strong>of</strong> all students in both the old and new mechanism. Using student preferences<br />
together with detailed information on student and school attributes, we estimate the factors that<br />
account for school demand. <strong>The</strong> estimated preference distribution is then used to evaluate the<br />
distributional consequences <strong>of</strong> the new mechanism and to assess mechanism design choices.<br />
<strong>The</strong> new mechanism is based on the student-proposing deferred acceptance mechanism, where<br />
truth-telling is a weakly dominant strategy for participants (Dubins and Freedman 1981, Roth<br />
1982). This motivates our empirical approach <strong>of</strong> recovering the preference distribution using<br />
the revealed preferences <strong>of</strong> students with discrete choice demand models. <strong>The</strong> richness <strong>of</strong> our<br />
data allow us to fit models with substantial student heterogeneity and unobserved school effects.<br />
While the incentive properties <strong>of</strong> the new mechanism motivate treating stated rankings as true<br />
preferences, there are some complications we investigate. First, the new mechanism only allows<br />
participants to rank at most 12 choices. Although 80% <strong>of</strong> applicants in our sample rank fewer than<br />
12, this constraint interferes with the dominant strategy property <strong>of</strong> the mechanism for students<br />
who find more than 12 schools acceptable (Abdulkadiroğlu, Pathak, and Roth 2009, Haeringer<br />
and Klijn 2009). Second, it is possible that the sudden adoption <strong>of</strong> the new mechanism led some<br />
families to use heuristics developed from the old mechanism, despite the DOE’s advice to rank<br />
schools truthfully. 2 For instance, some families might have ranked a “safety” school as their last<br />
choice, even though the new mechanism’s properties make this unnecessary if ranking fewer than<br />
12 choices.<br />
We start with benchmark models that use all ranked choices and assume the ranking is<br />
truthful, before turning to several alternatives. <strong>The</strong> alternatives are from models that treat only<br />
top choice as the most preferred alternative, that treat only the top three choices as the three<br />
most preferred, that restrict the student sample to those ranking less than 12 schools and treating<br />
their rankings as truthful, and that use all choices other than the last to eliminate a possible<br />
safety-school effect. We also report estimates assuming that stated choices are not necessarily<br />
preferred to unranked schools and from models with different assumptions on the outside option.<br />
We do not directly model strategic choices because we anticipate strategic considerations to be<br />
less important in mechanisms based on deferred acceptance than in other more manipulable<br />
mechanisms. 3<br />
Changes in student assignment due to the coordination <strong>of</strong> admissions <strong>of</strong>fers can happen<br />
through many channels. First, the new mechanism allows students to rank up to 12 choices,<br />
whereas the old mechanism only allowed for five. Second, the limited number <strong>of</strong> rounds <strong>of</strong><br />
<strong>of</strong>fers and acceptances can lead to coordination failures where students hold on to less preferred<br />
choices waiting to be <strong>of</strong>fered seats at more preferred schools once others decline. With few<br />
rounds <strong>of</strong> <strong>of</strong>fer processing, this may, in turn, lead to inefficiencies as many are left unassigned,<br />
2 <strong>The</strong> DOE’s school brochure advised: “You must now rank your 12 choices according to your true preferences”<br />
(DOE 2003).<br />
3 Hastings and Weinstein (2008) report that in Charlotte-Mecklensburg’s highly manipulable immediate acceptance<br />
mechanism, students respond to information on historical odds <strong>of</strong> assignment. Pathak and Sönmez (2013)<br />
formalize how constrained deferred acceptance is less manipulable than the immediate acceptance mechanism.<br />
2
a phenomenon termed congestion by Roth and Xing (1997). In the old mechanism, the district<br />
simply placed unassigned students to nearby schools in an over-the-counter process, even if they<br />
wanted to go elsewhere. In a centralized single-<strong>of</strong>fer system, school seats can be used effectively,<br />
as the computerized rounds <strong>of</strong> <strong>of</strong>fers and acceptances minimize coordination issues related to<br />
insufficient <strong>of</strong>fer processing time. Third, in New York’s old mechanism, schools were able to see<br />
the entire rank ordering <strong>of</strong> applicants, and <strong>of</strong>ten advertised they would only consider those who<br />
ranked them first. This property creates strategic pressure on student ranking decisions. Each<br />
<strong>of</strong> these features <strong>of</strong> uncoordinated assignment can result in student mismatch.<br />
On the other hand, coordinated assignment may make encourage movement throughout the<br />
district for little benefit. Ravitch (2011), for instance, argues that the elevation <strong>of</strong> choice in<br />
NYC “destroyed the concept <strong>of</strong> neighborhood schools” as “children scattered across the city in<br />
response to the lure <strong>of</strong> new, unknown small schools with catchy names, or were assigned to<br />
schools far from home.” Critics <strong>of</strong> the new system believed that denying principals’ information<br />
on students’ ranking <strong>of</strong> the school restricts a principal’s ability to attract “students who want<br />
them most” (Herszenhorn 2004). This logic implies that removing a school’s ability to know<br />
whether they were ranked first could lead fewer <strong>of</strong> their <strong>of</strong>fered students to enroll. It also may be<br />
in a district’s interest to advantage certain student subgroups to prevent them from leaving the<br />
district (Engberg, Epple, Imbrogno, Sieg, and Zimmer 2010). 4 Providing students with multiple<br />
<strong>of</strong>fers and letting them decide upon receiving <strong>of</strong>fers may make it easier for some students to decide<br />
what school is best for them. 5 Taken together, these points suggest that the new mechanism<br />
may involve a substantial redistribution among students, rather than better assignments for most<br />
students.<br />
<strong>The</strong> questions we examine are also relevant to other settings where uncoordinated assignment<br />
procedures have been replaced with coordinated ones. For instance, in England, where the 2003<br />
Admissions Code mandated coordination <strong>of</strong> admissions nationwide, a number <strong>of</strong> local education<br />
authorities, governing bodies similar to U.S. school districts, adopted common applications and<br />
centralized assignment. <strong>The</strong> English motivation parallels NYC since authorities wanted to ensure<br />
that “every child within a local authority area would receive one <strong>of</strong>fer <strong>of</strong> a school place on the same<br />
day. This would eliminate or largely eliminate multiple <strong>of</strong>fers and free up places for parents who<br />
would not otherwise be <strong>of</strong>fered a place” (Pennell, West, and Hind 2006). All 32 London boroughs<br />
coordinated to establish a Pan London Admissions Scheme based on deferred acceptance to make<br />
the admissions system “fairer” and “simpler,” and to “result in more parents getting an <strong>of</strong>fer <strong>of</strong><br />
a place for their child at one <strong>of</strong> their preferred schools earlier and fewer getting no <strong>of</strong>fer at all”<br />
(ALG 2005, Pathak and Sönmez 2013). In 2012, the Recovery <strong>School</strong> District in New Orleans<br />
also coordinated disparate school application processes within a common application and single<strong>of</strong>fer<br />
system, citing the need to eliminate a “frustrating, ad-hoc enrollment process handled at<br />
individual campuses” (Vanacore 2012).<br />
We find compelling evidence that the new mechanism is an improvement relative to the old<br />
one based on <strong>of</strong>fer processing and matriculation patterns. Students are more likely to enroll in<br />
4 Williams (2004) pr<strong>of</strong>iles families who had to pay a deposit to reserve a spot in a private or parochial school<br />
while awaiting their school placement in the first year <strong>of</strong> the new mechanism.<br />
5 Similar arguments for the flexibility provided by multiple <strong>of</strong>fer systems have been raised by critics <strong>of</strong> the<br />
National Residency Matching Program. See, e.g., Bulow and Levin (2006) and Agarwal (2013).<br />
3
the school they are assigned. In the old mechanism, 18.6% <strong>of</strong> students matriculated at schools<br />
other than where they were assigned at the end <strong>of</strong> the match, while in the new mechanism only<br />
11.4% students do. Multiple <strong>of</strong>fers and short rank order lists in the old mechanism advantage<br />
few students, but leave many without <strong>of</strong>fers. Roughly one-fifth <strong>of</strong> students obtained multiple<br />
<strong>of</strong>fers, but <strong>of</strong> those who accept a first round <strong>of</strong>fer, more than 70% take their most preferred <strong>of</strong>fer,<br />
suggesting that the majority <strong>of</strong> the students with multiple <strong>of</strong>fers did not benefit from having<br />
more than one option. Half <strong>of</strong> applicants obtain no first round <strong>of</strong>fer and 59% <strong>of</strong> these applicants<br />
are assigned in the over-the-counter process. <strong>The</strong> take-up rates for over-the-counter students<br />
are similar across mechanisms, but the number <strong>of</strong> students assigned in that round is three times<br />
larger in the old mechanism. In addition, 8.5% <strong>of</strong> applicants left the district after assignment in<br />
the old mechanism, while only 6.4% left under the new mechanism.<br />
<strong>The</strong> most significant difference between assignments under the two mechanisms is the distance<br />
between students and their assigned school. In the old mechanism, students travelled on average<br />
3.36 miles to reach their assigned school. Students travel 0.69 miles further in the new mechanism.<br />
All else equal, students prefer schools that are closer to home according to their preference<br />
rankings in the new mechanism. However, preferences are substantially heterogeneous. Estimates<br />
from models using all ranked choices show that student preferences trade <strong>of</strong>f proximity and school<br />
quality. Asian and white students and those with high baseline scores exhibit greater variance in<br />
their taste for school characteristics, net <strong>of</strong> distance, than black and Hispanics and those with low<br />
baselines scores, respectively. Students prefer schools with higher math achievement, but high<br />
baseline math students prefer them more. Students prefer schools with fewer subsidized lunch<br />
peers, but students from wealthier neighborhoods prefer them more. Students prefer schools with<br />
more white peers, but white and Asian students are more sensitive to this dimension than blacks<br />
and Hispanics. Variations from the demand model using first choice, top three choices, choices<br />
among those who rank fewer than 12, and dropping last choice show broadly consistent patterns.<br />
This fact together with evidence on both in and out-<strong>of</strong>-sample model fit reassure us about the<br />
suitability <strong>of</strong> using the estimated preferences to make statements about student welfare.<br />
<strong>The</strong> preference estimates suggest that the new mechanism has made it easier for students to<br />
express and obtain a choice they want. Even though school assignments are further away, based<br />
on our estimated distribution <strong>of</strong> preferences, the amount that students prefer these schools more<br />
than compensates for this difference. Ignoring the greater travel distance, the improvement<br />
associated with the matched school is distance-equivalent <strong>of</strong> 5.53 miles; net <strong>of</strong> distance, the<br />
average improvement is 4.79 miles. That is, on average the improved school match is equal to<br />
eight times the disutility from increased travel. Students across demographic groups, boroughs,<br />
baseline achievement levels all obtain a more preferred assignment on average from the new<br />
mechanism. <strong>The</strong> largest gains are for student groups who were more likely to be unassigned<br />
after the old mechanism’s first round. Given that some students eventually must attend less<br />
desirable schools under any mechanism, the pattern suggests the assignments produced by the<br />
new mechanism were a net improvement for most students and these allocative changes were<br />
not from zero-sum. <strong>The</strong>se gains are also seen whether utility is measured based on assignments<br />
made at the end <strong>of</strong> the high school match or based on subsequent enrollment.<br />
<strong>The</strong> reforms <strong>of</strong> the school choice mechanism in New York City also raise a number <strong>of</strong> optimal<br />
4
design questions that we examine using our preference estimates. As a benchmark, we compute<br />
the utilitarian optimal assignment given resource constraints on the number <strong>of</strong> school seats. <strong>The</strong><br />
new mechanism is 2.85 miles in distance-equivalent utility below this idealized benchmark. Had<br />
the mechanism produced a student-optimal stable matching, average student welfare improves<br />
by an equivalent <strong>of</strong> 0.11 miles. 6 An ordinally Pareto efficient matching is equivalent to an<br />
improvement <strong>of</strong> 0.60 miles. Unfortunately, these allocations cannot be implemented without<br />
harming the mechanism’s incentive properties and mechanisms that implement these allocations<br />
as equilibrium outcomes are not known. Nonetheless, the average welfare levels from these<br />
counterfactuals show that any potential gain from relaxing the constraints imposed by the new<br />
mechanism’s algorithm are smaller than those from coordinating assignments.<br />
<strong>The</strong> literature on school choice is immense and not easily summarized. We share a focus<br />
with papers interested in understanding how choice affects assignment and sorting <strong>of</strong> students<br />
(Epple and Romano 1998, Urquiola 2005), rather than the competitive effects <strong>of</strong> choice on student<br />
achievement (Hoxby 2003, Rothstein 2006). We also concentrate our study on allocative<br />
efficiency, rather than effects <strong>of</strong> choice options on subsequent test scores (Abdulkadiroğlu, Angrist,<br />
Dynarski, Kane, and Pathak 2011, Deming, Hastings, Kane, and Staiger 2011). This study<br />
contributes to a growing theoretical literature on centralized admissions to college (Balinski<br />
and Sönmez 1999) and K-12 public schools (Abdulkadiroğlu and Sönmez 2003). A number<br />
<strong>of</strong> recent papers use micro data from assignment mechanisms to understand school demand<br />
(Hastings, Kane, and Staiger 2009, He 2012), but these datasets comes from variants <strong>of</strong> the<br />
Boston mechanism, a mechanism where parents have a strong incentive to rank schools strategically.<br />
Niederle and Roth (2003)’s study <strong>of</strong> the introduction <strong>of</strong> a gastroentrology match is an<br />
important precursor, but the rich data we have allow us to examine the consequences <strong>of</strong> sorting,<br />
quantitatively evaluate student welfare, and assess design choices. Finally, our work complements<br />
theoretical papers investigating other assignment mechanisms using simulated data<br />
(Erdil and Ergin 2008, Kesten 2010, Abdulkadiroğlu, Che, and Yasuda 2012) or those that<br />
use submitted ordinal student preferences to examine theoretical counterfactuals, but not with<br />
empirically-grounded cardinalizations <strong>of</strong> utility (Abdulkadiroğlu, Pathak, and Roth 2009, Pathak<br />
and Sönmez 2008).<br />
<strong>The</strong> paper proceeds as follows. Section 2 provides additional details on high school assignment<br />
in New York City. Section 3 describes our data. Section 4 reports on descriptive features <strong>of</strong> <strong>of</strong>fer<br />
processing in the two mechanisms. In Section 5 we outline our empirical strategy, while Section 6<br />
reports estimates <strong>of</strong> the distribution <strong>of</strong> student preferences. Section 7 uses the demand estimates<br />
to compare student welfare across the two mechanisms, while Section 8 quantitatively assesses<br />
trade<strong>of</strong>fs in design choices. <strong>The</strong> last section concludes.<br />
6 Even though the mechanism is based on student-proposing deferred acceptance, the matching is not studentoptimal<br />
because school-side priorities are not strict (Erdil and Ergin 2008, Abdulkadiroğlu, Pathak, and Roth<br />
2009, Kesten 2010, Kesten and Kurino 2012).<br />
5
2 High <strong>School</strong> Choice in NYC<br />
2.1 <strong>School</strong> Options<br />
Forms <strong>of</strong> high school choice have existed in New York City for decades. Before 2002, high<br />
school assignment in New York City featured a hodgepodge <strong>of</strong> choice options mostly controlled<br />
by borough-wide high school superintendents. Significant admissions power resided with the<br />
schools because they could directly enroll students. Across the city, specific programs existed<br />
within schools, with curricula ranging from the arts to sciences to vocational training. It was not<br />
uncommon for multiple program types to exist within the same school, a feature that continues<br />
today. <strong>The</strong> set <strong>of</strong> school programs that developed at the time shaped the set <strong>of</strong> school options<br />
available in the uncoordinated mechanism. <strong>The</strong> 2002-03 High <strong>School</strong> directory describes seven<br />
types <strong>of</strong> programs, categorized according to their admissions criteria.<br />
At Specialized High <strong>School</strong>s, such as Stuyvesant and Bronx Science, there is only one type <strong>of</strong><br />
program, which admits students by admissions test performance on the Specialized High <strong>School</strong>s<br />
Admissions Test (SHSAT). 7 Audition programs interview students for pr<strong>of</strong>iciency in specific<br />
performing or visual arts, music, or dance. Screened programs evaluate students individually<br />
using an assortment <strong>of</strong> criteria including a student’s 7th grade report card, reading and math<br />
standardized scores, attendance and punctuality, interview, essay or additional diagnostic tests.<br />
Educational Option programs also evaluate students individually, but for half their seats. <strong>The</strong><br />
other half is allocated by lottery. Allocation <strong>of</strong> seats in each half targets a distribution <strong>of</strong><br />
student ability: 16 percent <strong>of</strong> seats should be allocated to high performing readers, 68 percent<br />
to middle performers, and 16 percent to low performers. Unscreened programs admit students<br />
by random lottery, while Zoned programs give priority to students who apply and live in the<br />
geographic zoned area <strong>of</strong> the high school. Limited Unscreened programs allocate seats randomly,<br />
but give priority to students who attend an information session or visit the school’s exhibit at<br />
high school fairs or open houses conducted each admissions season. We categorize programs<br />
into three main groups: Screened programs which also include Testing and Audition programs,<br />
Unscreened programs, which also include limited Unscreened and Zoned programs, and the rest<br />
are Educational Option programs.<br />
Throughout the last decade, a important component <strong>of</strong> education reform efforts in NYC<br />
involved closing and opening <strong>of</strong> new small high schools throughout the city, with roughly 400<br />
students. A big push for these small high schools came as part <strong>of</strong> the New Century High <strong>School</strong>s<br />
Initiative launched by Mayor Bloomberg and Chancellor Klein. Eleven new small high schools<br />
were opened in 2002 and 23 new small schools were opened in 2003, and the peak year <strong>of</strong> small<br />
high school openings was in 2004 (Abulkadiroğlu, Hu, and Pathak 2013). Most <strong>of</strong> these schools<br />
are small and have about 100 students per entering class. As a result, the new small high schools<br />
have a relatively small impact on overall enrollment patterns during our study period.<br />
7 Abdulkadiroğlu, Angrist, and Pathak (2013) describe their admissions process in more detail.<br />
6
2.2 Uncoordinated Admissions in 2002-03<br />
Admissions to the Specialized High <strong>School</strong>s and the LaGuardia High <strong>School</strong> <strong>of</strong> Music & Art and<br />
Performing Arts have been traditionally administered as a separate process from regular high<br />
schools, which did not change with the new mechanism. 8 In 2002-03, nearly 30,000 rising high<br />
schoolers applied for about 5,000 spots at one <strong>of</strong> the five Specialized High <strong>School</strong>s or LaGuardia<br />
by taking the SHSAT in late October and rank them in order <strong>of</strong> preference.<br />
About 80,000 students who are interested in regular high schools visit schools and attend citywide<br />
high school fairs before submitting their preference in the fall. In 2002-03, students could<br />
apply to at most five regular programs in addition to the Specialized High <strong>School</strong>s. Programs<br />
receiving a student’s application were able to see the applicant’s entire preference list, including<br />
where their program was ranked. Programs then decided whom to accept, place on a waiting<br />
list, or reject. Applicants were sent a decision letter from each program they had applied to and<br />
some obtained more than one <strong>of</strong>fer. Students were allowed to accept at most one admission and<br />
one wait-list <strong>of</strong>fer. After receiving responses to the first letters, programs with vacant seats could<br />
make new <strong>of</strong>fers to students from waiting lists. After the second round, students who do not have<br />
a zoned high school are allowed to participate in a supplementary round known as the variable<br />
assignment (VAS) process. In the supplementary round, students could rank up to eight choices,<br />
and students were assigned based on negotiation on seat availability between the enrollment<br />
<strong>of</strong>fice and high school superintendents. After replies to the second letter were received, a third<br />
letter with the new <strong>of</strong>fers was sent. New <strong>of</strong>fers did not necessarily go to wait-listed students in<br />
a predetermined order. Remaining unassigned students were assigned their zoned programs or<br />
administratively, when the central <strong>of</strong>fice tried to place kids as close to home as possible, ignoring<br />
their preferences. We refer to this final stage as the over the counter round. 9<br />
Three features <strong>of</strong> this assignment scheme motivated the NYC DOE to abandon it in favor <strong>of</strong> a<br />
new mechanism. First, there was inadequate time for <strong>of</strong>fers, wait list decisions, and acceptances<br />
to clear the market for school seats. DOE <strong>of</strong>ficials reported that in some cases high achieving<br />
students received acceptances from all <strong>of</strong> the schools they applied to, while many received none<br />
(Herszenhorn 2004). Comments by the Deputy <strong>School</strong>s Chancellor summarized the frustration:<br />
“Parents are told a school is full, then in two months, miracles <strong>of</strong> miracles, seats open up, but<br />
other kids get them. Something is wrong” (Gendar 2000).<br />
Second, some schools awarded priority in admissions to students who ranked them first on<br />
their application form. <strong>The</strong> high school directory advises that when ranking schools, students<br />
should “... determine what your competition is for a seat in this program” (DOE 2002). This<br />
recommendation puts strategic pressure on ranking decisions. Listing such a school first would<br />
improve the likelihood <strong>of</strong> an <strong>of</strong>fer at the risk <strong>of</strong> rejection from other schools, which take rankings<br />
8 <strong>The</strong> 1972 Hecht-Calandra Act is a New York State law that governs admissions to the original four Specialized<br />
high schools: Stuyvesant, Bronx High <strong>School</strong> <strong>of</strong> Science, Brooklyn Technical, and Fiorello H. LaGuardia High<br />
<strong>School</strong> <strong>of</strong> Music and Performance Arts. City <strong>of</strong>ficials indicated that this law prevents including these schools<br />
within the common application system without an act <strong>of</strong> the state legislature.<br />
9 Other accounts <strong>of</strong> New York City’s high school admissions process now refer to the over the counter round<br />
as the the placement <strong>of</strong> students who did not submit an high school application because they are new to New<br />
York City (Arvidsson, Fruchter, and Mokhtar 2013). Our analysis follows applicants through to assignment and<br />
therefore does not consider students who arrived to the process after the high school match.<br />
7
into account.<br />
Third, a number <strong>of</strong> schools managed to conceal capacity to fill seats later on with better<br />
students. For example, the Deputy Chancellor stated, “before you might have a situation where a<br />
school was going to take 100 new children for ninth grade, they might have declared only 40 seats,<br />
and then placed the other 60 outside the process” (Herszenhorn 2004). Overall, critics alleged<br />
that the old mechanism disadvantaged low-achieving students, and those without sophisticated<br />
parents (Hemphill and Nauer 2009).<br />
2.3 <strong>Coordinated</strong> Admissions in 2003-04<br />
<strong>The</strong> new mechanism was designed with input from economists (see Abdulkadiroğlu, Pathak,<br />
and Roth (2005) and Abdulkadiroğlu, Pathak, and Roth (2009)). When publicizing the new<br />
mechanism, the DOE explained that its goals were to utilize school places more efficiently and<br />
to reduce the gaming involved in obtaining school seats (Kerr 2003). In the first round, students<br />
apply to Specialized High <strong>School</strong>s when they take the SHSAT, just as in previous years. Offers<br />
are produced according to a serial dictatorship with priority given by SHSAT scores.<br />
In the main round, students can rank up to twelve regular school programs in their application,<br />
which are due in November. <strong>The</strong> DOE advised parents: “You must now rank your 12<br />
choices according to your true preferences” because this round is built on Gale and Shapley<br />
(1962)’s student-proposing deferred acceptance algorithm. <strong>School</strong>s with programs that prioritize<br />
applicants based on auditions, test scores or other criteria are sent lists <strong>of</strong> students who ranked<br />
the school, but these lists do not reveal where in the preference lists they were ranked. <strong>School</strong>s<br />
return orderings <strong>of</strong> applicants to the central enrollment <strong>of</strong>fice. <strong>School</strong>s that prioritize applicants<br />
using geographic or other criteria have those criteria applied by the central <strong>of</strong>fice. That <strong>of</strong>fice<br />
uses a single lottery to break ties amongst students with the same priority, generating a strict<br />
ordering <strong>of</strong> students at each school.<br />
<strong>Assignment</strong> is determined by the student-proposing deferred acceptance algorithm with student<br />
preferences over the schools, school capacities, and school’s strict ordering <strong>of</strong> students as<br />
parameters. <strong>The</strong> algorithm is run with all students in February. In this first round, only students<br />
who receive a Specialized High <strong>School</strong> <strong>of</strong>fer receive a letter indicating their regular school<br />
assignment, and are asked to choose one. After they respond, students that accept an <strong>of</strong>fer are<br />
removed, school capacities are adjusted and the algorithm is re-run with the remaining students.<br />
All students receive a letter notifying them <strong>of</strong> their assignment or whether they are unassigned<br />
after the main round.<br />
Unassigned students from the main round are provided a list <strong>of</strong> programs with vacancies<br />
and are asked to rank up to twelve <strong>of</strong> these programs. In 2003-04, the admissions criteria at<br />
the remaining school seats were ignored in this Supplementary round. Students are ordered by<br />
their random number, and the student-proposing deferred acceptance algorithm is run with this<br />
ordering in place at each school. Students who remain unassigned in the supplementary round<br />
are assigned administratively. <strong>The</strong>se students and any appealing students are processed on a<br />
case-by-case basis in the over the counter round.<br />
8
3 Data and Sample Definitions<br />
<strong>The</strong> NYC DOE provided us with several data sets for this study each linked by a unique student<br />
identification number: information on student choices and assignments, student demographics,<br />
and October student enrollment. <strong>The</strong> assignment file is maintained by the Enrollment <strong>of</strong>fice<br />
(formerly known as the Office <strong>of</strong> Student Enrollment and Planning Operations), which runs<br />
high school admissions. For 2002-03, the assignment files records a student’s initial rank order<br />
list, their <strong>of</strong>fers and rejections for each round, whether they participate in the supplementary<br />
round process, and their final assignment at the conclusion <strong>of</strong> the assignment process as <strong>of</strong> July<br />
2003. For 2003-04, the assignment files contain students’ choice schools in order <strong>of</strong> preference,<br />
priority information for each school, and assignments at the end <strong>of</strong> each <strong>of</strong> the rounds, and<br />
final assignment as <strong>of</strong> early August 2004. <strong>The</strong> student demographic file for both years contains<br />
information on sending school, home address, gender, race, limited English pr<strong>of</strong>iciency, special<br />
education status, and performance on 7th grade citywide tests. We use ArcGIS to compute the<br />
road distance between each student and school, and ArcGIS with StreetMap USA to geocode<br />
each student to a census block group corresponding to her address. Though we use road distance,<br />
we also computed subway distance using the Metropolitan Transportation Authority GIS files;<br />
the overall correlation between driving distance and subway commuting distance for all studentprogram<br />
pairs is 0.96. Further details are in the Data Appendix.<br />
Our analysis sample makes two restrictions. First, since we do not have demographic information<br />
for private school applicants, we restrict the analysis to students in NYC’s public middle<br />
schools at baseline. Second, we focus on students who are not assigned to Specialized high schools<br />
because that part <strong>of</strong> the assignment process did not change with the new mechanism. Most students<br />
prefer a Specialized High <strong>School</strong> to a regular ones, limiting the potential for interaction<br />
with the main round to differentially influence assignments as the mechanism changed. Given<br />
these restrictions, we have two main analysis files: the mechanism comparison sample and the<br />
demand estimation sample.<br />
<strong>The</strong> mechanism comparison sample is used for comparisons <strong>of</strong> the assignment across the<br />
two mechanisms. This sample is the largest set <strong>of</strong> students assigned through the high school<br />
assignment mechanism to a school that exists as <strong>of</strong> the time <strong>of</strong> the printing <strong>of</strong> the high school<br />
directory. We do not include students assigned to schools which are created afterwards because<br />
we are not able to evaluate the welfare from those assignments using rankings made before<br />
they existed. Any non-private applicant in the OSEPO files who does not opt for their current<br />
school and is assigned is a member <strong>of</strong> the mechanism comparison sample in 2002-03 and 2003-04.<br />
Columns (1) and (2) <strong>of</strong> Table 1 summarize student characteristics in the mechanism comparison<br />
samples across years. 3,500 fewer students are involved in the mechanism comparison 2003-04<br />
sample, which is mainly due to the students assigned to schools created after the printing <strong>of</strong> the<br />
high school directory or to closed schools (as shown in Table B1).<br />
New York City is the nation’s largest school district, and like many urban districts it is majority<br />
low-income and non-white. Nearly three-quarters <strong>of</strong> students are black or Hispanic, and<br />
about 10% <strong>of</strong> students are Asian. Brooklyn is home to the largest number <strong>of</strong> applicants, followed<br />
by Bronx and Queens, which each account for roughly one quarter <strong>of</strong> students. Manhattan and<br />
9
Staten Island, areas with high private and parochial school penetration, account for a considerably<br />
smaller share <strong>of</strong> students about 13 and six percent, respectively. Student characteristics <strong>of</strong><br />
participants are similar across years, suggesting applicant attributes did not change substantially<br />
as a result <strong>of</strong> the new mechanism.<br />
<strong>The</strong> demand sample contains participants in the main round <strong>of</strong> the new mechanism in the<br />
assignment files. <strong>The</strong> school choices expressed by this group <strong>of</strong> students represent the overwhelming<br />
majority <strong>of</strong> students. Among the set <strong>of</strong> main round participants, we exclude a small<br />
fraction <strong>of</strong> students who are classified as the top 2 percent because these students are guaranteed<br />
a school only if they rank it first and this may distort their incentives to rank schools truthfully.<br />
Additional details on the sample restrictions are in the Data appendix. Table 1 also shows that<br />
the student characteristics are similar across the sample used for comparing mechanisms and<br />
demand samples with one exception. <strong>The</strong> fraction <strong>of</strong> students who are classified as receiving a<br />
subsidized lunch differs across years because we only have access to subsidized lunch as measured<br />
as <strong>of</strong> the 2004-05 school year. For this reason, we use census block group family income as a<br />
student demographic.<br />
Data on schools were taken from New York State report card files provided by NYC DOE.<br />
Information on programs come from the <strong>of</strong>ficial NYC High <strong>School</strong> Directories made available to<br />
students before the application process. Table 2 summarizes school and program characteristics<br />
across years. <strong>The</strong>re is an increase in the number <strong>of</strong> schools from 215 to 235, and a corresponding<br />
decrease in the average number <strong>of</strong> students enrolled per school <strong>of</strong> about 40 students. This fact is<br />
driven by the replacement <strong>of</strong> some large schools with smaller schools that took place concurrently<br />
in 2003-04 described above. Despite this increase, there is little change in the average achievement<br />
levels <strong>of</strong> schools and school demographic composition as measured by report card data. Moreover,<br />
slightly more than half <strong>of</strong> the teachers are classified as inexperienced – having taught for less<br />
than two years – a pattern which holds for both years.<br />
Students in New York can choose among roughly 600 programs throughout the city. <strong>The</strong><br />
menu <strong>of</strong> program choices is immense, and programs vary substantially in focus, post-graduate<br />
orientation, and educational philosophy. For instance, the Heritage <strong>School</strong> in Manhattan is an<br />
Educational Option school where the arts play a substantial role in learning, while Townsend<br />
Harris High <strong>School</strong> in Queens is a Screened program with a rigorous humanities program making<br />
it among the most competitive in the city. Using information from high school directories, we<br />
identify each program’s type, language orientation, and speciality. With the new mechanism,<br />
there are more Unscreened programs and fewer Educational Option programs, a change driven<br />
by the conversion <strong>of</strong> many Educational Option programs to Unscreened programs, given their<br />
overlapping admissions criteria. We code language-focused programs as Spanish, Asian, or Other,<br />
and categorize program specialities into Arts, Humanities, Math and Science, Vocational, or<br />
Other. Not all programs have specialties, though about 70% fall into one <strong>of</strong> these classes.<br />
(Details on our classification scheme are in the Data appendix). <strong>The</strong> menu <strong>of</strong> language program<br />
<strong>of</strong>ferings or program specialities changes little across years.<br />
Given the sudden announcement <strong>of</strong> the new mechanism and the similar student attributes<br />
across years, it seems reasonable that there was little scope for participants to react by either<br />
opting out or in to the mechanism. Likewise, it does not appear that there is a large change<br />
10
in the residential locations <strong>of</strong> students. <strong>The</strong> distribution <strong>of</strong> students throughout the city looks<br />
nearly identical to that in the new mechanism, shown in the map in Figure 1. Moreover, though<br />
there are relatively small changes in the set <strong>of</strong> school options, these mostly involve an increase<br />
in the number <strong>of</strong> Unscreened programs, and a decrease in the number <strong>of</strong> Educational Option<br />
programs, and these two program types are similar. <strong>The</strong>se facts together motivate our approach<br />
<strong>of</strong> using preference estimates from choices expressed in 2003-04 to measure student welfare from<br />
the final assignments in 2002-03, and to attribute those to changes in the assignment mechanism<br />
rather than changes in the attributes <strong>of</strong> student participants or the menu <strong>of</strong> school options.<br />
4 Descriptive Evidence on Mechanism Performance<br />
4.1 Distance, Exit, and Matriculation<br />
One <strong>of</strong> the most important characteristics <strong>of</strong> a student’s school assignment is how far away it<br />
is. Figure 2 reports the overall distribution <strong>of</strong> road distance travelled by students in both the<br />
uncoordinated and coordinated mechanism. New York City spans a large geographic range, with<br />
nearly 45 miles separating the southern tip <strong>of</strong> Staten Island with the northern most points <strong>of</strong> the<br />
Bronx, and 25 miles traveling from the western edge <strong>of</strong> Manhattan near Washington Heights to<br />
Far Rockaway at the easternmost tip <strong>of</strong> Brooklyn. 10 <strong>The</strong> closest school for a typical student is<br />
0.82 miles from home, and students in the uncoordinated mechanism on average travelled 3.36<br />
miles to their assignment. In the coordinated mechanism, the average distance is 4.05 miles.<br />
<strong>The</strong> medians also increase from 2.25 to 3.04 miles. <strong>The</strong> increase in distance due to a coordinated<br />
mechanism is consistent with the pattern in Niederle and Roth (2003)’s study <strong>of</strong> a new centralized<br />
match in the gastroentrology labor market. Mobility in their study is defined as the percentage<br />
<strong>of</strong> gastroenterology fellows who change hospitals (or cities or states) after finishing their previous<br />
training (usually in internal medicine) and starting the gastroenterology fellowship. <strong>The</strong>y report<br />
a centralized mechanism is associated with mobility increases at the hospital, city, and state<br />
level.<br />
<strong>The</strong> increase in distance to assignment suggests that the scope <strong>of</strong> the market increased under<br />
the coordinated mechanism, but it does not directly imply a statement about welfare. Table 3<br />
presents further information on distance and <strong>of</strong>fer processing under both mechanisms. In the<br />
uncoordinated mechanism, more students obtain their final assignment in the last round than in<br />
the first round. Panel A <strong>of</strong> the table shows that 36% <strong>of</strong> students are assigned over the counter,<br />
compared to 35% in the first round. Since 33,894 students obtain one or more first round <strong>of</strong>fers<br />
(shown in Panel B), but only 24,292 students were finalized in the first round, 9,602 students<br />
with a first round <strong>of</strong>fer were finalized as <strong>of</strong>fers were made in subsequent rounds. <strong>The</strong> processing<br />
<strong>of</strong> these students took place as schools revised <strong>of</strong>fers based on first round rejections and made<br />
<strong>of</strong>fers anew in the second and third round. However, the large fraction <strong>of</strong> students assigned in<br />
the over-the-counter round suggests that three rounds were not sufficient to process all students.<br />
Students who are <strong>of</strong>fered earlier are assigned to and enroll at schools that are further away<br />
than students who are assigned in the last round. This fact is unsurprising given that the<br />
10 Table S1 reports on the relationship between road distance and subway distance. With the exception <strong>of</strong><br />
Manhattan, the overwhelming majority <strong>of</strong> students are assigned to a school in the same borough as their residence.<br />
11
DOE’s explicit aim was to find space for over-the-counter students based on their residence.<br />
After receiving an assignment, a student may opt for a private school, leave New York, or even<br />
drop out. Families may switch schools after their final assignments are announced, but before the<br />
school year starts for a number <strong>of</strong> reasons including moving within the city. In the uncoordinated<br />
mechanism, principals had greater discretion to enroll students and the DOE <strong>of</strong>ficials quoted<br />
above alleged that students with sophisticated parents might just show up at a school in the<br />
fall and ask for a place. Based on exit and matriculation patterns, students assigned earlier<br />
appear less satisfied with their assignment than those assigned in earlier rounds. <strong>The</strong> fraction <strong>of</strong><br />
students who exit NYC public schools is 13.5% among over-the-counter placements, which is 5%<br />
greater than the fraction who exit in earlier rounds. More than a quarter <strong>of</strong> students assigned in<br />
the over-the-counter round who are still in NYC public schools matriculate at schools other than<br />
where they were assigned. By comparison, the take-up <strong>of</strong> <strong>of</strong>fered assignments is much higher for<br />
those assigned in the first three rounds.<br />
Panel B shows that roughly one-fifth <strong>of</strong> students obtained multiple <strong>of</strong>fers in the uncoordinated<br />
mechanism. Though not reported in the table, 73% <strong>of</strong> those with multiple <strong>of</strong>fers receive two <strong>of</strong>fers,<br />
22% <strong>of</strong> receive three <strong>of</strong>fers, 5% receive four <strong>of</strong>fers, and 64 applicants received <strong>of</strong>fers at all five<br />
<strong>of</strong> their schools. Since a student can take at most one <strong>of</strong> these <strong>of</strong>fer, these numbers imply that<br />
there were a total <strong>of</strong> 17,263 extra <strong>of</strong>fers made. Based on exit and matriculation, students with<br />
multiple <strong>of</strong>fers are more satisfied with their assignment than students with zero or one <strong>of</strong>fer.<br />
<strong>The</strong>se students also travel further to their final assignment or enrolled school.<br />
Table A1 shows that <strong>of</strong> the students who accept a first round <strong>of</strong>fer, about 71% take their<br />
most preferred ranked <strong>of</strong>fer. This fact implies that the majority <strong>of</strong> students who had multiple<br />
<strong>of</strong>fers did not necessarily benefit from having more than one option if their top option is their<br />
most preferred. Multiple <strong>of</strong>fers come at the cost <strong>of</strong> more than half <strong>of</strong> the applicants receiving<br />
no first round <strong>of</strong>fer. Though not reported, 58% <strong>of</strong> students without any <strong>of</strong>fer are assigned in<br />
the over-the-counter process. This fact explains why their assigned schools are closer, they are<br />
more likely to exit NYC public schools, and more likely to enroll at a school other than their<br />
assignment than students with one or more <strong>of</strong>fer.<br />
4.2 Offer Processing by Student Characteristics<br />
Students from across all demographic groups, boroughs, and baseline achievement categories<br />
travel further to their assigned school except in Manhattan (shown in Table 10). <strong>The</strong> increased<br />
distance for the student subgroups are close to the overall average increase in distance <strong>of</strong> 0.69<br />
miles. High math baseline and SHSAT test takers tend not to travel as far, while black students<br />
and those from the Bronx and Queens travel further.<br />
As we’ve seen, changes in distance are related to the fraction who are assigned in the overthe-counter<br />
process. Table 4 reports the attributes <strong>of</strong> students across rounds and number <strong>of</strong> first<br />
round <strong>of</strong>fers compared to the overall population <strong>of</strong> applicants in the uncoordinated mechanism<br />
(shown in column (1)). Students from Manhattan, those with high math baseline scores, and<br />
those who have taken the SHSAT tend to obtain <strong>of</strong>fers earlier in the uncoordinated admissions<br />
process. <strong>The</strong>y are overrepresented among students either finalized in the first round or who<br />
receive multiple first round <strong>of</strong>fers. For example, 30% <strong>of</strong> the students who receive multiple first<br />
12
ound <strong>of</strong>fers take the SHSAT compared to 22.4% overall. Students from Staten Island, students<br />
who are white, and those from high income neighborhoods tend to systematically obtain <strong>of</strong>fers<br />
later in the process. Compared to the overall population, these groups are overrepresented in the<br />
over-the-counter round. About a quarter <strong>of</strong> students in the over-the-counter process are white,<br />
compared to 15% <strong>of</strong> participants in the overall distribution. <strong>The</strong> differences in student attributes<br />
across rounds are not as pronounced with the coordinated mechanism. In the over-the-counter<br />
process, 22.8% <strong>of</strong> students are SHSAT takers compared to 24.6% in the main round and 16.0%<br />
are white compared to 12.6% in the main round. <strong>The</strong>se facts are consistent with coordinated<br />
assignment more equitably distributing school access across rounds for different student groups.<br />
4.3 Mismatch in Over-the-Counter Round<br />
Before turning to estimates <strong>of</strong> student preferences, we compare the attributes <strong>of</strong> schools ranked<br />
by students processed in each round to the attributes <strong>of</strong> the assigned school. This comparison<br />
indicates how students processed in the main, supplementary, and over-the-counter rounds fare<br />
relative to what they wanted according to their choices.<br />
Students who are processed in earlier rounds are assigned to schools that have attributes more<br />
similar to the schools they ranked than students processed in later rounds. Table 5 shows that<br />
for those assigned in the main round, with the exception <strong>of</strong> distance, ranked schools tend to have<br />
slightly better attributes than assignments in both mechanisms. Ranked schools have higher<br />
English and Math achievement, more four-year college-going, fewer inexperienced teachers, and<br />
higher attendance rates.<br />
For students placed in the supplementary round, assignments are even worse than ranked<br />
schools. In the uncoordinated mechanism, for instance, the gap high math achievement between<br />
ranked and assigned schools is 0.8 percentage points for those assigned in the main round, while<br />
it 2.5 points in the supplementary round. <strong>The</strong> gap between ranked and assigned alternatives<br />
in teacher experience and subsidized lunch peers also larger in the supplementary round than<br />
in the main round. Since students participate in the supplementary round if did not obtain a<br />
main round assignment, it is not surprising that these students are assigned to schools with less<br />
desirable characteristics.<br />
<strong>The</strong> most striking pattern in Table 5, however, is for students who are assigned in the overthe-counter<br />
round. We’ve already seen that there are three times more students assigned in this<br />
round in the uncoordinated mechanism and that these students are assigned to schools much<br />
closer to home than those in other rounds. Aside from distance, each over-the-counter student’s<br />
assignment is considerably worse than the schools they ranked. <strong>The</strong>re is a large gap between the<br />
English and Math achievement levels, college-going, teacher experience, subsidized lunch levels,<br />
and attendance <strong>of</strong> ranked schools compared to assigned ones in the uncoordinated mechanism.<br />
For instance, there is a 6.8 percentage point gap in English achievement between ranked and<br />
assigned schools in the uncoordinated mechanism, which corresponds to a 40% standard deviation<br />
reduction in this measure.<br />
<strong>The</strong> differences between ranked and assignments are also large in the coordinated mechanism,<br />
though in some cases, they are not as stark. Achievement differences between ranked and assigned<br />
schools are narrower: for English, it’s a 5.0 percentage point gap (compared to 6.8) and for Math,<br />
13
it’s 3.6 vs. 4.3. On the other hand, assigned schools are not as close to home in the coordinated<br />
mechanism, and all else equal, families prefer closer schools. <strong>The</strong>refore, it is not possible to<br />
assert whether the over-the-counter process was better or worse in the two mechanisms. What<br />
is clear is that being processed in the over-the-counter round is undesirable for students in both<br />
mechanisms. As a result, a significant fraction <strong>of</strong> the changes in student welfare will be driven by<br />
the large reduction in the number <strong>of</strong> students who enter this round in the coordinated mechanism.<br />
5 Measuring Student Preferences<br />
5.1 Student Choices<br />
What makes a families rank particular schools and where do they obtain information about school<br />
options? In surveys, parents state that academic achievement, school and teacher quality are the<br />
most important school characteristics (Schneider, Teske, and Marschall 2002). In NYC, families<br />
obtain information about high schools and programs from many sources. Guidance counselors,<br />
teachers, peers, and other families in the neighborhood all provide input and recommendations.<br />
<strong>The</strong> <strong>of</strong>ficial repository <strong>of</strong> information on high schools is the NYC High <strong>School</strong> directory, a booklet<br />
that includes information about school size, advanced course <strong>of</strong>ferings, Regents and graduation<br />
performance, as well as the school’s address, and closest bus and subway. <strong>The</strong> directory also<br />
includes a paragraph description <strong>of</strong> each program together with a list <strong>of</strong> extracurricular activities<br />
and sports teams. Families learn about schools at High school fairs that are held on Saturdays<br />
and Sundays in the fall in each borough, from individual school open houses, from online school<br />
guides such as insideschools.org, and from books about high schools such as Hemphill (2007).<br />
Finally, local newspapers such as the New York Post and New York Daily News regularly publish<br />
rankings <strong>of</strong> high schools, using publicly available information from the New York State Report<br />
Cards.<br />
Even though the school ranking decision involves evaluating many different alternatives, the<br />
aggregate distribution <strong>of</strong> preferences displays some consistent regularities: students prefer closer<br />
and higher quality schools as measured by student achievement levels, shown in Table 6. 11 <strong>The</strong><br />
first row <strong>of</strong> the table shows that 20% <strong>of</strong> applicants rank 12 school choices; the majority rank nine<br />
or fewer choices and nearly 90% rank at least three choices. <strong>The</strong> average student’s top choice<br />
is 4.43 miles away from home. Given that the closest school is on average 0.82 miles away, this<br />
means that students do not simply rank the school closest to home. An average student’s first<br />
choice is nearly 0.40 miles closer than her second choice, and her second choice is about 0.25 miles<br />
closer than her third choice. Even though other school characteristics change with distance, the<br />
distance to ranked school increases monotonically until the 9th choice, which is 5.65 miles away<br />
on average. <strong>The</strong> distance to the 10-12th choices is lower than the 9th choice, but since less than<br />
half <strong>of</strong> students rank 10 schools, this may be due to compositional changes.<br />
Lower ranked schools are not only farther away, but they also less desirable on other measures<br />
<strong>of</strong> school quality. We measure quality using the Regents performance information in the NYC<br />
11 Table A2 provides additional information on school assignments. 31.9% <strong>of</strong> students receive their top choice,<br />
15.0% receive their second choice, and 2.4% receive a choice ranked 10th, 11th, or 12th. 17.5% <strong>of</strong> students are<br />
asked to participate in the supplementary round because they are unassigned in the main round.<br />
14
High <strong>School</strong> directory, which comes from NY state report cards. <strong>The</strong> fraction <strong>of</strong> students scoring<br />
an 85 or higher on the English or Math Regents exam decreases going down rank order lists.<br />
<strong>The</strong> percent <strong>of</strong> students attending a four year college also decreases as we go down rank order<br />
lists, and the fraction <strong>of</strong> teachers classified as inexperienced increases. Finally, schools enrolling<br />
a higher share <strong>of</strong> white and Asian students tend to be ranked higher than schools with smaller<br />
shares <strong>of</strong> these student groups.<br />
Using requests for individual teachers, Jacob and Lefgren (2007) find that parents in low<br />
income and minority schools value a teacher’s ability to raise student achievement more than in<br />
high income and non-minority schools. This difference across groups motivates our investigation<br />
<strong>of</strong> ranking behavior by baseline ability and neighborhood income. Both high and low baseline students<br />
prefer schools that are closer to home, but first choices are closer for low baseline students.<br />
High achieving students also tend to rank schools with high English and Math achievement,<br />
relative to low achievers, though both groups place less emphasis on achievement with further<br />
down their preference list. Similarly, students from low income neighborhoods tend to put less<br />
weight on Math and English achievement than students from high income neighborhoods, but<br />
both groups rank higher achieving schools higher. <strong>The</strong>se differences suggest the importance <strong>of</strong><br />
allowing for tastes for school achievement to differ by baseline achievement and income groups.<br />
Based on their submitted preferences, all else equal, students prefer attending a school closer<br />
to home. <strong>The</strong> fact that students in the new mechanism are assigned to schools further from home<br />
might suggest that the new mechanism led to assignments that are worse on average than the<br />
old mechanism. On the other hand, students may prefer schools outside <strong>of</strong> their neighborhood<br />
because they are a better fit. We must therefore weigh the greater travel distance in the new<br />
mechanism against the attributes <strong>of</strong> the school. Students do not rank the closest school to their<br />
home and instead trade <strong>of</strong>f school attributes with proximity. Our next task is to quantify how<br />
students evaluate distance relative to school attributes such as size, demographic composition,<br />
and average achievement levels based on their submitted preferences.<br />
5.2 Model and Estimation Strategy<br />
<strong>The</strong> comparison <strong>of</strong> the attributes <strong>of</strong> first choices relative to later choices provides rich information<br />
to identify how a student trades <strong>of</strong>f these features <strong>of</strong> schools in her preferences. To quantify these<br />
trade<strong>of</strong>fs, we work with a random utility model. Let i index students, j index programs, and<br />
let s j denote the school housing program j. We model student i’s indirect utility for program j<br />
using the following specification:<br />
u ij = δ sj + ∑ l<br />
α l z l ix l j + γd ij + ε ij , with<br />
δ sj = x sj β + ξ sj ,<br />
where z i is the vector <strong>of</strong> student characteristics, x j is a vector <strong>of</strong> program j’s characteristics, d ij<br />
is distance between student i’s home address and the address <strong>of</strong> program j, ξ sj is a school-specific<br />
unobserved vertical characteristic, and ε ij captures random fluctuations tastes via draws from an<br />
extreme value type-I distribution with variance normalized to π2<br />
6<br />
. Since students rank programs,<br />
it is natural to specify utility in terms <strong>of</strong> programs. However, we write the mean utility δ sj only<br />
15
in terms <strong>of</strong> school attributes since a program’s type and speciality are the only features that do<br />
vary within schools.<br />
This demand specification allows us to exploit the unusually large number <strong>of</strong> school and<br />
student characteristics and the student rank order lists in the micro data. Each model contains<br />
234 school fixed effects for each <strong>of</strong> the 235 schools, where one is dropped for our normalization. To<br />
take advantage <strong>of</strong> these characteristics, we estimate models with a large number <strong>of</strong> interactions,<br />
where each student characteristic is interacted with each school characteristic, and the logit<br />
functional form provides a parsimonious way to sidestep the need for numerical integration<br />
because we can write the probabilities in closed form. <strong>The</strong> logit assumption also comes with<br />
the Independence <strong>of</strong> Irrelevant Alternatives (IIA) property, which implies that if the choice<br />
set changes, the relative likelihood <strong>of</strong> ranking any two schools does not change. Most <strong>of</strong> our<br />
counterfactual exercises compare two different allocations by aggregating utilities for two different<br />
assignments. Our aim is to best fit the utilities <strong>of</strong> students given their observable characteristics.<br />
Unlike applications which relax the IIA property (Berry, Levinsohn, and Pakes 1995, Berry,<br />
Levinsohn, and Pakes 2004), we are not interested in predicting the consequences <strong>of</strong> changing<br />
prices or attributes for a given individual. In our setting, there are also relatively modest changes<br />
in the set <strong>of</strong> school options across years.<br />
In our preferred models, we do not explicitly include an outside option and instead normalize,<br />
without loss <strong>of</strong> generality, the value <strong>of</strong> δ for an arbitrarily chosen school to zero. This assumption<br />
is motivated by our primary interest in studying the allocation within inside options rather than<br />
substitution outside <strong>of</strong> the NYC public school system. As we describe below, we also find the<br />
behavioral implications <strong>of</strong> an outside option in rank order lists that do not rank all 12 alternatives<br />
unappealing. Indeed, the IIA property implies that our estimates would not change had we<br />
observed the actual rank <strong>of</strong> the outside option.<br />
<strong>The</strong> demand sample contains rankings <strong>of</strong> 69,907 participants in 2003-04 over 497 programs<br />
in 235 schools, representing a total <strong>of</strong> 542,666 school choices. We start by building a likelihood<br />
function assuming that choices are truthful before examining this assumption in detail. This<br />
assumption implies that student i ranks programs in order <strong>of</strong> the indirect utility she derives from<br />
the programs. We assume that all unranked schools are less preferred to all ranked schools. Let r i<br />
be the rank order list submitted by student i in our demand sample, with |r i | ≤ 12 denoting the<br />
length <strong>of</strong> the agent’s list and r ik denoting the kth choice. Let θ = (α, γ, δ) denote the parameters<br />
<strong>of</strong> interest. Given θ and our logit assumption, the likelihood that student i submits rank order<br />
list r i is:<br />
)<br />
L (θ|r i , z i , X, D i ) = Π |r i|<br />
k=1<br />
(u P irik > max u ij |θ, z i , x j , D i<br />
j∈J\∪ l
students ranking programs is much larger than the number <strong>of</strong> programs, the estimation error in<br />
δ is negligible compared to the error in β due to sample variance in the set <strong>of</strong> observed schools.<br />
6 Preference Estimates and Model Fit<br />
Our specifications follow other models <strong>of</strong> school demand and include average school test scores<br />
and racial attributes as characteristics (see, e.g., Hastings, Kane, and Staiger (2005)). <strong>The</strong>re are,<br />
<strong>of</strong> course, many school features influencing choices that we do not observe. <strong>The</strong>refore, unlike<br />
these earlier models, we include additive school fixed effects to capture school-level unobservables.<br />
As we have seen above, the distance to a school is an important dimension accounting for<br />
choices, so all demand models control for distance. We do not include more flexible controls<br />
for distance because it serves as our numeraire relative to which we measure the value <strong>of</strong> other<br />
school characteristics and a unit for welfare in the counterfactuals. In each specification, we also<br />
include three program type dummies and ten program speciality dummies. 12 <strong>The</strong> categorization<br />
<strong>of</strong> programs into specialities is described in the Data appendix.<br />
6.1 Demand Estimates<br />
Table 7 reports estimates for six specifications <strong>of</strong> our demand model. <strong>The</strong> first specification has<br />
school effects and 14 additional parameters (program specialty dummies, program type dummies,<br />
distance, and school characteristics). <strong>The</strong> school characteristics are size <strong>of</strong> 9th grade, percent<br />
white, attendance rate, percent subsidized lunch, and the Regents math performance. <strong>The</strong> second<br />
specification adds 34 parameters by interacting school characteristics with student racial<br />
characteristics. <strong>The</strong> third specification interacts a enrollment, attendance rate, and math performance<br />
with student baseline achievement. <strong>The</strong> next three specifications include interactions<br />
between all student demographics (gender, race, Special Ed, Limited English Pr<strong>of</strong>icient), math<br />
and English baseline achievement, and the school characteristics. We also include dummies for<br />
Spanish, Asian and Other Language Program, interacting these dummies with a student’s LEP<br />
status and whether they are Hispanic or Asian. This model has 349 parameters. We report three<br />
variations <strong>of</strong> this specification: using only the top ranked school, using only the top three ranked<br />
schools, and using all ranked schools. Since these specifications include many interactions, we<br />
only report a subset <strong>of</strong> coefficients, and present all <strong>of</strong> the estimates in Table C1.<br />
All else equal, students prefer schools closer to home. This fact can be seen by the negative<br />
coefficient, -0.31, on distance in the first specification in Table 7. As we add interactions, the<br />
estimate on distance is remarkably stable. Consistent with the patterns in Table 6, the size<br />
<strong>of</strong> this coefficient relative to the other coefficients also implies that distance has an important<br />
weight on choices relative to other attributes. <strong>The</strong> estimate <strong>of</strong> size <strong>of</strong> 9th grade or percent white<br />
implies that students are willing to travel for larger schools and for schools with more whites.<br />
Students also prefer schools with higher attendance rates, fewer subsidized lunch students, and<br />
higher math performance.<br />
12 <strong>The</strong> program type dummies are Unscreened, Screened, and Educational Option. <strong>The</strong> program specialty<br />
dummies are Arts, Humanities/Interdisciplinary, Business/Accounting, Math/Science, Career, Vocational, Government/Law,<br />
Other, Zoned, and General.<br />
17
Many interaction terms in columns (2)-(4) are significantly estimated and have the expected<br />
signs. For instance, students with high baseline math scores tend to prefer schools with higher<br />
attendance rates and high math performance more than those with low baseline scores. <strong>The</strong><br />
psuedo-R 2 indicates the fraction <strong>of</strong> variation in utility we can explain with observables. Though<br />
many <strong>of</strong> these interaction terms are significantly estimated, they do not lead to a large improvement<br />
the our psuedo-R 2 relative to the model without interactions. <strong>The</strong> psuedo-R 2 increases<br />
from 0.85 in column (1) to 0.92 in column (4). Thus, after accounting for distance, the school<br />
and program fixed effects, there are still unobserved components <strong>of</strong> student tastes.<br />
Aside from comparisons with the distance coefficient and the significance levels, the magnitude<br />
<strong>of</strong> the estimates in Table 7 are hard to directly interpret. We’d like to know, in particular,<br />
about the extent <strong>of</strong> student-level preference heterogeneity based on the estimates. In Table 8, we<br />
report on some simple summary measures <strong>of</strong> heterogeneity using the estimates from the model<br />
with interactions between school characteristics and student demographics and school characteristics<br />
and student baseline achievement reported in column (4) <strong>of</strong> Table 7. For each student<br />
group, we compute the distance-equivalent utility in miles from each school program by substituting<br />
student and school characteristics into our preference estimates. For this calculation,<br />
we ignore the unobserved taste shock, ɛ ij , and normalize the average utility for students in a<br />
category, net <strong>of</strong> distance, across programs to be zero. <strong>The</strong> first four columns report the average<br />
value split by quartile, while the last column reports the range from top to bottom quartile.<br />
Relative to the average across all students, Asian and white students are willing to travel<br />
more for schools than black and Hispanics. This fact can be seen in Panel A in column (5), where<br />
the willingness to travel to go from a top to bottom quartile school is a distance-equivalent <strong>of</strong><br />
11.00 and 11.93 miles for Asians and whites, while it is 8.56 and 8.40 for blacks and Hispanics,<br />
respectively. Similarly, high math baseline students exhibit greater willingness to travel for<br />
schools than low math baseline students. In a model without student level heterogeneity, the<br />
range across quartiles for rows split by race or by baseline achievement would be identical to the<br />
first row.<br />
<strong>The</strong> remaining panels <strong>of</strong> Table 8 report on willingness to travel for quartiles <strong>of</strong> school attributes.<br />
As before, we compute the distance-equivalent utility in miles, normalizing the mean<br />
utility to zero within student category. High math baseline students exhibit more spread in their<br />
valuation <strong>of</strong> high-achieving schools than low baseline students. Students from richer neighborhoods<br />
exhibit also more spread in their valuation <strong>of</strong> the percent <strong>of</strong> peers obtaining subsidized<br />
lunches. Finally, Asian and white students exhibit greater taste variation in school percent white<br />
than blacks and Hispanics. Each comparison reported here indicates that observable dimensions<br />
<strong>of</strong> student preference heterogeneity estimated from our demand model are important role in our<br />
welfare calculations.<br />
6.2 Assessing the Behavioral Assumptions<br />
Treating submitted rankings truthful, as in the estimates in the first four columns <strong>of</strong> Table 7, is<br />
a natural starting point and apparently generates sensible patterns. It is a benchmark because<br />
<strong>of</strong> the straightforward incentive properties <strong>of</strong> the mechanism and the advice that the NYC DOE<br />
provides in the 2003-04 High <strong>School</strong> Directory and in their information and outreach campaign.<br />
18
For instance, the DOE guide advises participants to “rank your twelve selections in order <strong>of</strong><br />
your true preferences” with the knowledge that “schools will no longer know your rankings.”<br />
Nonetheless, truthful behavior is a strong assumption worth investigating for a number <strong>of</strong> reasons.<br />
A first issue is whether students are deliberately ranking schools, given that there may be<br />
substantial frictions involved in evaluating roughly 500 school programs. Kling, Mullainathan,<br />
Shafir, Vermuelen, and Wrobel (2012), for instance, document “comparison frictions” in the<br />
evaluation <strong>of</strong> different Medicare Part D prescription drug plans. <strong>The</strong> choice between a vast<br />
number <strong>of</strong> school programs may be daunting and as a result, some may simply randomly list<br />
choices, especially further down their rank order list. If preferences were generated in this way<br />
by a large fraction <strong>of</strong> participants, then we’d expect that most <strong>of</strong> our point estimates to be<br />
imprecisely estimated or have unintuitive patterns, but they do not. Another way to see that<br />
choices matter for participants is to examine enrollment decisions by choice. Table A3 reports<br />
assignment and enrollment decisions for students who are assigned in the main round. <strong>The</strong><br />
table shows that 92.7% <strong>of</strong> students enroll in their assigned choice, and this number varies from<br />
88.4% to 94.5% depending on which choice a student receives. Interestingly, take-up is higher<br />
for students who receive lower ranked choices, while the fraction <strong>of</strong> students who exit is highest<br />
among students who obtain one <strong>of</strong> their top three choices. This fact suggests that either families<br />
are indifferent between later choices and simply enroll where they obtain an <strong>of</strong>fer or families have<br />
deliberately investigated later choices and are therefore willing to enroll in lower ranked schools.<br />
If families are more uncertain about lower ranked choices, then using information contained<br />
in all submitted ranks may provide a misleading account <strong>of</strong> student preferences. <strong>The</strong> last two<br />
columns <strong>of</strong> Table 7 present demand estimates using only the top and top three ranked schools.<br />
<strong>The</strong>se smaller samples use only 13% and 36% <strong>of</strong> the submitted choices, so they discard much <strong>of</strong><br />
the data set. Despite this, the coefficient on distance is similar across these two models and the<br />
one using all submitted ranks. Many <strong>of</strong> the interaction terms have similar signs to column (4),<br />
but as expected, they are estimated less precisely. On balance, the pattern <strong>of</strong> estimates shows<br />
remarkable consistency.<br />
<strong>The</strong> second issue with the assumption <strong>of</strong> truthful preferences is that students can rank at<br />
most 12 programs on school applications. When a student is interested in more than 12 schools,<br />
she has to carefully reduce her choice set down to at most 12 schools. If a student is only<br />
interested in 11 or fewer schools, this constraint in principle should not influence her ranking<br />
behavior (Abdulkadiroğlu, Pathak, and Roth 2009, Haeringer and Klijn 2009). It is a weakly<br />
dominant strategy to add an acceptable school to a rank order list as long as there is room for<br />
additional schools in the application form. However, 20.3% <strong>of</strong> students in our demand sample<br />
rank 12 schools. Some <strong>of</strong> these students may drop highly sought-after schools from top <strong>of</strong> their<br />
choice lists because <strong>of</strong> this constraint.<br />
To probe how restrictions on number <strong>of</strong> choices influence our preference estimates, Table A4<br />
report estimates dropping students ranking 12 choices from the sample. Despite the change in<br />
the composition <strong>of</strong> students, there is little change in our preference estimates. <strong>The</strong> coefficients<br />
on distance and school main effects are nearly identical with those in column (4), implying that<br />
the 12 choice constraint is unlikely to significantly alter our conclusions.<br />
Another concern with treated submitted rankings as truthful is that parents rank schools<br />
19
using heuristics carried over from the previous system. Despite the theoretical motivation and the<br />
DOE’s advice, parents might still deviate from truth-telling for reasons related to misinformation<br />
or lack <strong>of</strong> information about the new mechanism. Table A2 shows that students are more likely<br />
to be assigned their last choice than their penultimate choice. This pattern may be caused by<br />
strategic behavior if students apply to schools that they like and, as a safety option, rank a school<br />
in which they have a higher chance <strong>of</strong> admissions last. However, it may also be fully consistent<br />
with truth-telling. For example, students usually obtain borough priority or zone priority for<br />
schools in their neighborhoods. Ranking these schools improves their likelihood <strong>of</strong> being assigned<br />
to these schools in case they are rejected by their higher choices. If students consider applying and<br />
commuting to schools further away from their neighborhood for reasons like math and English<br />
achievement, they may as well stop ranking schools below their neighborhood schools once such<br />
considerations no longer justify the cost <strong>of</strong> commute. This preference pattern would produce the<br />
assignment pattern observed in the data. Column (3) in Table A4 presents estimates dropping<br />
the last choice <strong>of</strong> applicants; the coefficients on distance and school effects are nearly identical<br />
to column (1), implying that the possibility <strong>of</strong> strategically ranking safety schools seems unlikely<br />
to alter our conclusions.<br />
Finally, our approach avoids use <strong>of</strong> rank order lists to identify the value <strong>of</strong> the outside option<br />
relative to NYC public schools. <strong>The</strong> primary reason is that it that our counterfactuals exercises<br />
focus on the re-allocation <strong>of</strong> NYC public schools. However, one may be worried about bias in our<br />
estimates from this assumption. If ranking additional acceptable schools is cognitively costless,<br />
it may be appropriate to assume that unranked schools are worse than the outside option. Two<br />
facts suggest this is not true: the typical student ranks 9 out <strong>of</strong> 649 programs and 72.9% <strong>of</strong><br />
students who are participate in the supplementary round enroll at the school they are assigned<br />
in that round (shown in Table A3). Column (4) <strong>of</strong> Table A4 reports estimates <strong>of</strong> models only<br />
using information about the order <strong>of</strong> choices among ranked alternatives. We assume that when a<br />
student does not rank a school, that fact does not contain any information about their preferences:<br />
the school could either be ranked above an applicant’s top choice or below it. Our models, so<br />
far, treat ranked schools as more preferred than unranked schools. Perhaps unsurprisingly, the<br />
estimates from this model are considerably different than our other specifications. In particular,<br />
the importance <strong>of</strong> distance and school main effects are not comparable to the other specifications.<br />
<strong>The</strong> psuedo-R 2 is only 0.60 and the model fit is worse. As a result, the information on what<br />
schools are ranked or not ranked is important for our preference estimates.<br />
Ignoring the outside option does not yield biased estimates if the value <strong>of</strong> outside option is<br />
unrelated to preferences for inside options or if number <strong>of</strong> ranked schools is unrelated to the<br />
value <strong>of</strong> outside option. <strong>The</strong>se assumptions may not be satisfied, so the last column <strong>of</strong> Table A4<br />
reports estimates with an outside option, where we normalize the value <strong>of</strong> the outside option to<br />
zero. <strong>The</strong> IIA assumption implies that estimates should not be sensitive to the specification <strong>of</strong><br />
the outside option assumed to be preferred to all unranked schools. Consistent with this, the<br />
estimates are close to those shown in column (1). However, all <strong>of</strong> the school effects are less than<br />
0, implying that students prefer the outside option (i.e. not a NYC public school) to unranked<br />
schools. This unappealing implication contradicts the high take-up <strong>of</strong> public schools among our<br />
students.<br />
20
6.3 Model Fit<br />
An important necessary condition for using our preference estimates to make welfare statements<br />
is that they account for main moments <strong>of</strong> data. Figure 3 plots the predicted market shares<br />
<strong>of</strong> each program, computed using our preference estimates in column (4) <strong>of</strong> Table 7, against<br />
the program’s actual market share on a log scale. Market share is defined as the fraction <strong>of</strong><br />
students who rank a program anywhere on their rank order list. We compute predicted market<br />
shares by drawing the top 12 programs from the estimated preference distribution 100 times for<br />
each student and taking the average for each program. <strong>The</strong> figure shows a strong correlation<br />
between the two measures, an encouraging phenomenon likely driven by the presence <strong>of</strong> school<br />
fixed effects and the large number <strong>of</strong> student and school interactions. However, this relationship<br />
is not mechanical since the parameters are not estimated by inverting the fraction <strong>of</strong> students<br />
ranking a program first (Berry 1994). <strong>The</strong> single set <strong>of</strong> school fixed effects used to explain the<br />
rankings could have resulted in a poor fit.<br />
Another way to assess within-sample fit is to see what our estimates imply for the aggregate<br />
patterns by rank in Table 6. In Table 9, we use the estimates to compute the distribution <strong>of</strong><br />
preferences and aggregate them into cells corresponding to those reported in Table 6. Comparing<br />
the ranked program to the overall distribution <strong>of</strong> school characteristics, shown in the first column,<br />
the preference estimates capture many features <strong>of</strong> the distribution <strong>of</strong> preferences, though the<br />
estimated trade<strong>of</strong>fs in school attributes going down rank order lists is sometimes not as steep<br />
as what is observed. For instance, for first choices, school’s fraction high math achievement is<br />
14.9, while it drops to 13.8 for last choices. In the our sample, the corresponding range is 16.7<br />
to 10.4. Relative to the average attributes <strong>of</strong> schools, the model fit is much closer to the actual<br />
ranked distribution. <strong>The</strong> average distance to a high school in New York is 12.3 miles away from<br />
home, yet the top ranked school is 4.4 miles away. Moreover, the average percent white in New<br />
York’s schools is 10.8, but first choices are 19.1 white, while we predict them to be 17.4. Each<br />
predictions is closer to the actual properties <strong>of</strong> ranked schools than the aggregate distribution.<br />
<strong>The</strong> attributes <strong>of</strong> the schools ranked and what we predict using our demand estimates are<br />
relatively close, especially for choices from three to ten. <strong>The</strong>re are more significant differences<br />
between the prediction and actual choices for the top two choices. For instance, we predict<br />
that the average first choice is 5.1 miles away, while it is 4.4 miles away in the data. It is<br />
worth noting that a student’s closest school is 0.82 miles away, so our predictions are far better<br />
than a behavioral model where students simply rank their closest school first. <strong>The</strong> drop in<br />
predictive accuracy for the last three choices is probably driven by changes in the composition <strong>of</strong><br />
students who submit long rank order lists. Nonetheless, the model appears to capture the trend<br />
in preferences going down rank order lists.<br />
We next relate our estimated taste parameters to information that we have not used for<br />
estimating the distribution <strong>of</strong> preferences in an out-<strong>of</strong>-sample test. In Table 10, we regress<br />
indicators <strong>of</strong> exit and school matriculation, shown earlier, on our estimates <strong>of</strong> the observable<br />
components <strong>of</strong> student i’s utility for her assignment. This is an out-<strong>of</strong>-sample test because<br />
neither exit or matriculation decisions are used in estimation. Let y i indicate whether a student<br />
exits NYC public schools or does not exit, but enrolls in a school other than her assigned one.<br />
21
We fit equations like:<br />
y i = λû ia + γd ia + z i β + ɛ i ,<br />
where û ia is student i’s expected utility for program assignment a expressed in miles, d ia is the<br />
distance to assigned school, and z i represents the student demographic and baseline achievement<br />
measures in the demand model. Since distance is a key aspect <strong>of</strong> student preferences, we compare<br />
whether our imputed utility (which uses the submitted rankings) provides additional information<br />
beyond distance for the exit or matriculation decisions, conditional on student attributes.<br />
<strong>The</strong> estimates in column (1) show that if students obtain greater utility (measured in miles)<br />
for their assignment, they are less likely to exit in the coordinated mechanism. This pattern is<br />
reassuring since the demand estimates come from the main round <strong>of</strong> the coordinated mechanism,<br />
but we have not used information on the exit decision in estimation. Distance to school has no<br />
additional predictive power for the exit decision beyond the controls for student attributes in<br />
column (2). Matriculation is also negatively related to the utility from assignment in Panel B,<br />
while distance exhibits the opposite pattern. <strong>The</strong> magnitude <strong>of</strong> an increase distance is smaller<br />
than the magnitude <strong>of</strong> an decrease in estimated utility (in miles) on exit.<br />
An even more demanding comparison considers what our estimates imply for data from the<br />
uncoordinated mechanism. This comparison is more demanding because it uses data on preferences<br />
fitted from the coordinated mechanism to look at an out-<strong>of</strong>-sample decision in the uncoordinated<br />
mechanism for an entirely different student sample. It is reassuring that the relationship<br />
between utility, exit and matriculation is similar those reported using this distinct data from the<br />
uncoordinated mechanism. Just as with the coordinated mechanism, student exit is negatively<br />
related to a student’s utility from their assigned school, and the utility measure has a larger<br />
weight than distance. <strong>The</strong> magnitude <strong>of</strong> the estimate for predicting matriculation is similar to<br />
that in the coordinated mechanism. This out-<strong>of</strong>-sample comparison suggests that our preference<br />
estimates may be suitable for understanding the utility associated with the assignments in the<br />
uncoordinated mechanism.<br />
7 Comparing Mechanisms<br />
7.1 Measuring <strong>Welfare</strong><br />
<strong>The</strong> preference estimates in the last section allow us to compare the assignments produced by the<br />
uncoordinated and coordinated mechanisms, assuming that the preference estimates for students<br />
in the coordinated mechanism are similar to the underlying estimates in the uncoordinated<br />
mechanism. Since we do not observe the same student in both the uncoordinated and coordinated<br />
mechanism, it is necessary to make statements about groups <strong>of</strong> students rather than particular<br />
individuals.<br />
Consider a group <strong>of</strong> students with attribute G and a matching µ, which specifies the program<br />
for each student as µ(i), where if student i is unassigned µ(i) = i. Define the average welfare as<br />
a function <strong>of</strong> parameters θ as<br />
W µ G (θ) = 1 ∑<br />
u<br />
|G| iµ(i) (θ) ,<br />
i∈G<br />
22
which is the per-student average <strong>of</strong> the utilitarian welfare criterion for group G. For two different<br />
matchings, µ and µ ′ , corresponding set <strong>of</strong> students in group G(µ) and G(µ ′ ), the estimate <strong>of</strong> the<br />
welfare difference between the two matchings is given by:<br />
W µ µ′<br />
G<br />
(θ) − W<br />
G (θ) = 1<br />
|G(µ)|<br />
∑<br />
i∈G(µ)<br />
u iµ(i) (θ) −<br />
1<br />
|G(µ ′ )|<br />
∑<br />
i∈G(µ ′ )<br />
u iµ ′ (i) (θ) .<br />
Notice that the welfare difference depends only on the utility differences between the inside<br />
options at θ. <strong>The</strong>refore, when we compare the uncoordinated and coordinated mechanism, we<br />
normalize the utilities so that the mean utility for all students in the coordinated mechanism is<br />
zero.<br />
Furthermore, since no unassigned students are in the welfare samples, it is not necessary to<br />
impute a value to these students. As a result, our comparisons may understate the total welfare<br />
from the the new assignment system because it is only an intensive margin calculation and<br />
does not include the difference in exit rates that we described earlier. We are able to, however,<br />
assess the effects <strong>of</strong> the non-compliance by students who switched schools post-assignment using<br />
imputed utilities from the October enrollment in the subsequent school year. 13<br />
To operationalize our welfare comparisons, it is necessary to confront the fact that the precise<br />
allocation system used in the old mechanism cannot be easily simulated. To compute the welfare<br />
<strong>of</strong> the assignment, we need to calculate the utility conditional on a student being assigned to<br />
a given school program. This utility may differ from the utility given only observed student<br />
characteristics because <strong>of</strong> the unobserved taste term ɛ ij . A student’s submitted ranking reveals<br />
information about unobserved tastes. It is therefore necessary to model and account for this<br />
selection issue in our welfare calculations.<br />
In the coordinated mechanism, we observe the rankings submitted by students. Under the<br />
assumption that these reports are truthful, a program ranked kth yields the kth highest utility.<br />
For students in the coordinated mechanism who submit rank order list r i , we calculate the<br />
expected utility for each ranked choice and each unranked choice by simulating the utility from<br />
the estimated preference distribution, conditional on the relationships implied by the submitted<br />
ranks (see Appendix A for details on this calculation).<br />
<strong>The</strong> old mechanism’s uncoordinated nature makes the relationship between preferences and<br />
the final assignments less straightforward. Computing the expected utility conditional on the assignments<br />
requires strong assumptions about the old mechanism. Our approach exploits rankings<br />
from the uncoordinated mechanism’s first round. Such data is rarely available for in uncoordinated<br />
mechanisms, but are also difficult to interpret because <strong>of</strong> incentive properties <strong>of</strong> the system<br />
under which they were submitted. To avoid asymmetries in the welfare comparisons between<br />
mechanisms, we assume that the ranked school in the old mechanism are reported truthfully and<br />
all unranked schools are less preferred to ranked schools. This assumption is likely provides an<br />
optimistic case for the students that are assigned to a program to which they applied. 14 It is also<br />
safe to assume that students assigned in the supplementary or over-the-counter round do not<br />
13 <strong>The</strong> data tell us school a student is enrolled in, but not the program. If the assigned school is identical to the<br />
enrolled school, we assume that the enrolled program remains the same. For students switching to new schools,<br />
we use the program-size weighted mean as the utility measure.<br />
14 We also examined an alternative weaker assumption that schools ranked in the uncoordinated mechanism<br />
23
prefer those assignments to ones they applied to since they could have instead applied to those<br />
schools in the main round (and those schools were in fact available). Indeed, Table 5 makes clear<br />
that those schools have less desirable attributes.<br />
For both mechanisms, we compute the expected utility <strong>of</strong> a program ranked kth by student<br />
i as<br />
[<br />
E u ij<br />
| u (k+1)<br />
i<br />
]<br />
< u ij < u (k−1)<br />
i<br />
, r ik = j<br />
and the expected utility for all unranked schools as<br />
[<br />
]<br />
E u ij | u ij < u (|r i|)<br />
i<br />
, j ∉ ∪{r ik } , (2)<br />
where u (k)<br />
i<br />
is the kth highest value <strong>of</strong> {u ij } J j=1 and |r i| is the number <strong>of</strong> ranks submitted by<br />
student i. 15 Using these values, the welfare difference for group G from assignments µ and µ ′ is<br />
W µ G − W µ′<br />
G = 1<br />
|G(µ)|<br />
∑<br />
i∈G(µ)<br />
E[u iµ(i) |r i ] −<br />
1<br />
|G(µ ′ )|<br />
∑<br />
i∈G(µ ′ )<br />
E[u iµ ′ (i)|r i ],<br />
where E[u ij |r i ] denotes the conditional expectations in equations (1) and (2) above.<br />
7.2 <strong>Welfare</strong> across Mechanisms<br />
Most students are better <strong>of</strong>f under the coordinated mechanism. Figure 4 plots the overall distribution<br />
<strong>of</strong> student welfare from the two mechanisms. On average, students benefit by a distanceequivalent<br />
utility <strong>of</strong> 4.79 miles from the new mechanism. <strong>The</strong>re is a bimodal pattern <strong>of</strong> utility<br />
in the uncoordinated mechanism due to the students who are assigned in the over-the-counter<br />
round. Most <strong>of</strong> the mass in the first mode shifts rightward in the coordinated mechanism, given<br />
the sharp reduction in the number <strong>of</strong> over-the-counter students.<br />
Table 11 reports on welfare differences for the same student groups that we reported on <strong>of</strong>fer<br />
processing. For each student group, there is a positive gain from the new mechanism. Across<br />
racial groups, the gains are similar, with whites and Hispanics improving slightly more than<br />
blacks and Hispanics. <strong>The</strong>re are sharper differences across boroughs. Students from Staten<br />
Island, Queens, and the Bronx gain the most, while the smallest gain <strong>of</strong> any student group in<br />
the table comes from Manhattan residents. Low baseline math students gain more than high<br />
baseline math students or SHSAT test takers.<br />
One aspect <strong>of</strong> evaluating an assignment system is understanding whether initially bad allocation<br />
systems are undone post-assignment through aftermarket retrading. This view, sometimes<br />
associated with Coase (1960), implies that greater flexibility to switch assignments in the uncoordinated<br />
mechanism could undo the deleterious effects <strong>of</strong> initial assignments. To examine<br />
this possibility, we also compute the utility associated with the schools students enroll at in<br />
October <strong>of</strong> the following school year. Compared to assignments, the gains from the coordinated<br />
are better than those that are not ranked, but we did not utilize information on the ordering within ranked<br />
alternatives. Consistent with evidence on the importance <strong>of</strong> what is ranked vs. unranked, the welfare estimates<br />
under this assumption are similar to those we report.<br />
15 We do not observe applications for some students, and some choices are not available in the main round. In<br />
these cases, we impute the mean utility conditional on observables alone in the welfare calculations.<br />
(1)<br />
24
mechanism measured by enrollment are somewhat smaller, but are still large. For instance, the<br />
average student benefits by a distance-equivalent utility <strong>of</strong> 3.99 miles, which is 83% <strong>of</strong> the gain<br />
from assignments. <strong>The</strong> change in travel distance to enrolled school is also lower than the change<br />
in travel distance to assignment. About 0.3 <strong>of</strong> the 0.8 miles difference is due to this reduction<br />
in travel distance. Though a smaller gain from enrollment suggests that some <strong>of</strong> the old mechanism’s<br />
mismatch was undone in its aftermarket, these facts weigh against the argument that<br />
post-market reallocation will undo a large fraction <strong>of</strong> misallocation.<br />
Comparing the change in utility to the change in distance, it is possible to measure how<br />
much more match-specific utility students obtain from the coordinated mechanism. On average,<br />
the newly assigned school is equal to 5.53 miles <strong>of</strong> distance-equivalent utility. This implies that<br />
the improved school match is worth about eight times the costs associated with increased travel<br />
distance. <strong>The</strong> lowest gains are for Manhattan residents, a group which experiences no increase<br />
in travel distance. However, the welfare gains are not solely driven by changes in distance.<br />
For instance, across boroughs, Staten Island pupils only travel 0.34 miles further, but they<br />
experience the largest improvement <strong>of</strong> any borough at 6.32 miles. This suggests that mismatch<br />
was particularly severe in Staten Island, and is consistent with the substantially larger fraction<br />
<strong>of</strong> Staten Island residents who are assigned in the over-the-counter round in the uncoordinated<br />
mechanism. <strong>The</strong> important role <strong>of</strong> match-specific utility suggests that even if there are not<br />
enough good schools to assign everyone their top choice, preference heterogeneity generates an<br />
important role for matching mechanisms.<br />
<strong>The</strong> new mechanism has improved measures <strong>of</strong> equity <strong>of</strong> assignment. For instance, welfare<br />
gains are larger for low baseline math students than high baseline math students. <strong>The</strong>y are<br />
also higher for special education and limited English pr<strong>of</strong>icient students than SHSAT test-takers.<br />
<strong>The</strong>se differences within student groups closely track the rounds in which students were processed<br />
in the uncoordinated mechanism. For instance, substantially more high baseline students were<br />
assigned in the main round <strong>of</strong> the old mechanism compared to low baseline students. In many<br />
instances, student groups with higher fractions assigned in the over-the-counter round, shown in<br />
column (7), experience the largest gains. This may explain why there is relatively little difference<br />
between students who are either from rich or poor neighborhoods. Wealthier students may have<br />
been more effective at navigating the old mechanism. However, Staten Island residents, many<br />
who are white, experience the largest welfare gains, and Staten Island’s neighborhood income<br />
levels are higher on average than other parts <strong>of</strong> the city.<br />
We’ve already seen evidence for mismatch among those assigned over-the-counter in Table 5.<br />
Another way to examine the important role <strong>of</strong> over-the-counter assignments by further disaggregating<br />
students. We estimate a probit <strong>of</strong> whether a student is assigned over-the-counter on all<br />
student characteristics in the demand model. We also include census tract dummies to account<br />
for neighborhoods that may or may not have local high schools serving as student fallback. Given<br />
the large number <strong>of</strong> dummies, we are able to forecast whether a student is locally assigned with<br />
a high degree <strong>of</strong> accuracy. We then use this estimated relationship to compute each student’s<br />
propensity to be assigned over-the-counter in both the uncoordinated and coordinated mechanism.<br />
For each decile <strong>of</strong> this propensity, we compare the utility from assignment across both<br />
mechanisms. Figure 5 shows that distance to assignment is relatively unchanged across these ten<br />
25
student deciles. However, there is a clear monotonic pattern between over-the-counter propensity<br />
and the welfare improvement, whether comparing utility differences including distance or net <strong>of</strong><br />
distance.<br />
<strong>The</strong> facts clearly illustrate that the reduction in the number <strong>of</strong> over-the-counter round participants<br />
drive the welfare gains in the coordinated mechanism. But why were so many students<br />
assigned in that round in the old mechanism? <strong>The</strong>re are at least two features <strong>of</strong> the old mechanism<br />
that could generate this phenomenon. First, students are not able to rank more than five<br />
choices, so large numbers <strong>of</strong> unassigned indicate that students were not successful at calibrating<br />
their rankings to ensure acceptance at one <strong>of</strong> their five choices. In the coordinated mechanism,<br />
12.6% <strong>of</strong> students were assigned to choices 6-12 (shown in Table A2). Second, there are 17,263<br />
“extra” <strong>of</strong>fers in the first round, but only about 10,000 students finalized in the second and third<br />
rounds. This means that many <strong>of</strong> these extra <strong>of</strong>fers did not percolate through to other students.<br />
Both multiple <strong>of</strong>fers and short rank order lists in the uncoordinated mechanism interact<br />
to generate over-the-counter placements.<br />
8 Evaluating Mechanism Design Features<br />
Many <strong>of</strong> the new mechanism’s features were intended to address issues created by the old mechanism.<br />
Among the difficulties <strong>of</strong> the old mechanism its few rounds <strong>of</strong> <strong>of</strong>fer processing, multiple<br />
<strong>of</strong>fers, and poor student and school incentives. A single-<strong>of</strong>fer mechanism based on the studentproposing<br />
deferred acceptance algorithm coordinates <strong>of</strong>fers and simplifies the student ranking<br />
problem. Is the allocation produced by the new mechanism the best possible one or would alternatives<br />
have further improved overall student welfare? <strong>The</strong> alternatives we consider hold here<br />
fixed the set <strong>of</strong> schools and students, but change the allocation mechanism.<br />
<strong>The</strong> cardinalization <strong>of</strong> utility implied by our demand model allows us to compute student<br />
welfare maximum, by finding the assignment that maximizes the sum <strong>of</strong> student utility subject<br />
the feasibility constraints <strong>of</strong> the assignment. This program corresponds to the utilitarian optimal<br />
assignment ignoring school priorities and can be computed following Shapley and Shubik<br />
(1971). 16 We first draw complete student preference lists randomly conditional on submitted<br />
ranks, taking 1,000 draws from the distribution conditional on the observed ranks. We then<br />
compute the utilitarian optimal assignment for each draw.<br />
With this aggregate welfare maximizing level <strong>of</strong> utility as our benchmark, the first question<br />
we ask is how does the allocation produced by the current mechanism compares to this maximum.<br />
We compute assignments from the student-proposing deferred acceptance algorithm using<br />
stated preferences and a single tie-breaking rule 1,000 times. To avoid imputing a welfare to<br />
unassigned, if a student in the demand sample is left unassigned, we use their preferences to<br />
mimic NYC’s supplementary round, by assigning them under a serial dictatorship according to<br />
the random ordering <strong>of</strong> students. Student preferences for this round use the simulated ranks<br />
from the estimated model. Following the computation for the optimal assignment, the welfare<br />
calculation includes the effect <strong>of</strong> unobservable taste components ɛ ij conditional on their student<br />
16 <strong>The</strong> utilitarian allocation implies equal weights on students; in quasi-linear problems, such an allocation would<br />
be synonymous with the Pareto efficient allocation, but need not be here.<br />
26
ank (as described in Appendix A).<br />
<strong>The</strong> first column <strong>of</strong> Table 12 shows that the allocation produced by the new mechanism is 2.85<br />
miles from the maximum achievable only respecting feasibility. Relative to the difference between<br />
the uncoordinated and coordinated mechanism <strong>of</strong> 4.79 miles, this contrast implies that further<br />
improvements in the matching mechanism could at best be equal to 59% <strong>of</strong> gain from coordinating<br />
assignment. <strong>The</strong> utilitarian optimal assignment is an idealized benchmark, but is unlikely to<br />
be achieved given that it requires a cardinal mechanism and completely ignores the schools’<br />
priorities. <strong>The</strong>re are 208 screened programs in New York City in 2003-04, so implementing<br />
this allocation would involve overriding the preferences <strong>of</strong> these programs. It is also possible<br />
that students would express different rank orderings under a mechanism that implements the<br />
utilitarian outcome. <strong>The</strong>refore, we consider another alternative that does not completely abandon<br />
school priorities.<br />
<strong>The</strong> student-proposing deferred acceptance (DA) algorithm produces a stable matching,<br />
which cannot be blocked by a student and school. <strong>The</strong> presence <strong>of</strong> ties in school’s ranking<br />
<strong>of</strong> students, however, means that DA does not produce a student-optimal stable matching, even<br />
though it would if all school preferences are strict. However, a number <strong>of</strong> authors have pointed<br />
out that deferred acceptance cannot be improved upon without sacrificing strategy-pro<strong>of</strong>ness for<br />
students (Erdil and Ergin 2008, Abdulkadiroğlu, Pathak, and Roth 2009, Kesten 2010, Kesten<br />
and Kurino 2012).<br />
We quantify the cost <strong>of</strong> providing straightforward incentives for students by computing a<br />
student-optimal stable assignment that improves the DA assignment by assigning students higher<br />
in their choice lists, while respecting stability for strict school priorities. Such an assignment can<br />
be computed by the stable improvement cycles (SIC) algorithm developed by Erdil and Ergin<br />
(2008), which iteratively finds Pareto improving swaps for students, while still respecting the<br />
stability requirement for underlying weak priority ordering <strong>of</strong> schools. A total <strong>of</strong> 2,348 students<br />
in the demand sample can obtain a better assignment in a student-optimal stable matching, and<br />
this corresponds to an average difference <strong>of</strong> 2.74 miles <strong>of</strong> equivalent utility with the utilitarian<br />
optimal assignment. <strong>The</strong> benefit <strong>of</strong> a student-optimal stable matching corresponds to a difference<br />
<strong>of</strong> 0.11 miles relative to the current student-proposing deferred acceptance algorithm. <strong>The</strong> cost<br />
<strong>of</strong> this change is that the underlying mechanism is not based on a strategy-pro<strong>of</strong> algorithm. 17<br />
Another way to relax the constraints imposed in the algorithm is to abandon stability all<br />
together. While no stable mechanism eliminates strategic maneuvers by schools (Sönmez 1997),<br />
this may not be an issue in markets with many participants (Kojima and Pathak 2009). Stability<br />
may therefore be seen as an incentive constraint on the assignment mechanism to deal with<br />
strategic schools. 18 If a mechanism does not produce a stable outcome, it is possible that schools<br />
benefit by <strong>of</strong>fering students seats outside <strong>of</strong> the assignment process. Despite the SIC-outcome<br />
being student-optimal stable, it is not Pareto efficient for students. We quantify the cost <strong>of</strong><br />
providing school incentives by computing a Pareto efficient assignment that improves upon the<br />
student-optimal stable assignment by assigning students higher in their choice lists. A Pareto<br />
17 Azevedo and Leshno (2011) provide an example where the equilibrium assignment <strong>of</strong> the stable improvement<br />
cycles mechanism is Pareto inferior to the assignment from deferred acceptance when students are strategic.<br />
18 Balinski and Sönmez (1999) and Abdulkadiroğlu and Sönmez (2003) provide an alternative equity rationale<br />
for stability.<br />
27
efficient assignment can be found by transferring students from their assigned schools to their<br />
higher ranked choices via the Gale’s Top Trading Cycles algorithm (Shapley and Scarf 1974).<br />
Consequently, we calculate a Pareto efficient matching which dominates each student-optimal<br />
stable matching we simulate and estimate students’ utilities from this Pareto efficient matching<br />
in column (3) <strong>of</strong> Table 12. A total <strong>of</strong> 10,882 students obtain a more preferred assignment at a<br />
Pareto efficient matching, which is 2.24 miles away for each student from the utilitarian optimal<br />
assignment. Relative to the current mechanism, the cost <strong>of</strong> limiting the scope for strategizing<br />
by schools (by imposing stability) is 0.50 miles per student. Both the “cost <strong>of</strong> strategy-pro<strong>of</strong>ness<br />
for students” and the “cost <strong>of</strong> stability” are smaller than the “benefit <strong>of</strong> coordination” in the new<br />
mechanism.<br />
<strong>The</strong> finding that changes to the matching algorithm are smaller than changes associated with<br />
coordinating assignment informs a debate in a related auction market design context. Klemperer<br />
(2002) argued that “most <strong>of</strong> the extensive auction literature is <strong>of</strong> second-order importance for<br />
practical auction design,” and that “good auction design is mostly good elementary economics.”<br />
Although our conclusion is not as pessimistic for matching market design, our exercises shows<br />
that even if it were possible to implement a student-optimal stable matching or Pareto efficient<br />
matching, coordinating admissions produces larger gains.<br />
9 Conclusion<br />
Changes in NYC’s high school assignment system provide a unique opportunity to study the<br />
effects <strong>of</strong> centralizing and coordinating school admissions with detailed data on preferences,<br />
assignments, and enrollment from both mechanisms. <strong>The</strong> new coordinated mechanism is an<br />
improvement relative to the old uncoordinated mechanism on a variety dimensions. Relative to<br />
the uncoordinated mechanism, there is 7.2 percentage point increase in matriculation at assigned<br />
school. Multiple <strong>of</strong>fers and short rank order lists in the uncoordinated mechanism advantage few<br />
students and leave many without first round <strong>of</strong>fers. A large fraction <strong>of</strong> these unassigned students<br />
are placed in the over-the-counter round to schools with considerably worse characteristics than<br />
what they ranked. <strong>The</strong> increase in matriculation is driven by the three times reduction in the<br />
number <strong>of</strong> students assigned in the over-the-counter process in the coordinated mechanism.<br />
Our demand estimates show that Asians, whites, high math baseline and students from high<br />
income neighborhoods are willing to travel further for good schools. While the coordinated<br />
mechanism assigns students 0.69 miles further to their assignments, it’s easier for students to<br />
obtain the schools they want. <strong>The</strong> benefit <strong>of</strong> being assigned through the new mechanism is equal<br />
eight times the cost <strong>of</strong> additional travel according to our preferred estimates. <strong>The</strong> gains are<br />
positive on average for students from all boroughs, demographic groups, and baseline achievement<br />
categories. <strong>Welfare</strong> improvements are also seen whether utility is measured based on assignments<br />
made at the end <strong>of</strong> the high school match or based on subsequent enrollment, suggesting that<br />
post-match re-allocation does little to undo the market’s design.<br />
<strong>The</strong> new mechanism increased the equity <strong>of</strong> access to school choices, since students who were<br />
relatively more effective at navigating the uncoordinated mechanism gained less than students<br />
who were more likely to be processed in its over-the-counter round. Though gains are widespread,<br />
28
low math baseline students and those from Queens and Staten Island benefit more than SHSAT<br />
test-takers and students from Manhattan. Since a shortage <strong>of</strong> good schools implies that some will<br />
inevitably be assigned to less desirable schools, preference heterogeneity generates a significant<br />
role for a good assignment mechanism to generate allocative efficiencies.<br />
Our study <strong>of</strong> the new mechanism utilizes rankings submitted in its first year. Though we have<br />
examined several variations to the behavioral assumptions underlying our estimation strategy,<br />
it is possible that we’ve underestimated the long run performance <strong>of</strong> the new mechanism if<br />
participants alter their behavior as they learn more about how to navigate the process. After the<br />
first year, communication and outreach efforts increased via expanded school open houses and<br />
citywide high school fairs. <strong>School</strong>s have also gained more experienced with the new process. For<br />
instance, the DOE has reduced the number <strong>of</strong> students assigned over-the-counter by providing<br />
guidance on yield management, calibrating <strong>of</strong>fers based on expectations <strong>of</strong> attrition.<br />
<strong>The</strong> increase in student welfare from the new mechanism illustrates that there are considerable<br />
frictions to exercising choice in poorly designed assignment systems. <strong>The</strong> 2003 NYC change<br />
took place in an environment where participants already had some familiarity with choice since<br />
both the uncoordinated and coordinated system had a common application. <strong>The</strong> most important<br />
differences between the two mechanisms involved eliminating multiple <strong>of</strong>fers and allowing<br />
participants to submit longer rank order lists. We found that these features left many assigned<br />
in an over-the-counter process at schools with significantly worse attributes than where they<br />
would prefer to be placed. <strong>The</strong>se effects are relevant for settings where existing choice processes<br />
have been coordinated and streamlined, such as in London and Denver, two cities which recently<br />
adopted new mechanisms based on deferred acceptance.<br />
In other cities, the welfare gains may be even larger since school assignment processes are even<br />
more uncoordinated and decentralized than NYC’s old HS system. For instance, Boston’s charters<br />
only recently established a unified application timeline, but they still retain school-by-school<br />
admissions and <strong>of</strong>fers, and almost no coordination <strong>of</strong> waitlists and enrollment. With the exception<br />
<strong>of</strong> Denver and New Orleans, this situation is the norm across district and charter schools.<br />
However, discussions about unifying admissions and coordinating enrollment are currently under<br />
consideration in cities including Boston, Chicago, Newark, Philadelphia, and Washington DC<br />
(Darville 2013, Vaznis 2013). <strong>The</strong> NYC evidence suggests that students stand much to gain<br />
from making it easier to apply. However, the relative value <strong>of</strong> policies such as common timelines,<br />
a common application, single vs. multiple <strong>of</strong>fers, sophisticated matching algorithms, and good<br />
information and decision aids remains an open question.<br />
Finally, it’s worth emphasizing that our analysis has focused on the allocative aspects <strong>of</strong><br />
school choice and different school assignment procedures. An important question is whether<br />
allocative changes contribute to changes in the productive dimensions <strong>of</strong> assignment. That is,<br />
do different student assignment protocols affect levels <strong>of</strong> student achievement post-assignment?<br />
This far more difficult question requires understanding the link between choices and the education<br />
production function, but we hope to examine it further in future work.<br />
29
Table 1. Characteristics <strong>of</strong> Student Sample<br />
Mechanism Comparison<br />
Demand Estimation<br />
Uncoordinated <strong>Coordinated</strong> <strong>Coordinated</strong><br />
Mechanism Mechanism Mechanism<br />
(1) (2) (3)<br />
Number <strong>of</strong> Students 70,358 66,921 69,907<br />
Female 49.4% 49.0% 49.0%<br />
Bronx 23.7% 23.3% 23.7%<br />
Brooklyn 31.9% 34.1% 33.3%<br />
Manhattan 12.5% 11.8% 12.0%<br />
Queens 25.0% 24.8% 24.7%<br />
Staten Island 6.9% 6.0% 6.3%<br />
Asian 10.6% 10.9% 10.6%<br />
Black 35.4% 35.7% 35.7%<br />
Hispanic 38.9% 40.4% 40.3%<br />
White 14.7% 12.6% 13.0%<br />
Other 0.4% 0.4% 0.4%<br />
Subsidized Lunch* 54.0% 63.1% 63.7%<br />
Neighborhood Income 38,360 37,855 37,920<br />
Limited English Pr<strong>of</strong>icient 13.1% 12.6% 12.6%<br />
Special Education 8.2% 7.9% 7.5%<br />
SHSAT Test-‐Taker 22.4% 24.3% 23.9%<br />
Notes: Uncoordinated mechanism refers to 2002-‐03 mechanism and coordinated mechanism refers to the 2003-‐04 mechanism based on <br />
deferred acceptance. Subsidized lunch not available pre-‐assignment and comes from enrolled students as <strong>of</strong> 2004-‐05 school year. <br />
Neighborhood income is mean census block group family income from the 2000 census. SHSAT stands for Specialized High <strong>School</strong> Achievement <br />
Test.
Table 2. Descriptive Statistics for <strong>School</strong>s and Programs<br />
Uncoordinated Mechanism<br />
<strong>Coordinated</strong> Mechanism<br />
(1) (2) (3) (4)<br />
A. <strong>School</strong>s<br />
Number 215 235<br />
High English Achievement 19.1 (17.3) 19.3 (17.4)<br />
High Math Achievement 10.2 (13.5) 10.0 (13.4)<br />
Percent Attending Four Year College 47.8 (25.2) 47.2 (25.5)<br />
Fraction Inexperienced Teachers 54.7 (26.8) 55.6 (27.4)<br />
Percent Subsidized Lunch 62.5 (23.0) 62.6 (22.9)<br />
Attendance Rate (out <strong>of</strong> 100) 85.5 (6.0) 85.7 (6.0)<br />
Percent Asian 8.7 (11.4) 8.6 (11.2)<br />
Percent Black 38.5 (24.2) 38.4 (24.1)<br />
Percent Hispanic 41.9 (22.7) 42.1 (22.8)<br />
Percent White 10.9 (16.5) 10.9 (16.4)<br />
B. Programs<br />
Number 612 558<br />
Screened 233 208<br />
Unscreened 63 119<br />
Education Option 316 119<br />
Spanish Language 27 24<br />
Asian Language 10 9<br />
Other Language 6 7<br />
Arts 80 80<br />
Humanities 89 93<br />
Math and Science 53 60<br />
Vocational 55 59<br />
Other Specialties 163 162<br />
Notes: Panel A reports means in column (1) and standard deviations in column (2), unless otherwise noted. Panel B reports counts. Uncoordinated mechanism refers to 2002-‐<br />
03 mechanism and coordinated mechanism refers to the 2003-‐04 mechanism based on deferred acceptance. Data appendix presents information on the availability <strong>of</strong> school <br />
characteristics. High Math and High English achievement is the fraction <strong>of</strong> student that scored more than 85% on the Math A and English Regents tests in New York State <br />
Report Cards, respectively. Inexperienced teachers are those that have taught for less than two years.
Table 3. Offer Processing across Mechanisms<br />
Distance to <strong>School</strong> (in miles) Exit from NYC Public In NYC Public, but at <br />
Number <strong>of</strong> Students <strong>Assignment</strong> Enrollment <strong>School</strong>s <strong>School</strong> Other than <br />
(1) (2) (3) (4) (5)<br />
A. Uncoordinated Mechanism -‐ By Final <strong>Assignment</strong> Round<br />
Overall 70,358 3.36 3.50 8.5% 18.6%<br />
First Round 24,292 4.22 4.10 5.2% 9.5%<br />
Second Round 5,861 4.50 4.40 4.8% 11.3%<br />
Third Round 4,461 4.35 4.25 4.9% 14.1%<br />
Supplementary Round 10,170 4.61 4.37 7.8% 25.4%<br />
Over-‐the-‐Counter Round 25,574 1.61 2.09 13.5% 27.4%<br />
B. Uncoordinated Mechanism -‐ By Number <strong>of</strong> First Round Offers<br />
No Offers 36,464 2.80 3.12 10.4% 24.4%<br />
One Offer 21,328 3.89 3.85 7.1% 13.8%<br />
Many Offers 12,566 4.07 4.03 5.7% 9.8%<br />
C. <strong>Coordinated</strong> Mechanism -‐ By Final <strong>Assignment</strong> Round<br />
Overall 66,921 4.05 3.91 6.4% 11.4%<br />
Main Round 54,577 4.02 3.86 6.1% 9.9%<br />
Supplementary Round 5,201 5.10 4.90 4.8% 10.4%<br />
Over-‐the-‐Counter Round 7,143 3.50 3.52 9.6% 23.6%<br />
Notes: Columns (2) -‐ (5) report means. Uncoordinated mechanism refers to 2002-‐03 mechanism and coordinated mechanism refers to the 2003-‐04 mechanism based on deferred <br />
acceptance. Student distance calculated as road distance using ArcGIS. <strong>Assignment</strong> is the school assigned at the conclusion <strong>of</strong> the high school assignment process. Enrollment is <br />
the school a student enrolls in October following application. Assigned student exits New York City if they are not enrolled in any NYC public high school in October following <br />
application. Student in NYC Public, but at school other than assigned if enrolled in NYC public high school other than that assigned at end <strong>of</strong> match. Final assignment round is the <br />
round during which an <strong>of</strong>fer to the final assigned school first made.
Table 4. Offer Processing by Student Characteristics<br />
<strong>Coordinated</strong> Mechanism<br />
Final <strong>Assignment</strong> Round<br />
Number <strong>of</strong> First Round Offers<br />
All First Second Third<br />
Supplementar<br />
y<br />
Over the<br />
Counter None One Many<br />
Main<br />
Supplementa<br />
ry<br />
Over the<br />
Counter<br />
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)<br />
Students 70,358 24,292 5,861 4,461 10,170 25,574 36,464 21,328 12,566 54,577 5,201 7,143<br />
Female 49.4 52.1 52.4 51.0 48.9 46.1 47.1 51.4 52.7 49.4 48.8 46.8<br />
Bronx 23.7 24.1 26.0 34.7 33.2 17.1 23.6 24.6 22.6 23.3 20.0 25.3<br />
Brooklyn 31.9 32.6 34.4 30.6 35.9 29.4 29.8 33.8 34.9 34.7 35.0 29.0<br />
Manhattan 12.5 19.3 14.2 13.7 16.6 3.7 8.5 14.7 19.9 11.4 11.2 15.2<br />
Queens 25.0 21.7 24.2 20.3 14.4 33.4 27.0 23.8 21.3 24.1 31.9 25.2<br />
Staten Island 6.9 2.2 1.3 0.8 0.0 16.4 11.1 3.1 1.2 6.5 1.8 5.2<br />
Asian 10.6 10.3 9.7 9.2 4.0 14.0 11.0 10.0 10.6 11.0 10.3 10.5<br />
Black 35.4 38.6 40.0 37.7 45.2 27.1 32.1 38.6 39.6 35.6 40.1 33.5<br />
Hispanic 38.9 40.9 39.4 43.5 46.5 33.0 37.8 39.4 41.2 40.5 41.0 39.1<br />
White 14.7 9.8 10.5 9.3 3.8 25.5 18.6 11.7 8.3 12.6 8.2 16.0<br />
High Baseline Math 24.0 29.4 23.6 30.4 12.3 22.5 19.3 26.9 32.7 24.5 15.5 21.2<br />
Low Baseline Math 22.5 21.1 25.0 19.9 30.8 20.5 23.6 22.4 19.5 20.6 19.6 25.4<br />
Subsidized Lunch* 54.0 57.8 57.9 59.5 57.6 47.0 51.7 55.7 57.5 63.4 59.7 63.4<br />
Bottom Neighborhood<br />
Income Quartile<br />
24.1 27.6 27.5 27.9 38.8 13.4 21.6 26.0 27.9 25.1 23.2 25.7<br />
Top Neighborhood Income<br />
Quartile<br />
25.8 21.9 23.5 21.7 14.4 35.3 29.0 23.7 20.1 24.6 23.8 27.3<br />
Limited English Pr<strong>of</strong>icient 13.1 12.4 13.2 12.2 14.8 13.2 14.1 11.8 12.4 12.7 12.3 12.7<br />
Special Education 8.2 6.3 7.8 6.2 10.6 9.5 9.8 6.6 6.2 6.9 0.0 20.8<br />
SHSAT Test‐taker 22.4 27.8 26.6 34.3 15.9 16.8 17.5 26.2 30.0 24.6 22.9 22.8<br />
Notes: Uncoordinated mechanism refers to 2002‐03 mechanism and coordinated mechanism refers to the 2003‐04 mechanism based on deferred acceptance. Final assignment round is the round during which an <strong>of</strong>fer to the final assigned school was accepted.<br />
Subsidized lunch not available pre‐assignment and comes from enrolled students as <strong>of</strong> 2004‐05 school year. Neighborhood income is median family income from the 2000 census.
Table 5. <strong>School</strong> Access across Rounds <strong>of</strong> Offer Processing<br />
Main Round<br />
Supplementary Round<br />
Over-‐the-‐Counter Round<br />
Uncoordinated <strong>Coordinated</strong> <br />
Uncoordinated <br />
Uncoordinated <strong>Coordinated</strong> <br />
Mechanism Mechanism<br />
Mechanism <strong>Coordinated</strong> Mechanism<br />
Mechanism Mechanism<br />
Ranked <br />
Ranked <br />
Ranked <br />
Ranked <br />
Ranked <br />
Ranked <br />
<strong>School</strong>s Assigned <strong>School</strong>s Assigned <strong>School</strong>s Assigned <strong>School</strong>s Assigned <strong>School</strong>s Assigned <strong>School</strong>s Assigned<br />
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)<br />
Distance (in miles) 4.81 4.29 5.10 4.00 4.87 4.59 5.87 5.17 5.12 1.60 5.33 3.43<br />
High English Achievement 21.0 20.3 22.1 19.1 19.9 15.8 26.5 20.0 24.2 17.4 24.2 19.2<br />
High Math Achievement 12.4 11.6 13.0 10.7 11.8 9.3 16.6 14.2 14.8 10.5 14.3 10.7<br />
Percent Attending Four Year College 49.1 47.1 50.6 48.3 48.6 44.9 54.1 50.1 51.9 46.7 51.7 47.9<br />
Percent Inexperienced Teachers 45.3 45.7 46.6 43.8 46.0 41.5 45.3 36.9 41.8 39.3 47.8 42.1<br />
Percent Subsidized Lunch 59.9 60.5 57.6 56.7 62.0 61.8 53.5 51.0 53.8 50.3 57.2 53.1<br />
Attendance Rate (out <strong>of</strong> 100) 85.1 84.6 85.7 83.8 85.1 82.2 87.4 83.2 85.8 80.7 86.7 82.9<br />
Notes: Means unless otherwise noted. Analysis restricts the sample to students from the welfare sample with observed assignments. Uncoordinated mechanism refers to the 2002-‐03 mechanism <br />
and coordinated mechanism refers to the 2003-‐04 mechanism based on deferred acceptance. Main round in the uncoordinated mechanism corresponds to the first round. Rankings used are <br />
those submitted in the main round <strong>of</strong> the process. Student distance calculated as road distance using ArcGIS. High English and High Math achievement are the fractions <strong>of</strong> students scoring over <br />
85% on the English and Math A regents in New York State Report Cards, respectively. Inexperienced teachers are those who have been teaching less than 2 years.
Table 6. <strong>School</strong> Characteristics by Rank <strong>of</strong> Student Choice in <strong>Coordinated</strong> Mechanism<br />
Choice 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th<br />
A. All Students<br />
Students Ranking Choice 69,907 93% 89% 83% 76% 69% 62% 56% 50% 43% 34% 20%<br />
Distance in Miles ‐ Mean 4.43 4.81 5.05 5.21 5.38 5.49 5.59 5.63 5.65 5.58 5.43 5.12<br />
Median 3.51 3.95 4.20 4.37 4.57 4.63 4.72 4.77 4.78 4.71 4.59 4.24<br />
High English Achievement 27.8 25.6 24.3 23.2 22.2 21.4 20.8 20.5 19.9 19.4 19.1 18.6<br />
High Math Achievement 16.7 15.3 14.7 13.9 13.4 12.8 12.4 11.9 11.5 11.1 10.8 10.4<br />
Percent Attending Four Year College 53.7 54.1 52.8 51.7 50.9 50.1 49.3 49.3 49.0 48.3 47.6 47.5<br />
Fraction Inexperienced Teachers 43.0 43.8 45.0 46.2 47.1 48.1 48.7 49.5 49.5 50.2 49.9 49.9<br />
Fraction Subsidized Lunch 51.4 53.4 54.5 56.2 57.4 58.7 59.8 60.6 61.3 61.9 62.7 63.1<br />
Attendance Rate (out <strong>of</strong> 100) 87.2 86.7 86.4 86.1 85.9 85.7 85.5 85.5 85.2 85.0 84.8 84.5<br />
Percent Asian 13.8 13.7 13.3 12.7 12.2 11.5 11.0 10.6 10.1 9.7 9.5 9.4<br />
Percent Black 32.6 34.3 35.1 36.0 36.7 37.5 38.2 38.4 38.7 38.9 38.9 38.1<br />
Percent Hispanic 34.4 35.2 35.9 37.0 37.9 38.9 39.6 40.2 40.8 41.5 42.2 43.5<br />
Percent White 19.1 16.7 15.7 14.4 13.3 12.2 11.2 10.8 10.4 9.8 9.4 9.0<br />
B. Students with Low Baseline Math Scores<br />
Distance in Miles ‐ Mean 4.1 4.4 4.7 4.9 5.0 5.1 5.2 5.2 5.2 5.1 5.0 4.8<br />
High English Achievement 20.1 19.4 18.7 18.4 18.1 17.9 17.6 17.7 17.4 17.1 16.6 16.6<br />
High Math Achievement 10.9 10.9 10.5 10.1 10.0 9.7 9.6 9.4 9.4 9.1 8.8 8.8<br />
C. Students with High Baseline Math Scores<br />
Distance in Miles ‐ Mean 4.6 5.0 5.2 5.4 5.7 5.9 5.9 6.0 6.1 6.2 5.8 5.3<br />
High English Achievement 39.4 34.0 32.0 29.8 28.0 26.4 25.2 24.6 23.7 22.8 22.5 21.5<br />
High Math Achievement 26.0 21.4 20.5 19.1 18.2 17.3 16.4 15.6 15.2 14.2 13.9 12.8<br />
D. Students from Bottom Neighborhood Income Quartile<br />
Distance in Miles ‐ Mean 4.3 4.5 4.7 4.8 5.0 5.0 5.1 5.1 5.2 5.1 4.9 4.7<br />
High English Achievement 20.5 19.8 19.1 18.8 18.4 17.9 18.0 18.0 17.5 17.1 17.1 16.6<br />
High Math Achievement 11.4 10.9 10.5 10.4 10.1 9.9 10.0 9.9 9.6 9.3 9.1 8.7<br />
E. Students from Top Neighborhood Income Quartile<br />
Distance in Miles ‐ Mean 4.6 5.2 5.5 5.7 5.9 6.1 6.3 6.4 6.6 6.6 6.4 5.9<br />
High English Achievement 35.6 32.0 29.9 28.4 26.5 25.6 24.6 23.9 23.1 22.4 21.6 21.0<br />
High Math Achievement 23.3 20.7 19.6 18.7 17.7 16.8 16.0 15.1 15.0 14.1 13.2 12.7<br />
Notes: <strong>Coordinated</strong> mechanism refers to the 2003‐04 mechanism based on deferred acceptance. Data appendix provides details on availability on school characteristics. Student distance calculated as road<br />
distance using ArcGIS. High English and High Math achievement are the fractions <strong>of</strong> students scoring over 85% on the English and Math A regents in New York State Report Cards, respectively. Inexperienced<br />
teachers are those who have been teaching less than 2 years. High baseline math students score above the 75th percentile for 7th grade middle school math, low baseline math students score below the 25th<br />
percentile. Neighborhood income is median family income from the 2000 census.
Table 7. Student Preference Estimates<br />
<strong>School</strong> Characteristics x Student Demographics<br />
<strong>School</strong> Characteristics x <strong>School</strong> Characteristics x Student Baseline Achievement<br />
<strong>School</strong> Characteristics x Student Baseline <br />
Specifications: <strong>School</strong> Characteristics Student Race Achievement<br />
All Choices Top Choice Only Top Three Choices<br />
(1) (2) (3) (4) (5) (6)<br />
A. Preference for Distance<br />
Distance (in miles) -‐0.318*** -‐0.313*** -‐0.316*** -‐0.310*** -‐0.387*** -‐0.353***<br />
B. Preference for <strong>School</strong> Characteristics<br />
Size <strong>of</strong> 9th Grade (in 100s)<br />
Main effect 0.015*** 0.040*** 0.015*** 0.059*** 0.100*** 0.099***<br />
Baseline Math 0.004*** 0.001** 0.002 0.001<br />
Baseline English -‐0.006*** -‐0.003*** -‐0.009*** -‐0.005***<br />
Subsidized Lunch* -‐0.014*** -‐0.029*** -‐0.024***<br />
Neighborhood Income (in 1000s) -‐0.650*** -‐1.930*** -‐1.930***<br />
Special Education -‐0.007*** -‐0.017*** -‐0.012***<br />
Percent White<br />
Main effect 0.026*** 0.040*** 0.025*** 0.032*** 0.040*** 0.033***<br />
Asian -‐0.016*** -‐0.015*** -‐0.027*** -‐0.018***<br />
Black -‐0.022*** -‐0.020*** -‐0.035*** -‐0.024***<br />
Hispanic -‐0.011*** -‐0.009*** -‐0.016*** -‐0.010***<br />
Attendance Rate<br />
Main effect 0.055*** 0.103*** 0.062*** 0.075*** 0.131*** 0.117***<br />
Baseline Math 0.012*** 0.010*** 0.017*** 0.015***<br />
Baseline English 0.014*** 0.014*** 0.020*** 0.021***<br />
Subsidized Lunch* 0.000 -‐0.008*** -‐0.005***<br />
Neighborhood Income (in 1000s) 6.000*** 8.000*** 7.000***<br />
Percent Subsidized Lunch<br />
Main effect -‐0.002*** -‐0.007*** -‐0.003*** -‐0.004*** 0.005*** -‐0.003**<br />
Asian 0.000 -‐0.001 -‐0.004*** -‐0.001<br />
Black 0.002*** 0.001** -‐0.008*** -‐0.002***<br />
Hispanic 0.010*** 0.009*** 0.007*** 0.010***<br />
Subsidized Lunch* 0.000 -‐0.002** -‐0.001***<br />
Neighborhood Income (in 1000s) -‐0.722*** -‐1.000*** -‐0.736***
Table 7. Student Preference Estimates (cont.)<br />
<strong>School</strong> Characteristics x Student Demographics<br />
<strong>School</strong> Characteristics x <strong>School</strong> Characteristics x Student Baseline Achievement<br />
<strong>School</strong> Characteristics x Student Baseline <br />
Specifications: <strong>School</strong> Characteristics Student Race Achievement<br />
All Choices Top Choice Only Top Three Choices<br />
(1) (2) (3) (4) (5) (6)<br />
High Math Achievement<br />
Main effect 0.008*** 0.007*** 0.004*** 0.000 0.001 -‐0.001<br />
Baseline Math 0.008*** 0.007*** 0.012*** 0.009***<br />
Baseline English 0.005*** 0.005*** 0.006*** 0.005***<br />
Subsidized Lunch* -‐0.003*** -‐0.005*** -‐0.005***<br />
Neighborhood Income (in 1000s) 0.119* -‐0.712*** -‐0.242**<br />
Limited English Pr<strong>of</strong>icient -‐0.002*** -‐0.003* -‐0.005***<br />
Special Education -‐0.003*** 0.000 -‐0.001<br />
Spanish Language Program<br />
Limited English Pr<strong>of</strong>icient 5.147*** 5.755*** 5.472***<br />
Limited English Pr<strong>of</strong>icient x Hispanic -‐3.142*** -‐3.817*** -‐3.147***<br />
Asian Language Program<br />
Limited English Pr<strong>of</strong>icient 3.687*** 4.253*** 4.203***<br />
Limited English Pr<strong>of</strong>icient x Asian -‐2.131*** -‐3.299*** -‐2.717***<br />
Other Language Program<br />
Limited English Pr<strong>of</strong>icient 2.289*** 2.947*** 2.870***<br />
Program Type Dummies X X X X X X<br />
Program Specialty Dummies X X X X X X<br />
C. Model Diagnostics<br />
Number <strong>of</strong> ranks 542,666 542,666 542,666 542,666 69,907 197,245<br />
Number <strong>of</strong> parameters 248 272 280 349 349 349<br />
Log likelihood -‐2,726,580 -‐2,714,493 -‐2,710,042 -‐2,690,008 -‐302,141 -‐906,942<br />
Pseudo R-‐squared 0.85 0.89 0.89 0.92 0.96 0.95<br />
Fraction <strong>of</strong> variance explained by school 0.43 0.68 0.42 0.53 0.58 0.60<br />
observables<br />
Notes: Selected coefficients from two-‐step maximum likelihood estimation <strong>of</strong> demand system with 69,907 students and submitted ranks over 497 program choices in 235 schools. All parameter estimates in appendix Table C1. Student <br />
distance calculated as road distance using ArcGIS. Dummies for missing school attributes are estimated with separate coefficients. Estimates use all submitted ranks except in columns (4)-‐(6). Column (1) contains no interactions between <br />
student and school characteristics. Column (2) contains interactions <strong>of</strong> race dummies with all school characteristics. Column (3) contains interactions <strong>of</strong> baseline Math and English score with school characteristics. Columns (4)-‐(6) contain <br />
interactions <strong>of</strong> gender, race, achievement, special education, limited English pr<strong>of</strong>iciency, subsidized lunch, and median 2000 census block group family income with all school characteristics. High Math achievement is the fraction <strong>of</strong> student <br />
that scored more than 85% on the Math A in New York State Report Cards. Models estimate the utility differences amongst inside options only, with an arbitrarily chosen school's mean utility normalized to zero (this normalization is without <br />
loss <strong>of</strong> generality). Number <strong>of</strong> parameters include school fixed effects. Pseudo R-‐squared is the fraction <strong>of</strong> variance in utility explained by observables. * significant at 10%; ** significant at 5%; *** significant at 1%
Table 8. Estimated Heterogeneity in Student Preferences<br />
By Quartile<br />
Range from Top to <br />
Bottom Second Third Top Bottom Quartile<br />
(1) (2) (3) (4) (5)<br />
A. By Quartiles <strong>of</strong> Student Utility Net <strong>of</strong> Distance<br />
All Students -‐4.32 -‐0.85 1.12 4.82 9.14<br />
Asian -‐5.13 -‐1.34 1.08 5.87 11.00<br />
Black -‐4.15 -‐0.57 1.19 4.41 8.56<br />
Hispanic -‐3.89 -‐0.80 1.08 4.50 8.40<br />
White -‐5.84 -‐1.16 1.07 6.09 11.93<br />
High Math Baseline -‐4.97 -‐1.21 1.14 5.82 10.78<br />
Low Math Baseline -‐4.19 -‐0.79 1.14 4.72 8.91<br />
B. By Quartiles <strong>of</strong> <strong>School</strong> High Math Achievement<br />
All Students -‐1.54 -‐0.30 0.90 2.15 3.69<br />
High Math Baseline -‐2.30 -‐0.68 0.85 3.63 5.94<br />
Low Math Baseline -‐1.52 -‐0.30 0.91 2.13 3.65<br />
C. By Quartiles <strong>of</strong> <strong>School</strong> Percent Subsidized Lunch<br />
All Students 2.23 0.88 -‐1.00 -‐1.41 3.64<br />
Bottom Neighborhood Income Quartile 1.50 0.67 -‐0.54 -‐0.86 2.36<br />
Top Neighborhood Income Quartile 3.11 1.11 -‐1.60 -‐2.12 5.23<br />
D. By Quartiles <strong>of</strong> <strong>School</strong> Percent White<br />
All Students -‐2.04 -‐0.69 0.73 2.94 4.98<br />
Asian -‐3.01 -‐1.30 0.89 4.57 7.57<br />
Black -‐1.62 -‐0.43 0.69 2.10 3.72<br />
Hispanic -‐1.71 -‐0.42 0.81 2.54 4.25<br />
White -‐3.44 -‐1.73 0.44 5.14 8.58<br />
Notes: Each row defines a set <strong>of</strong> students and each column defines a set <strong>of</strong> programs based on either a program characteristic or the estimated desirability <strong>of</strong> the program. Utilities in <br />
distance-‐equivalent miles based on the estimates reported in column (4) <strong>of</strong> Table 7. Each cell reports the average normalized utility for the relevant student-‐school group, where we <br />
normalize the mean utility, net <strong>of</strong> distance, across programs for each student to zero.
Table 9. In-‐Sample Model Fit<br />
Average in <br />
Ranked Choice<br />
Overall <br />
Distribution 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th<br />
A. All Students<br />
<strong>Assignment</strong> Observed -‐ 32% 15% 10% 7% 5% 4% 3% 2% 1% 1% 1% 1%<br />
Distance Observed 12.7 4.4 4.8 5.1 5.2 5.4 5.5 5.6 5.6 5.6 5.6 5.4 5.1<br />
Simulated 5.1 5.2 5.3 5.3 5.4 5.4 5.5 5.5 5.6 5.7 5.7 5.7<br />
High English Achievement<br />
High Math Achievement<br />
Observed 19.3 27.8 25.6 24.3 23.2 22.2 21.4 20.8 20.5 19.9 19.4 19.1 18.6<br />
Simulated 24.8 24.8 24.5 24.3 24.0 23.9 23.7 23.6 23.5 23.5 23.4 23.3<br />
Observed 10.0 16.7 15.3 14.7 13.9 13.4 12.8 12.4 11.9 11.5 11.1 10.8 10.4<br />
Simulated 14.9 15.0 14.7 14.5 14.3 14.2 14.1 14.0 14.0 13.9 13.9 13.8<br />
Percent Attending Four Year <br />
College<br />
Fraction Inexperienced <br />
Teachers<br />
Fraction Subsidized Lunch<br />
Attendance Rate (out <strong>of</strong> <br />
100)<br />
Observed 47.2 53.7 54.1 52.8 51.7 50.9 50.1 49.3 49.3 49.0 48.3 47.6 47.5<br />
Simulated 52.8 53.1 53.1 53.0 52.9 52.7 52.6 52.5 52.4 52.3 52.2 52.1<br />
Observed 55.6 43.0 43.8 45.0 46.2 47.1 48.1 48.7 49.5 49.5 50.2 49.9 49.9<br />
Simulated 44.9 44.9 45.0 45.1 45.1 45.2 45.3 45.3 45.4 45.4 45.5 45.5<br />
Observed 62.6 51.4 53.4 54.5 56.2 57.4 58.7 59.8 60.6 61.3 61.9 62.7 63.1<br />
Simulated 53.5 53.7 54.0 54.2 54.4 54.6 54.7 54.9 55.0 55.1 55.3 55.4<br />
Observed 85.7 87.2 86.7 86.4 86.1 85.9 85.7 85.5 85.5 85.2 85.0 84.8 84.5<br />
Simulated 86.3 86.3 86.2 86.1 86.1 86.1 86.0 86.0 86.0 86.0 86.0 86.0<br />
Percent Asian Observed 8.6 13.8 13.7 13.3 12.7 12.2 11.5 11.0 10.6 10.1 9.7 9.5 9.4<br />
Simulated 13.0 13.0 13.0 13.0 13.0 13.0 12.9 12.9 12.9 12.9 12.9 12.9<br />
Percent Black Observed 38.4 32.6 34.3 35.1 36.0 36.7 37.5 38.2 38.4 38.7 38.9 38.9 38.1<br />
Simulated 34.5 34.7 34.9 35.1 35.2 35.4 35.5 35.6 35.7 35.8 35.9 35.9<br />
Percent Hispanic Observed 42.1 34.4 35.2 35.9 37.0 37.9 38.9 39.6 40.2 40.8 41.5 42.2 43.5<br />
Simulated 35.1 35.2 35.3 35.5 35.6 35.7 35.7 35.8 35.9 35.9 36.0 36.0<br />
Percent White Observed 10.9 19.1 16.7 15.7 14.4 13.3 12.2 11.2 10.8 10.4 9.8 9.4 9.0<br />
Simulated 17.4 17.1 16.7 16.5 16.2 16.0 15.9 15.7 15.6 15.4 15.3 15.2<br />
B. Students with Low Baseline Math Scores<br />
Distance Observed 12.3 4.1 4.4 4.7 4.9 5.0 5.1 5.2 5.2 5.2 5.1 5.0 4.8<br />
Simulated 5.0 5.1 5.1 5.1 5.2 5.2 5.2 5.3 5.3 5.4 5.4 5.4<br />
High Math Achievement Observed 10.0 10.9 10.9 10.5 10.1 10.0 9.7 9.6 9.4 9.4 9.1 8.8 8.8<br />
Simulated 10.4 10.4 10.4 10.4 10.4 10.4 10.4 10.4 10.4 10.4 10.3 10.3<br />
C. Students with High Baseline Math Scores<br />
Distance Observed 13.2 4.6 5.0 5.2 5.4 5.7 5.9 5.9 6.0 6.1 6.2 5.8 5.3<br />
Simulated 5.1 5.2 5.4 5.5 5.5 5.6 5.7 5.8 5.9 5.9 6.0 6.1<br />
High Math Achievement Observed 10.0 26.0 21.4 20.5 19.1 18.2 17.3 16.4 15.6 15.2 14.2 13.9 12.8<br />
Simulated 20.7 20.8 20.0 19.3 18.9 18.6 18.4 18.2 18.1 18.0 17.8 17.7<br />
Notes: Means. Observed characteristics <strong>of</strong> each choice are from demand sample. Simulated characteristics come from estimated preference distribution corresponding to the model in Table 7 column (4) assuming students rank their top 12 programs. <br />
Missing school characteristics are ommitted from both observed and simulated rows. <strong>The</strong> average in overall distribution shows average distance to school for all student-‐shool pairs and other school characteristics are an unweighted average across <br />
programs.
Table 10. Out <strong>of</strong> Sample Fit for Exit and Matriculation<br />
<strong>Coordinated</strong> Mechanism Uncoordinated Mechanism<br />
(1) (2) (3) (4)<br />
A. Exit from NYC Public <strong>School</strong>s<br />
Utility from <strong>Assignment</strong> (in miles) -‐0.0027*** -‐0.0030***<br />
(0.0002) (0.0002)<br />
Distance from <strong>Assignment</strong> 0.0001 -‐0.0026***<br />
(0.0003) (0.0003)<br />
Number <strong>of</strong> observations 66,921 66,921 70,358 70,358<br />
R-‐squared 0.0650 0.0623 0.0870 0.0856<br />
B. Enroll in <strong>School</strong> Other than Assigned<br />
Utility from <strong>Assignment</strong> (in miles) -‐0.0130*** -‐0.0156***<br />
(0.0003) (0.0003)<br />
Distance from <strong>Assignment</strong> 0.0120*** 0.0009*<br />
(0.0004) (0.0005)<br />
Number <strong>of</strong> observations 62,656 62,656 64,349 64,349<br />
R-‐squared 0.0601 0.0392 0.0610 0.0313<br />
Notes: Uncoordinated mechanism refers to 2002-‐03 mechanism and coordinated mechanism refers to the 2003-‐04 mechanism based on <br />
deferred acceptance. Exit is indicator that assigned student enrolled in a school outside NYC public system. Enroll in school other than assigned <br />
is an indicator that assigned student enrolled in NYC public high school other than that assigned at end <strong>of</strong> match. Distance calculated as road <br />
distance using ArcGIS. Utility estimates are come from specification in column (4) <strong>of</strong> Table 7, projected on observable student and school <br />
characteristics for the mechanism comparison sample in the uncoordinated and coordinated mechanism. Columns (1) and (3) report estimates <br />
<strong>of</strong> exit and enrollment indicators regressed on utility (in distance units), while columns (2) and (4) report estimates <strong>of</strong> exit and enrollment <br />
indicators regressed on on distance to assignment. All models include controls for student race, baseline Math, baseline English, no baseline <br />
Math score, no baseline English score, subsidized lunch, neighborhood income (from 2000 census block group median family income), limited <br />
English pr<strong>of</strong>iciency, and special education. * significant at 10%; ** significant at 5%; *** significant at 1%
Table 11. <strong>Welfare</strong> Comparison between Mechanisms<br />
Change in Utility<br />
(in miles)<br />
Change in Distance (in miles)<br />
2002-‐03 Offer Processing<br />
<strong>Assignment</strong> Enrollment <strong>Assignment</strong> Enrollment Main Rounds Supplementary Over-‐the-‐Counter<br />
(1) (2) (3) (4) (5) (6) (7)<br />
All Students 4.79 3.99 0.69 0.38 49.2% 14.5% 36.3%<br />
Female 4.45 3.63 0.68 0.37 51.8% 14.3% 33.9%<br />
Asian 4.75 3.75 0.63 0.33 46.6% 5.4% 48.0%<br />
Black 4.57 3.82 0.74 0.45 53.7% 18.4% 27.8%<br />
Hispanic 4.95 4.23 0.66 0.39 51.9% 17.3% 30.8%<br />
White 5.25 3.94 0.56 0.22 33.1% 3.8% 63.1%<br />
Bronx 5.26 4.57 0.93 0.64 53.5% 20.2% 26.2%<br />
Brooklyn 4.79 4.15 0.52 0.33 50.3% 16.2% 33.5%<br />
Manhattan 1.80 1.39 0.00 -‐0.09 69.9% 19.2% 10.8%<br />
Queens 5.41 4.01 1.13 0.57 43.2% 8.3% 48.5%<br />
Staten Island 6.32 6.20 0.34 0.00 13.5% 0.0% 86.5%<br />
High Baseline Math 3.14 2.12 0.53 0.20 58.5% 7.4% 34.1%<br />
Low Baseline Math 5.25 4.65 0.57 0.33 47.1% 19.8% 33.1%<br />
Subsidized Lunch* 4.52 3.77 0.59 0.31 52.9% 15.4% 31.7%<br />
Bottom Neighborhood Income <br />
4.38 3.96 0.57 0.42 56.4% 23.3% 20.3%<br />
Quartile<br />
Top Neighborhood Income Quartile 4.59 3.36 0.71 0.25 42.2% 8.1% 49.8%<br />
Special Education 5.38 4.40 0.76 0.43 39.2% 18.8% 42.0%<br />
Limited English Pr<strong>of</strong>icient 5.68 5.02 0.60 0.38 47.1% 16.3% 36.6%<br />
SHSAT Test-‐Takers 2.63 1.85 0.55 0.25 62.5% 10.3% 27.2%<br />
Notes: Utilities are in distance units (miles) averaged across students in the mechanism comparison sample in Table 1 using preference estimates in Table 7 column (4). <strong>Assignment</strong> is the school assigned at the <br />
conclusion <strong>of</strong> the high school assignment process. Enrollment is the school student enrolls in October following application. 2002-‐03 <strong>of</strong>fer process reports the fraction <strong>of</strong> students with row characteristic first <strong>of</strong>fered <br />
school finally assigned in the main round (rounds 1-‐3), the supplementary round or the over the counter process. Student distance calculated as road distance using ArcGIS. High baseline math students score above <br />
the 75th percentile for 7th grade relative to citywide distribution, while low baseline math students score below the 25th percentile. Subsidized lunch not available pre-‐assignment and comes from enrolled students <br />
as <strong>of</strong> 2004-‐05 school year. Neighborhood income is median census block group family income from the 2000 census.
Table 12. Alternative Mechanisms Relative to Utilitarian Optimal Benchmark<br />
Student-‐Proposing Student Optimal Ordinal Pareto Efficient <br />
Deferred Acceptance<br />
Matching<br />
Matching <br />
(1) (2) (3)<br />
Utility (in miles) -‐2.85 -‐2.74 -‐2.24<br />
Number <strong>of</strong> students reassignments <br />
relative to column (1)<br />
2,348 10,882<br />
Notes: Utility from alternative assignments relative to utilitarian optimal assignment computed using actual preferences ignoring all school-‐side <br />
constraints except capacity. Utility computed using estimates in Table 7 column (4). Mean utility from the utilitarian optimal assignment normalized to <br />
zero. Column (1) is from 500 lottery draws <strong>of</strong> student-‐proposing deferred acceptance with single tie-‐breaking using the demand estimation sample. If <br />
a student is unassigned, we mimic the Supplementary Round by assigning students according to a serial dictatorship using preferences drawn from the <br />
preference distribution estimated in Table 7 column (4). Student optimal matching computed by taking each deferred acceptance assignment and <br />
applying the Erdil-‐Ergin (2008) stable improvement cycles algorithm to find a student-‐optimal matching. Ordinal Pareto Efficient Matching computed <br />
by applying Gale's top trading cycles to the economy where the student-‐optimal matching determine student endowments, followed by the <br />
Abdulkadiroglu-‐Sonmez (2003) version <strong>of</strong> top trading cycles with counters.
Figure 1. <strong>School</strong> Locations and Students by New York City Census Tract in 2002-‐03 and 2003-‐04
Figure 2. Distribution <strong>of</strong> Distance to Assigned <strong>School</strong> in <br />
Uncoordinated (2002-‐03) and <strong>Coordinated</strong> (2003-‐04) Mechanism <br />
Mean (median) travel distance is 3.36 (2.25) miles in 2002-‐03 and 4.05 (3.04) miles in 2003-‐04. Top and bottom 1% are not shown in <br />
figure. Line fit from Gaussian kernel with bandwidth chosen to minimize mean integrated squared error using STATA’s kdensity <br />
command.
Figure 3. Log-‐Log Plot <strong>of</strong> Predicted vs. Actual Program Market Shares <br />
Market share means fraction <strong>of</strong> applicants ranking program anywhere on rank order list. Predicted program market shares based on <br />
preference estimates from the specification in column (4) <strong>of</strong> Table 6, using 100 draws from our estimates and assuming students <br />
rank their top 12 programs. Each point corresponds to the predicted and actual market share.
Figure 4. Student <strong>Welfare</strong> from Uncoordinated and <strong>Coordinated</strong> Mechanism <br />
Distribution <strong>of</strong> utility (measured in distance units) from assignment based on Table 7 column (4) estimates plotted with mean utility <br />
in 2003-‐04 normalized to 0. Average difference corresponds to 4.79 miles. Top and bottom 1% are not shown in figure. Line fit <br />
from Gaussian kernel with bandwidth chosen to minimize mean integrated squared error using STATA’s kdensity command.
Figure 5. Change in Student <strong>Welfare</strong> by Propensity to Assigned <br />
in the Over-‐the-‐Counter Round <strong>of</strong> Uncoordinated Mechanism <br />
<br />
Probability <strong>of</strong> over the counter assignment estimated from probit <strong>of</strong> over the counter on student census tract dummies and all student characteristics in the <br />
demand model except for distance. If student lives in tract where either all students are assigned over the counter or no students are assigned over the <br />
counter, for forecasting all students from those tracts are coded as over the counter. Each student is assigned to one <strong>of</strong> ten deciles <strong>of</strong> probability <strong>of</strong> over the <br />
counter based on these estimates. Differences across deciles in distance-‐equivalent utility including distance, distance-‐equivalent utility net <strong>of</strong> distance, and <br />
distance are plotted, where preference estimates come from column (4) <strong>of</strong> Table 7, conditional on submitted rankings. Distance is calculated as road <br />
distance using ArcGIS.
References<br />
Abdulkadiroğlu, A., J. Angrist, and P. Pathak (2013): “<strong>The</strong> Elite Illusion: Achievement<br />
<strong>Effects</strong> at Boston and New York Exam <strong>School</strong>s,” forthcoming, Econometrica.<br />
Abdulkadiroğlu, A., J. D. Angrist, S. M. Dynarski, T. J. Kane, and P. A. Pathak<br />
(2011): “Accountability and Flexibility in Public <strong>School</strong>s: Evidence from Boston’s Charters<br />
and Pilots,” Quarterly Journal <strong>of</strong> Economics, 126(2), 699–748.<br />
Abdulkadiroğlu, A., Y.-K. Che, and Y. Yasuda (2012): “Expanding Choice in <strong>School</strong><br />
Choice,” Working paper, Duke University.<br />
Abdulkadiroğlu, A., P. A. Pathak, and A. E. Roth (2005): “<strong>The</strong> New York City High<br />
<strong>School</strong> Match,” American Economic Review, Papers and Proceedings, 95, 364–367.<br />
(2009): “Strategy-pro<strong>of</strong>ness versus Efficiency in Matching with Indifferences: Redesigning<br />
the New York City High <strong>School</strong> Match,” American Economic Review, 99(5), 1954–1978.<br />
Abdulkadiroğlu, A., and T. Sönmez (2003): “<strong>School</strong> Choice: A Mechanism Design Approach,”<br />
American Economic Review, 93, 729–747.<br />
Abulkadiroğlu, A., W. Hu, and P. A. Pathak (2013): “Small High <strong>School</strong>s and Student<br />
Achievement: Lottery-Based Evidence from New York City,” NBER Working Paper, 19576.<br />
Agarwal, N. (2013): “An Empirical Model <strong>of</strong> the Medical Match,” Unpublished working paper,<br />
MIT.<br />
ALG (2005): “Association <strong>of</strong> London Government, Report to the Leader’s Committee on London<br />
Co-ordinated Admissions (Item 9), May 10,” May 10.<br />
Arvidsson, T. S., N. Fruchter, and C. Mokhtar (2013): “Over the Counter,<br />
Under the Radar: Inequitably Distributing New York City’s Late-Enrolling High<br />
<strong>School</strong> Students,” Annenberg Institute for <strong>School</strong> Reform at Brown University, Available<br />
at: http://annenberginstitute.org/publication/over-counter-under-radar-inequitablydistributing-new-york-city%E2%80%99s-late-enrolling-high-sc,<br />
Last accessed: October 2013.<br />
Azevedo, E. M., and J. D. Leshno (2011): “Can we make school choice more efficient? An<br />
example,” Unpublished mimeo, Harvard University.<br />
Balinski, M., and T. Sönmez (1999): “A Tale <strong>of</strong> Two Mechanisms: Student Placement,”<br />
Journal <strong>of</strong> Economic <strong>The</strong>ory, 84, 73–94.<br />
Berry, S. (1994): “Estimating Discrete-Choice Models <strong>of</strong> Product Differentiation,” Rand Journal<br />
<strong>of</strong> Economics, 25(2), 242–262.<br />
Berry, S., J. Levinsohn, and A. Pakes (1995): “Automobile Prices in Market Equilibrium:,”<br />
Econometrica, 63(4), 841–890.<br />
48
(2004): “Differentiated Products Demand Systems from a Combination <strong>of</strong> Micro and<br />
Macro Data: <strong>The</strong> New Car Market,” Journal <strong>of</strong> Political Economy, 112(1), 68–105.<br />
Bulow, J., and J. Levin (2006): “Matching and Price Competition,” American Economic<br />
Review, 96, 652–668.<br />
Coase, R. (1960): “<strong>The</strong> Problem <strong>of</strong> Social Cost,” Journal <strong>of</strong> Law and Economics, 3, 1–44.<br />
Darville, S. (2013): “NYC Sitting Out National Move to Tie Charter, District Admissions,”<br />
Gotham<strong>School</strong>s, September 20, http://gothamschools.org/2013/09/20/nyc-sitting-outnational-move-to-tie-charter-district-admissions/,<br />
Last accessed: September 2013.<br />
Deming, D. J., J. S. Hastings, T. J. Kane, and D. O. Staiger (2011): “<strong>School</strong> Choice,<br />
<strong>School</strong> Quality and Postsecondary Attainment,” forthcoming, American Economic Review.<br />
DOE (2002): “Directory <strong>of</strong> the New York City Public High <strong>School</strong>s: 2002-2003,” New York City<br />
Department <strong>of</strong> Education.<br />
(2003): “Directory <strong>of</strong> the New York City Public High <strong>School</strong>s: 2003-2004,” New York<br />
City Department <strong>of</strong> Education.<br />
Dubins, L. E., and D. A. Freedman (1981): “Machiavelli and the Gale-Shapley algorithm,”<br />
American Mathematical Monthly, 88, 485–494.<br />
Engberg, J., D. Epple, J. Imbrogno, H. Sieg, and R. Zimmer (2010): “Bounding the<br />
Treatment <strong>Effects</strong> <strong>of</strong> Education Programs that Have Lotteried Admission and Selective Attrition,”<br />
Unpublished working paper, University <strong>of</strong> Pennsylvania.<br />
Epple, D., and R. Romano (1998): “Competition Between Private and Public <strong>School</strong>s, Vouchers,<br />
and Peer Group <strong>Effects</strong>,” American Economic Review, 88(1), 33–62.<br />
Erdil, A., and H. Ergin (2008): “What’s the Matter with Tie-Breaking? Improving Efficiency<br />
in <strong>School</strong> Choice,” American Economic Review, 98, 669–689.<br />
Gale, D., and L. S. Shapley (1962): “College Admissions and the Stability <strong>of</strong> Marriage,”<br />
American Mathematical Monthly, 69, 9–15.<br />
Gendar, A. (2000): “Simpler HS Entry Eyed Ed Board Aims to Trim Selection Process, Improve<br />
<strong>School</strong>s,” New York Daily News, October 21.<br />
Haeringer, G., and F. Klijn (2009): “Constrained <strong>School</strong> Choice,” Journal <strong>of</strong> Economic<br />
<strong>The</strong>ory, 144, 1921–1947.<br />
Hastings, J., T. J. Kane, and D. O. Staiger (2005): “Parental Preferences and <strong>School</strong><br />
Competition: Evidence from a Public <strong>School</strong> Choice Plan,” NBER Working Paper, 11805.<br />
(2009): “Heterogenous Preferences and the Efficacy <strong>of</strong> Public <strong>School</strong> Choice,” Working<br />
paper, Yale University.<br />
49
Hastings, J., and J. M. Weinstein (2008): “Information, <strong>School</strong> Choice and Academic<br />
Achievement: Evidence from Two Experiments,” Quarterly Journal <strong>of</strong> Economics, 123(4),<br />
1373–1414.<br />
He, Y. (2012): “Gaming the Boston Mechanism in Beijing,” Unpublished mimeo, Toulouse.<br />
Hemphill, C. (2007): New York City’s Best Public High <strong>School</strong>s: A Parent’s Guide, Third<br />
Edition. Teachers College Press.<br />
Hemphill, C., and K. Nauer (2009): “<strong>The</strong> New Marketplace: How Small-<strong>School</strong> Reforms<br />
and <strong>School</strong> Choice have Reshaped New York City’s High <strong>School</strong>s,” Center for New York City<br />
Affairs, Milano the New <strong>School</strong> for Management and Urban Policy.<br />
Herszenhorn, D. M. (2004): “Council Members See Flaws in <strong>School</strong>-Admissions Plan,” New<br />
York Times, Metropolitan Desk, November 19.<br />
Hoxby, C. (2003): “<strong>School</strong> Choice and <strong>School</strong> Productivity (Or, Could <strong>School</strong> Choice be a<br />
Rising Tide that Lifts All Boats),” in <strong>The</strong> Economics <strong>of</strong> <strong>School</strong> Choice, ed. by C. Hoxby.<br />
Chicago: University <strong>of</strong> Chicago Press.<br />
Jacob, B., and L. Lefgren (2007): “What Do Parents Value in Education? An Empirical<br />
Investigation <strong>of</strong> Parents’ Revealed Preferences for Teachers,” Quarterly Journal <strong>of</strong> Economics,<br />
122(4), 1603–1637.<br />
Kerr, P. (2003): “Parents Perplexed by High <strong>School</strong> Admissions Rule: Reply,” New York Times,<br />
November 3.<br />
Kesten, O. (2010): “<strong>School</strong> Choice with Consent,” Quarterly Journal <strong>of</strong> Economics, 125(3),<br />
1297–1348.<br />
Kesten, O., and M. Kurino (2012): “On the (im)possibility <strong>of</strong> improving upon the studentproposing<br />
deferred acceptance mechanism,” WZB Discussion Paper SP II 2012-202.<br />
Klemperer, P. (2002): “What Really Matters in Auction Design,” Journal <strong>of</strong> Economic Perspectives,<br />
16(1), 169–189.<br />
Kling, J. R., S. Mullainathan, E. Shafir, L. C. Vermuelen, and M. V. Wrobel<br />
(2012): “Comparison Friction: Experimental Evidence from Medicare Drug Plans,” Quarterly<br />
Journal <strong>of</strong> Economics, 127, 199–235.<br />
Kojima, F., and P. A. Pathak (2009): “Incentives and Stability in Large Two-Sided Matching<br />
Markets,” American Economic Review, 99(3), 608–627.<br />
Niederle, M., and A. E. Roth (2003): “Unraveling reduced mobility in a labor market:<br />
Gastroenterology with and without a centralized match,” Journal <strong>of</strong> Political Economy, 111,<br />
1342–1352.<br />
Pathak, P. A., and T. Sönmez (2008): “Leveling the Playing Field: Sincere and Sophisticated<br />
Players in the Boston Mechanism,” American Economic Review, 98(4), 1636–1652.<br />
50
(2013): “<strong>School</strong> Admissions Reform in Chicago and England: Comparing Mechanisms<br />
by their Vulnerability to Manipulation,” American Economic Review, 103(1), 80–106.<br />
Pennell, H., A. West, and A. Hind (2006): “Secondary <strong>School</strong> Admissions in London,”<br />
Centre for Education Research, LSE CLARE Working paper No. 19.<br />
Ravitch, D. (2011): <strong>The</strong> Death and Life <strong>of</strong> the Great American <strong>School</strong> System. Basic Books.<br />
Roth, A. E. (1982): “<strong>The</strong> Economics <strong>of</strong> Matching: Stability and Incentives,” Mathematics <strong>of</strong><br />
Operations Research, 7, 617–628.<br />
Roth, A. E., and X. Xing (1997): “Turnaround Time and Bottlenecks in Market Clearing: Decentralized<br />
Matching in the Market for Clinical Psychologists,” Journal <strong>of</strong> Political Economy,<br />
105, 284–329.<br />
Rothstein, J. (2006): “Good Principals or Good Peers: Parental Valuation <strong>of</strong> <strong>School</strong> Characteristics,<br />
Tiebout Equilibrium, and the Incentive <strong>Effects</strong> <strong>of</strong> Competition Among Jurisdictions,”<br />
American Economic Review, 96(4), 1333–1350.<br />
Schneider, M., P. Teske, and M. Marschall (2002): Choosing <strong>School</strong>s: Consumer Choice<br />
and the Quality <strong>of</strong> American <strong>School</strong>s. Princeton University Press.<br />
Shapley, L., and H. Scarf (1974): “On Cores and Indivisibility,” Journal <strong>of</strong> Mathematical<br />
Economics, 1, 23–28.<br />
Shapley, L., and M. Shubik (1971): “<strong>The</strong> <strong>Assignment</strong> Game I: <strong>The</strong> core,” International<br />
Journal <strong>of</strong> Game <strong>The</strong>ory, 1(1), 111–130.<br />
Sönmez, T. (1997): “Manipulation via Capacities in Two-Sided Matching Markets,” Journal <strong>of</strong><br />
Economic <strong>The</strong>ory, 77, 197–204.<br />
Urquiola, M. (2005): “Does school choice lead to sorting? Evidence from Tiebout variation,”<br />
American Economic Review, 95(4), 1310–1326.<br />
Vanacore, A. (2012): “Centralized enrollment in Recovery <strong>School</strong> District gets first tryout,”<br />
Times-Picayune, New Orleans, April 16.<br />
Vaznis, J. (2013): “Barros Proposes Single Application for City, Charter <strong>School</strong>s,” Boston Globe,<br />
Education Desk, June 5.<br />
Williams, J. (2004): “HS delay pinches parents,” New York Daily News, March 25.<br />
51
Table A1. Offer Acceptances among Students who Obtain Multiple First Round Offers in Uncoordinated<br />
Mechanism<br />
Among those Accepting First<br />
Round Offer<br />
Fraction who<br />
Accept No First<br />
Fraction who<br />
Accept First<br />
Accept Offer<br />
Other than<br />
Most Preferred<br />
Number Round Offer Round Offer<br />
Accept Most<br />
Preferred<br />
(1) (2) (3) (4) (5)<br />
Two or more <strong>of</strong>fers 12,508 0.14 0.86 0.71 0.29<br />
Best <strong>of</strong>fer<br />
1 7,096 0.09 0.91 0.75 0.25<br />
2 3,159 0.17 0.83 0.67 0.33<br />
3 1,624 0.23 0.77 0.60 0.40<br />
4 629 0.30 0.70 0.58 0.42<br />
Notes: Table reports the fraction <strong>of</strong> students who received multiple first round <strong>of</strong>fers in the uncoordinated mechanism in 2002‐03. A<br />
student accepts her most preferred option if assigned to that option at the conclusion <strong>of</strong> the match. A student accepts first round<br />
<strong>of</strong>fer other than most preferred if assigned to an school ranked below the most preferred <strong>of</strong>fer. A student accepts none <strong>of</strong> her first<br />
round <strong>of</strong>fers if assigned to a school not <strong>of</strong>fered in the first round at the conclusion <strong>of</strong> the match.
Table A2. Main Round <strong>Assignment</strong>s in <strong>Coordinated</strong> Mechanism, by Length <strong>of</strong> Rank Order List<br />
Length <strong>of</strong> Rank Order List<br />
Choice Assigned All 1 2 3 4 5 6 7 8 9 10 11 12<br />
Total 69,907 4,597 3,282 4,128 4,622 4,952 4,776 4,406 4,390 4,558 6,135 9,849 14,212<br />
1 31.9% 88.6% 40.7% 35.2% 31.9% 27.9% 28.6% 27.1% 25.7% 25.6% 25.4% 26.2% 25.2%<br />
2 15.0% 39.8% 17.7% 15.1% 14.8% 14.6% 13.7% 13.9% 13.9% 15.2% 14.7% 14.6%<br />
3 10.2% 24.3% 11.6% 11.6% 10.6% 10.0% 10.8% 9.9% 10.4% 10.4% 10.5%<br />
4 7.3% 18.0% 9.3% 8.1% 7.9% 8.0% 7.6% 7.6% 7.8% 8.2%<br />
5 5.4% 12.8% 7.0% 7.0% 6.3% 6.1% 6.6% 6.2% 6.7%<br />
6 3.9% 10.2% 5.7% 4.9% 5.0% 4.9% 4.8% 5.3%<br />
7 2.9% 8.1% 4.3% 4.4% 4.0% 4.1% 4.3%<br />
8 2.0% 5.8% 3.4% 3.3% 2.9% 3.5%<br />
9 1.5% 4.0% 2.8% 2.7% 2.8%<br />
10 1.1% 3.2% 2.3% 2.6%<br />
11 0.8% 2.6% 2.2%<br />
12 0.5% 2.5%<br />
Unassigned 17.5% 11.4% 19.5% 22.8% 23.3% 23.6% 20.9% 20.6% 20.3% 20.1% 16.7% 15.3% 11.6%<br />
Notes: This table reports choices assigned after the main round in coordinated 2003-‐04 mechanism based on deferred acceptance.
Table A3. <strong>Assignment</strong> and Enrollment Decisions <strong>of</strong> Students in <strong>Coordinated</strong> Mechanism by Rank Order List Length<br />
Length <strong>of</strong> Rank Order List<br />
All 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th<br />
A. <strong>Assignment</strong> Offered in Main Round<br />
Number <strong>of</strong> Students 57,658 4,072 2,641 3,187 3,545 3,782 3,776 3,497 3,499 3,642 5,113 8,340 12,564<br />
Average Rank <strong>of</strong> <strong>Assignment</strong> 3.00 1.00 1.49 1.86 2.21 2.53 2.76 3.04 3.20 3.35 3.49 3.60 3.93<br />
Accept Main Round <strong>Assignment</strong> 92.7% 91.2% 88.5% 88.4% 90.2% 91.2% 92.3% 91.9% 93.0% 93.6% 94.5% 94.6% 94.3%<br />
Enroll at Private High <strong>School</strong> 2.5% 6.9% 7.4% 6.1% 4.5% 2.9% 2.4% 2.1% 1.9% 1.2% 1.0% 0.7% 1.0%<br />
Remain in Current <strong>School</strong> 1.2% 1.2% 2.0% 2.3% 1.9% 2.1% 1.4% 1.8% 1.4% 1.2% 0.9% 0.7% 0.6%<br />
Attend Specialized or Alternative <strong>School</strong> 0.1% 0.0% 0.0% 0.0% 0.0% 0.1% 0.1% 0.2% 0.1% 0.1% 0.2% 0.1% 0.0%<br />
Participate in Supplementary Round 0.3% 0.1% 0.2% 0.2% 0.4% 0.5% 0.6% 0.3% 0.6% 0.3% 0.3% 0.2% 0.3%<br />
B. Students Unassigned after Main Round<br />
Number <strong>of</strong> Students 12,249 525 641 941 1,077 1,170 1,000 909 891 916 1,022 1,509 1,648<br />
Participate in Supplementary Round 52.6% 26.1% 44.8% 54.0% 54.1% 56.2% 55.6% 55.7% 52.7% 46.5% 43.5% 49.6% 68.2%<br />
Enroll at Supplementary Round <strong>Assignment</strong> 72.9% 73.0% 85.0% 76.0% 75.5% 77.8% 73.0% 75.9% 74.5% 68.8% 71.7% 69.5% 66.3%<br />
Enroll at Private High <strong>School</strong> 2.8% 6.7% 6.1% 4.7% 3.5% 3.8% 2.2% 1.7% 1.6% 1.9% 2.2% 1.4% 1.9%<br />
Remaining in Current <strong>School</strong> 3.2% 6.7% 6.2% 5.6% 5.3% 4.4% 3.2% 3.3% 2.5% 2.0% 1.5% 0.9% 1.8%<br />
Attend Specialized or Alternative <strong>School</strong> 0.3% 0.8% 0.5% 0.6% 0.2% 0.1% 0.3% 0.3% 0.3% 0.1% 0.2% 0.5% 0.1%<br />
Notes: <strong>Assignment</strong> and enrollment decisions <strong>of</strong> students in the demand estimation sample under the coordinated mechanism. Panel A restricts to students that received an assignment in an NYC Public <strong>School</strong> in the Main Round. Panel B restricts to students that did not <br />
receive an assignment in the Main Round.
Table A4. Preference Estimates from Alternative Specifications<br />
All Drop Lists wth Dropping Last Ranked With Outside<br />
Choices 12 choices Choice Only Option<br />
(1) (2) (3) (4) (5)<br />
A. Preference for Distance<br />
Distance (in miles) -‐0.310*** -‐0.317*** -‐0.302*** -‐0.024*** -‐0.304***<br />
B. Preference for <strong>School</strong>s -‐ Main effects<br />
Size <strong>of</strong> 9th Grade (in 100s) 0.059*** 0.067*** 0.061*** 0.042 0.044***<br />
Percent White 0.032*** 0.032*** 0.029*** 0.001 0.030***<br />
Attendance Rate 0.075*** 0.082*** 0.087*** 0.061 0.053***<br />
Percent Free Lunch -‐0.004*** -‐0.004*** -‐0.004*** 0.005 -‐0.005***<br />
High Math Achievement 0.000 -‐0.001 0.002*** 0.003 0.002***<br />
Program Type Dummies X X X X X<br />
Program Specialty Dummies X X X X X<br />
C. Model Diagnostics<br />
Number <strong>of</strong> ranks 542,666 372,122 472,579 542,666 542,666<br />
Number <strong>of</strong> parameters 349 349 349 349 350<br />
Log likelihood -‐2,690,008 -‐1,818,749 -‐2,431,624 -‐753,816 -‐2,876,073<br />
Pseudo R-‐squared 0.92 0.92 0.92 0.60 0.86<br />
Fraction <strong>of</strong> variance explained by school observables 0.53 0.56 0.51 0.15 0.47<br />
Notes: Maximum likelihood estimates from demand systems with variations in the sample <strong>of</strong> students and alternative behavioral assumptions. All estimates have same controls as in column (4) <br />
<strong>of</strong> Table 7, which includes interactions with student demographic racial and baseline achievement and school characteristics. Dummies for missing school attributes are estimated with separate <br />
coefficients. <strong>The</strong> first column treats all choices as truthful, the second column drops students who have ranked 12 schools from the sample, the third column only uses information about relative <br />
comparisons among listed alternatives, assuming that unlisted alternatives are missing at random, and the fourth column drops the last ranked school from each students list. Columns (1) through <br />
(4) estimate utility amongst inside options only, with an arbitrarily chosen school's mean utility normalized to zero (this normalization is without loss <strong>of</strong> generality). Column (5) adds an outside <br />
option to the model, treating unranked schools as worse than the outside option. Pseudo R-‐squared in the fraction <strong>of</strong> variance in utility explained by observables. * significant at 10%; ** <br />
significant at 5%; *** significant at 1%
A<br />
Appendix: Simulating Utilities<br />
This appendix explains how we simulate utilities for students given the preference estimates.<br />
Consider a random utility function <strong>of</strong> the form<br />
u ij = V ij + ε ij ,<br />
where ε ij is distributed as F (ε) = exp (− exp (−ε)) and V ij is known. Suppose we observe<br />
ranking r i = (j 1 , ..., j K ) . We simulate the utilities conditional on the rankings by simulating the<br />
utility <strong>of</strong> kth choice conditional on the simulated utility <strong>of</strong> (k − 1)st choice. For the first choice,<br />
draws are simulated unconditionally.<br />
A.1 Ranked Choices<br />
Calculation <strong>of</strong> probabilities. <strong>The</strong> probability that ε ij1 is less than ¯s conditional on u ij1 being<br />
the largest in a set J when ε ij1 ≤ ¯s can be written as<br />
P (ε ij1 ≤ ¯s|u ij1 = max u ij ) = P (u ij 1<br />
= max u ij |ε ij1 ≤ ¯s) F (¯s)<br />
P (u ij1 = max u ij )<br />
=<br />
F j1 (¯s) F (¯s)<br />
P (u ij1 = max u ij ) .<br />
<strong>The</strong> quantities F (¯s) and P (u ij1 = max u ij ) are known. To compute F j1 (¯s), we have<br />
F j1 (¯s) =<br />
=<br />
=<br />
=<br />
∫ ¯s<br />
∏<br />
s=−∞<br />
j∈J\{j 1 }<br />
∫ ¯s<br />
s=−∞<br />
∫ ¯s<br />
s=−∞<br />
∫ ∞<br />
exp(−¯s)<br />
p that V ij +ε ij
<strong>The</strong> expression for the required probability simplifies to<br />
( (∑<br />
j<br />
P (ε ij1 ≤ ¯s|u ij1 = max u ij ) = exp − exp (−¯s)<br />
exp (V ))<br />
ij)<br />
+ 1 .<br />
exp (V ij1 )<br />
Conditional on all utilities for j ∈ J being less than û. <strong>The</strong> probability that ε ij1 is less<br />
than ¯s conditional on u ij being less than û for all j ∈ J, and j 1 the largest amongst the remaining<br />
choices is given by Bayes’ <strong>The</strong>orem.<br />
{<br />
0 ¯s ≥ û − V ij<br />
P (ε ij1 ≤ ¯s|u ij ≤ û, u ij1 = max u ij ) =<br />
Fij û<br />
1<br />
(¯s) otherwise<br />
Fij û<br />
1<br />
(¯s) = P (u ij 1<br />
= max u ij |u ij ≤ û, ε ij1 ≤ ¯s) P (u ij ≤ û, ε ij1 ≤ ¯s)<br />
P (u ij1 = max u ij |u ij < û) P (u ij ≤ û)<br />
= P (u ij 1<br />
= max u ij |u ij ≤ û, ε ij1 ≤ ¯s) F (¯s)<br />
P (u ij1 = max u ij |u ij ≤ û) F (û − V ij1 ) .<br />
Notice that P (u ij1 = max u ij |u ij ≤ û) is just P (u ij1 = max u ij |u ij ≤ û, ε ij1 ≤ ¯s) evaluated at<br />
¯s = û − V ij . <strong>The</strong>refore, we need to compute P (u ij1 = max u ij |u ij ≤ û, ε ij1 ≤ ¯s) . Since the event<br />
u ij1 = max u ij |ε ij1 ≤ ¯s is contained in the event u ij ≤ û, for ¯s ≤ û − V ij1 , we have that<br />
P (u ij1 = max u ij |u ij ≤ û, ε ij1 ≤ ¯s) P (u ij ≤ û) = P (u ij1 = max u ij |ε ij1 ≤ ¯s) . <strong>The</strong>refore,<br />
F û<br />
ij 1<br />
(¯s) =<br />
P (u ij1 = max u ij |ε ij1 ≤ ¯s) F (¯s)<br />
P (u ij1 = max u ij |ε ij1 ≤ û − V ij1 ) F (û − V ij1 )<br />
(<br />
∑<br />
j<br />
(exp (− (û − V ij1 )) − exp (−¯s))<br />
exp (V ij)<br />
exp (V ij1 )<br />
= exp<br />
)<br />
F (¯s)<br />
F (û − V ij1 ) .<br />
Inverse CDF. For simulations, we need the inverse cdf function.<br />
(<br />
∑<br />
F ûj<br />
j<br />
1<br />
(¯s) = exp {exp (− (û − V ij1 )) − exp (−¯s)}<br />
exp (V )<br />
ij) F (¯s)<br />
exp (V ij1 ) F (û − V ij1 )<br />
(<br />
(∑<br />
j<br />
= exp {exp (− (û − V ij1 )) − exp (−¯s)}<br />
exp (V ))<br />
ij)<br />
+ 1<br />
exp (V ij1 )<br />
{<br />
(∑<br />
j<br />
⇒ ¯s = − ln exp (− (û − V ij1 )) −<br />
exp (V −1<br />
}<br />
ij)<br />
+ 1)<br />
ln F ûj exp (V ij1 )<br />
1<br />
(¯s)<br />
Clearly, at F ûj 1<br />
(¯s) = 1, ¯s = û − V ij1<br />
as desired.<br />
A.2 Unranked Programs<br />
For a program that wasn’t ranked, we know that the utility for this program is less than all programs<br />
ranked by the student. Let û be the lowest utility from ranked programs. <strong>The</strong> probability<br />
distribution <strong>of</strong> ε ij <strong>of</strong> unranked programs is given by<br />
F û<br />
ij (¯s) =<br />
{<br />
0<br />
F (¯s)<br />
F (û−V ij )<br />
¯s > û − V ij<br />
otherwise .<br />
<strong>The</strong> value <strong>of</strong> û can be simulated using the procedure above.<br />
57
B<br />
Appendix: Subway Distances<br />
In New York, high school students who live within 0.5 miles <strong>of</strong> a school are not eligible for<br />
transportation. If a student lives between 0.5 and 1.5 miles the Metropolitan Transit Authority<br />
provides them with a half-fare student Metrocard that works only for bus transportation. If they<br />
reside 1.5 miles or greater, they obtain full fare transportation with a student Metrocard that<br />
works for subways and buses and is issued by the school transportation <strong>of</strong>fice.<br />
Since subway is a common mode <strong>of</strong> transportation in New York City, this appendix assesses<br />
how the driving distance measure we utilize in the paper differs from commuting distance using<br />
NYC’s subway system. Subway distance is defined as the sum <strong>of</strong> distance on foot to the student’s<br />
nearest subway station, travel distance on the subway network to a school’s nearest subway<br />
station, and the distance on foot to the school from that station. To compute these distances,<br />
we used ESRI’s ArcGIS s<strong>of</strong>tware and information on the NYC subway system using GIS files<br />
downloaded from Metropolitan Transit Authority’s website. Details on these sources are in the<br />
Data appendix.<br />
<strong>The</strong> overall correlation between driving distance and total commuting distance for all studentprogram<br />
pairs is 0.96. A regression <strong>of</strong> commuting distance on driving distance yields a coefficient<br />
<strong>of</strong> 0.77. Table S1 provides a summary <strong>of</strong> the correlations by the student and school borough.<br />
<strong>The</strong> correlations are higher than 0.84, except for schools in Staten Island, where the subway<br />
system is not quite as extensive as in other boroughs. In fact, it may be that driving distances<br />
are a more accurate measure <strong>of</strong> travel costs in Staten Island than subway distance.<br />
Panels B and C shows that most students are assigned to schools in their borough in both<br />
the uncoordinated and coordinated mechanism. In both mechanisms, a very small number <strong>of</strong><br />
students that do not live in Staten Island travel are assigned to schools there and conversely, only<br />
a small number <strong>of</strong> students living in Staten Island are assigned to school in a different borough.<br />
58
Table S1. Subway and Driving Distance and Cross-‐Borough <strong>School</strong> <strong>Assignment</strong><br />
<strong>School</strong> Borough<br />
Brooklyn Bronx Manhattan Queens Staten Island Total<br />
Student Borough (1) (2) (3) (4) (5) (6)<br />
A. Correlation between Subway and Driving Distance<br />
Brooklyn 0.91 0.90 0.95 0.91 0.92<br />
Bronx 0.93 0.90 0.97 0.91 0.76<br />
Manhattan 0.95 0.96 0.98 0.95 0.76<br />
Queens 0.91 0.91 0.95 0.87 0.85<br />
Staten Island 0.92 0.84 0.85 0.89 0.73<br />
B. Cross-‐Borough Travel in Uncoordinated Mechanism<br />
Brooklyn 20,877 13 1,073 502 12 22,477<br />
Bronx 41 15,187 1,382 66 1 16,677<br />
Manhattan 42 89 8,604 24 1 8,760<br />
Queens 493 15 586 16,498 0 17,592<br />
Staten Island 13 2 59 4 4,774 4,852<br />
C. Cross-‐Borough Travel in <strong>Coordinated</strong> Mechanism<br />
Brooklyn 20,035 39 1,858 846 40 22,818<br />
Bronx 85 13,335 2,049 84 8 15,561<br />
Manhattan 108 238 7,492 52 7 7,897<br />
Queens 584 26 1,028 14,972 9 16,619<br />
Staten Island 37 3 69 4 3,913 4,026<br />
Notes: Panel A reports on the correlation between student-‐school distance as computed by travel distance and by subway <br />
distance. Subway distance is the sum <strong>of</strong> distance on foot to the student's nearest subway station, travel distance on the <br />
subway network to a school's nearest subway station, and the distance on foot to the school from that location. Both measures <br />
<strong>of</strong> distance were computed using ArcGIS. Panels B and C report on the number <strong>of</strong> students in each borough who are <strong>of</strong>fered a <br />
school in each borough.
C<br />
Appendix: Data<br />
<strong>The</strong> data for this study come from the NYC Department <strong>of</strong> Education (DOE), the 2000 US Census,<br />
ArcGIS Business Analyst toolbox and GFTS NYC subway data from the NYC Metropolitan<br />
Transit Authority. <strong>The</strong>se sources provide us with data on students, schools, the rank-order lists<br />
submitted by the students, the assignment <strong>of</strong> students to schools or the distance between the<br />
students and schools on the road network or the subway system. Students and programs are<br />
uniquely identified by a number that can be used to populate fields and merge across DOE<br />
datasets. We geocode student and school addresses to merge with geo-spatial data.<br />
We also use three samples <strong>of</strong> students in our analysis: one sample to estimate demand and<br />
the other two to infer the welfare effects <strong>of</strong> the mechanism change. <strong>The</strong> welfare samples consists<br />
<strong>of</strong> public middle schools student that matriculate into NYC Public High <strong>School</strong>s in the academic<br />
years 2003-04 and 2004-05. <strong>The</strong> demand sample consists <strong>of</strong> public middle school students that<br />
participated in the main round <strong>of</strong> the mechanism in 2003-04. <strong>The</strong> demand sample and the welfare<br />
sample from 2003-04 are not nested because students participating in the mechanism may choose<br />
to enroll in a school outside the NYC Public <strong>School</strong> system whereas others may be assigned to<br />
a public school outside the main assignment process.<br />
C.1 Students<br />
<strong>Assignment</strong> and Rank Data<br />
Data on the assignment system come from the DOE’s enrollment <strong>of</strong>fice. <strong>The</strong> files indicate the<br />
final assignment <strong>of</strong> all students in both years in our analysis. We use these assignments as<br />
the basis <strong>of</strong> our baseline welfare calculations. In addition, the assignment system also provides<br />
separate files that detail the rank orders, applications or processes through which a student is<br />
assigned to a given school.<br />
We use the records from the main round in the new mechanism to obtain the rank order<br />
lists submitted by students and the assignment proposed by the mechanism. A total <strong>of</strong> 87,355<br />
students participated in the main round.<br />
For the old mechanism, the assignment system provides student choice and decision files for<br />
the main round. <strong>The</strong> former contains the ranked applications submitted by the students and the<br />
latter provides the decisions <strong>of</strong> the schools to accept/reject/waitlist the student and the students<br />
response to these <strong>of</strong>fers, if any. A total <strong>of</strong> 84,272 students participated in the main round.<br />
<strong>The</strong> old assignment system also contains several files documenting the supplementary variable<br />
assignment process (VAS) round.<br />
<strong>Assignment</strong> Rounds and Offers in the Old Mechanism<br />
<strong>The</strong> files in the old mechanism do not directly contain information on how students were assigned<br />
to their programs. However, we are able to determine whether a student applied to a particular<br />
program/school in the main process or the supplementary VAS process. We first append fields<br />
indicating whether a student applied to her assigned program in the main process. We also<br />
append a field indicating whether a student applied to her assigned school in the supplementary<br />
60
VAS process. It turns out that no final assignment appears in both the main and the VAS files.<br />
We therefore categorize the former assignments as main-round assignments and the latter as VAS<br />
assignments. We assume that all other assignments are through the over the counter process.<br />
Based on conversations with DOE <strong>of</strong>ficials, students were typically assigned to school closest to<br />
home that had open seats. Our understanding is that most <strong>of</strong> the students that participated in<br />
the VAS process did not have a default local school. An analysis <strong>of</strong> the geographic distribution<br />
<strong>of</strong> our definition <strong>of</strong> students assigned over the counter is consistent with this fact: many parts <strong>of</strong><br />
NYC have no students assigned through the over the counter process.<br />
Finally, we also append the number <strong>of</strong> <strong>of</strong>fers made to a particular student using a file with<br />
the initial response <strong>of</strong> schools to the student application.<br />
<strong>Assignment</strong> Rounds in the New Mechanism<br />
We use the NYC assignment files described above to determine the process through which a<br />
student was assigned a given school.<br />
<strong>The</strong> assignment files in the new mechanism contain, for every student program pair that<br />
is ranked in either the main round or the supplementary round, two fields indicating whether<br />
the student is eligible for the school and if the student was assigned to that school. A final<br />
assignment is treated as a main-round assignment if it appears as an eligible assignment in the<br />
main round. <strong>Assignment</strong>s that are not made in the main round are treated as a supplementary<br />
round assignment if they appear in the supplementary round files. All other assignments are<br />
treated as over the counter assignments.<br />
Student Characteristics<br />
<strong>The</strong> records from the NYC Department <strong>of</strong> Education contain the street address, previous and<br />
current grade, gender, ethnicity and whether the student was enrolled in a public middle school.<br />
Each student is identified by a unique number that allows us to merge these data with additional<br />
data from the NYC DOE on a student’s scores in middle school standardized tests, Limited<br />
English Pr<strong>of</strong>iciency status, and Special Education status. A separate file indicates subsidized<br />
lunch status as <strong>of</strong> the 2004-05 enrollment. If a student is not in that file, we code the student as<br />
not receiving a subsidized lunch.<br />
<strong>The</strong>re are several standardized tests taken by middle school students in NYC. To avoid the<br />
concern that two different tests may not be comparable indicators <strong>of</strong> student achievement, we<br />
identify the modal standardized tests in math and reading taken by students in our sample. <strong>The</strong>se<br />
are the May tests with codes CTB and TEM respectively. Of the students that did not take<br />
either <strong>of</strong> these tests in May, at most 10% (
fractions are 7.13% and 12.56%.<br />
Geographic Data<br />
We use the 2000 US Census to obtain block group family income. <strong>The</strong> addresses <strong>of</strong> students<br />
and their distance to school were calculated using ArcGIS. Corrections to the addresses, when<br />
necessary, were made using Google Map Tools followed by manual checks and corrections.<br />
<strong>The</strong> final set <strong>of</strong> addresses were geocoded using ArcGIS geocoder with the address-set in the<br />
Business Analyst toolbox (ver. 10.0). We first used an exact match to determine if a student’s<br />
address can be geocoded precisely to a ro<strong>of</strong>top. If the results were unreliable, we coded the<br />
student to the centroid <strong>of</strong> the zip-code. <strong>The</strong> vast majority <strong>of</strong> students were placed at the ro<strong>of</strong>top<br />
level. <strong>The</strong> OD Cost matrix tool in the Network Analyst toolbox was used to compute the<br />
distance by road for each student-school pair. <strong>The</strong> road network is also obtained from Business<br />
Analyst.<br />
Our computation <strong>of</strong> subway distances assumes that a student first walks to the closest subway<br />
stop, then uses the subway system to travel to the subway stop closest to the school, and finally<br />
walks from the subway to the school. <strong>The</strong> location <strong>of</strong> the subway stops is taken from the GTFS<br />
and geodata data on the NYC Metropolitan Transit Authority website. <strong>The</strong> network analyst<br />
toolbox is used to compute the walking distance and the GTFS data is used to compute the<br />
distance on the subway system between every pair <strong>of</strong> subway stops.<br />
Merging Student Records<br />
<strong>Assignment</strong> and other DOE files are matched using the unique student identifier linking these<br />
data. Each eighth-grade non-private middle school student in the Department <strong>of</strong> Education<br />
records could be merged uniquely with a student in the NYC assignment records. Less than<br />
0.45% <strong>of</strong> students with known assignments in the records <strong>of</strong> the NYC assignment system could<br />
not be merged with a student in the DOE records. <strong>The</strong>se students were not included in the<br />
analysis.<br />
C.2 Applicant Sample Construction<br />
Our goal is to consider first-time applicants to the NYC public (unspecialized) high school system<br />
that live in New York City attend a public middle school 8th grade. Below, we described the<br />
procedure used to construct the samples. <strong>The</strong> selection procedure is also summarized in Table<br />
B1.<br />
<strong>Welfare</strong> Sample<br />
<strong>The</strong> welfare samples is constructed from the NYC Department <strong>of</strong> Education’s records <strong>of</strong> all<br />
students enrolling in ninth grade at a high school in academic years 2003-04 and 2004-05.<br />
As our choice set in the demand analysis will be restricted to unspecialized, non-charter high<br />
schools in the public school system, we do not include students that matriculated to such schools<br />
in the welfare sample.<br />
62
Of the 92,623 eighth grade students matriculating into ninth grade at a NYC public school<br />
in 2002-03, 11,790 (12.73%) students went to a private middle school and were dropped. Another<br />
8,051 (8.69%) <strong>of</strong> students were not included in the analysis because their assignment was<br />
unknown, or because they matriculated at either a specialized high school or a charter school.<br />
Finally, we exclude students in schools that were closed (no assignments in the new system).<br />
In 2003-04, about 1.3% students had also participated in the old mechanism, presumably<br />
because these students repeated eighth grade. <strong>The</strong>se students were considered a part <strong>of</strong> the<br />
2002-03 sample and only their 2002-03 assignment into high school is considered in our analysis.<br />
We also drop private middle school students and those not assigned to public school. <strong>The</strong>se<br />
fractions were similar to the 2002-03 numbers, at 12.21% and 8.13% respectively. We also drop<br />
students that were assigned to new schools.<br />
<strong>The</strong>se selections into the sample leave us with 70,358 students in 2002-03 and 66,921 students<br />
in 2003-04. Students that may have been assigned to a high school program through a process<br />
other than the main round are included in these samples.<br />
Demand Sample<br />
This sample is sourced from the NYC <strong>Assignment</strong> system’s records on the participants in the<br />
main round <strong>of</strong> the mechanism. As discussed in the text, we use only data only from the main<br />
round <strong>of</strong> the mechanism as this round has the most desirable incentive properties.<br />
We do not want to exclude students on the basis <strong>of</strong> final assignment to avoid selecting on the<br />
choice to leave the public school system. In order to most closely match the construction <strong>of</strong> the<br />
welfare sample, we select the demand sample only on characteristics that can be considered as<br />
exogenous at the time <strong>of</strong> participation.<br />
Since we focus on first-time applicants in eighth grade, we exclude 747 students that were<br />
part <strong>of</strong> the 2002-03 files, and 5,311 students that were ninth graders. Presumably, these students<br />
were held back in eighth or ninth grade. This leaves us with a sample <strong>of</strong> 81,297 eighth grade<br />
students.<br />
Of the eighth-grade participants, 9,301 or 11.44% <strong>of</strong> students were from private middle schools<br />
and were dropped. We also excluded students designated as belonging to the top 2% <strong>of</strong> their<br />
middle school class as they are prioritized at education option schools, creating incentives to<br />
misreport their preferences. <strong>The</strong>se are 2.5% <strong>of</strong> the non-private eighth grade population.<br />
A total <strong>of</strong> 216 students did not rank any public schools in our sample. After excluding these<br />
students, a total <strong>of</strong> 69,907 students remain in the sample we will use for the demand analysis.<br />
C.3 Programs/<strong>School</strong>s<br />
NYC Department <strong>of</strong> Education <strong>School</strong> Report Cards<br />
<strong>The</strong> school characteristics were taken from the report card files provided by the NYC Department<br />
<strong>of</strong> Education. <strong>The</strong>se data provide information on a school’s enrollment statistics, racial<br />
composition <strong>of</strong> student body, attendance rates, suspensions, teacher numbers and experience,<br />
and Regents Math and English performance <strong>of</strong> the graduating class. A unique identifier for each<br />
school allows these data to be merged with data from our other sources.<br />
63
<strong>The</strong>re were significant differences in the file formats and field names across the years. To<br />
keep the school characteristics constant across years, we use the data from the 2003-04 report<br />
cards as the primary source. Except for data on the math and reading achievement, variable<br />
descriptions were comparable across years. For these comparable variables, we used the 2002-03<br />
data only when the 2003-04 data were not available. <strong>The</strong> coverage <strong>of</strong> the characteristics for the<br />
sample <strong>of</strong> schools is enumerated in Table B2.<br />
<strong>Assignment</strong> System and DOE files<br />
<strong>Assignment</strong> data contain a list <strong>of</strong> all school programs in the public school system along with<br />
an identifier for the associated high school. <strong>The</strong> department <strong>of</strong> education provided a separate<br />
file with data on the school addresses and identifiers that allow a merge with the assignment<br />
system database. A second identifier can be used to merge these data with other fields in the<br />
department <strong>of</strong> education records described above.<br />
Across the two years, the high school identifiers in the files were inconsistent for a small<br />
number <strong>of</strong> schools in our sample. <strong>The</strong>se were matched by name and address <strong>of</strong> the school. One<br />
school moved from Brooklyn to Manhattan and was investigated to ensure that the records were<br />
appropriately matched.<br />
Program Characteristics<br />
Program characteristics are taken from the DOE’s High <strong>School</strong> Directory, which is made available<br />
to students before the application process. Reliable data on program types was not available<br />
in 2002-03. For that year, the program types were imputed from the 2003-04 program types if<br />
the program was present in both years. Otherwise, the program was categorized as a general<br />
program.<br />
<strong>The</strong>re were a very large number <strong>of</strong> program types. <strong>The</strong>se were aggregated into fewer broad<br />
categories. <strong>The</strong> items in the list below are the aggregated categories that include all <strong>of</strong> the<br />
subcategories as described by the data.<br />
1. Arts: Dance, Instrument Performance, Musical <strong>The</strong>ater, Performing and Visual, Performing<br />
Arts, <strong>The</strong>ater, <strong>The</strong>ater Tech, Visual Arts, Vocal Performance.<br />
2. Humanities/Interdisciplinary: Education, Humanities/Interdisciplinary.<br />
3. Business/Accounting: Accounting, Business, Business Law, Computer Business, Finance,<br />
International Business, Marketing, Travel Business.<br />
4. Math/Science: Engineering, Engineering – Aerospace, Engineering – Electrical, Environmental,<br />
Math and Science, Science and Math.<br />
5. Career: Architecture, Computer Tech, Computerized Mech, Cosmetology, Journalism, Veterinary,<br />
Vision Care Technology.<br />
6. Vocational: Auto, Aviation, Clerical, Construction, Electrical Construction, Health, Heating,<br />
Hospitality, Plumbing, Transportation.<br />
64
7. Government/law: Law, Law Enforcement, Law and Social Justice, Public Service.<br />
8. Other: Communication, Expeditionary, Preservation, Sports.<br />
9. Zoned<br />
10. General: General, Unknown.<br />
Finally, some programs adopt a language <strong>of</strong> instruction other than English. We categorized<br />
the languages into Spanish, English, Asian Languages and Other.<br />
C.4 <strong>School</strong> Sample Construction<br />
We consider the assignment <strong>of</strong> eighth grade students in NYC public middle schools into public<br />
high schools that are not charters, specialized or parochial. Our analysis uses two school sample,<br />
one for each year in our analysis.<br />
To construct these samples, we started with the set <strong>of</strong> schools and programs in the assignment<br />
records. For the analysis <strong>of</strong> rank data, we added the set <strong>of</strong> school programs that were ranked by<br />
any student in our demand sample. This initial set consists <strong>of</strong> 743 (301) programs (schools) in<br />
2002-03 and 677 (293) programs (schools) in 2003-04.<br />
In 2003-04, this list contained 62 parochial school programs. We verified that each <strong>of</strong> the 130<br />
students matriculating to these school programs were private middle-schoolers. <strong>The</strong>se schools<br />
were dropped from the analysis because private middle-schoolers are not in population <strong>of</strong> interest.<br />
Subsequently, we dropped all charter and specialized high school programs and other school<br />
programs which do not have assignments and were not ranked by any student in our sample.<br />
A total <strong>of</strong> 9 continuing student programs accepted students only from their associated middle<br />
school. Since these programs cannot be chosen by students that were not in that school in<br />
eighth grade, we combine these programs with a generic program (e.g., unscreened, English, general/humanities/math).<br />
Rank order lists for students that ranked both the continuing students<br />
only program and the associated program were modified as described below.<br />
Finally, we dropped new and closed schools from the analysis. Closed schools were ones that<br />
admitted students in 2002-03, but not in 2003-04. <strong>The</strong> set <strong>of</strong> new schools was collected from<br />
a separate DOE directory <strong>of</strong> new schools. <strong>The</strong>se schools were not well advertised and very few<br />
students ranked them, making calculations with those schools unreliable.<br />
<strong>The</strong> number <strong>of</strong> schools and programs at each stage <strong>of</strong> our selection procedure is also summarized<br />
in Table B2.<br />
C.5 Program Capacities<br />
Program capacities are not provided separately in the data files. We have estimated program<br />
capacities from the actual match files and students’ final assignments. <strong>The</strong> capacity <strong>of</strong> each<br />
program is initially set to zero. If a student in our demand sample is assigned a program at the<br />
end <strong>of</strong> the assignment process, the capacity <strong>of</strong> the program is increased by one. Otherwise, if<br />
the student is assigned a program in the main round the capacity <strong>of</strong> the program is increased<br />
65
y one. Finally, if a student is not assigned in the main round and assigned a program in the<br />
supplementary round, the capacity <strong>of</strong> the program is increased by one.<br />
Education Option programs are divided into six buckets: High Select, High Random, Middle<br />
Select, Middle Random, Low Select and Low Random. <strong>The</strong> bucket capacities are calculated as<br />
above by taking into account the category <strong>of</strong> the assigned student. For example, if a student <strong>of</strong><br />
High category is assigned an Education Option program, then the capacity <strong>of</strong> a High bucket is<br />
increased by one. If the current capacity <strong>of</strong> the High Select bucket is less than or equal to that <strong>of</strong><br />
High Random, then the capacity <strong>of</strong> the High Select bucket is increased, otherwise the capacity<br />
<strong>of</strong> the High Random bucket is increased.<br />
C.6 Program Priorities<br />
<strong>The</strong> type <strong>of</strong> a program determines how students are priority-ordered for the program. <strong>The</strong><br />
data contains a list <strong>of</strong> all programs with program-specific information, including type, building<br />
number, street address etc. When students have the same priority, the tie is broken randomly.<br />
<strong>The</strong> random numbers are generated by computer during our simulations.<br />
<strong>The</strong> assignment data contains the following fields that determine a student’s priority order at<br />
programs. Priority group is a number assigned by the NYC Department <strong>of</strong> Education depending<br />
on students’ home addresses and location <strong>of</strong> programs etc. High school rank is a number assigned<br />
by each program. This may reflect an student’s ranking among all applicants to an Education<br />
Option program, or whether a student attended the information session <strong>of</strong> an limited unscreened<br />
program, etc. <strong>The</strong>se fields are provided for every student at every program that the student<br />
ranked. Students applying to Educational Option programs are placed into one <strong>of</strong> three categories<br />
based on their score on the 7th grade reading test: top 16 percent (high), middle 68 percent<br />
(middle), and bottom 16 percent (low). Student categories in the assignment data.<br />
Unscreened programs order students based on their random numbers only. Limited unscreened<br />
and formerly zoned programs order students first by priority group, and random number<br />
within priority group. Screened programs order students by priority group, then by high school<br />
rank. Each Education Option program orders all applicants for each <strong>of</strong> six buckets, High Select,<br />
High Random, Middle Select, Middle Random, Low Select and Low Random. A high bucket<br />
orders high category students first, then middle category students, then low category students.<br />
A middle bucket orders middle category students first, then high category students, then low<br />
category students. A low bucket orders low category students first, then high category students,<br />
then middle category students. A select bucket orders students within each category by priority<br />
order, then by high school rank. A random bucket orders students within each category by<br />
priority order.<br />
C.7 Miscellaneous Issues<br />
Modifications to the rank order list<br />
1. Some students ranked a program that were either charter schools or specialized high schools<br />
in the main round. <strong>The</strong>se programs are not in the sample <strong>of</strong> schools we consider and were<br />
likely ranked by the students in error. In such cases, programs were removed from the rank<br />
66
order lists and rank orders lists were made contiguous where all programs ranked below a<br />
program not in the sample were moved up in the rank order lists. <strong>The</strong>se programs were<br />
observed a total <strong>of</strong> 795 times in the data. Thirty students ranked only charter or specialized<br />
programs.<br />
2. <strong>The</strong> rank order lists <strong>of</strong> students that ranked continuing student program were modified as<br />
follows: First, the lists <strong>of</strong> all students that ranked only the continuing student program<br />
were modified so that the student ranked the associated generic program instead. When<br />
students ranked both the generic program and the associated continuing student program,<br />
the list was modified so that only the associated program was ranked, and at the highest <strong>of</strong><br />
the two ranked positions. All programs ranked at positions below the lower ranked <strong>of</strong> the<br />
two programs were moved up by one. A total <strong>of</strong> 46 students ranked both the continuing<br />
program and the generic program we mapped the continuing program to. In 17 cases, these<br />
ranks were not consecutive.<br />
67
Table B1. Student Sample Selection<br />
Mechanism Comparison<br />
Demand Estimation<br />
Uncoordinated <strong>Coordinated</strong> <strong>Coordinated</strong><br />
(1) (2) (3)<br />
Number in the NYC DOE student file 100,669 97,569<br />
Number <strong>of</strong> students in the rank data 87,355<br />
Excluding students in both 2002-‐03 and 2003-‐04 files from 2003-‐04 96,275 86,608<br />
Excluding ninth grade students 92,623 89,062 81,297<br />
Excluding private middle school students 80,833 78,183 71,996<br />
Excluding students with addresses outside the five boroughs 80,725 78,089 71,916<br />
Total number <strong>of</strong> students with known assignments to sample schools 75,515 73,989<br />
Excluding students attending specialized high schools 72,725 70,992<br />
Excluding students attending charter schools 72,681 70,886<br />
Excluding students in closed and new new schools 70,358 66,921<br />
Excluding top 2% students 70,123<br />
Excluding students that did not rank any sample schools 69,907<br />
Notes: Uncoordinated mechanism refers to 2002-‐03 mechanism and coordinated mechanism refers to the 2003-‐04 mechanism based on deferred acceptance. A student has invalid census <br />
information if address is missing, cannot be geocoded or places the student outside <strong>of</strong> New York City. A distance observation is invalid if it is missing or is greater than 65 miles.
Table B2. Construction <strong>of</strong> <strong>School</strong> Sample<br />
Uncoordinated Mechanism<br />
<strong>Coordinated</strong> Mechanism<br />
Programs <strong>School</strong>s Programs <strong>School</strong>s<br />
(1) (2) (3) (4)<br />
Programs where NYC public school students assigned 743 301 669 293<br />
Adding additional programs ranked by students 677 294<br />
Excluding parochial schools 681 239 677 294<br />
Excluding specialized schools 669 232 665 287<br />
Excluding charter schools 667 230 663 285<br />
Excluding programs with no assignments or ranking 637 225 648 284<br />
Combining continuing education programs 637 225 639 284<br />
Excluding new and closed schools 612 215 558 235<br />
Programs/schools ranked by students in sample 497 234<br />
Notes: Uncoordinated mechanism refers to 2002-‐03 mechanism and coordinated mechanism refers to the 2003-‐04 mechanism based on deferred acceptance. 13 continuining student <br />
programs were merged with a generic program at host school. Parochial schools in 2002-‐03 only have private middle school students assigned to them and are not ranked by students in <br />
the demand sample.
Table B3. Coverage <strong>of</strong> <strong>School</strong> Characteristics<br />
2002‐03 2003‐04 Both Years<br />
(1) (2) (3)<br />
Total number <strong>of</strong> schools in the sample 215 234 215<br />
Size <strong>of</strong> 9th grade 196 199 189<br />
Race 196 199 189<br />
Attendance Rate 196 199 189<br />
Percent Subsidized Lunch 196 198 189<br />
Percent <strong>of</strong> teachers less than 2 years experience 219 223 212<br />
High Math Achievement 198 200 191<br />
High English Achievement 180 177 173<br />
Percent Attending College 171 167 165<br />
Notes: Uncoordinated mechanism refers to 2002‐03 mechanism and coordinated mechanism refers to the mechanism<br />
based on deferred acceptance used in 2003‐04. Table reports the fraction <strong>of</strong> schools with the characteristics from New<br />
York State Report cards.
Table C1. All Coefficients from Student Preference Estimates<br />
<strong>School</strong> Characteristics x <br />
<strong>School</strong> Characteristics x Student Demographics<br />
<strong>School</strong> Characteristics x Student Baseline <br />
<strong>School</strong> Characteristics x Student Baseline Achievement<br />
Specifications: <strong>School</strong> Characteristics Student Race<br />
Achievement<br />
All Choices Top Choice Only Top Three Choices<br />
(1) (2) (3) (4) (5) (6)<br />
A. Preference for Distance<br />
Distance -‐0.318*** -‐0.313*** -‐0.316*** -‐0.310*** -‐0.387*** -‐0.353***<br />
B. Preference for <strong>School</strong> Characteristics<br />
Size <strong>of</strong> 9th grade (in 100s)<br />
Main effect 0.015*** 0.040*** 0.015*** 0.059*** 0.100*** 0.099***<br />
Female -‐0.007*** -‐0.009*** -‐0.008***<br />
Asian 0.019*** 0.016*** 0.052*** 0.034***<br />
Black -‐0.040*** -‐0.038*** -‐0.042*** -‐0.044***<br />
Hispanic -‐0.018*** -‐0.019*** -‐0.016*** -‐0.021***<br />
Baseline Math 0.004*** 0.001** 0.002 0.001<br />
Baseline English -‐0.006*** -‐0.003*** -‐0.009*** -‐0.005***<br />
No Baseline Math Score -‐0.027*** -‐0.010*** -‐0.033*** -‐0.019***<br />
No Baseline English Score 0.035*** 0.008*** 0.027*** 0.015***<br />
Subsidized Lunch -‐0.014*** -‐0.029*** -‐0.024***<br />
Neighborhood Income (in 1000s) -‐0.650*** -‐1.930*** -‐1.930***<br />
Limited English Pr<strong>of</strong>icient 0.007*** 0.018*** 0.016***<br />
Special Education -‐0.007*** -‐0.017*** -‐0.012***<br />
Percent White<br />
Main effect 0.026*** 0.040*** 0.025*** 0.032*** 0.040*** 0.033***<br />
Female -‐0.001** -‐0.001 0.000<br />
Asian -‐0.016*** -‐0.015*** -‐0.027*** -‐0.018***<br />
Black -‐0.022*** -‐0.020*** -‐0.035*** -‐0.024***<br />
Hispanic -‐0.011*** -‐0.009*** -‐0.016*** -‐0.010***<br />
Baseline Math 0.001*** 0.000* 0.000 0.000<br />
Baseline English 0.002*** 0.002*** 0.005*** 0.003***<br />
No Baseline Math Score -‐0.002*** 0.000 -‐0.003 -‐0.001<br />
No Baseline English Score 0.004*** 0.000 0.004* 0.000<br />
Subsidized Lunch 0.001*** 0.003*** 0.002***<br />
Neighborhood Income (in 1000s) 0.911*** 2.000*** 1.000***<br />
Limited English Pr<strong>of</strong>icient 0.000 0.000 0.001<br />
Special Education 0.001 0.006*** 0.003***
Table C1. All Coefficients from Student Preference Estimates (cont.)<br />
<strong>School</strong> Characteristics x <br />
<strong>School</strong> Characteristics x Student Demographics<br />
<strong>School</strong> Characteristics x Student Baseline <br />
<strong>School</strong> Characteristics x Student Baseline Achievement<br />
Specifications: <strong>School</strong> Characteristics Student Race<br />
Achievement<br />
All Choices Top Choice Only Top Three Choices<br />
(1) (2) (3) (4) (5) (6)<br />
B. Preference for <strong>School</strong> Characteristics (cont.)<br />
Attendance Rate<br />
Main effect 0.055*** 0.103*** 0.062*** 0.075*** 0.131*** 0.117***<br />
Female 0.002** 0.009*** 0.005***<br />
Asian 0.013*** 0.016*** 0.076*** 0.039***<br />
Black -‐0.053*** -‐0.036*** -‐0.040*** -‐0.046***<br />
Hispanic -‐0.053*** -‐0.037*** -‐0.040*** -‐0.047***<br />
Baseline Math 0.012*** 0.010*** 0.017*** 0.015***<br />
Baseline English 0.014*** 0.014*** 0.020*** 0.021***<br />
No Baseline Math Score -‐0.024*** -‐0.016*** -‐0.046*** -‐0.026***<br />
No Baseline English Score 0.010*** -‐0.004** 0.009* 0.000<br />
Subsidized Lunch 0.000 -‐0.008*** -‐0.005***<br />
Neighborhood Income (in 1000s) 6.000*** 8.000*** 7.000***<br />
Limited English Pr<strong>of</strong>icient 0.006*** 0.014*** 0.012***<br />
Special Education 0.004*** -‐0.008* 0.005*<br />
Percent Subsidized Lunch<br />
Main effect -‐0.002*** -‐0.007*** -‐0.003*** -‐0.004*** 0.005*** -‐0.003**<br />
Female -‐0.001*** 0.000 0.000<br />
Asian 0.000 -‐0.001 -‐0.004*** -‐0.001<br />
Black 0.002*** 0.001** -‐0.008*** -‐0.002***<br />
Hispanic 0.010*** 0.009*** 0.007*** 0.010***<br />
Baseline Math -‐0.001*** -‐0.001*** 0.000 -‐0.001**<br />
Baseline English -‐0.001*** -‐0.001*** -‐0.001* -‐0.001**<br />
No Baseline Math Score -‐0.001 0.001** 0.000 0.002<br />
No Baseline English Score 0.005*** 0.002*** 0.005*** 0.003***<br />
Subsidized Lunch 0.000 -‐0.002** -‐0.001***<br />
Neighborhood Income (in 1000s) -‐0.722*** -‐1.000*** -‐0.736***<br />
Limited English Pr<strong>of</strong>icient 0.000 0.003*** 0.002***<br />
Special Education 0.001*** -‐0.001 0.001
Table C1. All Coefficients from Student Preference Estimates (cont.)<br />
<strong>School</strong> Characteristics x <br />
<strong>School</strong> Characteristics x Student Demographics<br />
<strong>School</strong> Characteristics x Student Baseline <br />
<strong>School</strong> Characteristics x Student Baseline Achievement<br />
Specifications: <strong>School</strong> Characteristics Student Race<br />
Achievement<br />
All Choices Top Choice Only Top Three Choices<br />
(1) (2) (3) (4) (5) (6)<br />
B. Preference for <strong>School</strong> Characteristics (cont.)<br />
High Math Achievement<br />
Main effect 0.008*** 0.007*** 0.004*** 0.000 0.001 -‐0.001<br />
Female 0.002*** 0.002*** 0.003***<br />
Asian 0.009*** 0.010*** 0.014*** 0.013***<br />
Black -‐0.004*** 0.000 0.005*** 0.002**<br />
Hispanic -‐0.001* 0.004*** 0.009*** 0.007***<br />
Baseline Math 0.008*** 0.007*** 0.012*** 0.009***<br />
Baseline English 0.005*** 0.005*** 0.006*** 0.005***<br />
No Baseline Math Score 0.000 0.001 0.008*** 0.002**<br />
No Baseline English Score 0.002*** 0.000 -‐0.005*** 0.000<br />
Subsidized Lunch -‐0.003*** -‐0.005*** -‐0.005***<br />
Neighborhood Income (in 1000s) 0.119* -‐0.712*** -‐0.242**<br />
Limited English Pr<strong>of</strong>icient -‐0.002*** -‐0.003* -‐0.005***<br />
Special Education -‐0.003*** 0.000 -‐0.001<br />
Spanish Language Program 5.147*** 5.755*** 5.472***<br />
Limited English Pr<strong>of</strong>iciency -‐3.142*** -‐3.817*** -‐3.147***<br />
Limited English Pr<strong>of</strong>iciency x Hispanic<br />
Asian Language Program 3.687*** 4.253*** 4.203***<br />
Limited English Pr<strong>of</strong>iciency -‐2.131*** -‐3.299*** -‐2.717***<br />
Limited English Pr<strong>of</strong>iciency x Asian<br />
Other Language Program 2.289*** 2.947*** 2.870***<br />
Limited English Pr<strong>of</strong>iciency<br />
Program Type Dummies X X X X X X<br />
Program Specialty Dummies X X X X X X
Table C1. All Coefficients from Student Preference Estimates (cont.)<br />
<strong>School</strong> Characteristics x <br />
<strong>School</strong> Characteristics x <br />
Student Baseline <br />
<strong>School</strong> Characteristics x Student Demographics<br />
<strong>School</strong> Characteristics x Student Baseline Achievement<br />
Specifications: <strong>School</strong> Characteristics Student Race<br />
Achievement<br />
All Choices Top Choice Only Top Three Choices<br />
(1) (2) (3) (4) (5) (6)<br />
C. Model Diagnostics<br />
Number <strong>of</strong> ranks 542,666 542,666 542,666 542,666 69,907 197,245<br />
Number <strong>of</strong> parameters 248 272 280 349 349 349<br />
Log likelihood -‐2,726,580 -‐2,714,493 -‐2,710,042 -‐2,690,008 -‐302,141 -‐906,942<br />
Variance within students 4.93 4.98 5.00 5.62 8.96 7.43<br />
Variance between students 4.68 8.23 7.79 12.26 32.85 23.48<br />
Pseudo R-‐squared 0.85 0.00 0.89 0.00 0.89 0.91<br />
Fraction <strong>of</strong> variance explained by <br />
school observables<br />
0.43 0.00 0.68 0.00 0.42 0.63<br />
Notes: This table reports all parameter estimates corresponding to Table 6. Maximum likelihood estimates from demand systems with 69,907 students and submitted ranks over 497 program choices <br />
in 234 schools. Estimates use all submitted ranks except in columns (4)-‐(6). Column (1) contains no interactions between student and school characteristics. Column (2) contains interactions <strong>of</strong> race <br />
dummies with all school characteristics. Column (3) contains interactions <strong>of</strong> Math and English score with all school characteristics. Columns (4)-‐(6) contain interactions <strong>of</strong> gender, race, achievement, <br />
special education, limited English pr<strong>of</strong>iciency, subsidized lunch, and median 2000 census block group family income with all school characteristics. Number <strong>of</strong> parameters correspond to parameters <br />
outside <strong>of</strong> delta. Fraction <strong>of</strong> variance explained by school observables is R-‐squared <strong>of</strong> projection <strong>of</strong> mean utility on school attributes. * significant at 10%; ** significant at 5%; *** significant at 1%