Full Report - Center for Collaborative Education

!
!
!
5. Limitations of the Data
While the analysis conducted for this project offers
a review of a unique combination of data and variables,
we were constrained by a number of limitations
and clerical errors present in our SIMS, MCAS,
MEPA, OELL program data, BPS student data, and
MDESE school data sources. The study relies on
administrative data that the researchers themselves
did not collect. The use of administrative data of
any type poses challenges to researchers, since one
is not able to control its collection and completeness.
Variables with limitations to note include:
• ELL Program Variable for SY2009. At the time
of our initial data request, the research team
was unaware that OELL maintained records on
the specific ELL program placement of individual
students, in greater detail than is collected via
SIMS. While access to this data was obtained,
we were not able to import the data on a match
with the randomly generated student ID number
and therefore these data were entered by hand.
Being able to link these two datasets by the ID
number would have ensured greater accuracy for
this ELL program variable. The team attempted
to mitigate errors by working with OELL to enter
students’ program placement. In addition, it is
important to note that conversations with OELL
staff revealed that there was a lack of program
fidelity during the study period. For instance,
an SEI Spanish program in one school may be
implemented very differently from an SEI Span-
Table 4: Annual High School Dropout Rates. BPS, SY2009
ish program in another school; a school that
does not offer any ELL programs could be using
SEI instructional strategies consistently in its
classrooms. The lack of adherence to systematic
program definitions means that analysis of differences
in outcomes by program may be explained
by differences in practice within (and between)
program types.
• Dropout Variable. Dropout data may be most
subject to clerical errors or subjectivity on the
part of the staff person entering the data (particularly,
for instance, when a student is coded
as a dropout whose status/location is unknown).
After these data are submitted by districts,
MDESE checks to ensure that students coded
as dropouts have not re-enrolled in another
district within the state. Dropout rates reported
by MDESE reflect this adjustment. Our dataset
does not contain this check, and therefore our
dropout rates may be overstated. However, the
research team compared our dropout rate findings
with the dropout rates reported by MDESE.
For SY2009, our data show a lower high school
dropout rate than MDESE data. We are only able to
compare BPS and MDESE data for high school dropouts,
as MDESE does not publicly release middle
school dropout data.
Our data show higher dropout rates for Grades 9
and 11 than MDESE data, but our data show lower
dropout rates for Grades 10 and 12.
Total HS N HS Dropout N HS Dropout Rate
Total BPS (Source: MDESE) 18,037 1,308 7.3%
Total BPS (Source: data received for this study) 20,781 1,426 6.9%
Source for MDESE data: MDESE (2009c)
!
!
Table 5: Annual High School Dropout Rate by Grade. BPS, SY2009
9th grade 10th grade 11th grade 12th grade
Total BPS (Source: MDESE) 6.1% 7.5% 7.2% 8.5%
Total BPS (Source: data received for this study)
Source for MDESE data: MDESE (2009c)
!
!
7.0% 6.5% 7.8% 6.5%
Table 6: Annual High School Dropout Rate for LEP and EP Students. BPS, SY2008
MDESE Data for BPS, SY2008 BPS data received for this study, SY2008
Total HS EP HS LEP HS Total HS EP HS LEP HS
Dropout Rate Dropout Rate Dropout Rate Dropout Rate Dropout Rate Dropout Rate
7.6% 7.5% 8.3% 8.3% 8.3% 8.6%
Source for MDESE data is English Language Learners Subcommittee (2009).
Improving Educational Outcomes of English Language Learners in Schools and Programs in Boston Public Schools 115