Full Report - Center for Collaborative Education

school that they attend; if students in different schools demonstrate significant differences in
MCAS scores (“between-school variance”), it indicates that school-level factors have a significant
impact on individual students’ scores. If less than 10% of the variation in scores occurs
at the school level, another type of analysis would be more appropriate. Table 47 displays the
amount of variation in students’ scores that occurs between students in comparison to the
variation in scores that occurs between schools.
Table 47. Variation in MCAS Scores, 2-level Model. BPS, SY2009
Variable Level Percent of Explained Variation
ELA Math
Elementary School Student 84.3% 88.1%
School 15.8% 11.9%
Middle School Student 76.7% 78.6%
School 23.3% 21.4%
High School Student 56.8% 70.9%
School 43.2% 29.2%
!
Since variation that occurs due to school-level factors accounts for a significant amount of
variation in individual outcomes (over 10% at every level of schooling), multi-level modeling is
appropriate for this analysis. Interestingly, variation between schools increases as the school
level increases. In other words, although individual student factors were more important in
explaining the variation in LEP student academic achievement overall, school factors become
more important as school level increases in both subjects. School factors represent 16% of
the variation in MCAS ELA scores in elementary school, increasing to nearly half of the variation
in high school (43%); in MCAS Math scores school factors represent 12% of the variation
in elementary school, increasing to 29% of the variation in high school.
Once we determined that multi-level modeling was appropriate for this analysis, we developed
a two-level hierarchical linear model examining LEP students’ educational attainment
outcomes (as measured by MCAS ELA and Math scores) in conjunction with individual-level
and school-level characteristics. Again, all LEP students in Grades 3-12 who had MCAS scores
in the appropriate subject were included in this analysis. At the individual level, the variable
set included gender, attendance rate, English proficiency as measured by the student’s MEPA
score, special education (SPED) placement, and ELL program participation. The primary advantage
of a two-level model in which ELL program participation is an individual-level variable
is that it enables us to compare the academic achievement of LEP students in ELL programs
to that of LEP students not in ELL programs. The set of variables representing the school
environment included meeting Adequate Yearly Progress (AYP) goals in either ELA or Math
as appropriate, the percentage of the school population that is low-income, the school size
(small, medium, or large), and the percentage of the school population that is of limited English
proficiency (LEP). Although mobility was found to be significant in the descriptive analysis
at both the student and school levels, it was not part of this analysis because of the high correlation
between mobility and attendance at the student level and between mobility and the
percentage of the school population that is LEP (LEP density) at the school level. In this type
of analysis, high levels of correlation mean that only one of the correlated variables could be
used; for this analysis, attendance rate was included at the student level and LEP density was
Improving Educational Outcomes of English Language Learners in Schools and Programs in Boston Public Schools 81