Full Report - Center for Collaborative Education

salient trends across time. Appropriate statistical
tests – chi-square, t-test, or Fisher’s Exact test, with
α =.05 for all – were used to determine the significance
of the differences in outcomes among populations
and among LEP students enrolled in schools
showing different characteristics and in different
types of ELL programs. Effect size was calculated
where appropriate.
As noted earlier in this Appendix, the dropout
analysis conducted for this report was done in the
separate dropout database. Annual dropout rates
were determined for students enrolled in middle
school and high school grades. 3 For summer
dropouts, behavioral, academic (namely MEPA and
MCAS), ELL program and school characteristics
data from the prior year (the last school year completed)
were assigned to the student. For instance,
students who dropped out during the summer of
2008 were assigned their SY2008 values for these
SY2009 variables. Basic frequencies and cross-tabulations
were conducted and statistical significance
was determined by running chi-square 4 tests
(α =.05) and by determining the effect size.
In addition, an in-depth analysis was conducted to
explore the impact of student-level characteristics
and school environments on individual achievement
as measured by MCAS performance in the English
Language Arts and Math. 5 We identified hierarchical
linear modeling (HLM) as the preferred method
of analysis; due to the similarity of educational
experiences between students in a particular school,
traditional multiple regression techniques would
underestimate the correlation between school-level
variables and therefore the standard error, likely
resulting in spuriously significant relationships. Variables
of interest were identified through a review
of the literature, the descriptive analyses, and in
consultation with OELL.
Six two-level models were tested: MCAS ELA
performance for LEPs in SY2009 at elementary,
middle, and high school levels and MCAS Math
performance for LEPs in SY2009 at elementary,
middle, and high school levels. For the MCAS ELA
models, elementary grades included grades 4-5,
middle school grades included Grades 6-8, and high
school grades included Grades 9-12. For MCAS
Math models, elementary grades included Grades
3-5, middle school grades included Grades 6-8, and
high school grades included Grades 9-12.
Before including all explanatory variables in the
models, the intraclass correlation coefficient (ICC)
was calculated to verify that a hierarchical model
was needed (see Table 47). Next, we checked
multicollinearity to determine the model with valid
significance levels. Usually, higher correlations
among independent variables will result in a higher
condition index, and a variable may have to be removed
from the model for accurate estimation with
significance testing. Within the set of student-level
variables, Attendance Rate and Mobility were highly
correlated at the elementary and middle school
levels. The condition index was also high. Mobility
was removed from the model and Attendance
Rate was retained because the attendance variable
structure (ratio rather than categorical) provides
the opportunity for more detailed analysis. Percent
Mobility, a school-level variable representing the
percentage of the student population that changes
schools between October and June of a given
school year, was found to be strongly associated
with LEP Density. Percent Mobility was removed
from the model because LEP Density was considered
of more interest to this analysis. Finally, Highly
Qualified Teachers, a school-level variable representing
the percentage of the teaching staff that is
considered highly qualified, was also removed from
the analysis, because the structure of the variable
biased the analysis towards schools with highly
qualified teachers on staff.
118 Improving Educational Outcomes of English Language Learners in Schools and Programs in Boston Public Schools