Full Report - Center for Collaborative Education

Recommendations

Info

IN DEPTH: Using Hierarchical Linear Modeling (HLM) to Determine the Relative Importance of Individual- and School-Level Factors in LEP Students’ ELA and Math MCAS Outcomes This study has identified significant differences in student achievement among LEP students of different demographic backgrounds, in schools of different characteristics, and in different types of programs. Summarizing the individual factors that proved to be significant, we found that English proficiency and disability were significant in MCAS ELA pass rates at all grade spans. Mobility was significant in the MCAS ELA pass rates of elementary and middle school students and in the Math pass rates of middle schoolers. Gender proved significant in the MCAS pass rates of LEP students at the elementary and high school levels. We found that although there were apparent differences between students in ELL and not in ELL programs, this difference proved significant only in the MCAS Math pass rates of elementary school students. In terms of school factors, we found that the percentage of LEP students in a school was significant in the outcomes in all subjects and grade levels except elementary MCAS ELA pass rates. AYP also proved significant in the outcomes of all subjects and grades except high school ELA pass rates. Poverty status, size, and the proportion of teachers licensed in their teaching assignment were broadly significant. 6 In order to further investigate the impact of these factors among LEP students in BPS, additional analysis was undertaken to explore the impact of both student-level characteristics and school environments on individual achievement. The primary goal of this analysis was to identify the individual and school environment characteristics that have the greatest impact on LEP students’ academic achievement. We accomplished this by examining individual attainment of LEP students as measured by MCAS scores in conjunction with a set of student-level and school environment characteristics that were significant in our descriptive analysis. This analysis included all LEP students in Grades 3-12 who had scores for either MCAS ELA or MCAS Math. Although some other analyses in this report were restricted to students performing at MEPA Levels 4 and 5, this analysis included LEP students at all levels of English proficiency in order to capture the impact of English attainment on academic outcomes. One of the key challenges in analyzing educational outcomes is that student outcomes are influenced not only by the student’s individual demographic background and educational experience such as program enrollment (individual-level), but are also affected by school environmental factors, such as the size of the school (school-level). This means that there are multiple levels of analysis (in this case, individual-level and school-level factors) that must be taken into account in order to obtain accurate results. Hierarchical Linear Modeling (HLM) is a form of multi-level analysis frequently used in educational research to account for the correlations that occur when individual students have similar educational experiences. Using HLM allows us to disaggregate the results and examine the effects that different types of factors, such as individual- and school-level characteristics, have on student outcomes, thereby providing a more accurate analysis of students’ experiences (Bryk & Raudenbush, 1992). In order to confirm that multi-level modeling is appropriate for the analysis of LEP students’ outcomes, we calculated the intraclass correlation coefficient to determine whether school characteristics play an important role in determining individual students’ academic achievement. This analysis examines individual students’ MCAS scores while taking into account the 80 Improving Educational Outcomes of English Language Learners in Schools and Programs in Boston Public Schools
school that they attend; if students in different schools demonstrate significant differences in MCAS scores (“between-school variance”), it indicates that school-level factors have a significant impact on individual students’ scores. If less than 10% of the variation in scores occurs at the school level, another type of analysis would be more appropriate. Table 47 displays the amount of variation in students’ scores that occurs between students in comparison to the variation in scores that occurs between schools. Table 47. Variation in MCAS Scores, 2-level Model. BPS, SY2009 Variable Level Percent of Explained Variation ELA Math Elementary School Student 84.3% 88.1% School 15.8% 11.9% Middle School Student 76.7% 78.6% School 23.3% 21.4% High School Student 56.8% 70.9% School 43.2% 29.2% ! Since variation that occurs due to school-level factors accounts for a significant amount of variation in individual outcomes (over 10% at every level of schooling), multi-level modeling is appropriate for this analysis. Interestingly, variation between schools increases as the school level increases. In other words, although individual student factors were more important in explaining the variation in LEP student academic achievement overall, school factors become more important as school level increases in both subjects. School factors represent 16% of the variation in MCAS ELA scores in elementary school, increasing to nearly half of the variation in high school (43%); in MCAS Math scores school factors represent 12% of the variation in elementary school, increasing to 29% of the variation in high school. Once we determined that multi-level modeling was appropriate for this analysis, we developed a two-level hierarchical linear model examining LEP students’ educational attainment outcomes (as measured by MCAS ELA and Math scores) in conjunction with individual-level and school-level characteristics. Again, all LEP students in Grades 3-12 who had MCAS scores in the appropriate subject were included in this analysis. At the individual level, the variable set included gender, attendance rate, English proficiency as measured by the student’s MEPA score, special education (SPED) placement, and ELL program participation. The primary advantage of a two-level model in which ELL program participation is an individual-level variable is that it enables us to compare the academic achievement of LEP students in ELL programs to that of LEP students not in ELL programs. The set of variables representing the school environment included meeting Adequate Yearly Progress (AYP) goals in either ELA or Math as appropriate, the percentage of the school population that is low-income, the school size (small, medium, or large), and the percentage of the school population that is of limited English proficiency (LEP). Although mobility was found to be significant in the descriptive analysis at both the student and school levels, it was not part of this analysis because of the high correlation between mobility and attendance at the student level and between mobility and the percentage of the school population that is LEP (LEP density) at the school level. In this type of analysis, high levels of correlation mean that only one of the correlated variables could be used; for this analysis, attendance rate was included at the student level and LEP density was Improving Educational Outcomes of English Language Learners in Schools and Programs in Boston Public Schools 81
Page 1 and 2:
Center for Collaborative Education
Page 3 and 4:
This report, Improving Educational
Page 5 and 6:
VII. DROPPING OUT 48 A. What Are th
Page 7 and 8:
CHAPTER I. INTRODUCTION
Page 9 and 10:
Language Learners (OELL) has attemp
Page 11 and 12:
CHAPTER II. THE STUDY
Page 13 and 14:
epresented by the blocks in differe
Page 15 and 16:
Table 2. Variables, Definitions, an
Page 17 and 18:
CHAPTER III. ENROLLMENT AND CHARACT
Page 19 and 20:
Figure 1. Change in Enrollment of S
Page 21 and 22:
Native speakers of Haitian Creole a
Page 23 and 24:
In Sum Following a swift decline in
Page 25 and 26:
One of the foci of this study is th
Page 27 and 28:
School Poverty Rate. Income status
Page 29 and 30:
ed a significantly smaller proporti
Page 31 and 32:
In Sum In this chapter we focused o
Page 33 and 34:
One of the deepest and most far-rea
Page 35 and 36: the transfer of large numbers of LE
Page 37 and 38: education. There are no studies of
Page 39 and 40: IN DEPTH: Enrollment of English Lan
Page 41 and 42: Table 14. Nature of Primary Disabil
Page 43 and 44: choices. From the data gathered, it
Page 45 and 46: Becoming fully literate in English,
Page 47 and 48: Populations focused upon in this an
Page 49 and 50: Table 19. English Proficiency Level
Page 51 and 52: who attained the level of English p
Page 53 and 54: ed among those LEP students scoring
Page 55 and 56: High dropout rates among Boston Pub
Page 57 and 58: B What Is the Annual High School Dr
Page 59 and 60: C What Are the Rates of Attendance,
Page 61 and 62: Table 26. Median Attendance Rates o
Page 63 and 64: Table 29. Attendance, Out-of-School
Page 65 and 66: Table 31. Annual High School Dropou
Page 67 and 68: In Sum This chapter has focused on
Page 69 and 70: The tests of the Massachusetts Comp
Page 71 and 72: student achievement (Rumberger, 199
Page 73 and 74: Table 34. MCAS Math Pass Rates of S
Page 75 and 76: Table 36. MCAS ELA, Math, and Scien
Page 77 and 78: Pass Rates in MCAS ELA, Math, and S
Page 79 and 80: IN DEPTH: Attendance Rates of MCAS
Page 81 and 82: portions of core academic courses t
Page 83 and 84: D What are the MCAS ELA and Math Pa
Page 85: LEP Students at MEPA Performance Le
Page 89 and 90: Results: English Language Arts In a
Page 91 and 92: 1 Massachusetts meets the requireme
Page 93 and 94: A Overall Findings and General Reco
Page 95 and 96: Designation as student with a disab
Page 97 and 98: B Specific Findings and Recommendat
Page 99 and 100: Recommendation 7: Parents of LEP st
Page 101 and 102: Recommendation 14: Monitor indicato
Page 103 and 104: (2) We found instances in which stu
Page 105 and 106: Recommendation 30: The quality of i
Page 107 and 108: Commonwealth of Massachusetts. (201
Page 109 and 110: Nathan, J., & Thao, S. (2007). Smal
Page 111 and 112: Vaznis, J. (2011, Sept. 18). US fin
Page 113 and 114: Overview The report sought to answe
Page 115 and 116: Table 1. Variables, Definitions and
Page 117 and 118: In TBE learn solely in English. In
Page 119 and 120: Table 2: Variables, Definitions and
Page 121 and 122: ! ! ! 5. Limitations of the Data Wh
Page 123 and 124: Table 8: MCAS ELA Pass Rates of LEP
Page 125 and 126: Table 10. Variables Considered in H
Page 127 and 128: APPENDIX 2: Additional Tables and F
Page 129 and 130: ! Table 3.2. Characteristics of Eng
Page 131 and 132: ! ! Table 6.3. English Proficiency
Page 133 and 134: ! Table 7.3. Attendance, Out-of-Sch
Page 135 and 136: ! ! ! Figure 8.3. Science Pass Rate
Page 137 and 138:
Table 8.2. MCAS Math Pass Rates for
Page 139 and 140:
APPENDIX 3: Characteristics and Out
Page 141 and 142:
! ! ! ! ! ! Table 4. MCAS ELA and M
Page 143 and 144:
A discussion of our HLM analyses of
Page 145 and 146:
ELA scores on average than LEP stud
Page 147 and 148:
Improving Educational Outcomes of E
show all

Full Report - Center for Collaborative Education

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?