Shaping the Future - Jefferson County Public Schools
Shaping the Future - Jefferson County Public Schools Shaping the Future - Jefferson County Public Schools
Applying an Iterative Evaluation- based Professional Development Model to Elementary Literacy Instruction Dena Dossett, Ph.D Beverly Winsch, Ph.D Jefferson County Public Schools, KY Jefferson County Public Schools Shaping the Future
- Page 2 and 3: Jefferson County Public Schools •
- Page 4 and 5: Every 1 Reads • Every 1 Reads is
- Page 6 and 7: JCPS Elementary Literacy Initiative
- Page 8 and 9: JCPS Professional Development Model
- Page 10 and 11: Advantages of the JCPS PD Model •
- Page 12 and 13: E V A L U A T I O N JCPS Profession
- Page 14 and 15: Major Outcomes Continued Stage 3: D
- Page 16 and 17: Level 1 Evaluation: Participants’
- Page 18 and 19: Level 3 Evaluation: Organization Su
- Page 20 and 21: Level 5 Evaluation: Student Learnin
Applying an Iterative Evaluation-<br />
based Professional Development<br />
Model to Elementary Literacy<br />
Instruction<br />
Dena Dossett, Ph.D<br />
Beverly Winsch, Ph.D<br />
<strong>Jefferson</strong> <strong>County</strong> <strong>Public</strong> <strong>Schools</strong>, KY<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
<strong>Jefferson</strong> <strong>County</strong> <strong>Public</strong><br />
<strong>Schools</strong><br />
• 28 th largest school district in U.S.<br />
• 98,000 students (1/7 of all students in KY)<br />
• 153 <strong>Schools</strong><br />
• Preschool……………<br />
3- and 4-year4<br />
year-olds<br />
• K-5 5 (Elementary)…………<br />
………….. 87 schools<br />
• 6-88 (Middle)………………<br />
……………….. 23 schools<br />
• 9-12 (High)…………………<br />
…………………. 20 schools<br />
• Special Education…………<br />
…………. 3 schools<br />
• Alternative/O<strong>the</strong>r…………<br />
…………. 20 schools<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
<strong>Jefferson</strong> <strong>County</strong> <strong>Public</strong><br />
<strong>Schools</strong><br />
Racial Composition<br />
34% African-American<br />
59% White<br />
7% O<strong>the</strong>r<br />
Non-English Language<br />
Background students<br />
<br />
4,128 (4.2%) students<br />
823% growth since 1990<br />
<br />
40% of State NELB<br />
Exceptional Child<br />
Education<br />
13,595 students ages 3-21<br />
Home/Hospital students<br />
Homeless students<br />
4,600 (5%) students<br />
288% growth since 1993<br />
Single-parent households<br />
54% of total JCPS population<br />
82% of <strong>the</strong> JCPS African-American<br />
population<br />
Free or reduced-price lunch<br />
61% of Elementary students<br />
4% increase since 2002-03<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Every 1 Reads<br />
• Every 1 Reads is an education initiative that is<br />
designed to have every JCPS student reading at<br />
grade level within four years.<br />
• This goal will position <strong>the</strong> Louisville Metro<br />
area as a national pacesetter in education.<br />
• Currently, 18,000 students are<br />
not reading on grade level.<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
JCPS Elementary Literacy<br />
Initiative<br />
Assessment<br />
‣ Diagnostic and achievement tests at each grade level<br />
Curriculum<br />
‣ Established, research-based, standardized instruction<br />
for reading and writing<br />
‣ Prescriptive<br />
‣ Core Content Guides<br />
‣ 38 Literacy coaches<br />
Intervention<br />
‣ Extended School Services<br />
‣ Tier I, II, III (additional time, supplemental materials, accommodations)<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
JCPS Elementary Literacy<br />
Initiative<br />
Structure<br />
‣ Additional Time<br />
‣ Group, Regroup<br />
Professional Development<br />
‣ Narrow and focused<br />
‣ Long-term<br />
‣ Job-embedded<br />
‣ Data-driven<br />
‣ Research-based<br />
‣ All teachers trained in reading strategies<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
JCPS Elementary Literacy<br />
Initiative<br />
Assessment<br />
Curriculum/<br />
Instruction<br />
Intervention<br />
Professional<br />
Development<br />
Data<br />
Management<br />
System<br />
Diagnostic/<br />
Achievement<br />
tests at each<br />
grade level<br />
School-Based<br />
Analysis of<br />
Student Work<br />
5 Block Model:<br />
Community Reading &<br />
Conversation<br />
Guided Reading<br />
Word Work<br />
Writing<br />
Self-Selected Reading<br />
Teacher Resource<br />
Materials, School and<br />
Classroom Materials<br />
Tier 1<br />
5 Block Literacy Model<br />
150-180 Minutes<br />
Tier 2<br />
Accommodations &<br />
Supplemental<br />
Materials (during<br />
Tier 1 instruction)<br />
Earobics<br />
Comprehension Plus<br />
Tier 3<br />
Reading Recovery<br />
Reading Mastery<br />
District Based<br />
Summer<br />
Institute<br />
School Based<br />
Sessions<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
JCPS Professional<br />
Development Model<br />
TASK: Develop an Effective JCPS<br />
Framework that Links <strong>the</strong> Impact<br />
of PD to Student Achievement<br />
Issue 1: JCPS needs to standardize district-wide<br />
professional development<br />
Issue 2: Empirical connections between training<br />
and student achievement need to be<br />
established<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
JCPS Professional<br />
Development Model<br />
E<br />
V<br />
A<br />
L<br />
U<br />
A<br />
T<br />
I<br />
O<br />
N<br />
Stage Stage 1: 1:<br />
ANALYSIS ANALYSIS<br />
Stage Stage 2: 2:<br />
DEVELOPMENT<br />
DEVELOPMENT<br />
Stage Stage 3: 3:<br />
DELIVERY DELIVERY<br />
Stage Stage 4: 4:<br />
IMPLEMENTATION<br />
IMPLEMENTATION<br />
Needs<br />
Objectives<br />
Format<br />
Evaluation Tools<br />
Instruction<br />
Piloting<br />
Operational Training<br />
-Participant Reaction<br />
-Participant Knowledge<br />
Prep for School-Based Evaluation<br />
Organization Support/Change<br />
Participant Use of New<br />
Knowledge and Skills<br />
Student Learning Outcomes<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Advantages of <strong>the</strong> JCPS<br />
PD Model<br />
• Requires Deliberate, Purposeful Training<br />
Development<br />
• Uses Proven Instructional Design Principles<br />
• Standardizes PD Presentations<br />
• Generates Meaningful Evaluation Data<br />
• Results in Reusable Training Support Packages<br />
(TSPs)<br />
• Embeds Quality Assurance Process<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Organizational Support<br />
for <strong>the</strong> JCPS PD Model<br />
Inter-Disciplinary PD Panel with Expertise in:<br />
•Evaluation<br />
•Professional Development<br />
•Curriculum<br />
•School Support<br />
•Literacy<br />
•Assessment<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
E<br />
V<br />
A<br />
L<br />
U<br />
A<br />
T<br />
I<br />
O<br />
N<br />
JCPS Professional Development Process Flowchart<br />
Evidence<br />
insufficient<br />
Evidence<br />
insufficient<br />
Unfavorable<br />
evaluation<br />
results<br />
Content Expert/PD Panel Member conduct<br />
Stage 1: Analysis<br />
•Needs Assessment<br />
•Measurable Goals<br />
•Skills<br />
•Format<br />
PD System Committee(includes PD Panel<br />
member(s)) reviews analysis evidence<br />
Content Expert/PD Panel Member conduct<br />
Stage 2: Development<br />
•Measurable Objectives<br />
•Evaluation Tools<br />
•Instruction<br />
•Pilot TSP<br />
PD Panel reviews development evidence<br />
PD Presenter conducts Stage 3: Delivery<br />
• Operational delivery<br />
• Preparation for school based evaluation<br />
Content Experts review levels 1 and 2<br />
evaluation results<br />
PD Participant conducts<br />
Stage 4: Implementation<br />
Principal conducts Level 4 evaluation and with<br />
outside validation conducts levels 3 & 5<br />
Content Experts review level 3-5<br />
evaluation results<br />
Evidence<br />
sufficient<br />
Evidence<br />
sufficient<br />
Favorable<br />
evaluation<br />
results<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Major Outcomes of JCPS<br />
PD Model<br />
Stage 1: Analysis<br />
Multiple Data Sources Indicated an Implementation Issue with Teachers<br />
Delivering Guided Reading Instruction to Elementary Students.<br />
Stage 2: Development<br />
Created Professional Development Job Aid for Training Developers based<br />
on PD Model.<br />
Held Workshops with District Literacy Specialists who began TSP<br />
Development Process.<br />
Fur<strong>the</strong>r Analyses Revealed that <strong>the</strong> Classroom Organizational Support<br />
Systems/Materials for Guided Reading Instruction were Inadequate.<br />
Selected a guided reading program compatible with <strong>the</strong> major tenets of<br />
<strong>the</strong> PD model<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Major Outcomes Continued<br />
Stage 3: Delivery<br />
1400 teachers attended guided reading institute. Training topics<br />
included assessment, guided reading, shared reading, classroom<br />
management, and explicit strategy instruction<br />
All training was standardized. Collected participant reaction and<br />
knowledge data<br />
Established plan to measure 5 levels of evaluation of guided<br />
reading professional development institute<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Five Levels of Professional<br />
Development Evaluation*<br />
Level 1 - Participants’ Reactions<br />
Level 2 - Participants’ Learning<br />
Level 3 - Organization Support & Change<br />
Level 4 - Participants’ Use of New<br />
Knowledge & Skills<br />
Level 5 - Student Learning Outcomes<br />
*Adapted from Guskey, T.R. (2002) Evaluating Professional<br />
Development. Thousand Oaks, CA: Corwin Press<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Level 1 Evaluation:<br />
Participants’ Reactions<br />
Purpose<br />
Evaluation<br />
Question<br />
Data Collection<br />
Method<br />
Outcome<br />
Implication<br />
To measure participants’ reactions/ satisfaction with <strong>the</strong><br />
professional development experience<br />
Did <strong>the</strong> participants understand <strong>the</strong> training materials and<br />
find <strong>the</strong>m useful?<br />
Questionnaires administered at <strong>the</strong> end of <strong>the</strong> guided<br />
reading PD institute<br />
The percentage of participants responded as follows:<br />
85% - extend content knowledge and professional practices<br />
95% - materials were appropriate<br />
93% - information was relevant to classroom<br />
94% - likely to what <strong>the</strong>y have learned in <strong>the</strong>ir classroom<br />
Results used to improve program design and delivery of<br />
follow-up training sessions<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Level 2 Evaluation:<br />
Participants’ Learning<br />
Purpose<br />
Evaluation<br />
Question<br />
Data Collection<br />
Method<br />
Outcome<br />
Implication<br />
To measure participants’ new knowledge and skills<br />
Did <strong>the</strong> participants meet <strong>the</strong> presenters’ expected<br />
performance criteria (training objective)?<br />
Questionnaire; Product Demonstrations at end of PD<br />
session<br />
91.8% of participants were judged as meeting <strong>the</strong><br />
expected performance criteria for <strong>the</strong> end of session<br />
Results used to improve program content, format, and<br />
organization<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Level 3 Evaluation:<br />
Organization Support & Change<br />
Purpose<br />
Evaluation<br />
Question<br />
Data Collection<br />
Method<br />
Implication<br />
To measure <strong>the</strong> organization’s s capacity, support and facilitation<br />
of <strong>the</strong> implementation of participant’s s new knowledge and skills<br />
Did <strong>the</strong> training affect organizational climate and procedures?<br />
Were sufficient resources made available?<br />
Was implementation facilitated and supported?<br />
Questionnaires of principal, instructional coaches, literacy lead<br />
teachers, literacy teachers<br />
High Ratings Low Ratings<br />
PD supports school goals Financial Support<br />
Quality follow-up support Encouraging PD through<br />
Access to expertise incentives & resources<br />
Sufficient leadership & Collaboration in planning PD<br />
collegial support<br />
Results used to document and improve organization support<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Level 4 Evaluation:<br />
Participants’ Use of New Knowledge & Skills<br />
Purpose<br />
Evaluation<br />
Question<br />
Data<br />
Collection<br />
Method<br />
Outcome<br />
Implication<br />
To measure <strong>the</strong> degree and quality of implementation of<br />
participants’ use of new knowledge and skills<br />
Did participants effectively apply <strong>the</strong> new knowledge and<br />
skills?<br />
Classroom observations of guided reading lessons using 4<br />
point rubric conducted by instructional coaches twice a year<br />
and monthly by principals<br />
Baseline data:<br />
64% of primary level classrooms at full implementation<br />
51.3% of intermediate level classrooms at full implementation<br />
Results used to document and improve <strong>the</strong> implementation<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Level 5 Evaluation:<br />
Student Learning Outcomes<br />
Purpose<br />
Evaluation<br />
Question<br />
Data Collection<br />
Method<br />
Outcome<br />
Implication<br />
To measure <strong>the</strong> impact of PD on student achievement<br />
in reading<br />
What was <strong>the</strong> impact on student reading performance?<br />
Developmental Reading Assessment (administered 2<br />
times a year for all students, 3 times for struggling<br />
readers)<br />
Baseline: 48.1% of students met benchmark<br />
Current: 61.7% of students met benchmark<br />
Results used to improve teacher implementation and<br />
demonstrate <strong>the</strong> impact of teacher PD on student<br />
achievement<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>
Next Steps:<br />
• Level 1 and 2 evaluation data from <strong>the</strong> summer institute in<br />
guided reading will shape follow-up training during <strong>the</strong> school<br />
year<br />
• Level 3 data on organizational support from o<strong>the</strong>r key<br />
stakeholders will be collected and analyzed<br />
• Level 4 data on implementation will be collected in December<br />
and May<br />
• Level 5 student outcome data will continued to be analyzed<br />
during school year<br />
• PD panel will update and extend PD model<br />
• Evaluation of Guided Reading PD will be integrated with o<strong>the</strong>r<br />
evaluation efforts (implementation & outcome) of larger<br />
literacy initiative<br />
<strong>Jefferson</strong> <strong>County</strong><br />
<strong>Public</strong> <strong>Schools</strong><br />
<strong>Shaping</strong> <strong>the</strong> <strong>Future</strong>