Hillsborough County Lessons Learned Presentation

Lessons Learned

Communication, Communication, Communication
• Collaborative effort
• Support of all stakeholders
• Teachers, Principals, Supervisors, Students, Parents, Teacher’s
Union, Community
• Transparent Process
• Open two-way dialog
• Multiple methods of distributing and receiving information
• Teachers need to be able to review and verify the students
that count for them and their data multiple times.
Helping students achieve by helping teachers excel

What assessments will be included in the model?
• State testing data mandated.
• Standardized national assessments are readily available.
• When district assessments are used…
– consider the impact on the teacher
• Time to administer vs. quality of the data received
• Is the assessment aligned with the curriculum?
– consider the sample size
• Best practice suggests 10 teachers for a particular course to have
appropriate standard error ranges.
• Consider what will be done with results when the sample size of
teachers is less than 10…less than 3….
– consider the complexity of the assessment
– consider how the data will be recorded and reported
• Item bank/Testing platform
• How will you maintain as much objectivity as possible?
Helping students achieve by helping teachers excel

What assessments will be used for unique
populations?
• How will ESE students be measured?
– We looked at assessments recommended for use with specific populations.
– We are using the Brigance Test of Basic Skills for students in grades KG-2 and
11 & higher and the Battelle and 4 Year Old Standards for PK.
– We also have a district created rubric for assessing life skills in low
prevalence populations.
• How will electives like performing arts be measured?
– Our first round included performance items.
– We moved to written exams based on the content knowledge taught in these
courses.
– We are testing specials on-line at the elementary level.
• Group administration
• Art pictures embedded
• Music recordings
• Paper and pencil option
• What about courses where a teacher has multiple levels in the same section?
• How will the data be captured?
Helping students achieve by helping teachers excel

If district assessments will be used, what measures
will ensure their validity and reliability?
• Content validity
• Test security
• Item bank/testing platform
– Monitor real time data, rescore, rescan, remove zeros
– Conduct an item analysis each time an assessment is administered
– Revise exams if the point biserial and p value indicate a change is needed
• Minimize the use of stand alone pre-tests
– Prior year achievement for Reading and Math
• FCAT, Stanford, PSAT …….
– District semester exam/end of year test from prior year
• Importance to student – exam exemptions
• Large item bank so that multiple versions of the exam can be developed
• District staff dedicated to building exams and better assessments
— Collaborate with curriculum supervisors and teachers
Helping students achieve by helping teachers excel

Revised items are
stored for current or
later use.
Good items are kept
&
Poor items are
revised or deleted
Test Blueprint created by
curriculum supervisors
and teacher leaders* to
include standards and
number of items tested
Developing
and Revising
Assessment
Items or whole test
is field tested
during the year
*Teacher leaders are recommended and approved in conjunction with CTA.
Curriculum
supervisor and
teacher leaders
write and/or select
items and prepare
assessment for field
testing
Assessment is
transferred to
online portal
(Achievement
Series)

How will common exams be created and the autonomy
for teacher decision making be maintained?
• In most courses a common semester exam would be expected and the
norm.
• In cases where no prior exam existed, we created exams for the courses.
– We created ROTC exams.
• In other cases, the design of the course called for flexibility in pacing and
autonomous decision making by the teacher.
– We worked with IB teachers in order to find some common ground.
• In some courses a common exam was easily agreed upon.
• In some courses where pacing was varied, we agreed upon a common set of
questions per topic administered by all in the course of the year.
Helping students achieve by helping teachers excel

How can multiple response formats be
incorporated?
• The majority of exams are multiple choice and scanned into our
Achievement Series database.
• Some assessments are open response (Writing).
– Common scoring rubric
– Scorers trained and calibrated yearly on the rubric
– Teachers do not score the papers of their own students
• Some assessments will be gridded response.
– One example is IB Math Analysis
• Teacher had an excellent open response exam.
• In order to be as objective as possible, exam utilizes a gridded response that can
be evaluated off site.
Helping students achieve by helping teachers excel

• Process
Test Development
• Curriculum supervisors select teachers to work on each test – the
number of teachers varies based on the overall student enrollment.
• The content supervisors are responsible for content validity.
• The assessment department is responsible for item quality and overall
test quality.
• Current course teachers develop a test blueprint based on the course
objectives and teacher consensus.
• The weight of objectives or content on the test is based on the time
that content is addressed in the curriculum. This serves as a proxy for
importance.
• The blueprint includes content and Webb’s cognitive complexity.
• We try to focus on the culminating skills for each semester.

Test Development
• Teacher training in item writing
• We generally require the teachers to submit items for review as part of
the training process.
• Frequently teachers who are excellent in instructional delivery are not
necessarily good at item creation.
• Challenges
• Reliable Platform with authoring capabilities
• Method of scoring – timelines
• Administering and scanning correctly/
• Support staff in place during testing
• Tech support
• Maintaining test security
• Printing costs
• Writing items that measure higher levels of cognitive skills
• Endless process
Helping students achieve by helping teachers excel

Helping students achieve by helping teachers excel
When students succeed, we all win.