Critical Thinking Assessment Report 12/13 AY

DACC Assessment Report -- Critical Thinking Across the
Curriculum
Instructions: Complete a separate form for each assessment report for the past academic year and
submit to Susan Wood, DACC Assessment Coordinator by September 30 th .
Academic Year of Assessment: Spring 2013
Submission Date: Summer 2013
Program or Discipline: Campus-Wide Assessment
Course/s Number: ENGL/BOT/L SC/COMM/GOV/CCDE/SOC
Student Learning Outcome: Critical Thinking/Problem Solving (DACC General Education SLO)
Submitted by: Susan Wood on behalf of the Critical Thinking Assessment Group (Kathleen Baca,
Michelle Guzman-Armijo, Shana Mason, Kurt Depner, Pierre Laroche, Krista Kozel, David
Burleson, Hiranya Roychowdhury, Jose Aranda, Linda Skalic, Amy Garcia, Doug Layer)
Description of Assessment Procedure: In the fall of 2012 Critical Thinking Assessment Group (CTAG)
developed a CLA Performance task and rubric. The assignment adhered to the model used by the
Council of Aid for Education, who develops standardized versions of CLAs for national use and provided
training to CTAG in the spring of 2012. In spring 2013, members of CTAG piloted the assignment in their
classes. Instructors could give it as a homework assignment or ask students to complete it in class.
Students were not given help from their instructors apart from explaining the directions. Some
instructors used the assignment as extra credit while others counted it as a regular assignment or exam
in the class.
After the semester had ended, members of CTAG met to assess a random sample of student artifacts
produced as a result of the assignment. Using a rubric developed by CTAG and based largely on the
Facione and Facione Critical Thinking Rubric, 47 student artifacts were assessed. Each artifact was
assessed by two independent readers. There was very little norming done prior to the group reading.
Report of Assessment Data and Results: The rubric is comprised of seven criteria. They are:
1. Interpret evidence, statements, graphics, questions, etc.?
2. Identify relevant arguments (reasons and claims) pro and con?
3. Analyze and evaluate major alternative points of view?
4. Draw legitimate, judicious, non-fallacious conclusions?
5. Justify results and procedures, and explain assumptions and reasons?

6. Follow where evidence and reasons lead?
7. Adhere to the conventions of Standardized American English?
Each reader gave a score of 1, 2, 3 or 4 to each criterion.
1=very poor or missing
2=poor/weak
3=okay/satisfactory
4=strong/excellent
A score of 3 or 4 is considered acceptable; a score of 1 or 2 is considered not acceptable.
Of the seven criteria, on criterion 7 had an average that was acceptable; all others had averages that
were in the unacceptable range. The follow graph provides a visual of the averages:
Data were also collected on how many of the artifacts were in-class versus out-of-class assignments.
There were 19 in-class assignment artifacts and 28 out-of-class assignment artifacts.
Data were also compiled regarding splits. A split is considered any reading by the two independent
readers that bridged the acceptable/unacceptable range. In other words, if one reader gave a criterion
on a particular artifact a 4 and the second reader gave the same criterion on the same artifact a 2, this is
considered a split. Given that there were 47 artifacts assessed and each artifacts was scored for seven
criteria, that provides 329 separate pairs to examine for splits. Of the 329 pairs, 49 had splits, which is
15%. Another way to look at splits is by artifact. Of the 47 artifacts, there was at least one criterion over

which the two readers split in 27 of the artifacts; this means that 57% of the artifacts yielded at least
one criterion that produced a split.
The following graphs show results for out-of-class versus in-class options:
Whilte the trends are basically the same, the in-class option seems to produce better results, as shown
in the overall score average (28 points possible):

Analysis and Interpretation/Reflection on Results or Trends: Based on the data and the experience, the
group’s consensus is that:
1. as a school, we need to address how to incorporate more critical thinking into the curriculum –
especially since this is one of our general education SLOs.
2. the assignment and rubric need to be tweaked to better assess critical thinking
3. the procedure needs to be tweaked to produce more reliable data
4. faculty need to be normed at the beginning of the semester to the rubric so that there will be
fewer splits and more valid results
5. the curriculum in ENGL 111 may need to be examined and adjusted to provide a better
foundation for critical thinking
6. work done in a more controlled environment (in-class) produces better results
Plan for Improving the Assessment Process and/or Student Learning: The following lists CTAG’s plan for
improving this assessment:
1. create one or two new CLAs in fall semester
a. make sure questions are obviously not open-ended
b. create a grading key to accompany each CLA
2. provide students with some brief material regarding the function and form of a memo
3. include as one of the CLA task a request for students to create a matrix of sorts that shows
which documents they thought were useful and valid, and which ones weren’t
4. revise the rubric and reorder the criteria
5. be clear how many documents students to refer to and reflect this requirement in the rubric

6. set up a norming session at the beginning of the semester and before the common reading
7. invite a few more classes to participate in the spring 2014 assessment, including part-time
faculty
8. require that the memo students produce be written in-class rather than as a homework
assignment
9. address how to cite the documents provided in either the assignment or the rubric
10. provide professional development to more DACC faculty to better teach and assess critical
thinking