Grauerholz 2003 Assessing critical thinking sociology.pdf - Oncourse

488 TEACHING SOCIOLOGYcal thinking is discipline-specific criticalthinking.EMPIRICAL STUDIESOF CRITICAL THINKINGDespite the ambiguity and overuse of theterm critical thinking (as seen in the previoussection), there has been much moretheoretical work on this topic than empiricalinvestigation. In Paul Baker's (1981:326)insightful article on critical thinking, heasks: "If critical thinking is so important,why have so few sociologists designed testingmethods which can demonstrate theachievement of such educational aims?"Nearly two decades later, sociologists continueto recognize the value of teachingcritical thinking and many have proposedteaching strategies to encourage students'growth in this area, but very few have empiricallytested critical thinking. Obviously,developing reliable and valid measures presentsa daunting task, in part because thereis so little empirical research from which todraw and also because such highly complexlearning is not easily tapped by standardmeasurement techniques. As Browne andLitwin (1987:390) argue: "Critical thinkingis a process, not a body of knowledge likevocabulary that can be mastered. "Baker (1981) and Norris and Ennis (1989)provide extensive reviews of the literatureon critical thinking, including informationabout standardized exams to assess criticalthinking. For instance, Norris and Ennis(1989) offer a useful overview of commerciallyavailable tests as well as guidelinesfor constructing and using both multiplechoiceand open-ended tests of criticalthinking that could be adapted to coursematerial. However, most assessment toolshave been developed by non-sociologistsand do not relate specifically to sociologicalcontent. Also, Baker (1981) and Norris andEnnis (1989) do not offer any results ofempirical studies using these assessmenttools.A few sociologists have attempted tomeasure critical thinking. Logan (1976)developed a test using 20 items, each ofwhich was an example of uncritical thinking(e.g., "Americans should not allow theirforeign policy to continue to be dictated byan egomaniac who would yell 'Yahoo' inthe Taj Mahal just to hear the sound of hisown voice"). He first asked students to respondto statements in any way (to determinestudents' inclination to think critically,even when not prompted) and next to identifyproblems inherent in the statements (totest students' ability to think critically whenasked to do so).Shepelak et al. (1992) developed a questionnairethat taps students' use of and appreciationfor critical thinking, relying uponstudents' perceptions of change in their ownlevel of critical thinking ability. In addition,instructors assessed the level of change theyperceived in students' level of critical thinkingthrough "subjective impressions of theirachievements in class (e.g., papers andpresentations)" (Shepelak, et al. 1992:25).Similarly, Stoecker et al. (1993:336) used"in-depth quantitative and qualitativeevaluations" to assess critical thinking aswell as students' own assessment of their"ability to critically analyze an argument"(Stoecker et al. 1993:337).Such measures, however, often lack empiricalclarity and sensitivity to the sociologicalimagination. There have been a fewexceptions. For instance, Green and Klug's(1990) research on the effects of studentdebate on critical thinking incorporates adistinctly sociological approach. Facultygraders who evaluated students' essays weregiven the following guidelines for evaluatingthe essays:Are sociological concepts used to illuminateand analyze the issue? Does the writer indicatean awareness of historical, cultural, and socialstructural (e.g., class, gender, age, race) contexts?How adequate is the logic of the argument?Some common logical goals to look for:* uses evidence selectively, or uses out ofdate evidence, or uses examples of dubiouspertinence.* generalizes from personal experience or