Download Document - SDA Bocconi
Download Document - SDA Bocconi
Download Document - SDA Bocconi
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook
Table of Contents<br />
1. INTRODUCTION ................................................................................................................... 1<br />
2. WHAT IS (NOT) ASSESSMENT ............................................................................................ 3<br />
3. THE ASSESSMENT PROCESS ................................................................................................. 4<br />
STEP 1 ‐ ESTABLISHING LEARNING GOALS AND OBJECTIVES ........................................................................ 6<br />
STEP 2 ‐ ALIGNMENT OF CURRICULA WITH LEARNING GOALS ..................................................................... 7<br />
STEP 3 ‐ IDENTIFICATION OF ASSESSMENT MEASURES ................................................................................ 8<br />
STEP 4 ‐ COLLECTION, ANALYSIS AND DISSEMINATION OF ASSESSMENT INFORMATION .................................... 9<br />
STEP 5 ‐ USE OF ASSESSMENT INFORMATION FOR CONTINUOUS IMPROVEMENT .......................................... 10<br />
4. EVALUATION VS ASSESSMENT .......................................................................................... 11<br />
5. LEARNING GOALS AND LEARNING OBJECTIVES .................................................................. 12<br />
<strong>SDA</strong> BOCCONI’S DIMENSIONS AND LEARNING GOALS .............................................................................. 12<br />
DEFINING LEARNING GOALS ................................................................................................................ 14<br />
DEFINING LEARNING OBJECTIVES ......................................................................................................... 15<br />
6. TAXONOMIES .................................................................................................................... 18<br />
6.1 BLOOM’S TAXONOMY USED WHILE WRITING OBJECTIVES ................................................................. 19<br />
6.2 FINKS’S TAXONOMY USED WHILE WRITING OBJECTIVES .................................................................... 22<br />
7. RUBRICS ............................................................................................................................ 28<br />
HOW TO DESIGN RUBRICS ................................................................................................................... 29<br />
8. DEFINITIONS ..................................................................................................................... 32<br />
9. TEMPLATES ....................................................................................................................... 35<br />
COURSE SYLLABUS TEMPLATE ............................................................................................................. 35<br />
PROGRAM DIRECTORS ANNUAL REPORT SCHEME .................................................................................. 36<br />
RELEVANCE OF EACH COURSE FOR THE ACHIEVEMENT OF THE PROGRAM LEARNING GOALS ............................ 37<br />
CLOSING THE LOOP............................................................................................................................ 38<br />
CHECKPOINT ASSESSMENT FORMAT ..................................................................................................... 39<br />
EXAMPLE OF ASSESSMENT OF LEARNING OBJECTIVES FT MBA35 ............................................................ 42<br />
PEER OVERALL EVALUATIONS FT MBA 35 ............................................................................................ 47
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
1. INTRODUCTION<br />
In recent years, international institutions of higher education have<br />
recognized that the systematic assessment of student learning is<br />
essential to monitor quality and provide the information that leads<br />
to improvement.<br />
A full commitment to teaching and learning must include assessing<br />
and documenting what and how much students are learning and<br />
use this information to improve the educational experience offered<br />
by the institution.<br />
In many ways, assessment is a common sense process that the<br />
institution’s faculty already follows when it articulates the main<br />
goals for each course, checks if students have achieved them, and<br />
then uses these results to improve the course itself.<br />
This guide is designed to take advantage of what <strong>SDA</strong> <strong>Bocconi</strong><br />
School of Management is already doing by formalizing the process<br />
and sharing the individual efforts<br />
to the whole institution.<br />
If implemented effectively, the<br />
assessment of students’ learning is<br />
going to involve the mutual effort<br />
of students, administrators, and<br />
academic professionals.<br />
The idea at the heart of<br />
the assessment is that<br />
you need to know how<br />
you are doing before you<br />
can do better.<br />
The main focus of the assessment is the student, but it is also<br />
closely linked to the assessment of the effectiveness of the<br />
organization, which is important as a mean to monitor and improve<br />
the environment where teaching and learning occur.<br />
A good assessment can enhance quality at all levels of the School<br />
by providing it with the evidences necessary in order to steer<br />
effectively the decision‐making in many areas (thus including<br />
programmatic changes, classroom teaching modifications, support<br />
service adjustments, policy or procedure revisions, and structural<br />
reorganizations).<br />
The School is called to put a big effort in this process; hence,<br />
Assessment is a way to document the <strong>SDA</strong> <strong>Bocconi</strong> effectiveness<br />
and its ongoing improvement.<br />
1
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
A systematic, ongoing, cycle of goal setting, attainment<br />
measurement, and use results to make informed decisions is crucial<br />
to excellent performance.<br />
Thought in these terms, Assessment:<br />
Helps to determine whether the School’s mission has been<br />
accomplished<br />
Provides a rationale for modifying existing curriculum and<br />
instructional methods<br />
Enables the School to examine student’s growth and<br />
satisfaction<br />
2
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
2. WHAT IS (NOT) ASSESSMENT<br />
ASSESSMENT IS… a cyclical and systematic<br />
process<br />
ASSESSMENT IS NOT… a boring and<br />
annoying administrative activity<br />
ASSESSMENT AIMS TO… understand<br />
what students are/ are not learning<br />
ASSESSMENT DOES NOT AIM TO…<br />
horn in on a faculty member’s<br />
classroom<br />
ASSESSMENT PROVIDES…feedback<br />
to reinforce student learning<br />
ASSESSMENT DOES NOT<br />
PROVIDE… a means of punishment<br />
ASSESSMENT IMPROVES…<br />
Student learning<br />
ASSESSMENT DOES NOT INVOLVE…<br />
Exclusively faculty and students in<br />
classroom. It is mutually the<br />
responsibility of faculty and out of class<br />
educators.<br />
3
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
3. THE ASSESSMENT PROCESS<br />
Palomba and Banta (1999) define the outcomes assessment process<br />
as:<br />
The systematic collection, review, and use<br />
of information about educational programs<br />
undertaken for the purpose of improving<br />
student learning and development.<br />
“ … Measures of learning can assure external constituents such as<br />
potential students, trustees, public officials, supporters, and<br />
accreditors, that the organization meets its goals” (AACSB 2007,<br />
p.60).<br />
For this reason, at the heart of the AoL, there is the School’s<br />
mission that inspires its view of the future.<br />
Assessment is an ongoing, iterative process that<br />
1. ensures that students’ learning is measured and enhanced on a<br />
regular and continuous basis,<br />
2. uses results to inform decisions and make improvements.<br />
In order to implement it effectively, a careful planning is thus<br />
necessary.<br />
Learning goals and outcomes must be clearly specified, appropriate<br />
measures must be selected, data collection must be carefully<br />
executed and, most important, results must be shared for<br />
improvements to occur. The figure below illustrates the cycle of<br />
intertwined activities that facilitates continuous improvement.<br />
4
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Step 4: collection,<br />
analysis and<br />
dissemination of<br />
assement<br />
information<br />
Step 5: use of<br />
assessment<br />
information for<br />
contiunous<br />
improvement,<br />
Step 1: Establish<br />
including<br />
learning goals and<br />
documentation that<br />
objectives<br />
the ongoing process<br />
has been carried out<br />
on a systematic<br />
basis<br />
To contribute to the<br />
development of individuals<br />
and the environment where<br />
they operate by creating and<br />
disseminating innovative<br />
managerial knowledge.<br />
“Empowering lives through<br />
knowledge and imagination”<br />
Step 2: Alignment<br />
of curricula with<br />
adopted goals<br />
Step 3:<br />
Identification of<br />
assessment<br />
instruments and<br />
measures<br />
5
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Step 1 ‐ Establishing Learning Goals and Objectives<br />
What are students LO expected to achieve<br />
How do these LO relate to <strong>SDA</strong> <strong>Bocconi</strong>’s Mission<br />
Each learning goal should describe the desired educational<br />
outcome that students should be able to accomplish when<br />
they graduate from the program, regardless of their major<br />
or concentration.<br />
Learning goals are broad statements and, taken alone, are<br />
not suitable for assessment. Therefore, each learning goal<br />
must be translated into one or more learning objectives<br />
which describe a measurable attribute of the overall<br />
learning goal.<br />
For each learning objective, an assessment device or<br />
process must be developed to allow an evaluation of<br />
student performance on that objective providing a<br />
reasonable basis for conclusions about student<br />
performance on the overall learning goal.<br />
Program learning goals should include goals that address<br />
some, or part, of the general knowledge and skills areas<br />
(e.g., communication skills, problem‐solving abilities,<br />
ethical reasoning skills, language skills, technology skills,<br />
etc.) as well as management‐specific knowledge and skill<br />
areas (e.g., directly relate to management tasks that form<br />
the business foundation of degree requirements)<br />
6
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Step 2 ‐ Alignment of Curricula with Learning Goals<br />
Which curriculum components are more conducive to<br />
supporting LG<br />
What could be the appropriate measurement environment<br />
There should be clear evidence that the work students are<br />
doing in one or more classes directly supports students’<br />
achievement of the learning goals.<br />
The alignment of learning goals and curricula is critical. If<br />
learning goals are adopted but are not addressed in the<br />
curricula, the outcomes assessment process will be<br />
worthless.<br />
7
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Step 3 ‐ Identification of assessment measures<br />
Which instruments and measures will be used to determine<br />
the outcome achievement<br />
“Course‐embedded” measures relate to specific course<br />
assignments in a class where the students’ work on that<br />
assignment may also be used for outcomes assessment<br />
purposes.<br />
In the course embedded methods, course assignments or<br />
other student demonstrations from a course are evaluated<br />
for the purposes of AoL through a separate, distinct process<br />
that is driven by criteria established by the faculty (plural).<br />
Assessment measures should be selected after learning<br />
goals/objectives have been established.<br />
8
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Step 4 ‐ Collection, analysis and dissemination of<br />
assessment information<br />
When and how often should these outcomes be assessed<br />
Which means of dissemination will be used<br />
Once data are collected about students’ performance<br />
through the outcomes assessment process, it must be<br />
shared with and analyzed by the appropriate faculty<br />
committees and the leadership of the program.<br />
9
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Step 5 ‐ Use of assessment information for continuous<br />
improvement<br />
What results would indicate a satisfactory level of<br />
accomplishment<br />
How will these results be used to improve programs<br />
Regardless of the assessment processes that are deployed,<br />
the school program must demonstrate assessment data are<br />
being used to inform the leadership and faculty about the<br />
effectiveness of their educational programs.<br />
For AACSB accreditation reviews, schools should present<br />
examples of student’s performance measured on<br />
assessment criteria and document how and where<br />
assessment outcomes have been used for continuous<br />
improvement of curricula.<br />
10
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
4. EVALUATION VS ASSESSMENT 1<br />
Evaluation<br />
Focuses on the professor and the<br />
teaching performance<br />
individual performance<br />
Helps individuals know how they<br />
performed.<br />
Results in a grade for the activity or<br />
course.<br />
May or may not be used for<br />
improvement of instruction.<br />
Assessment<br />
Focuses on the student and the<br />
learning environment<br />
group performance<br />
Helps stakeholders know how<br />
students performed.<br />
Results in a plan for improvement<br />
Always used for improvement of<br />
instruction/ learning<br />
Grades, when used alone, are no longer regarded as adequate<br />
indicators of student learning.<br />
In particular,<br />
Grades are not specific and provide not sufficiently detailed<br />
information about which course outcomes students are<br />
mastering well and which are giving them trouble.<br />
Grading standards often vary widely among different<br />
instructors and do not indicate the same degree of mastery<br />
of course outcomes.<br />
Grade inflation (easy tests, generous grading, extra‐credit<br />
bonuses) sometimes presents a misleading indicator of<br />
student mastery of course outcomes.<br />
Faculty have the opportunity to use the course assessment process<br />
as a stimulus to discuss course content with each other and, based<br />
on the results of an assessment, determine how they can improve<br />
student.<br />
1 Adapted from http://www.pvc.maricopa.edu/AI/documents/Ass_sec1.pdf<br />
11
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
5. LEARNING GOALS AND LEARNING OBJECTIVES<br />
Goals express what the School wants its students to be.<br />
Objectives describe what the School wants its students to<br />
do.<br />
The definition of learning goals is a key element in how the school<br />
defines itself. This implies that the learning goals are closely related<br />
and connected to the School and the program’s mission.<br />
<strong>SDA</strong> <strong>Bocconi</strong>’s dimensions and learning goals<br />
<strong>SDA</strong> <strong>Bocconi</strong> School of Management has a description of the<br />
values, attitudes, knowledge, skills and behavior it is seeking to<br />
instill in its students. These skills and attitudes are broadly applied<br />
to all Master Programs.<br />
To help students to prepare for a successful professional career and<br />
to fully develop their potential as individuals, <strong>SDA</strong> <strong>Bocconi</strong> has<br />
defined three categories of Learning Dimensions which are a<br />
reference for all Master Programs, when defining and revising their<br />
Learning Goals and which are linked to the School’s Mission.<br />
At the forefront stays a strong and integrated managerial<br />
knowledge <strong>SDA</strong> <strong>Bocconi</strong> aims at creating, disseminating and<br />
conveying to teaching and learning experiences, both for students<br />
and Faculty.<br />
The second dimension is a deep understanding and awareness of<br />
the environment in which we operate, students, executives,<br />
companies, institutions and all the stakeholders and actors of<br />
economic systems, both domestic and global.<br />
The third is a sound equipment of managerial skills and culture,<br />
which make it possible to turn knowledge into a responsible and<br />
sustainable set of actions.<br />
12
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
In the beneath table the three dimensions and subsequent learning<br />
goals are represented.<br />
Dimension<br />
Learning Goal<br />
MANAGEMENT<br />
KNOWLEDGE<br />
1 Management Knowledge<br />
Graduates will be competent in the core<br />
functional areas and understand how these areas<br />
are integrated.<br />
ENVIRONMENT 2 Entrepreneurial mindset and creativity<br />
Graduates will acquire an entrepreneurial and<br />
creative mindset.<br />
3 Diversity and global business issues<br />
Graduates will understand the impact of diversity<br />
on organizations, and will have an understanding<br />
of the main global issues affecting companies,<br />
public/non‐profit organizations, banks and<br />
financial institutions.<br />
4 Ethics & Integrity<br />
Graduates will be conscious of the needs to<br />
comply with rules, regulations, social norms.<br />
MANAGEMENT<br />
SKILLS<br />
5 Strategic Thinking & Problem Solving Skills<br />
Graduates will formulate effective strategies and<br />
make decisions, using problem‐solving<br />
techniques.<br />
6 Teamwork & Leadership Skills<br />
Graduates will work effectively in teams, using<br />
interpersonal, communication and leadership<br />
skills.<br />
7 Project Management Skills<br />
Graduates will be able to effectively plan and<br />
implement projects and activities and to<br />
coordinate tasks and monitor their progress.<br />
13
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Defining learning goals<br />
Answering clearly and specifically is essential to obtain a consensus<br />
from faculty members in the program on the program’s vision,<br />
values and goals that will serve as guiding principles for developing<br />
outcomes and collecting data 2 .<br />
Hence, the first question you should answer is<br />
“What would the program accomplish”<br />
Characteristics of an effective mission statement<br />
RELEVANT: what is the aim of the program (it should be also<br />
related and aligned to the School’s mission)<br />
SPECIFIC: what is the student target<br />
CLEAR: what is the real contribute to the development of the<br />
participants<br />
INFLUENTIAL AND DISTINCTIVE: what is the reason why the<br />
program exists A mission should provide a sense of direction,<br />
and guide decision‐making processes<br />
Normally, four to ten learning goals are specified for each degree<br />
program. Core learning goals include general knowledge and skill<br />
goals and management‐specific learning goals.<br />
2 Adapted from http://www.stanford.edu/dept/presprovost/irds/assessment/downloads/mission_stmt.pdf<br />
14
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Defining learning objectives<br />
A learning objective is a statement of what students will be able to<br />
do, specifically what knowledge, skills and attitudes they will<br />
demonstrate to have acquired outside the classroom (in context)<br />
with what they have learned.<br />
This means that the point of view which should be adopted is<br />
“student‐based” rather than “instructor‐based” (e.g., “After this<br />
course, my students should be able to…” rather than “During this<br />
course I’ll talk about…”).<br />
A clear learning objective serves the following intents:<br />
Helps planners develop the program content<br />
Guides selection of teaching/learning activities that will best<br />
achieve objectives<br />
Serves to connect content and assessment<br />
Gives learners a clear picture of what to expect and what’s<br />
expected of them<br />
Forms the basis for evaluating learner, and program<br />
effectiveness<br />
Key elements of a useful learning objective 1<br />
SPECIFIC: according the learning goals, the program and the<br />
School’s mission<br />
MEASUREABLE: it makes the learning goals visible (serves as an<br />
indicator)<br />
ATTAINABLE: for target audience within scheduled time and<br />
specified conditions<br />
RESULTS‐ORIENTED: it should produces concrete outcomes<br />
TARGETED: to the learner and to the desired level of<br />
learning/thinking<br />
To create specific, measurable, and results‐oriented<br />
objectives:<br />
o<br />
o<br />
It’s helpful to finish the sentence, “After this course,<br />
you should be able to…”<br />
Start with an observable action word that captures<br />
what the learner should be able to do.<br />
15
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
o<br />
Avoid ill‐defined terms that are open to variable<br />
interpretation (e.g., understand, learn, grasp); use<br />
instead terms that describe directly observable<br />
behaviors.<br />
o When necessary, specify criteria concerning<br />
expected standard of performance<br />
To create attainable learning objectives:<br />
o<br />
Consider the beginning level of understanding/skill<br />
of your learners and craft your objective to move<br />
them to the next level.<br />
o Consider and specify when appropriate the<br />
conditions under which performance will take place<br />
(e.g., “On a written exam, describe…” or “With a<br />
standardized or actual context, demonstrate…”)<br />
o<br />
Limit number of objectives to major learning points<br />
you would like students to walk away with.<br />
To create objectives targeted to the audience and desired<br />
level of learning/thinking:<br />
o<br />
Ask yourself whether you want learners to be able<br />
to know, do and be, whenever possible, refer to<br />
commonly adopted taxonomies. We suggest<br />
Bloom’s and Fink’s taxonomies (see paragraphs 6<br />
and 7). Use the taxonomy to describe the desired<br />
level of learning/thinking (table 1 A and 1 B)<br />
o Match your action verb to the desired level (Table 2<br />
A or B).<br />
o Match learning objective with appropriate<br />
teaching/learning strategy (Table 3 A or B).<br />
Student performance on learning goals must be assessed<br />
systematically and routinely.<br />
No one approach to assurance of learning is prescribed.<br />
Benchmarking is not required.<br />
Assessment programs should include direct measures of<br />
learning. Course grades are not program assessment<br />
measures.<br />
Program assessment does not require every student to be<br />
assessed. Sampling is acceptable as long as an appropriate<br />
and representative sampling methodology is utilized.<br />
16
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Assessment results must be analyzed, disseminated, and<br />
utilized by the faculty toward curriculum planning.<br />
The work of individual students – not student teams – is to<br />
be used to assess learning outcomes (except for “Team<br />
Work” Learning Goal).<br />
17
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
6. TAXONOMIES<br />
"Taxonomy” simply means “classification”, so the taxonomies<br />
provided in the following paragraphs are an attempt (within the<br />
behavioral paradigm) to classify forms and levels of learning 3 .<br />
3<br />
The content of the tables are adapted from ©Teacher & Educational<br />
Development, University of New Mexico School of Medicine, 2005<br />
18
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
6.1 Bloom’s Taxonomy 4 Used While Writing Objectives<br />
Bloom’s classification of cognitive skills is widely used in instruction<br />
planning since its creation by a committee of educators chaired by<br />
Benjamin Bloom in 1956.<br />
It identifies three “domains” of learning:<br />
Cognitive: mental skills (Knowledge)<br />
Affective: growth in feelings or emotional areas (Attitude)<br />
Psychomotor: manual or physical skills (Skills)<br />
Each domain is organized as a series of levels or pre‐requisites. It is<br />
suggested that one cannot effectively — or ought not try to —<br />
address higher levels until those below them have been covered (it<br />
is thus effectively serial in structure). As well as providing a basic<br />
sequential model for dealing with topics in the curriculum, it also<br />
suggests a way of categorizing levels of learning, in terms of the<br />
expected ceiling for a given program. Thus in the Cognitive domain,<br />
training for technicians may cover knowledge, comprehension and<br />
application, but not concern itself with analysis and above, whereas<br />
full professional training may be expected to include this and<br />
synthesis and evaluation as well 5 .<br />
4 Bloom B., Taxonomy of Educational Objectives Handbook, 1956<br />
5 From http://www.learningandteaching.info/learning/bloomtax.htm<br />
19
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Table 1A: Example Action Verbs for Each Level of Learning<br />
Category<br />
Knowledge<br />
(Recall and<br />
Understanding)<br />
Application<br />
Problem‐<br />
Solving<br />
(Analyzing,<br />
Synthesizing,<br />
Evaluating)<br />
Example Action Verbs<br />
Associate<br />
Compare<br />
Contrast<br />
Define<br />
Calculate<br />
Demonstrate<br />
Draw<br />
Employ<br />
Advocate<br />
Analyze<br />
Assess<br />
Challenge<br />
Compose<br />
Describe<br />
Differentiate<br />
Distinguish<br />
Identify<br />
Estimate<br />
Give example<br />
Illustrate<br />
Locate<br />
Conclude<br />
Construct<br />
Create<br />
Critique<br />
Debate<br />
Indicate<br />
List<br />
Name<br />
Paraphrase<br />
Measure<br />
Operate<br />
Perform<br />
Prescribe<br />
Decide<br />
Defend<br />
Derive<br />
Design<br />
Evaluate<br />
Recognize<br />
Repeat<br />
Restate<br />
Review<br />
Record<br />
Set up<br />
Sketch<br />
Formulate<br />
Infer<br />
Judge<br />
Organize<br />
Plan<br />
Show<br />
State<br />
Summarize<br />
Tell<br />
Solve<br />
Trace<br />
Use<br />
Propose<br />
Rank<br />
Recommend<br />
Select<br />
Suggest<br />
Table 2A: Levels of Thinking/Learning<br />
Category Dimension Definition<br />
Knowledge Recalling Rote recall: Know common terms, specific facts,<br />
methods, procedures, concepts, principles<br />
Comprehending Interpolation or interpretation: Understand, estimate<br />
future implied consequences, justify methods and<br />
procedures<br />
Application Applying Using a concept in a new context: Apply theory, solve<br />
problems, construct graphs, demonstrate procedure<br />
Problem‐<br />
Solving<br />
Analyzing<br />
Synthesizing<br />
Evaluating<br />
Breaking something down and understanding its<br />
structure, the relationship between parts, the<br />
organizational principles:<br />
Recognize unstated assumptions and logical fallacies,<br />
distinguish between facts & inferences, determine<br />
relevance<br />
Building a structure/pattern from diverse elements: Write<br />
well‐organized essay, propose research question, develop<br />
plan for solving a problem, formulate a classification<br />
scheme<br />
Judging the value of ideas, works, solutions, materials:<br />
Judge logical consistency, adequacy of data in support of<br />
conclusions, value of work by internal & external<br />
standards<br />
20
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Table 3 A: Teaching/Learning Strategies Best Suited for Each Level of Learning<br />
Desired Dimension<br />
Knowing and<br />
comprehending<br />
Applying<br />
Analyzing<br />
Synthesizing<br />
Evaluating<br />
Suggested Presentational Strategies<br />
Presentation, lecture, question‐and‐answer, small group discussion,<br />
development of learning issues, self‐awareness exercises/tests, review<br />
sessions, teaching others, independent study, web‐based instruction<br />
Hands‐on, case study, simulation, role‐playing, action plan, teaching<br />
others, direct patient contact, guided practice with feedback, rolemodeling<br />
Question‐and‐answer, brainstorming, case study, problem‐solving,<br />
trouble‐shooting, role‐playing, article discussion<br />
Case study, writing, concept mapping, theory and model building,<br />
teaching others, developing research questions, direct enterprise contact<br />
Case study, critical review, self and group assessment/reflection,<br />
reflective writing, direct enterprise contact<br />
21
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
6.2 Finks’s Taxonomy 6 Used While Writing Objectives<br />
In 2003 Finks elaborated a new taxonomy. Individuals and<br />
organizations involved in higher education are expressing a need for<br />
important kinds of learning that do not emerge easily from the Bloom<br />
taxonomy, for example: learning how to learn, leadership and<br />
interpersonal skills, ethics, communication skills, character, tolerance,<br />
the ability to adapt to change, etc. My interpretation of these<br />
statements is that they are expressing a need for new kinds of<br />
learning, kinds that go well beyond the cognitive domain of Bloom's<br />
taxonomy and even beyond cognitive learning itself. This suggests<br />
that the time may have arrived when we need a new and broader<br />
taxonomy of significant learning.<br />
With an awareness of this need in mind, I have reviewed descriptions<br />
of high quality teaching and learning and have attempted the task of<br />
creating a new taxonomy, one that describes new kinds of significant<br />
learning 7 .<br />
6 Fink L.D., Creating Significant Learning Experiences, 2003<br />
7 Fink L.D., What is ''significant learning”,University of Oklahoma, 2003<br />
22
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Table 1B: Example Action Verbs for Each Dimension of Learning<br />
Dimension Action Verbs Objects<br />
FOUNDATIONAL KNOWLEDGE – What key information, ideas, perspectives are important for<br />
learners to know<br />
Understanding and<br />
Remembering, (developing a<br />
full understanding of the<br />
concepts associated with a<br />
subject to a degree that<br />
allows explanations,<br />
predictions, etc.)<br />
Associate, Compare,<br />
Contrast, Define, Describe,<br />
Explain, Give example,<br />
Identify, Illustrate, Indicate,<br />
List, Name, Paraphrase,<br />
Predict, Recite, Recognize,<br />
Repeat, Restate, Tell<br />
Facts, concepts, theories,<br />
relationships, models,<br />
perspectives, structures,<br />
organizations, purposes,<br />
proposals, problems, results,<br />
conclusions, plans<br />
APPLICATION – What kinds of thinking, complex projects, and skills is it important for learners<br />
to be able to do/manage<br />
Critical Thinking (analyzing<br />
and critiquing issues and<br />
situations)<br />
Practical Thinking<br />
(developing problem‐solving<br />
and decision‐making<br />
capabilities)<br />
Creative Thinking (creating<br />
new ideas, products, and<br />
perspectives)<br />
Managing Complex Projects<br />
(being able to coordinate and<br />
sequence multiple tasks in a<br />
single project/case and/or<br />
multiple projects/cases)<br />
Analyze, Assess, Audit,<br />
Catalog, Categorize, Classify,<br />
Compare, Contrast, Decipher,<br />
Deduce, Derive, Determine,<br />
Diagram, Differentiate,<br />
Dissect, Distinguish, Examine,<br />
Formulate, Hypothesize,<br />
Infer, Interpret, Label, Locate,<br />
Measure, Organize, Query,<br />
Separate, Trace<br />
Advise, Answer, Apply,<br />
Calculate, Certify, Choose,<br />
Consult, Debate, Decide,<br />
Determine, Diagnose,<br />
Evaluate, Give evidence,<br />
Judge, Justify,, Predict,<br />
Prescribe, Propose, Prove,<br />
Rank, Select, Solve, Suggest,<br />
Test<br />
Abstract, Adapt, Amend,<br />
Author, Compose, Construct,<br />
Convert, Create, Design,,<br />
Develop, Devise, Discover,<br />
Draw, Envision, Experiment,,<br />
Fabricate, Imagine, Improve,<br />
Refine, Reform, Sketch,<br />
Theorize, Transform, Write<br />
Administer, Assign, Coach,<br />
Communicate, Complete,<br />
Conduct, Coordinate,<br />
Delegate, Develop, Evaluate,<br />
Facilitate, Follow Up, Guide,<br />
Implement, Manage,<br />
Organize, Plan, Prioritize,<br />
Strategize, Supervise,<br />
Summarize, Teach, Time‐line,<br />
Train<br />
Ideas, issues, situations,<br />
proposals, processes, results,<br />
conclusions, theories,<br />
assumptions<br />
Problems, issues, conundrums<br />
Ideas, plans, products,<br />
objects, premises,<br />
perspectives, models,<br />
theories<br />
Tasks, timelines, cases,<br />
projects<br />
23
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Dimension Action Verbs Objects<br />
Performance Skills<br />
(developing capabilities in<br />
carrying out psychomotor<br />
activities)<br />
Conduct, Demonstrate, Do,<br />
Employ, Execute, Exhibit,<br />
Operate, Perform, Produce,<br />
Set up, Use<br />
Procedures, routines,<br />
processes, interviews<br />
INTEGRATION – What connections should learners be able to recognize and make within and<br />
beyond this learning experience<br />
Interdisciplinary Learning<br />
(connecting ideas, disciplines,<br />
perspectives, contexts),<br />
Learning Communities<br />
(connecting people),<br />
Learning and Living/Working<br />
(connecting, different realms<br />
of life)<br />
Associate, Combine,<br />
Compare, Concept map,<br />
Connect, Contrast, Correlate,<br />
Differentiate, Integrate, Link,<br />
Relate, Synthesize<br />
Ideas, disciplines,<br />
perspectives, contexts,<br />
people, domains, realms<br />
HUMAN DIMENSION – What should learners learn about themselves and about interacting with<br />
others<br />
Relationships (with, peers,<br />
supervisors, patients, others),<br />
Self‐Authorship (learning to<br />
create and, take responsibility<br />
for one’s own life),<br />
Leadership (becoming an<br />
effective leader), Ethics,<br />
Character Building (living by,<br />
ethical principles),<br />
Multicultural Education<br />
(being culturally, sensitive in<br />
interactions with others),<br />
Working as a Member of a<br />
Team, (knowing how to<br />
contribute to a team),<br />
Citizenship (of one’s<br />
profession, community,,<br />
nation state, other political<br />
entity), Environmental Ethics<br />
(having ethical, principles in<br />
relation to nonhuman world<br />
Acquire, Advise, Advocate,<br />
Balance, Be aware of, Behave,<br />
Collaborate, Communicate,<br />
Comply, Cooperate, Critically<br />
reflect, Decide to,<br />
Demonstrate, Describe,<br />
Educate, Embody, Empathize,<br />
Express, Feel confident, Give<br />
feedback, Help, Influence,<br />
Initiate, Inspire, Interact with,<br />
Involve, Lead, Mediate,<br />
Mobilize, Motivate,<br />
Negotiate, Nurture, Offer,<br />
Promote, Protect, Reconcile,<br />
Reform, Resolve, conflict,<br />
Respect, Respond, sensitively,<br />
See oneself as, Serve as role,<br />
model, Settle, Share, Show,<br />
Suggest, Support, Suspend,<br />
judgment, Sustain, Take<br />
responsibility, Unite<br />
Ethics, morality, principles,<br />
attitudes, values, beliefs,<br />
premises, conflicts, personal,<br />
social, cultural, and,<br />
environmental, implications<br />
CARING – What changes in learners’ feelings, interests, values are important<br />
Wanting to Be a Good<br />
Learner, (wanting to master,<br />
achieve high standards),<br />
Becoming Excited About a<br />
Particular, Activity/Subject<br />
(developing a keen interest),<br />
Developing a Commitment<br />
to Live, Right (i.e., deciding<br />
to take care of one’s,<br />
health/well‐being, live by a<br />
certain code)<br />
Agree to, Be ready to,<br />
Commit to, Decide to,<br />
Demonstrate, Develop,<br />
Discover, Explore, Express,<br />
Get excited about, Identify,<br />
Pledge, Recognize, value of,<br />
Renew interest, Revitalize,<br />
Share, State, Take time to,<br />
Value<br />
Attitudes, beliefs, feelings,<br />
interests, opinions, values<br />
24
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Dimension Action Verbs Objects<br />
LEARNING HOW TO LEARN – What should learners learn about learning, engaging in inquiry,<br />
and becoming self‐directed<br />
How to Be a Better Learner<br />
(engaging in, self‐regulated<br />
learning or deep learning),<br />
How to Inquire and<br />
Construct, Knowledge (how<br />
to engage in the scientific,<br />
method, historical method,<br />
other forms of inquiry), How<br />
to Pursue Self‐Directed or,<br />
Intentional Learning<br />
(developing a, learning<br />
agenda and plan, becoming<br />
an intentional, learner,<br />
becoming skilled in<br />
autodidaxy, being a, reflective<br />
practitioner)<br />
Construct knowledge about,<br />
Describe how to, Develop a<br />
learning plan, Frame useful<br />
questions, Generalize<br />
knowledge, Identify sources<br />
and resources, Identify your<br />
learning style & barriers,<br />
Identify what you need to<br />
know, Inquire, Predict<br />
performance, Reflect,<br />
Research, Self‐assess, Selfregulate,<br />
Self‐monitor, Set a<br />
learning agenda, Take<br />
responsibility for, Transfer<br />
knowledge<br />
Learning, acquisition of<br />
knowledge and skills, selfimprovement,<br />
self‐direction,<br />
accountability<br />
25
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Table 2B: Levels of Thinking/Learning<br />
Category Dimension Definition<br />
Foundational<br />
Knowledge<br />
Remembering &<br />
Understanding<br />
Knowing common terms, specific facts,<br />
methods and procedures, basic concepts,<br />
principles; understanding to a degree that<br />
allows for explanations, predictions<br />
Application Critical Thinking Analyzing and critiquing issues and situations<br />
Practical Thinking Solving problems and making decisions<br />
Creativity<br />
Creating/refining/ inventing new ideas,<br />
products, and perspectives<br />
Integration<br />
Human<br />
Dimension<br />
Caring<br />
Learning<br />
How to Learn<br />
Managing Complex<br />
Projects<br />
Performance Skills<br />
Interdisciplinary<br />
Learning<br />
Interpersonal<br />
Relationships<br />
Self‐Authorship<br />
Leadership<br />
Ethics, Character<br />
Building<br />
Multicultural<br />
Education<br />
Working as a Member<br />
of a Team<br />
Citizenship<br />
Environmental Ethics<br />
Wanting to be a good<br />
learner<br />
Becoming excited<br />
about a particular<br />
activity or subject<br />
Developing a<br />
commitment to live<br />
right<br />
How to be a better<br />
learner<br />
How to inquire and<br />
construct knowledge<br />
How to pursue selfdirected<br />
or intentional<br />
learning<br />
Coordinating and sequencing multiple tasks in<br />
a single project/case and/or multiple projects/<br />
cases<br />
Communicating and performing psycho‐motor<br />
activities<br />
Connecting different ideas,<br />
disciplines,perspectives, contexts<br />
Establishing effective working relationships<br />
with supervisors, peers, patients, and others<br />
Creating and taking responsibility for one’s<br />
own life<br />
Being an effective leader<br />
Developing character and living by ethical<br />
principles<br />
Becoming culturally sensitive in one’s<br />
interactions with others<br />
Knowing how to contribute to a team<br />
Being a responsible citizen of one’s<br />
profession, local community, nation state, and<br />
other political entity<br />
Having ethical principles in relation to the<br />
nonhuman world<br />
Wanting to master material, achieve high<br />
standards<br />
Developing a specific area interest<br />
For example, deciding to take care of one’s<br />
health and well‐being, to live by a certain code<br />
Engaging in self‐regulated learning or deep<br />
learning<br />
How to engage in the scientific method,<br />
historical method, and/or other forms of<br />
inquiry<br />
Developing a learning agenda and plan,<br />
becoming an intentional learner, becoming<br />
skilled in autodidaxy, being a reflective<br />
practitioner<br />
26
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Table 3B: Teaching/Learning Strategies Best Suited for Each Dimension of Learning<br />
Desired dimension<br />
Foundational Knowledge<br />
(understanding,<br />
remembering)<br />
Application (critical &<br />
practical thinking, creativity,<br />
managing projects,<br />
performance skills)<br />
Integration (connecting<br />
ideas,<br />
disciplines, people, realms)<br />
Human Dimension<br />
(leadership, ethics,<br />
teamwork;<br />
social, cultural, political,<br />
environmental implications)<br />
Caring (wanting to succeed,<br />
developing a keen interest,<br />
making a commitment)<br />
Learning to Learn (becoming<br />
a better learner, inquiring &<br />
constructing knowledge,<br />
being<br />
self‐directed)<br />
Suggested Teaching/Learning Strategies<br />
Presentation, lecture, question‐and‐answer, large and small<br />
group discussion, development of learning issues,<br />
independent study, review session, teaching others, game,<br />
web‐based instruction<br />
Hands‐on procedure, simulation, case study, role‐play, action<br />
plan, teaching others, question‐and‐answer, brainstorming,<br />
problem‐solving, trouble‐shooting, journal club, developing<br />
research questions, theory and model building, project,<br />
critical review, direct<br />
enterprise contact, precepting, guided practice with feedback<br />
What if…, compare and contrast, concept mapping, crossdisciplinary<br />
teams, cross‐disciplinary cases, multiple examples<br />
within & across contexts, theory & model building, integrated<br />
curriculum<br />
Case study, simulated situations, case presentations, working<br />
in diverse teams, authentic project,<br />
group project, direct enterprise contact, assigned leadership<br />
role, debate, journal club (e.g., using ethics articles)<br />
Authentic project, role modeling, self‐selection activity,<br />
debate, reflective writing, positive reinforcement, learning<br />
prescription<br />
Self‐assessment, self‐ and peer‐feedback, teaching others,<br />
reflective writing, formative assessment, self‐awareness<br />
exercise/inventory<br />
27
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
7. RUBRICS 8<br />
Once identified learning goals and learning objectives with their<br />
specific means of assessment, the next step is identifying a scoring<br />
procedure.<br />
A rubric is a one‐or two page document (usually a matrix) that<br />
describes how a program/specific assignment will be assessed and<br />
the expected student learning outcomes according to varying levels<br />
of quality, from excellent to poor. In this way it serves as an<br />
instructional tool as well as an evaluative one, since it gives students<br />
clear indications about how to improve.<br />
We can identify two main components of a rubric:<br />
1. The list of criteria, or "what counts" in a program or<br />
assignment;<br />
2. The gradations of quality, with descriptions of strong,<br />
middling, and problematic students’ work. The gradations<br />
of quality help students to spot weaknesses and give them<br />
concrete ways to improve their shortcomings.<br />
Using rubrics a tool of evaluation has several advantages:<br />
<br />
<br />
<br />
They are easy to use and to explain then simple grades;<br />
They make professors' expectations very clear;<br />
They provide students with more informative feedback<br />
about their strengths and areas in need of improvement<br />
and in turn they support students’ learning and<br />
development;<br />
8<br />
Adapted from http://www.stanford.edu/dept/presprovost/irds/assessment/downloads/rubrics.pdf<br />
28
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
How to design rubrics<br />
Framing a rubric effectively involve a four step process:<br />
1. Identify learning objectives (see paragraph 5);<br />
2. Identify for each objective its dimensions/traits;<br />
Example (source AACSB)<br />
Oral Communication Evaluation: checklist<br />
1. Preparation/content<br />
a. Opening statement with purpose<br />
b. Organization<br />
c. Content: currency & relevance<br />
d. Quality of slides<br />
e. Quality of conclusion<br />
2. Presentation<br />
a. Voice quality, pace<br />
b. Mannerism<br />
c. Professionalism<br />
d. Use of media<br />
e. Ability to answer questions<br />
3. Select the relevant dimensions; a rubric should fit into oneor<br />
two pages, or no more than 4 to 8 dimensions. In order to<br />
do this we could:<br />
a. List the identified dimensions;<br />
b. Rank them from the most important to least;<br />
c. Eliminate those who are at the bottom in order to<br />
reach the target number;<br />
4. Articulate the expected outcomes in gradation of quality.<br />
(They answer to the questions “yes, yes but, no but, no”, or<br />
“low performance, average, exemplary performance”)<br />
29
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Example: Oral Communication (source AACSB)<br />
Objectives<br />
Organization<br />
Content<br />
Quality of<br />
slides<br />
Low Performance<br />
0‐1 points<br />
No opening<br />
statement or<br />
irrelevant<br />
statement. Loses<br />
focus more than<br />
twice. Does not<br />
manage time<br />
effectively.<br />
Does not meet<br />
assignment<br />
requirements, is not<br />
current, not<br />
objective. Does not<br />
use appropriate<br />
analysis<br />
Sloppy and/or<br />
unprofessional. May<br />
be difficult to read.<br />
Many slides are<br />
superfluous.<br />
Average<br />
2‐3 points<br />
Has opening<br />
statement relevant<br />
to topic and gives<br />
outline of speech.<br />
Mostly organized,<br />
loses focus only<br />
once or twice. Some<br />
transitions. Allows<br />
enough time to<br />
deliver speech,<br />
although it could<br />
have been better<br />
edited.<br />
Fulfill assignment.<br />
Current. Uses<br />
appropriate sources<br />
and is objective.<br />
Reasonable<br />
analysis.<br />
Readable,<br />
professional,<br />
appropriate<br />
number.<br />
Exemplary<br />
Performance<br />
4‐5 points<br />
Has a clear opening<br />
statement that<br />
catches audience’s<br />
interest and gives<br />
overview. Stays<br />
focused<br />
throughout. Has<br />
transition<br />
statements, ties<br />
sections together.<br />
Very effective time<br />
management.<br />
Is more thorough<br />
than average.<br />
Research goes<br />
beyond minimum<br />
requirements.<br />
Strong analysis<br />
Readable,<br />
professional,<br />
imaginative and/or<br />
of high quality<br />
(without being a<br />
distraction)<br />
Earned<br />
Points<br />
Scoring:<br />
10‐19: Does not meet expectations<br />
20‐26: Meets expectations<br />
27‐30: Exceeds expectations<br />
Because rating performance involves human judgment, it is subject<br />
to error, due to personal biases, preceding tasks, and (lack of)<br />
standardization of scoring criteria.<br />
To avoid these common errors,<br />
1. Rate the performance of all students on one task before<br />
going to the next to keep scoring criteria in mind;<br />
2. Whenever possible rate performance without knowledge of<br />
student’s name to avoid halo effects;<br />
30
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
3. Use multiple raters for high‐stakes decisions; and<br />
4. Train your raters to calibrate their scoring by bringing them<br />
together to review their responses in order to identify<br />
patterns of inconsistent responses.<br />
31
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
8. DEFINITIONS<br />
Assessment tools: The instruments used to gather data<br />
about student learning. Tools can be both quantitative and<br />
qualitative. Examples of these tools are: pretests, group<br />
problem solving, performances and demonstrations,<br />
portfolios, peer observations.<br />
Assessment of Student Learning: An ongoing process<br />
aimed at understanding and improving student learning. It<br />
involves making our expectations explicit and public;<br />
setting appropriate criteria and standards for learning<br />
quality; systematically gathering, analyzing, and<br />
interpreting evidence to determine how well performance<br />
matches those expectations and standards; and using the<br />
resulting information to document, explain, and improve<br />
performance. (Angelo, 1995)<br />
Bloom's Taxonomy: The original model developed by<br />
Benjamin Bloom that established six different levels of<br />
learning:<br />
o Evaluation<br />
o Synthesis<br />
o Analysis<br />
o Application<br />
o Comprehension<br />
o Knowledge<br />
Classroom Assessment: A simple method faculty can use<br />
to collect feedback, early and often, on how well their<br />
students are learning what they are being taught. The<br />
purpose of classroom assessment is to provide faculty and<br />
students with information and insights needed to improve<br />
teaching effectiveness and learning quality. (Angelo, 1991)<br />
Continuous Improvement : Continuous improvement<br />
processes are ongoing, data‐supported, research‐driven<br />
processes carried out by a team for the purpose of<br />
improving the effectiveness of work systems and their<br />
results.<br />
Criteria: Statements applied to desired skills, attitudes, or<br />
knowledge that a student should learn to demonstrate<br />
competence. Criteria should be given to the student before<br />
an assessment or evaluation is performed.<br />
Curricula Alignment: The process of alignment the work<br />
students are doing in one or more classes directly to the<br />
achievement of the learning goals.<br />
32
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Direct Measures of Learning: Evidence about student<br />
learning based on student performance that demonstrates<br />
the learning itself. Can be value added, related to<br />
standards, qualitative or quantitative, embedded or not,<br />
using local or external criteria.<br />
Evaluation: The process of measuring the quality of a work<br />
performance, work product, or use of a process against a<br />
set of standards and criteria to make a judgment or<br />
determination if, or to what level, the standards have been<br />
met and students are achieving the instructional objectives.<br />
Fink's taxonomy: Revises Bloom's taxonomy in order to<br />
describe several ways in which learning can be significant.<br />
Fink defines learning in terms of change: "For learning to<br />
occur, there has to be some kind of change in the learner.<br />
No change, no learning. And significant learning requires<br />
that there be some kind of lasting change that is important<br />
in terms of the learner's life".<br />
Formative assessment: The gathering of information<br />
about student learning‐during the progression of a course<br />
or program and usually repeatedly‐to improve the learning<br />
of those students (Leskes, 2002)<br />
Indirect Measures of Learning: Evidence about how<br />
students feel about learning and their learning environment<br />
rather than actual demonstrations of outcome<br />
achievement. Examples include surveys, questionnaires,<br />
interviews, focus groups, and reflective essays.<br />
Learning dimensions: The description of the values,<br />
attitudes, knowledge, skills and behavior a School is<br />
seeking to instill in its students.<br />
Learning goal: Broad statement of desired results,<br />
achievement of which is usually accomplished after<br />
graduation.<br />
Learning Objective: Specific, measurable step to achieve a<br />
goal or outcome<br />
Learning Outcomes: Operational statements describing<br />
specific student behaviors that evidence the acquisition of<br />
desired knowledge, skills, abilities, capacities, attitudes or<br />
dispositions. Learning outcomes can be usefully thought of<br />
as behavioral criteria for determining whether students are<br />
achieving the educational objectives of a program, and,<br />
ultimately, whether overall program goals are being<br />
successfully met. (Allen, Noel, Rienzi & McMillin, 2002)<br />
Program Mission: A holistic vision of the values and<br />
philosophy of the program. The program mission needs to<br />
be consistent with campus and college missions. The overall<br />
question that the mission statement should answer is: What<br />
is the overall unique purpose of this program<br />
Rubric: A rubric is a one‐or two page document (usually a<br />
matrix) that describes how a program/specific assignment<br />
will be assessed and the expected student learning<br />
33
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
<br />
<br />
outcomes according to varying levels of quality, from<br />
excellent to poor.<br />
SMART: Acronym for characteristics of good goals,<br />
outcomes, and/or objectives: Specific, Measurable, Actionoriented,<br />
Realistic, Time‐Based<br />
Summative Assessment: is the process of gathering of<br />
data on student learning, based on specific assessment<br />
criteria, at the conclusion of a course, unit, field experience,<br />
or program. This data is the basis for judging the quality of<br />
student knowledge and skills and can be used for evaluative<br />
purposes (grades) and provide feedback for future<br />
improvement.<br />
34
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
9. TEMPLATES<br />
COURSE SYLLABUS TEMPLATE<br />
Master Program XXX – Term/Year<br />
Course Title:<br />
Professor’s<br />
picture<br />
Professor:<br />
E‐mail:<br />
Office:<br />
Phone:<br />
Assistant:<br />
E‐mail:<br />
Office:<br />
Phone:<br />
Learning Objectives<br />
Course Content<br />
Course Methodology<br />
Course Etiquette<br />
Required Readings<br />
Course Evaluation<br />
About The Instructor – Mr/Mrs<br />
How to get in touch:<br />
Syllabus<br />
Session,<br />
Date, time, duration<br />
Session,<br />
Date, time, duration<br />
Title<br />
Title<br />
35
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Program Directors Annual Report Scheme<br />
Master Program<br />
Annual Report, Year<br />
Master Director<br />
BACKGROUND<br />
Demand Analysis<br />
Competitor Analysis<br />
STRATEGY<br />
PERFORMANCE<br />
Market target<br />
Program Strategic Positioning<br />
Development aims and goals objectives<br />
Value Proposition Analysis (present and future)<br />
How to improve financial performance<br />
AOL Cycle: Closing the Loop (results and future actions)<br />
36
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Relevance of each course for the achievement of the program learning goals<br />
Relevance of each course for the achievement of the program learning<br />
goals<br />
Program: _______________________________________________<br />
COURSE<br />
Degree of relevance (*)<br />
LG 1 LG 2 LG 3 LG 4 LG 5 LG 6 LG 7<br />
(*) Please insert the degree of relevance as follows: LOW, MEDIUM, HIGH<br />
37
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Closing the loop<br />
LGs<br />
MANAGEMENT<br />
KNOWLEDGE<br />
Graduates will be competent<br />
in the core functional areas<br />
and understand how these<br />
areas are integrated (Master<br />
the knowledge required to<br />
manage organizations)<br />
Objectives<br />
Acquire proficiency in all<br />
functional areas of<br />
management<br />
Acquire the ability to<br />
integrate the functional<br />
knowledge adopting a<br />
general management<br />
perspective<br />
Measure<br />
ments<br />
When<br />
Results<br />
and<br />
Actions<br />
38
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Checkpoint Assessment Format<br />
ASSESSMENT OF LEARNING OBJECTIVES<br />
MASTER PROGRAM Edition #<br />
CHECKPOINT:<br />
PLEASE INSERT THE COURSE TITLE<br />
PROGRAM LEARNING OBJECTIVE(S) ASSESSED:<br />
PLEASE INSERT THE PROGRAM LEARNING OBJECTIVE(S)<br />
ASSESSED<br />
DATE:<br />
PLEASE INSERT TIME OF TEST<br />
ASSESSORS:<br />
PLEASE NSERT FACULTY NAME(S)<br />
CALL FOR ACTION:<br />
YES/NO<br />
39
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
PLEASE INSERT MASETR PROGRAM, #<br />
EDITION AND COURSE TITLE<br />
Assessment of learning Objectives ‐ # X/Y<br />
Learning Objective Assessed #1<br />
INSERT LEARNING GOAL ASSESSED<br />
INSERT THE LEARNING OBJECTIVE ASSESSED<br />
Assessment Methodology<br />
PLEASE DESCRIBE THE ADOPTED ASSESSMENT METHODOLOGY AND CRITERIA<br />
Assessment Results<br />
EXEMPLARY (3)<br />
GOOD (2)<br />
FAIR (1)<br />
PLEASE INSERT RUBRICS CORRESPONDING<br />
TO THE OBJECTIVE YOU CHOSE<br />
Exemplar<br />
%<br />
Good<br />
%<br />
Fair<br />
%<br />
PLEASE GIVE BREAKDOWN BASED ON THE ABOVE SCHEME AND PROVIDE THE SUPPORTING EVIDENCE FOR THE RESULTS<br />
(EVALUATION OF EACH STUDENT)<br />
40
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
PLEASE INSERT MASETR PROGRAM, # EDITION AND<br />
COURSE TITLE<br />
Learning Objectives: Comments & Actions<br />
Option A:<br />
Since the results have been in line with expectations, no change is required for next year.<br />
(Nonetheless, in order to ………… there are plans to change….)<br />
Option B:<br />
Since the results were not fully satisfactory, in the next edition of the Program the following changes, with<br />
related goals, will be undertaken:<br />
i) …<br />
ii) ...<br />
Option C:<br />
The results have been in line with expectations, but the following changes will anyway be undertaken in the next edition of the<br />
Program<br />
41
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Example of assessment of Learning Objectives<br />
FT MBA35<br />
ASSESSMENT OF LEARNING OBJECTIVES<br />
MBA 35 FT<br />
CHECKPOINT:<br />
Organizational Behavior and Negotiation<br />
PROGRAM LEARNING OBJECTIVE(S) ASSESSED:<br />
Team work and leadership skills – Obj. 1,2<br />
DATE:<br />
October‐November 2009<br />
ASSESSORS:<br />
Leonardo Caporarello<br />
CALL FOR ACTION:<br />
YES<br />
42
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
MBA 35 FT– Organizational Behavior and Negotiation<br />
Assessment of learning Objectives – 1/2<br />
Learning Objective Assessed #1<br />
Skills<br />
Graduates will work<br />
effectively in teams, using<br />
interpersonal,<br />
communication and<br />
Display the ability to work with others<br />
Assessment Methodology<br />
The ability of the candidates to work with others was assessed through specific self assessment tools for behavior and attitude orientation. During the class,<br />
candidates were given a series of team tasks to perform with the clear goal of observing and trying to understand group dynamics and team development.<br />
After each simulation theoretical framework for analysis was provided and discussed in plenary sessions. At the end of the Graduates should display the<br />
ability to work with others dynamics and team development. After each simulation theoretical framework for analysis was provided and discussed in plenary<br />
sessions. At the end of the course, part of the individual exam consisted of an individual analysis of a teamwork situation.<br />
Assessment Results<br />
EXEMPLARY (3)<br />
GOOD (2)<br />
FAIR (1)<br />
Graduates should display the<br />
ability to work with others<br />
Graduates participate wholeheartedly in group<br />
activity, gaining other people’s trust and<br />
benevolence. They know how to leverage on<br />
other people background and culture to<br />
establish fertile relationships in terms of<br />
opening new perspective, finding new solutions,<br />
raising new stimulating goals. However, they<br />
also know when to critically rely on third party<br />
contributions to overcome their own<br />
Blue Class<br />
Graduates participate in group activity,<br />
responsibly willingly, timely and fruitfully,<br />
bringing the required contribution.<br />
They know how to relate to people of different<br />
backgrounds & cultures, according the principle<br />
of mutual respect & support and fully recognize<br />
other group members’ contribution.<br />
Yellow Class<br />
MBA 35 Cohort<br />
Graduates search to minimize their<br />
contribution to group activities, incurring<br />
delays, avoiding commitment and taking<br />
a passive stance.<br />
They do not always recognize other<br />
people’s contribution.<br />
Exemplary<br />
Good<br />
9%<br />
46%<br />
Exemplary<br />
Good<br />
12%<br />
43&<br />
Exemplary<br />
Good<br />
10%<br />
45%<br />
Fair 46%<br />
Fair 45% Fair<br />
45%<br />
43
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
MBA 35 FT – Organizational Behavior and negotiation<br />
Learning Objectives: Comments & Actions<br />
Since the results were not completely in line with the targets expected, in the next edition of the program there will be given more<br />
specific task and theoretical frameworks on team structures and management reducing the impact of themes like power and politics on<br />
the module.<br />
44
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
MBA 35 FT– Organizational Behavior and Negotiation<br />
Assessment of learning Objectives – 2/2<br />
Learning Objective Assessed #2<br />
Teamwork & Leadership<br />
Skills<br />
Graduates will work<br />
effectively in teams, using<br />
interpersonal,<br />
communication and<br />
leadership skills.<br />
Be aware of the role and responsibilities of leadership and behave accordingly<br />
Assessment Methodology<br />
The ability of the candidates to identify roles and responsibilities and display the ability in conflict resolution was assessed through specific self assessment<br />
tools for behavior and in‐class simulations. After each simulation theoretical framework for analysis was provided and discussed in plenary sessions pointing<br />
out the relationship between output from the simulation and behavior adopted. At the end of the course, part of the individual exam consisted of a situation<br />
based multiple choice test and individual analysis of a teamwork situation.<br />
Assessment Results<br />
EXEMPLARY (3)<br />
GOOD (2)<br />
FAIR (1)<br />
Graduates should be aware<br />
of the role and<br />
responsibilities of leadership<br />
and behave accordingly<br />
Graduate possesses the ability to motivate,<br />
persuade, negotiate and resolve conflicts both<br />
in handling simple managerial tasks and in<br />
leading/participating in a wider change<br />
management process.<br />
Graduates possess the ability to motivate,<br />
persuade, negotiate, resolve conflicts in handling<br />
simple managerial tasks and in small groups of<br />
people, whilst lacking the standing & perspective<br />
to do the same when participating to a wider<br />
change management process and/or in larger<br />
groups.<br />
Graduates understand roles and<br />
responsibilities of leadership but have a<br />
tendency to eschew them, maybe<br />
sensing that they are personally not fitted<br />
for the role.<br />
Blue Class<br />
Exemplary<br />
Good<br />
Fair<br />
30%<br />
56%<br />
14%<br />
Yellow Class<br />
Exemplary<br />
Good<br />
Fair<br />
36%<br />
36%<br />
28%<br />
MBA 35 Cohort<br />
Exemplary<br />
Good<br />
Fair<br />
33%<br />
46%<br />
21%<br />
45
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
MBA 35 FT – Organizational Behavior and negotiation<br />
Learning Objectives: Comments & Actions<br />
The results were in line with the targets. However in the next edition of the program more time will be given to class discussion and<br />
simulation. Relevance will be given to class and teamwork participation, reducing the space reserved to theory and modeling.<br />
46
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Peer Overall Evaluations FT MBA 35<br />
MBA 35 FT<br />
LEARNING GOAL<br />
Teamwork & Leadership Skills<br />
Graduates will work effectively in teams, using interpersonal, communication and<br />
leadership skills.<br />
OBJECTIVES<br />
Display the ability to work with others<br />
Be aware of the role and responsibilities of leadership and behave accordingly<br />
Be able to communicate ideas effectively<br />
47
<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />
Assessment Handbook<br />
Assessment methodology<br />
Peer Evaluation – Study Group 1 st Term Managerial Excellence<br />
RATER: _______________________________________________________________<br />
CLASS: _________________________________________<br />
Please provide in the following table an objective evaluation of your colleagues based on observations you have made in the various group meetings<br />
and during any interpersonal interactions you may have had. The rating scale must be as follows:<br />
1=Rarely 2=Sometimes 3=Regularly 4=Very often 5=Always<br />
1.<br />
2.<br />
3.<br />
4.<br />
5.<br />
LAST NAME OF THE<br />
RATED GROUP MEMBER<br />
GROUP<br />
PARTICIPATION<br />
PERSONAL<br />
COMMITMENT<br />
GOAL<br />
ORIENTATION<br />
QUALITY<br />
ORIENTATION<br />
COMMUNICATION<br />
SKILLS<br />
LEADERSHIP<br />
GROUP S/he attends the group meetings, participates in the discussions, provides meaningful contributions to the discussions, and<br />
PARTICIPATION happily shares personal experiences, knowledge and information. S/he provides support, as needed, to peers and colleagues.<br />
PERSONAL S/he adds value to the group work through an active participation relevant to achieve the desired outcome. His/her contribution<br />
COMMITMENT is matched by a sense of involvement in the team process. S/he adheres to all of the group commitments and agreements.<br />
GOAL S/he is oriented to the desired outcome and strives to provide input that is relevant to the defined goal.<br />
ORIENTATION S/he contributes in a meaningful way to create the final document (or output) that is to be presented.<br />
QUALITY S/he is focused on the quality of the outcome and pays attention to all details to make the final result as perfect as possible.<br />
ORIENTATION Any necessary calculations is performed properly and not overdone. The focus on quality does not slow down the group.<br />
COMMUNICATION S/he communicates clearly, both verbally and in writing. S/he listens effectively and is tolerant of different opinions.<br />
SKILLS<br />
LEADERSHIP S/he is capable of guiding the group toward the desired results through a balanced people and goal-oriented approach.<br />
S/he is able to build team spirit, create a sense of cooperation, encourage input from all team members, and achieve consensus<br />
when necessary. S/he is able to persuade and influence colleagues in order reach consensus.<br />
48