29.01.2015 Views

Download Document - SDA Bocconi

Download Document - SDA Bocconi

Download Document - SDA Bocconi

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook


Table of Contents<br />

1. INTRODUCTION ................................................................................................................... 1<br />

2. WHAT IS (NOT) ASSESSMENT ............................................................................................ 3<br />

3. THE ASSESSMENT PROCESS ................................................................................................. 4<br />

STEP 1 ‐ ESTABLISHING LEARNING GOALS AND OBJECTIVES ........................................................................ 6<br />

STEP 2 ‐ ALIGNMENT OF CURRICULA WITH LEARNING GOALS ..................................................................... 7<br />

STEP 3 ‐ IDENTIFICATION OF ASSESSMENT MEASURES ................................................................................ 8<br />

STEP 4 ‐ COLLECTION, ANALYSIS AND DISSEMINATION OF ASSESSMENT INFORMATION .................................... 9<br />

STEP 5 ‐ USE OF ASSESSMENT INFORMATION FOR CONTINUOUS IMPROVEMENT .......................................... 10<br />

4. EVALUATION VS ASSESSMENT .......................................................................................... 11<br />

5. LEARNING GOALS AND LEARNING OBJECTIVES .................................................................. 12<br />

<strong>SDA</strong> BOCCONI’S DIMENSIONS AND LEARNING GOALS .............................................................................. 12<br />

DEFINING LEARNING GOALS ................................................................................................................ 14<br />

DEFINING LEARNING OBJECTIVES ......................................................................................................... 15<br />

6. TAXONOMIES .................................................................................................................... 18<br />

6.1 BLOOM’S TAXONOMY USED WHILE WRITING OBJECTIVES ................................................................. 19<br />

6.2 FINKS’S TAXONOMY USED WHILE WRITING OBJECTIVES .................................................................... 22<br />

7. RUBRICS ............................................................................................................................ 28<br />

HOW TO DESIGN RUBRICS ................................................................................................................... 29<br />

8. DEFINITIONS ..................................................................................................................... 32<br />

9. TEMPLATES ....................................................................................................................... 35<br />

COURSE SYLLABUS TEMPLATE ............................................................................................................. 35<br />

PROGRAM DIRECTORS ANNUAL REPORT SCHEME .................................................................................. 36<br />

RELEVANCE OF EACH COURSE FOR THE ACHIEVEMENT OF THE PROGRAM LEARNING GOALS ............................ 37<br />

CLOSING THE LOOP............................................................................................................................ 38<br />

CHECKPOINT ASSESSMENT FORMAT ..................................................................................................... 39<br />

EXAMPLE OF ASSESSMENT OF LEARNING OBJECTIVES FT MBA35 ............................................................ 42<br />

PEER OVERALL EVALUATIONS FT MBA 35 ............................................................................................ 47


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

1. INTRODUCTION<br />

In recent years, international institutions of higher education have<br />

recognized that the systematic assessment of student learning is<br />

essential to monitor quality and provide the information that leads<br />

to improvement.<br />

A full commitment to teaching and learning must include assessing<br />

and documenting what and how much students are learning and<br />

use this information to improve the educational experience offered<br />

by the institution.<br />

In many ways, assessment is a common sense process that the<br />

institution’s faculty already follows when it articulates the main<br />

goals for each course, checks if students have achieved them, and<br />

then uses these results to improve the course itself.<br />

This guide is designed to take advantage of what <strong>SDA</strong> <strong>Bocconi</strong><br />

School of Management is already doing by formalizing the process<br />

and sharing the individual efforts<br />

to the whole institution.<br />

If implemented effectively, the<br />

assessment of students’ learning is<br />

going to involve the mutual effort<br />

of students, administrators, and<br />

academic professionals.<br />

The idea at the heart of<br />

the assessment is that<br />

you need to know how<br />

you are doing before you<br />

can do better.<br />

The main focus of the assessment is the student, but it is also<br />

closely linked to the assessment of the effectiveness of the<br />

organization, which is important as a mean to monitor and improve<br />

the environment where teaching and learning occur.<br />

A good assessment can enhance quality at all levels of the School<br />

by providing it with the evidences necessary in order to steer<br />

effectively the decision‐making in many areas (thus including<br />

programmatic changes, classroom teaching modifications, support<br />

service adjustments, policy or procedure revisions, and structural<br />

reorganizations).<br />

The School is called to put a big effort in this process; hence,<br />

Assessment is a way to document the <strong>SDA</strong> <strong>Bocconi</strong> effectiveness<br />

and its ongoing improvement.<br />

1


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

A systematic, ongoing, cycle of goal setting, attainment<br />

measurement, and use results to make informed decisions is crucial<br />

to excellent performance.<br />

Thought in these terms, Assessment:<br />

Helps to determine whether the School’s mission has been<br />

accomplished<br />

Provides a rationale for modifying existing curriculum and<br />

instructional methods<br />

Enables the School to examine student’s growth and<br />

satisfaction<br />

2


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

2. WHAT IS (NOT) ASSESSMENT<br />

ASSESSMENT IS… a cyclical and systematic<br />

process<br />

ASSESSMENT IS NOT… a boring and<br />

annoying administrative activity<br />

ASSESSMENT AIMS TO… understand<br />

what students are/ are not learning<br />

ASSESSMENT DOES NOT AIM TO…<br />

horn in on a faculty member’s<br />

classroom<br />

ASSESSMENT PROVIDES…feedback<br />

to reinforce student learning<br />

ASSESSMENT DOES NOT<br />

PROVIDE… a means of punishment<br />

ASSESSMENT IMPROVES…<br />

Student learning<br />

ASSESSMENT DOES NOT INVOLVE…<br />

Exclusively faculty and students in<br />

classroom. It is mutually the<br />

responsibility of faculty and out of class<br />

educators.<br />

3


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

3. THE ASSESSMENT PROCESS<br />

Palomba and Banta (1999) define the outcomes assessment process<br />

as:<br />

The systematic collection, review, and use<br />

of information about educational programs<br />

undertaken for the purpose of improving<br />

student learning and development.<br />

“ … Measures of learning can assure external constituents such as<br />

potential students, trustees, public officials, supporters, and<br />

accreditors, that the organization meets its goals” (AACSB 2007,<br />

p.60).<br />

For this reason, at the heart of the AoL, there is the School’s<br />

mission that inspires its view of the future.<br />

Assessment is an ongoing, iterative process that<br />

1. ensures that students’ learning is measured and enhanced on a<br />

regular and continuous basis,<br />

2. uses results to inform decisions and make improvements.<br />

In order to implement it effectively, a careful planning is thus<br />

necessary.<br />

Learning goals and outcomes must be clearly specified, appropriate<br />

measures must be selected, data collection must be carefully<br />

executed and, most important, results must be shared for<br />

improvements to occur. The figure below illustrates the cycle of<br />

intertwined activities that facilitates continuous improvement.<br />

4


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Step 4: collection,<br />

analysis and<br />

dissemination of<br />

assement<br />

information<br />

Step 5: use of<br />

assessment<br />

information for<br />

contiunous<br />

improvement,<br />

Step 1: Establish<br />

including<br />

learning goals and<br />

documentation that<br />

objectives<br />

the ongoing process<br />

has been carried out<br />

on a systematic<br />

basis<br />

To contribute to the<br />

development of individuals<br />

and the environment where<br />

they operate by creating and<br />

disseminating innovative<br />

managerial knowledge.<br />

“Empowering lives through<br />

knowledge and imagination”<br />

Step 2: Alignment<br />

of curricula with<br />

adopted goals<br />

Step 3:<br />

Identification of<br />

assessment<br />

instruments and<br />

measures<br />

5


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Step 1 ‐ Establishing Learning Goals and Objectives<br />

What are students LO expected to achieve<br />

How do these LO relate to <strong>SDA</strong> <strong>Bocconi</strong>’s Mission<br />

Each learning goal should describe the desired educational<br />

outcome that students should be able to accomplish when<br />

they graduate from the program, regardless of their major<br />

or concentration.<br />

Learning goals are broad statements and, taken alone, are<br />

not suitable for assessment. Therefore, each learning goal<br />

must be translated into one or more learning objectives<br />

which describe a measurable attribute of the overall<br />

learning goal.<br />

For each learning objective, an assessment device or<br />

process must be developed to allow an evaluation of<br />

student performance on that objective providing a<br />

reasonable basis for conclusions about student<br />

performance on the overall learning goal.<br />

Program learning goals should include goals that address<br />

some, or part, of the general knowledge and skills areas<br />

(e.g., communication skills, problem‐solving abilities,<br />

ethical reasoning skills, language skills, technology skills,<br />

etc.) as well as management‐specific knowledge and skill<br />

areas (e.g., directly relate to management tasks that form<br />

the business foundation of degree requirements)<br />

6


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Step 2 ‐ Alignment of Curricula with Learning Goals<br />

Which curriculum components are more conducive to<br />

supporting LG<br />

What could be the appropriate measurement environment<br />

There should be clear evidence that the work students are<br />

doing in one or more classes directly supports students’<br />

achievement of the learning goals.<br />

The alignment of learning goals and curricula is critical. If<br />

learning goals are adopted but are not addressed in the<br />

curricula, the outcomes assessment process will be<br />

worthless.<br />

7


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Step 3 ‐ Identification of assessment measures<br />

Which instruments and measures will be used to determine<br />

the outcome achievement<br />

“Course‐embedded” measures relate to specific course<br />

assignments in a class where the students’ work on that<br />

assignment may also be used for outcomes assessment<br />

purposes.<br />

In the course embedded methods, course assignments or<br />

other student demonstrations from a course are evaluated<br />

for the purposes of AoL through a separate, distinct process<br />

that is driven by criteria established by the faculty (plural).<br />

Assessment measures should be selected after learning<br />

goals/objectives have been established.<br />

8


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Step 4 ‐ Collection, analysis and dissemination of<br />

assessment information<br />

When and how often should these outcomes be assessed<br />

Which means of dissemination will be used<br />

Once data are collected about students’ performance<br />

through the outcomes assessment process, it must be<br />

shared with and analyzed by the appropriate faculty<br />

committees and the leadership of the program.<br />

9


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Step 5 ‐ Use of assessment information for continuous<br />

improvement<br />

What results would indicate a satisfactory level of<br />

accomplishment<br />

How will these results be used to improve programs<br />

Regardless of the assessment processes that are deployed,<br />

the school program must demonstrate assessment data are<br />

being used to inform the leadership and faculty about the<br />

effectiveness of their educational programs.<br />

For AACSB accreditation reviews, schools should present<br />

examples of student’s performance measured on<br />

assessment criteria and document how and where<br />

assessment outcomes have been used for continuous<br />

improvement of curricula.<br />

10


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

4. EVALUATION VS ASSESSMENT 1<br />

Evaluation<br />

Focuses on the professor and the<br />

teaching performance<br />

individual performance<br />

Helps individuals know how they<br />

performed.<br />

Results in a grade for the activity or<br />

course.<br />

May or may not be used for<br />

improvement of instruction.<br />

Assessment<br />

Focuses on the student and the<br />

learning environment<br />

group performance<br />

Helps stakeholders know how<br />

students performed.<br />

Results in a plan for improvement<br />

Always used for improvement of<br />

instruction/ learning<br />

Grades, when used alone, are no longer regarded as adequate<br />

indicators of student learning.<br />

In particular,<br />

Grades are not specific and provide not sufficiently detailed<br />

information about which course outcomes students are<br />

mastering well and which are giving them trouble.<br />

Grading standards often vary widely among different<br />

instructors and do not indicate the same degree of mastery<br />

of course outcomes.<br />

Grade inflation (easy tests, generous grading, extra‐credit<br />

bonuses) sometimes presents a misleading indicator of<br />

student mastery of course outcomes.<br />

Faculty have the opportunity to use the course assessment process<br />

as a stimulus to discuss course content with each other and, based<br />

on the results of an assessment, determine how they can improve<br />

student.<br />

1 Adapted from http://www.pvc.maricopa.edu/AI/documents/Ass_sec1.pdf<br />

11


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

5. LEARNING GOALS AND LEARNING OBJECTIVES<br />

Goals express what the School wants its students to be.<br />

Objectives describe what the School wants its students to<br />

do.<br />

The definition of learning goals is a key element in how the school<br />

defines itself. This implies that the learning goals are closely related<br />

and connected to the School and the program’s mission.<br />

<strong>SDA</strong> <strong>Bocconi</strong>’s dimensions and learning goals<br />

<strong>SDA</strong> <strong>Bocconi</strong> School of Management has a description of the<br />

values, attitudes, knowledge, skills and behavior it is seeking to<br />

instill in its students. These skills and attitudes are broadly applied<br />

to all Master Programs.<br />

To help students to prepare for a successful professional career and<br />

to fully develop their potential as individuals, <strong>SDA</strong> <strong>Bocconi</strong> has<br />

defined three categories of Learning Dimensions which are a<br />

reference for all Master Programs, when defining and revising their<br />

Learning Goals and which are linked to the School’s Mission.<br />

At the forefront stays a strong and integrated managerial<br />

knowledge <strong>SDA</strong> <strong>Bocconi</strong> aims at creating, disseminating and<br />

conveying to teaching and learning experiences, both for students<br />

and Faculty.<br />

The second dimension is a deep understanding and awareness of<br />

the environment in which we operate, students, executives,<br />

companies, institutions and all the stakeholders and actors of<br />

economic systems, both domestic and global.<br />

The third is a sound equipment of managerial skills and culture,<br />

which make it possible to turn knowledge into a responsible and<br />

sustainable set of actions.<br />

12


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

In the beneath table the three dimensions and subsequent learning<br />

goals are represented.<br />

Dimension<br />

Learning Goal<br />

MANAGEMENT<br />

KNOWLEDGE<br />

1 Management Knowledge<br />

Graduates will be competent in the core<br />

functional areas and understand how these areas<br />

are integrated.<br />

ENVIRONMENT 2 Entrepreneurial mindset and creativity<br />

Graduates will acquire an entrepreneurial and<br />

creative mindset.<br />

3 Diversity and global business issues<br />

Graduates will understand the impact of diversity<br />

on organizations, and will have an understanding<br />

of the main global issues affecting companies,<br />

public/non‐profit organizations, banks and<br />

financial institutions.<br />

4 Ethics & Integrity<br />

Graduates will be conscious of the needs to<br />

comply with rules, regulations, social norms.<br />

MANAGEMENT<br />

SKILLS<br />

5 Strategic Thinking & Problem Solving Skills<br />

Graduates will formulate effective strategies and<br />

make decisions, using problem‐solving<br />

techniques.<br />

6 Teamwork & Leadership Skills<br />

Graduates will work effectively in teams, using<br />

interpersonal, communication and leadership<br />

skills.<br />

7 Project Management Skills<br />

Graduates will be able to effectively plan and<br />

implement projects and activities and to<br />

coordinate tasks and monitor their progress.<br />

13


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Defining learning goals<br />

Answering clearly and specifically is essential to obtain a consensus<br />

from faculty members in the program on the program’s vision,<br />

values and goals that will serve as guiding principles for developing<br />

outcomes and collecting data 2 .<br />

Hence, the first question you should answer is<br />

“What would the program accomplish”<br />

Characteristics of an effective mission statement<br />

RELEVANT: what is the aim of the program (it should be also<br />

related and aligned to the School’s mission)<br />

SPECIFIC: what is the student target<br />

CLEAR: what is the real contribute to the development of the<br />

participants<br />

INFLUENTIAL AND DISTINCTIVE: what is the reason why the<br />

program exists A mission should provide a sense of direction,<br />

and guide decision‐making processes<br />

Normally, four to ten learning goals are specified for each degree<br />

program. Core learning goals include general knowledge and skill<br />

goals and management‐specific learning goals.<br />

2 Adapted from http://www.stanford.edu/dept/presprovost/irds/assessment/downloads/mission_stmt.pdf<br />

14


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Defining learning objectives<br />

A learning objective is a statement of what students will be able to<br />

do, specifically what knowledge, skills and attitudes they will<br />

demonstrate to have acquired outside the classroom (in context)<br />

with what they have learned.<br />

This means that the point of view which should be adopted is<br />

“student‐based” rather than “instructor‐based” (e.g., “After this<br />

course, my students should be able to…” rather than “During this<br />

course I’ll talk about…”).<br />

A clear learning objective serves the following intents:<br />

Helps planners develop the program content<br />

Guides selection of teaching/learning activities that will best<br />

achieve objectives<br />

Serves to connect content and assessment<br />

Gives learners a clear picture of what to expect and what’s<br />

expected of them<br />

Forms the basis for evaluating learner, and program<br />

effectiveness<br />

Key elements of a useful learning objective 1<br />

SPECIFIC: according the learning goals, the program and the<br />

School’s mission<br />

MEASUREABLE: it makes the learning goals visible (serves as an<br />

indicator)<br />

ATTAINABLE: for target audience within scheduled time and<br />

specified conditions<br />

RESULTS‐ORIENTED: it should produces concrete outcomes<br />

TARGETED: to the learner and to the desired level of<br />

learning/thinking<br />

To create specific, measurable, and results‐oriented<br />

objectives:<br />

o<br />

o<br />

It’s helpful to finish the sentence, “After this course,<br />

you should be able to…”<br />

Start with an observable action word that captures<br />

what the learner should be able to do.<br />

15


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

o<br />

Avoid ill‐defined terms that are open to variable<br />

interpretation (e.g., understand, learn, grasp); use<br />

instead terms that describe directly observable<br />

behaviors.<br />

o When necessary, specify criteria concerning<br />

expected standard of performance<br />

To create attainable learning objectives:<br />

o<br />

Consider the beginning level of understanding/skill<br />

of your learners and craft your objective to move<br />

them to the next level.<br />

o Consider and specify when appropriate the<br />

conditions under which performance will take place<br />

(e.g., “On a written exam, describe…” or “With a<br />

standardized or actual context, demonstrate…”)<br />

o<br />

Limit number of objectives to major learning points<br />

you would like students to walk away with.<br />

To create objectives targeted to the audience and desired<br />

level of learning/thinking:<br />

o<br />

Ask yourself whether you want learners to be able<br />

to know, do and be, whenever possible, refer to<br />

commonly adopted taxonomies. We suggest<br />

Bloom’s and Fink’s taxonomies (see paragraphs 6<br />

and 7). Use the taxonomy to describe the desired<br />

level of learning/thinking (table 1 A and 1 B)<br />

o Match your action verb to the desired level (Table 2<br />

A or B).<br />

o Match learning objective with appropriate<br />

teaching/learning strategy (Table 3 A or B).<br />

Student performance on learning goals must be assessed<br />

systematically and routinely.<br />

No one approach to assurance of learning is prescribed.<br />

Benchmarking is not required.<br />

Assessment programs should include direct measures of<br />

learning. Course grades are not program assessment<br />

measures.<br />

Program assessment does not require every student to be<br />

assessed. Sampling is acceptable as long as an appropriate<br />

and representative sampling methodology is utilized.<br />

16


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Assessment results must be analyzed, disseminated, and<br />

utilized by the faculty toward curriculum planning.<br />

The work of individual students – not student teams – is to<br />

be used to assess learning outcomes (except for “Team<br />

Work” Learning Goal).<br />

17


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

6. TAXONOMIES<br />

"Taxonomy” simply means “classification”, so the taxonomies<br />

provided in the following paragraphs are an attempt (within the<br />

behavioral paradigm) to classify forms and levels of learning 3 .<br />

3<br />

The content of the tables are adapted from ©Teacher & Educational<br />

Development, University of New Mexico School of Medicine, 2005<br />

18


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

6.1 Bloom’s Taxonomy 4 Used While Writing Objectives<br />

Bloom’s classification of cognitive skills is widely used in instruction<br />

planning since its creation by a committee of educators chaired by<br />

Benjamin Bloom in 1956.<br />

It identifies three “domains” of learning:<br />

Cognitive: mental skills (Knowledge)<br />

Affective: growth in feelings or emotional areas (Attitude)<br />

Psychomotor: manual or physical skills (Skills)<br />

Each domain is organized as a series of levels or pre‐requisites. It is<br />

suggested that one cannot effectively — or ought not try to —<br />

address higher levels until those below them have been covered (it<br />

is thus effectively serial in structure). As well as providing a basic<br />

sequential model for dealing with topics in the curriculum, it also<br />

suggests a way of categorizing levels of learning, in terms of the<br />

expected ceiling for a given program. Thus in the Cognitive domain,<br />

training for technicians may cover knowledge, comprehension and<br />

application, but not concern itself with analysis and above, whereas<br />

full professional training may be expected to include this and<br />

synthesis and evaluation as well 5 .<br />

4 Bloom B., Taxonomy of Educational Objectives Handbook, 1956<br />

5 From http://www.learningandteaching.info/learning/bloomtax.htm<br />

19


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Table 1A: Example Action Verbs for Each Level of Learning<br />

Category<br />

Knowledge<br />

(Recall and<br />

Understanding)<br />

Application<br />

Problem‐<br />

Solving<br />

(Analyzing,<br />

Synthesizing,<br />

Evaluating)<br />

Example Action Verbs<br />

Associate<br />

Compare<br />

Contrast<br />

Define<br />

Calculate<br />

Demonstrate<br />

Draw<br />

Employ<br />

Advocate<br />

Analyze<br />

Assess<br />

Challenge<br />

Compose<br />

Describe<br />

Differentiate<br />

Distinguish<br />

Identify<br />

Estimate<br />

Give example<br />

Illustrate<br />

Locate<br />

Conclude<br />

Construct<br />

Create<br />

Critique<br />

Debate<br />

Indicate<br />

List<br />

Name<br />

Paraphrase<br />

Measure<br />

Operate<br />

Perform<br />

Prescribe<br />

Decide<br />

Defend<br />

Derive<br />

Design<br />

Evaluate<br />

Recognize<br />

Repeat<br />

Restate<br />

Review<br />

Record<br />

Set up<br />

Sketch<br />

Formulate<br />

Infer<br />

Judge<br />

Organize<br />

Plan<br />

Show<br />

State<br />

Summarize<br />

Tell<br />

Solve<br />

Trace<br />

Use<br />

Propose<br />

Rank<br />

Recommend<br />

Select<br />

Suggest<br />

Table 2A: Levels of Thinking/Learning<br />

Category Dimension Definition<br />

Knowledge Recalling Rote recall: Know common terms, specific facts,<br />

methods, procedures, concepts, principles<br />

Comprehending Interpolation or interpretation: Understand, estimate<br />

future implied consequences, justify methods and<br />

procedures<br />

Application Applying Using a concept in a new context: Apply theory, solve<br />

problems, construct graphs, demonstrate procedure<br />

Problem‐<br />

Solving<br />

Analyzing<br />

Synthesizing<br />

Evaluating<br />

Breaking something down and understanding its<br />

structure, the relationship between parts, the<br />

organizational principles:<br />

Recognize unstated assumptions and logical fallacies,<br />

distinguish between facts & inferences, determine<br />

relevance<br />

Building a structure/pattern from diverse elements: Write<br />

well‐organized essay, propose research question, develop<br />

plan for solving a problem, formulate a classification<br />

scheme<br />

Judging the value of ideas, works, solutions, materials:<br />

Judge logical consistency, adequacy of data in support of<br />

conclusions, value of work by internal & external<br />

standards<br />

20


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Table 3 A: Teaching/Learning Strategies Best Suited for Each Level of Learning<br />

Desired Dimension<br />

Knowing and<br />

comprehending<br />

Applying<br />

Analyzing<br />

Synthesizing<br />

Evaluating<br />

Suggested Presentational Strategies<br />

Presentation, lecture, question‐and‐answer, small group discussion,<br />

development of learning issues, self‐awareness exercises/tests, review<br />

sessions, teaching others, independent study, web‐based instruction<br />

Hands‐on, case study, simulation, role‐playing, action plan, teaching<br />

others, direct patient contact, guided practice with feedback, rolemodeling<br />

Question‐and‐answer, brainstorming, case study, problem‐solving,<br />

trouble‐shooting, role‐playing, article discussion<br />

Case study, writing, concept mapping, theory and model building,<br />

teaching others, developing research questions, direct enterprise contact<br />

Case study, critical review, self and group assessment/reflection,<br />

reflective writing, direct enterprise contact<br />

21


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

6.2 Finks’s Taxonomy 6 Used While Writing Objectives<br />

In 2003 Finks elaborated a new taxonomy. Individuals and<br />

organizations involved in higher education are expressing a need for<br />

important kinds of learning that do not emerge easily from the Bloom<br />

taxonomy, for example: learning how to learn, leadership and<br />

interpersonal skills, ethics, communication skills, character, tolerance,<br />

the ability to adapt to change, etc. My interpretation of these<br />

statements is that they are expressing a need for new kinds of<br />

learning, kinds that go well beyond the cognitive domain of Bloom's<br />

taxonomy and even beyond cognitive learning itself. This suggests<br />

that the time may have arrived when we need a new and broader<br />

taxonomy of significant learning.<br />

With an awareness of this need in mind, I have reviewed descriptions<br />

of high quality teaching and learning and have attempted the task of<br />

creating a new taxonomy, one that describes new kinds of significant<br />

learning 7 .<br />

6 Fink L.D., Creating Significant Learning Experiences, 2003<br />

7 Fink L.D., What is ''significant learning”,University of Oklahoma, 2003<br />

22


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Table 1B: Example Action Verbs for Each Dimension of Learning<br />

Dimension Action Verbs Objects<br />

FOUNDATIONAL KNOWLEDGE – What key information, ideas, perspectives are important for<br />

learners to know<br />

Understanding and<br />

Remembering, (developing a<br />

full understanding of the<br />

concepts associated with a<br />

subject to a degree that<br />

allows explanations,<br />

predictions, etc.)<br />

Associate, Compare,<br />

Contrast, Define, Describe,<br />

Explain, Give example,<br />

Identify, Illustrate, Indicate,<br />

List, Name, Paraphrase,<br />

Predict, Recite, Recognize,<br />

Repeat, Restate, Tell<br />

Facts, concepts, theories,<br />

relationships, models,<br />

perspectives, structures,<br />

organizations, purposes,<br />

proposals, problems, results,<br />

conclusions, plans<br />

APPLICATION – What kinds of thinking, complex projects, and skills is it important for learners<br />

to be able to do/manage<br />

Critical Thinking (analyzing<br />

and critiquing issues and<br />

situations)<br />

Practical Thinking<br />

(developing problem‐solving<br />

and decision‐making<br />

capabilities)<br />

Creative Thinking (creating<br />

new ideas, products, and<br />

perspectives)<br />

Managing Complex Projects<br />

(being able to coordinate and<br />

sequence multiple tasks in a<br />

single project/case and/or<br />

multiple projects/cases)<br />

Analyze, Assess, Audit,<br />

Catalog, Categorize, Classify,<br />

Compare, Contrast, Decipher,<br />

Deduce, Derive, Determine,<br />

Diagram, Differentiate,<br />

Dissect, Distinguish, Examine,<br />

Formulate, Hypothesize,<br />

Infer, Interpret, Label, Locate,<br />

Measure, Organize, Query,<br />

Separate, Trace<br />

Advise, Answer, Apply,<br />

Calculate, Certify, Choose,<br />

Consult, Debate, Decide,<br />

Determine, Diagnose,<br />

Evaluate, Give evidence,<br />

Judge, Justify,, Predict,<br />

Prescribe, Propose, Prove,<br />

Rank, Select, Solve, Suggest,<br />

Test<br />

Abstract, Adapt, Amend,<br />

Author, Compose, Construct,<br />

Convert, Create, Design,,<br />

Develop, Devise, Discover,<br />

Draw, Envision, Experiment,,<br />

Fabricate, Imagine, Improve,<br />

Refine, Reform, Sketch,<br />

Theorize, Transform, Write<br />

Administer, Assign, Coach,<br />

Communicate, Complete,<br />

Conduct, Coordinate,<br />

Delegate, Develop, Evaluate,<br />

Facilitate, Follow Up, Guide,<br />

Implement, Manage,<br />

Organize, Plan, Prioritize,<br />

Strategize, Supervise,<br />

Summarize, Teach, Time‐line,<br />

Train<br />

Ideas, issues, situations,<br />

proposals, processes, results,<br />

conclusions, theories,<br />

assumptions<br />

Problems, issues, conundrums<br />

Ideas, plans, products,<br />

objects, premises,<br />

perspectives, models,<br />

theories<br />

Tasks, timelines, cases,<br />

projects<br />

23


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Dimension Action Verbs Objects<br />

Performance Skills<br />

(developing capabilities in<br />

carrying out psychomotor<br />

activities)<br />

Conduct, Demonstrate, Do,<br />

Employ, Execute, Exhibit,<br />

Operate, Perform, Produce,<br />

Set up, Use<br />

Procedures, routines,<br />

processes, interviews<br />

INTEGRATION – What connections should learners be able to recognize and make within and<br />

beyond this learning experience<br />

Interdisciplinary Learning<br />

(connecting ideas, disciplines,<br />

perspectives, contexts),<br />

Learning Communities<br />

(connecting people),<br />

Learning and Living/Working<br />

(connecting, different realms<br />

of life)<br />

Associate, Combine,<br />

Compare, Concept map,<br />

Connect, Contrast, Correlate,<br />

Differentiate, Integrate, Link,<br />

Relate, Synthesize<br />

Ideas, disciplines,<br />

perspectives, contexts,<br />

people, domains, realms<br />

HUMAN DIMENSION – What should learners learn about themselves and about interacting with<br />

others<br />

Relationships (with, peers,<br />

supervisors, patients, others),<br />

Self‐Authorship (learning to<br />

create and, take responsibility<br />

for one’s own life),<br />

Leadership (becoming an<br />

effective leader), Ethics,<br />

Character Building (living by,<br />

ethical principles),<br />

Multicultural Education<br />

(being culturally, sensitive in<br />

interactions with others),<br />

Working as a Member of a<br />

Team, (knowing how to<br />

contribute to a team),<br />

Citizenship (of one’s<br />

profession, community,,<br />

nation state, other political<br />

entity), Environmental Ethics<br />

(having ethical, principles in<br />

relation to nonhuman world<br />

Acquire, Advise, Advocate,<br />

Balance, Be aware of, Behave,<br />

Collaborate, Communicate,<br />

Comply, Cooperate, Critically<br />

reflect, Decide to,<br />

Demonstrate, Describe,<br />

Educate, Embody, Empathize,<br />

Express, Feel confident, Give<br />

feedback, Help, Influence,<br />

Initiate, Inspire, Interact with,<br />

Involve, Lead, Mediate,<br />

Mobilize, Motivate,<br />

Negotiate, Nurture, Offer,<br />

Promote, Protect, Reconcile,<br />

Reform, Resolve, conflict,<br />

Respect, Respond, sensitively,<br />

See oneself as, Serve as role,<br />

model, Settle, Share, Show,<br />

Suggest, Support, Suspend,<br />

judgment, Sustain, Take<br />

responsibility, Unite<br />

Ethics, morality, principles,<br />

attitudes, values, beliefs,<br />

premises, conflicts, personal,<br />

social, cultural, and,<br />

environmental, implications<br />

CARING – What changes in learners’ feelings, interests, values are important<br />

Wanting to Be a Good<br />

Learner, (wanting to master,<br />

achieve high standards),<br />

Becoming Excited About a<br />

Particular, Activity/Subject<br />

(developing a keen interest),<br />

Developing a Commitment<br />

to Live, Right (i.e., deciding<br />

to take care of one’s,<br />

health/well‐being, live by a<br />

certain code)<br />

Agree to, Be ready to,<br />

Commit to, Decide to,<br />

Demonstrate, Develop,<br />

Discover, Explore, Express,<br />

Get excited about, Identify,<br />

Pledge, Recognize, value of,<br />

Renew interest, Revitalize,<br />

Share, State, Take time to,<br />

Value<br />

Attitudes, beliefs, feelings,<br />

interests, opinions, values<br />

24


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Dimension Action Verbs Objects<br />

LEARNING HOW TO LEARN – What should learners learn about learning, engaging in inquiry,<br />

and becoming self‐directed<br />

How to Be a Better Learner<br />

(engaging in, self‐regulated<br />

learning or deep learning),<br />

How to Inquire and<br />

Construct, Knowledge (how<br />

to engage in the scientific,<br />

method, historical method,<br />

other forms of inquiry), How<br />

to Pursue Self‐Directed or,<br />

Intentional Learning<br />

(developing a, learning<br />

agenda and plan, becoming<br />

an intentional, learner,<br />

becoming skilled in<br />

autodidaxy, being a, reflective<br />

practitioner)<br />

Construct knowledge about,<br />

Describe how to, Develop a<br />

learning plan, Frame useful<br />

questions, Generalize<br />

knowledge, Identify sources<br />

and resources, Identify your<br />

learning style & barriers,<br />

Identify what you need to<br />

know, Inquire, Predict<br />

performance, Reflect,<br />

Research, Self‐assess, Selfregulate,<br />

Self‐monitor, Set a<br />

learning agenda, Take<br />

responsibility for, Transfer<br />

knowledge<br />

Learning, acquisition of<br />

knowledge and skills, selfimprovement,<br />

self‐direction,<br />

accountability<br />

25


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Table 2B: Levels of Thinking/Learning<br />

Category Dimension Definition<br />

Foundational<br />

Knowledge<br />

Remembering &<br />

Understanding<br />

Knowing common terms, specific facts,<br />

methods and procedures, basic concepts,<br />

principles; understanding to a degree that<br />

allows for explanations, predictions<br />

Application Critical Thinking Analyzing and critiquing issues and situations<br />

Practical Thinking Solving problems and making decisions<br />

Creativity<br />

Creating/refining/ inventing new ideas,<br />

products, and perspectives<br />

Integration<br />

Human<br />

Dimension<br />

Caring<br />

Learning<br />

How to Learn<br />

Managing Complex<br />

Projects<br />

Performance Skills<br />

Interdisciplinary<br />

Learning<br />

Interpersonal<br />

Relationships<br />

Self‐Authorship<br />

Leadership<br />

Ethics, Character<br />

Building<br />

Multicultural<br />

Education<br />

Working as a Member<br />

of a Team<br />

Citizenship<br />

Environmental Ethics<br />

Wanting to be a good<br />

learner<br />

Becoming excited<br />

about a particular<br />

activity or subject<br />

Developing a<br />

commitment to live<br />

right<br />

How to be a better<br />

learner<br />

How to inquire and<br />

construct knowledge<br />

How to pursue selfdirected<br />

or intentional<br />

learning<br />

Coordinating and sequencing multiple tasks in<br />

a single project/case and/or multiple projects/<br />

cases<br />

Communicating and performing psycho‐motor<br />

activities<br />

Connecting different ideas,<br />

disciplines,perspectives, contexts<br />

Establishing effective working relationships<br />

with supervisors, peers, patients, and others<br />

Creating and taking responsibility for one’s<br />

own life<br />

Being an effective leader<br />

Developing character and living by ethical<br />

principles<br />

Becoming culturally sensitive in one’s<br />

interactions with others<br />

Knowing how to contribute to a team<br />

Being a responsible citizen of one’s<br />

profession, local community, nation state, and<br />

other political entity<br />

Having ethical principles in relation to the<br />

nonhuman world<br />

Wanting to master material, achieve high<br />

standards<br />

Developing a specific area interest<br />

For example, deciding to take care of one’s<br />

health and well‐being, to live by a certain code<br />

Engaging in self‐regulated learning or deep<br />

learning<br />

How to engage in the scientific method,<br />

historical method, and/or other forms of<br />

inquiry<br />

Developing a learning agenda and plan,<br />

becoming an intentional learner, becoming<br />

skilled in autodidaxy, being a reflective<br />

practitioner<br />

26


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Table 3B: Teaching/Learning Strategies Best Suited for Each Dimension of Learning<br />

Desired dimension<br />

Foundational Knowledge<br />

(understanding,<br />

remembering)<br />

Application (critical &<br />

practical thinking, creativity,<br />

managing projects,<br />

performance skills)<br />

Integration (connecting<br />

ideas,<br />

disciplines, people, realms)<br />

Human Dimension<br />

(leadership, ethics,<br />

teamwork;<br />

social, cultural, political,<br />

environmental implications)<br />

Caring (wanting to succeed,<br />

developing a keen interest,<br />

making a commitment)<br />

Learning to Learn (becoming<br />

a better learner, inquiring &<br />

constructing knowledge,<br />

being<br />

self‐directed)<br />

Suggested Teaching/Learning Strategies<br />

Presentation, lecture, question‐and‐answer, large and small<br />

group discussion, development of learning issues,<br />

independent study, review session, teaching others, game,<br />

web‐based instruction<br />

Hands‐on procedure, simulation, case study, role‐play, action<br />

plan, teaching others, question‐and‐answer, brainstorming,<br />

problem‐solving, trouble‐shooting, journal club, developing<br />

research questions, theory and model building, project,<br />

critical review, direct<br />

enterprise contact, precepting, guided practice with feedback<br />

What if…, compare and contrast, concept mapping, crossdisciplinary<br />

teams, cross‐disciplinary cases, multiple examples<br />

within & across contexts, theory & model building, integrated<br />

curriculum<br />

Case study, simulated situations, case presentations, working<br />

in diverse teams, authentic project,<br />

group project, direct enterprise contact, assigned leadership<br />

role, debate, journal club (e.g., using ethics articles)<br />

Authentic project, role modeling, self‐selection activity,<br />

debate, reflective writing, positive reinforcement, learning<br />

prescription<br />

Self‐assessment, self‐ and peer‐feedback, teaching others,<br />

reflective writing, formative assessment, self‐awareness<br />

exercise/inventory<br />

27


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

7. RUBRICS 8<br />

Once identified learning goals and learning objectives with their<br />

specific means of assessment, the next step is identifying a scoring<br />

procedure.<br />

A rubric is a one‐or two page document (usually a matrix) that<br />

describes how a program/specific assignment will be assessed and<br />

the expected student learning outcomes according to varying levels<br />

of quality, from excellent to poor. In this way it serves as an<br />

instructional tool as well as an evaluative one, since it gives students<br />

clear indications about how to improve.<br />

We can identify two main components of a rubric:<br />

1. The list of criteria, or "what counts" in a program or<br />

assignment;<br />

2. The gradations of quality, with descriptions of strong,<br />

middling, and problematic students’ work. The gradations<br />

of quality help students to spot weaknesses and give them<br />

concrete ways to improve their shortcomings.<br />

Using rubrics a tool of evaluation has several advantages:<br />

<br />

<br />

<br />

They are easy to use and to explain then simple grades;<br />

They make professors' expectations very clear;<br />

They provide students with more informative feedback<br />

about their strengths and areas in need of improvement<br />

and in turn they support students’ learning and<br />

development;<br />

8<br />

Adapted from http://www.stanford.edu/dept/presprovost/irds/assessment/downloads/rubrics.pdf<br />

28


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

How to design rubrics<br />

Framing a rubric effectively involve a four step process:<br />

1. Identify learning objectives (see paragraph 5);<br />

2. Identify for each objective its dimensions/traits;<br />

Example (source AACSB)<br />

Oral Communication Evaluation: checklist<br />

1. Preparation/content<br />

a. Opening statement with purpose<br />

b. Organization<br />

c. Content: currency & relevance<br />

d. Quality of slides<br />

e. Quality of conclusion<br />

2. Presentation<br />

a. Voice quality, pace<br />

b. Mannerism<br />

c. Professionalism<br />

d. Use of media<br />

e. Ability to answer questions<br />

3. Select the relevant dimensions; a rubric should fit into oneor<br />

two pages, or no more than 4 to 8 dimensions. In order to<br />

do this we could:<br />

a. List the identified dimensions;<br />

b. Rank them from the most important to least;<br />

c. Eliminate those who are at the bottom in order to<br />

reach the target number;<br />

4. Articulate the expected outcomes in gradation of quality.<br />

(They answer to the questions “yes, yes but, no but, no”, or<br />

“low performance, average, exemplary performance”)<br />

29


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Example: Oral Communication (source AACSB)<br />

Objectives<br />

Organization<br />

Content<br />

Quality of<br />

slides<br />

Low Performance<br />

0‐1 points<br />

No opening<br />

statement or<br />

irrelevant<br />

statement. Loses<br />

focus more than<br />

twice. Does not<br />

manage time<br />

effectively.<br />

Does not meet<br />

assignment<br />

requirements, is not<br />

current, not<br />

objective. Does not<br />

use appropriate<br />

analysis<br />

Sloppy and/or<br />

unprofessional. May<br />

be difficult to read.<br />

Many slides are<br />

superfluous.<br />

Average<br />

2‐3 points<br />

Has opening<br />

statement relevant<br />

to topic and gives<br />

outline of speech.<br />

Mostly organized,<br />

loses focus only<br />

once or twice. Some<br />

transitions. Allows<br />

enough time to<br />

deliver speech,<br />

although it could<br />

have been better<br />

edited.<br />

Fulfill assignment.<br />

Current. Uses<br />

appropriate sources<br />

and is objective.<br />

Reasonable<br />

analysis.<br />

Readable,<br />

professional,<br />

appropriate<br />

number.<br />

Exemplary<br />

Performance<br />

4‐5 points<br />

Has a clear opening<br />

statement that<br />

catches audience’s<br />

interest and gives<br />

overview. Stays<br />

focused<br />

throughout. Has<br />

transition<br />

statements, ties<br />

sections together.<br />

Very effective time<br />

management.<br />

Is more thorough<br />

than average.<br />

Research goes<br />

beyond minimum<br />

requirements.<br />

Strong analysis<br />

Readable,<br />

professional,<br />

imaginative and/or<br />

of high quality<br />

(without being a<br />

distraction)<br />

Earned<br />

Points<br />

Scoring:<br />

10‐19: Does not meet expectations<br />

20‐26: Meets expectations<br />

27‐30: Exceeds expectations<br />

Because rating performance involves human judgment, it is subject<br />

to error, due to personal biases, preceding tasks, and (lack of)<br />

standardization of scoring criteria.<br />

To avoid these common errors,<br />

1. Rate the performance of all students on one task before<br />

going to the next to keep scoring criteria in mind;<br />

2. Whenever possible rate performance without knowledge of<br />

student’s name to avoid halo effects;<br />

30


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

3. Use multiple raters for high‐stakes decisions; and<br />

4. Train your raters to calibrate their scoring by bringing them<br />

together to review their responses in order to identify<br />

patterns of inconsistent responses.<br />

31


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

8. DEFINITIONS<br />

Assessment tools: The instruments used to gather data<br />

about student learning. Tools can be both quantitative and<br />

qualitative. Examples of these tools are: pretests, group<br />

problem solving, performances and demonstrations,<br />

portfolios, peer observations.<br />

Assessment of Student Learning: An ongoing process<br />

aimed at understanding and improving student learning. It<br />

involves making our expectations explicit and public;<br />

setting appropriate criteria and standards for learning<br />

quality; systematically gathering, analyzing, and<br />

interpreting evidence to determine how well performance<br />

matches those expectations and standards; and using the<br />

resulting information to document, explain, and improve<br />

performance. (Angelo, 1995)<br />

Bloom's Taxonomy: The original model developed by<br />

Benjamin Bloom that established six different levels of<br />

learning:<br />

o Evaluation<br />

o Synthesis<br />

o Analysis<br />

o Application<br />

o Comprehension<br />

o Knowledge<br />

Classroom Assessment: A simple method faculty can use<br />

to collect feedback, early and often, on how well their<br />

students are learning what they are being taught. The<br />

purpose of classroom assessment is to provide faculty and<br />

students with information and insights needed to improve<br />

teaching effectiveness and learning quality. (Angelo, 1991)<br />

Continuous Improvement : Continuous improvement<br />

processes are ongoing, data‐supported, research‐driven<br />

processes carried out by a team for the purpose of<br />

improving the effectiveness of work systems and their<br />

results.<br />

Criteria: Statements applied to desired skills, attitudes, or<br />

knowledge that a student should learn to demonstrate<br />

competence. Criteria should be given to the student before<br />

an assessment or evaluation is performed.<br />

Curricula Alignment: The process of alignment the work<br />

students are doing in one or more classes directly to the<br />

achievement of the learning goals.<br />

32


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Direct Measures of Learning: Evidence about student<br />

learning based on student performance that demonstrates<br />

the learning itself. Can be value added, related to<br />

standards, qualitative or quantitative, embedded or not,<br />

using local or external criteria.<br />

Evaluation: The process of measuring the quality of a work<br />

performance, work product, or use of a process against a<br />

set of standards and criteria to make a judgment or<br />

determination if, or to what level, the standards have been<br />

met and students are achieving the instructional objectives.<br />

Fink's taxonomy: Revises Bloom's taxonomy in order to<br />

describe several ways in which learning can be significant.<br />

Fink defines learning in terms of change: "For learning to<br />

occur, there has to be some kind of change in the learner.<br />

No change, no learning. And significant learning requires<br />

that there be some kind of lasting change that is important<br />

in terms of the learner's life".<br />

Formative assessment: The gathering of information<br />

about student learning‐during the progression of a course<br />

or program and usually repeatedly‐to improve the learning<br />

of those students (Leskes, 2002)<br />

Indirect Measures of Learning: Evidence about how<br />

students feel about learning and their learning environment<br />

rather than actual demonstrations of outcome<br />

achievement. Examples include surveys, questionnaires,<br />

interviews, focus groups, and reflective essays.<br />

Learning dimensions: The description of the values,<br />

attitudes, knowledge, skills and behavior a School is<br />

seeking to instill in its students.<br />

Learning goal: Broad statement of desired results,<br />

achievement of which is usually accomplished after<br />

graduation.<br />

Learning Objective: Specific, measurable step to achieve a<br />

goal or outcome<br />

Learning Outcomes: Operational statements describing<br />

specific student behaviors that evidence the acquisition of<br />

desired knowledge, skills, abilities, capacities, attitudes or<br />

dispositions. Learning outcomes can be usefully thought of<br />

as behavioral criteria for determining whether students are<br />

achieving the educational objectives of a program, and,<br />

ultimately, whether overall program goals are being<br />

successfully met. (Allen, Noel, Rienzi & McMillin, 2002)<br />

Program Mission: A holistic vision of the values and<br />

philosophy of the program. The program mission needs to<br />

be consistent with campus and college missions. The overall<br />

question that the mission statement should answer is: What<br />

is the overall unique purpose of this program<br />

Rubric: A rubric is a one‐or two page document (usually a<br />

matrix) that describes how a program/specific assignment<br />

will be assessed and the expected student learning<br />

33


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

<br />

<br />

outcomes according to varying levels of quality, from<br />

excellent to poor.<br />

SMART: Acronym for characteristics of good goals,<br />

outcomes, and/or objectives: Specific, Measurable, Actionoriented,<br />

Realistic, Time‐Based<br />

Summative Assessment: is the process of gathering of<br />

data on student learning, based on specific assessment<br />

criteria, at the conclusion of a course, unit, field experience,<br />

or program. This data is the basis for judging the quality of<br />

student knowledge and skills and can be used for evaluative<br />

purposes (grades) and provide feedback for future<br />

improvement.<br />

34


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

9. TEMPLATES<br />

COURSE SYLLABUS TEMPLATE<br />

Master Program XXX – Term/Year<br />

Course Title:<br />

Professor’s<br />

picture<br />

Professor:<br />

E‐mail:<br />

Office:<br />

Phone:<br />

Assistant:<br />

E‐mail:<br />

Office:<br />

Phone:<br />

Learning Objectives<br />

Course Content<br />

Course Methodology<br />

Course Etiquette<br />

Required Readings<br />

Course Evaluation<br />

About The Instructor – Mr/Mrs<br />

How to get in touch:<br />

Syllabus<br />

Session,<br />

Date, time, duration<br />

Session,<br />

Date, time, duration<br />

Title<br />

Title<br />

35


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Program Directors Annual Report Scheme<br />

Master Program<br />

Annual Report, Year<br />

Master Director<br />

BACKGROUND<br />

Demand Analysis<br />

Competitor Analysis<br />

STRATEGY<br />

PERFORMANCE<br />

Market target<br />

Program Strategic Positioning<br />

Development aims and goals objectives<br />

Value Proposition Analysis (present and future)<br />

How to improve financial performance<br />

AOL Cycle: Closing the Loop (results and future actions)<br />

36


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Relevance of each course for the achievement of the program learning goals<br />

Relevance of each course for the achievement of the program learning<br />

goals<br />

Program: _______________________________________________<br />

COURSE<br />

Degree of relevance (*)<br />

LG 1 LG 2 LG 3 LG 4 LG 5 LG 6 LG 7<br />

(*) Please insert the degree of relevance as follows: LOW, MEDIUM, HIGH<br />

37


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Closing the loop<br />

LGs<br />

MANAGEMENT<br />

KNOWLEDGE<br />

Graduates will be competent<br />

in the core functional areas<br />

and understand how these<br />

areas are integrated (Master<br />

the knowledge required to<br />

manage organizations)<br />

Objectives<br />

Acquire proficiency in all<br />

functional areas of<br />

management<br />

Acquire the ability to<br />

integrate the functional<br />

knowledge adopting a<br />

general management<br />

perspective<br />

Measure<br />

ments<br />

When<br />

Results<br />

and<br />

Actions<br />

38


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Checkpoint Assessment Format<br />

ASSESSMENT OF LEARNING OBJECTIVES<br />

MASTER PROGRAM Edition #<br />

CHECKPOINT:<br />

PLEASE INSERT THE COURSE TITLE<br />

PROGRAM LEARNING OBJECTIVE(S) ASSESSED:<br />

PLEASE INSERT THE PROGRAM LEARNING OBJECTIVE(S)<br />

ASSESSED<br />

DATE:<br />

PLEASE INSERT TIME OF TEST<br />

ASSESSORS:<br />

PLEASE NSERT FACULTY NAME(S)<br />

CALL FOR ACTION:<br />

YES/NO<br />

39


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

PLEASE INSERT MASETR PROGRAM, #<br />

EDITION AND COURSE TITLE<br />

Assessment of learning Objectives ‐ # X/Y<br />

Learning Objective Assessed #1<br />

INSERT LEARNING GOAL ASSESSED<br />

INSERT THE LEARNING OBJECTIVE ASSESSED<br />

Assessment Methodology<br />

PLEASE DESCRIBE THE ADOPTED ASSESSMENT METHODOLOGY AND CRITERIA<br />

Assessment Results<br />

EXEMPLARY (3)<br />

GOOD (2)<br />

FAIR (1)<br />

PLEASE INSERT RUBRICS CORRESPONDING<br />

TO THE OBJECTIVE YOU CHOSE<br />

Exemplar<br />

%<br />

Good<br />

%<br />

Fair<br />

%<br />

PLEASE GIVE BREAKDOWN BASED ON THE ABOVE SCHEME AND PROVIDE THE SUPPORTING EVIDENCE FOR THE RESULTS<br />

(EVALUATION OF EACH STUDENT)<br />

40


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

PLEASE INSERT MASETR PROGRAM, # EDITION AND<br />

COURSE TITLE<br />

Learning Objectives: Comments & Actions<br />

Option A:<br />

Since the results have been in line with expectations, no change is required for next year.<br />

(Nonetheless, in order to ………… there are plans to change….)<br />

Option B:<br />

Since the results were not fully satisfactory, in the next edition of the Program the following changes, with<br />

related goals, will be undertaken:<br />

i) …<br />

ii) ...<br />

Option C:<br />

The results have been in line with expectations, but the following changes will anyway be undertaken in the next edition of the<br />

Program<br />

41


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Example of assessment of Learning Objectives<br />

FT MBA35<br />

ASSESSMENT OF LEARNING OBJECTIVES<br />

MBA 35 FT<br />

CHECKPOINT:<br />

Organizational Behavior and Negotiation<br />

PROGRAM LEARNING OBJECTIVE(S) ASSESSED:<br />

Team work and leadership skills – Obj. 1,2<br />

DATE:<br />

October‐November 2009<br />

ASSESSORS:<br />

Leonardo Caporarello<br />

CALL FOR ACTION:<br />

YES<br />

42


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

MBA 35 FT– Organizational Behavior and Negotiation<br />

Assessment of learning Objectives – 1/2<br />

Learning Objective Assessed #1<br />

Skills<br />

Graduates will work<br />

effectively in teams, using<br />

interpersonal,<br />

communication and<br />

Display the ability to work with others<br />

Assessment Methodology<br />

The ability of the candidates to work with others was assessed through specific self assessment tools for behavior and attitude orientation. During the class,<br />

candidates were given a series of team tasks to perform with the clear goal of observing and trying to understand group dynamics and team development.<br />

After each simulation theoretical framework for analysis was provided and discussed in plenary sessions. At the end of the Graduates should display the<br />

ability to work with others dynamics and team development. After each simulation theoretical framework for analysis was provided and discussed in plenary<br />

sessions. At the end of the course, part of the individual exam consisted of an individual analysis of a teamwork situation.<br />

Assessment Results<br />

EXEMPLARY (3)<br />

GOOD (2)<br />

FAIR (1)<br />

Graduates should display the<br />

ability to work with others<br />

Graduates participate wholeheartedly in group<br />

activity, gaining other people’s trust and<br />

benevolence. They know how to leverage on<br />

other people background and culture to<br />

establish fertile relationships in terms of<br />

opening new perspective, finding new solutions,<br />

raising new stimulating goals. However, they<br />

also know when to critically rely on third party<br />

contributions to overcome their own<br />

Blue Class<br />

Graduates participate in group activity,<br />

responsibly willingly, timely and fruitfully,<br />

bringing the required contribution.<br />

They know how to relate to people of different<br />

backgrounds & cultures, according the principle<br />

of mutual respect & support and fully recognize<br />

other group members’ contribution.<br />

Yellow Class<br />

MBA 35 Cohort<br />

Graduates search to minimize their<br />

contribution to group activities, incurring<br />

delays, avoiding commitment and taking<br />

a passive stance.<br />

They do not always recognize other<br />

people’s contribution.<br />

Exemplary<br />

Good<br />

9%<br />

46%<br />

Exemplary<br />

Good<br />

12%<br />

43&<br />

Exemplary<br />

Good<br />

10%<br />

45%<br />

Fair 46%<br />

Fair 45% Fair<br />

45%<br />

43


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

MBA 35 FT – Organizational Behavior and negotiation<br />

Learning Objectives: Comments & Actions<br />

Since the results were not completely in line with the targets expected, in the next edition of the program there will be given more<br />

specific task and theoretical frameworks on team structures and management reducing the impact of themes like power and politics on<br />

the module.<br />

44


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

MBA 35 FT– Organizational Behavior and Negotiation<br />

Assessment of learning Objectives – 2/2<br />

Learning Objective Assessed #2<br />

Teamwork & Leadership<br />

Skills<br />

Graduates will work<br />

effectively in teams, using<br />

interpersonal,<br />

communication and<br />

leadership skills.<br />

Be aware of the role and responsibilities of leadership and behave accordingly<br />

Assessment Methodology<br />

The ability of the candidates to identify roles and responsibilities and display the ability in conflict resolution was assessed through specific self assessment<br />

tools for behavior and in‐class simulations. After each simulation theoretical framework for analysis was provided and discussed in plenary sessions pointing<br />

out the relationship between output from the simulation and behavior adopted. At the end of the course, part of the individual exam consisted of a situation<br />

based multiple choice test and individual analysis of a teamwork situation.<br />

Assessment Results<br />

EXEMPLARY (3)<br />

GOOD (2)<br />

FAIR (1)<br />

Graduates should be aware<br />

of the role and<br />

responsibilities of leadership<br />

and behave accordingly<br />

Graduate possesses the ability to motivate,<br />

persuade, negotiate and resolve conflicts both<br />

in handling simple managerial tasks and in<br />

leading/participating in a wider change<br />

management process.<br />

Graduates possess the ability to motivate,<br />

persuade, negotiate, resolve conflicts in handling<br />

simple managerial tasks and in small groups of<br />

people, whilst lacking the standing & perspective<br />

to do the same when participating to a wider<br />

change management process and/or in larger<br />

groups.<br />

Graduates understand roles and<br />

responsibilities of leadership but have a<br />

tendency to eschew them, maybe<br />

sensing that they are personally not fitted<br />

for the role.<br />

Blue Class<br />

Exemplary<br />

Good<br />

Fair<br />

30%<br />

56%<br />

14%<br />

Yellow Class<br />

Exemplary<br />

Good<br />

Fair<br />

36%<br />

36%<br />

28%<br />

MBA 35 Cohort<br />

Exemplary<br />

Good<br />

Fair<br />

33%<br />

46%<br />

21%<br />

45


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

MBA 35 FT – Organizational Behavior and negotiation<br />

Learning Objectives: Comments & Actions<br />

The results were in line with the targets. However in the next edition of the program more time will be given to class discussion and<br />

simulation. Relevance will be given to class and teamwork participation, reducing the space reserved to theory and modeling.<br />

46


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Peer Overall Evaluations FT MBA 35<br />

MBA 35 FT<br />

LEARNING GOAL<br />

Teamwork & Leadership Skills<br />

Graduates will work effectively in teams, using interpersonal, communication and<br />

leadership skills.<br />

OBJECTIVES<br />

Display the ability to work with others<br />

Be aware of the role and responsibilities of leadership and behave accordingly<br />

Be able to communicate ideas effectively<br />

47


<strong>SDA</strong> <strong>Bocconi</strong> School of Management<br />

Assessment Handbook<br />

Assessment methodology<br />

Peer Evaluation – Study Group 1 st Term Managerial Excellence<br />

RATER: _______________________________________________________________<br />

CLASS: _________________________________________<br />

Please provide in the following table an objective evaluation of your colleagues based on observations you have made in the various group meetings<br />

and during any interpersonal interactions you may have had. The rating scale must be as follows:<br />

1=Rarely 2=Sometimes 3=Regularly 4=Very often 5=Always<br />

1.<br />

2.<br />

3.<br />

4.<br />

5.<br />

LAST NAME OF THE<br />

RATED GROUP MEMBER<br />

GROUP<br />

PARTICIPATION<br />

PERSONAL<br />

COMMITMENT<br />

GOAL<br />

ORIENTATION<br />

QUALITY<br />

ORIENTATION<br />

COMMUNICATION<br />

SKILLS<br />

LEADERSHIP<br />

GROUP S/he attends the group meetings, participates in the discussions, provides meaningful contributions to the discussions, and<br />

PARTICIPATION happily shares personal experiences, knowledge and information. S/he provides support, as needed, to peers and colleagues.<br />

PERSONAL S/he adds value to the group work through an active participation relevant to achieve the desired outcome. His/her contribution<br />

COMMITMENT is matched by a sense of involvement in the team process. S/he adheres to all of the group commitments and agreements.<br />

GOAL S/he is oriented to the desired outcome and strives to provide input that is relevant to the defined goal.<br />

ORIENTATION S/he contributes in a meaningful way to create the final document (or output) that is to be presented.<br />

QUALITY S/he is focused on the quality of the outcome and pays attention to all details to make the final result as perfect as possible.<br />

ORIENTATION Any necessary calculations is performed properly and not overdone. The focus on quality does not slow down the group.<br />

COMMUNICATION S/he communicates clearly, both verbally and in writing. S/he listens effectively and is tolerant of different opinions.<br />

SKILLS<br />

LEADERSHIP S/he is capable of guiding the group toward the desired results through a balanced people and goal-oriented approach.<br />

S/he is able to build team spirit, create a sense of cooperation, encourage input from all team members, and achieve consensus<br />

when necessary. S/he is able to persuade and influence colleagues in order reach consensus.<br />

48

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!