Birmingham MBChB Assessment Strategy - University of Birmingham

hatchingdealsLJ-0100109003*See Ad InsideEaton RapidsCommunity NewsYour very own newspaper for 19 years© 2011 Lansing Community NewspapersVol. 19 - No. 19 www.EatonRapidsCommunityNews.com September 18, 2011TMNew Eaton Rapids dog park in the worksMary Jo Whitemjwhite@lsj.comEATON RAPIDS — If agroup of dedicated animallovers have their way, EatonRapids may next year behome to a new park wheredogs can run unleashed.Group president BudMaxey, who owns a rescuedRottweiler named Baronwith his wife, Mary, says hesees more and more dogsbeing walked on city sidewalks.“This (the park) givesus a safe place,” he said.“Plus, dogs need the freedom.”The committee to formthe park has been hard atwork for several monthsand most recently met atcity hall Sept. 13 to electofficers and hear committeereports.They plan to update theParks and Recreation committeeon their proposalnext week and hope to havethe city council approve thepark plan by next January.Mayor Steven Platte,who offered technical computerassistance to thegroup, said at the meetinghe has not heard any oppositionto the park.A two-and-a-half-acresite on Eaton Pines Blvd.across from Rose Hill Cemeteryand just west of a cityownedsoccer field has beenproposed for the park.A trail around the fiveacres that would includeboth the dog park and thesoccer field is envisioned,too, where dogs could bewalked on leashes“We couldn’t have askedfor a better spot,” said localanimal behaviorist LaurieGriffith, the group’s newvice president, adding thatSee PARK, Page 4Ten members of a local group working to establish an Eaton Rapids dog park met Sept. 13at city hall. The group has formed committees and elected officers and intends to updatethe city’s Parks and Recreation committee on their proposal next week. A possible site forthe dog park has been identified on Eaton Pines Blvd. next to a city-owned soccer field. Lto r: Bud Maxey, Karen O’Briant, Jamie Snow, Laurie Griffith. MARY JO WHITE/LCNER Blue Star Memorial was dedicated Sept. 11There were well overone hundred peoplein attendance at thededication of the Blue StarMemorial Marker heldSept. 11 in Eaton Rapids.Many speakers, includingmembers from theEaton Rapids Home andGarden Club, the MichiganGarden Club, Inc., cityofficials, StateRepresentatives MikeShirkey and Deb Shaughnessyand State SenatorRick Jones’wife, CharleneJones, spoke.A quote from one of theMichigan Garden Club officialsstood out from a commentaryby Lester Holt ofNBC News:“We can’t single handedlyfix everything butwe can make small differencesthat can becomesomething larger.”The Blue Star chairpersonfor the central region,Lorrie Dyal, spoke aboutthe value of our veteranswith these words from anunknown author:“It is the veteran, notthe preacher, who hasgiven us freedom of religion.It is the veteran, notthe reporter, who has givenus freedom of the press.It is the veteran, notthe poet, who has given usfreedom of speech.It is the veteran, notthe campus organizer, whohas given us the freedomto assemble.It is the veteran, notthe lawyer, who has givenus the right to a fair trial.It is the veteran,not the politician, who hasgiven us the right to vote.It is the veteran, whosalutes the flag, who servesunder the flag, whose coffinis draped by the flag.”Brenda Wineman, copresidentof the EatonRapids Home and GardenClub, shared a poem called“The Dandelion” by Chelsea,a military brat, on anofficial flower.“She will blow and blowBut will never knowWhere the little dandelionWill GoWhere the little dandelionWill Grow.The wind takes it highThe wind takes it lowBut still she will neverMembers of the Eaton Rapids Home and Garden Club celebrate the dedication of theirBlue Star Memorial Marker Sept. 11. Left to right is Diane Hudson, Garry Montague,Brenda Wineman, Anne Sanders, Claudia Brown, Chris Belonge, Linda Montague, BecaBrooks, Mary Raymond, Karen Lewis, Billy Hood, Lori Boardman, and Glenna Hood. Alsoattending but not shown: Reggie Brooks, Patty Theile (who played taps), Jose Wixson(director of Sauk Trail Sound Sweet Adelines and photographer Allison Wineman). ALLISONWINEMANto come.knowWhere the little dandelionWill goWhere the little dandelionWill grow.”Garden club memberswill remember this auspiciousevent for many years— From garden clubco-presidents BrendaWineman and LindaMontague.How to reach us: Advertising: 517-377-1141 Circulation: 866-226-1812 News: 517-541-2531 Classifieds: 877-475-SELL or 877-391-SELL

Birmingham MBChB Assessment Strategy 2012/13ContentsIntroduction ............................................................................................................................................ 3Outcome of the course ....................................................................................................................... 4Links to relevant documentation ........................................................................................................ 5Assessment Sub-strategies ..................................................................................................................... 6Sub-strategy 1: All of the “Outcomes for Graduates” identified in TD are assessed using anappropriate range of assessment techniques .................................................................................... 6The MBChB blueprint ...................................................................................................................... 6Quality assurance of the MBChB blueprint..................................................................................... 6Modes of assessment ..................................................................................................................... 6Sub-strategy 2: The timing of assessments within the MBChB is appropriate ................................... 8Sub-strategy 3: Assessments are fit for purpose ................................................................................ 8Examiner training and feedback ..................................................................................................... 9Quality assurance of fitness for purpose (excluding content validity) ........................................... 9Sub-strategy 4: Appropriate standards are maintained ..................................................................... 9Quality assurance of passing standards ........................................................................................ 11Sub-strategy 5: Students receive regular, timely and useful feedback on their performance ......... 11Exam development process .................................................................................................................. 13Quality assurance of content validity ........................................................................................... 14Policies and procedures ........................................................................................................................ 15General information on assessments for all students .......................................................................... 21References ............................................................................................................................................ 22Page | 2

Birmingham MBChB Assessment Strategy 2012/13IntroductionThis strategy defines the over-arching approach to assessment on the MBChB. The aims of thisassessment strategy are therefore to:1. Embed assessment in the management of the MBChB,2. Ensure compliance with the QAA precepts, the GMC’s Tomorrow’s Doctors 2009 (TD) andUniversity of Birmingham (U of B) regulations (see below) relating to assessment and the use ofbest practice,3. Describe the mechanisms by which assessment is coordinated across the MBChB,4. Provide a resource for staff, students and other stakeholders with all assessment-related policiesin one place, and5. Encourage and facilitate development and improvement of assessment of the MBChB.In order to achieve these aims, five sub-strategies and 20 specific policies and procedures have beendeveloped. Each sub-strategy, policy and procedure is linked to the corresponding QAA precept(s),paragraph(s) in TD and U of B regulations and codes of practice. The strategy also includesinformation on the exam development process and a Phase-specific summary of assessmentrequirements for students.This assessment strategy has been prepared by Dr Celia Taylor, Senior Lecturer in Medical Education(Assessment), on behalf of the MBChB Assessments Group. The key aims of this group are to ensurethat assessments on the MBChB are fit for purpose, i.e. that (a) all of the GMC learning outcomes areassessed (assessments are valid), (b) assessments are designed and delivered according to prespecifiedbest-practice processes, and (c) assessments are fair and reliable. The group reports to theMBChB Curriculum Committee (MCC) and, where appropriate, to the individual Phase committees(via the Deputy Phase Leads, who sit on the Assessments Group). Any changes to this strategy willtherefore be approved by MCC and disseminated appropriately. Changes to the nature of anindividual assessment or programme of assessment within a Phase are considered in the section ofthis strategy on ‘policies and procedures’.The term “elements” is used in this document as the collective noun for the parts of the MBChB thatare assessed. Elements include modules (including special study modules (SSM)), student selectedactivities (SSA) and other required components.Every effort has been made to ensure the information in this strategy is correct at the time ofpublication for the current academic year. However, it will not form part of any contract betweenThe University of Birmingham and a student. Students will be informed of changes to assessmentrules by the usual mechanisms. The MBChB programme is currently being reviewed. Therefore theassessment details in this version of the assessment strategy are correct for the 2012/13 academicyear (unless otherwise stated). Revisions to the Phase 4 assessments will come into operation forthe 2013/14 academic year. Regardless of changes to the curriculum, this strategy will be reviewedannually.Page | 3

Birmingham MBChB Assessment Strategy 2012/13Outcome of the courseAt the end of the undergraduate course students passing all of the required components will receivea MBChB degree, which is a primary medical qualification (PMQ). Holding a PMQ is an entitlementto provisional registration with the General Medical Council. Provisionally registered doctors canonly practise in approved Foundation Year 1 posts: the law does not allow provisionally registereddoctors to undertake any other type of work.To obtain a Foundation Year 1 post students will need to apply during the final year of the coursethrough the UK Foundation Programme Office selection scheme, which allocates these posts tograduates on a competitive basis. So far, all suitably qualified UK graduates have found a place onthe Foundation Year 1 programme, but this cannot be guaranteed, for instance if there were to bean excessive number of competitive applications from non-UK graduates.Successful completion of the Foundation Year 1 programme is normally achieved within 12 monthsand is marked by the award of a Certificate of Experience and eligibility to apply for full registrationwith the General Medical Council. Doctors need full registration with a licence to practise forunsupervised medical practice in the NHS or private practice in the UK.Page | 4

Birmingham MBChB Assessment Strategy 2012/13type of assessment. Watching and giving feedback on others’ presentations incorporatesformative peer-assessment into the programme.PortfoliosPortfolios are used across the programme to promote self-assessment, reflection and life-longlearning. The use of portfolios in medical school is important given the need for all junior doctorsto maintain a similar portfolio. In Phase 1 of the programme, students’ personal mentors helpthem to develop the skills required to complete an effective portfolio.Sub-strategy 2: The timing of assessments within the MBChB is appropriateTD: 86, 112QAA: 6U of B Regulation on Assessment, progression and awardThe annual timetable of summative assessments and resits is shown in Appendix 3. Formativeassessment occurs throughout the programme so is not included on the timetable (see Sub-strategy5). Written and/or OSCE assessments are held once or twice per year in the main examinationperiods for each year group. Final written exams are held as close to the point of graduation as islogistically feasible and at a point that enables students to complete a post-exam period ofpreparation for their Foundation Programme posts. Due to practical constraints, final clinical examsare held in two parts (nine stations in November and nine in April). Other assessments are spreadover each year, with the timing of in-course assessments in Phases 3 and 4 dictated by students’rotations.Exam boards are held as soon as possible after the examination periods, so that results can bereleased to students in a timely manner and the ‘sealed list’ can be provided to the GMC on time.Sub-strategy 3: Assessments are fit for purposeTD: 86, 88, 111, 113, 115, 116QAA: 1i, 2, 3TD states that an assessment is fit for purpose if it is valid, reliable, generalisable, feasible and fair.An assessment is valid if it measures what it is intended to measure (scientific knowledge andapplication, clinical practice skills or professional behaviour and attitudes). A process of ensuringcontent validity has recently been implemented for the CC3 and CC4 examinations and is describedunder the section on the exam development process below. This process will be extended to coverexaminations in all phases of the programme. An assessment is reliable and generalisable if astudent would obtain a similar score and the same pass/fail result if a different version of theassessment was used on a different day and marked by a different assessor. An assessment isfeasible if it can be undertaken within the time, location, financial and other constraints imposed bythe University and NHS Trusts. Finally, an assessment is fair if scores reflect students’ knowledgePage | 8

Birmingham MBChB Assessment Strategy 2012/13In the CC4 OSCE, students are required to pass at least 12 of the 18 stations with a calibratedscore of 50% (using station-level cut scores).Rounding of scores to the nearest integer is only undertaken at module level and not at componentlevel. For example, a score of 39.6% on one rotation in the written CC3 exam would lead to thestudent failing based on the bona-fide attempt rule. All scores are truncated to four decimal places.Quality assurance of passing standardsOur External Examiners are crucial in helping us to ensure the standards required to graduate aremaintained. Birmingham is participating in the comparison of exit standards and the nationalprescribing skills assessment pilot projects run by the Medical Schools Council.Sub-strategy 5: Students receive regular, timely and useful feedback ontheir performanceTD 85, 111QAA: 9U of B Code of Practice on Taught programme and module assessmentStudents are formatively assessed and receive informal feedback during the programme in a varietyof ways, including:Participation in small group sessions.Discussions with academic and clinical staff, particularly students’ clinical academy tutors.Completing practice questions provided on WebCT.Contributing to module-based discussion boards on WebCT.Students receive feedback on their performance in all summative assessments. According toUniversity regulations, feedback is provided within four weeks. However an exception has beengranted for some written assignments where a longer period is required to ensure that high qualityfeedback can be provided for our large cohorts. In such cases, the appropriate documentation issubmitted to the University.The basic feedback on each summative assessment consists of the student’s mark, calibrated to apass mark of 50% and a histogram of all students’ scores so students can see where they sit in thescore distribution for that assessment. The additional specific feedback provided depends on thenature of the assessment, for example:Qualitative comments on written assessments (non-examination).Scores on individual OSCE stations, plus generic written comments about students’ strengths andweaknesses on each station.Scores for individual SAQs, the mean cohort mark and generic written comments about howstudents performed on that SAQ (Phase 1).Page | 11

Birmingham MBChB Assessment Strategy 2012/13Students failing an assessment are offered one-to-one feedback with an academic member of staff,where the student’s performance can be discussed in more detail; for example providing marks inthe different sections of the paper, highlighting areas of difficulty and any problems with examtechnique. Students who wish to access their exam scripts need to make a Subject Access Requestunder the Freedom of Information Act 2000 through the Data Protection office. Students are alsoprovided with access to pastoral care through the Student Development and Support Office (SDSO)as appropriate.All students have a Personal Mentor and, while the mentor cannot provide feedback on a student’sperformance in assessments, discussions with the mentor can help students to reflect on theirperformance and feedback received, as well as to recognise when they are receiving informalfeedback. These discussions may include issues such as study skills and time management, inaddition to academic and clinical performance in particular areas, and therefore help students toreact positively to the feedback they have received.At the beginning of each academic year (from Year 2/GEC 3 onwards), students will also receive(based on students starting an academic year together):1. Their weighted average score and cohort ranking for the previous year based on Honoursweightings,2. Their cumulative weighted average score and cohort ranking based on Honours weightings, and3. Their cumulative decile in the cohort based on UKFPO weightings. (Students who are intercalatingwill not receive a decile when they start their intercalation year. This is because UKFPO rulesspecify that students are ranked and allocated to a decile within the cohort of students who starttheir final year together.)Page | 12

Birmingham MBChB Assessment Strategy 2012/13They are then discussedwith the GEC teaching staffand relevant academicsinvolved in teaching GEC1.First review By the module lead. By the module lead. Written exams:By the module/sub-modulelead.OSCE:By the CC2 group.Second review(written exams)Final review andsign-off (writtenexams)Initial post-examchecksBy the module lead.As part of Modified Angoffprocess.across the body systems. Ablueprint is produced.Written exams:By the rotation lead.OSCE:By the CC3/4 committees.As part of Modified Angoffprocess.Followed by review byExternal Examiner.Written exams:Prior to scoring, item analysis of MCQ/EMQ responses is undertaken. Poorly performing items (particularly those with low facility) are highlightedto module/rotation leads to check. Any amendments (e.g. changing the correct answer key) need to be requested by the module/rotation lead viathe Phase lead. Amendments are undertaken by the Data Manager after confirmation from the Phase lead.OSCEs:Marksheets with missing data are identified and sent back to trusts for completion. Where they are purposefully missing (e.g. a student could notpercuss a patient in pain) the scoring system is amended accordingly for that student.Quality assurance of content validityMQAEC will review the match between the item specifications and actual question selection blueprints, using the same review cycle as for post-examevaluations.Page | 14

Policies and proceduresPolicy/procedureStudents are provided withtimely and appropriateguidance regardingassessmentTD 87, 99, 114QAA: 2Assessment processes arerobust(Sub-strategy 3)QAA: 5, 14U of B Code of Practice onthe Conduct of centrally coordinatedwrittenexaminationsMarking processes arerobust(Sub-strategy 4)QAA: 7, 8, 15U of B Code of Practice onTaught programme andmodule assessmentA predetermined process forprogress decisions isfollowedDetailsThe details of the summative assessments/required components thatstudents are required to pass/complete satisfactorily each year areprovided in the year-specific assessment summaries. A summary ofthis information is also given verbally to students via a talk at thestart of each year.Lectures/briefing sessions on different methods of assessment areprovided in the Learning Medicine module in Year 1 and in later yearsas required, with more detailed information and practice questionsavailable on WebCT.This Assessment Strategy will be available to students and staff viathe intranet, which includes links to all the relevant documentation.The question bank, draft and final papers are kept on a restrictedaccess server. Standard University processes regarding the handlingof printed exam papers are maintained to ensure exam security.Written exams are all held on University premises at venues deemedappropriate by the University.Most OSCE exams are held in NHS Trusts and we work closely withstaff in the Trusts to ensure that the venues are appropriate. ASenior Academic Invigilator is present for all CC4 OSCEs and Trustsnominate a Senior Invigilator for each site. In addition, academicmembers of staff are often present at external sites (and are alwayspresent for resits) to ensure the quality and robustness of theexamination sessions.Where required, a separate venue for quarantining students isprovided (e.g. between OSCE cycles where the same scenario is used)and an appropriate protocol followed (e.g. length of quarantine,preventing use of mobile phones).Standard University procedures regarding invigilation and briefingstudents at the beginning of written exams are followed (includingthe presence of an appropriately trained senior invigilator), whileinternally-written procedures are used for clinical exams.All written exams and assignments are marked anonymously.Correction of scores for guessing and negative marking are notundertaken.Pass marks are not adjusted for unreliability (i.e. we do not add astandard error of measurement to the pass mark).All raw scores are calibrated to a published pass mark of 50%.The Administration Team follow a predetermined series of checks onthe data prior to release of marks to students.From 2012/13, marks will be processed through the BirminghamIntegrated Record Management System (BIRMS). An appropriatelevel of security for these processes will be maintained, e.g. in termsof who is allowed to upload and amend student marks.Students are entitled to two attempts at each summativeassessment, with the exception of:a) The ‘Third Sit’ rule in Year 1: students are entitled to a third sitat a maximum of module(s) worth up to 20 credits, which will betaken as an external resit at the end of the following academic

Birmingham MBChB Assessment Strategy 2012/13Policy/procedureReasonable adjustments willbe made as appropriate tothe programmeTD 90, 118U of B Code of Practice onReasonable adjustmentsPlagiarism is not permittedU of B Code of Practice onPlagiarismStudents may request anextension to a deadlineU of B Code of Practice onTaught programme andmodule extensionsDetailspossible.Students who are registered with the University as being dyslexic willbe granted additional time to sit written exams and usually beallocated to a separate exam room. An additional 15 minutes perhour of examination time (i.e. 25%) is allowed. Students withdyslexia must notify the MBChB Office at the start of the academicyear if they wish to take up the offer of additional time.No adjustment to the assessments themselves or the standardrequired to pass assessments can be made, as per GMC regulations.Plagiarism is the act of a student claiming as his or her own,intentionally or by omission, work that is not theirs or has alreadybeen submitted by them for a previous assessment (auto-plagiarism).This includes fabricating evidence, results or data as well as copyingwork done by others.All submitted work will be liable to scrutiny in order to identify anyplagiarised element.A student suspected of plagiarism will be invited by way of a‘Notification Letter’ to a Plagiarism Meeting.Plagiarism will be categorised as either Serious or Moderate or asPoor Academic Practice. The consequences of plagiarism for thestudent will depend on the category assigned in accordance with theCode of Practice.Extensions to deadlines for the submission of written work can onlybe given by the year tutors. Extension decisions are made inaccordance with the University’s Code of Practice (see also the policyon extenuating circumstances).U of B Code of Practice onExtenuating circumstancesand fit to sit procedureSee Appendix 5 for moreinformationStudents who submit worklate will be penalisedU of B Code of Practice onTaught programme andmodule assessmentStudents who are late forOSCEs will be dealt withLate submissions (for either the hard copy or the electronic version ofan assignment) will be penalised by 5% per working day inaccordance with the University’s Code of Practice.Students who fail the assignment due to late submission will berequired to complete the standard resit process for that assignment(including incorporating feedback comments into the revisedsubmission).Both a hard copy and electronic version of resubmissions must beprovided by the resubmission deadline.If a student arrives too late to start their designated cycle in an OSCEwith the other candidates in that cycle then s/he will not be allowedPage | 18

Birmingham MBChB Assessment Strategy 2012/13Policy/procedurefairly across all sitesStudents are entitled toenter a written exam up to30 minutes after the starttime of that exam100% attendance isexpectedU of B Code of Practice onStudent attendance andreasonable diligenceStudents may not requestre-marking of MCQ/EMQsStudents may not requestre-marking of SAQs orwritten assignmentsDetailsto take the exam, but will be asked to leave. Unless the student hasaccepted extenuating circumstances for their late arrival, the studentwill have to forfeit the first sit of the exam and attempt the resit.This rule is in place to ensure that all students in all years at allvenues are treated equally. It is not always possible to add studentsto cycles, so it is fairest to notify students that this will not be doneregardless of the student’s reason for being late.If the student has extenuating circumstances for their late arrivalthese can be submitted in the usual way and the Exam Board willconsider whether or not the resit should be a first sit.No additional time will be allowed for students arriving late to awritten exam.100% attendance is expected in all rotations and modules. The U of BCode of Practice provides details on how any concerns regardingstudent attendance will be addressed.Students who suspect that they may have made a mistake whencompleting the answer sheets for an MCQ/EMQ paper (e.g. missedout a question and so got the subsequent answers out of line, orcompleted the wrong sheet for the question section) must advise therelevant year administrator in the MBChB Office by email within 24hours of the examination. Papers will not be re-marked following therelease of results. All OMR-marked MCQ/EMQ papers which are failsare routinely double checked to ensure that the student has used thecorrect mark sheet for each question section.Marking policies include procedures such as detailed markingguidelines and moderation to ensure that work has been markedappropriately.U of B Code of Practice onTaught programme andmodule assessmentStudents are entitled toappeal against a decisionmade by an Exam BoardU of B Codes of Practice onAppealsAs detailed in the Code of Practice, students may appeal on twogrounds:1. Procedural irregularity2. Extenuating circumstances unknown to the Board of ExaminersThere are no grounds for appeal against academic judgement. Astudent wishing to provide feedback on a particular assessmentshould do so via their year group representative and should notcontact academics responsible for the assessment directly.Students who appeal are not allowed to attend their next year ofstudies until the outcome of the appeal is known.Page | 19

Birmingham MBChB Assessment Strategy 2012/13Policy/procedureThe MBChB will comply withthe Data Protection Act1998A pre-determined policy forchanging assessments willbe followedTD 120DetailsAll assessment-related data are stored so as to comply with the DataProtection Act 1998.Assessments are regularly evaluated against best practice (as notedabove) and the recommendations of External Examiners considered.Changes to the assessments used in the MBChB may therefore beproposed and need to be agreed by the Assessments Group. Suchchanges are approved prior to the start of the academic year in whichthey are applied in accordance with University processes andcommunicated to students at the beginning of their academic year.If, during an academic year, it becomes clear that a plannedassessment is unworkable or inappropriate, an application will bemade to the University’s Programme Approval and ReviewCommittee for an extraordinary change to be made. Such changesmust be to the benefit of students: i.e. an easing of the assessment.Page | 20

Birmingham MBChB Assessment Strategy 2012/13General information on assessments for all studentsPlease refer to the year-specific assessment summaries and module handbooks for furtherinformation on assessments.All credit-bearing modules must be passed with a minimum mark of 50%. In addition, you mustperform satisfactorily in all required components (RC). The overall module total of studentsfailing a module on a bona-fide attempt rule but who have a module total of at least 50% willshow as 49% on IntraMed.Keep a photocopy (and electronic copy) of any assessments or assignments that you hand in. TheUniversity expects students to have back-up copies of work and does not accept IT failure as areason for non-submission.Keep a copy or note of any marks you are awarded. In job applications you may be asked yourmarks from across all years of the course (including years 1 and 2).Dictionaries and calculators MAY NOT be used in exams.Keep your clinical skills passport up-to-date and in a safe place, and back up the data on thewebsite.Unless otherwise stated in the assessment summaries, resit exams take the same format as mainsit exams.Taking an opportunity for assessment means making a bona-fide attempt at an examination orother assessment, in the academic opinion of the Board of Examiners or equivalent.Any material covered in the programme may be re-assessed at a later stage.Page | 21

Birmingham MBChB Assessment Strategy 2012/13ReferencesANGOFF, W. 1971. Scales, norms and equivalent scores. In: THORNDIKE, R. (ed.) EducationalMeasurement. Washington, DC: American Council on Education.CIZEK, G. & BUNCH, M. 2007. Standard Setting: A guide to establishing and evaluating performancestandards on tests, Thousand Oaks, CA, Sage Publications.LIVINGSTON, S. A. & ZIEKY, M. J. 1982. Passing Scores: A Manual for Setting Standards ofPerformance on Educational and Occupational Tests.NATIONAL BOARD OF MEDICAL EXAMINERS 2002. Constructing written test question for the basicand clinical sciences. 3rd Edition (Revised) ed. Philadelphia, PA: NBME.SHUMWAY, J. & HARDEN, R. 2003. AMEE Education Guide No. 25: The assessment of learningoutcomes. Dundee: AMEE.STEWART, M. A. 1995. Effective physician-patient communication and health outcomes: a review.CMAJ: Canadian Medical Association Journal, 152, 1423.TAYLOR, C. 2011. Development of a modified Cohen method of standard setting. Medical Teacher,33, e678-e682.Page | 22