Surgical Expertise in Neurosurgery: Integrating Theory Into ... - CHUQ
Surgical Expertise in Neurosurgery: Integrating Theory Into ... - CHUQ
Surgical Expertise in Neurosurgery: Integrating Theory Into ... - CHUQ
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.TOPICHISTORY AND DEVELOPING SIMULATION IN MEDICINEHISTORY AND DEVELOPING SIMULATION IN MEDICINE<strong>Surgical</strong> <strong>Expertise</strong> <strong>in</strong> <strong>Neurosurgery</strong>: Integrat<strong>in</strong>g<strong>Theory</strong> <strong>Into</strong> PracticeNicholas Gél<strong>in</strong>as-Phaneuf,MDRolando F. Del Maestro, MD,PhDNeurosurgical Neurosimulation ResearchCentre, Department of Neurology and<strong>Neurosurgery</strong>, Montreal NeurologicalInstitute and Hospital, McGill University,Montreal, Quebec, CanadaCorrespondence:Rolando F. Del Maestro, MD, PhD,Neurosurgical NeurosimulationResearch Centre,Montreal Neurological InstituteHospital, 3801 University, Room 438M,Montreal, Quebec, Canada, H3A 2B4.E-mail: rolando.delmaestro@mcgill.caReceived, June 17, 2013.Accepted, July 15, 2013.Copyright ª 2013 by theCongress of Neurological SurgeonsThe development of technical skills is a major goal of any neurosurgical tra<strong>in</strong><strong>in</strong>g program.Residency programs <strong>in</strong> North America are focused on achiev<strong>in</strong>g an adequate levelof tra<strong>in</strong><strong>in</strong>g to produce technically competent surgeons. The tra<strong>in</strong><strong>in</strong>g requirements andeducational environments needed to produce expert surgeons are <strong>in</strong>completelyunderstood. This review explores the theoretical implications of tra<strong>in</strong><strong>in</strong>g technical skillsto expertise rather than competency <strong>in</strong> a complex field such as neurosurgery. First, theterms technical expertise and technical competency are def<strong>in</strong>ed. Def<strong>in</strong>itions of thesequalities are lack<strong>in</strong>g <strong>in</strong> all surgical specialties. Second, the assessment of technical skillsof neurosurgeons are <strong>in</strong>vestigated us<strong>in</strong>g an expert performance approach. Thisapproach entails the design of tasks that can capture the level of expertise <strong>in</strong> a reproduciblemanner. One method to accomplish this <strong>in</strong>volves the use of novel simulatorswith validated performance metrics. Third, the tra<strong>in</strong><strong>in</strong>g of technical skills us<strong>in</strong>g simulationis studied <strong>in</strong> the optic of develop<strong>in</strong>g tra<strong>in</strong><strong>in</strong>g curricula that would target thedevelopment of expertise rather than simple competency. Such curricula should<strong>in</strong>clude objective assessments of technical skills, appropriate feedback, and a distributedschedule of deliberate practice. Implement<strong>in</strong>g a focus on the development ofexpertise rather than simple competency <strong>in</strong> surgical performance will lead to <strong>in</strong>novativedevelopments <strong>in</strong> the field of neurosurgical education. Novel technologies, such assimulation, will play important roles <strong>in</strong> the tra<strong>in</strong><strong>in</strong>g of future expert surgeons, andfocused technical skills curricula with a sound theoretical basis should guide thedevelopment of all such programs.KEY WORDS: Neurosurgical expertise, Neurosurgical practice, NeuroTouch, Simulation, <strong>Surgical</strong> theory<strong>Neurosurgery</strong> 73:S30–S38, 2013 DOI: 10.1227/NEU.0000000000000115 www.neurosurgery-onl<strong>in</strong>e.comEver s<strong>in</strong>ce humans have been perform<strong>in</strong>gsurgical treatments on their peers, a highdegree of competency has been a reasonableexpectation by both the patient and society.Recently, the assessment of technical skills competencyhas brought the critical topic of expertiseto the forefront. This review addresses technicalskills expertise <strong>in</strong> the doma<strong>in</strong> of neurosurgery toadvance the designs of research programs dedicatedto improv<strong>in</strong>g the tra<strong>in</strong><strong>in</strong>g, assessment, andcertification of “expert” surgeons. First, the def<strong>in</strong>itionsof technical skills expertise <strong>in</strong> surgery areexplored. There is a grow<strong>in</strong>g concern that expertise,as def<strong>in</strong>ed by surgeons, is only a representationof competency. The lack of clear def<strong>in</strong>itionsis presented and the consequences of this issue areABBREVIATIONS: OSATS, Objective StructuredAssessment of Technical Skill; VR, virtual realitydiscussed. The assessment of surgeons’ skills,primarily technical skills, are reviewed <strong>in</strong> thecontext of an expert performance approach toexpertise. As def<strong>in</strong>ed by Ericsson and Charness, 1expert performance is a laboratory technique thatcould allow the assessment of the extent ofexpertise <strong>in</strong> a context that is as close as possibleto the real tasks. Multiple tools to assess technicalskills of surgeons were designed and validated <strong>in</strong>the past 2 decades, and their results are reviewed.The evidence relat<strong>in</strong>g expert performance andsurgical simulation are discussed and the tra<strong>in</strong><strong>in</strong>gof neurosurgeons to achieve an “expert” level oftechnical performance reviewed. The literatureoutl<strong>in</strong><strong>in</strong>g the path to expertise <strong>in</strong> surgery isappraised <strong>in</strong> the context of a theoretical tra<strong>in</strong><strong>in</strong>gcurriculum focus<strong>in</strong>g on the comb<strong>in</strong>ation of expertperformance assessment, deliberate practice, andsimulation <strong>in</strong> neurosurgery.S30 | VOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENTwww.neurosurgery-onl<strong>in</strong>e.com
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.EXPERTISE IN NEUROSURGERYA DEFINITION OF EXPERTISE?Beg<strong>in</strong>n<strong>in</strong>g <strong>in</strong> the late 19th century, surgeons were tra<strong>in</strong>ed andassessed <strong>in</strong> a prolonged and organized fashion under thesupervision of senior doctors. 2 This novice/expert apprenticeshipprogram is the dom<strong>in</strong>ant method of tra<strong>in</strong><strong>in</strong>g surgeons. 3 Newtechnologies, society’s pressure, and a host of external constra<strong>in</strong>tssuch as lawsuits and reduced residents’ work<strong>in</strong>g hours haveresulted <strong>in</strong> a review of this model of teach<strong>in</strong>g. 2,4 It is nowbecom<strong>in</strong>g <strong>in</strong>creas<strong>in</strong>gly difficult to practice one’s surgical skills onpatients without hav<strong>in</strong>g reached a certa<strong>in</strong> level of “expertise.”Most authors agree that new methods of tra<strong>in</strong><strong>in</strong>g are required,but a fundamental (unanswered) question rema<strong>in</strong>s: Are thecurrent tra<strong>in</strong><strong>in</strong>g programs supposed to tra<strong>in</strong> surgeons to an“expert” or to only a competent level of performance?In 1990, a report discuss<strong>in</strong>g the perception of expertise <strong>in</strong>medic<strong>in</strong>e used <strong>in</strong>dicators of expertise such as years of experience,specialty board certification, and/or academic rank or responsibility.5 It is evident that these criteria are poorly correlated withsuperior cl<strong>in</strong>ical performance. An example of such failure of these<strong>in</strong>dicators to capture expertise <strong>in</strong>volved peer-nom<strong>in</strong>ated diagnosticexperts that did not objectively perform better than noviceson standard cases. 6 A systematic review that analyzed the numberof years <strong>in</strong> practice along with cl<strong>in</strong>ical performance founda negative correlation, suggest<strong>in</strong>g that more experience can beparadoxically associated with lower cl<strong>in</strong>ical performance. 7 Thesef<strong>in</strong>d<strong>in</strong>gs outl<strong>in</strong>e the lack of consensus <strong>in</strong> what constitutes an“expert” physician or surgeon and the difficult task of objectivelyevaluat<strong>in</strong>g performance.A precise def<strong>in</strong>ition of expertise <strong>in</strong> neurosurgery is needed. Thecurrent literature predom<strong>in</strong>antly focuses on the theme of surgical“competency” rather than surgical “expertise.” 8 This theme preference,orig<strong>in</strong>at<strong>in</strong>g from the grow<strong>in</strong>g importance of competency-basedassessment programs, has become the favored method ofevaluation by tra<strong>in</strong><strong>in</strong>g programs. Def<strong>in</strong>itions of surgicalcompetence usually <strong>in</strong>volve 2 aspects: the technical skills andthe “other skills.” Thelatterskillshavebeenoutl<strong>in</strong>edby2majorgroups <strong>in</strong> North America: the Royal College of Surgeons andPhysicians of Canada and the Accreditation Council onGraduate Medical Education <strong>in</strong> the United States. These 2organizations have highlighted aspects such as professionalism,communication skills, medical expertise, and collaboration skillsas important “other skills.” 9,10 A recent and very comprehensivedef<strong>in</strong>ition of competence states that surgical competenceencompasses knowledge and technical and social skills to solvefamiliar and novel situations to provide adequate patient care. 8This def<strong>in</strong>ition focuses, <strong>in</strong>terest<strong>in</strong>gly, on “adequate” rather than“excellent” patient care as a goal.To evaluate competency, 2 general approaches can be used: thebehaviorist approach, <strong>in</strong> which specific behaviors are rated, and theholistic approach, <strong>in</strong> which many aspects are comb<strong>in</strong>ed andevaluated. Bhatti and Cumm<strong>in</strong>gs 8 have concluded that theholistic approach is probably the best way to assess competency,but also the most difficult because of the lack of def<strong>in</strong>edmethodologies to measure “other skills.” This review concentrateson a behavioristic approach because technical skills are measurablebehaviors.Confusion is apparent when patients and doctors def<strong>in</strong>e surgicalcompetency. For a researcher and a neurosurgical tra<strong>in</strong><strong>in</strong>g program,competency is more than just pass<strong>in</strong>g an exam<strong>in</strong>ation. 8 Forthe patient and family, competency is the complex set of “skills”that contribute to an excellent outcome. The def<strong>in</strong>ition of expertand expertise can vary from specialty to specialty and even with<strong>in</strong>1 surgical specialty. Is a consensus def<strong>in</strong>ition of expertise thereforepossible and to what extent would such a def<strong>in</strong>ition affect thetra<strong>in</strong><strong>in</strong>g of neurosurgeons and, ultimately, patient care?With regard to patient care, the report “To Err Is Human” 11concluded that more than 98,000 deaths are caused by medicalerrors each year and that the majority of these errors werepreventable. After the <strong>in</strong>troduction of laparoscopic cholecystectomy,studies showed that surgeons who performed fewer than 30laparoscopic cholecystectomies had a fivefold <strong>in</strong>crease <strong>in</strong> bile duct<strong>in</strong>jury. 12 This prompted the general surgical community toreevaluate tra<strong>in</strong><strong>in</strong>g methods and to develop box-tra<strong>in</strong>er simulationsand virtual reality (VR) simulators. 2 In most generalsurgery studies, technical competency was usually def<strong>in</strong>ed as thenumber of procedures performed. Many neurosurgical proceduresare not as stereotypical and as easily classified as those ofgeneral surgery. In neurosurgery, there may be significantvariability <strong>in</strong> the surgical approach used to deal with a specificoperative lesion. This <strong>in</strong>herent variability may be the result of nothav<strong>in</strong>g the necessary phase III data that can be used to evaluatethe short- and long-term patient outcomes of different operativeapproaches. This leads to a major problem when one tries todef<strong>in</strong>e technical competency <strong>in</strong> a specific operative <strong>in</strong>tervention.To remedy this problem, a group of authors state that clearbenchmarks for every operation should be available anddeveloped to m<strong>in</strong>imize complications and improve results. 2 Thisprocess is <strong>in</strong>herently difficult and needs to be implemented for allneurosurgical procedures. No clear operational def<strong>in</strong>ition oftechnical competency/expertise <strong>in</strong> neurosurgery can, at present,be outl<strong>in</strong>ed.A major assumption <strong>in</strong> most studies regard<strong>in</strong>g expertise <strong>in</strong>medic<strong>in</strong>e is that novices will <strong>in</strong>evitably become experts withenough practice. Mylopoulos and Regehr 13 argue that rout<strong>in</strong>eexpertise can be achieved this way, but a more accuraterepresentation of expert performance probably relates to adaptiveexpertise. Rout<strong>in</strong>e experts are ak<strong>in</strong> to skilled technicians <strong>in</strong> theirspecific doma<strong>in</strong>, but they lack the tra<strong>in</strong><strong>in</strong>g to face novel problems.Adaptive experts stretch the boundaries of their own limits byus<strong>in</strong>g flexible and creative means to solve complex andunexpected situations. A qualitative study on this topic <strong>in</strong>volv<strong>in</strong>gundergraduate medical students identified that students believedthat it was beyond the scope of their tra<strong>in</strong><strong>in</strong>g to acquire<strong>in</strong>novative thought processes. 14 The reasons for these ideas areunclear, but this issue needs to be addressed if adaptive expertiseis to be a reasonable goal of neurosurgical tra<strong>in</strong><strong>in</strong>g. <strong>Surgical</strong>expertise is best described as an adaptive expertise because, likeNEUROSURGERYVOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENT | S31
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.GÉLINAS-PHANEUF AND DEL MAESTROpilots, neurosurgeons are tra<strong>in</strong>ed to deal with complex situations<strong>in</strong>volv<strong>in</strong>g unplanned and life-threaten<strong>in</strong>g events. 15The assumption beh<strong>in</strong>d all these issues is that understand<strong>in</strong>gwhat is required to be an expert and how experts achieve a level ofexpertise will allow the development of tra<strong>in</strong><strong>in</strong>g programs thatenable more neurosurgical residents and practic<strong>in</strong>g neurosurgeonsto become “experts” and ma<strong>in</strong>ta<strong>in</strong> their expertise. 16 How theCanMeds competency or the Accreditation Council on GraduateMedical Education core competencies model help achieve thegoal of <strong>in</strong>creas<strong>in</strong>g neurosurgical technical expertise needs furtherstudy.EXPERT PERFORMANCE IN SURGERY:CURRENT EVIDENCE<strong>Expertise</strong> can be represented by the ability to reproduceconsistently superior performance on a given task, on demand. 17This task, when it represents the essence of what an expert does <strong>in</strong>a given field, can be used <strong>in</strong> the expert performance paradigm. Itis the researchers’ chore to ensure that the task truly representspo<strong>in</strong>ts of expertise <strong>in</strong> the doma<strong>in</strong> and not just associatedepiphenomena. 18 The issue of experience, <strong>in</strong>struction, andexpertise has been extensively studied by the group led byEricsson over the past 2 decades. 1,17,18 To achieve a high level ofexpertise, experts need to master multiple technical factors,<strong>in</strong>clud<strong>in</strong>g the techniques <strong>in</strong>volved <strong>in</strong> their specialty and have themotivation to pursue the development and cont<strong>in</strong>ued developmentof expertise <strong>in</strong> these techniques. 17-19 Failure to cont<strong>in</strong>ue toimprove and develop these techniques with time results <strong>in</strong>a significant loss of performance (Figure 1). Ericsson 17,20 hasproposed, that an expert performance scheme could (and should)be adaptable to surgery. Research methodologies assess<strong>in</strong>g expertperformance are best carried out <strong>in</strong> constra<strong>in</strong>ed environments, beit a laboratory or an operat<strong>in</strong>g room. One must f<strong>in</strong>d a task thatrepresents with high fidelity an arena where the expert will giveFIGURE 1. Two trends for the development, ma<strong>in</strong>tenance, and decay of medicalperformance as a function of time, experience, and <strong>in</strong>struction. Adapted withpermission Ericsson KA. Acad Med. 2004;79(10 Suppl):S70-S81. Copyright Ó2004, Association of American Medical Colleges.a constant superior performance and that this superior performanceis actually derived from the alleged expertise. 20The current literature on the assessment of superior performance<strong>in</strong> surgery <strong>in</strong>volves technical skills perta<strong>in</strong><strong>in</strong>g to specificsurgical procedures. Simulator studies have <strong>in</strong>vestigated surgicaltechnical performance out of the operat<strong>in</strong>g room sett<strong>in</strong>g, but rarely“other skills.” 21 An objective assessment of technical skills hasalways been a goal of any neurosurgical tra<strong>in</strong><strong>in</strong>g program. Theseassessments are carried out to ensure that their students haveadequately mastered the different techniques required for theirspecific specialty. An extensive study has shown that checklists failto measure differences between various levels of expertise. 22Checklists only discrim<strong>in</strong>ate expertise up to a certa<strong>in</strong> po<strong>in</strong>t, afterwhich <strong>in</strong>termediate experts score as well as experienced surgeons.This was <strong>in</strong> contrast to global rat<strong>in</strong>g scales that ma<strong>in</strong>ta<strong>in</strong>ed theirability to differentiate levels of expertise over a larger range. 22,23The design of most studies on this topic <strong>in</strong>volves an expert-novicedesign to obta<strong>in</strong> various forms of validity of the tool be<strong>in</strong>gtested. 24 To <strong>in</strong>tegrate any technique or new technology <strong>in</strong>toa formal tra<strong>in</strong><strong>in</strong>g curriculum, it must be proved that tra<strong>in</strong><strong>in</strong>g isuseful and appropriate. A well-established series of validationsteps needs to be undertaken: face, content, construct, andconcurrent validity. Face and content validity determ<strong>in</strong>e thata technology is realistic and targets tra<strong>in</strong><strong>in</strong>g skills that are requiredto be tra<strong>in</strong>ed. Construct validity establishes that the scoresobta<strong>in</strong>ed correlate with actual operative technical skill bydiscrim<strong>in</strong>at<strong>in</strong>g experts from novices. This enables novices topractice and tra<strong>in</strong> <strong>in</strong> the technology until they reach theperformance of an expert. The f<strong>in</strong>al step, concurrent validity,is particularly important should that technology be used forassessment as it demonstrates that the skills acquired dur<strong>in</strong>gtra<strong>in</strong><strong>in</strong>g reflect performance <strong>in</strong> the operat<strong>in</strong>g room. Reznick’sgroup developed a tool to measure the performance of tra<strong>in</strong>eesand experts <strong>in</strong> the field of general surgery. This tool, theObjective Structured Assessment of Technical Skill (OSATS),has been shown to be both valid and reliable <strong>in</strong> multiplestudies. 25-27 However, the OSATS was never validated <strong>in</strong>neurosurgery, lead<strong>in</strong>g to the possibility of <strong>in</strong>accurate assessmentsif used without proper validation <strong>in</strong> neurosurgical studies. Agroup from McGill University designed another tool to evaluatethe performance of experts and novices dur<strong>in</strong>g live laparoscopicsurgery. This tool, the Global Operative Assessment of LaparoscopicSkills, has also been shown to be valid and reliable. 28 Avariety of others scales have been developed for other surgicalspecialties, <strong>in</strong>clud<strong>in</strong>g ophthalmology, gastroenterology, and earnose-throatsurgery. 29-31 There is currently no validated tool ofthis k<strong>in</strong>d <strong>in</strong> neurosurgery. A current project, the GlobalAssessment of Intraoperative Neurosurgical Skills, is try<strong>in</strong>g toaddresses this issue <strong>in</strong> neurosurgery. 32 Global Assessment ofIntraoperative Neurosurgical Skills uses the same pr<strong>in</strong>ciple ofexpert-novice comparison to validate the tool. A major limitation<strong>in</strong> design<strong>in</strong>g such a tool is the def<strong>in</strong>ition of what constitutes anexpert <strong>in</strong> neurosurgery. F<strong>in</strong>d<strong>in</strong>g the appropriate tasks that allowan accurate assessment of the full extent of expertise of a subjectS32 | VOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENTwww.neurosurgery-onl<strong>in</strong>e.com
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.EXPERTISE IN NEUROSURGERY<strong>in</strong> the operat<strong>in</strong>g room is a complex problem. A number of skillsbesides technical expertise <strong>in</strong>fluence the performance of anyneurosurgical procedure. The evaluation of the many technical,cognitive, and social contributions that affect overall globalperformance is difficult. For example, bra<strong>in</strong> tumors can beremoved us<strong>in</strong>g multiple techniques, the majority of which haveno randomized, controlled data to help determ<strong>in</strong>e whether onetechnical approach is equal or superior to another.Evaluations of performance need to be carried out by experts toachieve the highest level of validity and reliability. Both OSATSand Global Operative Assessment of Laparoscopic Skills usedexpert reviewers to rate performance. 26,28 This produced a level ofreliability high enough to be used <strong>in</strong> high-stake evaluations. 28Rat<strong>in</strong>g such scales can be carried out dur<strong>in</strong>g the procedure itselfor a posteriori us<strong>in</strong>g recorded video. It is critically important totra<strong>in</strong> the raters before us<strong>in</strong>g the tool to achieve a higher level of<strong>in</strong>terrater reliability. 33 F<strong>in</strong>ally, us<strong>in</strong>g video record<strong>in</strong>gs can shortenthe evaluation time required per tra<strong>in</strong>ee, with a mean reductionto 15 m<strong>in</strong>utes for 1 evaluation. 34 Assum<strong>in</strong>g that every novice willbecome an expert gives rise to the possibility that test<strong>in</strong>g simpletasks will lead to “the risk of narrow<strong>in</strong>g the def<strong>in</strong>ition of expertiseto performance of the mundane.” 13 The designer of suchassessment tools must then ensure the complex representativenessof the task <strong>in</strong>volved. This will not only lead to <strong>in</strong>creased facevalidity of the task but to a closer agreement between thelaboratory task and the actual operat<strong>in</strong>g room experience.Us<strong>in</strong>g Simulation to Reach <strong>Expertise</strong>This section discusses assessment and tra<strong>in</strong><strong>in</strong>g of surgeons onsimulators us<strong>in</strong>g the expert performance approach cognizant of thedifferences between real operations and simulated ones. First,a patient’s surgical lesion presents a variety of <strong>in</strong>dividualcharacteristics that are currently difficult to <strong>in</strong>tegrate <strong>in</strong>toa simulator or a laboratory sett<strong>in</strong>g, although patient rehearsal isbe<strong>in</strong>g developed. 19 However, the simulated task needs to be as“lifelike” as possible to rema<strong>in</strong> with<strong>in</strong> the boundaries of an expertperformance approach. 17 A simulated task is by nature constra<strong>in</strong>ed,but this has advantages when one evaluates different<strong>in</strong>dividuals because it ensures comparability. The transfer ofcompetency/expertise from the simulator to the operat<strong>in</strong>g roomneeds to be carefully evaluated. Simulators can be physicalmodels, 35 augmented reality models, such as ProMIS, 36 orcomplete VR models. 37,38 A variety of simulators are be<strong>in</strong>gdeveloped <strong>in</strong> the field of neurosurgery. 38-41 Simulators allowtra<strong>in</strong>ees to practice without risks to the patient <strong>in</strong> a safeenvironment and remove the stress of the operat<strong>in</strong>g roomenvironment from the task. 2 The question of the optimal site andsetup for a neurosurgical simulation center is not known, but itwould seem reasonable that such simulators be housed <strong>in</strong>locations that are easily accessible to residents. To maximizeresident, staff, and researcher <strong>in</strong>teraction, the NeurosurgicalSimulation Research Centre at the Montreal NeurologicalInstitute and Hospital, which <strong>in</strong>cludes a variety of simulatorsand resident work stations, was located near a patient care area,a short distance from the hospital operat<strong>in</strong>g rooms (Figure 2).There are theoretical advantages to high-fidelity simulators.Us<strong>in</strong>g Ericsson’s expert performance approach of Ericsson andCharness, 1 the closer one gets to the essence of expertise <strong>in</strong> a givenfield, the better are the <strong>in</strong>sights concern<strong>in</strong>g the differencesbetween novices and experts. This is supported by a randomizedstudy that has shown that a task-specific simulator yielded betterresults on specific sutur<strong>in</strong>g tasks <strong>in</strong> laparoscopic anatomises thantra<strong>in</strong><strong>in</strong>g <strong>in</strong> basic sutur<strong>in</strong>g tasks. 42 High-fidelity simulators havenot been analyzed <strong>in</strong> cost-effectiveness studies, and focusedresearch is necessary to justify the cost of development and use.The designers of simulation tra<strong>in</strong><strong>in</strong>g curricula need to developcost-efficient tra<strong>in</strong><strong>in</strong>g programs by provid<strong>in</strong>g more than justsimple technical skills on simulators because “learn<strong>in</strong>g how tocarry out a surgical procedure, after the core skills have beenmastered, is overwhelm<strong>in</strong>gly a cognitive and not a technicaltask.” 43A major limitation of assess<strong>in</strong>g expert performance on a simulatorarises when only “part-tasks” are used. It has beendemonstrated that superior performance us<strong>in</strong>g “part-tasks” canbe atta<strong>in</strong>ed <strong>in</strong>dependently without show<strong>in</strong>g superior performancewhen the whole task is analyzed. 18 Mov<strong>in</strong>g a simulated r<strong>in</strong>g frompeg to peg <strong>in</strong> a VR laparoscopic environment is not actuallyevaluat<strong>in</strong>g expertise but merely a component of the psychomotorskills required to achieve superior performance dur<strong>in</strong>g livesurgery. Conversely, a simple 2-dimensional crown preparationVR simulator <strong>in</strong> dentistry was able to differentiate experts fromFIGURE 2. The Neurosurgical Simulation Research Centre at the Montreal Neurological Institute and Hospital demonstrat<strong>in</strong>gNeuroTouch virtual reality neurosurgical simulators and research environment. On the right, prototype and box-tra<strong>in</strong>er on right.In the center, NeuroTouch with 3-dimensional stereoscopic viewer, haptic feedback, real-time bimanual simulation, TV systems ,and multiple computer work stations are all <strong>in</strong>tegral parts of the Center’s systems.NEUROSURGERYVOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENT | S33
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.GÉLINAS-PHANEUF AND DEL MAESTROnovices. 44 The aviation <strong>in</strong>dustry uses “almost real” to tra<strong>in</strong> pilots.Tasks are effectively complete, and part-tasks are not practicedbecause the actual cockpit is used and not a simplification of the<strong>in</strong>struments needed to pilot the plane. 45 Develop<strong>in</strong>g patientspecificreconstruction for tra<strong>in</strong><strong>in</strong>g and rehearsal is a future goal ofVR surgical simulators, 19 but its full <strong>in</strong>tegration <strong>in</strong>to a virtualneurosurgical operat<strong>in</strong>g room will take time. Some systems arecurrently try<strong>in</strong>g to <strong>in</strong>tegrate patient-specific data, notably dur<strong>in</strong>ganeurysm clipp<strong>in</strong>g simulation on the Dextroscope 46 and tumorresection on the NeuroTouch. 19Simulated tasks and performance metrics on such tasks need tobe validated before be<strong>in</strong>g <strong>in</strong>tegrated <strong>in</strong>to a tra<strong>in</strong><strong>in</strong>g curriculum.Kneebone 47 has outl<strong>in</strong>ed a conceptual framework of sucha program <strong>in</strong>tegrat<strong>in</strong>g simulation. He postulated that simulation<strong>in</strong> this context must atta<strong>in</strong> 4 goals: (1) deliberate practice <strong>in</strong> a safeenvironment, (2) availability of expert tutors who can effectivelyscaffold and enhance the teach<strong>in</strong>g material, (3) authenticsimulation that fits <strong>in</strong> a community of practice, and (4) anenvironment that takes <strong>in</strong>to account the emotional component oflearn<strong>in</strong>g. These 4 theoretical goals have not been adequatelyaddressed and evaluated regard<strong>in</strong>g simulation surgical tra<strong>in</strong><strong>in</strong>g.Currently, most performances on simulators are evaluated us<strong>in</strong>gperformance metrics generated by computer systems. All VRsimulators use different metrics that are not standardized acrosscompanies, mak<strong>in</strong>g mean<strong>in</strong>gful comparisons between simulatorsdifficult. 19,36,48-50 Even with reliable metrics, some studies havefailed to show consistent construct validity, further limit<strong>in</strong>g theiruse <strong>in</strong> a tra<strong>in</strong><strong>in</strong>g curriculum. 50 Future VR simulators should usenovel, specific, and quantifiable metrics that can be thoroughlyassessed for both construct validity and reliability beforewidespread adoption.The expert performance model outl<strong>in</strong>es that laboratory/simulatedtasks need to be constra<strong>in</strong>ed to ensure repeatability and objectiveassessment. 18 Once expert performance is achieved on thesimulator, one should determ<strong>in</strong>e whether this expertise willimprove the performance of the surgeon dur<strong>in</strong>g “real” surgeries.Recent randomized, controlled trials and meta-analyses haveaddressed these issues. 51-55 Data from these studies show thatexpert performance on a VR simulated task is associated withimproved levels of performance us<strong>in</strong>g both pig models 56 and actual“live” human surgeries. 55Simulators <strong>in</strong> neurosurgery can be used to explore 2 questionsthat relate directly to the question of neurosurgical expertise. First,how do expert neurosurgeons actually perform neurosurgicaloperations? Can questions such as what visual, tactile, and/orother cues are expert surgeons us<strong>in</strong>g dur<strong>in</strong>g the technicalcomponents of their operations be assessed us<strong>in</strong>g simulationtechnology? Second, with the proper simulation tools andcurriculum can the goal of neurosurgical tra<strong>in</strong><strong>in</strong>g programs beshifted from teach<strong>in</strong>g to competence to teach<strong>in</strong>g to expert level? Anumber of new simulation systems are be<strong>in</strong>g developed <strong>in</strong> the areaof neurosurgery. 19,38,40,57,58 A new project, NeuroTouch, iscurrently under development at the National Research Council ofCanada. 19,38,59 The goal of this project is to develop a VRneurosurgical simulator <strong>in</strong> 3 dimensions with haptic feedback(Figure 2). The usefulness of this or other simulators <strong>in</strong> theassessment of expert performance and resident tra<strong>in</strong><strong>in</strong>g rema<strong>in</strong>sto be def<strong>in</strong>ed.A Tra<strong>in</strong><strong>in</strong>g Curriculum Target<strong>in</strong>g <strong>Expertise</strong>: Is Ita Possible Goal?An expert performance approach to study<strong>in</strong>g neurosurgicaltechnical expertise should be transferable to a tra<strong>in</strong><strong>in</strong>g curriculum.Because the literature on the topic primarily <strong>in</strong>volves generalsurgery, a number of assumptions will need to be made to<strong>in</strong>corporate these research results <strong>in</strong>to the development ofneurosurgical curriculum. First, the theory of expert performanceis a general theory, with possible applications <strong>in</strong> a variety of fields,<strong>in</strong>clud<strong>in</strong>g neurosurgery. 17 Second, deliberate practice will bea s<strong>in</strong>e qua non requirement <strong>in</strong> such a program. 20 Third, expertswill be assumed to be highly competent neurosurgeons for thesake of the discussion because no consensual def<strong>in</strong>ition exists as towhat an expert neurosurgeon is. Fourth, feedback is an essentialbuild<strong>in</strong>g block of such a program. 20 F<strong>in</strong>ally, such a tra<strong>in</strong><strong>in</strong>gprogram should use current adult learn<strong>in</strong>g theories to achievemaximal efficiency. 60The pressure to change the tra<strong>in</strong><strong>in</strong>g curriculum, focus<strong>in</strong>g onsurgical skills, is an important issue faced by all neurosurgicalresidency tra<strong>in</strong><strong>in</strong>g committees. 61 There are no reasons to preventsimulation from play<strong>in</strong>g a major role <strong>in</strong> the neurosurgicaleducation of tra<strong>in</strong>ees, focused on improv<strong>in</strong>g their technical skillsacquisition. 62 The use of simulation <strong>in</strong> a curriculum “should bedriven by educational imperatives and not by technological<strong>in</strong>novation.” 43 Objective assessment of technical skills is possibleto a high degree of validity and reliability <strong>in</strong> surgery, with somedata be<strong>in</strong>g developed for neurosurgery (Gél<strong>in</strong>as-Phaneuf et al.2013, unpublished work). 63 Initial studies <strong>in</strong>volv<strong>in</strong>g medicalstudents and neurosurgical residents were conducted <strong>in</strong> a pilotvalidation study us<strong>in</strong>g a NeuroTouch tumor resection scenarioand demonstrated face, content, and construct validity <strong>in</strong>a competitive sett<strong>in</strong>g. 63 At present, studies are be<strong>in</strong>g conductedus<strong>in</strong>g “expert” neurosurgeons <strong>in</strong> noncompetitive sett<strong>in</strong>gs to<strong>in</strong>crease our understand<strong>in</strong>g of expert neurosurgical performance.Hence, the first requirement of an expert performance approachto assess and tra<strong>in</strong> neurosurgeons is possible because tasks (eg, theremoval of an <strong>in</strong>tracranial men<strong>in</strong>gioma or metastasis) can beobjectively measured <strong>in</strong> terms of performance. Deliberate practice<strong>in</strong> an expertise-oriented curriculum is essential. Multiplepublications have consistently demonstrated that expert performers<strong>in</strong> a variety of fields have more than 10 years of purposefulpractice to achieve an <strong>in</strong>ternational level of expertise. 20 In thefield of surgery, <strong>in</strong>nate talent has not been clearly demonstrated,61 and contradictive studies have associated higher thecorrelation of visuospatial skills to “expert” surgical performancerema<strong>in</strong>s unclear. 64-66 Deliberate practice paradigms will need tobe <strong>in</strong>corporated <strong>in</strong>to the design of any curriculum whose goal isto tra<strong>in</strong> to specific levels of expertise. Ericsson 20 demonstratedS34 | VOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENTwww.neurosurgery-onl<strong>in</strong>e.com
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.EXPERTISE IN NEUROSURGERYthat deliberate practice is a purposeful, goal-oriented method thatone uses to improve one’s performance. It is not <strong>in</strong>herentlypleasant, and it requires on the order of 10,000 hours to reach an<strong>in</strong>ternational level of performance. In neurosurgery, this level ofexpertise may not be acquired until neurosurgeons reach their 40sor early 50s (Figure 3). A recent commentary 67 calculated that10,000 hours of deliberate practice are reached after approximately6.9 years of tra<strong>in</strong><strong>in</strong>g <strong>in</strong> neurosurgery. In Canada, this isthe time equivalent to a regular tra<strong>in</strong><strong>in</strong>g program and 1 additionalyear of fellowship. A critical question, however, has not beenaddressed: how much of the time <strong>in</strong> the hospital and operat<strong>in</strong>groom can actually be termed deliberate practice as opposed to thedaily tasks of be<strong>in</strong>g a resident? If a neurosurgical tra<strong>in</strong><strong>in</strong>g programgoal is to use deliberate practice to improve performance, thensimulation may be a reasonable technology to achieve this goal.Simulation allows a resident time to practice both rout<strong>in</strong>e andcritical technical skills us<strong>in</strong>g a goal-directed expert performanceapproach.How to Incorporate Simulation <strong>Into</strong> Such a Curriculum?Most surgical skills are natural extensions of directed psychomotortasks. The model of skill acquisition described by Fitts andPosner 68 is the most widely cited <strong>in</strong> surgery. 4 This modeldescribes 3 stages of skill acquisition. The first stage <strong>in</strong>volvescognitive acquisition <strong>in</strong> which an <strong>in</strong>dividual learns the multiplesteps required to achieve the performance. Mistakes are frequentdur<strong>in</strong>g this stage, and it represents the base of the learn<strong>in</strong>g curve.The second stage is called the associative stage, and it results <strong>in</strong>a gradual improvement of the performance. Mistakes are rarer,and this is represented as the rapid <strong>in</strong>crease <strong>in</strong> skill on a learn<strong>in</strong>gcurve. The last stage, the automatic stage (or plateau) is theorizedto represent the accomplishment of enough practice to performthe task while free<strong>in</strong>g substantial amounts of cognitive <strong>in</strong>puts.This model raises a number of issues. First, authors disagree onthe need to achieve automaticity. Proponents of the Fitts andPosner model have shown that dual-task possibilities of experts <strong>in</strong>surgery are <strong>in</strong>creased when the task is automated. 69 Also, thepractice time required to achieve automaticity is variable fromone <strong>in</strong>dividual to the next, greater than 100 repetitions ona simulator <strong>in</strong> 1 study. 70 Ericsson 17,20,71 argued that automaticityis counterproductive to the achievement of expertise. Hesuggested that once a task has reached automaticity, additionalpractice will not improve the performance but will only ma<strong>in</strong>ta<strong>in</strong>the previous level of proficiency. This phenomenon has beennamed arrested development, and deliberate practice has beenhypothesized to be 1 method to counter this roadblock toacquir<strong>in</strong>g new skills 72 (Figure 4). The debate on automaticity <strong>in</strong>learn<strong>in</strong>g a psychomotor task is still open, but the current evidenceseems to po<strong>in</strong>t to a fusion of these 2 oppos<strong>in</strong>g views. Basicpsychomotor skills can and should be automatized, but the wholeperformance paradigm should always have the ability to beimproved through deliberate practice.Simulated tasks have been shown to be helpful when design<strong>in</strong>ga curriculum that implements the theoretical learn<strong>in</strong>g conceptspreviously outl<strong>in</strong>ed. A general surgery group <strong>in</strong>tegrated simulatedtasks address<strong>in</strong>g the 3 stages of the Fitts and Posner model <strong>in</strong>to thedesign of a competency-based curriculum. 73 They demonstratedthat the design of such a curriculum is feasible when assess<strong>in</strong>g tasksof different levels of complexity, and proficiency <strong>in</strong> difficult tasksimproved after tra<strong>in</strong><strong>in</strong>g on easier ones. This may have resulted frompartial automatization of the task. This group also demonstratedthat VR whole task simulation of laparoscopic cholecystectomy hadconstruct validity. 74 Dunphy and Williamson 60 reviewed variousmodels of expertise <strong>in</strong> medic<strong>in</strong>e, surgery, and nurs<strong>in</strong>g. Theycomb<strong>in</strong>ed the Fitts and Posner model with Vygotsky’s zone ofproximal development model, 75 result<strong>in</strong>g <strong>in</strong> a conceptual systemthat seems <strong>in</strong>tuitively correct. This new template has 4 stages,with the first 3 stages similar to those of Fitts and Posner butadd<strong>in</strong>g the ideas of outside regulation dur<strong>in</strong>g the first stage andself-regulation <strong>in</strong> the second stage, ideas taken from Vigotsky’sFIGURE 3. A model demonstrat<strong>in</strong>g the gradual <strong>in</strong>crease <strong>in</strong> surgical performanceas a function of age, before, dur<strong>in</strong>g, and after neurosurgical residency tra<strong>in</strong><strong>in</strong>g.The competence and expert levels are <strong>in</strong>dicated by the horizontal dashed l<strong>in</strong>es.FIGURE 4. Experts progress cont<strong>in</strong>uously by avoid<strong>in</strong>g rout<strong>in</strong>eautomaticity and ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g a schedule of deliberate practice.The second curve can represent automaticity as lead<strong>in</strong>g to arresteddevelopment when the participant stops deliberatepractice. Adapted with permission Ericsson KA. Acad Med.2004;79(10 Suppl):S70-S81. Copyright Ó 2004, Associationof American Medical Colleges.NEUROSURGERYVOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENT | S35
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.GÉLINAS-PHANEUF AND DEL MAESTROzone of proximal development model <strong>in</strong> which the teacher servesas a scaffold. This model possesses the possibility to “regress” <strong>in</strong>stages, allow<strong>in</strong>g for more deliberate practice. 60 Because feedbackis essential <strong>in</strong> a deliberate practice model, an understand<strong>in</strong>g ofthe <strong>in</strong>dividual components of performance that would be helpedby feedback is important. 19,72 Without knowledge of the criticalsteps of a surgical procedure, it is difficult to give objectivefeedback tailored to a specific <strong>in</strong>dividual task. 19,20 Cognitive taskanalyses have a def<strong>in</strong>ite role to play <strong>in</strong> determ<strong>in</strong><strong>in</strong>g the requiredsteps <strong>in</strong> a specific surgical procedure. 76 The type of feedbackprovided is important. Expert verbal feedback leads to moreretention of a skill than simple computer pr<strong>in</strong>touts ofperformance. 77 This may be an issue dur<strong>in</strong>g self-practice onsimulators, but regular feedback from the teacher dur<strong>in</strong>g formaltra<strong>in</strong><strong>in</strong>g sessions could compensate for this deficiency. 78 Theschedule of such practice is also important. There is evidencethat distributed practice over time is more useful for long-termretention than massed practice. 79 When a distributed curriculumwas compared with nondistributed practice controls, this showedbenefits on performance if distributed practice was scheduled. 80A curriculum aimed at achiev<strong>in</strong>g long-term expertise should thusallow for regular, preferably at least weekly, tra<strong>in</strong><strong>in</strong>g sessions withexpert feedback. 81 The role that script-based mental rehearsal <strong>in</strong>the acquisition and ma<strong>in</strong>tenance of neurosurgical technical skillsneeds to be further def<strong>in</strong>ed and could play an important role <strong>in</strong>resident teach<strong>in</strong>g. 82The tra<strong>in</strong><strong>in</strong>g of technical skills is only 1 component ofa surgical curriculum. The development of multiple other skillsis required to tra<strong>in</strong> a competent/expert neurosurgeon. This canprove difficult to implement <strong>in</strong> an expert performance fashionbecause measur<strong>in</strong>g objectively some of those skills, like professionalism,communication, and “other skills,” is either verycomplex or not yet available. Pilot studies <strong>in</strong> crisis situationmanagement <strong>in</strong> the operat<strong>in</strong>g room show promise <strong>in</strong> thetra<strong>in</strong><strong>in</strong>g of those important “other skills.” 82 The aviation<strong>in</strong>dustry program called “NOTECHS” (NOn-TECHnicalSkills), which can assess these skills <strong>in</strong> an operat<strong>in</strong>g roomenvironment, has achieved variable levels of reliability <strong>in</strong> thedifferent doma<strong>in</strong>s assessed. 83Is <strong>Expertise</strong> a F<strong>in</strong>al Endpo<strong>in</strong>t?The personal advantages of be<strong>in</strong>g an expert neurosurgeon arenot always obvious, and the monetary <strong>in</strong>centives are m<strong>in</strong>imal <strong>in</strong>current Canadian/American society. 84 What are the known andnovel methods that can be used to measure and improve thecompetency of residents and practic<strong>in</strong>g neurosurgeons to a moreexpert level of performance? Cont<strong>in</strong>ued learn<strong>in</strong>g after graduationis paramount <strong>in</strong> any expertise model. Limit<strong>in</strong>g oneself to onlyautomatized operat<strong>in</strong>g skills may result <strong>in</strong> arrested skillsdevelopment and decreased ability to deal with unexpectedand rare life-threaten<strong>in</strong>g <strong>in</strong>traoperative events. 72 Tra<strong>in</strong><strong>in</strong>g solelyon simulators is also a concern because “simulated” expertise isnot the desired endpo<strong>in</strong>t, only a means to an end. Deliberatepractice us<strong>in</strong>g simulators may arm the tra<strong>in</strong>ees with appropriatesurgical skills before they enter the operat<strong>in</strong>g room, hopefullyresult<strong>in</strong>g <strong>in</strong> (1) decreas<strong>in</strong>g surgical errors and improv<strong>in</strong>g patientoutcomes; (2) shortened learn<strong>in</strong>g times to master complexneurosurgical procedures; and (3) a lifelong <strong>in</strong>terest <strong>in</strong> ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>gand augment<strong>in</strong>g their surgical skills.CONCLUSIONThe concept of surgical expertise has been explored <strong>in</strong> thisreview with the <strong>in</strong>tent of develop<strong>in</strong>g a neurosurgical curriculumbased on solid theoretical assumptions. The current literature onthe subject is limited but can guide decisions when design<strong>in</strong>gneurosurgical tra<strong>in</strong><strong>in</strong>g programs. First, a clear consensual def<strong>in</strong>itionof expertise (or a high degree of competency) should bedeterm<strong>in</strong>ed to guide future research. Expert neurosurgical performanceneeds to be measured <strong>in</strong> realistic situations and used as thegoal <strong>in</strong> neurosurgical teach<strong>in</strong>g. Deliberate practice and feedbackshould be the cornerstone of any such program. High-fidelityneurosurgical simulators and validated performance measurementtools should help propel neurosurgical research and tra<strong>in</strong><strong>in</strong>g toaccomplish the goal of tra<strong>in</strong><strong>in</strong>g to expert surgical performance(Gél<strong>in</strong>as-Phaneuf et al. 2013, unpublished work). 63 Time will tellwhether neurosurgeons of the future will be more “expert,”whichever def<strong>in</strong>ition is used.DisclosuresThis work was supported by the Franco Di Giovanni, B-Strong, the TonyColann<strong>in</strong>o Foundations the Montréal English School Board, and the MontrealNeurological Institute and Hospital. Dr Gél<strong>in</strong>as-Phaneuf was funded by a generouscontribution from the Harold and Audrey Fisher Bra<strong>in</strong> Tumour ResearchAward. The authors have no personal f<strong>in</strong>ancial or <strong>in</strong>stitutional <strong>in</strong>terest <strong>in</strong> any ofthe drugs, materials, or devices described <strong>in</strong> this article.REFERENCES1. Ericsson KA, Charness N. Expert performance: its structure and acquisition. AmPsychol. 1994;49(8):725-747.2. Tsuda S, Scott D, Doyle J, Jones DB. <strong>Surgical</strong> skills tra<strong>in</strong><strong>in</strong>g and simulation. CurrProbl Surg. 2009;46(4):271-370.3. Jaffer A, Bednarz B, Challacombe B, Sriprasad S. The assessment of surgicalcompetency <strong>in</strong> the UK. Int J Surg. 2009;7(1):12-15.4. Reznick RK, MacRae H. Teach<strong>in</strong>g surgical skills: changes <strong>in</strong> the w<strong>in</strong>d. N Engl JMed. 2006;355(25):2664-2669.5. Elste<strong>in</strong> AS, Shulman LS, Sprafka SA. Medical problem-solv<strong>in</strong>g:a 10-yearretrospective. Eval Health Professions. 1990;13(1):5-36.6. Elste<strong>in</strong> A, Shulman L, Sprafka S. Medical Problem Solv<strong>in</strong>g: An Analysis of Cl<strong>in</strong>icalReason<strong>in</strong>g. Cambridge, MA: Harvard University Press; 1978.7. Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationshipbetween cl<strong>in</strong>ical experience and quality of health care. Ann Intern Med. 2005;142(4):260-273.8. Bhatti NI, Cumm<strong>in</strong>gs CW. Competency <strong>in</strong> surgical residency tra<strong>in</strong><strong>in</strong>g: def<strong>in</strong><strong>in</strong>gand rais<strong>in</strong>g the bar. Acad Med. 2007;82(6):569-573.9. Chou S, Cole G, McLaughl<strong>in</strong> K, Lockyer J. CanMEDS evaluation <strong>in</strong> canadianpostgraduate tra<strong>in</strong><strong>in</strong>g programmes: tools used and programme director satisfaction.Med Educ. 2008;42(9):879-886.10. Yaszay B, Kubiak E, Agel J, Hanel D. ACGME core competencies: where are we?Orthopedics. 2009;32(3):171.11. Kohn LT, Corrigan J, Donaldson MS. To Err Is Human: Build<strong>in</strong>g a Safer HealthSystem. Wash<strong>in</strong>gton, DC: National Academy Press; 2000.S36 | VOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENTwww.neurosurgery-onl<strong>in</strong>e.com
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.EXPERTISE IN NEUROSURGERY12. Moore MJ, Bennett CL. The learn<strong>in</strong>g curve for laparoscopic cholecystectomy. TheSouthern Surgeons Club. Am J Surg. 1995;170(1):55-59.13. Mylopoulos M, Regehr G. Cognitive metaphors of expertise and knowledge:prospects and limitations for medical education. Med Educ. 2007;41(12):1159-1165.14. Mylopoulos M, Regehr G. How student models of expertise and <strong>in</strong>novationimpact the development of adaptive expertise <strong>in</strong> medic<strong>in</strong>e. Med Educ. 2009;43(2):127-132.15. Tsang PS, Vidulich MA, eds. Pr<strong>in</strong>ciples and Practice of Aviation Psychology. BocaRaton, FL: CRC Press; 2002.16. Abernethy B, Poolton JM, Masters RS, Patil NG. Implications of an expertisemodel for surgical skills tra<strong>in</strong><strong>in</strong>g. ANZ J Surg. 2008;78(12):1092-1095.17. Ericsson KA. Deliberate practice and the acquisition and ma<strong>in</strong>tenance of expertperformance <strong>in</strong> medic<strong>in</strong>e and related doma<strong>in</strong>s. Acad Med. 2004;79(10 suppl):S70-S81.18. Ericsson KA, Lehmann AC. Expert and exceptional performance: evidence ofmaximal adaptation to task constra<strong>in</strong>ts. Annu Rev Psychol. 1996;47:273-305.19. Choudhury N, Gél<strong>in</strong>as-Phaneuf N, Delorme S, Del Maestro R. Fundamentals ofneurosurgery; virtual reality tasks for tra<strong>in</strong><strong>in</strong>g and evaluation of technical skills.World Neurosurg. 2012 Nov 23. [Epub ahead of pr<strong>in</strong>t].20. Ericsson KA. An expert-performance perspective of research on medical expertise:the study of cl<strong>in</strong>ical performance. Med Educ. 2007;41(12):1124-1130.21. Kahol K, Vankipuram M, Smith ML. Cognitive simulators for medical educationand tra<strong>in</strong><strong>in</strong>g. J Biomed Inform. 2009;42(4):593-604.22. Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklistsdo not capture <strong>in</strong>creas<strong>in</strong>g levels of expertise. Acad Med. 1999;74(10):1129-1134.23. Regehr G, MacRae H, Reznick RK, Szalay D. Compar<strong>in</strong>g the psychometricproperties of checklists and global rat<strong>in</strong>g scales for assess<strong>in</strong>g performance on anOSCE-format exam<strong>in</strong>ation. Acad Med. 1998;73(9):993-997.24. Gallagher AG, Ritter EM, Satava RM. Fundamental pr<strong>in</strong>ciples of validation, andreliability: rigorous science for the assessment of surgical education and tra<strong>in</strong><strong>in</strong>g.Surg Endosc. 2003;17(10):1525-1529.25. Reznick R, Regehr G, MacRae H, Mart<strong>in</strong> J, McCulloch W. Test<strong>in</strong>g technical skillvia an <strong>in</strong>novative “bench station” exam<strong>in</strong>ation. Am J Surg. 1997;173(3):226-230.26. Mart<strong>in</strong> JA, Regehr G, Reznick R, et al. Objective structured assessment oftechnical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273-278.27. Faulkner H, Regehr G, Mart<strong>in</strong> J, Reznick R. Validation of an objective structuredassessment of technical skill for surgical residents. Acad Med. 1996;71(12):1363-1365.28. Vassiliou MC, Feldman LS, Andrew CG, et al. A global assessment tool forevaluation of <strong>in</strong>traoperative laparoscopic skills. Am J Surg. 2005;190(1):107-113.29. Cremers SL, Ciol<strong>in</strong>o JB, Ferruf<strong>in</strong>o-Ponce ZK, Henderson BA. Objectiveassessment of skills <strong>in</strong> <strong>in</strong>traocular surgery (OASIS). Ophthalmology. 2005;112(7):1236-1241.30. L<strong>in</strong> SY, Laeeq K, Ishii M, et al. Development and pilot-test<strong>in</strong>g of a feasible,reliable, and valid operative competency assessment tool for endoscopic s<strong>in</strong>ussurgery. Am J Rh<strong>in</strong>ol Allergy. 2009;23(3):354-359.31. Vassiliou MC, Kaneva PA, Poulose BK, et al. Global Assessment of Gastro<strong>in</strong>test<strong>in</strong>alEndoscopic Skills (GAGES): a valid measurement tool for technical skills<strong>in</strong> flexible endoscopy. Surg Endosc. 2010;24(8):1834-1841.32. Gél<strong>in</strong>as-Phaneuf N, Okra<strong>in</strong>ec A, Fried GM, Choudhury N, Del Maestro RF. Theassessment of technical skills <strong>in</strong> neurosurgery: the development of the globalassessment of <strong>in</strong>traoperative neurosurgical skills (GAINS) scale. Paper presented at:Canadian Conference on Medical Education (Poster); May 2010; Sa<strong>in</strong>t-John’s,Newfoundland.33. Vassiliou MC, Feldman LS, Fraser SA, et al. Evaluat<strong>in</strong>g <strong>in</strong>traoperative laparoscopicskill: direct observation versus bl<strong>in</strong>ded videotaped performances. Surg Innov. 2007;14(3):211-216.34. Dath D, Regehr G, Birch D, et al. Toward reliable operative assessment: thereliability and feasibility of videotaped assessment of laparoscopic technical skills.Surg Endosc. 2004;18(12):1800-1804.35. Fried GM. FLS assessment of competency us<strong>in</strong>g simulated laparoscopic tasks.J Gastro<strong>in</strong>test Surg. 2008;12(2):210-212.36. Pellen MG, Horgan LF, Barton JR, Attwood SE. Construct validity of the ProMISlaparoscopic simulator. Surg Endosc. 2009;23(1):130-139.37. Maithel S, Sierra R, Korndorffer J, et al. Construct and face validity of MIST-VR,endotower, and CELTS: are we ready for skills assessment us<strong>in</strong>g simulators? SurgEndosc. 2006;20(1):104-112.38. Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch: a physicsbasedvirtual simulator for cranial microneurosurgery tra<strong>in</strong><strong>in</strong>g. Neurosugery. 2012;71(ons suppl 1):ons32-ons42.39. Ferroli P, Tr<strong>in</strong>gali G, Acerbi F, et al. Advanced 3-dimensional plann<strong>in</strong>g <strong>in</strong>neurosurgery. <strong>Neurosurgery</strong>. 2013;72(suppl 1):54-62.40. Chan S, Conti F, Salisbury K, Blev<strong>in</strong>s NH. Virtual reality simulation <strong>in</strong>neurosurgery: technologies and evolution. <strong>Neurosurgery</strong>. 2013;72(suppl 1):154-164.41. Malone HR, Syed ON, Downes MS, D’Ambrosio AL, Quest DO, Kaiser MG.Simulation <strong>in</strong> neurosurgery: a review of computer-based simulation environmentsand their surgical applications. <strong>Neurosurgery</strong>. 2010;67(4):1105-1116.42. Sabbagh R, Chatterjee S, Chawla A, Kapoor A, Matsumoto ED. Task-specificbench model tra<strong>in</strong><strong>in</strong>g versus basic laparoscopic skills tra<strong>in</strong><strong>in</strong>g for laparoscopicradical prostatectomy: a randomized controlled study. Can Urol Assoc J. 2009;3(1):22-30.43. W<strong>in</strong>dsor JA. Role of simulation <strong>in</strong> surgical education and tra<strong>in</strong><strong>in</strong>g. ANZ J Surg.2009;79(3):127-132.44. Suebnukarn S, Phatthanasathiankul N, Sombatweroje S, Rhienmora P,Haddawy P. Process and outcome measures of expert/novice performance ona haptic virtual reality system. J Dent. 2009;37(9):658-665.45. Hays RT, Jacobs JW, Pr<strong>in</strong>ce C, Salas E. Flight simulator tra<strong>in</strong><strong>in</strong>g effectiveness:a meta-analysis. Mil Psychol. 1992;4(2):63-74.46. Wong GK, Zhu CX, Ahuja AT, Poon WS. Craniotomy and clipp<strong>in</strong>g of<strong>in</strong>tracranial aneurysm <strong>in</strong> a stereoscopic virtual reality environment. <strong>Neurosurgery</strong>.2007;61(3):564-568.47. Kneebone R. Evaluat<strong>in</strong>g cl<strong>in</strong>ical simulations for learn<strong>in</strong>g procedural skills:a theory-based approach. Acad Med. 2005;80(6):549-553.48. Bajka M, Tuchschmid S, F<strong>in</strong>k D, Székely G, Harders M. Establish<strong>in</strong>g constructvalidity of a virtual-reality tra<strong>in</strong><strong>in</strong>g simulator for hysteroscopy via a multimetricscor<strong>in</strong>g system. Surg Endosc. 2010;24(1):79-88.49. Fayez R, Feldman LS, Kaneva P, Fried GM. Test<strong>in</strong>g the construct validity of theSimbionix GI Mentor II virtual reality colonoscopy simulator metrics: modulematters. Surg Endosc. 2010;24(5):1060-1065.50. Andreatta PB, Woodrum DT, Gauger PG, M<strong>in</strong>ter RM. LapMentor metricspossess limited construct validity. Simul Healthc. 2008;3(1):16-25.51. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality tra<strong>in</strong><strong>in</strong>g forsurgical tra<strong>in</strong>ees <strong>in</strong> laparoscopic surgery. Cochrane Database Syst Rev. 2009;(1):CD006575.52. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Systematic review ofrandomized controlled trials on the effectiveness of virtual reality tra<strong>in</strong><strong>in</strong>g forlaparoscopic surgery. Br J Surg. 2008;95(9):1088-1097.53. Sturm LP, W<strong>in</strong>dsor JA, Cosman PH, Cregan P, Hewett PJ, Maddern GJ. Asystematic review of skills transfer after surgical simulation tra<strong>in</strong><strong>in</strong>g. Ann Surg.2008;248(2):166-179.54. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J,Funch-Jensen P. Randomized cl<strong>in</strong>ical trial of virtual reality simulation forlaparoscopic skills tra<strong>in</strong><strong>in</strong>g. Br J Surg. 2004;91(2):146-150.55. Larsen CR, Soerensen JL, Grantcharov TP, et al. Effect of virtual reality tra<strong>in</strong><strong>in</strong>gon laparoscopic surgery: randomised controlled trial. BMJ. 2009;338:b1802.56. Stelzer MK, Abdel MP, Sloan MP, Gould JC. Dry lab practice leads toimproved laparoscopic performance <strong>in</strong> the operat<strong>in</strong>g room. JSurgRes.2009;154(1):163-166.57. Luciano CJ, Banerjee PP, Bellotte B, et al. Learn<strong>in</strong>g retention of thoracic pediclescrew placement us<strong>in</strong>g a high-resolution augmented reality simulator with hapticfeedback. <strong>Neurosurgery</strong>. 2011;69(1 suppl operative):ons14-ons19.58. Alaraj A, Lemole MG, F<strong>in</strong>kle JH, et al. Virtual reality tra<strong>in</strong><strong>in</strong>g <strong>in</strong> neurosurgery:review of current status and future applications. Surg Neurol Int. 2011;2:52.59. Neubauer A, Wolfsberger S. Virtual endoscopy <strong>in</strong> neurosurgery: a review.<strong>Neurosurgery</strong>. 2013;72(suppl 1):97-106.60. Dunphy BC, Williamson SL. In pursuit of expertise. Toward an educationalmodel for expertise development. Adv Health Sci Educ <strong>Theory</strong> Pract. 2004;9(2):107-127.61. Bell RH Jr. Why Johnny cannot operate. Surgery. 2009;146(4):533-542.62. Aggarwal R, Darzi A. Technical-skills tra<strong>in</strong><strong>in</strong>g <strong>in</strong> the 21st century. N Engl J Med.2006;355(25):2695-2696.63. Gél<strong>in</strong>as-Phaneuf N, Choudhury N, Al-Habib AR, et al. Assess<strong>in</strong>g performance <strong>in</strong>bra<strong>in</strong> tumor resection us<strong>in</strong>g a novel virtual reality simulator. Int J Comput AssistRadiol Surg. 2013 June 20. [Epub ahead of pr<strong>in</strong>t].NEUROSURGERYVOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENT | S37
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.GÉLINAS-PHANEUF AND DEL MAESTRO64. Wanzel KR, Hamstra SJ, Cam<strong>in</strong>iti MF, Anastakis DJ, Grober ED, Reznick RK.Visual-spatial ability correlates with efficiency of hand motion and successfulsurgical performance. Surgery. 2003;134(5):750-757.65. Wanzel KR, Hamstra SJ, Anastakis DJ, Matsumoto ED, Cusimano MD. Effect ofvisual-spatial ability on learn<strong>in</strong>g of spatially-complex surgical skills. Lancet. 2002;359(9302):230-231.66. Risucci DA. Visual spatial perception and surgical competence. Am J Surg. 2002;184(3):291-295.67. Omahen DA. The 10,000-hour rule and residency tra<strong>in</strong><strong>in</strong>g. CMAJ 2009;180(12):1272.68. Fitts PM, Posner MI. Human Performance. Belmont, CA: Brooks/Cole; 1967.69. Hsu KE, Man FY, Gizicki RA, Feldman LS, Fried GM. Experienced surgeons cando more than one th<strong>in</strong>g at a time: effect of distraction on performance of a simplelaparoscopic and cognitive task by experienced and novice surgeons. Surg Endosc.2008;22(1):196-201.70. Stefanidis D, Scerbo MW, Sechrist C, Mostafavi A, Heniford BT. Do novicesdisplay automaticity dur<strong>in</strong>g simulator tra<strong>in</strong><strong>in</strong>g? Am J Surg. 2008;195(2):210-213.71. Ericsson KA. Deliberate practice and acquisition of expert performance: a generaloverview. Acad Emerg Med. 2008;15(11):988-994.72. Ericsson KA. The <strong>in</strong>fluence of experience and deliberate practice on thedevelopment of superior expert performance. In: Ericsson KA, Charness N,Feltovich PJ, Hoffman RR, eds. The Cambridge Handbook of <strong>Expertise</strong> and ExpertPerformance. New York, NY: Cambridge University Press; 2006:683-703.73. Aggarwal R, Grantcharov T, Moorthy K, Hance J, Darzi A. A competency-basedvirtual reality tra<strong>in</strong><strong>in</strong>g curriculum for the acquisition of laparoscopic psychomotorskill. Am J Surg. 2006;191(1):128-133.74. Aggarwal R, Crochet P, Dias A, Misra A, Zipr<strong>in</strong> P, Darzi A. Development ofa virtual reality tra<strong>in</strong><strong>in</strong>g curriculum for laparoscopic cholecystectomy. Br J Surg.2009;96(9):1086-1093.75. Vygotsky LS. M<strong>in</strong>d <strong>in</strong> Society: The Development of Higher Psychological. Cambridge,MA: Harvard University Press; 1978.76. Peyre SE, Peyre CG, Hagen JA, et al. Laparoscopic Nissen fundoplicationassessment: task analysis as a model for the development of a procedural checklist.Surg Endosc. 2009;23(6):1227-1232.77. Porte MC, Xeroulis G, Reznick RK, Dubrowski A. Verbal feedback from an expertis more effective than self-accessed feedback about motion efficiency <strong>in</strong> learn<strong>in</strong>gnew surgical skills. Am J Surg. 2007;193(1):105-110.78. Kruglikova I, Grantcharov TP, Drewes AM, Funch-Jensen P. The impact ofconstructive feedback on tra<strong>in</strong><strong>in</strong>g <strong>in</strong> gastro<strong>in</strong>test<strong>in</strong>al endoscopy us<strong>in</strong>g high-fidelityvirtual-reality simulation: a randomized controlled trial. Gut. 2010;59(2):181-185.79. Moulton CA, Dubrowski A, Macrae H, Graham B, Grober E, Reznick R.Teach<strong>in</strong>g surgical skills: what k<strong>in</strong>d of practice makes perfect?: a randomized,controlled trial. Ann Surg. 2006;244(3):400-409.80. Webb TP, Weigelt JA, Redlich PN, Anderson RC, Brasel KJ, Simpson D.Protected block curriculum enhances learn<strong>in</strong>g dur<strong>in</strong>g general surgery residencytra<strong>in</strong><strong>in</strong>g. Arch Surg. 2009;144(2):160-166.81. Stefanidis D, Heniford BT. The formula for a successful laparoscopic skillscurriculum. Arch Surg. 2009;144(1):77-82.82. Undre S, Koutantji M, Sevdalis N, et al. Multidiscipl<strong>in</strong>ary crisis simulations: theway forward for tra<strong>in</strong><strong>in</strong>g surgical teams. World J Surg. 2007;31(9):1843-1853.83. Sevdalis N, Davis R, Koutantji M, Undre S, Darzi A, V<strong>in</strong>cent CA. Reliability ofa revised NOTECHS scale for use <strong>in</strong> surgical teams. Am J Surg. 2008;196(2):184-190.84. Hunt E. <strong>Expertise</strong>, talent and social encouragement. In: Ericsson K, Charness N,Feltovich PJ, Hoffman RR, eds. The Cambridge Handbook of <strong>Expertise</strong> and ExpertPerformance. New York, NY: Cambridge University Press; 2006:31-38.85. Aggarwal R, Grantcharov TP, Darzi A. Framework for systematic tra<strong>in</strong><strong>in</strong>g andassessment of technical skills. J Am Coll Surg. 2007;204(4):697-705.86. Marcus H, Vakharia V, Kirkman MA, Murphy M, Nandi D. Practice makesperfect? The role of simulation-based deliberate practice and script-based mentalrehearsal <strong>in</strong> the acquisition and ma<strong>in</strong>tenance of operative neurosurgical skills.<strong>Neurosurgery</strong>. 2013;72(suppl 1):124-130.S38 | VOLUME 73 | NUMBER 4 | OCTOBER 2013 SUPPLEMENTwww.neurosurgery-onl<strong>in</strong>e.com