AP Discrepancy Rates - College of American Pathologists

More documents

Recommendations

Info

Table 1. Definition of Discrepancy Discrepancy: A discrepancy has occurred if there is any difference between the original interpretation and the interpretation after the second review. Discrepancies were further classified by cause into one of the following categories: Change in margin status: The interpretation of the margin status was changed from benign to malignant or vice versa. Change in categoric interpretation: An interpretation was changed from one categoric diagnosis, such as benign, to another categoric diagnosis, such as malignant. For purposes of this study, interpretations were classified within categories that were graded by their probability of a malignant clinical outcome (eg, a benign diagnosis was assigned a 1; atypical, 2; suspicious, 3; and malignant, 4). We considered a difference of 2 or more steps between the original and the review interpretation as a discrepancy. For example, if the original diagnosis was benign, and the review diagnosis was malignant, the difference in steps between these 2 diagnoses was 4 1 3, and this case was considered discrepant. If the step difference between the original and review interpretations was 1, we decided that a discrepancy had not occurred. Change within the same category of interpretation: An interpretation was changed from one benign interpretation to another benign interpretation or from one malignant interpretation to another malignant interpretation. A change from one tumor type to another fell within this category. For example, if the original interpretation was adenocarcinoma and the review diagnosis was epithelioid sarcoma, this case was placed within this category of discrepancy. Change in patient information: There was a change in the organ site, such as the left ovary to the right ovary. Typographic error Table 2. Definitions of Effects of Discrepancies Effect of discrepancy on patient management: Discrepancies were classified into the following categories based on patient outcome: Harm (significant event): A discrepancy that resulted in patient harm (eg, inappropriate treatment, loss of life or limb, psychological event). The effect of the significant event on patient outcome was assessed using a 3-point Likert scale (1 severe effect, 2 moderate effect, 3 mild effect). The pathologists performing the review judged the significance of the event. Near miss: A discrepancy that was detected before harm occurred, such as a discrepancy that was detected at a clinical pathologic conference before treatment was initiated. No harm: A discrepancy that did not result in patient harm, such as a typographic error that had no bearing on patient management. MATERIALS AND METHODS Laboratories enrolled in the CAP’s volunteer Q-Probes quality improvement program participated in this study in 2003. The Q- Probes program and the format for data collection and handling have been previously described in detail. 21 Participants prospectively identified 100 consecutive surgical pathology or cytology specimens that were reviewed by a second pathologist after a first pathologist had signed out that case. In order to standardize the data-collection process across all participating laboratories, pertinent terms were defined (Tables 1 and 2). For each case, the participating laboratory recorded the specimen type (surgical pathology or cytology), organ or anatomic site (chosen from a specified list), primary reason for secondary review (chosen from a specified list), clarity of the report, presence or absence of a discrepancy (Table 1), effect of discrepancy on patient outcome (Table 2), and modification of report (if performed). We subclassified discrepancies into several main groups of causes that have been previously described in the pathology literature. 2 The detection of particular discrepancy subtypes depends partly on the method of secondary review, described in more detail later. For example, change in margin status is more likely to be detected by a frozen-permanent section review, and change in categoric interpretation is more likely to be detected by cytologic-histologic correlation review. We arbitrarily chose a disagreement of 2 steps as constituting a categoric interpretation discrepancy, although even a 1-step discrepancy is an error. Raab 2 reported that 2-step discrepancies have a much greater probability of clinical significance compared with 1-step discrepancies. In addition, because interobserver variability studies have shown that 1-step discrepancies are much more common, we did not want to collect data on a large subset of cases that had little effect on patient care. 2 For changes in categoric interpretation, the original and review diagnoses were recorded. The taxonomy of the effect of a pathology discrepancy on patient outcome was based on the medical patient safety literature, 23–26 which uses a taxonomy related to the effect of a failure of a planned action to be completed or the use of a wrong plan to achieve an aim. 1 Diagnostic error does not fit neatly into the category of an ‘‘action’’ error, and the resulting outcome of the patient is often difficult to assess. In particular, distinguishing between a no-harm and a near-miss event may be problematic. We defined a near-miss event as an error that was detected before harm occurred; an example is when a diagnostic error was picked up at a conference (or another means of secondary review) before a particular treatment protocol was initiated. In this case, if the error had not been detected, we assume that some degree of harm would have occurred. We recognize that harm (eg, psychologic) may still have occurred in this case, but we classified this event as a near miss, because the ability of the review pathologist to identify this type of harm was limited in this study. We defined a no-harm event as occurring when a diagnostic error occurred that would not cause patient harm even if the error had been undetected. An example of a no-harm event was a typographic error, such as writing the word ‘‘brest’’ instead of ‘‘breast’’ in the diagnostic line. We defined a significant event as an error resulting in patient harm. 1 We provided the review pathologist a 3-point Likert scale to grade the severity of this harm, recognizing that this grading was subjective. Grzybicki et al 27 showed that even with more well-defined criteria for patient harm, there is little agreement among pathologists as to the effect of an error on patient outcome; thus, although subclassifying harm needs further study, we wanted to measure the general view of the participant institutions in assessing harm. We did not require the pathologist to perform medical record review but only to assess the error severity based on the information available at the time of detection. In general, the pathologist had some degree of knowledge of the clinical outcome. This information varied depending on the method of detection; for example, more information may be available if a clinician requests secondary review compared to if a cytologic-histologic correlation is performed. Autopsy cases were the only anatomic pathology case type excluded from this study. Multiple specimens with a secondary review from the same case or specimens from different cases associated with the same patient were included in the study. Secondary review is the main method used to detect anatomic pathology errors. The recorded reasons why cases were reviewed were as follows: intradepartmental conference, request by clinician, interdepartmental conference, specified institutional quality 460 Arch Pathol Lab Med—Vol 129, April 2005 Patient Safety in Anatomic Pathology—Raab et al
assurance policy, and extradepartmental review. These methods encompass the majority of forms of secondary review, although participants also could choose the category of ‘‘other’’ if none of these methods applied. We chose to examine multiple methods, because single institutions often do not review a sufficient number of cases using a single method for statistical significance testing in the time frame provided by this Q-Probes study. We also wanted to determine if discrepancies were detected at different frequencies and had different clinical import depending on the method of detection. Institutional quality assurance policies include cytologic-histologic correlation, random review, and frozenpermanent section correlation. We recognize that these methods of review detect only a fraction of discrepancies (because in most institutions, the majority of cases do not undergo secondary review) and exclude those errors detected prior to sign-out. However, some of these forms of secondary review are the most likely means to detect clinically significant error, the subcategory of error that some authors believe is the most important to track and eradicate. The ‘‘clarity of the report’’ question referred to how understandable the original report was perceived to be from the standpoint of the review pathologist. The review pathologist determined the report clarity using a 4-point Likert scale (ie, clear, mildly unclear, moderately unclear, markedly unclear). A limitation in this assessment was that clinicians, who were the users of the diagnostic information, did not perform the analysis, and previous authors have indicated discrepancies between clinician and pathologist impression of report clarity. 28 We also recognize that the clarity assessment was subjective, because definitive criteria of clarity were not provided. The lack of clarity may be an example of error, particularly for reports assessed to be moderately or markedly unclear, because this lack of clarity could result in improper patient management. Laboratories could choose 3 options regarding computerized report modification after a discrepancy was detected: (1) the original report was deleted and replaced with a new report; (2) the original report was marked with a qualifier, and a new report was appended; or (3) the original report was not marked or modified, but a new report was appended. Institutional demographic and practice variable information was obtained by completion of a questionnaire. These data included annual specimen volume, number of institutional practicing pathologists, type and frequency of institutional review conferences (eg, breast, chest, or endocrine conference), and institutional quality assurance methods of secondary review. The discrepancy frequency (expressed as a percentage) was calculated for the aggregate data and for each institution. The discrepancy frequency was calculated as the number of discrepancies divided by the total number of cases reviewed, multiplied by 100. Discrepancies and report clarity assessments were subclassified by organ type (and further broken down by effect on patient outcome). The correlation of the main indicator variable (discrepancy frequency) was assessed for each of the predictor variables separately by using nonparametric Wilcoxon rank sum or Kruskal- Wallis tests. The 2 goodness-of-fit test was used to test for associations between the presence of a discrepancy and other caselevel variables. Statistically significant associations are defined at the P .05 level. The reason for case review was correlated with the effect of the discrepancy on patient outcome. Some participating institutions did not answer all of the questions on the demographics form or on the input forms. These institutions were excluded only from the tabulations and the analyses that required the missing data element. RESULTS A total of 74 institutions submitted data for this study. Most institutions (87.8%) were located in the United States, with the remainder located in Canada (n 7), Australia (1), and Saudi Arabia (1). Of the participating institutions, 31% were teaching hospitals, and 23% had a pathology residency program. The Joint Commission on Ac- Table 3. Percentile Distribution of Anatomic Pathology Discrepancy Rate All Institutional Percentiles N 10th 25th 50th (Median) 75th 90th Discrepancy rate, % 74 21.0 10.0 5.1 1.0 0.0 creditation of Healthcare Organizations inspected the majority of the institutions (68%) participating in this study, and the CAP inspected 82% of laboratories contributing data. The participants’ institutional sizes were as follows: 39.7% had fewer than 150 beds; 39.7%, 151 to 300 beds; 9.5%, 301 to 450 beds; 3.2%, 451 to 600 beds; and 7.9%, more than 600 beds. Private nonprofit institutions comprised 54.9% of the institutions; 11.1% were private forprofit institutions; 9.5% were state, county, or city hospitals; 6.3% were university hospitals; 3.2% were federal governmental institutions; and 15.9% were other. Fiftythree percent of the institutions were within a city, 28.1% were suburban, and 15.6% were rural. The annual mean number (and SD) of surgical pathology, nongynecologic cytology, and gynecologic specimens per institution were 16 241 (19 567), 1898 (2376), and 21 147 (44 903), respectively. The mean and median numbers of pathologists at each institution were 6 and 4, respectively. The 74 institutions performed secondary review of 6186 anatomic pathology specimens (5268 surgical pathology specimens and 847 cytology specimens), and each institution collected data on a range of from 2 to 100 specimens (median, 99 specimens). In aggregate, 415 discrepancies were reported, and the overall mean anatomic pathology discrepancy frequency was 6.7%. The overall mean surgical pathology discrepancy frequency was 6.8% (356 discrepancies of 5255 reviewed cases), and the overall mean cytology discrepancy frequency was 6.5% (55 discrepancies of 844 reviewed cases); neither of the specimen types had a higher discrepancy frequency (P .78). The distribution of the anatomic pathology discrepancy frequencies is listed in Table 3. Higher percentile ranks indicate lower discrepancy frequencies. The number of reviewed specimens, overall discrepancy frequency, and the discrepancy classification by organ type are shown in Table 4. The most common organ types reviewed in this study were female genital tract and breast. None of the organ types had a higher frequency of discrepancy than other organ types (P .15). For the organ types with higher volumes reviewed (200 cases), a change in categoric interpretation (ie, an error type more likely to be associated with harm) occurred more frequently in the female genital tract, male genital tract, and lymph node. A change in margin status occurred most frequently in breast specimens. In Table 5, the effect of the discrepancy on patient outcome and the report modification in response to a discrepancy are listed by organ type. Some form of harm was seen in the majority of organs, although specimen type (cytology or surgical pathology) or organ type did not correlate with effect on patient outcome (P .73 and P .83, respectively). Harm was observed in 20.8% (11 cases) of breast specimens and in 25.3% (21 cases) of female genital tract specimens in which a discrepancy was detected. Neither specimen type nor organ type correlated with the Arch Pathol Lab Med—Vol 129, April 2005 Patient Safety in Anatomic Pathology—Raab et al 461
Page 1: CAP Laboratory Improvement Programs
Page 5 and 6: Genital, female Breast Lung Genital
Page 7 and 8: ut this has not been thoroughly stu

AP Discrepancy Rates - College of American Pathologists

Create successful ePaper yourself

Delete template?

Save as template?