Systematic review, meta-analysis and economic modelling of ...

Systematic review, meta-analysis and economic modelling of ... Systematic review, meta-analysis and economic modelling of ...

10.07.2015 Views

Discussionchanges in sensitivity and specificity resulting from adding another biomarker at presentation are similar tothe changes resulting from using a high-sensitivity troponin assay with a low threshold for positivity. If oneassay can provide the same result as a combination, then it is likely to be more cost-effective.We also used our economic model to produce estimates of 1-year rates of death and non-fatal MI among(1) all patients presenting with suspected ACS and (2) those discharged after negative assessment, for themain strategies tested. These estimates show how using more sensitive strategies decreases the expectedrisks of adverse outcome and could be used by clinicians attempting to weigh up the risks and benefits ofdifferent strategies for the individual patient. They could also be used, given a sufficiently interested andinformed patient, to explain the potential risks and benefits of different strategies to the individual patient,potentially allowing them to participate in shared decision-making.Cost-effectiveness of biomarkers, computed tomographic coronary angiographyand exercise electrocardiography in troponin-negative patientsWe developed a second (prognostic) decision-analysis model to evaluate the cost-effectiveness of usinga biomarker (H-FABP), exercise ECG or CTCA to select troponin-negative patients for further investigationwith ICA if positive or current standard care if negative. These strategies were compared with currentstandard care for all and ICA for all. We assumed that current standard care involved further investigationaccording to NICE guidance for stable chest pain 11 if symptoms persisted or recurred. The benefit ofinvestigation clearly depended on the subsequent risk of death and non-fatal MI, and we had two sourcesfor this with contrasting estimates and implicit assumptions. Data from an observational study of patientsadmitted to hospital with suspected ACS 155 produced an estimate of 1.0% for death and 3.9% for MI upto 1 year, whereas data from a randomised trial of ED chest pain assessment 12 produced correspondingestimates of 0.19% and 0.24%. The difference in these estimates reflects patient selection and duration offollow-up. In using either data source in the model we make an implicit assumption about the duration ofeffect of initial testing. Using the Mills data 155 we assumed that initial testing influences outcomes up to1 year, whereas the RATPAC data 12 assumes that initial testing only influences outcomes up to 3 months.There obviously is a limit to the effect of initial testing compared with current standard care as standardcare involves subsequent investigation if symptoms recur or persist. However, it is not clear when thislimit is.The analysis showed that the estimate of the adverse event rate and associated implicit assumptionregarding the duration of potential effect of initial testing on outcome were important in determiningcost-effectiveness. If the higher estimates of adverse outcome and 1-year duration of effect were used,then CTCA was likely to be the optimal strategy at the NICE threshold for willingness to pay. If the lowerestimates of adverse outcome and 3-month duration of effect were used, then the no-testing strategywas likely to be optimal. A threshold analysis suggested that CTCA was likely to be cost-effective if theestimated combined risk of death and non-fatal MI within the duration of effect of initial testing were> 2% or 3%, depending on the threshold used for willingness to pay (£20,000 or £30,000/QALY). It isimportant to note that this analysis was driven by the effectiveness of the strategies rather than costs, andoutcomes associated with a high rate of referral for ICA were little better than no testing. This emphasisesthe importance of specificity in prognostic testing and the need to ensure that diagnostic thresholds areset and tests interpreted in a way that does not result in a large number of FP cases being referred for ICA.The value of information analysis associated with this model showed that around the NICE threshold,assumed to be between £20,000/QALY and £30,000/QALY, the EVPPIs associated with the baselinerisk of MI and the relative reduction in risk with treatment were relatively high, suggesting that furtherexperimental research of these parameters will potentially be cost-effective. Research estimating theeffect of treatment on patients identified as being at increased risk by CTCA is unlikely to be consideredethical, but research comparing a strategy of liberal compared with restrictive CTCA use (with treatmentbeing consequent on CTCA findings) would be more likely to be considered ethical and would provide anestimate of the effect of treatment.112NIHR Journals Library

DOI: 10.3310/hta17010 Health Technology Assessment 2013 Vol. 17 No. 1Strengths and limitations of the studySystematic review and meta-analysisThe systematic review and meta-analysis was undertaken in accordance with the guidelines publishedby the Centre for Reviews and Dissemination for undertaking systematic reviews 26 and the CochraneDiagnostic Test Accuracy Working Group on the meta-analysis of diagnostic tests. 27 Our literature searchwas extensive and retrieved a large number of studies. We deliberately developed selection criteria thatwould limit the review to high-quality studies using a relevant and well-recognised reference standard.This involved excluding studies that used the old World Health Organization definition of MI as areference standard and studies that used a composite outcome of ACS instead of MI alone (or did notreport MI alone) as their reference standard. This had the advantage of ensuring a reasonable degree ofhomogeneity among the reference standard tests and excluded studies that risked having a referencestandard (ACS) that included subjective clinical judgements and possibly elements of the index test.However, this approach could be criticised because it potentially excludes studies of important outcomes,such as unstable angina, that are not included in the reference standard.Our meta-analysis did not include direct comparison of different biomarkers or assays (i.e. comparingdifferent biomarkers or assays in the same cohorts). Our estimates of the diagnostic accuracy of differentbiomarkers or assays are therefore indirect (i.e. based on testing in different cohorts), so differences inaccuracy may be explained by differences in cohort characteristics rather than the biomarker or assayperformance. We did not undertake direct comparisons because, as Table 2 shows, most studies onlyanalysed one or two biomarkers. Where multiple biomarkers were analysed in the same cohort there waslittle consistency between studies in terms of the biomarkers or assays tested.Although we used a reasonably well-defined reference standard for the diagnostic biomarker review, therewas still substantial variation in the tests used to confirm the reference standard, particularly the troponinassay used, threshold for positivity and the timing of sampling. Alongside variation in study populationsand variation in index test assays, thresholds and timing, this probably explains the heterogeneity observedbetween the results of different studies.We were unable to be as selective when defining the reference standard, or outcome, for the prognosticstudies. There was substantial variation in the definition of MACEs and the duration and intensity offollow-up. In particular, some studies included process measures, such as revascularisation, in theirdefinition of MACEs. If the decision to undertake a process is made by someone who is aware of theindex test results then process measures are subject to bias and estimates of prognostic outcome will beconsequently inflated. Given the limitations of the primary data it could be argued that summary estimatesgenerated by our meta-analysis are misleading. We undertook meta-analysis because we felt that itwould be helpful to have an overall estimate of prognostic value but urge caution in the interpretation ofthese estimates.Although our literature search retrieved a large number of studies, it was limited by substantial variationin the terms used to describe tests and outcomes. As a result we retrieved a proportion of studies throughexpert contact, reviewing citation lists and other serendipitous means, rather than through the plannedsearches. This was particularly the case for the review of exercise ECG, where a wide range of differentterms was used to classify studies that reported follow-up of cohorts of patients receiving exercise ECGafter presentation with suspected ACS. Consequently, it is possible that despite our best efforts we havemissed potentially eligible studies that could have contributed to the review.Economic evaluationThe economic analysis used current best practice to develop the model and followed recommendationsproduced by NICE. 175 We used Bayesian methods to synthesise the data from the meta-analysis andgenerate probability distributions associated with the diagnostic accuracy in the model that fully reflect© Queen’s Printer and Controller of HMSO 2013. This work was produced by Goodacre et al. under the terms of a commissioning contract issued by the Secretary of Statefor Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journalsprovided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should beaddressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton SciencePark, Southampton SO16 7NS, UK.113

Discussionchanges in sensitivity <strong>and</strong> specificity resulting from adding another biomarker at presentation are similar tothe changes resulting from using a high-sensitivity troponin assay with a low threshold for positivity. If oneassay can provide the same result as a combination, then it is likely to be more cost-effective.We also used our <strong>economic</strong> model to produce estimates <strong>of</strong> 1-year rates <strong>of</strong> death <strong>and</strong> non-fatal MI among(1) all patients presenting with suspected ACS <strong>and</strong> (2) those discharged after negative assessment, for themain strategies tested. These estimates show how using more sensitive strategies decreases the expectedrisks <strong>of</strong> adverse outcome <strong>and</strong> could be used by clinicians attempting to weigh up the risks <strong>and</strong> benefits <strong>of</strong>different strategies for the individual patient. They could also be used, given a sufficiently interested <strong>and</strong>informed patient, to explain the potential risks <strong>and</strong> benefits <strong>of</strong> different strategies to the individual patient,potentially allowing them to participate in shared decision-making.Cost-effectiveness <strong>of</strong> biomarkers, computed tomographic coronary angiography<strong>and</strong> exercise electrocardiography in troponin-negative patientsWe developed a second (prognostic) decision-<strong>analysis</strong> model to evaluate the cost-effectiveness <strong>of</strong> usinga biomarker (H-FABP), exercise ECG or CTCA to select troponin-negative patients for further investigationwith ICA if positive or current st<strong>and</strong>ard care if negative. These strategies were compared with currentst<strong>and</strong>ard care for all <strong>and</strong> ICA for all. We assumed that current st<strong>and</strong>ard care involved further investigationaccording to NICE guidance for stable chest pain 11 if symptoms persisted or recurred. The benefit <strong>of</strong>investigation clearly depended on the subsequent risk <strong>of</strong> death <strong>and</strong> non-fatal MI, <strong>and</strong> we had two sourcesfor this with contrasting estimates <strong>and</strong> implicit assumptions. Data from an observational study <strong>of</strong> patientsadmitted to hospital with suspected ACS 155 produced an estimate <strong>of</strong> 1.0% for death <strong>and</strong> 3.9% for MI upto 1 year, whereas data from a r<strong>and</strong>omised trial <strong>of</strong> ED chest pain assessment 12 produced correspondingestimates <strong>of</strong> 0.19% <strong>and</strong> 0.24%. The difference in these estimates reflects patient selection <strong>and</strong> duration <strong>of</strong>follow-up. In using either data source in the model we make an implicit assumption about the duration <strong>of</strong>effect <strong>of</strong> initial testing. Using the Mills data 155 we assumed that initial testing influences outcomes up to1 year, whereas the RATPAC data 12 assumes that initial testing only influences outcomes up to 3 months.There obviously is a limit to the effect <strong>of</strong> initial testing compared with current st<strong>and</strong>ard care as st<strong>and</strong>ardcare involves subsequent investigation if symptoms recur or persist. However, it is not clear when thislimit is.The <strong>analysis</strong> showed that the estimate <strong>of</strong> the adverse event rate <strong>and</strong> associated implicit assumptionregarding the duration <strong>of</strong> potential effect <strong>of</strong> initial testing on outcome were important in determiningcost-effectiveness. If the higher estimates <strong>of</strong> adverse outcome <strong>and</strong> 1-year duration <strong>of</strong> effect were used,then CTCA was likely to be the optimal strategy at the NICE threshold for willingness to pay. If the lowerestimates <strong>of</strong> adverse outcome <strong>and</strong> 3-month duration <strong>of</strong> effect were used, then the no-testing strategywas likely to be optimal. A threshold <strong>analysis</strong> suggested that CTCA was likely to be cost-effective if theestimated combined risk <strong>of</strong> death <strong>and</strong> non-fatal MI within the duration <strong>of</strong> effect <strong>of</strong> initial testing were> 2% or 3%, depending on the threshold used for willingness to pay (£20,000 or £30,000/QALY). It isimportant to note that this <strong>analysis</strong> was driven by the effectiveness <strong>of</strong> the strategies rather than costs, <strong>and</strong>outcomes associated with a high rate <strong>of</strong> referral for ICA were little better than no testing. This emphasisesthe importance <strong>of</strong> specificity in prognostic testing <strong>and</strong> the need to ensure that diagnostic thresholds areset <strong>and</strong> tests interpreted in a way that does not result in a large number <strong>of</strong> FP cases being referred for ICA.The value <strong>of</strong> information <strong>analysis</strong> associated with this model showed that around the NICE threshold,assumed to be between £20,000/QALY <strong>and</strong> £30,000/QALY, the EVPPIs associated with the baselinerisk <strong>of</strong> MI <strong>and</strong> the relative reduction in risk with treatment were relatively high, suggesting that furtherexperimental research <strong>of</strong> these parameters will potentially be cost-effective. Research estimating theeffect <strong>of</strong> treatment on patients identified as being at increased risk by CTCA is unlikely to be consideredethical, but research comparing a strategy <strong>of</strong> liberal compared with restrictive CTCA use (with treatmentbeing consequent on CTCA findings) would be more likely to be considered ethical <strong>and</strong> would provide anestimate <strong>of</strong> the effect <strong>of</strong> treatment.112NIHR Journals Library

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!