The Pros and Cons of Threshold Values in

IGM
Pros and cons of threshold values
in economic evaluation
Rolf Holle
GSF – Research Centre for Environment and Health
Institute of Health Economics and Health Care Management (IGM)
1

IGM
δ
The price of two-dimensionality
Clinical research in the
pre-economic era
0
( )
• natural threshold for ∆E
• arbitrary threshold for uncertainty
α=5%
• „non-inferiority margin“ (EHEA
Guideline, Stat. in Med., 2006)
∆ E
∆C
Cost-effectiviveness studies
• arbitrary threshold for ∆E/∆C
• arbitrary threshold for uncertainty
α=5%
λ
∆ E
2

IGM
Fixed threshold for an ambiguous ICER ?
• Choice of the comparator
• Lack of standardization in cost measurement
• Measurement of HrQoL
• Theoretical foundation of QALY concept
• Time horizon and discounting
3

IGM
Three worlds
IDEAL world
• strictly limited budget
• simultaneous decision between
several new and existing programmes
• full information about opportunity costs
• decision in order to maximize health gain
NICE world
• vaguely limited budget
• one-by-one evaluation of new programmes
• very little information about opportunity cost
• guidance instead of decision
UGLY world
• no direct budgetary context
• opportunity cost irrelevant
• more marketing than guidance
„Everybody has won, and
all must have prizes“
4

IGM
IDEAL world
Weinstein & Zeckhauser model (1973): theoretical foundation of λ as
a criterion for maximizing health under limited budget
Assumptions:
• Full information on cost and outcomes of all existing programmes
• Perfect divisibility → socially unacceptable (only restriction to
subgroup possible, if ICERs vary over subgroup)
• Constant return to scale → irrelevant, because no divisibility
• Stopping existing programmes without extra costs
In this case:
• Threshold will represent opportunity costs at the margin, and
therefore will depend on size of budget and vary over time
• Budget constraints will refer to yearly costs, not to discounted costs
over lifetime horizons
5

IGM
NICE world
• threshold justified by call for transparency
• hard vs. soft threshold
• soft thresholds have evolved and seem to be accepted
Reference case is
necessary to make the
threshold more meaningful
10.000 £/QALY
30.000 £/QALY
Rawlins & Culyer, BMJ 2004;329:224-227
6

IGM
How soft is the threshold ?
NICE Guide to Technology Appraisal, 2004,
6.2.6.10:
• „Below a most plausible ICER of 20,000£/QALY, judgements … are
primarily based on the CE estimate.
• Above … 20,000 £/QALY, judgements are more likely to make more
explicit reference to factors including:
• degree of uncertainty …
• innovative nature of the technology
• features of condition and population ….
• wider societal costs and benefits.”
6.2.6.11:
• “Above an ICER of 30,000£/QALY the case for supporting the
technology on these factors has to be increasingly strong.”
• Con: undermines transparency of decisions
• Pro: allows for common sense and multi-attributive decision making
7

IGM
UGLY world
• ICER < some λ as a legitimation
• irrespective of time horizon, perspective, included costs
• poor standards of sensitivity analysis
• several guidelines exist (e.g. ISPOR)
Thresholds lead to bias in CE-results
• published results closely below
perceived threshold (e.g. 19,915£/QALY)
• results from independent reassessment
• 0: negative ICER (SE)
• 1: up to 15,000
• 2: up to 30,000
• 3: more than 30,000
• 4: negative ICER (NW)
(Miners et al. BMJ 2005)
8

IGM
G & B criticism
Gafni & Birch (2006) repeatedly claimed that
1. „the threshold approach has led to decisions that resulted in increased
expenditures …
2. … without any evidence of increases in total health gains.”
„NICE does not take affordability into account
when making judgment about CE.“
(Rawlins &Culyer, 2004)
Dent & Sadler (2002): to put guidance into
practice “primary care trusts have to put
aside 1m£ per year”
→ concern whether this is done sensibly
by intuition or soft evidence
BMJ News , Sept. 16, 2006:
„NICE is to root out ineffective treatments“
Dent & Sadler, BMJ, 2002
9

IGM
Consequences on Pricing
• In early phases of drug development companies need
an orientation whether a new drug may have a
chance of reimbursement and at what price.
• For innovative drugs without a reference price defined
by competing products, λ may help to avoid an
unacceptably high price
• But: Drug companies aim at high pricing below λ
threshold
10

IGM
Accounting for Uncertainty
Is uncertainty only a concern for ICER estimates in the upper range ?
Example: Acupuncture in low back pain (Ratcliffe et al, BMJ, 2006)
Abstract:
• ICER = 4,241 £/QALY
• more than 90% chance of being cost effective at £20,000 cost per QALY
threshold
Paper:
• 95%-CI: 191 to 28,026 £/QALY
• QALY difference not even significant
• “If 20,000 is taken as the maximum acceptable CE ratio, then
acupuncture for the treatment of low back pain seems cost effective.”
If ICER > λ one may find a subgroup with ICER < λ , but uncertainty will
increase
11

IGM
Use of λ in other methods
Several statistical methods and concepts (CE-Acceptability curves, Net
benefit analysis, Sample size estimation, Value-of-information analysis)
depend upon the value of a QALY.
Often the upper threshold of NICE (i.e. 30,000£/QALY) is taken.
Results usually displayed graphically depending on the chosen value of λ.
This adds complexity which some decision makers may not be able to digest.
In NB regression all p-values are a function of the continuous value of λ,
therefore
• presenting (and understanding) results becomes even more demanding
• variable selection will become even tougher
12

IGM
Use of λ in net benefit approach
∆C
ΝΒ(λ) 1
λ
NB(λ) = λ * ∆E –∆C
Line of constant NB(λ)
Line of constant ICER
„non-unique mapping between ICER and NB“ (Löthgren & Zethraeus, 2000)
∆E
13

IGM
Summary
• Theoretical foundation of threshold seems unrealistic,
• and would result in different, context specific threshold values
• the IDEAL world does not exist
• NICE threshold and reference case are appreciated as orientation
• decision makers and NICE guidance use threshold more or less
carefully
• in the NICE world a soft threshold is helpful
• Existing thresholds may invite bias in analysis and influence pricing,
• and they obscure the underlying heterogeneity of ICER estimation
• be careful in the UGLY world
14

IGM
Further reading:
Gafni & Birch, Soc Sci Med (2006):
Incremental cost-effectiveness ratios (ICERs):
The silence of the lambda
(with >50 references)
λ λ
λ
λ
15