Drug Information & Critical Appraisal Workshop: - Canadian Society ...

Drug Information and Critical
Appraisal Workshop:
Finding our way around the maze of drug
and medical information. Part I.
Originally presented at William Osler Health System on Jan 21, 2009; updated Sept
12, 2010
Miranda So, B.Sc., BSc.Phm., Pharm.D., RPh.,
Clinical Pharmacy Specialist—Infectious Diseases,
William Osler Health System
1

Learning Objectives
• To gain an overall appreciation of commonly
available drug information resources in hospital
pharmacy practice
• To be familiar with the levels of evidence
described in medical literature
• To apply critical appraisal principles in evaluating
a drug study
2

Outline
Part 1
Part 2
• Drug information • Exercise to evaluate a
resources: The Good, drug study
The Bad, The Ugly
(Disclaimer: presenter’s
personal opinion only)
• Critical Appraisal: the
Quest for Knowledge
• Formulary Submission:
The Essentials
3

Common Drug Information (DI) Resources
Good
Bad
Ugly
Micromedex
Dosing; indications;
compatibility (Trissel’s);
interactions;
comparison with other
agents; patient
education
Evidence/clinical trials
not most up-to-date;
online only and no
hand-held
Can be difficult to
navigate or to interpret
(e.g. adverse reactions
in form of case reports);
not on portable device
Lexi-comp /e-Lexicomp
Dosing; indications;
compatibility;
interactions; FDA
warning; patient
education; druginteraction
analysis;
easy-to-read lay-out;
print; online and
handheld
Certain data may be
over-simplified
Individual subscription
on handheld device
$$$
CPS (and e-CPS)
Detailed product
monograph provided by
manufacturers (mfr) with
some exceptions;
Canadian warnings
highlighted
Detailed product
monograph provided by
mfr; difficult to interpret
Search by brand-name
(though mildly improved
with e-CPS which allows
generic & brand); nohandheld
4

Common DI Resources
Up-to-date MD Consult Ovid/MEDLINE
Good
Easy to find information on
pathophysiology, diseases;
short and snappy overview
of guidelines, patient
education; helpful for
practising pharmacists
Access to primary (original
research) and secondary
literature (review articles)
with little search terms
restrictions unlike Ovid; e-
books; patient education;
alternative to Ovid
MEDLINE
Systematic way to
search for information;
Boolean logic operators
and Medical Subject
Heading search terms
used
Bad
Not in-depth and can be
MD-oriented; clinical trials
cited may not be “hot off
the press” (depends on
when material was
reviewed)
Can be difficult to find
specific information
Can be difficult to know
what search terms to
use
Ugly
Silver-platter format does
not stimulate critical
thinking in learners (e.g.
pharmacy students)
Can be difficult to find
specific information
Can be labour-intensive
to look through the list,
but can be overcome
with assigning “limits” 5

Common DI Resources (cont’d)
Ottawa IV Manual King Guide Trissel’s
Good
Easy and concise
information on
administration,
dosing, compatibility;
Canadian content.
Text and CD formats;
updated annually
Text is updated
quarterly; simple to
read; may be
accessed online
through e-Lexi-comp
subscription
Easy to read; specific
topics such as
calcium/phosphate
concentration in TPN
Bad
Text in loose-leaf
format—individual
pages may be lost
Text subscription and
updates in loose-leaf
format; labour
intensive to maintain
(may be lost easily)
Institutions that only
have access to text
will be required to
purchase a new
edition every couple
of years
Ugly -- How to apply their information to the clinical
situation at hand (e.g. US TPN formulations vs.
formulations available in Canada or accessible
to our institution) 6

Other DI Resources
• Regional DI centres (require membership at
personal or institutional level) and their
communications (e.g. newsletters)
• Institution-specific information on “intranet”, e.g.
formulary, IV monographs
• Health Canada Advisories
• Institute of Safe Medication Practice (ISMP) and
ISMP Canada bulletins
• University libraries (if affiliated)
7

Critical Appraisal
• Why?
• To practise evidence-based medicine (EBM)
requires pharmacists to distinguish high quality
drug information from those of lesser quality
• To facilitate knowledge translation
• To provide unbiased drug information to other
healthcare professionals and patients
8

Critical Appraisal Tools
• Scoring or criteria: e.g. Jadad scale (e.g. used in
meta-analyses to determine whether a trial should be
included)
• Website example:
o
www.equator-network.org
• “Worksheet to evaluate drug studies” from
Pharmacist’s Letter
• JAMA’s Users’ Guide Series
• CONSORT statements for RCTs www.consortstatement.org
• PRISMA (formerly QUORUM) checklist for
meta-analyses of RCTs
• BMJ checklists
http://resources.bmj.com/bmj/reviewers/peer-reviewersguidance
9

Levels of evidence
In order of highest to lowest evidence
• Systematic review/ meta-analysis of
randomized controlled trial (RCT)*quality limited
by quality of original data
• Randomized controlled trial
• Cross-over study
• Cohort study
• Case-control study
• Analysis of database
• Case series
• Case reports
10

Specific study design:
• Systematic review vs. meta-analysis
o
Meta-analysis uses statistical techniques to integrate
the results of the included studies; increases sample
size and power to detect differences between groups
• Non-inferiority design
o
o
o
One treatment “not worse” than another by a prespecified
margin (Δ) chosen based on statistical
reasoning and clinical judgement
When evaluating, consider: choice of comparing
agent; size of Δ—”is it meaningful?”
Δ should be smaller than the lower 95% CI for the
absolute risk difference observed between standard
therapy and placebo in the relevant superiority trial
Pater C., Current Controlled Trials in Cardiovascular Medicine 2004, 5:8
Piaggio G et al. JAMA 2006;295: 1152
11

Level of evidence often described in
guidelines
Quality of Evidence:
• I. from ≥1 properly
designed RCT
• II. from well designed
controlled, nonrandomized
trials
• III. From well designed
cohort or case-cohort
studies
• IV. From multiple case
series
• V. Opinion of authorities
Quality of
Recommendations
• Good evidence to support
• Fair evidence to support
• Poor evidence, but other
grounds used to make
recommendations
• Fair evidence to exclude
• Good evidence to
exclude
12

Determining the quality of evidence
Underlying methodology
A. RCT
B. Downgraded RCT or upgraded observational
studies
C. Well-done observational studies
D. Case series or expert opinion
Dellinger et al. Crit Care Med. 2008; 36:296
Atkins et al for the GRADE Working Group. BMJ 2004;328:1490.
Guyatt et al. BMJ 2008;336:995.
13

Determining the quality of evidence
Factors that might decrease
strength of evidence
• Poor quality of planning &
implementation of available of
RCTs suggest high
likelihood of bias
• Inconsistency of results
• Indirectness of evidence
• Imprecision of results
• High likelihood of reporting/
publication bias
Main factors that may increase
strength of evidence
• Large magnitude of effect
[direct evidence; relative risk
(RR)>2 with no confounders]
• V. large magnitude of effect
(RR>5 with no threats to
validity)
• Dose-response gradient
Dellinger et al. Crit Care Med. 2008; 36:296
Atkins et al for the GRADE Working Group. BMJ 2004;328:1490.
Guyatt et al. BMJ 2008;336:995.
14

Evaluating Clinical Studies
• Validity: rooted in study design
• Control confounding factors that may bias estimate
of treatment effect
• RCT evidence (generally) stronger than that of
observational studies
• Rigorous observational studies stronger than uncontrolled
case series
• Common themes to consider:
• Difference between study groups at baseline
• Allocation concealment
• Blinding
• Losses to follow up
• Intention-to-treat analysis
• Stopping early for benefit
Atkins et al for the GRADE Working Group. BMJ
2004;328:1490.
• Failure to report outcomes Guyatt et al. BMJ 2008;336:995. 15

Evaluating Clinical Studies: Common themes to
consider
• Clinically significant vs. statistically significant
results
• Consistency of results (compared to similar
studies)
• A priori vs. post-hoc analyses
• Safety: benefit vs. harm (can be
overlooked...especially if trial stopped early for
benefit)
• Conclusion consistent with results
• Application to your patient
Atkins et al for the GRADE Working Group. BMJ 2004;328:1490.
Guyatt et al. BMJ 2008;336:995.
16

Formulary submission: the Essentials
• Evidence-based—apply your skills of critical
appraisal
• Acknowledge limitations of studies coming
from manufacturer’s formulary submission
package
• Find the strongest evidence to support your
submission
• Applicability of evidenceusage criteria
• PEcon analysisimpact on the patient, the
hospital, the systemresources:
http://resources.bmj.com/bmj/authors/checkli
sts-forms/health-economics
http://cadth.ca/index.php/en/home
17

Part 2 of Workshop
• To prepare for Part 2, please read this article:
Gardiner D et al. Safety and Efficacy of
Intravenous Tigecycline in Subjects with
Secondary Bacteremia: Pooled Results from 8
Phase III Clinical Trials. Clin Infect Dis.
2010;50:229-38.
Go through a critical appraisal exercise in Part
2 of the presentation
18