Evidence-based Design of Communication Development ... - Phonak
Evidence-based Design of Communication Development ... - Phonak
Evidence-based Design of Communication Development ... - Phonak
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>Evidence</strong>, Ethics & Program<br />
Quality<br />
Martyn Hyde, PhD<br />
Mount Sinai Hospital,<br />
University <strong>of</strong> Toronto, Ontario, Canada<br />
mhyde@mtsinai.on.ca<br />
1
<strong>Evidence</strong>-<strong>based</strong> practice:<br />
a totem <strong>of</strong> the third millenium<br />
• ‘…has acquired the kind <strong>of</strong> sanctity <strong>of</strong>ten<br />
accorded to motherhood, home and the flag…’<br />
Feinstein A R, Horwitz I<br />
Problems in the ‘evidence’ <strong>of</strong> ‘evidence-<strong>based</strong><br />
medicine’<br />
Am J Med 1997;103:529-35<br />
2
growth <strong>of</strong> 'evidence <strong>based</strong>'<br />
publications in Medline<br />
number <strong>of</strong> eb articles<br />
2500<br />
2000<br />
1500<br />
1000<br />
500<br />
0<br />
1993 1995 1997 1999 2001 2003<br />
year<br />
3
Hilarious <strong>Evidence</strong><br />
• In a 2004 talk on human rights and abortion,<br />
Hilary Clinton delivered the ultimate insult to<br />
the Bush administration:<br />
‘ WASHINGTON IS AN EVIDENCE-FREE ZONE’<br />
4
Benefits <strong>of</strong> EBP<br />
• Devaluation <strong>of</strong> ‘expert’ opinion<br />
Expert opinion is the lowest level <strong>of</strong> evidence!<br />
old way:<br />
This is how I do it... We’ve always done it this way…<br />
Dr X says this is how to do it…<br />
Our pet expert is better than your pet expert…<br />
new way:<br />
What’s the real question? Is there evidence?<br />
How good is it? Let’s do an evidence review…<br />
5
More benefits <strong>of</strong> EBP<br />
• Raised awareness <strong>of</strong> methodology issues<br />
More training in scientific methodology<br />
Critical appraisal as an essential skill<br />
<strong>Evidence</strong> review as a routine tool<br />
Higher publication standards, editorially and in<br />
primary reviews<br />
6
EBP limitations: evidence<br />
structures with sand foundations?<br />
• MOST primary reports have serious flaws<br />
USPSTF and UNHS<br />
A handful <strong>of</strong> key evidence reports<br />
CWGCH and communication development<br />
No substantive evidence <strong>of</strong> effectiveness<br />
Gravel 2 days ago: 6,534 > 318 > 27 papers<br />
7
‘The gem cannot be polished<br />
without friction, nor man perfected<br />
without trials’<br />
Chinese proverb<br />
• Assuming RCT is relevant, ethical, practicable<br />
• Generalizability issues:<br />
Selected subjects, expert process delivery.<br />
Focus: few actions, outcomes, confounders.<br />
Group validity only, findings may not apply to<br />
specific individuals<br />
8
EBP limitations: systematic<br />
reviews and meta-analyses<br />
• Systematic reviews:<br />
Clear question? Right question?<br />
Search quality<br />
Bias in key evidence filters<br />
Subjectivity in quality ratings<br />
From clinical opinion to methodologic opinion?<br />
• Meta-analyses:<br />
Apples, oranges and unspecified fruits<br />
Studies differ in quality and relevance<br />
9
Cannibalism in the EB community<br />
• Primary literature<br />
• Systematic reviews and meta-analyses<br />
• Guidelines for SRs & metas<br />
• SRs <strong>of</strong> SRs<br />
• Metas <strong>of</strong> metas<br />
• CPGs<br />
• Guidelines for CPGs<br />
• SRs <strong>of</strong> CPGs……..<br />
10
‘The pure and simple truth is<br />
rarely pure and never simple’<br />
Oscar Wilde<br />
• Q: What is the prevalence <strong>of</strong> permanent<br />
childhood hearing impairment?<br />
• A: Well, it depends on impairment severity,<br />
frequency, type, laterality, age at<br />
determination, method and skill <strong>of</strong><br />
measurement, racial mix, healthcare<br />
practices, socioeconomic variables…...<br />
11
PCHI prevalence overview<br />
1. Fortnum H, Audiol Med 2003<br />
2. Canadian WG on Childhood Hearing 2004<br />
3<br />
2.5<br />
cases per thousand<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
0 20 40 60 80 100 120<br />
minimum criterion average dB HL<br />
12
‘Everything should be made as<br />
simple as possible, but not simpler’<br />
Albert Einstein<br />
• There is a hierarchy <strong>of</strong> evidentiability<br />
• Dimensions:<br />
Outcomes:<br />
Provider actions:<br />
Recipient preferences:<br />
Confounding variables:<br />
#, level (metric, ICF), stability<br />
Provider-recipient interactions:<br />
Process duration:<br />
#, complexity, automation<br />
#, diversity<br />
#, strength<br />
#, strength<br />
event < series < continuum<br />
13
Hierarchy examples, for EHDI<br />
• Screening<br />
outcomes simple, few events, no preferences, little interaction,<br />
automated, modest skills…<br />
• Definitive audiometry<br />
outcomes complex, few events, no preferences, little<br />
interaction, some automation, high skills<br />
• Amplification<br />
outcomes complex, event series, more preferences, more<br />
interaction, some automation, high skills<br />
• <strong>Communication</strong> development<br />
outcomes very complex, action continuum, many preferences,<br />
strong interactions, little automation, high skills<br />
14
EBP for difficult problems<br />
• Clinicians vary in skills for complex problems<br />
• ‘Deconstruct’ intuitive/gestalt behaviour <strong>of</strong><br />
successful clinicians<br />
• Piecewise construct validation and modelling<br />
<strong>of</strong> the care process<br />
• Sophisticated, situation-specific info & clinical<br />
decision support algorithms<br />
• Train others, improve minimum standards<br />
15
<strong>Evidence</strong>, values and ethics<br />
• Ethical principles are <strong>based</strong> on societal values<br />
• Values govern outcome choices and weight<br />
• eg: ‘all children deserve equal access to care’<br />
places massive weight on ANY screening<br />
outcome, so the incremental benefit <strong>of</strong> UNHS<br />
vs targeted screening becomes massive and<br />
outweighs standard benefit/harm arguments<br />
• The entire EBP process is subordinate to ethics<br />
16
EBP, ethics and EHDI programs<br />
• Large programs must conform to EBP zeitgeist<br />
• Ethical principles govern macrostructure<br />
• EBP assists microprocesses<br />
• Programs MUST be evaluable<br />
• Needless practice variation compromises<br />
evaluability and equity <strong>of</strong> care<br />
• EBP defines standard <strong>of</strong> care, where applicable<br />
• Individualized care, where EBP not applicable<br />
17
The curious case <strong>of</strong> EHDI and<br />
the USPSTF…(et al).<br />
• UNHS ===> language status at >3 years<br />
• Systematic review <strong>of</strong> the wrong question?<br />
• The vision analogy...<br />
• Hearing impairment is the target disorder<br />
• Early improved hearing is a 1 ry outcome, NOT<br />
a proxy or surrogate for language status<br />
• Access to early, effective communication,<br />
…outcome or basic family right…?<br />
• The equity <strong>of</strong> care argument…<br />
18
Useful evaluation tools<br />
• Primary literature<br />
Omni.ac.uk, Trisha Greenhalgh<br />
• Systematic reviews<br />
upenn ebmr<br />
DARE, ACP Journal Club, Cochrane, Ovid EBM<br />
• CPGs/PPGs<br />
AGREE instrument cluzeau<br />
19
Having two ears and one tongue,<br />
we should listen twice as much as we speak…<br />
Turkish proverb<br />
20