Dr. Ayasakanta Rout - James Madison University

10/19/2012
Coefficient of Agreement in Pairwise Comparison-
Application in Hearing Aid Research
Ayasakanta Rout, Ph.D.
James Madison University
Lincoln Gray
Lori Campanola
Relevant financial disclosure: None
Relevant non-financial disclosure: None
of 50
2
of 50
A word about hearing aid
development lifecycle
A word about hearing aid
development lifecycle
new DNR
What we
already know
•Basic compression
•Gain
•Physical components
•etc.
new DFS
Which New ones work
developments (α-testing)
new mic array
new style
What we
already know
Which New ones work
developments (α-testing)
Develop a hearing
aid Beta-testing with those
features
3 of 50
4
of 50
A word about hearing aid
development lifecycle
A word about hearing aid
development lifecycle
What we
already know
Which New ones work
developments (α-testing)
Develop a hearing
aid Field with tests those
features
Marketing
• In the absence of compelling evidence, and
due to time pressure, it is not uncommon to
launch a ‘novel’ feature in a hearing aid
supported by ‘best guess’.
• Unlike pharmaceutical research, there are not
any potentially hazardous side effects of
hearing aid signal processing.
Typically 3 - 4 years and millions of $$
5 of 50
6
of 50
1

frequency
10/19/2012
Multichannel noise reduction
• The incoming signal is split into multiple
frequency channels
• Signal to noise ratio in each channel is
estimated
– Frequency and depth of modulation
– Overall amplitude of the signal across all channels
– Co-modulation
– Harmonic structure
• Based on the analysis, gain reduction is
Advantages of multichannel noise
reduction
• Reduces gain only in channels where
predominantly noise is present.
– Listening Comfort
• Speech Intelligibility
– Not enough evidence
applied within individual channels 7 of 50
8
of 50
Several approaches are available
1. Temporal modulation based gain reduction
2. Modified Weiner filtering
3. Other forms of spectral subtraction
Temporal waveform
of speech and noise
Speech
Random noise
9 of 50
10
of 50
Harmonic structure of speech
How can we measure the level of
noise (gain) reduction
Creating appropriate test stimuli
time
11 of 50
12 of 50
2

Amplitude
10/19/2012
Spectrogram of ICRA noise
International Collegium of Rehabilitative Audiology
ICRA noise with a two-octave notch
centered around 2000 Hz
13 of 50
14
of 50
ICRA noise with the embedded
steady state noise (two octave)
ICRA noise with the embedded
steady state noise ( 1 / 3 octave)
15 of 50
16
of 50
ICRA noise with the embedded
steady state noise (one octave)
3D graphical representation
Frequency
17 of 50
18
of 50
3

Gain Reduction (dB)
Gain Reduction (dB)
Gain Reduction (dB)
10/19/2012
Methods
Methods
• Stimuli
• Digital BTEs with multichannel noise reduction
from five major manufacturers
• Stimulus duration: 2 minutes each
• Overall RMS amplitude equalized
• Total 20 stimuli
• Programmed for flat 60 dB HL
• Directionality, feedback suppression, and
manual volume control (where available)
disabled.
19 of 50
20
of 50
Methods
Manufacturer A
65 dB SPL
KEMAR
with
custom
earmold
ER-11
with
½ inch mic
5
0
-5
1/3-Octave 1-Octave 2-Octave
Computer
-10
-15
-20
-25
250 500 1000 2000 3000 4000
Location of Band Rejection Notch (Hz)
21 of 50
22
of 50
Manufacturer B
Manufacturer C
5
1/3-Octave 1-Octave 2-Octave
1/3-Octave 1-Octave 2-Octave
0
5
-5
-10
-15
0
-5
-10
-15
-20
-20
-25
250 500 1000 2000 3000 4000
Location of Band Rejection Notch (Hz)
-25
250 500 1000 2000 3000 4000
Location of Band Rejection Notch (Hz)
23 of 50
24
of 50
4

Gain Reduction (dB)
Gain Reduction (dB)
10/19/2012
Manufacturer D
Manufacturer E
1/3-Octave 1-Octave 2-Octave
1/3-Octave 1-Octave 2-Octave
5
250 500 1000 2000 3000 4000
10
5
0
0
-5
-5
-10
-15
-10
-15
-20
-20
-25
Location of Band Rejection Notch (Hz)
-25
250 500 1000 2000 3000 4000
Location of Band Rejection Notch (Hz)
25 of 50
26
of 50
Ranking based on gain reduction
D A B C E
Part II: Paired Comparison of Sound
Quality with Noise Reduction
Better
Worse
27 of 50
28 of 50
• Are there any differences in sound qualities of
noisy speech processed by these hearing aids
– Paired comparison of Noisiness
5 sentences
from HINT
@
+5 dB SNR
Manufacturer A
Manufacturer B
Manufacturer C
Manufacturer D
Manufacturer E
5
2
= 10 pairs
Methods
• Recording HINT sentences
– Five sentences presented at 65 dB SPL in sound
field at +5 dB SNR
– Output of hearing aids recorded on computer
• Paired comparison
– Every possible pairing of the five hearing aids
presented
29 of 50
30
of 50
5

10/19/2012
Subjects
• Young, normal hearing subjects (air
conduction thresholds at or better than 20 dB
HL for 250-8000 Hz) and normal
tympanograms
• Age 18-30 yrs
• N = 20 (12 M, 8 F)
Paired Comparison of Noisiness Results
• Hearing aids ranked in terms of less noisiness
1. Manufacturer D
2. Manufacturer A
3. Manufacturer E
4. Manufacturer C
5. Manufacturer B
31 of 50
32
of 50
D
A
E
C
B
What about the internal consistency
within and across listeners
Kendall and Smith (1939)
Kendall and Gibson (1990)
33 of 50
34 of 50
Circular Triads
1 = Column chosen as “less noisy”
0 = Row chosen as “less noisy”
Hearing Aid
A
>
Hearing Aid
B
A
A B C D E
Hearing Aid
B
Hearing Aid
C
>
>
Hearing Aid
C
Hearing Aid
A
Inconsistent
Response
B 0
C 0 1
D 0 1 1
E 0 1 1 1 _______
35 of 50
36
of 50
6

10/19/2012
Coefficient of Agreement for the Reduction
of Background Noise
COA2 Analysis for Reduction of
Background Noise Condition
Number of
Z Value
Circular Triples
Sentence 1 10
13.8
Sentence 2 5
13.5
Sentence 3 0
14
Sentence 4 6
13.6
Sentence 5 9
13
Subject Test-Retest Reliability for the
Reduction of Background Noise Condition
N = 6
Test-Retest Reliability of Six Subjects for the Less Background Noise Condition
Pearson Correlation Significance in Two-Tailed T-Test
Sentence 1
Sentence 2
Sentence 3
Sentence 4
0.975
0.989
0.931
0.98
0.005
0.001
0.022
0.003
Sentence 5
0.971
0.006
37 of 50
38
of 50
What About
Speech Intelligibility
• Are there any differences in sound
qualities of noisy speech processed by
these hearing aids
– Paired comparison of Speech Intelligibility
39 of 50
40
of 50
Methods
• Same as “noisiness condition”
• N=11 (6 M, 5 F)
Paired Comparison of Speech Intelligibility
• Hearing aids were ranked in terms of being
“more intelligible” as
1. Manufacturer C
2. Manufacturer A
3. Manufacturer E
4. Manufacturer B
5. Manufacturer D
41 of 50
42
of 50
7

10/19/2012
C
A
E
B
D
Coefficient of Agreement for the Speech
Intelligibility
COA2 Analysis for the
Speech Intelligibility Condition
Number of
Z Value
Sentence 1
Circular Triples
1
6
Sentence 2 2
5.8
Sentence 3 2
6.5
Sentence 4 3
5.4
Sentence 5 5
5.7
43 of 50
44
of 50
Subject Test-Retest Reliability for the
Speech Intelligibility Condition
Ranking Summary
Test-Retest Reliability of Six Subjects for the More Intelligible Condition
Pearson Correlation Significance in Two-Tailed T-Test
Sentence 1 0.951
0.013
Sentence 2 0.996 0
Sentence 3 0.989 0.001
Sentence 4 0.952 0.012
Sentence 5 0.928 0.023
Average gain
reduction
Best
Worst
D A B C E
Less noisy D A E C B
Speech
intelligibility
C A E B D
45 of 50
46
of 50
Conclusion
• Anticipated that hearing aids that best enhance
SNR would be judged as having more speech
intelligibility
– not supported by our data
• Few circular triads and good test-retest
reliability
• Hearing Aid D preformed best in objective
measurements but worse for speech
intelligibility
Conclusion
• Hearing Aid A preformed well across all
categories
• Hearing Aid C and E did not provide much gain
reduction for objective measurements but
were higher ranked regarding speech
intelligibility
• Hearing aid B provided gain reduction only for
low frequencies and ranked low for noisiness
and speech intelligibility
47 of 50
48
of 50
8

10/19/2012
Is there a correlation between the hearing aids
that were judged to have less background noise
and the hearing aids that were judged to have
more speech intelligibility
That’s all folks
routax@jmu.edu
Correlation Between Hearing Aids Being Judged as Having Less Background Noise
and Being Judged as Having More Intelligibility for Eleven Subjects
Pearson Correlation Significance in Two-Tailed T-Test
Sentence 1 -0.256 0.677
Sentence 2 -0.272 0.658
Sentence 3 -0.258 0.675
Sentence 4 -0.061 0.923
Sentence 5 -0.088 0.888
49 of 50
50
of 50
9