Behavioural Surveillance Surveys - The Wisdom of Whores

Should one- or two-tailed z-score values be used?
In the examples in Figure 4, we are
interested in detecting changes in one direction,
an increase in the proportion of sex workers
who always use a condom, and thus used a
value of Z 1-α
for a one-tailed test with a 95%
confidence (1.645). This will result in a
smaller sample size than if the corresponding
value for a two-tailed test had been used.
If we were interested in being able simultaneously
to detect a change of the same
magnitude in either direction, i.e. either an
increase or a decrease, we would use a
two-tailed test with a 90% confidence
level, with a value of 1.96 instead of 1.645.
In the latter case, the resulting required
sample size for each survey round would be
somewhat larger.
As a general rule, the prudent course of
action is to use two-tailed values of Z 1-α
/2.
However, BSS is often undertaken in the
context of prevention efforts which are
deliberately aiming to produce a change in
a given direction. In this case, it is reasonable
to use one-tailed tests.
The power of a study
In the context of BSS, power is shorthand
for the probability that a study will detect a
change in behavior of a specified magnitude,
if such a change did in fact occur. There is
no point carrying out a survey that does not
have the power to detect the changes you
aim to measure. To illustrate, suppose we
wanted to be able to measure a change of
10 percentage points in the proportion of sex
workers who always use a condom with their
clients. We compare two pairs of hypothetical
surveys taken 2 years apart: one with a sample
size of n=500 in each survey round and the
other with a sample size of n=200 per survey
round. While both surveys might indicate the
expected increase of 10 percentage points,
the increase may well not be statistically
significant for the survey with 200 respondents
per round. Thus, we would be forced to
conclude that there was no significant change
in this behavior over the study period , when
in fact there was a real increase that was not
detectable with a sample size of n=200 per
survey round. To ensure sufficient power, a
minimum value of Z 1-β
of .80 should be used.
This means you can be 80 percent sure that if
a change has occurred between survey
rounds, your study will pick it up. Where
resources permit, .90 is better yet.
The level of significance
In describing the results of a study, and
particularly the measurement of changes over
time, the phrase “statistically significant”
is frequently used. If a measured increase
in condom use over time is deemed to be
statistically significant, it means that surveillance
officials are confident that the observed
change could not have occurred by chance,
because of random differences in the characteristics
of respondents selected for the survey.
Survey designers must choose a level at which
they wish to be confident that observed
differences are not due to chance. Traditionally,
this level is set at 95 percent. In other
words, a statistically significant result is one
where investigators are 95 percent sure that
the observed change in behavior would
not have happened by chance. Measures
of significance are sometimes expressed as
p-values. A p-value is the inverse of the
level of significance. It indicates the probability
that the observed could have happened by
chance. A p-value of 0.05 means there is a
five percent chance that the observed change
could have occurred by chance. In other
words, it corresponds to a 95 percent level
of significance.
54
C H A PTER 4 B EHAV I OR A L S U R V EI L L A NC E S U R V EY S