Behavioural Surveillance Surveys - The Wisdom of Whores

Finally, the discussion of sample size
requirements below does not distinguish
between surveys to be undertaken using
probability sampling methods and those using
non-probability methods. It bears repeating
that conventional tests of statistical significance
and other statistical methods should not,
strictly speaking, be used in the analysis of
non-probability surveys. However, given that
the use of non-probability sampling methods
may be unavoidable in some settings, as a
practical matter the best that can be done in
such situations is to take steps to minimize
bias in the survey data. This implies assuming
that the data have distributional characteristics
similar to data gathered using probability
sampling methods. It may be possible to
ascertain how likely this is to be true by
examining basic socio-demographic variables
collected from respondents to non-probability
samples for any marked deviations from
expected distributions.
The formula for calculating sample sizes
The sample size required per survey round
for the measurement of change on a given
indicator is a function of five factors:
• the initial or starting level of the indicator;
• the magnitude of change you want to be
able to detect reliably;
• how sure you want to be that a change of
that magnitude would not have occurred by
chance (that is, the level of significance);
• how sure you want to be that you will
observe a change of that magnitude if it did
in fact occur (that is, the power);
• the percent of the population of interest
that - is eligible to be considered for the
indicator 1
An expression for the required sample size
for a given sub-population for each survey
round (n) is given by:
[Z 1-α
√2P (1-P)+Z 1-β
√P 1
1-P 1
) + P 2
(1-P 2
)] 2
n = D
(P 2
-P 1
) 2
Where:
D = design effect (see page 53);
P 1
P 2
= the estimated proportion at the time of
the first survey;
= the target proportion at some future
date, so that (P2 - P1) is the magnitude
of change you want to be able to
detect;
P = (P 1
+ P 2
)/2;
Z 1-α
Z 1-β
= the z-score corresponding to desired
level of significance
= the z-score corresponding to the
desired level of power
Standard values of Z 1-α
and Z 1-β
are provided
in Table 5, and the use of the above formula is
illustrated in Figure 4.
1
While it may appear that this fifth factor is similar
to the first, it is not so. Take one common indicator: the
proportion of respondents with non-regular partners in
the last year who used a condom with their last nonregular
partner. The starting level of the indicator may be
quite high. But the proportion of all respondents who had
non-regular partners in the last year might be quite low.
Both of these factors must be taken into account in
calculating the sample size necessary to measure a
significant change in the indicator.
B EHAV I OR A L S U R V EI L L A NC E SURV EY S CHAPTER 4
47