SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ...

A Guide to Small Area Estimation - Version 1.1 05/05/2006
fine geographic and level of detail as compared to psychological disability (which is
around 1%). This choice also depends on the quality of data which is discussed in the
next section.
D) What are the consequences for users’ decision making outcomes if the
small area data is incorrect, say, by 5%, 10%, 20%, etc? Which small
area estimates have the greatest priority in terms of accuracy
requirements?
The answer to this question will drive the level of quality and hence resources required
to produce small area estimates of acceptable quality to users. If large funds from a
government program are to be allocated to regions based on the small area estimates,
then a high level of quality assurance and validation is required. However, if all that is
required is an approximate guide to indicate areas where there may be unmet need, say
for program evaluation purposes, then broad quality checks may be adequate.
To assess how accurate small area estimates need to be before they start to adversely
impact upon decision making outcomes, it is important to understand the entire
decision making process and the way in which small area estimates feeding into that
process impact upon the outputs. This involves understanding the assumptions implicit
in the process. The analyst needs to work out, in consultation with users, how accurate
final decision making outcomes need to be. By working backwards, it may be possible
to work out what level of accuracy in the small areas estimates will give this level of
accuracy in the decision making outcomes. A sensitivity analysis is another approach
that can also be undertaken to determine how sensitive final decisions are to changes in
the small area estimates.
Zaslavsky and Schirm (2002) discuss, in the context of funding allocations, how
interactions between the provisions of the funding formula, data sources and estimation
procedures used to derive formula inputs can have unanticipated consequences that are
inconsistent with the policy goals of a program.
E) Are there any conceptual models, either social or economic, that are
believed to describe the process which influences the variable(s) for
which we are to calculate small area estimates
This is a great opportunity to get expert advice on what variables should have a
relationship with the population of interest. This will give a theoretical base to look at
certain variables which can then be confirmed by statistical analysis. A widely accepted
theoretical model or framework, published in the literature and/or supported by
empirical investigations can greatly assist in deciding which variables, interaction terms
and contextual effects should be included in the small area model or in validating the
predicted estimates. Should you decide to include other variables not included in the
framework or exclude variables that are included, you are aware of the potential need to
justify the decision.
Australian Bureau of Statistics 12