SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ...

A Guide to Small Area Estimation - Version 1.1 05/05/2006
Figure 4.3 : Trade-off between Quality and Cost / Time / Effort
Statistical expertise
Robustness of results
Understanding results
Interpretability
Issues
Validity of assumptions
User requirements
Availability of resources
Timeliness/deadlines
Quality
Simple
models
Complex
models
Level of
precision
Finer
disaggregation
Good auxiliary data
Cost/time/effort
As you can see from Figure 4.3, good quality auxiliary data is a crucial prerequisite for
obtaining quality small area estimates. Quality is of course a relative term and depends
very much upon the clients’ decision making requirements.
Assuming that we have good quality auxiliary data, we would expect more sophisticated
methods to provide results of a higher level of quality, as indicated by the upward slope
of the cost-quality curve. The same curve also indicates that somewhere in the
continuum there exists an optimal point (a level of precision) whereby any additional
effort/cost/time from that point on, has either marginal or declining effects on quality.
More elaborate techniques may give only marginal improvements in accuracy but
decrease timeliness, an important dimension of quality. Overall quality may also be
eroded when exceedingly smaller areas or finer disaggregations of the data are
demanded. For example, in the disability analysis, disaggregating disability by type of
impairment, level of severity and age group, in addition to the small area level, leads to
poor quality estimates, especially for the rarest impairment types such as sensory.
There are also other issues, as shown just above the cost-curve in Figure 4.3, that may
have significant bearing in relation to quality and cost of small area estimates. For
example, the use of more complex models may require a higher level of subject matter
knowledge and expertise to assist in understanding and interpreting model results.
Such knowledge is also important for testing the validity of assumptions inherent in the
model and in checking the robustness and sensitivity of model results.
Finally, there are important points that have to be made in relation to the quality versus
cost issue discussed above. Firstly, that simplicity is an important aspect of quality in
that it aids the interpretability of small area output. We do not intend to imply from
Figure 4.3 that simpler methods always imply poor quality estimates. More complex
methods should only be attempted where there are likely to be demonstrable gains in
Australian Bureau of Statistics 34