SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ...

More documents

Recommendations

Info

A Guide to Small Area Estimation - Version 1.1 05/05/2006 fine geographic and level of detail as compared to psychological disability (which is around 1%). This choice also depends on the quality of data which is discussed in the next section. D) What are the consequences for users’ decision making outcomes if the small area data is incorrect, say, by 5%, 10%, 20%, etc? Which small area estimates have the greatest priority in terms of accuracy requirements? The answer to this question will drive the level of quality and hence resources required to produce small area estimates of acceptable quality to users. If large funds from a government program are to be allocated to regions based on the small area estimates, then a high level of quality assurance and validation is required. However, if all that is required is an approximate guide to indicate areas where there may be unmet need, say for program evaluation purposes, then broad quality checks may be adequate. To assess how accurate small area estimates need to be before they start to adversely impact upon decision making outcomes, it is important to understand the entire decision making process and the way in which small area estimates feeding into that process impact upon the outputs. This involves understanding the assumptions implicit in the process. The analyst needs to work out, in consultation with users, how accurate final decision making outcomes need to be. By working backwards, it may be possible to work out what level of accuracy in the small areas estimates will give this level of accuracy in the decision making outcomes. A sensitivity analysis is another approach that can also be undertaken to determine how sensitive final decisions are to changes in the small area estimates. Zaslavsky and Schirm (2002) discuss, in the context of funding allocations, how interactions between the provisions of the funding formula, data sources and estimation procedures used to derive formula inputs can have unanticipated consequences that are inconsistent with the policy goals of a program. E) Are there any conceptual models, either social or economic, that are believed to describe the process which influences the variable(s) for which we are to calculate small area estimates This is a great opportunity to get expert advice on what variables should have a relationship with the population of interest. This will give a theoretical base to look at certain variables which can then be confirmed by statistical analysis. A widely accepted theoretical model or framework, published in the literature and/or supported by empirical investigations can greatly assist in deciding which variables, interaction terms and contextual effects should be included in the small area model or in validating the predicted estimates. Should you decide to include other variables not included in the framework or exclude variables that are included, you are aware of the potential need to justify the decision. Australian Bureau of Statistics 12
A Guide to Small Area Estimation - Version 1.1 05/05/2006 F) What administrative data is available and relevant as auxiliary information to support the modeling of the small area estimates? How is this data collected, for what purpose is it used, and how accurate is it likely to be? It is important to cast the net wide in considering all potential sources of auxiliary data that may help improve the goodness of fit and specification of the small area model. The importance of understanding differences between auxiliary data and the survey data cannot be overstated. Administrative datasets may not reflect the entire population of interest or be as reliable as it is captured during some other process (ie. tax collection). A careful assessment should be made of the differences in: o o o o o o o concepts, data item definitions, (standard) classifications used scope mode of data collection reference periods editing procedures across all the data sources in order to at least understand the limitations of the small area model. G) Will small area estimates be required to be disaggregated by other categories? Users often request a whole range of small area data at different levels that may actually be superfluous to their needs. Here it is useful to find out what is the minimum level of data and geographic detail required to meet their needs. Prioritise any further breakdowns either at the geographic or sub-population level so during the modeling time is best spent on the essential models. H) What previous studies have been used, if any, by the clients to undertake the policy/funding decision for which small area estimates are required? This allows the project to compare results to current or previous studies, which will give a good outline if it is consistent with other research. It also allows research into what problems have come up in the past. Australian Bureau of Statistics 13
Page 1 and 2: A Guide to Small Area Estimation -
Page 11: A Guide to Small Area Estimation -
Page 35: A Guide to Small Area Estimation -

SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ...

Create successful ePaper yourself

Delete template?

Save as template?