SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ...
SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ... SAE Manual Sections 1 to 4_1 (May 06).pdf - National Statistical ...
A Guide to Small Area Estimation - Version 1.1 05/05/2006 2. Assessing User Requirements 2.1 User Requirements Understanding user requirements for small area estimates is paramount for providing a high quality small area product that meets the client's decision making requirements. The importance of gaining a thorough understanding of user requirements at this initial phase cannot be over emphasised. Shortcuts taken at this phase will often lead to an inferior quality product and/or valuable time and resources lost along the way. With the right questions, users will be able to give a clear indication as to what information is critical in their decision making. Users are also a valuable resource in helping to determine the best potential sources for borrowing strength. Where complex techniques need to be applied, Methodology Division (MD) staff will need to be involved in performing the methodological work and it is highly recommended that MD staff are directly involved in the discussion with clients at the earliest possible opportunity. Table 2.1 below displays a checklist of the key questions to ask clients when commencing a small area exercise. Table 2.1 Checklist of Questions to Ask Users. Question A) What are the key policy making or program funding decisions that require small area data ? B) What are the organisation's strategic context, goals and desired outcomes, in which these decision making requirements are nested ? C) What small area data do users think would best meet their decision making requirements and what level of geography is required ? D) What are the consequences for users’ decision making outcomes if the small area data is incorrect, say, by 5%, 10%, 20%, etc? Which small area estimates have the greatest priority in terms of accuracy requirements ? E) Are there any conceptual models, either social or economic, that are believed to describe the process which influences the variable(s) for which we are to calculate small area estimates ? F) What administrative data is available and relevant as auxiliary information to support the modeling of the small area estimates? How is this data collected, for what purpose is it used, and how accurate is it likely to be ? G) Will small area estimates be required to be disaggregated by other categories ? H) What previous studies have been used, if any, to undertake the policy/funding decision for which small area estimates are required ? Australian Bureau of Statistics 10
A Guide to Small Area Estimation - Version 1.1 05/05/2006 A) What are the key policy making or program funding decisions that require small area data? Knowing how the small area data will be used as input to user’s decision making process is essential in ensuring the small area output meets user requirements. User decision making requirements can vary considerably. Some may be quite sophisticated and quantitatively based. Others may be quite informal and qualitatively based. In the former case, the decision making process should be identified and well understood as inherent assumptions may help determine just how accurate small area data really needs to be. It is also important to ensure, where possible, that the small area data is consistent and compatible with the users’ decision making process, and that the output of this process meets user expectations, not just the ABS small area output. A quality assessment should include measures of the fitness for purpose of small area output. However many users do not have sophisticated, quantitatively based decision making processes, and may have difficulty in articulating the very nature of the problem they wish to solve. Before undertaking the project it is worth investigating whether the small area estimates requested may suit the needs of a wider range of clients. Quite often similar data is required by different clients and can be useful for a wide range of users. By incorporating their needs into the project, this increases the value of the final product with minimal additional cost. B) What are the organisation's strategic context, goals and desired outcomes, in which these decision making requirements are nested? Need to ask users what the data problem is, why data needs to be obtained, the decision making processes used, what the users are trying to find out and why. This can be matched up with what is possible to estimate from the available data. Any possible limitations then can be identified early and additional information can be sought or the user can be made aware. When the final product is created the user has a good understanding of the limitations and the product is a close as is possible to what they need. C) What small area data do users think would best meet their decision making requirements and what level of geography is required? A minimum level of information on the variable of interest is needed in each small area. Given the available data, the user needs to be aware that a given level of the quality for the small area estimates is subject to a trade-off between the level of what geographic level and level of detail in the data is possible to model. That is, in the context of household based collections, a reasonably common characteristic of the variable of interest (say, greater than 10%) may be estimated at a reasonably fine level of geography such as Statistical Local Area (SLA). However, a variable of interest representing less than 1% of the population, can only be reliably estimated at a broader level of geography such as Statistical Sub-Division (SSD). For example, in the disability study estimates for physical disability (which accounts for more than 10%) could be obtained at a reasonably Australian Bureau of Statistics 11
- Page 1 and 2: A Guide to Small Area Estimation -
- Page 3 and 4: A Guide to Small Area Estimation -
- Page 5 and 6: A Guide to Small Area Estimation -
- Page 7 and 8: A Guide to Small Area Estimation -
- Page 9: A Guide to Small Area Estimation -
- Page 13 and 14: A Guide to Small Area Estimation -
- Page 15 and 16: A Guide to Small Area Estimation -
- Page 17 and 18: A Guide to Small Area Estimation -
- Page 19 and 20: A Guide to Small Area Estimation -
- Page 21 and 22: A Guide to Small Area Estimation -
- Page 23 and 24: A Guide to Small Area Estimation -
- Page 25 and 26: A Guide to Small Area Estimation -
- Page 27 and 28: A Guide to Small Area Estimation -
- Page 29 and 30: A Guide to Small Area Estimation -
- Page 31 and 32: A Guide to Small Area Estimation -
- Page 33 and 34: A Guide to Small Area Estimation -
- Page 35: A Guide to Small Area Estimation -
A Guide <strong>to</strong> Small Area Estimation - Version 1.1 05/05/20<strong>06</strong><br />
2. Assessing User Requirements<br />
2.1 User Requirements<br />
Understanding user requirements for small area estimates is paramount for providing a<br />
high quality small area product that meets the client's decision making requirements.<br />
The importance of gaining a thorough understanding of user requirements at this initial<br />
phase cannot be over emphasised. Shortcuts taken at this phase will often lead <strong>to</strong> an<br />
inferior quality product and/or valuable time and resources lost along the way. With the<br />
right questions, users will be able <strong>to</strong> give a clear indication as <strong>to</strong> what information is<br />
critical in their decision making. Users are also a valuable resource in helping <strong>to</strong><br />
determine the best potential sources for borrowing strength.<br />
Where complex techniques need <strong>to</strong> be applied, Methodology Division (MD) staff will<br />
need <strong>to</strong> be involved in performing the methodological work and it is highly<br />
recommended that MD staff are directly involved in the discussion with clients at the<br />
earliest possible opportunity.<br />
Table 2.1 below displays a checklist of the key questions <strong>to</strong> ask clients when<br />
commencing a small area exercise.<br />
Table 2.1 Checklist of Questions <strong>to</strong> Ask Users.<br />
Question<br />
A) What are the key policy making or program funding decisions that require small area data ?<br />
B) What are the organisation's strategic context, goals and desired outcomes, in which these<br />
decision making requirements are nested ?<br />
C) What small area data do users think would best meet their decision making requirements<br />
and what level of geography is required ?<br />
D) What are the consequences for users’ decision making outcomes if the small area data is<br />
incorrect, say, by 5%, 10%, 20%, etc? Which small area estimates have the greatest priority in<br />
terms of accuracy requirements ?<br />
E) Are there any conceptual models, either social or economic, that are believed <strong>to</strong> describe the<br />
process which influences the variable(s) for which we are <strong>to</strong> calculate small area estimates ?<br />
F) What administrative data is available and relevant as auxiliary information <strong>to</strong> support the<br />
modeling of the small area estimates? How is this data collected, for what purpose is it used,<br />
and how accurate is it likely <strong>to</strong> be ?<br />
G) Will small area estimates be required <strong>to</strong> be disaggregated by other categories ?<br />
H) What previous studies have been used, if any, <strong>to</strong> undertake the policy/funding decision for<br />
which small area estimates are required ?<br />
Australian Bureau of Statistics 10
Change language