Current Population Survey Design and Methodology - Census Bureau

Chapter 15.
Sources and Controls on Nonsampling Error
INTRODUCTION
For a given estimator, the difference between the estimate
that would result if the sample were to include the entire
population and the true population value being estimated
is known as nonsampling error. Nonsampling error can
enter the survey process at any point or stage, and many
of these errors are not readily identifiable. Nevertheless,
the presence of these errors can affect both the bias and
variance components of the total survey error. The effect
of nonsampling error on the estimates is difficult to measure
accurately. For this reason, the most appropriate
strategy is to examine the potential sources of nonsampling
error and to take steps to prevent these errors from
entering the survey process. This chapter discusses the
various sources of nonsampling error and the measures
taken to control their presence in the Current Population
Survey (CPS).
Sources of nonsampling error can include the following:
1. Inability to obtain information about all sample cases
(unit nonreponse).
2. Definitional difficulties.
3. Differences in the interpretation of questions.
4. Respondent inability or unwillingness to provide correct
information.
5. Respondent inability to recall information.
6. Errors made in data collection, such as recording and
coding data.
7. Errors made in processing the data.
8. Errors made in estimating values for missing data.
9. Failure to represent all units with the sample (i.e.,
undercoverage).
It is clear that there are two main types of nonsampling
error in the CPS. The first type is error imported from
other frames or sources of information, such as decennial
census omissions, errors from the Master Address File or
its extracts, and errors in other sources of information
used to keep the sample current, such as building permits.
All nonsampling errors that are not of the first type are
considered preventable, such as when the sample is not
completely representative of the intended population,
Current Population Survey TP66
U.S. Bureau of Labor Statistics and U.S. Census Bureau
within-household omissions, respondents not providing
true answers to a questionnaire item, proxy response, or
errors produced during the processing of the survey data.
Chapter 16 discusses the presence of CPS nonsampling
error but not the effect of the error on the estimates. The
present chapter focuses on the sources and operational
efforts used to control the occurrence of error in the survey
processes. Each section discusses a procedure aimed
at reducing coverage, nonresponse, response, or data processing
errors. Despite the effort to treat each control as a
separate entity, they nonetheless affect the survey in a
general way. For example, training, even if focused on a
specific problem, can have broad-reaching effects on the
total survey.
This chapter deals with many sources of and controls on
nonsampling error, but it is not exhaustive. It includes
coverage error, error from nonresponse, response error,
and processing errors. Many of these types of errors interact,
and they can occur anywhere in the survey process.
Although the full effects of nonsampling error on the survey
estimates are unknown, research in this area is being
conducted (e.g., latent class analysis; Tran and Mansur,
2004). Ultimately, the CPS attempts to prevent such errors
from entering the survey process and tries to keep these
that occur as small as possible.
SOURCES OF COVERAGE ERROR
Coverage error exists when a survey does not completely
represent the population of interest. When conducting a
sample survey, a primary goal is to give every unit (e.g.,
person or housing unit) in the target universe a known
probability of being selected into the sample. When this
occurs, the survey is said to have 100-percent coverage.
On the other hand, a bias in the survey estimates results if
characteristics of units erroneously included or excluded
from the survey differ from those correctly included in the
survey. Historically in the CPS, the net effect of coverage
errors has been an undercount of population (resulting
from undercoverage).
The primary sources of CPS coverage error are:
1. Frame omission. Frame omission occurs when the
list of addresses used to select the sample is incomplete.
This can occur in any of the four sampling
frames (i.e., unit, permit, area, and group quarters;
see Chapter 3). Since these erroneously omitted units
cannot be sampled, undercoverage of the target population
results. Reasons for frame omissions are:
Sources and Controls on Nonsampling Error 15–1