Current Population Survey Design and Methodology - Census Bureau
Current Population Survey Design and Methodology - Census Bureau
Current Population Survey Design and Methodology - Census Bureau
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Chapter 16.<br />
Quality Indicators of Nonsampling Errors<br />
(Updated coverage ratios, nonresponse rates, <strong>and</strong> other measures of quality can be found by clicking on ‘‘Quality Measures’’ at<br />
.)<br />
INTRODUCTION<br />
Chapter 15 contains a description of the different sources<br />
of nonsampling error in the CPS <strong>and</strong> the procedures<br />
intended to limit those errors. In the present chapter, several<br />
important indicators of potential nonsampling error<br />
are described. Specifically, coverage ratios, response variance,<br />
nonresponse rates, mode of interview, time-insample<br />
biases, <strong>and</strong> proxy reporting rates are discussed. It<br />
is important to emphasize that, unlike sampling error,<br />
these indicators show only the presence of potential nonsampling<br />
error, not an actual degree of nonsampling error<br />
present.<br />
Nonetheless, these indicators of nonsampling error are<br />
regularly used to monitor <strong>and</strong> evaluate data quality. For<br />
example, surveys with high nonresponse rates are judged<br />
to be of low quality, but the actual nonsampling error of<br />
concern is not the nonresponse rate itself, but rather nonresponse<br />
bias, that is, how the respondents differ from the<br />
nonrespondents on the variables of interest. Although it is<br />
possible for a survey with a lower nonresponse rate to<br />
have a larger nonresponse bias than a survey that has a<br />
higher nonresponse rate (if the difference between respondents<br />
<strong>and</strong> nonrespondents is larger in the survey with the<br />
lower nonresponse rate than it is in the survey with the<br />
higher nonresponse rate), one would generally expect that<br />
larger nonresponse indicates a greater potential for bias.<br />
While it is relatively easy to measure nonresponse rates, it<br />
is extremely difficult to measure or even estimate nonresponse<br />
bias. Thus, these indicators are simply a measurement<br />
of the potential presence of nonsampling errors. We<br />
are not able to quantify the effect the nonsampling error<br />
has on the estimates, <strong>and</strong> we do not know the combined<br />
effect of all sources of nonsampling error.<br />
COVERAGE ERRORS<br />
When conducting a sample survey, the primary goal is to<br />
give every person in the target universe a known probability<br />
of being selected for the sample. When this occurs, the<br />
survey is said to have 100 percent coverage. This is rarely<br />
the case, however. Errors can enter the system during<br />
almost any phase of the survey process, from frame creation<br />
to interviewing. A bias in the survey estimates<br />
results when characteristics of people erroneously<br />
included or excluded from the survey differ from those of<br />
individuals correctly included in the survey. Historically in<br />
the CPS, the net effect of coverage errors has been an<br />
<strong>Current</strong> <strong>Population</strong> <strong>Survey</strong> TP66<br />
U.S. <strong>Bureau</strong> of Labor Statistics <strong>and</strong> U.S. <strong>Census</strong> <strong>Bureau</strong><br />
underestimate of the size of the total population for most<br />
major demographic population subgroups before the<br />
population controls are applied (known as undercoverage).<br />
Coverage Ratios<br />
One way to estimate the coverage error present in a survey<br />
is to compute a coverage ratio. A coverage ratio is the<br />
outcome of dividing the estimated number of people in a<br />
specific demographic group from the survey by an independent<br />
population total for that group. The CPS coverage<br />
ratios are computed by dividing a CPS estimate using the<br />
weights after the first-stage ratio adjustment by the independent<br />
population controls used to perform the national<br />
<strong>and</strong> state coverage adjustments <strong>and</strong> the second-stage<br />
ratio adjustment. See Chapter 10 for more information on<br />
computation of weights. <strong>Population</strong> controls are not error<br />
free. A number of approximations or assumptions are<br />
required in deriving them. See Appendix C for details on<br />
how the controls are computed. Chapter 15 highlighted<br />
potential error sources in the population controls. Undercoverage<br />
exists when the coverage ratio is less than 1.0<br />
<strong>and</strong> overcoverage exists when the ratio is greater than<br />
1.0. Figure 16−1 shows the average monthly coverage<br />
ratios for September 2001 through September 2004.<br />
In terms of race, Whites have the highest coverage ratio<br />
(90.7 percent), while Blacks have the lowest (82.2 percent).<br />
Females across all races have higher coverage ratios<br />
than males. Hispanics 1 also have relatively low coverage<br />
rates. Historically, Hispanics <strong>and</strong> Blacks have lower coverage<br />
rates than Whites for each age group, particularly the<br />
20−29 age group. This is by no fault of the interviewers or<br />
the CPS process. These lower coverage rates for minorities<br />
affect labor force estimates because people who are<br />
missed by the CPS are on the average likely to be different<br />
from those who are included. People who are missed are<br />
accounted for in the CPS, but they are given the same<br />
labor force characteristics as those of the people who are<br />
included. This produces bias in the CPS estimates.<br />
This graph, as well as two other graphs of coverage ratios<br />
by race <strong>and</strong> gender, can be found at . (Their<br />
updates, with more current data, will be posted on this<br />
site as they are made available.) The three graphs provide<br />
1 Hispanics may be any race.<br />
Quality Indicators of Nonsampling Errors 16–1