In large health surveys like the National Health Interview Survey (NHIS, currently conducted by the U.S. Census Bureau) and the National Survey of Family Growth (NSFG, currently conducted by the University of Michigan), observations that interviewers record describing selected features of all sampled units represent a promising, cost-effective source of auxiliary information that survey statisticians can use for nonresponse adjustment of survey estimates and responsive survey designs. Interviewers are the eyes and ears of the survey organization out in the field, and a small literature has shown that interviewers can successfully record observations on features of sampled units that are correlated with both key survey measures and response propensity. Unfortunately, these observations have been shown to be prone to error, and a growing body of research has demonstrated that increasing levels of error in the observations can limit the effectiveness of nonresponse adjustments. Furthermore, initial work in this area has found that interviewers vary substantially in the accuracy of their observations, even after controlling for interviewer- and household-level covariates. No existing study has attempted to identify additional sources of this variance.
Motivated by literature in psychology and anthropology that examines sources of bias in human observation and preliminary work suggesting that interviewers vary in terms of their observational strategies, the proposed research aims to: 1) perform a qualitative analysis of several thousand justifications provided by NSFG interviewers for their observations on two key features of all sampled NSFG households; 2) use cluster analysis techniques to determine whether unique subgroups of NSFG interviewers exist based on the observational strategies evident in their justifications; and 3) use multilevel modelling techniques to compare the accuracy of the two observations among the identified subgroups of interviewers, controlling for other relevant correlates of observation accuracy. Successful completion of the proposed research will provide designers of health surveys with evidence of effective observational strategies that can help to standardize the collection of interviewer observations, and demonstrate a new research methodology that survey agencies can use to inform interviewer training.
Health survey data collected from representative samples of human populations are of critical importance for the implementation of policies and programs designed to evaluate and improve public health. Unfortunately, survey response rates are declining worldwide, and cost-effective methods are needed for reducing the nonresponse bias in survey estimates that can result. This project will provide evidence of effective strategies that interviewers can use to collect accurate observations on health-related features of both survey respondents and nonrespondents, improving nonresponse adjustments of health survey estimates that are based in part on the interviewer observations.
Funding:
Health and Human Services, Department of-National Institutes of Health
Funding Period:
09/21/2013 to 08/31/2016