In a recent publication in the Journal of Official Statistics, Sunghee Lee and her SRC colleagues Tuba Suzer-Gurtekin, James Wagner, and Rick Valliant applied the well-known Total Survey Error (TSE) framework to further understand potential sources of error in estimates produced by Respondent Driven Sampling (RDS). The various sources of error in RDS estimates have not been studied with this level of conceptual rigor before, and this article provides an important contribution to the literature in this area with several important take-away points for researchers using this type of non-probability sampling (which is often used to sample hard-to-reach populations). Researchers using RDS often make strong assumptions about the nonresponse and measurement errors (or lack thereof) associated with this technique, and this article provides important empirical evidence about the magnitudes of these types of errors.
The authors begin by providing a nice overview of the basics of RDS for those who may not be familiar with the technique, focusing on key recruitment concepts (such as recruitment coupons, which are required for study participation) and the importance of social networks for the technique. They also discuss how estimators are formed based on the RDS design, and the importance of the sizes of social networks for those estimators (interestingly, the only estimators proposed in this literature have been descriptive in nature, and estimators of parameters like regression coefficients have not yet been clearly defined). Next, they review some of the key assumptions underlying estimation based on this technique, and why some of these assumptions might be unreasonable in practice. Finally, they apply the TSE framework to RDS, and discuss the various sources of error that might arise when applying RDS in real data collections.
They then present the results of analyses of two publicly available data sets from studies that used RDS to collect information on persons at risk of HIV, and focus on the nonresponse error arising from the recruitment process in addition to measurement error in reports of network size. Importantly, these two public use data sets contain coupon distribution information from the recruitment process, which is essential for studying potential nonresponse errors in this context. The authors found that many recruitment chains, which are important for the RDS process, ended or ‘died’ very early (in both studies, less than one coupon was redeemed per potential recruiter), which is contrary to a common assumption of exponential growth in the chains, and that coupon redemption rates (as proxies of response rates) were relatively low. Furthermore, respondents within the same recruitment chain tended to have very similar characteristics (or, put differently, within-chain correlations were quite high). In addition, the authors presented evidence of potential nonresponse bias, finding that recruiters with lower income and higher health risks generated more recruits, and that peers with stronger relationships were more likely to accept recruitment coupons. These findings suggested that the recruitment process is not in fact random, which is another common assumption underlying RDS.
In terms of measurement error, self-reported network sizes were found to be quite variable and in some cases unreasonable (in one of the studies, 5.9% of reports were zero, and one report was 2,100), suggesting possible measurement error in these reports (which are essential for estimation based on the RDS technique). These results were important because RDS assumes that there is a positive relationship between network size and the probability of being selected for recruitment, and widely varying network sizes can lead to weighting effects on estimates (both in terms of bias and variance). The authors demonstrated how both estimates of descriptive statistics and the variances of those estimates were strongly affected by the highly variable weights that resulted. The authors also considered the effects of the weights on estimates of bivariate associations in logistic regression models, and found that the extreme weights can result in unexplainable and unreasonable associations. These results spoke to the importance of trying to apply different weight smoothing approaches when using these types of estimators.
In conclusion, the authors emphasize the importance of assessing these errors and assessing the sensitivity of estimates to different weighting approaches in a given study before trying to make inferences about hard-to-reach populations. They discuss the potential advantages of follow-up interviews, and how careful design of these interviews should be considered. They also discuss possible techniques for incorporating the errors that they discovered in inferential procedures, introducing many possible avenues for future research.
This article presents a careful study of the errors that can arise in RDS, and confirms that similar types of survey errors occurred in two independent RDS studies of at-risk populations. As is the case with any empirical studies that carefully apply the TSE framework to a given type of survey design, the errors identified should help future researchers applying such designs to make better inferences and think carefully about how these types of RDS designs can be improved.
Eleanor Singer died on June 3, 2017. She was 87.
She was a research professor emerita at the University of Michigan’s Survey Research Center, which is part of the Institute for Social Research (ISR). She had a long
and distinguished career at the University of Michigan, at Columbia University, and elsewhere.
‘Eleanor was a major figure in the field of survey methodology and she will be greatly missed by all who knew her,’ said ISR Director David Lam. ‘We are fortunate that she spent the last decades of her illustrious career at ISR, where she made major contributions to research, training, and the intellectual life of the Institute.’
In 2016, she received the Monroe G. Sirken Award in Interdisciplinary Survey Methods Research for ‘significant contributions in our understanding of survey participation, sources of nonresponse bias, and factors affecting survey responses; for pioneering research on the use and effects of incentives; and for leadership in developing awareness and understanding of ethical issues in survey research.’
Please read more about Eleanor Singer’s distinguished career, and her personal and professional impact on those who worked with her:
Eleanor Singer: In Memoriam (PDF)
SRC research assistant professor Hongwei Xu and his colleagues studied how early-life conditions affected cognitive function among the middle-aged and elder population in China in a recent paper published in the Journal of Aging and Health. They found that childhood socioeconomic status (SES) and nutrition were significantly associated with late-life memory performance. Their results also showed important geographical (urban vs rural) differences.
Cognitive function is an important dimension of well-being. Previous research has shown that it is directly associated with the ability to process information and perform activities in daily life, and a low level of cognitive function is also associated with higher demand for personal care and premature deaths among the elderly. Research has also shown that early-life conditions such as SES and health may influence cognitive function in older ages. Nevertheless, most of this research has been conducted in North America and European contexts, and whether these results can be generalized in other contexts remains a question.
Hongwei and his colleagues studied the relationship between early-life conditions and older-age cognitive function in China using the Chinese Health and Retirement Longitudinal Study (CHARLS), a nationally-representative sample of those aged 45 or older in China. They tested two hypotheses: whether childhood SES, nutrition, and health were associated with late-life cognitive function, and whether such effects were also explained by adulthood SES and health.
Only the 2011 (the very first) wave of the CHARLS data was used. Cognitive function was measured by memory performance via ‘word recall’, in which respondents were asked to recall 10 Chinese nouns right after the interviewer read them to the respondents (‘immediate recall’) as well as 4 minutes later (‘delayed recall’). Childhood SES was measured by father’s education, mother’s education, and childhood residence (urban vs rural). Childhood health was self-reported (and hence retrospective), and childhood nutrition was measured by knee height. Adulthood SES was measured by education, employment status, pension benefit, and household asset (number of luxury items in the house). Adulthood health was measured by the existence of chronical disease, functional limitation, and difficulties in performing activities of daily living (ADL).
Hongwei and his colleagues found that childhood SES and nutrition (but not health) were associated with memory performance in China. Additionally, living in an urban residence in childhood also had positive effect on late-life cognitive function, possibly because the Chinese government disproportionally allocated its resources in industrial cities and set up barriers to movements from rural to urban areas. The paper also found the association between childhood conditions and cognitive function can also be partially explained by the respondent’s own level of education. Adulthood SES and health did not seem to have additional explanatory power to the main effect between childhood conditions and late-life cognitive function.
That those who grew up in urban areas had better late-live cognitive outcomes was especially interesting. Today’s elderly population in China grew up at a time when the urban population had substantial advantages in food, education, employment, housing, and health care. Although the restriction in migration was eventually relaxed, it still is difficult even today to change the household registration (hukou) from rural to urban. Hence, Hongwei’s paper also adds to the literature that childhood residence affected late-life outcomes through a channel that is quite specific in the Chinese context.
The Affordable Care Act (ACA) was designed to increase health insurance coverage in the U.S. and has largely succeeded in doing so. An open question is whether and how the implementation of the law has affected health disparities. In particular, the United States experiences large and enduring racial and economic disparities not only in health insurance coverage but in income, wealth, and health. SRC’s Helen Levy, with co-authors Thomas Buchmueller, Zachary Levinson, and Barbara Wolfe used data from the American Community Survey (ACS) to study racial patterns in coverage and to track changes in coverage by race and ethnicity over the period when many of the major provisions of the act were implemented, from 2013 to 2014.
They also looked at the sources of coverage, studying nonelderly adults who were uninsured, covered by Medicaid, and covered by private health insurance. This is important because sources of coverage differ by race with Whites more likely to hold private insurance and Blacks and Hispanics more likely to hold public insurance. Interestingly, even before the ACA went into effect, race differences in income explained most of the gap in insurance coverage between Black and White Americans, and to a lesser extent between White and Hispanic US citizens. On the other hand, even at similar levels of income, noncitizen Hispanics were much less likely to hold health insurance.
The researchers found that Obamacare reduced racial and ethnic disparities in health insurance coverage. Large disparities in coverage continue, however. Coverage increased the most in states that expanded Medicaid. The researchers suggest that further increases in coverage will require Medicaid expansion by more states.
Buchmueller, T.C., Levinson, Z.M., Levy, H.G., Wolfe, B.L. (2016). Effect of the Affordable Care Act on Racial and Ethnic Disparities in Health Insurance Coverage. American Journal of Public Health. DOI: 10.2105/AJPH.2016.303155
In a recently published paper in the Journal of Human Resources, SRC researcher Joelle Abramowitz found that the Affordable Care Act (ACA) young adult provision was associated with young adults being less likely to marry. Her findings suggest that prior to the provision, some individuals married in order to obtain health insurance coverage and that the provision may enable young adults to make marriage decisions without the added consideration of insurance coverage, allowing them to search longer to find spouses who are better matches and form more stable marriages.
In 2010, the ACA young adult provision was implemented to allow those aged between 19 and 25 to be covered by their parents’ private insurance plans. The provision was intended to increase insurance coverage among young adults, who previously became ineligible for coverage under their parents’ plans when they turned 19 years old.
Without such a provision, the main channels to obtain private health insurance coverage for young adults were through working for an employer who provided health insurance or through obtaining coverage as a dependent on a spouse’s private insurance plan. By allowing young adults to stay on their parents’ plans, the ACA young adult provision facilitated an alternative source of health insurance coverage. It follows that such a provision could then affect young adults’ decisions to work or marry, as they no longer had to rely on employment or a spouse to obtain health insurance coverage.
To examine whether the ACA young adult provision influenced the marriage decisions of young adults, Joelle used the 2008 through 2013 American Community Survey (ACS) and compared the marriage patterns of those aged 23-25 (who were affected by the provision; the ‘treatment’ group) to those aged 28-30 (who were not affected by the provision; the ‘control’ group). To the extent that newly eligible individuals would have otherwise married in order to obtain health insurance coverage, the implementation of the ACA young adult provision would decouple their marriage decision from health insurance coverage decision, reducing their incentive to marry.
Indeed, this was what Joelle found in her paper. Assuming that the marriage patterns of the treatment and control groups would have followed the same patterns in the absence of the ACA young adult provision, the difference between the two groups net of the time trend (which, in statistics jargon, is the ‘difference-in-differences estimate’) can be interpreted as the effect of the young adult provision on the likelihood of marrying. Among those aged 23-25 who were not married in the year prior to the ACS survey, the likelihood of marrying during the survey year fell, representing approximately 65,000 fewer people in that age group marrying each year.
While this analysis did find that those aged 23-25 were less likely to marry, it does not necessarily suggest that individuals are less likely to eventually marry. Rather, the provision may only result in young adults delaying marriage either in order to search longer for a spouse who is a better match or to put off the marriage decision until they age out of their parent’s coverage at age 26. Distinguishing ‘marriage delayed’ versus ‘marriage foregone’ would require subsequent years of data allowing the researcher to follow the treated group into older ages and examine their marriage decisions.
Together with her SRC colleague Yvonne Terry-McElrath, SRC Research Associate Professor Megan Patrick recently published an article in Addiction entitled ‘High-Intensity Drinking by Underage Young Adults in the United States.’ This article addresses important gaps in knowledge with regard to both frequency of and trends in high-intensity drinking ( consuming 10+ or even 15+ drinks in a row at one occasion) , binge drinking, and sustained intoxication during the transition out of high school. The article presents descriptive and multivariate analyses of longitudinal data from the Monitoring the Future (MTF) study to understand changes in these behaviors over the course of one to two years after 12th grade; nine different cohorts from the MTF (first measured in 2005-2013 and followed up in 2006-2014) were ultimately analyzed in the study.
The article reports fairly high prevalence of binge drinking, high-intensity drinking, and sustained intoxication among this population, and shows that the overall prevalence of binge and HID increases during the transition out of high school. However, the article also shows that within individuals, these behaviors generally tend to decrease during the transition out of high school. In addition, the authors report that the prevalence of binge drinking and HID has in fact decreased across the nine MTF cohorts, and that the vast majority (roughly 3/4) of this population does not tend to engage in these behaviors. Four-year college students who were not living with their parents were found to be at highest risk of engaging in these behaviors when controlling for a variety of other socio-demographic covariates, suggesting that intervention strategies should particularly target this subgroup of young adults. This study was novel not only in its use of longitudinal MTF data, but also in its consideration of the living situations of young adults and how that moderates the relationships of various socio-demographic factors with high-risk drinking behaviors.
In light of these results, the article calls for early intervention to prevent high-risk drinking behaviors during the high school years, and also intervention for those exhibiting high-risk drinking behaviors during the high school years to aid the transition out of high school. Interestingly, the authors report that the prevalence of having 15+ drinks in a row at some point in the last two weeks has remained rare but stable across these cohorts. The absence of a decrease in this behavior over time for this population does introduce a need for possible interventions that could decrease what is undoubtedly an extremely high-risk behavior.
In sum, this article makes effective use of several cohorts of MTF data to answer important public health questions, and points to several opportunities for evidence-based interventions in a population that is very vulnerable to alcohol abuse and dependence.
By: Population, Neurodevelopment and Genetics Program
Why are disadvantaged children and adults disproportionately likely to experience attention disorders, chronic stress, and health problems? For more than a century, the nature/nurture debate has found unsatisfactory explanations, often casting blame on more vulnerable populations. Dan Keating’s book Born Anxious shows promising new research that transcends the nature/nurture stalemate and contains implications for many of us. It turns out that significant stress, particularly early on, can change a genetic process called ‘stress methylation,’ affecting us for life. Nurturing seems to be able to ameliorate or reverse this process. Anyone exposed to substantial prenatal or early childhood stress, without comfort, is at risk. Future generations may also be at risk: while this epigenetic change is not tied directly to DNA, it can be passed on in utero.
Keating and colleagues at the Human Development Program (HDP) at the Canadian Institute for Advanced Research used longitudinal data from the 1958 UK Birth Cohort to document that socioeconomic circumstances at birth have an impact throughout life, with sweeping consequences like academic problems at age 11 and health problems at age 30. They also found a ‘gradient effect,’ meaning that the outcomes became worse at each step down the socioeconomic scale. Still, socioeconomics were a stand-in for what was really causing the debilitating changes.
Animal studies showed the HDP team that a key factor was stress. Peer-raised monkeys, without parental nurturing, had an amped-up stress response, provoked quickly and slow to fade. A study of children from destitute Romanian orphanages showed that those who were not adopted into nurturing homes suffered as the deprived monkeys had, with a host of difficulties from stress to trouble forming relationships.
The missing piece was introduced by Michael Meaney in his work on epigenetics, the study of changes in gene expression rather than DNA itself. In stress methylation, the gene whose job it is to chill and shut off cortisol is silenced. Through stress methylation, in-utero and early-life stress can cause a permanent epigenetic change telling the newborn, child, teen, and then adult that it’s a dangerous world out there and you better live hard. Keeping the stress system on high alert leads to numerous problems in health and development throughout the life course, and is associated with early mortality.
Keating is now part of the SRC’s Population, Neurodevelopment and Genetics Program where he and colleagues study the many ramifications of and questions raised by these discoveries. His new book explains the current science on epigenetic stress changes, how they affect us as we age, how we may be able to heal, and what the implications are for our stress-permeated culture and future generations.
Daniel P. Keating. (2017). Born anxious: The lifelong impact of early life adversity and how to break the cycle. St. Martin’s Press.
By: Youth and Social Issues Program
E-cigarette use has increased rapidly in the past 5 years, from less than 1% to 16% of US high school seniors by 2015.
Now researchers wonder if this increase puts young people at greater risk for smoking regular cigarettes. Several local and regional studies have found that vaping (using e-cigarettes) among youth is associated with higher odds of smoking cigarettes.
A recent study by SRC researchers Richard Miech, Megan Patrick, Patrick O’Malley, and Lloyd Johnston confirms this with US nationally representative data from Monitoring the Future. The study also examines whether vaping makes cigarettes seem less harmful or, conversely, helps young people quit smoking. Students in the study completed a baseline survey in 2014 and a follow-up survey a year later.
Youth who had never smoked a cigarette by 12th grade were 4 times more likely to have smoked a year later if they reported vaping in 12th grade. Among these 12th graders who vaped, 31% experimented with cigarettes in the following year, compared to 7% of their non-vaping peers. This difference remained after taking into account sex, race, and parental education. Even among the non-smokers who considered cigarette smoking to be very harmful, vaping in 12th grade strongly predicted experimenting with cigarette smoking in the following year.
Vaping also seemed to have an effect on seeing cigarettes as less dangerous. Among non-smokers, those who vaped were 4 times more likely than their non-vaping peers to move away from seeing cigarettes as very harmful. This finding suggests that vaping may desensitize teens to the very real health hazards of smoking.
Contrary to common idea that vaping is used to quit smoking, vaping did not make smoking teens any more likely to have quit smoking a year later.
This study and others suggest that vaping may well be a one-way bridge to cigarette smoking for youth. According to Miech, teens who vape should be considered at high risk for future smoking.
Miech, R., Patrick, M. E., O’Malley, P. M., & Johnston, L. D. (2017). E-cigarette use as a predictor of cigarette smoking: Results from a 1-year follow-up of a national sample of 12th grade students. Tobacco Control. doi: 10.1136/tobaccocontrol-2016-053291.
When the suicide rate among U.S. Army personnel began rising in 2002, reaching record levels in 2007, 2008, and 2009, the Army took action by commissioning a large-scale study to try to understand why this was happening and what could be done about it. The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) was launched in 2009 to shed light on what puts soldiers at risk of suicide. Not surprisingly attempted suicide has increased along with the rate of increase in suicide and is an important indicator of suicide risk.
The study’s lead author Robert Ursano and SRC researcher Steve Heeringa, one of the principal investigators of Army STARRS, and others recently published findings in the Journal of the American Medical Association Psychiatry on sociodemographic, service-related, and mental health risks for suicide attempt among soldiers who were on active duty between 2004 and 2009.
They compared Regular Army-enlisted soldiers who were currently deployed, previously deployed, or never deployed. Soldiers who had never deployed had the highest rate of suicide attempt, followed by previously deployed and currently deployed soldiers.
Female soldiers were more likely than male soldiers to attempt suicide. Soldiers with fewer years of service were at greater risk compared to those with longer years of service. Not surprisingly, a recent mental health diagnosis-especially posttraumatic stress disorder, substance abuse, and depression-put soldiers at increased risk of suicide attempt.
They also explored the method and timing of suicide attempt. Those who were previously or currently deployed were most likely to attempt suicide with a firearm. The timing of when soldiers were at greatest risk of suicide attempt differed by deployment status. For those who were never deployed, risk was greatest early on, in the second month of active duty. This period is known to be a stressful period during basic training and Army acculturation. More research is needed to determine whether this is a pattern arising from expanded Army recruitment during war.
Ursano, RJ, et al. (2016). Risk Factors, Methods, and Timing of Suicide Attempts Among US Army Soldiers. JAMA Psychiatry, 73(7): 741-749.
The Michigan Program in Survey Methodology (MPSM) faculty Brady West and Fred Conrad, the Joint Program in Survey Methodology (JPSM) faculty Frauke Kreuter, and MPSM doctoral student Felicitas Mittereder published a new study in the Journal of Royal Statistical Society-Series A on the effect of conversational interviewing, funded by the National Science Foundation, the Institute for Employment Research in Nuremberg, and infas in Nuremberg.
Standardized interviewing in which interviewers are trained to read the questions as worded has been argued to minimize differences across interviewers in how they administer the questions and reduce interviewer variance in the responses. Previous studies have shown that conversational interviewing can provide more accurate responses when the questions may be ambiguous. In addition to higher cost associated with longer interviews, there has been no evaluation of whether allowing interviewers to deviate from the script may also be increasing interviewer variance.
In their study West, Conrad, Kreuter, and Mittereder employed a design with 15 geographic areas in Germany and 60 interviewers, randomly assigning two interviewers to conversational interviewing and two others to standardized interviewing within each area. The sample was selected from a database so that sample members were employed, have had more than one job, have had a part-time job, have had a spell without employment, and were at least 18 years old. These criteria ensured a more complex employment history and also provided a gold standard for some of the survey questions. Criteria for selecting the survey questions were that they elicited greater need for interviewer-respondent interaction and requests for clarification, or that they showed larger interviewer effects.
They fit multilevel models to the 55 survey questions to estimate the intra-interviewer correlations (ICC) and the shift in response distributions, controlling for the area effects in the interpenetrated interviewer design. Five questions showed significant or marginally significant differences in ICC, with standardized interviewing having lower between-interviewer correlations. However, for two of these items, the within-interviewer variances were larger for the standardized interviewing, potentially contributing to the ICCs. For one question for which administrative data were available, the reduction in bias overwhelmed the increase in interviewer variance. Fourteen questions showed significant or marginally significant shifts in response distributions, all in the expected direction of more accurate responding under the conversational interviewing condition. The authors conclude that they find support for the use of scripted, standardized language in survey questions, but not restricting interviewers from providing additional clarification.
Brady West, Frederick Conrad, FraukeKreuter, & Felicitas Mittereder. Can conversational interviewing improve survey response quality without increasing interviewer effects? Journal of the Royal Statistical Society, Series A: Statistics in Society. Online: December 2016.