Disentangling Mode and Nonresponse Effects in the World Trade Center Health Registry
Murphy, J. J., Brackbill, R., Yu, S. S., Wu, D., Walker, D., Turner, L., ... Saleska, E. L. (2012, May). Disentangling Mode and Nonresponse Effects in the World Trade Center Health Registry. Presented at AAPOR 2012, Orlando, FL.
The World Trade Center (WTC) Health Registry was established in 2003 for monitoring the health of people directly exposed to the WTC disaster. Wave 1 (W1) baseline data collection achieved a cohort of 71,437 individuals. Wave 2 (W2) was completed between November 2006 and December 2007 for a response rate of 68.6% (46,322 adults). Currently, the New York City Department of Health and Mental Hygiene (DOHMH) is conducting the third wave (W3) of data collection. Both W2 and W3 used a multi mode approach that included web, mail, and Computer Assisted Telephone Interviewing (CATI). In W2, mail surveys accounted for 46.1% of completed questionnaires, followed by Web surveys (41.8%) and CATI (12.1%).
At the time of CATI initiation, the response rate to mail and Web surveys was 59%. The CATI response rate was 21%. Estimates on certain key mental health measures, like probable post-traumatic stress disorder, were significantly lower for CATI respondents compared to all respondents (13.3% and 19.1%, respectively). This was partly explained by differences in respondent characteristics by mode, but even when controlling for these characteristics, mode differences remained. To better isolate the effect of mode, an experiment was incorporated into W3 with a subset of cases to disentangle mode effects from differential nonresponse. The experiment consists of three matched samples of 400 assigned to Web, mail, or CATI. By tracking who responds by these modes and estimates of key mental and physical health measures, we can better estimate the effects of mode independent of nonresponse error.
This presentation will summarize the methods and findings of the experiment and will be useful for others considering the tradeoffs of efficiency and potential mode effects in their data collection designs.