Identifying Causes of Verification Refusals on a Large Nationwide Field Study Using a Multilevel Model
Williams, D. C., Touarti, C. M., Clark, C. W., & Butler, J. S. (2011, May). Identifying Causes of Verification Refusals on a Large Nationwide Field Study Using a Multilevel Model. Presented at AAPOR 2011, Phoenix, AZ.
Verification data that are accurate and collected according to project protocols are an essential part of field studies. In particular, verification offers assurance to clients and the public that data are valid and reliable. On the National Survey on Drug Use and Health (NSDUH), respondents are asked to provide contact information so that project staff may call to check on the quality of completed work. Refusal of such information by respondents impedes the ability to verify said work or, at best, introduces delays and added expense to the process. Identifying the causal factors for the absence of verification contact data allows for remedial actions, thus reducing costs and increasing quality.
This paper will present the results of analyzed NSDUH data from all four quarters of 2009 and the first quarter of 2010. First conducted in 1971, NSDUH is sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA) and provides national, state and sub state data on substance use and mental health in the civilian, noninstitutionalized population age 12 and older. Approximately 140,000 household screenings and 67,500 interviews are completed annually.
Questions persist as to whether high verification refusal rates are caused by interviewer performance and noncompliance with protocols, privacy concerns among respondents, protocols and forms needing improvement, or community characteristics. This paper will use multilevel modeling to further examine the effects of field interviewer performance measures and area demographic characteristics on collection of verification contact information. Field interviewer performance will be measured by production, cost, and data quality indicators. The demographic characteristics of sampling areas, or segments, will be based on Census data. We will discuss the effect each of these variables has on verification data in order to identify explanations for patterns in verification refusal rates.