Identifying Potential Attrition Bias Using Paradata, Sampling Frame Information, and Survey Data
Creel, D. V., Mitchell, S. B., & Fahrney, K. (2011, May). Identifying Potential Attrition Bias Using Paradata, Sampling Frame Information, and Survey Data. Presented at AAPOR 2011, Phoenix, AZ.
Longitudinal surveys typically experience some amount of attrition from one round of data collection to the next. The major concern that arises because of attrition is the potential for estimates computed from the respondent data to be biased because respondents alone may not accurately represent the survey population. With a focus on attrition, this paper investigates the potential for attrition bias in the second round of a longitudinal survey. We evaluate the relationship between potential attrition bias and a wide array of data, including paradata from the data collection process, sampling frame data, and respondent data from the first round of the survey. In order to screen such a large number of possible variables that may be related to attrition bias, we use a tree-based methodology to identify a smaller group of variables to use in multiple regression and multiple logistic regression modeling to compare the proportional distribution of characteristics of respondents and nonrespondents to determine if nonresponse bias exists. This analysis is conducted for all respondents and important subgroups. To minimize potential bias, we discuss how we adjusted the weights using information from this evaluation. This research adds to the growing literature on the utility of paradata in evaluating nonsampling bias and provides researchers with a guide to identify and account for attrition bias.