An Evaluation of a Training Protocol Designed to Supplement the Coverage of an Address-based Sampling Frame
McMichael, J. P., Shook-Sa, B. E., McKamey, A. C., Stolzenberg, S. J., Morton, K. B., & Iannacchione, V. G. (2011, May). An Evaluation of a Training Protocol Designed to Supplement the Coverage of an Address-based Sampling Frame. Presented at AAPOR 2011, Phoenix, AZ.
Studies have shown that address-based sampling (ABS) frames for in-person surveys suffer from undercoverage of the household population, particularly in rural areas. To bridge this coverage gap, RTI International developed the Check for Housing Units Missed, or CHUM, (McMichael et al 2008) frame supplementation procedure for in-person surveys that use ABS as a sampling frame. The CHUM procedure enables field interviewers to locate dwelling units not included on the ABS list. When done correctly, an ABS frame supplemented with the CHUM can provide near-complete coverage of the household population. Incorrect implementation of the CHUM procedure can lead to undercoverage and/or frame multiplicities. Successfully training field interviewers to implement a frame supplementation procedure is essential to any study that utilizes an ABS sampling frame.
Correct implementation of the CHUM procedure requires field staff who are knowledgeable about the types of situations they will encounter when performing the procedure. To that end, RTI developed a training protocol for the CHUM procedure and evaluated it through an experimental set of field exercises designed to provide information about whether field staff could perform the procedure correctly; what procedures were the most challenging or difficult to perform in the field; and determine future improvements to the training materials and methods based on these findings. Field interviewers’ performance was evaluated on multiple domains including which component of the CHUM procedure was being performed, the difficulty of the CHUM interval, and whether performance improved after the field interviewers gained experience with the procedure. This paper describes the training protocol, the experimental design developed to evaluate field staff performance, and an analysis of the data collected from the field exercises including logistic regression modeling of the factors that contribute to whether the CHUM procedure was implemented correctly.