An experiment comparing concurrent and sequential choice+ mixed-mode data collection protocols in a self-administered health survey

Abstract

In self-administered web and paper mode surveys, the Choice+ protocol, which offers both modes concurrently but with a differentially higher incentive for responding by web, has been shown to increase the share of responses obtained by web and to increase response rates overall relative to a concurrent design with equal incentive amounts. This protocol was initially adopted for the 2020 Healthy Chicago Survey. After observing fewer than 20% of the 2020 completes coming via paper, however, an experiment was fielded in the 2021 administration to evaluate a sequential variant of the Choice+ protocol in which the paper mode was only offered to nonrespondents after an initial web-only invitation. This article reports on the key findings from this experiment. Although the sequential Choice+ protocol sacrifices 1.4 percentage points in the yield rate, it comes with a 15.2% lower cost-per-complete and produces a respondent set that looks very similar to the concurrent Choice+ protocol with respect to both demographics variables and key health outcomes measured by the survey. As a result of these findings, the sequential Choice+ variant has supplanted the concurrent Choice+ design as the primary data collection protocol in the Healthy Chicago Survey.