Managing complex multi-case study evaluations

Between 2010 and 2012, as part of the Communities Putting Prevention to Work (CPPW) initiative, the Centers for Disease Control and Prevention (CDC) funded 50 states, six US territories, and 50 communities to support high-impact, evidence-based, population-wide strategies to create healthy environments for their residents. CPPW is a locally driven initiative with a primary focus on prevention and control of tobacco use and obesity. As part of this initiative, CDC also funded an implementation evaluation to describe and understand how evidence-based, community-level improvements are applied in the field and contribute to improvements in health. We conducted this evaluation using multi-case study methods that best captured the local context and implementation processes. Large, cross-case evaluations present challenges that single or small multi-case evaluations do not. These challenges include creating flexible, but standard data collection instruments; ensuring the feasibility and utility of instruments and processes through pilot testing; promoting consistent data collection and quality; and managing a large qualitative data set and coding team. In this report, we document some strategies regarding data collection, management, and analysis that should be beneficial to other organizations supporting public health initiatives and to investigators in designing the strongest possible evaluations using large multi-case design.