Assessing reproducibility of analytic findings derived through national survey data integration efforts

Abstract

The quality and content of national population-based surveys are enhanced through
integrated designs that link additional medical, behavioral, environmental, socio-economic
and financial content from multiple sectors. A recent effort by the Committee on National
Statistics of the National Academy of Sciences is serving as a catalyst to advance future
national data integration efforts, as indicated in their recent report on Federal Statistics,
Multiple Data Sources, and Privacy Protection: Next Steps. These integrated data
platforms include content drawn from administrative sources and research studies to
enhance analytic capacity.
The integration of national survey data with content derived from multiple data sources has
the capacity to provide greater insights than possible from any of the component sources.
Based upon the level of granularity characterizing the data profiles of the component data
sets, the data integration effort can result in many-to-many linkages. While it is preferential
to restrict these data enhancement efforts to situations that permit one-to-one linkages, this
is often not possible. In this paper, we focus on a data integration effort that is impacted by
many-to-many linkages and provide a model for assessing the reproducibility of analytic
findings attributable to alternative linkage applications. Examples are provided using data
from the Medical Expenditure Panel Survey (MEPS) and cancer patient-level phase III
clinical datasets. This data integration effort permits studies assessing the influence of
health-related and socioeconomic factors, access to and use of health care services, health
behaviors and preferences in concert with clinical trial treatment effects on cancer patient
outcomes, heretofore not possible. In this study, we assess the reproducibility and stability
of the analytic findings identifying factors influencing patient outcomes as the linkages are
modified. Building similar evaluations into data integration efforts may serve to provide
additional evidence in support of the integrity resultant findings.