Evaluating the State Innovation Models (SIM) Initiative

Combining innovative evaluation and data analysis methods to track the impact of state-based health care transformation

Center for Medicare and Medicaid Innovation (Innovation Center)
The Urban Institute, National Academy for State Health Policy, The Henne Group

As regulators, legislators, conveners, and both suppliers and purchasers of health care services, state governments are uniquely positioned to use a wide array of policy levers, engage a broad range of stakeholders, and build on existing efforts to improve health care delivery and health outcomes. Recognizing this potential, in 2013 the Center for Medicare and Medicaid Innovation (the Innovation Center) awarded over $250 million in State Innovation Models (SIM) Initiative to six states—Arkansas, Maine, Massachusetts, Minnesota, Oregon, and Vermont—to test the ability of state governments to use their policy and regulatory levers to accelerate health care transformation efforts in their states.

In 2015, the Innovation Center awarded approximately $622 million to 11 new test states—Colorado, Connecticut, Delaware, Idaho, Iowa, Michigan, New York, Ohio, Rhode Island, Tennessee, and Washington. Additional states received smaller Model Design awards under the SIM Initiative in both Round 1 and Round 2 to develop State Health Care Innovation Plans.

Under the SIM Initiative, states use a variety of strategies to encourage health care providers to adopt health care models that promote coordination across provider types, integration of primary care and behavioral health care, and attention to population health. For example, states are changing payment models used by Medicaid and, where possible, aligning those payment models with new or existing efforts across Medicare and commercial payers. States are also offering technical assistance to primary care practices to implement new delivery system models and are developing or enhancing services—such as health information technology (health IT) and data analytic investment—that enable or improve model effectiveness.

Using Innovative Mixed Methods to Understand Impacts on Health and Health Care

The Innovation Center contracted with a team led by our experts to conduct independent evaluations of the SIM Initiative in both Round 1 and 2.

Both RTI-led evaluations use qualitative data collection and analysis to document what states are implementing using SIM funding and technical assistance from the federal government. We are also conducting a quantitative analysis to detect significant changes, if any, in health care and outcomes in the three years following implementation of the SIM Initiative. We synthesize results from these qualitative and quantitative analyses to provide a comprehensive understanding of the SIM Initiative over time.

Our team developed a framework for understanding how the SIM Initiative could affect key outcomes of health and health care. This framework offers examples to illustrate our approach to the evaluations. 

Framework for Understanding Implementation and Impact of the SIM Initiative

Framework for understanding implementation and impact of the SIM Initiative

Each state’s SIM Initiative includes

  • One or more health care delivery and payment reform models
  • Strategies to enable the operation of these models, such as health IT and data analytics investment and workforce development, and plans for integrating population health activities
  • Policy levers to allow or facilitate the spread of these models and strategies throughout the state.

The evaluation consists of analyzing how states are implementing the initiative, one or more quantitative analyses within each state focusing on the evaluation of specific delivery and payment reform models, and additional quantitative analyses that focus on statewide impact relative to a comparison group of states.

Informing Policy Makers

The experience of SIM Initiative states may have particular relevance for other ongoing and future health system reform efforts at the state level. Policy makers can look to lessons learned from how SIM states leverage policies and strategies intended to grow delivery system and payment models to reach more providers and consumers.

The evaluation results could also guide decisions to help build the infrastructure and capacity to operate delivery system and payment models, and help refine strategies based on stakeholder feedback and recognition of a changing environment.