• Report

Doing Reform Differently: Combining Rigor and Practicality in Implementation and Evaluation of System Reforms

Citation

Crouch, L., & DeStefano, J. (2017). Doing Reform Differently: Combining Rigor and Practicality in Implementation and Evaluation of System Reforms. (International Development Group Working Paper; No. 2017-01). Research Triangle Park, NC: RTI International.

Abstract

This paper brings together two promising intellectual trends in development: Doing Development Differently (DDD), and whole-system reform. In addition, it provides a framework for evaluating system reforms, as rigorously as possible. Doing Development Differently proposes an approach to development in which many different things are tried, in smallish ways, involving stakeholders; in one sense it is a reaction against “blueprint” development. These approaches have much to recommend them. While it may seem counterintuitive that whole-system reform can benefit from an approach that tries many different things, we posit that this may be the best way to carry out reform. The proposed approach to system reform also borrows from the idea of Problem-Driven Iterative Adaptation (PDIA). An issue that arises is that evaluating policy reform—especially an approach to reform characterized by DDD or PDIA—is very difficult methodologically. The difficulty occurs because there is no counterfactual, and there is no one single intervention. Many attempts are made, and the package that emerges may be far from what was designed. The paper argues that, nevertheless, there are ways to interject more rigor into the evaluation of these reforms than at first might seem possible. The education sector is used as a case in point.