Combining Rigor and Practicality in Implementation and Evaluation of System Reforms
Crouch, L., & DeStefano, J. (2017). Doing Reform Differently: Combining Rigor and Practicality in Implementation and Evaluation of System Reforms. RTI International. International Development Working Paper No. 2017-01
This paper brings together two promising intellectual trends in development: Doing Development Differently (DDD), and whole-system reform. In addition, it provides a framework for evaluating system reforms, as rigorously as possible. Doing Development Differently proposes an approach to development in which many different things are tried, in smallish ways, involving stakeholders; in one sense it is a reaction against “blueprint” development. These approaches have much to recommend them. While it may seem counterintuitive that whole-system reform can benefit from an approach that tries many different things, we posit that this may be the best way to carry out reform. The proposed approach to system reform also borrows from the idea of Problem-Driven Iterative Adaptation (PDIA). An issue that arises is that evaluating policy reform—especially an approach to reform characterized by DDD or PDIA—is very difficult methodologically. The difficulty occurs because there is no counterfactual, and there is no one single intervention. Many attempts are made, and the package that emerges may be far from what was designed. The paper argues that, nevertheless, there are ways to interject more rigor into the evaluation of these reforms than at first might seem possible. The education sector is used as a case in point.