Resetting targets Examining large effect sizes and disappointing benchmark progress

By Jonathan Michael Bradley Stern, Benjamin Piper

This paper uses recent evidence from international early grade reading programs to provide guidance about how best to create appropriate targets and more effectively identify improved program outcomes. Recent results show that World Bank and US Agency for International Development–funded large-scale international education interventions in low- and middle–income countries tend to produce larger impacts than do interventions in the United States, as measured by effect sizes. However, these effect sizes rarely translate into large gains in mean oral reading fluency scores and are associated with only small increases in the proportion of students meeting country-level reading benchmarks. The limited impact of these low- and middle–income countries’ reading programs on the proportion of students meeting reading benchmarks is in large part caused by right-skewed distributions of student reading scores. In other words, modest impacts on the proportion of students meeting benchmarks are caused by low mean scores and large proportions of nonreaders at baseline. It is essential to take these factors into consideration when setting program targets for reading fluency and comprehension. We recommend that program designers in lower-performing countries use baseline assessment data to develop benchmarks based on multiple performance categories that allow for more ambitious targets focused on reducing nonreaders and increasing beginning readers, with more modest targets aimed at improving oral reading fluency scores and increasing the percentage of proficient readers.

Bibliography

Stern, J. M. B., & Piper, B. (2019). Resetting targets: Examining large effect sizes and disappointing benchmark progress. (RTI Press Publication No. OP-0060-1904). Research Triangle Park, NC: RTI Press. https://doi.org/10.3768/rtipress.2019.op.0060.1904

© 2019 RTI International. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Authors

Jonathan Michael Bradley SternJonathan M. B. Stern, PhD, is a Senior Education Research and Evaluation Specialist at RTI International. His recent work has included the development and validation of reading and mathematics assessments for primary school learners; training and capacity building efforts for ministries of education on school inspection and monitoring, standard setting, and national assessment framework development; and leading research design efforts across projects. Dr. Stern has conducted research and provided technical assistance to programs in more than twenty countries across sub-Saharan Africa, the Middle East, and Asia.

Benjamin PiperBenjamin Piper, EdD, is the Senior Director of Africa Education for RTI International and is based in Nairobi. He provides support to large-scale education projects across the world. He led a multi-country study of large-scale teacher professional development modalities and is currently leading a study of instructional coaching practices. Dr. Piper was previously the Chief of Party of Tusome, PRIMR, and the National Tablets Programme. He is currently leading a multicountry study of highly effective large-scale education programs with funding from the Gates Foundation.

Contact RTI Press

To contact an author, request an exam or review copy, or seek permission to use copyrighted content, contact our editorial team.