Evaluation of the Innovation Lab Network (ILN)

From individual classrooms to state-level systems, educators in the United States are grappling with the question of how to ensure that students become critical thinkers, equipped with skills in communication and problem-solving. At the William and Flora Hewlett Foundation, a longtime benefactor of educational causes, this concept has become known as “Deeper Learning.”

The Hewlett Foundation organized its educational efforts around the theme of Deeper Learning by supporting programs that emphasize personalized learning and take assessment beyond test scores. Part of this initiative is funding for the Innovation Lab Network (ILN), an alliance of 12 states encouraging the adoption of innovative educational practices throughout state education systems.

Founded in 2009 by the Council of Chief State School Officers, a nationwide organization for heads of elementary and secondary education departments, the ILN helps states learn about and adopt policies that promote Deeper Learning. The Hewlett Foundation began funding the ILN in 2011, and in 2015, having made a significant investment in the ILN, chose RTI to evaluate the network’s impact.

Tracking Progress as States Establish New Educational Assessments and Other Innovations

A widespread initiative like the Innovation Lab Network can be a catalyst for change, but exactly how that change comes about can be difficult to pinpoint. To advise the Hewlett Foundation on the ILN’s effectiveness, we set out to look for changes in the status of Deeper Learning initiatives in the 12 states before and after the foundation’s involvement.

The ILN works toward five desired policy outcomes related to Deeper Learning. A key example is the use of multidimensional performance assessments to measure students’ abilities to think critically and solve problems. Instead of answering multiple-choice questions, students design and conduct investigations, analyze data, draw valid conclusions, and report findings. These assessments are a potential replacement for the standardized format that has dominated educational testing for many years.

We set out to determine how the member states were progressing toward this and other outcomes, taking stock of any catalysts and barriers along the way. In our evaluation, we worked directly with the ILN’s stakeholders, conducting in-depth qualitative interviews, an online survey, and a study of related grant proposals, progress reports, and education websites.

Throughout the process, we emphasized transparency, keeping the Hewlett Foundation informed at every step of the process. This commitment to collaboration, along with our deep expertise in educational issues and mixed-methods research, made us well suited to the project and the foundation’s needs.

Measuring the Effects of Network Participation on Public Policy

We learned several important lessons about how groups like the Innovation Lab Network can affect policy. Although many factors influence the ultimate outcome of educational initiatives, we learned that ILN members worked to enact ideas that surfaced in their monthly conversations.

Members told us that the ILN’s support influenced their commitment to promoting Deeper Learning in their state school systems. The connections and resources of the ILN proved helpful to states striving to make the transition to Deeper Learning, often against political resistance. Collaborating with the ILN also let members watch new programs taking hold in other states—and increased their sense of accountability for plans and action items.

Through our evaluation, we were able to assure the Hewlett Foundation that the Innovation Lab Network is an effective vehicle for promoting Deeper Learning. Not every state made equal progress toward the ILN’s desired outcomes, but they each demonstrated that participating benefits their overall effort. With this reassurance, the Hewlett Foundation gained valuable knowledge for continuing its work to get more states and schools involved with Deeper Learning, a concept that can lead to a better-educated society.