Using Human-in-the-Loop AI to Accelerate Economic Mobility Research

Changing economic well-being over a person’s lifetime often depends on a multitude of interconnected factors such as housing, education, employment, health, and access to human services. Governments, philanthropic organizations, and community leaders increasingly view economic mobility, or the ability for a group or individual to improve their economic well-being over time, as a priority investment area. This reflects a growing recognition that economic well-being is influenced by interconnected systems rather than any single sector alone. Stronger economic mobility helps create more stable communities, whereas weaker mobility contributes to widening income gaps and inequality.

Studying economic mobility can be challenging because the topic spans multiple domains, and data from each sector is often siloed. Fields that engage with the concept often define it differently and measure outcomes in inconsistent ways, making meaningful comparisons across studies difficult. These fragmented evidence-bases leave decision makers without a clear picture of which interventions actually drive change.

Policymakers and program leaders need better methods for understanding and tracking economic mobility to help them make decisions that boost economic opportunities within communities, reducing systemic poverty.

How RTI combined AI and expert review to study economic mobility

RTI sought to create a theory-driven framework by studying the domains that significantly influence wealth over time. Using internal RTI funding, we utilized DistillerSR, an AI-enabled software platform that automates and manages systematic literature review processes. Our team implemented a human-in-the-loop AI workflow using DistillerSR’s platform to support screening, prioritization, and data extraction while maintaining expert oversight. This strategy allowed RTI to leverage AI to quickly and efficiently build an economic mobility framework that is interpretable, valid, and defensible.

Our process included the following steps: 

1. Conducting a comprehensive literature search and importing records into DistillerSR.
2. Using AI-assisted screening and continuous reprioritization to identify relevant studies.
3. Leveraging DistillerSR’s Smart Evidence Extraction to automate data extraction from full-text articles.
4. Synthesizing findings and developing a theory-driven economic mobility framework.
    

Human-in-the-loop oversight for AI-assisted screening

RTI completed a comprehensive search of academic and other literature sources published from 2019 through 2026 across five interconnected domains: place-based strategy, workforce pathways, housing stability, early childhood support, and digital access. We imported all identified records, including their titles and abstracts, into DistillerSR, which organized each entry into a centralized evidence repository and removed any duplication. This process led to the identification of 397 unique records that were appropriate for further screening. 

Our AI-assisted screening procedure included:

• Two humans reviewed 104 sources: This step calibrated agreement between the human researchers and created a training set for DistillerSR.
• DistillerSR and one human reviewed 83 sources: This step confirmed agreement between the human researchers and DistillerSR.
• DistillerSR reviewed 210 sources: This step began after the team confirmed alignment between DistillerSR and the human researchers.

DistillerSR's AI-assisted screening capability independently reviewed 52.9% of references, while our team validated the results and conducted targeted quality assurance to maintain methodological rigor. During this process, 85 sources were excluded, and the remaining 312 sources advanced to a full-text review. 

The approach demonstrated how organizations responsible for evaluating complex evidence bases can use AI-enabled workflows to accelerate research synthesis while maintaining scientific rigor. 

Full-text review and data extraction

During the full-text review, our team assessed whether each publication met predetermined criteria, such as relevance to economic mobility, time frame, and applicability to the U.S. context. This process identified a total of 251 publications that met the criteria. Next, we used DistillerSR’s Smart Evidence Extraction technology to identify and extract relevant information from the full-text articles, reducing extraction time from more than 70 hours of manual work to approximately one hour without compromising data quality.

Additional human reviews to assess the AI’s accuracy were also conducted. A random sample of 26 records (approximately 10% of the final set of 251 extracted sources) was drawn from references with lower AI inclusion scores. Each sampled record was manually reviewed by a human reviewer to verify both inclusion decisions and the accuracy of extracted data fields. Only one record was identified as a false inclusion, indicating strong AI performance even among lower-confidence predictions and high reliability in the automated extraction process. 

Understanding results and developing an economic mobility framework

After screening the identified publications and extracting relevant information, the team analyzed the evidence to identify patterns and relationships across the data. This process enabled researchers to develop a theory-driven framework, connecting economic mobility levers, mechanisms, and outcomes across sectors. By bringing together information that was previously siloed, the framework can help policymakers understand how policies, programs, and investments may support long-term economic mobility.

Why human-in-the-loop AI strengthens research quality

Leveraging responsible AI while maintaining rigorous scientific oversight demonstrates that AI can provide the most value when combined with expert judgment rather than replacing it. In this project, RTI’s human-in-the-loop approach enabled a faster, scalable, and transparent evidence synthesis process for an economic mobility framework that can support complex policy and program decisions.

Learn more about RTI’s work in economic development research and artificial intelligence.