Key Takeaways
RTI applies a measurement-first approach to AI in survey research. AI tools are carefully evaluated for their impact on total survey error before implementation, ensuring efficiency gains do not come at the expense of data quality.
AI enhances survey quality and efficiency by augmenting expert work. Automation frees researchers to focus on innovation, while tools like RTI QUINTET improve quality control by identifying interviewer errors and prioritizing reviews more effectively.
Advanced AI applications enable more adaptive and tailored surveys. By analyzing paradata and piloting agentic AI workflows, RTI is improving outreach strategies, streamlining complex tasks, and unlocking new possibilities for more responsive and efficient survey design.
The need for tailored data coming from surveys has never been greater. Yet survey research faces many challenges including rising data collection costs, declining budgets, and lower participation rates to survey requests. Our federal, commercial, and foundation clients need precise, rich insights to make quick, data-driven decisions.
Our survey scientists and methodologists are meeting these challenges in part by integrating artificial intelligence (AI) across the survey research lifecycle—from survey and questionnaire design to data processing and coding to analysis and reporting. Our teams are already using AI to assist with survey coding, improve survey data quality, and deliver useful automations.
When applied responsibly, AI can strengthen core dimensions of data quality (e.g., accuracy, completeness, consistency, timeliness, relevance) in ways that are difficult to achieve at scale through manual processes alone. These quality improvements, in turn, increase efficiency, reduce cost, and provide decision-makers with the critical data insights they need much faster. RTI evaluates new AI applications against the total survey error framework, assessing its impact on measurement error, sampling error, coverage error, nonresponse error, and processing error. Rather than treating AI as a blanket solution, our rigorous evaluation process assesses how each tool interacts with these error sources, maximizing efficiency while safeguarding data quality. Our human-in-the-loop framework ensures that trained experts review AI outputs, override them when necessary, and continuously refine models based on real-world performance.
Our clients need survey insights faster and cheaper, without compromising data quality or scientific integrity, and we are ready to meet this need. While our researchers are exploring many ways AI can enhance the survey research process, this blog explores four practical use cases where AI and automation are already delivering results across the survey research lifecycle.
1. Increasing Efficiency in Survey Coding and Analysis
First, AI can automate repetitive tasks (e.g., survey coding, data cleaning, and analysis) to reduce processing time and free up research teams to focus their expertise on other parts of the survey process, such as reducing error and further improving data quality. By reducing the manual effort required on surveys, AI provides researchers more time to innovate on best practices for reaching sample members, crafting outreach messaging, and establishing adaptive and responsive data collection protocols. RTI is using automation and partial automation to improve efficiencies, while using robust quality control practices and systems built around AI to check its work.
For example, our team used AI to reduce burden and processing time on the National Science Foundation’s Survey of Earned Doctorates (SED), an annual survey of doctorate graduates at accredited U.S. institutions that tracks trends in doctoral education. Every year, the SED provides valuable insights into respondents’ fields of study, student demographics, post-graduation employment plans, and more. Using SMART, an AI-assisted coding application developed by RTI, we reduced the manual labor spent on the SED coding process by approximately 55% between 2022 and 2024—a savings of 303 hours.
SMART uses modern natural language processing techniques to recommend relevant codes interactively as coders review and assign each text response. The application reads strings of text from each survey response and suggests the most likely categories based on the semantic distance text as well as the way similar strings have been coded in the past. By automating coding and thematic analysis, AI enables researchers to gain actionable insights from survey data more quickly and efficiently.
Importantly, human coders still review every AI-suggested category and intercoder reliability checks ensure accuracy and consistency throughout. Learn more about how we were able to speed up data processing on the SED.
2. Improving Survey Data Quality
AI can also serve as a powerful quality control tool, catching errors that may otherwise go undetected. In survey interviewing, for instance, researchers rely on interviewers to follow standardized procedures, but mistakes happen. An interviewer might skip a probe, misread response options, or introduce subtle inconsistencies that compound across tens and even hundreds of interviews, ultimately introducing error into the data.
Traditionally, quality review has been limited to monitoring a small sample of interviews, leaving most interactions unexamined. AI makes it possible to review every interview in detail, identifying whether interviewers followed prescribed procedures and flagging systematic patterns, such as a question being consistently misadministered, that spot checks would likely miss. The result is not just error detection but a feedback loop that provides corrections during data collection, along with improvements to interviewer training and data collection protocols over time.
RTI is using AI in data processing on multiple projects. One such application is in imputation of missing data. Imputation models can be informed by hundreds, sometimes thousands of variables. Traditional approaches are limited both in how many variables can be used and for the same reason, in the ability to identify important interaction effects. By using machine learning methods, we are able to use more candidate variables and perhaps more importantly, to identify complex multivariate relationships within the data that otherwise would have led to model misspecification. The result is better-informed imputed values that reduce error in survey estimates.
One successful example of this in action is RTI QUINTET, a suite of AI tools developed by RTI for data analysis, transcription, search, and data quality checks on research projects.
Recently, RTI used this tool on a health care survey to identify places where the consent statement was not read verbatim during interviews. Usually, manual review of interview recordings is a time-consuming, labor-intensive process, requiring survey researchers to listen to hours of recorded surveys to identify discrepancies between the interviewer’s administration and the survey’s standardized questionnaire.
On this survey, RTI QUINTET quickly identified 17 wording errors that were made by interviewers and one instance where the consent statement was not read at all. The survey team avoided listening to hours of recorded interviews and enabled its quality assurance staff to prioritize potentially problematic survey recordings that were most likely to need follow-up.
Making Audio AI Data Analysis Easy and Modular
RTI QUINTET™ is an AI-powered tool that transforms large volumes of audio data into usable insights through transcription, search, and analysis, helping researchers improve data quality, reduce costs, and extract value from recordings.
3. Designing More Tailored Surveys Using Paradata
In survey research, paradata, data about the data collection process itself, such as the number and timing of contact attempts, interview duration, interviewer observations (e.g., reasons for refusals), and mode of outreach, has long been recognized as a valuable resource for improving fieldwork. The challenge is that analyzing paradata at scale has historically been logistically and budgetarily infeasible, limiting its use to retrospective evaluation rather than real-time operational decision-making.
AI changes that. By processing large volumes of paradata quickly, AI enables a more adaptive approach to survey operations—one in which data collection strategies are continuously refined based on what is and is not working in the field. For example, on a longitudinal survey where data have already been collected at Time A, using AI we can easily sift through paradata to identify trends and figure out what outreach strategy would be most efficient at Time B. These insights can potentially help researchers get in touch with respondents faster, ensure outreach is happening at the right time of day and day of the week, and send tailored survey requests that would better resonate with respondents compared to using generic messaging.
AI makes it feasible to conduct ad hoc analyses on paradata that were previously impractical given time and budget constraints. Over time, these analyses could help inform our decision-making, conduct surveys more efficiently, improve data quality, and reduce labor hours.
4. Unlocking New Possibilities for Surveys with Agentic AI
Although most researchers are already using large language models to enhance surveys, RTI's experts are now exploring the next frontier: agentic AI, in which AI systems execute multi-step workflows with minimal human intervention, delivering a level of automation that goes beyond single-task assistance.
Currently, RTI is conducting pilot studies to see what is possible for agentic AI. Our team has discovered that it is not necessary to reinvent the wheel in-house to build these agents; instead, one can take something off the shelf from a vendor (e.g., Anthropic, OpenAI) or other open-source alternatives (e.g., OpenCode) and build workflows on top of those agentic tools to execute more complex tasks.
For example, instead of asking language models to do simple tasks like read and summarize a document, we can ask the AI agent to conduct an entire workflow on complex Excel files, like a data harmonization process. The AI agent can analyze the Excel sheets, validate the data, and convert it to the correct format. To guide this work, RTI is developing specific agent skills (e.g., structured instructions and protocols that direct multiple AI agents through defined tasks) along with supporting tools that ensure these agents operate within our quality and governance standards. This approach allows us to scale agentic capabilities while maintaining the rigor and oversight our clients expect.
Learn More About AI and Survey Research
Our team is continually exploring new avenues for using AI to increase efficiency, reduce costs, and improve data quality. Learn more about how we are applying AI to enhance our surveys.