Piloting Data Submission for a Prospective Student-Level Data Network (SLDN)

All postsecondary institutions that participate in Title IV federal financial aid programs must submit data annually to the Integrated Postsecondary Education Data System (IPEDS). As a result, whenever there are changes to IPEDS, it attracts a lot of attention, and rightfully so. At the 2023 Association for Institutional Research (AIR) Forum in Cleveland, OH – and the 2 AIR Forums prior to that – our sessions discussing a potential federal student level data collection attracted a lot of attention given that the sessions concerned potential impacts on IPEDS. If proposed legislation is passed by the U.S. Congress and signed into law, a Student-Level Data Network (SLDN) would be created to collect student-level data. These data would replace the aggregate, student-based surveys in IPEDS. This change would be monumental—for data reporters, policymakers, researchers, and families. 

Student-Level Data Network: Collection Findings

Given the magnitude of the potential change, RTI International has invested internal resources to explore the potential impact of this shift. One major component of this investigation has been engaging with data respondents. After gaining feedback in such areas as (1) the variables required, (2) the reporting schedule, and (3) financial aid data, we conducted a pilot student-level data collection to answer two basic questions: If postsecondary institutions were abruptly asked to report student-level data needed for replicating aggregate IPEDS data, could they? And, if so, what challenges would they face?

To conduct the pilot, we partnered with four diverse entities that were willing to assist with the effort:

• Austin College –private, nonprofit, 4-year, primarily baccalaureate or above
• Henderson State University –public, 4-year, primarily baccalaureate or above 
• Arkansas State University-Beebe –public, primarily 2-year, offering associate degrees and certificates
• Arkansas State University System – oversees seven institutions (both 2-year and 4-year)

Using our knowledge of IPEDS and the SLDN (as described in the College Transparency Act), we provided an Excel template to each participant. We asked them to complete the template with anonymized student-level data, and then we aggregated the data and compared them to the published 2019–20 IPEDS data submissions from those same entities. Throughout the 6-month pilot project, we met periodically to pose and answer questions, discuss obstacles, and solicit feedback.    

The pilot project led to many interesting findings, which we presented at the 2023 AIR Forum:

• Overall, for all institutions, the student-level ENROLLMENT files closely matched the published IPEDS data. In some instances, totals matched, but subtotals for different populations did not.
• In subtotal categories that differed, some differences were large. Variations in subtotal counts were found in the following categories: degree-seeking vs. non-degree seeking, full-time vs. part-time, and entering vs. continuing students.
• Instructional activity calculations were close but not an exact match.
• For COMPLETIONS files, the results overall closely matched the IPEDS submissions, but subtotals for race/ethnicity, gender, and Classification of Instructional Programs (CIP) codes varied slightly. 
• No institutions provided financial aid data. These data are often stored in another system, and they are harder to access.

Institutional and system findings included the following:

• Some institutions did not collect all the detailed data requested (e.g., their data systems collected high school graduation year, but our template asked for month and year).
• Institutional methods for storing data internally might need to be adjusted (e.g., for housing/living arrangements, the data may be overwritten as situations change, so it would be necessary to freeze a snapshot of the data or otherwise archive them).
• Timing is important, and it will be essential to identify the point in time at which a measure should be reported (e.g., if credits attempted are reported at the beginning of the term and credits earned at the end of the term, situations will arise in which a student earns more credits than they attempted and tuition payment calculations will differ between the beginning and end of term).

Overall, everyone involved—including the institutional partners who worked hard to assemble these data—reported that the effort was informative and valuable. In addition to the findings above, institutions reported that (1) it was not as much work as they anticipated, and (2) the resulting data file containing the student-level information has come in handy to answer other data questions internally.  In fact, one institution is planning to reproduce the file again this year, so they have it available as a resource.

Although a switch to a SLDN will entail a lot of work by a lot of people, this pilot demonstrates that the burden associated with providing data to an SLDN may not be as significant as is often expressed by concerned individuals.  It should be noted, however, that the data collection method employed in this pilot project was relatively simple and might not accurately reflect the method that is ultimately determined when an SLDN is designed and implemented. 

Many questions remain to be answered, but the answers to the basic questions asked in this pilot were a resounding YES—institutions may, indeed, readily be able to report student-level enrollment and completions data that would meet the needs of an SLDN (and IPEDS), and we now have a better idea of potential challenges to be faced in doing so.

Learn more about RTI’s research in building a Student-Level Data Network.