Skip to Main Content

RTI uses cookies to offer you the best experience online. By and clicking “accept” on this website, you opt in and you agree to the use of cookies. If you would like to know more about how RTI uses cookies and how to manage them please view our Privacy Policy here. You can “opt out” or change your mind by visiting: http://optout.aboutads.info/. Click “accept” to agree.

Accept
RTI International
  • About
    • Office Locations
    • Executive Leadership
    • Corporate Governance
    • Partner with Us
      • U.S. Government
      • Clients and Funding Agencies
      • Industry and Commercial Clients
      • Foundations and Associations
      • Bilateral Agencies and Multilateral Banks
      • Universities and Academic Research Institutions
      • Suppliers and Small Businesses
    • Commitment to Quality
      • RTI's Client Listening Program
    • Ethics and Human Research Protection
    • Living Our Mission
    • Veteran Opportunities at RTI

    About

  • Practice Areas
    • Health
      • Public Health and Well-Being
      • Health Care Transformation
      • Behavioral Health
      • Health Behavior Change
      • Precision Medicine
      • RTI Health Solutions (RTI-HS)
      • RTI Center for Community Health Evaluation and Economics Research
      • Health Equity
      • RTI Health Advance
    • Transformative Research Unit for Equity​
      • Equity Capacity Building Hub
      • Social and Economic Justice Research Collaborative
      • Narrative Research and Community Engagement Lab
    • Education and Workforce Development
      • Early Childhood
      • K-12 Education
      • Postsecondary Education
      • Career and Adult Education and Workforce Development
      • Education Policy, Systems, and Governance
      • Education Research Methodologies
      • Education Technologies
    • International Development
      • Energy for Development
      • Environment
      • Global Food Security, Agriculture, and Nutrition
      • Global Health
      • International Education
      • Monitoring, Evaluation, Research, Learning, and Adapting (MERLA)
      • Youth and Economic Opportunity
      • Building Resilience Against COVID-19 in Developing Countries
      • Water, Sanitation, and Hygiene (WASH)
      • RTI Center for Governance
    • Climate Change
      • Clean Energy Technology and Renewables
      • Climate Finance
      • Climate Justice and Equity
      • Climate Planning, Preparedness and Resilience
      • Climate Policy
      • Climate Vulnerability, Adaptation, and Mitigation
      • Economic Impacts of Climate Change
    • Water
      • Food-Energy-Water Nexus
      • Water Quality
      • WASH (Water, Sanitation, Hygiene)
      • Water Resources Management
    • Energy Research
      • Carbon Capture and Utilization
      • Biomass Conversion
      • Natural Gas
      • Energy Efficiency
      • Industrial Water
      • Syngas Processing
    • Environmental Sciences
      • Air Quality
      • RTI Center for Water Resources
      • Urban Sustainability
      • Toxics
      • Building Resiliency in the FEW Nexus
      • Climate Change Sciences and Analysis
      • Environmental Policy
      • Environmental Justice
      • Sustainable Materials & Waste Management Solutions
    • Justice Research and Policy
      • RTI Center for Community Safety and Crime Prevention
      • RTI Center for Policing Research and Investigative Science
      • Child Well-Being and Family Strengthening
      • RTI Center for Forensic Sciences
    • Food Security and Agriculture
      • Market Systems Strengthening
      • Food Safety
      • Food and Nutrition
      • Global Food Security, Agriculture, and Nutrition
      • Climate-Smart Agriculture
      • Agricultural Innovation
      • Obesity Prevention
    • Innovation Ecosystems
      • Innovation Advising
      • Innovation for Economic Growth
      • Innovation for Emerging and Developing Economies
      • Innovation for Organizations
      • Research, Technology, and Innovation Policy
      • Technology Acceleration
    • Military Support
      • Military Behavioral Health
      • Military Health and Human Performance
      • Military Sexual Assault, Harassment, and Domestic Violence Prevention
      • Wearable Sensor Technologies
      • Military Health System Transformation

    Practice Areas

  • Services + Capabilities
    • Surveys and Data Collection
      • Survey Design
      • Instrument Development
      • Survey Methodologies
      • Data Collection
      • Establishment Surveys
      • Health Registries
      • Data Analysis and Reporting
      • Research Operations Center
    • Statistics and Data Science
      • Survey Statistics
      • Environmental Statistics
      • Coordinating Centers for Multisite Studies
      • Analysis and Design of Complex Data
      • Biostatistics
      • RTI Center for Data Science
    • Evaluation, Assessment and Analysis
      • Evaluation Design and Execution
      • Advanced Qualitative, Quantitative, and Mixed Methods
      • Evaluation, Monitoring, and Assessment
      • Economic Analysis
      • Evaluating Communication Interventions and Campaigns
      • Evidence Synthesis for Policy and Practice
      • Risk Assessment and Prediction
    • Program Design and Implementation
      • Systems Strengthening and Scaling
      • Capacity Assessment and Building
      • Policy Reform Support
      • Curriculum and Teacher Professional Development
      • Interventions and Prevention Programs
      • Implementation Science
    • Digital Solutions for Social Impact
      • Human-Centered Design of Digital Solutions
      • Digital Product Development
      • Digital Communication Campaigns
      • Digital Data Analytics
    • Research Technologies
      • Survey Technologies
      • Data Management and Decision Support Systems
      • Geospatial Science, Technology, and Visualization
      • ICT for Limited-Resource Settings
      • Mobile Applications
      • Web Applications
      • Bioinformatics
      • Interactive Computing
    • Drug Discovery and Development
      • Medicinal Chemistry
      • Molecular Design and Cheminformatics
      • Behavioral Pharmacology
      • Drug Metabolism and Pharmacokinetics (DMPK)
      • In Vitro Pharmacology, Bioassay Development, and High-Throughput Screening (HTS)
      • Isotope Labeling
      • Regulatory Consulting and Support for Medical Products
    • Analytical Laboratory Sciences
      • Bioanalytical and Toxicology Research
      • Forensic Sciences
      • Physicochemical Characterizations
      • Metabolomics
      • Proficiency Testing and Reference Materials
      • Microbiology
      • Analytical Chemistry and Pharmaceutics
    • Engineering & Technology R&D
      • Biomedical Technologies
      • Decarbonization Sciences
      • Environmental Exposure & Protection
      • Materials & Environment
      • Sustainable Energy Solutions

    Services + Capabilities

  • Centers
    • RTI Center for Advanced Methods Development
    • RTI Center for Communication Science
      • Communication Research
      • Communication Design
      • Communication Delivery
    • RTI Center for Data Science
    • RTI Center for Education Services
      • Teaching and Learning
      • Education Leadership
      • Peer Learning Networks
      • Strategic Consulting
    • RTI Center for Forensic Sciences
    • RTI Center for Global Noncommunicable Diseases
      • Program Financing & Economics for NCDs
      • Health Systems Strengthening for NCDs
      • Communication Science and Behavior Change for NCDs
      • Implementation Science for NCDs
    • RTI GenOmics, Bioinformatics, and Translational Research Center
      • Disability Studies
      • Ethics
      • Newborn Screening
    • RTI Center for Water Resources
      • Water Resources Sectors
      • Water Resources Services
      • Water Resources Tools
    • RTI Center for Governance
    • RTI Global Gender Center
    • North Carolina Center for Optimizing Military Performance
    • NCCU-RTI Center for Applied Research in Environmental Sciences
    • RTI Center for Climate Solutions

    Centers

  • Impact
    • Newsroom
    • Insights Blog
    • Events
    • Publications
    • RTI Press
      • About the RTI Press
      • Instructions for Authors
      • RTI Press Collections
    • Projects
    • Global Reach
      • Asia
      • Eastern Europe and Central Asia
      • RTI International India
      • Africa
      • Middle East and North Africa (MENA)
      • Latin America and the Caribbean (LAC)

    Impact

  • Experts
    • Our Experts
    • In-Depth With Our Experts
    • Related News
    • Experts In the Media
    • RTI Fellow Program

    Experts

  • Emerging Issues
    • COVID-19 Research
    • Artificial Intelligence
    • Global Health Security
    • Cannabis Research
    • Opioid Research
      • Interventions for Opioid Use Disorders
      • Preventing Opioid Misuse and Overdose
      • Treating Opioid Use Disorders
    • Policing Research and Investigative Science
    • Drone Research and Application
    • E-cigarette Research
    • Zika Virus Research
    • Integrated Governance

    Emerging Issues

  • COVID-19 Research + Response
  • Global Reach
  • Insights Blog
  • Newsroom
  • RTI Press
  • Publications
  • Partner With Us
  • Careers
  • Facebook IconTwitter IconInstagram IconYouTube IconLinkedin Icon
  • Home
  • Impact
  • RTI Press
  • Using ACASI to measure gender-based violence in Ugandan primary schools

Using ACASI to measure gender-based violence in Ugandan primary schools

By Maitri Punjabi, Julianne Norman, Lauren Edwards, Peter Muyingo.

March 2021 Open Access Peer Reviewed

DOI: 10.3768/rtipress.2021.rb.0025.2104

Check for Updates Download PDF
Punjabi, M., Norman, J., Edwards, L., & Muyingo, P. (2021). Using ACASI to measure gender-based violence in Ugandan primary schools. RTI Press. RTI Press Research Brief No. RB-0025-2104 https://doi.org/10.3768/rtipress.2021.rb.0025.2104
Copy citation
Share
  • Share on Facebook
  • Share on Twitter
  • Share on Linkedin
  • Email
Key Points
  • Audio Computer-Assisted Self-Interview (ACASI) provides respondents with privacy and confidentiality. It has been used by researchers administering surveys of a sensitive nature and is widely regarded as a useful tool in reducing social desirability bias in responses to sensitive surveys. However, ACASI has never been used to measure school-related gender-based violence (SRGBV), a survey normally administered face to face (FTF), among primary school–aged children.
  • A large-scale study was conducted in Uganda in 2019, where surveys on school climate, gender attitudes, social-emotional learning (SEL), and experiences of violence were administered to Primary Grade 3 pupils. These four different surveys of varying sensitivity were used to observe differences in responses across the ACASI and FTF administration types, with experiences of violence being the most sensitive.
  • Comparing responses between the ACASI and FTF groups, reported school climate did not show a difference, indicating that there may be low levels of social desirability bias around this topic. However, there was a difference in reporting by gender for the gender attitudes and SEL surveys, indicating the potential for social desirability bias present in the data collected via FTF.
  • Comparing responses from the SRGBV survey, reports of experiencing at least one act of sexual violence nearly doubled under ACASI compared with FTF, indicating extremely high levels of social desirability bias when talking about sexual violence in schools.

Abstract

School-related gender-based violence (SRGBV) remains difficult to measure because of high sensitivity and response bias. However, most SRGBV measurement relies on face-to-face (FTF) survey administration, which is susceptible to increased social desirability bias. Widely used in research on sensitive topics, Audio Computer-Assisted Self-Interview (ACASI) allows subjects to respond to pre-recorded questions on a computerized device, providing respondents with privacy and confidentiality. This brief contains the findings from a large-scale study conducted in Uganda in 2019 where primary grade 3 students were randomly selected to complete surveys using either ACASI or FTF administration. The surveys covered school climate, gender attitudes, social-emotional learning, and experiences of SRGBV. Through this study, we find that although most survey responses were comparable between ACASI and FTF groups, the reporting of experiences of sexual violence differed drastically: 43% of students in the FTF group versus 77% of students in the ACASI group reported experiencing sexual violence in the past school term. We also find that factor structures are similar for data collected with ACASI compared with data collected FTF, though there is weaker evidence for construct validity for both administration modes. We conclude that ACASI is a valuable tool in measuring sensitive sub-topics of SRGBV and should be utilized over FTF administration, although further psychometric testing of these surveys is recommended.

Creative Commons © 2023 RTI International. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Contents

  • Introduction
  • Background
  • Methodology
  • Findings
  • Recommendations
  • Acknowledgments
  • References

Using ACASI to Measure Gender-Based Violence in Ugandan Primary Schools

By MaitriPunjabiJulianneNormanLaurenEdwardsPeterMuyingo

Introduction

Incidences of school-related gender-based violence (SRGBV) are common in schools around the world. SRGBV can include “acts or threats of physical, sexual, or psychological violence or abuse that is based on gender stereotypes or that targets students on the basis of their sex, sexuality, or gender identities.”1 Incidences of violence for both girls and boys can occur on the way to and from school or while at school. Three broad categories of SRGBV include bullying and non-sexual harassment, corporal punishment, and sexual violence or harassment.1 Research, minimum standards, and monitoring frameworks, like that of the United Nations Girls Education Initiative’s (UNGEI’s) Whole School Approach2 have helped establish greater awareness of the extent of SRGBV and provide concrete steps for policy makers and implementers to address SRGBV. However, the research community is still just beginning to understand both the pervasiveness of SRGBV and its psychological and academic effects on students. Existing prevalence data, intended to inform donor programming and policy, are misrepresented for three reasons:

  1. SRGBV survey questions are especially sensitive. Responding can prove emotionally distressing to the respondent.3 Such sensitivity also poses ethical challenges and considerations that can make it difficult to ensure that the principle of “do no harm” is kept front and center of research studies.2

  2. Social desirability bias—an interest in responding to survey questions that align with cultural values and expectations—can cause respondents to exaggerate responses that align with culturally condoned behaviors or valued experiences or to under-report stigmatized acts or experiences.4 As a result, data reflect experiences or personal attributes perceived to be more socially accepted or desirable rather than the respondent’s truthful experiences.

  3. There are no right or wrong answers, but there are true and untrue answers. The existing literature argues that questions of a sensitive nature (i.e., questions subject to social desirability bias) threaten response accuracy and lead to systematic reporting (i.e., exaggerating socially desirable experiences and behaviors and underreporting less desirable ones). Tourangeau and Yan even argue the more sensitive the question, the greater the likelihood of misreporting.5

Although researchers can prepare to collect data on such sensitive topics by becoming familiar with the social norms and drivers of stigma in the context, it remains difficult to know to what degree social desirability bias will affect self-reported data. The literature on this topic (although results are mixed based on context and study design) argues that survey administration mode can address issues of misreported data caused by social desirability bias.5,6,7,8–14 This brief contributes to the body of literature on survey administration methods for SRGBV by providing early evidence on how to measure SRGBV prevalence among children. Specifically, this research compares Audio Computer-Assisted Self-Interview (ACASI), a confidential, self-report method of data collection, with the Face-to-Face (FTF) administration method across four surveys with varying degrees of sensitivity with Primary Grade 3 pupils in Uganda. This research also includes an analysis of the factor structure and construct validity of data collected using the two administration types.

Background

In contrast to the conventional FTF interview method, ACASI presents survey questions in an audio format through headphones—providing the respondent with true privacy. While completing the survey, respondents select their choices through the tablet without interacting with a data collector. ACASI gives the respondent privacy, which reduces social desirability bias and allows the researcher to place confidence in the quality of data collected.5 A vast amount of literature exists on the use of ACASI in collecting data from HIV-infected patients, on sexual behavior and drug use, and on intimate partner violence.6,7,15–20 ACASI gives the respondent privacy, which reduces social desirability bias and allows the researcher to place confidence in the quality of data collected. Respondents who use ACASI tend to report their experiences differently than those using the traditional FTF method when the experiences may be sensitive, taboo, or socially undesirable.8,21 At the time of writing this brief, no studies have tested the effectiveness of ACASI compared to FTF when administering SRGBV or school-centered surveys, especially with a population as young as 10 years of age in low- and middle-income countries. The study described in this brief addresses this gap in the literature on ACASI.

Rates of violence against children is particularly high in Uganda, with rates of physical violence among 13–17-year-old children ranging between 44% for girls and 59% for boys.22 In 2018, RTI International piloted the use of ACASI in 12 Ugandan public primary schools with Primary Class 5 (or P5; mean age  =  11.9 years) and P7 (mean age  =  13.8 years) students. Survey forms used for this study were adapted from RTI’s Survey of Student Experiences of SRGBV.23 Under the US Agency for International Development (USAID)-funded EdData II, Data for Education Research and Programming activity, RTI developed a suite of SRGBV surveys in 2016. RTI subsequently piloted this suite of surveys under the USAID-funded Uganda Literacy and Retention Activity in 2016 and refined the surveys based on the resulting analyses for use in occasion 1 of the longitudinal study in 2018. Our large-scale ACASI study coincided with the second occasion of the longitudinal study that occurred in 2019. To ensure comparability between the ACASI and FTF administration methods, the exact same surveys were used for ACASI as those used in the longitudinal study.

Results from the 2018 ACASI pilot in 12 schools suggested that ACASI could hold promise for reducing social desirability bias and eliciting more authentic responses. In 2019, we scaled up this study to include a randomized sample of students in 40 Ugandan public primary schools with a younger population of P3 students (mean age  =  10 years) and a suite of instruments related to SRGBV and school-centered dimensions.

Methodology

We visited 40 primary schools for the FTF and ACASI data collection in 2019. (Note that the survey and research protocol were approved by RTI’s Institutional Review Board and Uganda’s The AIDS Support Organization Research Ethics Committee.) These schools were preselected as the random sample of control schools under the US Agency for International Development (USAID)/Uganda Literacy Achievement and Retention Activity longitudinal study in 2018. Under this existing study, assessor teams visit the same schools every year, and the teams use the FTF method to assess students with surveys on school climate, gender attitudes, social-emotional learning (SEL), and experiences of violence. The ACASI administration included the same school climate, gender attitude, SEL, and experiences of violence surveys so that surveys with varying levels of sensitivity could be used to compare the data collected using both ACASI and FTF administration. The second round of data collection for the longitudinal study was conducted in July 2019, during which the ACASI study took place. ACASI assessor teams visited the same schools on the same day as the FTF teams. Because the 2019 data collection was the second occasion for the longitudinal study, 12 pupils in each school had already been selected the year before to take part in the FTF longitudinal study. Therefore, one day before the day of data collection, one member of the research team sampled 12 P3 students in each school out of those who were not already part of the longitudinal sample.* Similar to the sampling procedure for the longitudinal study in 2018, this was done by forming two lines of P3 boys and P3 girls present on the day before data collection, counting the total number of boys and number of girls in each line, and dividing by 8 to calculate a jump number for each sex. That jump number was used to sample 8 boys and 8 girls in each school, which included 2 replacements per group in case caregivers or pupils themselves did not show up to the school the following day for data collection. Lines were formed randomly by the assessor teams without influence of the teacher or school administrators. This ensured that random selection occurred for both studies at each school. All interviews were conducted in the school setting, with each student placed in a private location away from other students and teachers, usually an empty classroom or another available space. The assessor worked one-on-one with each student to first obtain assent at which time each student was informed about the nature of the questions, the availability of counselors onsite, and that if they noted they were in immediate danger, it would be reported. (Each student received a letter requesting their caregiver to come to the school to provide consent for their child to participate in the survey. Only students whose caregiver provided consent participated in the survey.) After this introduction, the FTF method continued with the assessor asking each question directly to the student. With the ACASI method, the assessor instead introduced the tablet and headphones and each individual survey. The protocol did not require same-sex enumerators for this introduction. ACASI recordings were done using a female voice for all surveys. The ACASI study used the exact same surveys that were already being administered as part of the longitudinal study (FTF group); this brief focuses on the administration method rather than the surveys themselves. (The full surveys were adapted from the following surveys: School Climate, Gender Attitudes, SEL, SRGBV, available here: https://shared.rti.org/resources?type[]=283.)

A total of 442 pupils were surveyed by the FTF (longitudinal study) assessors, and 412 pupils were surveyed by the ACASI study assessors. Survey weights were applied to these data. Table 1 shows some basic demographic information on the FTF and ACASI sampled students. The demographic makeup of both samples is similar, though about 50% of ACASI students reported being orphans, whereas only 17% of FTF students were reported as being orphans. This discrepancy could be because the FTF surveys asked parents or guardians of the students about disability and orphan status, whereas ACASI surveys asked students directly about their disability and orphan status. It is possible that the young children did not understand the definition of “orphan” when responding to the question. However, based on the sampling structure as defined above, it is improbable that students randomly sampled from the same schools would have true discrepancies in orphan status. Further cognitive interviewing around these questions could clarify how young children interpreted these terms.

Demographics of Sampled Students
Demographic FTF
n  =  442
[95% CI]
ACASI
n  = 412
[95% CI]
Average age 9.7 years 10.0 years
Percent girls 48.1%
[45.3–50.9]
48.5%
[44.8–51.1]
Percent with disability 3.3%
[1.0–5.5]
2.0%
[0.6–3.3]
Percent orphan 17.1%
[11.9–22.3]
47.3%
[39.8–54.7]
CI = Confidence Interval.

Findings

Research Question 1

Are the data collected from the target surveys using ACASI showing a difference in reporting as compared with data collected using the FTF method?

Research Question 1 is centered on determining whether a difference in reporting is observed between the two survey administration methods. We find that overall reporting for the school climate, gender attitudes, SEL, bullying, and corporal punishment surveys is similar between ACASI and FTF, whereas the sexual violence survey shows drastically increased reporting of experiencing sexual violence under ACASI. We summarized the data collected with each survey into key reporting metrics. The key metrics representing the percentage of responses favorable to (1) a positive school climate, (2) positive views on gender equity, or (3) a strong sense of agency and social awareness were derived for the school climate, gender attitudes, and SEL surveys, respectively. The key metrics of the experiences of violence surveys are the prevalence rates for each of the three forms of violence. Table 2 displays these key metrics for all surveys and both administration types, overall and by sex.† Calculations of key metrics is as follows: percentage of favorable responses = total # of non-missing responses that align with desired outcometotal # of non-missing responses prevalence= total # of pupils that reported experiencing at least ONE act of violencetotal # of pupils  Note: If a pupil did not respond to a question or selected “Don’t Know,” the question was treated as a missing response. Only non-missing responses were used in the calculation of percentage of favorable responses metrics. Missing responses did not affect the calculation of the prevalence metrics as long as the respondent answered at least one item in the survey.

The school climate, gender attitudes, and SEL surveys show similar results when comparing ACASI with FTF. This may be explained by the fact that there is likely less desire to understate the severity of an experience if that experience is not stigmatized or sensitive in nature. Interestingly, a slight increase in the percentage of favorable responses to the gender attitudes survey and a decrease in the percentage of favorable responses to the SEL survey is observed for girls under ACASI, perhaps indicating sex-specific social desirability bias at play. These findings suggest that girls may be less likely to provide authentic responses to questions around confidence, social awareness, and other dimensions of SEL if assessed FTF. The overall bullying and corporal punishment prevalence rates are also very similar for ACASI and FTF—though ceiling effects are being observed for both. We do observe an increase in the prevalence of corporal punishment for boys under ACASI compared with boys in the FTF group. The observed differences in gender attitudes, SEL, and corporal punishment reporting for specific sexes indicate that stigmatized behaviors could be differentiated by sex and highlight the importance of selecting the best method of survey administration.

The starkest findings from this study regarding a difference in reporting are related to the sexual violence survey. A drastic increase in prevalence of sexual violence is observed under ACASI for both boys and girls. About 43% of pupils who completed the sexual violence survey with the FTF administration reported experiencing an act of sexual violence at least once in the past term, whereas 77% of the ACASI group reported this experience. This substantial difference in reporting is supported by the existing literature on ACASI.5,6,10-12,14,24 Based on the literature, we assume that the sexual violence survey exhibits the greatest amount of misreporting because it is likely the most sensitive in this context and is likely subject to social desirability bias.22 Thus, we would expect pupils to respond more openly when they are given privacy and confidentiality through ACASI.7,8,19,21 Figure 1 provides a closer look at reporting for each individual item within the sexual violence survey, suggesting the increase in prevalence is not driven by a single question in the survey.

Key Metrics for FTF and ACASI Sampled Students, by Sex and Overall
Boys Girls Overall
Key Metric FTF
(n  = 223)
[95% CI]
ACASI
(n  = 213)
[95% CI]
p-value FTF
(n  = 219)
[95% CI]
ACASI
(n  = 193)
[95% CI]
p-value FTF
(n  =  442)
[95% CI]
ACASI
(n  = 412)
[95% CI]
p-value
Percentage of favorable responses to the school climate survey 71.7%
[70.1–73.3]
72.4%
[70.5–74.3]
P = 0.550 72.0%
[69.6–74.4]
69.5%
[67.2–71.8]
P = 0.139 71.8%
[70.3–73.3]
71.0%
[69.5–72.5]
P = 0.466
Percentage of favorable responses to the gender attitudes surveya 40.1%
[37.9–42.3]
41.8%
[39.6–44.0]
P = 0.284 43.3%
[40.6–46.0]
47.1%
[44.9–49.3]
P = 0.034 41.7%
[39.9–43.5]
44.3%
[42.7–45.9]
P = 0.031
Percentage of favorable responses to the SEL surveya 68.1%
[65.6–70.6]
68.5%
[65.1–71.9]
P = 0.839 67.3%
[64.5–70.1]
61.0%
[57.3–64.7]
P = 0.010 67.7%
[65.6–69.8]
65.0%
[62.5–67.5]
P = 0.105
Prevalence of bullyingb 95.3%
[92.0–98.6]
97.4%
[94.8–100]
P = 0.331 96.7%
[94.6–98.8]
96.7%
[94.3–99.1]
P = 0.976 96.0%
[94.0–98.0]
97.1%
[95.4–98.8]
P = 0.434
Prevalence of corporal punishmentb 92.5%
[88.5–96.5]
98.4%
[97.0–99.8]
P = 0.007 93.2%
[89.9–96.5]
93.2%
[89.2–97.2]
P = 0.999 92.8%
[90.4–95.2]
95.9%
[93.9–97.9]
P = 0.058
Prevalence of sexual violenceb 47.3%
[39.9–54.7]
81.7%
[75.2–88.2]
P < 0.001 39.0%
[31.5–46.5]
72.7%
[65.4–80.0]
P < 0.001 43.3%
[37.9–48.7]
77.3%
[71.9–82.7]
P < 0.001
CI = Confidence Interval.
Note: Sex was missing for 6 pupils in the ACASI group so these observations are excluded from the sex-disaggregated analyses.
Of these 35 pupils, about 30 pupils also did not receive the gender attitudes and SEL surveys. Therefore, these 30 pupils are not included in the gender attitudes and SEL analyses.
Because of technical issues with the tablet used for the ACASI administration, a total of 35 pupils in the ACASI sample did not receive the experiences of violence surveys. This happened across 14 schools with 1–2 pupils at each school, except for one case where 8 pupils in the same school did not receive these surveys via ACASI because of technical issues. Therefore, these 35 pupils are not included in the experiences of violence analyses.
Research Question 2

Does the factor structure of the target surveys using ACASI conform to that of the target surveys using the FTF data collection?

Item-Level Responses to Sexual Violence Survey for FTF and ACASI Sampled Pupils

To address Research Question 2, we conducted exploratory factor analysis (EFA).‡ EFA was conducted because prior psychometric analysis of these surveys indicated weak factor structures. Since there was weak evidence supporting the exact number of factors present within each survey, we did not deem confirmatory factor analysis as an appropriate test for this study. All EFA conducted for this study used orthogonal rotation methods. Factor analysis aims to reduce the large number of observed variables (e.g., individual survey questions) to a few interpretable latent variables, or factors (e.g., the overarching concepts we are trying to measure), that explain the maximum amount of variation in the data. Running EFA allows us to summarize findings related to the overall concepts rather than findings related to specific items. If factor structures change across different administrations of the same survey, this can mean that different constructs are being measured.

We find that the factor structures under the school climate, gender attitudes, SEL, and experiences of violence surveys for the FTF and ACASI administrations are roughly the same. (A factor loading cut-off of 0.20 was used to determine what items to drop during the EFA.) Figure 2 shows the FTF and ACASI factor structures with labels assigned according to item groupings and the associated root mean square error approximations to provide goodness of fit. (The gender attitudes and SEL surveys under ACASI and the gender attitudes survey under FTF do not meet the recommended levels for goodness of fit25 [RMSEA<0.08].) Though all other surveys show the same factor structure across administration modes, the sexual violence survey under the FTF administration shows a two-factor model whereas the same survey under the ACASI administration shows a one-factor model. This may be explained by the fact that social desirability bias has been shown to affect factor structure,26 making the direct comparison difficult for those surveys that may be susceptible to increased levels of social desirability bias. In fact, factor invariance tests show us that the gender attitudes, corporal punishment, and sexual violence surveys indicate at least metric invariance—factors do not largely differ across administration types. Table 3 provides results of factor invariance tests for each survey. Overall, it is evident that the type of administration does not largely affect the constructs being measured and using ACASI may not alter pupils' understanding of the items.

Factor Structures Present in the Data
Factor Invariance Tests
Survey Invariance Test χ2
(df)
Δχ2
(Δdf)
p-value Decision
School Climate Configural 1420.7
(752)
- - -
Metric 1466.1
(779)
45.4
(27)
0.0147 Reject
Gender Attitudes Configural 194.4
(86)
- - -
Metric 209.5
(95)
15.1
(9)
0.0909 Accept
Scalar 241.2
(104)
31.7
(9)
0.0002 Reject
SEL Configural 1309.4
(550)
- - -
Metric 1357.4
(574)
48
(24)
0.0025 Reject
Experiences of Violence – Bullying Configural 106.2
(54)
- - -
Metric 150.6
(62)
44.4
(8)
< 0.0001 Reject
Experiences of Violence – Corporal Punishment Configural 36.5
(28)
- - -
Metric 42.7
(34)
6.2
(6)
0.4044 Accept
Scalar 51
(40)
8.3
(6)
0.2169 Accept
Strict 80.1
(47)
29.1
(7)
0.0001 Reject
Experiences of Violence – Sexual Violence Configural 98.3
(28)
- - -
Metric 109.9
(34)
11.6
(6)
0.0683 Accept
Scalar 137.9
(40)
28
(6)
< 0.0001 Reject

Overall, it is evident that the type of administration does not largely affect the constructs being measured, and using ACASI may not alter pupils’ understanding of the items.

Research Question 3

Considering the target surveys using the conventional FTF method of data collection as the “focal test,” what is the construct validity of the target surveys using ACASI?

Research Question 3 focuses on construct validity, which is the degree of confidence that an identified construct is well-measured by its associated group of items. The average variance extracted (AVE) is a metric that calculates the amount of variance of a group of items that is captured by a single construct (versus the amount of variance caused by measurement error). A low AVE indicates that the construct cannot successfully capture variance coming from the individual items and therefore may not be a strong representation of those items. A high AVE indicates that the construct can capture much of the variance coming from the individual items, making it a good representation of those items. Generally, AVE values of at least 0.50 are seen as acceptable.27

The AVE values for all factors in the focal tests (i.e., School Climate, Gender Attitudes, SEL, and Experiences of Violence) are smaller than the acceptable 0.50 threshold under both FTF and ACASI. The consistency of low AVE values across administration types is indicative of the tool itself not showing strong construct validity, regardless of administration type. It is important to note that items with factor loadings of at least 0.20 were included in the factor designations. Because AVE calculations are based on factor loading values, increasing this threshold to items with loadings greater than 0.30 or 0.40 would result in higher AVE values.

It is also interesting to observe that the AVE for almost all surveys increases under the ACASI administration compared with FTF, though no survey shows sufficient evidence that construct validity is supported under either administration mode. Table 4 displays the AVE for each factor in the FTF and ACASI surveys. We recommend further survey adaptation and psychometric testing to assess construct validity of these surveys under both administration methods.

AVE for Each Factor of the FTF and ACASI Surveys
Survey Administration Average
Range (0–1.0)
School Climate FTF Factor 1: 0.133
Factor 2: 0.167
ACASI Factor 1: 0.235
Factor 2: 0.413
Gender Attitudes FTF Factor 1: 0.153
Factor 2: 0.192
ACASI Factor 1: 0.305
Factor 2: 0.308
SEL FTF Factor 1: 0.122
ACASI Factor 1: 0.381
Experiences of Violence – Bullying FTF Factor 1: 0.294
ACASI Factor 1: 0.356
Experiences of Violence – Corporal Punishment FTF Factor 1: 0.267
ACASI Factor 1: 0.311
Experiences of Violence – Sexual Violence FTF Factor 1: 0.357
Factor 2: 0.457
ACASI Factor 1: 0.406

Recommendations

Sample Limitations

Because the same exact students were not assessed with both methods, the administration of the survey may not be the only variable factor. However, because the samples of FTF and ACASI students were both chosen at random, these potential sample differences were likely mitigated.

Students who were part of the FTF sample had heard these surveys during the prior year, whereas the ACASI students had not.

Technical issues with tablets sometimes caused interruption in the middle of the ACASI-administered assessment, which produced incomplete results.

The findings from this study provide early evidence, in alignment with the literature, that the ACASI administration may obtain valid and reliable data when used with children to collect sensitive, self-reported data. Factor analysis of our collected data indicates that using ACASI largely does not affect the constructs being measured, though further piloting and testing is recommended to improve construct validity.

Increased responses to the corporal punishment (for boys) and sexual violence surveys administered with ACASI indicates that ACASI can mitigate social desirability bias when collecting such sensitive data from pupils. We also observe a difference for the SEL and gender attitudes surveys when using ACASI with girls, indicating that these surveys could be subject to sex-specific social desirability bias. The findings herein are strong evidence for rejecting the FTF interview method in favor of ACASI for the administration of the experiences of violence survey. The importance of using ACASI when administering surveys of a sensitive nature is clear, but further examination regarding financial and logistical tradeoffs across methods should be considered. Also, surveys should always be piloted, and psychometric analyses should be conducted when administering surveys in new contexts or with different populations. Further methods research should also examine survey administration types that have proven to mitigate response bias because of fear, threat of disclosure, and revicitimization5—a bias category that was not covered under this research. Such methods would include the anonymized list survey9 that should be further explored vis-à-vis ACASI and FTF with pupils in similar contexts.

It is of the utmost importance that the research community take responsibility when testing human subjects by ensuring the most confidential and comfortable survey administration is used. While using ACASI, children can feel more comfortable disclosing their experiences, knowing their privacy is safeguarded. Obtaining data through proper privacy measures, which reduce the potential for bias, is the first step toward determining the full breadth and depth of SRGBV. The data can be used to better inform programming that addresses the experiences and drivers of SRGBV to eradicate violence against children in schools.

Acknowledgments

The authors sincerely thank Dr. Elizabeth Randolph, Chris Cummiskey, Geri Burkholder, Simon King, Amy Mulcahy-Dunn, and the RTI M&E Team at USAID’s Literacy Achievement and Retention Activity for their support through the original piloting of this work in 2017 and scale-up in 2019. This work is a result of a team effort. This research was funded by RTI International through an Internal Research and Development (IR&D) award.

RTI Press Associate Editor: Jeffrey Rosen

References

1RTI International. Literature review on school-related gender-based violence: How it is defined and studied. Washington (DC): US Agency for International Development; 2016. https://pdf.usaid.gov/pdf_docs/PBAAF483.pdf
2United Nation’s Girls’ Education Initiative (UNGEI). A whole school approach to prevent school-related gender-based violence: minimum standards and monitoring framework. https://www.ungei.org/publication/whole-school-approach-prevent-school-related-gender-based-violence-1
3Ybarra ML, Langhinrichsen-Rohling J, Friend J, Diener-West M. Impact of asking sensitive questions about violence to children and adolescents. J Adolesc Health 2009;45(5):499-507. 10.1016/j.jadohealth.2009.03.00919837357
4Miller PH, Baxter SD, Royer JA, Hitchcock DB, Smith AF, Collins KL et al. Children’s social desirability: Effects of test assessment mode. Pers Individ Dif 2015;83:85-90. . 10.1016/j.paid.2015.03.03925870465
5Tourangeau R, Yan T. Sensitive questions in surveys. Psychol Bull 2007;133(5):859–83. 10.1037/0033-2909.133.5.85917723033
6Potdar R, Koenig MA. Does Audio-CASI improve reports of risky behavior? Evidence from a randomized field trial among young urban men in India. Stud Fam Plann 2005;36(2):107–16. 10.1111/j.1728-4465.2005.00048.x15991648
7Hewett PC, Mensch BS, Ribeiro MC, Jones HE, Lippman SA, Montgomery MR et al. Using sexually transmitted infection biomarkers to validate reporting of sexual behavior within a randomized, experimental evaluation of interviewing methods. Am J Epidemiol 2008;168(2):202–11. 10.1093/aje/kwn11318525081
8Falb K, Tanner S, Asghar K, Souidi S, Mierzwa S, Assazenew A et al. Implementation of Audio-Computer Assisted Self-Interview (ACASI) among adolescent girls in humanitarian settings: feasibility, acceptability, and lessons learned. Confl Health 2016;10(1):32. 10.1186/s13031-016-0098-128053657
9Cullen C. (2020). Method matters: underreporting of intimate partner violence in Nigeria and Rwanda. World Bank Policy Research Working Paper No. 9274. Available at SSRN: https://ssrn.com/abstract=3624515
10Kane JC, Murray LK, Sughrue S, DeMulder J, Skavenski van Wyk S, Queenan J et al. Process and implementation of Audio Computer Assisted Self-Interviewing (ACASI) assessments in low resource settings: a case example from Zambia. Glob Ment Health (Camb) 2016;3:e24. 10.1017/gmh.2016.1928596892
11Kelly CA, Hewett PC, Mensch BS, Rankin JC, Nsobya SL, Kalibala S et al. Using biomarkers to assess the validity of sexual behavior reporting across interview modes among young women in Kampala, Uganda. Stud Fam Plann 2014;45(1):43–58. 10.1111/j.1728-4465.2014.00375.x24615574
12Langhaug LF, Sherr L, Cowan FM. How to improve the validity of sexual behaviour reporting: systematic review of questionnaire delivery modes in developing countries. Trop Med Int Health 2010;15(3):362–81. 10.1111/j.1365-3156.2009.02464.x20409291
13Plummer ML, Wight D, Ross DA, Balira R, Anemona A, Todd J et al. Asking semi-literate adolescents about sexual behaviour: the validity of assisted self-completion questionnaire (ASCQ) data in rural Tanzania. Trop Med Int Health 2004;9(6):737–54. 10.1111/j.1365-3156.2004.01254.x15189466
14van der Elst EM, Okuku HS, Nakamya P, Muhaari A, Davies A, McClelland RS et al. Is audio computer-assisted self-interview (ACASI) useful in risk behaviour assessment of female and male sex workers, Mombasa, Kenya? PLoS One 2009;4(5):e5340. 10.1371/journal.pone.000534019412535
15Ghanem KG, Hutton HE, Zenilman JM, Zimba R, Erbelding EJ. Audio computer assisted self interview and face to face interview modes in assessing response bias among STD clinic patients. Sex Transm Infect 2005;81(5):421–5. 10.1136/sti.2004.01319316199744
16Jaspan HB, Flisher AJ, Myer L, Mathews C, Seebregts C, Berwick JR et al. Brief report: methods for collecting sexual behaviour information from South African adolescents—a comparison of paper versus personal digital assistant questionnaires. J Adolesc 2007;30(2):353–9. 10.1016/j.adolescence.2006.11.00217187853
17Beauclair R, Meng F, Deprez N, Temmerman M, Welte A, Hens N et al. Evaluating audio computer assisted self-interviews in urban South African communities: evidence for good suitability and reduced social desirability bias of a cross-sectional survey on sexual behaviour. BMC Med Res Methodol 2013;13(1):11. 10.1186/1471-2288-13-1123368888
18Stark L, Sommer M, Davis K, Asghar K, Assazenew Baysa A, Abdela G et al. Disclosure bias for group versus individual reporting of violence amongst conflict-affected adolescent girls in DRC and Ethiopia. PLoS One 2017;12(4):e0174741. 10.1371/journal.pone.017474128376108
19Adebajo S, Obianwu O, Eluwa G, Vu L, Oginni A, Tun W et al. Comparison of audio computer assisted self-interview and face-to-face interview methods in eliciting HIV-related risks among men who have sex with men and men who inject drugs in Nigeria. PLoS One 2014;9(1):e81981. 10.1371/journal.pone.008198124416134
20Population Council. (2019). Audio Computer-Assisted Self-Interviewing. New York, NY: Population Council. https://www.popcouncil.org/research/audio-computer-assisted-self-interviewing-acasi
21Mierzwa S, Souidi S, Akello C, Etima J, Ssebagala R, Nolan M et al. Case study: converting paper-based case report forms to an electronic format (e-CRF) with ACASI self-report integration. Online J Public Health Inform 2017;9(3):e198. 10.5210/ojphi.v9i3.792929403577
22Ministry of Gender, Labour, and Social Development, Republic of Uganda. Uganda violence against children survey: findings from a national survey. 2018. https://www.unicef.org/uganda/media/2156/file/Violence%20Against%20Children%20Survey%202018.pdf
23RTI International. Survey of Student Experiences of SRGBV. Uganda Literacy Achievement and Retention Activity. Research Triangle Park (NC): RTI International; 2018., https://shared.rti.org/content/survey-student-experiences-school-related-gender-based-violence-srgbv
24Hewett PC, Mensch BS, Erulkar AS. Consistency in the reporting of sexual behaviour by adolescent girls in Kenya: a comparison of interviewing methods. Sex Transm Infect. 2004;80(Suppl 2):ii43-8. .10.1136/sti.2004.013250
25Cangur S, Ercan I. Comparison of model fit indices used in structural equation modeling under multivariate normality. J Mod Appl Stat Methods 2015;14(1):14. 10.22237/jmasm/1430453580
26Navarro-González D, Lorenzo-Seva U, Vigil-Colet A. How response bias affects the factorial structure of personality self-reports. Psicothema 2016 Nov;28(4):465–70.27776617
27Civelek ME. Essentials of structural equation modeling. 2018. Zea E-Books. 64. https://digitalcommons.unl.edu/zeabook/64 10.13014/K2SJ1HR5

Click cover to download publication

Keep Exploring

  • icon-externallink-blue Created with Sketch.

    HTML version [Scholastica]

Sustainable Development Goals

  • Goal 5: Gender Equality
  • Goal 16: Peace, Justice, and Strong Institutions

Contact

To contact an author or seek permission to use copyrighted content, contact our editorial team

  • +1 919 541 6490
  • rtipress@rti.org

Meet the Experts

View All Experts

Julianne Norman

Lauren Edwards

Related Publications

View All Press
OCCASIONAL PAPER

Bringing an equity-centered framework to research

OCCASIONAL PAPER

The Preschool Entitlement

OCCASIONAL PAPER

Culturally informed community engagement

RESEARCH REPORT

Substance misuse prevention program attendance

RESEARCH BRIEF

COVID-19’s impact on clinical research

OCCASIONAL PAPER

Differentiated instruction in multigrade preprimary classrooms in Kenya

OCCASIONAL PAPER

The potential for solar-powered groundwater irrigation in Sub-Saharan Africa

OCCASIONAL PAPER

Social determinants of health

Recent Publications

View All Press
OCCASIONAL PAPER

Bringing an equity-centered framework to research

OCCASIONAL PAPER

The Preschool Entitlement

OCCASIONAL PAPER

Culturally informed community engagement

RESEARCH REPORT

Substance misuse prevention program attendance

RTI Logo
Partner With Us
  • US Government
  • Commercial
  • Foundations & Associations
  • Multilateral Donors
  • Universities
  • Suppliers
Site
  • Privacy Policy
  • Security Policy
  • Site Map
  • Terms of Use
  • Accessibility
  • Contact Us
Contact Us
Facebook Icon Twitter Icon Instagram Icon YouTube Icon Linkedin Icon
delivering the promise of science
for global good
RTI Health Solutions RTI Innovation Advisors RTI Health Advance

© 2023 RTI International. RTI International is a trade name of Research Triangle Institute. RTI and the RTI logo are U.S. registered trademarks of Research Triangle Institute.