Using ACASI to Measure Gender-Based Violence in Ugandan Primary Schools

Key Points • Audio Computer-Assisted Self-Interview (ACASI) provides respondents with privacy and confidentiality. It has been used by researchers administering surveys of a sensitive nature and is widely regarded as a useful tool in reducing social desirability bias in responses to sensitive surveys. However, ACASI has never been used to measure schoolrelated gender-based violence (SRGBV), a survey normally administered face to face (FTF), among primary school–aged children.


Introduction
Incidences of school-related gender-based violence (SRGBV) are common in schools around the world. SRGBV can include "acts or threats of physical, sexual, or psychological violence or abuse that is based on gender stereotypes or that targets students on the basis of their sex, sexuality, or gender identities. " 1 Incidences of violence for both girls and boys can occur on the way to and from school or while at school. Three broad categories of SRGBV include bullying and non-sexual harassment, corporal punishment, and sexual violence or harassment. 1 Research, minimum standards, and monitoring frameworks, like that of the United Nations Girls Education Initiative's (UNGEI's) Whole School Approach 2 have helped establish greater awareness of the extent of SRGBV and provide concrete steps for policy makers and implementers to address SRGBV. However, the research community is still just beginning to understand both the pervasiveness of SRGBV and its psychological and academic effects on students. Existing prevalence data, intended to inform donor programming and policy, are misrepresented for three reasons: 1. SRGBV survey questions are especially sensitive.
Responding can prove emotionally distressing to the respondent. 3 Such sensitivity also poses ethical challenges and considerations that can make it difficult to ensure that the principle of "do no harm" is kept front and center of research studies. 2 2. Social desirability bias-an interest in responding to survey questions that align with cultural values and expectationscan cause respondents to exaggerate responses that align with culturally condoned behaviors or valued experiences or to under-report stigmatized acts or experiences. 4 As a result, data reflect experiences or personal attributes perceived to be more socially accepted or desirable rather than the respondent's truthful experiences.
3. There are no right or wrong answers, but there are true and untrue answers. The existing literature argues that questions of a sensitive nature (i.e., questions subject to social desirability bias) threaten response accuracy and lead to systematic reporting (i.e., exaggerating socially desirable experiences and behaviors and underreporting less desirable ones). Tourangeau and Yan even argue the more sensitive the question, the greater the likelihood of misreporting. 5 Although researchers can prepare to collect data on such sensitive topics by becoming familiar with the social norms and drivers of stigma in the context, it remains difficult to know to what degree social desirability bias will affect selfreported data. The literature on this topic (although results are mixed based on context and study design) argues that survey administration mode can address issues of misreported data caused by social desirability bias. 5,6,7,[8][9][10][11][12][13][14] This brief contributes to the body of literature on survey administration methods for SRGBV by providing early evidence on how to measure SRGBV prevalence among children. Specifically, this research compares Audio Computer-Assisted Self-Interview (ACASI), a confidential, self-report method of data collection, with the Face-to-Face (FTF) administration method across four surveys with varying degrees of sensitivity with Primary Grade 3 pupils in Uganda. This research also includes an analysis of the factor structure and construct validity of data collected using the two administration types.

Background
In contrast to the conventional FTF interview method, ACASI presents survey questions in an audio format through headphones-providing the respondent with true privacy. While completing the survey, respondents select their choices through the tablet without interacting with a data collector.
ACASI gives the respondent privacy, which reduces social desirability bias and allows the researcher to place confidence in the quality of data collected. 5 A vast amount of literature exists on the use of ACASI in collecting data from HIVinfected patients, on sexual behavior and drug use, and on intimate partner violence. 6,7,[15][16][17][18][19][20] Respondents who use ACASI tend to report their experiences differently than those using the traditional FTF method when the experiences may be sensitive, taboo, or socially undesirable. 8,21 At the time of writing this brief, no studies have tested the effectiveness of ACASI compared to FTF when administering SRGBV or school-centered surveys, especially with a population as young as 10 years of age in low-and middle-income countries. The study described in this brief addresses this gap in the literature on ACASI.
Rates of violence against children is particularly high in Uganda, with rates of physical violence among 13-17-yearold children ranging between 44% for girls and 59% for boys. 22 In 2018, RTI International piloted the use of ACASI in 12 Ugandan public primary schools with Primary Class 5 (or P5; mean age = 11.9 years) and P7 (mean age = 13.8 years) students. Survey forms used for this study were adapted from RTI's Survey of Student Experiences of SRGBV. 23  Results from the 2018 ACASI pilot in 12 schools suggested that ACASI could hold promise for reducing social desirability bias and eliciting more authentic responses. In 2019, we scaled up this study to include a randomized sample of students in 40 Ugandan public primary schools with a younger population of P3 students (mean age = 10 years) and a suite of instruments related to SRGBV and school-centered dimensions.

Methodology
We visited 40 primary schools for the FTF and ACASI data collection in 2019. (Note that the survey and research protocol were approved by RTI's Institutional Review Board and Uganda's The AIDS Support Organization Research Ethics Committee.) These schools were preselected as the random sample of control schools under the US Agency for International Development (USAID)/Uganda Literacy Achievement and Retention Activity longitudinal study in 2018. Under this existing study, assessor teams visit the same schools every year, and the teams use the FTF method to assess students with surveys on school climate, gender attitudes, social-emotional learning (SEL), and experiences of violence. The ACASI administration included the same school climate, gender attitude, SEL, and experiences of violence surveys so that surveys with varying levels of sensitivity could be used to compare the data collected using both ACASI and FTF administration. The second round of data collection for the longitudinal study was conducted in July 2019, during which the ACASI study took place. ACASI assessor teams visited the same schools on the same day as the FTF teams. Because the 2019 data collection was the second occasion for the longitudinal study, 12 pupils in each school had already been selected the year before to take part in the FTF longitudinal study. Therefore, one day before the day of data collection, one member of the research team sampled 12 P3 students in each school out of those who were not already part of the longitudinal sample. * All interviews were conducted in the school setting, with each student placed in a private location away from other students and teachers, usually an empty classroom or another available space. The assessor worked oneon-one with each student to first obtain assent at which time each student was informed about the nature of the questions, the availability of counselors onsite, and that if they noted they were in immediate danger, it would be reported. (Each student received a letter requesting their caregiver to come to the school to provide consent for their child to participate in the survey. Only students whose caregiver provided consent participated in the survey.) After this introduction, the FTF method continued with the assessor asking each question directly to the student. With the ACASI method, the assessor instead introduced the tablet and headphones and each individual survey. The protocol did not require same-sex enumerators for this introduction. ACASI recordings were done using a female voice for all surveys. The ACASI study used the exact same surveys that were already being administered as part of the longitudinal study (FTF group); this brief focuses on the administration method rather than the surveys themselves. (The full surveys were adapted from the following surveys: School Climate, Gender Attitudes, SEL, SRGBV.) A total of 442 pupils were surveyed by the FTF (longitudinal study) assessors, and 412 pupils were surveyed by the ACASI study assessors. Survey weights were applied to these data. Table 1 shows some basic demographic information on the FTF and ACASI sampled students. The demographic makeup of both samples is similar, though about 50% of ACASI students reported being orphans, whereas only 17% of FTF students were reported as being orphans. This discrepancy could be because the FTF surveys asked parents or guardians of the students about disability and orphan status, whereas ACASI surveys asked students directly about their disability and orphan status. It is possible that the young children did not understand the definition of "orphan" when responding to the question. However, based on the sampling structure as defined above, it is improbable that students randomly sampled from the same schools would have true discrepancies in orphan status. Further cognitive interviewing around these questions could clarify how young children interpreted these terms.

Research Question 1
Are the data collected from the target surveys using ACASI showing a difference in reporting as compared with data collected using the FTF method?
Research Question 1 is centered on determining whether a difference in reporting is observed between the two survey administration methods. We find that overall reporting for the school climate, gender attitudes, SEL, bullying, and corporal punishment surveys is similar between ACASI and FTF, whereas the sexual violence survey shows drastically increased reporting of experiencing sexual violence under ACASI. We summarized the data collected with each survey into key reporting metrics. The key metrics representing the percentage of responses favorable to (1) a positive school climate, (2) positive views on gender equity, or (3) a strong sense of agency and social awareness were derived for the * Similar to the sampling procedure for the longitudinal study in 2018, this was done by forming two lines of P3 boys and P3 girls present on the day before data collection, counting the total number of boys and number of girls in each line, and dividing by 8 to calculate a jump number for each sex. That jump number was used to sample 8 boys and 8 girls in each school, which included 2 replacements per group in case caregivers or pupils themselves did not show up to the school the following day for data collection. Lines were formed randomly by the assessor teams without influence of the teacher or school administrators. This ensured that random selection occurred for both studies at each school.
school climate, gender attitudes, and SEL surveys, respectively.
The key metrics of the experiences of violence surveys are the prevalence rates for each of the three forms of violence.  Note: Sex was missing for 6 pupils in the ACASI group so these observations are excluded from the sex-disaggregated analyses.
a Of these 35 pupils, about 30 pupils also did not receive the gender attitudes and SEL surveys. Therefore, these 30 pupils are not included in the gender attitudes and SEL analyses.
b Because of technical issues with the tablet used for the ACASI administration, a total of 35 pupils in the ACASI sample did not receive the experiences of violence surveys. This happened across 14 schools with 1-2 pupils at each school, except for one case where 8 pupils in the same school did not receive these surveys via ACASI because of technical issues. Therefore, these 35 pupils are not included in the experiences of violence analyses. † Calculations of key metrics is as follows: prevalence = total # of pupils that reported experiencing at least ONE act of violence total # of pupils percentage of favorable responses = total # of non-missing responses that align with desired outcome total # of non-missing responses Note: If a pupil did not respond to a question or selected "Don't Know, " the question was treated as a missing response. Only non-missing responses were used in the calculation of percentage of favorable responses metrics. Missing responses did not affect the calculation of the prevalence metrics as long as the respondent answered at least one item in the survey.
violence at least once in the past term, whereas 77% of the ACASI group reported this experience. This substantial difference in reporting is supported by the existing literature on ACASI. 5,6,[10][11][12]14,24 Based on the literature, we assume that the sexual violence survey exhibits the greatest amount of misreporting because it is likely the most sensitive in this context and is likely subject to social desirability bias. 22 Thus, we would expect pupils to respond more openly when they are given privacy and confidentiality through ACASI. 7,8,19,21 Figure 1 provides a closer look at reporting for each individual item within the sexual violence survey, suggesting the increase in prevalence is not driven by a single question in the survey.

Research Question 2
Does the factor structure of the target surveys using ACASI conform to that of the target surveys using the FTF data collection?
To address Research Question 2, we conducted exploratory factor analysis (EFA). ‡ Factor analysis aims to reduce the large number of observed variables (e.g., individual survey questions) to a few interpretable latent variables, or factors (e.g., the overarching concepts we are trying to measure), that explain the maximum amount of variation in the data. Running EFA allows us to summarize ‡ EFA was conducted because prior psychometric analysis of these surveys indicated weak factor structures. Since there was weak evidence supporting the exact number of factors present within each survey, we did not deem confirmatory factor analysis as an appropriate test for this study. All EFA conducted for this study used orthogonal rotation methods.
findings related to the overall concepts rather than findings related to specific items. If factor structures change across different administrations of the same survey, this can mean that different constructs are being measured.
We find that the factor structures under the school climate, gender attitudes, SEL, and experiences of violence surveys for the FTF and ACASI administrations are roughly the same. (A factor loading cut-off of 0.20 was used to determine what items to drop during the EFA.) Figure 2 shows the FTF and ACASI factor structures with labels assigned Though all other surveys show the same factor structure across administration modes, the sexual violence survey under the FTF administration shows a two-factor model whereas the same survey under the ACASI administration shows a onefactor model. This may be explained by the fact that social desirability bias has been shown to affect factor structure, 26 making the direct comparison difficult for those surveys that may be susceptible to increased levels of social desirability bias. In fact, factor invariance tests show us that the gender attitudes, corporal punishment, and sexual violence surveys indicate at least metric invariance-factors do not largely differ across administration types. Table 3 provides results of factor invariance tests for each survey. Overall, it is evident that the type of administration does not largely affect the constructs being measured and using ACASI may not alter pupils' understanding of the items.

Figure 1. Item-Level Responses to Sexual Violence Survey for FTF and ACASI Sampled Pupils
Overall, it is evident that the type of administration does not largely affect the constructs being measured, and using ACASI may not alter pupils' understanding of the items.

Research Question 3
Considering the target surveys using the conventional FTF method of data collection as the "focal test, " what is the construct validity of the target surveys using ACASI?
Research Question 3 focuses on construct validity, which is the degree of confidence that an identified construct is well-measured by its associated group of items. It is also interesting to observe that the AVE for almost all surveys increases under the ACASI administration compared with FTF, though no survey shows sufficient evidence that construct validity is supported under either administration mode. Table 4 displays the AVE for each factor in the FTF and ACASI surveys. We recommend further survey adaptation and psychometric testing to assess construct validity of these surveys under both administration methods.

Sample Limitations
Because the same exact students were not assessed with both methods, the administration of the survey may not be the only variable factor. However, because the samples of FTF and ACASI students were both chosen at random, these potential sample differences were likely mitigated.
Students who were part of the FTF sample had heard these surveys during the prior year, whereas the ACASI students had not.
Technical issues with tablets sometimes caused interruption in the middle of the ACASI-administered assessment, which produced incomplete results.
The findings from this study provide early evidence, in alignment with the literature, that the ACASI administration may obtain valid and reliable data when used with children to collect sensitive, self-reported data. Factor analysis of our collected data indicates that using ACASI largely does not affect the constructs being measured, though further piloting and testing is recommended to improve construct validity.
Increased responses to the corporal punishment (for boys) and sexual violence surveys administered with ACASI indicates that ACASI can mitigate social desirability bias when collecting such sensitive data from pupils. We also observe a difference for the SEL and gender attitudes surveys when using ACASI with girls, indicating that these surveys could be subject to sex-specific social desirability bias. Further research on reporting of SEL competencies and gender attitudes and norms across survey administration modes is warranted. The findings herein are strong evidence for rejecting the FTF interview method in favor of ACASI for the administration of the experiences of violence survey. The importance of using ACASI when administering surveys of a sensitive nature is clear, but further examination regarding financial and logistical tradeoffs across methods should be considered. Also, surveys should always be piloted, and psychometric analyses should be conducted when administering surveys in new contexts or with different populations. Further methods research should also examine survey administration types that have proven to mitigate response bias because of fear, threat of disclosure, and revicitimization 5 -a bias category that was not covered under this research. Such methods would include the anonymized list survey 9 that should be further explored vis-à-vis ACASI and FTF with pupils in similar contexts.
It is of the utmost importance that the research community take responsibility when testing human subjects by ensuring the most confidential and comfortable survey administration is used. While using ACASI, children can feel more comfortable disclosing their experiences, knowing their privacy is safeguarded. Obtaining data through proper privacy measures, which reduce the potential for bias, is the first step toward determining the full breadth and depth of SRGBV. The data can be used to better inform programming that addresses the experiences and drivers of SRGBV to eradicate violence against children in schools.