Survey Research Capabilities


Data Collection and Management

Data Collection

We have more than 45 years of experience providing our clients with a full range of survey data collection services from one-on-one personal interviewing to leading-edge, computer-assisted methodologies. Our data collection activities and expertise include the following:

  • Field surveys and personal interviews -- Using a vast, geographically distributed national network of over 100 supervisors and 7,000 interviewers for field work
  • Telephone surveys -- conducting telephone interviews with the latest advances in computer–telephony integration and data networking, using VoIP technology
  • Mail surveys -- Conducting custom mail surveys of diverse populations
  • Computer-assisted interviewing -- Using and tailoring leading-edge, commercial software and developing applications to facilitate survey interviews
  • Mixed-mode data collection -- Mixing and balancing different methods of collecting data to improve response rates and cost efficiencies for special populations or projects
  • Focus groups -- Assembling selected groups of individuals to evaluate the effectiveness of a given data collection method
  • Sensitive topics and behaviors -- Conducting research on sensitive subject matter such as drugs, sex, and abortion
  • Tracing operations -- Customizing and conducting follow-up surveys by tracing hard-to-find interviewees over time

Using a wide range of data collection tools and systems, our researchers and technicians gather and process information for government, industry, and education alike. We use both standard and state-of-the-art instruments and apply them to meet the unique requirements of every client.

Data Collection Systems

We have developed complex data collection systems to address unique needs of studies, such as surveys with multiple stages (e.g., screening, selection, and interviewing), surveys in which the same sample members may be contacted through multiple modes, the collection of data for the same sample member from multiple sources (e.g., interviews with children, their parents or guardians, and their caseworkers), the abstraction of records for consenting adults, and the collection and processing of biospecimens in addition to the survey interview. To this end, various data collection systems have been developed by us, such as Hatteras -- a unified system that allows collection of data for the same study through a Web survey, telephone, and face-to-face interviewing.

More Information: Data Capture and Management

Data Management

How data are managed during data collection, processed, and compiled into analytical data sets is essential to the ability to meet the study's objectives. Data are reviewed early during the data collection period to verify that no anomalies are present. At the end of data collection, survey data are examined for inconsistencies; variables are recoded, labeled, and, in some studies, compiled from multiple sources and various data structures into analytic datasets; and documentation is created.

More Information: Data Capture and Management


Methodologies

Study Design

With evolving challenges in the field of survey methodology and improved understanding of survey errors, there is a need for increasingly complex study designs. Studies conducted by us have addressed these challenges using innovative approaches, such as implementing multiple frames, multiple modes, and planning and allowing for changes in the design during data collection. Systems have been developed to support more complex designs, such as the development of Hatteras -- a unified system for data collection through multiple modes. We take pride in having the capabilities, statistical expertise, a wide range of substantive expertise, and experience to construct designs that are specific to the circumstances and objectives of a particular study.

In addition, RTI is recognized as a leader in the development of new approaches to collecting survey data. Our survey methodologists are expert in designing experimental research that will address questions regarding the most appropriate approach to studying a particular topic within the recognized cost and schedule constraints.

More Information: Experimental Design, Study Design

Sampling

Sampling varies greatly across studies, from systematic selection from listed samples, to multiframe, stratified, clustered, multiphase sampling designs. Designs are constructed for individual studies, optimizing for the intended analysis and cost. Sampling procedures to improve coverage of the population, through incorporation of multiple frames such as landline and cell phone frames, and alternative sources of frames, such as U.S. postal addresses, are continually being developed and improved.

Considerable development has been undertaken by us in small area estimation, developing software that provides small area estimates with substantially reduced sample size, while accounting for complex survey design features.

More Information: Sampling, Small Area Estimation

Quality Control

Quality control plans and procedures are developed for individual studies across the different stages of the survey process. These include the verification of sampling procedures, systematic review of instruments during their construction using the questionnaire appraisal system, pretesting of programmed instruments, examination of data after the start of data collection, supervising and monitoring of interviewers, consistent implementation of training protocols, interview verification, and the use of computer-assisted recorded interviewing (CARI).

Weighting and Imputation

In addressing current challenges, surveys become increasingly complex, such as through the use of multiple sampling frames and subsampling during data collection. Simply accounting for the initial selection probabilities is often insufficient. We have developed weights that account for different selection probabilities across sample members, as well as for multiple selection probabilities for individuals.

While attention to unit nonresponse is reflected in the survey weights, we have developed methods that reduce the loss in precision due to weighting, while maintaining the adjusted survey point estimates. Through the use of General Exponential Models, multiple weights are calibrated simultaneously to achieve efficient analytic survey weights.

Our researchers are familiar with imputation techniques, from rule-based deterministic imputation, to hotdeck, and regression-based multiple imputation. Differences exist between imputation approaches, and based on the analytic objectives, the most appropriate methods are recommended for individual studies.

More Information: Weighting, Imputation, Nonsampling Error Analysis

Analysis

We have extensive experience in reporting results, such as descriptive statistics, needed to inform client decisions on government policies. Statistics and their variance estimates are estimated to incorporate the study design and postsurvey adjustments. The SUDAAN software has been developed at RTI to allow for a multitude of possible complex survey designs, providing corrected survey estimates.

Many research questions can not be answered through descriptive statistics. Modeling of survey data, such as to examine relationships between survey variables or classify respondents is also performed by us, through methods such as structural equations modeling, hierarchical linear modeling, latent class analysis, item response theory, and multidimensional scaling.

More Information: Small Area Estimation, Nonsampling Error Analysis, SUDAAN, Longitudinal Analysis, Qualitative Analysis, Psychometrics

Cross-Cultural Issues and Translation

Conducting surveys in different languages and across ethnic populations, both in the U.S. and in multiple countries, complicates the study design and can introduce new sources of error in survey estimates. Our survey methodologists and translators conduct research to inform questionnaire construction and implementation that minimizes survey error that is introduced through translation, cultural differences that alter the meaning of questions, and other sources. Understanding differences in cognitive processes across cultures as well as pretesting instruments in different languages is further assisted by our dedicated facilities, such as a cognitive laboratory and focus group rooms.

Contact: Marjorie Hinsdale-Shouse

Human Subjects

We conduct all research involving human subjects in accordance with federal regulations (45 CFR 46 and 21 CFR 50 and 56). We hold a Federalwide Assurance (FWA) from the Department of Health and Human Services' Office for Human Research Protections (OHRP). We assume full responsibility for performing all research involving human subjects (regardless of funding agency) in accordance with that Assurance, including compliance with all federal, state, and local laws as they may relate to the research.

We have created three institutional review boards (IRBs), which review all research involving human subjects that is conducted by RTI researchers. The IRBs are responsible for ensuring that the rights of research participants who take part in RTI studies are protected. The IRB chairs and members for the three IRBs are selected from a variety of RTI disciplines and from the surrounding community to ensure proper review of human research activities.

More Information: Human Research Protection

Questionnaire Design and Evaluation

Design and development of survey instruments are core activities across all research areas at RTI. Testing and refining instruments prior to implementation not only minimizes measurement error but also helps to control costs, minimize respondent burden, and maintain project schedules. Our researchers have expertise in a wide variety of instrument development sciences and techniques. Our research facilities are equipped with the latest technologies for conducting evaluations of the wide range of instruments used in survey data collection today, including interviews conducted using laptop computers, event history calendars, time and activities diaries, and self-administered paper questionnaires.

We offer expertise in questionnaire design and evaluation, including usability testing, cognitive pretesting, and psychometric evaluation.

Usability Testing

Our staff are skilled at designing and conducting usability tests. We have conducted usability tests of computer-assisted personal interviewing (CAPI) and audio computer-assisted self-interviewing (ACASI) instruments. Our cognitive and usability testing laboratory is fully equipped for testing the usability of all types of computer-based applications.

Usability testing allows in-depth evaluation of how respondents and interviewers interact with computerized questionnaires. For example, we have the resources to evaluate

  • Effect of screen layout on willingness to participate
  • Respondent comprehension and attentiveness to consent screens and privacy policy screens
  • Respondent perceptions of data security

The usability lab includes cameras that can capture the user's hands on the computer keyboard and his/her facial expressions. It is also equipped with a video feed that captures the image on the computer screen. These images are time synchronized to facilitate analysis. Audio microphones that capture the voices of respondents and interviewers during the interviews are also available. Behavior coding software can be used to create files suitable for quantitative analysis. A portable usability lab, suitable for traveling to a respondent's home or more remote locations, is also available.

Cognitive Pretesting

We have considerable expertise in cognitive interviewing and cognitive appraisal methods. These pretesting methods provide important tools for examining the thought processes that affect the quality of answers provided to survey questions. Intensive cognitive interviewing of individuals representing a population to be surveyed can provide valuable information about how respondents view the question-answer process, access short-term and long-term memory, and process complex terminology.

The results of these investigations are used to

  • Improve question wording and flow
  • Evaluate understanding of the instrument among special populations like children, non-native English speakers, and the elderly
  • Determine the appropriateness of particular interview administration methods

We also excel in cognitive appraisal coding. This approach makes extensive use of technical review by design experts. Our questionnaire appraisal system (QAS-2003) identifies important features of questionnaire items that either enhance or interfere with response accuracy. The coding scheme applies a taxonomy of problem categories -- such as comprehension, task definition, information retrieval, judgment, and response generation -- that are used to identify possible revisions in item wording, response wording, and questionnaire formats. In addition, the system also identifies potential problems due to cross-cultural or translation issues.

Psychometric Evaluation

Our expert psychometricians have designed studies to assess the psychometric properties of questionnaires, such as reliability, validity, test dimensionality, and differential item functioning. We analyze data from the pretesting phase of questionnaire development and after a study has been completed.

Our psychometricians use many advanced techniques, including item response theory (IRT), structural equation modeling (SEM), and latent growth curve analysis. We regularly consult with staff from the neighboring L.L. Thurstone Psychometric Laboratory at the University of North Carolina for specialized analysis of unique psychometric problems.

More Information: Psychometrics, Qualitative Analysis


Security

Data Security

Data security plans are created and adhered to throughout the life of a project. Protections are in place for transmitting data to and from interviewers in the field and for securing data onsite at RTI. Our information technology infrastructure, which is in compliance with all requirements as set forth in Federal Information Processing Standards Act (FIPS) 199, represents a notable strength for RTI. The requirements to implement IT security for research projects at the rigorous FIPS 199 moderate level are currently beyond the capabilities of many corporations, universities, and government agencies. We have networks that meet NIST standards for both low and moderate risk, allowing us to provide the appropriate level of security for the information. We have created a self-contained enhanced security network (ESN) to address FIPS 199 security requirements for moderate risk data (i.e. PHI and PII).

More Information: Data and Network Security

Disclosure Avoidance and Data Delivery

Extensive care is taken to protect respondent confidentiality when releasing survey data. Survey data is processed to remove identifying information, and models have been estimated to inform the possibility of identifying individuals in a dataset with a particular set of available variables. There are multiple methods of protecting respondents' confidentiality in a dataset that are evaluated in the selection of procedures for a particular study. When identifying information is transferred to a client, additional measures are followed, such as the separation of identifying information from survey data, encryption, and password protection.

More Information: Statistical Disclosure Control