Standard measurement protocols for pediatric development research in the PhenX Toolkit

PhenX Project Overview

In 2007, the National Human Genome Research Institute (NHGRI) recognized that identifying and promoting a common currency of standard measures would facilitate data sharing and cross-study analyses and potentially increase the scientific impact of individual studies.⁵ The goal for the resultant PhenX Toolkit is to address a broad range of research topics or domains, with a relatively small number of recommended measurement protocols (~15) per domain, to facilitate investigators’ selection of protocols for their study designs. In general, the protocols in the Toolkit are well established, recommended by experts in the scientific community, and low burden in terms of cost, time, and logistic complexity for both investigators and participants. The Toolkit includes protocols in the languages in which they are available from the source; for example, when the source provides the Toolkit with the protocol in Spanish and Chinese, it is made available to users. All Toolkit protocols are identified by WGs of experts in the relevant research domain.

The PhenX Steering Committee (SC) provides ongoing, overarching guidance to the PhenX project, prioritizes the research domains for the Toolkit, and establishes the initial scope of each domain. PhenX WGs operate within the guidance provided by the SC, including established criteria to inform the selection of protocols to be included in the PhenX Toolkit. For a detailed discussion of SC guidance and the PhenX consensus process, see Maiese et al.⁴

PhenX Pediatric Development Working Group

In 2018, the PhenX SC prioritized the addition of a PhenX Pediatric Growth and Development domain to address the need for a set of well-established research protocols for pediatric research. Upon initial review of existing PhenX Toolkit measurement protocols, the SC determined that the Toolkit had robust content on pediatric growth (e.g., anthropometric protocols for measuring the height, weight, and other growth characteristics of a child). Thus, the SC decided on a more focused domain for Pediatric Development. The initial scope for this domain included the areas of development, exposures, social environment, and child lifestyle. The scope, in turn, guided the identification and selection of six experts to serve on the PDWG, which was led by a chair and supported by a PhenX SC Liaison. The PDWG comprised researchers from five academic institutions and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), and members had expertise in perinatology, pediatrics, developmental and clinical psychology, public health, and epidemiology.

The PDWG followed the standard process for selecting measurement protocols, which includes an introductory call, a scope review call, a protocol consensus workshop, and a final recommendations meeting. Additional interim meetings were scheduled as needed. Because the PhenX Toolkit covers a wide range of research domains, there were already numerous protocols available for administration to pediatric populations. These included protocols for commonly collected sociodemographic data (e.g., ethnicity and race, annual family income, household roster); diet and nutrition; speech, hearing, and language; and child-specific assessments of temperament, intelligence, autism, school attendance and performance, and environmental exposures. The PDWG was tasked with building on the existing Toolkit content by focusing on the selection of protocols that are unique or critical to pediatric development research needs and that complement existing PhenX Toolkit content.

The PDWG discussed the age ranges to cover with the new protocols. The guidance was to include a continuum from birth through adolescence, as relevant for a given construct; prenatal exposures were also considered applicable for some constructs. The PDWG decided not to pursue the initially proposed scope elements of physical environmental exposures in childhood (e.g., exposure to pollutants, toxins, chemicals, medications) to focus more tightly on pediatric development. Each WG member assumed responsibility for scope elements for which they had expertise. During the Protocol Consensus Workshop, each member presented their findings from a literature review and made recommendations about which protocols to include, given the guidance from the SC. Their consideration included identifying existing protocols already in the Toolkit that sufficiently addressed their assigned scope element to avoid adding competing protocols to the Toolkit.

Following PhenX guidance, the PDWG prioritized consideration of existing, well-established measures that are in the public domain. Measurement protocols were assessed in terms of the burden and requirements for both study participants and investigators. Burden of use was assessed in terms of time, equipment, training, and data collection requirements. The PDWG considered measures outside of the public domain if those measures were well-established and best addressed the construct of interest or if suitable measures did not exist in the public domain. Because the PhenX Toolkit is a US effort, the primary focus was on measures for use in domestic research, but the PDWG considered international measurement standards as appropriate.

The PDWG initially selected 20 protocols for distribution to and feedback from the scientific community, including registered users of the PhenX Toolkit and other project participants, such as National Institutes of Health (NIH) liaisons, prior and current members of other PhenX WGs, and PhenX SC members. Respondents were asked to provide comments about the appropriateness, usability, and limitations of the proposed protocols and the value of including each in the PhenX Toolkit. There was general agreement about the inclusion of these recommended protocols, with lack of public access noted as a limitation for those protocols that were not the public domain. In the final meeting to review feedback from outreach, the PDWG considered and agreed with this limitation but concluded that the four proprietary protocols were the most widely used and that there were not alternative protocols publicly available that met the criteria of being well-established or providing sufficient coverage of the scope element.

The PDWG selected 18 measurement protocols (described in Table 1) to include in the new Pediatric Development domain of the PhenX Toolkit. As with all PhenX Toolkit protocols, a bioinformatics pipeline processed information about these 18 measurement protocols for loading into the Toolkit, and protocol-specific data dictionaries and data collection worksheets were developed.⁶ The PDWG agreed that an additional 2 protocols were important resources but were more appropriate as Supplemental Information reference materials. The first Supplemental Information protocol, Child’s Living Environment, was from the Avon Longitudinal Study of Parents and Children (ALSPAC). This protocol includes extensive use of British English words (e.g., English heating systems, currency), potentially limiting its usefulness in US-based research, which is the main target of the NIH-funded PhenX Toolkit. However, some researchers may find the protocol valuable as a model for collecting data regarding a child’s living environment. Another ALSPAC-derived protocol, Child’s Lifetime Household Composition, was included as one of the 18 Pediatric Development measurement protocols, because it included very limited use of British English. The other Supplemental Information protocol, the Youth Alcohol Screener, does not provide sufficient details to ensure consistent data collection and thus did not meet criteria for inclusion in the Toolkit; however, it may be of interest to researchers requiring a brief screening measure.

Measurement Protocols in the PhenX Toolkit

Table 1 presents the 18 Pediatric Development measurement protocols selected by the PDWG. These protocols cover a range of areas of child and caregiver functioning and psychosocial exposures relevant to pediatric development research. Specifically, the Pediatric Development domain includes validated measurement protocols of neonatal abstinence syndrome and opioid withdrawal syndrome; child care/preschool quality; academic performance; quality of peer relationships; bullying experiences; child mental health; psychological resilience; household stability, quality, and functioning; and caregiver mental health and parenting style. The domain includes both self-report and other-report measurement protocols, where relevant, to provide flexibility in study design and comprehensive assessments of children’s functioning and their environments.

The protocols are largely questionnaires or interviews that can be flexibly administered with a low burden in a variety of settings with minimal training; only one protocol, the Neonatal Abstinence Syndrome (NAS) and Neonatal Opioid Withdrawal Syndrome (NOWS), requires independent clinical observation with specialized training. The PDWG considered existing Toolkit content carefully when recommending these measurement protocols for inclusion in the Pediatric Development domain. Researchers are encouraged to use the life-stage filter within the search function of the Toolkit to identify the large number of measures that could be relevant to their investigations of child health and development, including many measures outside of the Pediatric Development domain. The measurement protocols added by the PDWG build on the existing content to add depth to the Toolkit for pediatric research.

Limitations

Although freely available protocols were preferred for inclusion in the Toolkit, the PDWG selected some protocols that are categorized in the Toolkit as “limited availability,” meaning that users must contact the source to obtain access (e.g., register for permission to use, pay fees for usage). Such measures were chosen when the PDWG determined that a suitable, freely available protocol did not exist. Additionally, in its review of the literature, the PDWG acknowledged important areas for which measures could not be identified that met PhenX Toolkit requirements (e.g., well-established, low burden) or have not yet been developed. As described below, these are areas for further research and exploration. Also included below is a description of challenges the PDWG encountered around the language used in some of the selected measures for inclusion in the Toolkit.

Child Sleep

The PDWG recognized the critical importance of sleep to child health and well-being.^13–18 However, the group was unable to identify a broadly validated instrument that it could recommend for use in pediatric populations. The PDWG sought input from the research community during outreach but received no protocol suggestions.

Anxiety in Pregnancy

The PDWG noted the large body of scientific literature demonstrating the significant impact of maternal mental health in pregnancy on a range of pediatric health outcomes.^19–22 Pregnancy and the postpartum period present specific demands that may not allow for standard mental health assessments to be used validly in the peripartum period but rather require tailored instruments. For example, diagnostic criteria for depression include several symptoms that may be present in pregnancy and not be indicative of psychological distress (e.g., changes in appetite, weight, and sleep). At the time the PDWG was convened, the PhenX Toolkit already included a validated measurement protocol of maternal depressive symptoms during the peripartum period (Edinburgh Postnatal Depression Scale²³) that takes into account these considerations. However, the PDWG could not identify an appropriate instrument for assessing anxiety in pregnancy.

Although several clinical interviews and questionnaires exist for evaluating general anxiety in the PhenX Toolkit (e.g., anxiety subscale of the Symptom Check List 90;²⁴ Structured Clinical Interview for DSM-5^® Disorders-Clinician Version²⁵), these protocols do not distinguish clinical anxiety symptoms from common experiences resulting from pregnancy and in caring for a newborn infant (e.g., somatic symptoms). They therefore may have decreased validity during the peripartum period. Furthermore, these measures do not address pregnancy-related anxiety (i.e., anxiety specific to worries about the health of the fetus/infant and concerns about the labor and delivery process). The extant literature suggests that pregnancy-related anxiety is distinct from general anxiety and depressive symptoms and has specific impact on maternal and child outcomes.^26–28 Although the PDWG identified several pregnancy-related anxiety measures in the literature, none of them met PhenX Toolkit criteria for inclusion because they lacked sound theoretical or psychometric properties or sufficient scope and depth for adequate assessment. Given that the pregnancy and postpartum periods are known risk periods for the onset or exacerbation of maternal mental health difficulties and that maternal mental health during these periods can have major, long-term impact on child health,^29–31 the PDWG encourages researchers to develop standardized, validated instruments that specifically address the unique considerations of assessing maternal mental health comprehensively during these periods.

Parenting and Caregiving Quality

A large body of research indicates the importance of parenting/caregiving quality, particularly in early life, on child health outcomes, and direct observation is considered the “gold standard” for assessing caregiving quality.^32–36 However, assessing caregiving quality by direct observation is quite burdensome given the need to have highly trained coders score observations that are collected in a standardized format. Therefore, the PDWG elected to recommend the Alabama Parenting Questionnaires, well-established instruments that are easy to use and involve much less burden on the investigator and the participant. In addition, the recommended Home Observation for Measurement of the Environment protocol, also well established, may be used to capture additional information about aspects of the caregiving environment. Studies that involve caregiving quality as a central construct may elect to use an established observation measure but consider using the PDWG-recommended questionnaires as adjunct measures.

Similarly, standardized observational data collection protocols exist for assessing the quality of a child’s daycare, child care, or preschool environment. However, the PDWG determined that such measures were highly burdensome because they require trained coders to conduct hours of observation in a child’s care setting. Further, the PDWG concluded that such observational measures would provide data at a level of detail unlikely to be needed by pediatric studies outside of those centrally focused on daycare or preschool quality. Rather, the PDWG chose a protocol that addresses parents’ or caregivers’ perceptions of their child’s care, including questions related to the reliability, cost, and quality of care, because these characteristics were determined to be particularly relevant for child health outcomes and measurable via caregiver report.^37–40

Evolving Landscape and Language

The PDWG confronted challenges when considering protocols with questions based on outdated assumptions (e.g., heterosexual nuclear family structure, obsolete technology available in the home). Ultimately, the PDWG elected to include these measures in the Pediatric Development domain because the protocols were the best available to assess the constructs considered essential for inclusion. However, because existing protocols are not modified for inclusion in the PhenX Toolkit, the PDWG recognizes that these measures may need to be modified by users for their specific study populations and encourages investigators to undertake efforts to update these measures, including conducting studies to document the validity of revised measures.

Additionally, although the PDWG chose measurement protocols that have been validated in various populations, the group recognizes the need for further research on how the protocols may function differently in different sociodemographic groups. Of the 18 pediatric protocols added to the Toolkit, three are available from the source in languages other than English; two of these are not open source and therefore cannot be displayed in the Toolkit but can be obtained by contacting the publisher. Before implementing any PhenX Toolkit measurement protocol in a study, investigators should consider the suitability of the measurement protocol for their particular study population and research questions.

Biomarkers

The PDWG recognized the importance of biomarkers in predicting and tracking pediatric outcomes and considered a range of wearable devices and exposure measures. However, the group was unable to identify any biomarker measures that had achieved sufficient technological advancement and met PhenX requirements of being clearly defined, well established, low burden, and broadly validated to be recommended at this time for wide implementation. These protocols are expected to evolve such that they can be considered by a future PhenX WG focused on biomarker assessments.