Skip to Main Content

RTI uses cookies to offer you the best experience online. By and clicking “accept” on this website, you opt in and you agree to the use of cookies. If you would like to know more about how RTI uses cookies and how to manage them please view our Privacy Policy here. You can “opt out” or change your mind by visiting: http://optout.aboutads.info/. Click “accept” to agree.

Accept
RTI International
  • About
    • Office Locations
    • Executive Leadership
    • Corporate Governance
    • Partner with Us
      • U.S. Government
      • Clients and Funding Agencies
      • Industry and Commercial Clients
      • Foundations and Associations
      • Bilateral Agencies and Multilateral Banks
      • Universities and Academic Research Institutions
      • Suppliers and Small Businesses
    • Commitment to Quality
      • RTI's Client Listening Program
    • Ethics and Human Research Protection
    • Living Our Mission
    • Veteran Opportunities at RTI

    About

  • Practice Areas
    • Health
      • Public Health and Well-Being
      • Health Care Transformation
      • Behavioral Health
      • Health Behavior Change
      • Precision Medicine
      • RTI Health Solutions (RTI-HS)
      • RTI Center for Community Health Evaluation and Economics Research
      • Health Equity
      • RTI Health Advance
    • Transformative Research Unit for Equity​
      • Equity Capacity Building Hub
      • Social and Economic Justice Research Collaborative
      • Narrative Research and Community Engagement Lab
    • Education and Workforce Development
      • Early Childhood
      • K-12 Education
      • Postsecondary Education
      • Career and Adult Education and Workforce Development
      • Education Policy, Systems, and Governance
      • Education Research Methodologies
      • Education Technologies
    • International Development
      • Energy for Development
      • Environment
      • Global Food Security, Agriculture, and Nutrition
      • Global Health
      • International Education
      • Monitoring, Evaluation, Research, Learning, and Adapting (MERLA)
      • Youth and Economic Opportunity
      • Building Resilience Against COVID-19 in Developing Countries
      • Water, Sanitation, and Hygiene (WASH)
      • RTI Center for Governance
    • Climate Change
      • Clean Energy Technology and Renewables
      • Climate Finance
      • Climate Justice and Equity
      • Climate Planning, Preparedness and Resilience
      • Climate Policy
      • Climate Vulnerability, Adaptation, and Mitigation
      • Economic Impacts of Climate Change
    • Water
      • Food-Energy-Water Nexus
      • Water Quality
      • WASH (Water, Sanitation, Hygiene)
      • Water Resources Management
    • Energy Research
      • Carbon Capture and Utilization
      • Biomass Conversion
      • Natural Gas
      • Energy Efficiency
      • Industrial Water
      • Syngas Processing
    • Environmental Sciences
      • Air Quality
      • RTI Center for Water Resources
      • Urban Sustainability
      • Toxics
      • Building Resiliency in the FEW Nexus
      • Climate Change Sciences and Analysis
      • Environmental Policy
      • Environmental Justice
      • Sustainable Materials & Waste Management Solutions
    • Justice Research and Policy
      • RTI Center for Community Safety and Crime Prevention
      • RTI Center for Policing Research and Investigative Science
      • Child Well-Being and Family Strengthening
      • RTI Center for Forensic Sciences
    • Food Security and Agriculture
      • Market Systems Strengthening
      • Food Safety
      • Food and Nutrition
      • Global Food Security, Agriculture, and Nutrition
      • Climate-Smart Agriculture
      • Agricultural Innovation
      • Obesity Prevention
    • Innovation Ecosystems
      • Innovation Advising
      • Innovation for Economic Growth
      • Innovation for Emerging and Developing Economies
      • Innovation for Organizations
      • Research, Technology, and Innovation Policy
      • Technology Acceleration
    • Military Support
      • Military Behavioral Health
      • Military Health and Human Performance
      • Military Sexual Assault, Harassment, and Domestic Violence Prevention
      • Wearable Sensor Technologies
      • Military Health System Transformation

    Practice Areas

  • Services + Capabilities
    • Surveys and Data Collection
      • Survey Design
      • Instrument Development
      • Survey Methodologies
      • Data Collection
      • Establishment Surveys
      • Health Registries
      • Data Analysis and Reporting
      • Research Operations Center
    • Statistics and Data Science
      • Survey Statistics
      • Environmental Statistics
      • Coordinating Centers for Multisite Studies
      • Analysis and Design of Complex Data
      • Biostatistics
      • RTI Center for Data Science
    • Evaluation, Assessment and Analysis
      • Evaluation Design and Execution
      • Advanced Qualitative, Quantitative, and Mixed Methods
      • Evaluation, Monitoring, and Assessment
      • Economic Analysis
      • Evaluating Communication Interventions and Campaigns
      • Evidence Synthesis for Policy and Practice
      • Risk Assessment and Prediction
    • Program Design and Implementation
      • Systems Strengthening and Scaling
      • Capacity Assessment and Building
      • Policy Reform Support
      • Curriculum and Teacher Professional Development
      • Interventions and Prevention Programs
      • Implementation Science
    • Digital Solutions for Social Impact
      • Human-Centered Design of Digital Solutions
      • Digital Product Development
      • Digital Communication Campaigns
      • Digital Data Analytics
    • Research Technologies
      • Survey Technologies
      • Data Management and Decision Support Systems
      • Geospatial Science, Technology, and Visualization
      • ICT for Limited-Resource Settings
      • Mobile Applications
      • Web Applications
      • Bioinformatics
      • Interactive Computing
    • Drug Discovery and Development
      • Medicinal Chemistry
      • Molecular Design and Cheminformatics
      • Behavioral Pharmacology
      • Drug Metabolism and Pharmacokinetics (DMPK)
      • In Vitro Pharmacology, Bioassay Development, and High-Throughput Screening (HTS)
      • Isotope Labeling
      • Regulatory Consulting and Support for Medical Products
    • Analytical Laboratory Sciences
      • Bioanalytical and Toxicology Research
      • Forensic Sciences
      • Physicochemical Characterizations
      • Metabolomics
      • Proficiency Testing and Reference Materials
      • Microbiology
      • Analytical Chemistry and Pharmaceutics
    • Engineering & Technology R&D
      • Biomedical Technologies
      • Decarbonization Sciences
      • Environmental Exposure & Protection
      • Materials & Environment
      • Sustainable Energy Solutions

    Services + Capabilities

  • Centers
    • RTI Center for Advanced Methods Development
    • RTI Center for Communication Science
      • Communication Research
      • Communication Design
      • Communication Delivery
    • RTI Center for Data Science
    • RTI Center for Education Services
      • Teaching and Learning
      • Education Leadership
      • Peer Learning Networks
      • Strategic Consulting
    • RTI Center for Forensic Sciences
    • RTI Center for Global Noncommunicable Diseases
      • Program Financing & Economics for NCDs
      • Health Systems Strengthening for NCDs
      • Communication Science and Behavior Change for NCDs
      • Implementation Science for NCDs
    • RTI GenOmics, Bioinformatics, and Translational Research Center
      • Disability Studies
      • Ethics
      • Newborn Screening
    • RTI Center for Water Resources
      • Water Resources Sectors
      • Water Resources Services
      • Water Resources Tools
    • RTI Center for Governance
    • RTI Global Gender Center
    • North Carolina Center for Optimizing Military Performance
    • NCCU-RTI Center for Applied Research in Environmental Sciences
    • RTI Center for Climate Solutions

    Centers

  • Impact
    • Newsroom
    • Insights Blog
    • Events
    • Publications
    • RTI Press
      • About the RTI Press
      • Instructions for Authors
      • RTI Press Collections
    • Projects
    • Global Reach
      • Asia
      • Eastern Europe and Central Asia
      • RTI International India
      • Africa
      • Middle East and North Africa (MENA)
      • Latin America and the Caribbean (LAC)

    Impact

  • Experts
    • Our Experts
    • In-Depth With Our Experts
    • Related News
    • Experts In the Media
    • RTI Fellow Program

    Experts

  • Emerging Issues
    • COVID-19 Research
    • Artificial Intelligence
    • Global Health Security
    • Cannabis Research
    • Opioid Research
      • Interventions for Opioid Use Disorders
      • Preventing Opioid Misuse and Overdose
      • Treating Opioid Use Disorders
    • Policing Research and Investigative Science
    • Drone Research and Application
    • E-cigarette Research
    • Zika Virus Research
    • Integrated Governance

    Emerging Issues

  • COVID-19 Research + Response
  • Global Reach
  • Insights Blog
  • Newsroom
  • RTI Press
  • Publications
  • Partner With Us
  • Careers
  • Facebook IconTwitter IconInstagram IconYouTube IconLinkedin Icon
  • Home
  • Impact
  • RTI Press
  • Eliciting stakeholder perceptions using a novel online engagement platform

Eliciting stakeholder perceptions using a novel online engagement platform

A case study on nano-agrifoods

By Juliana M. Ruzante, Ellen Thomas Shumaker, Sidney Holt, Susan Mayer, Adam Kokotovich, Maude Cuchiara, Andrew R. Binder, Jennifer Kuzma, Khara Grieger.

January 2022 Open Access Peer Reviewed

DOI: 10.3768/rtipress.2022.op.0071.2201

Check for Updates Download PDF
Ruzante, J. M., Shumaker, E. T., Holt, S., Mayer, S., Kokotovich, A., Cuchiara, M., Binder, A. R., Kuzma, J., & Grieger, K. (2022). Eliciting stakeholder perceptions using a novel online engagement platform: A case study on nano-agrifoods. RTI Press. RTI Press Occasional Paper No. OP-0071-2201 https://doi.org/10.3768/rtipress.2022.op.0071.2201
Copy citation
Share
  • Share on Facebook
  • Share on Twitter
  • Share on Linkedin
  • Email
Key Points
  • Stakeholder engagement is an important component in developing policies on critical issues such as the use and development of novel methods and technologies, including biotechnologies and nanotechnologies.
  • Understanding the perspectives, needs, and concerns of stakeholder groups can facilitate the development of transparent and trusted policy recommendations.
  • In this work, we used an online stakeholder engagement platform (OSEP) to engage stakeholders and capture their perceptions and views about the application of nanotechnology in food and agriculture (nano-agrifood) and the role of responsible innovation in the development of nano-agrifood products.
  • The OSEP provided a reliable and interactive environment for stakeholders to share their views and exchange ideas.
  • Such OSEPs should be further explored as novel tools for engaging stakeholders on a range of issues from emerging technologies to public health.

Abstract

Stakeholder engagement is an important component in developing policies on critical issues such as the use and development of novel methods and technologies, including biotechnologies and nanotechnologies. Understanding the perspectives, needs, and concerns of stakeholder groups can facilitate the development of transparent and trusted policy recommendations. Innovative online research platforms have been developed as alternatives to typical stakeholder engagement methods such as in-person focus groups, interviews, and online and paper surveys. These platforms facilitate the engagement of geographically and linguistically (i.e., individuals who speak different languages) diverse stakeholders using a wide range of methods, from virtual focus groups to surveys. Stakeholders can participate at their own leisure and anonymously, which can facilitate more open interactions on issues where viewpoints may differ. In this work, we used an online stakeholder engagement platform (OSEP) to engage stakeholders and capture their perceptions and views about the application of nanotechnology in food and agriculture (nano-agrifood) and the role of responsible innovation in the development of nano-agrifood products. The OSEP provided a reliable and interactive environment for stakeholders to share their views and exchange ideas. Such OSEPs should be further explored as novel tools for engaging stakeholders on a range of issues from emerging technologies to public health.

Creative Commons © 2023 RTI International. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Contents

  • Introduction
  • Case Study: Responsible Innovation of Nanotechnology in Food and Agriculture
    • Background
      • Online Stakeholder Engagement Platform
        • Platform Development
          • Stakeholder Participation
            • Evaluation of the Platform
            • Discussion
            • Acknowledgments
            • References

            Eliciting Stakeholder Perceptions Using a Novel Online Engagement Platform: A Case Study on Nano-Agrifoods

            By Juliana M.RuzanteEllen ThomasShumakerSidneyHoltSusanMayerAdamKokotovichMaudeCuchiaraAndrew R.BinderJenniferKuzmaKharaGrieger

            Introduction

            Stakeholder engagement is an important component in developing policies and regulations on critical issues such as the use and development of new methods of product innovation, such as biotechnologies and nanotechnologies. It offers an opportunity to engage with stakeholders and to address their concerns and answer their questions. More importantly, sound, transparent, and trusted policy recommendations can be developed by understanding various stakeholder groups’ perspectives, needs, and concerns. Furthermore, stakeholder engagement can identify areas of commonality and agreement or disagreement and reveal the underlying motivations and rationales for differences of opinion and perspective, aiding communication between policy makers and various stakeholder groups. Accounting for stakeholder needs and preferences is especially important during responsible development of new technologies to ensure that research and innovation processes are open, inclusive, and consider diverse perspectives, ultimately creating new technologies that align with societal values, needs, and expectations (Owen et al., 2013; RRI Tools, 2021; Stilgoe et al., 2013).

            Stakeholder engagement is routinely called for in governance, risk governance, and policy development within food and agriculture sectors, including those that develop or utilize new food and agriculture technologies. Stakeholder engagement can help incorporate perspectives and concerns from diverse viewpoints to address complex environmental and public health challenges efficiently (Cummings & Kuzma, 2017; International Risk Governance Center, 2017, 2020). For instance, stakeholder engagement can help identify desirable and undesirable technology attributes and inform the problem formulation stage of risk analysis and regulatory decisions (Kokotovich et al., 2020; Kuzma, 2019). In food and agriculture, there are several types of stakeholder groups that are directly or indirectly involved and that can be impacted by decisions and policies. Typical stakeholders in food and agriculture include the general public (i.e., consumers, primary shoppers), academia, government, industry, international, and non-profit organizations.

            Decision methods such as multi-criteria decision analysis (MCDA) can incorporate and weigh the different stakeholders’ views, perceptions, and preferences systematically and transparently. These methods have been used to help improve decision-making in several areas such as engineering, environmental health (Linkov & Moberg, 2012), and food safety (Food and Agriculture Organization of the United Nations, 2017; Ruzante et al., 2010). However, before starting any approach such as MCDA, stakeholder preferences need to be captured and characterized.

            The first step in stakeholder engagement is to identify key stakeholders in the field of interest and determine how and when they can be involved in policy development (Lemke & Harris-Wai, 2015). The motivation for engaging stakeholders and potential challenges expected during engagement activities also need to be defined. Many methods exist to engage, gather, and quantify stakeholders’ perspectives and views. Focus groups are efficient in engaging participants in a targeted manner and are often used to gather information and identify themes concerning participants’ perceptions, beliefs, and behaviors on a topic in a short timeframe (typically 60–90 minutes); however, it is also more difficult to maintain anonymity, which might be a disadvantage when it comes to discussing potentially polarizing or controversial issues. In-person focus groups are generally used to bring together individuals from a similar geographic area. Therefore, several focus groups need to be conducted to obtain representative points of view and ensure that geographically diverse groups have a chance to discuss their different points of view about an issue, being potentially expensive. Furthermore, focus groups might also need to be hosted outside working hours to include a representative group of participants, which might add a burden to researchers. In-person interviews and mail or online questionnaires are also used to gather stakeholders’ opinions, values, and perceptions.

            With mail and online methods, respondent identity can be protected; however, it is often not possible to interact or converse directly with stakeholders on the subject at hand. Community forums and public comments are other methods used to gather stakeholder input, but these may lack the level of active interaction, depth, and dynamic discussion of focus groups and suffer from nonrepresentative samples. Additionally, consensus conferences also capture perceptions and decision-making under “high information” circumstances and provide insights from a more diverse sample of participants (Einsiedel et al., 2001). Consensus conferences have usually been convened to assess how non-experts might respond to a variety of controversial topics. A desirable method to gather stakeholder input would likely include attributes from all of these methods: geographically diverse participation, interactive group discussion, ease of administration, and anonymous participation.

            Over the past 10 to 15 years, innovative online research platforms have been developed as alternatives to typical stakeholder engagement approaches, which in many ways come closer to having all desirable attributes. These online stakeholder engagement platforms (OSEPs) overcome many of the limitations related to in-person focus groups, questionnaires, and other methods. They allow researchers to engage geographically diverse stakeholders using a wide range of methods, from virtual focus groups to digital diaries and message boards (Hussey & Zerfas, 2020; Belpasso, 2019). Moreover, stakeholders can participate at their leisure—within the parameters set by the researchers—and from wherever is most comfortable for them. Another advantage of OSEPs is that they allow stakeholders to engage with one another anonymously, which can facilitate more open and uninhibited engagement on issues where they might have different points of view. This anonymity can also help mitigate issues around power dynamics, which can reduce participation by certain stakeholder groups or individuals who may defer to others who “know best.” Most of all, these OSEPs offer flexibility. They can accommodate both fixed-duration projects and efforts to generate ongoing or long-term engagement. CMNTY, EngagementHQ, Incling, The HiVE, and VisionsLive are a few examples of OSEPs that have made stakeholder engagement increasingly convenient for both researchers and stakeholders. These OSEPs have also made it easier to navigate stakeholder engagement during the COVID-19 pandemic, when gathering in person was not possible. One potential disadvantage is that they require stakeholders to have access to the internet, which can unintentionally limit participants’ full availability or privilege certain participant groups.

            In this paper we demonstrate the potential benefits (from researchers’ and participants’ points of view) and the application of commercially available OSEPs to elicit stakeholder perceptions on topics where there is disagreement and the views of stakeholders are not clearly defined. In the case study presented, we used an OSEP to explore stakeholder views of responsible innovation (RI) in the context of nanotechnology applications in food and agriculture (nano-agrifood).

            Case Study: Responsible Innovation of Nanotechnology in Food and Agriculture

            Background

            The use of nanotechnology and engineered nanomaterials in food and agriculture sectors may provide potential health benefits, such as the use of nano-emulsions to create nutrient-fortified foods that are appealing to consumers (Aswathanarayan & Vittal, 2019), and potential environmental benefits, as the use of nano-pesticides and nano-fertilizers that may have less ecological impact compared with conventional agrochemicals (Fraceto et al., 2016). However, their full impacts on health, the environment, and society are not yet fully understood and therefore require further investigation on their environmental, health, and safety (EHS) aspects and ethical, legal, and societal implications (ELSI) (Cummings et al., 2021; Grieger et al., 2016a,b; Iavicoli et al., 2017; Steenis & Fischer, 2016). RI is a governance paradigm that promotes the principles of anticipation, inclusion, reflexivity, and responsiveness to incorporate not only aspects of EHS and ELSI in early innovation stages but also processes that researchers can utilize to reflect upon and modify their innovation practices to develop products that align with societal needs and expectations (Stilgoe et al., 2013). It is often presumed that when successfully practiced, RI also leads to better ground-up or “upstream” policymaking.

            In this case study, we were interested in characterizing stakeholders’ views of RI for nano-agrifood applications across the farm-to-fork continuum. The outcomes from the case study selected here included identifying approaches and best practices that may help address stakeholders’ concerns surrounding nano-agrifoods, with the goal of ultimately improving RI for nano-agrifoods, and are reported elsewhere (Grieger et al., 2021).

            Online Stakeholder Engagement Platform

            In the present case study, we developed a series of stakeholder engagement surveys and discussion forums on the CMNTY platform (CMNTY Corporation, 2021) to allow for an open, inclusive, and effective means for deliberation on the topic of RI for nano-agrifoods. More specifically, it focused on applications of nano-agrifood across the food continuum because stakeholder perceptions and attitudes differ across various nano-agrifood applications as it has been shown by Steenis & Fischer (2016). Multiple platforms, including CMNTY, EngagementHQ, Incling, The HiVE, and VisionsLive all have very similar functionalities: flexibility for sharing platform content, options to solicit stakeholder feedback using multiple research methods (i.e., individual questionnaires and group discussions), ability to maintain user anonymity; however, we did not conduct a comparative analysis among these platforms, and CMNTY (2021) was selected for convenience because some of the researchers had previous experience with the platform. All study procedures were developed and submitted to the Institutional Review Board at North Carolina State University for approval. The following section outlines the specific steps taken to develop and conduct stakeholder engagement using CMNTY.

            Platform Development

            Recruitment. We identified stakeholder candidates based on their affiliations (i.e., government, nongovernmental organization [NGO] think tanks, advocacy organizations, industry, and academia) through the research team’s connections in food and agricultural industries, research consortia, and networks. We conducted a literature review to identify authors publishing articles in the fields of interest. We also reviewed the program and speaker lists of high-profile conferences in nano-agrifoods and conducted a broader internet search on relevant stakeholder groups in nano-agrifood innovations to obtain potential candidates to join the study. We did not recruit consumers and the general public directly, but we did include advocacy organizations representing these groups.

            In total, we invited 442 stakeholders to participate. The study offered a $100 incentive upon completion of all activities. The benefits for participation included opportunities to contribute to stakeholder discussions that can influence recommendations on nano-agrifood policy, be involved in research that develops best practices for RI of agrifoods, voice key perceptions and concerns, and identify approaches to address concerns to support agrifood RI. Of the 442 invited, 62 individuals agreed to participate. The final group of participants represented a range of agrifood sectors, including government, NGO, industry, and academia.

            Platform Setup. We set up the CMNTY platform to elicit responses from stakeholders using a mixed-methods approach that included independent activities where participants had no interaction with others (e.g., questionnaires) and a group activity where discussions between participants were expected (e.g., message board–style discussion forum). The home page included a welcome message that provided participants with clear instructions on how to complete the various activities and in what order to complete them. Figure 1 provides a screenshot of the welcome page. None of the activities were actively moderated by the research team; however, researchers were available to answer any clarifying questions via the chat feature.

            Platform homepage

            We used two initial questionnaires to assess participant familiarity with nanotechnology and engineered nanomaterials in food and agriculture and to understand their own definition of RI and related practices. Following the initial questionnaires, participants individually reviewed five case studies based on existing uses of nanotechnology in food and agriculture. Each case study explored a different application and rationale for using nanomaterial. Questions following the description of each case study aimed to quantify how each stakeholder viewed the usefulness, safety, benefits, and other considerations associated with the case study, including whether they thought each case study was an example of RI. Figure 2 shows a screenshot of the questionnaire page that participants completed independently.

            Questionnaire page

            Following the case studies, we asked participants to rank each of the five case studies from most to least aligned with their own definition of RI. We also asked them to share the three most important factors that influenced their ranking. The study included a “Situating Worldviews” multiple-choice questionnaire to gauge participants’ general perceptions and beliefs on science and technology more broadly.

            For the group activity, we created a message board discussion forum for participants to discuss topics with threaded comments. Participants could comment in response to the discussion prompt and view and respond to comments posted by other participants. The message board contained five discussion prompts, including what types of barriers exist concerning RI for nano-agrifoods and how they can be overcome.

            A final insights questionnaire captured participants’ concluding views concerning RI for nano-agrifoods, and an exit survey asked participants about their experience using the platform—specifically, whether they thought it worked well—and to give them an opportunity to provide any additional feedback.

            Table 1 summarizes platform activities, numbered in the order that participants progressed through the activities.

            Overview of platform activities.
            Familiarity with Nanotechnology
            Individual Questionnaires (Multiple-Choice or Open Response) What is Responsible Innovation?
            Case Studies
            a. Fresh Cut Fruit and Surface Browning
            b. Laying Hens and Salmonella Infection
            c. Dietary Supplements and Micronutrient Copper
            d. Fruit and Citrus Greening Disease
            e. Infant Formula and Aesthetic Appearance
            Ranking Case Studies
            Situating Worldview
            Final Insights and Reflections
            Exit Survey
            Forum Discussions
            (Message board format)
            a. Company Actions for Responsible Innovation
            b. Regulatory Agency Actions for Responsible Innovation
            c. Barriers to Responsible Innovation
            d. Disclosure of Nanomaterials
            e. Examples of Responsible Innovation

            Stakeholder Participation

            The discussion forum and in the consent form that participants signed before starting the activities clearly described expectations around the level and type of engagement. The consent form outlined that participants would regularly participate in the platform by (1) completing the individual questionnaires, (2) completing a group forum exercise, and (3) participating on the platform at least two times over 3 weeks. The expected level of effort (2 to 4 hours) was also noted. In the discussion forum, we asked participants to respond to each question by adding their own answers and by commenting on one or two responses made by other participants. We also encouraged participants to log in and review the discussion board multiple times over the course of the study. In terms of etiquette for engaging in online discussions, we asked participants to practice respectful communication and disagreement. It was clear that there would be no tolerance for disrespectful (i.e., offensive, hateful, or obscene) comments. Members of the research team monitored the platform for new activity and encouraged participation through regular reminder emails. However, there was no real-time moderation of the discussions, and research team members did not post comments on the message board.

            Evaluation of the Platform

            Of the 62 who accepted the invitation to participate, 55 completed all activities on the platform: government (16.4%; n = 9); NGO – think tank (12.7%; n = 7); NGO – advocacy (18.2%; n = 10); industry (18.2%; n = 10); and academia (34.5%; n = 19). This high completion rate (89%) coupled with the high quality of responses (i.e., thorough and considered responses) speaks to the effectiveness of OSEPs for online stakeholder engagement, as discussed below. Despite the instructions provided, the engagement level varied widely across participants. The discussion forum garnered 401 responses (170 original posts and 231 replies), and the number of responses per participant ranged from 1 to 17, with an average of 7.

            The majority of study participants (80%, n = 44 of 55) agreed or strongly agreed that the OSEP worked well for this study. The most frequently cited advantage (n = 12 participants cited this advantage) of the platform was its message board–style discussion feature. One participant shared, “The [discussion] forum was very interesting, and it was very helpful to read and evaluate the cases individually and then engage in the discussion. It was a good exercise to make me think about RI and also learn from others.” Other participants commented that they were “impressed by the quality of the discussion on the message boards” and that “it was eye-opening to see so many different points of view.” In fact, exposure to diverse viewpoints was cited as an advantage of the OSEP (n = 6 participants cited this advantage), speaking to OSEP’s ability to engage diverse stakeholder audiences effectively and efficiently.

            Participants also appreciated that the OSEP was easy to use and navigate (n = 6), that it facilitated a mixed-methods research approach (n = 6), and that it allowed for self-pacing (n = 6) and anonymity (n = 3). One participant who strongly agreed that the platform worked well for the study noted, “Really liked this format. Easy to navigate, easy to pause and continue as my schedule and energy allowed. I also really enjoyed engaging with other subjects in the forum. Excellent hybrid of survey and focus group research methods.” Several other participants commented on the mixed-methods approach, specifically the benefits of independent assessment followed by group discussion. For instance, one participant shared:

            I thought it worked well because it provided material for all to read, and questions for all to answer, to provide a common background for the discussion between participants whom I suspect have a wide range of familiarity with nanotechnology. The fact that the questions prior to the group discussion required rankings as well as written responses allowed positions on the topics to be identified and the reasons why explained. Having gone through this all “together” seemed to me to break the ice and allow a wide-ranging, and respectful discussion to follow.

            This comment speaks to the notion of “leveling the playing field” by ensuring that all participants have an opportunity to read study materials and consider their stance before the group discussion, a benefit that was reiterated by another participant who appreciated the ability to self-pace: “I liked having time to field the questions and formulate my answers. I do not always think as well when I am placed in a group.”

            Although the feedback on the platform was overwhelmingly positive, participants identified a few downsides to engaging stakeholders in this manner, namely the lack of “live interaction,” which created an inability to ask clarifying questions and to discuss perspectives in depth, which is particularly problematic for a complex topic such as RI in food and agriculture. For instance, one participant noted, “It was sometimes difficult to understand what was meant in the discussions. In a live situation, you could ask someone to clarify their statement or to check your understanding of [what] they said. You can’t easily do that in this format.”

            Discussion

            The inclusion of stakeholder perspectives is essential for responsibly designing and developing nano-agrifoods. In the case study described here, a commercially available OSEP was used to obtain stakeholders’ individual perspectives and their asynchronous interactions with other stakeholders assessing the same information. The goal was to connect with and learn from a wide range of stakeholders directly involved in nano-agrifoods, including members of NGOs, advocacy organizations, academia, industry, and government, using a single platform; however, it could have also been used to connect and engage the general public and consumers just as efficiently. Moreover, the platform supported a diverse array of research methods, which allowed the researchers to use a mixture of questionnaires and group discussions to obtain a rich dataset without overly burdening participants. As an example of this array of research methods, the researchers designed this study so that stakeholders first answered a series of questionnaires independently before engaging in a group discussion. Several study participants noted the advantage of this stepwise approach and felt that it facilitated a more robust discussion.

            From the researchers’ point of view, using an OSEP allowed both individual questionnaires and a group discussion forum to be conducted on the same platform, increasing the ease of administration and tracking of participant progress. The platform also provided in-platform analysis capabilities that allowed researchers to identify emerging trends in the results from the multiple-choice questionnaires and an easy way to export results. Finally, the OSEP subscription included regular meetings with company employees who could answer questions about how to best integrate our research into the platform.

            This study had several limitations that can serve as lessons learned for researchers planning to use OSEPs. First, according to participants, a weakness was that it lacked “live” or dynamic interaction. Although we used the discussion forum to engage stakeholders in a conversation, this forum is modeled after a message board in which participants respond at their leisure to a static prompt rather than in “real time” to moderated questions. The OSEP used here does offer a virtual focus group feature; however, we did not use this data collection method for the current study because the primary goal was to have all stakeholders participating in discussions together. Depending on the goals of the research study, it may be beneficial to utilize the “live” features of OSEPs to facilitate a more interactive engagement. Discussion boards could be timed for specific days or participants could be randomly assigned to a particular day to allow more time for exchange to facilitate engagement and restrict participants from posting all comments at one time.

            Another limitation was that despite the instructions provided in the consent form and in message boards to describe what was expected of study participants, several individuals shared that they did not clearly understand the goals of the project or of the stakeholder engagement. Therefore, it is important to consider the use of multiple methods through which participants can engage in real time with researchers so that the goals of the engagement are better understood. Repetition of expectations throughout the activities might also be helpful when asking stakeholders to participate in an OSEP such as the one used here.

            Although one advantage of OSEPs is that stakeholders can participate whenever is most convenient for them, future studies that utilize these platforms may find it beneficial to have predetermined, advertised windows of time (i.e., a specific time each day or week of the study) for research team members to engage with stakeholders in real time to answer any clarifying questions and facilitate conversations where needed to facilitate a “two-way” interaction between stakeholders and the research team as a way to address the previously discussed limitations. However, one disadvantage of ongoing researcher participation is that participants may be more reticent to express their views freely for fear of being incorrect or in deference to what they view as “experts” conducting the forum behind the scenes. There is likely an appropriate balance to strike in this regard.

            Access to the internet was not an issue for the stakeholder groups targeted in this case study; however, depending on the groups that need to be involved in the discussion, reliable internet access needs to be considered so this does not become an excluding factor for participation and the engagement process can be truly inclusive. That would be an important factor to be considered if using these platforms to engage with the general public.

            Overall, our use of an OSEP resulted in positive experiences for most participants, both for being part of research evaluating real-world applications of nano-food innovations and for furthering their own thinking about RI in the food sector. This particular OSEP provided a reliable and engaging environment for stakeholders to exchange and discuss ideas and could be further explored by policy makers and other decision makers as an innovative tool to engage different types of stakeholders.

            Acknowledgments

            The authors would like to state that there are no conflict of interest involving authors and CMNTY, the software tool used in the case study and cited in this paper.

            The authors would like to acknowledge Christopher L. Cummings from North Carolina State University for his contributions to the development of the stakeholder platform activities and study design. This research was conducted with funding from the United States Department of Agriculture, National Institute for Food and Agriculture.

            RTI Press Associate Editor: Barbara B. Biesecker

            References

            Aswathanarayan, J. B., & Vittal, R. R. (2019). Nanoemulsions and their potential applications in food industry. Frontiers in Sustainable Food Systems, 3, 95. Advance online publication. 10.3389/fsufs.2019.00095
            Belpasso, R. (2019, April 5). How an insights platform empowers adolescent health promotion. CMNTY. https://www.cmnty.com/blog/community-platform-adolescent-health-promotion/
            CMNTY Corporation. (2021). Software. https://www.cmnty.com/
            Cummings, C. L., & Kuzma, J. (2017). Societal Risk Evaluation Scheme (SRES): Scenario-based multi-criteria evaluation of synthetic biology applications. PLoS One, 12(1), e0168564. 10.1371/journal.pone.016856428052080
            Cummings, C. L., Kuzma, J., Kokotovich, A., Glas, D., & Grieger, K. (2021). Barriers to responsible innovation of nanotechnology applications in food and agriculture: A study of US experts and developers. NanoImpact, 23, 100326. 10.1016/j.impact.2021.100326
            Einsiedel, E. F., Jelsøe, E., & Breck, T. (2001). Publics at the technology table: The consensus conference in Denmark, Canada, and Australia. Public Understanding of Science (Bristol, England), 10(1), 83–98. 10.1088/0963-6625/10/1/306
            Food and Agriculture Organization of the United Nations. (2017). Food safety risk management: Evidence-informed policies and decisions, considering multiple factors. https://www.fao.org/documents/card/en/c/I8240EN/
            Fraceto, L. F., Grillo, R., de Medeiros, G. A., Scognamiglio, V., Rea, G., & Bartolucci, C. (2016). Nanotechnology in agriculture: Which innovation potential does it have? Frontiers in Environmental Science, 4. Advance online publication. 10.3389/fenvs.2016.00020
            Grieger, K. D., Merck, A. W., Cuchiara, M., Binder, A. R., Kokotovich, A., Cummings, C. L., & Kuzma, J. (2021). Responsible innovation of nano-agrifoods: Insights and views from U.S. stakeholders. NanoImpact, 24, 100365. 10.1016/j.impact.2021.100365
            Grieger, K. D., Hansen, S. F., Mortensen, N. P., Cates, S., & Kowalcyk, B. (2016a). International implications of labeling foods containing engineered nanomaterials. Journal of Food Protection, 79(5), 830–842. 10.4315/0362-028X.JFP-15-33527296434
            Grieger, K. D., Harrington, J., & Mortensen, N. (2016b). Prioritizing research needs for analytical techniques suited for engineered nanomaterials in food. Trends in Food Science & Technology, 50, 219–229. 10.1016/j.tifs.2016.02.004
            Hussey, S., & Zerfas, Y. (2020). ‘Peoplebank’: Creating a common language in online health engagement. Bangthetable. https://www.bangthetable.com/blog/peoplebank-creating-a-common-language-in-online-health-engagement/
            Iavicoli, I., Leso, V., Beezhold, D. H., & Shvedova, A. A. (2017). Nanotechnology in agriculture: Opportunities, toxicological implications, and occupational risks. Toxicology and Applied Pharmacology, 329, 96–111. 10.1016/j.taap.2017.05.02528554660
            International Risk Governance Center. (2017). Introduction to the IRGC risk governance framework (Revised Version). https://irgc.org/risk-governance/irgc-risk-governance-framework/
            International Risk Governance Center. (2020). Involving stakeholders in the risk governance process. https://irgc.org/wp-content/uploads/2020/10/IRGC-Stakeholder-Engagement-Resource-Guide_Version2_2020-3.pdf
            Kokotovich, A. E., Delborne, J. A., Elsensohn, J., & Burrack, H. (2020). Emerging technologies for invasive insects: The role of engagement. Annals of the Entomological Society of America, 113(4), 266–279. 10.1093/aesa/saz064
            Kuzma, J. (2019). Procedurally robust risk assessment framework for novel genetically engineered organisms and gene drives: Risk assessment for GEOs and gene drives. Regulation & Governance. 15(4), 1144–1165. 10.1111/rego.12245
            Lemke, A. A., & Harris-Wai, J. N. (2015). Stakeholder engagement in policy development: Challenges and opportunities for human genomics. Genetics in Medicine, 17(12), 949–957. 10.1038/gim.2015.825764215
            Linkov, I., & Moberg, E. (2012). Multi-criteria decision analysis: Environmental applications and case studies (1st ed.). CRC Press.
            Owen, R., Stilgoe, J., Macnaghten, P., Gorman, M., Fisher, E., & Guston, D. (2013). A framework for responsible innovation. In R. Owen, J. Bessant, & M. Heintz (Eds.), Responsible innovation: Managing the responsible emergence of science and innovation in society (pp. 27–50). John Wiley & Sons, Ltd. 10.1002/9781118551424.ch2
            RRI Tools. (2021). What is RRI? https://rri-tools.eu/about-rri
            Ruzante, J. M., Davidson, V. J., Caswell, J., Fazil, A., Cranfield, J. A. L., Henson, S. J., Anders, S. M., Schmidt, C., & Farber, J. M. (2010). A multifactorial risk prioritization framework for foodborne pathogens. Risk Analysis, 30(5), 724–742. 10.1111/j.1539-6924.2009.01278.x
            Steenis, N. D., & Fischer, A. R. H. (2016). Consumer attitudes towards nanotechnology in food products. British Food Journal, 118(5), 1254–1267. 10.1108/BFJ-09-2015-0330
            Stilgoe, J., Owen, R., & Macnaghten, P. (2013). Developing a framework for responsible innovation. Research Policy, 42(9), 1568–1580. 10.1016/j.respol.2013.05.008

            Click cover to download publication

            Sustainable Development Goals

            • Goal 2: Zero Hunger
            • Goal 3: Good Health and Well Being

            Contact

            To contact an author or seek permission to use copyrighted content, contact our editorial team

            • +1 919 541 6490
            • rtipress@rti.org

            Meet the Experts

            View All Experts
            Juliana Ruzante

            Juliana Ruzante

            Headshot of Susan Mayer

            Susan Mayer

            Related Publications

            View All Press
            OCCASIONAL PAPER

            The Preschool Entitlement

            OCCASIONAL PAPER

            Differentiated instruction in multigrade preprimary classrooms in Kenya

            OCCASIONAL PAPER

            The potential for solar-powered groundwater irrigation in Sub-Saharan Africa

            BOOK

            Knowledge system development

            RESEARCH REPORT

            Integrated governance

            OCCASIONAL PAPER

            Reach Health Assessing Cost-Effectiveness for Family Planning (RACE-FP) methodology report

            OCCASIONAL PAPER

            Eliciting stakeholder perceptions using a novel online engagement platform

            Article

            Implementation of DHIS2 for disease surveillance in Guinea

            Recent Publications

            View All Press
            OCCASIONAL PAPER

            Bringing an equity-centered framework to research

            OCCASIONAL PAPER

            The Preschool Entitlement

            OCCASIONAL PAPER

            Culturally informed community engagement

            RESEARCH REPORT

            Substance misuse prevention program attendance

            RTI Logo
            Partner With Us
            • US Government
            • Commercial
            • Foundations & Associations
            • Multilateral Donors
            • Universities
            • Suppliers
            Site
            • Privacy Policy
            • Security Policy
            • Site Map
            • Terms of Use
            • Accessibility
            • Contact Us
            Contact Us
            Facebook Icon Twitter Icon Instagram Icon YouTube Icon Linkedin Icon
            delivering the promise of science
            for global good
            RTI Health Solutions RTI Innovation Advisors RTI Health Advance

            © 2023 RTI International. RTI International is a trade name of Research Triangle Institute. RTI and the RTI logo are U.S. registered trademarks of Research Triangle Institute.