Eliciting stakeholder perceptions using a novel online engagement platform

Stakeholder engagement is an important component in developing policies and regulations on critical issues such as the use and development of new methods of product innovation, such as biotechnologies and nanotechnologies. It offers an opportunity to engage with stakeholders and to address their concerns and answer their questions. More importantly, sound, transparent, and trusted policy recommendations can be developed by understanding various stakeholder groups’ perspectives, needs, and concerns. Furthermore, stakeholder engagement can identify areas of commonality and agreement or disagreement and reveal the underlying motivations and rationales for differences of opinion and perspective, aiding communication between policy makers and various stakeholder groups. Accounting for stakeholder needs and preferences is especially important during responsible development of new technologies to ensure that research and innovation processes are open, inclusive, and consider diverse perspectives, ultimately creating new technologies that align with societal values, needs, and expectations (Owen et al., 2013; RRI Tools, 2021; Stilgoe et al., 2013).

Stakeholder engagement is routinely called for in governance, risk governance, and policy development within food and agriculture sectors, including those that develop or utilize new food and agriculture technologies. Stakeholder engagement can help incorporate perspectives and concerns from diverse viewpoints to address complex environmental and public health challenges efficiently (Cummings & Kuzma, 2017; International Risk Governance Center, 2017, 2020). For instance, stakeholder engagement can help identify desirable and undesirable technology attributes and inform the problem formulation stage of risk analysis and regulatory decisions (Kokotovich et al., 2020; Kuzma, 2019). In food and agriculture, there are several types of stakeholder groups that are directly or indirectly involved and that can be impacted by decisions and policies. Typical stakeholders in food and agriculture include the general public (i.e., consumers, primary shoppers), academia, government, industry, international, and non-profit organizations.

Decision methods such as multi-criteria decision analysis (MCDA) can incorporate and weigh the different stakeholders’ views, perceptions, and preferences systematically and transparently. These methods have been used to help improve decision-making in several areas such as engineering, environmental health (Linkov & Moberg, 2012), and food safety (Food and Agriculture Organization of the United Nations, 2017; Ruzante et al., 2010). However, before starting any approach such as MCDA, stakeholder preferences need to be captured and characterized.

The first step in stakeholder engagement is to identify key stakeholders in the field of interest and determine how and when they can be involved in policy development (Lemke & Harris-Wai, 2015). The motivation for engaging stakeholders and potential challenges expected during engagement activities also need to be defined. Many methods exist to engage, gather, and quantify stakeholders’ perspectives and views. Focus groups are efficient in engaging participants in a targeted manner and are often used to gather information and identify themes concerning participants’ perceptions, beliefs, and behaviors on a topic in a short timeframe (typically 60–90 minutes); however, it is also more difficult to maintain anonymity, which might be a disadvantage when it comes to discussing potentially polarizing or controversial issues. In-person focus groups are generally used to bring together individuals from a similar geographic area. Therefore, several focus groups need to be conducted to obtain representative points of view and ensure that geographically diverse groups have a chance to discuss their different points of view about an issue, being potentially expensive. Furthermore, focus groups might also need to be hosted outside working hours to include a representative group of participants, which might add a burden to researchers. In-person interviews and mail or online questionnaires are also used to gather stakeholders’ opinions, values, and perceptions.

With mail and online methods, respondent identity can be protected; however, it is often not possible to interact or converse directly with stakeholders on the subject at hand. Community forums and public comments are other methods used to gather stakeholder input, but these may lack the level of active interaction, depth, and dynamic discussion of focus groups and suffer from nonrepresentative samples. Additionally, consensus conferences also capture perceptions and decision-making under “high information” circumstances and provide insights from a more diverse sample of participants (Einsiedel et al., 2001). Consensus conferences have usually been convened to assess how non-experts might respond to a variety of controversial topics. A desirable method to gather stakeholder input would likely include attributes from all of these methods: geographically diverse participation, interactive group discussion, ease of administration, and anonymous participation.

Over the past 10 to 15 years, innovative online research platforms have been developed as alternatives to typical stakeholder engagement approaches, which in many ways come closer to having all desirable attributes. These online stakeholder engagement platforms (OSEPs) overcome many of the limitations related to in-person focus groups, questionnaires, and other methods. They allow researchers to engage geographically diverse stakeholders using a wide range of methods, from virtual focus groups to digital diaries and message boards (Hussey & Zerfas, 2020; Belpasso, 2019). Moreover, stakeholders can participate at their leisure—within the parameters set by the researchers—and from wherever is most comfortable for them. Another advantage of OSEPs is that they allow stakeholders to engage with one another anonymously, which can facilitate more open and uninhibited engagement on issues where they might have different points of view. This anonymity can also help mitigate issues around power dynamics, which can reduce participation by certain stakeholder groups or individuals who may defer to others who “know best.” Most of all, these OSEPs offer flexibility. They can accommodate both fixed-duration projects and efforts to generate ongoing or long-term engagement. CMNTY, EngagementHQ, Incling, The HiVE, and VisionsLive are a few examples of OSEPs that have made stakeholder engagement increasingly convenient for both researchers and stakeholders. These OSEPs have also made it easier to navigate stakeholder engagement during the COVID-19 pandemic, when gathering in person was not possible. One potential disadvantage is that they require stakeholders to have access to the internet, which can unintentionally limit participants’ full availability or privilege certain participant groups.

In this paper we demonstrate the potential benefits (from researchers’ and participants’ points of view) and the application of commercially available OSEPs to elicit stakeholder perceptions on topics where there is disagreement and the views of stakeholders are not clearly defined. In the case study presented, we used an OSEP to explore stakeholder views of responsible innovation (RI) in the context of nanotechnology applications in food and agriculture (nano-agrifood).

The inclusion of stakeholder perspectives is essential for responsibly designing and developing nano-agrifoods. In the case study described here, a commercially available OSEP was used to obtain stakeholders’ individual perspectives and their asynchronous interactions with other stakeholders assessing the same information. The goal was to connect with and learn from a wide range of stakeholders directly involved in nano-agrifoods, including members of NGOs, advocacy organizations, academia, industry, and government, using a single platform; however, it could have also been used to connect and engage the general public and consumers just as efficiently. Moreover, the platform supported a diverse array of research methods, which allowed the researchers to use a mixture of questionnaires and group discussions to obtain a rich dataset without overly burdening participants. As an example of this array of research methods, the researchers designed this study so that stakeholders first answered a series of questionnaires independently before engaging in a group discussion. Several study participants noted the advantage of this stepwise approach and felt that it facilitated a more robust discussion.

From the researchers’ point of view, using an OSEP allowed both individual questionnaires and a group discussion forum to be conducted on the same platform, increasing the ease of administration and tracking of participant progress. The platform also provided in-platform analysis capabilities that allowed researchers to identify emerging trends in the results from the multiple-choice questionnaires and an easy way to export results. Finally, the OSEP subscription included regular meetings with company employees who could answer questions about how to best integrate our research into the platform.

This study had several limitations that can serve as lessons learned for researchers planning to use OSEPs. First, according to participants, a weakness was that it lacked “live” or dynamic interaction. Although we used the discussion forum to engage stakeholders in a conversation, this forum is modeled after a message board in which participants respond at their leisure to a static prompt rather than in “real time” to moderated questions. The OSEP used here does offer a virtual focus group feature; however, we did not use this data collection method for the current study because the primary goal was to have all stakeholders participating in discussions together. Depending on the goals of the research study, it may be beneficial to utilize the “live” features of OSEPs to facilitate a more interactive engagement. Discussion boards could be timed for specific days or participants could be randomly assigned to a particular day to allow more time for exchange to facilitate engagement and restrict participants from posting all comments at one time.

Another limitation was that despite the instructions provided in the consent form and in message boards to describe what was expected of study participants, several individuals shared that they did not clearly understand the goals of the project or of the stakeholder engagement. Therefore, it is important to consider the use of multiple methods through which participants can engage in real time with researchers so that the goals of the engagement are better understood. Repetition of expectations throughout the activities might also be helpful when asking stakeholders to participate in an OSEP such as the one used here.

Although one advantage of OSEPs is that stakeholders can participate whenever is most convenient for them, future studies that utilize these platforms may find it beneficial to have predetermined, advertised windows of time (i.e., a specific time each day or week of the study) for research team members to engage with stakeholders in real time to answer any clarifying questions and facilitate conversations where needed to facilitate a “two-way” interaction between stakeholders and the research team as a way to address the previously discussed limitations. However, one disadvantage of ongoing researcher participation is that participants may be more reticent to express their views freely for fear of being incorrect or in deference to what they view as “experts” conducting the forum behind the scenes. There is likely an appropriate balance to strike in this regard.

Access to the internet was not an issue for the stakeholder groups targeted in this case study; however, depending on the groups that need to be involved in the discussion, reliable internet access needs to be considered so this does not become an excluding factor for participation and the engagement process can be truly inclusive. That would be an important factor to be considered if using these platforms to engage with the general public.

Overall, our use of an OSEP resulted in positive experiences for most participants, both for being part of research evaluating real-world applications of nano-food innovations and for furthering their own thinking about RI in the food sector. This particular OSEP provided a reliable and engaging environment for stakeholders to exchange and discuss ideas and could be further explored by policy makers and other decision makers as an innovative tool to engage different types of stakeholders.

The authors would like to state that there are no conflict of interest involving authors and CMNTY, the software tool used in the case study and cited in this paper.

The authors would like to acknowledge Christopher L. Cummings from North Carolina State University for his contributions to the development of the stakeholder platform activities and study design. This research was conducted with funding from the United States Department of Agriculture, National Institute for Food and Agriculture.

RTI Press Associate Editor: Barbara B. Biesecker