Public and Stakeholder Engagement in Developing Human Heritable Genome Editing Policies: What Does it Mean and What Should it Mean?

As scientific research pushes the boundaries of knowledge, new discoveries and technologies often raise ethical and social questions. Public responses vary from surprise, to unrealistic optimism about imminent new treatments, confusion, and absolute opposition. Regardless of the intent, the use of a precise gene editing tool on human embryos, such as CRISPR-Cas9, is an example of such a controversial emerging technology. Substantive disagreement about the appropriate research pathways and permissible clinical applications is to be expected. Many ethical concerns, especially related to genetic manipulation of human embryos, are rooted in deeply held moral, religious, or ideological beliefs that science alone cannot address. Today, more scientists and scientific societies as well as policy makers are calling for public and stakeholder engagement in developing guidelines and policies governing scientific practice. We conducted a critical interpretive review of the literature on public and stakeholder engagement in science policy development regarding emerging technologies to determine the ideals that should guide engagement efforts of entities developing recommendations or guidelines on policy for such technologies. We identify and describe five ideals. To illustrate possible applications of these ideals, we review the engagement efforts described in three reports on heritable human genome editing and assess those efforts in light of these ideals. Finally, we recommend possible avenues for engagement that would advance those goals.


INTRODUCTION
Scientific research and technology continue to push the boundaries of what we know and what we can do. However, these changes often raise new ethical and social questions. One example is CRISPR-Cas9 and its use in heritable human genome editing (HHGE). While discussions regarding HHGE date back for decades, many were relegated as 'science fiction' due to limitations in technological feasibility (Frankel and Chapman, 2001;Evans, 2002;Dresser, 2004). More common were discussions related to ethical issues associated with clinical uses of genetic technologies, such as gene transfer technology, that was not expected to alter the germline (King and Cohen-Haguenauer, 2008).
In 2012, CRISPR technology was introduced and over the past decade publications and research using CRISPR has exploded (Doudna and Charpentier, 2014;Ledford, 2015). CRISPR is a precise and easy to use gene editing tool which allows for the manipulation of DNA within cells and has potential clinical uses. By 2015, scientists already began publishing research using CRISPR to edit genes in human embryos (Cyranoski and Reardon, 2015;Liang et al., 2015;Ma et al., 2017). Despite the fact that none of the embryos were transferred for gestation, these experiments were controversial and led to substantive disagreements regarding if and how the technology should be used on humans, resulting in calls for additional discussions and international fora and even a call for a moratorium on the research (Baltimore et al., 2015a;Baltimore et al., 2015b;Hurlbut et al., 2015;Kaiser and Normile, 2015;Landphier et al., 2015;Pollack, 2015).
The controversy regarding HHGE came to the forefront of public attention in November 2018 when Chinese scientist HE Jiankui announced that he had used CRISPR to edit a gene in human embryos and transferred them into women, resulting in two twins born in 2018 and a third child born later (Regalado, 2018;Begley and Joseph, 2018). The public seemed shocked by the announcement, which was followed by a flurry of media attention on HE Jiankui, the experiments and anyone associated with them (Regalado, 2018;Begley and Joseph, 2018;Begley, 2018;Begley, 2019;Cohen, 2019). Scientists were also taken aback by the experiments. CRISPR discoverer Jennifer Doudna described being "horrified," United States National Institute of Health (NIH) director Francis Collins found the experiments to be "profoundly disturbing," and Nobel laureate David Baltimore, said it was "a failure of self-regulation by the scientific community" (NASEM, 2019).
These events highlight the need to better understand the public's and stakeholder concerns related to emerging technologies. When scientists move beyond what the public deems acceptable, public backlash can be significant and at times could also undermine research the public otherwise would deem legitimate, such as the use of CRISPR on adult somatic cells which do not contribute to the germline. Following the 2018 incident, many scholars called for further discussions regarding acceptable practices regarding HHGE as well as increased public or stakeholder engagement (PSE) on what research should and should not be conducted Saha et al., 2018;Hurlbut, 2019;Lander et al., 2019;Matthews and Iltis, 2019).
Beyond HHGE, calls for PSE linked to science policy development for emerging technologies have arisen in recent years for topics ranging from nanotechnology, human embryo research, and shale gas to vaccine mandates (International Society for Stem Cell Research, 2021;Jones, 2014;Norheim et al., 2021;Pham, 2016;NRC, 2012;NASEM, 2017;North et al., 2014;Jasanoff, 2004;Warnock, 1984). However, it is often unclear what PSE is, what its goals are, how to achieve them, and ultimately how the data collected from PSE can be used effectively to inform policy recommendations and decision making.
PSE is an important part of science policy development, especially when reviewing controversial areas that concern deeply held moral and religious belief and areas where there are significant ambiguities or uncertainties, such as HHGE. Genetic research has had a long history of PSE, especially after the human genome project started and the US NIH began funding ethical, legal and social impact research related genetic research. Understanding patients' and public concerns helps to highlight issues that scientists or physicians may not otherwise address such as the right of access to research findings, equitable representation in research, or determining what is disability versus diversity when viewing genetic differences (McGuire et al., 2020).
In order to successfully develop public policy, PSE must be conducted effectively and thoughtfully, being as inclusive as possible to obtain the often numerous and divergent views found in a pluralistic society. Otherwise, it runs the risk of missing major public questions and concerns or not defining the appropriate issues related to the technology. Ultimately, science policy is implemented by policymakers and not committees issuing recommendations or guidelines. If policy recommendations fail to address public concerns, especially in the United States with polarized politics, they are likely to be ignored or result in unintended limitations to the broader research field.
In this paper, we identify five ideals that should guide PSE efforts when developing science policy recommendations or guidelines on emerging technologies. These ideals emerge from a critical interpretive review of the PSE literature on science policy development, especially those focused on controversial issues. We use these ideals to assess recent engagement efforts described in three seminal reports on human heritable genome editing (HHGE) from the United Kingdom Nuffield Council on Bioethics (NCB), the United States National Academies of Science Engineering and Medicine (NASEM), and the collaboration between NASEM and the United Kingdom Royal Society (NASEM-RS) with an international commission (National Academies of Sciences, Engineering, and Medicine (NASEM), 2020; NASEM, 2017; NCB, 2018). These three reports were selected as they provide the most recent work on HHGE and are used to guide scientific research, especially where local oversight is not robust. The purpose of analyzing these three reports is to illustrate how these five ideals can be understood in practice and used to inform future efforts in the science policy development arena. Reviewing these efforts (and associated public documents), we identify gaps, and recommend improvements that would advance the stated PSE goals from each report. We argue that the efforts made by these three groups, while notable, were not always adequate and more robust PSE efforts are warranted going forward.

DEFINING PSE FOR SCIENCE POLICY DEVELOPMENT
The terms 'public' and 'stakeholder' often are used interchangeably, especially when referring to PSE. However, they are distinctly different groups in terms of how science policy affects them. For this paper, we use the term 'stakeholders' to refer to "interested or affected parties," who often are organized into groups (North et al., 2014). For HHGE, stakeholders include scientists who conduct the work or are in the broad developmental biology field interested in the results. HHGE stakeholders also include patients and their advocates who believe they or other similarly affected individuals in the future would benefit from gene editing as well as those who donate their gametes or embryos for this research and disability advocates who see genetic variants and "mutations" as forms of human variation that do not need to be "fixed." In addition, stakeholders include those who fund the research (public and private entities), regulate the research, conduct broader social science research on the subject, and hold strong ethical, moral or religious beliefs related to genome editing.
In contrast, we use 'public' as a proxy for the general public who might not have previous knowledge or experience of a topic or are not recognized as specialists (Lezaun and Soneryd, 2007;North et al., 2014;Nuffield Council on Bioethics (NCB), 2012;Reed et al., 2018). Some authors prefer the term 'publics' to avoid the implication that the public is a homogeneous group representing a single set of experiences and perspectives. Our use of the singular is not meant to obscure differences among public perspectives.
Our use of the term PSE, therefore, intentionally covers a broad range of populations. Where there is reason to distinguish between the public and stakeholders, we do so. We hold that including both stakeholders and the public is important in science policy development regarding emerging technologies.
It is important to note what PSE is and what is not. PSE is often confused with public outreach. Outreach is one-way communication with the public. Examples include printed or digital educational materials and lectures open to broad audiences with little to no audience interaction beyond answering a few questions after a presentation. While outreach is important to explain new research and developments in science, it is not PSE. Nevertheless, at times unidirectional communication is erroneously identified as part of PSE.
PSE requires multi-way communication or a dialogue among scientists, stakeholders, and/or the public, such as a presentation of new ideas (a lecture or publication) followed by facilitated discussion. It requires listening and synthesizing outside information, perspectives, and thoughts in the process of developing recommendations or policy (Pieczka and Escobar, 2013). This is especially important regarding developing public policy for controversial issues in science including HHGE.
A wide variety of mechanisms have been used for PSE. Key differences among the mechanisms include whether they are asynchronous or synchronous (live versus recorded), the level of participant activity (from passive to active engagement), who is intentionally included and likely to participate, and whether their primary purpose is to secure consensus or map perspectives and identify issues. Some PSE is invited, through speakers, public comments or calls for information. This is especially common when specific stakeholders and views have been predetermined to be important. Other engagement is more open, allowing uninvited members of the public to share their opinions and perspectives. These exchanges permit those who were missed or overlooked to participate and can remove potential bias that can occur when selecting stakeholders.
Justifications and goals of PSE can inform the design of future efforts and the assessment of past efforts (Stirling, 2012). Some might see PSE instrumentally, as a tool to promote research, dispel myths, or avoid public backlash (National Research Council (NRC), 1996). Some PSE advocates point to the role such efforts play in building trust in science and an appreciation for the legitimacy and importance of scientific research among people with different points of view. PSE may help to secure funding, increase acceptance of results, reduce controversy, and, where relevant, improve adherence to scientific recommendations (Adashi et al., 2020;Kyle and Dodds, 2009;NASEM, 2017; National Academies of Sciences, Engineering, and Medicine (NASEM), 2020; Norheim et al., 2021;NRC, 2012;Pham, 2016). Another reason for PSE is that public involvement in deliberations is required as a matter of principle in a democratic society. Scholars have suggested that people who are affected by a decision should have a fair opportunity to participate in decision-making (Adashi et al., 2020;Kyle and Dodds, 2009;Irwin, 2014;Neuhaus, 2018;Norheim, et al., 2021;National Research Council (NRC), 1996). For others, the primary justification for PSE is that science is public good and a social enterprise that is not only informed and shaped by society but also transforms society (Jasanoff, 2004). Experts from various disciplines are needed to explore the possible implications of scientific developments, particularly as they relate to social and economic effects (Adashi et al., 2020; Nuffield Council on Bioethics (NCB), 2012; van Est, 2011). The public and stakeholders can help to determine how science can most effectively respond to or advance public interests and to make the best decisions possible. Including the public can help identify the challenges to which science should respond and inform the goals of science (Jones, 2014;Barbosa et al., 2020).
Furthermore, science shapes society with the introduction of new knowledge and developments, and with the allocation of social resources used to fund research in lieu of other social goals. In turn, society shapes science by guiding research priorities and establishing regulations and laws that authorize or prohibit different practices (Jones, 2014). For example, the field of global health research emerged after societal pressure and funding from outside of the traditional scientific enterprise, such as the Bill and Melinda Gates Foundation, encouraged researchers to shift their focus (Matthews and Ho, 2008). Understanding science as a public good and social enterprise, a view defended convincingly by numerous science and technology scholars, supports the importance of robust PSE in science policy development. It can improve the quality of scientific research, protect affected parties, and lead to better, more relevant results (NASEM, 2017; National Academies of Sciences, Engineering, and Medicine (NASEM), 2020; NCB, 2018; Norheim et al., 2021;North et al., 2014;NRC, 2012

METHODS
To determine a set of ideals for PSE in science policy, we conducted a critical interpretive review of the literature on PSE in science policy development regarding controversial emerging technologies. The critical interpretive review methodology was developed in bioethics, where the relevant literature comes from multiple disciplines, to capture "key ideas from existing literature" to answer a research question (McDougall 2015, p. 525). We sought to answer the question: What ideals, norms, or principles should guide efforts to engage the public or stakeholders when developing science policy regarding controversial emerging technologies? The critical interpretive review methodology was more appropriate for addressing this question than a systematic review, a methodology designed to capture all relevant studies on an intervention to assess the intervention, because our focus was not comparing the effectiveness of specific PSE approaches or techniques (McDougall, 2015). Our goal was to identify the ideals that should guide PSE processes overall. An additional reason for choosing the critical interpretive review method was that literature from multiple disciplines and different types of publications, including journal articles, reports, and books, would be relevant to answering our research question. Two authors (KRWM and ASI) manually searched the literature and identified 24 publications for inclusion. A third author (SH) received instruction on data coding, extraction, and reporting, and all three authors read these publications and discussed them to begin to identify themes and inform a more rigorous literature search. One author (ASI) conducted a literature search using embase and PubMed on June 1, 2021. The search terms and results are reported in Table 1 and Table 2.
The search resulted in a total of 1,084 publications (743 from embase and 267 from PubMed). The 24 publications identified manually were added to this group for a total of 1,034 publications considered. After duplicates were removed, 988 records remained. ASI screened the titles and abstracts of all records. To be included, publications had to address a controversial emerging technology or the concept of emerging technologies and public policy. They also had to at least implicitly address one or more norms, principles, or ideals that PSE should meet or one or more goals, purposes, or justifications of PSE that could shape our understanding of the principles, norms, or ideals that should inform PSE. Initial review led to exclusion of 819 records. The remaining 145 articles were read and assessed for eligibility. After removing 127 publications, 42 publications remained and were used in answering the research question. All three authors reviewed publications to identify key themes and shared findings using Google Docs. Some publications were read in full by all three authors, some were read in full by two authors, and in a few cases, only one author read the full text. Through critical discussion of the findings, the five ideals discussed below were identified. Figure 1 reports the search and screening process.

FIVE IDEALS FOR EFFECTIVE PSE
Many scholars have defended the need for PSE and discussed methods for effective PSE (Adashi et al., 2020;Burgess, 2014;Guston, 2014;Jasanoff, 2003 andJones, 2014;Kouper, 2010;Kyle and Dodds, 2009;Neuhaus, 2018;Nisbet, 2009;Norheim et al., 2021;Pham, 2016;Pieczka and Escobar, 2013;Selin et al., 2017;Stilgoe et al., 2014;Stix, 2021;Trench, 2006;Varner, 2014;  Wilsdon and Willis, 2014). Our analysis of the literature revealed five ideals for effective PSE that should guide committees when conducting assessments and making policy recommendations or decisions regarding emerging, and especially controversial, areas of science research. As defined and described in Table 3, PSE should be: 1) comprehensive, 2) transparent, 3) inclusive, 4) methodologically sound, and 5) accountable. These ideals can ensure that PSE improves decision making, especially around controversial emerging technologies, by addressing the right issues and engaging the broadest audience, including marginalized and often missed voices, to improve the quality of decisions and increase trust and legitimacy of guidelines, recommendations, and policies (Norheim et al., 2021). Using these ideals will also allow the resulting recommendations to have a stronger public policy impact, which often relies on, especially in the United States, public approval as many of the policymaker implementing the recommendations are publicly elected. Unfortunately, many activities labeled as PSE do not accomplish or reflect all five ideals. Some PSE relies on the deficit model, which assumes that members of the public are ignorant and that if they understood the science more fully, they would approve of it (Trench, 2006;Irwin, 2014;Jones, 2014;Simis et al., 2016;NASEM, 2017). This usually leads to unidirectional outreach that consists of experts explaining science rather than true PSE. Such activities fail to meet the goals, justifications, and ideals of PSE, and they do not qualify as PSE as we have defined it.
Other activities capture some of the ideals but not all. For example, common mechanisms employed by the US federal government include issuing notices of proposed new rules in advance with an open-comment period and publishing responses to the comments; live-streaming committee meetings, town hall meetings or other open assemblies (and posting the recording); and developing standing advisory panels that include experts and non-experts (NASEM, 2017;Norheim et al., 2021). These methods can be comprehensive, transparent, and methodologically sound, when properly deployed. However, they have limited inclusion insofar as they only passively seek feedback from the public. There is minimal or no dissemination or push for broad participation as they require the public to be proactive and find the announcements in the public registry on their own. Much of the US government communication also presumes a high literacy level, further limiting participation.
Some PSE models, such as the Expert and Citizen Assessment of Science and Technology (ECAST), serve as better examples (Weller et al., 2021). In an effort to find ways to engage citizen participation to improve science and technology policy, ECAST conducts participatory technology assessments on topics such as biodiversity, climate change, and NASA's asteroid project. Citizen participants identify questions and share feedback on emerging developments in science and technology, ultimately to help drive more thoughtful policymaking. The model can be comprehensive (participants choose the direction of the discussion and discussions can begin early), transparent, methodologically sound and accountable, allowing one to see what transpired as well as follow up on results. In addition, the organizers make efforts to be inclusive by advertising broadly and encouraging participation from often missed voices. PSE efforts regarding human embryo research and in vitro fertilization (IVF) in the United Kingdom and United States during the 1970s and 1980s also can inform present efforts. They were comprehensive, transparent, inclusive, methodologically sound and accountable. In 1978, the Ethics Advisory Board of the United States Department of Health, Education and Welfare (now the Department of Health and Human Services) was tasked with determining whether IVF and embryo research were ethically acceptable. The board requested comments (written and oral) from stakeholders in various related fields, conducted 11 hearings across the country in nine cities and received more than 2,000 documents which were reviewed by the committee. A similar committee was created to review IVF in the United Kingdom, led by Dame Mary Warnock (Warnock, 1984). The committee spent a year hosting public and private meetings, collecting evidence and opinions from different stakeholder and public perspectives on IVF. After the report was released, the committee gathered additional public feedback (Hammond-Browning, 2015). Both committees developed recommendations, which have lasted for decades, on what types of human embryo research should be permitted. In addition to the recommendations, both also described their deliberation processes. The United Kingdom report, known as the Warnock Report, led to the Human Fertilisation and Embryology Act of 1990 and the creation of the Human Fertilization and Embryology Authority (HFEA) to oversee IVF and human embryo research in the United Kingdom (Matthews and Moralí, 2020). While the United States report did not lead to policy change, it was the first to recommend a limit on human embryo research to 14 days after fertilization, which has been implemented in policies across the world (including the United Kingdom) (Matthews and Moralí, 2020).
For PSE to be successful, it should strive to achieve all five ideals. Time or budget restrictions can make it challenging to fully develop all aspects. But, with advanced planning and some imagination by those organizing the PSE, many, if not all, could be achieved.

PSE FOR HHGE AND PUBLIC POLICY DECISION MAKING
HHGE has the potential to affect all of society and raises a host of ethical issues. Because of the controversial nature of the research as well as its broad social impact for generations, PSE is an important aspect of policy and guideline development. Discussions about the use of gene editing technology are not new, some dating back to the 1960s (Evans 2002;Dresser 2004). Beyond questions related to whether the research ought to be conducted, there has also been discussion regarding which genes should and should not be considered for engineering. These discussions are directly or indirectly influenced by decadeslong discussions related to eugenics, once hailed as the key to a healthy population and now condemned (Cavaliere 2018). Because of the nature of this research, scientists have often argued for caution related to HHGE as well as strong policies

Ideal Definition
Comprehensive Engagement should begin early in the process of scientific inquiry and prior to technology development, if possible. Public and stakeholder input should inform the questions committees examine and focus on broad questions about the direction of science as well as on particular applications of scientific findings or new technologies Transparent All relevant information should be disclosed in a timely and accessible fashion, including: who developed and sponsored the work; who was invited to participate and why they were included; what processes for soliciting feedback were followed; what input was given. Mechanisms should be in place to learn about the meetings, the deliberative process, including disagreements among committee members and unresolved questions, as well as ways to provide uninvited input Inclusive Stakeholders and members of the public should be allowed a fair opportunity and encouraged to participate. Information must be communicated in ways that are understandable to people with different levels of literacy and science literacy. People with a broad range of beliefs, perspectives, backgrounds, and experiences should be welcomed and encouraged to participate, including members of groups who have traditionally not been able to participate meaningfully in PSE. Easily accessible mechanisms should be available to provide input, even by people not specifically invited to participate Methodologically Sound Engagement should be conducted using evidence-based methods that are aligned to specific project goals or objectives Accountable Activities should be assessed using standard metrics to evaluate the extent to which they meet the specified objectives, such as inclusion, and fulfill the other ideals as well as to determine whether the ultimately inform the recommendations developed. This includes documenting processes for receiving and reviewing input. Insofar as some input is dismissed or not incorporated, these decisions should be documented and justified Sources: Adashi et al., 2020;Barbosa et al., 2020;Burgess, 2014;Cormick, 2009;Fisher, 2011;Guston, 2014;Haywood and Besley, 2014;Heidari et al., 2016;Irwin, 2014;Jones, 2014;Jasanoff, 2003;Jasanoff, 2004;Kyle and Dodds, 2009;Kaner et al., 2014;Kouper, 2010;Lezaun and Soneryd, 2007;Longstaff and Secko, 2016;NASEM, 2017 (p 110, 182);NRC, 2012;Neuhaus, 2018;Nisbe,t 2009;Norheim et al., 2021;North et al., 2014; Nuffield Council on Bioethics(NCB), 2012; Office of Health Equity (OHE), 2019; Pham, 2016;Posner et al., 2016;Reed et al., 2018;Scheufele et al., 2021;Selin et al., 2017;Smith et al., 2021;Stilgoe et al., 2014;Stix, 2021;Stirling, 2008;Stirling, 2012;Stix, 2021;Sturgis, 2014;van Est, 2011;Varner, 2014;Warnock, 1984;Weller et al., 2021;Wilsdon and Willis, 2014. Frontiers in Political Science | www.frontiersin.org September 2021 | Volume 3 | Article 730869 6 to guide the research (Baltimore et al., 2015a;Landphier et al., 2015;Lander et al., 2019).
To address HHGE research and its potential clinical use, three major recent reports assessed the issue and made policy recommendations: the United Kingdom NCB, 2018 report, the United States NASEM, 2017 report, and the 2020 joint report by NASEM and the United Kingdom Royal Society (NASEM-RS) with an international commission. Each report developed a consensus document from a committee that reviewed existing scientific research and knowledge as well ethical and policy challenges. As part of their mandate, each committee indicated that they engaged stakeholders and the public as part of developing their recommendations.
The 2018 NCB report was linked to an earlier NCB assessment of genome editing published in 2016. Both reports were associated with public outcry after the 2015 publication by Chinese scientists using CRISPR to edit human embryos (Cyranoski and Reardon, 2015;Liang et al., 2015). The 2016 report reviewed the current state of gene editing and major concerns across several different fields of research (NCB, 2016). However, the committee determined that HHGE required a detailed assessment of its own. As a result, a second committee was formed, releasing their report in 2018 (NCB, 2018). This committee was guided by a working group with eight members with experience in developmental and cellular biology, law, sociology, and bioethics. As part of their assessment, the committee directed surveys to specific individuals and organizations, conducted a public survey, held fact-finding meetings with experts in associated areas (including developmental biology, law and bioethics), and interviewed reproductive and disability rights advocates ( Table 4). They also relied on research conducted by the 2016 committee that also included PSE. The final 2018 report as well as associated documents (including survey questions and responses) are available on the NCB public website.
Scientists at the 2015 International Summit on Human Gene Editing called for additional discussions on HHGE, resulting in the 2017 NASEM report (Baltimore et al., 2015b;NASEM, 2017). The committee included 22 members with expertise in basic science, clinical research and medicine, law and regulation, ethics and religion, patient advocacy, and the biomedical industry, with seven members from outside the United States. The group held four public meetings that included public comment sessions (three in Washington, DC and one in Paris), and invited 37 expert speakers ( Table 4). The final report, committee information, and videos of public hearings are available on the NASEM website.
After the 2018 announcement of the birth of twins with genetically altered DNA at the Second International Summit on Human Genome Editing, there was another international call for renewed discussions regarding the permissibility of HHGE. As a result, the NASEM collaborated with The Royal Society (United Kingdom) to form the "International Commission on the Clinical Use of Human Germline Genome Editing" (National Academies of Sciences, Engineering, and Medicine (NASEM), 2020). This group had 18 members, which included experts in biological science, medicine, ethics, psychology, regulation and law from 10 different countries. They held two public meetings (Washington, DC and London) with 44 invited experts, four webinars, and hosted a "public call for evidence during fall 2019," which received 83 responses ( Table 4). The final report, committee information, and videos of public hearings are available on the NASEM website.
In this section, we assess the PSE efforts described in these three reports in light of the five ideals and identify areas for improvement ( Table 3). Analyzing these reports allows us to not just define the ideals, but to determine how they are or are not used conducting PSE for science policy development for emerging technologies. For the assessment, we reviewed what was discussed specifically within the report and in publiclyaccessible materials including (but not limited to) websites, videos of public sessions, and public documents associated with the committee. While each report described some PSE activities and its importance, as illustrated below, the PSE activities reported did not satisfy all five ideals: 1) comprehensive, 2) transparent, 3) inclusive, 4) methodologically sound, and 5) accountable. Entities assigning tasks and charges to committees for science policy should use these ideals in establishing the parameters for committees' work, including allowing sufficient time and resources for effective PSE.

Comprehensive
The ideal of comprehensiveness applies both at a broad level (when and how decision-makers call for PSE) as well as the scope of PSE efforts committees are tasked with (Fisher, 2011;Kyle and Dodds, 2009;Scheufele et al., 2021;Smith et al., 2021;Stix, 2021;van Est, 2011). One measure of the former is how far along research has progressed before PSE begins. Unfortunately, these efforts to develop recommendations regarding HHGE policy in the United States and United Kingdom did not begin until after the 2015 publication reporting the first human embryo to be edited (Cyranoski and Reardon, 2015;Liang et al., 2015). The 2017 NASEM and NCB committees were organized soon afterwards, while NASEM-RS report was a response to the HE Jiankui CRISPR-edited baby announcement (Cohen, 2019). While other policy reports have been produced in the past, all three of the selected reports were started after major research boundaries were crossed. This was not the responsibility of the committees but rather speaks to the failure in the science policy community to address these issues early.
Comprehensiveness also refers to the breadth of the questions and topics considered. Of the three committees, two focused the scope of their work on research governance (NCB and NASEM-RS reports). Notably, they presumed that HHGE would progress and did not seem to consider any possibility of halting it. The NCB committee focused on the nature of the genome and genome interventions relative to other technologies, the obligations of scientists to society, and the principles that should inform legal and regulatory frameworks governing genome editing as well as questions about the application and possible impacts of genome editing. The NASEM-RS committee's charge was to "defin [e] a responsible pathway for clinical use of HHGE, should a decision be made by any nation to permit its use" (NASEM-RS, 2020) The topics covered by the experts were predominantly related to how and for which conditions HHGE could and should be used.
In contrast, the NASEM committee was charged to "examine the scientific underpinnings as well as the clinical, ethical, legal, and social implications of the use of human genome-editing technologies in biomedical research and medicine" (NASEM, 2017). The committee interpreted this charge as excluding most discussions of whether HHGE research ought to be conducted, and focused on issues related to the governance of HHGE. In its final meeting the committee did expand its dialogues to include relevant social considerations, such as what implications the United States' history of race and genetics might have for genome editing and how moral views and public policy are or should be connected. Other ethical and moral questions had little to no discussion. However, in assuming that HHGE research would continue and by limiting discussion to how the research should be conducted rather than remaining open to broader questions about the permissibility of such work, the three committees missed an opportunity to understand why some publics oppose the research. As a result, the recommendations were not responsive to some significant concerns.

Transparent
Transparency is an important part of trust-building for committees reviewing and assessing science policy (Longstaff and Secko, 2016;Norheim et al., 2021;Posner et al., 2016;Wilsdon and Willis, 2014). All three committees' made efforts to be transparent; they developed websites with the charge and purpose of the committee, the meeting schedule, and the final reports; made the working group memberships known; and included lists of experts consulted. The NCB was the only committee to share a public list of the individuals and organizations who submitted comments as wells as the public survey questions, dissemination plan, and results on their website (NCB, 2017;NCB, 2018). The NASEM and NASEM-RS committees held public meetings that were live-streamed with links on their websites for future viewing. The NASEM committee also shared information through social media.
None of the committees adequately explained the methodology used to select experts for input. NCB engaged experts in meetings, surveys, and interviews, but did not explain why particular experts were included in different PSE activities. NASEM-RS had a public survey, but unlike the NCB, they did not publish the survey questions, responses or how the survey was publicized or disseminated in the report or on the websites. While the NCB reported reviewing comments during their eighth meeting, the NASEM and NASEM-RS committees did not mention how or if public comments or survey responses were integrated in their deliberations, only offering that "information provided to the Commission [committee] from outside sources or through online comment is available by request through the National Academie's Public Access Records Office" (NASEM, 2017; National Academies of Sciences, Engineering, and Medicine (NASEM), 2020). After repeated requests for materials, we emailed leadership at the NASEM and received NASEM's information provided to the NASEM and NASEM-RS committees. These materials included Frontiers in Political Science | www.frontiersin.org September 2021 | Volume 3 | Article 730869 8 speaker slides from the public sessions, 28 comments sent to the 2017 NASEM committee (via the current projects system), the questions for the NASEM-RS survey and seven invited responses to the NASEM-RS committee. Missing were the public comments received and the online de-identified submissions to the public call for evidence for the NASEM-RS committee.
Inclusive PSE should seek a broad range of sometimes divergent views and opinions on subjects, including people with different backgrounds, expertise or experiences (Adashi et al., 2020;Barbosa et al., 2020;Cormick, 2009;Fisher, 2011;Heidari et al., 2016;Kyle and Dodds, 2009;Neuhaus, 2018;Scheufele et al., 2021;Smith et al., 2021;Stirling, 2012;Stix, 2021;Sturgis, 2014). Each committee attempted to be inclusive. All three committees included people with expertise in biology, sociology, bioethics and the law; however, the NASEM-RS committee was predominately biologists with only three nonbiologists (out of 18 members). To obtain outside perspectives, each committee invited commentary from additional experts, NCB and NASEM-RS conducted public surveys, and NASEM and NASEM-RS held open hearings with public comments sessions and had email addresses publicized for comments. One particularly noticeable omission appears to have been soliciting input from representatives of faith communities.
NCB's survey was sent to several governmental and nongovernmental organizations, advocacy groups, and the Royal Society to help publicize it and increase participation. They received 320 responses. In addition, the NCB committee invited feedback on a series of questions from selected groups of stakeholders, obtaining responses from seven individuals and seven organizations. This allowed for diverse voices and opinions to be captured, as seen in their public documents. They made significant efforts to be inclusive.
In contrast, the NASEM and NASEM-RS's PSE activities seemed less inclusive, relying on more passive methods, which made it less likely to have broad participation. Both allowed comments during public sessions, but these sessions were poorly attended, often with only two or three people providing in-person comments and three or four online submitted comments being read or summarized by a committee or staff member. Both committees hosted invited expert speakers. The NASEM committee heard from a few speakers on ethics, focused largely on the United States history of racism and eugenics. Most of the NASEM-RS experts were doctors and scientists (29 of 44) with only three patient advocates. As noted previously, this imbalance suggests that NASEM and the Royal Society saw governance of HHGE their primary focus rather than broader questions about permissibility.
The NASEM-RS committee was geographically diverse and included representatives from 10 countries: Canada, China, France, India, Japan, Malaysia, South Africa, Sweden, the United States and the United Kingdom. The committee PSE included a public survey and four webinars. The webinars only allowed committee members were allowed to ask presenters questions, no public questions although anyone was allowed to watch the webinar. There was limited information on the public survey beyond the questions posed (responses nor demographic information about respondents were included in the public materials NASEM shared). They did note obtaining 83 responses "from every continent and included academic leaders, lawyers, social scientists, philosophers, and representatives from disability advocacy groups, journals, national ethics councils, industry, and scientific societies" (National Academies of Sciences, Engineering, and Medicine (NASEM), 2020). There was no indication that the survey was translated into any other language outside of English. Considering that a goal for the report was for it be adopted globally, making the survey and other materials available in different languages would have made international engagement more meaningful.

Methodologically Sound
PSE activities should be methodologically sound, utilizing evidence-based methods aligned to achieve specific PSE goals (Scheufele et al., 2021). However, none of the reports note why particular methods were adopted. Reviewing other NCB and NASEM reports indicates that the methods chosen are used frequently for reports at NCB and NASEM. NCB often uses similar methods for its reviews: public and stakeholder surveys and interviews, invited experts for comments, and project websites. The 2018 NCB report indicated that there are assessments of these methods during the process, specifically noting that the public survey deadline was extended to advertise it on social media in an effort to increase participation from younger individuals and those with lower educational achievements.
The NASEM and NASEM-RS methods were also consistent with other NASEM projects: project websites, specific email addresses, and recorded open sessions with time for public comments. In fact, both reports used almost identical language to describe their PSE: The committee's [Commission's] data-gathering meetings provided opportunities for the committee [Commission] to interact with a variety of stakeholders. Each public meeting included a public comment period, in which the committee [Commission] invited input from any interested party. The committee [Commission] also worked to make its activities as transparent and accessible as possible. The study website was updated regularly to reflect the recent and planned [Commission] activities [of the committee]. Study outreach also included a study-specific email address for comments and questions. A subscription to [regular] email updates was available to share further information and solicit additional comments and input to the committee [Commission]. Live video streams with closed captioning were provided throughout the course of the study to allow the opportunity for input from those unable to attend meetings in person . . . [I]nformation provided to the committee [Commission] from outside sources or through the online comment tool is available by request through the National Academie's Public Access Records Office. (NASEM, 2017, p.275-276;NASEM-RS 2020, p.188-189).
While consistency in PSE is encouraging, this near duplication of language in the NASEM-RS and NASEM reports could indicate that PSE is not as much a priority as a box to check. NASEM is required by United States law some level of transparency and openness in order to be comply with the United States Federal Advisory Committee Act (FACA) guidelines. Some of their PSE can be interpreted as aimed at compliance rather than reflecting a deep commitment to PSE. Moreover, most of the PSE approaches implemented were passive. They posted meetings on their website and expected individuals to find the meetings or calls on their own or come from a pre-identified list of names or emails interested in the topic. They did not indicate any new methods to increase survey or comment participation, nor did, for example, NASEM-RS note reviewing survey responses to ensure broad participation (like NCB did). Furthermore, it is unclear why these PSE mechanisms were used over other widely recommended methods, why surveys were not included in the 2017 NASEM report, or why the NASEM-RS survey had such a poor response (NASEM 2017;Norheim et al., 2021). Different methods likely would have been required to obtain participation from more diverse stakeholders and a broader segment of the public.

Accountable
To be accountable, PSE efforts should include plans for assessment and be assessed both to ensure they are meeting their goals and to measure their impact on recommendations or guidelines issued (Cormick, 2009;Stilgoe et al., 2014;Heidari et al., 2016;Longstaff and Secko, 2016;Selin et al., 2017;Neuhaus, 2018;Scheufele et al., 2021;Stix, 2021). This includes evaluating whether a committee was successful in meeting the other four ideals (comprehensive, transparent, inclusive, and methodologically sound): did they addressed the range of relevant questions and issues, operated and reported their work transparently, reached a broad range of stakeholders and members of the public, and adopted methods that were suited to particular goals? Further, if they failed to meet all ideals, where their reasons and rationales for doing so?
Accountability also means determining the extent to which PSE activities and the information gleaned through those efforts helped shape the recommendations or guidance. If PSE is conducted effectively and integrated within the committee deliberations, the policy recommendations should reflect public and stakeholder input, making it more likely that they will be actionable for policy makers. Policy makers, especially elected policy makers that rely on public support and votes, are often less likely to adopt recommendations that do not adequately reflect public and stakeholder priorities and concerns.

CONCLUSION
PSE is an important part of policymaking for science and technology research and development (Jones, 2014;Posner et al., 2016). As the public are often both the funders and the users of the products of research, their participation in goal setting and establishing research boundaries is vital for science policy to serve public interests and for the public to accept and support emerging research.
Assessing three reports related to HHGE based on the five ideals of PSE-comprehensiveness, transparency, inclusiveness, methodological soundness, and accountability-we found some successes and failures. Deadlines likely limited the PSE efforts each committee could undertake. On average, each committee had less than a year to complete the projects. This timeline was extremely ambitious considering the committees hosted multiple meetings, conducted surveys, collected data, reviewed relevant literature, developed recommendations, and wrote a consensus report.
Often PSE is approached from the perspective that if one explains science effectively enough, then the public will approve of it. As a result, skepticism or questions regarding research are often viewed and labeled as anti-science, even if they do not challenge scientific knowledge (Stirling, 2008). Public discussions also make some scientists uncomfortable when they project the future trajectory of technology and consider possible long-range applications that scientists cannot predict (Stirling, 2008;Jones, 2014). These discussions can get complicated and result in recommendations that the research community does not want because they restrict basic research. For the case of HHGE, public dialogues could find a public uninterested in pursuing the technology at this time, perhaps prematurely limiting research, from the scientists' perspective. On the contrary, PSE could help clarify concerns regarding research to promote alternatives or compromises.
Ultimately, to be effective at PSE for science policy development, institutions performing it must continue to assess and learn new ways to better engage with broader audiences. Guidance from other fields, including public health and business administration, already exists to advise approaches to community engagement for implementing better decisionmaking processes (Kaner et al., 2014; Office of Health Equity (OHE), 2019). Additional PSE methods could be used to increase inclusiveness including focus groups, Delphi groups, town meetings in various cities (especially those further and less accessible to major policy centers), surveys, and webinars with public questions and surveying. These methods also require effective communication and advertising to the public to encourage participation, using traditional and social media and strategies to include under-represented or marginalized communities. New tools and technologies are constantly being developed that can help scientists and scholars engage more effectively and be more inclusive. It is up to institutions to continue to test PSE models to determine the most appropriate methods for their tasks or charges. There are always lessons to be learned from previous efforts and improvements can always be made.
There have also been calls to move PSE upstream, to begin when new research and technologies are being explored instead of waiting for them to be ready to implement or at least substantially developed, allowing the engagement to be more comprehensive (Jones, 2014). Doing this might require a more formal institutionalized system of PSE, as recommended by the NCB and others (Guston, 2014;NCB, 2018).
Beyond PSE evaluation, entities producing public policy recommendations or issuing guidance scientific research should evaluate other earlier processes used to develop reports to make their work more effective. For instance, lessons can be learned from the NASEM, 2017 report concerning the need to avoid ambiguity and offer clear, specific and actionable guidance. As noted in their report, NASEM's recommendations were intentionally vague in defining what HHGE should and should not be done: It is important to note that such concepts as "reasonable alternatives" and "serious disease or condition" embedded in these criteria are necessarily vague. Different societies will interpret these concepts in the context of their diverse historical, cultural, and social characteristics, taking into account input from their publics and their relevant regulatory authorities. Likewise, physicians and patients will interpret them in light of the specifics of individual cases for which germline genome editing may be considered as a possible option. (NASEM, 2017, p. 8, p. 8).
As a result, the Chinese scientist HE Jiankui and his colleagues misinterpreted the recommendations. They believed their experiments to genetically modified human embryos, which resulted in three live births, were justified and consistent with these recommendations (Begley, 2018;Begley and Joseph, 2018;Cohen, 2019). HIV/AIDS is considered a serious disease or condition and the gene HE Jiankui mutated, CCR5, is linked to increased resistance to HIV. The couples targeted by HE Jiankui had HIV + men. While IVF with sperm washing can be used to avoid transmitting the virus in such cases, Chinese law prohibits IVF for individuals with HIV. HE Jiankui offered IVF with sperm washing to couples who would participate in his experiment aimed at making the future children resistant to HIV. HE Jiankui and his colleagues appear to have reasoned that, in the Chinese context, editing embryos to prevent a serious disease was justified since there were no "reasonable alternatives" for these couples. It is unclear whether the NASEM committee members foresaw that their guidelines might be interpreted in this way. Furthermore, after HE Jiankui's experiments were made public, two committee members, Sharon Terry and Luigi Naldini, joined a group of scientists who suggested that the committee's 2017 recommendations were insufficient and endorsed having a moratorium on HHGE (Lander et al., 2019). The NASEM-RS committee was formed to clarify HHGE recommendations and guidelines, suggesting the 2017 recommendations did not provide effective guidance.
These lessons suggest that to best determine the appropriate policies for research in HHGE, more inclusive and comprehensive PSE is urgently needed. While the three reports on HHGE did a good job gauging interest and concerns from vest stakeholders, there is still a need for meaning engagement with the broader public ). This public engagement should also allow for diverse opinions and questions regarding the goal and products of the work being analyzed. Only with thoughtful engagement and a continued willingness to examine and learn from the past are we likely to see a policy developed that is respectful of the publics it is serving and effective at guiding science.

AUTHOR CONTRIBUTIONS
AI and KM developed the concept of the article and both contributed to the developing the draft. AI conducted the literature review and screened publications. SH reviewed selected literature, highlighted relevant passages within reports for consideration in conceptualizing and assessing public and stakeholder engagement, and formatted references. All authors reviewed and edited the final manuscript.