Black Boxes and the Role of Modeling in Environmental Policy Making

Modeling is essential for modern science, and science-based policies are directly affected by the reliability of model outputs. Artificial intelligence has improved the accuracy and capability of model simulations, but often at the expense of a rational understanding of the systems involved. The lack of transparency in black box models, artificial intelligence based ones among them, can potentially affect the trust in science driven policy making. Here, we suggest that a broader discussion is needed to address the implications of black box approaches on the reliability of scientific advice used for policy making. We argue that participatory methods can bridge the gap between increasingly complex scientific methods and the people affected by their interpretations


INTRODUCTION
Computer algorithms and models are critical in modern science. They have been embraced by academic circles for decades, permeating influence all the way from policy-making to people's everyday life. Today, any person can check the weather forecast in the local news or on a cell phone, without noticing the colossal cyber infrastructure behind the information provided. From classical physics to complex models and algorithms run on supercomputers, the issue of understanding and trusting them is irrelevant for someone who simply wants to choose which clothes to wear tomorrow. People are happy to believe that the forecast is based on the best available knowledge. In this case, science has become part of people's daily life and is well accepted, despite the end user's superficial understanding of the methods and tools involved.
In many other cases, the increasing complexity of scientific methods and models tends to disconnect the general public from science, as the underlying logic and theories become more difficult to understand even for other scientists. When it comes to artificial intelligence, for example, some researchers argue that the entire field is becoming too complex for anyone to understand (Voosen, 2017). In fields such as machine learning, even top scientists sometimes struggle to make sense of the mechanisms driving their own results.
When the internal logic of a model is hidden, it is often referred to as a black box model. The term "black box" traditionally comes from empirical models that simply relate inputs and outputs, without knowledge or assumptions about the internal functions or processes. Similar black box concepts can be used to assist modular approaches, in which the inner workings of smaller components are temporarily ignored, so interactions in the higher structure can be better understood. Nonetheless, today even process based models become similar to black box ones because the complexity of the processes and interactions included makes them too difficult to understand, especially by policy makers and stakeholders with no special training. Indeed, as scientific understanding of the system deepens, as new methods emerge, and computational power increases, our access to the internal logic of models tends to become more obscure. This does not make such models less accurate or useful in comparison to more transparent causal or process-based models. However, the question is how can we maintain trust in and usability of such black box models in environmental policy-making?

MODEL COMPLEXITY AND POLICY-MAKING
Socio-ecological systems are inherently-and increasingly-complex, involving interactions of numerous factors, stakeholder groups and sectors at multiple temporal and spatial scales (Voinov et al., 2016). Given such complexity, observations alone are incapable of informing the management of environmental problems or the use of natural resources (Schmolke et al., 2010). In addition to observations, an understanding of the system's logic is needed. This leads us to an intrinsic paradox in the use of models to support environmental policymaking. By definition, a model is a simplified representation of the key processes and factors that explain the behavior of a system in a certain context within certain boundaries. Replacing a system with a model and then analyzing it, provides justified predictions for the future and allows the exploration of alternative policies and management scenarios (Schmolke et al., 2010). However, with the advent of modern artificial intelligence tools that enable, for example, increasingly efficient machine learning and integrated modeling studies, the scientific approaches are now heading to the opposite direction with increasing model complexity.
Already three decades ago, scientists warned that models providing results that are too abstract and distant from the realities of political challenges are almost certain to fail as tools for policy-making (Kraemer and King, 1986). We argue that this may be also the case when the processes driving modeling become inaccessible for non-scientists, e.g., due to high complexity. A well-known example can be seen in the field of climate change mitigation, where scientists have not yet succeeded in convincing all policy makers and the public in either the problem or the results produced by climate models. It is notorious that the lobbying by climate change deniers can influence funding for climate research, as well as affect the implementation of policies (Stern et al., 2016). Climate change is, of course, a complex example, where multiple conflicting interests are in place, and science itself is by far not the only player. However, when it is difficult for the general public to understand the theory behind climate models it certainly weakens the scientific arguments underlying the policy advice (Stern et al., 2016;Visschers, 2018).
Achieving public confidence and trust is thus a major challenge for models designed to support environmental policy. Problem framing defines the questions a model can answer, and leads to identifying relevant variables and their interactions. Hence, the conceptual phase of model design is highly dependent on the participation of experts and stakeholders altogether. In such a context, the less transparent are the models, the bigger is the challenge for integrating stakeholders into the policy making process. Current protocols, as the ones set by the European Commission guidelines for better regulation, warns that successful modeling requires communicating to decision makers how a model works and the strengths and limitations of a chosen modeling approach. However, official guidelines do not yet foresee how fields such as artificial intelligence will affect public confidence in scientific advice.
The challenges imposed by the lack of transparency and understandability of emerging technologies should not, however, create a barrier to the advancement of science. Instead of closing themselves in their black boxes, researchers should invest in alternative strategies to keep policy makers and stakeholders involved, in order to maintain their trust independently of their access to the models, by explaining why the models behave as they do. Studies have shown that stakeholders involved in modeling do not necessarily need to understand the scientific method as such, if their role is, for example, to provide practical information for the modeler (Ritzema et al., 2010;Mäntyniemi et al., 2013;Voinov et al., 2016). To maintain stakeholders' trust in a model, the results of which will affect their lives, modelers need to clarify the assumptions and inference logic behind the models and justify their choices .

STRATEGIC ROLE OF COMMUNICATION
Using scientific advice for policy making involves transmitting knowledge from researchers to non-scientists and/or nonspecialists and back. If this link is poorly maintained, the level of complexity in the scientific methods becomes irrelevant, as even the simplest approaches can be criticized and discarded. Transmitting the right message is therefore important for extending the reach of scientific advice, given that numerous barriers are in place for effectively communicating scientific results to policy makers (Cartwright et al., 2016). For example, communicating the uncertainties intrinsic to a model is essential (Stern et al., 2016;Visschers, 2018). As demonstrated in fisheries science, misunderstanding and misinterpretation of model uncertainties can reduce confidence in the policy advice and lead to decision paralysis in situations when stakeholders cannot or do not want to take appropriate actions (Cartwright et al.,

ACTIONABLE RECOMMENDATIONS
As the increasing complexity of models starts to influence policy making, it is important for scientists to create new approaches to communicate their underlying assumptions, reasoning, data and methods to stakeholders. Here, we define a stakeholder as a person, group, or organization involved in or affected by a course of action (in this case, actions guided by models). These may also include scientists from different disciplines, as complex socio-environmental problems are by nature transdisciplinary and require input from various disciplines to provide relevant scientific advice. When models serve decision making that directly affects people's lives, as in the case of many natural resource management actions, it is important to make the stakeholders part of the modeling process.
For instance, some of the tools also categorized under artificial intelligence, such as graphical Bayesian Networks (Korb and Nicholson, 2010), can help users to better perceive and understand complex and uncertain systems. Visualization of the framing and causal structures of inference combined with the possibility to interactively adjust scenarios, update the level of knowledge and observe the system's reactions in terms of dynamically updating visual probability distributions, improves the understanding about the conditional aspects and impact mechanisms of potential management interventions under consideration (Carriger et al., 2018;Kaikkonen et al., 2021). Bayesian methods can be designed to use the input from stakeholders as an intrinsic part of the model, in which case local knowledge can be used to provide priors for the model, leading to more realistic outputs (i.e., posteriors). In general, approaches that allow end-users to manipulate model parameters can be a powerful learning tool, as even if the internal logic of the model is hidden, users can still understand how changes in the system affect the final results.
Participatory modeling (PM) is an interactive learning process that employs the knowledge and/or values of stakeholders to create modeling and communication tools . The level of engagement of stakeholders in PM can vary. The most intense participation occurs when the people affected by the results are engaged in all stages of a modeling process, from problem identification and framing, model designing, parameter selection, data collection, parameter estimation, model validation, and communication of the results, all the way to using the model for policy advice (Voinov et al., 2016). Most often, however, stakeholders only participate in some of the modeling stages. When it comes to models that from the stakeholder perspective are seen as "black boxes", a full participation of stakeholders, as described above, may not be feasible as the model neither includes clear model structures nor understandable parameters. Still, stakeholder involvement in defining appropriate questions, key indicators and/or management objectives and their prioritization is not only possible (McCabe and Halog, 2018;Laurila-Pant et al., 2019), but highly desirable for keeping the stakeholders informed and involved and for improving the quality of the policy making process. Stakeholder engagement is also essential to explain how uncertainties are treated in modeling.
Participatory approaches that bring together complex models and stakeholders have been implemented in a large variety of applications dealing with environmental or natural resource policy. For example, during the next years, European fisheries are transitioning to new rules aiming at reducing discards and mandatorily bringing all catches to land. These new rules will significantly affect fishery activities in Europe. As it is impossible to completely avoid unwanted catches, developing models to identify the most suitable fishing grounds to avoid them will be critical for maintaining the competitiveness as well as acceptability of the European fishing industry. To improve the performance of such models while at the same time allowing an easier assimilation and stronger legitimacy of the tools at local levels, researchers have created a framework for combining stakeholders' knowledge with a Bayesian model of fishery discards (Maeda et al., 2017). This framework allows fishermen to input their prior knowledge of the fishing grounds into the model (i.e., to indicate in which areas they think by-catches are high), to then combine it with observed data and provide posterior probabilities of fishery discards in the form of maps. In this case, although the core algorithms are mathematically demanding and remain a "black box" from the perspective of the fishermen, the credibility of the overall framework is strengthened as the end users are involved in the modeling process from the beginning and see that their knowledge is used and preferences are taken into account, before receiving the final product.
PM has also been applied for supporting policies related to oil spill accidents, water resources management, and agricultural practices, among many others (Vayssières et al., 2011;Lehikoinen et al., 2013;Basco-Carrera et al., 2017;Hedelin et al., 2017). A common ground among these applications is the capability for integrating local knowledge and views, leading to a stronger transfer of values and perspectives among modelers and stakeholders. Such benefits have increased trust in scientific information during the process of environmental decisionmaking (Hedelin et al., 2017;Ulibarri, 2018). Given the requirement to gain stakeholders' and policy makers' trust in science for policy, the importance of PM may be growing in environmental policy making as models are even more often seen as black boxes.
Despite the benefits of PM, engaging stakeholders in scientific projects is not easy (Lavery, 2018). Among the challenges is the poor understanding of the outcomes that participatory approaches can produce, or the lack of a clear definition of who should be involved as a stakeholder (Lavery, 2018). This, as well as lack of evidence in the needs for stakeholder engagement, can significantly affect resource allocation in projects involving PM implying that budgets for stakeholder participation remain too small. We believe that addressing these challenges will become increasingly important, as the complexity of scientific methods together with the need for participatory approaches continues to grow. Thus, the next challenge for the scientific community is to find an appropriate balance between the prediction capacity and understandability of the models used.
The popularity of PM studies has grown remarkably in recent years (Voinov et al., 2016) and proper communication practices become a key component in science (Gluckman, 2014). There are, therefore, good opportunities for reconciling advanced technologies and people benefiting from them. While scientific advice may be, in the future, increasingly based on complex models, the environmental policies benefiting from these tools should head into the direction of more transparent and collaborative processes. It may be difficult to communicate the computational details of a black box model, but it is still necessary to communicate the logic behind the results, as well as the policy process that uses the models (Röckmann et al., 2012;Glynn et al., 2018;Jordan et al., 2018). To keep the participatory processes up with the advancements of the fast-developing computation capacity and modeling technology, substantial effort should be spent on developing tools and skills to support stakeholder participation and engagement.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.