Multi-Criteria Frameworks to Improve Evaluation of the Effectiveness of Environmental Governance in the Amazon

In the Amazon, environmental governance (EG) is an important multi-stakeholder approach to the sustainable management of natural resources. There is by now a large theoretical literature on the many requirements for effective EG. There is also extensive empirical literature on EG in the Amazon. While empirical work in the Amazon has advanced our understanding of EG, it also has important limitations. There remains a need to unpack the many requirements for effective EG to identify violations of governance criteria to explain cases where EG is ineffective. There is also a need for multi-case comparative analyses of EG across diverse parts of the Amazon to identify common explanations for ineffective EG. Therefore, we review multi-criteria frameworks for the evaluation of EG, noting their contributions and differences. We draw on three recent frameworks for evaluating EG to suggest an integrated framework that can be applied to multiple cases for comparative evaluation. We argue that the use of an integrated framework for the identification of violations of specific criteria for effective EG can guide targeted conservation action. We then briefly discuss five study cases in the Amazon to outline how we could apply the integrated framework to identify unmet criteria to account for ineffective EG. We identify commonly unmet criteria for ineffective EG across the five cases. The identification of common criteria violated across cases can provide a basis for broader conservation strategies to improve EG.


INTRODUCTION
In regions like the Amazon, where diverse stakeholders make competing claims on natural resources, and where socio-environmental conflicts are common, governance approaches can be especially valuable. Multi-stakeholder planning processes for environmental governance (EG) seek to arrive at collective decisions about how natural resources will be managed. However, governance approaches involve numerous details concerning who participates, how the processes are organized, and what criteria must be sufficed to satisfy stakeholders and ensure sustainable resource management. Those details have led to variations in how EG is defined, studied, and practiced.
Consequently, EG has a large and diverse theoretical literature with varying terminology and assumptions. Numerous perspectives on EG have emerged, such as community-based natural resource management (e.g., Ostrom et al., 1999;Dietz et al., 2003;Plummer et al., 2013), multilevel EG (e.g., Lemos and Agrawal, 2006;Newig and Fritsch, 2009), and global EG (e.g., Lemos and Agrawal, 2006;Biermann and Pattberg, 2008). While they share commonalities in their commitment to multi-stakeholder processes, such perspectives differ in terms of scale, the particular stakeholders involved, the processes to be pursued, what outcomes are important, and how resources will be managed.
Similarly, EG in the Amazon has been the subject of a diverse body of research. Many studies regard EG in the Amazon as a question of managing protected areas and indigenous territories (e.g., Soares-Filho et al., 2006;Nolte et al., 2013;Schleicher et al., 2017). Other studies focus on state-based regulatory tools, featuring some combination of rules like deforestation limits and enforcement mechanisms like monitoring systems (e.g., Piketty et al., 2015;Arvor et al., 2018). Several studies equate governance with marketbased mechanisms, and therefore examine decisions by private sector actors that reduce environmental impacts, as via adoption of new technologies in management practices (e.g., Tritsch and Arvor, 2016;Thaler, 2017;Arvor et al., 2018). Other research broadens the focus to consider state-society interactions, including bottom-up initiatives where civil society organizations contest state policies (e.g., de Koning, 2014;Damonte, 2018). Such interactions may also be positive, as via co-management arrangements (e.g., Ezzine-de-Blas et al., 2011), including polycentric governance systems in which networks of stakeholders cross national and other jurisdictional boundaries (e.g., Perz et al., 2008).
While the literature on EG in the Amazon involves several significant advances, it also bears some important limitations.
One key issue is that studies of EG in the Amazon tend to pursue analyses based on one or a few indicators, often focusing on specific outcomes while paying little attention to processes, or offering simplistic theoretical frameworks to inform methodological decisions. These shortcomings reflect the difficulty of relating sophisticated theoretical frameworks to empirics for evaluating EG. It is undoubtedly useful to focus on few key indicators, especially a bottom-line outcome like illegal resource use or forest loss, to translate dense theory for purposes of empirical tractability. But it is also clear that EG involves complicated planning processes and diverse stakeholders, which require more inclusive analytical frameworks that recognize the complications involved. A second key issue is that most EG work in the Amazon focuses on specific study sites involving a single case, or treats the entire basin or a specific country's part of the Amazon as a unitary whole. There remains a need for comparative analyses of governance processes among cases to recognize local differences in EG processes. Such analyses can then pursue inclusive comparisons to draw out broader lessons, while also acknowledging casespecific contrasts.

ENVIRONMENTAL GOVERNANCE EVALUATION FRAMEWORKS
A valuable step forward in the literature on EG has been to set forth analytical frameworks for the systematic evaluation of the effectiveness of EG as a process. Such frameworks offer multiple contributions to the literature on EG. One is that they unpack the many theorized requirements for effective EG to permit more inclusive empirical analyses. These frameworks identify multiple criteria that must be met if EG is to be effective. Evaluation frameworks thus honor theoretical accounts that posit the importance of many different aspects of EG processes. Another contribution is that multi-criteria frameworks offer a set of indicators by which EG processes can be evaluated, which affords a basis to identify multiple possible explanations for ineffective EG in terms of criteria not being met. Whereas EG has many elements, and EG processes in developing regions like the Amazon are often not effective, there are likely to be multiple explanations. A third contribution is that evaluation frameworks help address the challenge of relating complicated EG processes to outcomes. Because evaluation frameworks permit the identification of multiple criteria not being met in EG processes, they offer a broader basis for explanations of outcomes. This affords empirical identification of explanations for ineffective EG, rather than assuming them based on observed outcomes alone. A final contribution of evaluation frameworks is that they provide a more or less standardized set of requirements by which EG processes can be evaluated. That permits comparisons among cases, notably in terms of inclusive comparisons to identify common explanations across cases. In sum, multi-criteria evaluation frameworks provide a helpful basis for identifying unmet criteria in EG processes. In cases of ineffective EG, that affords a basis for targeted modifications in EG processes to yield improved outcomes.
Recent work on EG thus includes proposals for frameworks to evaluate the effectiveness of EG. Bennett and Satterfield (2018) draw on the conservation literature to propose a sophisticated evaluation framework. Their proposal is organized around four key objectives: (1) "effectiveness" (the extent to which EG permits sustained ecosystem functioning), (2) "equity" (the degree to which EG processes are inclusive and fair), (3) "responsiveness" (how well EG can be adapted to different contexts and changing circumstances), and (4) "robustness" (the persistence of functioning institutions). Those objectives in turn each have multiple attributes. Waddington et al. (2019) draw on the public service provision literature in developing countries to propose the more elegant "PITA" framework, which stands for participation, inclusion, transparency, and accountability. Those elements are commonly noted in the broader governance literature and are thus standard requisites for governance. Vizeu Pinheiro et al.
(2020) offer a third option by suggesting a framework to evaluate EG on the national level in Latin American countries. Their 11part framework includes many specific economic sectors, but the first four criteria offer a general framework for evaluating EG: (1) "regulation and enforcement, " (2) "civic engagement, " (3) "fundamental environmental and social rights, " and (4) "access to and quality of justice." As in Bennett and Satterfield (2018), the items in Vizeu Pinheiro et al. (2020)'s framework serve as broad topical headings that include multiple criteria to evaluate EG. These frameworks move beyond governance in theory to spell out specific evaluation criteria of governance in practice. These evaluation frameworks also go beyond most of the empirical literature on EG by expanding the range of criteria to guide an analysis of EG effectiveness in practice. Thus, they hold out the prospect of evaluating the effectiveness of EG in terms of multiple criteria across cases, thereby identifying targeted improvements in EG processes to yield better outcomes.
That said, there remain significant challenges to evaluating the effectiveness of EG via the application of these frameworks. These sources come to the question of governance from somewhat different directions. While Bennett and Satterfield (2018)  (2020) include transparency. The latter pays particular attention to the role of the state in regulation, rights, and justice. Both Bennett and Satterfield (2018) and Waddington et al. (2019) highlight issues of access to information, coordination among stakeholders, and capacity for planning and implementation. It becomes evident that each source makes distinct contributions that could be used to construct a more integrated framework for the evaluation of EG. Hence there is more work to be done on frameworks to evaluate the effectiveness of EG; otherwise, key criteria might be missed, and diagnoses of ineffective EG might be inclined to highlight specific perspectives.
We therefore pursued the integration of the evaluation frameworks from Bennett and Satterfield (2018) Table 1. We proceeded by first listing the evaluation criteria in each framework. We then identified similar criteria in two or more of the frameworks and placed those criteria in the same category. We found that the frameworks shared many criteria in common. "Accountability" and "Participation" are similarly covered in all three sources, so we retained those criteria in our integrated framework. At the same time, the more detailed frameworks had more criteria. In some cases, we combined criteria in a framework where they seemed conceptually similar. For example, we combined "Anticipatory, " "Adaptive" and "Innovative" from Bennett and Satterfield into one criterion. This exercise produced 11 distinct criteria for evaluation of EG. In Table 1 at the left, we present the criteria from Bennett and Satterfield (2018) in blue, Waddington et al. (2019) in green, and Vizeu Pinheiro et al. (2020) in orange. At the right, we relate the evaluation criteria in our sources to the criteria we identified for our integrated framework. Table 2 provides basic definitions of the criteria in our integrated framework. While the meanings of the concept behind each criterion are not exactly the same as those in our sources in every case, we sought to ensure that the meanings differed among the criteria in our resulting framework. We regard the criteria in Table 2 as a provisional framework for the evaluation of EG. One advantage of an integrated framework is that it draws on recent source frameworks that make complementary contributions and uses a single list that combines those contributions. Another advantage of an integrated framework is that it permits the application of a common list of criteria to multiple cases for comparative analysis.

COMPARATIVE EVALUATION OF THE EFFECTIVENESS OF ENVIRONMENTAL GOVERNANCE IN THE AMAZON
The Amazon is a very large region that spans nine countries and thus encompasses an enormous diversity of local and national circumstances. Extant empirical work on EG in the Amazon is extensive, but rarely comparative in a systematic fashion. The diversity of local conditions, and the common observation of difficulties in realizing effective EG, call for comparative work to identify patterns in the explanations for ineffective EG. If it is obvious enough that there are many explanations for ineffective EG, it is somewhat less clear how such explanations relate to specific criteria for effective EG. Reframing explanations for ineffective EG in terms of unmet criteria for effective EG in a shared evaluation framework can provide a basis for comparative analysis. Inclusive comparisons would permit the identification of criteria more commonly violated across cases, whether local sites or countries. In turn, identifying criteria more often unmet could serve to prioritize strategies to meet those criteria and thereby more broadly advance effective EG.
Certain prominent sites within the Amazon with which we are familiar offer useful illustrations of ineffective EG. In Bolivia, the Isiboro Sécure Indigenous Territory and National Park (TIPNIS) is a complex space combining protected areas and indigenous lands. The TIPNIS is now a site of contention over EG as the Bolivian state under President Evo Morales sought to build a road to facilitate coca production and various forms of resource extraction. The Bolivian state has actively undermined effective EG there, notably by fracturing indigenous groups and organizations who had built a tradition of EG in previous decades. The upper Madera watershed at the frontier between Bolivia and Brazil faces major challenges to EG, in ongoing disputes over the planning and construction of hydroelectric dams. High-level negotiations between the two national governments proceeded with limited inclusion of local peoples, resulting in mobilization for protest and independent data collection on dam impacts. State inattention to local protests and data showing significant impacts of dams on fish populations and local livelihoods make evident that EG is ineffective in the bi-national watershed. Elsewhere in the Brazilian Amazon, in southern Amazonas-northern Rondonia, several highways  into the lowlands to claim land and extract natural resources, which threatens protected areas and indigenous lands. This has led to conflicts over resources, including a rise in violence, as the state has given indications that it supports extractive interests in the name of national development. The incursions of extractive interests, the emergence of conflicts and the lack of state support for local peoples all serve to undermine EG. Finally, in Madre de Dios, Peru, paving of the Inter-Oceanic Highway coincided with the world financial crisis and the boom in commodities, which stimulated a gold rush in rivers along the road. Gold mining in Madre de Dios is mostly informal and often illegal, highlighting problems of EG, as mining causes deforestation and washes mercury into aquatic ecosystems. Stakeholders seeking improved EG have faced challenges of coordination to pursue sustainable strategies for resource management. While these cases are diverse in terms of national governments, topical foci, and other details, they offer the opportunity for inclusive comparisons to identify common explanations for ineffective EG. One issue concerns the problem of state authoritarianism, which appears in multiple cases. In Bolivia, the state imposed its will indirectly by engaging in corruption and spreading misinformation to undermine trust among local indigenous organizations in the TIPNIS area; in Brazil, the Bolsonaro administration was openly hostile to environmental enforcement and recognizing indigenous lands, whether in southern Amazonas-northern Rondonia or elsewhere. Despite the contrasts in the substance of state authoritarianism in these two cases, when interpreted in light of evaluation frameworks, we nonetheless find some commonalities in the Table 2 EG criteria violated, including the lack of transparency, accountability, coordination among stakeholders, inclusion of marginal groups, and public participation. The use of the integrated evaluation framework in Table 2 for comparative analysis thus permits unpacking of explanations for ineffective EG in terms of specific criteria and affords identification of common criteria violated.
Another issue that arises in multiple cases concerns the advancement of infrastructure projects without adequate planning for effective EG. In the upper Madeira case, licensing of hydroelectric dams involves inadequate or purely procedural environmental impact assessments (EIAs), and inattention to contributions to planning from independent scientific and local knowledge. In Madre de Dios, paving of the Inter-Oceanic Highway proceeded without a complete EIA and with limited regional capacity to enforce environmental regulations, which permitted the rapid expansion of informal and illegal gold mining. The same is true of the planning for the BR-319 in southern Amazonas and northern Rondonia. In terms of evaluation criteria in Table 2, in both cases, there was a lack of access to information among stakeholders, inclusion during the planning process, coordination between states and other stakeholders, and recognition of rights of stakeholders affected by illegal activities.
The third commonality across multiple cases concerns the negligent treatment of indigenous groups. In Colombia, indigenous groups are well-organized to govern their territories, and indigenous territorial autonomy is recognized in the constitution; but in practice, extractive interests and sometimes even the state perpetrate threats and violence against indigenous peoples. In the TIPNIS, the indigenist government under the Morales administration had a highland indigenous base; but that base exhibited a tradition of resource use that does not recognize the value of standing forests, so supporters sought access to land to clear forest and unsustainably exploit natural resources in the Amazon. In Brazil under Bolsonaro, the state has exhibited outright hostility to demarcating indigenous territories, while promoting new infrastructure projects subverting indigenous people's autonomy. In each of these cases, in terms of the Table 2 criteria, there is a lack of coordination between the state and indigenous stakeholders, recognition of indigenous territorial rights, inclusion of indigenous stakeholders in planning, and indigenous participation in planning.

DISCUSSION
Our goal was to present an integrated framework for the evaluation of the effectiveness of EG. The framework builds on and leverages the contributions of previous frameworks that draw on established EG literature. However, the foregoing analysis with the integrated framework is suggestive rather than definitive. Space limitations do not permit a full analysis of explanations for ineffective EG across cases in terms of the evaluation criteria in Table 2. Our point for present purposes is that an integrated framework for evaluation of EG permits a comparative analysis across cases to identify unmet criteria. According to our analysis, the most commonly unmet criteria are lack of transparency and access to information, accountability, coordination among stakeholders, inclusion of vulnerable and indigenous groups, and public participation. The fact that explanations for ineffective EG can be unpacked in terms of violations of multiple criteria for effective EG indicates that the evaluation of EG cannot be reductive. By unpacking explanations for ineffective EG, it becomes easier to see that multiple requisites are lacking, which in turn permits the identification of targeted strategies to address those shortcomings. While it is daunting to seek to address broad explanations such as "state authoritarianism" or "lack of adequate planning" or "negligent treatment of indigenous peoples, " it is arguably more tractable to address specific evaluation criteria like the lack of inclusion or effective participation or coordination among stakeholders in a planning process.
We also argue that the examination of multiple cases of EG provides the basis for comparative analysis to identify similarities in explanations. Via inclusive comparisons of the details from multiple cases, we can learn broader lessons about the effectiveness of EG. We have pointed out some similarities in explanations for ineffective EG, and then unpacked those similarities in terms of shared violations of criteria for effective EG. The identification of violations of common criteria for EG can serve to prioritize specific conservation strategies for broader impact. Conversely, a comparative analysis could focus on differences among cases, including instances where some criterion for EG was sufficed, to see if a strategy for fulfilling that criterion in another case might be viable. Whether by the pursuit of inclusive comparisons or identification of contrasts among cases, comparative analysis can lead to the delineation of criteria for EG that can guide conservation strategy.
We conclude by returning to the point that our integrated framework is provisional. More can be done to refine frameworks for the evaluation of EG and add specific criteria to particular contexts. We note that the frameworks on which we draw are relevant to but not developed specifically for the Amazon. This implies that Amazonian scholars and practitioners can usefully engage the work on evaluation frameworks for effectiveness of EG. We suggest our integrated framework as a useful point of departure for further adaptation to the context of the Amazon.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

AUTHOR CONTRIBUTIONS
SP drafted the manuscript based on a collaborative manuscript by all of the co-authors. PM-G, MS, and AG edited the manuscript. SP, PM-G, and MS contributed to the background and theoretical discussion of the evaluation frameworks presented. MA, AB, SF, AG, and MR provided data and narratives on the case studies noted and provided comments on the draft manuscript. All authors have read the full manuscript.

FUNDING
This study has received funding from the Tropical Conservation and Development Program (TCD) at UF. Portions of the research for the upper Madera, southern Amazonas-northern Rondonia, and Colombia cases were supported by the Gordon and Betty Moore Foundation (Grant #7715).