Multi-Criteria Frameworks to Improve Evaluation of the Effectiveness of Environmental Governance in the Amazon

Introduction

In regions like the Amazon, where diverse stakeholders make competing claims on natural resources, and where socio-environmental conflicts are common, governance approaches can be especially valuable. Multi-stakeholder planning processes for environmental governance (EG) seek to arrive at collective decisions about how natural resources will be managed. However, governance approaches involve numerous details concerning who participates, how the processes are organized, and what criteria must be sufficed to satisfy stakeholders and ensure sustainable resource management. Those details have led to variations in how EG is defined, studied, and practiced.

Consequently, EG has a large and diverse theoretical literature with varying terminology and assumptions. Numerous perspectives on EG have emerged, such as community-based natural resource management (e.g., Ostrom et al., 1999; Dietz et al., 2003; Plummer et al., 2013), multilevel EG (e.g., Lemos and Agrawal, 2006; Newig and Fritsch, 2009), and global EG (e.g., Lemos and Agrawal, 2006; Biermann and Pattberg, 2008). While they share commonalities in their commitment to multi-stakeholder processes, such perspectives differ in terms of scale, the particular stakeholders involved, the processes to be pursued, what outcomes are important, and how resources will be managed.

Similarly, EG in the Amazon has been the subject of a diverse body of research. Many studies regard EG in the Amazon as a question of managing protected areas and indigenous territories (e.g., Soares-Filho et al., 2006; Nolte et al., 2013; Schleicher et al., 2017). Other studies focus on state-based regulatory tools, featuring some combination of rules like deforestation limits and enforcement mechanisms like monitoring systems (e.g., Piketty et al., 2015; Arvor et al., 2018). Several studies equate governance with market-based mechanisms, and therefore examine decisions by private sector actors that reduce environmental impacts, as via adoption of new technologies in management practices (e.g., Tritsch and Arvor, 2016; Thaler, 2017; Arvor et al., 2018). Other research broadens the focus to consider state-society interactions, including bottom-up initiatives where civil society organizations contest state policies (e.g., de Koning, 2014; Damonte, 2018). Such interactions may also be positive, as via co-management arrangements (e.g., Ezzine-de-Blas et al., 2011), including polycentric governance systems in which networks of stakeholders cross national and other jurisdictional boundaries (e.g., Perz et al., 2008).

While the literature on EG in the Amazon involves several significant advances, it also bears some important limitations. One key issue is that studies of EG in the Amazon tend to pursue analyses based on one or a few indicators, often focusing on specific outcomes while paying little attention to processes, or offering simplistic theoretical frameworks to inform methodological decisions. These shortcomings reflect the difficulty of relating sophisticated theoretical frameworks to empirics for evaluating EG. It is undoubtedly useful to focus on few key indicators, especially a bottom-line outcome like illegal resource use or forest loss, to translate dense theory for purposes of empirical tractability. But it is also clear that EG involves complicated planning processes and diverse stakeholders, which require more inclusive analytical frameworks that recognize the complications involved. A second key issue is that most EG work in the Amazon focuses on specific study sites involving a single case, or treats the entire basin or a specific country’s part of the Amazon as a unitary whole. There remains a need for comparative analyses of governance processes among cases to recognize local differences in EG processes. Such analyses can then pursue inclusive comparisons to draw out broader lessons, while also acknowledging case-specific contrasts.

Environmental Governance Evaluation Frameworks

A valuable step forward in the literature on EG has been to set forth analytical frameworks for the systematic evaluation of the effectiveness of EG as a process. Such frameworks offer multiple contributions to the literature on EG. One is that they unpack the many theorized requirements for effective EG to permit more inclusive empirical analyses. These frameworks identify multiple criteria that must be met if EG is to be effective. Evaluation frameworks thus honor theoretical accounts that posit the importance of many different aspects of EG processes. Another contribution is that multi-criteria frameworks offer a set of indicators by which EG processes can be evaluated, which affords a basis to identify multiple possible explanations for ineffective EG in terms of criteria not being met. Whereas EG has many elements, and EG processes in developing regions like the Amazon are often not effective, there are likely to be multiple explanations. A third contribution is that evaluation frameworks help address the challenge of relating complicated EG processes to outcomes. Because evaluation frameworks permit the identification of multiple criteria not being met in EG processes, they offer a broader basis for explanations of outcomes. This affords empirical identification of explanations for ineffective EG, rather than assuming them based on observed outcomes alone. A final contribution of evaluation frameworks is that they provide a more or less standardized set of requirements by which EG processes can be evaluated. That permits comparisons among cases, notably in terms of inclusive comparisons to identify common explanations across cases. In sum, multi-criteria evaluation frameworks provide a helpful basis for identifying unmet criteria in EG processes. In cases of ineffective EG, that affords a basis for targeted modifications in EG processes to yield improved outcomes.

Recent work on EG thus includes proposals for frameworks to evaluate the effectiveness of EG. Bennett and Satterfield (2018) draw on the conservation literature to propose a sophisticated evaluation framework. Their proposal is organized around four key objectives: (1) “effectiveness” (the extent to which EG permits sustained ecosystem functioning), (2) “equity” (the degree to which EG processes are inclusive and fair), (3) “responsiveness” (how well EG can be adapted to different contexts and changing circumstances), and (4) “robustness” (the persistence of functioning institutions). Those objectives in turn each have multiple attributes. Waddington et al. (2019) draw on the public service provision literature in developing countries to propose the more elegant “PITA” framework, which stands for participation, inclusion, transparency, and accountability. Those elements are commonly noted in the broader governance literature and are thus standard requisites for governance. Vizeu Pinheiro et al. (2020) offer a third option by suggesting a framework to evaluate EG on the national level in Latin American countries. Their 11-part framework includes many specific economic sectors, but the first four criteria offer a general framework for evaluating EG: (1) “regulation and enforcement,” (2) “civic engagement,” (3) “fundamental environmental and social rights,” and (4) “access to and quality of justice.” As in Bennett and Satterfield (2018), the items in Vizeu Pinheiro et al. (2020)’s framework serve as broad topical headings that include multiple criteria to evaluate EG. These frameworks move beyond governance in theory to spell out specific evaluation criteria of governance in practice. These evaluation frameworks also go beyond most of the empirical literature on EG by expanding the range of criteria to guide an analysis of EG effectiveness in practice. Thus, they hold out the prospect of evaluating the effectiveness of EG in terms of multiple criteria across cases, thereby identifying targeted improvements in EG processes to yield better outcomes.

That said, there remain significant challenges to evaluating the effectiveness of EG via the application of these frameworks. These sources come to the question of governance from somewhat different directions. While Bennett and Satterfield (2018) draw on the conservation literature, Waddington et al. (2019) work from the public service provision literature, and Vizeu Pinheiro et al. (2020) write on behalf of the Inter-American Development Bank. Consequently, the three frameworks differ substantially. Whereas Bennett and Satterfield (2018) list nearly 20 evaluation criteria, the PITA framework in Waddington et al. (2019) has only four, and Vizeu Pinheiro et al. (2020) include 10 criteria. Further, the three frameworks differ in their substance. While all three sources include participation, inclusion, and accountability, Bennett and Satterfield (2018) add criteria for adaptation and flexibility; Waddington et al. (2019) and Vizeu Pinheiro et al. (2020) include transparency. The latter pays particular attention to the role of the state in regulation, rights, and justice. Both Bennett and Satterfield (2018) and Waddington et al. (2019) highlight issues of access to information, coordination among stakeholders, and capacity for planning and implementation. It becomes evident that each source makes distinct contributions that could be used to construct a more integrated framework for the evaluation of EG. Hence there is more work to be done on frameworks to evaluate the effectiveness of EG; otherwise, key criteria might be missed, and diagnoses of ineffective EG might be inclined to highlight specific perspectives.

We therefore pursued the integration of the evaluation frameworks from Bennett and Satterfield (2018); Waddington et al. (2019), and Vizeu Pinheiro et al. (2020), shown in Table 1. We proceeded by first listing the evaluation criteria in each framework. We then identified similar criteria in two or more of the frameworks and placed those criteria in the same category. We found that the frameworks shared many criteria in common. “Accountability” and “Participation” are similarly covered in all three sources, so we retained those criteria in our integrated framework. At the same time, the more detailed frameworks had more criteria. In some cases, we combined criteria in a framework where they seemed conceptually similar. For example, we combined “Anticipatory,” “Adaptive” and “Innovative” from Bennett and Satterfield into one criterion. This exercise produced 11 distinct criteria for evaluation of EG. In Table 1 at the left, we present the criteria from Bennett and Satterfield (2018) in blue, Waddington et al. (2019) in green, and Vizeu Pinheiro et al. (2020) in orange. At the right, we relate the evaluation criteria in our sources to the criteria we identified for our integrated framework.

TABLE 1

Table 1. Integration of governance evaluation criteria.

Table 2 provides basic definitions of the criteria in our integrated framework. While the meanings of the concept behind each criterion are not exactly the same as those in our sources in every case, we sought to ensure that the meanings differed among the criteria in our resulting framework. We regard the criteria in Table 2 as a provisional framework for the evaluation of EG. One advantage of an integrated framework is that it draws on recent source frameworks that make complementary contributions and uses a single list that combines those contributions. Another advantage of an integrated framework is that it permits the application of a common list of criteria to multiple cases for comparative analysis.

TABLE 2

Table 2. Integrated environmental governance framework.

Comparative Evaluation of the Effectiveness of Environmental Governance in the Amazon

The Amazon is a very large region that spans nine countries and thus encompasses an enormous diversity of local and national circumstances. Extant empirical work on EG in the Amazon is extensive, but rarely comparative in a systematic fashion. The diversity of local conditions, and the common observation of difficulties in realizing effective EG, call for comparative work to identify patterns in the explanations for ineffective EG. If it is obvious enough that there are many explanations for ineffective EG, it is somewhat less clear how such explanations relate to specific criteria for effective EG. Reframing explanations for ineffective EG in terms of unmet criteria for effective EG in a shared evaluation framework can provide a basis for comparative analysis. Inclusive comparisons would permit the identification of criteria more commonly violated across cases, whether local sites or countries. In turn, identifying criteria more often unmet could serve to prioritize strategies to meet those criteria and thereby more broadly advance effective EG.

Certain prominent sites within the Amazon with which we are familiar offer useful illustrations of ineffective EG. In Bolivia, the Isiboro Sécure Indigenous Territory and National Park (TIPNIS) is a complex space combining protected areas and indigenous lands. The TIPNIS is now a site of contention over EG as the Bolivian state under President Evo Morales sought to build a road to facilitate coca production and various forms of resource extraction. The Bolivian state has actively undermined effective EG there, notably by fracturing indigenous groups and organizations who had built a tradition of EG in previous decades. The upper Madera watershed at the frontier between Bolivia and Brazil faces major challenges to EG, in ongoing disputes over the planning and construction of hydroelectric dams. High-level negotiations between the two national governments proceeded with limited inclusion of local peoples, resulting in mobilization for protest and independent data collection on dam impacts. State inattention to local protests and data showing significant impacts of dams on fish populations and local livelihoods make evident that EG is ineffective in the bi-national watershed. Elsewhere in the Brazilian Amazon, in southern Amazonas-northern Rondonia, several highways cross a complex mosaic of protected areas and indigenous lands. The question of paving the BR-319 highway has become a particular point of contention, as the Brazilian state under President Jair Bolsonaro has promulgated a set of policies hostile to environmental regulations and indigenous rights. In particular, the Bolsonaro administration has populated agencies with military personnel and sought to impose policies on the ground, reflecting its authoritarian tendencies, which are anathema to multi-stakeholder EG. In Colombia, years of armed insurgency against the state meant limited extractive activity in the Amazon, but the cessation of insurgency has changed the situation. Extractive interests have followed official and unofficial roads into the lowlands to claim land and extract natural resources, which threatens protected areas and indigenous lands. This has led to conflicts over resources, including a rise in violence, as the state has given indications that it supports extractive interests in the name of national development. The incursions of extractive interests, the emergence of conflicts and the lack of state support for local peoples all serve to undermine EG. Finally, in Madre de Dios, Peru, paving of the Inter-Oceanic Highway coincided with the world financial crisis and the boom in commodities, which stimulated a gold rush in rivers along the road. Gold mining in Madre de Dios is mostly informal and often illegal, highlighting problems of EG, as mining causes deforestation and washes mercury into aquatic ecosystems. Stakeholders seeking improved EG have faced challenges of coordination to pursue sustainable strategies for resource management.

While these cases are diverse in terms of national governments, topical foci, and other details, they offer the opportunity for inclusive comparisons to identify common explanations for ineffective EG. One issue concerns the problem of state authoritarianism, which appears in multiple cases. In Bolivia, the state imposed its will indirectly by engaging in corruption and spreading misinformation to undermine trust among local indigenous organizations in the TIPNIS area; in Brazil, the Bolsonaro administration was openly hostile to environmental enforcement and recognizing indigenous lands, whether in southern Amazonas-northern Rondonia or elsewhere. Despite the contrasts in the substance of state authoritarianism in these two cases, when interpreted in light of evaluation frameworks, we nonetheless find some commonalities in the Table 2 EG criteria violated, including the lack of transparency, accountability, coordination among stakeholders, inclusion of marginal groups, and public participation. The use of the integrated evaluation framework in Table 2 for comparative analysis thus permits unpacking of explanations for ineffective EG in terms of specific criteria and affords identification of common criteria violated.

Another issue that arises in multiple cases concerns the advancement of infrastructure projects without adequate planning for effective EG. In the upper Madeira case, licensing of hydroelectric dams involves inadequate or purely procedural environmental impact assessments (EIAs), and inattention to contributions to planning from independent scientific and local knowledge. In Madre de Dios, paving of the Inter-Oceanic Highway proceeded without a complete EIA and with limited regional capacity to enforce environmental regulations, which permitted the rapid expansion of informal and illegal gold mining. The same is true of the planning for the BR-319 in southern Amazonas and northern Rondonia. In terms of evaluation criteria in Table 2, in both cases, there was a lack of access to information among stakeholders, inclusion during the planning process, coordination between states and other stakeholders, and recognition of rights of stakeholders affected by illegal activities.

The third commonality across multiple cases concerns the negligent treatment of indigenous groups. In Colombia, indigenous groups are well-organized to govern their territories, and indigenous territorial autonomy is recognized in the constitution; but in practice, extractive interests and sometimes even the state perpetrate threats and violence against indigenous peoples. In the TIPNIS, the indigenist government under the Morales administration had a highland indigenous base; but that base exhibited a tradition of resource use that does not recognize the value of standing forests, so supporters sought access to land to clear forest and unsustainably exploit natural resources in the Amazon. In Brazil under Bolsonaro, the state has exhibited outright hostility to demarcating indigenous territories, while promoting new infrastructure projects subverting indigenous people’s autonomy. In each of these cases, in terms of the Table 2 criteria, there is a lack of coordination between the state and indigenous stakeholders, recognition of indigenous territorial rights, inclusion of indigenous stakeholders in planning, and indigenous participation in planning.

Discussion

Our goal was to present an integrated framework for the evaluation of the effectiveness of EG. The framework builds on and leverages the contributions of previous frameworks that draw on established EG literature. However, the foregoing analysis with the integrated framework is suggestive rather than definitive. Space limitations do not permit a full analysis of explanations for ineffective EG across cases in terms of the evaluation criteria in Table 2. Our point for present purposes is that an integrated framework for evaluation of EG permits a comparative analysis across cases to identify unmet criteria. According to our analysis, the most commonly unmet criteria are lack of transparency and access to information, accountability, coordination among stakeholders, inclusion of vulnerable and indigenous groups, and public participation. The fact that explanations for ineffective EG can be unpacked in terms of violations of multiple criteria for effective EG indicates that the evaluation of EG cannot be reductive. By unpacking explanations for ineffective EG, it becomes easier to see that multiple requisites are lacking, which in turn permits the identification of targeted strategies to address those shortcomings. While it is daunting to seek to address broad explanations such as “state authoritarianism” or “lack of adequate planning” or “negligent treatment of indigenous peoples,” it is arguably more tractable to address specific evaluation criteria like the lack of inclusion or effective participation or coordination among stakeholders in a planning process.

We also argue that the examination of multiple cases of EG provides the basis for comparative analysis to identify similarities in explanations. Via inclusive comparisons of the details from multiple cases, we can learn broader lessons about the effectiveness of EG. We have pointed out some similarities in explanations for ineffective EG, and then unpacked those similarities in terms of shared violations of criteria for effective EG. The identification of violations of common criteria for EG can serve to prioritize specific conservation strategies for broader impact. Conversely, a comparative analysis could focus on differences among cases, including instances where some criterion for EG was sufficed, to see if a strategy for fulfilling that criterion in another case might be viable. Whether by the pursuit of inclusive comparisons or identification of contrasts among cases, comparative analysis can lead to the delineation of criteria for EG that can guide conservation strategy.

We conclude by returning to the point that our integrated framework is provisional. More can be done to refine frameworks for the evaluation of EG and add specific criteria to particular contexts. We note that the frameworks on which we draw are relevant to but not developed specifically for the Amazon. This implies that Amazonian scholars and practitioners can usefully engage the work on evaluation frameworks for effectiveness of EG. We suggest our integrated framework as a useful point of departure for further adaptation to the context of the Amazon.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author Contributions

SP drafted the manuscript based on a collaborative manuscript by all of the co-authors. PM-G, MS, and AG edited the manuscript. SP, PM-G, and MS contributed to the background and theoretical discussion of the evaluation frameworks presented. MA, AB, SF, AG, and MR provided data and narratives on the case studies noted and provided comments on the draft manuscript. All authors have read the full manuscript.

Funding

This study has received funding from the Tropical Conservation and Development Program (TCD) at UF. Portions of the research for the upper Madera, southern Amazonas-northern Rondonia, and Colombia cases were supported by the Gordon and Betty Moore Foundation (Grant #7715).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank the Gordon and Betty Moore Foundation for supporting work in the upper Madera, southern Amazonas-northern Rondonia, and Colombia mosaics. The Inter-American Foundation supported the research on the TIPNIS area. The authors also thank the TCD as well as the Governance and Infrastructure in the Amazon (GIA) project team, led by Robert Buschbacher, Bette Loiselle, Andrea Chavez, SP, Marianne Schmink, Claudia Romero, and Angelica Almeyda, for their support. The authors thank Christine Overdevest and Jazmin Gonzalez-Tovar for providing relevant literature and/or comments on earlier drafts of this manuscript. Errors herein are the responsibility of the authors.

References

Arvor, D., Daugeard, M., Tritsch, I., De Mello-Thery, N. A., Thery, H., and Dubreuil, V. (2018). Combining socioeconomic development with environmental governance in the Brazilian Amazon: the Mato Grosso agricultural frontier at a tipping point. Environ. Dev. Sustain. 20, 1–22. doi: 10.1007/s10668-016-9889-1

CrossRef Full Text | Google Scholar

Bennett, N. J., and Satterfield, T. (2018). Environmental governance: a practical framework to guide design, evaluation, and analysis. Conserv. Lett. 11:113. doi: 10.1111/conl.12600

CrossRef Full Text | Google Scholar

Biermann, F., and Pattberg, P. (2008). Global environmental governance: taking stock, moving forward. Annu. Rev. Environ. Resour. 33, 277–294. doi: 10.1146/annurev.environ.33.050707.085733

CrossRef Full Text | Google Scholar

Damonte, G. H. (2018). Mining formalization at the margins of the state: small-scale miners and state governance in the peruvian amazon. Dev. Change 49, 1314–1335. doi: 10.1111/dech.12414

CrossRef Full Text | Google Scholar

de Koning, J. (2014). Unpredictable outcomes in forestry-governance institutions in practice. Soc. Nat. Resour. 27, 358–371. doi: 10.1080/08941920.2013.861557

CrossRef Full Text | Google Scholar

Dietz, T., Ostrom, E., and Stern, P. C. (2003). The struggle to govern the commons. Urban Ecol. 302, 611–622. doi: 10.1007/978-0-387-73412-5_40

CrossRef Full Text | Google Scholar

Ezzine-de-Blas, D., Börner, J., Violato-Espada, A. L., Nascimento, N., and Piketty, M. G. (2011). Forest loss and management in land reform settlements: implications for REDD governance in the Brazilian Amazon. Environ. Sci. Policy 14, 188–200. doi: 10.1016/j.envsci.2010.11.009

CrossRef Full Text | Google Scholar

Lemos, M. C., and Agrawal, A. (2006). Environmental governance. Annu. Rev. Environ. 31, 297–325. doi: 10.1146/annurev.energy.31.042605.135621

CrossRef Full Text | Google Scholar

Newig, J., and Fritsch, O. (2009). Environmental governance: participatory, multi-level – and effective? Env. Pol. Gov. 19, 197–214. doi: 10.1002/eet.509

CrossRef Full Text | Google Scholar

Nolte, C., Agrawal, A., Silvius, K. M., and Soares Filho, B. S. (2013). Governance regime and location influence avoided deforestation success of protected areas in the Brazilian Amazon. Proc. Natl. Acad. Sci. U. S. A. 110, 4956–4961. doi: 10.1073/pnas.1214786110

PubMed Abstract | CrossRef Full Text | Google Scholar

Ostrom, E., Burger, J., Field, C. B., Norgaard, R. B., and Policansky, D. (1999). Revisiting the commons: local lessons, global challenges. Science 284, 278–282. doi: 10.1126/science.284.5412.278

PubMed Abstract | CrossRef Full Text | Google Scholar

Perz, S., Brilhante, S., Brown, F., Caldas, M., Ikeda, S., Mendoza, E., et al. (2008). Road building, land use and climate change: prospects for environmental governance in the Amazon. Philos. Trans. R. Soc. B Biol. Sci. 363, 1889–1895. doi: 10.1098/rstb.2007.0017

PubMed Abstract | CrossRef Full Text | Google Scholar

Piketty, M.-G., Poccard-Chapuis, R., Drigo, I., Coudel, E., Plassin, S., Laurent, F., et al. (2015). Multi-level governance of land use changes in the Brazilian Amazon: lessons from Paragominas, State of Pará. Forests 6, 1516–1536. doi: 10.3390/f6051516

CrossRef Full Text | Google Scholar

Plummer, R., Armitage, D. R., and de Loë, R. C. (2013). Adaptive comanagement and its relationship to environmental governance. Ecol. Soc. 18:21. doi: 10.5751/ES-05383-180121

PubMed Abstract | CrossRef Full Text | Google Scholar

Schleicher, J., Peres, C. A., Amano, T., Llactayo, W., and Leader-Williams, N. (2017). Conservation performance of different conservation governance regimes in the Peruvian Amazon. Sci. Rep. 7:11318. doi: 10.1038/s41598-017-10736-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Soares-Filho, B. S., Nepstad, D. C., Curran, L. M., Cerqueira, G. C., Garcia, R. A., Ramos, C. A., et al. (2006). Modelling conservation in the Amazon basin. Nature 440, 520–523. doi: 10.1038/nature04389

PubMed Abstract | CrossRef Full Text | Google Scholar

Thaler, G. M. (2017). The land sparing complex: environmental governance, agricultural intensification, and state building in the Brazilian Amazon. Ann. Am. Assoc. Geogr. 107, 1424–1443. doi: 10.1080/24694452.2017.1309966

CrossRef Full Text | Google Scholar

Tritsch, I., and Arvor, D. (2016). New pattern of socio-economic development and deforestation in the Brazilian Amazon. Land Use Policy 59, 446–455. doi: 10.1016/j.landusepol.2016.09.018

CrossRef Full Text | Google Scholar

Vizeu Pinheiro, M., Rojas Sanchez, L., Chamness Long, S., and Ponce, A. (2020). Environmental Governance Indicators for Latin America & the Caribbean. Available online at: https://publications.iadb.org/publications/english/document/Environmental-Governance-Indicators-for-Latin-America–the-Caribbean.pdf. (accessed October 22, 2020).

Google Scholar

Waddington, H., Sonnenfeld, A., Finetti, J., Gaarder, M., John, D., and Stevenson, J. (2019). Citizen engagement in public services in low- and middle-income countries: a mixed-methods systematic review of participation, inclusion, transparency and accountability (PITA) initiatives. Campbell Syst. Rev. 15:e1025. doi: 10.1002/cl2.1025

CrossRef Full Text | Google Scholar