Improving stakeholder engagement in climate change risk assessments: insights from six co-production initiatives in Europe

1. Introduction

While climate adaptation efforts are rolling out across the globe, so far most actions toward adaptation prompt research and planning rather than solutions and implementation (European Commission, 2021). Despite considerable scientific advancements, conventional research falls short in supporting adaptation processes as it rarely offers usable and actionable information for societal actors and is thus not effective in terms of achieving impact on policy and practice (Klein and Juhola, 2014; Bremer and Meisch, 2017; Palutikof et al., 2019). To address this challenge, a growing number of scientists and policymakers call for the re-conceptualization of the role of experts, practitioners, and citizens in the production and use of scientific knowledge, recognizing that different types of knowledge are considered necessary to come together in transdisciplinary processes (European Commission and Directorate-General for Research and Innovation, 2009).

This apparent shift from conventional, science-driven, top-down models to transdisciplinary approaches (Klein et al., 2001) is gaining popularity in the climate services domain (Brasseur and Gallardo, 2016; Vaughan et al., 2018; Vincent et al., 2018; Bremer et al., 2019; Daniels et al., 2020). Climate services are concerned with the development, delivery and use of climate-related knowledge that support long-term planning and decision-making for climate adaptation. They include a wide array of products and services, such as climate change scenarios and projections, climate impact indicators, vulnerability studies, climate risk assessments, socio-economic indicators, general guidance, and tailored user support and training (Máñez et al., 2014).

There are increasing calls for refocusing the climate services lens toward a truly collaborative, process-oriented, and user-driven approach that enables the use of integrated climate information (decision-relevant climate and non-climate information) and thereby increase its usability and uptake (Daniels et al., 2020). This means moving away from supply-driven, one-directional delivery of climate information from providers (e.g., climatologists, meteorologists) to users (e.g., decision-makers, city planners, and extension officers) (Brasseur and Gallardo, 2016; Daniels et al., 2020; Nyadzi et al., 2022).

In recent years, new collaborative and process-oriented climate service frameworks have been introduced (Vincent et al., 2018; Carter et al., 2019; Daniels et al., 2020) to support the design and implementation of transdisciplinary processes spanning across the science and society interface and thereby translating climate information into actionable, climate-resilient decision-making. This is important if we are to bring about fundamental, long-term societal benefits (such as shared understanding, trust-building, expanded networks and partnerships, engagement, ownership, and enhanced individual and institutional capacities) in the face of climate risks (Beier et al., 2017; Vincent et al., 2018; Gerger Swartling et al., 2019; Daniels et al., 2020). This mode of transdisciplinary knowledge generation we hereafter refer to as knowledge co-production, defined as research processes that transcends the divide between academia and society by involving multiple knowledge perspectives (Norström et al., 2020). While these terms are often used interchangeably, we see co-production as a key feature of transdisciplinary research (cf., Polk, 2015; Wibeck et al., 2022).

In this paper, we aim to further refine methods for co-producing climate services to support risk-informed decision-support and adaptation action. By drawing on insights from six case studies of climate risk assessments (one type of climate services) in Northern and Central Europe, we seek to better understand how the associated challenges and opportunities arising in co-production processes play out in different contexts and how lessons learned can help bring further clarity to what methods work when, where, and how (Lang et al., 2012; Harvey et al., 2019; Norström et al., 2020). Hence, we purposively go beyond the aspirational and methodological dimensions of co-production that have dominated the recent sustainability debate and literature (Lemos et al., 2018; Turnhout et al., 2020), to advance our understanding of the practice of applying a co-production approach in regional and local climate risk assessment initiatives.

Common for all selected cases of knowledge co-production processes is their application of a standardized framework for climate vulnerability and risk assessment, the impact chain method, outlined in the Vulnerability Sourcebook (Fritzsche et al., 2014; Zebisch et al., 2021) and its supplements (Zebisch et al., 2017). We qualitatively analyse the processes (including contextual factors) and their effects (outputs, outcomes, and impacts) that emerge from the cases and put them in the context of good practice in co-production of climate services.

In the following section we outline the key concepts of the paper, continuing with a description of the method applied. We then present a synthesis of the results based on the analysis of the six case studies, a discussion of the main findings as well as conclusions with focus on how co-production processes can be improved with wider lessons also for good practice climate services.

2. Co-produced climate services in the context of impact chain-based climate risk assessments

2.1. The impact chain method

In this paper, we define climate risk assessments as one type of climate service that can improve risk-informed decision-support and adaptation action (e.g., Máñez et al., 2014). The impact chain method is widely used in climate risk assessments and documented as a useful tool to develop climate information, communicate climate risk and complex cause-effect relationships, and identify and monitor adaptation options (Zebisch et al., 2021; Petutschnig et al., 2023).

With a starting point in the IPCC (2014) definition of the concept of risk, impact chains are based on a combination of top-down and bottom-up participatory approaches, applicable to local to national scales and different settings. The structure of the impact chain represents the main cause-effect chain: a hazard (e.g., a heavy rain event) may lead to a sequence of intermediate impacts (e.g., erosion upstream that contributes to flooding downstream), which in interaction with the vulnerability (e.g., widespread poverty) of exposed elements of the social-ecological system (e.g., a medium-sized city next to a river) finally lead to a risk or multiple risks.

Following the Vulnerability Sourcebook (Fritzsche et al., 2014), the impact chain method consists of eight modules and subsequent steps (Table 1). A key component is the participation of stakeholders with diverse knowledge as well as context- and location-specific information (Menk et al., 2022). Participatory methods are advocated in all steps, to validate the results and ensure ownership and sustainability. However, stakeholder engagement varies across stages: the first three and the last of these modules are highly participative, while the remaining four modules require operational quantification. This does not, however, exclude stakeholders from being part of the more operational modules. For example, participatory weighting methods are common in climate risk and vulnerability assessments (Haque et al., 2012; Barquet and Cumiskey, 2018).

TABLE 1

Table 1. The impact chain method as outlined in the Vulnerability Sourcebook (Fritzsche et al., 2014; Zebisch et al., 2017, 2021).

A review of the impact chain method shows that stakeholder involvement in climate risk assessments is typically challenging, as it is both time and resource intensive and there is a fine balance to consider diverging interests and different opinions (Menk et al., 2022). However, the review also identifies potential benefits such as increased legitimacy of results; increased self-awareness of climate vulnerability and risk; and enhanced opportunities to validate results and verify adaptation measures (Menk et al., 2022). Yet, like any other participatory process, there are also challenges related to mobilizing enough capacity, resources, and expertise (Page and Dilling, 2019; Norström et al., 2020; Grainger et al., 2021).

2.2. Key elements of good practice in co-production of climate services

Co-production is one of the key factors that contribute to successful climate services, commonly defined as “(perceived) usability” (Boon et al., 2022) and found to support adaptation and climate action (e.g., Lemos and Morehouse, 2005). Co-production and inclusive planning processes that span diverse areas of expertise can help build trust and capacities; develop a common understanding; promote learning, commitment, local ownership; and create networks and partnerships. These are all essential components of science-informed decision making for adaptation (Jönsson and Gerger Swartling, 2014; Rodela and Gerger Swartling, 2019; Daniels et al., 2020; André et al., 2021). While user-producer interactions are essential, they require careful consideration to the design and implementation to generate desired results (Boon et al., 2022).

Frameworks for good practice in co-production exist, which can serve as a guide to overcoming challenges and maximizing benefits. Here, we look at process-centric frameworks developed to support co-produced climate services, notably Vincent et al. (2018), Carter et al. (2019), and Daniels et al. (2020). While these frameworks emphasize different aspects of the co-production process, they have in common a focus on the users and the role of the process to facilitate the development of relevant and applicable climate services, while building resilience and stakeholders' long-term capacities to address climate risks and adapt.

First, the authors recommend to co-explore and consider stakeholder needs and the decision-making context both in the design of the process and the outputs produced. Stakeholder engagement is situating research and analysis within a broader planning or decision-making process (Beier et al., 2017). To ensure a decision-driven process (Vincent et al., 2018), relevant (adaptation) issues and stakeholder information needs have to be co-explored early on in the process (Daniels et al., 2020). It is also key to understand where and how the climate service and its outputs will be used as well as the wider context for stakeholder's ability to participate (e.g., Carter et al., 2019). In a similar vein, the timing and delivery of information must be aligned with the decision-making context to ensure knowledge uptake with consideration to stakeholders' preferred formats and means of communication (Carter et al., 2019).

The way knowledge co-production is conceptualized and implemented, including its aims and terminologies, affects what effects emerge from such processes (Fazey et al., 2014). Considering the lack of conceptual coherence as regards knowledge co-production aims, definitions and practices (Lang et al., 2012; Bremer and Meisch, 2017; Bremer et al., 2019; Chambers et al., 2021), part of the assessment of stakeholder needs is also to bring to the surface the aims of the process with the ambition to “ensure value-added for all involved” (Carter et al., 2019).

Moreover, the authors identifies the importance of having a flexible, iterative, and learning-based approach (cf. Boon et al., 2022). Flexibility is needed as it is not possible to fully map out the co-production process at the start, and monitoring and learning may be required to refine the product and process as a result of continuous knowledge exchange (Vincent et al., 2018). Focus and learning objectives should be established in the initial phase of the process to facilitate monitoring, evaluation, and learning (MEL) (Daniels et al., 2020).

As regards stakeholder involvement, Carter et al. (2019) emphasize the need to embrace diversity, respect differences, and ensure inclusivity. It is acknowledged that expertise central to climate-informed assessments and decision-making processes comes not only from science but also from on-the-ground politics and practice. The most effective decisions thus emerge from incorporating diverse perspectives and disciplines (Daniels et al., 2020). Such well-designed, collaborative, knowledge integration processes bring together insights from people with experience in government, private sector, civil society, and climate science and support the true sharing of power and of knowledge (Daniels et al., 2020).

Further, it is recommended to build human capacity (cf. Palomo et al., 2016) and establish trustful relationships (cf. Culwick et al., 2019). Without trust, Carter et al. (2019) point to the risk that the outputs produced remain underutilized. It can also inhibit future engagements. However, enabling and sustaining trustful relationships and science-stakeholder interactions require both time and resources to achieve sustainability outcomes. Research shows that the costs of pursuing co-production are potentially high in terms of time, money, facilitation expertise, and individual commitment on the part of participants, compared to more conventional modes of knowledge production (Lemos et al., 2018). This highlights the importance of carefully designed processes (cf. Boon et al., 2022) where skilled facilitators are central to mediating between experts and stakeholders as well as ensuring that the process is transparent and fair (e.g., Carter et al., 2019; Daniels et al., 2020).

3. Material and methods

This research builds on work undertaken within the UNCHAIN¹ project which aimed to improve climate change risk assessment methods in general, and the impact chain method in particular. While the project had five sub-goals defined corresponding to specific research and methodological innovation areas (see further details in Petutschnig et al., 2023), this paper addresses specifically the innovation on the co-production of knowledge. The five research innovations in UNCHAIN were tested through 11 case studies in seven countries across Europe, selected to challenge and further develop the impact chain method and related research and innovations areas. The case study research approach enabled in-depth analysis of different contexts, obtained through an exploratory, iterative and inductive stance (Yin, 2009; Stjelja, 2013).

Our study focuses on six of the 11 case studies that have applied stakeholder engagement methods and co-production techniques in the concerned local/regional climate risk assessments (Table 2). The cases represented a diverse set of climate risks, sectors, and European countries: multiple hydrometeorological hazards (3), transboundary climate risks (TCR) (5, 6), critical infrastructure (1, 4), and agriculture (2) (Figure 1). They were designed and implemented independently by case study researchers under the guidance of a common case study protocol developed to support the knowledge co-production process as well as the proposed modules and steps outlined in the Vulnerability Sourcebook (Fritzsche et al., 2014). To extract challenges and opportunities for future climate risk assessments, we synthesized different experiences of applying the impact chain method and its participatory elements. Given the many differences, we avoid making a cross-case comparison. Even though the case studies did not relate to all the impact chain modules and steps as outlined in Table 1, they all implemented the first two modules that required active participation from stakeholders.

TABLE 2

Table 2. Case study overview.

FIGURE 1

Figure 1. The geographical location of case studies.

To examine how the case studies included knowledge co-production and the effects of these processes, an evaluation framework with qualitative indicators was developed. The framework used synthesized knowledge gaps found in the literature (Leander et al., 2020) as a baseline to construct both general and specific research questions and related indicators. These were validated against a review of 25 peer-reviewed articles suggesting co-production evaluation practices (Englund et al., 2022).

Noting the difficulties of attributing research impact to a specific intervention (Belcher and Palenberg, 2018; Reed et al., 2021) we adopted a so called “system perspective” approach focussing on capturing different factors that contributed to the results (Belcher and Palenberg, 2018). To this end, we found the Wall et al. (2017) framework for evaluating co-produced climate science particularly useful, acknowledging multiple components—including internal, external and process related factors—of relevance. This framework also corresponds to key factors identified in the wider literature on evaluating co-production processes (Englund et al., 2022). In brief, we applied three overarching categories centered on: (1) the knowledge co-production process, (2) co-production effects, and (3) contextual factors (Table 3).

TABLE 3

Table 3. Framework for analysis (building on Wall et al., 2017).

Indicators related to the process focused on assessing both input- and process-specific components including stakeholders' and researchers' preconditions and capacities to participate effectively, as well as the nature of interaction and knowledge exchange. To assess the effects, we focussed on indicators capturing different types of outputs (e.g., peer reviewed articles or technical reports), outcomes and impacts. Following Wall et al. (2017, p. 100) we defined outcomes as the “tangible and more conceptual results” of both the outputs produced and from the process itself. In line with Wiek et al. (2014), outcomes are generated during project life cycle whereas impacts refer to more long-term effects (see also e.g., Belcher and Palenberg, 2018). Impact related indicators thus aim to capture aspects such as how results were used to inform adaptation action or decision-making (Wall et al., 2017). Lastly, contextual factors refer to factors outside the process which may be important for understanding whether and how the results are being used or not. This could for example relate to political will and access to financial resources (Wall et al., 2017).

The different knowledge co-production processes were analyzed through a collaborative and iterative approach involving contributors of each case study (see Chambers et al., 2021). To collect insights from the cases a combination of methods was applied; surveys, interviews, and a workshop complemented with relevant case study documentation and research observations and reflections (Table 4). The three lead authors of the paper led the work whereas the co-authors and case study researchers contributed with results and empirical knowledge from their respective case studies and validated emergent findings (c.f., Chambers et al., 2021).

TABLE 4

Table 4. Outline of the evaluation process undertaken by the case study research team.

Guided by the evaluation framework, a survey was developed for case study researchers consisting of 32 quantitative and qualitative questions covering both basic information about the case study and the knowledge co-production process, effects, and context (see Supplementary material). The survey was completed by the case study research teams, one per case study. Complementary unstructured interviews were then conducted with one or several individuals in each case study for more in-depth insights and contextual information. In total ten researchers participated in the interviews.

We also collected stakeholder inputs to capture perceptions of the process and the results, and for validation of results. To this end, a protocol with questions was developed and adapted to the specific case studies including a selection of key questions related to the outputs, outcomes, and impacts. Responses were collected through interviews or surveys with key stakeholders in five of the case studies. Case studies 1 and 3 received responses from three stakeholders, case study 6 engaged six stakeholders whereas case study 2 gathered input from one stakeholder. Case study 4 conducted a feedback analysis with four representatives of three real estate companies that were involved in the study (see Attoh et al., 2022 for details). For case study 5 we relied on stakeholder feedback and contributions with 15 interviews of experts completed during the production stage, and a validation workshop with the Paris stakeholders for sharing and discussing the case study results. All in all, in the analysis we focused on process-related aspects, building primarily on researcher experiences and insights from designing and facilitating the co-production processes.

The results from the interviews and surveys were analyzed using an inductive approach based on the evaluation framework and suggested indicators. We developed codes as the analysis progressed, distilling themes and commonalities as they emerged and then organized the codes according to the categories outlined in the evaluation framework—process, effects, and context. We synthesized each code separately by clustering data into classes that consisted of similar objects. A workshop was held with case study researchers and co-authors to collectively discuss and synthesize results from the evaluation. The results were analyzed in terms of good practice co-produced climate services with a focus on user-driven and process-centric frameworks and principles (Vincent et al., 2018; Carter et al., 2019; Daniels et al., 2020).

4. Results

In this section, we present results from the analysis of the co-production processes in the six climate risk assessment initiatives. We begin with outlining process-related aspects and then proceed with describing the various effects (outputs, outcomes, and impacts) identified. Throughout, we relate to external and contextual factors that were found to be important to both the processes and the results.

4.1. Process and nature of stakeholder engagement

4.1.1. Identification and selection of stakeholders

In each case, the knowledge co-production process was conducted within two-years, and engaged on average 17 participants from academia, national agencies, municipalities, civil society, private enterprises, and politicians (Table 2). Stakeholders were mainly identified through existing networks and research teams' previous relationships with stakeholders in respective case study location. We found that local champions were key in the process of setting up the case studies. With support from these local contacts additional stakeholders were identified and invited for participation if relevant in the climate risk context being investigated. The climate risk context was key in the process of identifying stakeholders, yet it turned out to be challenging in some case studies. Case study 1 noted for example, that the case study topic limited the number of interested stakeholders hence they slightly modified the geographical scope of the research. Case study 4 differed as stakeholders were predetermined as they requested the climate risk assessment of researchers.

4.1.2. Problem definition and expectations

Four of the six case studies were initiated by researchers. The problem definitions related to case study contexts were in many cases already formulated when seeking project funding. Most of the cases, however, refined and adapted the problem definition based on stakeholder priorities and needs (Table 5). In Sweden (3) this was done by inviting them to a workshop to discuss and gather feedback on the aim and scope of the case study, allowing the problem definition to represent inputs from both researchers and stakeholders. Yet a difference between expectations remained in some of the case studies. For example, there were occasions when stakeholders expected to identify adaptation measures whereas the case study was designed to test the impact chain method (case study 2). In the Netherlands (4) expectations from stakeholders exceeded what was possible to achieve in terms of scientific deliveries due to data availability and state of the art. The researchers carefully explained why some expectations remained unmet to address the mismatch in expectations. In the end, this clarification improved acceptance and ownership of the process.

TABLE 5

Table 5. Nature of knowledge co-production processes in case studies.

In the analysis we found three overall aims underpinning the six case studies: informing decision-making; methodological development or improvement of the impact chain method; and exploring a specific research topic and contributing to scientific progress (Table 5). The first aim was characterized by extensive stakeholder engagement and communication where stakeholders could provide input to the framing of the case study. In contrast, the two latter themes engaged stakeholders to improve research findings rather than designing knowledge fit for informing decision-making.

4.1.3. Co-production activities

Depending on the specific aim, the participatory processes were purposefully structured differently. Most cases applied an iterative participatory process in which the climate risk assessment was validated and refined based on stakeholder input. The cases employed different approaches to co-production including online and in-person workshops, group interviews, individual exchanges, and field visits (Table 5). Individual exchanges entailed informal meetings with researchers and stakeholders to prepare for the climate risk and vulnerability assessment and build rapport.

Some cases involved their stakeholders on an ad-hoc basis where different stakeholders attended workshops at different points in time depending on their expertise. Others facilitated a continuous dialogue where the same stakeholders were engaged throughout the process. For example, case study 4 involved stakeholders for collaboration in all parts of the process. For the railway sector, there was one in-person workshop and 15 interviews, and for the financial sector, there were two workshops and five interviews conducted. In addition, several phone calls were made between the researchers and stakeholders throughout the process. Stakeholders provided data and shared detailed insights on the challenge itself, helping the researchers to better understand the problem.

Case study 1 followed the steps outlined in the Vulnerability Sourcebook, but through an iterative refinement process. They did not involve the same stakeholders in all workshops, but the group changed depending on workshop purpose. Some stakeholders were involved in several workshops whereas some were only involved in one. Two workshops were conducted to construct the impact chain, the process then continued with further refinements. In the words of the research team:

After the impact chain workshops, we introduced the relations between the identified elements in the impact chain. We made some suggestions on how these impacts could be related, which we shared with the stakeholders and made a detailed list of all the changes that were made. We received feedback and then we adapted the impact chain accordingly. Three times the impact chain was circulated back and forth. It was a continuous discussion. We ended up with results that were feasible for both us and stakeholders.

Case study 3 combined a mix of workshops and group interviews. The first workshop was held online and aimed at introducing the project and to further define case study objectives. This was done through an open discussion with the stakeholders on risks and current challenges to the locality. Stakeholders were then invited to a second online workshop in which they brainstormed around what social groups might render vulnerable in the case of flooding. Based on results from the first and second workshop the research team developed an impact chain outlining social vulnerability to flooding. The third stakeholder iteration was structured as group interviews where a list of possible vulnerable groups was used for more in-depth discussions of drivers of vulnerability. Stakeholders were then invited to a validation workshop in person in which they provided feedback on the final impact chain.

Case study 2 took a somewhat different approach. Most information was collected from stakeholders prior to the workshops to instead focus the discussions on drivers of risks and adaptation measures. As described by the research team:

The first workshop was at the beginning of the case study. It was an online workshop presenting the project and then looking at the impact chain and drivers, as well as adaptation measures the different stakeholders were undertaking. We shared a survey with the participants prior to the workshop asking: In your opinion what is amplifying drought impacts? What are the drivers? What can we do about it? We collected this information before the workshop so we could show it to them to support the discussion. We did not make an impact chain during the workshop, we only discussed drivers and adaptation measures.

This was also the situation in case study 5. The researchers conducted online interviews with selected key stakeholders to gather input and identify indicators to present in the impact chain. Stakeholders then provided data and validated the impact chain during a workshop. The actual development of the impact chain was done in-house without any involvement of stakeholders. Toward the end of the process, a workshop to identify adaptation measures was held together with city of Paris stakeholders.

Case study 6, on the other hand, facilitated several interactions with local stakeholders. During the initial phase of the process, the research team had two initial meetings with the municipal administration and local politicians to explain the project and to set the scene. They then had smaller meetings with the project leaders from the local and regional municipalities to discuss how to structure the workshops:

Before the first workshop, we gathered a small group for an online meeting to prepare them as group leaders and enable them to take charge of working with the impact chain method. The first workshop was on local climate risks, using the impact chain method. Stakeholders contributed with real content to the analysis and decided what was essential to include. Before the second workshop, we developed a flow chart which we sent out to the participants for evaluation beforehand.

Since most case studies started in 2020, the co-production processes were adapted to the specific restrictions imposed in response to the COVID-19 pandemic. All case studies, therefore, used virtual forms of interactions, of which most used digital participatory platforms and online visualization tools such as Miro, Mural, or Mentimeter to support the process. As these online platforms enable user-generated content, they allowed participants to co-create perspectives and jointly develop impact chains and identify risk factors. In the Swedish case study (3), stakeholders were engaged using Mentimeter to co-design the research scope and objectives. Mentimeter allowed the stakeholders to suggest and vote for important risks for the municipality to consider, while disseminating results in real-time to support group discussions.

4.1.4. Capacity to engage

We identified several factors related to both stakeholders' and researchers' capacities and resources to engage effectively in the co-production processes. All researchers had prior experience with stakeholder engagement and four case studies had researchers with experience in using the impact chain method. Stakeholders, on the other hand, varied in terms of their previous experiences, knowledge, and skills regarding the adaptation challenges being addressed. Some were unfamiliar with scientific terminology to define and discuss climate risks and struggled to differentiate between exposure, vulnerability, and risk (case studies 2 and 4). The cases working with TCR (case study 5 and 6) required a high level of abstract thinking, which stakeholders found difficult. Similarly, researchers involved in the Swedish case study (3) noted that stakeholders sometimes found it difficult to think beyond their day-to-day field of work to also include social groups and their vulnerabilities.

As noted by all case study leaders, knowledge co-production is time- and resource-intensive and the time allocated to stakeholder engagement varied across cases. Many stakeholders had other tasks and duties to fulfill which limited their time availability for the study. Researchers also reported limited resources and consequently time to engage effectively with stakeholders. Here, we noticed the importance of unforeseen, external factors outside the system, such as the COVID-19 pandemic and the 2022 Russian invasion of Ukraine that imposed additional challenges as some stakeholders were unable to continue to participate. For example, in Germany (1) some stakeholders were prevented from participating as they were occupied with responding to the disruptions triggered by the pandemic, and later managing the energy crisis resulting from the invasion. In France (5), stakeholders needed to prioritize the flow of migration from Ukraine.

While most found the online meetings useful for example as they required less time investment on the part of participating stakeholders, there were also occasions when communication was hindered by poor internet connection (case study 5) or digital literacy (case study 1) which inhibited active participation and knowledge exchange.

4.2. Co-production effects

The assessment of case studies took place in 2022 meaning that there was only limited time for outcomes and impacts to emerge by the time of writing of this article. Another challenge is the apparent difficulty of attributing outcomes to particular activities (VanderMolen et al., 2020) which was not a central focus in our analysis. Despite these constraints, we identified a variety of effects emerging from the six co-production processes (Table 6).

TABLE 6

Table 6. Overview case study results in the form of output, outcome and impacts.

4.2.1. Outputs and communication

The survey showed that all case studies generated a diverse set of tangible outputs including excel tools, project reports, and journal articles (Table 6). Visual representations of the results included impact chains, interactive dashboards, and maps such as risk and vulnerability hotspot maps. Case studies 2–4 presented the findings in a final validation workshop, whereas the remaining case studies communicated their outputs via email. Most outputs were posted online for a wider audience. Some cases described how they adapted the outputs to stakeholder needs, for example by avoiding scientific jargon, using the local language, and keeping the written content brief. However, few case studies indicated that they had involved stakeholders in planning the communication of results as well as feedback. Researchers involved in case study 5 further noted that stakeholders shared the results internally which is a sign of the perceived relevance of the findings for a wider group of stakeholders.

4.2.2. Outcomes

Looking more specifically at the outcomes, we observe from the survey that scientific progress was the most common, especially regarding improvements in the impact chain method. Methodological innovations included modeling dynamic interactions, assessing transboundary risk, and developing feedback loops and casual relationships. Also, related to research advancements, all case studies developed plans for future collaboration and identified new research questions and initiatives. Future collaboration was foreseen, both among the researchers themselves, as well as with the involved stakeholders.

The second most reported outcome was a change in perceptions and increased awareness among stakeholders who experienced an improved understanding of climate change impacts and the significance of adaptation and risk assessments. In the Netherlands (4), stakeholders gained a better theoretical understanding of key concepts such as climate risks and uncertainty. In the Swedish case study (3), stakeholders increasingly considered the social dimension of flood risk in addition to its physical and climatic parameters. Similarly, stakeholders in the Norwegian case study (6) improved their awareness and understanding of TCRs. Three case studies also indicated mutual learning as an outcome, where the impact chains seem to have served as a boundary object supporting this to happen. Stakeholders learned about the research topics, whereas the researchers developed an understanding of the decision-making context. In Germany (1), mutual learning evolved through an iterative process in which the impact chain was circulated and adapted three times to integrate knowledge from stakeholders. Similar results were also found in Austria (2), where the co-development of the impact chain helped to reduce complexity while fostering creativity which improved stakeholder and researcher understanding of the topic of agricultural drought.

From the survey results, we also noted that the exchanges between researchers and different stakeholders increased the understanding of the roles of others. For example, representatives from the agricultural and industry sector in Norway (6) enjoyed learning about others' perspectives. However, according to the survey results, trust-building seems to have occurred in two case studies only (case studies 1 and 3). It appears that the restrictions implemented in response to the COVID-19 pandemic prevented trust from emerging in the other four case studies. This is likely because there were fewer opportunities for informal exchange which had negative implications for the quality of interaction and trust-building. For example, in Sweden (3) trust-building was facilitated by continuous interactions and a final face-to-face workshop that brought stakeholders and researchers together which improved the dialogue and collaboration. At the same time, all case studies had initiated plans for future collaboration, which is a clear sign of good relationships and possibly also mutual trust. Also, the fact that most cases relied on previous relationships when initiating the case studies indicates that case studies were characterized by high levels of trust from the very beginning.

4.2.3. Early signs of impact and the role of external factors

While we cannot see any clear impacts (as too early in the process), all case studies indicated that they supported ongoing climate and policy initiatives. For example, parallel to the research, case studies provided input to policies under development including climate adaptation strategies, heat action plan, agricultural plan, climate vulnerability study, flood protection plan, investment decisions, and municipal plan. Ongoing policy development was perceived to increase the relevance of the case studies. They also provided an entry point for results to be integrated into policymaking, facilitating the process of informing adaptation planning and decision-making. For example, the German (1) and Austrian (2) case studies supported policies and action plans drafted as a response to the 2018 heatwave. Thereupon, the heatwave in 2018 served as a window of opportunity for researching extreme heat and water scarcity. Based on researchers reports, their stakeholders perceived the topic as relevant already before the UNCHAIN project started.

More concrete examples of actual impact include the stakeholder engagement process in the city of Paris (5) that paved the way for the local government's decision to incorporate the TCR dimension in the municipal policy agenda and in their systems for assessing risk. Similar results appeared in the Norwegian case study (6) where reports provided by the project have been incorporated into two main municipal plans. These two examples further point to the importance of contextual factors where external events appeared to affect the uptake of knowledge. Researchers noticed that TCR appeared to gain importance on the public agenda, starting to make its way into planning and decision-making. Hence, the timing of the case studies coincided with increased attention paid to these issues. In addition, the 2022 energy crisis and the Russian invasion of Ukraine further highlighted the importance of considering TCRs, increasing the perceived relevance of the case studies (5 and 6) addressing such topics.

5. Discussion

In this section, we discuss challenges and opportunities arising from the studied co-production processes and how they played out in different case-specific risk assessment contexts, with the ambition to improve the usability of the impact chain method and climate services in Europe and beyond.

Overall, the analysis shows that the six case studies across Northern and Central Europe relate to elements of good practice co-production (see section 2.2) in different ways and that the impact chain method supported this process. To a varying degree and through a diverse set of approaches and formats, the participatory processes enabled the co-exploration of stakeholder needs and adaptation pathways in the respective localities. However, as case studies were informed by the overarching aim of methodological development, the seeming knowledge fit for decision-making was not the primary objective. Moreover, case studies built human capacity and trustful relationships, and involved skilled facilitators.

While this study did not go into depth with the question of how case studies embraced diversity, we note that different types of stakeholders were involved (academia, national agencies, municipalities, civil society, private enterprises, and politicians), and that they in most cases were identified and invited to participate in close dialogue with local contact persons. In addition, how case studies respected differences and ensured inclusivity has not been captured in the analysis. Researchers reported however that they used different methods and techniques to engage with stakeholders including online tools. Obviously, the online format might have benefited some stakeholders whereas other might have disadvantaged.

The COVID-19 crisis further illustrates the importance of both the external environment and how it shapes the co-production process, and the need for flexible approaches that are sensitive to contextual factors. However, here we see a challenge to both consider project-specific demands and limitations which makes it difficult to embrace a truly collaborative and stakeholder-driven approach. Moreover, aspects related to communication, timing, and delivery of results were not in focus in any of the case studies even though we found examples of how outputs were adapted to stakeholder needs.

Based on the analysis of the results we identify three domains to foster more collaborative and user-driven processes that support the acceleration of adaptation action and resilience: formulating joint learning objectives and expected outcomes; communicating and presenting results; and supporting iterative learning.

5.1. Formulating joint learning objectives and expected outcomes

Being “research-output-oriented” in nature, the empirical cases reported in this study were largely driven by what Chambers et al. (2021) frame as “Mode 1: Researching solutions”. This has further implications for the type of outcomes and outputs that can realistically be expected. For example, we found that case studies that explicitly aimed to further develop the impact chain method generated an improved understanding of the topics and concepts in focus among stakeholders. Although the results seemed policy relevant in terms of informing ongoing planning processes, the extent to which involved stakeholders applied the results in adaptation planning remains unclear at the time of writing this article even though we see early signs of impact.

These research-focused approaches appear as one obvious explanation why stakeholders were only partly involved in defining the problems and risks. Following Carter et al. (2019) case studies operated along the spectrum of consultative and immersive co-production approaches. In some cases that were more on the consultative side of the spectrum, research objectives were defined by the researchers during the proposal development stage and hence without any involvement of stakeholders. In most cases, however, the problem definition was the product of a compromise between researchers' initial problem formulation and stakeholder feedback regarding their needs and expectations from the collaborative process. If the goal is to generate actionable knowledge and stakeholder empowerment, then the role of stakeholders and their inputs need to be more prominently featured throughout the process.

Consequently, in line with previous research (Reed et al., 2018), we argue that it is critical to engage stakeholders early on to ensure that their perspectives and needs are considered throughout the process. Inclusive priority setting and equal power sharing are generally aspired to in genuinely participatory processes. Studies show that power imbalances may be a challenge as elite actors are often able to shape these processes to serve their own interests (Parkinson, 2012). In co-production initiatives, power inequalities may be further compounded by the strong authority attributed to scientific expertise in relation to other knowledge systems (Turnhout et al., 2020). Thus, the sheer involvement of stakeholders throughout the process is not sufficient to address power dynamics. This challenge is not specific to the impact chain method but applies to any process that seeks to integrate different knowledge bases and expertise. In co-production processes it appears critical to facilitate, manage and co-ordinate the complex web of psychological, social, cultural and institutional interactions that are in play, and apply a constant critical reflective practice and dialogue to foster more equal relational co-production and co-design processes (Farr, 2018). This approach aligns with that of Daniels et al. (2020) who propose a framework for designing transdisciplinary knowledge integration processes based on co-exploration and co-production processes using a wide array of knowledge. Such a collaborative learning approach provides a structure for understanding decision needs; guiding actors in designing and delivering an effective transdisciplinary knowledge integration process; and, enhancing capacities (both individual and institutional), working relationships and networks necessary for longer-term change and action. Applied in the context of UNCHAIN, such a truly collaborative approach can assist in clarifying both stakeholders' and researchers' expectations of the process and identifying the knowledge and capacity gaps in relation to adaptation, while also mitigating power imbalances.

In this context, we note that the impact chain method (Fritzsche et al., 2014) provides good support and structure, especially through the first module and the scoping phase of the risk assessment. We, however, suggest incorporating a Theory of Change (ToC) to describe and illustrate how and why change is expected to occur and its impact, as well as who might be affected (van Es et al., 2015). A ToC engages stakeholders and researchers in a collaborative backward-mapping process, bridging potential contrasting values, epistemological beliefs, and diverging expectations. Stakeholders and researchers first co-explore desired long-term objectives, followed by designing a pathway of change that outlines intermediate learning objectives, activities and outputs, and assumptions (Fazey et al., 2014; van Es et al., 2015). The ToC fits well in the initial module of the impact chain method, scoping, as it allows stakeholders and researchers to co-explore issues and context in depth and formulate joint learning objectives and expected outcomes. Thereafter, the ToC can be used to monitor and evaluate the co-production process and encourage reflection and learning as new insights emerge (Englund et al., 2022).

5.2. Communication and presentation of results

One of the rationales for co-producing climate services is to increase the usability and uptake of results (e.g., Chiputwa et al., 2020; Boon et al., 2022). The process, if implemented effectively, can lead to science made more accessible to decision-makers and an increase in the perceived saliency, credibility and legitimacy of research outcomes (Cvitanovic et al., 2019). Making science more usable is, however, not only about the content and quality but also how the results are presented and communicated (Lemos and Morehouse, 2005; Vincent et al., 2018; André et al., 2021).

In our analysis, we found that few, if any, had involved stakeholders in the communication of results, for example by discussing preferred forms and format, resolution of data, and scale (e.g., spatial and/or temporal) and the timing of deliverables. Timing is, for example, important to consider in relation to case study planning and decision contexts (Carter et al., 2019). The outputs produced were mostly in the form of written reports and presentations, shared via email, and hence little scope for discussion and feedback. It thus appears that, while stakeholders had been actively engaged in previous steps of the process, they seem to have been more passively involved in the final impact chain module.

The current impact chain method provides little guidance on how results should be communicated to stakeholders. We therefore see a need to discuss this early in the process and clearly involve stakeholders in the communication as well as their preferences for how they want to receive the results (e.g., formats, scale, timing etc.). Previous research (e.g., Vincent et al., 2018) emphasize that, in order to be effective to users, scientific information needs to be communicated in a format and language that is relevant and understandable to them. However, there is often not one single type of user, which is why different formats might be preferred to ensure that the information is accessible and actionable to all relevant stakeholders. To guide the process, we suggest that relevant (tangible) outputs and desired outcomes are identified early on, ideally in the scoping phase of the process when co-exploring stakeholder needs. This could also be further connected with an assessment of stakeholder capacity building needs.

5.3. Iterative learning

Our findings indicate that the impact chain method can support an iterative feedback process. Most case studies invited stakeholders to validate and refine the climate risk assessment. We found, however, that stakeholders were sometimes involved on an ad hoc basis. Moreover, we observe challenges when stakeholder needs must be reconsidered. As highlighted by some case studies, external events can trigger changes in project plans or even objectives. Other case studies experienced a mismatch in expectations. This highlights the need for an iterative and flexible approach to allow for the reflection and processing of information as new knowledge emerges throughout the process. To this end, we suggest integrating a mechanism for practices to adapt as new information emerges. One promising approach is to draw from certain principles of interaction that mediate the consequences of practices that suppress uncertainty to gain or maintain control, and instead aim for more adaptive management (Armitage et al., 2011; Bremer and Meisch, 2017; Arora, 2019).

Further, co-production processes are rarely evaluated (Lemos et al., 2018), yet a growing body of research suggests that monitoring and evaluation can support iterative learning and adaptive management in complex endeavors (Patton, 2010; van Tulder and Keen, 2018; Englund et al., 2022; Visman et al., 2022). A monitoring framework allows stakeholders and researchers to reflect whether learning outcomes are achieved and adjust the implementation process accordingly, hence stimulating a continuous real-time feedback loop that connects evaluation findings and decision-making. To ensure contextual relevance, the co-production process must adapt as new information or needs emerge. An iterative approach can thus support the impact chain method in learning and feedback by monitoring the progress, refining the climate risk assessment, and adapting to new circumstances.

While the impact chain method is a standardized approach for conducting climate risk assessments, the absence of iterative learning and flexibility has less to do with the method per se but rather a potential discrepancy in research project design and funding agency requirements. The development of climate services—in this study in the form of climate risk assessments—therefore needs to increasingly emphasize capacity building and long-term climate resilience beyond the scope of a specific project. In line with Vincent et al. (2018) and Daniels et al. (2020) we observe that iterative learning and adaptive management require a process-centric approach when co-producing climate risk assessments. This approach is underpinned by sustained engagement and interaction that allow for iterative learning and co-benefits to emerge, for example related to networks, empowerment, and trust.

6. Conclusions

This article has presented findings on the practice of knowledge co-production which represents one of six research innovations of the impact chain method investigated in the UNCHAIN project. The study is based on a qualitative analysis of six European climate risk assessment initiatives that collectively testify to the potential benefits of combining good practice knowledge co-production beyond what is currently practiced in impact chain studies, and the potential barriers to undertaking such co-production approach in a real-world context. While the structured and stepwise approach of the impact chain method proved beneficial to the knowledge co-production process per se, in reality there was a predominantly expert-driven approach to stakeholder-informed climate risk assessments, where stakeholder perspectives and needs remained somewhat hidden or (at least partially) untapped.

At the same time, we have observed that the collaborations and interactions have contributed to a number of benefits on the part of participating researchers and stakeholders. These include awareness raising and mutual learning where stakeholders, on the one hand, have gained understanding of climate risks, impacts and vulnerabilities, whereas researchers have deepened their knowledge about local and regional decision-making contexts and the need for tailor-made climate risk assessments. Plans for future collaboration also indicate that case studies have been successful in establishing good relationships to further build on, which may ultimately foster deeper researcher-stakeholder interactions in the longer term. Some case studies reported scientific progress and methodological innovations emerging from the co-production approach to climate risk assessments. Importantly, although the climate risk assessment processes are relatively recent results have to some extent proven to inform and contribute to ongoing adaptation policy and planning processes.

However, challenges remain as to how to adopt and integrate a flexible and iterative approach to co-production, where stakeholder needs and capacities are reassessed during the process, especially to account for external events and circumstances. Altogether these lessons demonstrate the complexity involved in co-production processes that aim to support actionable climate services. In this paper, we argue that these challenges can be overcome through due attention to joint iterative learning facilitated through co-developing a Theory of Change (ToC) and by introducing monitoring, evaluation and learning (MEL) frameworks to support a flexible approach while providing an opportunity for joint discussion and feedback.

In line with Lemos et al. (2018), we see co-production as a mechanism to enhance the uptake of scientific knowledge informing adaptation planning and decision-making, yet it cannot be an end-goal in itself. To move beyond awareness raising to adaptation action, which is called for by the EU Adaptation Strategy (European Commission, 2021), co-production processes need to be carefully designed and facilitated as well as further reflected upon (cf. Bremer and Meisch, 2017). As one first step we propose future research to assess the value of applying more flexible, iterative and reflexive participatory approaches that foster long-term capacity building. This capacity enhancement is required both within academia to engage effectively with stakeholders, and in practice to equip stakeholders with actionable climate services.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

KA: conceptualization, methodology, validation, formal analysis, investigation, writing—original draft, writing—review and editing, project administration, and funding acquisition. ÅG: conceptualization, methodology, validation, investigation, writing—original draft, writing—review and editing, supervision, project administration, and funding acquisition. ME: conceptualization, methodology, validation, formal analysis, investigation, writing—original draft, and writing—review and editing. LP: validation, investigation, visualization, writing—original draft, and writing—review and editing. EA: methodology, validation, investigation, writing—original draft, and writing—review and editing. KM: validation, investigation, writing—original draft, and writing—review and editing. DL and TB: validation, investigation, and writing—review and editing. AC: validation, investigation, writing—original draft, writing—review and editing, and funding acquisition. MH: writing—original draft and writing—review and editing. MB: investigation and writing—review and editing. ER: validation, investigation, writing—review and editing, supervision, and funding acquisition. All authors contributed to the article and approved the submitted version.

Funding

The project UNCHAIN: Unpacking Climate Impact Chains was funded through the EU funding mechanisms Joint Programming Initiative (JPI) and Assessment of Cross (X)-sectoral climate impacts and pathways for Sustainable transformation (AXIS), with support from the EU (Grant No.: 776608). All partners are granted financial support through their national funding agency: FORMAS (SE, 2018-02737), DLR/BMBF (DE, grant number FKZ 01LS1908B), AEI (ES), FFG (AT, reference 872000), ANR (FR), NOW (NL), and RCN (NO) with co-funding from H2020. We also acknowledge funding from the Swedish Civil Contingencies Agency (MSB) and Formas—a Swedish Research Council for Sustainable Development for contributing to the implementation of the Swedish case study through the HydroHazards project (Grant No.: MSB 2019-0651).

Acknowledgments

The authors wish to thank all stakeholders for their participation in the project. We are also grateful to all UNCHAIN researchers and to Elin Leander, Stockholm Environment Institute (SEI) that contributed to the initial knowledge review on knowledge co-production in climate risk assessments.

Conflict of interest

AC and MB are employed by Ramboll France SAS.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fclim.2023.1120421/full#supplementary-material

Footnotes

1. ^https://www.vestforsk.no/en/project/unpacking-climate-impact-chains-new-generation-action-and-user-oriented-climate-change-risk

References

André, K., Järnberg, L., Gerger Swartling, Å., Berg, P., Segersson, D., Amorim, J. H., et al. (2021). Assessing the quality of knowledge for adaptation–experiences from co-designing climate services in Sweden. Front. Clim. 3, 636069. doi: 10.3389/fclim.2021.636069

CrossRef Full Text | Google Scholar

Armitage, D., Berkes, F., Dale, A., Kocho-Schellenberg, E., and Patton, E. (2011). Co-management and the co-production of knowledge: Learning to adapt in Canada's Arctic. Glob. Environ. Chang. 21, 995–1004. doi: 10.1016/j.gloenvcha.2011.04.006

CrossRef Full Text | Google Scholar

Arora, S. (2019). Admitting uncertainty, transforming engagement: towards caring practices for sustainability beyond climate change. Reg. Environ. Chang. 19, 1571–1584. doi: 10.1007/s10113-019-01528-1

CrossRef Full Text | Google Scholar

Attoh, E. M. N. A. N., de Bruin, K., Goosen, H., van Veldhoven, F., and Ludwig, F. (2022). Making physical climate risk assessments relevant to the financial sector – Lessons learned from real estate cases in the Netherlands. Clim. Risk Manage. 37, 100447. doi: 10.1016/j.crm.2022.100447

CrossRef Full Text | Google Scholar

Barquet, K., and Cumiskey, L. (2018). Using participatory Multi-Criteria Assessments for assessing disaster risk reduction measures. Coastal Eng. 134, 93–102. doi: 10.1016/j.coastaleng.2017.08.006

CrossRef Full Text | Google Scholar

Beier, P., Hansen, L. J., Helbrecht, L., and Behar, D. (2017). A how-to guide for coproduction of actionable science. Conserv. Lett. 10, 288–296. doi: 10.1111/conl.12300

CrossRef Full Text | Google Scholar

Belcher, B., and Palenberg, M. (2018). Outcomes and Impacts of Development Interventions: Toward Conceptual Clarity. Am. J. Eval. 39, 478–495. doi: 10.1177/1098214018765698

PubMed Abstract | CrossRef Full Text | Google Scholar

Boon, E., Wright, S. J., Biesbroek, R., Goosen, H., and Ludwig, F. (2022). Successful climate services for adaptation: What we know, don't know and need to know. Clim. Serv. 27, 100314. doi: 10.1016/j.cliser.2022.100314

CrossRef Full Text | Google Scholar

Brasseur, G. P., and Gallardo, L. (2016). Climate services: Lessons learned and future prospects. Earth's Fut. 4, 79–89. doi: 10.1002/2015EF000338

CrossRef Full Text | Google Scholar

Bremer, S., and Meisch, S. (2017). Co-production in climate change research: reviewing different perspectives. Clim. Change 8, 1–22. doi: 10.1002/wcc.482

CrossRef Full Text | Google Scholar

Bremer, S., Wardekker, A., Dessai, S., Sobolowski, S., Slaattelid, R., and van der Sluijs, J. (2019). Toward a multi-faceted conception of co-production of climate services. Clim. Serv. 13, 42–50. doi: 10.1016/j.cliser.2019.01.003

CrossRef Full Text | Google Scholar

Carter, S., Steynor, A., Vincent, K., Visman, E., and Waagsaether, K. (2019). Co-production of African weather and Climate Service. Manual. Cape Town: Future Climate for Africa and Weather and Climate Information Services for Africa Available online at: https://futureclimateafrica.org/coproduction-manual (accessed May 26, 2023).

Google Scholar

Chambers, J. M., Wyborn, C., Ryan, M. E., Reid, R. S., Riechers, M., Serban, A., et al. (2021). Six modes of co-production for sustainability. Nat. Sustain. 4, 983–996. doi: 10.1038/s41893-021-00755-x

CrossRef Full Text | Google Scholar

Chiputwa, B., Wainaina, P., Nakelse, T., Makui, P., Zougmoré, R. B., Ndiaye, O., et al. (2020). Transforming climate science into usable services: The effectiveness of co-production in promoting uptake of climate information by smallholder farmers in Senegal. Clim. Serv. 20, 100203. doi: 10.1016/j.cliser.2020.100203

CrossRef Full Text | Google Scholar

Culwick, C., Washbourne, C.-L., Anderson, P. M. L., Cartwright, A., Patel, Z., and Smit, W. (2019). CityLab reflections and evolutions: nurturing knowledge and learning for urban sustainability through co-production experimentation. Curr. Opin. Environ. Sustain. 39, 9–16. doi: 10.1016/j.cosust.2019.05.008

CrossRef Full Text | Google Scholar

Cvitanovic, C., Howden, M., Colvin, R. M., Norström, A., and Mison, P. F. E. (2019). Maximising the benefits of participatory climate adaptation research by understanding and managing the associated challenges and risks. Environ. Sci. Policy 94, 20–31. doi: 10.1016/j.envsci.2018.12.028

CrossRef Full Text | Google Scholar

Daniels, E., Bharwani, S., Gerger Swartling, Å., Vulturius, G., and Brandon, K. (2020). Refocusing the Clim. Serv. lens: Introducing a framework for co-designing “transdisciplinary knowledge integration processes” to build climate resilience. Clim. Serv. 19, 1–15. doi: 10.1016/j.cliser.2020.100181

CrossRef Full Text | Google Scholar

Englund, M., André, K., Gerger Swartling, Å., and Iao-Jörgensen, J. (2022). Four Methodological Guidelines to Evaluate the Research Impact of Co-produced Climate Service. Front. Climate 119, 909422. doi: 10.3389/fclim.2022.909422

CrossRef Full Text | Google Scholar

European Commission (2021). Forging a climate-resilient Europe - the new EU Strategy on Adaptation to Climate Change. COM(2021) 82 final. Brussels: European Commission. Available online at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2021:82:FIN (accessed May 26, 2023).

Google Scholar

European Commission Directorate-General for Research Innovation (2009). Challenging futures of science in society : emerging trends and cutting-edge issues : report of the MASIS expert group setup by the European Commission. Publications Office. Available online at: https://data.europa.eu/doi/10.2777/467 (accessed May 26, 2023).

Google Scholar

Farr, M. (2018). Power dynamics and collaborative mechanisms in co-production and co-design processes. Crit. Soc. Policy 38, 623–644. doi: 10.1177/0261018317747444

CrossRef Full Text | Google Scholar

Fazey, I., Bunse, L., Msika, J., Pinke, M., Preedy, K., Evely, A. C., et al. (2014). Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Global Environ. Change 25, 204–220. doi: 10.1016/j.gloenvcha.2013.12.012

CrossRef Full Text | Google Scholar

Fritzsche, K., Schneiderbauer, S., Bubeck, p, Kienberger, S., Buth, M., Zebisch, M., et al. (2014). The Vulnerability Sourcebook. Concept and guidelines for standardized vulnerability assessments. Germany: Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbHGIZ.

Google Scholar

Gerger Swartling, Å., Tenggren, S., André, K., and Olsson, O. (2019). Joint knowledge production for improved Clim. Serv.: Insights from the Swedish forestry sector. Environ. Policy Gover. 29, 97–106. doi: 10.1002/eet.1833

CrossRef Full Text | Google Scholar

Grainger, S., Murphy, C., and Vicente-Serrano, S. M. (2021). Barriers and opportunities for actionable knowledge production in drought risk management: embracing the frontiers of co-production. Front. Environ. Sci. 9, 602128. doi: 10.3389/fenvs.2021.602128

CrossRef Full Text | Google Scholar

Haque, A. N., Grafakos, S., and Huijsman, M. (2012). Participatory integrated assessment of flood protection measures for climate adaptation in Dhaka. Environ. Urbaniz. 24, 197–213. doi: 10.1177/0956247811433538

CrossRef Full Text | Google Scholar

Harvey, B., Cochrane, L., and Epp, M. V. (2019). Charting knowledge co-production pathways in climate and development. Environ. Policy Gover. 29, 107–117. doi: 10.1002/eet.1834

CrossRef Full Text | Google Scholar

IPCC (2014). “Climate Change 2014: Synthesis Report,” in Contribution of Working Groups I, II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, eds. R.K. Pachauri and L.A. Meyer (Geneva, Switzerland: IPCC).

Google Scholar

Jönsson, A. M., and Gerger Swartling, Å. (2014). Reflections on science–stakeholder interactions in climate change adaptation research within swedish forestry. Soc. Nat. Resour. 27, 1130–1144. doi: 10.1080/08941920.2014.906013

CrossRef Full Text | Google Scholar

Klein, J. T., Grossenbacher-Mansuy, W., Häberli, R., Bill, A., Scholz, R. W., and Welti, M. (2001). Transdisciplinarity: Joint Problem Solving Among Science, Technology, and Society: An Effective Way for Managing Complexity. Berlin: Springer Science and Business Media. doi: 10.1007/978-3-0348-8419-8

CrossRef Full Text | Google Scholar

Klein, R. J. T., and Juhola, S. (2014). A framework for Nordic actor-oriented climate adaptation research. Environ. Sci. Policy 40, 101–115. doi: 10.1016/j.envsci.2014.01.011

CrossRef Full Text | Google Scholar

Lang, D. J., Wiek, A., Bergmann, M., Stauffacher, M., Martens, P., Moll, P., et al. (2012). Transdisciplinary research in sustainability science: practice, principles, and challenges. Sustain. Sci. 7, 25. doi: 10.1007/s11625-011-0149-x

CrossRef Full Text | Google Scholar

Leander, E., André, K., Dale, B., Hanssen Korsbrekke, M., Menk, L., and Gerger Swartling, Å. (2020). “User-interface and stakeholder involvement,” in Methods for Climate Change risk assessments: An international knowledge review, ed. C. Aall (Sogndal: Western Norway Research Instititute (WNRI)).

Google Scholar

Lemos, M. C., Arnott, J. C., Ardoin, N. M., Baja, K., Bednarek, A. T., Dewulf, A., et al. (2018). To co-produce or not to co-produce. Nat. Sustain. 1, 722–724. doi: 10.1038/s41893-018-0191-0

CrossRef Full Text | Google Scholar

Lemos, M. C., and Morehouse, B. J. (2005). The co-production of science and policy in integrated climate assessments. Global Environ. Change A. 15, 57–68. doi: 10.1016/j.gloenvcha.2004.09.004

CrossRef Full Text | Google Scholar

Máñez, M., Zölch, T., and Cortekar, J. (2014). Mapping of Climate Service Providers - Theoretical Foundations and Empirical Results: A German Case Study Report 15 - Climate Service Center Germany. Available online at: https://www.climate-service-center.de/about/news_and_events/news/063447/index.php.en (accessed September 14, 2022).

Google Scholar

Menk, L., Terzi, S., Zebisch, M., Rome, E., Lückerath, D., Milde, K., et al. (2022). Climate change impact chains: a review of applications, challenges, and opportunities for climate risk and vulnerability assessments. Weath. Clim. Soc. 14, 619–636. doi: 10.1175/WCAS-D-21-0014.1

CrossRef Full Text | Google Scholar

Norström, A. V., Cvitanovic, C., Löf, M. F., West, S., Wyborn, C., Balvanera, P., et al. (2020). Principles for knowledge co-production in sustainability research. Nat. Sustain. 3, 182–190. doi: 10.1038/s41893-019-0448-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Nyadzi, E., Nyamekye, A. B., and Ludwig, F. (2022). “Making Clim. Serv. Actionable for Farmers in Ghana: The Value of Co-Production and Knowledge Integration,” in Indigenous Knowledge and Climate Governance: A Sub-Saharan African Perspective Sustainable Development Goals Series, eds. E. E. Ebhuoma and L. Leonard (Cham: Springer International Publishing) 97–110. doi: 10.1007/978-3-030-99411-2_8

CrossRef Full Text | Google Scholar

Page, R., and Dilling, L. (2019). The critical role of communities of practice and peer learning in scaling hydroclimatic information adoption. Weath. Clim. Soc. 11, 851–862. doi: 10.1175/WCAS-D-18-0130.1

CrossRef Full Text | Google Scholar

Palomo, I., Felipe-Lucia, M. R., Bennett, E. M., Martín-López, B., and Pascual, U. (2016). “Chapter six - disentangling the pathways and effects of ecosystem service co-production,” in Advances in Ecological Research Ecosystem Services: From Biodiversity to Society, Part 2., eds. G. Woodward and D. A. Bohan (London: Academic Press) 245–283. doi: 10.1016/bs.aecr.2015.09.003

CrossRef Full Text | Google Scholar

Palutikof, J. P., Leitch, A. M., Rissik, D., Boulter, S. L., Campbell, M. J., Perez Vidaurre, A. C., et al. (2019). Overcoming knowledge barriers to adaptation using a decision support framework. Clim. Change. 153, 607. doi: 10.1007/s10584-018-2177-3

CrossRef Full Text | Google Scholar

Parkinson, J. (2012). “Democratizing deliberative systems,” in Deliberative Systems: Deliberative Democracy at the Large Scale, eds. J. Parkinson and J. Mansbridge (Cambridge: Cambridge University Press). doi: 10.1017/CBO9781139178914

CrossRef Full Text | Google Scholar

Patton, M. Q. (2010). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York, NY: Guilford Press.

PubMed Abstract | Google Scholar

Petutschnig, L., Rome, E., Lückerath, D., Milde, K., Aall, C., Gerger Swartling, Å., et al. (2023). Research advancements for impact chain based climate risk and vulnerability assessments. Front. Clim. 5, 699. doi: 10.5281/zenodo.7634699

CrossRef Full Text | Google Scholar

Polk, M. (2015). Transdisciplinary co-production: Designing and testing a transdisciplinary research framework for societal problem solving. Futures 65, 110–122. doi: 10.1016/j.futures.2014.11.001

CrossRef Full Text | Google Scholar

Reed, M. S., Ferr,é, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M., et al. (2021). Evaluating impact from research: A methodological framework. Res. Policy 50, 1–17. doi: 10.1016/j.respol.2020.104147

CrossRef Full Text | Google Scholar

Reed, M. S., Vella, S., Challies, E., de Vente, J., Frewer, L., Hohenwallner-Ries, D., et al. (2018). A theory of participation: what makes stakeholder and public engagement in environmental management work? Restor. Ecol. 26, S7–S17. doi: 10.1111/rec.12541

CrossRef Full Text | Google Scholar

Rodela, R., and Gerger Swartling, Å. (2019). Environmental governance in an increasingly complex world: Reflections on transdisciplinary collaborations for knowledge coproduction and learning. Environ. Policy Gover. 29, 83–86. doi: 10.1002/eet.1842

CrossRef Full Text | Google Scholar

Stjelja, M. (2013). The Case Study Approach: Some Theoretical, Methodological and Applied Considerations. Available online at: https://apps.dtic.mil/sti/citations/ADA588465 (accessed November 9, 2022).

Google Scholar

Turnhout, E., Metze, T., Wyborn, C., Klenk, N., and Louder, E. (2020). The politics of co-production: participation, power, and transformation. Curr. Opin. Environ. Sustain. 42, 15–21. doi: 10.1016/j.cosust.2019.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

van Es, M., Guijt, I., and Vogel, I. (2015). Hivos ToC Guidelines: Theory of Change Thinking in Practice. A Stepwise Approach. The Hauge: Hivos. Available online at: https://hivos.org/document/hivos-theory-of-change/ (accessed February 16, 2021).

Google Scholar

van Tulder, R., and Keen, N. (2018). Capturing collaborative challenges: designing complexity-sensitive theories of change for cross-sector partnerships. J. Bus. Ethics. 150, 315–332. doi: 10.1007/s10551-018-3857-7

PubMed Abstract | CrossRef Full Text | Google Scholar

VanderMolen, K., Meadow, A. M., Horangic, A., and Wall, T. U. (2020). Typologizing stakeholder information use to better understand the impacts of collaborative climate science. Environ. Manage. 65, 178–189. doi: 10.1007/s00267-019-01237-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Vaughan, C., Dessai, S., and Hewitt, C. (2018). Surveying Climate service: what can we learn from a bird's-eye view? Weath. Clim. Soc. 10, 373–395. doi: 10.1175/WCAS-D-17-0030.1

CrossRef Full Text | Google Scholar

Vincent, K., Daly, M., Scannell, C., and Leathes, B. (2018). What can Climate Service learn from theory and practice of co-production? Clim. Serv. 12, 48–58. doi: 10.1016/j.cliser.2018.11.001

CrossRef Full Text | Google Scholar

Visman, E., Vincent, K., Steynor, A., Karani, I., and Mwangi, E. (2022). Defining metrics for monitoring and evaluating the impact of co-production in Climate Service. Clim. Serv. 26, 100297. doi: 10.1016/j.cliser.2022.100297

CrossRef Full Text | Google Scholar

Wall, T. U., Meadow, A. M., and Horganic, A. (2017). Developing Evaluation Indicators to Improve the Process of Coproducing Usable Climate Science. Wea. Clim. Soc. 9, 95–107. doi: 10.1175/WCAS-D-16-0008.1

CrossRef Full Text | Google Scholar

Wibeck, V., Eliasson, K., and Neset, T.-S. (2022). Co-creation research for transformative times: Facilitating foresight capacity in view of global sustainability challenges. Environ. Sci. Policy 128, 290–298. doi: 10.1016/j.envsci.2021.11.023

CrossRef Full Text | Google Scholar

Wiek, A., Talwar, S., O'Shea, M., and Robinson, J. (2014). Toward a methodological scheme for capturing societal effects of participatory sustainability research. Res. Eval. 23, 117–132. doi: 10.1093/reseval/rvt031

CrossRef Full Text | Google Scholar

Yin, R. (2009). The SAGE Handbook of Applied Social Research Methods. Thousand Oaks: SAGE Publications, Inc.).

Google Scholar

Zebisch, M., Schneiderbauer, S., Fritzsche, K., Bubeck, P., Kienberger, S., Kahlenborn, W., et al. (2021). The vulnerability sourcebook and climate impact chains – a standardised framework for a climate vulnerability and risk assessment. Int. J. Clim. Change Strat. Manage. 13, 35–59. doi: 10.1108/IJCCSM-07-2019-0042

CrossRef Full Text | Google Scholar

Zebisch, M., Schneiderbauer, S., Renner, K., Below, T., Brossman, M., Ederer, W., et al. (2017). Risk Supplement to the Vulnerability Sourcebook. Guidance on how to apply the Vulnerability Sourcebook's approach with the new IPCC AR5 concept of climate risk. Bonn, Germany: Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ).

Google Scholar