Prospects and challenges of regional modeling frameworks to inform planning for food, energy, and water systems: Views of modelers and stakeholders

1 Introduction

A growing body of scholarship and practice highlights the deep couplings and complexities of food, energy, and water systems (FEWS) at multiple scales (Keairns et al., 2016; Berardy and Chester, 2017; D'Odorico et al., 2018; National Academies of Sciences, Engineering, and Medicine [NAS], 2021; Viglia et al., 2022). Primary examples are demonstrated by demands on water resources for agricultural irrigation, energy generation, ecological systems, and residential and industrial consumption. Complexity across systems is amplified by climate change, increased throughput, technological change, and globalization. The impacts of increasing demand, stronger couplings, and a changing climate are amplifying tensions among FEWS, raising questions about how they are linked as well as how to deploy engineered infrastructure (e.g., dams, irrigation, water treatment plants) and nature-based infrastructure (e.g., land, aquatic systems, ecosystems) to manage them and improve system resilience and sustainability (Miara et al., 2017).

The ways that systems are coupled and the ways that trade-offs arise in deployment of policy actions to manage them in the face of climate change have been the growing focus of modeling (Daher and Mohtar, 2015; Keairns et al., 2016; Kling et al., 2017; Haimes, 2018; Nie et al., 2019; National Academies of Sciences, Engineering, and Medicine [NAS], 2021). Kling et al. (2017), pg. 151 argue that

“Existing models tend to individually examine strategies to address environmental problems. However, FEW systems often generate multiple environmental impacts, some of which occur as complements, such that addressing one leads to co-benefits by reducing others … Developing models that incorporate these multiple impacts can lead to more holistic approaches … ”

Various tools and frameworks have been applied to this topic, including life cycle analysis (Mannan et al., 2018) and linked biophysical and socioeconomic models (Howells et al., 2013; van Vuuren et al., 2015; Kling et al., 2017; Miara et al., 2017; Vörösmarty et al., 2023) to help scientists and policymakers analyze coupled systems, reveal trade-offs and propagation of impacts, and explore future dynamics. Such tools have been implemented in a variety of contexts and at multiple scales. These range from localized, to regional, to national scales and they explore scenarios defined by climate stressors, new policies, and new technologies.

Modelers can approach the development of frameworks to model FEWS in a variety of ways. One approach is through participatory processes that result in new models (González-Rosell et al., 2020; Schmitt-Olabisi et al., 2020). Another is for a team of modelers to integrate existing validated models related to food, energy, and water, and other systems into a single model (Welsh et al., 2013; US DOE, 2020; Thompson et al., 2021). These are generally referred to as integrated assessment models, where the models are tightly coupled (Weyant, 2017; Kling et al., 2017). A third approach is to “soft-link” existing models, such that they are connected via a semi-coupled confederation of individual models, or what some call a modeling framework (MF) (Howells et al., 2013).

In this paper we explore opportunities and challenges of coupling existing models into a MF with the goal of assisting planners and decision makers to consider linkages among FEWS and the implications of management strategies to promote resilience in the face of a changing climate. Our focus is on frameworks that link existing models to provide information at the regional scale of watersheds or multiple states (e.g., New England). These encompass both strongly coupled models (i.e., integrated assessment models) and soft-linked models, or modeling frameworks (MFs). The development and application of MFs for FEWS give rise to many challenges (Webler et al., 2011; Kling et al., 2017; Fisher-Vanden and Weyant, 2020). These include challenges to validating coupled models, providing meaningful information to decision makers, the “inertia” of existing models whose features may be difficult to modify, built-in assumptions and input and output parameters that may not align well with the interests of policymakers, and ensuring appropriate “hand-offs” of outputs from one model as inputs to another model. In addition, scientists and stakeholders may have different ways of conceptualizing FEWS (Villamor et al., 2020). Roles of stakeholders in developing modeling frameworks and scenarios can vary and present their own set of challenges (McBridge et al., 2019; Villamor et al., 2022).

The context for our study is the C-FEWS project (Climate-induced extremes on the linked food, energy, water system), in which a framework is being developed to enable a systematic assessment of future policy options to manage and adapt FEWS to changing climate extremes and other environmental stressors from the present-day to 2100 (Vörösmarty et al., 2023). The C-FEWS framework is based on a semi-coupled confederation of existing models for climate, energy, food, and water systems. The C-FEWS project is representative of MFs.

We interviewed ten modelers associated with the C-FEWS project and ten regional policymakers, NGO representatives, and researchers (henceforth referred to as stakeholders) to gather their perspectives and insights on the roles of MFs of FEWS, the challenges of creating MFs to inform planning, and the information needed to support planning in integrated FEWS. In addition, we analyzed comments of participants in a meeting of the project’s Stakeholder Advisory Group and the discussions among modelers in project meetings.

2 Materials and methods

We gathered information from modelers and regional stakeholders in four ways. First, we conducted semi-structured interviews with ten modelers associated with the C-FEWS project. The interviewer used a series of questions to guide the interviews, but there was room to explore topics as they arose. Interviews typically lasted one hour and were all conducted via the telephone or zoom. After acquiring voluntary informed consent, all interviews were recorded and transcribed. We inquired of modelers’ opinions and experiences about:

• How specific models and their outputs have been used to inform planners and decision-makers,

• The caveats and embedded assumptions and uncertainties they think are most important for stakeholders to know,

• The primary challenges to developing regional FEWS modeling frameworks,

• What they believe stakeholders in regional sectors want to learn from regional FEWS modeling frameworks, and

• What they hope to learn from engaging with the stakeholders as part of the project.

Second, we interviewed ten stakeholders from New England who engage in regional planning across food, energy, and water systems. Potential interviewees were identified via web searches and key informants. They included staff from non-governmental organizations, researchers participating in regional planning, and staff from government agencies. We told them that we wanted to learn how models can aid in regional policy and decision making across multiple sectors such as food, energy, water, housing, transportation, and habitat management. For the sake of brevity, we focused this study only on the New England region. As with modelers, after acquiring informed consent, we recorded and transcribed interviews. We inquired of stakeholder’s beliefs, opinions, and experiences about:

• The information that has been or would be most helpful in their regional work across multiple sectors,

• What they think makes regional modeling frameworks helpful to their regional planning work, including opportunities to participate in their development, information produced, and methods of communicating results, and

• Key questions at the regional scale that FEWS modeling frameworks could help answer.

Co-authors independently read and coded interview transcripts (RH, ST and TW) using the qualitative data analysis technique of grounded theory (Glaser and Strauss, 1967; Webler et al., 2011; Corbin and Strauss, 2014). In this approach, segments of text that relate to a theme or idea are identified. A segment of coded text on a given theme is contrasted and compared with other coded segments to find commonalities and differences. We began by coding with the research questions in mind. We included additional topics that emerged in the interviews.

Third, we reviewed comments of 17 participants in a meeting, in November 2020, of a Stakeholder Working Group created as part of the C-FEWS project. The Stakeholder Working Group is intended to engage people from diverse sectors and organizations to co-design and then explore scenarios of how the food-energy-water system responds to climate extremes, such as drought, extreme precipitation events, and increasing temperatures, and how natural and engineered infrastructure can be employed as policy “levers” to minimize environmental and societal damage. During the meeting Stakeholder Working Group members provided initial insights into the kinds of scenarios that would be of interest to explore with the suite of models in response to two questions.

• What are your major concerns regarding the state and trajectories of FEWS across the Midwest and Northeast regions?

• What are policy and management levers you anticipate could be used to realize alternative outcomes?

Finally, we reviewed and coded recordings of a sample of project meetings where processes of linking models and the harmonization of model features such as parameter definitions, time steps, and grid resolutions were discussed and resolved. During January 2020 through September 2021 the project team met 22 times on zoom, which ranged from 2h to a full day (the project team has consisted of 15–20 faculty and graduate students). We coded 15 of those meetings. Similar to the process of coding interview transcripts, zoom meeting recordings were coded using a grounded theory qualitative data analysis approach.

3 Results

In the following sections we present the results of our interviews with modelers and stakeholders. We present them together in four sections, based on themes that emerged from our analysis:

• the benefits of regional FEWS MFs.

• what information can be gained from regional FEWS MFs

• challenges of linking pre-existing models, and

• suggestions for designing useful modeling frameworks.

3.1 The benefits of regional FEWS MFs

We inquired of the modelers and stakeholders about what they think are the benefits to stakeholders from MFs. Both modelers and stakeholders highlighted opportunities for learning and informing stakeholders’ thinking. The use of scenarios to explore possible futures is a key way to support learning. Both modelers and stakeholders believe MFs are useful to inform policies, but they should not determine policies.

3.1.1 FEWS MFs can help stakeholders make better informed choices about courses of action

Choices can be better informed when consequences and trade-offs of potential actions are understood. A stakeholder we interviewed talked about the role that models can play in helping to understand the implications of particular policy choices. She spoke of an effort to explore the implications of promoting viable and secure local food systems. By exploring different scenarios, MFs could help stakeholders understand the implications of policies for which they want to advocate. However, stakeholders may also find MF outputs do not confirm their beliefs, which can make them less interested in their use because of the questions that they may evoke among other stakeholders, decisions makers, or the public. One stakeholder we interviewed put it this way:

“The modeling could help [us understand the system response better] but could also lead to a lot of unpleasant questions and concerns.”

The modelers agreed with the point made by the stakeholders that MFs can help stakeholders better understand consequences of potential actions. Modelers highlighted the power of stakeholders to use MFs to explore future consequences of actions, such as new regulations, management approaches, or new technologies. As one modeler explained, MFs can help people

“Translate where we are now with land cover distribution to particular issues that people are wanting to find out for some time in the future.”

An aspect of exploring future consequences is the consideration of how actions in one sector or region may impact other sectors or regions. The idea of exploring trade-offs across food, energy, and water systems is central to the goal of the C-FEWS framework and members of the modeling team suggested that MFs could help stakeholders understand these tradeoffs across systems.

“[The MF could] look at an improvement [in the agriculture sector] to get irrigation but it’s not an improvement to have irrigation [if there is] a dry spell. Somebody will be pumping water out of the river to feed the corn, and they are not feeding downstream the power station that is producing hydro [energy] or needs cooling water. One sector's benefiting so its reliability goes up or its risks go down but downstream users’ risk goes up and their reliability goes down in their systems … take a look at your sector, realize that you’re not the only sector and then [through the modeling] unveil the set of pinch points or tradeoffs.”

Another modeler described it this way:

“I argued that if they rolled out a carpet and understood what’s going on across the carpet, they would get a better indication of where there were opportunities to do sustainable development versus non-sustainable development [….] Yeah, all the politics are local, however infrastructure build [creates] opportunities for planning things out at the regional scale, in fact that’s what we would like to do, right? Maybe we can design an experiment to show that if you think locally, you might optimize but you lose the regionality. That would be a very good scenario for the group to consider.”

Stakeholders also spoke about the ways that MFs can help improve policies. For example, they suggested using models in different ways, such as forecasting and backcasting to explore how desired futures can be achieved as well as helping to understand system dynamics. For instance, one stakeholder interviewee mentioned that she wants to understand the implications of a decarbonized electricity grid, which is a question of forecasting. Another stakeholder was interested in backcasting to explore how to achieve a desired outcome:

“I do think that sitting down and sketching out in 50 years, “this is what an ideal world would look like,” would be great. And then backtracking to this point.”

Participants in the Stakeholder Working Group meeting emphasized how these approaches can be used to better understand the impacts of a changing climate and the role policy levers, new technologies, and management systems play in shaping outcomes. They also talked about how MFs could make trade-offs among policy goals more transparent.

Stakeholders who are active in state or national politics and policy making sometimes explained that they used models to help understand the opportunities and challenges of a future system state. In other words, they are more interested in the so-called “30,000 foot view” and not “fine-tuned modeling.” What they want from models are general trends over longer periods of time or models that help them understand what a significantly altered system is like.

3.1.2 FEWS MFs can help stakeholders explore scenarios to enhance understandings

Modelers described the important role of examining a range of scenarios to help stakeholders understand the dynamics of coupled food, energy, and water systems and the implications of different management strategies. As one modeler put it:

“You do want to consider […] multiple scenarios, because it really depends on what humans do in the next century […] and how much temperature change and climate change we’re going to have. So we look at a range of scenarios.”

Given that scenarios are important, we inquired about the role of stakeholders in defining those scenarios. A modeler made a distinction between stakeholders helping to develop the MF versus helping to define scenarios that can be explored by the framework:

“I think their major role is, is to work with us, to codesign the scenarios, the storyline. What do you guys want in terms of answers to your questions? There’s no time for us to sit down and codesign the computer code to do that so you'll have to trust us that we have some competency there but what are you, what's your worry 30 years from now? Are you worried about heat waves? Are you worried about fuel mix? Are you worried about cost per kilowatt? What's your concern as we go into the future? I think the codesign of the scenarios is where we engage them, not at the level of the model building.”

Another modeler felt that scenarios defined by a modeling team could help stakeholders understand the capabilities and limitations of a MF. In his words scenarios could “demonstrate some results to give a taste for what we could do.” The modeler went on to say that it can also be beneficial to present scenarios to stakeholders to provoke discussion:

“There’s some hand holding that has to be done. We can say, ‘We looked at this scenario versus this scenario and we revealed this. Have you thought about this in your operations or in your planning or in your management strategies?’ That’s pretty targeted because you have to know ahead of time what it is they might be interested in and make sure the results are sound enough that they’re not gonna say well this does not make sense because you did not do x, y, and z.”

A stakeholder made a similar point but emphasized that achieving better understandings of possible future scenarios and the capabilities and limitations of a MF depends on transparency about goals, intentions, and assumptions.

3.2 What information can be gained from regional FEWS modeling frameworks

Stakeholders emphasized that useful MFs should provide information that has value to them. For example, if a model reports information on projected corn production, but the stakeholder is interested in alfalfa production, there is obviously a mismatch.

The stakeholders we interviewed and participants in the Stakeholder Working Group meeting emphasized that they wanted model outputs that relate directly to the decisions and policies they make or that they are trying to influence. They highlighted their desire for information about spatial and temporal factors and about trade-offs relevant to their decision making and planning. For instance, one stakeholder mentioned that her organization wants to understand how the transition to clean electricity will improve air quality and reduce greenhouse gas emissions. It is understood that there will also be downsides to policy change. In this case, jobs, grid reliability, or electricity prices may change. Information about such impacts was important to this stakeholder.

3.2.1 Information about economic costs

Cost of a policy option was a variable of widespread interest among stakeholders and the regulatory and political officials with whom they need to collaborate. This stakeholder emphasized that regulators at the state are hyper-focused on cost:

“[…] the mentality of regulators tends to be how much is it going to cost people today? And how do we do it the cheapest way?”

Stakeholders who communicate directly to publics noted the importance of being able to tell people what a given policy action would cost:

“The problem is that a lot of these federal or even regional policies—at least as advocacy is concerned—we're really interested in being able to tell people, how is this going to affect them personally, and so on, being able to translate that to, this is going to cost you X number of dollars.”

3.2.2 Information about distributional equity

Some of the data that stakeholders seek are about justice and distributional equity. One stakeholder pointed out how past policy actions have sometimes made injustices worse:

“There are a lot of people that have been historically oppressed by environmental actions. So I think that we need to really consider that as we move forward. So yeah, looking at the diversity, equity and inclusion piece of it is also important.”

Another stakeholder focused on land justice - how much and which parcels of agricultural land are owned by people of different races:

“When you deal with climate change, land is a critical entity, and ownership and access to decision making on land is disproportionately, you know, like, the more wealthy white people. So that’s a big issue.”

3.2.3 Information about uncertainty

A thread through many discussions about the value of information for stakeholders was about the value of reporting uncertainty information. Modelers had different opinions about whether stakeholders wanted this information. For example, a modeler said:

“I think if we got to the point where ‘Hey this result is interesting and it could be used to inform planning’, but what is the uncertainty around it? I think uncertainties are very important. We had some ways to address it in terms of standard methods that are in the literature, but I think that capturing the effect of uncertainty on an outcome in a model is getting a lot more attention and has a lot more value and I think planners are becoming more and more aware of that too.”

Other modelers reported that, in their experience, many stakeholders are uninterested in uncertainty information:

“We present [information about uncertainty] but my impression was that they were not particularly keen about that, they were not demanding that as I recall, they were just demanding well “what”s the loss of capacity’ and then we say ‘oh it’s about 12% give or take’ but they were not demanding that and it's really funny to me that among the modelers we’re super concerned about uncertainty … I think we’re more concerned about that than [stakeholders] we’ve worked with … ”

“What we think is important may not be so important [to them]. They just want the number.”

In contrast to the view of the modeler, all the stakeholders we talked to expressed a desire for MFs that can characterize uncertainties related to food, energy, and water systems (production, distribution, supply, etc.) as a result of climate change and the implications of uncertainties for decision making. They emphasized the importance of disclosing uncertainties in conjunction with outputs because uncertainties can affect planning about, for example, water management and water storage capacity.

For some stakeholders, uncertainty is about not knowing what people would do. For example, when we asked one stakeholder about how important it was for him to know about uncertainty, he mentioned the uncertainty of knowing whether urban migration to rural areas would change. Another mentioned uncertainty about Federal Government programs. Uncertainty in this sense is more akin to scenario design than it is to data stochasticity. In models, many variables are not point estimates, but are probabilistic.

3.2.4 Information about relevant regions, scales, and sectors

Modelers recognized that the value of models lay in providing information that is relevant to decision makers, and that means information that is at relevant spatial and temporal scales:

“We quickly learned that we need to break down the science by what people, again really want to know for their particular interests. So, if you're from the Midwest, you want to know what’s happening in the Midwest, you want to know what’s likely to happen in the Midwest, you probably care a little less about what’s going on in the rest of the country. You know maybe you have some interest in, maybe you have relatives in California so you’re interested in the West or the Deep South or something, but you’re primarily gonna be interested in the Midwest. Let’s say you work on energy or transportation or water issues or you’re a farmer, … they’re gonna be interested in how we look at particular sectors of society.”

Stakeholders in the working group meeting and in our interviews also emphasized the value of outputs at relevant spatial and temporal scales. While modelers may struggle to downscale from 100 to 4 km², users sometimes wanted model output on scales even finer than this. For instance, some local planners wanted models to provide useful information about individual properties. FEMA floodplain maps were brought up in one interview as an example of the granularity that some stakeholders need. But even as FEMA’s maps provide sufficient spatial precision, they do not provide the temporal granularity that some stakeholders sought (even as they understood this was not the task Congress allocated to FEMA). FEMA’s maps do not predict future flood risks; they are based entirely on historical rainfall and storm surge data. Thus, they lack the temporal dimensions of interest to some stakeholders. For flood risk, some stakeholders sought spatial granularity and high-confidence future projections.

The desire for information at fine scales is in part driven by the question of where decisions are being made that affect food, energy, and water systems. This point was further emphasized by interviewees that advocated for MFs to provide information at the scale of states, because it is at the state level that many policies and regulations are proposed and enacted. While describing a New England regional food system planning initiative one interviewee noted that,

“The work is at the state level with all of the different state actors. How are we going to contribute to this? To which we can say here’s what Vermont's contribution should be because we have a lot of farms, we have more farmland, here’s what they need to do … [the work] is going to need to happen more at the state level and even the local level, as opposed to on a regional focus … it’s got to happen on the state level, because you're dealing with every state has some kind of a Department of Agriculture, and they have grant funds, and they have access to them, they have different regulations, and all that kind of stuff.”

3.2.5 Information about thresholds and inflection points

Stakeholders reported an interest in understanding sensitivities, thresholds, and inflection points. An interviewee explained that causal relations might be linear, but only up to a point. After that they may transition quickly to a different slope or even become exponential. For example, it is often noted that transitioning from fossil fuels to renewables is quite feasible up until 85% of demand is met (Denholm et al., 2022). After that inflection point, gains become much more difficult to achieve. If models could help stakeholders locate potential thresholds and inflection points, the policy expectations could be better managed. For similar reasons, some stakeholders desired information about the aggregation of many small scale (individual) decisions. During the Stakeholder Working Group meeting, participants asked how MFs can help make sense of the cumulative impacts of many small-scale decisions and actions for FEWS.

Modelers agreed. Modelers thought that MFs offer an opportunity for stakeholders to learn about thresholds of change. For example, a modeler described a situation related to extreme heat:

“One of the things we often look at is what is the number of days above 95 degrees. Well why? Well, being a Midwesterner, I know that corn seed will not develop if the corn is developing during a period when it’s above 95 degrees and so are 95 degrees days becoming more common? It’s not that common in Illinois right now for example, but by mid-century we could see half a month to a month of 95 degrees and if that happens to be in July when the corn is setting, that will affect production so we know that’s a kind of threshold, that will be of interest. So, we’re always looking for what is the new sense of a threshold that matters.”

3.3 Challenges of linking pre-existing models

The focus of the C-FEWS project is the development of a regional FEWS MF built from a suite of “soft linked” existing biophysical models (Vörösmarty et al., 2023). The core models are connected through the exchange of data inputs and outputs, an approach used in other contexts (Howells et al., 2013). Interviews with the C-FEWS project team members yielded insights into four key challenges of that arise from linking existing models into a FEWS MF. While some of them can related to modeling of complex systems generally, they are exacerbated by linking existing models into a modeling framework.

3.3.1 Complexity of coupling individual models

The effort to link existing water, food, energy, nutrient flux, and climate models takes considerable effort. There are challenges to matching time steps, grid sizes, data resolution, spatial regions, and metrics for input and output variables. The effort required - in time and labor - to address these differences can be significant. The suite of models managed within the C-FEWS modeling framework define parameters and embed assumptions—as all models do—but in ways that are sometimes inconsistent and potentially incompatible to the other models and not well understood among the modelers:

“[Other modelers] are making very coarse statements about the models that were totally wrong. They were not understanding the data set we gave them.”

For example, modelers have to work through key differences in critical definitions such as how to define and measure carbon sequestration or even what defines the Northeast and Midwest regions:

“[Name of modeler] has a different idea of what the Midwest and Northeast is than what we do. We use the national climate assessment states, they’re using something broader because they are worried about the watersheds. We have to communicate better.”

Often, these differences were rooted in disciplinary traditions, as this statement conveys:

“The economists tend to talk about crops, they are really talking about dollars, but they relate that to bushels. And bushels are a kind of a weird unit for us because we are looking at grams of carbon, which you can convert to grams dry weight but bushels are not dry weight necessarily because you’ve got water in there.”

Reconciling differences is obviously vital, and the project team achieves this through regular meetings. However, discovering differences can take time and it is not always easy to find where misunderstandings lie. As one modeler told us:

“We have all these data harmonization issues and these technical wrestling things to the ground, staging them, making sure everyone understands the formats, the different input structures for the different models. [….] the models were developed for different purposes, the models have different time steps, different time horizons, SPARROW is a steady state model and now we’re making it into a time series model so all of these, the minutiae of getting the models set up, running them, harmonizing the data, harmonizing the outputs, all that stuff takes an enormous amount of effort.”

3.3.2 Cascading parameter changes

The effort involved in harmonizing independent component models is exacerbated if a component model is updated or modified. While a change may seem an incremental improvement, it can lead to cascading changes in parameter values in the linked models. This is particularly problematic when the linked models are particularly sensitive to the change and when those changes are unexpected or difficult to see. A modeler spoke to this issue:

“I know from my past experience that anytime a model changes, you think, “Oh it”s a minor change.’ But it has an impact. That’s just the nature of numerics and it’s not necessarily the science”

3.3.3 Long run times and large sets of output data

Component biophysical models of FEWS and models of climate systems and suites of coupled models made up of a set of complex models can have very long run times and very large storage requirements:

“The complicated models, as you know, take months to set this stuff up and to run it god knows how long, and then you have to check, and then if something is wrong you have to re-run it.”

Such lengthy run times compromise the utility of the model to some stakeholders, especially as they need to be tested and validated.

In addition, they often require significant computer storage. One modeler suggested that the fundamental limit to MFs is not the data or knowledge of the system, but the computing power to run these extremely complex models:

“There is not enough computer storage in the world to deal with everything we can produce so we’re always struggling with that ourselves. What is the minimum we can get away with, because then we can do more runs if we do not have to store as much. And yet for their model runs they need a certain amount of information.”

The last quote hints at another issue: when results are generated, the volume of data produced can be overwhelming if modelers and stakeholders do not work closely together to determine the value of information being generated. A modeler reflected on his experience, saying that stakeholders:

“… would like to have it on a finer time scale. They would like to have it almost as fine as you can give it to them but then when you dump hourly data or even daily data on them, that's too much, it’s a lot of information.”

3.3.4 A multiplicity of output options

The question of how MFs can be useful for regional stakeholders is tied up with assumptions that modelers and stakeholders make about appropriate purposes of MFs and, as the previous sections demonstrate, preferences for information in particular contexts.

Stakeholders may desire information about questions that MFs are not capable of providing, and this is especially relevant in the context of linking existing models for which there is constrained flexibility to configure models to answer stakeholders’ questions. For example, during the Stakeholder Working Group meeting, one person wanted to know more about the impacts of urban flooding on local transportation and energy systems. Others wanted to learn about adoption and diffusion rates for new technologies and the implications of population and demographic shifts in the Northeast. Interviews with stakeholders revealed a strong interest in the distributional impacts of policy and technology changes (i.e., equity). These are issues that the models cannot shed detailed light on.

A modeler put the challenge this way:

“When we found out that one of the stakeholders was interested in street flooding, well immediately we have to say that we do not have that capability. But would you like to talk about the frequency of flooding events, how often they happen? Is there more extremity in the future that we’ll see compared to today? Something like that we could talk about. There are certain things that are off the table. We could have designed a high-resolution flood model, or set ourselves up to accommodate an existing algorithm, like the US Army Corps of Engineers’ HEC-RAS model. We could have done that, but it would not be in the spirit of what we're trying to do regionally.”

A more general issue raised by modelers we interviewed relates to transferability of models from one area to another:

“The crop models are notorious for being perfect at a particular site, but then once you move them to a different site, same crop, if you go from the US to Europe for example, they do not do well.”

Modelers’ perceptions of what is interesting or useful are based on what they think stakeholders want to know, although some modelers recognized that what might be preferred actions to modelers may not be what stakeholders prefer:

“We could come up with a […] very practical solution. Say, go use the land resources more than the water resources. Get off the once through circulating power stations and get into the land use question with solar and wind and at the end of the day, that’s probably a better outcome at least in a theoretical sense, right? But if you’re working with a stakeholder, you can actually take what would be satisfying theoretically to us as scientists and you could practically say, ‘Hey you better think about this, maybe use the land a little bit more sensibly or [what if] you use the land instead of the water?’ That is a message you can directly transfer into the world of the stakeholder.”

At the same time, some stakeholders were open to considering information from models they do not customarily use, as was summarized in this imaginary dialogue one interviewee shared:

Modeler: “What kind of information are you interested in?”

Stakeholder: “What kind of information can you give me?”

While such a conversation may seem promising, modelers emphasized to us that this was not an easy question to answer, as models may incorporate thousands of variables.

3.4 Suggestions for designing useful modeling frameworks

During the course of our interviews, modelers and stakeholders expressed four ideas for how to make MFs more useful to stakeholders. While the suggestions apply to MFs generally, they raise challenging questions for MFs based on linked existing models.

3.4.1 Simplify to help stakeholders make sense of outputs

Modelers realized that stakeholders do not want to learn and understand all the details of the science behind the models. They also do not want to be presented with a dozen or more output variables that move in different directions. While they may elect to learn more detail, at first, all stakeholders want are summary statistics, according to the modelers:

“I learned early on that it’s useful to develop special metrics that might be useful to people that want to understand the science but do not have the background to fully understand the science. That, in a sense, simplify the science for them”

Cost is assumed to be an obvious summary measure of interest to many stakeholders, although stakeholders expressed interest in measures that cannot be reduced to dollar values, such as equity:

“I think you’ve got to boil it off into ways a policymaker would be interested. For better or for worse, because this makes the world goes round, you’ve got to look at it in terms of economic value … You know if you see one portfolio yielding $26.8 billion and the other yielding $19.4, you can boil off all that detail that [the models calculate] that would be completely irrelevant to the planners, but they see $19 vs. $26, that says something to them.”

To be useful, stakeholders also spoke of their desire to acquire an intuitive sense of how the model works such that they are not surprised at the results of running different scenarios. This is an issue that also speaks to simplification and relates to transparency and clear presentation of outputs. Several stakeholders expressed a need for transparency in models, noting that this quality becomes vital when policy decisions need to be justified to the public or to elected officials with decision making authority. Transparency does not mean that the mathematics behind the models have to be understood by everyone. Instead, it refers to honesty about the strengths and limitations of the model and–importantly–what assumptions are built into the models and what impact those assumptions have on the outputs.

However, it is possible to overly simplify. Interestingly, stakeholders rarely suggested a desire for simplified models or outputs, in contrast to the modelers’ expectations of what stakeholders want. Instead of highlighting a need for simplification stakeholders emphasized what makes information obtained from models useful or actionable.

3.4.2 Use multiple metrics of interest to intended stakeholders

While cost is viewed as relevant to most stakeholders, some modelers discussed risk or climate resilience as possible variables of interest:

“One of the things that could capture what the stakeholders might be interested in is […] risk or maybe resiliency so that they would see that you know their particular sectors gets more—okay here’s one thing so let’s say the food sector is getting hit by a lot of droughts so they invest in irrigation right so what does the irrigation do? It reduces the risk of having a catastrophic crop failure.”

Another example of a summary statistic assumed to be helpful for stakeholders is the direction and magnitude of change of values:

“What they are really interested in is the direction of change. Are things getting better or worse, that’s where the threshold kind of comes in as a secondary issue and so like you may be doing great till you reach a certain spot, and then things start dying off, so things are not so great.”

3.4.3 Make the model outputs readily accessible to users

Additionally, modelers and stakeholders recognized that the format of output data also matters with graphic outputs having strength as communicative tools. A modeler put it this way:

“You’re always looking for ways of how you simplify this science … graphics of such and such, you know temperature change, they can get that. They are not gonna understand how you got that, but they can understand temperature change or precipitation change or you know things that we simplify the science down to something that you can easily take a bite out of. Metrics are useful that way, certain types of graphics are useful that way, it’s always a matter of how do you translate things into what people can then really grasp because they do not have the scientific background you have.”

Modelers also emphasized the importance of data being made available and accessible in models.

“People want “just the facts,” you know? They want to be able to make decisions for themselves. And so this is where I think that modeling is actually useful, because we’re able to produce data, and often-times just enormous amounts of data, probably way more data than they would ever need. The real key to this is getting the data, the data that people actually need, and then presenting it in a way that’s nice and clear.”

3.4.4 Integrate dynamics created by human and organizational behaviors

Reviews of integrated assessment modeling have repeatedly pointed out the absence of sophisticated modeling of socio-economic factors and human and organizational behaviors, including adaptive responses, implementation dynamics, and feedbacks to the climate and FEWS (Weyant, 2017; Kling et al., 2017; Nielsen et al., 2020; National Academies of Sciences, Engineering, and Medicine [NAS], 2021). The absence of information about human and organizational behaviors is related to the challenge of answering the questions of interest to stakeholders. Stakeholders that we interviewed expressed an interest in understanding the cumulative impacts of many individual actors, such as private property owners of woodlots in New England and farmers in the Midwest.

A modeler, reflecting on the need for input about human and organizational responses, noted,

“We should think about how humans are changing agriculture practices. That’s the major point. Implementation of those practices in the model is not a challenge or issue, but discussing and thinking about how the humans are changing their behavior is the challenging part. That part we are not discussing. We are only discussing the outcome of the model if you change this or change that, but we are not actually thinking about how the humans are changing their behaviors.”

Some assumptions might seem quite trivial, but as this modeler pointed out, the entire credibility of model can rest on those assumptions having lasting validity:

“All bets are off if agriculture stops selling our products abroad because whatever, we’ve found we cannot produce enough for ourselves or vice-versa.”

4 Discussion

The C-FEWS project is representative of integrated MFs that couple existing climate, energy, water, food, nutrient, and chemical balance models to provide information about regional dynamics and explore how to mitigate impacts from climate extremes by interventions. Interventions can include a mix of engineered and natural infrastructures, emerging technologies, efficiency gains, and policy and regulatory instruments. A regional modelling framework is valuable because many dynamics of FEWS manifest at regional scales. Mitigation strategies, for example, can span multiple jurisdictional boundaries.

The modelers and stakeholders we interviewed agree that MFs should provide policy and decision makers with valuable and actionable knowledge. Some stakeholders we interviewed reported relying on many kinds of models in their work but had yet to be exposed to an MF. Still, they recognized the value of a MF that could operate across conventional policy and governance domains and they understood that the tight coupling of systems makes the response of FEWS to a stressor more complex.

The results of our study reveal much overlap in the beliefs of modelers and stakeholders about the benefits of regional FEWS MFs and the information that would be valuable and actionable. Differences were more a matter of emphasis or reflect a focus on particular contexts. Stakeholders we interviewed, for example, agreed with modelers that economic value can be a useful metric for comparing the impacts of different policy or technology interventions. However, stakeholders also expressed interest in measures of distributional impacts and equity. While some modelers acknowledged the importance of these issues, non-economic value trade-offs were either not a central focus in their efforts to create an MF or they adopted a utilitarian ethic that assumes important tradeoffs can be measured by monetary values. Similarly, some modelers felt that stakeholders do not want to know about uncertainties, whereas all the stakeholders we interviewed indicated it is very important for them to gain knowledge about uncertainties. Finally, modelers emphasized the need for simplification of modeling results, while stakeholders did not view simplification as important as obtaining actionable information and demonstrating credibility. The broader concerns of stakeholders we interviewed, in contrast to modelers, are consistent with prior observations about the limited scope of integrated assessment models. For example, Villamor et al. (2022), pg. 7 note that in integrated assessment models, “biophysical dimensions continue to take precedence; integration of social fields currently takes place primarily through the lens of economics, with less attention given to fields including law, policy, and stakeholder participation.”

4.1 Challenges to developing MFs to inform policy making

In developing the C-FEWS framework we found that several challenges arose and many can be traced back to the decision to employ a suite of existing models. A key challenge is the considerable effort for modelers to learn about each other’s models and assess the compatibility of assumptions, parameter definitions, and analytic approaches (e.g., statistical or mechanistic models). Reconciling inconsistencies and differences takes a great deal of work and coordination. This is also true for assessing the capabilities and validity of integrated or semi-coupled models, a challenge that received a great deal of attention from the modeling team. One of our interviewees described it this way:

“There’s lots of sumo wrestlers around that table right? All the modelers are sumo wrestlers the way I would look at it. It is like a dance, like we’re around that circle and here comes TEM [one model in the C-FEWS framework] and it’s got its spatial resolution, it’s got its temporal context and here comes TP2M [another model in the C-FEWS framework], it’s got another set of time steps or space resolution and it’s organized differently because it's a drainage basin model. Here we’re trying to get useful outputs from the sumo wrestler called TEM and the other sumo wrestler which is TP2M and then we get in the ring and we try to figure out what the time steps should be and what information we share…. On top of it, we’ve got the climate datasets which everyone is sharing and then we’re discovering as we probe the whole thing there’s issues with the way the models run, there's continuity, missing values of things, funny crazy step functions, you know all this stuff that it needs to fit into all the other pieces of the project and as you saw, it takes an enormous amount of effort to get [the couplings and connections] right.”

Another set of challenges revolve around the capabilities of MFs to provide the information that stakeholders desire. First, modelers do not and cannot know with great certainty what stakeholders will want to know in every particular case. Instead, our interviews demonstrate anew what has been found in the past: modelers make assumptions about what stakeholders want to know and, while these are often reasonable and based on experience of working with stakeholders, they are not always accurate (Webler et al., 2011). Second, stakeholders themselves may not agree about what kind of information is useful as their preferences for information are context dependent. This is particularly relevant to regional FEWS models that address multiple sectors and many different decision and policy contexts. While in principal modelers may want to provide information that is useful for stakeholders, it is not a simple question to determine what is useful from regional MFs for stakeholders with regional interests.

While a large literature emphasizes that models designed with stakeholders’ input are perceived by stakeholders to be legitimate, credible, and salient (Gray et al., 2016; Weyant, 2017; Villamor et al., 2022), achieving a sense of ownership and designing models around stakeholder questions are more difficult when pre-existing models are used. Legitimacy, credibility, and saliency are attributes that have long been recognized as relevant to the ways that scientific information informs policy making (Berkhout et al., 2002; Cash et al., 2003; Pahl-Wostl, 2008). When existing models are used to build an MF, they are unlikely to be as responsive to decision makers’ needs as would those built from scratch with stakeholder participation (González-Rosell et al., 2020). It may also be difficult to meet the goal expressed by Stern (2021), pg. 873 that “The level of sophistication desirable in a model should be driven by its intended use.” Previously developed models may not be easily modified to meet needs. However, modelers can adopt strategies that overcome these downsides and seek to leverage advantages of MFs based on existing models. Existing models likely have familiarity, credibility, and legitimacy earned from a long history of applications and extensive documentation. There are also likely gains in efficiency (of time and social expenditures) because it is arguably less resource and time intensive to connect existing models than to build new modeling frameworks from scratch.

4.2 A need for talk

These challenges point to a need for talking–talking among modelers and talking among modelers and stakeholders. Modelers and stakeholders need to talk to find the “sweet spot” between what information regional MFs can reasonably provide and what multiple stakeholders with potentially diverse interests and preferences want to know.

To this point, a recent NRC committee recommended dialogue between modelers and stakeholders but went no further than to recommend an analytic-deliberative process; an adaptive, learning-based dialogue that needs to be tailored to the specific context (see also NRC, 1996; NRC, 2008; National Academies of Sciences, Engineering, and Medicine [NAS], 2021). There is no “recipe” for how to best involve stakeholders in such dialogue (Tuler and Webler, 2010). A promising approach is to employ diagnostic questions to guide design (NRC, 2008). The use of diagnostic questions can inform choices affected by modelers and stakeholders having different conceptual models of systems, conceptions of how to make MFs useful in specific contexts, and preferences for how to engage stakeholders.

Diagnostic questions for developing MFs that can provide useful information to stakeholders can build on a framework to understand “decision landscapes” (Webler et al., 2015). Questions should identify potential users, their preferences for information, preferences for engagement, and their capacity to use modeling outputs. They should help modelers match to the extent possible MF capabilities with stakeholder needs. By engaging stakeholders in dialogue modelers can help stakeholders understand what can and cannot be modeled and the implications of choices in the design of the MF. Example questions include.

- Who are the stakeholders that may be interested in the MF, and its particular spatial and temporal scales?

- What kinds of decisions or actions are stakeholders engaged in?

- What are the stakeholders hoping to achieve with the MF (e.g., identifying consequences, exploring trade-offs, forecasting, backcasting)?

- What information (i.e., outputs) do stakeholders want to know to answer their questions?

- What information needs to be shared to demonstrate credibility of the MF?

- What are stakeholders preferences for sharing MF results (e.g., forms of data visualization)?

Another reason for talk among modelers and stakeholders is to design scenarios (Wiebe et al., 2018; McBride et al., 2019; National Academies of Sciences, Engineering, and Medicine [NAS], 2021). Scenarios summarize possible futures and decision pathways and can help decision makers become more aware of the possible consequences of given decisions; coupled with models they are “learning machines” that support exploration and learning (Berkhout et al., 2002; Pahl-Wostl, 2008; Tuler et al., 2017; Dorin and Joly, 2020). Modelers and stakeholders in this study highlighted the important role of scenarios to explore possible effects of climate extremes and the effects of interventions on different sectors and communities.

The articulation of scenarios requires conversation between modelers and stakeholders and transparency about what can be modeled within a particular MF (Videira et al., 2017; Webler et al., 2017). Given the lengthy time periods it can take an MF to run, it is important to put careful consideration into the choice of scenarios. Three aspects of designing scenarios require careful consideration between modelers and stakeholders, including: what is modeled, which of many possible inputs and outputs should be chosen, and how outcomes should be represented, including the representation of uncertainty (Pahl-Wostl, 2008). Choices about what to model (e.g., biophysical systems, socio-economic systems, behavioral responses) and input and output parameters determine what stakeholders can learn about.

One way to approach the challenge arising from coupling existing models into an MF is for modelers to define exemplary scenarios that can demonstrate the capabilities of an MF, followed by dialogue to co-design additional scenarios that are of particular interest to the stakeholders. Another approach, combined with stakeholder participation in the design of scenarios, is to develop models with reduced complexity that mimic aspects of a more complex MF (Dargin et al., 2019; Bokhari et al., 2023¹). Such models would lend themselves to a more interactive and rapid exploration of elements of an integrated FEWS.

5 Conclusion

As anthropogenic climate change drives weather extremes that threaten the productive capacity and resilience of FEWS, policymakers require information that will better enable them to manage risk (National Academies of Sciences, Engineering, and Medicine [NAS], 2021). Integrated assessment modeling frameworks that describe complex interactions in FEWS have grown appreciably over the last two decades. Such frameworks allow policymakers to explore the consequences of proposed actions by revealing trade-offs between subsystems and, through an iterative process, develop policies and decisions that improve resilience of the system to weather-related shocks. The C-FEWS project attempts to capitalize on existing models and to make them broadly useful to a wide range of policymakers by coupling component models in a manner such that region-wide scenarios can be run that examine the impacts of climate adaptation strategies across multiple systems.

In interviews with modelers building the C-FEWS MF we identified challenges that this group of modelers faced while designing and building a regional modeling framework. In interviews with stakeholders who are anticipated users of MF outputs, we learned about their expectations and hopes for MFs. We found that modelers and stakeholders realize they need to speak to each other to ensure that the MF is relevant to the intended users.

While modelers are appropriately focused on the accuracy of the models, communication plays a critical role. Communication within the modeler teams is key to ensure all are using the same definitions, that there is transparency about model inputs, assumptions, and outputs, that there is agreement on how uncertainty is estimated and managed in the models. Communication between modelers and stakeholders is critical to ensure that the meaning of the model outputs is accurately understood, that model outputs are of use to stakeholders, and that modelers are running scenarios of interest and use to stakeholders.

Because they aspire to be relevant to stakeholders across a large geography, MFs about FEWS face the challenge that it is not possible to meet the needs and preferences of all stakeholders with different roles and agendas across diverse sectors in a region. Instead, modelers and stakeholders need to talk within and among themselves to identify that “sweet spot” where MFs provide much of what most stakeholders need, minimize significant changes to existing models, and make reasonable demands on computer capacity.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Social and Environmental Research Institute IRB Committee. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

TW and ST contributed equally to all aspects of this work and share first authorship. RH contributed to interviews and data analysis. CV, JM, and DW contributed to the conceptualization of the research, interpretation of results, and editing.

Funding

Financial support for this work was provided by the U.S. National Science Foundation’s Innovations at the Nexus of Food, Energy and Water Systems Program (INFEWS/T1 Grant #1856012). Publication of this article is supported by the US National Science Foundation (NSF) Sustainability Research Network (SRN) award #1444745.

Acknowledgments

We would like to thank all the people who generously donated their time to be interviewed for this study. We are especially grateful for the willingness of C-FEWS project team members to reflect on their experiences and provide a peek behind the curtain of developing a modeling framework. We thank Thomas Dietz for his suggestions about the design of this study and comments on an early draft of the paper. We also benefited from the comments of two reviewers.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

¹Bokhari, H. H., Najafi, E., Dawidowicz, J., Wuchen, L., Maxfield, N., Vörösmarty, C. J., et al. (2023). Simulating basin-scale linkages of the food-energy-water nexus with reduced complexity modeling

References

Berardy, A., and Chester, M. V. (2017). Climate change vulnerability in the food, energy, and water nexus: Concerns for agricultural production in Arizona and its urban export supply. Environ. Res. Lett. 12 (3), 035004. doi:10.1088/1748-9326/aa5e6d

CrossRef Full Text | Google Scholar

Berkhout, F., Hertin, J., and Jordan, A. (2002). Socio-economic futures in climate change impact assessment: Using scenarios as “learning machines. Glob. Environ. Change 12 (2), 83–95. doi:10.1016/s0959-3780(02)00006-7

CrossRef Full Text | Google Scholar

Cash, D., Clark, W. C., Alcock, F., Dickson, N. M., Eckley, N., and Jäger, J. (2003). Salience, credibility, legitimacy and boundaries: Linking research, assessment and decision making. Available at: https://dash.harvard.edu/handle/1/32067415.

Google Scholar

Corbin, J., and Strauss, A. (2014). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage Publications.

Google Scholar

D'Odorico, P., Davis, K. F., Rosa, L., Carr, J. A., Chiarelli, D., Dell'Angelo, J., et al. (2018). The global food-energy-water nexus. Rev. Geophys. 56 (3), 456–531. doi:10.1029/2017rg000591

CrossRef Full Text | Google Scholar

Daher, B. T., and Mohtar, R. H. (2015). Water–energy–food (WEF) nexus tool 2.0: Guiding integrative resource planning and decision-making. Water Int. 40 (5-6), 748–771. doi:10.1080/02508060.2015.1074148

CrossRef Full Text | Google Scholar

Dargin, J., Daher, B., and Mohtar, R. H. (2019). Complexity versus simplicity in water energy food nexus (WEF) assessment tools. Sci. Total Environ. 650, 1566–1575. doi:10.1016/j.scitotenv.2018.09.080

PubMed Abstract | CrossRef Full Text | Google Scholar

Denholm, P., Brown, P., Cole, W., Mai, T., and Sergi, B. (2022). Examining supply-side options to achieve 100% clean electricity by 2035. Golden, CO: National Renewable Energy Laboratory. NREL/TP-6A40-81644. Available at: https://www.nrel.gov/docs/fy22osti/81644.pdf.

Google Scholar

Dorin, B., and Joly, P. B. (2020). Modelling world agriculture as a learning machine? From mainstream models to agribiom 1.0. Land Use Policy 96, 103624. doi:10.1016/j.landusepol.2018.09.028

CrossRef Full Text | Google Scholar

Fisher-Vanden, K., and Weyant, J. (2020). The evolution of integrated assessment: Developing the next generation of use-inspired integrated assessment tools. Annu. Rev. Resour. Econ. 12 (1), 471–487. doi:10.1146/annurev-resource-110119-030314

CrossRef Full Text | Google Scholar

Glaser, B., and Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.

Google Scholar

González-Rosell, A., Blanco, M., and Arfa, I. (2020). Integrating stakeholder views and system dynamics to assess the water–energy–food nexus in andalusia. Water 12 (11), 3172. doi:10.3390/w12113172

CrossRef Full Text | Google Scholar

S. Gray, M. Paolisso, R. Jordan, and S. Gray (Editors) (2016). Environmental modeling with stakeholders: Theory, methods, and applications (Springer).

Google Scholar

Haimes, Y. Y. (2018). Risk modeling of interdependent complex systems of systems: Theory and practice. Risk Anal. 38 (1), 84–98. doi:10.1111/risa.12804

PubMed Abstract | CrossRef Full Text | Google Scholar

Howells, M., Hermann, S., Welsch, M., Bazilian, M., Segerstrom, R., Alfstad, T., et al. (2013). Integrated analysis of climate change, land-use, energy and water strategies. Nat. Clim. Change 3, 621–626. doi:10.1038/nclimate1789

CrossRef Full Text | Google Scholar

Keairns, D. L., Darton, R. C., and Irabien, A. (2016). The energy-water-food nexus. Annu. Rev. Chem. Biomol. Eng. 7, 239–262. doi:10.1146/annurev-chembioeng-080615-033539

PubMed Abstract | CrossRef Full Text | Google Scholar

Kling, C. L., Arritt, R. W., Calhoun, G., and Keiser, D. A. (2017). Integrated assessment models of the food, energy, and water nexus: A review and an outline of research needs. Annu. Rev. Resour. Econ. 9, 143–163. doi:10.1146/annurev-resource-100516-033533

CrossRef Full Text | Google Scholar

Mannan, M., Al-Ansari, T., Mackey, H. R., and Al-Ghamdi, S. G. (2018). Quantifying the energy, water and food nexus: A review of the latest developments based on life-cycle assessment. J. Clean. Prod. 193, 300–314. doi:10.1016/j.jclepro.2018.05.050

CrossRef Full Text | Google Scholar

McBride, M. F., Duveneck, M. J., Lambert, K. F., Theoharides, K. A., and Thompson, J. R. (2019). Perspectives of resource management professionals on the future of New England’s landscape: Challenges, barriers, and opportunities. Landsc. Urban Plan. 188, 30–42. doi:10.1016/j.landurbplan.2018.10.019

CrossRef Full Text | Google Scholar

Miara, A., Macknick, J. E., Vörösmarty, C. J., Tidwell, V. C., Newmark, R., and Fekete, B. (2017). Climate and water resource change impacts and adaptation potential for US power supply. Nat. Clim. Change 7 (11), 793–798. doi:10.1038/nclimate3417

CrossRef Full Text | Google Scholar

National Academies of Sciences, Engineering, and Medicine [NAS] (2021). Global change research needs and opportunities for 2022-2031. Washington, DC: The National Academies Press.

Google Scholar

National Research Council (1996). Understanding risk. Washington, DC: National Academies Press.

Google Scholar

National Research Council (2008). Public participation in environmental assessment and decision making. Washington, DC: National Academies Press.

Google Scholar

Nie, Y., Avraamidou, S., Xiao, X., Pistikopoulos, E. N., Li, J., Zeng, Y., et al. (2019). A Food-Energy-Water Nexus approach for land use optimization. Sci. Total Environ. 659, 7–19. doi:10.1016/j.scitotenv.2018.12.242

PubMed Abstract | CrossRef Full Text | Google Scholar

Nielsen, K. S., Stern, P. C., Dietz, T., Gilligan, J. M., van Vuuren, D. P., Figueroa, M. J., et al. (2020). Improving climate change mitigation analysis: A framework for examining feasibility. One Earth 3 (3), 325–336. doi:10.1016/j.oneear.2020.08.007

CrossRef Full Text | Google Scholar

Pahl-Wostl, C. (2008). “Participation in building environmental scenarios,” in Environmental futures: The practice of environmental scenario analysis. Editor J. Alcamo (Elsevier), 105–122.

CrossRef Full Text | Google Scholar

Schmitt-Olabisi, L., McNall, M., Porter, W., and Zhao, J. (2020). Innovations in collaborative modeling. East Lansing: MSU Press.

Google Scholar

Stern, P. C. (2021). On modeling worldviews in quantitative decision support. Risk Anal. 41 (6), 870–873. doi:10.1111/risa.13749

PubMed Abstract | CrossRef Full Text | Google Scholar

Thompson, J., Ganapathysubramanian, B., Chen, W., Dorneich, M., Gassman, P., Krejci, C., et al. (2021). Iowa urban FEWS: Integrating social and biophysical models for exploration of urban food, energy, and water systems. Front. big Data 4, 662186. doi:10.3389/fdata.2021.662186

PubMed Abstract | CrossRef Full Text | Google Scholar

Tuler, S., Dow, K., Webler, T., and Whitehead, J. (2017). “Learning through participatory modeling: Reflections on what it means and how it is measured,” in Including stakeholders in environmental modeling. Editors S. Grey, M. Paolisso, R. Jordan, and S. Grey (Switzerland: Springer International Publishing), 25–45.

CrossRef Full Text | Google Scholar

Tuler, S., and Webler, T. (2010). How preferences for public participation are linked to perceptions of the context, preferences for outcomes, and individual characteristics. Environ. Manag. 46 (2), 254–267. doi:10.1007/s00267-010-9515-1

CrossRef Full Text | Google Scholar

US DOE (US Dep. Energy) (2020). Multisector dynamics. Washington, DC: Off. Sci., US Dep. Energy. Available at: https://climatemodeling.science.energy.gov/program/multisector-dynamics.

Google Scholar

van Vuuren, D. P., Kok, M., Lucas, P. L., Prins, A. G., Alkemade, R., van den Berg, M., et al. (2015). Pathways to achieve a set of ambitious global sustainability objectives by 2050: Explorations using the IMAGE integrated assessment model. Technol. Forecast. Soc. Change 98, 303–323. doi:10.1016/j.techfore.2015.03.005

CrossRef Full Text | Google Scholar

Vidiera, N., Antunes, P., and Santos, R. (2017). “Engaging stakeholders in environmental and sustainability decisions with participatory system dynamics modeling,” in Including stakeholders in environmental modeling. Editors S. Grey, M. Paolisso, R. Jordan, and S. Grey (Switzerland: Springer International Publishing), 241–265.

Google Scholar

Viglia, S., Brown, M. T., Love, D. C., Fry, J., Neff, R. A., and Hilborn, R. (2022). Wild caught Alaska sockeye salmon: A case study of the food energy water nexus for a sustainable wild catch fishery. J. Clean. Prod. 369, 133263. doi:10.1016/j.jclepro.2022.133263

CrossRef Full Text | Google Scholar

Villamor, G. B., Griffith, D. L., Kliskey, A., and Alessa, L. (2020). Contrasting stakeholder and scientist conceptual models of food-energy-water systems: A case study in magic valley, southern Idaho. Socio-Environ. Syst. Modell. 2, 16312. doi:10.18174/sesmo.2020a16312

CrossRef Full Text | Google Scholar

Villamor, G. B., Sharma-Wallace, L., van Noordwijk, M., Barnard, T., and Meason, D. F. (2022). A systematic review of participatory integrated assessment at the catchment scale: Lessons learned from practice. Curr. Res. Environ. Sustain. 4, 100167. doi:10.1016/j.crsust.2022.100167

CrossRef Full Text | Google Scholar

Vörösmarty, C. J., Melillo, J. M., Wuebbles, D. J., Jain, A. K., Ando, A. W., Chen, M., et al. (2023). The C-FEWS framework: Supporting studies of climate-induced extremes on food, energy, and water systems at the regional scale. Front. Environ. Sci. 11, 1069613. (this issue). doi:10.3389/fenvs.2023.1069613

CrossRef Full Text | Google Scholar

Webler, T., Tuler, S., and Dietz, T. (2011). Modellers' and outreach professionals' views on the role of models in watershed management. Environ. Policy and Gov. 21 (6), 472–486. doi:10.1002/eet.587

CrossRef Full Text | Google Scholar

Webler, T., Tuler, S., Goble, R., and Schweizer, P. (2015). Characterizing landscapes of regional risk governance. Int. J. Perform. Eng. 11 (6), 521–534. doi:10.23940/ijpe.15.6.p605.mag

CrossRef Full Text | Google Scholar

Webler, T., Stancioff, E., Goble, R., and Whitehead, J. (2017). “Participatory modeling and community dialog about vulnerability of lobster fishing to climate change,” in Including stakeholders in environmental modeling. Editors S. Grey, M. Paolisso, R. Jordan, and S. Grey (Switzerland: Springer International Publishing), 267–287.

CrossRef Full Text | Google Scholar

Welsh, W. D., Vaze, J., Dutta, D., Rassam, D., Rahman, J. M., Jolly, I. D., et al. (2013). An integrated modelling framework for regulated river systems. Environ. Model. Softw. 39, 81–102. doi:10.1016/j.envsoft.2012.02.022

CrossRef Full Text | Google Scholar

Weyant, J. (2017). Some contributions of integrated assessment models of global climate change. Rev. Environ. Econ. Policy 11 (1), 115–137. doi:10.1093/reep/rew018

CrossRef Full Text | Google Scholar

Wiebe, K., Zurek, M., Lord, S., Brzezina, N., Gabrielyan, G., Libertini, J., et al. (2018). Scenario development and foresight analysis: Exploring options to inform choices. Annu. Rev. Environ. Resour. 43, 545–570. doi:10.1146/annurev-environ-102017-030109

CrossRef Full Text | Google Scholar