Adaptive Planning, Monitoring, and Evaluation for Long-Term Impact: Insights From a Water Supply Case in Bangladesh

Most water and development interventions aim to contribute to long-term sustainable impacts. Given the uncertainties involved in these longer-term water development projects, adaptive planning approaches hold promise to connect planning, implementation and evaluation. Recent innovations report promising results, but find limited wide-spread practice due to a relatively large distance from current operational realities. Therefore, in this article we set out to investigate the real-world benefits and barriers for adaptive planning, monitoring and evaluation as a tool for water development interventions. To do so, we have stripped the advanced theoretic adaptive planning approach to its essentials, for exploration in an ongoing water development project. Application of the resulting three essential adaptive planning steps to a water development project in Khulna, Bangladesh shows that these steps are feasible and can support a more adaptive planning and management. In particular, they have helped to surface critical assumptions and uncertainties, as well as associated adaptation actions for the case. These were related to spatial development, water quality, finance, and management capacity. However, results also show that the actual use of the proposed steps is likely to be hindered by strategic misrepresentation. Our findings suggest this as an additional and more fundamental barrier to the widespread use of adaptive planning practices. We reflect on this barrier of strategic misrepresentation and on possible mechanisms to counter it, in order to enable water development actors to make their planning and evaluation arrangements more adaptive.

Introduction

Long-term sustainability has been one of the most important and challenging ambitions of water and development interventions. The Sustainable Development Goals underscore the importance of sustainable impacts, which endure over longer periods of time (United Nations, 2015). At the same time, many development partners struggle to ensure that water infrastructure continues to function and that reforms within water sector organizations continue to be effective also after development projects have finished. In recent years, this has led many of the key players in the development sector to focus on long term and lasting impacts in their development strategies (World Bank Group, 2013; ICAI, 2015; Ministry of Foreign Affairs, 2016). For instance, the Dutch government has formulated a 15-year sustainability clause for all its official development aid in water supply and sanitation (Ministry of Foreign Affairs, 2016).

As a consequence, monitoring and evaluation arrangements are required to assess the long-term impacts of water and development interventions. It is no longer sufficient to account for the direct use of development funds and the immediate project outputs and short-term results. Also longer term impacts need to be accounted for. A problem, of course, is that long-term impacts of development interventions are difficult to predict or control at the inception of new projects. Furthermore, after a long period of time it is very hard to attribute observed change or inertia to a specific development project that was initiated many years ago. This creates challenges that cannot be solved only by better ex-post evaluation designs and methods. Rather, it also requires a rethinking of linkages between evaluation, monitoring, implementation and planning for long-term sustainability (Patton, 2011). This echoes observations that for sustainable development, the fields of planning and evaluation are likely to grow closer together (Chelimsky, 2019, p. 83).

In recent years, adaptive planning approaches have emerged as a new way to support long-term planning under uncertainty (Lempert et al., 2003; Haasnoot et al., 2013; Walker et al., 2013; Marchau et al., 2019). Similarly, the implications of adaptive planning for evaluation have been explored as part of developmental evaluation (Patton, 2011; Larson, 2018) and as a logical extension of adaptive planning approaches (Hermans et al., 2017). Adaptive planning approaches have been applied to support water and delta management in countries such as the United States (Lempert and Groves, 2010), the Netherlands (Bloemen et al., 2017), and the United Kingdom (Ranger et al., 2013). For developing countries, adaptive planning has been explored by the UK Department for International Development (Ramalingam et al., 2014), Australian Aid (Ladner, 2015), and the World Bank (Choy et al., 2018), with explorations in the water sector for strategic delta planning in Vietnam (Mekong Delta Plan, 2013), and Bangladesh (BDP2100, 2017).

The research question that we pose in this article, is if these adaptive planning methods can add value for the planning, monitoring and evaluation cycles in international development, where so far they have not been reported extensively. We seek to assess if adaptive planning methods can help to plan, monitor and evaluate real-world water development interventions for long-term sustainability. In doing so, our ambition is not so much in offering yet another conceptual innovation, but in testing the real-world applicability of recent innovations in adaptive planning and management.

We first review existing planning and management approaches for development projects, along with literature on adaptive planning approaches. This review is used to propose some relatively straightforward steps to make existing planning approaches more adaptive. The application of these steps is reported for a project in the water, sanitation, and hygiene (WASH) sector in Bangladesh. The results provide food for discussion and thought, with which we end this article.

Approaches to Plan, Monitor, and Evaluate the Long-Term Sustainability of Development Interventions

Recent Developments to Deal With Long-Term Sustainability, Complexity, and Uncertainty in the Evaluation Domain

Approaches such as results-based management, impact evaluations, and performance management have dominated planning, monitoring, and evaluation in most international development organizations in the past years (United Nations Development Group, 2011; OECD, 2017; United Nations Joint Inspection Unit, 2017). Although there is debate about the benefits and limitations of these approaches (e.g., Smith and Radin, 2009; Raimondo, 2018) it is fair to expect that, for the time being and for the foreseeable future, these remain the dominant approaches to planning, management and evaluation.

A key feature of the results-based and impact-oriented management approaches is the use of “causal linkages in a hierarchy of results (inputs, activities, outputs, outcomes, impacts) based on a theory of how change happens” (United Nations Joint Inspection Unit, 2017, p. 20). In the planning stage of development interventions, typically a theory of change or a logic-model (Morell, 2019) is developed that specifies the expected causal chain through which an intervention would lead to a desired impact. Although most of these contemporary management approaches also have the ambition to describe change processes in ways that go beyond straightforward linear logics (United Nations Joint Inspection Unit, 2017), there are not yet mainstreamed responses for dealing with non-linear complexity (OECD, 2017).

The ambition to develop novel methods that can deal with complexity and uncertainty has inspired a new wave of research and investigation. In the evaluation field, systems approaches are looked into for guidance on dealing with complexity, path dependence, actor diversity, emergence, uncertainty, and non-linearity (Patton, 2011; Ramalingam et al., 2014; Larson, 2018; Moore et al., 2019). There are approaches that focus more on the actor diversity and social implications of complexity and there are approaches that focus more on non-linearity and uncertainty in systems. Examples of the first are the actor-oriented evaluation approach by Van Ongevalle et al. (2014), game-theory methods for evaluation (Hermans et al., 2014) and methods such as network analysis and agent-based modeling (Ramalingam et al., 2014; Moore et al., 2019). Examples of approaches to deal with uncertainty and emergence in systems include outcome mapping (Earl et al., 2001), developmental evaluation (Patton, 2011; Lawrence et al., 2018), problem-driven iterative adaptation (Andrews et al., 2013), complexity informed theories of change (Ramalingam et al., 2014), outcome harvesting (Wilson-Grau and Britt, 2012), and strategy testing (Ladner, 2015).

Beyond the evaluation field, in the more general field of planning and policy analysis, attention has also focused on systems approaches as ways to better deal with complexity and uncertainties (Walker et al., 2013; Carey et al., 2015; Marchau et al., 2019). Assumption-based planning approaches stressed the importance of identifying the assumptions upon which the success of the plan most heavily rest. Assumption-based planning was developed in the late 1980s based on systems thinking (Dewar et al., 1993), strategic assumption surfacing and testing had already been developed earlier (Mason and Mitroff, 1981). Similarly, scenario-based planning methods were developed to deal with a variety of relevant uncertainties and their development over time (Schwartz, 1991; Enserink et al., 2010; Walker et al., 2013). More recently, these assumption-based planning and scenario approaches have been combined into adaptive planning approaches such as adaptive policymaking (Walker et al., 2013), dynamic adaptive policy pathways (Haasnoot et al., 2013) and adaptation pathways (Maru et al., 2014). Monitoring and evaluation designs are essential components of these approaches and can be based on pre-defined plan vulnerabilities—along with outputs, outcomes and impacts (Hermans et al., 2017; Walker et al., 2019). Researchers are now using these methods to study community adaptation as part of adaptive delta management (Kulsum et al., 2020), which can be considered part of a wider scientific endeavor around notions of adaptation, resilience and vulnerability, also in the context of countries like Bangladesh (Kulsum et al., 2020; Nath et al., 2020).

Despite the theoretical and methodological advances related to understanding and managing adaptation, these adaptive approaches are not yet mainstreamed as planning and evaluation methods (Carey et al., 2015). Ramalingam et al. (2014) observed that the analysis in their pilot applications remained at a relatively high level of abstraction, removed from the more operational side of things. Recommendations that were being made based on these approaches, did not always correctly factor in the existing organizational andpolitical constraints (Ramalingam et al., 2014, p. 37). Others have similarly observed that existing institutional or organizational restrictions might not leave room to act on the recommendations to adapt or change part of projects or programmes: Novel methods seem too far removed from current skills and expertise with planners, evaluators, and evaluation commissioners (Hermans et al., 2014; Ladner, 2015) and many of the new approaches are time and labor intensive (Ladner, 2015; Lawrence et al., 2018).

Turning methodological innovations and adaptive planning theory into practice, is thus likely to be served by some relatively simple or smart extensions to make existing planning and evaluation approaches more adaptive. Exploring the use of such relatively simple extensions will create a better understanding of the feasibility of underlying basic concepts, as a basis for further-reaching and more demanding innovations toward adaptive planning, monitoring and evaluation.

A Proof of Concept Approach to Test the Worth of Adaptive Planning for Evaluation

What could be a practical and basic approach that improves adaptive planning, monitoring, and evaluation? Building on these adaptive planning approaches and the growing use of systems-approaches in evaluation and in water planning, an approach can be proposed to strengthen planning, monitoring, and evaluation for complex and uncertain problems. Following the underlying basis of assumption-based planning, planners are triggered to think beforehand of ways a plan might fail and to design adaptive actions to guard against such failure.

The approach starts from a proposed plan or policy that can provide a starting point for a more adaptive planning approach. The existing plan provides what Walker et al. (2013) call a “basic policy.” The adaptive planning approaches suggest a fairly simple and practical approach that can add value while being feasible in practice. It consists of three additional steps, not yet included in mainstream planning and management approaches (Figure 1):

1) Highlighting the key impacts and critical assumptions of the planned development intervention based on its theory of change or logic model,

2) Identifying relevant uncertainties by using contextual scenarios, and

3) Formulating adaptation actions and their timing.

FIGURE 1

Figure 1. Three additional adaptive planning steps.

Together, these three steps will result in more adaptive plans, including elements for monitoring and evaluation to help ensure timely adaptation. The practice of most water and development interventions is that they take shape through shorter-term projects that fit into longer-term programmes and strategies. For instance, a 2 year capacity development campaign in one or more regional hospitals can be part of a larger 10 year strategy to improve public health. The above three steps can be used for both types of interventions. Starting at the project level, they could be used to extend project plans, by identifying both relatively short-term and more longer-term uncertainties and adaptation actions can be incorporated. The longer-term uncertainties and adaptation actions for any given short-term project plan, can be used to feed into adaptive monitoring arrangement of longer-term strategies—and vice versa.

Methods: Exploring Adaptive Planning for a Real-World Project

Case Study Approach

The three additional adaptive planning steps have been designed as a relatively smart extension of current practices. The expectation is that these steps are more practical to use than some of the known innovations that are more demanding in terms of time and expertise, and that are more distant from current operational practices. However, even for an expected “simple” approach, unforeseen challenges, barriers, opportunities, and benefits may arise when applied in practice. Exploring these will help assess underlying causes that support or hinder uptake of similar, and more demanding and advanced approaches.

We used a proof of concept case (Kendig, 2016) to explore the practical feasibility of a more adaptive approach. Given that a known limitation of current innovative approaches is their relatively large distance from operational realities, we used an operational project case as a basis. This was a project with a clearly delineated timespan. Still, the ambition for this project was to contribute to longer-term sustainable impacts. Many interventions take place through projects that, collectively and over time, should create meaningful long-term impact. In line with this, both short-term uncertainties were covered, directly relevant for the actual project, as well as longer-term uncertainties that could affect the objective of long-term sustainable impacts.

The case that we used was a project to improve the urban drinking water supply infrastructure and services in Khulna, Bangladesh. For this Khulna Water Supply Project (Asian Development Bank, 2020), the three steps of the adaptive planning approach were applied early in 2018, when the project was nearing its final stages. This meant that the results of our “test” would not directly benefit this particular project, but also, it meant that more clarity could be expected to exist with various stakeholders about the uncertainties and complications encountered. This would help our test of the usefulness of the new basic adaptive approach.

Data Collection and Analysis

Two different strands of activities were done, whereby each time, the existing project plan was scrutinized following the three-step approach for adaptive planning (Figure 2). One activity was with international sectoral WASH experts, experienced in similar projects elsewhere. A second activity was with the direct project stakeholders and key informants in Bangladesh. This process is described further below; a more complete description of this case application is provided in (Versteeg, 2018).

FIGURE 2

Figure 2. Application of the adaptive planning approach for the Khulna Water Supply project case.

International Expert Workshop

For the first activity, a half-day workshop was designed to test the adaptive planning approach with five international WASH experts in Amsterdam, the Netherlands, on December 20, 2017. The workshop design followed the three systematic steps of the adaptive results-based planning approach. Five experts from Simavi, a Dutch NGO, participated. Four experts were WASH programme officers, a fifth expert was a planning, monitoring, and evaluation manager. These international experts were familiar with international WASH development projects, but not with the specifics of the project in Khulna city in Bangladesh. Project documents and a few key informant interviews had been done to extract the relevant elements in the project design. Based on this secondary project information, researchers developed a project logic model with key outputs, outcomes, impacts and assumptions.

During the first part of the workshop, the experts reviewed the logic model of the project, assessing the expected causal chain through which project interventions were expected to lead to the desired impact. This helped to define impact and identify critical assumptions. As part of this step, experts also looked into the assumptions that were stated in the original project documents, as part of the proposed loan and technical assistance grant by the Asian Development Bank.

In the second part of the workshop the international experts formulated relevant uncertainties that could affect long-term planning of the project, as well as associated adaptation actions. These were generated using a handful of plausible future scenarios for Khulna City in 2025, which had been prepared based on project documents prior to the workshop, and which were introduced to the international experts. The scenarios were developed with a contextual scenario approach (Schwartz, 1991; Enserink et al., 2010).

Interviews With Local Key Informants

For the second activity, in January and February 2018, 11 local key informants were interviewed, project sites were visited in Bangladesh and relevant documentation was studied. Interviewees represented organizations directly involved in the project planning and management (e.g., loan provider, executive organization) as well as organizations in the direct environment of the project (e.g., the local municipality). ADB, JICA, Khulna City Corporation (KCC), Khulna Development Authority (KDA), the Khulna Department of Public Health Engineering (DPHE) were interviewed. The Khulna Water Supply and Sewerage Authority (KWASA) was a key organization in this project and therefore multiple representatives of KWASA were interviewed separately, representing different roles: general management, financial management, engineering, and project management.

During the first part of the interviews, the causal chain through which the project interventions were expected to lead to the desired project impact was discussed. This was based on the logic model that was also used during the expert workshop. In the second step the local key informants were asked to formulate relevant uncertainties in response to different features from plausible future scenarios for Khulna city in 2025. These scenarios were the same as for the workshop with the international experts as reported in the previous section. The third step was to formulate actions to adapt the project plan to cope with the identified uncertainties. Discussion about uncertainties and adaptation actions alternated as natural response to the evolvement of the semi-structured interviews.

Results for the Khulna Drinking Water Supply Project

Introduction to the Khulna Water Supply project

Khulna, the third largest city in Bangladesh, is located in the south west area of the country (Figure 3) and has a population of 1.5 million (Khulna City Corporation, 2014). To cope with increasing water demand and to improve water supply services, the Khulna Water Supply and Sewerage Authority (KWASA) started the Khulna Water Supply Project in 2009, assisted by the Japan International Co-operation Agency (JICA) and the Asian Development Bank (ADB) (Asian Development Bank, 2011). The Khulna Water Supply Project was designed for a 6-year period (2011–2017) and later extended until 2018. The project aim was to expand the access to water supply services for the inhabitants of Khulna.

FIGURE 3

Figure 3. Location map of Khulna city in Bangladesh.

At the time of the project's inception, Khulna's water supply was based on the use of groundwater, pumped from nearby aquifers. However, with the expansion of the city and the depletion of groundwater tables, new water sources were needed to meet the growing demands. The Khulna Water Supply Project would build a surface water treatment plant that would use river water as a source, and the project would extend the distribution network to cover the whole city. This should meet the urban drinking water demand until 2025 without increasing the groundwater abstraction and thus without further undermining the sustainability of this resource. The desired long-term impact of the project was to improve the quality of life for all residents and businesses in Khulna.

Adaptive Planning With International WASH Experts

Step 1: Critical Assumptions

The international experts reviewed the project logic model. This logic model outlined three key project components around the construction of surface water treatment facilities, a drinking water distribution network, and strengthening management capacity for KWASA (Figure 4). The experts questioned the logic of the project and in particular the original project assumptions. The project assumptions that Khulna City would not expand to include fringe areas and that the population of Khulna City would not grow rapidly seemed questionable. But also for the groundwater abstraction by private wells, they observed that this could not be expected without additional action to control this. To the international experts, these assumptions seemed to signal more wishful thinking and desired developments, rather than fairly realistic expectations, even within the 6-year time horizon of the project.

FIGURE 4

Figure 4. Logic model for Khulna Water Supply Project.

Step 2: Relevant Uncertainties

Contextual scenarios were used to support the discussion of relevant uncertainties. The underlying drivers for these scenarios were urbanization (migration and population projections) and climate change (mainly for future water conditions). This resulted in a two-by-two scenario logic with four plausible scenarios (Figure 5). The international experts used these scenarios to examine the vulnerability of the project assumptions in the face of different plausible future events and developments.

FIGURE 5

Figure 5. Contextual scenarios for Khulna city in 2025.

Relevant uncertainties were formulated that were likely to be part of the intervention chain. The identified uncertainties could be clustered into different categories, related to spatial development, water quality and quantity, financial situation of Khulna WASA and uncertainty about the management capacity of Khulna WASA (Table 1). For these uncertainties, no statement was made on the distribution of short-term or longer-term uncertainties. However, they were identified based on longer-term scenarios for Khulna City in 2025. As Table 1 shows, uncertainties were identified in relation to all main project components. For instance, salinity increases in the river water were considered a risk that could influence the water treatment efforts and costs, the ability to effectively improve KWASA management capacity was uncertain, as well as the project assumptions regarding the financial underpinnings.

TABLE 1

Table 1. Uncertainties identified by experts during the adaptive planning workshop.

Step 3: Adaptation Actions

Within the third step, experts were asked to identify and consider different actions as ways to adapt the current project plan to meet possible conditions deriving from the future scenarios. The type of adaptation actions mentioned by the international experts concerned collaboration among beneficiaries, social and demand-driven approaches, policy activities, and institutional capacity strengthening (Table 2).

TABLE 2

Table 2. Adaptation actions identified in the adaptive planning expert workshop.

Although the timing of actions was an important component in the adaptive planning approach, this last component could not be covered during the adaptive planning workshop. Prior steps had consumed more time than anticipated, leaving no workshop time for the international experts to address the timing of the adaptation actions. Nevertheless, they shortly argued that the adaptation actions should be implemented directly after project completion, because all actions would be needed to adapt the current project plan to meet future unforeseen developments. These final considerations underscored that many of the adaptation actions were formulated not in response to highly uncertain developments, but perhaps more in response to basic project assumptions that were considered overly optimistic by the international experts. Also, this indicates that even with a long-term horizon, short-term improvements are easily identified.

Adaptive Planning With Project Experts and Local Stakeholders

The usefulness and practical feasibility of the proposed adaptive planning approach was also explored based on semi-structured interviews with eleven local key informants. The interviews used the same preparatory materials as the expert workshop, summarized in the text above and in Figures 4, 5.

Step 1: Critical Assumptions

The local key informants did not critically question the validity of the project assumptions. Also, they did not question if the project's intervention logic would need to be reconstructed at some point in time. The local key informants argued the project's logic model had been developed at the start of the project and provided the design of the causal chain through which project interventions would lead to the desired project impact. They considered the assumptions to be part of this design and therefore to be fixed for the duration of the project.

Although the local key informants would agree that at least some assumptions were vulnerable to be overturned by events within the planning time horizon, there was no intention to re-assess these assumptions during the course of project. The local key informants did not question the relationship between the longer-term impacts of the project and the underlying assumptions. Rather, the main concern of the local project stakeholders was toward the activities and outputs of the project (Figure 4). Project success was formulated as the realization of the short-term project activities.

Step 2: Relevant Uncertainties

Whereas the international experts identified four clusters of uncertainties (see Tables 1, 2), the discussion with local stakeholders resulted also in two clusters: urban growth and financial sustainability.

The first cluster of urban growth uncertainty was similar to the uncertainty used to design the contextual scenarios. Urban growth—the population growth and expansion of Khulna city—would have important consequences for the service area of Khulna WASA and for the design of their projects. The importance of these uncertainties was illustrated by local stakeholders with an example from the project. In 2011, the Khulna Water Supply Project started with the construction of a water treatment plant that was to supply sufficient drinking water to all citizens of Khulna city. The treatment plant was designed for 110,000 m³/day. However, in 2018 it was known that the water treatment plant should have the capacity to produce a little more than 143,000 m³/day, in order to serve the 1,500,000 citizens of Khulna city (2018).

The second relevant cluster of financial uncertainties was markedly different from the predefined scenarios. The financial sustainability of Khulna WASA, as the project's key stakeholder, would affect the operation and maintenance of the drinking water infrastructure developed in the project, as well as the financial sustainability of the project. The main source of income for Khulna WASA was collecting revenue from the sale of available water services. The KWASA income at the time of the interviews was reported to be Tk. 3,500,000 per month (roughly equivalent to 39,000 Euro). It goes without saying that these amounts would not leave any room for savings or investments. The low water tariffs in Khulna city and the significant amount of non-revenue water were the key reasons for this low income. Non-revenue water referred to water that was lost in the system—either through physical losses or through administrative losses due to unpaid bills or illegal connections. Both the low water tariffs and high amounts of non-revenue water, were not so much future uncertainties, but existing problems. The uncertainty related to the question if these problems could be tackled successfully in the future.

Step 3: Adaptation Actions

This resulted in four actions to adapt the current project plan (Table 3). Two of the adaptation actions were defined to cope with unforeseen urbanization developments for Khulna WASA's water supply services. The other two adaptation actions were defined to cope with the uncertain financial situation of Khulna WASA.

TABLE 3

Table 3. Results of interviews with local key informants: uncertainties and adaptation actions.

The first action mentioned by the local key informants to cope with urbanization developments, was the expansion of the capacity of the project's drinking water treatment plant. Several times during the interviews, the local key informants mentioned that an additional drinking water treatment plant would be needed to serve all citizens of Khulna city. The construction of additional treatment capacity was seen as a necessary follow-up project for Khulna WAS by local experts, including one of the loan-providing stakeholders. A second, somewhat similar adaptation action, was to add an already existing drinking water treatment plant to the drinking water system of Khulna WASA for residential usage. A separate treatment plant was operated to supply water to government organizations only and was reported to have excess capacity. This government organizations' drinking water treatment plant thus could be used to complement the supply capacity to cope with the growing demand for potable water in the larger Khulna city area.

In response to the uncertainty about the financial sustainability of Khulna WASA, district metering areas were mentioned as an adaptation action. In Khulna the distribution networks could be organized in 5 blocks and then further divided into 10 district metering areas per block. Amounts of inflow, outflow, and consumption could be diligently monitored for each area, to identify and minimize non-revenue water. Another adaptation action was to increase the water tariff in Khulna city. An appropriate tariff setting for the water usage would make operation and maintenance by Khulna WASA financially viable. It was suggested for tariffs to be raised. According to the national Water and Sewage Authority (WASA) Act (1996), Khulna WASA was allowed to increase water rates up to 5% in a financial year. Within these legal limits, are more appropriate tariff setting could be achieved within 6 years.

Little time was left for the discussion of the timing of adaptation actions, but it was nevertheless discussed partly during many interviews. The local key informants argued that Khulna WASA would not be able to provide potable water to all citizens of Khulna city without additional actions, and therefore would not achieve the aimed impact of Khulna' Water Supply Project. They argued that therefore the adaptation actions should be implemented as soon as possible.

Some local key informants also argued for a certain sequence in the timing of adaptation actions. The construction of an additional or expanded drinking water treatment plant would be time consuming and would require a high investment. Adding the existing treatment plant for governmental use to the supply network would probably be less time consuming and would require a smaller investment. Its effect would be less, but it might be useful as a short-term immediate step. Also increasing the water tariff could be a relatively easy because Khulna WASA did not depend on other parties as long as it would stay within the 5% bandwidth. Introducing district metering areas could be a time-consuming action but could also help to significantly reduce the amount of non-revenue water and therefore big revenue losses.

Combining the Results From International Experts and Local Stakeholders

The designed adaptive results-based planning approach was applied twice to explore its usefulness and practical feasibility for real-world development project to support long-term sustainability. Both the international experts and the local key informants executed the three additional steps of the proposed adaptive approach. The steps worked with both the international experts and the local key informants, but in different ways.

The international experts were well capable of the surfacing critical assumptions. Their review of the project resulted in the identification of various critical assumptions for the Khulna Water Supply project, many of which seemed to have been violated almost from the onset of the project. The international experts could also discuss a rich set of uncertainties, based on their experiences with similar projects in various parts of the world and triggered by the pre-developed scenarios. The international exports were not able to formulate very specific adaptation actions. They lacked the context specific knowledge to formulate adaptations actions adequate for the local conditions. The adaptation actions raised by the international experts were mostly general actions: increase collaboration across stakeholders, train the staff involved and increase enforcement to reduce illegal water use.

Where the international experts readily criticized key project assumptions, the local key informants did not do the same. They appeared to focus on justifying the project rather than critically scrutinizing its underlying assumptions. Also, the local project stakeholders focused on the immediate project outputs much more than on its longer-term impacts. The use of future scenarios was a very useful step for this group, which enabled them to identify and discuss some uncertainties without openly criticizing existing project designs. In most interviews, these discussions of uncertainties were directly related to the discussion of adaptation actions to cope with potential plan vulnerabilities. These locally generated adaptation actions were quite specific and operational, in contrast to the adaptation actions identified by the international experts.

Discussion

The Application of the Adaptive Planning Steps

The results for the Khulna Water Supply Project show that the steps identified for more adaptive planning are feasible, perhaps with the exception of the last sub-step related to the timing of adaptation action.

The first steps helped to find critical assumptions that can affect long-term sustainability. In this case, a key insight was that many of the key planning assumptions seemed to be violated already at the time of the study, before the project construction cycle was finished. This was suggested in strong signals by the international experts and was more tacitly acknowledged by the local project stakeholders. Rather than calling the Khulna Water Supply Project an outright failure, and thereby jeopardizing its completion and downplaying the significant and much needed improvements it did represent, local stakeholders readily identified several specific adaptation actions for future follow-up projects.

Adaptation actions were identified, but one could question if these actions were actually adaptation actions as understood in the current adaptive planning approaches. For one, the vulnerability of the current project was such that additional actions seemed required and useful already on the short-term and for fairly certain developments, even before longer-term uncertainties and adaptations would come into play. When the new treatment plant constructed under the current project would become operational, it would not be able to meet all water demands, while also the management and financial capacity of the Khulna WASA would remain a key source of concern.

Also, the practical feasibility and room for maneuver was not critically reviewed. How much additional funds would be required for the local level adaptation actions, and how much social and political support would they have? We worked with local key informants, which suggests some level of practical feasibility, but only in a rather limited sense. For an actual test, more would need to be done to critically assess practical feasibility of adaptation actions.

Strategic Misrepresentation in Project Planning

An important point across our case findings, is the violation of key project assumptions. Literature suggests that this might not be a unique phenomenon. According to Flyvbjerg (2007), planners and promoters misrepresent costs, benefits, and risks of projects, in order to increase the likelihood that their project gains approval and funding. Planners and promoters purposely define scenarios of success and gloss over the potential for failure. Pickrell (1992) and Altshuler and Luberoff (2004) found that municipal governments officials applying for senior-level government funding have an incentive to underestimate the cost of their projects. They do so to make the projects more attractive to provincial or federal governments.

Politicians and project promoters have an incentive to underestimate the costs of their desired infrastructure plans to make the plans more appealing to voters. And contractors competitively bidding for projects may strategically underestimate costs. They know that once they are awarded with the job, they can drive up the price through change orders. This results in the pursuit of ventures that are unlikely to come in on budget or on time, or to deliver the promised benefits. Competition between projects and authorities creates political and organizational pressures that in turn create an incentive structure that makes it rational for project promoters to emphasize benefits and deemphasize costs and failures. A project that looks highly beneficial on paper is more likely to get funded than one that does not (Flyvbjerg, 2007). Flyvbjerg (2007) thus identified strategic misrepresentation about the cost, benefits, and risks involved as a main problem in major infrastructure developments. Literature suggests that similar mechanisms are at play in development aid, with incentive structures existing with donors and recipient countries points (e.g., Savedoff et al., 2005; Morton, 2013; Watkins and Swidler, 2013).

Strategic misrepresentation has important implications for adaptive planning approaches. Managing projects for long-term impacts starts with realistic planning, and the development of different alternative pathways to respond to different types of uncertainties. Strategic misrepresentation leads to an unrealistic project. None of the key stakeholders has anything to gain from scrutinizing project plans for critical assumptions and uncertainties, as this is likely to make the overoptimistic planning assumptions explicit and thus put project funding at risk. However, if the critical assumptions and uncertainties cannot be surfaced, they cannot provide a basis for longer-term monitoring and adaptive planning either.

Using Adaptive Thinking to Counter Strategic Misrepresentation

In order to enable long-term sustainability in development projects, it is important to objectively assess and discuss the criticality of the project assumptions. With strategic misrepresentation, this is not likely to happen. Our case findings also help us to identify some counter-mechanisms though, which can be incorporated in planning and funding cycles as reality checks to counter the more “cynical” incentives in the planning system.

Based on the insights of this research, it can be recommended that different components of the adaptive planning approach for long-term sustainable WASH services should be practiced by different involved parties. In this, it is important to use the diversity of expertise and roles, including directly involved local experts, independent experts at some distance, methodological experts that can help in the facilitation of the process and funding agencies. One possible way of doing this, based on our case exploration, could be the following:

• Independent (not directly involved) sectoral experts can be called upon to identify and review critical assumptions for project impacts;

• Contextual scenarios can be used with directly involved project stakeholders to enable them to discuss critical assumptions in non-threatening ways, and to get their knowledge about context-specific and feasible adaptation actions

• An independent or impartial analyst or planning expert would facilitate these steps and prepare the necessary inputs, such as contextual scenarios (cf. Ladner, 2015).

• Ensuring that such independence and rigor is applied in the process, could be a role of funding agencies and donors—even if we acknowledge this can be problematic in light incentives for strategic misrepresentation discussed above.

Local project stakeholders could be tasked with the enforcement of these mechanisms, but are not likely to do so. As argued by Morton (2013) in any relationship, the provision of funds from one entity to another establishes a power imbalance that favors the funding agency, with the funding agency able to set the terms of the relationship, and the recipient ultimately forces to serve these terms. Thus, it would be the funding agencies that would need to oversee and demand these mechanisms to be put in place for any project funded. This requires also a change within many of these funding agencies. It requires accountability and performance management systems that acknowledge complexity and the need to adapt.

Conclusion

Managing water and development interventions for long-term sustainability requires more adaptive forms of planning, monitoring, and evaluation. Monitoring and evaluation are key to measure, understand and manage long-term sustainability. These need to be linked to planning systems that allow for reasonable adaptations within a longer-term planning time horizon. The dominant planning and management approaches for water development, such as results-based management, currently leave limited room for this. A rethinking of the linkages between planning, implementation and evaluation over longer periods of time is thus required. Recent innovations in this field report promising results, but also limited wide-spread practice due to a relatively large distance from current operational realities.

Therefore, we have used insights from adaptive planning approaches to develop a first smart and simple extension to make mainstream planning and management practices more adaptive. The resulting approach takes a proposed plan as its starting point, and then adds three adaptive planning steps: (1) defining the critical assumptions in the project, (2) defining relevant uncertainties by using contextual scenarios, and (3) defining adaptation actions and specifying the timing of these. This adaptive planning approach was explored for its usefulness and practical feasibility for a water supply project in Bangladesh.

The steps proved feasible and resulted in the identification of assumptions, uncertainties, and adaptation actions. Different roles were being filled by international experts and local project stakeholders. International experts were skilled at highlighting critical assumptions and vulnerabilities in the plan, while local stakeholders were much better at identifying useful adaptation actions, triggered by pre-developed contextual scenarios.

Case results also suggested an important barrier to the real-world application of the adaptive approaches, less highlighted in earlier studies in the water and delta management fields. Strategic misrepresentation disturbs realistic project planning. Yet adaptively managing projects for long-term impacts starts with realistic planning, and the development of different alternative pathways to respond to different types of uncertainties. Incentives in current planning and management systems steer actors away from realistic planning and acknowledgment of uncertainties. We expect that some of these negative incentives for strategic misrepresentation could be countered by ensuring an appropriate but different role for key planning actors: Independent international experts, local project stakeholders, independent adaptive planning experts, and project funding agencies and donors. Adding and institutionalizing these different roles in water and development planning arrangements, will help a more realistic and hence adaptive approach to planning, monitoring and evaluation for long-term impact in the water sector.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

NV conducted the initial literature review, collected field data, and designed and facilitated the expert workshop. SA and LH supported NV with the case work in Bangladesh and at Simavi. BV contributed to the creative process, setup and guidance of the research. NV and LH took the lead in writing the manuscript and in manuscript revisions. All authors contributed to the article and approved the submitted version.

Funding

Data collection and field work in Bangladesh for this research was supported by funds from TU Delft Global Initiative, the Netherlands Organization for Scientific Research NWO (project W.07.69.104 UDW Shifting Grounds) and Simavi. Finalization of the manuscript was supported by the DUPC Impact Initiative (project 108704).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We acknowledge Dr. md. Mustafa Saroar, Tanmoy Chakraborty and Khondaker Mohammed Mohiuddin Ekram from Khulna University of Engineering and Technology, and all the others who supported the field research in Bangladesh. Furthermore, we acknowledge the contributions from Angela van den Broek and Maurits Ertsen who provided additional guidance to NV during the research process.

References

Altshuler, A. A., and Luberoff, D. E. (2004). Mega-Projects: The Changing Politics of Urban Public Investment. Washington, DC: Brookings Institution Press.

Google Scholar

Andrews, M., Pritchett, L., and Woolcock, M. (2013). Escaping capability traps through problem driven iterative adaptation (PDIA). World Dev. 51, 234–244. doi: 10.1016/j.worlddev.2013.05.011

CrossRef Full Text | Google Scholar

Asian Development Bank (2011). Proposed Loan and Technical Assistance Grant People's Replublic of Bangladesh: Khulna Water Suppply Project. Report and Recommendation of the President to the Board of Directors. Project 42171. Asian Development Bank.

Google Scholar

Asian Development Bank (2020). Projects: Bangladesh: Khulna water supply project. Available online at: https://www.adb.org/projects/42171-013/main#project-pds (accessed December 2, 2020)

Google Scholar

BDP2100 (2017). Bangladesh Delta Plan 2100. Investment Plan for The Bangladesh Delta Plan 2100. Volume 1: The Plan. General Economic Division of the Bangladesh Planning Commission. July 2017.

Bloemen, P., Reeder, T., Zevenbergen, C., Rijke, J., and Kingsborough, A. (2017). Lessons learned from applying adaptation pathways in flood risk management and challenges for the further development of this approach. Mitig. Adapt. Strateg. Global Change 23, 1083–1108. doi: 10.1007/s11027-017-9773-9

CrossRef Full Text | Google Scholar

Carey, G., Malbon, E., Carey, N., Joyce, A., Crammond, B., and Carey, A. (2015). Systems science and systems thinking for public health: a systematic review of the field. BMJ Open 2015:e009002. doi: 10.1136/bmjopen-2015-009002

PubMed Abstract | CrossRef Full Text | Google Scholar

Chelimsky, E. (2019). “Improving the match between sustainability questions and evaluation practice: some reflections and a checklist,” in Evaluating Sustainability. New Directions for Evaluation, ed G. Julnes, 162 (Hoboken, NJ: Wiley), 69–86. doi: 10.1002/ev.20363

CrossRef Full Text | Google Scholar

Choy, A., Stanton-Geddes, Z., and Kryspin-Watson, J. (2018). Go with the flow—adaptive management for urban flood risk. Sustainable Cities Blog, The World Bank. Available online at: http://blogs.worldbank.org/sustainablecities/go-flow-adaptive-management-urban-flood-risk (accessed April 27, 2018)

Dewar, J. A., Builder, C. H., Hix, W. M., and Levin, M. H. (1993). Assumption-Based Planning; A Planning Tool for Very Uncertain Times. MR-114-a. RAND, Santa Monica. Available online at: https://www.rand.org/content/dam/rand/pubs/monograph_reports/2005/MR114.pdf

Google Scholar

Earl, S., Carden, F., and Smutylo, T. (2001). Outcome Mapping: Building Learning and Reflection into Development Programs. Ottawa, ON: IDRC.

Google Scholar

Enserink, B., Hermans, L., Kwakkel, J., Thissen, W., Koppenjan, J., and Bots, P. (2010). Policy Analysis of Multi-Actor Systems. The Hague: Boom Publishers.

PubMed Abstract | Google Scholar

Flyvbjerg, B. (2007). Policy and planning for large-infrastructure projects: problems, causes, cures. Environ. Plan. B Plan. Des. 34, 578–597. doi: 10.1068/b32111

CrossRef Full Text | Google Scholar

Haasnoot, M., Kwakkel, J. H., Walker, W. E., and Ter Maat, J. (2013). Dynamic adaptice policy pathways: a method for crafting robust decisions for a deeply uncertain world. Global Environ. Change 23, 485–498. doi: 10.1016/j.gloenvcha.2012.12.006

CrossRef Full Text | Google Scholar

Hermans, L., Cunningham, S., and Slinger, J. (2014). The usefulness of game theory as a method for policy evaluation. Evaluation 20, 10–25. doi: 10.1177/1356389013516052

CrossRef Full Text | Google Scholar

Hermans, L. M., Haasnoot, M., Ter Maat, J., and Kwakkel, J. H. (2017) Designing monitoring arrangements for collaborative learning about adaptation pathways. Environ. Sci. Policy 69, 29–38. doi: 10.1016/j.envsci.2016.12.005

CrossRef Full Text | Google Scholar

ICAI (2015). DFID's Approach to Delivering Impact. Independent Commission for Aid Impact. Report 45–June 2015. Available online at: https://icai.independent.gov.uk/wp-content/uploads/ICAI-report-DFIDs-approach-to-Delivering-Impact.pdf (accessed October 20, 2020).

Kendig, C. E. (2016). What is proof of concept research and how does it generate epistemic and ethical categories for future scientific practice? Sci. Eng. Ethics 22, 735–753. doi: 10.1007/s11948-015-9654-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Khulna City Corporation (2014). Basic Statistics. Available online at: http://www.khulnacity.org/Content/index.php?page=About_KCCandZ2candpid=30 (accessed October 20, 2020).

Google Scholar

Kulsum, U., Timmermans, J., Khan, M. S. A., and Thissen, W. (2020). A conceptual model-based approach to explore community livelihood adaptation under uncertainty for adaptive delta management. Int. J. Sustain. Dev. World Ecol. 27, 583–595. doi: 10.1080/13504509.2019.1654555

CrossRef Full Text | Google Scholar

Ladner, D. (2015). Strategy Testing: An Innovative Approach to Monitoring Highly Flexible aid Programs. Working Politically in Practice Series–Case Study No, 3. San Francisco, CA: The Asia Foundation.

Larson, A. (2018). Evaluation amidst complexity: eight evaluation questions to explain how complex adaptive systems affect program impact. Evaluation 24, 353–362. doi: 10.1177/1356389018781357

CrossRef Full Text | Google Scholar

Lawrence, R. B., Rallis, S. F., Davis, L. C., and Harrington, K. (2018). Developmental evaluation: bridging the gaps between proposal, program, and practice. Evaluation 24, 69–83. doi: 10.1177/1356389017749276

CrossRef Full Text | Google Scholar

Lempert, R. J., and Groves, D. G. (2010). Identifying and evaluating robust adaptive policy responses to climate change for water management agencies in the American west. Technol. Forecast. Soc. Change 77, 960–974. doi: 10.1016/j.techfore.2010.04.007

CrossRef Full Text | Google Scholar

Lempert, R. J., Popper, S. W., and Bankes, S. C. (2003). Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis. MR-1626-RPC. Santa Monica: RAND. doi: 10.7249/MR1626

CrossRef Full Text | Google Scholar

Marchau, V. A. W. J., Walker, W. E., Bloemen, P. J. T. M., and Popper, S. W. (2019). Decision Making Under Deep Uncertainty. From Theory to Practice. New York, NY: Springer. doi: 10.1007/978-3-030-05252-2

CrossRef Full Text | Google Scholar

Maru, Y. T., Smith, M. S., Sparrow, A., Pinho, P. F., and Dube, O. P. (2014). A linked vulnerability and resilience framework for adaptation pathways in remote disadvantaged communities. Global Environ. Change 28, 337–350. doi: 10.1016/j.gloenvcha.2013.12.007

CrossRef Full Text | Google Scholar

Mason, R. O., and Mitroff, I. I. (1981). Challenging Strategic Planning Assumptions: Theory, Cases, and Techniques. Hoboken, NJ: John Wiley and Sons.

Mekong Delta Plan (2013). Mekong Delta Plan. Long Term Vision and Strategy for a Safe, Prosperous and Sustainable Delta. Available online at: https://www.wur.nl/upload_mm/2/c/3/b5f2e669-cb48-4ed7-afb6-682f5216fe7d_mekong.pdf (accessed August 29, 2018)

Ministry of Foreign Affairs (2016). WASH Strategy 2016-2030, Contributing to Water, Sanitation and Hygiene for all Forever. The Hague: Ministry of Foreign Affairs.

Moore, M., Parsons, B., and Jessup, P. (2019). How evaluators can use a complex systems lens to get “untrapped” from limiting beliefs and assumptions. Am. J. Eval. 40, 75–91. doi: 10.1177/1098214018756578

CrossRef Full Text | Google Scholar

Morell, J. A. (2019). Linking management and evaluation: project schedules as program models. Am. J. Eval. 40, 472–489. doi: 10.1177/1098214018784198

CrossRef Full Text | Google Scholar

Morton, B. (2013). An Overview of International NGOs in Development Cooperation. New York, NY: United Nations Development Program.

Nath, S., van Laerhoven, F., Driessen, P., and Nadiruzzaman, M. (2020). Capital, rules or conflict? Factors affecting livelihood-strategies, infrastructure-resilience, and livelihood-vulnerability in the polders of Bangladesh. Sustain. Sci. 15, 1169–1183. doi: 10.1007/s11625-020-00818-6

CrossRef Full Text | Google Scholar

OECD (2017). “Strengthening the results chain: synthesis of case studies of results-based management by providers,” in OECD Development Policy Papers. Poliy Paper 7 by Rosie Zwart, Paris, OECD, doi: 10.1787/544032a1-en

CrossRef Full Text | Google Scholar

Patton, M. Q. (2011). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York, NY: Guilford Press.

PubMed Abstract | Google Scholar

Pickrell, D. H. (1992). A desire named streetcar fantasy and fact in rail transit planning. J. Am. Plan. Assoc. 58, 158–176. doi: 10.1080/01944369208975791

CrossRef Full Text | Google Scholar

Raimondo, E. (2018). The power and dysfunctions of evaluation systems in international organizations. Evaluation 24, 26–41. doi: 10.1177/1356389017749068

CrossRef Full Text | Google Scholar

Ramalingam, B., Laric, M., and Primrose, J. (2014). From Best Practice to Best Fit: Understanding and Navigating Wicked Problems in International Development. ODI Working Paper. London: Overseas Development Institute.

Google Scholar

Ranger, N., Reeder, T., and Lowe, J. (2013). Addressing ‘deep' uncertainty over long-term climate in major infrastructure projects: four innovations of the Thames Estuary 2100 Project. Euro J. Decision Proces. 1, 233–262. doi: 10.1007/s40070-013-0014-5

CrossRef Full Text | Google Scholar

Savedoff, W. D., Levine, R., and Birdsall, N. (2005). When Will We Ever Learn? Recommendations to Improve Social Development Through Enhanced Impact Evaluation. Consultation Draft. Washington, DC: Center for Global Development.

Google Scholar

Schwartz, P. (1991). The Art of the Long View: Paths to Strategic Insight for Yourself and Your Company. New York, NY: Currency Doubleday.

Smith, D., and Radin, B. A. (2009). Point/counterpoint. Performance management and measurement. J. Policy Anal. and Manag. 28, 496–516.

Google Scholar

United Nations (2015). Transforming Our World: The 2030 Agenda for Sustainable Development. A/RES/70/1. Available online at: https://sustainabledevelopment.un.org/post2015/transformingourworld/publication (accessed October 20, 2020).

Google Scholar

United Nations Development Group (2011). Results-Based Management Handbook. Harmonizing RBM Concepts and Approaches for Improved Development Results at Country Level. Available online at: https://undg.org/wp-content/uploads/2016/10/UNDG-RBM-Handbook-2012.pdf (accessed October 20, 2020).

United Nations Joint Inspection Unit (2017). Results-Based Management in the United Nations Development System. Analysis of Progress and Policy Effectiveness. Full report. Geneva. Sukai Prom Jackson, Joint Inspection Unit. Available online at: https://www.unjiu.org/sites/www.unjiu.org/files/rbm_full_report_english.pdf (accessed October 20, 2020).

Van Ongevalle, J., Huyse, H., and Van Petegem, P. (2014). Dealing with complexity through actor-focused planning, monitoring and evaluation (PME). Evaluation 20, 447–466. doi: 10.1177/1356389014551487

CrossRef Full Text | Google Scholar

Versteeg, N. (2018). Adaptive planning for sustainable WASH (MSc thesis). Delft University of Technology, Delft. Available online at: http://resolver.tudelft.nl/uuid:d46800ab-7070-41e2-9634-ca02b2f053f8 (accessed October 20, 2020).

Google Scholar

Walker, W. E., Marchau, V., and Kwakkel, J. H. (2013). “Uncertainty in the framework of policy analysis,” in Public Policy Analysis: New Developments, eds W. A. H. Thissen and W. E. Walker (New York: Springer), 215–261. doi: 10.1007/978-1-4614-4602-6_9

CrossRef Full Text | Google Scholar

Walker, W. E., Marchau, V. A. W. J., and Kwakkel, J. H. (2019). “Dynamic adaptive planning,” in Decision Making Under Deep Uncertainty, eds V. A. W. J. Marchau, W. E. Walker, P. J. T. M. Bloemen, and S. W. Popper (New York: Springer), 53–68. doi: 10.1007/978-3-030-05252-2_3

CrossRef Full Text | Google Scholar

Watkins, S. C., and Swidler, A. (2013). Working misunderstandings: donors, brokers, and villagers in Africa's AIDS industry. Popul. Dev. Rev. 38, 197–218. doi: 10.1111/j.1728-4457.2013.00560.x

CrossRef Full Text | Google Scholar

Wilson-Grau, R., and Britt, H. (2012). Outcome Harvesting. Cairo: Ford Foundation.

Google Scholar

World Bank Group (2013). A Stronger, Connected, Solutions World Bank Group: An Overview of the World Bank Group Strategy. Washington, DC. World Bank. Available online at: https://openknowledge.worldbank.org/handle/10986/16093 (accessed October 20, 2020).