Monitoring, evaluation, accountability, and learning (MEAL) in anticipatory action—earth observation as a game changer
Markus Enenkel
Karen Dall2 
Charles K. Huyck
Shanna N. McClain- 1Harvard Humanitarian Initiative, Cambridge, MA, United States
- 2German Red Cross, Berlin, Germany
- 3ImageCat, Long Beach, CA, United States
- 4National Aeronautics and Space Administration, Washington, DC, United States
- 5Australian Red Cross, Melbourne, VIC, Australia
For many decades, humanitarian assistance relied on emergency response, triggering both funding and operational activities only after disaster impacts had been recorded. In recent years, many humanitarian actors have joined forces to complement traditional, reactive mechanisms with a forward-looking approach that can be activated before a disaster strikes. Anticipatory action (AA) uses forecasts of extreme weather events and combines them with risk information to identify and implement locally-led early actions with the goal of protecting lives and livelihoods more efficiently. AA is still a relatively new approach. Hence, monitoring, evaluation, accountability and learning (MEAL) is crucial to measure its effectiveness and adjust where necessary, as well as for (government) donors that want to see the added value of their investment maximized. However, evidence-based studies that investigate potential limitations and the exact impact pathway of AA at household level are time-consuming, costly, and therefore scarce. Satellite earth observation can become a game changer in AA by strengthening the evidence base via rapid, low-cost assessments. Both commercial and freely available satellite-derived data have reached an unprecedented level of quality, spatial, and temporal resolution. Simultaneously, there are major uncertainties regarding where, when, how, and under what conditions satellite data can support MEAL for AA at all. We argue that satellite data for an advanced MEAL framework should be considered already in the design phase of AA projects and that the translation of satellite data into actionable information will require a cross-cutting community of practice.
Introduction
Empirical evidence indicates that initiating humanitarian activities based on early warnings can help to protect lives and livelihoods while simultaneously reducing costs (Lopez et al., 2020). Nevertheless, only a fraction of humanitarian funding, sometimes cited as around one percent (Weingärtner and Spencer, 2019), is issued through pre-agreed triggers and plans. The overall objective is to make 1 billion people safer from disasters by 2025 (Risk-informed Early Action Partnership, 2021) through a combination of weather or climate forecasts, pre-agreed action plans, and pre-agreed finance—in short: anticipatory action1 (AA). The transition toward a forward-looking humanitarian system requires a strong evidence base and monitoring tools to evaluate and learn what works and how to do better (UN World Food Programme, 2021). Even though the Anticipation Hub of the Red Cross Red Crescent Movement (RCRC) has already developed an evidence database for AA2, the generation of robust evidence remains complex, costly, and time-consuming. Based on the monitoring, evaluation, accountability, and learning (MEAL) framework, this study explores how satellite earth observation (EO) could support the assessment of the added-value and limitations of AA. The following sections correspond to each of the four MEAL components (in a slightly different order). We end with a hypothetical use case and a conclusion that aims to pave the way for the first actual use cases.
Monitoring of hazards and vulnerabilities
Seasonal forecasts typically achieve lead times of up to 7 months and spatial resolution of around 35 kilometers (Johnson et al., 2019). Simultaneously, EO-derived information is contributing to the monitoring of hazards at different spatial resolutions, e.g., up to 25 cm in the optical domain (Denis et al., 2017). However, AA does not only require information on hazards but on the exposure and vulnerabilities of affected communities (e.g., by identifying housing types which are more prone to cyclones) to enable impact-based forecasting (International Federation of Red Cross Red Crescent Societies, 2020a). EO-derived information has the potential to support these assessments and even the transition from a static to a much more dynamic vulnerability monitoring. This includes, for instance, agricultural applications that are capable of predicting near-future yield variations (Vreugdenhil et al., 2021) or even economic wellbeing (Yeh et al., 2020) based on freely available imagery (e.g., the Sentinel programme of the European Space Agency and the European Commission).
Evaluating the impact of AA on the ground
The evaluation component focuses on critical questions about the performance of AA, such as: Did the model trigger early action at the right time in the right region? Or did the pre-agreed early action lead to the desired socioeconomic benefit? In addition to complementing the risk assessment before the manifestation of a hazard on the ground (e.g., via the identification of flood prone areas), EO-derived information could support the evaluation of early action with regard to socioeconomic benefits. Most importantly, direct and indirect evaluations need to be clearly distinguished. Direct evaluations could for instance include an assessment of the condition of fortified roofs, an early action implemented in the Philippines while the region was affected by a tropical cyclone (International Federation of Red Cross Red Crescent Societies, 2019), or an assessment whether crops were better protected from floods by digging trenches as an early action in Zambia (International Federation of Red Cross Red Crescent Societies, 2020b). Indirect evaluations might focus on the monitoring of agricultural production after the distribution of drought-tolerant seeds as an early action before drought as implemented under the UN Central Emergency Relief Fund (CERF) in Somalia (United Nations Central Emergency Response Fund, 2021). Based on such evaluations, a series of improvements could be initiated, such as the revision of early actions or the development of scenarios based on longitudinal studies to better understand counterfactuals (e.g., what is the actual impact of drought on people's livelihoods vs. what is the impact if the distribution of seeds allows them to replant?).
Learning from the evaluation/activation
As AA is still a relatively new approach, each activation and subsequent evaluation is an opportunity to learn what works and how to do it better next to generating evidence of the effectiveness (UN World Food Programme, 2021). As shown above, satellite data can support the monitoring and evaluation process, but cannot directly relate to learning. However, learning means understanding, synthesizing and communicating findings to ultimately adjust decision-making processes. Learning requires joint discussions of data and service providers, national stakeholders (e.g., meteorological agencies), and AA practitioners, which allows the integration of different expertise and thus the enhancement of AA. An institutionalized learning approach that allows learning across all stakeholders involved could rely on national and regional technical working groups that have already been established for AA, e.g., the Southern Africa Forecast-based Financing (FbF) regional technical working group (Anticipation Hub3. A general learning framework for AA exists, but EO-based MEAL is not mentioned yet (UN Office for the Coordination of Humanitarian Affairs, 2021). The obstacle is thus to sensitize members of existing technical working groups about the offer of EO for MEAL in AA as well as to facilitate a trust-based matchmaking between the EO and the AA community.
Accountability: Access and ownership
AA is designed as a primarily locally-led mechanism that relies on national risk ownership. However, in practice, the concept of “owning” risk needs to extend to the ownership of data, data-driven analyses and decision-support services to enable humanitarian assistance independently from international organizations. Ultimately, risk ownership is also closely tied to the accountability of humanitarian practitioners toward communities at risk, implementing partners, and donors. In combination with technical capacity building, freely available EO-derived information, such as rainfall estimates, that is well-calibrated and validated, consistent over space and time, and easily accessible has the potential to improve transparency, accountability and even empower communities.
In order to make EO actionable for MEAL on a local level, a certain skill set is required to bridge the gap between the EO ecosystem and communities at risk. This skill set should be centered on the capacity to reverse-engineer AA based on critical gaps in local capacities regarding specific hazards, the ability to quantify and communicate the added-value of EO in the context of MEAL, and the competence to manage expectations. More concretely, people working at the intersection of EO and MEAL need to understand the strengths and limitations of EO as well as the standard operation procedures and evaluation requirements of humanitarian organizations—a skill set that is currently evolving through cross-cutting collaboration, such as the Anticipation Hub's “EO4AA” working group.
A hypothetical use case for drought-induced food insecurity
The following scenario is based on a simplified, partly-idealized use case. We do not highlight the areas in which EO could strengthen the design or tracking of AA to concentrate exclusively on the MEAL component. The seasonal climate forecast in March predicts a weak onset of the season around May/June. Since agriculture in our region of interest is exclusively rainfed, delayed planting at the start of the rainy season often leads to crop yield deficits at the end of the season. The Early Action Protocol (EAP) is designed to trigger early action in the form of unconditional cash transfers and the distribution of drought-tolerant seeds if a predefined threshold for precipitation and/or temperature is reached. A pre-activation survey already indicated the location of communities with the highest level of vulnerability.
The onset of the season unfolds as predicted and local experts confirm that the seeds that had already been planted are not germinating due to lack of moisture. Usually, smallholder farmers in our region of interest are struggling to buy new seeds, but the EAP foresees a cash distribution of US$ 80 and 10 kg of corn seeds per household via local organizations. Most farmers decide to buy vaccines for their livestock, because they are concerned that indirect drought effects, such as diseases, increase their animals' mortality. Different satellite systems keep monitoring both the skill of the climate forecast and crop conditions throughout the season.
At the end of the season, crop cutting experiments and satellite data allow a very accurate estimation of crop production, which is estimated to be 25% below average, but crop failure could be avoided. A neighboring region that was not yet part of the EAP for drought, is facing a much more severe impact on crop production (losses of more than 50%). Based on EO-derived information, the assessment teams, which consist of international experts and local stakeholders, conclude that over 85% of farmers that had received a second round of seeds managed to grow enough maize to feed their families until the next harvest. Satellite data do not result in any direct conclusions about the use of cash. However, a commercial satellite data provider decides to make very high resolution satellite data available to count livestock (Laradji et al., 2020). These data indicate the communities that had received cash transfers lost virtually no animals, while neighboring villages had to face dramatic losses in livestock, confirming the assumption of local experts. The national working group comes together to evaluate the performance of AA during the previous agricultural season, to recap what worked and what did not, to summarize and communicate lessons learned. Despite the fact that both communities were not directly comparable regarding the characteristics of the drought impact or their vulnerability profiles, further regions will be covered by AA in the next rainy season. Ultimately, AA cost a fraction of the general food distribution that other communities had required, while allowing local stakeholders to deal with the drought shock in a more efficient and dignified way.
Discussion
The advent of the latest generation of EO satellites paved the way for all kinds of use cases that require weather-independent monitoring through RADAR, very-high spatial resolution optical imagery or a combination of various independent datasets to exploit the convergence of evidence, such as in the case of drought monitoring (Enenkel et al., 2016). So far, the performance assessment of forward-looking humanitarian activities has not been a use case. We argue that the main barriers for EO to support evidence-based MEAL are mostly not caused by technical limitations, but by uncertainties related to their added-value and a relatively high entry barrier for evaluation teams that are not used to working with EO-derived information.
What is needed are interdisciplinary approaches that connect social and physical sciences—similar to the concept of climate science translators (Enenkel and Kruczkiewicz, 2022), but covering the entire nexus of human and environmental interactions. As a consequence, we expect three interrelated technical and non-technical activities to pave the way for EO in the context of MEAL:
(1) A cross-cutting community of practice with MEAL practitioners, EO data/service providers, and AA experts; awareness raising about the availability and potential of free and commercial EO data via evidence-based use cases
(2) Standards to establish a harmonized, repeatable EO-driven MEAL process that enables the generalization of findings over time, and
(3) A long-term strategy to establish EO data-driven approaches as an assessment instrument that complements studies using socioeconomic data.
One forum for those activities are the previously mentioned technical working groups. Moreover, it will be crucial to generate the first use cases by piloting these EO-driven MEAL approaches. Here, the Anticipation Hub can support finding suitable pilots to initiate and facilitate exchanges of the AA and EO community. As AA is primarily designed as a locally-led mechanism that relies on local risk ownership, upcoming AA projects will need to proactively tackle potential bottlenecks. These bottlenecks are primarily related to establishing a truly locally-led mechanism for the design, implementation, and evaluation of AA programs. In parallel, technical capacity building and trust-building will be needed to support local stakeholders regarding the access to and operational work with EO-based assessment strategies. Hence, the relatively high entry barrier into EO technologies requires targeted capacity building for MEAL as a core element of the AA design process. EO data and services can increase the efficiency and lower the costs of MEAL, paving the way for assessment processes as a core element of the AA design process rather than just an ad-hoc task at the end of the project.
Author contributions
ME: idea, study design, and manuscript lead. KD: manuscript co-lead. CH: focus on monitoring and evaluation. SM: focus on learning and discussion. VB: focus on accountability. All authors contributed to the article and approved the submitted version.
Funding
This research was funded in part by the NASA Disasters Research Program under grant #80NSSC19K1112.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1. ^Anticipatory action, forecast-based financing and early warning/early action are used synonymously.
2. ^https://www.anticipation-hub.org/experience/evidence-database/evidence-list
3. ^https://www.anticipation-hub.org/exchange/working-groups
References
Denis, G., Claverie, A., Pasco, X., Darnis, J.-P., de Maupeou, B., Lafaye, M., et al. (2017). Towards disruptions in Earth observation? New Earth Observation systems and markets evolution: possible scenarios and impacts. Acta Astronaut. 137 415–433. doi: 10.1016/j.actaastro.2017.04.034
Enenkel, M., and Kruczkiewicz, A. (2022). The humanitarian sector needs clear job profiles for climate science translators - more than ever during a pandemic. Bull. Am. Meteorol. Soc. 103, E1088–E1097. doi: 10.1175/BAMS-D-20-0263.1
Enenkel, M., Steiner, C., Mistelbauer, T., Dorigo, W., Wagner, W., See, L., et al. (2016). A combined satellite-derived drought indicator to support humanitarian aid organizations. Remote Sens. 8, 340. doi: 10.3390/rs8040340
International Federation of Red Cross Red Crescent Societies (2019). Philippines: Typhoon Early Action Protocol Summary. Available online at: https://adore.ifrc.org/Download.aspx?FileId=381877#:~:text=ThreeEarlyActionsareconsidered,shelterstrengtheningkits(SSK) (accessed May 12, 2022).
International Federation of Red Cross Red Crescent Societies (2020a). Zambia: Floods (Early Action Protocol Summary). Available online at: https://www.google.com/url?q=https://reliefweb.int/attachments/f375048a-dc48-3ce3-bbc8-7f415f9c7a41/EAP2020ZM01_Summary.pdfandsa=Dandsource=docsandust=1652778002208632andusg=AOvVaw205C7WmhoY9GVBtQMitZJV (accessed May 12, 2022).
International Federation of Red Cross Red Crescent Societies UK Met Office UK Aid, Risk-informed Early Action, Partnership, Asia Regional Resilience to a Changing. (2020b). The Future of Forecasts: Impact-Based Forecasting for Early Action. Available online at: https://www.forecast-based-financing.org/wp-content/uploads/2020/09/Impact-based-forecasting-guide-2020.pdf (accessed May 12, 2022).
Johnson, S. J., Stockdale, T. N., Ferranti, L., Balmaseda, M. A., Molteni, F., Magnusson, L., et al. (2019). SEAS5: the new ECMWF seasonal forecast system. Geosci. Model Dev. 12 1087–1117. doi: 10.5194/gmd-12-1087-2019
Laradji, I., Rodriguez, P., Kalaitzis, F., Vazquez, D., Young, R., Davey, E., et al. (2020). Counting cows: tracking illegal cattle ranching from high-resolution satellite imagery. arXiv preprint arXiv:2011.07369. doi: 10.48550/arXiv.2011.07369
Lopez, A., Coughlan de Perez, E., Bazo, J., Suarez, P., van den Hurk, B., and van Aalst, M. (2020). Bridging forecast verification and humanitarian decisions: a valuation approach for setting up action-oriented early warnings. Weather Clim. Extrem. 27, 100167. doi: 10.1016/j.wace.2018.03.006
Risk-informed Early Action Partnership (2021). REAP Framework for Action. Available online at: https://www.early-action-reap.org/sites/default/files/2021-06/20210627_Reap_Framework.pdf (accessed March 22, 2022).
UN Office for the Coordination of Humanitarian Affairs (2021). A Learning Framework for Anticipatory Action. Available online at: https://anticipatory-action-toolkit.unocha.org/wp-content/uploads/2021/07/Learning-framework-AA-for-2021-final-draft-30-March.pdf (accessed May 13, 2022).
UN World Food Programme (2021). Monitoring and Evaluation of Anticipatory Actions for Fast and Slow-Onset Hazards Guidance and Tools for Forecast-Based Financing. Available online at: https://docs.wfp.org/api/documents/WFP-0000135356/download/?_ga=2.194488104.468843460.1652684207-476009298.1607178933 (accessed May 18, 2022).
United Nations Central Emergency Response Fund (2021). CERF Allocation for Somalia. Available online at: https://cerf.un.org/what-we-do/allocation/2022/summary/21-RR-SOM-47081 (accessed May 17, 2022).
Vreugdenhil, M., Pfeil, I., Brocca, L., Camici, S., Enenkel, M., and Wagner, W. (2021). Satellite Soil Moisture for Yield Prediction in Water Limited Regions (PICO). Available online at: https://meetingorganizer.copernicus.org/EGU21/EGU21-12549.html
Weingärtner, L., and Spencer, A. (2019). Analysing Gaps in the Humanitarian and Disaster Risk Financing Landscape. Overseas Development Institute and START Network. Available online at: https://www.anticipation-hub.org/Documents/Policy_Papers/ODI_paper_02_ANALYSING_GAPS_IN_THE_HUMANITARIAN_AND_DISASTER_RISK_FINANCING_LANDSCAPE.pdf (accessed March 29, 2022).
Keywords: earth observation, monitoring and evaluation, anticipatory action, emergency preparedness, early warning, humanitarian action
Citation: Enenkel M, Dall K, Huyck CK, McClain SN and Bell V (2022) Monitoring, evaluation, accountability, and learning (MEAL) in anticipatory action—earth observation as a game changer. Front. Clim. 4:923852. doi: 10.3389/fclim.2022.923852
Received: 19 April 2022; Accepted: 01 August 2022;
Published: 23 August 2022.
Edited by:
Clara Rodriguez Morata, Columbia University, United StatesReviewed by:
Jason MacLean, University of New Brunswick Fredericton, CanadaCopyright © 2022 Enenkel, Dall, Huyck, McClain and Bell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Markus Enenkel, menenkel@hsph.harvard.edu