Progress and Challenges of Demand-Led Co-Produced Sub-Seasonal-to-Seasonal (S2S) Climate Forecasts in Nigeria

This paper identifies fundamental issues which prevent the effective uptake of climate information services in Nigeria. We propose solutions which involve the extension of short-range (1 to 5 days) forecasts beyond that of medium-range (7 to 15 days) timescales through the operational use of current forecast data as well as improve collaboration and communication with forecast users. Using newly available data to provide seamless operational forecasts from short-term to sub-seasonal timescales, we examine evidence to determine if effective demand-led sub-seasonal-to-seasonal (S2S) climate forecasts can be co-produced. This evidence involves: itemization of forecast products delivered to stakeholders, with their development methodology; enumeration of inferences of forecast products and their influences on decisions taken by stakeholders; user-focused discussions of improvements on co-produced products; and the methods of evaluating the performance of the forecast products. We find that extending the production pipeline of short-range forecast timescales beyond the medium-range, such that the medium-range forecast timescales can be fed into existing tools for applying short-range forecasts, assisted in mitigating the risks of sub-seasonal climate variability on socio-economic activities in Nigeria. We also find that enhancing of collaboration and communication channels between the producers and the forecast product users helps to: enhance the development of user-tailored impact-based forecasts; increases users' trusts in the forecasts; and, seamlessly improves forecast evaluations. In general, these measures lead to more smooth delivery and increase in uptake of climate information services in Nigeria.

This paper identifies fundamental issues which prevent the effective uptake of climate information services in Nigeria. We propose solutions which involve the extension of short-range (1 to 5 days) forecasts beyond that of medium-range (7 to 15 days) timescales through the operational use of current forecast data as well as improve collaboration and communication with forecast users. Using newly available data to provide seamless operational forecasts from short-term to sub-seasonal timescales, we examine evidence to determine if effective demand-led sub-seasonal-to-seasonal (S2S) climate forecasts can be co-produced. This evidence involves: itemization of forecast products delivered to stakeholders, with their development methodology; enumeration of inferences of forecast products and their influences on decisions taken by stakeholders; user-focused discussions of improvements on co-produced products; and the methods of evaluating the performance of the forecast products.
We find that extending the production pipeline of short-range forecast timescales beyond the medium-range, such that the medium-range forecast timescales can be fed into existing tools for applying short-range forecasts, assisted in mitigating the risks of sub-seasonal climate variability on socio-economic activities in Nigeria. We also find that enhancing of collaboration and communication channels between the producers and the forecast product users helps to: enhance the development of user-tailored impact-based forecasts; increases users' trusts in the forecasts; and, seamlessly improves forecast evaluations. In general, these measures lead to more smooth delivery and increase in uptake of climate information services in Nigeria.

INTRODUCTION Climate Information Services and Challenges in Nigeria
The delivery of climate information services in Nigeria has come of age. Climate services evolve around the timely transfer of meteorological data and forecast products to users in several socio-economic sectors. There has been increasing growth of population and economy, as well as urbanization in the country. This has consequently resulted in an increase in the activities of different climate-sensitive sectors, such as agriculture and food security, construction, energy, health, water resources and transport. Though, these changes are largely driving the uptake of climate information services in Nigeria, there are, however, two major challenges that have been preventing the appropriate uptake of climate information services in the country.
Currently and operationally, weather forecasts in Nigeria are limited to two days in advance. Outlooks cover up to 5 days at most. Medium-range (7 to 15 days) weather and or climate forecasts are therefore not covered (Olaniyan et al., 2018), thereby making short-range (1 to 5 days) weather forecasts operationally predominant. While these short-range forecasts are useful for reactive decisions, they are less effective for long-term planning and action toward disaster risk reduction particularly in key areas such as health, food security, environment, and water resources (Omotosho and Abiodun, 2007).
Additionally, collaborations between the country's meteorological and hydrological agency and key stakeholders (i.e., forecast end-users) is weak. There have been no jointly developed user-tailored impact-based forecasts (Nkiaka et al., 2020). The end-users contribute in no way to the forecaster's operational algorithms. The outcome of the poor relationship is the end-users' inability to interpret and consume the services offered by the forecasters.
In view of the aforementioned challenges facing the delivery of climate information services in Nigeria, timely and reliable sub-seasonal-to-seasonal (S2S) climate forecast algorithms that will extend the short-range forecast timescales beyond that of medium-range timescales becomes necessary for operational purposes. At the same time, scientists and forecasters need to develop a platform that opens and utilizes communication channels with forecast end-users.
The deployment of this S2S climate forecast algorithm, that extends the short-range forecast timescales beyond that of medium-range timescales, may help in mitigating the risks of sub-seasonal climate variability on socio-economic activities in Nigeria. Stakeholders' platform for collaboration will enhance the joint development of user-tailored impact-based forecasts; increase users' trusts in the forecasts; and, seamlessly help in the evaluation of the performance of the forecasts. This paper aims to document the process of co-producing new forecast products to improve the resilience of climate-sensitive sectors in Nigeria. The purpose is to outline this process and identify outstanding challenges in order to guide future efforts in climate service development in West Africa in general.
This study comprises of three sections. While this section introduces the concept of the paper by highlighting the challenges faced by climate information services in Nigeria, the subsequent sub-sections briefly describe the climate-induced problems in Nigeria (section Climatic problems in Nigeria), activities of the Global Challenges Research Fund African Science Weather Information for Forecasting Techniques (GCRF African-SWIFT; https://africanswift.org/) project as it relates to the concept of this paper (section The GCRF African-SWIFT project), and the S2S datasets that are utilized to jointly produce new forecast products that are operationally being issued to end-users (section Real-time state-of-the-art S2S climate forecast datasets). Analysis of demand-led co-produced forecast products analyses the demand-led co-produced forecast products in terms of the decisions taken by the stakeholders, improvement in the forecast products from the stakeholders' perspectives, feedbacks from stakeholders and the performance evaluation of the forecast products. The section will also discuss the development of a range of prototype products in response to the users' needs. Some of the decisions taken by the users based on these forecasts and the outcomes of those decisions will be discussed in sub-section decisions taken by the stakeholders based on the co-produced forecast products. While the refinement of the forecasts and the development of new forecasts based on the feedback from users are discussed in sub-section improvement in the forecast products from the stakeholders' perspectives, sub-section performance evaluation of the forecast product(s) discusses the process of evaluation of the forecasts. Conclusions from the progress so far, identification of remaining challenges and recommendations are presented in section conclusions.

Climatic Problems in Nigeria
The majority of the population in Nigeria consists of small-holder farmers whose economy and livelihood depend on small-scale, rain-fed agriculture. The country is therefore highly vulnerable to large variations in diurnal, monthly, seasonal and interannual weather and climate events (Gbobaniyi et al., 2013). These often include extreme events such as severe and squally thunderstorms (Omotosho and Abiodun, 2007;Oguntunde et al., 2014), boreal spring / summer heat waves (Ragatoa et al., 2018;Lawal et al., 2019), boreal winter dust storms that dangerously impair horizontal and vertical visibilities (Marais et al., 2014), excessive dryness and drought (Kasei et al., 2010;Oguntunde et al., 2014;Ayanlade et al., 2018;Gbode et al., 2019;Ajayi and Ilori, 2020), and heavy rainfall and urban flooding (Adekola and Lamond, 2018). All these extreme climatic events are projected to occur more frequently in future due to global warming (IPCC (Inter-governmental Panel on Climate Change)., 2014; Zhao et al., 2017;Klutse et al., 2018). The frequent occurrences of extreme weather events are likely to lead to more losses in agriculture, which is the mainstay of the nation's economy. Therefore, a combination of reliable and timely S2S climate forecasts tailored toward stakeholders' demand would go a long way to mitigate the impacts of climate-induced problems on socio-economic activities.

The GCRF African-SWIFT Project
The GCRF African-SWIFT project has been funding and hosting a series of operational weather forecasting "testbeds" in Africa. Four countries, Senegal, Ghana, Nigeria, and Kenya, are participating in this project through their national meteorological establishments and academic institutions. The testbeds have been bringing together researchers and forecasters, as well as forecast users from these participating African countries (Roberts et al., 2021). Operational forecasters work in tandem with stakeholders at the testbeds and the reports of their work are documented and archived internally.
Forecasters and stakeholders utilize real-time predictions from state-of-the-art forecasting models to co-produce new and user-tailored sub-seasonal climate forecast products. These products are designed to aid decision-making in sectors such as health, food security, agriculture, environment, water resources, disaster risk reduction and energy. The testbeds mimic similar successful testbeds that have been held in the United States of America in the past (Ralph et al., 2013). The only difference here is that pilot products are continually being used, feedbacks from users encouraged, and forecast performances evaluated in real time to improve their practical application for decisionmaking (Carter et al., 2019;Hirons et al., 2021). The testbeds therefore have become avenues for real-time sub-seasonal climate forecast products.

Real-Time State-Of-The-Art S2S Climate Forecast Datasets
The GCRF African-SWIFT project, being one of the 16 projects taking part in the World Meteorological Organization (WMO) S2S Prediction Project Real-time Pilot Initiative (http:// s2sprediction.net/), provides some real-time S2S climate forecast datasets to national meteorological services of the participating countries. The S2S Prediction Project is a joint initiative of the World Weather Research Programme (WWRP) and the World Climate Research Research Programme (WCRP).
Accessing and utilizing the S2S climate forecast datasets is as a follow up to the spirit of the testbeds as agreed by the participating countries. Nigerian Meteorological Agency (NiMet; www.nimet.gov.ng) is one of the participants and beneficiaries.
These S2S climate forecast datasets, used in this study, are from the state-of-the-art generations of models from (though not limited or restricted to) the European Centre for Medium-range Weather Forecasts (ECMWF) seasonal forecasts System 4 (Vitart et al., 2012). On weekly temporal scale, GCRF African-SWIFT project provides daily forecasts from the ECMWF for 1 to 46 days in advance (Vitart et al., 2017). General configurations of the ECMWF-S2S forecast datasets have been described by Vitart et al. (2017).
Performances of the ECMWF-S2S, over Nigeria, have been evaluated by Olaniyan et al. (2018Olaniyan et al. ( , 2019. In addition, Vellinga et al. (2013) and Batté et al. (2018) have also shown that the ECMWF-S2S climate forecast has skill in representing weather and sub-seasonal climate forecasts over Nigeria's neighboring countries. Furthermore, de Andrade et al. (2021) who utilized several verification metrics to assess the ECMWF-S2S' weekly precipitation forecast quality at lead times of one to four weeks ahead (weeks 1-4) during different seasons over Africa, and in comparison to other models, found that predictions from the ECMWF-S2S model are more skillful than those from other models, especially in the first two weeks. de Andrade et al. (2021) also found that the ECMWF-S2S model's forecast quality is linked to the strength of climate drivers, vis-à-vis teleconnections such as Indian Ocean dipole, El Niño-Southern Oscillation, and the Madden-Julian oscillation. ECMWF-S2S dataset has 51 ensemble members thereby allowing analyses of the range of possible climate events, uncertainties in climate events and probabilities of occurrence of climate events across all meteorological variables. Operationally, availability of the ECMWF-S2S forecast data offers new opportunities to enhance the delivery of weather and climate services in Nigeria.
In the spirit of the testbeds, the real-time ECMWF-S2S forecast datasets are being routinely analyzed to develop some sub-seasonal climate forecast products in conjunction with some sub-set of forecast users, i.e., the stakeholders. These stakeholders are from weather-sensitive economic sectors for whom subseasonal climate information would aid there planning and decisions. The co-produced forecast products are then utilized by stakeholders to take numerous decisions that prevent losses, both materially and financially, in various sectors of the economy. Timely provision of these sub-seasonal climate forecasts constitute parts of early warning systems that afford Nigerians sufficient time to mitigate against severe weather events that normally hinder socio-economic activities (Nkiaka et al., 2019(Nkiaka et al., , 2020. It is appropriate to note that these co-produced subseasonal climate forecast products are not daily forecasts, but an average over arrays of days, e.g., weekly, dekadal, etc., that will seemingly extend and improve the existing short-range forecast timescales. For clarity, the co-produced sub-seasonal climate forecast products will indicate if an extremely wet or dry period (week or dekad) is coming but not exactly how much rain will fall on a particular day. The forecasts will also be able to indicate the anticipation of extremely hot or cold periods but not the exact magnitude of hotness or coldness of a particular coming day.

ANALYSIS OF DEMAND-LED CO-PRODUCED FORECAST PRODUCTS
Mimicking series of testbeds involving the forecast producers and the stakeholders, of which the modus-operandi and the applications of their reports have been documented by Nkiaka et al. (2020) and Roberts et al. (2021), this section discusses the user demand for forecast information on sub-seasonal timescales to inform decision making in a range of sectors. The section also discusses the development of a range of prototype products in response to the users' needs. Here, all the direct and derived subseasonal climate products that were operationally given out in the form of forecasts to stakeholders are presented on Table 1. There are statistical descriptions as well as statistical metric of each forecast product. The reason(s) each product were issued to stakeholders are also displayed on the table. Periods within the year, in terms of months, peculiar to each forecast product are also included. To depict regions that may experience heavier amount of rainfall accumulation within a dekad. A flood alert is released if this persist for more than a dekad over a region. Issued June to September.

Cumulative dekadal rainfall
Total amount of rainfall within a dekad, i.e., total amount for each 10 days in each month or for each 10 days of the forecast periods.
To depict total amount of rainfall, that may be experienced over a region, for each 10 days of the forecast periods, or specifically for each 10 days of a month. This assists farmers to make irrigation decision based on their determined thresholds. Issued throughout the rainy seasons (April to October) to monitor rainfall performance.
mm Precipitation anomaly Variability of a climatic parameter defined as its deviations from its long-term average for a specified period of time, i.e., anomaly of a climatic parameter = projected amount of a climatic parameter during the specified period minus the climatology of that climatic parameter for the specified period.
To depict regions projected to have normal, below normal or above normal amount of rainfall. Persistence of certain category helps to determine areas that are likely to experience business as usual, drought or flood, respectively, depending on the degrees of deviations from the climatology. Issued throughout the rainy seasons (April to October). mm Temperature anomaly for maximum and minimum near-surface temperatures To depict regions projected to have normal, below normal or above normal near-surface temperatures. Persistence of certain category helps to determine areas that are likely to experience business as usual, cold or heat waves, respectively, depending on the degrees of deviations from the climatology. Issued throughout the year. o C Top 20 cm soil moisture anomaly To depict regions projected to have normal, below normal or above normal moisture within the top 20 cm of the surface soil. Persistence of certain category helps to determine areas that are likely to experience business as usual, dryness or wetness, respectively, depending on the degrees of deviations from the climatology. This also assists farmers in monitoring drought impacts on agriculture and crop production hence make irrigation decision based on their determined thresholds. Issued throughout the year to monitor the moisture conditions at the top 20 cm of the surface soil.
Kgm −3 or % Frontiers in Climate | www.frontiersin.org Stakeholders are given the opportunity to describe their idea of meteorological parameters that would aid or suit their operational algorithms at weekly meetings between the forecast producers and users. Forecasters will then try to turn these ideas into tangible localized forecast products, as presented in Table 1, using available datasets. This method allows forecast users to contribute effectively to the forecasters' operational algorithms, thereby finding it easier to interpret and consume the services offered by the forecasters.
NiMet has a forecasting mandate, conferred on it by the WMO, to support other West African nations (e.g., Liberia, Sierra Leone). To give credence to this claim an extract from sub-section decisions taken by the stakeholders based on the co-produced forecast products of WMO report on "Request for WMO Expert Assistance to Improve NWP at NiMet" reference number 01918/2019-DRA-AFLDC is quoted below: "Following a quick tour de table introduction, the Director General of NiMet and also the Permanent Representative (PR) of Nigeria with WMO welcomed the team to Abuja. He expressed, in the following terms, the reasons why he requested the assistance of WMO to assist with the assessment of the NiMet NWP program: there is a strong desire to modernize and improve the capacity of the NWP program, not only for the provision of better services to Nigerians, but also to neighboring countries which require some assistance in the area of training and forecasting services, like Sierra Leone and Liberia which are currently supported by the NiMet".
This explains the reasons some figures, for instance Figures 1-3, are plotted over West Africa, and not over Nigeria alone.

Decisions Taken by the Stakeholders Based on the Co-Produced Forecast Products
Based on the reports from the testbed meetings we found that numerous decisions were taken by stakeholders based on the co-produced forecast products. On a general note, the forecast products were utilized to plan and take decisions that prevent losses in terms of lives, property, and finances. Some of the instances relating to some Nigerian socio-economic sectors are discussed here.

Health Sector
Health workers normally look out for areas where low surface relative humidity (≤ 30%; Figure 1A) coincides with high nearsurface temperature (≥ 38 • C; Figure 1B). They use this to determine the likely areas of meningitis epidemics (Molesworth, 2003;Mera et al., 2014). Areas of high near-surface temperature (≥ 38 • C) are also used as an indicator for the migration of tsetse flies. These flies are notoriously known for blood sucking and transmission of African trypanosomiasis, commonly referred to as sleeping sickness, across the country (Ford and Katondo, 1977;Rogers et al., 1996). Health sector stakeholders are interested in these specific diseases, as well as other health-related issues that are traceable to high near-surface temperatures, e.g., dehydration that stresses the human vascular systems (Flynn et al., 2005;Thornton, 2010;Lim et al., 2015); malaria epidemics (common FIGURE 1 | Samples of co-produced probabilities (x100%) of (A) surface relative humidity ≤ 30%; (B) near-surface maximum temperature ≥ 38 • C; and, (C) dekadal rainfall accumulation ≥ 25 mm but < 50 mm, produced from the ECMWF-S2S base forecast of March 16, 2020 and valid from April 5, 2020 to April 14, 2020. Probability of, for instance, 0.8 on the scale = 80%. but deadly disease caused by protozoan parasites that are spread through anopheles mosquitoes' bites: Sachs and Malaney, 2002;Klutse et al., 2014); etc. Warnings and advisories are sent out to members of the public the moment health-threatening (userdefined) thresholds are detected. Preparations for health-related emergencies then commence.

Natural Disaster and Humanitarian Sector
Natural disaster and humanitarian emergency managers utilize rainfall thresholds such as the probability of dekadal rainfall accumulation ≥ 50 mm but < 100 mm (e.g., Figure 1C) to warn valley dwellers of the possibility of flooding (Adekola and Lamond, 2018). With such forecasts, humanitarian managers will be prompted to relocate the would-be-affected people to higher dry lands or reinforce shelters and riverbanks. Usually this requires coordination with health workers, as well as make food available in order to mitigate all the negative effects that will come with the anticipated flood. One example of the successes of such warnings was a flood event that occurred in July 2020 on the outskirts of Abuja, the capital of Nigeria (https://africanswift.org/2020/10/19/thescience-behind-abujas-summer-flood/). One of the S2S climate forecasts released in July 2020 made a statement that "widespread flooding is possible and should be expected, especially over the northwestern, central and the Niger Delta areas. Agricultural and human losses, over the low lands and river basins, are not ruled out", and that "the probability of rainfall exceeding 50 mm was 100%. This was expected to be about 15 mm above normal, coupled with the anticipation of excessive soil moisture that would likely aid run-off." Specifically, the forecast declared that Abuja should expect rainfall accumulation of about 75-100 mm within 20-29 July 2020, the last dekad in July 2020. However, some damages to lives and property were still recorded despite the forecast being s-ent to an appropriate stakeholder to warn people ahead of time.
Another example is in August 2020, timely release of a similar forecast saved numerous lives and property worth billions FIGURE 3 | Sample of a co-produced anticipated cumulative dekadal rainfall amount (mm) produced from the ECMWF-S2S base forecast of July 20, 2020. of dollars in Bama, a city in the northeastern part of the country (https://reliefweb.int/disaster/fl-2020-000196-nga).

Agricultural Sector
While livestock farmers are interested in the absolute values of surface relative humidity and temperatures to determine and avert the negative effects of heat stress and thermal discomfort indices (Sylla et al., 2018); crop farmers are more interested in the anomalies of soil moisture, rainfall accumulation and near-surface temperature (Figures 2A-C). Crop farmers use the combinations of these three anomalies to quantify various degrees of agricultural drought (Luo et al., 2017;Ayanlade et al., 2018). Crop farmers without irrigation infrastructures take decisions to expand their farmlands if rainfall deficits are projected to occur repeatedly. This helps them to compensate for loss in production (Zaveri et al., 2020). Crop farmers utilize absolute values of soil moisture to determine, based on threshold, the water intake rate of the topsoil and/or saturation, which may consequently lead to excessive run-off, thereby resulting in flood and top-soil erosion (Abdulfatai et al., 2014). They also utilize rainfall accumulation forecasts (Figure 3) to take decisions relating to water supply, flood control, and potential evaporation. With rainfall being the only source of free water input into agricultural operations, and that its future changes remain likely uncertain (Monerie et al., 2017), farmers are concerned about the likely future increase in potential evaporation due primarily to an enhanced warming of the climate (Zougmoré et al., 2016).

General Public and Insurance
Insurance companies benefit from the co-produced forecasts as it assists them in the operational task of preparing for major pay-outs. This is true for individual members of the public, as well as corporate socio-economic sectors, as they are beginning to embrace weather-index-based insurance. Weatherindex-based insurance provides a safety net to the exposed individuals, as well as fragile weather dependent socio-economic sectors (Collier et al., 2009). Weather-index-based insurance is gradually becoming essential as a risk mitigation strategy that individuals and weather dependent socio-economic sectors may use to mitigate adverse climate shocks and natural disasters encountered during social and economic operations (Fonta et al., 2018).

Water Resources and Construction Sectors
Dam, power and construction managers utilize rainfall accumulation forecasts (Figure 3) to take decisions relating to water supply, electricity generation, flood control, recreation, irrigation, etc. Rainfall accumulation forecasts are used to compute runoff / streamflow as a proxy for water availability because runoff is closely related to changes in precipitation (Roudier et al., 2014). Runoff is harvested and stored as water in dams. In Nigeria, availability of power supply has been linked to water level in dams because a greater percentage of electricity is generated from hydro-power, when compared to other sources of power generation. Construction managers rely on rainfall accumulation forecasts to decide whether to move heavy equipment to muddy sites.

Improvement in the Forecast Products From the Stakeholders' Perspectives
Continuous interactions with several stakeholders indicate that improvement in the co-produced forecast products, from the stakeholders' points of view, are of four standpoints. 1. graphics of presentation of the forecast products; 2. understanding and simplicity of language of presentation; 3. local / point specific (instead of spatial) presentation of forecast products on daily timescale; and 4. the inclusion of specific tailored forecast products.
Stakeholders are continuously requesting for forecast products to be graphically presented in a simplified and comprehensive manner. These requests are, however, complex in nature. This is because there is no consensus among the stakeholders on the specificity of the graphic design of the forecast products. Forecast producers (in this case NiMet, inspired by the WMO's guidelines on graphical presentations of public weather services products-https://library.wmo.int/ doc_num.php?explnum_id=9190), therefore, have to group stakeholders according to similarity in their presentation preferences due to resource constraints.
Nigeria is a diverse nation with more than 250 ethnic groups, each having their dominant languages spoken within their territories (https://www.britannica.com/place/ Nigeria/Languages). Fortunately, a substantial fraction of the stakeholders are highly literate; however, the majority of them, especially the privately owned socio-economic quarters, would prefer that forecast products be presented in certain local languages, instead of in English which is the nationally spoken language. Furthermore, stakeholders prefer that forecast presentations should be devoid of scientific terminologies which may be difficult for them to understand, regardless of the language used. Forecasters are therefore undergoing training, designed by NiMet, in "operational" writing for the benefits of non-scientists. As with graphical presentation, resource constraints impose a strong limit how much the linguistic preferences can be satisfied.
Stakeholders tend to prefer that forecast products are presented on a daily timescale for a local or specific coordinate (e.g., Figure 4), instead of spatial, weekly or dekadal presentation. They reason that this will afford them the opportunity to be aware of the likelihood of some events well in advance and thus be prepared. However, this can be unwieldy. For example, there are 774 autonomous local councils in Nigeria. Assuming that we have say 20 stakeholders in each local council and we are to produce a sample of, say, Figure 4 for about 10 meteorological variables, per stakeholder each for a different locality, this will generate more than 1,50,000 graphics of different images altogether. Unfortunately, such demands have come to show that some stakeholders are yet to fully grasp what information an S2S model can accurately produce. Current S2S forecasts are able to reliably forecast the state of the atmosphere averaged over a period of days, e.g., weekly, dekadal, etc. and are currently not capable of determining exact daily rainfall totals over a very specific local region. This mismatch between S2S forecast capability and specific user needs will require significant resource and communication to overcome effectively.
On the last standpoint of further improvement, stakeholders would like some new forecast products to be included. Specifically, health sector stakeholders require maps of likely areas of meningitis and malaria outbreaks. Livestock farmers are interested in the graphics that will depict heat stress and thermal discomfort indices. Crop farmers will appreciate information that directly contain drought and flood prone areas, and potential evapotranspiration. Though, production of these parameters are likely to take some liabilities off the shoulders of the stakeholders, it will however require significant resource from the forecast producers to do these, as well as over-concentrate productions in the hands of forecast producers. In this case, increase in computational resource and training of skilled forecast producers needs to be considered by the forecast producers. As forecasts are only useful if they are reliable, therefore, performance evaluation of these new products is required and may also delay the reliable production and roll-out of these much-needed parameters.

Performance Evaluation of the Forecast Product(S)
Performance evaluations of the forecast products are carried out using the following four strategies-−1. monitoring of extreme weather events reported by electronics and print media in the country; 2. eye-witness narrations of extreme weather events; 3. comparisons of the forecast products to weather observation reports from NiMet's weather observatories; and 4. utilization of some quantitative statistical metrics documented by research outputs.
Extreme weather events reported by electronics and print media in Nigeria (from January 2020 to September 2020) were collected, compiled, and collated. Enumerations of the events as reported by the media houses indicate that the model (i.e., the forecast products) captured about 80% of severe weather events over the country. While there was some damage to lives and property, the media houses also reported that many lives and properties were saved by the timely release of the forecasts.
Eye-witness reports are another way of evaluating the performance of the co-produced forecast products. They are in the form of feedbacks from the forecast product users. Official letters of acknowledgment and appreciation, sent to NiMet, by some stakeholders also formed parts of the performance evaluation. Eye-witness reports could also be in the form of news and publications from either the print or electronic media, as they typically report any extreme weather event, which has been predicted some days or weeks earlier, as it affects people and or certain communities, along with the trails of devastations it leaves behind. Two examples of such events (e.g., https://africanswift. org/2020/10/19/the-science-behind-abujas-summer-flood/ and FIGURE 4 | A sample of likely anticipated daily precipitation amount (mm) over Ibadan (latitude 7.38 • N, longitude 3.95 • E; days from July 7 to August, 2020), produced from the ECMWF-S2S base forecast of July 6, 2020. Ibadan is a city that normally experience the famous little dry season (i.e., August-break; Olaniyan et al., 2018) from end of July through August every year. Top and bottom of the bars indicate the likely maximum and minimum precipitation as predicted, respectively, by at least one ensemble member in each case.
FIGURE 5 | Spatio-temporal correlation between daily near-surface mean temperature values from the ECMWF-S2S forecasts and the gridded observations over Nigeria.
https://reliefweb.int/disaster/fl-2020-000196-nga) were earlier mentioned in sub-section natural disaster and humanitarian sector of this paper.
Furthermore, some works have documented the skill and reliability of the ECMWF-S2S forecasts, e.g., Tompkins and Feudale (2010), Vellinga et al. (2013), Batté et al. (2018), Olaniyan et al. (2018), de Andrade et al. (2021, etc. They found that ECMWF-S2S climate forecast data has skill in representing weather and sub-seasonal climate simulations and forecasts over Nigeria, and where applicable over West Africa. de Andrade et al. (2021), using several verification metrics, found that predictions from the ECMWF-S2S model are more skillful than those from other models, especially in the first two weeks. They also found that the model's forecast quality is linked to the strength of climate drivers, vis-à-vis teleconnections such as Indian Ocean dipole, El Niño-Southern Oscillation, and the Madden-Julian oscillation.
It is also found that the forecast products contain reliable information when the forecast products are compared to weather observations over the country, using some quantitative statistical metrics. For example, there is moderately positive linear association between the near-surface temperature forecasts and the real-time observations. The spatio-temporal correlation between daily near-surface mean temperature values from ECMWF-S2S forecasts and the gridded observations, over Nigeria, yielded values that are generally > 0.5 (Figure 5). Also, for a particular precipitation threshold, ensemble spread, and skill were determined using the fractions skill score (FSS; Skok and Roberts, 2016). Here, FSS is able to give a direct measure of the error in the placement of the precipitation (figure not shown). FSS over the country is able to indicate that the ECMWF-S2S precipitation forecasts are potentially useful by returning values that are > 0.5.

CONCLUSIONS
Under the status quo in Nigeria, decision-makers in weathersensitive sectors only had short-term weather and climate information available to them. This resulted in a void of information for timely, longer-term planning decisions or early warning of extreme weather and climate events. Having timely, reliable and useful information on these longer timescales would go toward building a more resilient society and economy in Nigeria. Through the GCRF African-SWIFT S2S forecasting testbed, new S2S climate forecast products have been co-developed with forecast users and forecast producers in Nigeria in a way not seen before. This paper has examined the steps taken to achieve this and some of the challenges encountered. Dike et al. (2018) itemized some challenges facing African climate scientists. These challenges, by extension, also affect the operational capabilities of some African meteorological institutions. Operationally, there is a disconnection between the forecast producers and the users of forecast products. This has been having negative effects on climate information services in Africa, vis-à-vis, Nigeria. This huge gap has been filled to some extent, by an operational weather forecasting 'testbeds' as part of the GCRF African-SWIFT project. The testbeds saw previously unestablished collaborations between forecast producers and forecast users developing. By developing a joint understanding both were able to work in tandem for the benefits of the socioeconomic sectors of their respective countries. Though, the intervention provided by the GCRF African-SWIFT project has been able to cut down some effects of the challenges; some issues, such as communication, however, remain unresolved.
Communication remains a considerable challenge. For example, forecast users request products in "operational" writing devoid of scientific terminologies which may be difficult for the stakeholders to understand. However, further effort is required to better train forecast producers in the art of "operational" writing, something which will take time and resource. The challenge does not end there, the very fact that forecasts are required to be interpreted into more than 200 local languages presents an almost unsurmountable communication challenge.
Furthermore, the mismatch that has become apparent between what some users request (e.g., daily rainfall totals weeks in advance in a specific location), and what S2S forecast models are capable of accurately producing (e.g., weekly averaged likelihood of wet/dry conditions over a region) also highlights the need for better communication. In agreement with existing studies (e.g., Lemos et al., 2018), it is clear from the evidence presented here that there is considerable merit in applying resource to understanding and improving the co-production "process", as well as it's intended "products" or "outcomes".
It is believed that the barriers preventing the smooth delivery of climate information services in Nigeria will be surmounted, by continuous adherence to steps analyzed in this paper. The end result will be that of mutual benefits to both the forecast producers, and the forecast users. Forecast users will continue to have access to timely and reliable climate services in their desired formats while forecast producers will continue to understand the applications of atmospheric science as it evolves round numerous socio-economic sectors.
Two things have been achieved here: one, short-range forecast timescales have been extended beyond the mediumrange forecast timescales by over 40 days; and, two, collaboration and communication channels have been opened between the forecast producers and the forecast users.
Socio-economic planning and productions in health, food security, agriculture, environment, water resources, etc. have become more effective than before as weather forecasts and outlooks become more reliably extended. Users now contribute effectively to the forecasters' operational algorithms, thereby finding it easier to interpret and consume the services offered by the forecasters. The first achievement helps in mitigating the risks of sub-seasonal climate variability on socioeconomic activities in Nigeria. The second achievement helps to enhance the development of user-tailored impact-based forecasts; increase users' trusts in the forecasts; and, seamlessly help in the evaluation of the performance of the forecasts. Overall, smooth delivery of climate information service has been substantially enhanced.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

AUTHOR CONTRIBUTIONS
All authors have approved this work for publication listed having made a substantial direct and intellectual contribution to the work in the order of activities listed against their names as follow: KL conceptualization, investigation, methodology, validation, resources, writing (original draft, review & editing), visualization, supervision, software, formal analysis, data processing. EO methodology, software, formal analysis, resources, data processing, visualization. II, SO, VA, ECO, VI, RN, and OF investigation. LH conceptualization, investigation, methodology, resources, writing (review & editing), project administration. ET and VB investigation, resources. JT, IG, and DS conceptualization, investigation, methodology, writing (review & editing). EA and JO investigation, funding acquisition. LY investigation, resources, funding acquisition. EN conceptualization, investigation, resources. MN investigation, resources, supervision, funding acquisition. SW conceptualization, investigation, methodology, validation, resources, formal analysis, writing (review & editing), visualization, funding acquisition, project administration.

ACKNOWLEDGMENTS
This work was supported with grants from the following projects: U.K. Research and Innovation as part of the GCRF African-SWIFT (work packages R2, R5 and R6) program-grant number NE/P021077/1; BNP Attribution Project of the African Climate and Development Initiative (ACDI: www.acdi.uct.ac.za) of the University of Cape Town, South Africa; and, the 2019 National Research Fund Intervention (TETFund/DR&D/CE/NRF/STI/73/VOL.1) of the Tertiary Education Trust Fund (TETFUND) of Nigeria. The realtime data used in this study has been provided through the S2S Prediction Project Real-time Pilot. The S2S Prediction Project is a joint initiative of the WWRP and the WCRP.