We compare the COSMO EPS precipitation forecasts with the deterministic forecasts of its global driving model ICON, the deterministic forecasts of the European Centre for Medium-Range Weather Forecasts (ECMWF), and the ensemble forecasts of two GRAPES (Global and Regional Assimilation and PrEdiction System) EPSs, which are running operationally at China Meteorological Administration (CMA) (for details, see Table 4).

The GRAPES model has a fully compressible dynamical core based on the non-hydrostatic approximation, a semi-implicit and semi-Lagrangian scheme for time integration, and a height-based terrain-following sigma coordinate system (Chen et al., 2019). GRAPES mesoscale EPS (GRAPES MEPS) has 15 ensemble members covering all regions of China, for which the initial conditions and lateral boundary conditions are downscaled from the corresponding members of the GRAPES global EPS (GRAPES GEPS; Zhang et al., 2014; Xia et al., 2019). GRAPES GEPS uses gridded statistical interpolation (GSI) for the operational data assimilation, while ensemble data assimilation (EDA) is employed by ECMWF. The global model ICON running operationally at DWD adopts a hybrid method with an assimilation cycle of 3 h, which consists of an ensemble transform Kalman filter (ETKF) coupled with a three-dimensional variational analysis (3D-VAR).

The global precipitation measurement (GPM) with a horizontal resolution of 0.1° × 0.1° provided by the new generation of precipitation observation satellites is used as the observation. The GPM satellite was launched on 27 February 2017. It inherits and improves the algorithm and the detection technology of the TRMM (Tropical Rainfall Measuring Mission) satellite. The dual-frequency radar observation system is first adopted to provide precipitation data with higher spatiotemporal resolution over a wider spatial area, contributing to more accurate descriptions of the precipitation distribution (Liu et al., 2017; Huffman et al., 2019). All forecast data are regridded to the observation grid.

In addition, the estimated precipitation rates derived from ten S-band Doppler radars covering the Guangdong Province via classic Z–R relations are used for latent heat nudging (LHN) experiments, which are further discussed in Section 3.3. The average separation of these radars is 100–150 km, and the maximum observed range is 230 km.

The spatial distribution of observed precipitation is compared with different model forecasts in Figure 2. For an EPS such as COSMO, the precipitation distribution is represented by its ensemble median forecast in Figure 2. The precipitation caused by the extreme storm event that accumulated over the period 27–31 August 2018 is mainly concentrated in the southeast coastal areas and gradually decreases from southeast to northwest. Only COSMO EPS and ECMWF predict the heavy precipitation over the eastern coastal regions where the 5-day total precipitation exceeds 250 mm, but the location of the heavy precipitation in both of them exhibits a larger eastward bias than the observations. The coverage area of heavy precipitation predicted by ECMWF is relatively small, while COSMO EPS’ prediction is closer to the reality. However, COSMO EPS also predicts a second heavy precipitation center in the northern inland, which is not observed. In other regions of Guangdong Province, the precipitation predicted by COSMO EPS is much weaker than the observations, especially in the southern Pearl River Delta. ICON and GRAPES MEPS both underestimate the precipitation in eastern Guangdong while predicting well in the central and northern inland regions. GRAPES GEPS overforecasts precipitation in most regions of Guangdong. The corresponding gridpoint-based scores of different model forecasts are consistent with the visual appearance (not shown). Although ICON and GRAPES MEPS have the smallest MAE and largest PCC values, their ETS values for heavy precipitation above 250 mm are almost zero because they both underforecast the extreme precipitation. In contrast, COSMO EPS obtains the largest MAE and smallest PCC values, but it performs best for heavy precipitation exceeding 250 mm by showing the highest ETS value among all models.

The occurrence frequency of daily precipitation according to its amount as quantified by the cumulative distribution function (CDF) varies considerably between model forecasts and observations (Figure 3). During the period of 27–31 August 2018, the CDFs of GRAPES MEPS and GRAPES GEPS increased rapidly with the increase of the precipitation amount, which are quite different from the observed distributions. The CDFs will reach 1 when the daily precipitation predicted by GRAPES MEPS is just around 40–60 mm, and GRAPES GEPS forecasts are generally below 80 mm. This phenomenon reflects the fact that both GRAPES models (including regional and global EPS) underpredict this extreme rainstorm event in Guangdong. ICON and ECMWF perform similarly, with both models underestimating the occurrence of the lower precipitation amounts and overestimating the occurrence of precipitation amounts above 60 mm, especially on 29 and 30 August 2018. Compared with these four models, the CDF curves of COSMO EPS are much closer to the observations, except for 31 August 2018. Especially for higher precipitation amounts, COSMO EPS CDFs almost coincide with the observed CDFs, but it overestimates the occurrence of precipitation below 40 mm. Visually and also quantified by the non-parametric two-sample Kolmogorov–Smirnov test, which does not require restrictive assumptions about the distribution of the test statistic (Orskaug et al., 2011), COSMO EPS reproduces the observed CDF best. But none of the model CDFs—including the one produced by COSMO EPS—stands the test for similarity with the observations (not shown). The gridpoint-based statistics (i.e., MAE, PCC, and ETS) for daily precipitation verification are shown in Figures 5A–D. COSMO EPS performs again only below average compared to the other models in terms of MAE and PCC, but shows the best forecasts for the daily heavy precipitation above 50 mm during 27–29 August 2018. GRAPES GEPS performs well for the heavy precipitation that occurred on 30 and 31 August 2018. ICON and GRAPES GEPS exhibit relatively high performance for daily precipitation above 25 mm, but have almost no forecasting skill for heavy precipitation.

To focus on the heavy precipitation, Figure 4 only displays the daily precipitation objects with intensity exceeding 25 and 50 mm identified by the MODE. For the object identification, the precipitation field is spatially smoothed with a convolution radius of four grid points in our evaluations in order to account for an effective resolution of a model being about seven grid points (Skamarock 2004). The quantification of similarities between the forecast objects and the observed objects by OTS metric is shown in Figures 5E,F. The precipitation on 27 August 2018 is in the early stage of development. The observed areas with daily precipitation above 25 mm are mainly located in northern, western, and eastern Guangdong. Only ECMWF predicts the observed rain belt in northern Guangdong, contributing to its highest OTS value. COSMO EPS predicts heavy precipitation over the east coast, consistent with the observations, but with stronger intensities. The other three models all underpredict the precipitation. On 28 August, the heavy precipitation gradually concentrated in the Pearl River Delta. The precipitation distribution predicted by ICON is the one most similar to the observations. COSMO EPS predicts a wide range of heavy precipitation in coastal areas, most of which are not observed. The coverage area of heavy precipitation expands significantly on 29 August and is located in the coastal regions with the rain belt northeast–southwest oriented. To a certain extent, the heavy precipitation in eastern Guangdong has been predicted by COSMO EPS and ECMWF. In contrast, both ICON and GRAPES MEPS underestimate the precipitation above 50 mm but predict well the precipitation above 25 mm. Therefore, the former two models obtain the highest OTS values for the threshold of 50 mm, and the latter two models have the highest OTS values for the threshold of 25 mm. On 30 August, the heavy precipitation moves to eastern Guangdong. The distributions of heavy precipitation above 50 mm predicted by COSMO EPS and GRAPES GEPS are similar. Their forecast coverage area of heavy precipitation in eastern Guangdong is much smaller than the observations, and heavy precipitation predicted in other regions has not actually happened. The coverage area and intensity of heavy precipitation decrease on 31 August, which is mainly located in Heyuan city. All five models’ precipitation distributions are different from the observations. COSMO EPS and GRAPES GEPS predict some regions in which heavy precipitation occurs, but with incorrect locations.

In order to obtain multiple meteorological variables such as pressure, temperature, specific humidity, relative humidity, and wind, the ECMWF reanalysis data (ECMWF Reanalysis v5, ERA5) are used as the observations in this section. First, the large-scale circulation background during the occurrence and development of this extreme rainstorm event is analyzed, as shown in Figure 6. During the whole period, the South Asia high (200 hPa) is continuously and stably located over the Tibetan Plateau, and divergent airflow persists over South China. The middle and high latitudes of the middle troposphere (500 hPa) are controlled by the circulation of two troughs and one ridge. The high-pressure ridge is located on the west side of Lake Baikal and moves slowly. The western Pacific subtropical high exhibits an abnormal shape with double ridges, of which the north and south ridge line are, respectively, located at about 5°N and 30°N. Therefore, a belt-shaped high-pressure dam is formed between the Yangtze–Huaihe river basin and the Yellow-Huaihe river basin. The southwest monsoon is strong in the lower troposphere (850 hPa), but it is blocked by the high-pressure dam when advancing northward, causing it to mainly affect areas south of 30°N. The southwest monsoon continuously transports warm and moist airflow to the monsoon depression to strengthen its development. Coupled with the slow-moving large-scale weather systems, the monsoon depression has a lasting impact on Guangdong, providing favorable conditions for the occurrence and development of extreme rainstorms (Cai et al., 2019).

Combined with the analysis of gridpoint-based (i.e., PCC, MAE, and ETS) and object-based (i.e., MODE) evaluations, the regional model COSMO EPS and the global model ECMWF, respectively, predict well the precipitation above 50 and 25 mm. Therefore, a more in-depth comparative analysis of these two models from the aspects of circulation background and water vapor conditions will be conducted. Figure 7 shows the distributions of wind flow and relative humidity in the lower troposphere (850 hPa). Guangdong Province is mainly under the control of the monsoon depression on 27–28 August. The southwest airflow in the southeast side of the monsoon depression transports water vapor from the South China Sea to Guangdong. The monsoon depression then moves westward, with low-level winds shifting from southwesterly to southerly. The southerly wind over the coastal areas has proven to be an important factor for the continuous extreme rainstorm (Li et al., 2020). The circulation distributions of COSMO EPS and ECMWF are roughly the same as the observations, while the water vapor distributions predicted by COSMO EPS are more similar to the observations.

The evolution of the low-level jet (LLJ) over time in the northern South China Sea (113°E) is shown in Figure 8. From 00 UTC on 27 August to 18 UTC on 31 August, the LLJ progresses gradually northward and strengthens. The LLJ core front with a wind speed greater than 12 m s⁻¹ on 850 hPa lands on the coastal areas of Guangdong at 00 UTC on 29 August. It is conducive to the convergence of water vapor and dynamic uplift here, and then promotes the occurrence and development of the extreme rainstorm. Subsequently, the LLJ intensifies to above 14 m s⁻¹ and is located over the southern Pearl River Delta, contributing to the heavy precipitation occurring in most of the coastal areas. After 06 UTC on 30 August, the monsoon depression begins to weaken; thus, the LLJ weakens and then the precipitation in the coastal areas gradually disappears. COSMO EPS predicts the evolution and intensity of LLJ more closely to the reality than ECMWF.

The transport of water vapor is necessary to maintain continuous precipitation; thus, Figure 9 calculates the integrated atmosphere (1,000–200 hPa) water vapor flux divergence. The observed convergence at 18 UTC on 29 and 30 August are significantly stronger than that at 06 UTC, which is related to the characteristics of monsoon precipitation (Guo et al., 2019). However, both COSMO EPS and ECMWF do not capture this feature. Starting from 06 UTC on 29 August, the observed water vapor flux convergence occurs in the coastal areas of Guangdong Province, and the maximum convergence exceeds 70 × 10⁻⁶ g cm⁻²·s⁻¹. At 18 UTC on 31 August, the water vapor flux convergence significantly weakens, and the maximum center is shifted westward. The area with strong convergence on 29 August predicted by COSMO EPS is located in the coastal area, and then it mainly appears in the eastern and northern parts of Guangdong. ECMWF predicts that there will be strong convergence in eastern Guangdong on 29 August and the convergence intensity will weaken in the following 2 days. The distributions of water vapor flux convergence predicted by both COSMO EPS and ECMWF are quite different from the observations, while the convergence intensity predicted by COSMO EPS is generally stronger than that of ECMWF.

The regional model forecasts by COSMO EPS and GRAPES MEPS compared so far are initialized and driven at the boundaries by the predictions of their global models. Thus, there are no mesoscale data assimilation involved. However, several COSMO experiments with different initial conditions show that the forecasts of this extreme rainstorm event in Guangdong Province are very sensitive to the initial conditions (not shown). Therefore, in this section, we feed the in-built LHN scheme of COSMO with the precipitation rate retrieved by radar in order to explore its impact on the forecast quality of COSMO EPS. The LHN scheme is developed for a 2.8-km resolution and introduces increments in temperature and moisture to the thermodynamic equation, which is expected to enable the model dynamic to react in generating similar precipitation patterns as observed (Stephan et al., 2008). It assumes that the latent heat release is proportional to the model surface rain rate (Jones and Macpherson, 1997). Accordingly, the LHN scheme adds positive temperature increments to increase model precipitation and negative increments to decrease it. We run COSMO LHN experiments with two setups. The results of the EPS without LHN (COSMO-noLHN EPS, first setup) have been already discussed in Section 3.1. For the second setup, we stop LHN each day after an analysis window of 3 h (COSMO-3hrLHN EPS). Thus, we have each day a 21-h free (not nudged) forecast.

The results of two setups for the 21 h of the free forecast from the LHN runs are summarized in Figure 10. Without LHN (Figure 10C), the results are similar to the ones discussed already in the previous subsection. Note that differences between Figure 2B and Figure 10C relate to the shorter periods—21 instead of 24 h/day—for the latter. COSMO-3hrLHN EPS reproduces the overall observed precipitation pattern for amounts above 50 mm, which significantly improves the gridpoint-based scores compared to the COSMO-noLHN experiment (not shown). Although the precipitation intensity in eastern Guangdong predicted by COSMO-3hrLHN EPS is still too weak, its location has moved westward compared with COSMO-noLHN EPS, which is closer to the observations. The daily precipitation (21 h/day) is spatially evaluated by the MODE with thresholds of 25 and 50 mm, and the OTS values are listed in Table 5. For the threshold of 25 mm, the OTS values of COSMO-3hrLHN EPS are generally higher than those of COSMO-noLHN EPS, indicating the forecast skill improvements by LHN. However, there are no obvious differences between the COSMO-3hrLHN EPS and COSMO-noLHN EPS predictions for the threshold of 50 mm.