There were three primary datasets used in this study. Two are from climate model runs, and the other is from NASA satellite observational data. The observational dataset used in this study is the NASA Global Precipitation Measurements Integrated Multi-Satellite Retrievals (GPM IMERG). Further details about the satellites and precipitation retrieval process can be found in Bolvin (2020). The GPM IMERG Final Run data were used in this study, and the data have a spatial resolution of 0.1° globally, or roughly 11 km. This resolution is higher than that of the data in both model runs. Only the total precipitation rate data (precipitationCal) are used for this study. The units of this variable are in millimeters per hour (mm/h).

The primary climate model simulations used in this study are from the Energy Exascale Earth System Model (E3SM) Version 2. Details about this model and its data can be found in Golaz et al. (2022), and only a brief overview is provided here. The low resolution (LR) and North American Regionally Refined Model (NARRM) runs from E3SM are used (Tang et al., 2023).

Both low resolution (LR) and North American Regionally Refined Model (NARRM) simulations from E3SM were used in the current study. While both datasets are based on output from the same E3SM, for the purposes of this study, the LR run is considered a GCM as it has a spatial resolution of approximately 100 km globally. Similarly, the NARRM run is considered an RRM as it has a spatial resolution of approximately 25 km, covering a significant portion of North America as shown in Figure 1, with 100 km spacing between grid points elsewhere. It should be noted that because some cloud parameterizations in E3SM suffer from poor scale-awareness, the NARRM adopts the hybrid timestep approach, i.e., a combination of low-resolution physics timesteps and the high-resolution dynamics timesteps, for the atmosphere model to achieve reasonable global climate for CMIP6 production simulations (Tang et al., 2023). Such choice should be taken into account when interpreting NARRM results relative to the LR counterpart. In these datasets, only the convective precipitation rate (PRECC) and stable precipitation rate (PRECL) data are used. The sum of PRECC and PRECL yields the total precipitation rate (PRECT). The units of these precipitation variables are all in millimeters per hour (mm/h).

All three datasets were analyzed at three temporal scales: monthly, daily, and 3-hourly. Although the data are available for several years between 2000 and 2020, only the year 2014 is chosen as a representative year for the evaluation using the inter-resolution framework. The analyzed simulated and observed data amounted to approximately 233 gigabytes.

In addition to climate model datasets, elevation data from the Joint Institute for the Study of the Atmosphere and Ocean (JISAO) were used. These elevation data primarily served to assess the topographical complexity of various regions within CONUS (Mitchell, 2014).

This analysis focuses on the precipitation of the continental United States (CONUS), which encompasses the region between −130° and −60° longitude and 20°–60° latitude.

Due to the significant disparities in spatial resolutions among the three datasets, quantitative comparisons necessitate the regridding of the data to common resolution and pixels. As the E3SM RRM run contains irregularly spaced grid points in part of its simulations, a specific regridding process is necessary for this study. Consequently, the datasets are remapped onto the Hierarchical Equal-Area isoLatitude Pixelization (HEALPix) pixels at the respective spatial resolutions (Górski et al., 2005). HEALPix is a rearrangement of spherical model data that essentially simplifies the comparisons between models and observations of different spatial resolutions. The details of the mathematics behind how the data are rearranged and interpolated are outlined in Górski et al. (2005). This pixelization has been used in several cosmology and engineering studies such as Gao et al. (2019), Krachmalnicoff and Tomasi (2019), and Fernique et al. (2015). However, HEALPix has largely remained unused in atmospheric science and climate studies despite the fact that the pixelization is designed for spherical mediums such as Earth’s atmosphere.

HEALPix possesses several important properties that are pertinent to this research. Firstly, at any given spatial resolution, all pixels have equal areas (Górski et al., 2005). This characteristic mitigates the problem of spherical distortion encountered in Cartesian grids. Secondly, the conversion between different spatial resolutions in HEALPix merely entails a straightforward partitioning of pixels of lower-resolution pixels to form four nested pixels with higher resolution (Fu and Lee, 2022). The partitioning process preserves the geometric similarity of the grid points as illustrated in Figure 2. These two properties of HEALPix greatly simplify the comparison process among all remapped datasets. Once the data is in the HEALPix arrangement, it can be regridded to lower resolutions, as needed, using the aforementioned partition averaging process. Table 1 provides an overview of the HEALPix levels and the associated spatial resolutions (Górski et al., 2005).

These properties of the HEALPix remapping make it the preferred framework compared to other remapping techniques used in other RCM studies. For example, Kalognomou et al. (2013) uses a bilinear interpolation remapping method, and Eum et al. (2011) uses a distance-based weighted average remapping method. These methods have simple calculations when using Cartesian-based grids on a sphere (as is the case for the models used in those studies). However, the calculations for remapping become rather complex when applied to the E3SM grid due to its nonuniform spacing as seen in Figure 1. As such, using HEALPix greatly simplifies the remapping process for models with irregularly-spaced non-Cartesian grids. Furthermore, because the previously-defined CONUS region has a relatively large meridional extent, any remapping techniques used have to account for the reduced surface area of grid points that are more poleward when computing overall statistics. The equal-area property of HEALPix simplifies this calculation and further justifies its use over other remapping frameworks.

In the HEALPix arrangement, the LR, NARRM, and GPM IMERG datasets have spatial resolutions of 203 km, 51 km, and 13 km, respectively. These are coarser spatial resolutions than those of the original data, but the reduction is necessary to allow for the appropriate comparisons between each of the datasets.

Before discussing the metrics, it is important to address the topographical complexity of CONUS. This study defines topographical complexity by the spatial variance/standard deviation of the elevation, and a more topographically complex region will have a higher spatial variance in elevation.

There are several key climate model metrics analyzed in this study, each with a different purpose for analysis. The first metric is the bias in regionally averaged precipitation. This was chosen to evaluate the precipitation simulations in the regions with varying topographical complexity. In the current study, the CONUS was divided into seven different regions as defined in the United States National Climate Assessment: the Northwest (NW), Southwest (SW), Northern Great Plains (GPN), Southern Great Plains (GPS), Southeast (SE), Northeast (NE), and Midwest (MW) (USGCRP, 2018). The topographical complexity of each region, as well as which states the regions encompass, are shown in Figure 3. The spatially averaged precipitation values were calculated for each region, across the three temporal scales, and for each of the original dataset within the entire year of 2014. The HEALPix remapping was not used during the calculation of bias in regionally averaged precipitation.

Next, we analyze the distribution of precipitation rates using the Kolmogorov-Smirnov (KS) test at the 99% confidence level on the cumulative distribution functions (CDF). Each pair of datasets (LR vs. NARRM, LR vs. IMERG, NARRM vs. IMERG) is compared to obtain the KS test statistics. These statistics determine the similarity of the precipitation rate distributions across datasets, and also assess the presence of biases in the distributions of LR- and NARRM-simulated precipitation. For this metric, the frequencies of precipitation rate values are calculated using the raw precipitation data across the entire spatial domain of CONUS and the temporal domain of year 2014. Since the data are discrete, Gaussian kernel density estimates are used to approximate the CDFs. HEALPix remapping was not implemented for these two metrics as the analysis of the raw precipitation data is necessary to assess the effects of predicted precipitation bias.

Furthermore, we assess how the spatial variance in precipitation changes with different HEALPix resolution levels. The purpose of this metric is to highlight the extent to which spatial variability of precipitation increases as spatial resolution improves. The HEALPix-remapped data for all three datasets are regridded to each of the coarser HEALPix spatial resolution levels. At each HEALPix level, the spatial variance of precipitation is calculated over the CONUS domain. This process is repeated for the seven regions to assess any regional differences or topographical effects.

From there, we compare the HEALPix resolution levels to the relative change in spatial variance of precipitation. This metric offers insight into the value of higher-resolution climate simulations. Mathematically, it is calculated by taking the spatial variance in precipitation at each HEALPix spatial resolution level and computing the difference in variance values between a given HEALPix level and the previous level. The difference values are then divided by the variance value of the previous HEALPix level. The resulting proportions represent the relative percent decrease in spatial variance of precipitation between a given HEALPix level and the previous level. This process is repeated for the seven regions.

Lastly, we assess the differences in spatial variance of precipitation between model simulations and observations. This metric directly evaluates the performance of the precipitation simulations compared to observations. Mathematically, it is calculated by dividing the spatial variance in precipitation of both E3SM remapped datasets by the spatial variance in precipitation of the GPM IMERG remapped dataset. The resulting proportions indicate the percentages of observed spatial variance of precipitation explained by E3SM simulations. This is performed over the whole CONUS domain and the seven regions.

It should be noted that the temporal domain of 2014 is not identical between the model run data and the observational data; The E3SM simulations for the year 2014 do not precisely replicate the observed precipitation conditions for a given temporal instant in that year. As such, the raw spatial precipitation rate maps at specific temporal moments will appear significantly different between E3SM runs and GPM IMERG. However, it is worth mentioning that all of the aforementioned metrics involve some form of temporal averaging in some way, and the metrics primarily focus on the climatological aspects of CONUS precipitation. Therefore, it is acceptable to make comparisons between E3SM and GPM IMERG despite the disparity in temporal alignment.

The results of Regionally Averaged Precipitation, shown in Figure 4, exhibit regional variations. In the NW, SW, and GPN regions, both E3SM simulations show that the spatiotemporal mean precipitation rates are 1 and 2 times higher than those of GPM IMERG. Conversely, in the SE region, spatiotemporal mean precipitation rates in both E3SM runs are approximately 70%–90% of the GPM IMERG precipitation rate. For the NE, MW, and GPS regions, both E3SM simulations capture the spatial-temporally averaged precipitation well, with differences between simulated and observed precipitation rates not exceeding 20%.

The results of the KS tests are presented in Table 2. Notably, for all temporal resolutions, the test statistics values are lower for E3SM NARRM vs. GPM IMERG compared to E3SM LR vs. GPM IMERG. This indicates that E3SM NARRM captures the observed precipitation rate distribution more accurately than E3SM LR. This result is not surprising considering the inherent resolution differences between the two model runs. It should be noted that all KS test statistics are significant at the 99% confidence level, and the corresponding p-values are provided in Table 3.

For reference, the probability distribution function (PDF) and cumulative distribution function (CDF) of all datasets are shown in Figure 5. Two additional results are worth highlighting. Firstly, as shown in the PDFs, E3SM LR simulates a significantly lower frequency of higher precipitation rates than E3SM NARRM does, and E3SM NARRM also exhibits lower frequencies of higher precipitation rates than GPM IMERG. This result is expected given that the difference in spatial resolution, with GPM IMERG having the highest and E3SM LR having the lowest resolution among the three datasets. This pattern holds true at the monthly, daily, and 3-hourly scales. Additionally, increasing the temporal resolution leads to an overall increase in the frequency of all precipitation rates, regardless of the dataset.

The second additional result involves the CDFs of the three datasets. Specifically, at lower precipitation rates, the CDF values in both E3SM datasets are lower than those of GPM IMERG, before converging closer to each other at high precipitation rates. This indicates that the E3SM runs simulate a higher proportion of lower precipitation rates compared to observations. Again, the pattern holds true at each of the monthly, daily, and 3-hourly temporal scales. This bias is common in many climate models, and E3SM is no exception. Further details about this bias are discussed in Pendergrass and Hartmann (2014) and Taylor et al. (2022).

There are two important results to highlight from analyzing the sensitivity of the spatial variance in precipitation with respect to spatial resolution. These are shown in Figure 6. Firstly, in both models and observations, as the HEALPix level and spatial resolution increase, so does the spatial variance in precipitation. While this result is not surprising, the slopes of the line plots in the figures provide insight into the extent of additional spatial information is captured in precipitation simulations with higher spatial resolutions. Similarly, when examining temporal resolution, the variance increases as temporal resolution increases. This trend holds true when considering the seven regions within CONUS.

The second result pertains to the comparison between E3SM LR to E3SM NARRM. Despite having a higher HEALPix spatial resolution, the spatial variance in precipitation of the NARRM is lower than that of the LR. This pattern remains consistent across changes in spatial resolution, temporal resolution, and the CONUS regions. This result has implications for E3SM precipitation simulations, which will be discussed later.

The results for the relative change in spatial variance of precipitation between spatial resolutions, as depicted in Figure 7, highlight the importance of high-resolution simulations by themselves. As the spatial resolution increases, the increase in spatial variance from the previous spatial resolution level to the next decreases. Still, it is worth noting that spatial variance in precipitation increases by approximately 50% between 25 km and 13 km at the three-hourly temporal scale. The patterns hold consistent when considering changes in temporal resolution in all datasets, as well as changes in regional domains. All of these increases between spatial resolution levels demonstrate how higher spatial resolution capture additional spatial information about precipitation.

Results for this metric are shown in Figure 8. The first notable result here is that the percentage of spatial variance in precipitation explained by models generally decreases as spatial resolution increases across all temporal scales, except for 407 km resolution at the monthly timescale. However, the pattern generally flips when analyzing the seven CONUS regions individually, where models explain more variance in precipitation at the higher spatial resolutions. The exception is the GPN region, where the trends are quite inconsistent with both temporal and spatial resolution changes. The second notable result is that despite having a higher spatial resolution than E3SM LR, E3SM NARRM explains less of the observed spatial variance in precipitation at all spatial resolutions, timescales, and in all of the CONUS regions. The implications of these results on the E3SM runs’ performance are discussed in the following section.

In this study, a framework was developed for comparing model simulations and observations of different spatial resolutions. Multiple model and observational datasets were used, and they were regridded to the appropriate HEALPix resolution as needed. This regridding allowed for quantitative comparisons of precipitation between the datasets at different spatial resolutions. This framework was tested on the E3SM LR and E3SM NARRM model runs, as well as the GPM IMERG observations, which have significantly different spatial resolutions. The remapping onto the HEALPix pixels was necessary to compare the three datasets. This remapping process greatly simplified the comparisons between all three datasets as it standardized not only a common spatial resolution but also a common layout and/or arrangement for all the data. HEALPix enabled the inter-resolution comparisons that would have otherwise been mathematically challenging.

Several metrics were calculated to evaluate the precipitation from E3SM model against GPM IMERG observations. The primary objective is to determine the viability of this framework in comparing multiple datasets with different native spatial resolutions. The chosen metrics were relevant for evaluating the HEALPix framework’s viability. While some results regarding E3SM’s performance may seem counterintuitive in the context of climate model development, this study’s findings demonstrate that the framework produced meaningful results for comparing E3SM LR, E3SM NARRM, and GPM IMERG despite their significant differences in their native resolutions. The successful application of this framework opens up possibilities for its use in other climate model versus observational comparisons, particularly for inter-resolution comparisons.

As aforementioned, several results about E3SM’s performance showed that the E3SM RRM performs worse than the E3SM LR in CONUS precipitation simulations. Firstly, the RRM has a lower spatial variance in precipitation than the LR at all resolutions. Furthermore, the RRM explains less of the observed precipitation variance than the LR. This is because the NARRM is developed for climate production campaigns (e.g., CMIP6), hence, it needs to meet the criteria of reasonably simulating global climate. NARRM adopted the hybrid time step method (i.e., a combination of an LR physics time step and the high-resolution dynamics time steps) in the atmosphere model to achieve the reasonable global performance without sacrificing too much of the benefits from high-resolution dynamics (Tang et al., 2023). The hybrid time step approach cannot take full advantage of the high-resolution dynamics at 25 km due to large time-truncation errors. Additionally, 25 km spatial resolution is not sufficiently high enough to accurately simulate small-scale precipitation features. To accurately represent these small-scale processes, a much higher spatial resolution, typically around 4 km or higher, is often required, and model parameterizations must be set appropriately to account for any spatial coarsening effects. Consequently, the 25 km resolution of the RRM resulted in a lack of spatial variance that would otherwise be present in simulated precipitation at a higher resolution than 25 km. Any small-scale precipitation features present in the RRM are effectively filtered out when remapping to coarser resolutions, and this has the effect of generally lowering the spatial variance in precipitation.

In both of the E3SM simulations, as the spatial resolution (HEALPix Level) increases, the observed spatial variance in precipitation explained by both model runs also decreases. This result can be attributed to the E3SM’s ability to simulate larger-scale precipitation features such as atmospheric rivers, cold fronts, and monsoonal systems. The 100 km spatial resolution of the LR may be sufficiently high enough to accurately simulate these large-scale precipitation features. As such, simulating these features in the RRM at 25 km resolution did not significantly contribute to the overall spatial variance since there was minimal additional information gained by simulating these large-scale features at a higher resolution.

In conclusion, when considering total precipitation simulations, the advantages of increasing the spatial resolution in the E3SM RRM generally do not outweigh the drawbacks in terms of increased computational costs. The RRM’s 25 km resolution may not be high enough to accurately capture small-scale precipitation processes, while the LR already performs well in simulating larger-scale precipitation features at its resolution. Because E3SM performs better in simulating large-scale precipitation features than simulating small scale-features, increasing the HEALPix resolution would ultimately decrease the proportion of observed precipitation variance that E3SM explains because the added resolution would not be able to sufficiently explain the small-scale precipitation features that become increasingly more present as HEALPix resolution increases.

Overall, this research has laid the foundation for comparing models and observations across different spatial scales, demonstrating the viability of the developed framework. Indeed, the counterintuitive results obtained in this study regarding E3SM precipitation simulations should not be generalized to all variables, climate models, or regions outside of the CONUS. It is important to recognize the complexity of climate modeling and the wide range of factors that can influence model performance. Moving forward, there are several promising avenues for further research. Expanding the analysis to include other variables simulated by E3SM or other climate models would provide a more comprehensive understanding of the model’s performance across different spatial resolutions. Additionally, extending the study beyond the CONUS to different regions would help assess the generalizability of the findings. To better evaluate the computational costs associated with analyzing datasets, developing quantifiable metrics is crucial. Incorporating these metrics into the research would enable comparisons of computational costs between different datasets and facilitate a more accurate cost-benefit analysis. This analysis would help determine whether the benefits of high-resolution climate model simulations outweigh the drawbacks in terms of computational power.