Abstract
With the ongoing expansion of global observation networks, it is expected that we shall routinely analyze records of geophysical variables such as temperature from multiple collocated instruments. Validating datasets in this situation is not a trivial task because every observing system has its own bias and noise. Triple collocation is a general statistical framework to estimate the error characteristics in three or more observational-based datasets. In a triple colocation analysis, several metrics are routinely reported but traditional multiple-panel plots are not the most effective way to display information. A new formula of error variance is derived for connecting the key terms in the triple collocation theory. A diagram based on this formula is devised to facilitate triple collocation analysis of any data from observations, as illustrated using three aerosol optical depth datasets from the recent Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE). An observational-based skill score is also derived to evaluate the quality of three datasets by taking into account both error variance and correlation coefficient. Several applications are discussed and sample plotting routines are provided.
1 Introduction
The development of atmospheric sciences and other branches of Earth system science is inextricably linked to our capability of improving and expanding current atmospheric observing systems (Crutzen and Ramanathan, 2000; Stith et al., 2018; Bluestein et al., 2022). For instance, a dense weather station network provides us with synoptic weather conditions while sounding profiles from radiosondes and dropsondes give us a wealth of information for the vertical structure of the atmosphere over land and ocean (Stith et al., 2018). In addition to in situ observations, remotely sensed instruments such as weather radars are able to detect severe weather systems in a timely manner. The advent of satellite meteorology has reshaped our data inventory by making global observations possible (Atlas, 1997). Research aircraft in field campaigns provides a unique platform to collect measurements from both in situ and remotely sensed instruments for specific research problems with pressing societal needs (Bluestein et al., 2022).
With more and more new observing systems available, it has become commonplace for more than one instrument observing the same geophysical variable at the same location. Data validation and calibration then become necessary because no observing systems are perfectly built. Stoffelen (1998) recognized this issue and proposed a general statistical framework called triple collocation. For any geophysical variables, triple collocation estimates the error characteristics of three independent datasets without requiring any dataset being the ground truth. Since then, the method has been applied to a wide range of variables such as soil moisture (e.g., Dorigo et al., 2010), sea surface temperature (e.g., O’Carroll et al., 2008), and leaf area index (e.g., Fang et al., 2012), to name a few. The theory of triple collocation has also been clarified, tested, and extended over the years (e.g., Zwieback et al., 2012; Draper et al., 2013; McColl et al., 2014; Su et al., 2014; Yilmaz and Crow, 2014; Gruber et al., 2016; Tsamalis, 2022).
Several performance metrics such as standard error (Stoffelen, 1998) and correlation coefficient (McColl et al., 2014) have been developed for triple collocation analysis. These metrics are usually reported in the form of a map that shows one aspect such as a metric for one dataset (e.g., Dorigo et al., 2010) or a figure that shows another aspect for three datasets (e.g., Tsamalis, 2022). Multiple-panel plots are common in the literature (e.g., McColl et al., 2014; Deng et al., 2023) and some examples are given in Supplementary Material. But this kind of plot is not the most effective way to summarize results. A table that shows multiple aspects may help but different researchers have varying designs and notations (e.g., Su et al., 2014). The purpose of this study is to present a single diagram for the triple collocation analysis that can succinctly compare multiple aspects of multiple datasets. The diagram is facilitated by deriving a new formula of error variance. Some applications of the new triple collocation diagram are illustrated using datasets collected from a recent field campaign, i.e., the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE).
2 Methods
2.1 Classical and extended triple collocation
We first briefly review the derivation of classical and extended triple collocation. An error model is needed to specify each dataset as a function of the unknown ground truth (Zwieback et al., 2012). So far in the literature, linear error models are the most popular type with one example being the affine model,where represents dataset ; is the unknown ground truth; , , and are the additive bias, multiplicative bias, and random error from dataset , respectively.
Based on the affine error model, the variance/covariance equations for the collocated datasets are then given as.where is the total variance, is the truth variance, and is the error variance. Note that . Three datasets result in three equations for variance and three more for covariance, amounting to six equations in total.
Three further assumptions are usually made to simplify the equations (Gruber et al., 2020). First, the mean random error is zero . Second, the random error is uncorrelated with the ground truth . Third, the random error of different datasets is uncorrelated with each other . In virtue of these assumptions, Eqs 2, 3 become.
The six equations from Eqs. 4, 5 now have seven unknowns (, and ) so the system has no unique solution. McColl et al. (2014) proposed that unknown variables can be combined to reduce the number of unknowns. Thus, in theory, the error variance can be solved using
In practice, the signal variance for different datasets (Eqs 7–9) can be computed from combining the covariance terms from Eq. 4.
To extend the classical triple collocation analysis, McColl et al. (2014) derived a new metric, the correlation coefficient, from the ordinary least squares framework.where is correlation coefficient of the dataset with respect to an unknown ground truth, and sgn is signum function. Note that the sign of all correlation coefficients is ambiguous but practically all observing systems are positively correlated with the ground truth, i.e., taking the positive root.
2.2 Basis of the triple collocation diagram
To construct the triple collocation diagram, an alternative error variance equation is needed to connect all variance terms in Eq. 6 with the correlation coefficient (Eqs 10–12). To start with, we use the following basic statistical identities (Dekking et al., 2005; Thomson and Emery, 2014).where and are constants; and are random variables.
Combining two identities (Eqs 13, 14) giveswhere is a constant.
Applying the above identity (Eq. 15) to the error model Eq. 1 gives Eq. 16, and subsequently Eqs 17, 18.where the left hand side (lhs) of Eq. 17 uses Eq. 1, the lhs of Eq. 18 uses Eq. 13. On the right hand side (rhs) of Eq. 18, the correlation coefficient between each dataset and the ground truth follows the traditional definition, which is the ratio of the covariance of two datasets to the product of their corresponding standard deviations (Wilks, 2011). To our knowledge, Eq. 18 is derived here for the first time.
The alternative error variance Eq. 18 is inspired by the Taylor diagram (Taylor, 2001) which is based on the law of cosines. The alternative equation also resembles the law of cosines (Figure 1A): three sides for three standard deviation terms (error standard deviation , signal standard deviation , and total standard deviation ) and the angle between and for correlation coefficient. Triple collocation estimates error properties of observational-based datasets with respect to an unknown ground truth; on the contrary, the Taylor diagram aims to summarize model performance against an observational-based reference. Therefore, the three sides become model standard deviation, reference standard deviation, and the unbiased root-mean-square error between the model and reference fields, and the angle is the correlation coefficient between the two fields (Taylor, 2001).
FIGURE 1
We emphasize that all datasets are assumed to be positively correlated with the ground truth in Eqs 10–12, i.e., taking the positive root. Therefore, is also positive using Eq. 8 in McColl et al. (2014). This assumption ensures that all standard deviation terms are positive and is acute.
The original error variance Eq. 6 resembles the Pythagoras’ theorem (Figure 1B), a special case of the law of cosines. Therefore, the angle between and has to be 90. Note that there is nothing mysterious about the right angle as it is merely a consequence of the random error and ground truth being uncorrelated from one of our assumptions. It is clear that the alternative error variance equation is needed because the original equation does not provide information on the angle .
To summarize, the alternative error variance equation incorporates correlation coefficient, which is absent in the original equation. The new equation forms the basis of the proposed triple collocation diagram.
3 Data
To demonstrate applications of the new triple collocation diagram, we focus on a measurement related to the abundance of aerosol particles. Aerosols are highly variable in space and time leading to high uncertainty in estimating total anthropogenic radiative forcing (Forster et al., 2021). Three aerosol optical depth (AOD) retrieval datasets are used in this study. AOD quantifies the column-integrated aerosol loading, but more specifically the sum of light scattering and absorption by aerosols (Seinfeld and Pandis, 2006).
Two of the retrievals come from the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) which is one of the National Aeronautics and Space Administration (NASA) Earth Venture Suborbital-3 (EVS-3) missions (Sorooshian et al., 2019; Sorooshian et al., 2023). During ACTIVATE, two NASA Langley Research Center aircraft, the low-flying HU-25 Falcon and high-flying King Air, were strategically deployed to collect in-situ and remotely-sensed measurements over the western North Atlantic Ocean. From February 2020 to June 2022, 174 and 168 flights were successfully completed by the King Air and Falcon, respectively. Among these, 162 were joint flights. AOD was retrieved by two remote sensing instruments onboard the King Air, the Research Scanning Polarimeter (RSP) (Cairns et al., 1999; Cairns et al., 2003) and Second Generation High Spectral Resolution Lidar (HSRL-2) (Hair et al., 2008). The third retrieval comes from the Moderate Resolution Imaging Spectroradiometer (MODIS). Two MODIS sensors have been onboard the Terra and Aqua satellites since 1999 and 2002, respectively. The two satellites observe the Earth along a sun-synchronous orbit at an altitude of around 700 km with a period of 99 min, providing MODIS data at three nadir spatial resolutions (Remer et al., 2005; Levy et al., 2013).
We compare AOD at 532 nm from the three retrievals. For RSP, the aerosol properties are retrieved using the Microphysical Aerosol Properties from Polarimetry (MAPP) algorithm (Stamnes et al., 2018), which then become part of the level 2 aerosol product. For HSRL-2, AOD is derived using extinction coefficients from the difference in molecular return signals (Hair et al., 2008). The original 1-km level 2 RSP and HSRL-2 AOD data are horizontally averaged to a spatial resolution of 3 km, which results in 6,988 pairs. For MODIS, we use the MODIS Collection 6.1 level 2 3-km aerosol product for both satellites: MOD04_3K from Terra and MYD04_3K from Aqua. This product is entirely retrieved using the Dark Target (DT) algorithm (Remer et al., 2013). The native MODIS AOD at 550 nm is converted to 532 nm using the Ångström exponent calculated from the MODIS AOD at 470 nm and 550 nm. For each pair of RSP and HSRL-2 data, we collocate the nearest MODIS data point within 60 min and 25 km, which results in 2,344 triplets for triple collocation analysis. All triplets are collocated over the ocean.
4 Applications
4.1 A bird’s-eye view of the triple collocation analysis
Based on the alternative error variance Eq. 18, a triple collocation diagram is designed to display multiple aspects of triple collocation analysis. A prototype is shown in Figure 2 with three AOD datasets: RSP, HSRL-2, and MODIS. For comparison, an additional figure (see Supplementary Figure S1) is prepared to display the same information using the traditional multi-panel method.
FIGURE 2
For those who are familiar with Taylor diagram, it is straightforward to use the schematic from Figure 1 to recognize the three standard deviation terms and correlation coefficient in Figure 2. The circles on the polar grid represents the total standard deviation. The projections of the total standard deviation on the -axis and -axis represent the signal standard deviation and error standard deviation, respectively. As a result, it is relatively easy to compare the magnitudes for either signal or error standard deviation.
During ACTIVATE, HSRL-2 has the smallest error standard deviation and the highest correlation coefficient while RSP has the largest error standard deviation and the lowest correlation coefficient among the three datasets. The different performance of RSP and HSRL-2 is justified from the perspectives of retrieval algorithm and instrument uncertainty. First, some assumptions in the RSP retrieval algorithm are not warranted during the field campaign. For example, the RSP algorithm assumes that all aerosols are spherical but the in-situ instruments and HSRL-2 show that non-spherical sea salt and dust are often detected in winter and summer, respectively. Second, past field campaigns showed that, compared to the Sun photometer measurements, the root-mean-squared difference for RSP and HSRL-2 are (Fu et al., 2020) and (Shinozuka et al., 2013), respectively.
Other metrics can also be easily estimated or computed from the diagram. The values of total standard deviation can be computed using . The values of signal standard deviation can then be computed using Eq. 6. Signal standard deviation is important for computing the unbiased signal-to-noise ratio or unbiased noise-to-signal ratio , which indicates whether signal variations can be set apart from noise variations (Gruber et al., 2016). Total standard deviation is also important because fractional root mean square error (Draper et al., 2013) is defined as the ratio of error standard deviation to total standard deviation . Total standard deviation also provides information on the measurement sensitivity through (Rodgers and Nicewander, 1988).
4.2 Tracking performance metrics
The triple collocation diagram is not limited to handling three datasets only. For example, seeing that RSP exhibits the largest errors among three datasets (Figure 2), a simple data quality flag is devised to filter out data points with inferior retrievals based on a performance cost function in the RSP retrieval algorithm (Stamnes et al., 2018). Around 50% of triplets are filtered out, and the difference before and after applying the quality flag is shown in Figure 3. Filtered datasets are indicated by squares without changing the color theme. Colored arrows have also been added to follow the changes in performance metrics. Appreciable improvement is seen with RSP data after filtering as the correlation coefficient rises to 0.84 and error standard deviation falls to 0.044 (or a 31% decrease). Note also that if the correlation coefficient increases with the direction of the arrow, the signal-to-noise ratio also increases. An additional figure (see Supplementary Figure S2) is prepared to display the same information using the traditional multi-panel method.
FIGURE 3
4.3 Triple collocation skill score
While various performance metrics have been developed for the triple collocation analysis, it is beneficial to summarize the different aspects of the performance in a single measure. Similar problems have led to the development of skill scores such as Brier skill score and ranked probability skill score in weather and climate prediction (Wilks, 2011) and efficiencies such as Nash–Sutcliffe efficiency and Kling–Gupta efficiency in hydrology. For triple collocation, we propose a skill score,where is the normalized error variance adopted from Koh et al. (2012). ranges between 0 and 1.
The same datasets in Figure 3 are shown in Figure 4, which is contoured with the proposed skill score (Eq. 19). Before filtering, HSRL-2 owns the highest skill score; after filtering, RSP gains in skill score. While is proportional to by definition, it varies in a more linear scale in the normal range between 0.1 and 0.9.
FIGURE 4
5 Discussion
The utility of displaying multiple performance metrics in the triple collocation diagram deserves some discussion. The ACTIVATE field campaign provides a rare opportunity to validate the aerosol measurements from two airborne remote sensing instruments on the same platform. The common approach is to compare with Sun photometer measurements from the Aerosol Robotic Network (AERONET) but this is not feasible in our case because no AERONET site is available in the remote ocean over the study region. To get around this issue, we added one more independent satellite dataset and proceeded with the triple collocation analysis.
In the process, we got to be aware of the diverse applications in the literature and the development of various performance metrics. For example, was developed as an alternative of fRMSE (McColl et al., 2014). There are two reasons for showing multiple performance metrics in our current approach. First, it is impractical to predetermine a single performance metric for users as the choice is usually field or application dependent. For example, in aerosol studies, conventionally people are more concerned about the error standard deviation of an instrument compared to the Sun photometer because this quantity can be used to estimate the uncertainty of aerosol radiative forcing (Hansen et al., 1995; Chylek et al., 2003; Mishchenko et al., 2004), which remains the largest uncertainty of our future climate projection. There are usually no preference on the performance metrics for other variables, such as sea surface temperature and soil moisture. Second, it is in line with a current triple collocation community guideline (Gruber et al., 2020) to present a comprehensive picture of error characteristics. As such, our strategy is to display the various variance terms and correlation coefficient and provide guidance to compute other metrics (such as and ) to suit the needs of individual practitioners. Note that the performance metrics including , , , and are functions of under the triple collocation assumptions listed in Section 2.
6 Summary
Triple collocation is a statistical technique that considers three collocated datasets of any geophysical variable and estimates the error characteristics of each dataset without assuming any of the three datasets represent ground truth. Triple collocation analysis involves reporting several performance metrics which can at times become cumbersome using traditional ways of presentation such as multiple-panel plots and long tables.
An alternative equation is derived for the error variance which forms the basis of the proposed triple collocation diagram as a visual aid for condensing multiple aspects of the triple collocation analysis into a two-dimensional polar plot. An observational-based skill score is also obtained to compare performance of three datasets by considering both error variance and correlation coefficient. Three potential applications of the diagram, as illustrated in Figures 2–4, are showcased and discussed. The Taylor diagram has been used to compare the performance of multiple models with an observational-based reference dataset. The proposed diagram complements the Taylor diagram by comparing the performance of multiple observational-based datasets. For interested readers, a routine to draw the triple collocation diagram with several examples is available (see Data Availability Statement).
Statements
Data availability statement
All the data used in this work are publicly available. The ACTIVATE data can be downloaded from https://asdc.larc.nasa.gov/project/ACTIVATE (NASA/LARC/SD/ASDC, 2021). The MODIS Terra and Aqua data can be downloaded from https://doi.org/10.5067/MODIS/MOD04_3K.061 (MODIS Science Team, 2017b) and https://doi.org/10.5067/MODIS/MYD04_3K.061 (MODIS Science Team, 2017a). The code repository for this work is at https://github.com/leongwaisiu. Further inquiries can be directed to the corresponding author.
Author contributions
LWS: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Software, Validation, Visualization, Writing–original draft, Writing–review and editing. XZ: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing–review and editing, Data curation, Formal Analysis, Investigation, Software, Validation, Visualization. AS: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing–review and editing, Data curation, Formal Analysis, Investigation, Software, Validation, Visualization. BC: Data curation, Formal Analysis, Investigation, Software, Validation, Visualization, Writing–review and editing. CAH: Data curation, Formal Analysis, Investigation, Software, Validation, Visualization, Writing–review and editing. DP: Data curation, Formal Analysis, Investigation, Software, Validation, Visualization, Writing–review and editing. JSS: Data curation, Formal Analysis, Investigation, Software, Validation, Visualization, Writing–review and editing. RAF: Data curation, Formal Analysis, Funding acquisition, Investigation, Resources, Software, Validation, Visualization, Writing–review and editing. JWH: Data curation, Formal Analysis, Funding acquisition, Investigation, Resources, Software, Validation, Visualization, Writing–review and editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work is supported by NASA grant 80NSSC19K0442 for ACTIVATE, a NASA Earth Venture Suborbital-3 (EVS-3) investigation, which is funded by NASA’s Earth Science Division and managed through the Earth System Science Pathfinder Program Office.
Acknowledgments
We thank the pilots and aircraft maintenance personnel of NASA Langley Research Services Directorate for their work in conducting the ACTIVATE flights. Comments from two reviewers led to an improvement in presentation. We thank Karl Taylor for his comments on an earlier version of the manuscript.
Conflict of interest
Author DP was employed by Analytical Mechanics Associates Inc.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Authors BC, RAF, and DP declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frsen.2024.1395442/full#supplementary-material
References
1
AtlasR. (1997). Atmospheric observations and experiments to assess their usefulness in data assimilation (gtSpecial IssueltData assimilation in meteology and oceanography: theory and practice). J. Meteorological Soc. Jpn. Ser. II75, 111–130. 10.2151/jmsj1965.75.1B_111
2
BluesteinH. B.CarrF. H.GoodmanS. J. (2022). Atmospheric observations of weather and climate. Atmosphere-Ocean60, 149–187. 10.1080/07055900.2022.2082369
3
CairnsB.RussellE. E.LaVeigneJ. D.TennantP. M. W. (2003). “Research Scanning Polarimeter and airborne usage for remote sensing of aerosols,” in Proceedings of SPIE. Editors ShawJ. A.TyoJ. S. (Bellingham, WA: SPIE), 5158, 33–44. 10.1117/12.518320
4
CairnsB.RussellE. E.TravisL. D. (1999). “Research Scanning Polarimeter: calibration and ground-based measurements,” in Proceedings of SPIE. Editors GoldsteinD. H.ChenaultD. B. (Denver, CO: SPIE), 3754, 186–196. 10.1117/12.366329
5
ChylekP.HendersonB.MishchenkoM. (2003). Aerosol radiative forcing and the accuracy of satellite aerosol optical depth retrieval. J. Geophys. Res. Atmos.108, 1–8. 10.1029/2003jd004044
6
CrutzenP. J.RamanathanV. (2000). The ascent of atmospheric sciences. Science290, 299–304. 10.1126/science.290.5490.299
7
DekkingF. M.KraaikampC.LopuhaäH. P.MeesterL. E. (2005). A modern introduction to probability and statistics: understanding why and how. London, UK: Springer-Verlag.
8
DengX.ZhuL.WangH.ZhangX.TongC.LiS.et al (2023). Triple collocation analysis and in situ validation of the CYGNSS soil moisture product. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens.16, 1883–1899. 10.1109/JSTARS.2023.3235111
9
DorigoW. A.ScipalK.ParinussaR. M.LiuY. Y.WagnerW.de JeuR. A. M.et al (2010). Error characterisation of global active and passive microwave soil moisture datasets. Hydrology Earth Syst. Sci.14, 2605–2616. 10.5194/hess-14-2605-2010
10
DraperC.ReichleR.de JeuR.NaeimiV.ParinussaR.WagnerW. (2013). Estimating root mean square errors in remotely sensed soil moisture over continental scale domains. Remote Sens. Environ.137, 288–298. 10.1016/j.rse.2013.06.013
11
FangH.WeiS.JiangC.ScipalK. (2012). Theoretical uncertainty analysis of global MODIS, CYCLOPES, and GLOBCARBON LAI products using a triple collocation method. Remote Sens. Environ.124, 610–621. 10.1016/j.rse.2012.06.013
12
ForsterP.StorelvmoT.ArmourK.CollinsW.DufresneJ.-L.FrameD.et al (2021). “The earth’s energy budget, climate feedbacks, and climate sensitivity,” in Climate change 2021: the physical science basis. Contribution of working group I to the sixth assessment report of the intergovernmental panel on climate change. Editors Masson-DelmotteV.ZhaiP.PiraniA.ConnorsS.PéanC.BergerS.et al (Cambridge, United Kingdom and New York, NY, USA: Cambridge University Press), 923–1054. 10.1017/9781009157896.009
13
FuG.HasekampO.RietjensJ.SmitM.Di NoiaA.CairnsB.et al (2020). Aerosol retrievals from different polarimeters during the acepol campaign using a common retrieval algorithm. Atmos. Meas. Tech.13, 553–573. 10.5194/amt-13-553-2020
14
GruberA.De LannoyG.AlbergelC.Al-YaariA.BroccaL.CalvetJ.-C.et al (2020). Validation practices for satellite soil moisture retrievals: what are (the) errors?Remote Sens. Environ.244, 111806. 10.1016/j.rse.2020.111806
15
GruberA.SuC.-H.ZwiebackS.CrowW.DorigoW.WagnerW. (2016). Recent advances in (soil moisture) triple collocation analysis. Int. J. Appl. Earth Observation Geoinformation45, 200–211. 10.1016/j.jag.2015.09.002
16
HairJ. W.HostetlerC. A.CookA. L.HarperD. B.FerrareR. A.MackT. L.et al (2008). Airborne high spectral resolution lidar for profiling aerosol optical properties. Appl. Opt.47, 6734. 10.1364/ao.47.006734
17
HansenJ.RossowW.CarlsonB.LacisA.TravisL.Del GenioA.et al (1995). Low-cost long-term monitoring of global climate forcings and feedbacks. Clim. Change31, 247–271. 10.1007/BF01095149
18
KohT.WangS.BhattB. C. (2012). A diagnostic suite to assess NWP performance. J. Geophys. Res. Atmos.117, 1–20. 10.1029/2011jd017103
19
LevyR. C.MattooS.MunchakL. A.RemerL. A.SayerA. M.PatadiaF.et al (2013). The Collection 6 MODIS aerosol products over land and ocean. Atmos. Meas. Tech.6, 2989–3034. 10.5194/amt-6-2989-2013
20
McCollK. A.VogelzangJ.KoningsA. G.EntekhabiD.PilesM.StoffelenA. (2014). Extended triple collocation: estimating errors and correlation coefficients with respect to an unknown target. Geophys. Res. Lett.41, 6229–6236. 10.1002/2014gl061322
21
MishchenkoM. I.CairnsB.HansenJ. E.TravisL. D.BurgR.KaufmanY. J.et al (2004). Monitoring of aerosol forcing of climate from space: analysis of measurement requirements. J. Quantitative Spectrosc. Radiat. Transf.88, 149–161. 10.1016/j.jqsrt.2004.03.030
22
MODIS Science Team (2017a). MODIS/Aqua aerosol 5-min L2 swath 3km. 10.5067/MODIS/MYD04_3K.061
23
MODIS Science Team (2017b). MODIS/Terra Aerosol 5-Min L2 Swath 3km. 10.5067/MODIS/MOD04_3K.061
24
NASA (2021). Aerosol Cloud meTeorology Interactions oVer the western ATlantic experiment (ACTIVATE). 10.5067/SUBORBITAL/ACTIVATE/DATA001
25
O’CarrollA. G.EyreJ. R.SaundersR. W. (2008). Three-way error analysis between AATSR, AMSR-E, and in situ sea surface temperature observations. J. Atmos. Ocean. Technol.25, 1197–1207. 10.1175/2007jtecho542.1
26
RemerL. A.KaufmanY. J.TanréD.MattooS.ChuD. A.MartinsJ. V.et al (2005). The MODIS aerosol algorithm, products, and validation. J. Atmos. Sci.62, 947–973. 10.1175/jas3385.1
27
RemerL. A.MattooS.LevyR. C.MunchakL. A. (2013). MODIS 3 km aerosol product: algorithm and global perspective. Atmos. Meas. Tech.6, 1829–1844. 10.5194/amt-6-1829-2013
28
RodgersJ. L.NicewanderW. A. (1988). Thirteen ways to look at the correlation coefficient. Am. Statistician42, 59–66. 10.1080/00031305.1988.10475524
29
SeinfeldJ. H.PandisS. N. (2006). Atmospheric chemistry and physics: from Air pollution to climate change. 2nd edn. Hoboken, New Jersey: John Wiley & Sons, Inc.
30
ShinozukaY.JohnsonR. R.FlynnC. J.RussellP. B.SchmidB.RedemannJ.et al (2013). Hyperspectral aerosol optical depths from TCAP flights. J. Geophys. Res. Atmos.118 (12), 180–212. 10.1002/2013jd020596
31
SorooshianA.AlexandrovM. D.BellA. D.BennettR.BetitoG.BurtonS. P.et al (2023). Spatially coordinated airborne data and complementary products for aerosol, gas, cloud, and meteorological studies: the NASA ACTIVATE dataset. Earth Syst. Sci. Data15, 3419–3472. 10.5194/essd-15-3419-2023
32
SorooshianA.AndersonB.BauerS. E.BraunR. A.CairnsB.CrosbieE.et al (2019). Aerosol–cloud–meteorology interaction airborne field investigations: using lessons learned from the U.S. West Coast in the design of ACTIVATE off the U.S. East Coast. Bull. Am. Meteorological Soc.100, 1511–1528. 10.1175/bams-d-18-0100.1
33
StamnesS.HostetlerC.FerrareR.BurtonS.LiuX.HairJ.et al (2018). Simultaneous polarimeter retrievals of microphysical aerosol and ocean color parameters from the “MAPP” algorithm with comparison to high-spectral-resolution lidar aerosol and ocean products. Appl. Opt.57, 2394. 10.1364/ao.57.002394
34
StithJ. L.BaumgardnerD.HaggertyJ.HardestyR. M.LeeW.-C.LenschowD.et al (2018). 100 years of progress in atmospheric observing systems. Meteorol. Monogr.59 (2), 2.1–2.55. 10.1175/amsmonographs-d-18-0006.1
35
StoffelenA. (1998). Toward the true near-surface wind speed: error modeling and calibration using triple collocation. J. Geophys. Res. Oceans103, 7755–7766. 10.1029/97jc03180
36
SuC.-H.RyuD.CrowW. T.WesternA. W. (2014). Beyond triple collocation: applications to soil moisture monitoring. J. Geophys. Res. Atmos.119, 6419–6439. 10.1002/2013jd021043
37
TaylorK. E. (2001). Summarizing multiple aspects of model performance in a single diagram. J. Geophys. Res. Atmos.106, 7183–7192. 10.1029/2000jd900719
38
ThomsonR. E.EmeryW. J. (2014). Data analysis methods in physical oceanography. 3rd edn. Amsterdam, Netherlands: Elsevier.
39
TsamalisC. (2022). Clarifications on the equations and the sample number in triple collocation analysis using SST observations. Remote Sens. Environ.272, 112936. 10.1016/j.rse.2022.112936
40
WilksD. S. (2011). Statistical methods in the atmospehric sciences. 3rd edn. San Diego, California: Academic Press.
41
YilmazM. T.CrowW. T. (2014). Evaluation of assumptions in soil moisture triple collocation analysis. J. Hydrometeorol.15, 1293–1302. 10.1175/jhm-d-13-0158.1
42
ZwiebackS.ScipalK.DorigoW.WagnerW. (2012). Structural and statistical properties of the collocation technique for error characterization. Nonlinear Process. Geophys.19, 69–80. 10.5194/npg-19-69-2012
Summary
Keywords
triple collocation, ACTIVATE, RSP, HSRL-2, polarimeter, lidar, MODIS, aerosol
Citation
Siu LW, Zeng X, Sorooshian A, Cairns B, Ferrare RA, Hair JW, Hostetler CA, Painemal D and Schlosser JS (2024) Summarizing multiple aspects of triple collocation analysis in a single diagram. Front. Remote Sens. 5:1395442. doi: 10.3389/frsen.2024.1395442
Received
03 March 2024
Accepted
12 June 2024
Published
16 July 2024
Volume
5 - 2024
Edited by
Danfeng Hong, Chinese Academy of Sciences (CAS), China
Reviewed by
Chun-Hsu Su, Bureau of Meteorology, Australia
Joshua Roundy, University of Kansas, United States
Updates
Copyright
© 2024 Siu, Zeng, Sorooshian, Cairns, Ferrare, Hair, Hostetler, Painemal and Schlosser.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Leong Wai Siu, leongwaisiu@arizona.edu
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.