Has the Impact of Rising CO₂ on Plants been Exaggerated by Meta-Analysis of Free Air CO₂ Enrichment Studies?

Meta-analysis is a statistical approach that combines the findings of multiple experimental studies to quantify a population effect (Field and Gillett, 2010; Quintana, 2015). This technique has become increasingly popular to gage plant responses to carbon dioxide (Long et al., 2004; Ainsworth and Long, 2005; Ainsworth and Rogers, 2007), ozone (Feng et al., 2008), nutrient status (Koricheva et al., 1998), herbivory (Hawkes and Sullivan, 2001) and drought (Pinheiro and Chaves, 2011). The synthesis of pools of data from related studies should in theory permit more accurate prediction of the impact of environmental change on plants. Indeed, the results of meta-analytic studies are increasingly used to model plant responses to climate change and inform perspectives on the likely impacts on photosynthesis, carbon sequestration, and food security (Long et al., 2004, 2006; Ainsworth, 2008; Wu et al., 2011). Here, we illustrate how the limitations of this approach are not being critically applied in the plant sciences. One area where meta-analysis has been widely utilized is in the study of plant responses to increased atmospheric carbon dioxide concentration ([CO₂]) in free air [CO₂] enrichment (FACE) studies (e.g., Long et al., 2004; Ainsworth and Long, 2005; Ainsworth and Rogers, 2007; Bishop et al., 2014). We use the meta-analysis of FACE experiments as an example of the limitations inherent in this approach that result in an overemphasis of the effect of [CO₂], and thus distort our understanding of crop responses to [CO₂]. We acknowledge that growth under FACE has a direct impact upon photosynthesis and growth through CO₂-fertilization; however, meta-analytic approaches have exaggerated the predicted impact of rising [CO₂].

Meta-analysis utilizes the effect size of numerous studies to produce an average effect size for a given factor (Field and Gillett, 2010; Quintana, 2015). As such, the meta-analysis is entirely dependent upon the input of studies, and whether those studies represent a true reflection of the treatment effect size. The most highly cited (Long et al., 2004; Ainsworth and Long, 2005; Ainsworth and Rogers, 2007) and recent meta-analytic studies (Bishop et al., 2014; Baig et al., 2015) of plant responses to FACE rely upon data from peer-reviewed studies indexed in the ISI Web of Science and/or Scopus. However, the possibility of reporting bias influencing the selection of studies is not considered. The issue of reporting bias is widely acknowledged in medicinal science; it is estimated that studies that demonstrate a positive effect are 94% more likely to be submitted (Greenwald, 1975) and then published (Coursol and Wagner, 1986) in leading journals. These journals are most likely to be indexed and their studies included in meta-analyses (Figure 1). This skew toward positive studies is driven by publication bias (where journals prefer to publish positive studies), data availability bias (studies with a large effect size are more likely to be written up in comparison to those where the replication is insufficient to demonstrate a significant effect) and reviewer bias (where reviewers favor manuscripts reporting strong treatment effects confirming a prevailing consensus; Dwan et al., 2013). Funnel plots are one of the most common methods to observe possible reporting bias in meta-analysis datasets. Asymmetry in funnel plots is indicative of bias and can be assessed using regression (Egger et al., 1997), rank correlation (Begg and Mazumdar, 1994) and the ‘trim and fill’ method [where estimated ‘missing studies’ are imputed to create a more symmetrical funnel plot (Duval and Tweedie, 2000)].

FIGURE 1

To test for and assess the possible impacts of bias in FACE studies, we followed the methodology of previous meta-analysis of FACE by analysing data from studies indexed in the ISI Web of Science (Long et al., 2004; Ainsworth and Long, 2005; Ainsworth and Rogers, 2007; Bishop et al., 2014; Baig et al., 2015). We compiled photosynthesis, stomatal conductance and yield data from 103 studies of C3 herbaceous plants to FACE (a full list of articles and species used in the meta-analysis is given in Supplementary Information). We then performed a random effects meta-analysis using the metafor package (Viechtbauer, 2010) in R statistical software following Field and Gillett (2010) and Quintana (2015) (Figure 2). Bias in the dataset was assessed using regression (Egger et al., 1997), rank correlation (Begg and Mazumdar, 1994), trim and fill (Duval and Tweedie, 2000) and weighting analysis of the studies (Vevea and Woods, 2005).

FIGURE 2

Duval and Tweedie, 2000) uses the existing data to impute estimated ‘missing studies’ that are represented as gray symbols. The black solid vertical line indicates the mean effect size of the meta-analysis after the trim and fill. The dashed vertical line indicates the mean effect size computed by the meta-analysis before the ‘missing studies’ were imputed. The difference between the black solid line and dashed vertical line represents the effect of reporting bias on effect size as indicated by the trim and fill method. The gray box below the funnel plot shows the Begg – Mazumdar (Begg and Mazumdar, 1994) rank correlation coefficient using Kendall’s τ and Egger’s regression test (Egger et al., 1997) to assess the probability of publication bias within the datasets. To gage the impact of hypothetical publication bias in the literature on the meta-analysis of photosynthesis (d), stomatal conductance (e) and yield (f) we included increasing numbers of studies with randomly generated small effect sizes (r) of -0.1 to 0.1 (Cohen, 1992). The black solid line indicates the mean effect size for the meta-analysis and the gray shading either side represents 95% confidence intervals. Solid circles indicate the points where moderate (labeled ms) and severe (labeled ss) selection calculated using the model of Vevea and Woods (2005) would occur.

Begg and Mazumdar’s (1994) rank correlation and Egger’s et al. (1997) regression test indicated significant asymmetry in the funnel plots suggestive of reporting bias for all three parameters. The inclusion of estimated missing studies using the trim and fill method (Duval and Tweedie, 2000) resulted in a more balanced spread of the data and also reduced the effect size of FACE on photosynthesis by 43%, stomatal conductance by 32% and yield by 41%. The model of Vevea and Woods (2005) performs a sensitivity analysis, applying weight functions of the effect sizes of studies within a meta-analysis to determine the impact of moderate or severe reporting bias on effect size. Assuming that our dataset has experienced moderate selection, this would indicate that reporting bias has induced 5 to 15% increases in effect size.

It is particularly difficult to quantify the true effect of bias on a meta-analysis (Field and Gillett, 2010; Quintana, 2015). It is possible to survey non-indexed so-called ‘gray’ literature that is not subject to peer-review, directly approach researchers for non-significant unpublished data or submit contrasting ‘sample’ articles or questionnaires to journals to quantify rates of acceptance/rejection. However, all of these methods are time consuming and subject to limitations. We therefore decided to assess the potential impact of bias on meta-analysis of FACE by incorporating an increasing proportion of studies showing small effect sizes (randomly generated r values of -0.1 to 0.1: Cohen (1992) and re-running the meta-analyses as a ‘sensitivity test’ of the published data). Assuming that the current published literature is not subject to any bias, photosynthesis (r = 0.542), stomatal conductance (r = -0.447), and yield (r = 0.398) all showed significant effects of elevated [CO₂], and the significance of this effect remained even at the highest levels of hypothetical reporting bias. A hypothetical publication bias of 30% induced reductions in [CO₂] effect size of 43.7% in photosynthesis, 27.6% in stomatal conductance and 27.5% in yield. The decline in effect size becomes more apparent at the 80–90% level found in medicinal science (Greenwald, 1975; Coursol and Wagner, 1986). Such reductions in effect size will have critical implications for studies where the output of meta-analyses are used to predict the photosynthetic (Ainsworth and Rogers, 2007) and yield (Long et al., 2006; Bishop et al., 2014; Challinor et al., 2014) responses of plants to rising [CO₂].

Our analysis is indicative of high levels of bias within published meta-analytic studies of plant responses to FACE that have resulted in over-estimation of the effect size of elevated [CO₂]. As a result the outputs of these studies should be treated with a degree of caution. We propose that sensitivity testing of meta-analytic studies of plant responses to FACE be undertaken as standard in the future (e.g., Vevea and Woods, 2005), and efforts made to further encourage the publication of studies reporting non-significant outcomes and compilation of non-significant data for researchers wishing to undertake meta-analysis.

Statements

Summary

Keywords

photosynthesis, stomatal conductance, yield, food security, atmospheric CO₂, elevated CO₂

Citation

Haworth M, Hoshika Y and Killi D (2016) Has the Impact of Rising CO₂ on Plants been Exaggerated by Meta-Analysis of Free Air CO₂ Enrichment Studies?. Front. Plant Sci. 7:1153. doi: 10.3389/fpls.2016.01153

Received

14 June 2016

Accepted

19 July 2016

Published

03 August 2016

Volume

7 - 2016

Edited by

Urs Feller, University of Bern, Switzerland

Reviewed by

Eero Nikinmaa, University of Helsinki, Finland; Matthew Paul, Rothamsted Research, UK

Updates

Check for updates

Copyright

© 2016 Haworth, Hoshika and Killi.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Matthew Haworth, haworth@ivalsa.cnr.it

This article was submitted to Agroecology and Land Use Systems, a section of the journal Frontiers in Plant Science

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.