MetaWin 3: open-source software for meta-analysis
Michael S. Rosenberg- Center for Biological Data Science, Virginia Commonwealth University, Richmond, VA, United States
The rise of research synthesis and systematic reviews over the last 25 years has been aided by a series of software packages providing simple and accessible GUI interfaces which are intuitively easy to use by novice analysts and users. Development of many of these packages has been abandoned over time due to a variety of factors, leaving a gap in the software infrastructure available for meta-analysis. To fulfill the continued demand for a GUI-based meta-analytic system, we have now released MetaWin 3 as free, open-source, multi-platform software. MetaWin3 is written in Python and developed from scratch relative to earlier versions. The codebase is available on Github, with pre-compiled executables for both Windows and macOS available from the MetaWin website. MetaWin includes standardized effect size calculations, exploratory and publication bias analyses, and allows for both simple and complex explanatory models of variation within a meta-analytic framework, including meta-regression, using traditional least-squares/moments estimation.
1 Introduction
Research synthesis is generally defined as a review of primary research with the intent to integrate findings; meta-analysis is a particular form of quantitative research synthesis with a focus on combining and comparing effect sizes across studies (Glass, 1976; Hedges and Olkin, 1985; Koricheva and Gurevitch, 2013). While meta-analytical studies have been particularly popular in the medical and social sciences, they have been performed for virtually every area of academic research.
The rise of quantitative research synthesis and meta-analysis over the last 25 years was aided by a series of software packages providing simple and accessible GUI interfaces which are intuitively easy to use for novice analysts and users. MetaWin (Rosenberg et al., 1997) was first published as a small, commercial software package that made meta-analytical calculations more accessible to the burgeoning research synthesis community, particularly in the ecological sciences, and helped introduce the use of resampling methods into the meta-analytic statistical repertoire (Adams et al., 1997). Version 2 of the software (Rosenberg et al., 2000) was substantially expanded over the original version, easier to use, and more flexible and powerful. These software have been cited thousands of times in fields including agriculture, anthropology, biology, business, chemistry, economics, education, engineering, forestry, geography, geology, medicine, physics, and psychology, among others.
Versions 1 and 2 of MetaWin were written in Pascal and Delphi and depended on several commercial licensed packages. These developmental environments and decisions combined to restrict the software to the Windows operating system and prevented any practical open-source release as the software was otherwise uncompilable without these components. These also served to restrict further development and updates as the developmental components and systems gradually became outdated and incompatible with newer operating systems.
Despite these limitations, MetaWin has continued to be regularly used and cited for over 20 years after its original release; would-be users still regularly request copies of this relatively ancient software. The long-term popularity has likely been driven by the simple GUI-interface, which contrasts it with powerful, more difficult-to-use alternatives available in R. In an attempt to fill in the gap between MetaWin 2 and the R-based meta-analytic community, OpenMEE (Wallace et al., 2016) was created as open-source, cross-platform software with a GUI interface, but which used R computation on the backend. OpenMEE appeared to fill many of the needs that MetaWin served, but the software became quickly abandoned with development apparently halted in 2016. To fulfill the continued demand for a GUI-based meta-analytic system, we have now released MetaWin 3.
2 Materials and methods
MetaWin 3 is free, open-source, and multi-platform, unlike its predecessors. It has been written from scratch relative to the earlier versions, entirely in Python, with the codebase openly available on Github and pre-compiled (using PyInstaller) executables for both Windows and macOS available from the MetaWin website. The code has minimal external dependencies, and relies on only four established, and heavily used Python packages: PyQt6, NumPy, SciPy, and Matplotlib.
The focus of MetaWin is to provide access to meta-analytic fundamentals with an easy-to-use GUI. Output includes citations and references to appropriate literature sources based on method choices made by the users; similarly, all graphical output includes auto-generated sample captions (including references, when appropriate) to ease in interpretation (see below). MetaWin imports and exports data from tables (row × column) in standard text formats (e.g., CSV). Textual output can be exported as plain text, HTML, or markdown. Figures can be exported in a variety of standard graphical formats, including both common vector (e.g., SVG, EPS) and raster (e.g., PNG, TIF, JPG) options.
The major functions in MetaWin are roughly divided into four primary categories: effect size calculations, publication bias exploration, meta-analytic computation, and additional graphical output.
Effect size calculations are for standardized effect measures well established in the meta-analytic literature (Table 1). Effect size calculation is optional within the MetaWin framework, as a user can always import a pre-calculated effect size and its variance for use in the general analytical computation.
TABLE 1. Standardized effects size metrics included within MetaWin.
Publication bias exploration (Table 2) includes both graphical approaches (e.g., funnel plots) and analytical methods such as Egger’s Regression, Rank Correlation Tests, and Trim-and-Fill Analysis. Fail-safe number calculation is included as a subcomponent of basic meta-analysis (see below).
TABLE 2. Publication bias methods included in MetaWin.
There are currently 8 primary analytical methods implemented within MetaWin 3, three of which represent simple and exploratory models, while the other five represent more complex explanatory structural models (Table 3), frequently referred to as meta-regression. These include a simple linear model or more complex multivariate models using a general linear model (GLM) regression framework. Most of the methods allow for both fixed- or random-effects variance model implementation; a mixed-effects model meta-analysis (Mengersen et al., 2013) is performed by using the grouped meta-analysis structure with random-effects variance chosen. Heterogeneity is estimated using both Q- and I2-statistics (Hedges and Olkin, 1985; Higgins and Thompson, 2002; Huedo-Medina et al., 2006). Most analyses include one or more optional resampling tests for determining confidence intervals or significance testing (Adams et al., 1997). All of these methods currently implement traditional moments/least-squares estimators for meta-analysis (Rosenberg, 2013). The basic analysis also includes fail-safe number estimation (Rosenthal, 1979; Orwin, 1983; Rosenberg, 2005).
TABLE 3. Primary meta-analytic methods included in MetaWin.
While most of the analytical methods can optionally produce an associated graph or figure, additional figures which have proven useful for meta-analysis can be created outside of the main methods (examples are shown in the panels of Figure 1). These include scatter plots, weighted histograms, forest plots, normal quantile plots (Wang and Bushman, 1998), and Galbraith radial plots (Galbraith, 1988; 1994). Figures are customizable and exportable. Additionally, the underlying data can be exported for re-creation in a user’s preferred plotting software. All figures include automatically generated captions to aid in interpretation and communication (Figure 2).
FIGURE 1. Examples of graphical output from MetaWin. (A) Funnel plot with contour confidence intervals. (B) A power enhancement funnel plot. (C) A forest plot of individual effect sizes and the overall mean and median, including confidence intervals based on both an assumption of a normal distribution and bootstrapping. (D) A forest plot of a cumulative meta-analysis.
FIGURE 2. Example of the graphical output from a Trim-and-Fill Analysis. The following caption is automatically generated by MetaWin as part of the figure creation, including the associated reference information: “Funnel plot of effect vs. precision, showing the results of a Trim and Fill Analysis (Duval and Tweedie, 2000a; Duval and Tweedie, 2000b). Original data are represented by black circles, inferred “missing” data by open circles with a fire engine red border. The dashed silver line represents the mean effect size of the original data, the dashed fire engine red line the mean effect size including the inferred data.”
3 Results and discussion
The immediate goal of the current release of MetaWin was to minimally replicate what older versions of the software could do in a modern package, along with adding some obvious enhancements reflecting methodological advancement from the past 20 years. Over half of the analyses listed in Tables 2, 3 were not available in earlier versions of the software. This package now forms a platform to add additional features and analytical methods, for example, potentially including maximum likelihood (Mengersen and Schmid, 2013) and/or Bayesian inference (Schmid and Mengersen, 2013) solutions, which would allow greater flexibility in model specification and distributional assumptions then the currently implemented least-squares approach. Many additional analytical ideas and approaches in meta-analysis have been developed across a broad research synthesis community and could be added to MetaWin in the future, depending on demand, fit, and computational complexity, including additional effect size metrics, analysis models, plots, publication-bias and outlier estimators, etc.
4 Availability
MetaWin is free and released under a GPL-3.0 license. Pre-compiled versions of MetaWin for both Windows and macOS can be downloaded directly from https://www.metawinsoft.com. The entire codebase is open-source and available on Github at https://www.github.com/msrosenberg/MetaWin. The help manual can be found on the MetaWin website and is also accessible offline bundled within the pre-compiled executables.
Data availability statement
Publicly available datasets were analyzed in this study. This data can be found here: https://www.github.com/msrosenberg/MetaWin.
Author contributions
MR: Conceptualization, Methodology, Project administration, Resources, Software, Validation, Writing–original draft.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported in part by Virginia Commonwealth University.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Adams, D. C., Gurevitch, J., and Rosenberg, M. S. (1997). Resampling tests for meta-analysis of ecological data. Ecology 78, 1277–1283. doi:10.1890/0012-9658(1997)078[1277:rtfmao]2.0.co;2
Begg, C. B. (1994). “Publication bias,” in The handbook of research synthesis. Editors H. Cooper, and L. V. Hedges (New York: Russell Sage Foundation), 399–409.
Begg, C. B., and Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics 50, 1088–1101. doi:10.2307/2533446
Berlin, J. A., Laird, N. M., Sacks, H. S., and Chalmers, T. C. (1989). A comparison of statistical methods for combining event rates from clinical trials. Statistics Med. 8, 141–151. doi:10.1002/sim.4780080202
Chalmers, T. C. (1991). Problems induced by meta-analyses. Statistics Med. 10, 971–980. doi:10.1002/sim.4780100618
Dersimonian, R., and Laird, N. M. (1986). Meta-analysis in clinical trials. Control. Clin. Trials 7, 177–188. doi:10.1016/0197-2456(86)90046-2
Duval, S., and Tweedie, R. (2000a). A nonparametric Trim and fill method of accounting for publication bias in meta-analysis. J. Am. Stat. Assoc. 95, 89–98. doi:10.2307/2669529
Duval, S., and Tweedie, R. (2000b). Trim and Fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56, 455–463. doi:10.1111/j.0006-341x.2000.00455.x
Egger, M., Smith, G. D., Schneider, M., and Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629–634. doi:10.1136/bmj.315.7109.629
Galbraith, R. F. (1988). A note on graphical presentation of estimated odds ratios from several clinical trials. Statistics Med. 7, 889–894. doi:10.1002/sim.4780070807
Galbraith, R. F. (1994). Some applications of radial plots. J. Am. Stat. Assoc. 89, 1232–1242. doi:10.1080/01621459.1994.10476864
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educ. Res. 5, 3–8. doi:10.3102/0013189x005010003
Greenland, S. (1987). Quantitative methods in the review of epidemiologic literature. Epidemiol. Rev. 9, 1–30. doi:10.1093/oxfordjournals.epirev.a036298
Hedges, L. V., Gurevitch, J., and Curtis, P. S. (1999). The meta-analysis of response ratios in experimental ecology. Ecology 80, 1150–1156. doi:10.1890/0012-9658(1999)080[1150:tmaorr]2.0.co;2
Hedges, L. V., and Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press, Inc.
Higgins, J. P. T., and Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics Med. 21, 1539–1558. doi:10.1002/sim.1186
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., and Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol. Methods 11, 193–206. doi:10.1037/1082-989x.11.2.193
Koricheva, J., and Gurevitch, J. (2013). “Place of meta-analysis among other methods of research synthesis,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 3–13.
Kossmeier, M., Tran, U. S., and Voracek, M. (2020). Power-enhanced funnel plots for meta-analysis: the sunset funnel plot. Z. für Psychol. 228, 43–49. doi:10.1027/2151-2604/a000392
L'abbé, K. A., Detsky, A. S., and O'rourke, K. (1987). Meta-analysis in clinical research. Ann. Intern. Med. 107, 224–233. doi:10.7326/0003-4819-107-2-224
Lajeunesse, M. J., Rosenberg, M. S., and Jennions, M. D. (2013). “Phylogenetically independent meta-analysis,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 284–299.
Light, R. J., and Pillemer, D. B. (1984). Summing up: the science of reviewing research. Cambridge: Harvard University Press.
Mantel, N., and Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. J. Natl. Cancer Inst. 22, 719–748. doi:10.1093/jnci/22.4.719
Mengersen, K. L., and Gurevitch, J. (2013). “Using other metrics of effect size in meta-analysis,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 72–85.
Mengersen, K. L., and Schmid, C. H. (2013). “Maximum likelihood approaches to meta-analysis,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 125–144.
Mengersen, K. L., Schmid, C. H., Jennions, M. D., and Gurevitch, J. (2013). “Statistical models and approaches to inference,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 89–107.
Normand, S.-L. T. (1999). Meta-analysis: formulating, evaluating, combining, and reporting. Statistics Med. 18, 321–359. doi:10.1002/(sici)1097-0258(19990215)18:3<321::aid-sim28>3.3.co;2-g
Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. J. Educ. Statistics 8, 157–159. doi:10.2307/1164923
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., and Rushton, L. (2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J. Clin. Epidemiol. 61, 991–996. doi:10.1016/j.jclinepi.2007.11.010
Rosenberg, M. S. (2005). The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 59, 464–468. doi:10.1554/04-602
Rosenberg, M. S. (2013). “Moment and least-squares based approaches to meta-analytic inference,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 108–124.
Rosenberg, M. S., Adams, D. C., and Gurevitch, J. (1997). MetaWin: statistical software for meta-analysis with resampling tests, 1.0. Sunderland, Massachusetts: Sinauer Associates.
Rosenberg, M. S., Adams, D. C., and Gurevitch, J. (2000). MetaWin: statistical software for meta-analysis, 2.0. Sunderland, Massachusetts: Sinauer Associates.
Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychol. Bull. 86, 638–641. doi:10.1037/0033-2909.86.3.638
Schmid, C. H., and Mengersen, K. L. (2013). “Bayesian meta-analysis,” in Handbook of meta-analysis in ecology and evolution. Editors J. Koricheva, J. Gurevitch, and K. L. Mengersen (Princeton, NJ: Princeton University Press), 145–173.
Sterne, J. a.C., and Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J. Clin. Epidemiol. 54, 1046–1055. doi:10.1016/s0895-4356(01)00377-8
Wallace, B. C., Lajeunesse, M. J., Dietz, G., Dahabreh, I. J., Trikalinos, T. A., Schmid, C. H., et al. (2016). Open<scp>MEE</scp>: intuitive, open-source software for meta-analysis in ecology and evolutionary biology. Methods Ecol. Evol. 8, 941–947. doi:10.1111/2041-210x.12708
Keywords: effect sizes, meta-analysis, meta-regression, research synthesis, software
Citation: Rosenberg MS (2024) MetaWin 3: open-source software for meta-analysis. Front. Bioinform. 4:1305969. doi: 10.3389/fbinf.2024.1305969
Received: 02 October 2023; Accepted: 08 January 2024;
Published: 08 February 2024.
Edited by:
Sankar Subramanian, University of the Sunshine Coast, AustraliaReviewed by:
Marcel Van Tuinen, University of Groningen, NetherlandsMichail Tsagris, University of Crete, Greece
Copyright © 2024 Rosenberg. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Michael S. Rosenberg, msrosenberg@vcu.edu