metaSEM: an R package for meta-analysis using structural equation modeling

The metaSEM package provides functions to conduct univariate, multivariate, and three-level meta-analyses using a structural equation modeling (SEM) approach via the OpenMx package in the R statistical platform. It also implements the two-stage SEM approach to conducting fixed- and random-effects meta-analytic SEM on correlation or covariance matrices. This paper briefly outlines the theories and their implementations. It provides a summary on how meta-analyses can be formulated as structural equation models. The paper closes with a conclusion on several relevant topics to this SEM-based meta-analysis. Several examples are used to illustrate the procedures in the supplementary material.


INTRODUCTION
Meta-analysis is a popular technique for synthesizing research findings in the social, behavioral, educational, and medical sciences (e.g., Hedges and Olkin, 1985;Whitehead, 2002;Borenstein et al., 2009;Schmidt and Hunter, 2015). There are several standalone programs for conducting meta-analyses, e.g., Comprehensive Meta-Analysis (Borenstein et al., 2005). There are also macros or packages to fit some meta-analytic models in standard statistical packages such as SPSS (Lipsey and Wilson, 2000), and SAS (Arthur et al., 2001). R (R Development Core Team, 2014) is a popular open source statistical platform for computations and data analysis. There are also several R packages available for meta-analysis (e.g., Viechtbauer, 2010;Lumley, 2012;Schwarzer, 2014).
The metaSEM package (Cheung, 2014b) is another R package for conducting meta-analyses. It formulates univariate, multivariate, and three-level meta-analytic models as structural equation models (Cheung, 2008(Cheung, , 2013b(Cheung, , 2014c via the OpenMx package (Boker et al., 2011). It also implements the two-stage structural equation modeling (TSSEM) approach Chan, 2005, 2009;Cheung, 2014a) to fit fixed-and randomeffects meta-analytic structural equation modeling (MASEM) on correlation or covariance matrices. This paper outlines the metaanalytic models implemented in the metaSEM package (Cheung, in press). There are two main objectives of this paper. First, it provides an succinct summary on how various meta-analytic models can be formulated as structural equation models. Readers may refer to the references for more details and advantages of formulating meta-analytic models as structural equation models. Second, it illustrates how to conduct these analyses using the metaSEM package. Complete R code, output, and remarks are included in the supplementary material. Users may refer to http:// courses.nus.edu.sg/course/psycwlm/Internet/metaSEM/ on how to install the metaSEM package.

STRUCTURAL EQUATION MODELING BASED META-ANALYSIS
SEM is a multivariate technique to fit and test hypothesized models. Let y be a p × 1 vector of a sample of continuous data from a multivariate normal distribution where p is the number of observed variables. It is hypothesized that the model for the first and the second moments are functions of θ, where θ is a vector of parameters that can be regression coefficients, error variances, factor loadings, and factor variances. The model is: where μ and are the population mean vector and covariance matrix, respectively. Maximum likelihood (ML) estimation method is the most common estimation method in SEM. The −2 * log-likelihood (−2LL) for the ith case is, where p i is the number of filtered variables with complete data in the ith case, μ i (θ) and i (θ) are the model implied mean vector and the model implied covariance matrix for the ith case, respectively. Since there is a subscript i in Equation 2, the model implied mean vector and covariance matrix may vary across cases. Thus, it automatically handles incomplete data by selecting the complete data in the log-likelihood function with the full information maximum likelihood (ML or FIML) estimation method (Enders, 2010).
To obtain the parameter estimates, we may take the sum of the −2LL i over all cases and minimize it. Iterative methods are used to obtain the parameter estimates. When it is convergent, the asymptotic sampling covariance matrix of the parameter estimates may be obtained from the inverse of the Hessian matrix. The standard errors (SEs) of the parameter estimates are calculated by taking the square root of the diagonal elements of the asymptotic sampling covariance matrix. The parameter estimates divided by their SEs follow a z distribution under the null hypothesis. A likelihood ratio (LR) statistic may also be used to compare nested models. The model fit and the significance of individual parameters can be tested (e.g., Kline, 2011).

UNIVARIATE FIXED-EFFECTS MODEL
The following subsections briefly introduce how various metaanalytic models can be formulated as structural equation models. Let us begin with the meta-analytic model with only one effect size y i in the ith study (Cheung, 2008). y i can be any effect size, such as the odds ratio, raw mean difference, standardized mean difference, correlation coefficient, or its Fisher's z transformed score. When the sample sizes in the primary studies are reasonably large, y i can be assumed to be normally distributed with a variance of v i (e.g., see Borenstein et al., 2009, for the formulas of common effect sizes). The univariate fixed-effects model for the ith study is: where β F is the common effect under the fixed-effects model, and Var(e i ) = v i is the known sampling variance. To conduct a univariate fixed-effects meta-analysis in SEM, we may fit the following model implied moments: Since v i is known, the only parameter in the model is β F . Figure 1 shows the graphical model of the fixed-effects meta-analysis.

UNIVARIATE RANDOM-EFFECTS MODEL
Since the primary studies are conducted by different researchers in different settings, these studies are unlikely not direct replicates of each other. It is reasonable to expect that the population effect sizes may not be the same. A random-effects model allows studies to have their own study-specific effect. The model for the ith study is: where β R is the average population effect under the randomeffects model, and Var(u i ) = τ 2 is the heterogeneity variance that has to be estimated. To fit the model in SEM, we may consider the following model implied moments: In the literature of meta-analysis, v i and τ 2 + v i are known as the conditional and the unconditional variances, respectively. Under this model we have to estimate both β R and τ 2 . Figure 2 shows the graphical model of the random-effects meta-analysis. Various estimation methods, such as methods of moments, ML estimation and restricted maximum likelihood (REML) estimation may be used to estimate τ 2 (e.g., Borenstein et al., 2009). The default estimation method in the SEM-based meta-analysis is ML estimation, while the REML estimation method may also be used to minimize the slight negative bias on the estimated variance component using the ML estimation method (Cheung, 2013a).

Quantifying heterogeneity
To test the homogeneity of the population effect sizes, we may compute the Q statistic (Cochran, 1954), where w i = 1/v i . Under the null hypothesis of the homogeneity of effect sizes, the Q statistic has an approximate chi-square distribution with (k − 1) degrees of freedom (df s). The Q statistic may be significant simply because of the large number of studies. Conversely, a large Q statistic may be non-significant because of the small number of studies. Therefore, the significance of the Q statistic should not be used to determine whether a fixed-or a random-effects model is used in the analysis. One popular index quantifying the degree of heterogeneity of effect sizes is the I 2 (Higgins and Thompson, 2002). The general formula is whereṽ is a typical within-study sampling variance. I 2 can be interpreted as the proportion of the total variation of the effect size that is due to the between study heterogeneity. Higgins and Thompson (2002) defined the typical within-study sampling variance using the Q statistic: One advantage of usingṽ Q as the typical within-study sampling variance is that I 2 can be simplified to I 2 Q = Q − (k − 1)/Q. Two more definitions ofṽ have also been proposed in the literature. Takkouche et al. (1999) suggested that the harmonic mean of v i can be used as the typical within-study sampling variance, Xiong et al. (2010) also discussed an estimator of I 2 that is based on the arithmetic mean:ṽ All of the above definitions are available in the metaSEM package. Users may choose among them by specifying the argument I2 = "I2q" based on the Q statistic (the default), I2 = "I2hm" based on the harmonic mean, and I2 = "I2am" based on the arithmetic mean.

UNIVARIATE MIXED-EFFECTS MODEL
The mixed-effects meta-analysis extends the random-effects meta-analysis by using study characteristics as predictors.
Assuming that x i is an (m + 1) × 1 vector of predictors including a constant of 1 where m is the number predictors in the ith study, the mixed-effects model is: where β is a a (m + 1) × 1 vector of regression coefficients including the intercept. To fit the model in SEM, we may use the following model implied conditional mean and variance: (13) Figure 3 shows the graphical model of the mixed-effects metaanalysis with one predictor. A phantom variable P is introduced to specify the predictor x i . Since x i is specified via definition variables (see Cheung, 2010), x i is treated as a design matrix rather than as variables.

FIGURE 3 | Univariate mixed-effects meta-analysis with one predictor.
Mathematically, it is clear that the random-effects metaanalysis is a special case of the mixed-effects meta-analysis by fixing x = 1 as a constant of ones, while the fixed-effects metaanalysis is a special case of the random-effects meta-analysis by fixing τ 2 = 0. It should be noted that the assumptions and interpretations on the fixed-and random-effects models are different.

Explained variance
Besides testing whether the predictors are significant, researchers may want to quantify the degree of prediction. The percentage of variance explained by the inclusion of predictors, can be calculated by comparing theτ 2 0 without a predictor and theτ 2 1 with predictors (Raudenbush, 2009). When the calculated R 2 is negative, it is usually truncated to zero.

MULTIVARIATE META-ANALYSIS
When the research questions become more complicated, a single effect size may not be sufficient to summarize the effect in the primary studies. Multiple effect sizes are required to quantify the effect of the studies. Let us assume that there are a total of p effect sizes with m predictors in k studies. Since it is likely that different numbers of effect sizes are reported in the primary studies, we assume that there are p i effect sizes in the ith study. The model for the multivariate mixed-effects meta-analysis in the ith study is: where y i is a p i × 1 vector of effect sizes, B i is a p i × (m+1) matrix of regression coefficients including the intercepts, x i is a (m + 1) × 1 matrix of predictors including 1 in the first column, Z i is a p i × p filter matrix selecting the effect sizes that are present, u i is a p × 1 study-specific random effects, and e i is a p i × 1 sampling error. We assume that Var(e i ) = V i is known in the ith study and that Var(u i ) = T 2 is the variance component of the between-study heterogeneity that has to be estimated. The model handles missing effect sizes by selecting the complete effect sizes only in the above equation. Since x i is a design matrix, missing value is not allowed in x i . When there are missing values in x i , the whole study will be deleted before the analysis is conducted.
The −2LL of the above model is: To fit the multivariate mixed-effects meta-analysis in SEM, we use the following model implied conditional mean vector and covariance matrix (Cheung, 2013b): www.frontiersin.org January 2015 | Volume 5 | Article 1521 | 3 Cheung metaSEM FIGURE 4 | Multivariate mixed-effects meta-analysis with two effect sizes per study and one predictor. Figure 4 shows the graphical model of the multivariate mixedeffects meta-analysis with two effect sizes per study and one predictor. A phantom variable P is introduced to specify the predictor x i . The multivariate random-effects meta-analysis is a special case of the multivariate mixed-effects meta-analysis by using X i = 1 as the design matrix; the random-effects meta-analysis is a special case of the fixed-effects meta-analysis by fixing T 2 = 0. Moreover, the univariate meta-analysis is also a special case of the multivariate meta-analysis with only one effect size. The I 2 and R 2 in a univariate meta-analysis may also be calculated for each effect size in a multivariate meta-analysis.

THREE-LEVEL META-ANALYSIS
Effect sizes are assumed to be independent in most meta-analytic models. However, the effect sizes can be non-independent for various reasons. For example, the effect sizes reported in the same study may be more similar than the effect sizes reported in other studies. When the degree of dependence is known, the multivariate meta-analysis introduced in the Section 2.4 can be used to model the dependence. When the degree of dependence is not known, a three-level meta-analytic model may be used to address the dependence among the effect sizes (e.g., Konstantopoulos, 2011;Cheung, 2014c;Van den Noortgate et al., 2013). The model is: where y ij is the effect size for the ith effect size in the jth cluster, β is an (m + 1) × 1 vector of regression coefficients including the intercept, x ij is the (m + 1) × 1 predictors including 1 in the first element for the ith study at the jth cluster, u (2)ij and u (3)j are the random effects at level 2 and level 3, respectively, and Var(e ij ) = v ij is the known sampling variance of the effect size. To fit the three-level meta-analytic model in SEM, we may use the following model implied moments for the conditional mean and variance: where Var(u (2)ij ) = τ 2 (2) and Var(u (3)j ) = τ 2 (3) are the heterogeneity at level 2 and level 3, respectively (Cheung, 2014c).

Quantifying heterogeneity and explained variance
Similar to the I 2 defined in a random-effects meta-analysis, we may define the degree of heterogeneity for a three-level metaanalysis in level 2 and level 3 as, whereṽ is the typical within-study sampling variance defined in a random-effects meta-analysis. I 2 (2) and I 2 (3) can be interpreted as the proportion of the total variation of the effect size that is due to the level 2 and level 3, respectively. Sinceṽ is sample specific, one limitation of I 2 (2) and I 2 (3) is that they are not estimating any population quantities. Cheung (2014c) introduced two intra-class correlations (ICCs), and Both ICC (2) and ICC (3) are estimating their population counterparts τ 2 (2) /(τ 2 (2) + τ 2 (3) ) and τ 2 (3) /(τ 2 (2) + τ 2 (3) ), respectively. ICC (2) and ICC (3) can be interpreted as the percentage of the population heterogeneity due to level 2 and level 3, respectively.
When there are predictors, we may calculate the R 2 for level 2 and level 3 in a similar manner to that defined before, When the estimates are negative, they are usually truncated to zero.

META-ANALYTIC STRUCTURAL EQUATION MODELING
SEM is a popular modeling techniques in the social and behavioral sciences. When there are more and more studies addressing similar research questions using similar variables, there is a need to compare and synthesize these findings. MASEM combines ideas of meta-analysis and SEM by pooling correlation (or covariance) matrices and testing structural equation models on the pooled correlation (or covariance) matrix (e.g., Viswesvaran and Ones, 1995;Cheung and Chan, 2005;Becker, 2009). There are two stages in conducting the analysis. In the first stage of the analysis, the correlation (or covariance) matrices are pooled together.
In the second stage of the analysis, the pooled correlation (or covariance) matrix is used to fit structural equation models. Chan (2005, 2009) proposed a fixed-effects TSSEM. The fixed-effects TSSEM approach has been extended to the random-effects TSSEM by Cheung (2014a). Regardless of whether a fixed-or a random-effects model is used, the metaSEM package handles this automatically. In other words, parameter estimates, SEs, and goodness-of-fit indices in the stage 2 analysis have already taken the stage 1 model into account.

STAGE 1 ANALYSIS
The main objective of the stage 1 analysis is to pool the correlation (or covariance) matrices together. There are two classes of models in meta-analysis-fixed-effects models and randomeffects models (see Hedges and Vevea, 1998;Schmidt et al., 2009). Fixed-effects models are used for conditional inferences based on the selected studies. They are intended to draw conclusions on the studies included in the meta-analysis. Researchers are mainly interested in the studies used in the analysis. The assumption in fixed-effects models is usually, but not always, that all studies share common effect sizes. The stage one analysis in both the fixed-and the random-effects TSSEM is based on the ML estimation method. Thus, the parameter estimates are unbiased and efficient when the missing correlation coefficients are missing completely at random (MCAR) or missing at random (MAR) (e.g., Enders, 2010).

Fixed-effects model
Under the fixed-effects (or more correctly the common effects) model, it is assumed that the population correlation (or covariance) matrices are the same while there are study-specific correlation (or covariance matrices) under the random-effects model. To simplify the presentation, I will mainly focus on the analysis of correlation matrices. Generalizing to analysis of covariance matrices is a straight-forward process (see Cheung and Chan, 2009). A covariance matrix in the ith study can be decomposed into a product of the matrices of correlations and standard deviations: where i (θ) is the model implied covariance matrix, D i is the diagonal matrix of standard deviations, and P i is the correlation matrix. Under the assumption of the homogeneity of correlation matrices, we may obtain a common correlation matrix by imposing the constraint P = P 1 = P 2 = . . . = P k , where D i may vary across studies. When there are missing correlations, the missing data are filtered out. If we want to obtain a common covariance matrix under the assumption of the homogeneity of covariance matrices, we may also add the constraint D = D 1 = D 2 = . . . = D k . An LR statistic can be used to test the null hypothesis of homogeneity of correlation matrices P 1 = P 2 = . . . = P k . Moreover, various goodness-of-fit indices may also be used to evaluate the appropriateness of the "close" fit of the homogeneity of correlation matrices.

Random-effects model
Since the primary studies are independently conducted by different researchers, the samples, measures, and research focuses are likely different. The assumption of homogeneity of correlation matrices may not be reasonable. A random-effects TSSEM is usually more appropriate to analyze the data (Cheung, 2014a). When a random-effects model is used, the correlation matrices are treated as vectors of multivariate effect sizes. Let r i = vechs(R i ) be the p(p − 1)/2 × 1 vector of a sample correlation for p variables by taking the column-wise non-redundant elements from R i . If an analysis of the covariance matrices is conducted, the p(p + 1)/2 × 1 vectorized multivariate effect sizes become s i = vech(S i ).
The model for the sample correlation vector r i is: where ρ R is the p(p − 1)/2 × 1 vector of average population correlation vector under a random-effects model, Var(u i ) = T 2 is the variance components of the random effects, and Var(e i ) = V i is the known conditional sampling covariance matrix. The multivariate random-effects meta-analysis introduced in Section 2.4 may be used to conduct the stage 1 analysis with a random-effects model. When there are many variables or not enough data (studies) in the analysis,T 2 can be non-positive definite. The results cannot be trusted. One workaround is to fix T 2 to a diagonal matrix rather than as a symmetric matrix. This can be done easily by specifying the argument RE.type = "Diag" when calling the tssem1() function.

STAGE 2 ANALYSIS
After the stage 1 analysis with either a fixed-or a randomeffects model, a vector of pooled correlations r and its asymptotic covariance matrix V are available after the analysis. It should be noted thatT 2 is not directly involved in fitting the correlation structure in the stage 2 analysis. However, the presence of T 2 is required so that the heterogeneity of the random effects has been properly taken in the stage 1 analysis.
Most applications of MASEM use the pooled correlation matrix as if it was an observed correlation matrix to fit structural equation models. Cheung and Chan (2005) discussed some of these problems. For example, the elements of the pooled correlation matrix are usually based on different studies. Researchers usually use an ad-hoc sample size, such as the harmonic or arithmetic means of the individual sample sizes, as the sample size in fitting structural equation models. Unless all the correlation coefficients are based on the same number of studies, the precision of some correlation coefficients are over-estimated while others are under-estimated. Another issue is that the pooled correlation matrix is analyzed as it was a covariance matrix. It is generally incorrect to analyze the correlation matrix in SEM, although most published articles using MASEM have treated the pooled correlation matrix as a covariance matrix. Many SEM experts (e.g., Cudeck, 1989) have warned about the problems of analyzing the correlation matrix instead of the covariance matrix in primaryresearch applications of SEM. Specifically, the chi-square statistics and (or) the SEs of parameter estimates may be incorrect.
The TSSEM approach addresses these issues. The weighted least square (WLS) estimation is used to fit the proposed models in the stage 2 analysis. A correlation structural model ρ(γ ) is fitted with the WLS estimation method by minimizing the following fit function, F(γ ) = (r − ρ(γ )) V −1 (r − ρ(γ )).
An LR statistic and various goodness-of-fit indices may be used to judge whether the proposed structural model is appropriate, while SEs may be used to test the significance of individual parameter estimates.

CONCLUSION
This paper introduced various meta-analytic models using a SEM approach. More importantly, these models have all been implemented in the metaSEM package which is freely available as an R package. Due to space constraint, I did not include topics, such as using REML as the estimation method (see Cheung, 2013a), constructing likelihood-based confidence interval (LBCI) (see , and using alternative random-effects MASEM (see Cheung and Cheung, under review). Readers may refer to the relevant papers and the metaSEM package for details. Some of these models can be implemented in standard SEM software such as Mplus (Muthén and Muthén, 2012). Since SEM software was not designed for meta-analysis, transformations on the effect sizes are required to meet the distribution assumptions (see e.g., Cheung, 2008Cheung, , 2013b. To conclude, SEM provides a flexible framework to develop meta-analytic techniques. Many of the techniques available in SEM can be easily extended to metaanalysis. The supplementary material include some examples on how to analyze these models using the metaSEM package.

FUNDING
Preparation of this paper was supported by the Academic Research Fund Tier 1 (FY2013-FRC5-002; R-581-000-158-112) from the Ministry of Education, Singapore.