We conducted a literature search in PubMed, Web of Science, PsycINFO, Scopus, and Google Scholar from the inception of the databases to October 15th, 2022 to identify eligible studies by querying all TMS studies (including original studies, reviews, and meta-analyses) on language comprehension and production in healthy participants, published in English. The search terms were any combination of (“Language, Syntax, Grammar, Semantics, Meaning, Phonetics, OR Phonology”) AND (“TMS” OR “transcranial magnetic stimulation”). The reference lists of previous reviews and meta-analyses (Vigneau et al., 2006; Flöel, 2012; Ren et al., 2014; Otal et al., 2015; Klaus and Schutter, 2018; Beynel et al., 2019; Bucur and Papagno, 2019; Hartwigsen and Saur, 2019; Hong et al., 2021; Zhang et al., 2021) were also screened in case any related studies were overlooked. The eligible study selection criteria are listed as following:

Each study underwent three screening steps for final inclusion: (1) Removal of duplicates (this was done by XQ, ZW, YC, QX, ZL, and LL); (2) Screening of titles and abstracts (XQ and ZW first independently evaluated all the studies and then conducted a check together. Studies met with disagreement were entered into the third step); (3) Full-text review (by XQ and ZW). Disagreement was resolved through group discussions among all authors. In addition, we also calculated Cohen’s kappa coefficient to assess inter-rater reliability (k = 0.803, p < 0.001), obtaining strong consistency (McHugh, 2012). The procedures of the study search and selection are depicted in Figure 1.

Thirty-seven eligible studies were identified. The means and standard deviations of RT and ACC, and the sample sizes were extracted. For each study, the following information was collected: literature information (authors and publication year), participant characteristics (sample size, gender, age, and native language), tasks, cortical targets, TMS protocols (methods of localization, stimulation types, timing, frequencies, and intensities), and study designs (control conditions and group designs). Table 1 provides an overview of the included studies.

XQ, ZW, YC, QX, ZL, and LL independently extracted data from each study. XQ and ZW further checked the extracted data together. Missing data pertinent to the current study were obtained and authorized by e-mailing the authors. The extracted data were recorded in excel spreadsheets. Data extraction and management were conducted manually.

We used mean and standard deviation to calculate Hedges’g (abbreviated as “g” hereafter) to estimate effect size, which provides a less biased estimate of the true effect than Cohens’d, especially for studies with small sample sizes (Hedges and Olkin, 1985). In line with Klaus and Schutter (2018), this meta-analysis mainly focused on the absolute effect sizes, that is, the magnitude of the neuromodulatory TMS effects regardless of the effect directions (i.e., improvement of behavioral performance as facilitation, and disruption of behavioral performance as inhibition). We performed data aggregation to avoid multiple similar data points from a single experiment entering the analysis, ensuring that each experiment provided only one measured value (Borenstein et al., 2009). Nevertheless, if several conditions were tested for more elaborate comparisons of the experimental variables within one experiment, the effect sizes would be calculated separately for each comparison (see also Klaus and Schutter, 2018). For example, if a study compared the conditions within various brain regions, for each brain region, the effect sizes were aggregated as one and included in the meta-analysis.

Since the included studies encompassed within-subject designs, it was difficult to assume statistical independence in the meta-analysis, and the issue of effect size dependency should be considered (Hunter and Schmidt, 2004). In previous studies, there were five commonly used methods to deal with correlated effect size: (a) Ignore dependencies; (b) Average dependent effect sizes across studies; (c) Extract only one effect size from each study; (d) Introduce the correlation coefficient r. However, these methods are suggested to be confronted with problems such as the exaggeration of relevant significance tests (Cheung, 2014), the lowering of the statistical power of meta-analysis and the precision of parameter estimation (Van den Noortgate et al., 2013) and too conservative estimation of coefficient value (Hedges and Pigott, 2001; Cheung, 2014). The fifth method (e) adopts modeling, such as the frequently used multilevel random effects model. Compared to the aforementioned four methods, using multilevel random effects model to estimate the effect sizes in meta-analysis is more accurate, effective and flexible. The model can incorporate multiple predictors to account for heterogeneity between studies or add additional random effects to address the various dependencies of effect sizes within and between studies (Fernández-Castilla et al., 2020). Therefore, to solve the problem of effect size dependency especially for within-subject designs, a three-level random effects model was adopted for the meta-analysis in this study (see Supplementary material for more details regarding the five methods and related rationales).

We also conducted the likelihood ratio test (LRT) to explore whether the three-level random effects model was more suitable for this meta-analysis compared to the traditional two-level random effects model. There was a significant difference between the traditional random effects model and the three-level random effects model in ACC (LRT = 9.99, p = 0.007), and a trend toward significance in RT (LRT = 2.80, p = 0.09). This indicated that compared with the traditional random effects model, the three-level model provided a better model fit.

Study heterogeneity was estimated by Cochran’s Q and p-value (Huedo-Medina et al., 2006), and all effect sizes were entered into the three-level random effects model. We also conducted sensitivity analysis to verify the robustness of the results.

According to the Cochrane guideline (Higgins et al., 2019), we used the Cochrane risk of bias tool (RoB 2.0, Sterne et al., 2019) to evaluate the quality of the included studies in the following aspects: (a) randomization process; (b) deviations from intended interventions; (c) missing outcome data; (d) measurement of the outcome; (e) selection of the reported result. Two researchers assessed the quality of each study. Studies met with disagreement were evaluated through pairing with a third party for group assessment.

Moreover, to account for the fact that the overall effect size might be overestimated because of publication bias (Rothstein and Bushman, 2012), we tested for a potential publication bias by constructing a “funnel plot”. If there was no bias, the graph would present an inverted funnel shape, and the distribution of the points (i.e., the included studies) would be roughly symmetrical. In case of a publication bias, the funnel would be skewed (Van den Bussche et al., 2009). However, concerns about the subjectivity of funnel plots have been raised. To ensure reliability, we further quantified publication bias by the Egger Test (Egger et al., 1997).

Moderator analyses were performed, since experimental parameters such as the language tasks (especially for different language functions, including semantic, syntactic, and phonological tasks), cortical targets [i.e., brain regions of interest, containing frontal, temporal, and parietal regions, as well as more specific regions including IFG, STG, and MTG. Please note that owing to the lack of adequate sample sizes, we did not perform meta-analysis on other specific regions such as the premotor cortex (PC), the anterior temporal lobe (ATL) and the angular gyrus (AG)], stimulation parameters of the TMS protocols [i.e., methods of brain region localization, stimulation types (further analysis on cTBS but not iTBS (only one study) was conducted considering the inadequate sample sizes), timing, frequencies, and intensities], and experimental designs (i.e., control conditions and group design types) are crucial for the examination of TMS effects on language performance in healthy participants. Effect sizes with an associated 95% confidence intervals (CI) were calculated when at least two studies were available for a particular estimate (see also Valentine et al., 2010). The analysis-structure of both the overall and the moderator analyses are summarized in Figure 2.

All effect size computations, summary analyses, sensitivity analysis, risk of bias tests and the publication bias tests were conducted by using the “metafor 3.0-2” (Viechtbauer, 2010) implemented in R (version 4.0.4, R Core Team, 2021). The results of all effect analyses of RT and ACC were graphically synthesized in forest plots.

Thirty-seven studies including 797 participants and 77 effect sizes were computed for the overall effect analysis. The results (see also Figures 3, 4) showed that for both RT and ACC, TMS could exert significant neuromodulatory effects on language performance in healthy participants (RT: g = 0.16, 95% CI: 0.04–0.29, Z = 2.496, p = 0.013; ACC: g = 0.14, 95% CI: 0.04–0.24, Z = 2.689, p = 0.007). The sensitivity analysis revealed that the results were still significant after removing the studies with the maximum and the minimum weight, which indicated the stability of RT and ACC in the overall analysis (see Table 2 for details).

The risk of bias was depicted in Figure 5. The results showed a low risk of bias in terms of randomization process. As for the risk of deviations from intended interventions (i.e., effects of blindness), all studies were single blind except for one double-blind study. Regarding the blinding of participants, studies using “vertex,” “other brain regions,” and “others” as the control conditions simulated the auditory and somatosensory sensations caused by the active stimulation, resulting in a low risk of bias. With “sham TMS,” although the auditory sensations could be similar to that of active stimulation, the somatosensory sensations were different, which might lead to mild concerns of blindness. In comparison, the control condition of “no TMS” could be easily differentiated from the active stimulation by the participants, resulting in potential high risk of bias. Finally, for the risk of missing the outcome data, inaccurate measurement of the outcomes, and reporting selected results, no evidence indicating an unclear or high risk of bias was found in the included studies. To summarize, although there might be a few risks of blinding, the overall quality could still be acceptable, and no study was eliminated because of low quality.

The publication bias tests showed no significant results in RT or ACC (RT: Egger: p = 0.69; ACC: Egger: p = 0.23), indicating that the overall effect sizes should not be enhanced by publication biases (see also Figure 6).

The moderator analyses results were summarized in Tables 3, 4 for RT and ACC, respectively, and displayed in Figure 2 (see Supplementary material for the forest plots, including moderator analyses and additional analyses).

Transcranial magnetic stimulation significantly modulated task performance for both semantic and phonological tasks, contrasting its non-significant effects on syntactic tasks.

For semantic tasks, TMS effects were manifested both on RT and ACC. These robust modulatory effects might be attributed to two reasons. First, semantic processing recruits broadly distributed but highly interactive regions in the left hemisphere such as the inferior frontal, and posterior temporo-parietal cortices (Hartwigsen et al., 2010, 2016; Papeo et al., 2015; Passeri et al., 2015). TMS studies, utilizing “condition-and-perturb” or “perturb-and-measure” paradigms on semantic processing revealed that TMS effects on the stimulation site might spread to other regions that are structurally or functionally connected to the stimulated area (Hartwigsen et al., 2016; Vitale et al., 2021). This suggests that the broad semantic network might be affected as a whole when targeting a semantic key region, leading to a significant change in semantic task performance. Second, potential compensatory effects within larger networks seem to be strictly constrained by the experimental factors. As pinpointed by Klaus et al. (2020), the compensatory effects of the semantic network might only be observed when TMS intensity and executive control components of the task are relatively low. As a result, semantic task performance can not be maintained to the original level by compensatory effects.

Phonological processing also involves a distributed neural network, but previous work suggests that each region might make relatively independent and unique contributions to phonological processing (Hartwigsen et al., 2010), thus potentially reducing effective compensatory effects among regions and leading to significant effects on task performance. Another finding worth noting is that only ACC but not RT was affected by TMS. This could be due to the fact that phonological processing relies on a larger domain-general verbal working memory system (Deschamps et al., 2014, 2020), and previous studies have shown that TMS effects on working memory tasks tend to be manifested especially on ACC (Mottaghy et al., 2003; Nixon et al., 2004; Romero et al., 2006; Osaka et al., 2007; Acheson et al., 2011). As pointed out by Deschamps et al. (2014), most of the tasks adopted by studies investigating phonological processing actually recruited verbal working memory, such as the same/different judgment task (Deschamps et al., 2014), the delayed auditory discrimination task (Deschamps et al., 2020), and phonological decision tasks (Hartwigsen et al., 2016), all involving sub-vocal rehearsal and the maintenance of phonological information in working memory. Another potential explanation for the observed effects is that compared to the retrieval and decision phase, perturbation of the encoding phase tends to cause changes in ACC as opposed to RT (Karabanov et al., 2015). Most of the included phonological studies in our meta-analysis focused on the posterior inferior frontal gyrus (pIFG), supramarginal gyrus (SMG), and posterior superior temporal gyrus (pSTG), which have been proven to play a role in the rehearsal and encoding phase, but not in the retrieval or decision phase of phonological working memory tasks (Nixon et al., 2004; Kahn et al., 2005; Kirschen et al., 2006).

Compared to semantic and phonological studies, the number of syntactic studies was relatively small, hampering the possibility to obtain a stable significant result and a conclusive interpretation. However, some studies did show a significant TMS effect on syntactic tasks (Uddén et al., 2017; Ishkhanyan et al., 2020; van der Burght et al., 2022). It is also possible that the overall insignificant results are related to the degree to which sub-regions are differentiated. Taking Finocchiaro et al. (2015) as an example, this study probed the function of anterior, middle and posterior parietal sites in thematic role assignment and only found a significant TMS effect for the posterior site. Therefore, for more comprehensive and deeper understanding of TMS neuromodulatory effects on syntactic processing, future studies should adopt different syntactic tasks and focus on specialized sub-regions (such as IFG and pSTG).

With regard to stimulation sites, none of the three broad regions (including frontal, temporal, and parietal regions) showed significant TMS effects on RT or ACC. Further analyses at a finer-scale only revealed marginally significant TMS effects on ACC in IFG, but not in STG and MTG.

These results might be interpreted from four directions. First, the basic rationale of TMS studies is to explore, or to be more specific, to verify the potential causal relationship between cognitive functions and certain brain regions based on previous neuroimaging and clinical studies (Flöel, 2012; Papeo et al., 2013; Hartwigsen, 2015). Therefore, the results can be both verification as well as falsification. When analyzing the brain regions alone regardless of the effects of other moderators such as language functions, it seems equally possible to obtain significant as well as non-significant effect. For example, the findings of Bonnì et al. (2015) contradicted the existing clinical studies, showing that TMS on left ATL had no behavioral effects on written word processing. Krieger-Redwood et al. (2013) found that, despite positive evidence from neuroimaging results, TMS over the primary motor cortex (PMC) did not disrupt the mapping of speech sounds onto semantic categories. As a result, future meta-analyses should investigate interactions and try to separate the relationships between different moderators.

Second, it has become increasingly evident that some key language-related brain regions such as IFG and STG can be divided into finer anatomical structures specialized for different functions (Whitney et al., 2012; Klaus and Hartwigsen, 2019; Deschamps et al., 2020; Piai et al., 2020). This brought new challenges for studies to localize the precise stimulation site for the target brain regions underlying certain functions, along with the already existing barriers regarding the limited spatial resolution of TMS (between 0.5 and 1 cm, Sliwinska et al., 2015) and the variance in precision of different methods of localization, leading to the failure to capture significant modulatory effects of TMS. Moreover, it has also been demonstrated that these sub-regions are quite sensitive to task difficulty. For example, in Whitney et al. (2012), stimulation to the anterior inferior frontal gyrus (aIFG) only affected semantic tasks with higher executive control demands, while leaving more automatic tasks unaffected. Future TMS studies are therefore recommended to differentiate not only between specific task types for each language function (i.e., semantic, syntactic and phonological) but also between different degrees of task difficulty within each task.

Third, unlike long-term effects of recovery in patients with brain lesions (Walsh and Cowey, 2000; Krieger-Redwood et al., 2013), the “virtual lesion” caused by TMS may be compensated by rapid functional reorganization within the distributed neural network for language in a rather short time (Hartwigsen et al., 2013, 2016; Klaus and Hartwigsen, 2019), making the transient TMS effects harder to detect. Combined with the finer division of regions and corresponding functions, the implication for future studies is to aim for the network instead of single nodes and target key/hub nodes within the networks for different functions.

Finally, for the marginally significant TMS effects on IFG, we reason that this may reflect the relatively large number of studies investigating this region. Since IFG is the language hub where the classic language region, Broca’s region resides, studies probing semantic, syntactic, or phonological processing could all take IFG into account and have indeed proven its involvement in these three language functions (Krieger-Redwood and Jefferies, 2014; Zhang et al., 2018; Ishkhanyan et al., 2020). On the other hand, the modulatory effects of TMS on STG and MTG are still rather unstable owing to the lack of adequate studies and need to be confirmed by future meta-analysis including more studies.

The present results support the coordinates of focal stimulation sites (brain regions) extracted from previous studies over the localization determined by the researchers themselves. Taking a closer look at the specific approaches for targeting, we found that studies determining stimulation sites by themselves mostly utilized coarse-grained targeting such as scalp measurement in reference to certain landmarks on the skull (e.g., the inion) (Finocchiaro et al., 2008; Jackson et al., 2015) or standard electrode cap from the EEG 10-20 system (Franzmeier et al., 2012; Jost et al., 2020). These approaches neither account for inter-individual differences in the anatomical structures beneath the scalp nor for the differences in the functional organization of the brain (Sandrini et al., 2011; Beynel et al., 2019). Some studies also combined anatomical magnetic resonance imaging (MRI) scans with the use of frameless stereotaxic neuro-navigation systems to realize more accurate “online” localization of the target site (Zhang et al., 2018; Deschamps et al., 2020), but still, this approach lacks the precision regarding inter-individual differences in structure-to-function relationships. By contrast, relying on anatomical coordinates from previous fMRI studies or meta-analyses with the same task paradigm or tasks probing similar language processing under investigation (Krieger-Redwood and Jefferies, 2014; Passeri et al., 2015) seems to be more promising. This function-guided approach has been proven to be the optimal localization approach with higher experimental power, especially when individual fMRI localizers within the same participants are used (Sack et al., 2009; Sandrini et al., 2011; Beynel et al., 2019).

When considering the stimulation types, we found that rTMS could exert significant neuromodulatory effects on language task performance. This may be because rTMS could prolong the stimulation time, thus accumulating and enhancing the effect sizes (Sandrini et al., 2011). As for TBS, this protocol may be more susceptible to inter-individual differences in focal neuronal states, neural compensation mechanisms, and the specific location within the structurally complex brain regions (Silvanto and Pascual-Leone, 2008; Thut and Pascual-Leone, 2010; Hamada et al., 2013; Vernet et al., 2013; Jost et al., 2020), and thus elicited variable modulatory effects in the present meta-analysis (e.g., Hartwigsen et al., 2013; Bonnì et al., 2015; Jung and Lambon Ralph, 2016).

Our moderator analysis on the timing parameter (i.e., when to apply TMS) demonstrated that online TMS could exert significant neuromodulatory effects especially on RT while offline TMS elicited significant effects on ACC. During online TMS, stimulation is administered immediately before or during the task to transiently disrupt the ongoing neural processing (sometimes referred to as “virtual lesion”). However, it has also been proposed that such online disruption, unlike an actual lesion that would terminate the ongoing process, may rather induce neuronal noise in the targeted area (Devlin and Watkins, 2007; Sandrini et al., 2011). Consequently, online TMS might mainly result in a quantitative change of response efficiency (i.e., as reflected in the response speed), with the quality of response (i.e., the accuracy rates) being spared. Furthermore, according to the “state-dependency” concept (Silvanto and Pascual-Leone, 2008; Miniussi et al., 2013), the induced noise may not be completely random but dependent on the brain state induced by the task, and could turn into part of the signal if it synchronizes with the ongoing neural activity. For example, some studies discovered that TMS given immediately prior to the task could pre-activate related neuron populations and facilitate picture naming speed (Töpper et al., 1998; Mottaghy et al., 1999). Collectively, these findings support the notion that the transient online TMS effect is more likely to affect response efficiency but may not be detrimental enough to disrupt response quality. On the contrary, offline TMS is given before a task, with the aftereffects typically lasting for up to 30 min after the stimulation (Hartwigsen, 2015; Beynel et al., 2019). The accumulated rTMS effects are not restricted to the stimulated sites but may spread to other connected brain regions within a network. Such long-lasting remote effects may modulate the whole network and disrupt or facilitate processing, leading to a perturbation or enhancement in task accuracy.

The current results confirmed that both high and low TMS frequencies could affect healthy participants’ language performance, with high frequencies exerting more stable effects. This accords with a series of studies focusing on the influence of specific stimulation parameters (including frequency) on TMS effects (Sparing et al., 2001; Sollmann et al., 2015, 2018; Nettekoven et al., 2021), which support the idea that higher frequencies may induce more reliable disruption of language functions. There are two possible explanations for this finding. The first explanation is related to potential side effects, especially discomfort or pain during stimulation. The distraction caused by physical discomfort (e.g., twitching and contractions of face muscles) or more severe side effects (e.g., dysarthria resulting from stimulation-induced contraction of cranial muscles, Sollmann et al., 2018) are non-specific TMS effects and are very likely to confound the interpretation of the results. Therefore, it has been proposed that higher frequencies correlated with lower pain levels and were therefore more optimal for obtaining reliable TMS effects (Nettekoven et al., 2021). Secondly, it is likely that using TMS frequencies matching with the natural frequency band of endogenous brain oscillations increases the probability of TMS pulses to interfere with cortical processing at the appropriate timing (Thut and Miniussi, 2009; Miniussi et al., 2013; Nettekoven et al., 2021). Indeed, evidence from MEG studies has associated language-related processing with brain oscillations in higher frequency bands, such as the beta (17–25 Hz) and the low gamma band (26–50 Hz) (Hirata et al., 2010; Hinkley et al., 2020; Youssofzadeh et al., 2020). It is also worth noting that different language brain regions may be sensitive to different stimulation frequencies. However, studies exploring optimal frequencies for distinct regions are still lacking, leaving room for further progress.

Our additional analysis examining the direction of TMS effects revealed no significant results. Yet, we did find that high-frequency rTMS was prone toward inhibition as manifested by the ACC decrease. This supports the notion that a simple transfer of the relationship between frequency type and effect direction (i.e., high frequency for facilitation, and low frequency for inhibition, see Bailey et al., 2001; Fitzgerald et al., 2006; Murdoch and Barwood, 2013) from the motor to the language system does not hold. Rather, this relationship may be influenced by multiple factors such as task types, stimulation intensities, and target brain regions (Vallar and Bolognini, 2011; Hartwigsen, 2015; Beynel et al., 2019). Nevertheless, no strong conclusion could be made given the non-significant results.

The moderator analysis also showed that compared to the intensities calibrated according to the AMT, RMT could exert more significant TMS effects on language performance. This result is not surprising and directly relates to higher stimulation intensities. RMT is typically defined as the lowest amount of stimulator output (intensity) necessary to produce a motor evoked potential (MEP) in the resting muscle exceeding 50 μV in at least 50% of the total trials. In contrast, AMT is assessed under voluntary pre-contraction of the target muscle, requiring MEP sizes of at least 150 μV (Sandrini et al., 2011). A a result, individual RMTs are usually considerably higher (approximately 15%) than AMTs and consequently leading to a higher TMS intensity, which was proposed to introduce more severe perturbations, rendering functional compensation more difficult (Klaus et al., 2020), and exert stronger remote, long-distance effects spreading across specialized networks (Hartwigsen, 2015). Also, researches (although rare) distinguishing between RMT and AMT (e.g., Wassermann, 2002) argued that experimental error and other unstable determinants of threshold may account for about 36% of the cross-subject variability at rest and about 50% during voluntary contraction, which suggests that compared to RMT, AMT is more vulnerable to factors such as coil placement, stimulation frequency and other unknown physiological sources of individual variability and is therefore a less stable reference to determine the stimulation intensity.

Regarding control conditions, sham TMS outperformed other solutions. It is noteworthy that TMS on presumably unrelated control sites might elicit unwanted effects due to their connections with target sites, thus potentially confounding experimental and control conditions (Franzmeier et al., 2012; Papeo et al., 2015; Klaus and Hartwigsen, 2019; Vitale et al., 2021). Therefore, researchers should be very cautious when selecting control sites. It should also be emphasized that the inclusion of sham TMS alone does not control for potential side effects of the stimulation such as muscle twitches and pain. Consequently, such conditions are not sufficient and studies without active control sites are more prone to false positives (Jost et al., 2020; Vitale et al., 2022). The optimal TMS study on language should include both an active control site and sham stimulation.

As for the group design types, within-subject designs were less affected by the individual variance which might submerge the TMS effects in between-subject designs (see also Passeri et al., 2015; Zhang et al., 2018). For example, Zhang et al. (2018) adopted a between-subject design and considered this a major limitation due to the large individual variance. Passeri et al. (2015) further emphasized that the neuromodulatory TMS effects are largely affected by the different degrees of language-related brain region lateralization of individuals. Therefore, the individual brain’s structural and functional variance might be critical for identifying TMS effects.