Memory is not an isomorphic construct and, of the various forms of memory, episodic memory is the form that is absolutely critical for maintaining one's self-identity. According to Tulving (2002), episodic memory is the type of declarative memory that represents knowledge of particular events that one has experienced as well as their contextual acquisition (e.g., autobiographical memory), such as when and where an event occurred (e.g., who you met at a dinner party last week). It is measured by retrieval accuracy on declarative memory tests such as recall and recognition. Episodic memory requires and extends a related type of declarative memory, i.e., semantic memory or one's general knowledge of the world. Semantic memory involves retrieving knowledge (i.e., facts, concepts, meaning) without the accompanying contextual information (e.g., knowing how many planets are in the solar system, but without remembering in vivid detail when or where this was learned). The critical role of the hippocampus and surrounding structures of the medial temporal lobes (MTL) in the encoding, consolidation, and retrieval of episodic memories has been highlighted by striking cases of amnesia following either surgical resection (as in patient HM), disease (as in Clive Wearing), or brain trauma (as in patient KC), and have been elucidated by decades of animal/behavioral, neuropsychological, neuroimaging, and neurostimulation research (for reviews see, Moscovitch et al., 2016; Eichenbaum, 2017).

The hippocampus alone does not support episodic memory, however. In order for an event to be remembered, it must first be encoded through some combination of perceptual (e.g., visual, acoustic), affective, contextual, and semantic processes that integrate an event with its contextual associations to form episodic memories. Additionally, executive control processes are thought to facilitate the multi-modal binding of features and associations for episodic memory representations and are thought to be mediated by regions of the prefrontal cortex (PFC) and parietal cortex (PC; Buckner et al., 1999). Encoding and retrieval processes are inherently intertwined. Once encoded, successful retrieval demonstrates access to a memory, which also serves as a measure for successful encoding. For example, the encoding specificity principle (Tulving and Thomson, 1973) and related concept of “transfer-appropriate-processing” (Morris et al., 1977; Blaxton, 1989) emphasize the role of a reinstatement of the encoding context during retrieval for episodic memory. At the neural level, context reinstatement is operationalized by the degree of overlap in neural activity elicited during encoding and retrieval, and functional neuroimaging studies have revealed the involvement of regions in the prefrontal (Prince, 2005), medial temporal (Rugg and Vilberg, 2013), and parietal cortices (Wagner et al., 2005) (for review, see Craik and Rose, 2012).

Although encoding and retrieval are intertwined, there are important differences in the neural correlates of successful episodic encoding and retrieval. For example, there is often a hemispheric asymmetry in left vs. right PFC activation during encoding vs. retrieval, depending on the task stimuli, familiarity, and the cognitive processes engaged (see below). With regards to rTMS, it is important to note that the neural substrates of episodic memory are impacted by the underlying state of activation in a stimulated brain region that is driven by the task demands and stimuli (e.g., words vs. pictures; Miniussi et al., 2013; Romei et al., 2016).

It is generally theorized that, in younger adults, the left PFC is critical for encoding, while the right PFC is to be preferentially involved in retrieval (Habib et al., 2003). This is known as the hemispheric encoding-retrieval asymmetry (HERA) model, which has been supported by some evidence from rTMS findings (Rossi et al., 2004). However, the role of the left and right PFC in episodic memory may also vary due to numerous factors, such as task stimuli. For example, in two studies, left DLPFC stimulation impaired memory for verbal information (Rami et al., 2003), while right DLPFC stimulation impaired memory for non-verbal stimuli (Epstein et al., 2002; Floel et al., 2004). However, differences between the parameters of stimulation such as frequency and intensity across studies complicate direct comparison.

The effects of rTMS on episodic memory has also been shown to vary with an individual's familiarity with the type of information that is to be remembered. For example, online 20 Hz rTMS to the left DLPFC during retrieval impaired novel compared to familiar information (Sandrini et al., 2003). Analogously, rTMS has been shown to preferentially affect memory for contextual information of studied associations. For example, online 10 Hz rTMS to the left inferior frontal gyrus (IFG) during encoding of novel faces paired with a “context” stimulus (e.g., the word “lawyer”) impaired subsequent recognition of faces compared to faces studied without a context stimulus (i.e., faces studied with “no context”; Feurra et al., 2010). Thus, effects of rTMS on episodic memory likely depends not only on the specific rTMS parameters (e.g., frequency, intensity, stimulation timing, or target site), but also on the cognitive processes that are tapped by the type of information presented or the way in which it is to be remembered.

In a similar vein, the effects of rTMS on episodic memory likely also depends on the cognitive strategies involved during encoding and/or retrieval (Manenti et al., 2010b, 2012; Hawco et al., 2013). For example, Innocenti et al. (2010) had individuals engage in “deep” (semantic) or “shallow” (perceptual) levels of processing while encoding words and applying online 10 Hz rTMS to either the left or right DLPFC. There were dissociable effects on recognition of words that were deeply or shallowly encoded, with left DLPFC stimulation abolishing the typical benefit of deep processing on memory (Innocenti et al., 2010). Relatedly, in a face-name pair encoding task, participants received fMRI-guided 10 Hz rTMS to either the left or right DLPFC during retrieval. After subdividing performance for those who reported using a retrieval strategy from those who reported using no retrieval strategy, it was concluded that right DLPFC stimulation disrupted retrieval for strategy users, while left DLPFC stimulation disrupted retrieval for non-strategy users (Manenti et al., 2010b).

Taken together, the effects of rTMS on episodic memory likely depend not only on the specific parameters of the protocol, but also on the brain state and cognitive processes engaged at the time of stimulation. This brain-state dependence is consistent with research that has used TMS in a wide variety of domains that has shown that the effects of TMS to a targeted region interacts with the ongoing, endogenous neural activity in that region and functionally coupled regions at the time of stimulation. Thus, while patterns are complicated across conditions, this principle enables researchers to investigate the causal role of the targeted and functionally coupled regions in specific neurocognitive mechanisms through the systematic manipulation of different task conditions (for review see, Romei et al., 2016).

According to dual-process models of episodic memory, the cognitive processes engaged during retrieval can be supported by two separate processes (i.e., familiarity and recollection). A central theme of dual-process models is that previously studied information supports recognition by being generally more “familiar” than new/non-studied information. A distinct retrieval process, “recollection,” can further facilitate recognition of previously studied information through the retrieval of associated spatial, temporal, or contextual information (Yonelinas, 2002). The remember/know procedure is commonly implemented to examine differences in the nature or subjective experience of retrieval. Specifically, during a recognition task, participants indicate if the decision to respond that a presented item was previously seen (i.e., old) was based on recollection of the encoding details or based on a feeling of familiarity; they are instructed to respond with “remember” or “know,” respectively, (Tulving, 1985).

rTMS enables the ability to causally investigate the underlying neural substrates of familiarity and recollection processes. For example, Turriziani et al. (2008) applied online 20 Hz rTMS to the left or right DLPFC during either encoding or retrieval (i.e., remember/know paradigm) of faces. They found that recollection (remember responses) was impaired following right compared to left DLPFC stimulation during encoding. However, familiarity was impaired following left and right DLPFC stimulation during encoding. These findings suggest that both the left and right DLPFC support familiarity and recollective retrieval processes, but the left vs. right DLPFC may contribute differentially depending on the type of retrieval process engaged.

The traditional view of episodic memory has focused on the hippocampus and its role in binding specific details of an experience (such as what events have occurred) with when (temporal), where (spatial), or how (source) the events occurred (Tulving, 1985, 2002). A large body of work on patient H.M., who had extensive surgical resection of the bilateral hippocampi and surrounding structures of the MTL in young adulthood, revealed the importance of the hippocampus in the formation of new episodic memories (Squire, 2009). However, inferences from brain lesion studies are often confounded by compensatory reorganization mechanisms (e.g., plasticity) that occur following brain injury and/or damage that is not isolated to specific brain regions. Alternatively, TMS can be used to causally modulate neural activity in healthy brain regions and networks at the precise time that a hypothesized cognitive operation is thought to occur; it affords a direct test of how specific brain regions contribute to episodic memory. However, the limited spatial ability to target deep regions in the MTL has led to substantial focus on targeting neocortical regions. The combination of TMS with neuroimaging (e.g., fMRI, EEG) and functional connectivity analyses provides a promising route to understanding brain network interactions, and has enabled further understanding of how TMS related changes in local neural activity propagate through polysynaptic connections to distal regions that are functionally connected at the time of stimulation (Shafi et al., 2012). For example, several studies have shown effects on episodic memory from targeting nodes in neocortical networks in prefrontal cortex or parietal cortex that are functionally connected with the hippocampus (e.g., Wang et al., 2014).

The PFC in particular has received substantial attention as a stimulation site in rTMS protocols. For example, a fMRI-localized rTMS study targeted the left inferior prefrontal cortex (LIPFC) online during a word-pair encoding task with 7 Hz rTMS. In comparison to two active control conditions (i.e., RIPFC, left superior partial cortex; LPC), stimulation of the LIPFC enhanced subsequent memory on an immediate word recognition test. The authors suggested that this causal link between LIPFC and episodic memory may have been driven by LIPFC stimulation triggering elaborative processing of word pairs that led to more distinctive memory cues (Köhler et al., 2004). One important design aspect of this study that we will briefly return to later is the fMRI-localization targeting procedure that accounts for individual differences in the structure and function of stimulated brain regions for more accurate targeting of regions of interest in a specific task.

While PFC-MTL interactions are a cornerstone of neurocognitive models of episodic memory, it is also clear that there is extensive, often task-dependent recruitment of distributed networks of areas that process sensory, semantic, or emotional information to support various aspects of episodic memory (Ciaramelli et al., 2008; Cabeza and Moscovitch, 2013). Thus, research has examined the posterior parietal cortex due to its hypothesized role in attending to memory representations. It has been suggested that regions of parietal cortex are critical for episodic retrieval because of their role in either the formation of memory representations or shifts in attention for decision making processes, or both (Dobbins et al., 2003).

For example, Manenti et al. (2010a) implemented an rTMS-fMRI design to investigate the causal functioning of both the parietal cortex and DLPFC during encoding and retrieval. The authors applied online 10 Hz rTMS to either the left or right DLPFC or individualized regions of the left or right parietal cortex (either supramarginal or angular gyrus) during either encoding or retrieval. The authors demonstrated word recognition decrements (i.e., slower reaction times) following rTMS to the left parietal cortex (compared to sham). Furthermore, those with larger rTMS-related memory decrements were those who demonstrated a larger degree of right parietal activation during retrieval in the initial fMRI-localization task (Manenti et al., 2010a). One implication from these findings is that different aspects of episodic memory performance (e.g., accuracy, reaction times, subjective confidence) may be supported differentially by the PFC vs. PC.

Additionally, the null memory results reported (Manenti et al., 2010a) contrast with previously reported findings of Innocenti et al. (2010), despite the use of many of the same parameters (i.e., 10 Hz, left and right DLPFC, encoding, and retrieval of words). One possible explanation may be attributable to differences in other parameters, such as stimulation intensity (i.e., 100 vs. 90%). Similarly, the findings of Manenti et al. (2010a) also differed from Köhler et al. (2004) despite implementing many of the same parameters (i.e., online stimulation at the same intensity to the PFC and PC during word encoding). However, differences in the stimulation frequency (i.e., 7 vs. 10 Hz) might account for the different results.

It should be noted that studies often select a stimulation site to have a selective (or, more likely, stronger) effect on one cognitive domain, but it must be acknowledged that stimulated brain regions may be implicated in numerous related cognitive processes. For example, a meta-analysis revealed that rTMS to the DLPFC improved working memory performance (e.g., accuracy, reaction times; Brunoni and Vanderhasselt, 2014). The studies included in the Brunoni and Vanderhasselt (2014) meta-analysis targeted the DLPFC because of its role in short-term (STM)/working memory (WM). Thus, subsequent assessments of the rTMS effects on long-term episodic memory should be interpreted based on possible modulation of STM/WM processes on encoding and maintenance processes that may subsequently result in modulation of long-term memory. Indeed, a number of studies have examined the effects of rTMS to the DLPFC during STM/WM and has assessed the effects of rTMS on subsequent long-term memory (Nilakantan et al., 2017; Marin et al., 2018). For example, Marin et al. (2018) found that continuous theta burst (cTBS) to the right DLPFC prior to the encoding of objects and their spatial location was impaired for subsequent long-term memory tasks (i.e., object recognition and spatial location recall), but short-term memory performance (i.e., short-term spatial location recall) was left unaffected compared to controls.

Taken together, the findings reviewed thus far build upon previous correlational neuroimaging findings that the PFC, PC, and MTL are important for episodic memory, but systematic examination of encoding, consolidation, and retrieval processes, as well as potential interactions among neocortical regions, is needed. For example, the functional roles of these regions in recollection and familiarity (Yonelinas, 2005) remains to be fully understood.

How rTMS can enhance or impair memory (or leave it unaffected) depends on numerous factors regarding the ways in which the rTMS protocols are implemented. A detailed review of all technical considerations regarding rTMS is beyond the scope of the current paper (for details, see Kammer et al., 2001; Rossi et al., 2009; Sauvé and Crowther, 2014). The main stimulation characteristics considered here are stimulation frequency, intensity, and timing. A common approach in selecting a stimulation frequency generally involves applying a protocol hypothesized to either enhance cortical excitability, such as a high frequency (>5 Hz) or intermittent theta burst (iTBS) protocol, or suppress cortical excitability, such as a low frequency (<1 Hz) or cTBS protocol (Huang et al., 2005; Hallett, 2007). However, these hypotheses stem largely from physiological findings that have been investigated in the motor cortex with the assumption that this remains true for other cortical regions.

Another way in which different stimulation frequencies may modulate neural activity is through altering neural oscillatory activity through the resetting of natural oscillations, and in some cases, driving neural entrainment that may influence communication between brain regions and cognitive performance (Luber and Lisanby, 2014). Importantly, this modulation of neural activity is assumed to be specific to functionally relevant frequencies. Because relevant oscillations may vary in a continuous fashion between tasks, across phases of a task, and/or between individuals, considerable research is needed to fully understand how rTMS frequency can modulate episodic memory processes. A major aim of the current meta-analysis is to help elucidate the aggregate and complex interactions associated with the effects of different rTMS frequencies on episodic memory.

One of the many factors that will influence the effectiveness of rTMS is stimulation intensity. Generally, researchers will adjust stimulation intensity based upon an individual's resting or active motor threshold (RMT, AMT; Ngomo et al., 2012), which is defined as the minimal stimulator intensity needed to induce a muscle evoked potential in 5 out of 10 occurrences in either a relaxed (RMT) or actively engaged muscle (AMT). Prior work has found evidence that low (below) and high (above) stimulation intensities mirror facilitatory vs. inhibitory neural activity that is also associated with enhancements vs. decrements in behavioral outcomes (Silvanto et al., 2018). In addition, recent findings have shown that the effects of stimulation intensity on cortical excitability also may interact with specific frequencies, which in turn can lead to divergent effects on cognitive performance (Chung et al., 2018). Furthermore, the effects of varying stimulation intensity may covary with numerous factors such as the “brain state” or the cortical excitability of targeted and functionally coupled regions at the time of stimulation (Romei et al., 2016). Thus, a systematic investigation of the effects of stimulation intensity and how it interacts with frequency is needed. Elucidating the effects of varying stimulation intensity on episodic memory was another aim of the current study.

When stimulation is applied (i.e., stimulation timing parameter) is also an important factor that enables researchers to investigate the time course of episodic memory processes. The timing of the stimulation protocol reflects differences in whether stimulation occurs simultaneously with a cognitive process (online; e.g., rTMS during picture encoding) or prior to or following a cognitive process (offline; e.g., rTMS prior to picture encoding). One reason that online and offline stimulation may affect episodic memory differently is that they may affect different mechanisms, with online stimulation having an immediate influence on brain activity while offline stimulation often results in after-effects up to an hour after stimulation (Bergmann et al., 2016). In other words, online approaches can interfere or enhance neural activity through immediate depolarization of targeted neurons, while offline approaches can modulate long-term potentiation- or long-term depression-like plasticity mechanisms to either enhance or inhibit episodic memory processes. In a previous meta-analysis of different forms of non-invasive brain stimulation on various cognitive measures, differences in stimulation timing were found, with larger enhancing effects occurring for online stimulation (Hsu et al., 2015). Importantly, stimulation timing may interact with stimulation intensity, which could lead to facilitating or impairing effects on cognition (Silvanto and Cattaneo, 2017).

The implementation of a proper control conditions is important to note given the variety of different control comparisons that are available. A proper control comparison is critical in order to assess whether the observed effects on performance are due specifically to the effects of applying TMS to the targeted region as opposed to some other variable, such as non-specific effects associated with the sensations of stimulation (Sandrini et al., 2011). Active stimulation control conditions that involve stimulating some other brain region that is not hypothesized to be as involved in the specific process of interest as the experimentally targeted region at the same intensity and frequency in both conditions are ideal because they most closely replicate the same aspects of the experimental/treatment TMS procedures. Sham stimulation controls (e.g., sham coil or tilting the coil away from the scalp) are implemented to mimic the non-specific effects of TMS (i.e., the discharging sound and scalp contact of the coil). “No TMS” conditions provide the least amount of control for non-specific effects of rTMS. In order to test if the type of control condition moderated the size of rTMS effects on episodic memory we examined whether there were systematic differences between effect sizes associated with comparisons to active, active vertex, sham, or no TMS control conditions.

An exhaustive PubMed search was conducted to select studies for the review and meta-analysis with a combination of the following search terms: “transcranial magnetic stimulation,” “TMS,” “rTMS,” “Theta burst stimulation,” “TBS,” “memory,” “episodic memory,” “long term memory,” “associative memory,” “recollection,” “mild cognitive impairment.” The search for studies ended in July 2018. Our literature search yielded 484 studies from PubMed, and references from these relevant articles were examined for possible inclusion in our analysis. In line with previous protocols, when the title and abstract were insufficient to identify the study for inclusion, the full-text underwent further examination. In cases of uncertainty, articles were examined by the authors and research assistants and removed based on mutual agreement. The most frequent reasons for exclusion were that articles (a) did not apply TMS or (b) did not assess episodic memory, see Figure 1.

For inclusion in the systematic review, the studies were limited to (1) single pulse TMS, paired pulse TMS, rTMS, or TBS protocols that measured (2) episodic memory with a free-recall, cued-recall, or recognition test (3) in healthy younger adults, (4) healthy older adults, or (5) older adults with a clinical condition, (6) and those that were published in English.

A total of 59 studies were incorporated into the systematic review based on the search terms and inclusion criteria. However, inclusion in the meta-analysis was focused on rTMS studies of healthy younger adults. Seven of the remaining studies were excluded from the meta-analysis because there was insufficient reporting of data necessary to calculate effect sizes (e.g., means, SDs, t-, or F-values). Thus, 37 articles examining the effects of rTMS on episodic memory in healthy younger adults were included in the meta-analysis. See Figure 1 for a flow diagram illustrating this process. A total number of 245 effect sizes were extracted from the selected studies. For experiments that pooled participants' data across multiple studies, only participants who were uniquely recruited for each study were analyzed. Details about the calculation of effect sizes are described below in the data extraction section.

In order to assess the methodological quality of the selected studies, we were guided by the assessment criteria recommended by Higgins et al. (2008), which is to categorize study characteristics as low risk, unknown risk, or high risk based on whether there was random allocation of participants to conditions, blinding of participants, blinding of experimenter, blinding of experimenter assessing the outcome measure(s), selective reporting of outcome measures, and participant attrition. Characteristics were coded as unknown when insufficient details were provided; studies were categorized as high risk when it was clear that the conditions were not met, such as when it was implausible for the participants and/or experimenters to have been blind to the administration of the experimental vs. control conditions. We also extended these criteria to include additional assessments pertinent to TMS studies, namely the targeting procedure and type of control condition. For targeting procedure, if a study used structural or functional MRI neuronavigation, international 10–20 EEG positioning system, or distance measurements, they were classified as low risk, unknown risk, and high risk, respectively. A variety of control conditions were implemented, such as active, sham, and no TMS conditions, that were classified, as low, unknown, and high risk, respectively.

Moderators that were hypothesized to modulate the size of effects of rTMS on episodic memory were coded for each study. In order to code for moderators, the full-text articles were examined for explicitly stated information regarding the moderators of interest. In situations where it was not clearly stated, the moderators were inferred from the study protocol (e.g., the study did not state if stimulation occurred “online” or “offline,” but, if the task figures showed that stimulation occurred prior to a cognitive task, then we coded stimulation timing as offline). Studies were independently coded by both the first author and research assistants to establish reliable coding and any discrepancies were resolved by the authors.

Means (M), standard deviations (SD), and sample sizes (n) were obtained for relevant episodic memory outcome measures for each study. When standard errors (SE) were reported, they were converted to SD. In cases where there was insufficient statistical information provided in the text or tables, they were collected from graphical representations using GetData Graph Digitizer data extraction software (downloaded at http://www.getdata-graph-digitizer.com). In the event that insufficient data could be obtained, attempts were made to contact the corresponding authors for additional data. Descriptive statistics and effect size calculations were done independently by the first author and research assistants to cross-check the calculations and resolve any discrepancies. The table of experiments included in the meta-analysis, their moderator variables, and all of the effect sizes that were extracted from the experiments are presented in an open-access database here: https://osf.io/6kbqe/?view_only=226e600d6e1546748b52f6bbfc593140. For additional information or to contribute additional data, please contact the first author. Our intention for sharing the data is so that effect size calculations can be confirmed and so that additional studies can be continuously added as the literature grows, which will facilitate future meta-analysis and systematic investigation.

Effect sizes for between-subject comparisons were computed with the following formula:

When studies reported comparisons between gain scores (e.g., Post-experimental—Pre-experimental) the effect size was calculated with the following formula:

Effect sizes for within-subject comparisons were calculated with the following formula:

This was done to obtain an accurate effect size estimate for within-subject designs (Cumming, 2012) using the commonly recommended approach for meta-analyses (Lakens, 2013). Alternative routes that address dependencies in within-subject designs, such as averaging effect sizes, can lead to a reduction in statistical power and the loss of information can result in less precise estimates and SE's (Hedges and Pigott, 2001; Cheung, 2014). In a similar vein, limitations also arise when calculating Cohen's d_rm, which attempts to account for the dependencies by correcting for the correlation in within-subject designs. As correlations between conditions are rarely reported, estimations may be too conservative, especially when correlations are high (Lakens, 2013).

Hedge's g and g_av effect sizes were calculated, which provides a less biased estimate of the true effect than Cohen's d, especially for studies with small sample sizes (Hedges and Olkin, 1985). In order to do this, we first calculated Cohen's d and then applied a Hedge's g correction (see Cumming, 2012) with the following formula:

With the exception of 5 studies, we were able to collect multiple effect sizes per study. Thus, we can no longer assume statistical independence, as within-study effect sizes may be more similar than effect sizes from other studies. Thus, dependent effect sizes may contain redundant information that becomes generally less informative when effect sizes are correlated. This form of dependency may occur when multiple effect sizes can be calculated from multiple outcome measures (Van den Noortgate et al., 2013). For example, in examining the effects of rTMS on episodic memory, a study can test a set of stimulation parameters (e.g., 10 Hz rTMS at 100% AMT to the DLPFC) on multiple memory outcomes, such as recall, recognition and source memory. A second dependency occurs when effect sizes are correlated in situations where an experimental condition is compared to multiple control groups or multiple experimental conditions are compared to one control group. For instance, many of the studies compared experimental stimulation to combinations of active control, sham, or no stimulation conditions, while other effect sizes were comparisons between multiple experimental stimulation conditions to a single control condition. This can be problematic in meta-analyses and increase type 1 errors if dependent effect sizes are assumed to contain unique information. Therefore, to address multiple dependency concerns we conducted a three-level random effects meta-analysis with Hedge's g and g_av corrections, which is detailed below (for additional information on dependencies and three-level models see, Van den Noortgate et al., 2013).

We conducted a three-level random effects model using the Metafor package in R (Viechtbauer, 2010). The typical random effects approach allows for effect sizes to vary due to sampling variation and due to differences between studies. In order to account for studies that contain multiple effect sizes, we can extend this model to a three-level random effects model to take into account the correlation of within-studies effect sizes. This approach does not assume independence among effect sizes as is done with other meta-analysis techniques. Multilevel modeling allows for an alternative for dealing with the dependency of effect sizes when correlations between outcomes are not reported or there is insufficient data to estimate the dependency (for additional details, see Assink and Wibbelink, 2016). The hierarchical structure of the data allows effect sizes to vary between participants (level 1; sampling variance), outcomes (level 2; within studies), and studies (level 3; between studies). In this case, individual studies (denoted by the subscript k) can contain one or multiple outcomes (denoted by the subscript j). The studies (i.e., Study ID) and individual effect sizes (i.e., Effect Size ID) were entered into the model as random effects. The simplest model with no moderators and three residuals (v_k, u_jk, e_jk) that are assumed to be normally distributed with a mean of zero is:

where an observed effect size for outcome j within study k is represented by ES_jk. The overall population effect size mean across all outcomes and studies is represented by β_0. The v_k element is the random mean deviation effect in study k from the overall effect. The random deviation of jth population effect in study k from the mean effect in study k is denoted by u_jk. Lastly, e_jk is the random residual error from the population effect due to sampling variation. Parameter estimates in the meta-analysis include the between-study variance (σv2) and within-study variance (σu2), and the sampling variance is not reported because it is calculated from reported data prior to the analysis. Therefore, observed significant between-study variance differences reflect that effect sizes systematically vary between studies to a greater degree than would be expected due to random differences in sampling variance (due to chance). Meanwhile significant within-study variance above zero signifies that the observed effects within a study vary across individuals to a greater degree than chance.

In order to implement a parsimonious model, we examined if our three-level model provided a better model fit over more simplistic models with multiple ANOVAs. There was a significant difference between a single and two-level model (within-study variance), likelihood ratio test (LRT) = 41.75, p < 0.01. There was also a significant difference between two and three-level models (between-study variance), LRT = 41.83, p < 0.01. Thus, the three-level modeling of the dependent effect sizes provided a better model fit than treating the effect sizes as independent.

A prevalent issue in systematic reviews and meta-analyses is how to address missing data due to publication bias. This problem occurs when significant results are more likely to be published than non-significant findings. If left unaccounted for, publication bias may lead to spurious results. In this instance, publication bias may lead to the true effect of rTMS on episodic memory being inflated. We used two methods to assess the degree of possible publication bias in the data: funnel plots and an extension of Egger's regression method (Egger et al., 1997)

The proportion of studies classified as low, medium, and high risk for the different categories of the quality assessment analysis are presented in Figure 2. Overall, many of the studies implemented appropriate methodological protocols for random allocation of participants to conditions, blinding of the experimenters assessing effects on the outcome measure(s), and non-selective reporting of outcome measures. However, several studies either did not implement proper blinding of participants and/or experimenters, or did not provide enough information to confirm proper blinding procedures. There was low concern for the degree of participant attrition that was reported. For our additional quality assessment measure of control type, we found that the majority of studies implemented either active or sham controls. With regards to the rTMS targeting procedure, we found that the majority of studies implemented either low risk neuronavigational procedures or used the 10–20 system for targeting cortical regions of interest.

Several approaches were used to examine possible publication bias. First, we examined the enhanced-contour plot (i.e., effect size vs. 1/standard error). The reference line is set to g = −0.06 with the contour shading representing p-value significance. The enhanced-contour plot revealed some asymmetry, but no systematic patterns of missing non-significant effect sizes, see Figure 3. Given the numerous effect sizes collected, the larger effect sizes observed in the funnel plots are expected. Importantly, they appear to be symmetrically distributed. An examination of influential points revealed no Cook's d values above 0.5.

Next, we examined the residual plots (residuals vs. standard error) for each moderator variable, see Figure 4. The reference line is set to average effect size with contour shading once again representing p-value significance. The residual plots revealed some asymmetry across the moderator variables.

Interpreting funnel plots is inherently difficult (Terrin et al., 2005) and may be further complicated in three-level models (e.g., observed asymmetry may not be problematic). For example, clustered data points due to the three-level structure could potentially be misinterpreted as bias. Therefore, in a more formal test of asymmetry we entered standard error into the model. This revealed no significant moderating effect of standard error on effect size, F_{(1, 244)} = 1.76, p = 0.18, indicating that the funnel plot was not significantly asymmetrical. Collectively, this suggests that publication bias is unlikely to be problematic in this analysis. However, future research is needed to properly address how to handle detecting and correcting for missing data (e.g., publication bias) in three-level models.

We initially entered individual effect sizes (i.e., Effect Size ID) and studies (i.e., Study ID) as random effects. This model revealed the average effect size (g = −0.06, SE = 0.08) was not significantly different from zero, t₍₂₄₄₎ = −0.75, p = 0.45, see Figure 5. Thus, we do not see an overall effect of rTMS on episodic memory performance, which is unsurprising given that most studies selected and implemented protocol parameters to either enhance or impair episodic memory functioning.

We tested the heterogeneity of effect sizes to examine if differences between effect sizes were systematic and not due to random sampling variance. If a substantial amount of between- or within-study effect size variance is found, potential moderators can be added into the model to explain this variability. There was significant within-study variance (level 2), estimate = 0.19, p < 0.01 and between-study variance (level 3), estimate = 0.12, p < 0.01, demonstrating that effect sizes varied both within and between studies. Consequently, 44.58% of the variance is accounted for by differences in effect sizes within studies, 27.25% by variance between studies, and 28.07% by random sampling variance. The significant variability across both levels highlights the substantial differences in effect sizes between studies and within the same study. Therefore, we next conducted a moderation analysis in order to assess whether the observed differences in effect sizes were moderated by other factors.

A moderation test revealed a significant moderating effect of rTMS frequency on episodic memory effects, F_{(6, 224)} = 2.40, p = 0.03, see Table 2. Specifically, 1 Hz (β = 0.59, SE = 0.20) rTMS protocols led to significantly larger enhancing effects compared to 10 Hz, (β = −0.28, SE = 0.20), 20 Hz (β = −0.22, SE = 0.14), iTBS (β = −0.42, SE = 0.36), and cTBS (β = −0.05, SE = 0.15). The comparisons between other frequencies were not significant. The aggregate effects of stimulation intensity or timing, or the targeted hemisphere, cortical region, memory process, or retrieval type, or the type of control condition were not significant moderators of rTMS effect sizes, see Table 2.

Moderation analyses revealed that stimulation frequency modulated the effect of rTMS on episodic memory in younger adults, with larger facilitatory effects of 1 Hz (g = 0.59) rTMS compared to 10 (g = −0.28), 20 (g = −0.22), iTBS (g = −0.42), and cTBS (g = −0.05) protocols. The facilitatory effect of 1 Hz rTMS may appear counter-intuitive because low frequency (e.g., <1 Hz) rTMS is typically thought to lead to inhibitory effects on neural processing of targeted regions, whereas, many high frequency protocols are generally thought to lead to enhancing cortical excitability (Walsh and Cowey, 2000; Rossi and Rossini, 2004). Relatedly, the numerically larger negative effects of iTBS compared to cTBS paradigms is also surprising given that cTBS is typically assumed to have long-term-depression-like inhibitory effects (LTD) that can suppress cortical excitability in targeted regions whereas iTBS is assumed to lead to long-term potentiation-like effects (LTP) that can enhance cortical excitability (Huang et al., 2005). However, these findings stem from physiological studies that modulated cortical activity in the motor cortex (Jelić et al., 2015), while the studies in the present meta-analysis generally targeted prefrontal and parietal regions. One possible explanation is that the effects of TMS may differ according to the structural and functional architecture of the targeted region (e.g., motor cortex vs. prefrontal/parietal cortex). However, strong inferences based on these findings are currently limited due to the numerous differences between studies in the specific rTMS parameters and experimental designs that were used, which may interact to affect episodic memory.

Interaction analyses revealed facilitatory effects of 1 Hz rTMS for both online and offline stimulation. However, the effects of 1 Hz rTMS was driven by stimulating below MTI. While the underlying mechanisms of 1 Hz rTMS on episodic memory are poorly understood, one possible explanation comes from the principle of stochastic resonance, whereby neural communication is enhanced by low levels of noise that can push a neural signal past threshold (Schwarzkopf et al., 2011; Silvanto and Cattaneo, 2017). Thus, 1 Hz rTMS may enhance episodic memory by interacting with intensity to enhance the signal-to-noise ratio in the stimulated network of neurons. However, this process is affected by a variety of other factors that further complicate the process. For example, the effects of MTI are known to be sensitive to “brain states” and changes in brain activity can influence if below- or above-MTI leads to facilitatory or inhibitory effects (Romei et al., 2016).

Stimulation intensity also interacted with 20 Hz rTMS with attenuating effects of below-MTI compared to at-MTI. These findings are not consistent with stochastic resonance, where below MTI stimulation should lead to enhancing effects. However, the situations in which different levels of intensity will lead to either enhancing or inhibitory effects have also been known to depend on brain states. One reason for the observed effects may be due to differences in experimental designs, as task stimuli or cognitive processes engaged likely resulted in different activation states in the targeted brain regions. Relatedly, previous work has shown that cortical excitability varies with different MTI levels following iTBS (Chung et al., 2018) protocols. Thus, it is possible that the enhancing or inhibitory effects of rTMS may vary with MTI, which may also differ depending on the frequency of stimulation.

Additional analyses revealed that the effects of episodic memory performance in young adults depended on frequency and stimulation timing. Specifically, the enhancing effects of 1 Hz compared to 20 Hz was driven by online protocols. On the other hand, both 1 Hz and 20 Hz rTMS protocols had enhancing effects compared to either iTBS or cTBS paradigms when stimulation occurred offline. Interestingly, online 20 Hz rTMS protocols had inhibitory effects on episodic memory; however, offline 20 Hz rTMS had enhancing effects on episodic memory. These findings suggest that 20 Hz rTMS modulates episodic memory through different mechanisms when applied online vs. offline. For example, online paradigms may disrupt ongoing neural activity while offline paradigms may potentiate episodic memory processes through LTP-like effects.

Taken together, while the meta-analysis suggests novel perspectives on the effects of interacting factors on rTMS studies of episodic memory, the limited numbers of studies with combinations of factors spanning the entire parameter space may have hindered our ability to reliably estimate the role of many factors (e.g., targeted hemisphere, cortical region, memory process, and retrieval type) that are hypothesized to moderate the effects of rTMS on episodic memory. Moreover, the current meta-analysis did not find a moderating effect of the type of control condition (i.e., active, active vertex, sham, or no TMS) that was implemented. Importantly, these findings do not suggest that there are no differences between using active, vertex, sham, or no TMS control conditions. There are substantial gains in the quality of inference that can be obtained with comparison to proper control conditions. Inferences about the role of a particular brain region in a specific process are strengthened when effects are contrasted against conditions in which active stimulation at the same level of intensity and frequency is applied to either the targeted brain region at a different time point or to a different brain region that is not hypothesized to be involved in the process of interest at the time of stimulation. Therefore, researchers should still take careful consideration in the type of control condition that is most appropriate and feasible for specific experimental designs (for further information on control conditions see, Davis et al., 2013; Duecker and Sack, 2015).

In the next section we discuss how the meta-analysis on younger adults relates to the pattern of observed effects of rTMS effects on episodic memory functioning in healthy older adults and those with clinical disorders. This systematic review of the effects of rTMS on older adults revealed several similar findings that were observed in the meta-analysis on younger adults, thereby providing some sense of cross-validation of the findings in different populations.

A similar pattern of effects of 1 Hz rTMS on episodic memory that was revealed in the meta-analysis on young adults was found in 1 Hz rTMS studies on older adults with below-MTI stimulation enhancing performance (Turriziani et al., 2012) while above-MTI had no effect (Davis et al., 2017). Turriziani et al. (2012) administered offline 1 Hz rTMS at 90% MT to the left or right DLPFC of patients with mild cognitive impairment following encoding of faces, buildings, and words and found enhanced recognition memory. In contrast, Davis et al. (2017) administered offline 1 or 5 Hz rTMS at 120% MT to the left DLPFC of healthy older adults prior to an encoding task and observed null effects on associative memory (although there were differential effects on brain network activity related to successful memory performance, for details see Davis et al., 2017). Although several parameters differed between these studies, the difference in outcomes of 1 Hz rTMS as a function of MTI is consistent with the findings of studies included in the meta-analysis irrespective of differences among those studies. However, there are not enough studies available to compare all combinations of parameters of offline 1 Hz rTMS below-, at-, or above-MTI stimulation.

Twenty Hz rTMS findings in healthy older adults found a similar pattern that was revealed in the meta-analysis with impairments to episodic memory when 20 Hz rTMS was administered online or below MTI. Rossi et al. (2004) reported attenuating effects of online 20 Hz rTMS at 90% MT to the left DLPFC during picture retrieval. In addition, Manenti et al. (2011) found attenuating effects on episodic memory when online 20 Hz rTMS at 90% RMT was applied to left vs. right DLPFC during either encoding or retrieval. However, this effect was largest in low performing adults when stimulation was applied to left compared to right DLPFC during encoding. Thus, it is possible that the effects of 20 Hz rTMS on episodic memory may rely on more complex interactions that were not possible to examine in the current meta-analysis due to the limited number of studies that have investigated the combinations of parameters.

Two studies on older adults with memory disorders found a similar pattern that was revealed in the meta-analysis with enhancing effects of multi-session administration of offline 20 Hz rTMS (Wang et al., 2014; Nilakantan et al., 2017). In prodromal Alzheimer's patients, Koch et al. (2018) reported that 10 sessions (5 sessions/week) with 2-week separation between cross over stimulation (active vs. sham) of offline 20 Hz rTMS at 100% RMT to the precuneus enhanced delayed recall (Koch et al., 2018). Similarly, in an individual with mild cognitive impairment, Cotelli et al. (2012) found enhanced associative memory compared to healthy controls following 10 sessions (5 sessions/week) of offline 20 Hz rTMS at 100% MT to the left parietal cortex (Cotelli et al., 2012).

Importantly, the enhancing effect of offline 20 Hz contrasts with an overall impairment of online 20 Hz rTMS on episodic memory (see Table 2). However, offline 20 Hz studies implemented multi-session protocols (e.g., 5 sessions/week for 2 weeks) while all nine online 20 Hz studies implemented single session protocols (see Table 1), so this confounds interpretation of the potential difference in the direction of effects of 20 Hz rTMS when applied offline vs. online. This also reveals a gap in the literature as few studies have implemented multi-session protocols with younger adults. Collectively, these findings are encouraging with regards to the potential for applying multi-session offline 20 Hz rTMS protocols to enhance episodic memory in older adults, even those with MCI or AD.

Three studies implemented 5 Hz rTMS and found enhanced behavioral outcomes (Solé-Padullés et al., 2006; Peña-Gomez et al., 2012) or modulation of neural activity (Davis et al., 2017). In low performing older adults, Solé-Padullés et al. (2006) reported enhancing effects when offline 5 Hz rTMS at 80% MT was applied prior to encoding face-name associations. In addition, in carriers of a major genetic risk factor for Alzheimer's disease (Apolipoprotein ε4 allele; APOE), enhancing effects were reported following offline 5 Hz rTMS at 80% MT prior to a face-name encoding task (Peña-Gomez et al., 2012). In healthy older adults, Davis et al. (2017) reported modulated neural activity, but no differences in memory performance following offline 5 Hz rTMS at 120% MT prior to an associative encoding task. There are no studies on young adults that administered offline 5 Hz rTMS. This prevents making comparisons and highlights the need for future work to address this gap.

Five studies applied offline 10 Hz rTMS to older adults or clinical populations that included a wide age range and all but two found enhancing effects on episodic memory (see Table 1). The offline 10 Hz studies that found null effects used a different stimulation intensity than the others (Rektorova et al., 2005) or applied rTMS to motor cortex in fibromyalgia patients (Baudic et al., 2013). The studies that found enhancing effects of offline 10 Hz used stimulation intensities below- or above-MTI (Drumond Marra et al., 2015; Qiao et al., 2016; Kavanaugh et al., 2018). These findings diverged from the results of the meta-analysis that revealed a trend for impairments of online 10 Hz rTMS in younger adults. Differences in the stimulation timing between studies is one potential reason for the different effects; however, direct comparison is complicated because no studies to date have applied offline 10 Hz in young adults.

It should also be noted that the current meta-analysis focused on accuracy as the outcome measure of episodic memory performance. We did not address the potential moderating effects of rTMS and study design parameters on other measures of episodic memory performance such as reaction times and confidence ratings. These are important issues to investigate in future research as the body of literature accumulates.

For example, while the meta-analysis on accuracy measures failed to reveal a main effect or interaction with the cortical region that was targeted or the memory process that was stimulated, based on the systematic review of the published studies, it seemed to be the case that several studies found that applying rTMS to regions of parietal cortex affected the subjective experience of remembering as evidenced by modulation of confidence ratings even in the absence of effects on objective measures of accuracy. For example, Ye et al. (2018) recorded participants playing an action-adventure video game and then applied offline 1 Hz rTMS at 110% AMT to the precuneus prior to snapshot images from the video game and then tested subjects' temporal order memory and confidence ratings. The authors found decreases in confidence ratings during retrieval following precuneus compared to vertex stimulation, despite no differences in memory performance (Ye et al., 2018).

In a similar fashion, Bonnì et al. (2015) applied offline cTBS at 100% MT to the precuneus, posterior parietal cortex, or vertex prior to associative (color-context) memory tests of studied object images (e.g., apple outlined in red). While there were no differences in hit rates among the conditions, there was a significant decrease in the number of source memory errors (e.g., correct judgment that the apple was outlined in red not green) that were made following precuneus stimulation (Bonnì et al., 2015). Relatedly, offline 20 Hz rTMS at 100% MT to the left angular gyrus prior to retrieval of previously encoded pictures led to decreased recognition relative to superior parietal lobe stimulation, and a decrease in recollective details (by altering response bias) compared to either superior parietal or vertex stimulation (Sestieri et al., 2013). Moreover, offline cTBS at 70% RMT to the left angular gyrus prior to encoding auditory stimuli reduced subjective confidence ratings for source memory while leaving free- and cued- recall of word pairs unaffected (Yazar et al., 2014). Taken together, it seems that rTMS to parietal regions tends to affect the subjective experience of remembering even when there are null effects on episodic memory accuracy. However, too few studies have been published to allow for meta-analysis of the effects of moderators of the size of effects on episodic memory. The same was true for other important measures of episodic memory performance such as reaction times. Future research should address these limitations.

Another limitation has to do with the complication of interpreting the effects of rTMS on episodic memory due to the design of many studies, particularly studies with multiple rTMS sessions or offline rTMS, because it is difficult to know if the effects of rTMS are modulating encoding, consolidation, and/or retrieval processes. Relatedly, to our knowledge, only one study implemented rTMS to definitively target consolidation-based processes. In a pair of studies, Sandrini et al. (2013) investigated the role of right DLPFC in the reconsolidation of memories for studied words of objects by administering offline 1 Hz rTMS at 100% MT. The first experiment spanned 3 days. On day one, participants learned a list of words with contextual reminders (e.g., specific room and colored bag containing the words). On day two, participants were assigned to receive either rTMS to right DLPFC during a pure consolidation phase without reactivating the memories or rTMS to right DLPFC or the vertex before reactivating the memories in a contextual reminder reconsolidation phase. Finally, on day three, participants underwent a free recall task in which participants recalled as many words as possible. The findings revealed that rTMS prior to memory reactivation led to memory enhancements compared to the no reactivation and vertex control conditions. To examine if rTMS effects were specific to reconsolidation and rather than either encoding or retrieval processes, a follow up experiment used a similar procedure with the exception that memory was assessed 1 h after stimulation instead of the next day. Given that reconsolidation processes take several hours, finding no memory differences across the conditions after 1 h supported the conclusion that the DLPFC was specifically involved in reconsolidation processes (Sandrini et al., 2013). The use of multiple active control conditions further helped the researchers to isolate the effects to reconsolidation processes (as opposed to initial encoding or retrieval). This study demonstrates the unique feasibility of using rTMS to systematically manipulate (re)consolidation processes. However, because there is so little research that has investigated the effects of rTMS on consolidation processes, the current meta-analysis was restricted to studies that have investigated effects on encoding and retrieval stages. This highlights an area where future TMS research is needed.

Another limitation of the current meta-analysis is the possibility that we may have underestimated the true effect of rTMS on episodic memory performance across conditions. The quality assessment revealed that a large number of studies targeted cortical regions using the international 10–20 electrode positioning system. This approach is imprecise and does not take into account inter-individual differences in the location of specific brain structures. Structurally or functionally guided TMS targeting using a neuronavigation system with each subject's structural and/or functional MRI can more precisely stimulate the targeted brain region of interest. Modeling work that has contrasted the variety of targeting methods has shown that larger TMS effects are obtained with individual fMRI-guided TMS neuronavigation when targeting parietal regions than targeting with the 10–20 EEG positioning system, which yielded smaller effect sizes (Sack et al., 2009). Power analyses revealed that a sample size of 47 was required to detect a significant effect when the 10–20 EEG positioning system is used for TMS targeting (Sack et al., 2009). Given that a large number of studies implemented the 10–20 targeting system and the selected studies had an average sample size of 24.19 in the meta-analysis, it is possible that the existing effects sizes available in the published literature underestimate the true effect of rTMS on episodic memory. Another important aspect to note is that only published effect sizes could be included in the meta-analysis and, therefore, excluding any unpublished null effects would obscure and overestimate the true effect of rTMS on episodic memory. Reassuringly, the steps we took to investigate a potential publication bias suggests that this was unlikely to be of major concern. However, it should be noted that absence of small-study biases does not prove the absence of publication bias, and further research is needed to properly address publication bias in multi-level models.

Below we highlight and discuss potentially important factors that need to be systematically investigated over the next 30 years of TMS research and beyond. One pervasive issue in the field has been the highly variable findings across both behavioral and neural outcomes. Certain individuals may respond to a set of rTMS parameters, while other individuals may have null or negative effects to the same rTMS protocol. One possible solution to this heterogeneity in the response to rTMS is to implement more personalized rTMS parameters titrated to match each individual's physiological characteristics that likely underlie the variability between individuals. We have already noted the need to use structural and functional neuronavigated targeting procedures. Other characteristics that are likely to be important concern the titration of the frequency of stimulation to the precise frequency of endogenous oscillations of interest in targeted brain regions.

For example, in a pair of studies, Waldhauser et al. (2016) implemented a personalized rTMS-EEG design to examine if reactivation of sensory information is necessary for recognition of studied items and/or their studied context/source. In experiment one, participants intentionally encoded visual stimuli (i.e., objects) presented laterally (left and right) to a fixation cross; at test, participants made old/new judgments and source judgments about whether the item initially appeared on the left or right side of the screen while undergoing concurrent EEG recording. Identification of a time window (33.5–204.5 ms), frequency (17.5 Hz), and source (MNI coordinates ± 40, −78, 0) of oscillatory behavior for successful source memory judgments were obtained by examining the lateralization of alpha/beta activity during encoding and retrieval. In a second experiment using the same encoding/retrieval procedure they applied rTMS (17.5 Hz) to the occipital lobe (MNI coordinates ± 40, −78, 0) during retrieval from 33.5 to 204.5 ms after cue presentation (Waldhauser et al., 2016). Importantly, when rTMS was applied using the specific temporal, spatial, and frequency parameters related to the task demands for successful episodic retrieval, there was a decrease in source memory performance relative to control.

In a similar study, Hanslmayr et al. (2014) compared online rTMS-EEG at 90% RMT in the beta frequency range (18.7 Hz) to 10.7, 6.8 Hz, and sham stimulation to the left IFG during word encoding. On a subsequent free recall task, participants showed memory impairment following 18.7 Hz stimulation compared to sham. Critically, despite targeting the same task-dependent region, there were no differences in memory performance for 10.7 or 6.8 Hz stimulation compared to sham. At the neural level, the degree of beta entrainment following 18.7 Hz stimulation was correlated with memory impairment, with higher levels of beta entrainment leading to greater decrements in memory performance (Hanslmayr et al., 2014). The findings that artificial beta synchronization leads to memory impairment supports electrophysiology findings that alpha/beta desynchronization is associated with memory enhancement (Hanslmayr et al., 2016). Thus, the implementation of more nuanced rTMS frequencies affords the possibility to more effectively modulate episodic memory.

In this review, many of the studies that assessed the effects of rTMS on episodic memory involved spatial locations/associations with stimuli (e.g., object-location associations). While spatial context information is often one important element of episodic memory, there are other important aspects of episodic memories (e.g., object, temporal, social, and emotional information) that may result in dissociable effects of rTMS to different brain regions. Furthermore, it is possible that the type of episodic information (e.g., visual, auditory, multi-modal) may interact with different aspects of episodic memory (e.g., object, temporal, and spatial). Future research should consider these potentially important moderating variables when investigating the effects of rTMS on various aspects of episodic memory.

The majority of rTMS findings have focused on researching the brain as a “black box” in an open-loop manner, whereby a priori input stimulation parameters remain constant and one observes the corresponding output. Emerging evidence from multi-modal approaches has begun to clarify how stimulation parameters such as timing and frequency modulate brain-behavior relationships. This has also led to a shift in focus from mapping segregated brain regions to a more interactive-systems approach by attempting to understand how fast-changing patterns of neural activity interact with local and distal brain regions to give rise to human behavior. Understanding the frequency and temporally specific manner that different cortical regions interact has helped uncover how neural processing is linked with behavior. However, the optimal stimulation parameters that lead to episodic memory enhancement or detriment have not been clearly identified and the sparseness of available findings await replication and extension with systematic manipulations to map the parameter space. Moreover, there has been little consideration paid to how memory outcomes may be dependent on interactions between stimulation protocols and temporal patterns of initial brain state activity and ongoing neural activity during a task. The results of the current meta-analysis reveal that it is of paramount importance that future studies take this into consideration in order to determine optimal stimulation parameters for episodic memory (for additional information see, Bergmann et al., 2016)

While this “black box” approach has proven fruitful, the brain is a dynamic system capable of receiving, modulating, and generating both further inputs (modified representations, novel associations) as well as outputs (Berkman and Lieberman, 2011). A wave of technological advancements has made it possible to make stimulation adjustments based on real-time neural signals. This has given rise to an informed closed-loop paradigm where neural activity from EEG/MEG provide amplitude, phase, or power of oscillatory activity markers that allow for stimulation in a temporally specific fashion. This has also paved the way for fully adaptive closed-loops that aim to use stimulated neural markers to not only trigger stimulation but also to use neural activity to adaptively alter stimulation protocols in real time (Karabanov et al., 2016).

Given the wealth of knowledge that has been obtained using multimodal approaches, extending these typically two-way combinations (e.g., TMS-EEG and TMS-fMRI) to three-way TMS-EEG-fMRI tools may provide a more holistic view of brain functioning. Such a combination with TMS enables novel stimulation findings based on the temporal precision of EEG to be complemented with fMRI to examine changes in functionally related neural activity in cortical and subcortical structures with high spatial resolution. The feasibility of combining TMS-EEG-fMRI has already been established to be safe and provide reliable signal recordings (Peters et al., 2013).

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.