Test–Retest Reliability of the Effects of Continuous Theta-Burst Stimulation

Abstract

Objectives:

The utility of continuous theta-burst stimulation (cTBS) as index of cortical plasticity is limited by inadequate characterization of its test–retest reliability. We thus evaluated the reliability of cTBS aftereffects, and explored the roles of age and common single-nucleotide polymorphisms in the brain-derived neurotrophic factor (BDNF) and apolipoprotein E (APOE) genes.

Methods:

Twenty-eight healthy adults (age range 21–65) underwent two identical cTBS sessions (median interval = 9.5 days) targeting the motor cortex. Intraclass correlation coefficients (ICCs) of the log-transformed, baseline-corrected amplitude of motor evoked potentials (ΔMEP) at 5–60 min post-cTBS (T5–T60) were calculated. Adjusted effect sizes for cTBS aftereffects were then calculated by taking into account the reliability of each cTBS measure.

Results:

ΔMEP at T50 was the most-reliable cTBS measure in the whole sample (ICC = 0.53). Area under-the-curve (AUC) of ΔMEPs was most reliable when calculated over the full 60 min post-cTBS (ICC = 0.40). cTBS measures were substantially more reliable in younger participants (< 35 years) and in those with BDNF Val66Val and APOE ε4– genotypes.

Conclusion:

cTBS aftereffects are most reliable when assessed 50 min post-cTBS, or when cumulative ΔMEP measures are calculated over 30–60 min post-cTBS. Reliability of cTBS aftereffects is influenced by age, and BDNF and APOE polymorphisms. Reliability coefficients are used to adjust effect-size calculations for interpretation and planning of cTBS studies.

Introduction

Transcranial magnetic stimulation (TMS) is a method for focal non-invasive stimulation of the brain through electromagnetic induction (Barker et al., 1985). Application of TMS within the recommended guidelines (Rossi et al., 2009; Rossini et al., 2015) is a safe means of triggering or modulating neural activity in a given brain region or network (Pascual-Leone et al., 2011; Fox et al., 2012; Valero-Cabré et al., 2017). A form of repetitive TMS (rTMS) known as continuous theta-burst stimulation (cTBS) consists of 50 Hz bursts of three TMS pulses repeated at 5 Hz for a total of 600 pulses over 40 s (Huang et al., 2005). The average amplitude of motor evoked potentials (MEPs) induced by single TMS pulses is reduced by approximately 25% for up to 50 min following cTBS of the primary motor cortex (M1) (Wischnewski and Schutter, 2015). This neuromodulatory effect is thought to involve mechanisms similar to long-term depression (LTD) (Pascual-Leone et al., 1994; Huang et al., 2005; Hallett, 2007). Therefore, the pattern of cTBS-induced changes in MEPs provides a neurophysiologic index of the mechanism of cortical plasticity (Pascual-Leone et al., 2005, 2011; Oberman et al., 2010, 2012, 2014, 2016; Suppa et al., 2016).

The neuromodulatory effect of cTBS applied to M1 or other brain regions has been investigated for its potential as a neurophysiological biomarker and a therapeutic intervention in several neurological and psychiatric disorders (Koch et al., 2009, 2012; Eberle et al., 2010; McClintock et al., 2011; Cazzoli et al., 2012; Oberman et al., 2012, 2014; Di Lazzaro et al., 2013, 2016; Mori et al., 2013; Cantone et al., 2014; Chuang et al., 2014; Forogh et al., 2014; Li et al., 2014; Suppa et al., 2014; Carrette et al., 2016). Despite its growing popularity, however, cTBS responses show large inter-individual (Hamada et al., 2013; Goldsworthy et al., 2014; López-Alonso et al., 2014; Vallence et al., 2015; Guerra et al., 2017; Heidegger et al., 2017; Hordacre et al., 2017; Jannati et al., 2017) and intra-individual variability (Vernet et al., 2014; Vallence et al., 2015) that can limit the utility of cTBS for assessing brain plasticity in clinical populations.

Only two published studies have assessed the reproducibility of cTBS aftereffects (Vernet et al., 2014; Vallence et al., 2015). The first (Vernet et al., 2014) used a relatively small sample size (n = 10) and did not report reliability coefficients of cTBS aftereffects, which can be compared with the reliability coefficients of other TMS measures (Carroll et al., 2001; Kimiskidis et al., 2004; Christie et al., 2007; Livingston and Ingersoll, 2008; Bastani and Jaberzadeh, 2012; Ngomo et al., 2012; Hinder et al., 2014; Liu and Au-Yeung, 2014; Sankarasubramanian et al., 2015; Schambra et al., 2015; Hermsen et al., 2016; Fried et al., 2017). The second study (Vallence et al., 2015) used an input–output curve approach that allowed assessment of cTBS aftereffects elicited over a range of stimulation intensities, but at the cost of fewer time-points. Specifically, assessments were only performed at 0, 15, and 30 min post-cTBS. This excluded the earliest time points, i.e., 5 and 10 min post-cTBS, which typically exhibit the maximal cTBS effects (Wischnewski and Schutter, 2015) and later time-points, i.e., 40–60 min post-cTBS, which capture the longer-lasting TBS effects and have been found to be useful in differentiating clinical populations such as individuals with Alzheimer’s disease (Freitas et al., 2011), autism spectrum disorder (Oberman et al., 2012), diabetes (Fried et al., 2016), and schizophrenia (McClintock et al., 2011) from healthy individuals. For comparison, at least three studies have assessed the reliability of intermittent theta-burst stimulation (iTBS) aftereffects (Hinder et al., 2014; Fried et al., 2017; Schilberg et al., 2017).

Full characterization of the test–retest reliability of cTBS aftereffects is essential to properly interpret results and plan for future studies. We thus aimed to address this need by systematically assessing the test–retest reliability of cTBS aftereffects in 5- or 10-min intervals for 60 min post-cTBS in a sizeable sample of healthy adults. We also calculated adjusted effect sizes for cTBS aftereffects by taking into account the reliability (or lack thereof) of each cTBS measure (Friedman, 1968; Wright, 2014; Fried et al., 2017). In addition, we explored the effects of age group on the reproducibility of cTBS aftereffects, as well as of single-nucleotide polymorphisms (SNPs) in brain-derived neurotrophic factor (BDNF) and apolipoprotein E (APOE) genes, which have been found to influence neuroplasticity (White et al., 2001; Cheeran et al., 2008; Nichol et al., 2009; Antal et al., 2010; Peña-Gomez et al., 2012; Lee et al., 2013; Chang et al., 2014; Di Lazzaro et al., 2015; Jannati et al., 2017). Our results can improve the utility of cTBS as a neurophysiologic index of cortical plasticity in neurological and psychiatric disorders, help elucidate the sources of intra-individual variability in cTBS responses, and ensure adequate sample size and power in future cTBS studies in clinical populations.

Materials and Methods

Participants

Twenty-eight healthy adults (25 males, age range: 21–65) participated in the study, which was approved by the local Institutional Review Board in accordance with the Declaration of Helsinki. All participants provided written informed consent prior to enrollment and received monetary compensation upon completion. None of the participants had any TMS contraindication (Rossi et al., 2009), and all had normal physical and neurological examinations. Individual and group-level demographics are presented in Tables 1 and 2, respectively.

Neuropsychological Testing

Mini-Mental State Examination (Folstein et al., 1975; Crum et al., 1993) and the Abbreviated Battery of Stanford–Binet IV intelligence scale (Thorndike et al., 1986), including Verbal Knowledge and Non-Verbal Fluid Reasoning subscores, were completed.

Genetic Analyses

Saliva samples from 22 participants were assessed for BDNF Val66Met polymorphism and the presence of APOE ε4 allele, as reported previously (Jannati et al., 2017). Aliquot (700 μL) extraction of genomic DNA was performed on saliva samples collected using the Oragene Discover OGR-250 Kit (DNA Genotek Inc., Ottawa, ON, Canada). DNA was extracted from samples using standard methodology and the prepIT L2P reagent (DNA Genotek Inc., 2015). The rs6265 SNP of the BDNF gene, and the rs429358 and the rs7412 SNPs of the APOE gene were analyzed using a TaqMan single-tube genotyping assay, which uses polymerase chain reaction (PCR) amplification and a pair of fluorescent dye detectors that target the SNP. During PCR, the polymerase released the fluorescent probe into solution where it was detected using endpoint analysis in an 7900HT Real-Time instrument from Applied Biosystems, Inc. (Foster City, CA, United States).

Transcranial Magnetic Stimulation

Two identical TMS visits (7–33 days apart; median interval = 9.5 days) were conducted. The starting times of the two visits were 0–254 min apart (interquartile range = 1–103 min; median = 12 min). The inter-visit intervals and starting-time differences for individual subjects are presented in Table 1.

All TMS procedures followed the recommended guidelines endorsed by the International Federation of Clinical Neurophysiology (Rossi et al., 2009; Rossini et al., 2015). Participants were seated in a comfortable chair with the right arm and hand in a natural pronated ∼90° angle on a pillow in front of them. They were instructed to keep their right hand as still and relaxed as possible throughout the experiment. They were also monitored for drowsiness and were asked to keep their eyes open during the TMS application. Single TMS pulses and cTBS were applied to the left primary motor cortex (M1) at 120% of individual resting motor threshold (RMT) and 80% of active motor threshold (AMT), respectively, as biphasic pulses with an antero-posterior–postero-anterior (AP-PA) induced current direction using a MagPro X100 stimulator and a MC-B70 Butterfly Coil (outer diameter: 97 mm; MagPro, MagVenture A/S, Farum, Denmark). The coil was held tangentially to the participant’s head surface, with the handle pointing occipitally and positioned at 45° relative to the mid-sagittal axis of the participant’s head. The optimal spot for the maximal responses of the right first dorsal interosseous (FDI) muscle (“motor hotspot”) was localized. A Polaris infrared-optical tracking system (Northern Digital Inc., Waterloo, ON, Canada) and a Brainsight TMS neuronavigation system (Rogue Research Inc., Montreal, QC, Canada) with a brain MRI template (for 21 participants) or the participant’s brain MRI (for the remaining 7 participants) was used to ensure consistent targeting throughout the experiment.

Surface electromyogram (EMG) was recorded from the right FDI with a PowerLab 4/25 data-acquisition device and LabChart 8 software (AD Instruments, Colorado Springs, CO, United States). Electrodes were placed over the FDI belly (negative) and the first interphalangeal joint of the second finger (positive). The ground electrode was placed over the ipsilateral ulnar styloid process. The TMS system delivered triggered pulses that synchronized the TMS and EMG systems. EMG signal was digitized at 1 kHz for 500 ms following each stimulus trigger and 100 ms pre-trigger, amplified with a range of ±10 mV (band-pass filter 0.3–1000 Hz).

Each TMS session began by localizing the motor hotspot for FDI and assessment of the RMT, defined as the lowest intensity of stimulation that elicited MEPs ≥ 50 μV in at least five of ten pulses in the relaxed right FDI. To assess pre-cTBS cortico-motor reactivity, three blocks of 30 single TMS pulses were applied to M1, with a 5–10 min inter-block interval and at a random 4–6 s inter-pulse interval. In each block, individual MEPs > 2.5 SD from the mean were excluded. Baseline MEP amplitude was calculated as the average of the peak-to-peak amplitude of MEPs in the three blocks. The AMT was then assessed as the lowest intensity that elicited MEPs ≥ 200 μV in at least five of ten pulses with the FDI slightly contracted. After a 5-min break, during which participants were instructed to maintain hand relaxation to control the effects of voluntary hand movements on cTBS responses (Iezzi et al., 2008), cTBS was applied as 200 bursts of three pulses at 50 Hz, repeated at 200-ms intervals for 40 s (for a total of 600 pulses). Cortico-motor reactivity was reassessed at 5, 10, 15, 20, 30, 40, 50, and 60 min post-cTBS (T5–T60).

Statistical Analyses

Stata software version 13.1 (StataCorp, College Station, TX, United States) and MATLAB and Statistics and Machine Learning Toolbox R2016b (The MathWorks, Natick, MA, United States) were used for data analysis. Data from each TMS visit included: (a) RMT and AMT, expressed as percentage of maximum stimulator output; (b) baseline MEP amplitude, calculated as the average of baseline MEP amplitude in 3 blocks of 30 single TMS pulses; and (c) percent change in the average amplitude of 30 MEPs at T5–T60 relative to baseline (%Δ) for each participant.

The Shapiro–Wilk test found significant deviations in MEP values from normal distribution; thus, natural log-transformed, baseline-corrected MEP amplitude at each post-cTBS time point (ΔMEP) was averaged over all participants separately for each visit. The following measures were also calculated: absolute MEP modulation at T5–T60 (|ΔMEP|), maximum suppression and maximum modulation of MEPs during 60 min post-cTBS, area under-the-curve (AUC) and the absolute AUC value (|AUC|) of ΔMEPs over T0–T10, …, and T0–T60 intervals. Cumulative AUC and |AUC| measures up to each time-point were calculated as the summed products of the average ΔMEP and the average |ΔMEP|, respectively, across each two consecutive time-points and the time in minutes between them.

Grand-average values for all cTBS measures were calculated separately for both visits and were compared against zero using one-sample t-tests. Visit-B minus visit-A difference (Δ_B-A) and |Δ_B-A| were calculated for each neurophysiological measure (Table 3). All analyses were two-tailed, and the α level was set to 0.05. When explicitly noted, false discovery rate (FDR) was used to adjust p-values for multiple testing (Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001).

ΔMEPs at T10 and T40 were previously found to be the best predictors of inter-individual variability in cTBS aftereffects in visit-A (Jannati et al., 2017). Thus, to assess the effect of potential covariates on the intra-individual variability of cTBS aftereffects at T10 and T40, we conducted linear mixed-effects (LME) regression analyses with ΔMEPs at T10 or T40 as dependent variable, Visit (visit-A vs. visit-B) as a within-subject factor, and potential covariates including RMT, AMT, baseline MEP amplitude, number of days between the two visits (Inter-visit Interval), and the absolute inter-visit difference in starting time (in minutes) (Time Difference) as between-subject factors. Based on previous studies that found in many situations a regression model is likely to be reliable when the number of candidate predictors is smaller than one-tenth of the number of subjects (Harrell, 2015) (p. 72), up to three between-subjects predictors were considered for simultaneous inclusion in any regression model.

To assess test–retest reliability, intraclass correlation coefficients (ICCs) (Portney and Watkins, 2009) were calculated in the form of absolute agreement between the two visits for all neurophysiological measures. ICCs were calculated using a two-way mixed-effects model, with fixed column (C) effects and random row (R) effects (McGraw and Wong, 1996):

where ICC(A,1) represents the degree of absolute agreement of measurements made under the two fixed levels of the column factor. k = the number of raters/measurements per subject; MS_R = mean square for rows (representing the individual subjects); MS_E = mean square error; MS_C = mean square for columns (representing the two visits); n = the number of subjects.

Using this formula, ICC = 1 indicates maximum reliability and ICC ≤ 0 indicates no reliability [in the case that the within-group variance is equal to or higher than the between-groups variance (Kenny et al., 2002)]. ICC values were interpreted as follows (Portney and Watkins, 2009): (i) ICC < 0.25: very low to no reliability; (ii) 0.25 ≤ ICC < 0.50: low reliability; (iii) 0.50 ≤ ICC < 0.75: moderate reliability; and (iv) ICC ≥ 0.75: high reliability. ICC values were statistically compared using two-way mixed-effects F statistics (McGraw and Wong, 1996, Table 8). The effects on the ICCs of covariates that had a significant effect on ΔMEPs were assessed by including the covariate in the corresponding mixed-effects regression model and re-calculating the residual intraclass correlation.

Lack of reliability of a measure of interest attenuates the observed effect size compared to the population parameter (Hunter and Schmidt, 1994). Following previously applied methodology (Friedman, 1968; Wright, 2014; Fried et al., 2017), we assessed how test–retest reliability (or lack thereof) of TMS measures would attenuate small, medium, and large effect sizes, i.e., Cohen’s d values of 0.2, 0.5, and 0.8 (Cohen, 1992), respectively, which assume perfect reproducibility. First, each idealized Cohen’s d is converted to an r (Cohen, 1988) (p. 23):

This idealized r is then adjusted for unreliability using the ICC (Wright, 2014):

Finally, the adjusted r is converted back to an adjusted d (Friedman, 1968) (p. 246):

Exploratory Analyses

Though unintended, the age of our participants conformed to a bimodal distribution. Thus, to explore the impact of age on our reliability measures, the total sample was subdivided into two distinct age groups with a 10-year gap and a ∼27-year difference in mean age: a Younger group with age < 35 (n = 16; range: 21–34; mean ± SD, 25.3 ± 4.3) and an Older group with age ≥ 45 (n = 12; range: 45–65; mean ± SD, 52.1 ± 6.5). To explore the effect of age on the test–retest reliability of cTBS aftereffects: (i) the ICC values of TMS measures were calculated separately for the two age groups; (ii) separate repeated-measures analyses of variance (Rm-ANOVAs) were conducted with ΔMEP at T10 or T40 as dependent variable, Age Group as a between-subjects factor, Visit as a within-subject factor, and Age Group × Visit interaction. Because the proportion of Hispanic participants was significantly higher in the Younger group than in the Older group, we assessed the effect of Ethnicity as a categorical covariate in these Rm-ANOVAs. We also re-calculated all the ICCs for the Younger group while controlling for Ethnicity.

To explore the roles of BDNF and APOE SNPs in the reliability of TMS measures, we calculated the ICC values of neurophysiological measures over the two visits separately for participants with BDNF Val/Val (Met–; n = 14) and Val/Met (Met+; n = 8) genotypes as well as for those with APOE ε2/ε3 or ε3/ε3 (ε4–; n = 12) and APOE ε2/ε4 or ε3/ε4 (ε4+; n = 10) genotypes.

Because BDNF was previously found to influence the cTBS aftereffect at T10 (Jannati et al., 2017), we assessed the cTBS aftereffects separately for BDNF Met– and Met+ participants in each visit and conducted a Rm-ANOVA with ΔMEP at T10 as dependent variable, BDNF Status (Met– vs. Met+) as a between-subjects factor, Visit (visit-A vs. visit-B) as a within-subject factor, and BDNF Status × Visit interaction. Further, we assessed the effect of BDNF Status as a covariate in the LME regression analyses at T10.

Results

Demographics, neuropsychological measures, inter-visit interval, starting times of the two visits, and inter-visit differences in starting time for individual participants are presented in Table 1. Statistical comparisons of these measures between the two age groups are presented in Table 2.

Genetic Analyses

Available BDNF and APOE results and comparisons of all available measures between BDNF/APOE subgroups are presented in Tables 1 and 2, respectively.

Among 22 participants with available DNA results, the frequencies of BDNF Val/Val and Val/Met genotypes were 0.64 and 0.36, respectively, while the frequencies of APOE ε2/ε3, ε3/ε3, and ε3/ε4 genotypes were 0.14, 0.41, and 0.46, respectively. BDNF and APOE subgroups were comparable in all available measures (Table 2).

Baseline Neurophysiological Measures

The RMT, AMT, and baseline MEP amplitude in each visit and their inter-visit differences are summarized in Tables 2 and 3, respectively.

There were no significant differences in any of the baseline neurophysiological measures in either visit between age or genetic subgroups (Table 2). There was also no significant difference between the two visits in any of the baseline neurophysiological measures for the whole sample (p’s > 0.14), for each age group (p’s > 0.17), or for each BDNF (p’s > 0.08) or APOE (p’s > 0.28) subgroup.

The LME regression analyses of ΔMEP at T10 found a significant, negative effect of Time Difference in all models (p’s < 0.025), but no significant effect of any of the baseline neurophysiological measures or Visit (p’s > 0.21). LME regression analyses of ΔMEP at no other time point found a significant effect of Time Difference (p’s > 0.41).

cTBS-Induced Plasticity Results

Grand-average ΔMEPs in visits A and B are shown in Figure 1. ΔMEP and |ΔMEP| values, maximum suppression and maximum modulation as well as their inter-visit differences are summarized in Table 3. Grand-average ΔMEPs did not significantly differ from zero at any time point in either visit (p’s > 0.11). There was also no significant difference in grand-average ΔMEP or |ΔMEP| between the two visits at any time point (p’s > 0.16).

FIGURE 1

The ΔMEPs in the Older group were significantly greater than zero at T20 in visit-A (P_FDR = 0.029), but not at any other time point in either visit (p’s > 0.18). The ΔMEPs in the Younger group did not significantly differ from zero at any time point in either visit (p’s > 0.06). The ΔMEPs in the BDNF Met– group were significantly less than zero at T10 in visit-A (P_FDR = 0.042), but not at any other time point in either visit (p’s > 0.14). The ΔMEPs in the BDNF Met+ group were not significantly different from zero at any time point in either visit (P_FDR’s > 0.05). cTBS aftereffects in both visits for the two age subgroups and the two BDNF subgroups are presented in Figure 2 and 4 respectively.

FIGURE 2

The Rm-ANOVA on the ΔMEP at T10 found a significant effect of BDNF status, F(1, 20) = 8.28, p = 0.009, = 0.29, but no significant effects of Visit or BDNF × Visit interaction (p’s > 0.10). BDNF Met-carrier status had a significant positive effect in all LME regression analyses of ΔMEP at T10 (s > 0.28, p’s < 0.027). There was no other significant effect in any of the LME models (p’s > 0.08).

To control for potential effects of gender, race/ethnicity, and handedness on cTBS-induced plasticity measures, we calculated ΔMEP’s at T5–T60 in a subgroup of White, non-Hispanic, and right-handed males (n = 11, Table 1). In this smaller, but more-homogenous subsample, ΔMEPs did not significantly differ from zero at any time point in either visit (p’s > 0.23).

Test–Retest Reliability of TMS Measures

Measures of inter-visit variability and test–retest reliability for RMT, AMT, baseline MEP amplitude and cTBS measures, as well as reliability-adjusted effect sizes for each TMS measure in the whole group are presented in Table 3.

The ICCs of baseline neurophysiological measures was not significantly different between the two age groups (p’s > 0.19; Figure 3). In contrast, ΔMEPs in the Younger group were significantly more reliable than in the Older group at T10 and T60 (P_FDR’s < 0.015), but not at other individual time points (P_FDR’s > 0.058). Similarly, |AUC| measures were significantly more reliable in the Younger group than in the Older group over T0–T20 and beyond (P_FDR’s < 0.001; Figure 3). The reliability of other cumulative ΔMEP measures was not significantly different between the two age groups (p’s > 0.058). After adjusting for Ethnicity, none of the ICCs in the Younger group crossed our pre-defined boundaries for interpretation of ICC values (see “Materials and Methods” section).

FIGURE 3

To control for potential effects of gender, race/ethnicity, and handedness on the test–retest reliability of cTBS aftereffects, we calculated the ICC values of baseline neurophysiological measures and ΔMEP’s at T5–T60 among White, non-Hispanic, and right-handed males (n = 11, Table 1). The ICC values of RMT, AMT, and baseline MEP amplitude in this subgroup were 0.86 (p < 0.001), 0.96 (p < 0.001), and 0.75 (p = 0.003), respectively. The ICC values of ΔMEP at T5, T10, T15, T20, T30, T40, T50, and T60 in this subgroup were 0.54 (p = 0.031), –0.09 (p = 0.602), 0.18 (p = 0.297), 0.71 (p = 0.005), 0.86 (p < 0.001), 0.67 (p = 0.008), 0.79 (p = 0.001), and 0.17 (p = 0.309), respectively.

The ICCs of baseline neurophysiological measures were not significantly different between either the BDNF or the APOE subgroups (p’s > 0.16; Figure 5). In contrast, ΔMEPs were significantly more reliable in BDNF Met– participants than in BDNF Met+ participants at T20–T40 (P_FDR’s < 0.023), but not at other time points (p’s > 0.24). Maximum suppression, maximum modulation, and AUC of ΔMEPs over T0–T30 and beyond were significantly more reliable in BDNF Met– participants than in BDNF Met+ participants (P_FDR’s < 0.032; Figure 5). The reliability of other cumulative ΔMEP measures was not significantly different between the two BDNF subgroups (P_FDR > 0.21).

ΔMEPs were significantly more reliable in APOE ε4– participants than in APOE ε4+ participants at T5 and T20–T40 (P_FDR’s < 0.024), but not at other time points (P_FDR’s > 0.07). All AUC measures were significantly more reliable in APOE ε4– participants than in APOE ε4+ participants (P_FDR’s < 0.021; Figure 5). There were no significant differences in the reliability of maximum suppression or maximum modulation between the two APOE subgroups (P_FDR’s > 0.27).

Discussion

Test–retest reliability of TMS measures influences their utility as potential neurophysiologic biomarkers or targets for therapeutic intervention. As the use of plasticity-inducing rTMS protocols becomes more common, it is necessary to investigate the magnitude and sources of their inter- and intra-individual variability. While some of the factors that contribute to the inter-individual variability of these types of plasticity metrics among healthy individuals have been identified (Cheeran et al., 2008, 2009; Antal et al., 2010; Hamada et al., 2013; Goldsworthy et al., 2014; López-Alonso et al., 2014; Nettekoven et al., 2014, 2015; Vallence et al., 2015; Suppa et al., 2016; Hordacre et al., 2017; Jannati et al., 2017), few studies have assessed the intra-individual reliability of cTBS responses (Vernet et al., 2014; Vallence et al., 2015) and no study, to our knowledge, has systematically assessed the test–retest reliability of cTBS aftereffects during 60 min post-cTBS. The present study was designed to fill this gap by assessing the test–retest reliability of cTBS aftereffects at 5- or 10-minute intervals (T5–T60) and of cumulative cTBS aftereffects during 60 min post-cTBS in healthy adults. Furthermore, in order to provide guidance and reference for future studies, we calculated adjusted effect sizes that take into account the test–retest reliability of cTBS measures. Finally, we explored the influences of age group and common SNPs in BDNF and APOE genes on the reliability of cTBS aftereffects.

Overall Reliability of Baseline Neurophysiological Measures

Resting motor threshold had high test–retest reliability (Table 3), which was comparable with the ICC values reported in most previous studies (Carroll et al., 2001; Kimiskidis et al., 2004; Livingston and Ingersoll, 2008; Bastani and Jaberzadeh, 2012; Ngomo et al., 2012; Hinder et al., 2014; Schambra et al., 2015; Fried et al., 2017; Davila-Pérez et al., 2018) and somewhat higher than other studies that found RMT ICCs in the 0.75–0.80 range (Fleming et al., 2012; Liu and Au-Yeung, 2014; Sankarasubramanian et al., 2015; Hermsen et al., 2016). The AMT also had high test–retest reliability (Table 3), which was comparable with the results of previous studies (Ngomo et al., 2012; Hinder et al., 2014; Fried et al., 2017).

Baseline MEP amplitude had moderate test–retest reliability (Table 3). The ICC of baseline MEP amplitude found in the present study (ICC = 0.70) was moderate compared to the wide range of ICC values for baseline MEP amplitude (–0.16 to 0.87) reported in previous studies (Kamen, 2004; McDonnell et al., 2004; Christie et al., 2007; Fleming et al., 2012; Ngomo et al., 2012; Hinder et al., 2014; Hermsen et al., 2016; Fried et al., 2017; Davila-Pérez et al., 2018). Variability of baseline MEP amplitude was previously found to be associated with variability of TBS aftereffects (Hordacre et al., 2017; Fried et al., 2017). The moderate reliability of baseline MEP amplitude in the present study suggests that such variability was not the main cause of the low reliability of some of the cTBS measures reported here.

Overall Reliability of cTBS Aftereffects

The finding that differences between the start times of the two visits influenced the cTBS aftereffects at T10 could be due to the effect of circadian rhythm on the neuromodulatory effects of rTMS arising from changes in cortical excitability and synaptic efficiency during the day (Cohen et al., 2005). While the present results cannot definitively conclude that circadian factors influenced the intra-individual variability in plasticity at T10, future studies could attempt to control for the time of day or, perhaps even better, to individualize visits to coincide with the same relative point in each subject’s circadian cycle.

The finding that T5 had one of the lowest between-visit variabilities among post-cTBS time points is consistent with the findings of a previous study (Vernet et al., 2014). Importantly, however, the low between-visit variability of cTBS aftereffects at T5 and T50 at the group level in the present study did not translate to high test–retest reliability measures, which take into account both within-individual and between-individuals variability; while T50 was the most reliable post-cTBS time point (ICC = 0.53), T5 had very low reliability (ICC = 0.16). This pattern of results underlines the importance of calculating the ICCs of TMS measures rather than relying only on measures of inter-visit variability at the group level. Further, the low ICCs at T5 and T10 indicate that the time points expected to show maximal effects of cTBS (Wischnewski and Schutter, 2015) do not necessarily exhibit high test–retest reliability. This remained true for the cumulative measures of cTBS aftereffects over the first 20 min post-cTBS.

The very low test–retest reliability of ΔMEPs at T10 (ICC = 0.11) could be due to two factors: (1) The ΔMEPs at T10 in both visits could be the most influenced by BDNF polymorphism (Jannati et al., 2017). Consistent facilitation of MEPs, at least numerically, in BDNF Met+ participants in both visits may have resulted in higher test–retest reliability of T10 ΔMEP in that subgroup (Figure 5). (2) Despite the relatively long inter-visit interval in the present study, T10 seemed to exhibit a metaplastic-like effect similar to those reported with shorter intervals (Maeda et al., 2000; Gentner et al., 2008; Valero-Cabré et al., 2008; Oberman et al., 2016) in the overall results (Figure 1), as well in the Younger group (Figure 2A) and the BDNF Met– subgroup (Figure 4A). In all three cases, the direction of the neuromodulatory effect of cTBS at T10 was reversed, at least numerically, from inhibitory in visit-A to facilitatory in visit-B. Such reversals, when predominant at the individual level, would substantially reduce the test–retest reliability of cTBS aftereffects at T10. Although a previous iTBS study found only inter-visit intervals shorter than 7 days to be associated with metaplastic changes after iTBS in aging adults (Fried et al., 2017), the initial cTBS in the present study may have set into motion subtle changes that were still present when the second cTBS was applied. Further, it is possible that demographic, genetic, and state-dependent factors modulate the metaplastic(-like) effects of successive TBS sessions (Opie et al., 2017).

FIGURE 4

The low test–retest reliability of several cTBS aftereffects resulted in adjustment of large- and medium effect sizes to medium and small effect sizes, respectively (Table 3). While calculating the cumulative cTBS measures improved the overall ICCs over T0–T30 and beyond (Figure 3, 5), the ICCs of the cumulative measures for the whole sample remained below 0.5. Attenuation of idealized effect sizes by this level of reproducibility indicates that detecting significant differences in cTBS responses between healthy and clinical populations may require sample sizes that are substantially larger than those used in most previous cTBS studies (Wischnewski and Schutter, 2015; Chung et al., 2016; Suppa et al., 2016), unless steps can be implemented to improve the reliability of this technique.

FIGURE 5

The finding that within White, non-Hispanic, and right-handed males (n = 11), none of the ΔMEPs at any time point in either visit differed significantly from zero suggests that controlling for demographic factors such as gender, race/ethnicity, and handedness is not enough to overcome the large inter-individual variability in cTBS responses in either visit. We found, despite comparable reliability of baseline neurophysiological measures, T5 and T20–T50 ΔMEPs were substantially more reliable in this more-homogenous subgroup, suggesting that heterogeneity of these demographic factors influence the reliability of cTBS aftereffects in the whole sample. While the small sample precludes definitive conclusions about the effects of gender, handedness, and race/ethnicity, these results nonetheless appear to suggest that demographic variation plays a role in the test–retest reliability of cTBS responses.

Age and Reliability of cTBS Aftereffects

The significant Age Group × Visit interaction effect on T10 ΔMEP indicates that cTBS aftereffects at T10 in the Younger group, but not in the Older group, switched from inhibitory in visit-A to facilitatory in visit-B (Figure 2). Such reversal could be due to metaplasticity or some other state-dependent factor. The distinct patterns of cTBS aftereffects at T10 among Younger and Older groups could be due to two factors: (i) Based on animal studies that have found an age-related reduction in the efficiency of gamma-aminobutyric acid- (GABA-) mediated inhibition (Milbrandt et al., 1994; Billard et al., 1995; McQuail et al., 2012), it is possible that older participants have less-efficient GABAergic synaptic transmission, which is presumed to be involved in cTBS-induced plasticity (Stagg et al., 2009; Trippe et al., 2009). The resulting reduced inhibitory effects of cTBS, potentially combined with cumulative facilitatory effects of successive single TMS pulses (Pellicciari et al., 2016), could have resulted in facilitation of MEPs in the Older group, at least in visit-A. (ii) The finding that the Younger group, but not the Older group, showed metaplastic-like changes at T10 could be due to age-related differences in the priming effect of TBS (Opie et al., 2017), i.e., a stronger priming effect of cTBS in visit-A among younger participants.

Similarly, the finding that despite comparable ICCs of baseline neurophysiological measures in the two age groups, cTBS aftereffects were substantially less reliable in older participants (Figure 3) could be due to the age-related decrease in the efficiency of GABAergic synaptic transmission reported in animal studies (Milbrandt et al., 1994; Billard et al., 1995; McQuail et al., 2012). These results indicate that in order to retain adequate power to detect differences in cTBS measures of plasticity in future studies, it may be necessary to adjust effect sizes separately for younger and older age groups.

BDNF and APOE Polymorphisms and Reliability of cTBS Aftereffects

Despite comparable ICC values of RMT, AMT, and baseline MEP amplitude in BDNF Met– and Met+ groups, cTBS aftereffects at several time points and most cumulative measures of cTBS aftereffects were substantially more reliable in BDNF Met– participants than in Met+ participants (Figure 5). This pattern of results could be due to the following: BDNF Met carrier status is known to be associated with impaired N-Methyl-D-aspartic acid- (NMDA-)dependent LTD (Woo et al., 2005), aberrant GABAergic synaptic transmission (Abidin et al., 2008), reduced cTBS-induced inhibition of MEPs (Chung et al., 2016), and “paradoxical” cTBS-induced facilitation of MEPs in visit-A reported in our previous study (Jannati et al., 2017) and a few other studies (Gentner et al., 2008; Goldsworthy et al., 2012; Hellriegel et al., 2012; Brownjohn et al., 2014). The finding that BDNF Met+ participants showed MEP facilitation, at least numerically, at T10 in both visits (Figure 4B) supports an association between BDNF Met+ status and facilitatory response to cTBS at T10 in the present sample. The noticeably lower test–retest reliability of cTBS aftereffects in BDNF Met+ participants (Figure 5) could be due to the less-efficient cTBS-induced plasticity caused by aberrant GABAergic inhibition (Abidin et al., 2008), assumed to be involved in the LTD-like effects of cTBS (Stagg et al., 2009; Trippe et al., 2009).

Despite comparable ICC values of RMT, AMT, and baseline MEP amplitude in APOE ε4– and ε4+ participants, cTBS aftereffects at several time points and most cumulative measures of cTBS aftereffects were substantially more reliable in APOE ε4– participants than in APOE ε4+ participants (Figure 5). These results could be due to the influence of APOE ε4 on NMDA-mediated synaptic plasticity, which has been found to be involved in TBS aftereffects (Huang et al., 2007; Chen et al., 2010). These results are also consistent with the less-efficient rTMS-induced activation of brain networks in APOE ε4 carriers (Peña-Gomez et al., 2012).

The small number of participants in the BDNF and APOE subgroups study limits the generalizability of the present findings on genetic influences on the test–retest reliability of cTBS measures. Assuming that the noticeable differences in reliability of cTBS aftereffects in BDNF and APOE subgroups observed here (Figure 5) are confirmed in future studies, it would be advantageous to consider the expected proportions of BDNF and APOE subgroups and adjust effect sizes for each SNP subgroup accordingly. For example, the minor allele frequencies of rs6265 (BDNF), rs429358 (APOE), and rs7412 (APOE) SNPs in the admixed American population in the 1000 Genomes Project (Auton et al., 2015) are 0.1527, 0.1037, and 0.0476, respectively. As long as that the SNP frequencies among participants do not significantly deviate from Hardy–Weinberg equilibrium proportions (Guo and Thompson, 1992; Wigginton et al., 2005), these frequencies provide good approximations to the frequencies of minor BDNF and APOE alleles in future cTBS studies.

In comparisons of test–retest reliability of cTBS measures between age or genetic subgroups, the finding that the subgroups in each case were comparable in gender, race/ethnicity, handedness, and the reliability of baseline neurophysiological measures indicates that differences in heterogeneity of demographic factors and baseline cortical excitability did not play a major role in the observed differences in test–retest reliability of cTBS aftereffects.

In addition to considering age, genetic polymorphisms, inter-visit interval (Fried et al., 2017), the time of day (Cohen et al., 2005), and the use of neuronavigation (Julkunen et al., 2009), other factors that could improve the test–retest reliability of TMS measures include: ensuring comparable blood glucose levels and caffeine intake before and during each visit (Specterman et al., 2005; Cerqueira et al., 2006; Badawy et al., 2013), comparable amount and quality of sleep the night before each visit (Civardi et al., 2001; Kreuzer et al., 2011), comparable intensity and duration of exercise before each visit (Samii et al., 1997; Lentz and Nielsen, 2002), comparable phase of the menstrual cycle across visits (Smith et al., 1999; Hattemer et al., 2007), the use of robotic arms such as the TMS-Robot (Axilum Robotics, Schiltigheim, France), which can reduce trial-to-trial variability of MEP amplitude (Foucher et al., 2012), comparable baseline MEP amplitude across plasticity visits (Fried et al., 2017), and implementing closed-loop systems that trigger TMS pulses timed to real-time, EEG-defined indices of brain states (Zrenner et al., 2016, 2018).

Conclusion

The present study assessed the test–retest reliability of cTBS aftereffects in healthy adults. cTBS aftereffects at most individual time points had low to moderate reliability. Cumulative cTBS measures over the first 30 min and beyond were relatively more reliable. Effect sizes adjusted for reliability of cTBS aftereffects are provided to help future studies retain adequate power for comparing M1 cTBS responses between healthy and clinical populations. Those calculations resulted in adjustment of several large and medium effect sizes to medium and small effect sizes, respectively, thereby substantially increasing the estimates of the required sample size to detect a significant difference in cTBS responses between healthy and clinical populations. Exploratory analyses found cTBS aftereffects were substantially more reliable in younger participants (age < 35 years) and those with BDNF Met– and APOE ε4– genotypes.

References

• 1

  AbidinI.EyselU. T.LessmannV.MittmannT. (2008). Impaired GABAergic inhibition in the visual cortex of brain-derived neurotrophic factor heterozygous knockout mice: GABA release is impaired in visual cortex of BDNF heterozygous KO mice.J. Physiol.5861885–1901. 10.1113/jphysiol.2007.148627

  • CrossRef
  • Google Scholar

• 2

  AntalA.ChaiebL.MoliadzeV.Monte-SilvaK.PoreiszC.ThirugnanasambandamN.et al (2010). Brain-derived neurotrophic factor (BDNF) gene polymorphisms shape cortical plasticity in humans.Brain Stimul.3230–237. 10.1016/j.brs.2009.12.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3

  AutonA.AbecasisG. R.AltshulerD. M.DurbinR. M.AbecasisG. R.BentleyD. R.et al (2015). A global reference for human genetic variation.Nature52668–74. 10.1038/nature15393

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4

  BadawyR. A. B.VogrinS. J.LaiA.CookM. J. (2013). Cortical excitability changes correlate with fluctuations in glucose levels in patients with epilepsy.Epilepsy Behav.27455–460. 10.1016/j.yebeh.2013.03.015

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5

  BarkerA. T.JalinousR.FreestonI. L. (1985). Non-invasive magnetic stimulation of human motor cortex.Lancet3251106–1107. 10.1016/S0140-6736(85)92413-4

  • CrossRef
  • Google Scholar

• 6

  BastaniA.JaberzadehS. (2012). A higher number of TMS-elicited mep from a combined hotspot improves intra- and inter-session reliability of the upper limb muscles in healthy individuals.PLoS One7:e47582. 10.1371/journal.pone.0047582

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7

  BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing.J. R. Stat. Soc. B57289–300. 10.1111/j.2517-6161.1995.tb02031.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8

  BenjaminiY.YekutieliD. (2001). The control of the false discovery rate in multiple testing under dependency.Ann. stat.291165–1188. 10.1186/1471-2105-9-114

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9

  BillardJ.LamourY.DutarP. (1995). Decreased monosynaptic GABAB-mediated inhibitory postsynaptic potentials in hippocampal CA1 pyramidal cells in the aged rat: pharmacological characterization and possible mechanisms.J. Neurophysiol.74539–546. 10.1152/jn.1995.74.2.539

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10

  BrownjohnP. W.ReynoldsJ. N. J.MathesonN.FoxJ.ShemmellJ. B. H. (2014). The Effects of individualized theta burst stimulation on the excitability of the human motor system.Brain Stimul.7260–268. 10.1016/j.brs.2013.12.007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11

  CantoneM.Di PinoG.CaponeF.PiomboM.ChiarelloD.CheeranB.et al (2014). The contribution of transcranial magnetic stimulation in the diagnosis and in the management of dementia.Clin. Neurophysiol.1251509–1532. 10.1016/j.clinph.2014.04.010

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12

  CarretteS.BoonP.DekeyserC.KloosterD. C. W.CarretteE.MeursA.et al (2016). Repetitive transcranial magnetic stimulation for the treatment of refractory epilepsy.Expert Rev. Neurother.161093–1110. 10.1080/14737175.2016.1197119

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  CarrollT. J.RiekS.CarsonR. G. (2001). Reliability of the input–output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation.J. Neurosci. Methods112193–202. 10.1016/S0165-0270(01)00468-X

  • CrossRef
  • Google Scholar

• 14

  CazzoliD.MuriR. M.SchumacherR.von ArxS.ChavesS.GutbrodK.et al (2012). Theta burst stimulation reduces disability during the activities of daily living in spatial neglect.Brain1353426–3439. 10.1093/brain/aws182

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  CerqueiraV.de MendonçaA.MinezA.DiasA. R.de CarvalhoM. (2006). Does caffeine modify corticomotor excitability?Neurophysiol. Clin.36219–226. 10.1016/j.neucli.2006.08.005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16

  ChangW. H.BangO. Y.ShinY.-I.LeeA.Pascual-LeoneA.KimY.-H. (2014). BDNF polymorphism and differential rTMS effects on motor recovery of stroke patients.Brain Stimul.7553–558. 10.1016/j.brs.2014.03.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17

  CheeranB. J.RitterC.RothwellJ. C.SiebnerH. R. (2009). Mapping genetic influences on the corticospinal motor system in humans.Neuroscience164156–163. 10.1016/j.neuroscience.2009.01.054

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18

  CheeranB. J.TalelliP.MoriF.KochG.SuppaA.EdwardsM.et al (2008). A common polymorphism in the brain-derived neurotrophic factor gene (BDNF) modulates human cortical plasticity and the response to rTMS: BNDF polymorphism modulates response to rTMS.J. Physiol.5865717–5725. 10.1113/jphysiol.2008.159905

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19

  ChenY.DurakoglugilM. S.XianX.HerzJ. (2010). ApoE4 reduces glutamate receptor function and synaptic plasticity by selectively impairing ApoE receptor recycling.Proc. Natl. Acad. Sci. U.S.A.10712011–12016. 10.1073/pnas.0914984107

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20

  ChristieA.FlingB.CrewsR. T.MulwitzL. A.KamenG. (2007). Reliability of motor-evoked potentials in the ADM muscle of older adults.J. Neurosci. Methods164320–324. 10.1016/j.jneumeth.2007.05.011

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21

  ChuangW.-L.HuangY.-Z.LuC.-S.ChenR.-S. (2014). Reduced cortical plasticity and GABAergic modulation in essential tremor.Mov. Disord.29501–507. 10.1002/mds.25809

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22

  ChungS. W.HillA. T.RogaschN. C.HoyK. E.FitzgeraldP. B. (2016). Use of theta-burst stimulation in changing excitability of motor cortex: a systematic review and meta-analysis.Neurosci. Biobehav. Rev.6343–64. 10.1016/j.neubiorev.2016.01.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23

  CivardiC.BoccagniC.VicentiniR.BolampertiL.TarlettiR.VarrasiC.et al (2001). Cortical excitability and sleep deprivation: a transcranial magnetic stimulation study.J. Neurol. Neurosurg. Psychiatry71809–812. 10.1136/jnnp.71.6.809

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  CohenD. A.Pascual-LeoneA.PressD. Z.RobertsonE. M. (2005). Off-line learning of motor skill memory: a double dissociation of goal and movement.Proc. Natl Acad. Sci. U.S.A.10218237–18241. 10.1073/pnas.0506072102

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25

  CohenJ. (1988). Statistical Power Analysis for the Behavioral Sciences.Hilsdale. NJ: Lawrence Earlbaum Associates.

  • Google Scholar

• 26

  CohenJ. (1992). A power primer.Psychol. Bull.112155–159.

  • Google Scholar

• 27

  CrumR. M.AnthonyJ. C.BassettS. S.FolsteinM. F. (1993). Population-based norms for the mini-mental state examination by age and educational level.JAMA2692386–2391. 10.1001/jama.1993.03500180078038

  • CrossRef
  • Google Scholar

• 28

  Davila-PérezP.JannatiA.FriedP. J.Cudeiro MazairaJ.Pascual-LeoneA. (2018). The effects of waveform and current direction on the efficacy and test–retest reliability of transcranial magnetic stimulation.Neuroscience39397–109. 10.1016/j.neuroscience.2018.09.044

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29

  Di LazzaroV.CaponeF.Di PinoG.PellegrinoG.FlorioL.ZolloL.et al (2016). Combining robotic training and non-invasive brain stimulation in severe upper limb-impaired chronic stroke patients.Front. Neurosci.10:88. 10.3389/fnins.2016.00088

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30

  Di LazzaroV.PellegrinoG.Di PinoG.CorbettoM.RanieriF.BrunelliN.et al (2015). Val66Met BDNF gene polymorphism influences human motor cortex plasticity in acute stroke.Brain Stimul.892–96. 10.1016/j.brs.2014.08.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31

  Di LazzaroV.RothwellJ. C.TalelliP.CaponeF.RanieriF.WallaceA. C.et al (2013). Inhibitory theta burst stimulation of affected hemisphere in chronic stroke: a proof of principle, sham-controlled study.Neurosci. Lett.553148–152. 10.1016/j.neulet.2013.08.013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32

  DNA Genotek Inc. (2015). Laboratory Protocol for Manual Purification of DNA from 0.5 mL of Sample. Available at: http://www.dnagenotek.com/US/pdf/PD-PR-006.pdf (accessed January 12, 2016).

  • Google Scholar

• 33

  EberleM.-C.WildgruberD.WasserkaB.FallgatterA. J.PlewniaC. (2010). Relief from chronic intractable auditory hallucinations after long-term bilateral theta burst stimulation.Am. J. Psychiatry1671410–1410. 10.1176/appi.ajp.2010.10070988

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  FlemingM. K.SorinolaI. O.NewhamD. J.Roberts-LewisS. F.BergmannJ. H. M. (2012). The effect of coil type and navigation on the reliability of transcranial magnetic stimulation.IEEE Trans. Neural Syst. Rehabil. Eng.20617–625. 10.1109/TNSRE.2012.2202692

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  FolsteinM. F.FolsteinS. E.McHughP. R. (1975). Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician.J. Psychiatr. Res.12189–198.

  • Google Scholar

• 36

  ForoghB.Yazdi-BahriS.-M.AhadiT.FereshtehnejadS.-M.RaissiG. R. (2014). Comparison of two protocols of transcranial magnetic stimulation for treatment of chronic tinnitus: a randomized controlled clinical trial of burst repetitive versus high-frequency repetitive transcranial magnetic stimulation.Neurol. Sci.35227–232. 10.1007/s10072-013-1487-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37

  FoucherJ.LorgouillouxK.TurekJ.PhamB.-T.EloweJ.BayleB.et al (2012). “Robotic assistance in coil positioning improves reliability and comfort,” in Proceedings of the 3rd Annual Conference of the German Society for Brain Stimulation - Modulating Emotions, (Berlin).

  • Google Scholar

• 38

  FoxM. D.HalkoM. A.EldaiefM. C.Pascual-LeoneA. (2012). Measuring and manipulating brain connectivity with resting state functional connectivity magnetic resonance imaging (fcMRI) and transcranial magnetic stimulation (TMS).NeuroImage622232–2243. 10.1016/j.neuroimage.2012.03.035

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  FreitasC.Mondragón-LlorcaH.Pascual-LeoneA. (2011). Noninvasive brain stimulation in Alzheimer’s disease: systematic review and perspectives for the future.Exp. Gerontol.46611–627. 10.1016/j.exger.2011.04.001

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40

  FriedP. J.JannatiA.Davila-PérezP.Pascual-LeoneA. (2017). Reproducibility of single-pulse, paired-pulse, and intermittent theta-burst tms measures in healthy aging, type-2 diabetes, and alzheimer’s disease.Front. Aging Neurosci.9:263. 10.3389/fnagi.2017.00263

  • CrossRef
  • Google Scholar

• 41

  FriedP. J.SchilbergL.BremA.-K.SaxenaS.WongB.CypessA. M.et al (2016). Humans with type-2 diabetes show abnormal long-term potentiation-like cortical plasticity associated with verbal learning deficits.J. Alzheimer’s Dis.5589–100. 10.3233/JAD-160505

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42

  FriedmanH. (1968). Magnitude of experimental effect and a table for its rapid estimation.Psychol. Bull.70245–251. 10.1037/h0026258

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43

  GentnerR.WankerlK.ReinsbergerC.ZellerD.ClassenJ. (2008). Depression of human corticospinal excitability induced by magnetic theta-burst stimulation: evidence of rapid polarity-reversing metaplasticity.Cereb. Cortex182046–2053. 10.1093/cercor/bhm239

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44

  GoldsworthyM. R.Müller-DahlhausF.RiddingM. C.ZiemannU. (2014). Inter-subject variability of LTD-like plasticity in human motor cortex: a matter of preceding motor activation.Brain Stimul.7864–870. 10.1016/j.brs.2014.08.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45

  GoldsworthyM. R.PitcherJ. B.RiddingM. C. (2012). The application of spaced theta burst protocols induces long-lasting neuroplastic changes in the human motor cortex.Eur. J. Neurosci.35125–134. 10.1111/j.1460-9568.2011.07924.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  GuerraA.López-AlonsoV.CheeranB.SuppaA. (2017). Variability in non-invasive brain stimulation studies: reasons and results.Neurosci. Lett.10.1016/j.neulet.2017.12.058 [Epub ahead of print],

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47

  GuoS. W.ThompsonE. A. (1992). Performing the exact test of Hardy-Weinberg proportion for multiple alleles.Biometrics48361–372. 10.2307/2532296

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48

  HallettM. (2007). Transcranial magnetic stimulation: a primer.Neuron55187–199. 10.1016/j.neuron.2007.06.026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49

  HamadaM.MuraseN.HasanA.BalaratnamM.RothwellJ. C. (2013). The role of interneuron networks in driving human motor cortical plasticity.Cereb. Cortex231593–1605. 10.1093/cercor/bhs147

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 50

  HarrellF. E.Jr. (2015). Regression Modeling Strategies: with Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis.Berlin: Springer.

  • Google Scholar

• 51

  HattemerK.KnakeS.ReisJ.RochonJ.OertelW. H.RosenowF.et al (2007). Excitability of the motor cortex during ovulatory and anovulatory cycles: a transcranial magnetic stimulation study.Clin. Endocrinol.66387–393. 10.1111/j.1365-2265.2007.02744.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52

  HeideggerT.Hansen-GoosO.BatlaevaO.AnnakO.ZiemannU.LötschJ. (2017). A data-driven approach to responder subgroup identification after paired continuous theta burst stimulation.Front. Hum. Neurosci.11:382. 10.3389/fnhum.2017.00382

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 53

  HellriegelH.SchulzE. M.SiebnerH. R.DeuschlG.RaethjenJ. H. (2012). Continuous theta-burst stimulation of the primary motor cortex in essential tremor.Clin. Neurophysiol.1231010–1015. 10.1016/j.clinph.2011.08.033

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 54

  HermsenA.HaagA.DuddekC.BalkenholK.BugielH.BauerS.et al (2016). Test–retest reliability of single and paired pulse transcranial magnetic stimulation parameters in healthy subjects.J. Neurol. Sci.362209–216. 10.1016/j.jns.2016.01.039

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55

  HinderM. R.GossE. L.FujiyamaH.CantyA. J.GarryM. I. (2014). Inter-and intra-individual variability following intermittent theta burst stimulation: implications for rehabilitation and recovery.Brain Stimul.7365–371. 10.1016/j.brs.2014.01.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  HordacreB.GoldsworthyM. R.VallenceA.-M.DarvishiS.MoezziB.HamadaM.et al (2017). Variability in neural excitability and plasticity induction in the human cortex: a brain stimulation study.Brain Stimul.10588–595. 10.1016/j.brs.2016.12.001

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57

  HuangY.-Z.ChenR.-S.RothwellJ. C.WenH.-Y. (2007). The after-effect of human theta burst stimulation is NMDA receptor dependent.Clin. Neurophysiol.1181028–1032. 10.1016/j.clinph.2007.01.021

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58

  HuangY.-Z.EdwardsM. J.RounisE.BhatiaK. P.RothwellJ. C. (2005). Theta burst stimulation of the human motor cortex.Neuron45201–206. 10.1016/j.neuron.2004.12.033

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 59

  HunterJ. E.SchmidtF. L. (1994). “Correcting for sources of artificial variation across studies,” in The Handbook of Research Synthesis, edsCooperH.HedgeL. V. (New York, NY: Russell Sage Foundation), 323–336.

  • Google Scholar

• 60

  IezziE.ConteA.SuppaA.AgostinoR.DinapoliL.ScontriniA.et al (2008). Phasic voluntary movements reverse the aftereffects of subsequent theta-burst stimulation in humans.J. Neurophysiol.1002070–2076. 10.1152/jn.90521.2008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 61

  JannatiA.BlockG.ObermanL. M.RotenbergA.Pascual-LeoneA. (2017). Interindividual variability in response to continuous theta-burst stimulation in healthy adults.Clin. Neurophysiol.1282268–2278. 10.1016/j.clinph.2017.08.023

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 62

  JulkunenP.SäisänenL.DannerN.NiskanenE.HukkanenT.MervaalaE.et al (2009). Comparison of navigated and non-navigated transcranial magnetic stimulation for motor cortex mapping, motor threshold and motor evoked potentials.NeuroImage44790–795. 10.1016/j.neuroimage.2008.09.040

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 63

  KamenG. (2004). Reliability of motor-evoked potentials during resting and active contraction conditions.Med. Sci. Sports & Exercise361574–1579. 10.1249/01.mss.0000139804.02576.6a

  • CrossRef
  • Google Scholar

• 64

  KennyD. A.MannettiL.PierroA.LiviS.KashyD. A. (2002). The statistical analysis of data from small groups.J. Personal. Soc. Psychol.83126–137. 10.1037//0022-3514.83.1.126

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65

  KimiskidisV. K.PapagiannopoulosS.SotirakoglouK.KazisD. A.DimopoulosG.KazisA.et al (2004). The repeatability of corticomotor threshold measurements.Neurophysiol. Clin.34259–266. 10.1016/j.neucli.2004.10.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66

  KochG.BonniS.GiacobbeV.BucchiG.BasileB.LupoF.et al (2012). Theta-burst stimulation of the left hemisphere accelerates recovery of hemispatial neglect.Neurology7824–30. 10.1212/WNL.0b013e31823ed08f

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 67

  KochG.BrusaL.CarrilloF.Lo GerfoE.TorrieroS.OliveriM.et al (2009). Cerebellar magnetic stimulation decreases levodopa-induced dyskinesias in Parkinson disease.Neurology73113–119. 10.1212/WNL.0b013e3181ad5387

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 68

  KreuzerP.LangguthB.PoppR.RasterR.BuschV.FrankE.et al (2011). Reduced intra-cortical inhibition after sleep deprivation: a transcranial magnetic stimulation study.Neurosci. Lett.49363–66. 10.1016/j.neulet.2011.02.044

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 69

  LeeM.KimS. E.KimW. S.LeeJ.YooH. K.ParkK.-D.et al (2013). Interaction of motor training and intermittent theta burst stimulation in modulating motor cortical plasticity: influence of BDNF Val66Met polymorphism.PLoS One8:e57690. 10.1371/journal.pone.0057690

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 70

  LentzM.NielsenJ. F. (2002). Post-exercise facilitation and depression of M wave and motor evoked potentials in healthy subjects.Clin. Neurophysiol.1131092–1098. 10.1016/S1388-2457(02)00031-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 71

  LiC.-T.ChenM.-H.JuanC.-H.HuangH.-H.ChenL.-F.HsiehJ.-C.et al (2014). Efficacy of prefrontal theta-burst stimulation in refractory depression: a randomized sham-controlled study.Brain1372088–2098. 10.1093/brain/awu109

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72

  LiuH.Au-YeungS. S. Y. (2014). Reliability of transcranial magnetic stimulation induced corticomotor excitability measurements for a hand muscle in healthy and chronic stroke subjects.J. Neurol. Sci.341105–109. 10.1016/j.jns.2014.04.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73

  LivingstonS. C.IngersollC. D. (2008). Intra-rater reliability of a transcranial magnetic stimulation technique to obtain motor evoked potentials.Int. J. Neurosci.118239–256. 10.1080/00207450701668020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 74

  López-AlonsoV.CheeranB.Río-RodríguezD.Fernández-del-OlmoM. (2014). Inter-individual variability in response to non-invasive brain stimulation paradigms.Brain Stimul.7372–380. 10.1016/j.brs.2014.02.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 75

  MaedaF.KeenanJ. P.TormosJ. M.TopkaH.Pascual-LeoneA. (2000). Modulation of corticospinal excitability by repetitive transcranial magnetic stimulation.Clin. Neurophysiol.111800–805. 10.1016/s1388-2457%2899%2900323-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 76

  McClintockS. M.FreitasC.ObermanL. M.LisanbyS. H.Pascual-LeoneA. (2011). Transcranial magnetic stimulation: a neuroscientific probe of cortical function in schizophrenia.Biol. Psychiatry7019–27. 10.1016/j.biopsych.2011.02.031

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 77

  McDonnellM. N.RiddingM. C.MilesT. S. (2004). Do alternate methods of analysing motor evoked potentials give comparable results?J. Neurosci. Methods13663–67. 10.1016/j.jneumeth.2003.12.020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 78

  McGrawK. O.WongS. P. (1996). Forming inferences about some intraclass correlation coefficients.Psychol. Methods130–46. 10.1037/1082-989x.1.1.30

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 79

  McQuailJ. A.BañuelosC.LaSargeC. L.NicolleM. M.BizonJ. L. (2012). GABAB receptor GTP-binding is decreased in the prefrontal cortex but not the hippocampus of aged rats.Neurobiol. Aging331124.e1–1124.e12. 10.1016/j.neurobiolaging.2011.11.011

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 80

  MilbrandtJ. C.AlbinR. L.CasparyD. M. (1994). Age-related decrease in GABAB receptor binding in the Fischer 344 rat i inferior colliculus.Neurobiol. Aging15699–703.

  • Google Scholar

• 81

  MoriF.RossiS.PiccininS.MottaC.MangoD.KusayanagiH.et al (2013). Synaptic plasticity and PDGF signaling defects underlie clinical progression in multiple sclerosis.J. Neurosci.3319112–19119. 10.1523/JNEUROSCI.2536-13.2013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 82

  NettekovenC.VolzL. J.KutschaM.PoolE.-M.RehmeA. K.EickhoffS. B.et al (2014). Dose-dependent effects of theta burst rTMS on cortical excitability and resting-state connectivity of the human motor system.J. Neurosci.346849–6859. 10.1523/JNEUROSCI.4993-13.2014

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 83

  NettekovenC.VolzL. J.LeimbachM.PoolE.-M.RehmeA. K.EickhoffS. B.et al (2015). Inter-individual variability in cortical excitability and motor network connectivity following multiple blocks of rTMS.NeuroImage118209–218. 10.1016/j.neuroimage.2015.06.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 84

  NgomoS.LeonardG.MoffetH.MercierC. (2012). Comparison of transcranial magnetic stimulation measures obtained at rest and under active conditions and their reliability.J. Neurosci. Methods20565–71. 10.1016/j.jneumeth.2011.12.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 85

  NicholK.DeenyS. P.SeifJ.CamaclangK.CotmanC. W. (2009). Exercise improves cognition and hippocampal plasticity in APOE ε4 mice.Alzheimers Dementia5287–294. 10.1016/j.jalz.2009.02.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 86

  NIH Office of Extramural Research (2001). NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research. Available at: https://grants.nih.gov/grants/funding/women_min/guidelines_amended_10_2001.htm (accessed February 1, 2016).

  • Google Scholar

• 87

  ObermanL.Ifert-MillerF.NajibU.BashirS.WoollacottI.Gonzalez-HeydrichJ.et al (2010). Transcranial magnetic stimulation provides means to assess cortical plasticity and excitability in humans with fragile X syndrome and autism spectrum disorder.Front. Synaptic Neurosci.2:26. 10.3389/fnsyn.2010.00026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 88

  ObermanL. M.EldaiefM.FecteauS.Ifert-MillerF.TormosJ. M.Pascual-LeoneA. (2012). Abnormal modulation of corticospinal excitability in adults with Asperger’s syndrome: modulation of excitability in Asperger’s.Eur. J. Neurosci.362782–2788. 10.1111/j.1460-9568.2012.08172.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 89

  ObermanL. M.Ifert-MillerF.NajibU.BashirS.Gonzalez-HeydrichJ.PickerJ.et al (2016). Abnormal mechanisms of plasticity and metaplasticity in autism spectrum disorders and fragile X syndrome.J. Child Adoles. Psychopharmacol.26617–624. 10.1089/cap.2015.0166

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 90

  ObermanL. M.Pascual-LeoneA.RotenbergA. (2014). Modulation of corticospinal excitability by transcranial magnetic stimulation in children and adolescents with autism spectrum disorder.Front. Hum. Neurosci.8:627. 10.3389/fnhum.2014.00627

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 91

  OpieG. M.VosnakisE.RiddingM. C.ZiemannU.SemmlerJ. G. (2017). Priming theta burst stimulation enhances motor cortex plasticity in young but not old adults.Brain Stimul.10298–304. 10.1016/j.brs.2017.01.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 92

  Pascual-LeoneA.AmediA.FregniF.MerabetL. B. (2005). The plastic human brain cortex.Annu. Rev. Neurosci.28377–401. 10.1146/annurev.neuro.27.070203.144216

  • CrossRef
  • Google Scholar

• 93

  Pascual-LeoneA.FreitasC.ObermanL.HorvathJ. C.HalkoM.EldaiefM.et al (2011). Characterizing brain cortical plasticity and network dynamics across the age-span in health and disease with TMS-EEG and TMS-fMRI.Brain Topogr.24302–315. 10.1007/s10548-011-0196-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 94

  Pascual-LeoneA.Valls-SoléJ.WassermannE. M.HallettM. (1994). Responses to rapid-rate transcranial magnetic stimulation of the human motor cortex.Brain117847–858. 10.1093/brain/117.4.847

  • CrossRef
  • Google Scholar

• 95

  PellicciariM. C.MiniussiC.FerrariC.KochG.BortolettoM. (2016). Ongoing cumulative effects of single TMS pulses on corticospinal excitability: an intra- and inter-block investigation.Clin. Neurophysiol.127621–628. 10.1016/j.clinph.2015.03.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 96

  Peña-GomezC.Solé-PadullésC.ClementeI. C.JunquéC.BargallóN.BoschB.et al (2012). APOE status modulates the changes in network connectivity induced by brain stimulation in non-demented elders.PLoS One7:e51833. 10.1371/journal.pone.0051833

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 97

  PortneyL. G.WatkinsM. P. (2009). Foundations of Clinical Research: Applications to Practice, 3rd Edn. Upper Saddle River, NJ: Prentice Hall.

  • Google Scholar

• 98

  RossiS.HallettM.RossiniP. M.Pascual-LeoneA. (2009). Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research.Clin. Neurophysiol.1202008–2039. 10.1016/j.clinph.2009.08.016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 99

  RossiniP. M.BurkeD.ChenR.CohenL. G.DaskalakisZ.Di IorioR.et al (2015). Non-invasive electrical and magnetic stimulation of the brain, spinal cord, roots and peripheral nerves: basic principles and procedures for routine clinical and research application. An updated report from an I.F.C.N. Committee.Clin. Neurophysiol.1261071–1107. 10.1016/j.clinph.2015.02.001

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 100

  SamiiA.WassermannE. M.HallettM. (1997). Post-exercise depression of motor evoked potentials as a function of exercise duration.Electroencephalogr. Clin. Neurophysiol.105352–356. 10.1016/S0924-980X(97)00033-7

  • CrossRef
  • Google Scholar

• 101

  SankarasubramanianV.RoelleS. M.BonnettC. E.JaniniD.VarnerinN. M.CunninghamD. A.et al (2015). Reproducibility of transcranial magnetic stimulation metrics in the study of proximal upper limb muscles.J. Electromyogr. Kinesiol.25754–764. 10.1016/j.jelekin.2015.05.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 102

  SchambraH. M.OgdenR. T.Martínez-HernándezI. E.LinX.ChangY. B.RahmanA.et al (2015). The reliability of repeated TMS measures in older adults and in patients with subacute and chronic stroke.Front. Cell Neurosci.9:335. 10.3389/fncel.2015.00335

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 103

  SchilbergL.SchuhmannT.SackA. T. (2017). Interindividual variability and intraindividual reliability of intermittent theta burst stimulation-induced neuroplasticity mechanisms in the healthy brain.J. Cogn. Neurosci.291022–1032. 10.1162/jocn_a_01100

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 104

  SmithM. J.KeelJ. C.GreenbergB. D.AdamsL. F.SchmidtP. J.RubinowD. A.et al (1999). Menstrual cycle effects on cortical excitability.Neurology532069–2069. 10.1212/wnl.53.9.2069

  • CrossRef
  • Google Scholar

• 105

  SpectermanM.BhuiyaA.KuppuswamyA.StruttonP.CatleyM.DaveyN. (2005). The effect of an energy drink containing glucose and caffeine on human corticospinal excitability.Physiol. Behav.83723–728. 10.1016/j.physbeh.2004.09.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 106

  StaggC. J.WylezinskaM.MatthewsP. M.Johansen-BergH.JezzardP.RothwellJ. C.et al (2009). Neurochemical effects of theta burst stimulation as assessed by magnetic resonance spectroscopy.J. Neurophysiol.1012872–2877. 10.1152/jn.91060.2008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 107

  SuppaA.HuangY.-Z.FunkeK.RiddingM. C.CheeranB.Di LazzaroV.et al (2016). Ten years of theta burst stimulation in humans: established knowledge, unknowns and prospects.Brain Stimul.9323–335. 10.1016/j.brs.2016.01.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 108

  SuppaA.MarsiliL.Di StasioF.BerardelliI.RoselliV.PasquiniM.et al (2014). Cortical and brainstem plasticity in Tourette syndrome and obsessive-compulsive disorder.Mov. Disord.291523–1531. 10.1002/mds.25960

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 109

  ThorndikeR. L.HagenE. P.SattlerJ. M. (1986). The Stanford-Binet Intelligence Scale: Guide for Administering and Scoring.Rolling Meadows, IL: Riverside Publishing Company.

  • Google Scholar

• 110

  TrippeJ.MixA.Aydin-AbidinS.FunkeK.BenaliA. (2009). Theta burst and conventional low-frequency rTMS differentially affect GABAergic neurotransmission in the rat cortex.Exp. Brain Res.199411–421. 10.1007/s00221-009-1961-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 111

  Valero-CabréA.AmengualJ. L.StengelC.Pascual-LeoneA.CoubardO. A. (2017). Transcranial magnetic stimulation in basic and clinical neuroscience: a comprehensive review of fundamental principles and novel insights.Neurosci. Biobehav. Rev.83381–404. 10.1016/j.neubiorev.2017.10.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 112

  Valero-CabréA.Pascual-LeoneA.RushmoreR. J. (2008). Cumulative sessions of repetitive transcranial magnetic stimulation (rTMS) build up facilitation to subsequent TMS-mediated behavioural disruptions.Eur. J. Neurosci.27765–774. 10.1111/j.1460-9568.2008.06045.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 113

  VallenceA.-M.GoldsworthyM. R.HodylN. A.SemmlerJ. G.PitcherJ. B.RiddingM. C. (2015). Inter- and intra-subject variability of motor cortex plasticity following continuous theta-burst stimulation.Neuroscience304266–278. 10.1016/j.neuroscience.2015.07.043

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 114

  VernetM.BashirS.YooW.-K.ObermanL. M.MizrahiI.Ifert-MillerF.et al (2014). Reproducibility of the effects of theta burst stimulation on motor cortical plasticity in healthy participants.Clin. Neurophysiol.125320–326. 10.1016/j.clinph.2013.07.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 115

  WhiteF.NicollJ. A. R.RosesA. D.HorsburghK. (2001). Impaired neuronal plasticity in transgenic mice expressing human apolipoprotein E4 compared to E3 in a model of entorhinal cortex lesion.Neurobiol. Dis.8611–625. 10.1006/nbdi.2001.0401

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 116

  WiggintonJ. E.CutlerD. J.AbecasisG. R. (2005). A note on exact tests of hardy-weinberg equilibrium.Am. J. Hum. Genet.76887–893. 10.1086/429864

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 117

  WischnewskiM.SchutterD. J. L. G. (2015). Efficacy and time course of theta burst stimulation in healthy humans.Brain Stimul.8685–692. 10.1016/j.brs.2015.03.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 118

  WooN. H.TengH. K.SiaoC.-J.ChiaruttiniC.PangP. T.MilnerT. A.et al (2005). Activation of p75NTR by proBDNF facilitates hippocampal long-term depression.Nat. Neurosci.81069–1077. 10.1038/nn1510

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 119

  WrightK. (2014). “Adjusting effect sizes in light of reliability estimates,” in Paper Presented at the Annual Meeting of the Southwest Educational Research Association, (New Orleans, LA).

  • Google Scholar

• 120

  ZrennerC.BelardinelliP.Müller-DahlhausF.ZiemannU. (2016). Closed-loop neuroscience and non-invasive brain stimulation: a tale of two loops.Front. Cell. Neurosci.10:92. 10.3389/fncel.2016.00092

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 121

  ZrennerC.DesideriD.BelardinelliP.ZiemannU. (2018). Real-time EEG-defined excitability states determine efficacy of TMS-induced plasticity in human motor cortex.Brain Stimul.11374–389. 10.1016/j.brs.2017.11.016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar