Classification of Cognitive Impairment and Healthy Controls Based on Transcranial Magnetic Stimulation Evoked Potentials

1. Introduction

Cognitive impairment (CI) refers to a cognitive function decline beyond typical aging, which is increasingly prevalent in the elderly and loom as a public health issue (Montine et al., 2021).

Clinically, Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE) are commonly used for routine cognitive screening. MoCA is more sensitive than MMSE in detecting mild cognitive impairment (MCI) (Ciesielska et al., 2016). For Alzheimer's disease (AD) diagnosis, there are two main types of biomarkers: biophysiological biomarkers such as β amyloid in cerebrospinal fluid (CSF)/plasma/serum and brain imaging markers (Frisoni et al., 2017; Ng et al., 2019; Cullen et al., 2021). For example, Aβ42/Aβ40 ratio in CSF and blood (Buchhave et al., 2012; Hansson et al., 2018), Positron Emission Tomography (PET) of beta-amyloid and tau proteins (Leuzy et al., 2019; Rabinovici et al., 2019). However, the invasiveness of the collection of body fluids and high cost of PET limit their large-scale use. In recent years, some articles have reported that combining different neuroimaging modalities together can effectively detect CI. For example, combining functional MRI (fMRI) and Diffusion Tracking Imaging (DTI) can reflect functional connectivity changes in neuronal networks between CI and Healthy Controls (HC) (Ye and Bai, 2018). Multimodal fusion combines the advantages of each modality, but it is undeniable that complex data fusion algorithms impose huge challenges for clinical application.

Due to the above problems, some researchers have turned their attention to find a quick, noninvasive, and inexpensive method to detect CI, especially for mild and moderate patients without obvious behavioral symptoms. As a non-invasive, high time resolution method, the electroencephalogram (EEG) has been widely used in clinical examinations. In recent decades, the abnormalities in the resting state EEG of patients with CI have been discovered, such as a shift of the power spectrum to lower frequencies, a decrease in the coherence of fast rhythms, a decreased complexity of EEG patterns (Jeong, 2004).

With the development of non-invasive neuromodulation technology, the combination of Transcranial Magnetic Stimulation and Electroencephalogram (TMS-EEG) allows external input to specific cortical areas of subjects in a controlled and quantitative way for direct functional assessment (Hallett, 2007; Kimiskidis, 2016; Cao et al., 2021). When TMS pulses are applied to the cortex, trans-synaptic activation of local and distal cortical networks is obtained (Tremblay et al., 2019). The sum of synaptic potentials can be recorded simultaneously by high time resolution, multichannel scalp EEG. There are a series of positive and negative deflections after TMS, known as TMS Evoked Potentials (TEPs). The TEPs last 300 ms or more and can be recorded by either local or distal electrodes (Komssi et al., 2002), reflecting the spread of activation over cortical regions that are functionally connected and indicating the state of the brain further (Nikulin et al., 2003). Compared with resting-state EEG, TMS-EEG provides controlled stimulation without the involvement of the participation, which is more stable and objective.

At present, some researchers have used TMS-EEG to assess patients with CI. For example, the prefrontal TMS-evoked activity was able to track disease progression in Alzheimer's Disease (AD) and the P30 amplitude was predictive of the MMSE score in patients with AD (Bagattini et al., 2019). The Motor-Evoked Potentials (MEPs) produced by paired pulses on the primary cortex can be used as indicators in the classification of different subtypes of MCI (Benussi et al., 2021).

In this study, we hypothesized that TEPs resulting from stimulation of the left DLPFC may be associated with the cognitive status, thus, the features of TEPs could further differentiate CI and HC. On the premise of preserving time and space features simultaneously, we extracted some concise, interpretable linear features of TEPs in seven regions of interest (ROIs). We aimed to classify CI and HC automatically through machine learning based on the spatiotemporal features of TEPs and find potential biomarkers for clinical diagnosis.

2. Materials and Methods

The framework is shown in Figure 1. We removed artifacts of TMS-EEG at first. Then, we divided all the trials of each participant into three segments to the augment dataset. In order to preserve the features of time and space simultaneously, we focused on the TEP's N100 and P200 components in seven ROIs. Next, we explored some concise, interpretable linear features of these two components including Local Mean Field Power (LMFP), Latency, Amplitude, Standard Deviation, Average Value (AVG), Area Under the Curve (AUC), and Range. Finally, we used three common machine learning algorithms: K-Nearest Neighbor (KNN), Support Vector Machine (SVM), and Random Forest (RF) to obtain the label of each segment, and voted to get participant's final prediction result (Cover and Hart, 1967; Vapnik, 1999; Liaw and Wiener, 2002).

FIGURE 1

Figure 1. The framework of CI detection based on TEPs. First, the N100 and P200 components of TEPs were selected after removing artifacts. Then, all trials were divided into three segments. Subsequently, seven features were extracted from different segments and regions of interest (ROIs) respectively. Finally, machine learning was used to classify features, and voted on each segment to get the predicted result.

2.1. TMS-EEG Data Acquisition

2.1.1. Study Participants

All participants in this study were recruited from the Department of Neurology in Shenzhen People's Hospital (The First Affiliated Hospital, Southern University of Science and Technology). This study was approved by the Institutional Review Board of Shenzhen People's Hospital. All participants provided written informed consent.

A total of 43 subjects participated in this study, including 21 patients with CI and 22 HC. The inclusion criteria of CI group were: (a) Clinicians highly suspect the subject has AD based on the clinical diagnostic criteria (National Institute of Neurological and Communicative Disorders and stroke and the AD and Related Disorders Association (NINCDS-ADRDA), the Diagnostic and Statistical Manual of Mental Disorders-V (DSM-V) criteria); (b) Mild and moderate CI diagnosed (10 < MoCA <26) (Nasreddine et al., 2005); (c) aged 50–75 years old. The exclusion criteria were: (a) blood vessels and other types of dementia; (b) severe CI (MoCA <10); (c) a history of other psychiatric or other neurological disorders, such as schizophrenia, Parkinson's disease, and multiple sclerosis; (d) any contraindication for TMS, such as a metallic implanted device in or near the head and aneurysms.

All of the subjects in the control group met the following criteria: (a) aged 50–75 years old; (b) never complained of cognition or memory problems; (c) no history of any psychiatric or neurological disorders, brain injury, cranial neurosurgery, alcohol or drug abuse, or any severe chronic systemic illness; (d) no contraindication for TMS.

There was no significant difference in age between the two groups (p = 0.406). Demographic information was summarized in Table 1.

TABLE 1

Table 1. Demographic subjects.

2.1.2. TMS-EEG Recordings

The dorsolateral prefrontal cortex (DLPFC) is a key node of various cognitive functions such as memory, attention, and execution (Carlén, 2017). As we aimed to research cognition related function, the left DLPFC is also a recommended target for TMS treatment (Ahmed et al., 2012). Therefore, we chose left DLPFC to be the target of stimulation in this study.

All of the subjects in this study underwent a TMS-EEG protocol. A total of 100 single-pulse TMS pulses were delivered using the MagPro X100 with MagOption(MagVenture, Copenhagen, Denmark). The coil (figure-8 coil, Coil B65; external wing diameter, 90 mm) was placed over F3 (International 10/20 EEG system) to target the left DLPFC. The Inter-Stimulus Interval (ISI) was 3s jittered, and the stimulation intensity was 120% Resting Motor Threshold (RMT). The RMT is determined as the minimum stimulus intensity that produces a MEP exceeding 50 μV in a minimum of five out of ten trials in the relaxed right abductor pollicis brevis.

While receiving TMS, EEG signals of subjects were collected by BrainAmp DC amplifier (Brain Products, Munich, Germany) with a 64-channel EEG system. Participants were asked to remain still and relaxed during the EEG recording. The sampling rate was maintained at 5 kHz, and electrode impedances were maintained below 5 kΩ by applying the conductive gel. FCz was used as the reference while AFz was the ground during the EEG recording. All recordings took place in a temperature-controlled and electrically shielded room. Participants were asked to listen to white noise through earphones in order to mask the loud click accompanied by TMS coil discharge. A foam layer was placed under the coil to inhibit bone conduction and scalp sensation caused by the vibration of the coil (Rogasch et al., 2014).

2.2. TMS-EEG Data Preprocessing

2.2.1. Remove Artifacts

The TMS-EEG data in this study were preprocessed offline with TMS-EEG Signal Analyser (TESA) toolbox (Rogasch et al., 2017). TESA is an open source extension for EEGLAB (Delorme and Makeig, 2004), which is used for cleaning and analyzing TMS-EEG data. Both EEGLAB and TESA ran in Matlab (R2020b).

The data were divided into two-second epochs (−1,000 to 1,000 ms, the time of stimulation was marked as 0 s) and then baseline corrected (−500 to −50 ms). In order to remove the huge electromagnetic artifacts associated with TMS, the large amplitude TMS pulse artifact was removed between −5 and 15 ms and cubic interpolation was used to replace the removed data. For more efficient computing, the sampling rate of data was reduced from 5 to 1 kHz. Epochs and channels contaminated seriously were removed during visual inspection.

The first round of Independent Component Analysis (ICA) was performed to remove large value artifacts including TMS-evoked muscle artifacts and decay artifacts. Subsequently, the data were band-pass (1–80 Hz) and band-stop (48–52 Hz) filtered. It was followed by the second round of ICA to remove other relatively small value artifacts including auditory artifacts, blinks, eye movement, persistent scalp muscle activity, and electrode noise. Finally, the missing channels removed in the preprocessing were interpolated using spherical interpolation and all channels were re-referenced to the common average (Rogasch et al., 2014).

2.2.2. Time-Locked Averaging and GMFP

After data preprocessing, we got clean TMS-EEG trials (1s before stimulation, 1s after stimulation). TEPs were computed by averaging selected artifact-free single trial. The grand average TEPs of the two groups were shown in Figures 2A,B.

FIGURE 2

Figure 2. TMS Evoked Potentials (TEPs) components and ROIs selection. The grand average butterfly plot of all channels' TEPs of (A) HC group and (B) CI group. The electorde (F3) under the TMS coil is indicated in red. (C) The GMFP of two groups, the gray areas indicate two time windows of N100 (100–160 ms) and P200 (180–280 ms). (D) Schematic diagram of seven ROIs.

TMS Evoked Potentials can be recorded from the local electrode to the stimulation site, also from the electrode located in the distant cortical region. The Global Mean Field Power (GMFP) is usually calculated as a measure of global cortical excitability if ROIs are not specified. GMFP is the standard deviation (SD) of all channels at a given sampling point (Esser et al., 2006). The calculation formula was shown in Equation (1). The GMFP curve was shown in Figure 2C, which also indicated that the two groups may have differences in the two time windows of N100 and P200 (gray areas).

$\begin{array}{ll}GMFP(i)=\sqrt{\frac{{\displaystyle {\sum }_{j=1}^K}{\left(V_{\text{j}}(i)-V_{mean}\right)}^2}{K}}& (1)\end{array}$

where K is the number of all channels (K = 62 in this study), V_j(i) is defined as the voltage measured with channel j at sampling point i, and V_mean represents the average of the voltages across all channels.

2.2.3. Data Augmentation and Gaussian Smoothing

In order to improve the stability of subsequent machine learning, data augmentation technique was applied. We divided all the trials (the trials remaining after artifacts removing) of each subject into three segments (1–30, 31–60, and 61-end), and then averaged the trials in each segment, which meant that each subject had three segments available for training.

The TEPs obtained by a smaller number of trials are not as smooth as all the trials. In order to make the subsequent feature extraction more reliable, especially the Latency and Amplitude, we used Gaussian window method (the length of Gaussian window is set to 20) to smooth the data after averaging (Gwosdek et al., 2011).

2.2.4. TEPs' Time Windows and ROIs Selection

We found four typical characteristic peaks in Figures 2A,B which were consistent with previous research (N40, P60, N100, and P200) (Rosanova et al., 2012; Rogasch et al., 2015). In these typical peaks, N100 and P200 are widely regarded as the two most reliable and reproducible peaks (Kerwin et al., 2018). Considering individual variation led to the advancement or delay of the latency of characteristic peaks, we chose two relatively wide time windows (100–160 ms, 180–280 ms) to include peaks in the window to the full extent.

For the spatial features of TEPs, we selected 7 ROIs according to previous research (Kerwin et al., 2018). The seven ROIs are left Frontal (Fl, F1/F3/FC3/FC5), right Frontal (Fr, F2/F4/FC4/FC6), Central (C, Cz/C1/C2), Centroparietal (Cp, CPz/CP1/CP2/Pz/P1/P2), left Parietal (Pl, CP3/CP5 /P3/P5), right Parietal (Pr, CP4/CP6/P4/P6), and Occipital (O, Oz/O1/O2), as shown in Figure 2D.

2.3. Temporal-Spatial Features Extract

The average TEPs recorded by all channels in each ROI was calculated as the TEP of this ROI, as shown in Equation (2).

$\begin{array}{ll}X(i)=\frac{1}{k}{\displaystyle \sum _{j=1}^k}{V}_j(i)& (2)\end{array}$

where X(i) is TEP in the selected ROI, k is the number of channels in this ROI, V_j(i) is defined as the voltage measured with channel j at the sampling point i.

In order to describe the details of the two peaks (N100 and P200) as much as possible, we calculated a series of linear features in the selected time windows and regions. The features we extracted were introduced below:

• Latency and Amplitude. The Latency and Amplitude are the most common approaches for quantifying TEP (Tremblay et al., 2019). That is, the time and amplitude of the largest peak (negative or positive).

• Local Mean Field Power. The LMFP refers to SD across specific channels in the selected ROI (i.e., electrodes of interest, EOI). We calculated at every sampling point in the given time window and then averaged them. The LMFP reflects the dispersion degree of the signals recorded by the electrodes in this region indicating local excitability changes (Pellicciari et al., 2013), as shown in Equation (3).

$\begin{array}{ll}{f}_{LMFP}=\frac{1}{N}{\displaystyle \sum _{i=1}^N}\sqrt{\frac{{\sum }_{j=1}^k{\left(V_j(i)-V_{mean}\right)}^2}{k}}& (3)\end{array}$

where N is the number of all sampling points in the time window.

• Standard Deviation (STD). The STD is the standard deviation of the signal value in the selected time window, reflecting the degree of dispersion of the signal, as shown in Equation (4). This is an important time domain feature in EEG also called activity. The activity is quantified by means of the amplitude variance (Hjorth, 1970).

$\begin{array}{ll}{f}_{STD}=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N}{\left(X(i)-\frac{1}{N}{\displaystyle \sum _{i=1}^N}X(i)\right)}^2}& (4)\end{array}$

• Area Under Curve (AUC). The AUC is the area of the envelope between the signal and the time axis. The upper part of the time axis is positive and the lower part is negative. We used numerical integration to calculate the area, as shown in Equation (5). AUC was also called cortical-evoked activity (CEA) in previous research (Rajji et al., 2013).

$\begin{array}{ll}{f}_{AUC}=\frac{1}{2f}{\displaystyle \sum _{i=1}^{N-1}}\left(X(i)+X(i+1)\right)& (5)\end{array}$

where f is the sampling rate.

• Average Value. The AVG is the average signal value in the selected time window.

• Range. The Range is the difference between the maximum value and the minimum value of the signal in the selected time window, reflecting the fluctuation degree of the signal.

2.4. Machine Learning

In each ROI, we extracted 7 features of N100 and P200 respectively. Finally, 98 (2 time windows^*7 ROIs^*7 features) features were obtained in each segment of each subject.

Due to the possible correlation of different features, the t-Distributed Stochastic Neighbor Embedding (t-SNE) (van der Maaten and Hinton, 2008) was used to reduce the dimension. After features' dimension reduction, the feature array was normalized to [−1, 1].

In this study, three machine learning algorithms were used. SVM was implemented in the LIBSVM toolbox (Chang and Lin, 2011) with default parameters (linear kernal). Other classifiers [RF(n_tree = 7) and KNN(k = 5)] were also implemented in Matlab.

Since the features were divided into three segments, three labels that had the same weight were obtained for each participant after the classifier's prediction. We used the voting strategy to fusion three labels. The most pointed category was considered the final label of the subject.

In order to evaluate the performance of the classifiers and to simulate the reality of real CI recognition as much as possible, we adopted a leave-one-out cross-validation (LOOCV) strategy, keeping the minimum subject subset containing all the segment of a subject as the test set and employing all the others for training.

It is necessary to evaluate the classification effect of the model using appropriate indicators. For the binary classification problem, the test set can be divided into: True Positive (TP), False Positive (FP), False Negative (FN), and True Negative (TN). In this study, the subjects with CI were defined as positive samples, HC were defined as negative samples. The several evaluation indicators we used in this section are as follows:

$\begin{array}{ll}Accuracy=\frac{TP+TN}{TP+TN+FP+FN}& (6)\end{array}$

$\begin{array}{ll}Sensitivity=\frac{TP}{TP+FN}& (7)\end{array}$

$\begin{array}{ll}Specificity=\frac{TN}{TN+FP}& (8)\end{array}$

$\begin{array}{ll}\text{F1-score}=\frac{2TP}{2TP+FP+FN}& (9)\end{array}$

2.5. Statistics

2.5.1. Cluster-Based Permutation Test

Electroencephalogram data has both time and space structure (sampled in multiple channels and multiple time points). Therefore, the difference between CI and HC was the evaluation of a very large number of channel-time pairs, which was a multiple comparisons problem (MCP). For TEPs, we used cluster-based permutation statistics at the whole scalp level to take into account any combination of space and time, while controlling the MCP (Maris and Oostenveld, 2007). We performed an independent t-test for the two groups in the selected time windows (100–160 ms, 180–280 ms). If the test statistic value observed in at least two adjacent channels was lower than the threshold value of 0.05, then this value was considered in the cluster arrangement. We performed 5,000 iterations of trial randomization to generate permutation distributions and controlled multiple comparisons across spaces (P < 0.025, two-tailed test).

3. Results

3.1. Classification Results Based on Different Time Windows

According to the section 2.2.4, we extracted features in different components of TEPs (N100 and P200). Then, we explored the performance of the classifiers by using different components' features. To reduce the dimension of the features matrix, we used t-SNE to reduce the dimension of data.

Table 2 showed the classification results. All components mean merging the features of N100 and P200 components. The best classification performance was achieved by using the features of N100. The highest accuracy of 88.37% was achieved by SVM, with a specificity of 95.45%, the sensitivity of 80.95%, and the F1-score of 87.18%. The features of all components reached slightly weaker but still reasonable classification performance. The classification results of P200 were not satisfactory, the highest accuracy was 79.07%, and other classifiers had even worse results. The sensitivity, specificity, and F1-score were also lower than the N100 in different classifiers.

TABLE 2

Table 2. Classification results by all classifiers in different components.

3.2. Classification Results Based on Different ROIs

Regions of interest were divided according to the section 2.2.4 and we extracted features from each ROI to explore which ROIs are more sensitive to CI. Since there were only 14 features (2 time windows^*7 features) in each ROI, the features were directly put into the classifier after normalization without dimensionality reduction.

Table 3 showed the classification results. It could be concluded that different brain regions had great influence on the classification results, and the Fr and Pl region showed the best performance, which achieved 83.72% by SVM. Moreover, when RF was used, the features of Fr were sensitive to the distinction of positive samples, which meant that the patients with CI could be well recognized and the probability of the patients with CI being diagnosed as normal was reduced. The features of the Fl region showed slightly weaker but still reasonable classification performances. The features of the C region can distinguish negative samples well, but it was not good enough to distinguish patients with CI. The features of the Cp region, O region, and Pr region were basically unable to complete the task of distinguishing normal people from patients with CI.

TABLE 3

Table 3. Classification results by all classifiers in different regions.

3.3. Statistical Results

We first performed a cluster-based permutation test on the TEPs of the two groups. The results of statistical analysis were shown in Figure 3. The topographic map was generated with Yang's topoplot_bcl function based on EEGLAB's topoplot function (Li et al., 2018). The asterisk indicates that the p-value is less than 0.01. In the N100 time window (100–160 ms), right frontal region, left parietal region, and occipital region were significantly different between the two groups (p < 0.01). In the P200 time window (180–280 ms), bilateral frontal region, bilateral parietal region, and occipital region were significantly different between the two groups (p < 0.01).

FIGURE 3

Figure 3. Comparison of TEPs using cluster-based permutation tests. Red means TEP of cognitive impairment (CI) is higher than HC, blue means TEP of CI is lower than healthy controls (HC). The asterisk indicates that p < 0.01. (A) N100: CI vs. HC. (B) P200: CI vs. HC.

Furthermore, we used the violin plot to describe the distribution of 14 features of the right frontal region, as shown in Figure 4. We also performed t-test on the features in Figure 4, the results were shown in the Table 4. All the features of N100 in the right frontal region were significantly different between the two groups (p < 0.01). There was no difference in the latency of P200 in the right frontal region between the two groups. This may explain why the features of N100 performed better than P200 on classification, and even better than using them simultaneously.

FIGURE 4

Figure 4. Distribution of spatial features in right frontal region. (A) N100: LMFP, (B) N100: STD, (C) N100: Latency, (D) N100: Amplitude, (E) N100: AVG, (F) N100: AUC, (G) N100: Range, (H) P200: LMFP, (I) P200: STD, (J) P200: Latency, (K) P200: Amplitude, (L) P200: AVG, (M) P200: AUC, and (N) P200: Range.

TABLE 4

Table 4. Local features in right frontal region.

Then, we used the t-SNE to reduce the dimension of the best performing N100 features to 3, and then draw them in three-dimensional space. The result was shown in Figure 5A, red represented CI, and blue represented HC. It revealed that the points in CI were more dispersed than normal.

FIGURE 5

Figure 5. Feature visualization and importance comparison. (A) The visualization features map based on t-Distributed Stochastic Neighbor Embedding (t-SNE) dimension-reduction. (B) The feature importance based on XGBoost.

Finally, for the 7 features of N100 in the right frontal region, we used XGBoost to evaluate the importance of features (Chen and Guestrin, 2016). The feature importance ranking was listed in Figure 5B. The LMFP, AVG, Latency, and Amplitude were the most important, which exactly were consistent with the most widely used features to measure TEPs in previous studies (Tremblay et al., 2019). Our results verified the conclusions of the predecessors and also instructed doctors to pay more attention to these features in future clinical practice.

4. Discussion

The study showed that machine learning can effectively identify CI. In the time domain, the features of N100 had the best performance. In the aspect of spatiality, the features of the right frontal region and left parietal region had the best performance. Then, we discussed our results from these two dimensions, and particularly emphasized the influence of sensory contamination within the TEP. Finally, we discussed the limitations of our study.

Previous studies have used TMS to stimulate the motor cortex and have demonstrated that patients with AD have increased cortical excitability, which is manifested by a decrease in RMT and an increase in MEP (Alagona et al., 2001; Di Lazzaro et al., 2004). There is also evidence of high excitability in patients with AD after TMS stimulating the left DLPFC, which is manifested as an increase in CEA, and this increase is negatively correlated with overall cognitive and executive functions (Joseph et al., 2021). In our study, the CI group had higher GMFP, suggesting higher cortical excitability which is consistent with previous studies. This hyper-excitability may reflect the plastic reorganization of the sensorimotor system and may be used as a compensatory mechanism to offset the loss of cortical volume and protect the motor function of the patient (Niskanen et al., 2011; Bagattini et al., 2019).

It was found in a previous study that the reliability of the potentials induced by TMS in the left DLPFC are highly consistent and the measurement error is small. The most reliable peaks are generally located at 100 and 200 ms (Kerwin et al., 2018). High reproducibility is necessary as a potential neurobiomarker. Although there was some prior knowledge that CI may affect P30 (Bagattini et al., 2019), considering the first peak may be affected by preprocessing (–5–15 ms removed and cubic interpolation), we still focused on the N100 and P200 two components to preserve the temporal features.

In the time domain, the features of N100 have the best performance according to the four indicators in the classifier. In the CI group, the change of N100 and P200 may be related to the alterations of GABAergic activity. Gamma-aminobutyric acid (GABA) is an inhibitory neurotransmitter. Its natural function is binding to GABA-A receptors and GABA-B receptors on the neurons to modulate and block impulses between nerve cells (Gou et al., 2013). There is some evidence that the amplitude of the TEPs component is related to GABAergic activity. GABA-A receptors agonists (alprazolam and diazepam) and GABA-B receptors agonists (baclofen) can modulate the amplitude of N100 or P200 (Premoli et al., 2014; Murphy et al., 2016). There are also some studies stated that the alterations of GABAergic circuits may contribute to CI by disrupting the overall network function (Li et al., 2016). This may be the neuropathological basis for the difference of N100 and P200 between the two groups, which was the precondition for classification.

In the aspect of spatiality, the features of the right frontal region and left parietal region had the best performance in the SVM classifier. From the statistical results, the right prefrontal region and the left parietal region both had significant differences in these two selected time windows, which may explain why the features of these two ROIs perform better in the classifiers. Interestingly, the significant difference between CI and HC is reflected in the anterior and posterior regions, and the trend is opposite over time. This seems to indicate the abnormal changes in the connectivity of the brains of patients with CI. Previous studies have shown that the damaged excitatory-inhibitory balance between anterior and posterior regions might represent a maladaptive pathogenic mechanism (Bagattini et al., 2019).

In addition, we want to emphasize the issue of sensory contamination within the TEP. There is no doubt that the auditory complex can overlap with N100 and P200 (Conde et al., 2019), but in fact we have used strict online and offline methods to avoid the impact of auditory and somatosensory evoked potential as much as possible. In the data collection process, we played white noise by earphones for the subjects. In addition, a foam layer was placed under the coil to inhibit bone conduction and weaken the scalp sensation caused by the vibration of the coil. During data preprocessing, we paid attention to the removal of auditory artifacts in the second ICA run which were characterized by a topography centering over Cz to Fz and a time course with bipolar peaks at approximately 100–200 ms (Rogasch et al., 2014). From another perspective, all subjects received the same protocol and all the EEG were produced in the same environment and preprocessed by the same method. Therefore, even if the online and offline methods cannot completely remove the auditory and somatosensory evoked potentials, the remaining potentials have the same effect on both two groups, which does not contribute to the difference between groups, let alone the impact on classification. In summary, there is no reason to think that the difference in the selected time windows is related to sensory contamination.

In this study, we observed that all classification results had high specificity but unsatisfied sensitivity, meaning that CI subjects were not well distinguished from HC. It is due to the heterogeneity of cognitive related disease, time of illness, and disease progression may lead to more scattered features. There was a limitation that all patients with CI enrolled were highly suspected of AD with mild to moderate symptoms based on clinical diagnosis and MoCA, excluded vascular and other types of dementia, but not on biological markers of CSF or PET. The definition of AD is controversial throughout articles (Jellinger, 2020; Milà-Alomà et al., 2021). The focus of this study was the extraction method of TEPs' features and machine learning rather than strict AD diagnostic criteria. To be conservative and rigorous, although all subjects in the CI group were highly suspected of AD, we did not define them as AD but summarized it with CI in this study.

A further limitation is that the sample size in this study was small in the field of machine learning. Although we have used data augmentation and voting strategy to obtain a more robust model, more data will still be needed in subsequent studies to meet the real and complex clinical needs.

5. Conclusion

We found that the TEPs induced by TMS over the left DLPFC has significant differences between CI and HC. Machine learning based on the spatiotemporal features of TEPs is effective for the classification of CI and HC.

In the time domain, the features of N100 had the best performance in the SVM classifier, with an accuracy of 88.37%. In the aspect of spatiality, the features of the right frontal region and left parietal region had the best performance in the SVM classifier, with an accuracy of 83.72%. By using XGBoost to evaluate the importance of features, the LMFP, AVG, Latency, and Amplitude contributed the most in classification. It is suggested that clinicians should pay close attention to the important features above, which may be potential biomarkers for diagnosing CI.

In this study, the features selected were all simple and linear, the classification algorithms used were popular and sophisticated. Therefore, our research particularly emphasized the interpretability and clinical usability. These findings prove that machine learning based on spatiotemporal features of TEP has the potential to automatically clinical auxiliary diagnosis of CI.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of Shenzhen People's Hospital. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

JZ: writing—original draft preparation, TMS-EEG data artifacts removing, features extraction, and statistical analysis. HL: writing—original draft preparation, machine learning, and feature analysis. LZ: writing—reviewing, conceptualization, and methodology. HR: data collection. XSu and XL: TMS equipment operation and EEG acquisition. GD, XZ, XJ, JF, and XSh: investigation, writing—reviewing, and editing. TW and XH: writing—reviewing and editing, investigation, methodology, and supervision. YG: writing—reviewing and editing, clinical diagnosis, investigation, and methodology. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by Shenzhen Science and Technology Innovation Commission (KCXFZ20201221173400001 and KCXFZ20201221173411032), Natural Science Fund of Guangdong Province (2021A1515010983). Shenzhen Key Medical Discipline Construction Fund (No. SZXK005), and Special Funds for the Cultivation of Guangdong College Students' Scientific and Technological Innovation (Climbing Program Special Funds, pdjh2021c0018).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ahmed, M. A., Darwish, E. S., Khedr, E. M., Ali El Serogy, Y. M., and Ali, A. M. (2012). Effects of low versus high frequencies of repetitive transcranial magnetic stimulation on cognitive function and cortical excitability in alzheimer's dementia. J. Neurol. 259, 83–92. doi: 10.1007/s00415-011-6128-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Alagona, G., Bella, R., Ferri, R., Carnemolla, A., Pappalardo, A., Costanzo, E., et al. (2001). Transcranial magnetic stimulation in alzheimer disease: motor cortex excitability and cognitive severity. Neurosci. Lett. 314, 57–60. doi: 10.1016/s0304-3940(01)02288-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Bagattini, C., Mutanen, T. P., Fracassi, C., Manenti, R., Cotelli, M., Ilmoniemi, R. J., et al. (2019). Predicting alzheimer's disease severity by means of tms–eeg coregistration. Neurobiol. Aging 80, 38–45. doi: 10.1016/j.neurobiolaging.2019.04.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Benussi, A., Grassi, M., Palluzzi, F., Cantoni, V., Cotelli, M. S., Premi, E., et al. (2021). Classification accuracy of tms for the diagnosis of mild cognitive impairment. Brain Stimul. 14, 241–249. doi: 10.1016/j.brs.2021.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Buchhave, P., Minthon, L., Zetterberg, H., Wallin, Å. K., Blennow, K., Hansson, O., et al. (2012). Cerebrospinal fluid levels ofβ-amyloid 1-42, but not of tau, are fully changed already 5 to 10 years before the onset of alzheimer dementia. Arch. Gen. Psychiatry 69, 98–106. doi: 10.1001/archgenpsychiatry.2011.155

PubMed Abstract | CrossRef Full Text | Google Scholar

Cao, K.-X., Ma, M.-L., Wang, C.-Z., Iqbal, J., Si, J.-J., Xue, Y.-X., et al. (2021). Tms-eeg: an emerging tool to study the neurophysiologic biomarkers of psychiatric disorders. Neuropharmacology 197:108574. doi: 10.1016/j.neuropharm.2021.108574

PubMed Abstract | CrossRef Full Text | Google Scholar

Carlén, M.. (2017). What constitutes the prefrontal cortex? Science 358, 478–482. doi: 10.1126/science.aan8868

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, C., and Lin, C. (2011). Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–39. doi: 10.1145/1961189.1961199

CrossRef Full Text | Google Scholar

Chen, T., and Guestrin, C. (2016). Xgboost: a scalable tree boosting system, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD '16, (Association for Computing Machinery) (New York, NY), 785–794.

PubMed Abstract | Google Scholar

Ciesielska, N., Sokołowski, R., Mazur, E., Podhorecka, M., Polak-Szabela, A., Kdziora-Kornatowska, K., et al. (2016). Is the montreal cognitive assessment (moca) test better suited than the mini-mental state examination (mmse) in mild cognitive impairment (mci) detection among people aged over 60? meta-analysis. Psychiatr Pol. 50, 1039–1052. doi: 10.12740/PP/45368

PubMed Abstract | CrossRef Full Text | Google Scholar

Conde, V., Tomasevic, L., Akopian, I., Stanek, K., Saturnino, G. B., Thielscher, A., et al. (2019). The non-transcranial tms-evoked potential is an inherent source of ambiguity in tms-eeg studies. Neuroimage 185, 300–312. doi: 10.1016/j.neuroimage.2018.10.052

PubMed Abstract | CrossRef Full Text | Google Scholar

Cover, T., and Hart, P. (1967). Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13, 21–27. doi: 10.1109/TIT.1967.1053964

PubMed Abstract | CrossRef Full Text | Google Scholar

Cullen, N. C., Leuzy, A., Palmqvist, S., Janelidze, S., Stomrud, E., Pesini, P., et al. (2021). Individualized prognosis of cognitive decline and dementia in mild cognitive impairment based on plasma biomarker combinations. Nat. Aging 1, 114–123. doi: 10.1038/s43587-020-00003-5

CrossRef Full Text | Google Scholar

Delorme, A., and Makeig, S. (2004). Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21. doi: 10.1016/j.jneumeth.2003.10.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Lazzaro, V., Oliviero, A., Pilato, F., Saturno, E., Dileone, M., Marra, C., et al. (2004). Motor cortex hyperexcitability to transcranial magnetic stimulation in alzheimer's disease. J. Neurol. Neurosurgery Psychiatry 75, 555–559. doi: 10.1136/jnnp.2003.018127

PubMed Abstract | CrossRef Full Text | Google Scholar

Esser, S., Huber, R., Massimini, M., Peterson, M., Ferrarelli, F., Tononi, G., et al. (2006). A direct demonstration of cortical ltp in humans: a combined tms/eeg study. Brain Res. Bull. 69, 86–94. doi: 10.1016/j.brainresbull.2005.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Frisoni, G. B., Boccardi, M., Barkhof, F., Blennow, K., Cappa, S., Chiotis, K., et al. (2017). Strategic roadmap for an early diagnosis of alzheimer's disease based on biomarkers. Lancet Neurol. 16, 661–676. doi: 10.1016/S1474-4422(17)30159-X

PubMed Abstract | CrossRef Full Text | Google Scholar

Gou, Z., Wang, X., and Wang, W. (2013). Evolution of neurotransmitter gamma-aminobutyric acid, glutamate and their receptors. Dongwuxue Yanjiu 33, E75–E81. doi: 10.3724/SP.J.1141.2012.E05-06E75

PubMed Abstract | CrossRef Full Text | Google Scholar

Gwosdek, P., Grewenig, S., Bruhn, A., and Weickert, J. (2011). Theoretical Foundations of Gaussian Convolution by Extended Box Filtering, in Scale Space and Variational Methods in Computer Vision. SSVM 2011. Lecture Notes in Computer Science, vol 6667, eds A.M. Bruckstein, B.M. ter Haar Romeny, A. M. Bronstein, M. M. Bronstein (Berlin: Springer), 447–458.

Google Scholar

Hallett, M.. (2007). Transcranial magnetic stimulation: a primer. Neuron 55, 187–199. doi: 10.1016/j.neuron.2007.06.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Hansson, O., Seibyl, J., Stomrud, E., Zetterberg, H., Trojanowski, J. Q., Bittner, T., et al. (2018). Csf biomarkers of alzheimer's disease concord with amyloid-β pet and predict clinical progression: a study of fully automated immunoassays in biofinder and adni cohorts. Alzheimers Dementia 14, 1470–1481. doi: 10.1016/j.jalz.2018.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Hjorth, B.. (1970). Eeg analysis based on time domain properties. Electroencephalogr. Clin. Neurophysiol. 29, 306–310. doi: 10.1016/0013-4694(70)90143-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Jellinger, K.. (2020). Towards a biological definition of alzheimer disease. Int. J. Neurol Neurother. 7:95. doi: 10.23937/2378-3001/1410095

CrossRef Full Text | Google Scholar

Jeong, J.. (2004). Eeg dynamics in patients with alzheimer's disease. Clin. Neurophysiol. 115, 1490–1505. doi: 10.1016/j.clinph.2004.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Joseph, S., Knezevic, D., Zomorrodi, R., Blumberger, D. M., Daskalakis, Z. J., Mulsant, B. H., et al. (2021). Dorsolateral prefrontal cortex excitability abnormalities in alzheimer's dementia: findings from transcranial magnetic stimulation and electroencephalography study. Int. J. Psychophysiol. 169, 55–62. doi: 10.1016/j.ijpsycho.2021.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Kerwin, L. J., Keller, C. J., Wu, W., Narayan, M., and Etkin, A. (2018). Test-retest reliability of transcranial magnetic stimulation eeg evoked potentials. Brain stimul. 11, 536–544. doi: 10.1016/j.brs.2017.12.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Kimiskidis, V.. (2016). Transcranial magnetic stimulation (tms) coupled with electroencephalography (eeg): biomarker of the future. Rev. Neurol. 172, 123–126. doi: 10.1016/j.neurol.2015.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Komssi, S., Aronen, H. J., Huttunen, J., Kesäniemi, M., Soinne, L., Nikouline, V. V., et al. (2002). Ipsi- and contralateral eeg reactions to transcranial magnetic stimulation. Clin. Neurophysiol. 113, 175–184. doi: 10.1016/s1388-2457(01)00721-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Leuzy, A., Chiotis, K., Lemoine, L., Gillberg, P. G., Almkvist, O., Rodriguez-Vieitez, E., et al. (2019). Tau pet imaging in neurodegenerative tauopathies-still a challenge. Mol. Psychiatry 24, 1112–1134. doi: 10.1038/s41380-018-0342-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, A.-S., Miao, C.-G., Han, Y., He, X., and Zhang, Y. (2018). Electrophysiological correlates of the effect of task difficulty on inhibition of return. Front. Psychol. 9:2403. doi: 10.3389/fpsyg.2018.02403

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Y., Sun, H., Chen, Z., Xu, H., Bu, G., and Zheng, H. (2016). Implications of gabaergic neurotransmission in alzheimer's disease. Front. Aging Neurosci. 8:31. doi: 10.3389/fnagi.2016.00031

PubMed Abstract | CrossRef Full Text | Google Scholar

Liaw, A., and Wiener, M. (2002). Classification and regression by randomforest. R News 23. Available online at: https://cogns.northwestern.edu/cbmg/LiawAndWiener2002.pdf

Google Scholar

Maris, E., and Oostenveld, R. (2007). Nonparametric statistical testing of eeg-and meg-data. J. Neurosci. Methods 164, 177–190. doi: 10.1016/j.jneumeth.2007.03.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Milà-Alomà, M., Salvadó, G., Shekari, M., Grau-Rivera, O., Sala-Vila, A., Sánchez-Benavides, G., et al. (2021). Comparative analysis of different definitions of amyloid-β positivity to detect early downstream pathophysiological alterations in preclinical alzheimer. J. Prevent. Alzheimers Dis. 8, 68–77. doi: 10.14283/jpad.2020.51

PubMed Abstract | CrossRef Full Text | Google Scholar

Montine, T. J., Bukhari, S. A., and White, L. R. (2021). Cognitive impairment in older adults and therapeutic strategies. Pharmacol. Rev. 73, 152–162. doi: 10.1124/pharmrev.120.000031

PubMed Abstract | CrossRef Full Text | Google Scholar

Murphy, S. C., Palmer, L. M., Nyffeler, T., Müri, R. M., and Larkum, M. E. (2016). Transcranial magnetic stimulation (tms) inhibits cortical dendrites. Elife 5:e13598. doi: 10.7554/eLife.13598

PubMed Abstract | CrossRef Full Text | Google Scholar

Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al. (2005). The montreal cognitive assessment, moca: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 695–699. doi: 10.1111/j.1532-5415.2005.53221.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Ng, T. K. S., Ho, C. S. H., Tam, W. W. S., Kua, E. H., and Ho, R. C.-M. (2019). Decreased serum brain-derived neurotrophic factor (bdnf) levels in patients with alzheimer's disease (ad): a systematic review and meta-analysis. Int. J. Mol. Sci. 20:257. doi: 10.3390/ijms20020257

PubMed Abstract | CrossRef Full Text | Google Scholar

Nikulin, V. V., Kičić, D., Kähkönen, S., and Ilmoniemi, R. J. (2003). Modulation of electroencephalographic responses to transcranial magnetic stimulation: evidence for changes in cortical excitability related to movement. Eur. J. Neurosci. 18, 1206–1212. doi: 10.1046/j.1460-9568.2003.02858.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Niskanen, E., Könönen, M., Määttä, S., Hallikainen, M., Kivipelto, M., Casarotto, S., et al. (2011). New insights into alzheimer's disease progression: a combined tms and structural mri study. PLoS ONE 6:e26113. doi: 10.1371/journal.pone.0026113

PubMed Abstract | CrossRef Full Text | Google Scholar

Pellicciari, M. C., Brignani, D., and Miniussi, C. (2013). Excitability modulation of the motor system induced by transcranial direct current stimulation: a multimodal approach. Neuroimage 83, 569–580. doi: 10.1016/j.neuroimage.2013.06.076

PubMed Abstract | CrossRef Full Text | Google Scholar

Premoli, I., Castellanos, N., Rivolta, D., Belardinelli, P., Bajo, R., Zipser, C., et al. (2014). Tms-eeg signatures of gabaergic neurotransmission in the human cortex. J. Neurosci. 34, 5603–5612. doi: 10.1523/JNEUROSCI.5089-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Rabinovici, G. D., Gatsonis, C., Apgar, C., Chaudhary, K., Gareen, I., Hanna, L., et al. (2019). Association of amyloid positron emission tomography with subsequent change in clinical management among medicare beneficiaries with mild cognitive impairment or dementia. JAMA 321, 1286–1294. doi: 10.1001/jama.2019.2000

PubMed Abstract | CrossRef Full Text | Google Scholar

Rajji, T. K., Sun, Y., Zomorrodi-Moghaddam, R., Farzan, F., Blumberger, D. M., Mulsant, B. H., et al. (2013). Pas-induced potentiation of cortical-evoked activity in the dorsolateral prefrontal cortex. Neuropsychopharmacology 38, 2545–2552. doi: 10.1038/npp.2013.161

PubMed Abstract | CrossRef Full Text | Google Scholar

Rogasch, N. C., Daskalakis, Z. J., and Fitzgerald, P. B. (2015). Cortical inhibition of distinct mechanisms in the dorsolateral prefrontal cortex is related to working memory performance: A tms–eeg study. Cortex 64, 68–77. doi: 10.1016/j.cortex.2014.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Rogasch, N. C., Sullivan, C., Thomson, R. H., Rose, N. S., Bailey, N. W., Fitzgerald, P. B., et al. (2017). Analysing concurrent transcranial magnetic stimulation and electroencephalographic data: a review and introduction to the open-source tesa software. Neuroimage 147, 934–951. doi: 10.1016/j.neuroimage.2016.10.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Rogasch, N. C., Thomson, R. H., Farzan, F., Fitzgibbon, B. M., Bailey, N. W., Hernandez-Pavon, J. C., et al. (2014). Removing artefacts from tms-eeg recordings using independent component analysis: importance for assessing prefrontal and motor cortex network properties. Neuroimage 101, 425–39. doi: 10.1016/j.neuroimage.2014.07.037

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosanova, M., Casarotto, S., Pigorini, A., Canali, P., Casali, A., and Massimini, M. (2012). Combining transcranial magnetic stimulation with electroencephalography to study human cortical excitability and effective connectivity. Neuromethods 67, 435–457. doi: 10.1007/7657_2011_15

PubMed Abstract | CrossRef Full Text | Google Scholar

Tremblay, S., Rogasch, N. C., Premoli, I., Blumberger, D. M., Casarotto, S., Chen, R., et al. (2019). Clinical utility and prospective of tms–eeg. Clin. Neurophysiol. 130, 802–844. doi: 10.1016/j.clinph.2019.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

van der Maaten, L., and Hinton, G. (2008). Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605.

Google Scholar

Vapnik, V. N.. (1999). An overview of statistical learning theory. IEEE Trans. Neural Netw. 10, 988–999. doi: 10.1109/72.788640

PubMed Abstract | CrossRef Full Text | Google Scholar

Ye, Q., and Bai, F. (2018). Contribution of diffusion, perfusion and functional mri to the disconnection hypothesis in subcortical vascular cognitive impairment. Stroke Vascular Neurol. 3, 131–139. doi: 10.1136/svn-2017-000080

PubMed Abstract | CrossRef Full Text | Google Scholar