Machine Learning Based Classification of Resting-State fMRI Features Exemplified by Metabolic State (Hunger/Satiety)
Arkan Al-Zubaidi
Alfred Mertins2 
Marcus Heldmann
Thomas F. Münte- 1Department of Neurology, University of Lübeck, Lübeck, Germany
- 2Institute for Signal Processing, University of Lübeck, Lübeck, Germany
- 3Institute of Psychology II, University of Lübeck, Lübeck, Germany
- 4Department of Psychiatry and Psychotherapy, Kiel University - Christian-Albrechts, Kiel, Germany
Objective: Resting-state functional magnetic resonance imaging (rs-fMRI) has become an essential measure to investigate the human brain’s spontaneous activity and intrinsic functional connectivity. Several studies including our own previous work have shown that the brain controls the regulation of energy expenditure and food intake behavior. Accordingly, we expected different metabolic states to influence connectivity and activity patterns in neuronal networks.
Methods: The influence of hunger and satiety on rs-fMRI was investigated using three connectivity models (local connectivity, global connectivity and amplitude rs-fMRI signals). After extracting the connectivity parameters of 90 brain regions for each model, we used sequential forward floating selection strategy in conjunction with a linear support vector machine classifier and permutation tests to reveal which connectivity model differentiates best between metabolic states (hunger vs. satiety).
Results: We found that the amplitude of rs-fMRI signals is slightly more precise than local and global connectivity models in order to detect resting brain changes during hunger and satiety with a classification accuracy of 81%.
Conclusion: The amplitude of rs-fMRI signals serves as a suitable basis for machine learning based classification of brain activity. This opens up the possibility to apply this combination of algorithms to similar research questions, such as the characterization of brain states (e.g., sleep stages) or disease conditions (e.g., Alzheimer’s disease, minimal cognitive impairment).
Highlights
- We compare fALFF, DC, and ReHo for classifying human metabolic states within an rs-fMRI scan based on SVM.
- We combine an rs-fMRI based voxel-wise frequency-domain approach with sequential forward floating selection method to identify brain areas modulated as a function of hunger/satiety.
- It turns out that fALFF is a reliable and stable index of spontaneous brain activity.
Introduction
Resting-state functional magnetic resonance imaging (rs-fMRI) has been increasingly applied to study activity and connectivity of the resting brain and involves the recording of the blood-oxygen-level-dependent (BOLD) signal without imposing a task (Biswal et al., 2010; X.-N. Zuo and Xing, 2014). This approach can be used to assess intrinsic and spontaneous brain activity. Analysis techniques of rs-fMRI have evolved rapidly over the past few years and are based on correlation methods (Bullmore and Sporns, 2009; Sporns, 2011; Wee et al., 2012; Jie et al., 2014), partial correlation (Salvador et al., 2005; Marrelec et al., 2006, 2007), graph theory based analysis (Chen and Herskovits, 2007; Bullmore and Sporns, 2009) and sparse representation methods (Lee et al., 2011; Wee et al., 2014), among others. Because of its simplicity (short scan time, no stimulation equipment needed, no task requirements), the rs-fMRI method has become particularly popular for the characterization of clinical conditions, for example pinpointing to differences between healthy participants and patients with Parkinson’s disease (Göttlich et al., 2013; Tahmasian et al., 2015), Alzheimer’s disease (AD) (Sheline and Raichle, 2013; Dennis and Thompson, 2014), bilateral vestibular failure (Göttlich et al., 2014a), schizophrenia (Alderson-Day et al., 2016; Hu et al., 2017), and obsessive-compulsive disorder (Göttlich et al., 2014b; Gürsel et al., 2018), to name but a few targeted neuropsychiatric conditions.
In neuroimaging, machine learning classifier (MLC) methods are applied to fMRI data to detect model-free brain activity and to use these brain activity patterns to differentiate between groups or conditions (Cox and Savoy, 2003; Pereira et al., 2009). The application of MLC to fMRI data is often referred to as multi-voxel (i.e., analyzing more than one voxel at once) pattern analysis (MVPA). MVPA is a helpful tool to investigate how a pattern of brain activity is related to different cognitive states (Haynes and Rees, 2006; Norman et al., 2006; Mahmoudi et al., 2012). The process of applying the MVPA approach to fMRI data can be broken down into three stages (Pereira et al., 2009; Poldrack et al., 2011). First, feature extraction, which converts the BOLD fMRI signals to the relevant variables, i.e., features, which will be used to train and to test the classifier. Second, feature selection, which determines features that have to be included in the classifier analysis in order to improve the classification. Third, cross-validation methods that divide the data into training and testing samples and determine the accuracy of the classifier in generalization to new data.
Support vector machine (SVM; Cortes and Vapnik, 1995; Vapnik, 2013) is a powerful method available to perform MVPA. In contrast to alternative MVPA methods (such as linear discriminant analyses), SVM provides better prediction accuracy, having the advantage of being relatively insensitive to the sample size of the training dataset (O’Toole et al., 2007; Mountrakis et al., 2011). Furthermore, SVM has additional advantages regarding efficiency, simplicity, robustness and is less susceptible to noise (Mountrakis et al., 2011; Meier et al., 2012).
The application of SVM to fMRI data at group level has several advantages over traditional univariate (i.e., individual) voxel-based methods, like the general linear model (GLM). For instance, SVM allows to identify voxels or brain regions of interest that are informative for classifying groups by accumulating the information in an efficient way across many spatial locations. While in GLM analysis, these voxels or brain regions could appear statistically insignificant, although they might carry some information about differences between states or groups (Haynes and Rees, 2006; Norman et al., 2006). Thus, SVM provides insight into the defining differences between the two states or groups (Cox and Savoy, 2003; Orru et al., 2012).
In many cases, fMRI data have a small number of samples and a large number of variables or features. This often leads to overfitting in classification, which in turn leads to deceptive diagnostic results and poor generalization performance (Pereira et al., 2009; Wang, 2009). To avoid the danger of overfitting, most of the MVPA-based fMRI studies applied both methodologies, feature-selection algorithms to remove redundant information and MLC methods that are less sensitive to a high dimensionality, such as linear SVM. Finally, cross-validation analyses are performed to evaluate the classification accuracy and generalizability for unseen data (Cox and Savoy, 2003; Mitchell et al., 2004; Kamitani and Tong, 2005; Zhang et al., 2005).
Rs-fMRI yields data comprise multiple data points per subject and/or condition, among other things, raising the question of whether it might be possible to distinguish between different conditions (e.g., disease present or not) using classification algorithms from the realm of machine learning. Indeed, several recent publications have tackled this question. For example, Abós et al. (2017) obtained functional connectomes from the rs-fMRI in healthy controls (HC) and 70 Parkinson’s disease patients [of which one third had a mild cognitive impairment (MCI)]. Using a SVM trained on features selected through randomized logistic regression with leave-one-out cross-validation (LOOCV), they could separate patients with MCI from those not having MCI with an accuracy of about 83% in the training sample. In a smaller validation sample of 25 Parkinson patients (8 MCI), classification accuracy with regard to MCI was 80% using the features found in the training sample (Abós et al., 2017). This suggests that SVM classification based on metrics obtained from rs-fMRI can indeed yield meaningful results. Likewise, applying a graph theoretical approach to rs-fMRI to characterize functional connectivity in patients with MCI, AD and age-matched HC (total sample n = 168), followed by SVM based classification, Khazaee et al. (2015a) were able to accurately classify the subjects into three groups (HC, MCI, and AD) with 88.4% accuracy. The same research group (Hojjati et al., 2017) tried to distinguish patients with MCI who later converted to an AD from MCI patients who did not. Again a SVM, using features derived from local and global graph measures, was used. This approach yielded a specificity of 91.4% and sensitivity of 83.2% regarding the conversion to the AD. Bi et al. (2018) attempted to classify patients with autism spectrum disorder (ASD) from HC using random SVM cluster and reported classification accuracy based on the optimal feature to be 96%. These are just a few examples illustrating that rs-fMRI derived features can be used for classification of conditions using machine learning algorithms. What it is less clear, however, is which method of rs-fMRI analysis delivering the most discriminating features might be best in distinguishing different metabolic states (hunger vs. satiety).
In the present investigation, we therefore sought to compare the accuracies of three different connectivity parameters or features (the predictor variables used for classification) extracted from rs-fMRI fluctuations. These features assess local and global functional connectivity as well as changes in the brain activity as indicated by the amplitude of the BOLD signal, i.e., regional homogeneity (ReHo), degree of centrality (DC), and fractional amplitude of low-frequency fluctuations (fALFF), respectively. Briefly, ReHo characterizes the local connectivity of a brain voxel to its nearest neighboring brain voxels (Zang et al., 2004; Jiang and Zuo, 2015) by determining the coherence among spontaneous BOLD signals that might reflect spontaneous neuronal activity (Sato et al., 2012). ReHo has been applied to widely differing neuropsychiatric conditions (Cao et al., 2006; Liu et al., 2006; He et al., 2007; Wu et al., 2009; Paakki et al., 2010). DC is derived from graph theory based analysis and describes the global connectivity (global connectedness) of a given voxel with the voxels in the entire brain, by computing the number of connections above a certain threshold (Buckner et al., 2009; Bullmore and Sporns, 2009; Zuo et al., 2012). Again, DC has seen widespread application in neuropsychiatric conditions (Buckner et al., 2009; Beucke et al., 2013; Di Martino et al., 2013; Göttlich et al., 2013; Hou et al., 2014). Finally, to quantify spontaneous local brain activity, the amplitude of the BOLD signals has been used. This can be assessed by the amplitude of low-frequency fluctuations (ALFF) and its derivative fALFF (Zang et al., 2007; Zou et al., 2008). While ALFF describes the local spontaneous brain activity across the whole brain, by assessing the amplitude in a given voxel or brain area in the low-frequency range (0.01–0.08 Hz), fALFF is a normalized derivation of ALFF representing the ratio of low-frequency range amplitudes (0.01–0.08 Hz) relative to the entire frequency range (e.g., 0–0.25 if TR=2 s) amplitudes. Both ALFF and fALFF have high temporal stability (Küblböck et al., 2014) and test-retest reliability (X. N. Zuo and Xing, 2014). In contrast to ALFF, fALFF has been reported to have higher specificity in detecting local spontaneous brain activities, especially in gray matter (Zou et al., 2008; Zuo et al., 2010). Moreover, fALFF is recommended to be used instead of ALFF (X.-N. Zuo and Xing, 2014), since it is more robust against non-specific signal components, such as physiological noise (Zuo et al., 2010). In the present study, we performed fALFF on rs-fMRI data to describe the local spontaneous brain activities.
The aim of feature selection algorithms is to reduce the dimensionality of feature space and computation time, as well as to enhance the accuracy of optimization methods by ignoring redundant, irrelevant or noisy features (Tang et al., 2014; Jović et al., 2015). In general, the feature selection algorithms are classified in two categories, according to the type of objective functions that one chooses to work with: filter methods and wrapper methods (Pereira et al., 2009; Mwangi et al., 2014). Filter methods select the feature subsets based on statistical properties (such as interclass distance, mutual information, entropy or statistical independence) of the features to filter out poorly informative ones without employing any classification algorithm. In contrast, wrapper methods rate the feature subsets based on their predictive accuracy to improve the performance of classification when applying a particular classifier (such as SVM or the k-nearest neighbor). Filter methods are advantageous because they perform quickly, afford a more general solution and tend to select large feature subsets. Wrapper methods are expensive because they need more time to train the classifier of each subject many times (i.e., cross-validation), but often do not deteriorate from the problem of overfitting (Burrell et al., 2007) and provide more accurate results comparable to filter methods (Guyon and Elisseeff, 2003; Maldonado and Weber, 2009).
There are several strategies to apply wrapper methods (Mwangi et al., 2014). For instance, sequential forward selection (SFS) and sequential forward floating selection (SFFS) are easy to execute and are assumed to provide useful results. Although the SFFS strategy requires massive computational resources, it performs better and is more effective for solving small- and medium-scale problems than simpler strategies like SFS (Kudo and Sklansky, 2000). However, the SFS strategy reduces the computational costs for the feature subset selection. Accordingly, Burrell et al. (2007) concluded that SFS was a reasonable alternative to select a small subset of features for fMRI data. In this work, we compared between SFS and SFFS strategies for creating feature subsets to distinguish different metabolic states.
The emphasis of the present work is on the ability to classify the metabolic states (hunger vs. satiety) by the MVPA approach. Therefore, we first estimate and compare the prediction accuracy of classification (hunger vs. satiety) based on different features of rs-fMRI data (ReHo, DC and fALFF). Second, we identify brain regions containing discriminating information between different metabolic states. To this end, we apply supporting linear SVM as a classifier and two feature selection strategies (SFS and SFFS) to identify those brain regions that most efficiently differentiate between hungry and satiated states based on rs-fMRI data. Finally, we employ a cross-validation scheme and permutation tests to validate the reliability of classifier and significance testing, respectively (see Figure 1 for classification procedure).
Figure 1. Full analysis procedure of hunger classification based on rs-fMRI data.
Materials and Methods
Experimental Design
We investigated 24 healthy male volunteers aged 20 to 30 years (mean ± SEM: 24.5 ± 0.6 years) with a body mass index (BMI) within the normal weight range of 20–25 kg/m2 (mean ± SEM in kg/m2: 23.4 ± 0.3), recruited from the University and the local population of Lübeck, Germany.
Each subject was investigated in two sessions (i.e., two metabolic states), once under fasting (36 h fasting) and once under standardized eating conditions (five meals throughout 36 h). The order of the two sessions was randomized across participants, with a break of at least 1 week between sessions.
In the hunger condition, subjects fasted (no food or beverages, except water) from 11 pm the night before the examination started until the end of the rs-fMRI recording. In the satiety condition, three standardized meals per day were provided. Standardized meals were served according to the recommendations of the clinical diabetes counseling department at the University Medical Campus Schleswig-Holstein (UMCSH): Breakfast (25% protein, 50% carbohydrate and 25% fat), lunch (20% protein, 63% carbohydrate and 17% fat) and dinner (22% protein, 60% carbohydrate and 18% fat) were provided at 9 am, 12 pm and 7 pm, respectively. In both sessions, subjects arrived at the sleep lab at 8 am. The first blood sample for defining the blood glucose levels was taken at 8:45 am. More details on the exact timing for obtaining blood samples can be found in our previous work (please see the experimental design and Figure 1 in Al-Zubaidi et al., 2018). All subjects stayed and slept overnight in the laboratory at the UMCSH. On the next morning, blood samples were drawn at 8:45 am and at fixed time points throughout noon until the MRI was performed at 1 pm for the rs-fMRI recording (duration 6 min). Subjects were instructed to lie still inside the scanner having the eyes closed and trying to avoid any particular cognitive activity. For each condition, subjects rated hunger feelings 20 min before the MRI sessions started. This was done by using a visual analog scale from 0 (not hungry at all) to 9 (very hungry).
Blood sugar levels were lower while feelings of hunger were more intense during hunger compared to satiety before the rs-fMRI scans (Figure 2). These findings confirm the success of our fasting treatment. Results of blood sugar levels and hunger ratings can be found in Al-Zubaidi et al. (2018).
Figure 2. Average blood glucose levels (A) and rating of hunger feeling (B) per experimental condition. ∗ and ∗∗∗ represent the significant differences between conditions, at a threshold of p < 0.01 and p < 0.0001, respectively.
Image Acquisition
We used a 3-T Philips Achieva scanner (Philips Medical Systems, Best, Netherlands) and a standard eight-channel phased array head coil to record structural and functional images. Anatomical scans consisting of 180 sagittal slices were acquired by applying a T1-weighted 3D turbo gradient-echo sequence with SENSE (image matrix 240 × 240; voxel dimensions 1 × 1 × 1 mm; field of view 240 × 240 mm2; 1 mm slice thickness; 9° flip angle). For the rs-fMRI recording (duration 6 min), subjects were instructed to lie still inside the scanner with their eyes closed and to not engage in any particular cognitive activity. Further, 178 whole-brain fMRI volumes were acquired in an interleaved fashion with a T2∗-weighted single-shot gradient-echo EPI sequence (TR = 2000 ms; TE = 28 ms; voxel dimensions 3 × 3 × 3 mm; field of view 192 × 192 mm2; 80° flip angle; 40 slices).
Preprocessing
Part of the preprocessing of the functional images was carried out using FSLv5.01 to implement independent component analysis (ICA)–based strategy for automatic removal of motion artifacts (ICA-AROMA) for head motion correction (Pruim et al., 2015a). It has been reported that ICA-AROMA improves the specificity (i.e., the signals of interest) and sensitivity (i.e., motion-related noise) of rs-fMRI activation and connectivity analyses (Pruim et al., 2015a). To improve inter-subject alignment (Klein et al., 2009; McLaren et al., 2010), the spatial preprocessing of the data was performed with the statistical parametric mapping 12b (SPM12b).
The rs-fMRI images were preprocessed as follows: (i) we discarded the first eight functional volumes from each participant’s two runs to allow steady-state tissue magnetization; (ii) we manually reoriented all functional volumes to the anterior commissure; (iii) we implemented head movement correction on data by realigning all volumes to the middle time-point volume using MCFLIRT (Jenkinson et al., 2002); (iv) we applied ICA-AROMA to the data in order to identify and remove motion-related components using four spatial and temporal features (Pruim et al., 2015b). Briefly, functional images were submitted to the MELODIC toolbox for running a probabilistic independent component analysis (PICA) with automatic estimation of the number of hidden independent components (i.e., source signals) to find a set of components for each participant per recording individually (Beckmann and Smith, 2004). A predetermined classifier was then applied on independent components to represent the motion-related artifact components, which were identified at least by assessing each component to one of the following criteria: (1) maximum correlation with realignment parameters, (2) high-frequency temporal content >35%, (3) spatial content in edge voxels and cerebrospinal fluid (CSF) >10%. Finally, we removed the motion-relevant components from rs-fMRI data that had been realigned by using a linear regression approach. The structural and denoised functional images were further preprocessed with data processing assistant for resting-state fMRI (DPARSF) toolbox as follows (Yan, 2010): (v) we co-registered the T1 structural image to the mean functional image; (vi) we ran a segmentation protocol to distinguish gray matter, white matter and CSF; (vii) we applied bias correction and spatial normalization of the T1 structural image and adjusted them to the MNI template using DARTEL algorithm (Ashburner and Friston, 2005); (viii) we performed nuisance regression (including white matter and CSF signals) to reduce the impact of undefined physiological effects on rs-fMRI signals (Liu, 2016); (ix) we spatially normalized functional images and gray matter images to the MNI-template using the normalization parameters estimated by DARTEL algorithm with a 3 mm isotropic voxel size; (x) we performed spatial smoothing with a 6 mm full width at half maximum (FWHM) Gaussian kernel. Rs-fMRI signals were smoothed after calculating ReHo (Zang et al., 2004) not before (see ReHo paragraph); (xi) we applied temporal band-pass filtering (0.01–0.08 Hz) on the rs-fMRI signal to reduce the effect of low-frequency drift, such as respiratory, and high-frequency noise, such as heart activity. As suggested by Zou et al. (2008) no such band-pass filter was used when computing the fALFF (see fALFF paragraph); (xii) finally, we masked all functional images with a gray matter mask. The gray matter mask was calculated by averaging the grey matter images of all subjects. To generate the binary mask, we defined the common voxels between the average gray matter image and the gray matter template (without cerebellar lobules) derived from the Automated-anatomical-labeling (AAL) atlas (Tzourio-Mazoyer et al., 2002) by using the xjView toolbox2. In some subjects we were not able to measure the whole cerebellum. Therefore, the cerebellar cortex was excluded from the gray matter mask and regions of interest (ROIs).
Feature Extraction From rs-fMRI Data
Feature extraction is used to reduce the dimension of the original data space to a new feature space. This new feature space helps to minimize the training time taken by the classifier (Chen et al., 2010). To encompass different aspects of rs-fMRI fluctuations, we extracted and compared three of the most common features according to their accuracy to select those brain regions that best distinguish different metabolic states by using linear SVM together with feature selection strategies. In the beginning, we submitted the preprocessed data to the DPARSF toolbox (Yan, 2010) and extracted three features from each subject per section as described in the following paragraphs. Then, those features were analyzed as explained in Section of Feature Selection.
First, local connectivity of brain regions was described using ReHo (for regional homogeneity), which is a measure of the temporal homogeneity among brain voxels and the neighboring brain voxels within the low-frequency range of rs-fMRI signals (Zang et al., 2004). The rationale behind ReHo is based on the assumption that the BOLD signal has characteristics that depend on neuronal activities, and therefore, the time series of neighboring voxels in a functional brain area will be highly similar or synchronized when that area supports specific goals or representations (Jiang and Zuo, 2015). The ReHo index for a particular voxel is calculated by using Kendall’s coefficient concordance (KCC) approach. In this study, the KCC was calculated using the following formula (Zang et al., 2004):
where KCC is the ranging coefficient, from 0 to 1 (no to maximal coherence), of a given voxel in relation to its nearest neighbors, Ri represents the rank sum of i th time point as and rij is the rank of the i th time point of the j th voxel. refers to the average of the Ri and n represents the length of the time series (here n = 170 time points). K is the number of voxels within the targeted clusters (here K = 27, the given voxel (which is the center voxel) plus its 26 immediate neighbors). The KCC value was then put to the center voxel of the respective cluster. From the KCC of all voxels, the so-called ReHo map can be constructed. Thus, an individual ReHo map was computed for each subject per session.
Second, DC (for degree centrality) is used to investigate the global connectivity of brain regions, which is defined as the number of connections of one voxel in the brain to whole brain voxels (Buckner et al., 2009). This measure depends on graph theoretical approaches. The individual DC map was generated by correlating the time course of each voxel in the brain with all other voxels in the brain and calculating the number of connections above a definite threshold (Buckner et al., 2009; Al-Zubaidi et al., 2018). The temporal connection between two voxels was defined by applying Pearson’s correlation coefficient (r) approach. Then, the individual correlation coefficients were used to generate a correlation matrix = , 1 = i,j = N, where N is the number of voxels within the whole-brain mask and rij is the temporal Pearson’s correlation of time series between i th and j th voxels measuring the similarity between two voxels. The correlation matrix was thresholded at 0.25 to build a binary undirected and unweighted network matrix dij as follows
The binary connectivity matrix dij was used to define the degree centrality of voxel Di by the following
Third, fALFF was employed to provide information on the magnitude of the BOLD signals, which reflects the neural activity of each brain voxel or region within a network of interest (Fujino et al., 2017; Al-Zubaidi et al., 2018). It refers to the ratio of rs-fMRI signal fluctuation in the low-frequency range proportional to the entire frequency range (Zou et al., 2008). DPARSF toolbox has a built-in fast Fourier transform (FFT) to convert time series for each voxel to the frequency domain and compute the power spectrum. This procedure estimates the amplitude of each frequency as the square root of the power spectrum. The total amplitude of the low-frequency range (0.01–0.08 Hz) was divided by that of the full frequency range 0–0.25 Hz.
Feature Selection for Hunger/Satiety Status Classification
After generating ReHo, DC and fALFF maps from rs-fMRI data for each subject per condition, we used the AAL atlas to define the ROIs. The AAL atlas is a well-established anatomical parcellation of the brain into 45 ROIs per hemisphere when excluding the cerebellar lobules. Mean values of ReHo, DC and fALFF were calculated for each ROI and used to create a feature (region) vector, i.e., R[1, ..., 90], with 90 dimensions for each map. Those features are listed in Table 1.
Table 1. List of the anatomical regions (AAL atlas) of interest and their labels in the region vector.
In the classical classification problem, the goal of feature selection is to automatically search and select the best feature subset for the classification purpose. Here, we applied sequential feature selection algorithms to select the optimal feature subset (region subset) that best captured differences between hunger and satiety. This type of selection algorithm contains two components. The first element is a sequential search strategy to select and establish the best future subset, which evaluates additional features by a criterion function. In this study we used two strategies, namely SFS and SFFS. The SFS procedure starts by identifying the first feature with the highest classification rate and feeds it to a new empty candidate set. Other features are selected sequentially by adding a local feature to the first feature or the last subset of features in the candidate set, and testing a new feature combination until the highest classification rate (objective function) is achieved. The processing continues until further features do not enhance the objective function. However, the SFS algorithm is suboptimal and suffers from the “nesting effect” (Pudil et al., 1994), while SFFS offers the flexibility to discard features that were earlier selected and to re-evaluate features that had been discarded previously. This theoretical advantage notwithstanding, Burrell et al. (2007) showed that the computationally less demanding SFS could provide a reasonable alternative to SFFS to select features for discriminating between epileptic and non-epileptic activity of epileptic patients, indicating that both strategies had similar difficulties to separate patterns of functional and dysfunctional brain activities in epileptic patients. In this study, we compared SFS and SFFS strategies in order to figure out if SFS already provides near-optimal results. SFS and SFFS strategies were performed using the “sequential” function in MATLAB and sequential floating feature selection toolbox3, respectively.
The second component in the feature selection strategy is an objective (criterion) function to evaluate over all possible feature subsets. In this work, the misclassification rate of the linear SVM classifier was set as an objective function (Guyon and Elisseeff, 2003). The combination of SFFS and SVMs has previously been used, for example to assess Gabor features for classification of Parkinson’s disease risk assessment based on transcranial sonography images (Al-Zubaidi et al., 2013). To evaluate the feature subset, the data were divided into test and training samples using a LOOCV scheme. Accordingly, independent samples were used for training and testing. For each LOOCV loop, the training samples were submitted to train an SVM model, and the test sample was applied to that model to evaluate the feature subset. In the end, the average of the values returned by LOOCV loops was calculated and used to assess each candidate’s feature subset (Kohavi and John, 1997).
The classification accuracy (CA) was derived using a LOOCV strategy with confusion matrix (CM) and calculation of classification error rate (ER). In our study, the CM comprises information about the actual and predicted classifications generated by linear SVM. By comparing the results of the SVM classifier (hunger or satiety) with the reference data, we documented the outcomes of the CM in the present study as given in Table 2. For significance testing (Pereira et al., 2009), we estimated the empirical distribution by calculating the error rate 10,000 times for random label permutations in a cross-validation procedure. P < 0.05 implies that classification results differ significantly from chance.
Table 2. Confusion matrix.
Results
The experiments showed that fALFF was marginally better than ReHo and DC in distinguishing between hunger and satiety states in the healthy brain (Table 3). The region subset obtained by SFFS resulted in higher classification accuracy than SFS, both higher than no feature selection (90 regions). Using a linear SVM classifier with an LOOCV strategy, we observed that the fALFF region subset selected by SFFS identified the hunger state with the highest classification accuracy of 81% and with the most balanced overall performance. In our SFFS results (Table 3), the regions 45 and 46 are the left and right inferior occipital lobe (Table 1), respectively, and region 50 is the right superior occipital lobe. Also, regions 5 and 35 are medial (Calcarine) and inferior (Fusiform) surfaces of the occipital lobe, respectively. Furthermore, regions 17 and 18 are left and right frontal gyrus, respectively, region 52 is the right Olfactory cortex, region 56 is the right Paracentral lobule and region 73 is the left Rolandic operculum. SFFS-identified brain regions, which distinguished best between hunger and satiety for each rs-fMRI feature, are shown in Figure 3. Furthermore, SFFS fALFF (Figure 4) was most stable in the permutation test (ER = 0.19 / p = 0.0001) closely followed by DC (ER = 0.21/p = 0.0004) and ReHo (ER = 0.29/p = 0.0068).
Table 3. Classification accuracy of rs-fMRI data using different models of brain connectivity/activity and features selection algorithms with linear SVM classifier.
Figure 3. Brain regions that provided relevant information to distinguish between hunger and satiety states in healthy lean participants. The performance of these regions was evaluated by linear SVM classifier and SFFS algorithm. All images are in neurological orientation, i.e., right = right and left = left.
Figure 4. Empirical distributions of incorrect classification generated via 10000 times of random label permutations for region sets selected by SFFS. Red line shows the actual classification error.
Discussion
The primary goal of the present research was to assess the ability to classify different brain states by applying a MVPA approach, i.e., feature selection strategies and linear SVM, on various features, i.e., connectivity parameters, derived from rs-fMRI data. This approach was carried out on a data set comprising two conditions (hungry and satiated) in a repeated measures design. As the two metabolic states, i.e., hunger and satiety, were induced for a rather long time (36 h), experimental conditions can be treated as the ground truth to compare and evaluate the classification scheme.
An advanced preprocessing, including ICA-AROMA, was carefully applied, ensuring the removal of motion artifacts and other structured noise from the data [e.g., cardiac pulsation artifacts (Pruim et al., 2015b)]. Thus, classification in the current case is deemed to reflect true brain differences rather than extracerebral differences (e.g., motion) between the conditions.
A critical question with regard to classification is the selection of the best approach for feature selection. The inclusion of all possible features and the computation of all possible combinations of features is computationally not feasible at present. Therefore, sequential search techniques have gained some popularity. These work by choosing the best individual feature and then adding a second feature that yields the best classification accuracy in combination with the first feature. This procedure is repeated for a third and any subsequent features until the addition of further features does not yield an improvement of classification rates. Alternatively, the user can predetermine the maximum number of features. The SFFS method tries to optimize feature selection by adding an elimination step to this sequential search process. Concretely, at each level, it is examined whether the classification rates improve, if any of the selected features are eliminated. If this happens, the feature set is reduced by this feature. Then, the search continues based on the new set. Jain and Zongker (1997), who compared 14 different methods for feature selection applying them to the problem of handwriting recognition. In this case, the SFFS method outperformed other feature selection schemes. It has to be pointed out, however, that conventional feature selection approaches, including SFS and SFFS as well as filter approaches (Kira and Rendell, 1992; Almuallim and Dietterich, 1994), have recently been supplemented by metaheuristic methods for feature selection. In this regard, Zhu et al. (2007) have suggested a genetic algorithm combined with local search in a hybrid wrapper and filter feature selection algorithms. Others, like Neshatian and Zhang (2009) and Gu et al. (2018), proposed new optimizations methods including such advanced feature selection procedures. Applying genetic algorithms and new optimization functions on rs-fMRI data are beyond the scope of this study and will be considered in future work.
In the present study, the subset of regions obtained by the SFFS algorithm provided the highest classification rate for all rs-fMRI maps (Table 3). Using fALFF, SFFS and SVM classification, we were able to demonstrate that patterns of amplitude BOLD signals in five brain regions [paracentral lobule, Rolandic operculum, olfactory cortex, lateral occipital (inferior division) gyrus, and inferior frontal (opercular) gyrus; Figure 3] can distinguish between metabolic states (hunger vs. satiety) with 81% accuracy.
The Rolandic operculum, which belongs to somatosensory regions, is activated during the anticipation and consumption of food (Stice et al., 2008; Stice and Yokum, 2016), in response to palatable food receipt (Stice et al., 2013) and has been associated with the processing of high- and low-caloric food pictures (Stice et al., 2010). Among many functions, paracentral gyrus is known to respond to highly rewarding stimuli (Stice and Yokum, 2016). A study using independent component analysis to estimate functional connectivity (FC) parameters showed that the connectivity strength of the paracentral gyrus in the sensorimotor network was increased during hypoglycemia relative to euglycemia (Bolo et al., 2015). Furthermore, Van Duinkerken et al. (2012) reported that the change of sensorimotor FC was associated with basal glycemic levels in type 1 diabetes mellitus patients. Thus, the paracentral lobule seems to be part of the reward system and the sensorimotor network. The olfactory cortex (OLFC), whose activity was modulated by metabolic states as well, is involved in the experience and processing of negative affective states, including anxiety and depression (Krusemark et al., 2013). Consistent with that, a rs-fMRI study in rodents demonstrated that ReHo of the OLFC is increased in stress-exposed rats compared to a control group (Li et al., 2018). In our study, the fasting for 36 h might have led to a stress increase, which might be reflected in an increase in OLFC activity. Moreover, the satiety state might have reduced peripheral hunger signals compared to the hunger state, and accordingly, might have influenced brain regions related to somatosensory processes, such as Rolandic operculum, and parts of the sensorimotor network like the paracentral lobule.
The inferior frontal gyrus (IFG) has been suggested to be involved in cognitive control (Hare et al., 2009; Sundermann and Pfleiderer, 2012). IFG activation during response inhibition has been associated with a reduced desire for food and with successful impulse regulation (Gabrieli et al., 2015; Hollmann et al., 2012; Lopez et al., 2014). In addition, stronger IFG activity in response to orosensory stimulation was found in successful weight loss maintainers compared to people who were obese or normal weighted (Sweet et al., 2012). In our experiment, participants had to refrain from eating during the hunger state and from overfeeding during the satiety state, which may have contributed to the finding that the IFG is partially important for classifying between different metabolic states.
The lateral occipital cortex (LOC) is part of the visual association cortex and is activated in response to the perception of emotionally salient stimuli, such as food, which is thought to be a correlate of heightened attention (Killgore and Yurgelun-Todd, 2007; van der Laan et al., 2011). For instance, a recent rs-fMRI study using SVM on graph theory analysis indicates that the LOC is partly important for classification between high-caloric (potato chips) vs. low-caloric (zucchini) food ingestion on the brain of healthy subjects (Mendez-Torrijos et al., 2018). Furthermore, it has been suggested that the processing of visual salience of a stimulus depends on the affective state of the individual and the motivational value of a stimulus (Killgore and Yurgelun-Todd, 2007). Considering the general role of the LOC in the visual processing of food stimuli, this region might potentially facilitate the detection/perception of such cues in a deprived state. Note that these interpretations are based on reverse inference of resting-state data and should thus be taken with caution.
However, some studies have used rs-fMRI to investigate changes in baseline brain activity of lean or obese participants during both hunger and satiety states. For instance, Lohmann et al. (2010) showed increased centrality, which was measured by eigenvector centrality analysis, of the anterior precuneus (APCUN) during the hunger state relative to the satiety state of 22 normal volunteers. Consistent with that, our previous study (Al-Zubaidi et al., 2018) revealed that the fALFF was increased in the APCUN and posterior cingulate cortex (PCC) when comparing hunger against satiety states. Additionally, Wright et al. (2016) used seed-based connectivity analysis to estimate the FC parameters of 19 healthy participants. They reported that the FC between the posterior insula and superior frontal gyrus, and between the hypothalamus and IFG, were enhanced during the hunger state. Furthermore, it has been found that both 20 lean and 20 obese subjects had increased ReHo connectivity from hunger to the satiety state in the orbitofrontal cortex and inferior temporal lobe (Zhang et al., 2015). These studies reported different brain regions that might be related to the changes in the metabolic state compared to the results of the current study, except IFG. These inconsistencies here might be associated with the different experimental paradigms and neuroimaging modalities.
Importantly, the statistical analysis at group levels of the studies mentioned above was computed by using GLM approaches to define the significant brain regions at baseline activity that are sensitive to changes in metabolic state. In GLM approaches, the p-values are the successful statistical tests to represent significant brain regions that show different brain activities in the average sense of one or more brain features when compared between different groups and/or experimental conditions. On the other hand, SVM classifier aims to automatically classify each subject into one of the groups or experimental design in the study. Thus, the overall classification accuracy is usually used to measure the success of the studies. In general, it is easier to demonstrate group or experimental condition differences compared to predict a single subject (Arbabshirani et al., 2017). Furthermore, the significant variables or features that show the difference between group or experiment do not necessarily have high classification accuracy and vice versa (Lo et al., 2015; Arbabshirani et al., 2017). Hence, brain regions that showed significant differences between hunger and satiety states in previous studies do not necessarily mean that these regions can predict the subject class with high accuracy and vice versa.
Furthermore, our results indicate that fALFF analysis is more informative than ReHo and is slightly more precise than DC for classification of resting brain changes during hunger and satiety, probably because fALFF is an index of the power of the BOLD signal. Against this, ReHo and DC parameters refer to dynamics of BOLD connectivity, either with some (in this case, 26) neighboring voxels (ReHo) or with all voxels (DC) in the gray matter of the brain.
One other important question is, whether SFFS based classification is superior to simply trying to classify states using statistically significant group differences. Baker et al. (2012) answered this question on an EEG data set from AD patients, MCI patients and HC. They used an SFFS algorithm and a t-test to classify patients and found that the SFFS technique resulted in improved classification rates compared to the t-test for four feature types (average power, coherence, correlation, and phase synchrony). They concluded that the SFFS algorithm selects reliable features for classification where statistically significant features fail in classification.
The sample size of most fMRI studies is often relatively small due to the high costs of scanning time and subject stamina (Poldrack et al., 2017). However, sample size impacts the trade-off between accuracy and generalizability (Schnack and Kahn, 2016). For instance, in the context of rs-fMRI features and SVM classification methods, several studies reported high classification accuracies (92 ± 9%) with relatively small sample sizes (20 ± 5 subjects per group), when classifying groups of brain disease patients and healthy subjects (Lord et al., 2012; Tang et al., 2012; Fekete et al., 2013; Wang et al., 2013; Wei et al., 2013; Cheng et al., 2015; Khazaee et al., 2015b). Here, the high classification accuracy is driven by the heterogeneity between groups (Schnack and Kahn, 2016; Arbabshirani et al., 2017). In contrast, studies with large sample sizes are assumed to result in classification models with a higher degree of generalizability, allowing for a better prediction in samples drawn from other populations. Their classification models capture a more complete picture of disease patterns, but at the cost of lower accuracy, which is likely due to the within-group heterogeneity (Anderson and Cohen, 2013; Krystal et al., 2013; Guo et al., 2014; Schnack and Kahn, 2016; Arbabshirani et al., 2017).
One limitation of the present study is the ability to generalize, since the sample size of 24 subjects is relatively small. Accordingly, larger samples are needed to confirm our findings. However, we would like to argue that our results are not driven by the heterogeneity between samples, because we have chosen a within-subject design in a well-controlled experimental setting. Also, we evaluated whether the rs-fMRI features in conjunction with sequential feature selection strategies were sufficiently reliable to predict the subject’s metabolic state using the LOOCV scheme. Thus, independent training and testing samples were used to estimate the SVM model parameters and to validate the classification model. In this case, the CA was derived by averaging the resulting classification accuracies over all samples. All in all, the homogeneity of our samples and the high CA results in an increased validity of our findings, determining brain patterns that are able to discriminate between different metabolic states.
As rs-fMRI has received widespread attention over the past 10 years, the possibility of reliable classification of disease conditions or subject states (e.g., sleep stages) paves the way for using rs-fMRI as a diagnostic tool on an individual patient/subject level. In addition the applications mentioned in the introduction, such as the prediction of conversion of MCI to AD (Hojjati et al., 2017), many other diagnostic and research questions lend themselves to this approach, e.g., the differentiation of typical Parkinson’s disease from atypical Parkinsonian syndromes (c.f., Tang et al., 2010). From our data, we conclude that fALFF, in combination with SFFS based feature selection, is a useful and straightforward way to proceed in tackling such research questions.
Ethics Statement
This study was carried out in accordance with the recommendations of the Declaration of Helsinki. The protocol was approved by the ethics committee of the University of Lübeck. Before participation, each participant gave written informed consent to the study.
Author Contributions
KJ-C and TM designed the study and wrote the protocol. AA-Z, MH, and KJ-C participated in the data collection. AA-Z managed the literature searches, wrote the first draft of the manuscript and performed data analysis. AA-Z and AM designed the statistical analyses. MH and TM helped with interpretation of data. All authors contributed to the manuscript revision and approved the final version.
Funding
This research was supported by a grant of the German Research Foundation to the Research Training Group 1957 “Adipocyte-Brain Crosstalk” and the SFB TR 134, project C1.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We acknowledge financial support by Land Schleswig-Holstein within the funding program Open Access Publikationsfonds.
Abbreviations
AAL, Automated-Anatomical-Labeling; AD, Alzheimer’s disease; APCUN, anterior precuneus; BMI, body mass index; BOLD, Blood-oxygen-level-dependent; CA, classification accuracy; CFS, cerebrospinal fluid; CM, confusion matrix; DARTEL, diffeomorphic anatomical registration through exponentiated Lie algebra; DC, degree of centrality; DPARSFA, data processing assistant for resting-state fMRI advanced edition; EPI, echo-planar imaging; ER, error rate; fALFF, fractional amplitude of low-frequency fluctuations; FC, function connectivity; fMRI, functional magnetic resonance imaging; FSL, FMRIB Software Library; GLM, general linear model; HC, healthy controls; ICA-AROMA, independent component analysis (ICA)-based strategy for automatic removal of motion artifacts; IFG, inferior frontal gyrus; KCC, Kendall’s coefficient concordance; LOC, lateral occipital cortex; LOOCV, h-out cross-validation; MCI, mild cognitive impairment; MLC, machine learning classifier; MNI, Montreal Neurological Institute; MVPA, multivariate voxel-pattern analysis; OLFC, olfactory cortex; ReHo, regional homogeneity; ROI, region of interest; rs-fMRI, resting-state functional magnetic resonance imaging; SFFS, sequential forward floating selection; SFS, sequential forward selection; SPM, statistical parametric mapping; SVM, support vector machine; TE, echo time; TR, repetition time.
Footnotes
- ^http://fsl.fmrib.ox.ac.uk/fsl/
- ^http://www.alivelearn.net/xjview8
- ^http://splab.cz/en/download/software/software-pro-sekvencni-selekci-priznaku
References
Abós, A., Baggio, H. C., Segura, B., García-Díaz, A. I., Compta, Y., Martí, M. J., et al. (2017). Discriminating cognitive status in Parkinson’s disease through functional connectomics and machine learning. Sci. Rep. 7:45347. doi: 10.1038/srep45347
Alderson-Day, B., Diederen, K., Fernyhough, C., Ford, J. M., Horga, G., Margulies, D. S., et al. (2016). Auditory hallucinations and the brain’s resting-state networks: findings and methodological observations. Schizophr. Bull. 42, 1110–1123. doi: 10.1093/schbul/sbw078
Almuallim, H., and Dietterich, T. G. (1994). Learning Boolean concepts in the presence of many irrelevant features. Artif. Intell. 69, 279–305. doi: 10.1016/0004-3702(94)90084-1
Al-Zubaidi, A., Chen, L., Hagenah, J., and Mertins, A. (2013). “Robust feature for transcranial sonography image classification using rotation-invariant Gabor filter,” in Bildverarbeitung für die Medizin 2013, Informatik Aktuell, eds H. P. Meinzer, T. Deserno, H. Handels, and T. Tolxdorff (Berlin: Springer).
Al-Zubaidi, A., Heldmann, M., Mertins, A., Jauch-Chara, K., and Münte, T. F. (2018). Influences of hunger, satiety and oral glucose on functional brain connectivity: a multimethod resting-state fMRI study. Neuroscience 382, 80–92. doi: 10.1016/j.neuroscience.2018.04.029
Anderson, A., and Cohen, M. (2013). Decreased small-world functional network connectivity and clustering across resting state networks in schizophrenia: an fMRI classification tutorial. Front. Hum. Neurosci. 7:520. doi: 10.3389/fnhum.2013.00520
Arbabshirani, M. R., Plis, S., Sui, J., and Calhoun, V. D. (2017). Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. Neuroimage 145, 137–165. doi: 10.1016/j.neuroimage.2016.02.079
Ashburner, J., and Friston, K. J. (2005). Unified segmentation. Neuroimage 26, 839–851. doi: 10.1016/j.neuroimage.2005.02.018
Baker, M. C., Kerr, A. S., Hames, E., and Akrofi, K. (2012). “An SFFS technique for EEG feature classification to identify sub-groups,” in Proceedings of the 25th International Symposium on Computer-Based Medical Systems (CBMS), (Rome: IEEE), 1–4.
Beckmann, C. F., and Smith, S. M. (2004). Probabilistic independent component analysis for functional magnetic resonance imaging. IEEE Trans. Med. Imaging 23, 137–152. doi: 10.1109/TMI.2003.822821
Beucke, J. C., Sepulcre, J., Talukdar, T., Linnman, C., Zschenderlein, K., Endrass, T., et al. (2013). Abnormally high degree connectivity of the orbitofrontal cortex in obsessive-compulsive disorder. JAMA Psychiatry 70, 619–629. doi: 10.1001/jamapsychiatry.2013.173
Bi, X., Wang, Y., Shu, Q., Sun, Q., and Xu, Q. (2018). Classification of autism spectrum disorder using random support vector machine cluster. Front. Genet. 9:18. doi: 10.3389/fgene.2018.00018
Biswal, B. B., Mennes, M., Zuo, X.-N., Gohel, S., Kelly, C., Smith, S. M., et al. (2010). Toward discovery science of human brain function, supporting information. Proc. Natl. Acad. Sci. U.S.A. 107, 4734–4739. doi: 10.1073/pnas.0911855107
Bolo, N. R., Musen, G., Simonson, D. C., Nickerson, L. D., Flores, V. L., Siracusa, T., et al. (2015). Functional connectivity of insula, basal ganglia, and prefrontal executive control networks during hypoglycemia in type 1 diabetes. J. Neurosci. 35, 11012–11023. doi: 10.1523/JNEUROSCI.0319-15.2015
Buckner, R. L., Sepulcre, J., Talukdar, T., Krienen, F. M., Liu, H., Hedden, T., et al. (2009). Cortical hubs revealed by intrinsic functional connectivity: mapping, assessment of stability, and relation to Alzheimer’s disease. J. Neurosci. 29, 1860–1873. doi: 10.1523/JNEUROSCI.5062-08.2009
Bullmore, E., and Sporns, O. (2009). Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10, 186–198. doi: 10.1038/nrn2575
Burrell, L., Smart, O., Georgoulas, G. K., Marsh, E. D., and Vachtsevanos, G. J. (2007). “Evaluation of feature selection techniques for analysis of functional MRI and EEG,” in International Conference on Data Mining (DMIN’07), Las Vegas, NV, 256–262.
Cao, Q., Zang, Y., Sun, L., Sui, M., Long, X., Zou, Q., et al. (2006). Abnormal neural activity in children with attention deficit hyperactivity disorder: a resting-state functional magnetic resonance imaging study. Neuroreport 17, 1033–1036. doi: 10.1097/01.wnr.0000224769.92454.5d
Chen, L., Seidel, G., and Mertins, A. (2010). “Multiple feature extraction for early Parkinson risk assessment based on transcranial sonography image,” in Proceedings of the 2010 International Conference on Image Processing (Nice: France).
Chen, R., and Herskovits, E. H. (2007). Clinical diagnosis based on bayesian classification of functional magnetic-resonance data. Neuroinformatics 5, 178–188. doi: 10.1007/s12021-007-0007-2
Cheng, H., Newman, S., Goñi, J., Kent, J. S., Howell, J., Bolbecker, A., et al. (2015). Nodal centrality of functional network in the differentiation of schizophrenia. Schizophr. Res. 168, 345–352. doi: 10.1016/j.schres.2015.08.011
Cox, D. D., and Savoy, R. L. (2003). Functional magnetic resonance imaging (fMRI)“brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270. doi: 10.1016/s1053-8119(03)00049-1
Dennis, E. L., and Thompson, P. M. (2014). Functional brain connectivity using fMRI in aging and Alzheimer’s disease. Neuropsychol. Rev. 24, 49–62. doi: 10.1007/s11065-014-9249-6
Di Martino, A., Zuo, X. N., Kelly, C., Grzadzinski, R., Mennes, M., Schvarcz, A., et al. (2013). Shared and distinct intrinsic functional network centrality in autism and attention-deficit/hyperactivity disorder. Biol. Psychiatry 74, 623–632. doi: 10.1016/j.biopsych.2013.02.011
Fekete, T., Wilf, M., Rubin, D., Edelman, S., Malach, R., and Mujica-Parodi, L. R. (2013). Combining classification with fMRI-derived complex network measures for potential neurodiagnostics. PLoS One 8:e62867. doi: 10.1371/journal.pone.0062867
Fujino, J., Tei, S., Jankowski, K. F., Kawada, R., Murai, T., and Takahashi, H. (2017). Role of spontaneous brain activity in explicit and implicit aspects of cognitive flexibility under socially conflicting situations: a resting-state fMRI study using fractional amplitude of low-frequency fluctuations. Neuroscience 367, 60–71. doi: 10.1016/j.neuroscience.2017.10.025
Gabrieli, J. D. E., Ghosh, S. S., and Whitfield-Gabrieli, S. (2015). Prediction as a humanitarian and pragmatic contribution from human cognitive neuroscience. Neuron 85, 11–12. doi: 10.1016/j.neuron.2014.10.047
Göttlich, M., Jandl, N. M., Wojak, J. F., Sprenger, A., Von Der Gablentz, J., Münte, T. F., et al. (2014a). Altered resting-state functional connectivity in patients with chronic bilateral vestibular failure. Neuroimage Clin. 4, 488–499. doi: 10.1016/j.nicl.2014.03.003
Göttlich, M., Krämer, U. M., Kordon, A., Hohagen, F., and Zurowski, B. (2014b). Decreased limbic and increased fronto-parietal connectivity in unmedicated patients with obsessive-compulsive disorder. Hum. Brain Mapp. 35, 5617–5632. doi: 10.1002/hbm.22574
Göttlich, M., Münte, T. F., Heldmann, M., Kasten, M., Hagenah, J., and Krämer, U. M. (2013). Altered resting state brain networks in Parkinson’s disease. PLoS One 8:e77336. doi: 10.1371/journal.pone.0077336
Gu, S., Cheng, R., and Jin, Y. (2018). Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput. 22, 811–822. doi: 10.1007/s00500-016-2385-6
Guo, W., Su, Q., Yao, D., Jiang, J., Zhang, J., Zhang, Z., et al. (2014). Decreased regional activity of default-mode network in unaffected siblings of schizophrenia patients at rest. Eur. Neuropsychopharmacol. 24, 545–552. doi: 10.1016/j.euroneuro.2014.01.004
Gürsel, D. A., Avram, M., Sorg, C., Brandl, F., and Koch, K. (2018). Frontoparietal areas link impairments of large-scale intrinsic brain networks with aberrant fronto-striatal interactions in OCD: a meta-analysis of resting-state functional connectivity. Neurosci. Biobehav. Rev. 87, 151–160. doi: 10.1016/j.neubiorev.2018.01.016
Guyon, I., and Elisseeff, A. (2003). An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182. doi: 10.1016/j.aca.2011.07.027
Hare, T. A., Camerer, C. F., and Rangel, A. (2009). Self-control in decision-Making involves modulation of the vmPFC valuation system. Science 324, 646–648. doi: 10.1126/science.1168450
Haynes, J.-D., and Rees, G. (2006). Neuroimaging: decoding mental states from brain activity in humans. Nat. Rev. Neurosci. 7, 523–534. doi: 10.1038/nrn1931
He, Y., Wang, L., Zang, Y., Tian, L., Zhang, X., Li, K., et al. (2007). Regional coherence changes in the early stages of Alzheimer’s disease: a combined structural and resting-state functional MRI study. Neuroimage 35, 488–500. doi: 10.1016/j.neuroimage.2006.11.042
Hojjati, S. H., Ebrahimzadeh, A., Khazaee, A., and Babajani-Feremi, A. (2017). Predicting conversion from MCI to AD using resting-state fMRI, graph theoretical approach and SVM. J. Neurosci. Methods 282, 69–80 doi: 10.1016/j.jneumeth.2017.03.006
Hollmann, M., Hellrung, L., Pleger, B., Schlögl, H., Kabisch, S., Stumvoll, M., et al. (2012). Neural correlates of the volitional regulation of the desire for food. Int. J. Obes. 36, 648–655. doi: 10.1038/ijo.2011.125
Hou, J. M., Zhao, M., Zhang, W., Song, L. H., Wu, W. J., Wang, J., et al. (2014). Resting-state functional connectivity abnormalities in patients with obsessive-compulsive disorder and their healthy first-degree relatives. J. Psychiatry Neurosci. 39, 304–311. doi: 10.1503/jpn.130220
Hu, M.-L., Zong, X.-F., Mann, J. J., Zheng, J.-J., Liao, Y.-H., Li, Z.-C., et al. (2017). A review of the functional and anatomical default mode network in schizophrenia. Neurosci. Bull. 33, 73–84. doi: 10.1007/s12264-016-0090-1
Jain, A., and Zongker, D. (1997). Feature selection: evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell. 19, 153–158. doi: 10.1109/34.574797
Jenkinson, M., Bannister, P., Brady, M., and Smith, S. (2002). Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17, 825–841. doi: 10.1016/S1053-8119(02)91132-8
Jiang, L., and Zuo, X.-N. (2015). Regional homogeneity: a multimodal, multiscale neuroimaging marker of the human connectome. Neuroscientist 22, 486–505. doi: 10.1177/1073858415595004
Jie, B., Shen, D., and Zhang, D. (2014). “Brain connectivity hyper-network for MCI classification,” in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 8674, eds P. Golland, N. Hata, C. Barillot, J. Hornegger, and R. Howe (Cham: Springer).
Jović, A., Brkiæ, K., and Bogunoviæ, N. (2015). “A review of feature selection methods with applications, in: information and communication technology, electronics and microelectronics (MIPRO),” in Proceedings of the 38th International Convention On IEEE, Piscataway, NJ, 1200–1205.
Kamitani, Y., and Tong, F. (2005). Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8, 679–685. doi: 10.1038/nn1444
Khazaee, A., Ebrahimzadeh, A., and Babajani-Feremi, A. (2015a). Application of advanced machine learning methods on resting-state fMRI network for identification of mild cognitive impairment and Alzheimer’s disease. Brain Imaging Behav. 10, 799–817. doi: 10.1007/s11682-015-9448-7
Khazaee, A., Ebrahimzadeh, A., and Babajani-Feremi, A. (2015b). Identifying patients with Alzheimer’s disease using resting-state fMRI and graph theory. Clin. Neurophysiol. 126, 2132–2141. doi: 10.1016/j.clinph.2015.02.060
Killgore, W. D. S., and Yurgelun-Todd, D. A. (2007). Positive affect modulates activity in the visual cortex to images of high calorie foods. Int. J. Neurosci. 117, 643–653. doi: 10.1080/00207450600773848
Kira, K., and Rendell, L. A. (1992). “A practical approach to feature selection,” in Proceedings of the 9th International Workshop on Machine Learning (San Mateo, CA: Morgan Kaufmann).
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M.-C., et al. (2009). Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. Neuroimage 46, 786–802. doi: 10.1016/j.neuroimage.2008.12.037
Kohavi, R., and John, G. H. (1997). Wrappers for feature subset selection. Artif. Intell. 97, 273–324. doi: 10.1016/S0004-3702(97)00043-X
Krusemark, E. A., Novak, L. R., Gitelman, D. R., and Li, W. (2013). When the sense of smell meets emotion: anxiety-state-dependent olfactory processing and neural circuitry adaptation. J. Neurosci. 33, 15324–15332. doi: 10.1523/JNEUROSCI.1835-13.2013
Krystal, J. H., Anticevic, A., Savic, A., Pearlson, G. D., Glahn, D. C., Winkler, A. M., et al. (2013). Characterizing thalamo-cortical disturbances in schizophrenia and bipolar illness. Cereb. Cortex 24, 3116–3130. doi: 10.1093/cercor/bht165
Küblböck, M., Woletz, M., Höflich, A., Sladky, R., Kranz, G. S., Hoffmann, A., et al. (2014). Stability of low-frequency fluctuation amplitudes in prolonged resting-state fMRI. Neuroimage 103, 249–257. doi: 10.1016/j.neuroimage.2014.09.038
Kudo, M., and Sklansky, J. (2000). Comparison of algorithms that select features for pattern classifiers. Pattern Recognit. 33, 25–41. doi: 10.1016/s0031-3203(99)00041-2
Lee, K., Tak, S., and Ye, J. C. (2011). A data-driven sparse GLM for fMRI analysis using sparse dictionary learning with MDL criterion. IEEE Trans. Med. Imaging. 30, 1076–1089. doi: 10.1109/TMI.2010.2097275
Li, J., Yang, R., Xia, K., Wang, T., Nie, B., Gao, K., et al. (2018). Effects of stress on behavior and resting-state fMRI in rats and evaluation of Telmisartan therapy in a stress-induced depression model. BMC Psychiatry 18:337. doi: 10.1186/s12888-018-1880-y
Liu, H., Liu, Z., Liang, M., Hao, Y., Tan, L., Kuang, F., et al. (2006). Decreased regional homogeneity in schizophrenia: a resting state functional magnetic resonance imaging study. Neuroreport 17, 19–22. doi: 10.1097/01.wnr.0000195666.22714.35
Liu, T. T. (2016). Noise contributions to the fMRI signal: an overview. Neuroimage 143, 141–151. doi: 10.1016/j.neuroimage.2016.09.008
Lo, A., Chernoff, H., Zheng, T., and Lo, S.-H. (2015). Why significant variables aren’t automatically good predictors. Proc. Natl. Acad. Sci. U.S.A. 112, 13892–13897. doi: 10.1073/pnas.1518285112
Lohmann, G., Margulies, D. S., Horstmann, A., Pleger, B., Lepsien, J., Goldhahn, D., et al. (2010). Eigenvector centrality mapping for analyzing connectivity patterns in fMRI data of the human brain. PLoS One 5:e10232. doi: 10.1371/journal.pone.0010232
Lopez, R. B., Hofmann, W., Wagner, D. D., Kelley, W. M., and Heatherton, T. F. (2014). Neural predictors of giving in to temptation in daily life. Psychol. Sci. 25, 1337–1344. doi: 10.1177/0956797614531492
Lord, A., Horn, D., Breakspear, M., and Walter, M. (2012). Changes in community structure of resting state functional connectivity in unipolar depression. PLoS One 7:e41282. doi: 10.1371/journal.pone.0041282
Mahmoudi, A., Takerkart, S., Regragui, F., Boussaoud, D., and Brovelli, A. (2012). Multivoxel pattern analysis for fMRI data: a review. Comput. Math. Methods Med. 2012:961257. doi: 10.1155/2012/961257
Maldonado, S., and Weber, R. (2009). A wrapper method for feature selection using Support Vector Machines. Inf. Sci. 179, 2208–2217. doi: 10.1016/j.ins.2009.02.014
Marrelec, G., Horwitz, B., Kim, J., Pélégrini-Issac, M., Benali, H., and Doyon, J. (2007). Using partial correlation to enhance structural equation modeling of functional MRI data. Magn. Reson. Imaging 25, 1181–1189. doi: 10.1016/j.mri.2007.02.012
Marrelec, G., Krainik, A., Duffau, H., Pélégrini-Issac, M., Lehéricy, S., Doyon, J., et al. (2006). Partial correlation for functional brain interactivity investigation in functional MRI. Neuroimage 32, 228–237. doi: 10.1016/j.neuroimage.2005.12.057
McLaren, D. G., Kosmatka, K. J., Kastman, E. K., Bendlin, B. B., and Johnson, S. C. (2010). Rhesus macaque brain morphometry: a methodological comparison of voxel-wise approaches. Methods 50, 157–165. doi: 10.1016/j.ymeth.2009.10.003
Meier, T. B., Desphande, A. S., Vergun, S., Nair, V. A., Song, J., Biswal, B. B., et al. (2012). Support vector machine classification and characterization of age-related reorganization of functional brain networks. Neuroimage 60, 601–613. doi: 10.1016/j.neuroimage.2011.12.052
Mendez-Torrijos, A., Kreitz, S., Ivan, C., Konerth, L., Rösch, J., Pischetsrieder, M., et al. (2018). Snack food as a modulator of human resting-state functional connectivity. CNS Spectr. 23, 321–332. doi: 10.1017/S1092852918000767
Mitchell, T. M., Hutchinson, R., Niculescu, R. S., Pereira, F., Wang, X., Just, M., et al. (2004). Learning to decode cognitive states from brain images. Mach. Learn. 57, 145–175. doi: 10.1371/journal.pone.0178410
Mountrakis, G., Im, J., and Ogole, C. (2011). Support vector machines in remote sensing: a review. ISPRS J. Photogramm. Remote Sens. 66, 247–259. doi: 10.1016/j.isprsjprs.2010.11.001
Mwangi, B., Tian, T. S., and Soares, J. C. (2014). A review of feature reduction techniques in neuroimaging. Neuroinformatics 12, 229–244. doi: 10.1007/s12021-013-9204-3
Neshatian, K., and Zhang, M. (2009). “Pareto front feature selection: using genetic programming to explore feature space,” in Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, New York, NY, 1027–1034.
Norman, K. A., Polyn, S. M., Detre, G. J., and Haxby, J. V. (2006). Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430. doi: 10.1016/j.tics.2006.07.005
Orru, G., Pettersson-Yeo, W., Marquand, A. F., Sartori, G., and Mechelli, A. (2012). Using support vector machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci. Biobehav. Rev. 36, 1140–1152. doi: 10.1016/j.neubiorev.2012.01.004
O’Toole, A. J., Jiang, F., Abdi, H., Pénard, N., Dunlop, J. P., and Parent, M. A. (2007). Theoretical, statistical, and practical perspectives on pattern-based classification approaches to the analysis of functional neuroimaging data. J. Cogn. Neurosci. 19, 1735–1752. doi: 10.1162/jocn.2007.19.11.1735
Paakki, J. J., Rahko, J., Long, X., Moilanen, I., Tervonen, O., Nikkinen, J., et al. (2010). Alterations in regional homogeneity of resting-state brain activity in autism spectrum disorders. Brain Res. 1321, 169–179. doi: 10.1016/j.brainres.2009.12.081
Pereira, F., Mitchell, T., and Botvinick, M. (2009). Machine learning classifiers and fMRI: a tutorial overview. Neuroimage 45, S199–S209. doi: 10.1016/j.neuroimage.2008.11.007
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., et al. (2017). Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat. Rev. Neurosci. 18, 115–126. doi: 10.1038/nrn.2016.167
Poldrack, R. A., Mumford, J. A., and Nichols, T. E. (2011). Handbook of Functional MRI Data Analysis. Cambridge: Cambridge University. doi: 10.1038/nrn.2016.167
Pruim, R. H. R., Mennes, M., Buitelaar, J. K., and Beckmann, C. F. (2015a). Evaluation of ICA-AROMA and alternative strategies for motion artifact removal in resting state fMRI. Neuroimage 112, 278–287. doi: 10.1016/j.neuroimage.2015.02.063
Pruim, R. H. R., Mennes, M., van Rooij, D., Llera, A., Buitelaar, J. K., and Beckmann, C. F. (2015b). ICA-AROMA: a robust ICA-based strategy for removing motion artifacts from fMRI data. Neuroimage 112, 267–277. doi: 10.1016/j.neuroimage.2015.02.064
Pudil, P., Novovièová, J., and Kittler, J. (1994). Floating search methods in feature selection. Pattern Recognit. Lett. 5, 1119–1125. doi: 10.1016/0167-8655(94)90127-9
Salvador, R., Suckling, J., Coleman, M. R., Pickard, J. D., Menon, D., and Bullmore, E. (2005). Neurophysiological architecture of functional magnetic resonance images of human brain. Cereb. Cortex 15, 1332–1342. doi: 10.1093/cercor/bhi016
Sato, J. R., Hoexter, M. Q., Fujita, A., and Rohde, L. A. (2012). Evaluation of pattern recognition and feature extraction methods in ADHD prediction. Front. Syst. Neurosci. 6:68. doi: 10.3389/fnsys.2012.00068
Schnack, H. G., and Kahn, R. S. (2016). Detecting neuroimaging biomarkers for psychiatric disorders: sample size matters. Front. Psychiatry 7:50. doi: 10.3389/fpsyt.2016.00050
Sheline, Y. I., and Raichle, M. E. (2013). Resting state functional connectivity in preclinical Alzheimer’s disease. Biol. Psychiatry 74, 340–347. doi: 10.1093/cercor/bhu259
Sporns, O. (2011). The human connectome: a complex network. Ann. N. Y. Acad. Sci. 1224, 109–125. doi: 10.1111/j.1749-6632.2010.05888.x
Stice, E., Burger, K. S., and Yokum, S. (2013). Relative ability of fat and sugar tastes to activate reward, gustatory, and somatosensory regions1-3. Am. J. Clin. Nutr. 98, 1377–1384. doi: 10.3945/ajcn.113.069443
Stice, E., Spoor, S., Bohon, C., Veldhuizen, M. G., and Small, D. M. (2008). Relation of reward from food intake and anticipated food intake to obesity: a functional magnetic resonance imaging study. J. Abnorm. Psychol. 117, 924–935. doi: 10.1037/a0013600
Stice, E., and Yokum, S. (2016). Gain in body fat is associated with increased striatal response to palatable food cues, whereas body fat stability is associated with decreased striatal response. J. Neurosci. 36, 6949–6956. doi: 10.1523/JNEUROSCI.4365-15.2016
Stice, E., Yokum, S., Bohon, C., Marti, N., and Smolen, A. (2010). Reward circuitry responsivity to food predicts future increases in body mass: Moderating effects of DRD2 and DRD4. Neuroimage 50, 1618–1625. doi: 10.1016/j.neuroimage.2010.01.081
Sundermann, B., and Pfleiderer, B. (2012). Functional connectivity profile of the human inferior frontal junction: involvement in a cognitive control network. BMC Neurosci. 13:119. doi: 10.1186/1471-2202-13-119
Sweet, L. H., Hassenstab, J. J., McCaffery, J. M., Raynor, H. A., Bond, D. S., Demos, K. E., et al. (2012). Brain response to food stimulation in obese, normal weight, and successful weight loss maintainers. Obesity 20, 2220–2225. doi: 10.1038/oby.2012.125
Tahmasian, M., Bettray, L. M., van Eimeren, T., Drzezga, A., Timmermann, L., Eickhoff, C. R., et al. (2015). A systematic review on the applications of resting-state fMRI in Parkinson’s disease: Does dopamine replacement therapy play a role? Cortex 73, 80–105. doi: 10.1016/j.cortex.2015.08.005
Tang, C. C., Poston, K. L., Eckert, T., Feigin, A., Frucht, S., Gudesblatt, M., et al. (2010). Differential diagnosis of parkinsonism: a metabolic imaging study using pattern analysis. Lancet Neurol. 9, 149–158. doi: 10.1016/S1474-4422(10)70002-8
Tang, J., Alelyani, S., and Liu, H. (2014). “Feature selection for classification: a review,” in Data Classification: Algorithms and Applications (Boca Raton, FL: CRC Press), 37–64. Available at: https://asu.pure.elsevier.com/en/publications/feature-selection-for-classification-a-review
Tang, Y., Wang, L., Cao, F., and Tan, L. (2012). Identify schizophrenia using resting-state functional connectivity: an exploratory research and analysis. Biomed. Eng. Online 11:50. doi: 10.1186/1475-925X-11-50
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., et al. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15, 273–289. doi: 10.1006/nimg.2001.0978
van der Laan, L. N., de Ridder, D. T. D., Viergever, M. A., and Smeets, P. A. M. (2011). The first taste is always with the eyes: a meta-analysis on the neural correlates of processing visual food cues. Neuroimage. 55, 296–303. doi: 10.1016/j.neuroimage.2010.11.055
Van Duinkerken, E., Schoonheim, M. M., Sanz-Arigita, E. J., IJzerman, R. G., Moll, A. C., Snoek, F. J., et al. (2012). Resting-state brain networks in type 1 diabetic patients with and without microangiopathy and their relation to cognitive functions and disease variables. Diabetes 61, 1814–1821 doi: 10.2337/db11-1358
Wang, X., Jiao, Y., Tang, T., Wang, H., and Lu, Z. (2013). Altered regional homogeneity patterns in adults with attention-deficit hyperactivity disorder. Eur. J. Radiol. 82, 1552–1557. doi: 10.1016/j.ejrad.2013.04.009
Wang, Z. (2009). A hybrid SVM–GLM approach for fMRI data analysis. Neuroimage 46, 608–615. doi: 10.1016/j.neuroimage.2009.03.016
Wee, C.-Y., Yap, P.-T., Zhang, D., Wang, L., and Shen, D. (2012). Constrained sparse functional connectivity networks for MCI classification. Med. Image Comput. Comput. Assist. Interv. 15(Pt. 2):212–219 doi: 10.1016/j.micinf.2011.07.011
Wee, C. Y., Yap, P. T., Zhang, D., Wang, L., and Shen, D. (2014). Group-constrained sparse fMRI connectivity modeling for mild cognitive impairment identification. Brain Struct. Funct. 219, 641–656. doi: 10.1007/s00429-013-0524-8
Wei, M., Qin, J., Yan, R., Li, H., Yao, Z., and Lu, Q. (2013). Identifying major depressive disorder using Hurst exponent of resting-state brain networks. Psychiatry Res. Neuroimaging 214, 306–312. doi: 10.1016/j.pscychresns.2013.09.008
Wright, H., Li, X., Fallon, N. B., Crookall, R., Giesbrecht, T., Thomas, A., et al. (2016). Differential effects of hunger and satiety on insular cortex and hypothalamic functional connectivity. Eur. J. Neurosci. 43, 1181–1189. doi: 10.1111/ejn.13182
Wu, T., Long, X., Zang, Y., Wang, L., Hallett, M., Li, K., et al. (2009). Regional homogeneity changes in patients with parkinson’s disease. Hum. Brain Mapp. 30, 1502–1510. doi: 10.1002/hbm.20622
Yan, C. G. (2010). DPARSF: a MATLAB toolbox for “pipeline” data analysis of resting-state fMRI. Front. Syst. Neurosci. 4:13. doi: 10.3389/fnsys.2010.00013
Zang, Y., Jiang, T., Lu, Y., He, Y., and Tian, L. (2004). Regional homogeneity approach to fMRI data analysis. Neuroimage 22, 394–400. doi: 10.1016/j.neuroimage.2003.12.030
Zang, Y.-F., He, Y., Zhu, C.-Z., Cao, Q.-J., Sui, M.-Q., Liang, M., et al. (2007). Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI. Brain Dev. 29, 83–91. doi: 10.1016/j.braindev.2006.07.002
Zhang, B., Tian, D., Yu, C., Zhang, J., Tian, X., von Deneen, K. M., et al. (2015). Altered baseline brain activities before food intake in obese men: A resting state fMRI study. Neurosci. Lett. 584, 156–161. doi: 10.1016/j.neulet.2014.10.020
Zhang, L., Samaras, D., Tomasi, D., Volkow, N., and Goldstein, R. (2005). “Machine learning for clinical diagnosis from functional magnetic resonance imaging,” in Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2 Washington, DC. doi: 10.1016/j.neulet.2014.10.020
Zhu, Z., Ong, Y. S., and Dash, M. (2007). Wrapper-filter feature selection algorithm using a memetic framework. IEEE Trans. Syst. Man Cybern. Part B Cybern. 37, 70–76. doi: 10.1109/TSMCB.2006.883267
Zou, Q. H., Zhu, C. Z., Yang, Y., Zuo, X. N., Long, X. Y., Cao, Q. J., et al. (2008). An improved approach to detection of amplitude of low-frequency fluctuation (ALFF) for resting-state fMRI: Fractional ALFF. J. Neurosci. Methods 172, 137–141. doi: 10.1016/j.jneumeth.2008.04.012
Zuo, X. N., Di Martino, A., Kelly, C., Shehzad, Z. E., Gee, D. G., Klein, D. F., et al. (2010). The oscillating brain: complex and reliable. Neuroimage 49, 1432–1445. doi: 10.1016/j.neuroimage.2009.09.037
Zuo, X. N., Ehmke, R., Mennes, M., Imperati, D., Castellanos, F. X., Sporns, O., et al. (2012). Network centrality in the human functional connectome. Cereb. Cortex 22, 1862–1875. doi: 10.1093/cercor/bhr269
Keywords: brain functional activity and connectivity, feature selection, resting-state fMRI, hunger, satiety, support vector machine
Citation: Al-Zubaidi A, Mertins A, Heldmann M, Jauch-Chara K and Münte TF (2019) Machine Learning Based Classification of Resting-State fMRI Features Exemplified by Metabolic State (Hunger/Satiety). Front. Hum. Neurosci. 13:164. doi: 10.3389/fnhum.2019.00164
Received: 04 December 2018; Accepted: 06 May 2019;
Published: 28 May 2019.
Edited by:
Karsten Mueller, Max Planck Institute for Human Cognitive and Brain Sciences, GermanyReviewed by:
Michael Rullmann, Universitätsklinikum Leipzig, GermanyHaiko Schloegl, Leipzig University, Germany
Copyright © 2019 Al-Zubaidi, Mertins, Heldmann, Jauch-Chara and Münte. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Arkan Al-Zubaidi, arkan.al-zubaidi@neuro.uni-luebeck.de; Thomas F. Münte, thomas.muente@neuro.uni-luebeck.de