ORIGINAL RESEARCH article
Volume 17 - 2023 | https://doi.org/10.3389/fnins.2023.1177424
A deep learning framework for identifying Alzheimer's disease using fMRI-based brain network
- 1School of Information Technology Engineering, Tianjin University of Technology and Education, Tianjin, China
- 2Tianjin Key Laboratory of Information Sensing and Intelligent Control, School of Automation and Electrical Engineering, Tianjin University of Technology and Education, Tianjin, China
Background: The convolutional neural network (CNN) is a mainstream deep learning (DL) algorithm, and it has gained great fame in solving problems from clinical examination and diagnosis, such as Alzheimer's disease (AD). AD is a degenerative disease difficult to clinical diagnosis due to its unclear underlying pathological mechanism. Previous studies have primarily focused on investigating structural abnormalities in the brain's functional networks related to the AD or proposing different deep learning approaches for AD classification.
Objective: The aim of this study is to leverage the advantages of combining brain topological features extracted from functional network exploration and deep features extracted by the CNN. We establish a novel fMRI-based classification framework that utilizes Resting-state functional magnetic resonance imaging (rs-fMRI) with the phase synchronization index (PSI) and 2D-CNN to detect abnormal brain functional connectivity in AD.
Methods: First, PSI was applied to construct the brain network by region of interest (ROI) signals obtained from data preprocessing stage, and eight topological features were extracted. Subsequently, the 2D-CNN was applied to the PSI matrix to explore the local and global patterns of the network connectivity by extracting eight deep features from the 2D-CNN convolutional layer.
Results: Finally, classification analysis was carried out on the combined PSI and 2D-CNN methods to recognize AD by using support vector machine (SVM) with 5-fold cross-validation strategy. It was found that the classification accuracy of combined method achieved 98.869%.
Conclusion: These findings show that our framework can adaptively combine the best brain network features to explore network synchronization, functional connections, and characterize brain functional abnormalities, which could effectively detect AD anomalies by the extracted features that may provide new insights into exploring the underlying pathogenesis of AD.
Alzheimer's disease (AD), the most common cause of dementia, is a progressive, degenerative brain disease (Prince, 2018) which is the main cause of disability among older adults, affecting around 50 million people worldwide (Prince et al., 2019). By 2050, it is estimated that 1 in every 85 individuals will be affected by this condition, and the number of affected individuals is projected to double within the next 20 years (Sarraf and Tofighi, 2016). Alzheimer's disease (AD) has been associated with several common abnormalities in the patient's brain: (1) amyloid-related imaging abnormalities (ARIA) (Salloway et al., 2022); (2) damaged areas of nerve fibers and the occurrence of neurofibrillary tangles (Moloney et al., 2021); (3) disconnection of brain network connectivity compared to normal aging individuals (Rosenberg et al., 2020). These pathological or physiological heterogeneities directly or indirectly affect the diagnosis of AD.
To overcome these challenges, the diagnosis of Alzheimer's disease requires meticulous medical evaluation, patient history records, mental state examinations (MMSE), as well as various neurobiological and physical examinations (Duc et al., 2020). Additionally, functional magnetic resonance imaging (fMRI) has rapidly advanced due to its safety, non-invasiveness, and high spatial resolution, making it the most commonly used method for analyzing regular brain changes, different activities, and studying functional connectivity and synchronization between brain regions (Ibrahim et al., 2021). Currently, the identification of AD primarily relies on resting-state functional magnetic resonance imaging (rs-fMRI), which differs from task-based functional magnetic resonance imaging (task-based fMRI). Resting-state fMRI does not affect the individual's ability to recognize and execute task instructions, making it highly useful for studying the cognitive decline caused by AD. During these tests, patients remain in a resting state where they neither engage in any activities nor have any specific thoughts, making the data acquisition task simple and enabling the assessment of routine brain changes (Tong et al., 2017). The rs-fMRI network has been demonstrated to be highly sensitive to AD (Zhang et al., 2021). Magnetic resonance imaging (MRI) and functional magnetic resonance imaging (fMRI) techniques reveal lower MRI signal intensity, reduced brain tissue and cortical volume, enlarged ventricles, affecting brain regions and neural networks associated with cognition, memory, planning, and decision-making (Janghel and Rathore, 2021), ultimately leading to cognitive decline symptoms and decreased complexity of brain networks in AD patients (Cai et al., 2018). Therefore, distinguishing the visual differences between AD data and images from elderly participants with traditional aging effects requires in-depth information and knowledge, combined with additional clinical clues for accurate data classification (e.g., MMSE) (Duc et al., 2020). However, throughout the process, a tool or algorithm is needed to classify imaging data based on rs-fMRI, such as analyzing rs-fMRI data to establish brain network features for quantitative analysis, and differentiate healthy individuals from AD patients for appropriate treatment approaches.
Synchrony has been widely used on the brain network analysis, including the correlation coefficient, Granger causality, phase synchronization index (PSI), et al. (Dauwels et al., 2010b; Cai et al., 2018). Particularly, PSI, as a nonlinear synchrony analysis, can quantify the relationship between the instantaneous phases of AD signals (Zheng and Zhang, 2013; Szymanski et al., 2017). Cai et al. (2018) constructed PSI network by EEG signals, and found that the brain network connectivity of AD patients was abnormal and the small-world property was weakened. Nobukawa et al. (2020) estimated the functional connectivity of AD brains by building brain networks using the phase lag index through EEG signals, and found that brain connectivity and complexity reduced in AD patients. According to the previous synchrony analysis, functional networks of multiple signals could be constructed (Chen et al., 2019b) based on complex network and graph theory. Functional connectivity can describe the relationship between multiple channels in spatially remote brain regions, and quantify their synchronous relationship in the complex brain system (Bi et al., 2020; Sporns, 2022). Therefore, it has been widely applied to fMRI to understand the brain network structure, development and evolution of AD. Zhang et al. (2019) established brain network from fMRI data by using Pearson correlation, and found the loss of small-world properties and the destruction of whole brain tissue network in AD brain. Si et al. (2019) proposed a mutual information brain network model, and revealed that AD brain had characteristics of a longer average time to process information transfer between brain regions and a reduction in data processing capacity than healthy control (HC). Note that functional networks can represent information transmission between different brain regions.
As one of the mainstream deep learning algorithms, convolutional neural networks (CNNs) have made tremendous progress in various fields, such as image recognition and clinical diagnosis (Chen et al., 2019a). CNNs can learn features automatically from large-scale datasets, and then identify global and local patterns (Samek et al., 2016; Mahmud et al., 2018). Li et al. (2019) proposed a hybrid convolutional and recurrent neural network for more detailed hippocampus analysis using structural MRI to recognize AD brain and obtained a high accuracy rate. Venugopalan et al. (2021) used stacked de-noising auto-encoders to extract features from clinical and genetic data of AD, and used CNN for imaging data, then found that hippocampus, amygdala brain areas and the rey auditory verbal learning test (RAVLT) were significant changes in AD brain. Despite variable structures of CNNs, the deep features extracted from CNNs were used to explain their strong recognition power (Bi et al., 2020; Gao et al., 2022). One aim of this study was to find possible explanations of the learned features based on the connectivity matrices derived from PSI matrix of fMRI signals.
In this paper, a new framework, which combined the PSI, CNN and support vector machine (SVM) measures, is proposed and used for recognition and detection of AD from fMRI. Topological and deep features are extracted to quantify the different local and global patterns between AD and HC brain networks. The rest of this paper are organized as follows: In Section 2, the ADNI dataset and data preprocessing are described. In Section 3, the proposed methods for AD classification are illustrated, including methods for constructing brain networks using PSI, CNN structure, deep features and topological feature extraction. Section 4 presents the analytical results, which consists of deep and topological features analysis, statistical analysis, classification analysis, followed by a discussion in Section 5. The conclusions are drawn in Section 6.
In this study, data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu/) (Jack Jr et al., 2008). The primary objective of ADNI is to assess whether a combination of sequential magnetic resonance imaging (MRI), positron emission tomography (PET), other biomarkers, as well as clinical and neuropsychological evaluations, can be used to measure the progression of mild cognitive impairment (MCI) and early-stage Alzheimer's disease (AD). To date, over 1,000 scientific publications have used ADNI data. A number of other initiatives related to AD and other diseases have been designed and implemented using ADNI as a model. Additionally, the fMRI data were collected by a total 3T fMRI data of 118 patients with AD and 127 healthy older adults (HC) were downloaded from the Alzheimer Disease Neuroimaging Initiative (Jack Jr et al., 2008). The main characteristics of the subjects are reported in Table 1, which presents the baseline clinical and demographic variables of both groups. Additionally, the fMRI data were collected by 3T Philips fMRI scanner, where the specific scanning parameters are as follows: TR/TE is 3000ms/30ms, flip Angle is 80°, imaging matrix is 64 × 64, voxel size is 3.31mm × 3.31mm × 3.31mm, 48 slices.All original image files are available to the general scientific community. In this table, AD represents Alzheimer's disease patients, NC represents normal controls, MMSE refers to Mini-Mental State Examination, and CDR represents Clinical Dementia Rating.
2.2. fMRI preprocessing
All preprocessing was performed using the Matlab toolbox SPM12 (https://www.fil.ion.ucl.ac.uk/spm/software/spm12/) and CONN (https://www.nitrc.org/search/?type_of_search=group&q=conn). (1) Data discard: The first 10 time points were discarded for scanner calibration and for subjects to get used to the circumstance; (2) Slice timing: The time offset between adjacent slices were removed by the time slice correction image; (3) Realignment: Motion artifacts generated on different images between individual subjects were eliminated; (4) Normalization: The images after realignment were spatial normalized to the standard EPI template with 3 × 3 × 3 resolution; (5) Smoothing: The normalized images were further spatially smoothed with a Gaussian kernel of 6mm full width at half maximum (FWHM); (6) Filtering: The temporal filtering (0.01Hz<f<0.08 Hz) was applied to the time series of each voxel to reduce the effect of low-frequency drifts and high-frequency noise such as respiratory and cardiac rhythms; (7) ROI signal extraction: the original image was converted into ROI signals using the AAL atlas based 116 Brain Regions (AAL-116) template. Figure 1 shows the raw and pre-processed fMRI.
3.1. Functional brain networks construction
For the identification of functional and structural changes in the brain of AD patients through constructing brain networks, the brain network, as a graph, has two crucial components: nodes and edges. Therefore, this study employs the phase synchronization method to investigate the phase coupling of signals within and between regions of interest (ROIs), which are extracted using the AAL atlas, serving as the nodes. The functional connectivity between two signals is calculated using the phase synchronization index (PSI) and represents the edges for constructing the brain functional network. Due to the sensitivity of Phase Synchronization Index (PSI) to low-frequency signals, which reflect the regulation and synchronization of neural activity, utilizing phase synchronization analysis to process low-frequency signals in fMRI aligns well with relevant concepts and mechanisms in neurophysiology. Therefore, it is a commonly used metric for measuring the strength of functional connectivity in networks. PSI describes the instantaneous phase relationship between time series. Through the Hilbert transformation of the signal sequence (Thuraisingham et al., 2012), that is, the one-dimensional real signal is transformed into a two-dimensional signal on the complex plane. Hx(t) represents Hilbert transform from x(t), which can be defined as follows (Yu et al., 2020):
where pv is the Cauchy principal value. The instantaneous amplitude A(t) and the instantaneous phase φ(t) can be computed by:
The phase difference between two signals is defined as:
where fm and fn are the center frequencies of two signals. Besides, m and n are integers that should satisfy the condition m·fn = n·fm. The phase difference is calculated by setting m = n = 1. Therefore, the PSI can be defined as:
where 〈·〉 refers to the averaging across time. Obviously, PSI is a symmetrical measure (i.e. PSIXY = PSIYX) and within the range [0, 1]. According to graph theory, the PSI matrix in this study can be transformed into an unweighted binary adjacency matrix by applying a given threshold (= 0.3). In the representation of the functional network graph, only connections with PSI values greater than the threshold are realized, resulting in corresponding entries of the matrix being 1, while others are 0. Therefore, the functional connectivity network can be established by PSI, as shown in Figure 2. The 116 × 116 connection matrix is displayed through BrainNet Viewer toolbox (http://www.nitrc.org/projects/bnv/) from full views, according to three brain regions of different sizes of connectivity.
Figure 2. PSI matrix of regional brain network: FL: frontal lobe, LS: limbic system, OL: occipital lobe, PL: parietal lobe, BG: basal ganglia, TL: temporal lobe, C: cerebellum, and CV: cerebellar vermis.
3.2. Convolutional neural network
Convolutional neural network (CNN) is a deep learning network, which can maintain the original data hierarchical structure and apply translation invariance to convert information to feature information for classification (Azulay and Weiss, 2018). It is designed to learn the latent and intrinsic features from images in a supervised way. These features are appropriate in categorizing the anatomical structures and diagnosing the abnormal structures. CNN has a hierarchical structure, where each layer learns feature representations at different levels of abstraction. The lower-level convolutional layers primarily learn low-level features such as edges and textures, while deeper convolutional layers gradually learn higher-level features such as shapes and structural components. This hierarchical feature learning ability enables CNN to effectively capture semantic information in image data. In this study, the convolutional operation in CNN is used to generate extractable deep features, thereby analyzing the object features and texture features of the input data from an image perspective. It provides insights into the input data from both global and local perspectivesIn the convolution layer of a CNN, there are many neurons connected spatially and share the weight and bias, and it uses a set of convolutional kernels to obtain the feature mapping of the input image (Wang et al., 2021). In the convolution layer, the input image is mapped with a set of kernels to generate a new feature map Ck and this process is called convolution. The feature value at location (x, y) in the kth feature map of ith layer, wk is the weight of the kernel, bk is bias of the kernel, is calculated:
where Ii[x, y] is the value at location (x, y) of input image to the ith layer.
Pooling layers are used to reduce the dimensionality of computed feature maps Singh et al. (2020). In CNN models, three commonly used pooling operations are max pooling, min pooling, and average pooling (Satti et al., 2020; Tasnim et al., 2021). After a series of convolutional and pooling operations, the output of the final convolutional layer is propagated to the classification layer. Let Rl be the output of the last convolutional layer, and Rl is computed as follows:
where k represents the number of kernels used in the last convolutional layer, wk denotes the weights of the kernels, f(Il−1) represents the activation values of the (l − 1) convolutional layer, and frelu represents the computation through the ReLU activation function.
Finally, the classification layer uses the softmax activation function to predict the class of an input image. The softmax function calculates the class probability value (Wang et al., 2018):
Figure 3 shows the structures of proposed CNNs in this paper, which has two convolutional (Conv) layers and two fully connected (Fc) layers: (1) In the first Conv layer, there are 8 Conv kernels/filters with a kernel size of 3 × 3, stride of 2, pad of 2, and max pooling size of 2; (2) In Conv2, there are 16 Conv kernels/filters with a kernel size 3 × 3, stride 1 and pad 1; (3) ReLU activation functions are used for all Conv/Fc layers and sigmoid activation functions for output layers, respectively.
3.3. Feature extraction
Deep features can extract both local and global feature information from images, while brain network topological features provide connectivity patterns and functional associations between brain regions. Combining these two types of features can better describe the functional and structural information in brain imaging data, thereby enhancing the performance of disease classification. Additionally, integrating deep features with brain network features can provide a more comprehensive and intuitive visual analysis, aiding researchers in understanding the intrinsic characteristics and brain functionality of brain imaging data. This section describes the eight topological features to be extracted: degree (DG), node betweenness (NB), edge betweenness (EB), assortativity (AS), clustering coefficient (CC), global efficiency (GE), local efficiency (LE) and average characteristic path length (APL), see details in Supplement file. In addition, eight deep features F1-F8 were automatically extracted from the last convolutional layer of 2D-CNN.
3.4. Statistical analysis
Single-factor analysis of variance (ANOVA) was used to assess the statistical differences between the AD and HC groups. ANOVA returns several statistical measures, including the sum of squares (SS), degrees of freedom (df), mean squares (MS = SS/df), F-value, and P-value. The F-value is the ratio of the between-group mean squares (MSB) to the within-group mean squares (MSW), indicating the extent of between-group differences (Wang et al., 2015). The P-value is inversely related to the F-value and represents the probability of error when between-group differences are not significant. In statistics, it is generally considered significant when P<0.01. A larger F-value or a smaller P-value indicates more significant between-group differences, and vice versa.
In addition, the receiver operating characteristic (ROC) curve was used to visually evaluate the ability of these features to distinguish AD patients from normal controls. Through variance analysis, the between-group differences were found to be significant. This statistical method summarizes the performance of two classifiers over a range of possible thresholds from 0 to 1. Sensitivity refers to the true positive rate, while specificity refers to the true negative rate.
Among them, false negatives (FN) are the number of AD patients misclassified as normal controls, and false positives (FP) are the number of normal controls misclassified as patients. True positives (TP) and true negatives (TN) are the counts of correctly identified AD patients and normal controls, respectively. Accuracy quantifies the total number of correctly classified subjects and is defined as follows:
The area under the ROC curve (AUC) represents the performance of the classification. For a perfect classifier, the AUC is 1, while an AUC of 0.5 indicates a test with no value (Ferraris, 2019).
The application process for AD detection is as follows:
(1) Preprocessing of fMRI Images: Apply standard preprocessing steps such as motion correction, slice-timing correction, and spatial normalization to the fMRI images. These steps enhance the data quality and provide the raw data for subsequent analysis.
(2) Extraction of ROI Signals using AAL Atlas: Extract the average time series signals from each region of interest (ROI) using the Automated Anatomical Labeling (AAL) atlas. These signals represent the brain activity in the corresponding regions.
(3) Calculation of PSI to Measure Functional Connectivity: Calculate the PSI to quantify the functional connectivity between brain regions. Construct a connectivity matrix based on PSI values and extract 8 topological features (DG, NB, EB, AS, CC, GE, LE, FL) from the network. These measures represent the overall connectivity and information exchange capabilities of the brain network.
(4) Analysis and Extraction of Deep Features using CNN: Train a CNN model on the PSI dataset to learn discriminative patterns in fMRI images. Extract 8 deep features from the learned CNN model. These features provide intuitive representations of object characteristics and texture features within the PSI network, offering new insights into the affected regions in the brain associated with AD from a global and local perspective.
(5) Feature Selection and Classification: Select relevant features from the topological features derived from the PSI brain network and the deep features extracted from the CNN model for classification. Evaluate the impact of using the same features or different feature combinations on the classification accuracy. Employ a SVM classifier to classify AD and non-AD samples. Apply 5-fold cross-validation to ensure the robustness of the classification results. The training and test sets are split with a ratio of 4:1. This process aids in model training and evaluation while ensuring the reliability of the classification results.
The intelligent detection process is illustrated in Figure 4.
4.1. Analysis of brain network
To study the cognitive impairment-related brain mechanisms of AD, the PSI was used to construct the functional connectivity after selecting the same threshold, so the adjacency matrix was obtained (Figure 5A), and the brain network connection corresponding to the matrix was displayed through BrainNet Viewer toolbox from full views (frontal, profile, top), as shown in Figure 5B. Compared to the HC group, the brain networks of the AD group were abnormal. To be specific, the values of PSI matrix of AD group were ranged in [0, 0.4], which were much smaller than that of the HC group ranged in [0.6, 1], reflected in the colder color of the PSI matrix in AD group. Correspondingly, connections between nodes in the AD group were sparser and weaker than that in the HC group, which were shown by the thickness and density of the connecting lines, especially in the frontal lobe (FL), parietal lobe (PL) and basal ganglia (BG), marked by red and green color. That is, the AD group had weaker ability of information transmission and less activation in local brain regions. This partial loss of functional connectivity across brain regions results from degenerative changes in the AD brains.
Figure 5. Comparison of whole-brain functional connectivity between the AD and HC group: (A) connectivity matrices of the two groups. (B) Connectivity full views (frontal, profile, top) of the two groups.
In order to further study the difference between the AD and HC group, eight topological features of brain network matrix: DG, NB, EB, AS, CC, GE, LE, APL were extracted and performed one-way ANOVA analysis, as shown in Figure 6, Table 2. In Figure 6, box indicates the first and third quartiles, red line indicates the median, whiskers mark the minimum and the maximum, notches indicate the 95% confidence interval, and the red crosses indicate outliers. Moreover, all the values of features in this figure were normalized to the range of [0, 1]. All the topological features except for the AS presented significant differences between AD and HC groups, indicated by the P-values of the seven features smaller than 0.01. The values of DE, CC, LE and GE of the AD group are smaller than that of the HC group, suggesting that the connectivity, aggregation and compensatory capacity of the AD network was decreased, and thus the integration ability between AD brain regions was poor; while the values of NB, EB and APL of AD group increased, indicating the small-world property of brain networks was weakened, the information transfer capacity and the rate of information transmission between network nodes of AD network were decreased.
4.2. Analysis of 2D-CNN
To illustrate the effectiveness of CNN on classifying the brain connectivity of AD and HC, and determine the certain lesion region in the AD brain image, the visualization feature maps of convolutional layers by CNN were plotted in Figure 7 to exhibit the abnormality of functional connectivity of AD patients. Figures 7A, B showed the feature maps of AD and HC PSI matrix, respectively. Feature maps generated from convolutional layers 1 and 2 of the CNN model were extracted to show how the convolutional layers work. In each convolutional layer, there were multiple kernels, for which a feature map could be generated by sliding the filters over the PSI matrix (convolution). Therefore, the features map dimension changes drastically from one convolutional layer to the next according to the filters number. In Figure 7, the left, middle and right panels represented the input layer, convolutional layer 1 and convolutional layer 2, respectively. After different convolution operation in convolutional layers, the feature map changed, such as the dimension of features maps of different layers. For example, the input layer was 116 × 116, and convolutional layer 1 and convolutional layer 2 were 58 × 58 and 29 × 29, respectively. However, object features and texture features can preserve certain key information of the original input data of the PSI matrix. Compared to the HC group, the regions of FL, PL, and BG of AD in convolutional layer 1 were much lighter in color, indicating abnormalities in connectivity may be in these brain regions. The same anomaly also occurred in convolutional layers 1 and 2, marked by red and green colors. Thus, the comparison of the feature maps of different layers of the CNN and the exploration of the feature maps could reveal some interesting insights.
Figure 7. The feature maps of 2D-CNN for (A) AD group and (B) HC group. Panels on the left, middle and right represented the input layer, convolutional layer 1 and convolutional layer 2 of 2D-CNN model.
Eight deep features, which could be divided into two categories: object features and texture features, extracted from the last convolutional layer of the CNN: after normalization, F1-F8 were further extracted and one-way ANOVA analysis was performed to characterize the abnormality of AD brain, as shown in Figure 8, Table 3. All the eight topological features, had obvious group differences with P-values smaller than 0.01. Compared to the HC group, F1, F3, F4, F6 and F8 of the AD group were decreased, while F2, F5, F7 of were increased. Notably, the deep features in Table 3 could better display the local and global abnormal regions in AD brain than topological features in Table 2 with larger F-values and smaller P-values statistically.
4.3. Correlation analysis of features
To gain more insight into the relationship between the different measures, the correlations between those measures were calculated using Pearson correlation. Figure 9 showed the correlation between the deep features and the topological features in AD and HC, where the colder the color, the weaker the correlation, and vice versa. Seen from Figure 9, the correlation matrix could be divided into four parts: top left, bottom right, and top right which was symmetrical with the bottom left part, which represented the correlation between any two topological features: DG, NB, EB, AS, CC, GE, LE, APL, the correlation between any two deep features: F1 F8 and the correlation between topological features and deep features, respectively. For the part of one, it could find that there was strong correlation among the topological features: (1) DE, CC, GE and LE; (2) NB, EB and APL; For the part of three, the correlation between deep features in the HC group was much larger than in theAD group, especially for F2 with F5, F3 with F4/F6/F8, and F5 with F7/ F8; for the part of two, the correlations of deep features F2, F6, F8 with the topological features except AS were relatively strong. Clearly, the reduced feature correlation in AD patients compared to the HC group is due to the effect of reduced inter-regional brain correlations. In addition, the relevance of deep features and topological features can capture the changes in the global pattern of AD from the structural and functional brain network loss and convolutional kernels, and thus can more effectively characterize AD than topological or deep features alone.
Figure 9. The Pearson correlation matrix of (A) AD and (B) HC group. Region 1 shows correlation between topological features; Region 2 shows correlation between topological features and deep features; Region 3 shows correlation between deep features.
To further demonstrate the relevance between the deep and topological features and investigate the effect on distinguishing AD from HC, scatter plots are plotted to display the correlation results between the deep and topological features in Figure 10, which was composed of 64 subfigures, where each row represented the eight deep features and each column represented the eight topological features. After normalization, all eight deep features were correlated with any of the eight topological measures, which indirectly verifies the effect of deep features. In each subplot, the red and green points represented the AD and HC group, respectively. In order to find the difference between the two groups of data more intuitively, the black line was fitted to show the discrimination of different groups. When two features were combined, the normalized values of the features were clustered in a distribution between the two groups and can thus be easily discriminated. Interestingly, the topological feature AS, which has no significant group differences originally, could become effective attributing to depth features (marked with blue rectangle), thereby suggesting that the CNN model can learn some features that represent the global patterns of the brain network that are not described by the topological features.
Figure 10. The correlation results between the deep features and the topology features of brain connectivity, where the blue dashed box represents topology feature AS with no significant difference (P-values> 0.05). The black line denoted the discrimination of different groups.
To further quantitatively investigate whether the correlation between the deep and topological features could effectively distinguish AD from HC, the classification accuracy is shown in Tables 4, 5. The corresponding accuracy of each subplot could distinguish between AD and HC, with higher accuracy for the combined features formed by the deep features F1, F2, F3, F4 and F7 with topological features. In summary, when data passing through successive CNN layers, the complexity of the data decreases as the size of the output data decreases, but the correlation with the topological features is preserved, indicating that the global and local patterns on the brain network are preserved during CNN learning. Therefore, it is feasible to improve the classification accuracy of AD and HC by extracting the depth features of the last convolutional layer.
Table 5. The results of classification accuracy between the deep features and the topology features.
4.4. Classification analysis
The results in the previous section show that there is a certain correlation between TF and BF. Therefore, PSI can be combined with 2D-CNN for classification recognition. After 5-fold crossover, eight topological features and eight depth features are combined with these sixteen features to be fed into an SVM classifier for classification. As shown in Figure 11, Table 6, the three methods of topological features and deep features could distinguish between AD and HC, but the classification by topological features is generally effective. The combination of hybrid method had the best ACC, AUC, SEN and SPE for classifying AD and HC. It indicated that the accuracy of AD recognition can be greatly improved by combining the brain structural features extracted by PSI brain network and the whole brain global pattern features extracted by 2D-CNN. All of them could be used to explore the underlying pathogenesis of AD, which may provide new insights for analysis.
Figure 11. The ROC curve of PSI brain network features, 2D-CNN deep features and PSI and 2D-CNN combination features. TPR is true positive rate. FPR is false positive rate.
Table 6. ACC and AUC classified by SVM after using 8 features for 2D-CNN models and PSI brain network.
Alzheimer's disease is a progressive and incurable neurodegenerative disease, which leads to degradation of cognitive (Chhatwal et al., 2018). Although the pathological mechanisms underlying the neuropathological changes in AD remain unclear, some neuropathological hallmarks have been reported, such as neuritic extracellular amyloid plaques and intracellular neurofibrillary tangles, which would lead to energy slowing down, decrease of complexity and synchrony, and disconnection of the connectivity (Dauwels et al., 2010a). By complex networks theory and graph theory, recent research on brain functional networks extracted from fMRI (Table 7) and the structural networks extracted from MRI have shown that AD patients have different degrees of local or even global topology abnormalities and “disconnection” symptoms (Xu et al., 2008; Sanz-Arigita et al., 2010; Córdova-Palomera et al., 2017; Khatri and Kwon, 2022; Xing et al., 2022).
Additionally, some consistent conclusions of AD networks were obtained that connectivity between brain regions decreased (Si et al., 2019; Zhang et al., 2019) and small-world attribute declined (Yu et al., 2018), specific brain hubs degenerated, clustering coefficients reduced and path lengths closed to the values of random networks (Yu et al., 2018; Ferreira et al., 2019). The connection and aggregation of the network was decreased, and the compensatory ability of the network was reduced, which result in blocking of information transmission. Besides, some specific lesion areas in AD brain have been suggested by complex network analysis, and the roles of frontal lobe (FL), parietal lobe (PL) and basal ganglia (BG) in the development and progression of AD have only recently begun to receive attention. (Ferreira et al., 2019) found that the frontal cortex loss its modular connectivity with the subcortical gray matter structures and nodal global efficiency was decreased in the middle frontal cortex, which was associated with decreased motor function and cognitive function in AD patients. Srivishagan et al. (2020) applied the group measures to the structural connectomes of AD subjects based on the brain-lobes and demonstrated that the strength of the parietal lobes has been heavily affected in AD, and the impairment of the parietal lobe was reflected in the reduced ability to integrate sensations. Chen et al. (2022) found that AD patients predominantly decreased connectivity in the basal ganglia, demonstrating that the motor control and motor learning were declined.
Principally, CNN can identify and amplify these pathological changes and lesions on the network, due to the number of involved brain regions increases as the receptive field expands (Chen et al., 2019a). For lower layers of the CNN, a smaller receptive field may capture local patterns of the brain network, while a larger receptive field for higher layers may reflect global patterns. In CNN models, 2D-CNN and 3D-CNN are commonly used in the fMRI image recognition, so the above two CNNS were also used and compared in this paper, as shown in Figure 12, Table 8. Since 2D-CNN has only two dimensions, the convolutional kernel can move in both directions, which requires less convolutional computation and is more efficient compared to 3D-CNN. 2D-CNN can be used to amplify abnormalities in AD brain networks. In particular, the CNN structure was applied to PSI matrix (2D data) and fMRI data after pre-processing (3D data) for recognition, and the results of the accuracy and loss rate were showed in Figure 12, Table 8. For both types of CNNs, the test set shows the same trend as the training set with less fluctuations and smoother curves, and training accuracy is 100% after 100 iterations, which validates the effectiveness for both 2D-CNN and 3D-CNN. For the training set (Figures 12A, C), the 2D-CNN model could achieve 95.833% of test accuracy, which was higher than 3D-CNN; for the loss rate (Figures 12B, D), the loss rate of 2D-CNN was 33.396%, greatly lower than that of 3D-CNN. Clearly, 2D-CNN after constructing the brain network through PSI may achieve better recognition efficiency than 3D-CNN directly identifying pre-processed fMRI data.
Figure 12. The accuracy and loss rates for training and testing convergence of 2D-CNN, and 3D-CNN, where blue curves represent the test set, and orange curves represent the training set.
The analysis of the features shows eight deep features extracted from last convolutional layer in 2D-CNN had high correlations (p < 0.01) with seven topological features except AS (shown in Figure 12), thereby indicating that the global patterns described by the eight deep features were of certain importance in the AD identification problem and the CNN models excavated those patterns. However, it is still impossible to understand CNN from features alone, and the CNN model remains a black box. The underlying pathological relationship between these deep features and AD is still uncertain. As a result, further targeted clinical therapies cannot be pursued based on these findings. In contrast, with a precise methodological definition, topological features can reveal which lesions are present in the brain network of AD and can be helpful in assisting clinical diagnosis.
However, there were some limitations in this study. (1) The depth of the CNN model proposed in this paper and the parameters between the layers have optimization possibilities. (2) The courses of AD patients (LMCI, EMCI, et al.) was not taken into account, all classification results presented in this paper were two classifications for AD and HC. In the following studies, we will conduct more comprehensive analyses and use multivariate classification methods involving different AD processes. (3) It is important to acknowledge that overfitting was not extensively addressed in this study, which may limit the applicability of the proposed methods described in the paper.
In future work, we can enhance our research through various means, such as employing intelligent algorithms for parameter optimization of CNN, incorporating LSTM modules to optimize the structure of CNN, using Softmax activation function or multiple output nodes to perform multi-classification of different stages of Alzheimer's disease (AD) patients. At last, it would be valuable to explore additional strategies to combat overfitting, such as data augmentation techniques, dropout regularization, or ensemble methods. Considering the potential impact of overfitting on the reliability and generalizability of the results, it is important for future studies to prioritize addressing this issue and conducting thorough validation on independent datasets. By doing so, the validity and applicability of the proposed approach can be further substantiated.
In this paper, a new framework using PSI and 2D-CNN was applied on fMRI data in order to investigate the abnormalities of AD brain. Firstly, after pre-processing, the PSI analysis was applied to ROI signals for synchronization analysis thus mapped into a PSI network, and eight topological features were extracted. Compared with HC group, the brain network connectivity of AD group was sparser, connection strength was weaker, and the small-world property of the network was reduced. Specifically, the information transmission capacity and the rate of information transmission between network nodes of AD network was decreased. Secondly, the 2D-CNN was further applied to the PSI matrix to explore the local and global patterns of the brain network by extracting eight deep features from the 2D-CNN convolutional layers, which shows significant group differences by ANOVA analysis. Besides, the focus area could be exhibited from the feature map of 2D-CNN, such as frontal lobe (FL), parietal lobe (PL) and basal ganglia (BG). Then correlation analysis was implemented by Pearson analysis. As a result, topological features and deep features were correlated, verifying the possibility and effectiveness of the combination of the PSI and 2D-CNN methods. Finally, SVM was applied to classify AD fMRI by combing the PSI and 2D-CNN measures, which has the best performance (accuracy: 98.869%) compared to only using PSI or 2D-CNN.
The result reveals that the 2D-CNN model found some deep features that showed global patterns and texture patterns representing the abnormal connectivity in the brain network of AD patients and other patterns that could not described by the topological features. This paper demonstrates that 2D-CNN has the amazing learning ability and could identify potential biases in AD patients by extracting deep features of convolutional layers. Those features may provide new insights into the underlying pathogenesis of AD.
Data availability statement
Publicly available datasets were analyzed in this study. This data can be found here: Alzheimer's Disease Neuroimaging Initiative: ADNI https://adni.loni.usc.edu/.
Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.
RW, QH, and HW designed the study and analyzed the results. RW, QH, and LS provided the code support of algorithm. QH carried out the experiments. RW and QH wrote the original draft. RW, QH, CH, and YC polished the sentences of the article. RW, QH, HW, and YC modified the format of figures and tables. QH and HW finalize format of the manuscript. RW, CH, and YC provided the funding. All authors contributed to the article and approved the submitted version.
This research was funded by the Natural Science Foundation of Tianjin (grant No. 22JCYBJC00470), Tianjin Science and Popularization of Science and Research Development Project (grant No. 22KPXMRC00170), Science and Technology Think Tank Youth Talent Program, China (grant No. 20220615ZZ07110153), Tianjin Municipal Education Commission Scientific Research Program (grant No. 2022KJ106), Tianjin Municipal Special Program of Talent Development for Excellent Youth Scholars (grant No. TJTZJHQNBJRC-2-2), and National Natural Science Foundation of China Project (grant No. 62103301).
We would like to express our gratitude to the older people and the young people who assisted in completing the experiment.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2023.1177424/full#supplementary-material
Bi, X., Zhao, X., Huang, H., Chen, D., and Ma, Y. (2020). Functional brain network classification for alzheimer's disease detection with deep features and extreme learning machine. Cognit. Comput. 12, 513–527. doi: 10.1007/s12559-019-09688-2
Cai, L., Wei, X., Wang, J., Yu, H., Deng, B., and Wang, R. (2018). Reconstruction of functional brain network in alzheimer's disease via cross-frequency phase synchronization. Neurocomputing 314, 490–500. doi: 10.1016/j.neucom.2018.07.019
Chen, J., Wang, H., Wang, Q., and Hua, C. (2019b). Exploring the fatigue affecting electroencephalography based functional brain networks during real driving in young males. Neuropsychologia 129, 200–211. doi: 10.1016/j.neuropsychologia.2019.04.004
Chen, P., Yao, H., Tijms, B. M., Wang, P., Wang, D., Song, C., et al. (2022). Four distinct subtypes of alzheimer's disease based on resting-state connectivity biomarkers. Biol Psychiatry. 93, 759–769. doi: 10.1016/j.biopsych.2022.06.019
Chhatwal, J. P., Schultz, A. P., Johnson, K. A., Hedden, T., Jaimes, S., Benzinger, T. L., et al. (2018). Preferential degradation of cognitive networks differentiates alzheimer's disease from ageing. Brain 141, 1486–1500. doi: 10.1093/brain/awy053
Córdova-Palomera, A., Kaufmann, T., Persson, K., Alnaes, D., Doan, N. T., Moberget, T., et al. (2017). Disrupted global metastability and static and dynamic brain connectivity across individuals in the alzheimer's disease continuum. Sci. Rep. 7, 1–14. doi: 10.1038/srep40268
Dauwels, J., Vialatte, F., Musha, T., and Cichocki, A. (2010b). A comparative study of synchrony measures for the early diagnosis of alzheimer's disease based on eeg. Neuroimage 49, 668–693. doi: 10.1016/j.neuroimage.2009.06.056
Duc, N. T., Ryu, S., Qureshi, M. N. I., Choi, M., Lee, K. H., and Lee, B. (2020). 3d-deep learning based automatic diagnosis of alzheimer's disease with joint mmse prediction using resting-state fmri. Neuroinformatics 18, 71–86. doi: 10.1007/s12021-019-09419-w
Ferraris, V. A. (2019). Commentary: should we rely on receiver operating characteristic curves? from submarines to medical tests, the answer is a definite maybe! J. Thoracic Cardiovasc. Surg. 157, 2354–2355. doi: 10.1016/j.jtcvs.2018.10.100
Ferreira, D., Pereira, J. B., Volpe, G., and Westman, E. (2019). Subtypes of alzheimer's disease display distinct network abnormalities extending beyond their pattern of brain atrophy. Front. Neurol. 10, 524. doi: 10.3389/fneur.2019.00524
Gao, L., Hu, Z., Li, R., Lu, X., Li, Z., Zhang, X., et al. (2022). Multi-perspective feature extraction and fusion based on deep latent space for diagnosis of Alzheimer's diseases. Brain Sci. 12, 1348. doi: 10.3390/brainsci12101348
Ibrahim, B., Suppiah, S., Ibrahim, N., Mohamad, M., Hassan, H. A., Nasser, N. S., et al. (2021). Diagnostic power of resting-state fmri for detection of network connectivity in Alzheimer's disease and mild cognitive impairment: a systematic review. Hum. Brain Mapp. 42, 2941–2968. doi: 10.1002/hbm.25369
Jack Jr, C. R., Bernstein, M. A., Fox, N. C., Thompson, P., Alexander, G., Harvey, D., et al. (2008). The alzheimer's disease neuroimaging initiative (adni): Mri methods. J. Magnetic Reson. Imag. 27, 685–691. doi: 10.1002/jmri.21049
Khatri, U., and Kwon, G.-R. (2022). Alzheimer's disease diagnosis and biomarker analysis using resting-state functional mri functional brain network with multi-measures features and hippocampal subfield and amygdala volume of structural mri. Front. Aging Neurosci. 14, 818871. doi: 10.3389/fnagi.2022.818871
Li, F., Liu, M., and Alzheimer's Disease Neuroimaging Initiative. (2019). A hybrid convolutional and recurrent neural network for hippocampus analysis in Alzheimer's disease. J. Neurosci. Methods 323:108–118. doi: 10.1016/j.jneumeth.2019.05.006
Mahmud, M., Kaiser, M. S., Hussain, A., and Vassanelli, S. (2018). Applications of deep learning and reinforcement learning to biological data. IEEE Trans. Neural Netw. Learn. Syst. 29(6):2063–2079. doi: 10.1109/TNNLS.2018.2790388
Moloney, C. M., Lowe, V. J., and Murray, M. E. (2021). Visualization of neurofibrillary tangle maturity in alzheimer's disease: a clinicopathologic perspective for biomarker research. Alzheimer's Dement. 17, 1554–1574. doi: 10.1002/alz.12321
Nobukawa, S., Yamanishi, T., Kasakawa, S., Nishimura, H., Kikuchi, M., and Takahashi, T. (2020). Classification methods based on complexity and synchronization of electroencephalography signals in alzheimer's disease. Front. Psychiatry 11, 255. doi: 10.3389/fpsyt.2020.00255
Prince, M., Wimo, A., Guerchet, M., Ali, G., Wu, Y., Prina, M., et al. (2019). Alzheimer's Disease International: World Alzheimer Report 2015: The Global Impact of Dementia: An Analysis of Prevalence, Incidence, Cost and Trends. 2015. London: Alzheimer's Disease International.
Rosenberg, A., Mangialasche, F., Ngandu, T., Solomon, A., and Kivipelto, M. (2020). Multidomain interventions to prevent cognitive impairment, alzheimer's disease, and dementia: From finger to world-wide fingers. J. Prevent. Alzheimer's Dis. 7, 29–36. doi: 10.14283/jpad.2019.41
Salloway, S., Chalkias, S., Barkhof, F., Burkett, P., Barakos, J., Purcell, D., et al. (2022). Amyloid-related imaging abnormalities in 2 phase 3 studies evaluating aducanumab in patients with early alzheimer disease. JAMA Neurol. 79, 13–21. doi: 10.1001/jamaneurol.2021.4161
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., and Müller, K.-R. (2016). Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Netw. Learn. Syst. 28, 2660–2673. doi: 10.1109/TNNLS.2016.2599820
Sanz-Arigita, E. J., Schoonheim, M. M., Damoiseaux, J. S., Rombouts, S. A., Maris, E., Barkhof, F., et al. (2010). Loss of “small-world” networks in alzheimer's disease: graph analysis of fmri resting-state functional connectivity. PLoS ONE 5, e13788. doi: 10.1371/journal.pone.0013788
Sarraf, S., and Tofighi, G. (2016). Classification of Alzheimer's disease structural MRI data by deep learning convolutional neural networks. arXiv [Preprint]. arXiv: 1607.06583. doi: 10.48550/arXiv.1607.06583
Si, S., Wang, B., Liu, X., Yu, C., Ding, C., and Zhao, H. (2019). Brain network modeling based on mutual information and graph theory for predicting the connection mechanism in the progression of alzheimer's disease. Entropy 21, 300. doi: 10.3390/e21030300
Srivishagan, S., Perera, A. A., Hojjat, A., Ratnarajah, N., and Initiative, A. D. N. (2020). Brain network measures for groups of nodes: Application to normal aging and alzheimer's disease. Brain Connect. 10, 316–327. doi: 10.1089/brain.2020.0747
Szymanski, C., Pesquita, A., Brennan, A. A., Perdikis, D., Enns, J. T., Brick, T. R., et al. (2017). Teams on the same wavelength perform better: inter-brain phase synchronization constitutes a neural substrate for social facilitation. Neuroimage 152, 425–436. doi: 10.1016/j.neuroimage.2017.03.013
Tasnim, Z., Chakraborty, S., Shamrat, F., Chowdhury, A. N., Nuha, H. A., Karim, A., et al. (2021). Deep learning predictive model for colon cancer patient using cnn-based classification. Int. J. Adv. Comput. Sci. Appl. 12, 0120880. doi: 10.14569/IJACSA.2021.0120880
Thuraisingham, R. A., Tran, Y., Craig, A., and Nguyen, H. (2012). “Frequency analysis of eyes open and eyes closed eeg signals using the hilbert-huang transform,” in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. San Diego: IEEE, 2865–2868. doi: 10.1109/EMBC.2012.6346561
Tong, T., Gray, K., Gao, Q., Chen, L., Rueckert, D., Initiative, A. D. N., et al. (2017). Multi-modal classification of alzheimer's disease using nonlinear graph fusion. Pattern Recognit. 63, 171–181. doi: 10.1016/j.patcog.2016.10.009
Venugopalan, J., Tong, L., Hassanzadeh, H. R., and Wang, M. D. (2021). Multimodal deep learning models for early detection of alzheimer's disease stage. Sci. Rep. 11, 1–13. doi: 10.1038/s41598-020-74399-w
Wang, M., Lu, S., Zhu, D., Lin, J., and Wang, Z. (2018). “A high-speed and low-complexity architecture for softmax function in deep learning,” in 2018 IEEE asia pacific conference on circuits and systems (APCCAS). Chengdu: IEEE, 223–226. doi: 10.1109/APCCAS.2018.8605654
Wang, Z., Zhao, W., Du, W., Li, N., and Wang, J. (2021). Data-driven fault diagnosis method based on the conversion of erosion operation signals into images and convolutional neural network. Process Safety Environm. Prot. 149, 591–601. doi: 10.1016/j.psep.2021.03.016
Xu, Y., Xu, G., Wu, G., Antuono, P., Rowe, D. B., and Li, S.-J. (2008). The phase shift index for marking functional asynchrony in alzheimer's disease patients using fmri. Magn. Reson. Imaging 26, 379–392. doi: 10.1016/j.mri.2007.07.007
Yu, H., Lei, X., Song, Z., Wang, J., Wei, X., and Yu, B. (2018). Functional brain connectivity in Alzheimer's disease: an eeg study based on permutation disalignment index. Physica A. 506, 1093–1103. doi: 10.1016/j.physa.2018.05.009
Yu, H., Zhu, L., Cai, L., Wang, J., Liu, C., Shi, N., et al. (2020). Variation of functional brain connectivity in epileptic seizures: an eeg analysis with cross-frequency phase synchronization. Cogn. Neurodyn. 14, 35–49. doi: 10.1007/s11571-019-09551-y
Zhang, T., Zhao, Z., Zhang, C., Zhang, J., Jin, Z., and Li, L. (2019). Classification of early and late mild cognitive impairment using functional brain network of resting-state fmri. Front. Psychiatry 10, 572. doi: 10.3389/fpsyt.2019.00572
Keywords: fMRI, Alzheimer's disease, 2D-CNN, phase synchronization index, ROI
Citation: Wang R, He Q, Han C, Wang H, Shi L and Che Y (2023) A deep learning framework for identifying Alzheimer's disease using fMRI-based brain network. Front. Neurosci. 17:1177424. doi: 10.3389/fnins.2023.1177424
Received: 01 March 2023; Accepted: 24 July 2023;
Published: 08 August 2023.
Edited by:Yutong Bai, Capital Medical University, China
Reviewed by:Haitao Yu, Tianjin University, China
Martin Dyrba, Helmholtz Association of German Research Centers (HZ), Germany
Copyright © 2023 Wang, He, Han, Wang, Shi and Che. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yanqiu Che, firstname.lastname@example.org
†These authors have contributed equally to this work