Multimodal Brain Network Jointly Construction and Fusion for Diagnosis of Epilepsy

Brain network analysis has been proved to be one of the most effective methods in brain disease diagnosis. In order to construct discriminative brain networks and improve the performance of disease diagnosis, many machine learning–based methods have been proposed. Recent studies show that combining functional and structural brain networks is more effective than using only single modality data. However, in the most of existing multi-modal brain network analysis methods, it is a common strategy that constructs functional and structural network separately, which is difficult to embed complementary information of different modalities of brain network. To address this issue, we propose a unified brain network construction algorithm, which jointly learns both functional and structural data and effectively face the connectivity and node features for improving classification. First, we conduct space alignment and brain network construction under a unified framework, and then build the correlation model among all brain regions with functional data by low-rank representation so that the global brain region correlation can be captured. Simultaneously, the local manifold with structural data is embedded into this model to preserve the local structural information. Second, the PageRank algorithm is adaptively used to evaluate the significance of different brain regions, in which the interaction of multiple brain regions is considered. Finally, a multi-kernel strategy is utilized to solve the data heterogeneity problem and merge the connectivity as well as node information for classification. We apply the proposed method to the diagnosis of epilepsy, and the experimental results show that our method can achieve a promising performance.


INTRODUCTION
Brain network analysis has been widely applied to analysis and diagnosis of brain diseases, such as epilepsy and Alzheimer's disease (Osipowicz et al., 2016). It mainly benefits from more and more neuroimaging technologies that can give us insight into the neuroanatomical correlates of cognition. functional MRI (fMRI) and diffusion tensor imaging (DTI) are of remarkable importance and widely used to construct brain networks (Osipowicz et al., 2016). Inspired by graph theory, brain network abstracted as a set of nodes and edges, is developed to describe the correlation or interaction among the different regions of the brain. In brain network, nodes represent region-of-interest (ROIs), and the edges between nodes represent the correlation between different brain regions (Fornito et al., 2016). Functional MRI (fMRI) can reflect temporal correlations between BOLD signals in brain regions, while diffusion tensor imaging (DTI) can be used to reveal the physical connectivity between the functionally relevant gray matter regions (Osipowicz et al., 2016).
According to Fornito et al. (2016), even a slight disruption in the small-world character of the functional brain networks would suggest a disruption in the integrity of the cognitive state systems involved in causing the disease. In practical application, functional connectivity (FC) can be constructed from fMRI and structural connectivity (SC) can be constructed from DTI. So that the FC can be used to detect the consistency of brain activities while the SC can measure neural fiber physical connections between different brain regions . It is widely acknowledged that both FC and SC are able to contribute significant information for brain disease diagnosis. Many recent researches have proved that combining the two modalities to construct brain network is an promising technique. Specifically, compared to the singlemodal brain network, the multi-modal brain network can achieve better analysis and diagnosis results (Song et al., 2020).
But majority of existing network-based analysis, which fuse FC and SC, can be divided into the following two categories . In the first category, some approaches based on data fusion strategy have been adopted, such as principal component analysis (PCA), multi-view embedding as well as multi-kernel learning (MKL) . More specifically, these approaches are applied to combining structural and functional network properties, which can reveal the balance of local and global efficiency between structural and functional networks (Rudie et al., 2013). Multi-kernel technology has been proved as an effective way in fusing multimodal data (Zhang et al., 2011), and many experiments were done to demonstrate that the more discriminatory results can achieve than using only single modality (Zhang et al., 2011;Dyrba et al., 2015). In the second category, the commonly used strategy is using one of the modality to assist another modality . However, this strategy does not make full use of the complementary information between the two modalities. What is more, most of the existing research ignore the global brain region correlation and the information of nodes. Although some papers propose to consider node information, only some simple topology attribute measurement methods, such as clustering coefficients or average degree are utilized. More potential and significant node information is ignored, which can be extracted by higher-order methods. The difference between such two mechanisms is depicted in Figures 1A,B.
On the one hand, in the brain network construction, the previous fusion brain methods construct the functional and structural brain networks independently, which cannot comprehensively reflect the interaction between structural and functional image data. On the other hand, in the feature extraction of brain network, the previous methods mainly analyze the connectivity features or the simple topological features, such as clustering coefficients or average degree, from the graph of the brain network. Different from above two categories, we seek to develop a multi-modal jointly construction method shown in Figure 1C, in which the interaction between functional and structural data promotes the discovery of hybrid structural and functional connectivity and high-order node information of the network can be obtained.
In this paper, we proposed a unified multi-modal brain network (UM2BN) construction and fusion method. First, we make spatial calibration, and then build the correlation model among all brain regions with functional data by lowrank representation, so that the global brain region correlation can be captured. Simultaneously, the local manifold with structural data is embedded into this model, so the local structural information can be preserved by manifold learning. Second, the PageRank algorithm is adaptively used to evaluate the significance of different brain regions. Finally, a multikernel strategy is utilized to merge the connectivity and node significance information from the constructed unified network for classification.
The main contributions can be summarized as the following four folds: 1. A unified framework to construct brain network by combing both FC and SC is proposed, in which the space alignment and brain network construction are carried out under the same framework and promote each other. 2. The relationship between multiple brain regions can be comprehensively considered instead of only considering two brain regions by adding a low rank constraint. And the local structural information is preserved by manifold learning. 3. The significant node information is adaptively extracted by PageRank algorithm from the unified brain network. Compared with only using simple attributes such as clustering coefficient or average degree, significant high-order node information can be obtained, which may help to capture the slight change in brain network. 4. In order to solve the data heterogeneity problem, an effective multi-kernel technology is utilized to fuse information of connectivity as well as node importance for classification.
The experimental results show that, compared with a series of previous brain network analysis approaches, our approach can achieve a promising performance in the diagnosis of epilepsy on a real epilepsy dataset. A preliminary version of this work has been reported (Yang et al., 2020). The rest of this paper is organized as follows. In section 2, we introduce related works. Then, we present the proposed multi-modal brain network jointly construction and fusion method in section 3. Section 4 introduces materials used in the study and provides the experimental results on epilepsy dataset. In section 5, we give an analysis of the experimental results. Finally, we summarize our work in section 6.

Pearson Correlation Based Brain Network Construction
Functional magnetic resonance imaging (FMRI) uses blood oxygenation level dependent (BOLD) changes in brain blood flow and oxygen consumption to detect the activity of neurons. Because of its high temporal and spatial resolution, fMRI is widely used in the field of brain functional network research. Pearson-based method is the most widely used in functional network construction methods. Suppose there are two brain regions, whose BOLD signals are denoted by the vectors x and y. The connectivity strength between these two brain regions is measured by where Cov(x,y) denotes the covariance of x and x, and Var(x) and Var(y) denote the variance of x and the variance of y.
For each pair of the regions, its connectivity strength can be calculated by above correlation. All the connectivities from the same subject form a brain network. As can be seen from the above description, the connectivity calculation in Pearson-based method only consider simple pairwise relationship of its attached brain regions, which may ignore the latent influence of other brain regions.

PageRank Algorithm
With the rapid development of the information technology, the number of users and web pages in the network grows very quickly. The network topology structure that reflects the relationship between users and web pages are becoming more and more complex, making it more difficult to provide users with high quality, relevant web pages based on user queries (Pandurangan et al., 2002). To address this issue, many algorithms have been proposed, among which PageRank algorithm is widely used because it can well reflect the high-order information of nodes (Pandurangan et al., 2002). The PageRank algorithm is based on two hypotheses. One is quantitative hypothesis: The more nodes connected to one node, the more important one node is. The other is quality hypothesis: If one node is more important, then other nodes connected to it are also more important. According to Gleich (2015), PageRank algorithm can be used not only in web learning, but also in social networks and bioinformatics. For example, Markovich et al. (2017) and Roul and Sahoo (2021) are both typical applications in web learning. Priyanta et al. (2019) has proved that PageRank algorithm can be used as an important tool for social network analysis. Jiang et al. (2017) use PageRank to diffuse information on two-layer graph model in protein structure analysis, and Liu et al. (2020) use PageRank to move the homologous proteins of query proteins to the neighbors of the query proteins in a protein similarity network. The PageRank algorithm indicates that if a node has important links to it, its links to other nodes are also significant (Xing and Ghorbani, 2004). Therefore, both the brain network and Web network can be abstracted as graph model, which is composed of a series of nodes and connectivities.
The higher-quality pages point to page A, the more important page A is. In graph model, brain network and Web network have similar properties. Brain areas and brain connectivities correspond to pages and links in the Web network, respectively. The importance of a brain region in a brain network is related to the importance of other brain areas transferred by connectivities. If a brain region is connected to a more important brain region, its importance will be higher. Therefore, in our model, we use the PageRank algorithm to evaluate the similarity of brain regions. The advantage of this method is that the importance of global brain regions and brain connections are considered in the model calculation. It is suitable to apply PageRank algorithm in brain network analysis, because brain network also has these properties (Gleich, 2015).

Notation
Denote boldface uppercase letter as a matrix (e.g., X), boldface lowercase letter as a vector (e.g., x), and lowercase letter as a

G
The DTI matrix reflects the physical connections.

D
The degree matrix of G, D ii = i G ij or j G ij .

L
The Laplacian matrix of G, L = D − G.
W The brain network matrix.

U
The projection matrix.

tr(X)
The trace of matrix X.

X T
The transpose matrix of X.

X ij
The ith row and jth column element of matrix X.
x i The ith row of matrix X.
x j The jth column of matrix X. · * The trace or nuclear norm of a matrix.

· F
The Frobenius norm of the matrix.
x, y The inner product between vector x and vector y.
scale (e.g., x). Further, we summarize the important symbols and definitions used in this article in Table 1.

Problem Formulation of Unified Brain Network
Our approach aims to learn a unified brain network representation combing structural connectivity and brain activities for disease diagnosis. More specifically, we utilize the matrix W ∈ R N×N to represent the unified brain network, the entry of W is W ij , which reflects not only the information of FC between ith brain region and jth brain region, but also the information from SC. Suppose x i ∈ R K×1 is a feature vector of fMRI time series for ith brain region, X = [x 1 , x 2 , ..., x N ] is the feature matrix of brain connection, in which N indicates the number of brain regions and K indicates the number of time points of time series. The matrix G ∈ R N×N reflects the physical connections, whose entry G ij ≥ 0 is the physical fiber quantity between brain region i and brain region j. The detail steps for constructing the unified brain network with functional and structural data proposed in this paper are as follows. First, we suppose the feature matrix of brain activities can be linearly represented by the weight matrix W. It can be expressed by X = XW, and further written as is a feature vector of fMRI time series for ith brain region. In order to make (2) better depict the correlation between brain regions, space alignment method is adopted. Inspired by Zhang et al. (2016b), we introduce a projection matrix U ∈ R M×K to transform the original k-dimensional data into m-dimensional compact representations X. It is expressed by X = UX, where X ∈ R M×N and X ∈ R K×N . Combined with (2), the model can be expressed as where · F is utilized to constrain the projection matrix U. Noteworthy, we make space alignment and brain network construction under a unified framework, so the two can promote each other for constructing a more precise brain network structure. Then, in order to consider the global brain region information, the low-rank (Han et al., 2019;Yu et al., 2019;Wang and Guo, 2020) constraint is introduced.
Rank of matrix W is a small number based on the assumption that each brain region can be approximately represented by a combination of only a few other brain regions. So objective function can be defined as Because rank minimization problem is non-convex. Inspired by Wang et al. (2018Wang et al. ( , 2019, (4) is reformulated to where · * is the trace or nuclear norm of a matrix. In addition, because DTI can reflects physical connectivity between functionally related gray matter regions, the matrix G is introduced to make the correlation model more discriminative. It has been proved that the foundation of FC is SC (Honey et al., 2009;Stam et al., 2016). Thus, we assume that the more fiber bundles exist between ith brain region and jth brain region, the closer the distance between w i and w j is. Inspired by He and Niyogi (2004), construct the following objective function: On the whole, the final objective function is defined to jointly minimizing the above problem where α, β, and γ are positive scalars weight the corresponding terms in (7).
where L = D − G (Xu et al., 2015;Yu et al., 2019). Thus, we transform the objective function as The matrix W is the solution above problem, and the unified brain network represented by it contains the information of both FC and SC. Figure 2 gives a schematic illustration of our proposed method for constructing multi-modal unified brain network with significant information of nodes. This paper is an extension of the work in MICCAI (Yang et al., 2020), but we need to point out that it has improvement FIGURE 2 | The flow chart of our proposed method for constructing unified brain network with functional and structural data and fusing connectivity as well as node information for classification.
in methodology and carried out more extensive experiments to evaluate the performance. The original method was performed in original feature space, and this work is conducted in feature space aligned by the projection shown in Equation (8). Although both of these two methods represent the signal from one brain region by signals from other brain regions, the choice of feature space directly affects the performance of the above representation model. The space alignment and brain network construction are carried out under the same framework and promote each other. In other words, the feature space of correction can help to establish more effective brain network, and the latter can also provide guidance for correction.

Alternating Optimization Algorithm
In order to solve this problem, some alternative optimization methods can be adopted. Here, the alternating direction method of multipliers (ADMM) algorithm (Xu et al., 2015) is utilized. First of all, we make the problem separable by introducing two auxiliary variables. And then (8) can be reformulated as We solve problem (9) by minimizing the following augmented Lagrange multiplier (ALM) function L where Y 1 , Y 2 , and Y 3 are Lagrange multipliers, and µ > 0 is a penalty parameter. ADMM algorithm is an iterative method that solves for each variable in a coordinate descent manner. The update formulas for those variables are as follows: where ϑ λ (X) = US λ ( )V T is a thresholding operator with respect to a singular value λ; is the soft-thresholding operator; X = U V T is the singular value decomposition of X.
2. Fix the other variables and update P by using (11). 3. Fix the other variables and update Q by using (12).
4. Fix the other variables and update U by using (13).
5. Fix the other variables and update X by using (14).

Node Information
After building the unified brain network, PageRank algorithm is utilized to evaluate the importance of different brain regions. According to Yan and Ding (2011), a simplified version of PageRank is defined as and P(u) is the set of nodes that connect to node u. N v denotes the number of links of node v and s u represents the score of node u. The calculation process of the algorithm is shown in Figure 3. Suppose the vector s = [s 1 , s 2 , ..., s n ] ∈ R N×1 reflects the importance of different brain regions. Its entry s u is the score of the uth brain region with restriction N u=1 s u = 1. And the larger the number of s u is, the more important the uth brain region is. First of all, we transform the matrix W into a matrix B ∈ {0, 1} N×N through a threshold factor r (if W vu ≥ r, then the corresponding entry B vu is set to one; otherwise, the entry B vu is set to zero). And the value of vector s is initialized as s = [ 1 N , 1 N , ..., 1 N ].
Then, we introduce a transfer matrix T ∈ R N×N to iterative update vector s, based on the iterative updating formula for s = T × s, until the convergence condition is satisfied.

Multi-Kernel SVM Classification
After obtaining the connectivity and node importance information from the constructed unified network, a multikernel strategy is adopted to fuse these heterogeneous information. The reasons why we adopt the multi-kernel fusion method are as follows. The previous study (Zhang et al., 2011;Huang et al., 2019) has shown that multi-kernel can effectively fuse multi-modal data and has been successfully applied into the diagnosis of brain diseases. For example, Zhang et al. (2011) used multi-kernel to fuse the multi-modal brain image data for Alzheimer's disease classification. Huang et al. (2019) employed multi-kernel to combine functional and structural connectivity features, and applied it into epilepsy classification. In addition, the multi-kernel method can handle heterogeneous multimodal data fusion by kernel trick. In our method, the obtained brain region features and brain connectivities are with different dimensionalities. Considering above two aspects, we adopted the multi-kernel method to fuse the multi-modal feature in brain network. The first modality is the connectivity of unified brain network, and the second modality is the node importance information of unified brain network. More specifically, k( is defined as a mixed kernel between the multi-modal training sample x i and x j , and k(x i , x) = m c m k (m) (x (m) i , x (m) ) is defined as the mixed kernel between the multi-modal training sample x i and the test sample x. m c m = 1 is restricted and a coarse-grid search is adopted to search the optimal values. Then, we fuse two kernels into a mixed kernel and a standard SVM classification algorithm is performed.

Materials and Preprocessing
Before experiments, we collected raw rs-fMRI data and DTI data from 306 peoples. Including 114 normal controls (NC), 103 patients with frontal lobe epilepsy (FLE), and 89 patients with temporal lobe epilepsy (TLE). All the subjects are with righthanded.The information of the dataset are shown in Table 2. Those data are collected by Siemens Trio 3T scanner at Jinling Hospital, Nanjing, China. The scan parameters are as follows: repetition time = 2,000ms; echo time = 30ms; and flip angle =90 • ; 30 transverse slices; field of view (FOV)= 240 × 240mm; slice thickness = 4mm; interstice gap = 0.4mm; voxel size = 3.75 × 3.75 × 3.75mm; DTI scans were obtained by using spin echobased echo planar imaging sequence. The scan parameters are as The functional network is constructed through fMRI and the structural network through DTI. The SPM8 in the DPARSF toolbox is utilized to pre-process all rs-fMRI images. Specifically, slice time are collected, corrected, rearranged, and normalized to the EPI template to obtain the initial functional time series. Then, the de-trending process is performed to remove spurious sources of variance. Utilizing the AAL atlas, we divided the resulting volumes consist of 240 time points into 90 regions of brain interest (ROIs), so that those time series reflect the information about brain activities. The DTI data are processed by the PANDA suite. First, the FSL toolbox is utilized to correct the DTI distortion, remove the eddies, and extract the brain mask from the non-diffusion weighted (B0) image. Then, the TrackVis is used to obtain fiber images by deterministic tracking method, and defined anatomic areas using AAL conventions based on each subject's co-registered T1 images. Finally, the quantity of fibers can naturally be viewed as the edge strength of the structural network.
In the PC (Betzel et al., 2016) method, the functional connectivity matrix is defined by Pearson's correlation coefficient. Then, we extract the upper triangular element of the functional connectivity matrix and compress it into a vector for each subject. Finally, the standard SVM is exploited for classification.
In the LSR (Qiao et al., 2016) method, a functional brain network (FBN) is constructed which jointly learns from partial correlation and sparse representation. Then, based on the matrixregularized network learning framework, we further formulate it as a sparse low-rank graph learning problem. Finally, t-test is used for feature selection and SVM is adopted for classification.
The WSGR (Yu et al., 2017) method ensures the construction of more biologically meaningful brain network by integrating connectivity strength, group structure, and sparsity. In contrast to traditional sparse representation, a connectivity strength weight matrix is defined based on Pearson's correlation matrix for l-1 norm, and a group partition for l-21 norm is added in constructing brain networks. Then, a linear SVM is exploited for classification.
The SSGSR (strength and similarity guided GSR)  method, which exploits both BOLD signal temporal correlation-based "low-order" FC and intersubject LOFC-profile similarity-based "high-order" FC as two priors to jointly guide the GSR-based network modeling. Then, the upper triangular element of the GSR-based network is extracted, and compressed into a vector. Finally, a linear SVM is used for classification.
In HOFC (Chen et al., 2016) method, an FC profile is calculated for each brain region first. Then, based on these FC profiles, a second layer of correlations is computed between all pairs of brain regions to generate an HOFC network. Then, for each subject, the upper triangular element of the HOFC network is extracted, and compressed it into a vector. Finally, a standard SVM is used.
The tHOFC (Zhang et al., 2016a) method is similar to HOFC method. More specifically, topographical profile similarity-based HOFC (tHOFC) is one types of HOFC method. Both of them have the idea of computing "correlation of correlations." Nonetheless, instead of measuring the similarity of the original rs-fMRI signals with the traditional FC, tHOFC measures the similarity of LOFC profiles between each pair of brain regions.
In the GCNN (Mao et al., 2018) model, a specific convolutional operator is designed for brain network which applies a row scanning on adjacent matrix to generate the feature map. Classification results are acquired by the softmax function based on these feature maps. This GCNN model can deal with both FC and SC. It is worth noting that the FC is defined by Pearson's correlation coefficients, while the SC is obtained by using the same approach as our proposed method.
In the SGCN (Ktena et al., 2017) method, a pair of brain networks are defined by FC matrices and a common graph structure is defined by the anatomy of brain. Then, the common graph structure is used for spectral graph convolutional networks. The model including an inner product layer, which is used to combined node representations from two brain networks, and a single fully connected output layer is used to output the similarity between brain networks, subsequently. Finally, the KNN classifier is exploited to disease diagnosis.
In the GK (Kang et al., 2012) method, a random walk graph kernel is used to measure the similarity between brain networks based on the number of common walks in the two networks. Then, a kernel matrix is constructed by these pairwise similarity. At last, the kernel matrix is fed into the SVM for disease diagnosis directly.
In the MK (Dyrba et al., 2015) method, a linear kernel matrix is calculated based on the feature vectors used in the PC methods, while the graph kernel matrix is calculated same as the GK method. Then, the two kernel matrices are linearly combined into a mixed kernel matrix by grid search to find the optimal parameters. At last, the standard SVM algorithm is performed for classification.
The CCA algorithm is used to fuse connectivity and node importance information of the brain network. These multimodal data of different dimensions are mapped to the same subspace, giving them the same dimension and the greatest linear correlation. Finally, a more discriminative set of features is obtained from multi-modal information.
The KCCA algorithm is an extension of CCA algorithm, and is also used to fuse multi-modal data, namely the connectivity of the brain network and the node information. Unlike the CCA method, it uses a kernel function to map the raw data into a higher-dimensional space and then looks for nonlinear relationships between different modalities.
In the M2TFS method, the functional connectivity matrix defining by Pearson correlation coefficient is viewed as the first modality, the DTI data are viewed as the second modality. First, M2TFS denotes the feature learning on each modality as a single task. Then, it uses group-sparsity regularizer to capture the intrinsic relatedness among multiple tasks. Furthermore, a new manifold-based Laplacian regularizer is introduced to preserve the data distribution information from each task. Finally, a multikernel SVM method to fuse multi-modal data for classification.
To verify the effect of the space alignment as well as node importance information, we omit these two parts respectively, and then calculate the classification accuracies. In the first case (i.e., our method without space alignment), we directly use the original fMRI data to construct the unified brain network. In the second case (i.e., our method without node importance), we directly utilize the information of brain network connectivity for disease diagnosis without taking into account the node information.

Experimental Setup
In order to evaluate the performance of our proposed method, we apply it to four different classification experiments, including NC vs. FLE, NC vs. TLE, FLE vs. TLE, and NC vs. (FLE and TLE). We use the following measures to ensure the fairness of the comparison. For parameter setting, we perform our method and the comparison methods with grid search parameter selection, and choose the one that performed best as the parameter. For the splitting of the dataset in experiment, we used five-fold cross-validation to verify the model's performance. The whole dataset was divided into five exclusive subsets, which have the same or similar size. The experiment was repeated for 5 times, the reported result was the average accuracy. Noting that it is necessary to ensure that the test set data is not used in the model training process. Classification accuracy (ACC) is used as an indicator to evaluate the classification performance.

Results on Epilepsy Data
Experimental results of all methods are summarized in Table 3. As can be seen from Table 3, in the four classification tasks, the accuracies of the proposed method are 73.3, 75.5, 67.9, and 75.0%, respectively. Compared with other methods, the method in this paper achieves the highest accuracies in the three tasks: NC vs. FLE, NC vs. TLE, NC vs. (FLE and TLE). Although SGCN algorithm has the highest classification accuracy in the FLE vs. TLE task, the discrepancy between it and the accuracy of our method 67.9% is relatively small. In addition, observing the Table 3, we can also draw the following three conclusions.
First, the fMRI-based method achieved better results than the method based on only DTI data, indicating that fMRI data contained more effective information than DTI data in the epilepsy disease diagnosis tasks. In general, the multi-modal method combining fMRI and DTI data achieved higher classification accuracy than using only single modality. This indicates that the multi-modal classification method is indeed an effective method in brain disease diagnosis.
Second, the classification accuracies can be improved by combining the node information measured by PageRank algorithm. More specifically, we utilize our proposed method without node importance to classification, and the accuracies are only 69.0, 72.0, 63.8, and 71.5%. Compared with adding node importance information, the accuracies are relatively poor. This can indicate that the node importance of brain network measured by PageRank algorithm does contain important information, which is helpful for the diagnosis of epilepsy.
Third, the space alignment operation improves the final classification accuracies. Our goal is to construct a more precise brain network structure by making space alignment and brain network construction under a unified framework. The results show that 1.7, 1.3, 0.9, and 1.4% can be improved in the four classification tasks of NC vs. FLE, NC vs. TLE, FLE vs. TLE, NC vs. (FLE and TLE), respectively.

Comparison of Other Topological Attributes
We also conducted experiment to verify that the PageRank algorithm introduced to evaluate the importance of different brain regions is reasonable and can achieve good results. Other commonly used topological attribute measures, e.g., Clustering coefficient (Clustering), Average degree (Avgdeg), Closeness centrality (Closeness), and Radiality are, respectively, used to evaluate the importance of brain regions, and the results of PageRank algorithm are compared. Specifically, we maintain the same experimental setup, changing only the method used to estimate the node information. The experimental results are reflected in Figure 4. As can be seen from Figure 4, we can see that, compared with other measurement methods, introduced PageRank algorithm achieves the highest accuracies in all four different experiments. Thus, it can be verified that the PageRank algorithm is reasonable and effective in the assessment of unified brain network node importance information.

Discussion on the Effect of Space Alignment and DTI Constraint
When the characteristic dimension is large, the distance between samples tends to be consistent, and the information of the relationship between samples is easy to be concealed. So it is usually necessary to reduce the dimension of the data in advance. However, this process is independent of the subsequent structural information mining and cannot guarantee the original data structure information contained in the feature space after dimension reduction. In order to solve this problem, we introduce space alignment into the construction process of unified brain network. Space alignment and brain network construction are carried out under a unified framework, and the two promote each other. On the one hand, a better feature space is beneficial to the mining of precise brain network structure information. On the other hand, precise brain structure information is also helpful to the selection of a better feature space. In addition, we use DTI to constrain the correlation between the two brain regions [expressed in (5)]. According to Honey et al. (2009) andStam et al. (2016), a reasonable hypothesis is that the more fibers between brain region i and brain region j, the closer W i and W j is. Therefore, Equation (5) is constructed inspired by He and Niyogi (2004). In order to validate the rationality and effectiveness of space alignment and DTI constraint, we delete the linear feature extraction part and the DTI constraint part in the final objective function, respectively, but guarantee other experimental setup are the same. We show the classification accuracies in Table 4. As can be seen from Table 4, during the course of brain network construction, deleting any part of two in the objective function will reduce the classification accuracies. It shows that both space alignment and DTI constraint contribute to the more precise brain network structure. Results demonstrate that the two promote each other, which can improve the performance of disease diagnosis.

Analysis of Convergence
In order to verify the rationality of the solving process of the objective function (8), we randomly select three samples and, respectively, draw the convergence curve in Figures 5A-C. As can be seen from Figure 5 that with the increase of iteration times, the value of the objective function first decreases then tends to be stable. And it can be seen that our proposed method can converge to a certain value within 600 iterations.

Analysis of Parameter Sensitivity
During the experiment, there are three hyperparameters that affect the construction of brain network, i.e., α, β, and γ . In order to study the influence of these parameters, we conduct the following experiments: First, we fix the parameter γ as the default value 1.0, study the influence of two hyperparameters α and β, and reflect the experimental results in Figure 6. Among them, both of α and β range from {0.01,0.05,. . . ,50}. It can be seen that when the range of parameters α and β is within {0.01,0.05,. . . ,50}, our proposed algorithm is relatively stable. After that, we set both parameters α and β as 1.0, study the influence of parameter γ on classification accuracies, and reflected the results in Figure 7A. It can be seen from Figure 7A that when the value range of γ parameter is {0.01,0.05,. . . ,50}, the classification accuracies change only slightly for four different classification tasks. It can be concluded that the proposed method is relatively stable under different values of α, β, and γ . In addition, the parameter m used in the linear feature extraction will also affect the final experimental results. We also conduct experiments to study the influence of this parameter on the classification accuracies, as shown in Figure 7B. It can be seen that when parameter m changes within the range of {20, . . . , 220}, the classification accuracies of the four different tasks will also change. For the NC vs. FLE task, the optimal m value range is 100-180. For the NC vs. TLE task, the optimal m value range is 120-160. For the FLE vs. TLE task, the optimal m value is 100-160. And for the NC vs (FLE and TLE) task, the optimal m value is in the range of 60-80. Before using the PageRank algorithm to evaluate the importance of nodes in the brain network matrix W, a thresholding operation needs to be performed on the initial brain network. The parameter r is to control the sparsity of the network in thresholding. The larger the value of r is, the sparser the network after thresholding is. In the experiment, we control the value range of r in the range of [0.1, 0.2,...., 0.8], and determine the optimal value of r by searching the candidate set.
The experimental results are shown in Figure 7C. As can be seen from this figure, the experimental accuracy is sensitive to the value of r. If the value of r is too large or too small, it will cause poor accuracies. The reason might be as follows. When the value of r is too large, there are too few or even no edges are preserved in the network, and the PageRank algorithm cannot extract the relationship among most of the nodes. When the value of r is too small, the network is very dense, which leads to average the importance of the node. Experimental results show that in the NC vs. FLE and NC vs. TLE tasks, the best value of r is 0.3; in the FLE vs. TLE and NC vs. (FLE and TLE) tasks, the best value of r is 0.4.

Analysis of Information Brain Regions
This work adopted the indicator proposed in Zhu et al. (2018) to evaluate the brain region importance. We establish the discriminant model from brain connectivities to labels by non-negative elastic-net sparse constraint. Then the connectivities can be ranked according to the value of the representation coefficients.In order to prove the effectiveness of the proposed method, we select top 12 significant alterations of connectivity and reported in Figure 8 for three classification tasks, respectively. It can be found that some brain connectivities involving brain regions such as Parahippocampal gyrus, Precuneus, and Superior temporal gyrus have significant changes. It means that these connectivities are the key to distinguish between normal people and FLE patients. There are similar findings in the literatures (Woodward et al., 2014;Zhang et al., 2019). Similarly, in the NC vs. TLE task, brain regions such as Parahippocampal and Amygdala are selected, which was supported by the work of Reinsberger et al. (2010) and Qiao et al. (2016). In the FLE vs. TLE task, some connectivities about brain regions such as Middle frontal gyrus, ParaHippocampal, and Amygdala have significant changes. Reinsberger et al. (2010) and Exner et al. (2002) have also mentioned this finding. Extensive evidence proved the effectiveness of the proposed method.

Limitations and Future Work
Although the experimental results show that our method has achieved good results in epilepsy diagnosis. But it still has some limitations. First, the nodes used in this study are defined by the AAL template, which divides the brain into only 90 ROIs. In the future, we will try to use other templates to divide the brain more finely. Second, PageRank algorithm is utilized to extract node information of the unified brain network, and then fuse connectivity and node information for classification. The experimental results show that, compared with other common topological attribute measurement methods, the PageRank algorithm can get better results. However, there are currently some variants of PageRank algorithm, such as weighted PageRank (Xing and Ghorbani, 2004). In future work, we will try to use variants of PageRank methods, which may improve classification accuracies. Third, in order to verify the effectiveness of our method, we apply it to the multi-modal epilepsy dataset. The experimental results show that our method achieves good results in four different classification tasks. In the future, we will consider extending our method to other brain disease diagnosis tasks to further explore the application value of our proposed method.

CONCLUSION
In this paper, a unified brain network construction algorithm is proposed, which is jointly learned from both functional and structural data, and make full use of complementary information between each other. In our method, we make the space alignment and multi-modal brain network construction under a unified framework, so that the two can promote each other. Instead of only considering two brain regions, we comprehensively consider the global brain regions relationship by low-rank constraint. And the local structural information can be preserved by extending the local manifold learning into this model. What is more, we take into account not only the connectivity, but also the node importance information of the unified brain network, extracted by PageRank algorithm. Finally, a multi-kernel strategy is utilized to solve the data heterogeneity problem and merge the connectivity as well as node information for classification. At last, we apply the proposed method (JCFBN) to the epilepsy diagnosis, and the experimental results show that our method can achieve a promising performance on all four classification experiments.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

AUTHOR CONTRIBUTIONS
QZ and JY conceived the experiment and completed the manuscript. BX and LS preprocessed the data. ZH provided clinical guidance on biomarkers. DZ was responsible for data analysis. All the authors listed have made practical contributions to this work and agree with the manuscript.