Robust Detection of Impaired Resting State Functional Connectivity Networks in Alzheimer's Disease Using Elastic Net Regularized Regression

Introduction

Many studies have identified altered functional connectivity networks in resting state examinations of Alzheimer's disease (AD) patients compared to controls using functional imaging techniques such as FDG-PET or resting state functional MRI (rs-fMRI) (for a recent review see Teipel et al., 2016). Typically AD dementia impairs functional connectivity in the default mode network (DMN; Greicius et al., 2004), but AD pathological changes and ensuing functional disruptions have been shown to extend beyond the regions of the DMN (Agosta et al., 2012; Grothe et al., 2016).

To identify the network characteristics of AD-related changes in functional imaging data, most studies have employed stepwise or multiple linear regression approaches (Agosta et al., 2012; Koch et al., 2012; Sheline and Raichle, 2013). However, features from rs-fMRI and other functional imaging data are often highly collinear across regions, and linear regression approaches are known to be highly sensitive toward collinearity (James et al., 2013; Section 3.3.6). In the presence of a high number of features relative to the number of available observations (Tibshirani, 2011) and when features are collinear (Hoerl and Kennard, 1970; Tibshirani, 1996), regularization techniques have been established for dimension reduction and feature selection. More recently, regularized models, using an elastic net penalty (Zou and Hastie, 2005; Zou and Zhang, 2009), have been applied to multimodal neuroimaging studies to reduce the effect of multicollinearity on feature selection (Trzepacz et al., 2014; Teipel S. J. et al., 2015; Schouten et al., 2016; de Vos et al., 2016).

Here, we used rs-fMRI data from a multicenter study to compare accuracy of group separation, as well as stability of regional feature selection and ensuing identification of cortical networks discriminating AD patients and controls between cross-validated regularized logistic regression with an elastic net penalty and classical stepwise logistic regression. We hypothesized that elastic net logistic regression would lead to more generalizable feature selection and more consistent network identification than classical stepwise logistic regression. Of note, the principles of these methods, except the elastic net penalty, represent textbook knowledge from statistical learning literature, but adoption of these methods to the burning issue of highly collinear features in neuroimaging research is still slow.

Materials and Methods

For the current study, we used data from 53 patients with clinically probable AD according to NINCDS-ADRCA criteria (McKhann et al., 1984) and 118 healthy elderly control individuals that have been retrieved retrospectively from four sites within the framework of the “German resting-state initiative for diagnostic biomarkers” (http://www.psymri.org). Distribution of demographic characteristics of participants across sites is summarized in Table 1.

TABLE 1

Table 1. Demographic characteristics.

All participants were free of any significant neurological, psychiatric, or medical condition (except for AD in patients), in particular cerebrovascular apoplexy, vascular dementia, depression, or subclinical hypothyroidism, as well as substance abuse. Healthy controls were required to have no cognitive complaints and scored within one standard deviation of the age and education adjusted norm in all subtests of the Consortium to Establish a Registry of Alzheimer's Disease (CERAD) cognitive battery (Morris et al., 1989).

Written informed consent was provided by all subjects, or their representatives. The study was approved by local ethics committees at each of the participating centers, and has been conducted in accord with the Helsinki Declaration of 1975.

Imaging and Data Acquisition

The data used in this study were obtained from four different 3.0 Tesla MRI scanners. Acquisition parameters for the rs-fMRI sequences are given in Table 2. In one center (site I), the subjects were instructed to keep their eyes open, whereas in the remaining centers (sites II-IV) all subjects were requested to close their eyes, relax, but not to fall asleep. Functional MRI was based on echo-planar imaging using scan durations between 6 and 8.7 min for the rs-fMRI sequence. The number of acquired time points was between 120 and 200 with a voxel size ranging from 2 × 2 × 2.6 up to 3.28 × 3.28 × 4.4 mm³ (Table 2). For anatomical reference, high-resolution T1-weighted gradient echo sequences with an isotropic resolution of 1 mm³ were also obtained from all scanners during the same session.

TABLE 2

Table 2. Scanner characteristics.

MR Processing

Functional MRI data processing was carried out using Data Processing Assistant for Resting-State fMRI (DPARSF 3.2) (Chao-Gan and Yu-Feng, 2010), considering the recommendations from a recent systematic evaluation of processing alternatives (Shirer et al., 2015). After the removal of the first six images to account for gradient field stabilization, the rs-fMRI data was slice time corrected and realigned to the temporal mean image. The anatomical T₁-weighted image of each participant was coregistered to the mean functional image and subsequently segmented into gray matter, white matter, and cerebrospinal fluid (CSF) partitions using the Voxel-based Morphometry (VBM8) toolbox (Gaser et al., 1999) that extends Statistical Parametric Mapping (SPM8) (Friston et al., 2007). The Diffeomorphic Anatomical Registration Through Exponentiated Lie algebra (DARTEL) algorithm (Ashburner, 2007) was applied to normalize the T₁-weighted images to the Montreal Neurological Institute (MNI) reference coordinate system using the default brain template included in VBM8. The deformation fields generated by DARTEL were used to project the functional scans from each subjects' native image space into the MNI reference space. We combined this step with the reslicing of all functional data to an isotropic resolution of 3 mm. The subsequent nuisance regression included covariates of head movement (rotation, translation, and first and second order derivatives) and the mean time courses for the global brain signal, the white matter segment signal, and the CSF segment signal. Although global signal regression was found to introduce negative correlations (Murphy et al., 2009; Shirer et al., 2015), studies consistently reported that it effectively increases the signal-to-noise ratio (Yan et al., 2013; Power et al., 2014; Shirer et al., 2015). Recently, Shirer et al. evaluated the influence of global signal regression on group separation but only found a minor, non-significant effect (Shirer et al., 2015). Subsequently, the images were band-pass filtered using the frequency band 0.1–0.01 Hz. For each individual the time series of signal was extracted for each of the 84 functionally defined regions of the Greicius atlas (Shirer et al., 2012). Pearson's correlation coefficients were computed for the 3486 possible pairs of correlations between these 84 regions (Shirer et al., 2012). Finally, Pearson correlation coefficients of the signal time courses were adjusted to be normally distributed using Fisher's Z-transform (Fisher, 1915): z = 0.5 ln [(1+r)/(1–r)].

Statistical Analysis

Demographic Characteristics

Baseline demographic characteristics were compared between AD and control cases using parametric and non-parametric tests as required: age and years of education were compared between groups using Student's t-test, gender distribution using Chi² test, and neuropsychological test results using non-parametric Mann-Whitney U-test.

Prediction of Group Membership

We compared two regression models for prediction of group membership (AD vs. controls) in respect to two outcomes, (i) the accuracy of prediction as determined by the area under the receiver operating characteristics curve (AUC), and (ii) the consistency of feature selection.

The two regression models encompassed:

1. bidirectional (backward and forward) stepwise unpenalized logistic regressions using the function step in R (The R Foundation for Statistical Computing). The function weights the choices via the Akaike information criterion (AIC), which takes account of the total number of fitted parameters.

2. penalized logistic regression models with an elastic net penalty, as determined using the R package glmnet (available at http://cran.r-project.org/web/packages/glmnet/index.html). Elastic net regression is controlled by two parameters, (i) alpha, which sets the degree of mixing between two types of regularized regression, namely ridge regression (regularization by squared L₂ norm; alpha = 0) and the Lasso (Least Absolute Shrinkage and Selection Operator, regularization by L₁ norm; alpha = 1), and (ii) lambda, defining the strength of regularization (Friedman et al., 2010). Alpha was selected to be 0.5, corresponding to a full elastic net penalty, which minimized the partial likelihood deviance of the model (see Figure 1). Lambda was determined using grid search with 100-fold cross-validation. The optimal lambda was determined as the mean across 100 iteratively determined lambda values minimizing the deviance of the model. The optimal lambda value was determined for each bootstrap iteration in the training data and applied to the test data as defined below. Details of this method can be found in the appendix.

FIGURE 1

Figure 1. Selection of alpha parameter for penalized logistic regression. Misclassification error plotted against the range of lambda values (plotted on a logarithmic scale) for different values of α for a penalized logistic regression on the rs-fMRI data. Numbers on top of each graph indicate the number of selected variables. Error bars indicate the bootstrapped standard deviation for the misclassification error for each lambda value. The left bottom plot shows the different deviance curves on a unified scale, indicating that α = 0.5 yields the lowest deviance together with α = 0, corresponding to a ridge regression model.

Both models were determined using strict cross-validation procedures. Random samples of 2/3 of the data were drawn 1000 times to train the prediction models (training data). For both regression models, the prediction accuracy was determined using the remaining 1/3 as test data. Parameter optimization, i.e., selection of optimal lambda and (stepwise) feature selection, was conducted in the training data and subsequently applied to the test data. Prior to model building, the feature space was restricted through determining the set of variables which correlate with diagnosis with a Pearson's correlation coefficient of |r| > 0.35 in the training data, resulting in an average number of 36 included predictor variables across the bootstrapped repetitions.

In a second analysis, dummy coded center variables were forced as additional variables into the models to determine the effect of center on model accuracy and feature selection.

To check for multicollinearity of the stepwise logistic regression models, we determined the variance inflation factor (VIF) (Belsley, 1991) for each independent variable on the set of the remaining independent variables using the function vif in R package “car” (available at https://cran.r-project.org/web/packages/car/index.html).

Results

Demographic Characteristics

Demographic characteristics are summarized in Table 1. AD patients and controls were not significantly different in age (t = 1.67, 169 df, p = 0.96) or sex distribution (Chi² = 0.68, 1 df, p = 0.41). Both groups differed significantly in years of education (t = −4.72, 168 df, p < 0.001), with less years of education in the AD cases, and, as expected, AD patients scored significantly lower than healthy controls in the MMSE score (p < 0.001).

Prediction of Group Membership

The median VIF across all stepwise regression models and variables was 86, indicating a very high collinearity in the large majority of models. Mean area under the ROC curves in the test samples was 70% for the stepwise selection, and 80% for the elastic net regression models for the discrimination between AD cases and controls. The mean AUC and corresponding 2.5/97.5 percentile confidence intervals for both models are shown in Figure 2. The selected features are shown in Table 3 for both models, with seven features selected in at least 50% of 1000 cross-validation repetitions for the elastic net and two features selected for the stepwise logistic regression model. Figures 3, 4 show the frequency distribution of feature selection, suggesting that features were more homogeneously and more often selected in the cross-validation repetitions for the elastic net compared to the stepwise logistic regression models, with a median value of 10 features with the stepwise regression and 22 features in the elastic net regression.

FIGURE 2

Figure 2. Areas under the ROC curves for stepwise and elastic net logistic regression. AUC and 2.5/97.5 percentile confidence intervals for stepwise logistic regression without scanner (step.) and with scanner forced into the model (step. plus), and for elastic net logistic regression without scanner (EN) and with scanner forced into the model (EN plus).

TABLE 3

Table 3. Selected features.

FIGURE 3

Figure 3. Feature selection frequency plot. Frequency of selected features (based on 1000 bootstrap iterations) for elastic net and stepwise logistc regression. Please note that the x-axis represents the features that were sorted according to their frequency independently within each model. Therfore, the same position on the x-axis does not indicate the same feature for the elastic net and the stepwise logistic regression models, respectively.

FIGURE 4

Figure 4. Number of features selected per model. Histograms plotting the frequency with which a number of features was selected across all bootstrapping iterations for elastic net (blue) and stepwise logistc regression (red).

When we repeated the analyses with dummy coded center covariates forced into the models, AUC was 81% for the elastic net penalty, and selected features above 50% frequency were unchanged. For the stepwise regression, AUC decreased to 68%, and no feature was selected with a frequency above 45%.

Discussion

In accordance with our hypothesis, we found more accurate group discrimination between AD dementia cases and controls and more homogeneous feature selection from resting state fMRI data when using regularized logistic regression with an elastic net penalty compared with a classical stepwise logistic regression. These findings support the notion that regularized regression is superior to classical stepwise feature selection for dealing with highly collinear multidimensional functional imaging data. The features retrieved from penalized regression point to alterations of an extended functional network in mild AD dementia, compromising the dorsal DMN, but also key regions for language processing, object recognition and attention.

As illustrated by the high VIF with a median value of 86 (values above five are considered indicative for serious multicollinearity; Belsley, 1991), the regional rs-fMRI values exhibited a high degree of collinearity that compromised unbiased feature selection and determining the relevance of single features. The problem of dealing with multidimensional, multicollinear data is well-known in the statistical literature under the term of “the curse of dimensionality” (Bellman, 1961). Penalized regression has been developed since the 1940s to deal with this problem, encompassing techniques like ridge regression (Hoerl, 1970), the Lasso (Tibshirani, 1996), and more recently elastic net regression (Zou and Hastie, 2005), which combines both regularization techniques within the same model. Different to ridge regression, and similar to the Lasso, elastic net regression not only shrinks the feature coefficients but sets some of the coefficients to zero, thus reducing the dimensionality of the feature space. Different to the Lasso, elastic net regression is designed to select highly correlated features as a group rather than selecting only a single feature out of such a set of highly correlated variables, thus preserving a potentially meaningful correlation structure of the original feature space (Zou and Hastie, 2005).

Previous neuroimaging studies have successfully applied elastic net regression to multimodal neuroimaging data for feature selection for dementia prediction in subjects with mild cognitive impairment (MCI), and AD cases (Trzepacz et al., 2014; Teipel S. J. et al., 2015; de Vos et al., 2016). A previous study has applied this approach to rs-fMRI data of people with mild to moderate AD dementia from one scanner (Schouten et al., 2016), reaching 77% accuracy in the mild AD subgroup. In our multicenter study, cross-validated accuracy of 80% discrimination between AD cases and controls from elastic net regression was higher than the accuracy in this previous study (Schouten et al., 2016), but still lower than results from previous monocenter studies lacking cross-validation (Koch et al., 2012; Balthazar et al., 2014). Our findings level of accurcy agrees with estimates from previous cross-validated monocenter studies using non-linear machine learning techniques for classification (Challis et al., 2015; Dyrba et al., 2015). One recent study yielded 100% group discrimination between 20 AD patients and 20 controls using support vector machine classification (Khazaee et al., 2015). From the method description, however, the feature selection prior to cross-validated machine learning was based on the entire data set and was not part of the cross-validation.

Features selected by the elastic net regression were more consistent across repeated cross-validations than features selected by stepwise regression. Previous research on rs-fMRI in AD dementia has often focused on the DMN regions (Greicius et al., 2004; Koch et al., 2012; Balthazar et al., 2014). This approach reduces potential problems from collinearity through a priori feature selection. At the same time, it restricts the analysis to a single preselected functional network. Using elastic net regression, we retrieved the dorsal part of the DMN as key part of altered functional connectivity in AD. This agrees with previous analyses based on preselected DMN regions (Greicius et al., 2004; Koch et al., 2012; Balthazar et al., 2014) and underscores the overall validity of our approach. In addition, we found decreased functional connectivity in AD in the superior temporal gyrus, a region that is involved in language processing (Zhuang et al., 2014), and prefrontal parts of the salience network, prefrontal and parietal components of executive control networks, as well as the medial occipital gyrus as part of the ventral visual stream involved in object recognition (Teipel et al., 2007) and recognition of limb movements (Astafiev et al., 2004). These findings support the extended nature of AD pathology affecting several higher order cognitive networks, as previously found in topographic lesion driven studies (Grothe et al., 2016) and rs-fMRI analysis in small samples of 12 to 16 AD cases and 12–22 controls (Zhou et al., 2010; Agosta et al., 2012; Dai et al., 2012), and one large scale study (Brier et al., 2012). Different to two of these previous studies (Zhou et al., 2010; Agosta et al., 2012), we found only reductions, but no increases of functional connectivity in AD. This difference may have two possible causes. The first possible cause would be different severity of disease within the dementia stage of AD. However, the MMSE scores were similar between the AD cases of our and the previous studies. Another cause may be the different metric used as prediction features: we used correlation between regions irrespective of preselected networks, whereas the previous studies used regional loadings on independent components associated with specific functional networks (Agosta et al., 2012; Zhou et al., 2010).

Compared with elastic net regression, stepwise regression yielded only 70% accuracy. In addition, selection of the most relevant features was much less consistent across the 1000 iterations, compromising only two functional connections between sensorimotor and visuospatial regions, and no connection involving the DMN. These findings suggest that feature selection in step-wise regression was more sensitive to multicollinearity, where small differences in explained variance drive almost arbitrarily selection of a single feature among a set of highly collinear variables (Farrar and Glauber, 1967).

Stepwise logistic regression was sensitive to scanner effects, with a slight drop in prediction accuracy and a further loss of consistency in feature selection when scanner was forced into the model. In contrast, elastic net regression was insensitive to scanner effects; both accuracy of group discrimination and frequency of feature selection were unaffected when we repeated the analyses with scanner forced into the cross-validated models. This finding is of particular relevance given the sensitivity of rs-fMRI data to multiscanner effects, as has been reported in test-retest studies of rs-fMRI even in healthy people repeatedly scanned at the same scanner (Meindl et al., 2010; Chen et al., 2015; Lin et al., 2015; Orban et al., 2015; Shirer et al., 2015; Jovicich et al., 2016), including long-term evaluation after more than 12 months (Chou et al., 2012; Guo et al., 2012; Blautzik et al., 2013). Moreover, the use of multiple scanners typically results in high variability of signal-to-noise and contrast-to-noise ratios, particularly when using field strengths of 3T and higher (Magnotta et al., 2006; Lin et al., 2015; Jovicich et al., 2016).

We need to consider two main limitations of our study. First, the scan protocols were different between scanners. Multiscanner acquisition helps to increase sample size, a problem of many previous monocenter studies. In addition, estimates of accuracy derived from multicenter data may more easily generalize to future use of an imaging technology in routine care than estimates derived from single center data acquisition. We employed preprocessing steps that had been shown in a previous study to reduce multiscanner effects (Shirer et al., 2015), and used cross-correlation of regional signal time series which in a previous study had yielded more stable results across scanners than other connectivity metrics, such as cross-coherence or partial cross-correlation (Fiecas et al., 2013). Secondly, the reference standard in our sample was a clinical diagnosis of AD dementia, but independent PET or CSF based biomarker validation was not available in the majority of cases. Data came from expert centers experienced in the early diagnosis of AD. Still, a final judgment of the added value of rs-fMRI for AD diagnosis must await systematic evaluation of diagnostic accuracy in multicenter data from biomarker stratified cases.

In summary our findings point to an extended network of functional disconnection, including the dorsal DMN, but also involving functional networks employed in attention, object recognition and language processing. In a multicenter sample of AD and control cases, elastic net regression yielded cross-validated diagnostic accuracy that approached, but did not reach, the benchmark for a useful biomarker of AD (Consensus-Group, 1998); diagnostic approaches based on stepwise regression came not even close to this benchmark. These findings question the future wide-spread use of rs-fMRI as a stand-alone diagnostic marker of AD (Teipel S. et al., 2015). This does not exclude an important role of rs-fMRI as add-on diagnostic marker (Dai et al., 2012) and to identify mechanisms of functional disconnection and resilience in future prospective studies. Our data suggest that regularized regression should be preferred over still more widely used but less robust stepwise feature selection to retrieve homogeneous and stable estimates of altered functional networks in AD.

Ethics Statement

Institutional Review Board of the University Medicine Rostock. Written informed consent was provided by all subjects, or their representatives. The study was approved by local ethics committees at each of the participating centers, and has been conducted in accord with the Helsinki Declaration of 1975. For people with dementia, informed consent involves oral and written presentation of study procedures, and information of caregivers.

Author Contributions

ST: Conception of the work; acquisition, analysis, and interpretation of data for the work; drafting the work; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. MG, MD: Acquisition, analysis, and interpretation of data for the work; revising the work critically for important intellectual content; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CM, TG, CS, ME, NF, EM, SK, VB, MW: Interpretation of data for the work; revising the work critically for important intellectual content; final approval of the version to be published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

ST received support by a grant of the Federal Ministry of Research (BMBF) (AgeGain, 1GQ1425B).

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/article/10.3389/fnagi.2016.00318/full#supplementary-material

References

Agosta, F., Pievani, M., Geroldi, C., Copetti, M., Frisoni, G. B., and Filippi, M. (2012). Resting state fMRI in Alzheimer's disease: beyond the default mode network. Neurobiol. Aging 33, 1564–1578. doi: 10.1016/j.neurobiolaging.2011.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Ashburner, J. (2007). A fast diffeomorphic image registration algorithm. Neuroimage 38, 95–113. doi: 10.1016/j.neuroimage.2007.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Astafiev, S. V., Stanley, C. M., Shulman, G. L., and Corbetta, M. (2004). Extrastriate body area in human occipital cortex responds to the performance of motor actions. Nat. Neurosci. 7, 542–548. doi: 10.1038/nn1241

PubMed Abstract | CrossRef Full Text | Google Scholar

Balthazar, M. L., de Campos, B. M., Franco, A. R., Damasceno, B. P., and Cendes, F. (2014). Whole cortical and default mode network mean functional connectivity as potential biomarkers for mild Alzheimer's disease. Psychiatry Res. 221, 37–42. doi: 10.1016/j.pscychresns.2013.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Bellman, R. (1961). Adaptive Control Processes. A Guided Tour. Princeton, NJ: Princeton University Press.

Google Scholar

Belsley, D. A. (1991). Conditioning Diagnostics: Collinearity and Weak Data in Regression. Chichester: John Wiley & Sons.

Blautzik, J., Keeser, D., Berman, A., Paolini, M., Kirsch, V., Mueller, S., et al. (2013). Long-term test-retest reliability of resting-state networks in healthy elderly subjects and with amnestic mild cognitive impairment patients. J. Alzheimers. Dis. 34, 741–754. doi: 10.3233/JAD-111970

PubMed Abstract | CrossRef Full Text | Google Scholar

Brier, M. R., Thomas, J. B., Snyder, A. Z., Benzinger, T. L., Zhang, D., Raichle, M. E., et al. (2012). Loss of intranetwork and internetwork resting state functional connections with Alzheimer's disease progression. J. Neurosci. 32, 8890–8899. doi: 10.1523/JNEUROSCI.5698-11.2012

PubMed Abstract | CrossRef Full Text

Challis, E., Hurley, P., Serra, L., Bozzali, M., Oliver, S., and Cercignani, M. (2015). Gaussian process classification of Alzheimer's disease and mild cognitive impairment from resting-state fMRI. Neuroimage 112, 232–243. doi: 10.1016/j.neuroimage.2015.02.037

PubMed Abstract | CrossRef Full Text | Google Scholar

Chao-Gan, Y., and Yu-Feng, Z. (2010). DPARSF: a MATLAB toolbox for “pipeline” data analysis of resting-state fMRI. Front. Syst. Neurosci. 4:13. doi: 10.3389/fnsys.2010.00013

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, B., Xu, T., Zhou, C., Wang, L., Yang, N., Wang, Z., et al. (2015). Individual variability and test-retest reliability revealed by ten repeated resting-state brain scans over one month. PLoS ONE 10:e0144963. doi: 10.1371/journal.pone.0144963

PubMed Abstract | CrossRef Full Text | Google Scholar

Chou, Y. H., Panych, L. P., Dickey, C. C., Petrella, J. R., and Chen, N. K. (2012). Investigation of long-term reproducibility of intrinsic connectivity network mapping: a resting-state fMRI study. AJNR Am. J. Neuroradiol. 33, 833–838. doi: 10.3174/ajnr.A2894

PubMed Abstract | CrossRef Full Text | Google Scholar

Consensus-Group (1998). Consensus report of the working group on: “Molecular and Biochemical Markers of Alzheimer's Disease.” The Ronald and Nancy Reagan Research Institute of the Alzheimer's Association and the National Institute on Aging Working Group. Neurobiol. Aging 19, 109–116.

Dai, Z., Yan, C., Wang, Z., Wang, J., Xia, M., Li, K., et al. (2012). Discriminative analysis of early Alzheimer's disease using multi-modal imaging and multi-level characterization with multi-classifier (M3). Neuroimage 59, 2187–2195. doi: 10.1016/j.neuroimage.2011.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

de Vos, F., Schouten, T. M., Hafkemeijer, A., Dopper, E. G., van Swieten, J. C., de Rooij, M., et al. (2016). Combining multiple anatomical MRI measures improves Alzheimer's disease classification. Hum. Brain Mapp. 37, 1920–1929. doi: 10.1002/hbm.23147

PubMed Abstract | CrossRef Full Text | Google Scholar

Dyrba, M., Grothe, M., Kirste, T., and Teipel, S. J. (2015). Multimodal analysis of functional and structural disconnection in Alzheimer's disease using multiple kernel SVM. Hum. Brain Mapp. 36, 2118–2131. doi: 10.1002/hbm.22759

PubMed Abstract | CrossRef Full Text | Google Scholar

Farrar, D. E., and Glauber, R. R. (1967). Multicollinearity in regression analysis: the problem revisited. Rev. Econ. Stat. 49, 92–107. doi: 10.2307/1937887

CrossRef Full Text | Google Scholar

Fiecas, M., Ombao, H., van Lunen, D., Baumgartner, R., Coimbra, A., and Feng, D. (2013). Quantifying temporal correlations: a test-retest evaluation of functional connectivity in resting-state fMRI. Neuroimage 65, 231–241. doi: 10.1016/j.neuroimage.2012.09.052

PubMed Abstract | CrossRef Full Text | Google Scholar

Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika 10, 507. doi: 10.2307/2331838

CrossRef Full Text | Google Scholar

Folstein, M. F., Folstein, S. E., and McHugh, P. R. (1975). Mini-mental-state: a practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 12, 189–198. doi: 10.1016/0022-3956(75)90026-6

PubMed Abstract | CrossRef Full Text

Friedman, J., Hastie, T., and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33, 1–22. doi: 10.18637/jss.v033.i01

PubMed Abstract | CrossRef Full Text | Google Scholar

Friston, K. J., Ashburner, J., Kiebel, S., Nichols, T., and Penny, W. D. (2007). Statistical Parametric Mapping: The Analysis of Funtional Brain Images, 1st Edn (Amsterdam; Boston, MA: Elsevier/Academic Press).

Google Scholar

Gaser, C., Volz, H.-P., Kiebel, S., Riehemann, S., and Sauer, H. (1999). Detecting structural changes in whole brain based on nonlinear deformations—application to schizophrenia research. Neuroimage 10, 107–113. doi: 10.1006/nimg.1999.0458

PubMed Abstract | CrossRef Full Text | Google Scholar

Greicius, M. D., Srivastava, G., Reiss, A. L., and Menon, V. (2004). Default-mode network activity distinguishes Alzheimer's disease from healthy aging: evidence from functional MRI. Proc. Natl. Acad. Sci. U.S.A. 101, 4637–4642. doi: 10.1073/pnas.0308627101

PubMed Abstract | CrossRef Full Text | Google Scholar

Grothe, M. J., Teipel, S. J., and Alzheimer's Disease Neuroimaging Initiative (2016). Spatial patterns of atrophy, hypometabolism, and amyloid deposition in Alzheimer's disease correspond to dissociable functional brain networks. Hum. Brain Mapp. 37, 35–53. doi: 10.1002/hbm.23018

PubMed Abstract | CrossRef Full Text | Google Scholar

Guo, C. C., Kurth, F., Zhou, J., Mayer, E. A., Eickhoff, S. B., Kramer, J. H., et al. (2012). One-year test-retest reliability of intrinsic connectivity network fMRI in older adults. Neuroimage 61, 1471–1483. doi: 10.1016/j.neuroimage.2012.03.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoerl, A. E. (1970). Ridge regression. Biometrics 26, 603.

Hoerl, A. E., and Kennard, R. W. (1970). Ridge regression - biased estimation for nonorthogonal problems. Technometrics 12, 55–67. doi: 10.1080/00401706.1970.10488634

CrossRef Full Text | Google Scholar

James, G. A., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning with Applications in R. New York, NY: Springer.

Google Scholar

Jovicich, J., Minati, L., Marizzoni, M., Marchitelli, R., Sala-Llonch, R., Bartres-Faz, D., et al. (2016). Longitudinal reproducibility of default-mode network connectivity in healthy elderly participants: a multicentric resting-state fMRI study. Neuroimage 124(Pt A), 442–454. doi: 10.1016/j.neuroimage.2015.07.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Khazaee, A., Ebrahimzadeh, A., and Babajani-Feremi, A. (2015). Identifying patients with Alzheimer's disease using resting-state fMRI and graph theory. Clin. Neurophysiol. 126, 2132–2141. doi: 10.1016/j.clinph.2015.02.060

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, W., Teipel, S., Mueller, S., Benninghoff, J., Wagner, M., Bokde, A. L., et al. (2012). Diagnostic power of default mode network resting state fMRI in the detection of Alzheimer's disease. Neurobiol. Aging 33, 466–478. doi: 10.1016/j.neurobiolaging.2010.04.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, Q., Dai, Z., Xia, M., Han, Z., Huang, R., Gong, G., et al. (2015). A connectivity-based test-retest dataset of multi-modal magnetic resonance imaging in young healthy adults. Sci. Data 2:150056. doi: 10.1038/sdata.2015.56

PubMed Abstract | CrossRef Full Text | Google Scholar

Magnotta, V. A., Friedman, L., and First, B. (2006). Measurement of signal-to-noise and contrast-to-noise in the fBIRN multicenter imaging study. J. Digit. Imaging 19, 140–147. doi: 10.1007/s10278-006-0264-x

PubMed Abstract | CrossRef Full Text | Google Scholar

McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., and Stadlan, E. M. (1984). Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of the Department of Health and Human Services Task Force on Alzheimer's disease. Neurology 34, 939–944. doi: 10.1212/WNL.34.7.939

PubMed Abstract | CrossRef Full Text | Google Scholar

Meindl, T., Teipel, S., Elmouden, R., Mueller, S., Koch, W., Dietrich, O., et al. (2010). Test-retest reproducibility of the default-mode network in healthy individuals. Hum. Brain Mapp. 31, 237–246. doi: 10.1002/hbm.20860

PubMed Abstract | CrossRef Full Text | Google Scholar

Morris, J. C., Heyman, A., Mohs, R. C., Hughes, J. P., van Belle, G., Fillenbaum, G., et al. (1989). The Consortium to Establish a Registry for Alzheimer's Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer's disease. Neurology 39, 1159–1165. doi: 10.1212/WNL.39.9.1159

PubMed Abstract | CrossRef Full Text | Google Scholar

Murphy, K., Birn, R. M., Handwerker, D. A., Jones, T. B., and Bandettini, P. A. (2009). The impact of global signal regression on resting state correlations: are anti-correlated networks introduced? Neuroimage 44, 893–905. doi: 10.1016/j.neuroimage.2008.09.036

PubMed Abstract | CrossRef Full Text | Google Scholar

Nasreddine, Z. S., Phillips, N. A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al. (2005). The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 695–699. doi: 10.1111/j.1532-5415.2005.53221.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Orban, P., Madjar, C., Savard, M., Dansereau, C., Tam, A., Das, S., et al. (2015). Test-retest resting-state fMRI in healthy elderly persons with a family history of Alzheimer's disease. Sci. Data 2:150043. doi: 10.1038/sdata.2015.43

PubMed Abstract | CrossRef Full Text | Google Scholar

Power, J. D., Mitra, A., Laumann, T. O., Snyder, A. Z., Schlaggar, B. L., and Petersen, S. E. (2014). Methods to detect, characterize, and remove motion artifact in resting state fMRI. Neuroimage 84, 320–341. doi: 10.1016/j.neuroimage.2013.08.048

PubMed Abstract | CrossRef Full Text | Google Scholar

Schouten, T. M., Koini, M., de Vos, F., Seiler, S., van der Grond, J., Lechner, A., et al. (2016). Combining anatomical, diffusion, and resting state functional magnetic resonance imaging for individual classification of mild and moderate Alzheimer's disease. Neuroimage Clin. 11, 46–51. doi: 10.1016/j.nicl.2016.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Sheline, Y. I., and Raichle, M. E. (2013). Resting state functional connectivity in preclinical Alzheimer's disease. Biol. Psychiatry 74, 340–347. doi: 10.1016/j.biopsych.2012.11.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Shirer, W. R., Jiang, H., Price, C. M., Ng, B., and Greicius, M. D. (2015). Optimization of rs-fMRI pre-processing for enhanced signal-noise separation, test-retest reliability, and group discrimination. Neuroimage 117, 67–79. doi: 10.1016/j.neuroimage.2015.05.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Shirer, W. R., Ryali, S., Rykhlevskaia, E., Menon, V., and Greicius, M. D. (2012). Decoding subject-driven cognitive states with whole-brain connectivity patterns. Cereb. Cortex 22, 158–165. doi: 10.1093/cercor/bhr099

PubMed Abstract | CrossRef Full Text | Google Scholar

Teipel, S., Drzezga, A., Grothe, M. J., Barthel, H., Chetelat, G., Schuff, N., et al. (2015). Multimodal imaging in Alzheimer's disease: validity and usefulness for early detection. Lancet Neurol. 14, 1037–1053. doi: 10.1016/S1474-4422(15)00093-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Teipel, S., Grothe, M. J., Zhou, J., Sepulcre, J., Dyrba, M., Sorg, C., et al. (2016). Measuring cortical connectivity in Alzheimer's Disease as a brain neural network pathology: toward clinical applications. J. Int. Neuropsychol. Soc. 22, 138–163. doi: 10.1017/S1355617715000995

PubMed Abstract | CrossRef Full Text | Google Scholar

Teipel, S. J., Bokde, A. L., Born, C., Meindl, T., Reiser, M., Moller, H. J., et al. (2007). Morphological substrate of face matching in healthy ageing and mild cognitive impairment: a combined MRI-fMRI study. Brain 130(Pt 7), 1745–1758. doi: 10.1093/brain/awm117

PubMed Abstract | CrossRef Full Text | Google Scholar

Teipel, S. J., Kurth, J., Krause, B., Grothe, M. J., and Alzheimer's Disease Neuroimaging Initiative (2015). The relative importance of imaging markers for the prediction of Alzheimer's disease dementia in mild cognitive impairment - beyond classical regression. Neuroimage Clin. 8, 583–593. doi: 10.1016/j.nicl.2015.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. B 58, 267–288.

Google Scholar

Tibshirani, R. (2011). Regression shrinkage and selection via the lasso: a retrospective. J. R. Stat. Soc. B 73, 273–282. doi: 10.1111/j.1467-9868.2011.00771.x

CrossRef Full Text | Google Scholar

Trzepacz, P. T., Yu, P., Sun, J., Schuh, K., Case, M., Witte, M. M., et al. (2014). Comparison of neuroimaging modalities for the prediction of conversion from mild cognitive impairment to Alzheimer's dementia. Neurobiol. Aging 35, 143–151. doi: 10.1016/j.neurobiolaging.2013.06.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Yan, C. G., Cheung, B., Kelly, C., Colcombe, S., Craddock, R. C., Di Martino, A., et al. (2013). A comprehensive assessment of regional variation in the impact of head micromovements on functional connectomics. Neuroimage 76, 183–201. doi: 10.1016/j.neuroimage.2013.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhou, J., Greicius, M. D., Gennatas, E. D., Growdon, M. E., Jang, J. Y., Rabinovici, G. D., et al. (2010). Divergent network connectivity changes in behavioural variant frontotemporal dementia and Alzheimer's disease. Brain 133(Pt 5), 1352–1367. doi: 10.1093/brain/awq075

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhuang, J., Tyler, L. K., Randall, B., Stamatakis, E. A., and Marslen-Wilson, W. D. (2014). Optimally efficient neural systems for processing spoken language. Cereb. Cortex 24, 908–918. doi: 10.1093/cercor/bhs366

PubMed Abstract | CrossRef Full Text | Google Scholar

Zou, H., and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67, 301–320. doi: 10.1111/j.1467-9868.2005.00503.x

CrossRef Full Text | Google Scholar

Zou, H., and Zhang, H. H. (2009). On the adaptive elastic-net with a diverging number of parameters. Ann. Stat. 37, 1733–1751. doi: 10.1214/08-AOS625

PubMed Abstract | CrossRef Full Text | Google Scholar