A Nonlinear Simulation Framework Supports Adjusting for Age When Analyzing BrainAGE
- 1Laureate Institute for Brain Research, Tulsa, OK, United States
- 2Department of Mathematics, University of Tulsa, Tulsa, OK, United States
- 3Tandy School of Computer Science, University of Tulsa, Tulsa, OK, United States
- 4Department of Psychiatry, University of California, San Diego, La Jolla, CA, United States
Several imaging modalities, including T1-weighted structural imaging, diffusion tensor imaging, and functional MRI can show chronological age related changes. Employing machine learning algorithms, an individual's imaging data can predict their age with reasonable accuracy. While details vary according to modality, the general strategy is to: (1) extract image-related features, (2) build a model on a training set that uses those features to predict an individual's age, (3) validate the model on a test dataset, producing a predicted age for each individual, (4) define the “Brain Age Gap Estimate” (BrainAGE) as the difference between an individual's predicted age and his/her chronological age, (5) estimate the relationship between BrainAGE and other variables of interest, and (6) make inferences about those variables and accelerated or delayed brain aging. For example, a group of individuals with overall positive BrainAGE may show signs of accelerated aging in other variables as well. There is inevitably an overestimation of the age of younger individuals and an underestimation of the age of older individuals due to “regression to the mean.” The correlation between chronological age and BrainAGE may significantly impact the relationship between BrainAGE and other variables of interest when they are also related to age. In this study, we examine the detectability of variable effects under different assumptions. We use empirical results from two separate datasets [training = 475 healthy volunteers, aged 18–60 years (259 female); testing = 489 participants including people with mood/anxiety, substance use, eating disorders and healthy controls, aged 18–56 years (312 female)] to inform simulation parameter selection. Outcomes in simulated and empirical data strongly support the proposal that models incorporating BrainAGE should include chronological age as a covariate. We propose either including age as a covariate in step 5 of the above framework, or employing a multistep procedure where age is regressed on BrainAGE prior to step 5, producing BrainAGE Residualized (BrainAGER) scores.
Aging is a biological process that can affect behavioral and cognitive dimensions. Biological age as measured by telomere length deviates from an individual's chronological age as a result of environment, lifestyle, and genetics (Shammas, 2011). However, other measures of biological age that may be particularly relevant to psychopathology can involve structural and functional changes in the brain.
Several imaging modalities, including T1-weighted structural imaging (Franke et al., 2010), diffusion tensor imaging (Han et al., 2014; Lin et al., 2016), and functional MRI (Tian et al., 2016) have been used in conjunction with machine learning algorithms to predict an individual's age. Recently, integration of neuroimaging data of different feature types and across multiple modalities has been shown to improve age prediction (Erus et al., 2015; Liem et al., 2017; Gutierrez Becker et al., 2018). While the details vary according to modality, the general strategy has been to (1) extract image-related features, (2) build a model on a training set composed of healthy participants using these features to predict participant age, (3) apply that model to a testing set, producing a predicted age for each individual, (4) compute the difference between a participant's predicted age and chronological age (often referred to as Brain Age Gap Estimate, BrainAGE, or brain predicted age difference, brain-PAD), (5) test for relationships between other variables of interest and BrainAGE, and (6) make inferences about accelerated or delayed brain aging (Cole and Franke, 2017). Variables of interest have included physical fitness (Ritchie et al., 2017), physical activity (Steffener et al., 2016), cognitive impairment after traumatic brain injury (Cole et al., 2015), mortality risk in elderly participants (Cole et al., 2018), acute ibuprofen administration in healthy participants (Le et al., 2018), or status of various diseases and disorders such as diabetes (Franke et al., 2013), Alzheimer's disease (Gaser et al., 2013; Löwe et al., 2016), psychiatric disorders (Koutsouleris et al., 2014; Nenadić et al., 2017), and human immunodeficiency virus (Wilkins, 2017).
Support Vector Regression (SVR) with a radial kernel is a commonly used machine learning algorithm to predict age and compute BrainAGE (Franke et al., 2010), along with other methods such as Gaussian process and relevant vector regression (Drucker et al., 1997). The residual error of these age-predicting models, BrainAGE, is necessarily correlated with age, which results in an overestimation of the age of younger individuals and an underestimation of the age of older individuals. This is due to the fact that these algorithms, like all regression methods, are subject to the fundamental phenomenon of “regression toward the mean” (Galton, 1886). A theoretical basis for this phenomenon is presented in section Theoretical Basis for the Age-BrainAGE Correlation. In practice, the correlation between chronological age and BrainAGE is visually evident in many figures of chronological vs. predicted age (Franke et al., 2010; Cole et al., 2018). While most studies involving BrainAGE have not discussed the age-BrainAGE correlation, some have accounted for this correlation by using predicted age as the primary outcome, which is similar to the correction we propose (Erus et al., 2015; Habes et al., 2016).
The age-BrainAGE correlation may affect the apparent relationship between BrainAGE and variables of interest when these other variables are also related to age. In the clinical neuroscience domain, for example, we may be interested in covariates including physiological variables such as body composition or psychological measures of mood or testing performance, some of which have clear relationships with age. In this study, we examine the detectability of multiple covariate effects in both real and simulated data. Using real data, we characterized relationships between BrainAGE, age, and other variables of interest. Then, we generated a known “ground truth” with characteristics similar to what we observed in real data. In our simulation model, age has a direct effect on the variables of interest, which may in turn affect simulated imaging features. We include both linear and nonlinear effects at each level.
The goals of the current study are: (1) to highlight the universal correlation between chronological age and BrainAGE in theory and practice and (2) develop a general framework for simulating age-dependent data that can be used to investigate the effect of the age-BrainAGE correlation in subsequent analyses. One of the challenges of determining the best practices for using BrainAGE in statistical modeling is related to the fact that variables of interest may be related to age, but not directly related to accelerated or delayed brain aging. In that case, spurious relationships with BrainAGE may be observed. Our results strongly support the proposal that models including BrainAGE as an independent variable should be adjusted for chronological age as well.
We begin with a theoretical explanation for regression toward the mean and the concurrent correlation between the residuals and observed values for any regression. Then, we show in our own data the relationships between chronological age, BrainAGE, and other covariates of interest as a basis for the parameters in our simulations. Finally, we describe a simulation approach to generate data with a comparable age effect on brain image features and show how the age-BrainAGE correlation can contribute to observed relationships, even when the simulated independent variables do not associate with imaging features. The R scripts for simulation and analysis are publicly available on the GitHub repository https://github.com/lelaboratoire/BrainAGE-simulation.
Theoretical Basis for the Age-BrainAGE Correlation
Regression Toward the Mean
Consider n data points (yi, xi), i = 1, …, n used to fit a simple linear regression y = α+βx+ε. Least-square estimation leads to
where rxy is the Pearson correlation between x and y, sx and sy are the standard deviation, respectively. Substituting the formulas into the fitted values yields
In this setting, regression toward the mean refers to the phenomenon that the standardized predicted value of y is closer to its mean than that of x to its mean for any imperfect correlation, −1 < rxy < 1. The weaker the correlation, the greater the extent of regression toward the mean. For perfect correlations (|rxy| = 1), the standardized distance between the predicted value in y to its mean equals that of x to its mean and there is no regression toward the mean. The implication for BrainAGE is that the age of younger individuals tends to be overestimated and the age of older individuals tends to be underestimated.
Partition of Variance or Analysis of Variance (ANOVA)
In the general setting y = f(X) + ε, where X can be any dimension and f(·) can be any regression model, the variance of y is partitioned into a part that can be explained by X, and a part due to random error: . Then
For , and
where and .
Thus, unless predicts y perfectly with . The correlation formulas suggest that the correlation between residual and y decreases with the correlation between y and ŷ, i.e., prediction accuracy of . Supplementary Figure 1 illustrates this phenomenon using a simple simulation where y was a function of x plus random normal noise. As the noise decreases (and fit increases), the correlation between y and the residuals decreases as well.
In the context of BrainAGE, the goal is to find that best predicts chronological age (y) using brain measures as X, and BrainAGE is computed as . Because never predicts chronological age perfectly, BrainAGE remains correlated with age. When BrainAGE is used as the response variable in subsequent analyses to make inferences on a covariate Z, it is important to check whether Z is associated with chronological age. If Z is not associated with chronological age, then one may simply evaluate the bivariate association between BrainAGE and Z. On the other hand, if Z is associated with both chronological age and BrainAGE, chronological age may confound the relationship between BrainAGE and Z (Elwood, 1992) and should be taken into account. Confounding effects can be addressed at study design (e.g., randomization and matching) or in statistical analysis [e.g., stratification of the confounder or including the confounder as a covariate (Pourhoseingholi et al., 2012)]. For example, Franke et al. (2010) considered a variable Z that represents two groups (ill vs. healthy) and selected two groups of individuals with similar chronological age (so Z is not associated with chronological age) to compare their BrainAGE. In the current work, we include chronological age as a covariate and evaluate this approach in the context of BrainAGE.
We used two separate datasets to illustrate the correlation between BrainAGE and chronological age and the effect this can have on associations with covariates of interest. All data were collected at the Laureate Institute for Brain Research between 2009 and 2017. All protocols were approved by Western Institutional Review Board (www.wirb.com). Participants signed written informed consent and received financial compensation for their participation.
Structural MRI data were collected from 475 healthy volunteers (mean age ± sd = 30.5 ± 10.3 years; age range = 18–60 years; 259 female) between 2009 and 2017. Each participant was scanned in a 3T GE MR750 whole body scanner. Scans were acquired using axial T1-weighted MP-RAGE sequences with a 24 cm FOV, 256 × 256 acquisition matrix, 8-degree flip angle and 0.9375 × 0.9375 mm in-plane resolution with no gap. Other parameters varied within the following ranges: 5.736 to 6.292 ms TR, 1.896–2.104 ms TE, 0.9–1.2 mm slice thickness, with either an 8- (General Electric, Milwaukee, WI) or 32- (Nova Medical Inc., Wilmington MA) channel phased array coil. Healthy neuropsychiatric status was assessed using either the MINI-international Neuropsychiatric Interview (Sheehan et al., 1998) or the Structured Clinical Interview for DSM-IV (First et al., 2002).
Structural MRI data were collected from 489 (mean age ± sd = 34.6 ± 10.6 years; age range = 18–56 years; 312 female) participants as part of Tulsa 1000, a longitudinal observational study including people with mood/anxiety, substance use, eating disorders, and healthy controls. Inclusion criteria for the participant populations were Patient Health Questionnaire ≥10, Overall Anxiety Severity and Impairment Scale ≥8, Drug Abuse Screening Test >3, or SCOFF ≥2. Exclusion criteria included a history of significant brain trauma, neurological disorders, change in medication within 6 week prior to scanning, bipolar disorder, and schizophrenia. Scanning parameters for this dataset were: 24 cm FOV, 256 × 256 acquisition matrix, 186 axial slices, 0.9 mm slice thickness with no gap, TR/TE = 5/2.012 ms, using an 8-channel phased array coil (General Electric, Milwaukee, WI). Testing and training sets differed on mean age (t = 6.2, p < 0.0001, mean difference 4.2 years) and sex composition (χ2 = 8.2, p = 0.004).
All participants in the testing dataset also underwent an intense battery of assessments including self-report, clinical interviews, neuropsychological testing, and body composition analysis. For full details, please see (Victor et al., 2018). From these, we selected 154 measures, which were used to illustrate the normal range of correlations with age and how these can affect the relationship between BrainAGE and covariates of interest.
All images in both the testing and training sets were processed using Freesurfer version 6.0.0 (Dale et al., 1999) in order to produce gray/non-gray matter masks. Then, using a procedure similar to Franke (Franke et al., 2010) but implemented in AFNI, all gray matter masks were transformed to MNI space via affine transformation, smoothed with an 8 mm gaussian kernel, and downsampled to 8 × 8 × 8 mm voxels. This produced a set of 3,707 voxels per participant, with the value at each voxel representing the fraction of that voxel comprised of gray matter.
R (version 3.2.2) and R package caret (version 6.0.76) were used to fit a support vector regression (SVR) model with radial basis functions. The ε (tolerance margin) was fixed at and cost parameters were tuned using 5 repeats of 10-fold cross validation in the training set. The hyperparameter space was sampled using a grid search that fixed ε at 0.000145 and allowed cost to vary from 0.25 to 4,096. The final best model (cost = 2) was then applied to the testing set to produce one predicted age for each participant. BrainAGE was taken to be predicted age minus chronological age.
Additionally, we define the Brain Age Gap Estimate Residualized (BrainAGER) to be the residual of the regression of BrainAGE on age to remove the remaining linear bias of age. This way, we have a measure of deviation from expected age that is linearly uncorrelated with chronological age.
To investigate the effect of the age-BrainAGE correlation on subsequent modeling results, we simulated hierarchical correlation structures among brain features, chronological age and covariates using a generative biological model (Figure 1). We then generated two groups of independent variables. Within each group of variables, some are dependent on age and others are not. One group was used in the simulation of neuroimaging features, while the other was not. We randomly split the data set into two subsets, trained SVR on the training set and computed BrainAGE on the testing set. On the testing set, we conducted linear regressions of BrainAGE on all independent variables, both with and without chronological age. With 1,000 replications, we assessed the significance of the contribution from the independent variables by examining the distribution of the resulting p-values.
Figure 1. BrainAGE simulation and analysis framework (Equations 2–4). (A) Linear and non-linear age basis functions (ABFs) fi (orange, black, and blue lines). For a particular individual i, the ABFs are combined to create volume k's gray matter proportion vik (orange, black, and blue arrows) and age-dependent covariates of interest, Zij(A) with a different set of coefficients αj. (B) Some of the Zij are then fed back into the wik when generating volume vik, which leads to two levels of age association between covariate and BrainAGE. (C) Proportional gray matter volume (volumetric data) generated from non-linear combinations of ABFs. (D) Predicted-age and BrainAGE computed from simulated volumetric data and simulated chronological age with Support Vector-based regression; (E) Test for association between BrainAGE and covariates of interest.
A realistic simulation model should capture the properties of normal age-related brain volumetric data, such as brain region-dependent changes and nonlinear chronological age dependence (Fjell et al., 2013). A realistic simulation should also include the ability to generate age-dependent deviations from the normal population and age-dependent covariates that may influence BrainAGE nonlinearly. We consider a biological causal path model and develop a novel age-basis-function approach for simulating BrainAGE data with covariates (Figure 1 and Supplementary Figure 2).
Denoting age by A, we assumed an underlying (unobserved) biological process represented by m functions of age, denoted as fm(A), which we referred to as age basis functions (ABFs). Here, without a function space defined, the term “basis” is used loosely to indicate the elementary functions that can be combined linearly to form any variable of interest y:
In this study, we implemented three monotone decreasing ABFs that can generate a wide range of non-linear functions (Supplementary Figure 3), and used these ABFs to simulate covariates of interest and the features extracted from an imaging modality.
A covariate of interest Zj for participant i with chronological age Ai was generated by
where αmj is a covariate-specific weight and the covariate-specific error denotes a Gaussian noise with mean 0 and standard deviation σj.
Simulating imaging modality
The proportional gray-matter volume for voxel k of a participant i with chronological age Ai was generated by
or, in short, vik = f(Ai)+ ϵi, where ϵi represents Gaussian noise with mean 0 and standard deviation σν. This setting allows capturing within-participant correlations (4b) and spatial dependence within participants (4c):
Note that the weight function wmik(Ai) allows the weights of ABFs to vary across individuals and volumes, and as a function of an individual's chronological age.
To further make the imaging modality dependent on some covariates, we let
where wmk is the population mean weight for ABF fm at voxel k, and the participant level departures Di depends on the first q variables (covariates):
Other measurable variables, Zj>q, do not contribute to the weights deviation. In addition to the age-related imaging features that are generated from the ABFs, we also added 25% “background” features that do not correlate with age. Other parameters such as standard deviation of the noise ϵ were chosen with the objective of yielding R2 and MAE values that closely match our empirical results when the volumetric features were used as inputs to the support vector regression (SVR) model to estimate chronological age. Nevertheless, the choice of parameters and even the simulation design matrix do not affect the overall improvement in the regression that includes age as an explanatory variable from the regression without age.
Finally, we carried out linear regressions of the covariates of interest on BrainAGE, with and without including age as an explanatory variable in the model. Over 100 replications, we assessed the detectability of the covariates as significant contributors to BrainAGE by examining their p-value distributions. In the ideal case, we should detect relationships between BrainAGE and covariates .
1. Draw 1,000 age values from the uniform distribution U(20, 80).
2. For each m = 1, 2, 3, draw 100 wmik values from N(0, σw) for each region k.
3. Set αmj = 0 for some m and j (Supplementary Table 1). Randomly draw the remaining αmj from the uniform distribution U(−2, −1) to construct the j covariate for each participant i (Equation 4).
4. Construct the volumetric data set. For each imaging feature k of participant i (Equation 2), add noisy volumetric features that do not correlate with age.
5. Randomly apply 50% of the (age, volumetric) data for training and 50% for validation. Train the SVR model using the R package e1071 with hyperparameters set as default on the training set and apply the model on the validation set to compute the BrainAGE scores.
6. On the testing set, run linear regressions of BrainAGE on all covariates, with and without age.
7. Assess the significance of the covariates by looking at the confidence intervals of their coefficients as well as the distribution of the resulting p-values.
In steps 3 and 4, we simulated 16 covariate types in each of 1,000 replicate data sets (Supplementary Table 1). The 16 variables were simulated by using all 8 possible combinations of the three age basis functions. Half of them contributed to the weights wmik (A), which consequently affected the gray matter density. For example, Z2 and Z10 were both derived from only the linear basis function f1, but Z10 does not influence the aging.
Additionally, the complete simulation procedure was carried out for two scenarios: one with relatively large and another with relatively small effects of the covariates on BrainAGE. This was achieved by modifying the constant γ in Equation (3) so that, in one case, the final weights wmik have a larger fold change on the original weights. In particular, the fold change is computed as
where is the average of Dmik(A) across all ages.
Covariate Correlations With Age
Observed Pearson correlations between age and the 154 clinical variables ranged from −0.33 (PROMIS physical function) to 0.29 (waist circumference) (Supplementary Figure 4). Because any confounding effect of the correlation between age and covariates of interest is likely to be worse with larger correlations, we focused on simulated covariates that correlated with age with an r of up to 0.3.
Age Prediction Accuracy and Bias
After fitting on the training dataset, SVR achieved a mean absolute error of 4.84 years and explained 64% of the variance in age in the testing dataset (Figure 2a). This is comparable to the cross-validated performance on the training set, where MAE was 5.1 years and R2 was 0.59. The correlation between age and predicted age was 0.82. On the other hand, regression toward the mean lead to a negative relationship between age and BrainAGE (r = −0.63, Figure 2c). After removing the linear trend as shown in Figure 2c, we observed no relationship between age and BrainAGER (r = 0.001, Figure 2e). More explicitly, BrainAGE had a positive expected value at low chronological age and a negative expected value at high chronological age, while BrainAGER has an expected value of 0 regardless of actual age.
Figure 2. Similar out-of-sample R2 when applying SVR to predict age as well as negative correlation between BrainAGE and chronological age between T1000 data and simulated data. (a,b) Chronological age vs. predicted age in the testing dataset, with a mean absolute error (MAE) of 4.78 years and R2 = 0.65 in (a) and MAE = 5.15, R2 = 0.841 in (b). Overlaying black 45-degree line and blue regression line showed regression toward the mean. (c,d) Chronological age vs. BrainAGE (r = −0.63). Negative correlation between BrainAGE and chronological age indicates younger participants tend to have positive BrainAGE and old participants tend to have negative BrainAGE. (e,f) After removing the linear trend in (b,c), there is no relationship between age and BrainAGER (r = 0.001). BrainAGER has an expected value of 0, regardless of chronological age.
Relationships Among Age-Covariate, Covariate-BrainAGE, and Covariate-BrainAGER Correlations
In order to investigate the effect that the correlation between BrainAGE and chronological age can have on the conclusions of an imaging study, we computed the correlations between each of the covariates and age, BrainAGE and BrainAGER. Larger age-covariate correlations lead to larger differences in measured correlation between that covariate and BrainAGER or BrainAGE (Figure 3A, colored points far from the 45° line). When age did not correlate with a covariate, BrainAGE, and BrainAGER tended to give similar results (gray points, near the 45° line). When age positively correlated with covariates (e.g., BMI), BrainAGER gave more positive values, and when age negatively correlated with covariates (e.g., PROMIS physical function), BrainAGER yields more negative values. Similarly, the greater the variance explained by age, the greater the squared difference in r between using BrainAGE or BrainAGER (Figure 3B).
Figure 3. Relationship between age-covariate correlation and the difference in measured correlation. The difference between using BrainAGE and BrainAGER depends on the age-covariate relationship. (A) Covariate-BrainAGER correlations as a function of the covariate-BrainAGE correlation, with points colored according to the Age-Covariate correlation. The 45-degree line is shown, and covariates more strongly related to age are further from the line. (B) The squared difference in r between using BrainAGE and BrainAGER as a function of the variance explained by age.
Table 1 shows the top 22 variables that are significantly correlated with either BrainAGE or BrainAGER after FDR correction for 154 tests. Notably, 17 variables were related to BrainAGE, and the strongest relationships were among variables strongly correlated with age, including body composition (percent body fat r = −0.2, percent body water r = 0.2, percent dry lean mass r = 0.2) and sensation seeking (r = 0.18). BrainAGER was only significantly correlated with six variables including waist to hip ratio (r = 0.15), color naming scaled (r = −0.15), and lean body mass (r = 0.17).
Table 1. Correlation and significance after FDR adjustment of each covariate with BrainAGE (rBrainAGE, pBrainAGE) or BrainAGER (rBrainAGER, pBrainAGER).
Negative Correlation Between BrainAGE and Chronological Age in Simulated MRI Data
We set the parameters of our simulation algorithm to achieve realistic characteristics of experimental data, such as correlation distribution between volumes and chronological age and the negative correlation between computed BrainAGE and chronological age. This negative correlation was also present in previous models such as with Gaussian Process Regression (Cole et al., 2018) and Relevant Vector Regression (Franke et al., 2010). Simulated results closely mirrored empirical results. The simulated testing data had MAE of 4.58 years and R2 of 0.71 (Figure 2b). In our simulated data, we observed an overestimation of younger participant's ages and an underestimation of older participant's ages (Figure 2d). After removing the effect of age on BrainAGE, simulated BrainAGER had an expected value of 0 regardless of actual age (Figure 2f).
Reduction of False Discoveries in Regression That Include Age as Explanatory Variable
In the linear models regressing BrainAGE on the 16 covariates of interest with simulated large effect sizes (FC = 1.255), we observed the following: when age was not included as an explanatory variable, many age-related covariates were shown to have statistically significant association with BrainAGE (Figures 4A,C), even when they did not contribute to the weights that made up the neuroimaging features (Figure 4, orange boxplots above the horizontal). These false positives (FP) were simply the result of the relationship between these covariates and chronological age that are part of the BrainAGE's defining formula. Moreover, several covariates that were simulated to contribute to the brain structure volumes had p-values on average above 0.05 (Figure 4, blue boxplots below the horizontal).
Figure 4. Significance of linear regression of covariates with BrainAGE for 100 replicate simulations. Each data set contains 16 age-dependent covariates with differing age dependencies (linear and nonlinear) and effects on volumetric variation. Blue boxes are variables that have a direct (TRUE) effect on BrainAGE, orange boxes are variables that do not have a direct effect on BrainAGE (FALSE), and this effect is relatively large in the top (A,B) and small in the bottom (C,D) plots. Boxplots on the left (A,C)do not use age as an explanatory variable and models on the right (B,D) include age as an explanatory variable. “Significance” was measured by –log(p). Horizontal line is at –log(0.05).
When age was included in the regression as an extra explanatory variable, the significance increased (p-values decreased) for all variables that were generated to have an association with the imaging features, even variables that were already detected in the previous regression without age (Figures 4B,D). Further, the decrease in significance (increase in p-values) for unrelated covariates indicated a significant decrease in the number of false positives. Variation in the p-values across covariates came from their different (linear and nonlinear) age dependencies and effects on volumetric variation. In other words, the real “significance” of a covariate depended on from which age basis functions it was generated and how it affected the brain features (w1k, w2k, or w3k). Simulations with a smaller effect size (FC = 1.170, Figures 4C,D) showed a similar effect, though attenuated, for covariates that were contributors to wmk. The positive rate (true and false) across 100 replications is quantified in Supplementary Table 2. Values in this table represent the portion of each boxplot above the horizontal line, which is the TP rate for covariates that had an influence on imaging features and FP rates for covariates that did not.
This study aims to highlight the relationship between chronological age and BrainAGE and its transitive effect on the relationship between BrainAGE and covariates of interest that are also related to age. We propose a solution to this problem: either use BrainAGER, or in the simple case of post-hoc linear regression, use chronological age as a covariate in subsequent analyses. We developed a simulation framework to generate data with complex, but known, relationships between the original imaging features, age, and a set of covariates that may also be related to age. Then, we were able to quantify the effect that accounting for age has on the ability to detect actual and spurious correlations with covariates in subsequent analyses.
Our main findings can be separated into three parts: analytical, empirical, and simulated data results. The analytical results provide a theoretical basis for the age-BrainAGE correlation, and the analyses using real and simulated data demonstrate this effect in practice. For the empirical data, there were three main findings: (1) many variables that may be of interest are correlated with age with Pearson coefficients of up to r = 0.3, (2) BrainAGE is strongly negatively correlated with chronological age (r = −0.63 in our dataset), (3) BrainAGER provides a measure of deviation between predicted and actual age that is not dependent on age, and has substantially different correlations with covariates that are correlated with age when compared to BrainAGE.
Since it is unknown which covariates are actually related to premature aging, we then developed a simulation framework to generate synthetic data. Simulated data showed: (1) similar characteristics to actual data when used to train and test a model on separate datasets, and (2) increased detectability of true positives and decreased occurrence of false positives when accounting for the age-covariate relationship, with this being modulated by the size of the simulated effect on physiology.
Based on our observations in both real and simulated data, we recommend that the relationship between chronological age and BrainAGE should be accounted for. The two methods proposed in this study are either: (1) regress age on BrainAGE, producing BrainAGER, which is centered on 0 regardless of a participant's actual age or (2) include age as a regressor when doing follow-up analyses. In fact, these two methods will produce the same coefficients in the case of linear regression, with slightly larger t-statistics in the second case. The advantage of using BrainAGER is simplicity and generalizability; it could be used as the dependent variable in any arbitrary model, rather than being confined to simple linear regression. While the focus of this study is not to show specific correlates of premature aging, it is worth noting that 17 variables significantly correlated to BrainAGE whereas only 6 were related to BrainAGER, with 1 variable (PROMIS Alcohol Negative Consequences) overlapping between the two sets (Table 1). Thus, accounting for the age-BrainAGE relationship results in a vastly different set of positive findings and would lead to a remarkably different interpretation of these data. More explicitly, not correcting the age-BrainAGE correlation would lead to an extensive set of spurious results in this dataset.
There are a few cases where the age-BrainAGE correlation is not relevant. When comparing two groups with matched age, any differences in BrainAGE are not likely to be caused by the relationship with age. When the individuals being examined are in a restricted age range, there is not likely to be much contribution from the age-BrainAGE correlation. Also, when the variable of interest is not related to age, removing the effect of age makes almost no difference (Figure 3B). However, when these cases are not true, our findings suggest that we should include age as an explanatory variable in a final model that aims to detect association of brain anomalies with covariates of interest.
The magnitude of the age-BrainAGE correlation is directly related to the accuracy of the prediction model. The fact that the residuals are correlated with observed values is a characteristic of regression in general, regardless of the specific data domain, and has a theoretical basis described in section Theoretical Basis for the Age-BrainAGE Correlation. Several factors may decrease the model performance on our testing set, and thereby increase the age-BrainAGE correlation. Specifically, the distribution of age ranges in our samples is non-uniform, which may lead to more weight being given to the middle of the distribution. There are substantial differences between the testing and training sets we used including age, sex, and diagnosis. It may therefore be possible to improve model performance on the testing set by subsampling the training set to have a more uniform distribution of ages and to match the testing set on several factors. However, model performance is already comparable across testing and training sets (R2 of 0.59 and MAE of 5.1 years, compared to 0.64 and 4.84) and is comparable with what has been previously reported.
Although the simulation was carefully designed and executed, because of the model's complexity, we have not fully explored all scenarios with different simulation parameters. However, we have identified effect size as the most important parameter and showed how it influenced the results. When varying other parameters, we still observed a reduction in the number of false positives when age is included as an explanatory variable in the final regression (results not shown). Moreover, while determining the parameters, we aimed to obtain realistic patterns as we observed in real data, such as similar distributions of the correlation values.
By constructing and studying an appropriate generative model containing covariates that have linear and non-linear relationship with age, we demonstrated that the correlation between covariates and age should be considered when making inferences about the relationship between BrainAGE and these covariates.
This study was carried out in accordance with the recommendations of the Belmont report and the U.S. Department of Health and Human Services PART 46: PROTECTION OF HUMAN SUBJECTS. The protocol was approved by the Western IRB. All subjects gave written informed consent in accordance with the Declaration of Helsinki.
TL, RK, BM, H-WY, WT, and MP contributed to the design of the study. TL and RK wrote the manuscript. TL performed simulation analyses and RK performed empirical analyses. The T-1000 investigators contributed to the conception of the study, selection of the assessments and discussion regarding the interpretation of the results. All authors revised the manuscript critically for important intellectual content and approved the final manuscript.
This work has been supported in part by The William K. Warren Foundation, the National Institute of Mental Health Award Numbers K23MH112949 (SSK), K23MH108707 (RLA), K01MH096175-01 (WKS), and the National Institute of General Medical Sciences Center Grant Award Number 1P20GM121312. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnagi.2018.00317/full#supplementary-material
Cole, J. H., Leech, R., Sharp, D. J., and Alzheimer's Disease Neuroimaging, Initiative (2015). Prediction of brain age suggests accelerated atrophy after traumatic brain injury. Ann. Neurol. 77, 571–581. doi: 10.1002/ana.24367
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A., and Vapnik, V. (1997). “Support vector regression machines,” in Advances in Neural Information Processing Systems 9 (Cambridge, MA: MIT Press), 155–161.
Erus, G., Battapady, H., Satterthwaite, T. D., Hakonarson, H., Gur, R. E., Davatzikos, C., et al. (2015). Imaging patterns of brain development and their relationship to cognition. Cereb. Cortex 25, 1676–1684. doi: 10.1093/cercor/bht425
First, M. B., Spitzer, R. L., Miriam, G., and Williams, J. B. W. (2002). Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Non-patient Edition. (Washington, DC: SCID-I/NP).
Fjell, A. M., Westlye, L. T., Grydeland, H., Amlien, I., Espeseth, T., Reinvang, I., et al. (2013). Critical ages in the life course of the adult brain: nonlinear subcortical aging. Neurobiol. Aging 34, 2239–2247. doi: 10.1016/j.neurobiolaging.2013.04.006
Franke, K., Ziegler, G., Klöppel, S., and Gaser, C. (2010). Estimating the age of healthy subjects from T1-weighted MRI scans using kernel methods: exploring the influence of various parameters. NeuroImage 50, 883–892. doi: 10.1016/j.neuroimage.2010.01.005
Gaser, C., Franke, K., Klöppel, S., Koutsouleris, N., Sauer, H., and Initiative, A. D. N. (2013). BrainAGE in mild cognitive impaired patients: predicting the conversion to Alzheimer's Disease. PLoS ONE 8:e67346. doi: 10.1371/journal.pone.0067346
Gutierrez Becker, B., Klein, T., and Wachinger, C. (2018). Gaussian process uncertainty in age estimation as a measure of brain abnormality. NeuroImage 175, 246–258. doi: 10.1016/j.neuroimage.2018.03.075
Habes, M., Janowitz, D., Erus, G., Toledo, J. B., Resnick, S. M., Doshi, J., et al. (2016). Advanced brain aging: relationship with epidemiologic and genetic risk factors, and overlap with Alzheimer disease atrophy patterns. Transl. Psychiatry 6:e775. doi: 10.1038/tp.2016.39
Koutsouleris, N., Davatzikos, C., Borgwardt, S., Gaser, C., Bottlender, R., Frodl, T., et al. (2014). Accelerated brain aging in schizophrenia and beyond: a neuroanatomical marker of psychiatric disorders. Schizophr. Bull. 40, 1140–1153. doi: 10.1093/schbul/sbt142
Le, T. T., Kuplicki, R., Yeh, H.-W., Aupperle, R. L., Khalsa, S. S., Simmons, W. K., et al. (2018). Effect of Ibuprofen on BrainAGE: a randomized, placebo-controlled, dose-response exploratory study. Biol. Psychiatry Cogn. Neurosci. Neuroimaging. 3, 836–843. doi: 10.1016/j.bpsc.2018.05.002
Liem, F., Varoquaux, G., Kynast, J., Beyer, F., Kharabian Masouleh, S., Huntenburg, J. M., et al. (2017). Predicting brain-age from multimodal imaging data captures cognitive impairment. NeuroImage 148, 179–188. doi: 10.1016/j.neuroimage.2016.11.005
Lin, L., Jin, C., Fu, Z., Zhang, B., Bin, G., and Wu, S. (2016). Predicting healthy older adult's brain age based on structural connectivity networks using artificial neural networks. Comput. Methods Progr. Biomed. 125, 8–17. doi: 10.1016/j.cmpb.2015.11.012
Löwe, L. C., Gaser, C., Franke, K., and Initiative, for the A. D. N. (2016). The effect of the APOE genotype on individual BrainAGE in normal aging, mild cognitive impairment, and Alzheimer's Disease. PLoS ONE 11:e0157514. doi: 10.1371/journal.pone.0157514
Nenadić, I., Dietzek, M., Langbein, K., Sauer, H., and Gaser, C. (2017). BrainAGE score indicates accelerated brain aging in schizophrenia, but not bipolar disorder. Psychiatry Res. Neuroimaging 266, 86–89. doi: 10.1016/j.pscychresns.2017.05.006
Pourhoseingholi, M. A., Baghestani, A. R., and Vahedi, M. (2012). How to control confounding effects by statistical analysis. Gastroenterol. Hepatol. Bed Bench 5, 79–83. Available online at: https://search.crossref.org/?q=%22How+to+control+confounding+effects+by+statistical+analysis%22
Ritchie, S. J., Tucker-Drob, E. M., Cox, S. R., Dickie, D. A., Del C Valdés Hernández, M., Corley, J., et al. (2017). Risk and protective factors for structural brain ageing in the eighth decade of life. Brain Struct. Funct. 222, 3477–3490. doi: 10.1007/s00429-017-1414-2
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. (1998). The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychiatry 59, 22–33.
Steffener, J., Habeck, C., O'Shea, D., Razlighi, Q., Bherer, L., and Stern, Y. (2016). Differences between chronological and brain age are related to education and self-reported physical activity. Neurobiol. Aging 40, 138–144. doi: 10.1016/j.neurobiolaging.2016.01.014
Tian, L., Ma, L., and Wang, L. (2016). Alterations of functional connectivities from early to middle adulthood: clues from multivariate pattern analysis of resting-state fMRI data. NeuroImage 129, 389–400. doi: 10.1016/j.neuroimage.2016.01.039
Victor, T. A., Khalsa, S. S., Simmons, W. K., Feinstein, J. S., Savitz, J., Aupperle, R. L., et al. (2018). Tulsa 1000: a naturalistic study protocol for multilevel assessment and outcome prediction in a large psychiatric sample. BMJ Open 8:e016620. doi: 10.1136/bmjopen-2017-016620
Keywords: BrainAGE, simulation, false positives, SVR, MRI, aging
Citation: Le TT, Kuplicki RT, McKinney BA, Yeh H-W, Thompson WK, Paulus MP and Tulsa 1000 Investigators (2018) A Nonlinear Simulation Framework Supports Adjusting for Age When Analyzing BrainAGE. Front. Aging Neurosci. 10:317. doi: 10.3389/fnagi.2018.00317
Received: 30 March 2018; Accepted: 21 September 2018;
Published: 24 October 2018.
Edited by:Katja Franke, Universitätsklinikum Jena, Germany
Reviewed by:Guray Erus, University of Pennsylvania, United States
Franziskus Liem, Universität Zürich, Switzerland
Copyright © 2018 Le, Kuplicki, McKinney, Yeh, Thompson, Paulus and Tulsa 1000 Investigators. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Rayus T. Kuplicki, email@example.com
†These authors have contributed equally to this work
‡Tulsa 1000 Investigators: Robin L Aupperle, Jerzy Bodurka, Yoon-Hee Cha, Justin S. Feinstein, Sahib S. Khalsa, Jonathan Savitz, W Kyle Simmons, Teresa A Victor