Edited by: Ludovico Minati, Tokyo Institute of Technology, Japan
Reviewed by: M. Justin Kim, University of Hawai’i at Mānoa, United States; Guido van Wingen, University of Amsterdam, Netherlands; Ahmed El-Gazzar, University of Amsterdam, Netherlands, in collaboration with reviewer GW
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
In recent years, deep learning (DL) has become more widespread in the fields of cognitive and clinical neuroimaging. Using deep neural network models to process neuroimaging data is an efficient method to classify brain disorders and identify individuals who are at increased risk of age-related cognitive decline and neurodegenerative disease. Here we investigated, for the first time, whether structural brain imaging and DL can be used for predicting a physical trait that is of significant clinical relevance—the body mass index (BMI) of the individual. We show that individual BMI can be accurately predicted using a deep convolutional neural network (CNN) and a single structural magnetic resonance imaging (MRI) brain scan along with information about age and sex. Localization maps computed for the CNN highlighted several brain structures that strongly contributed to BMI prediction, including the caudate nucleus and the amygdala. Comparison to the results obtained via a standard automatic brain segmentation method revealed that the CNN-based visualization approach yielded complementary evidence regarding the relationship between brain structure and BMI. Taken together, our results imply that predicting BMI from structural brain scans using DL represents a promising approach to investigate the relationship between brain morphological variability and individual differences in body weight and provide a new scope for future investigations regarding the potential clinical utility of brain-predicted BMI.
Over the last few years, the use of deep learning (DL) has become increasingly widespread in the analysis of neuroimaging data in several different application domains (
The potential of these methods lies partly in that—in contrast to conventional mass univariate analytical methods—machine learning in general and DL in particular allow statistical inferences at the individual level (
The above findings demonstrate how computational models aimed at predicting a certain biometric trait have potential clinical applicability. Here we investigated whether structural brain imaging and machine learning can be used for predicting a physical trait that is of significant clinical relevance—the body mass index (BMI) of the individual. The prevalence and disease burden of excessive body weight is on the rise globally (
Training a machine learning algorithm to predict individual BMI based on brain imaging data has several potential applications. On the one hand, once sufficiently accurate prediction performance is achieved, it is possible to investigate which features (e.g., structural properties of the brain) contribute significantly to the predicted value. This has the potential to provide complementary information regarding the relationship between brain structure and body weight, besides conventional neuroimaging approaches. On the other hand, it can pave the way for potential clinical applications, inasmuch as the discrepancy between the true and the predicted BMI might be related to individual differences in food intake regulation and associated propensity for future weight gain. This would be analogous to that how the difference between brain-predicted and chronological age is used to quantify health risks.
Here we apply, for the first time to our knowledge, DL to predict individual BMI based on brain imaging data. In particular, we employ a CNN for BMI prediction based on T1-weighted structural MR images, as well as information about the participants’ age and sex. This approach has the advantage of being able to use minimally preprocessed neuroimaging data as input and automatically learn a hierarchical set of representations suitable for solving the task at hand (
Once a well-performing model has been obtained and tested on new data, a logical next step is to try to make sense of why the model predicts what it predicts. While deep neural networks are usually regarded as “black boxes,” it is possible to give reasonable explanations for their predictions without elucidating the underlying mechanisms (
Since the present study represents one of the first attempts to apply Grad-CAM for analyzing neuroimaging data, we also intended to investigate the neural underpinnings of individual differences in body weight using a more conventional neuroimaging approach and compare the obtained results. To this end, we performed automatic anatomical processing using the FreeSurfer software and general linear modeling to examine the relationship between brain morphology and BMI. FreeSurfer implements the automatic reconstruction of the cortical surface as well as subcortical structure segmentation using a probabilistic atlas (
All analyses reported in this article include participants from the UK Biobank population cohort
All participants provided informed consent to participate in the UK Biobank study. The UK Biobank Research Ethics Committee (REC) approval number is 11/NW/0382. Detailed information on the consent procedure of UK Biobank are available at the following URL:
Data were acquired on Siemens Skyra 3T MRI scanners (Siemens Healthcare, Erlangen, Germany) at the UK Biobank imaging centers in Cheadle, Newcastle, and Reading. A standard Siemens 32-channel RF receive head coil was applied. The brain imaging protocol included a T1-weighted 3D magnetization-prepared rapid gradient echo (MPRAGE) sequence for structural imaging, using in-plane acceleration (iPAT = 2) and a field-of-view (FOV) of 208 × 256 × 256 with isotropic 1 mm spatial resolution.
Raw T1-weighted images were preprocessed by the UK Biobank team using an automated processing pipeline based on FSL tools (
Data on weight were collected using a Tanita BC418MA body composition analyzer (Tanita Corporation of America, Inc., Arlington Heights, IL, United States). A Seca 240 cm height measure (Seca Deutschland, Hamburg, Germany) was used to obtain standing height measurement from participants. Body mass index was calculated as follows:
Further details on the anthropometric measurements can be obtained from the following URL:
The age of each participant was derived from the date of birth (data-fields 34, 52) and the date of the imaging visit (data-field 21,003 instance 2) and was given in years with precision to the month. Sex was self-reported (data-field 31) and coded as 0 for female and 1 for male.
We used a CNN to predict BMI. The prediction of the model is based on three inputs from each subject:
T1-weighted brain image in MNI152 space, encoded in a Numpy
Chronological age of the participant in years with precision to the month.
Sex of the participant (0 for female and or 1 for male).
The output of the network is a single scalar corresponding to the predicted BMI of the subject.
A schematic illustration of the network architecture is given in
Schematic illustration of the architecture of the convolutional neural network used for predicting body mass index. The network comprises repeated blocks of 3D spatially separable convolutional layers followed by batch normalization and ReLU, with every other block followed by a pooling layer to subsample the input. Global average pooling is used to map the feature maps of the last block to a vector (with a single scalar for each feature map) that is fed into a fully connected hidden layer followed by a single output unit for BMI prediction. Dashed lines denote concatenation, S denotes stride.
Every other batch normalization layer is followed by max pooling (filter shape 3 × 3 × 3, stride = 2) to subsample the input images, and global average pooling is implemented after the last batch normalization layer to calculate the average intensity value of each feature map computed by the last convolutional layer. The output of this operation, along with the values representing age and sex, is fed into a fully connected hidden layer with 128 units and ReLU activation function. This hidden layer is connected to a single output unit, the activation of which corresponds to the predicted BMI value.
The CNN has 231,681 parameters overall, out of which 230,961 parameters are trainable. The model was implemented in Python using TensorFlow 1.13.
To examine whether information about age and sex was crucial for BMI prediction we also trained a network that was identical to the one described above, except that the values representing age and sex were not concatenated to the output of the global average pooling operation nor were they fed to the network in any other way.
The weights of the convolutional and fully connected layers were initialized using Xavier initialization (
The brain images of all participants were randomly assigned to disjoint training (
A single NVIDIA Quadro M4000 GPU was used to train the CNN, with a runtime of about 1 h per epoch.
We used transfer learning to investigate the generalizability of our approach. Transfer learning refers to the method of training a neural network on one dataset (the source domain) and then adapting the model to a different dataset and/or task (the target domain) by transfer and fine-tuning of the previously learned model weights. In our case, the UK Biobank dataset constituted the source domain and the Information eXtraction from Images (IXI) dataset
Images were randomly divided into disjoint training (
In order to obtain localization maps highlighting brain regions that are important for BMI prediction, we used a modified version of the Grad-CAM (
where
In the original formulation of Grad-CAM, which was developed to provide class-discriminative visualizations, a ReLU was applied to
Localization maps were computed for each individual in the UK Biobank test set. They were upsampled to match the size of the input images using spline interpolation (for details, see section “Neuroimaging”). Intensity values were standardized to have zero mean and unit variance. As all brain images were registered to MNI152 space, a voxelwise grand average localization map across all test subjects could be computed. The resulting map was thresholded at two standard deviations from the mean and superimposed on the ch2bet MRIcron
Based on the visualization provided by the modified Grad-CAM method, we performed further exploratory analyses to investigate the association between BMI and morphological variability in the human brain using the UK Biobank data. To this end, we randomly selected a subset of 200 participants from the test set, with the only constraint being that the male–female ratio and the distribution of chronological age and BMI remain similar to those in the overall test set. We used FreeSurfer 6.0
The volume-based stream of FreeSurfer (
The surface-based stream of FreeSurfer (
Based on the grand average localization map, we directly investigated the association between the morphology of the right middle temporal gyrus and BMI. In particular, we computed partial correlations to examine the relationship between BMI and surface area, mean thickness and curvature while controlling for age, sex, and total cortical gray matter volume.
Overall, results showed that our CNN model can be used to predict BMI with high accuracy. Prediction error on the validation set reached a minimum after 32 epochs (MAE = 2.41 kg/m2, STDAE = 1.93 kg/m2). The model generalized well to the brain images in the test set (
BMI prediction accuracy on the UK Biobank dataset. The scatterplot depicts the true (horizontal axis) and the CNN-predicted BMI (vertical axis) on the test set (
When training the network without feeding information about age and sex to it, it took longer to reach a minimum of prediction error on the validation set (after 41 epochs, MAE = 2.36 kg/m2, STDAE = 2.09 kg/m2). Nevertheless, the model generalized well to the test set images: MAE = 2.41 kg/m2; STDAE = 2.11 kg/m2; RMSE = 3.20 kg/m2; Pearson
When fine-tuning learned weights on the IXI dataset, validation error reached a minimum after 44 epochs (MAE = 2.53 kg/m2; STDAE = 2.00 kg/m2). We obtained reasonable BMI prediction on the IXI test set (
BMI prediction accuracy on the IXI dataset. The scatterplot depicts the true (horizontal axis) and the CNN-predicted BMI (vertical axis) on the test set (
The grand average localization map across all the 2000 subjects’ images in the test set is depicted in
Grand average localization map highlighting brain regions that strongly contribute to predicted BMI. Activation values are
Based on the localization map, two subcortical regions, the left caudate and amygdala, were selected for volumetric analysis in a subset of the test subjects (
BMI and subcortical volumes. Scatterplots depict the volumes of the caudate
Regarding the analysis of cortical morphometry, no significant association between BMI and cortical thickness or curvature was observed after correcting for multiple comparisons (FDR threshold at 0.05). However, a positive relationship was observed between BMI and the area of the isthmus cingulate in the right hemisphere (
Vertex-wise analysis of surface area using FreeSurfer. BMI is significantly associated with surface area in a right hemisphere cluster encompassing the isthmus cingulate cortex (when age, sex, and total cortical gray matter volume are controlled for). The cluster survived false discovery rate correction at threshold
In this proof-of-concept study, we established that a deep CNN can be used to predict individual BMI with high accuracy, based on a single structural MRI brain scan and information about age and sex. This finding is in line with the results of several previous studies showing gray and white matter structural alterations in obese individuals (
In particular, the localization map produced by the Grad-CAM method highlighted a set of brain regions including a portion of the left medial temporal lobe in the vicinity of the amygdala. The relationship between amygdalar volume and BMI was also confirmed by using FreeSurfer-based subcortical segmentation and partial correlation correcting for age and sex, which showed that higher BMI was associated with larger amygdalar volume. Previous studies using voxel-based (
Besides the commonalities, several discrepancies have been observed between the results of the Grad-CAM-based localization and the vertex-wise analysis using FreeSurfer. On the one hand, the vertex-wise analysis yielded a significant association between BMI and the surface area in a region corresponding to the isthmus cingulate in the right hemisphere. While at least one previous study reported a relationship between BMI and the morphology of the posterior cingulate cortex (
Besides being a promising tool for neuroscientific investigation, brain-predicted BMI may also have practical utility. We managed to adapt the CNN model to a novel dataset, suggesting that our method is more generally applicable to a variety of different MR scanner types. Coming back to the relationship between the amygdala and body weight, this brain structure has been shown to be involved in the evaluation of food cues (
Our findings provide proof of concept that individual BMI can be predicted with high accuracy from a single MRI scan using DL methods and suggest a relationship between the morphology of subcortical structures and body weight.
Publicly available datasets were analyzed in this study. This data can be found here: this research has been conducted using the UK Biobank Resource under Application Number 27236. All analyses reported in this paper include participants from the UK Biobank population cohort (
The studies involving human participants were reviewed and approved by UK Biobank Research Ethics Committee (REC; approval number: 11/NW/0382). The participants provided their written informed consent to participate in this study.
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This research has been conducted using the UK Biobank Resource under Application Number 27236.