Artificial Intelligence in Cardiac Imaging With Statistical Atlases of Cardiac Anatomy

In many cardiovascular pathologies, the shape and motion of the heart provide important clues to understanding the mechanisms of the disease and how it progresses over time. With the advent of large-scale cardiac data, statistical modeling of cardiac anatomy has become a powerful tool to provide automated, precise quantification of the status of patient-specific heart geometry with respect to reference populations. Powered by supervised or unsupervised machine learning algorithms, statistical cardiac shape analysis can be used to automatically identify and quantify the severity of heart diseases, to provide morphometric indices that are optimally associated with clinical factors, and to evaluate the likelihood of adverse outcomes. Recently, statistical cardiac atlases have been integrated with deep neural networks to enable anatomical consistency of cardiac segmentation, registration, and automated quality control. These combinations have already shown significant improvements in performance and avoid gross anatomical errors that could make the results unusable. This current trend is expected to grow in the near future. Here, we aim to provide a mini review highlighting recent advances in statistical atlasing of cardiac function in the context of artificial intelligence in cardiac imaging.


INTRODUCTION
The main function of the heart is to pump blood to the lungs and body. In order to maintain the equilibrium state of normal blood circulation, the heart continuously adapts its structure, shape, and function in response to physiological challenges and long-term environmental factors. From the onset of injury or disease, the heart starts a cascade of structural and morphological adaptations, known as cardiac remodeling. Common cardiac remodeling includes left ventricular dilatation, increasing ventricular mass, hypertrophy, aortic dilation, and systolic/diastolic functional alterations. When this condition is prolonged, cardiac function may deteriorate until symptoms become clinically evident and may eventually lead to heart failure (1). Here, we define cardiac remodeling to encompass a wide spectrum of physiological processes from adaptive remodeling in athlete's hearts (2) and normal aging process (3) to adverse remodeling in hypertensive heart disorder (4) and ischemia (5). It is therefore critical in the management of patients with heart disease to identify and quantify the different types of cardiac remodeling and associations with environmental and clinical factors and to predict the likelihood of adverse outcomes in the future.
The associations between traditional risk factors of cardiovascular disease (including smoking, raised blood pressure, raised serum cholesterol, and diabetes mellitus) and developing cardiac disease were discovered from large epidemiological studies such as the Framingham Heart Study (6). To better understand the mechanism of subclinical disease, before symptoms are clinically evident, modern imaging examinations were later included, such as in the Multi-Ethnic Study of Atherosclerosis (MESA) (7) and the UK Biobank study (8). These large-scale studies have enabled a massive increase of imaging data available for the investigation of variations in cardiac geometry and function by using statistical shape analysis, as well as providing training data for machine learning algorithms.
Modern cardiac imaging modalities include echocardiography, computed tomography (CT), and magnetic resonance imaging (MRI). Each modality has its own advantages and disadvantages, but MRI has unique attributes over the other modalities that have enabled large-scale imaging studies in the general population, including the study of 6,000 preclinical subjects in the MESA and 100,000 asymptomatic subjects in the UK Biobank. MR images are acquired without ionizing radiation, and tomographic analysis can be performed without any geometrical assumption. In a single examination session, cardiac MRI can provide anatomical and functional images of the heart and great vessels in multiple views with high contrastto-noise ratio, as well as high spatiotemporal resolution blood flow, microstructural tissue characterization, myocardial strain, blood perfusion, and scar images.
In this mini review, we focus on the rapid developments of machine learning combined with cardiac atlases. Although examples were taken mainly from cardiac MRI studies, these methods are generally extensible to other modalities. We first show how statistical shape analysis has enabled better understanding of cardiac shape remodeling within and between pathological groups. We then discuss current developments in machine learning to utilize the robustness of cardiac anatomy derived from statistical atlases to improve image analysis, including motion atlases to highlight the utility of dynamic data analysis vs. static analysis. Table 1 compares representative papers in each category. We conclude with a discussion of future perspectives of cardiac atlases in the context of artificial intelligence (AI) in cardiac imaging.

STATISTICAL CARDIAC ATLASES
Statistical atlases consist of maps of cardiac shape and function, which can be used to quantify the variation in the population and quantify the differences between cohorts. They can also be used to quantify shape scores in individual patients relative to standard population groups. For example, the Cardiac Atlas Project 1 (24) provides repositories of thousands of cardiac MRI studies (25) and benchmark data for the development of automated analysis algorithms, including segmentation of images (26) and shape analysis (27).
Two common atlas construction pipelines are shown in Figure 1, where both approaches lead to a comparable statistical analysis (28). In the first approach (9), images are analyzed to obtain the locations of cardiac landmarks (valve positions and the margins of the interventricular septum) and ventricular contours. The points are then mapped into 3D, and slice shifts due to breath-hold mis-registration are corrected. A 3D shape model template is then customized to the location of the landmarks and contours by minimizing the point-to-surface distances between the landmarks/contours and the model surfaces. Homologous points are then sampled from the surfaces and used to construct a point distribution model. This surface template fitting approach has also been translated to echocardiographic images where temporal resolution is much higher, as demonstrated in (20).
The second approach uses 3D images to establish a mean image template before generating cardiac mesh data. In (11), a high-resolution 3D MR template image and myocardial mesh are used. Each short axis image stack is then corrected for breath-hold mis-registration and registered to the template image using non-rigid image registration methods. For each case, a registration map is stored to give a mapping from subject space to template space at each voxel. The template mesh is then propagated to each subject using the inverse registration map. A point distribution model can then be calculated from the resulting homologous points. A similar approach was demonstrated in (12) by using CT images, with the advantage of high resolution and no breath-hold mis-registration in CT data.
Both these approaches benefit from recent advances in machine learning methods. Firstly, deep learning segmentation networks for cardiac images have been developed to enable fast generation of contours and landmarks (17,29); and secondly, deep learning has enabled fast computation of registration maps, which can be trained without extensive manual image annotation using image similarity as the loss function (23,30,31).

ATLAS MEASURES OF CARDIAC REMODELING
Let s ∈ R 3P be a shape vector with P homologous points in 3D. To extract shape parameters from a cohort or pathology group, a linear generative model is commonly applied, that is, where s ∈ R 3P is the mean shape estimated from the cohort, ∈ R M×3P is the linear decomposition matrix (defining modes of shape variation), and b ∈ R M is the shape parameter vector. If N is the number of patient shapes in the cohort and M < N, then Equation (1) is called a dimension reduction technique. Because each 3D point in this point distribution model encapsulates approximately the same anatomical location in the heart, the relative locations of neighboring positions are highly correlated, enabling the dimension reduction method to distill a small number of shape parameters. The most common dimension reduction method is principal component analysis (PCA), whereby shape modes are ordered by the amount of variance explained. Most of the shape variations can then be explained in terms of the first few principal modes of variation. In the MESA baseline imaging study, the PCA mode explaining the most shape variation was associated with the size of the heart, even after correction for patient height (9). This is a common finding because the first mode often relates to the amplitude of the studied descriptors. The second mode was associated with sphericity. Clinically, these first two PCA modes are known to be associated with adverse outcomes in both symptomatic disease and asymptomatic cohorts (32)(33)(34)(35).
PCA regression enables evaluation of the relationships between the PCA scores and clinical factors such as diabetes (9,28). However, PCA is an unsupervised dimension reduction method, and component modes do not in general map to recognizable shape characteristics (9,28). Supervised dimension reduction methods such as information maximizing component analysis have shown promise for quantifying the differences between a patient group and a control group, or two patient groups (36). Another approach is to combine dimensionality reduction with direct correlation with clinically defined remodeling indices such as ventricular volumes, wall thickness, and sphericity, by using the partial least squares method. Zhang et al. (37) applied this method in conjunction with a sequential orthogonalization algorithm to construct orthogonal shape scores, which are optimally matched with known clinical indices of remodeling. More general ways of characterizing the shape probability distribution have been investigated (38).
Gilbert et al. (28) found that both volume and surface cardiac atlases showed similar morphometric characteristics and similar relationships between risk factors and left ventricular shape. Thus, shape scores derived from atlases are robust to differences in construction methodology and quantify real anatomical relationships with cardiovascular risk factors. Morphometric scores were found to be more sensitive to cardiovascular risk factors than traditional measures of mass and volume. Mauger et al. (10) used a biventricular shape model to study right and left ventricular interactions in the UK Biobank study. A subdivision surface biventricular shape model was automatically customized to manually draw contours using a diffeomorphic least squares optimization algorithm. A control group sub-cohort consisting of 630 participants with no cardiovascular risk factors and normal cardiac parameters was used as a reference group to quantify shape differences due to traditional risk factors. Morphometric scores were computed using linear regression to quantify shape variations associated with prediction variables including sex, age, height, high cholesterol, high blood pressure, obesity, and smoking as well as diabetes, previous myocardial infarction, and angina. This regression approach enabled quantification of the effects of each prediction variable while controlling for the effects of the others.
In congenital heart disease, atlas-based analysis of shape variations can provide quantitative measures of deterioration before detection of symptoms. Sheehan et al. (39) developed a method for patient customization using a linear combination of database templates. This knowledge-based reconstruction method has shown accurate and rapid analysis of right ventricular shapes and volumes in patients with tetralogy of Fallot (39), dextro-transposition of the great arteries (40), and other types of congenital heart disease (41). A more dilated and spherical right ventricle was found in patients with transposition of the great arteries after atrial switch, with regional reduction in function at the base (42,43). These methods assume that the patient heart geometry is accurately represented by a linear combination of cases in the database. An alternative approach is to jointly estimate the shape and the underlying statistical shape model so that the statistical model can be automatically updated while analyzing new cases (44). Shape model templates have been constructed to describe common congenital pathologies, such as congenitally corrected transposition of the great arteries, enabling a wide range of pathologies to be accurately characterized (45). In singleventricle pathologies, with tricuspid atresia and Fontan repair, shape mode scores were able to quantify differences in shape and function, with more spherical ED shapes being associated with reduced longitudinal shortening (46). Atlas analysis in association with biomechanical analysis may be able to identify mechanisms underlying changes in function with developing disease (47).

DEEP LEARNING NETWORKS WITH CARDIAC SHAPE PRIORS
Deep learning is currently the state-of-the-art method for medical image feature extraction and supervised analysis. Its superior performance has surpassed any other traditional machine learning algorithms in many applications, including cardiac imaging (48,49). This success is mainly attributed to the automatic generation of optimal features, rather than relying on handcrafted features. This means that without significantly modifying the architecture, deep learning allows transfer of techniques, thereby shifting the data domain from one application, for example, natural image analysis, to another, for example, cardiac imaging. In addition, transfer learning directly reuses a pretrained network and fine-tunes to a new application domain. Examples include transfer learning of retinal image segmentation into cardiac vessels (50) or predicting cardiovascular risk from retinal fundus images (51). This flexibility and reusability of deep neural network architectures have led to rapid development. However, there are some limitations. Deep learning is prone to overfitting and usually cannot infer the anatomical correctness of the prediction results. The network's parameters are also sensitive to the data or cohort used during training (implicit bias). Statistical atlases or shape priors can therefore be integrated with deep learning to overcome these limitations. Thus, anatomical correctness can be imposed by enabling the network to learn the biological constraints as well as the measurement correlations.
Machine learning methods can add new quantitative analysis techniques to examine the relationships between shape features and clinical status, in addition to the traditional methods of linear or logistic regression. These are now being applied to statistical shape atlases to characterize differences in patient groups and predict outcomes. In the STACOM 2015 shape analysis challenge (27), various machine learning algorithms were compared on a benchmark dataset, and 11 groups participated to determine cardiac shapes of patients with myocardial infarction from healthy subjects. Five groups used the z-scores (standardized b vector in Equation 1) in different ways to classify myocardial infarction shapes. The training accuracies ranged between 0.93 and 0.98, whereas the test accuracies were 0.83-0.98. Shape atlases have been useful in identifying genetic mutations affecting left ventricular (LV) mass (52). Shape features associated with disease can be interpreted through visualizations using deep generative networks (53).
Incorporating cardiac anatomy in deep learning was demonstrated by Oktay et al. (13) with an anatomically constrained neural network. Two separate autoencoder networks were appended after the final predicted segmentation mask and the ground truth mask layers, which extracted features from mask images separately. A global shape similarity loss function calculated from the output of autoencoder networks was introduced as a way to constrain the optimization to follow the same shapes as the ground truth. Their results showed improved super resolution and segmentation accuracies in the long-axis view 2 by correcting mis-registration between image slices. Another shape-based loss function was also proposed by Yang et al. (54) to segment the right ventricle.
Alternatively, shape priors can be introduced directly inside a network (14)(15)(16). Zotti et al. (14) inserted a cardiac shape probability map before the final layer of a U-Net architecture to ensure that the output segmentation masks were valid. Chen et al. (15) also modified a U-Net architecture with cardiac shape priors, but they modified the bottom layer (feature extraction layer) by inserting short-axis and long-axis feature vectors trained independently from short-axis and long-axis cardiac MRI, respectively. Duan et al. (16) embedded a more specialized shape refinement subnetwork into the main segmentation and super resolution network. The subnetwork consisted of shape affine alignment, atlas selection, and non-rigid free form deformation registration operations. The network was able to generate smooth high-resolution 3D cardiac mesh data from low-resolution cardiac MRI.

DEEP LEARNING FOR STATISTICAL CARDIAC ATLASES
The ability of deep learning to learn non-linear relationships between different data domains and the high focus on segmentation have enabled several studies to directly link cardiac imaging and statistical shape analysis. In Equation (1), patientspecific shape parameters with population reference of are represented by b ∈ R M vectors. A statistically plausible new shape of s can be generated by setting values of b within ±2 √ σ , where σ is the eigenvalues from the PCA. Shape generation can also be performed by sampling from a probability distribution function learned from an atlas (38). Attar et al. (17) proposed a neural network model that learns how to directly predict shape parameters b given a combination of cardiac MRI and patient characteristics metadata [age, weight, height, body mass index (BMI), body surface area (BSA), heart rate, systolic blood pressure (SBP), diastolic blood pressure (DBP), sex, smoking status, and alcohol consumption]. Hence, the network was trained to predict statistically plausible b vector from images and metadata parameters to generate a 3D cardiac shape by using (Equation 1). Also, Clough et al. (18) used a variational autoencoder to generate interpretable representations of patients with low ejection fraction. This aids the interpretability of machine learning algorithm, which is vital to their acceptance in the clinical community.
A different approach to embed statistical shape parameterization into deep neural network was proposed by Painchaud et al. (19). A separate adversarial variational autoencoder was trained to generate a latent space of cardiac anatomy from mask images and was then connected to another anatomical variational autoencoder to correct errors after segmentation. Hence, this network (19) indirectly learned patient-specific parameters in the latent space without actually modeling how the latent space should be parameterized as in (17). The disentanglement of latent spaces is an active area of research and shows promise in factorizing anatomical representations from modality characteristics (55).

DYNAMIC ATLASES
Many of the features associated with cardiac pathology are manifest as changes in motion rather than changes in static shape. As the heart is responsible to deliver sufficient blood into the circulation system, the onset of cardiac diseases forces the heart to adapt its motion. Changes in cardiac shape deformation, myocardial strain, and strain rate are examples of important dynamic remodeling indices when building a cardiac motion or dynamic atlas. However, building a dynamic atlas is sometimes limited by the temporal resolution of the acquired imaging data, although combining two modalities, such as MRI and echocardiography (20), can increase the temporal resolution of the atlas considerably.
There are a significant number of cardiac applications that can get the benefit of machine learning from cardiac motion. In pulmonary hypertension, a motion atlas is combined with the latent space of autoencoder network to predict the survival rate (21). A machine learning system that combines motion atlas with non-motion data (ECG and clinical reports) has been demonstrated in the selection of patients with dyssynchrony for cardiac resynchronization therapy (22). The study of dynamic atlases will be a fruitful area of future research. Deep learning methods for combined shape and motion analysis are now being developed (23), which can be used to extend previous methods for motion atlasing (11).

DISCUSSION
A statistical atlas of cardiac anatomy is a powerful tool to analyze a patient-specific remodeling compared with the reference population. An abnormal cardiac shape can be quantified against a population reference, regional wall motion differences can be compared across pathological groups, and a hypothetical cardiac shape can further be predicted from a longitudinal study. Apart from that, a statistical atlas can be used as a reference by machine learning algorithms to constrain their analysis within valid anatomic boundaries.
In summary, we have reviewed three ways to integrate a statistical atlas into a machine learning framework. The first approach is to directly use individual shape atlas parameters, for example, the z-scores, as the training data. This approach needs homologous points generated from a shape modeling technique derived from images and a registration method to align points to remove variations in the global position and orientation. The effectiveness of this approach was demonstrated in the STACOM 2015 challenge. The second approach is to use statistical atlases as shape priors either as a way to measure shape similarity in a loss function or to add shape features to be learned inside the network. The third approach is to predict statistical shape parameters or a location in a shape-based feature space directly from images. This is a promising field for deep learning, because it can generate relationships between two completely different data domains.
In the future, statistical atlases show promise for augmenting deep learning methods, and vice versa. An atlas can add robustness to the prediction results because additional information on a reference population is included during the learning process. Atlases will also increase the interpretability of the AI process, which is critical for the acceptance of AI in health care.