^{1}

^{1}

^{2}

^{3}

^{1}

^{*}

^{1}

^{2}

^{3}

Edited by: Valeriya Naumova, Simula Research Laboratory, Norway

Reviewed by: Edoardo Milotti, University of Trieste, Italy; Alexandre De Castro, Brazilian Agricultural Research Corporation (EMBRAPA), Brazil

This article was submitted to Computational Physics, a section of the journal Frontiers in Physics

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

An abdominal aortic aneurysm (AAA) is a gradual enlargement of the aorta that can cause a life-threatening event when a rupture occurs. Aneurysmal geometry has been proved to be a critical factor in determining when to surgically treat AAAs, but, it is challenging to predict the patient-specific evolution of an AAA with biomechanical or statistical models. The recent success of deep learning in biomedical engineering shows promise for predictive medicine. However, a deep learning model requires a large dataset, which limits its application to the prediction of the patient-specific AAA expansion. In order to cope with the limited medical follow-up dataset of AAAs, a novel technique combining a physical computational model with a deep learning model is introduced to predict the evolution of AAAs. First, a vascular Growth and Remodeling (G&R) computational model, which is able to capture the variations of actual patient AAA geometries, is employed to generate a limited

The aorta is a major artery in which blood flows through the circulatory system. To consider an enlargement of an aorta as an aneurysm, a relative criterion can be used such as that the enlargement of the aorta is greater than 50% of the normal diameter. On the other hand, an absolute criterion can also be used. For example, in the region of the infrarenal aorta, which lies between the renal branches and the iliac bifurcation, an aorta with maximum diameter greater than 3 cm is considered an Abdominal Aortic Aneurysm (AAA). Because the risk from open surgery or endovascular repair outweighs the risk of AAA rupture, surgical treatments are not recommended with AAAs less than 5.5 cm in diameter [

Traditionally, the maximum diameter has been considered a significant risk factor of AAA rupture and has been commonly used as the criterion for screening, surveillance, and intervention decision making. Current clinical practices for diameter measurement methods, however, vary depending on parameters such as plane of acquisition, axis of measurement, position of callipers, and selected diameter [

In recent years, notable advances in statistical tools have been implemented to predict the maximum diameter with longitudinal AAA scanning data. Sweeting and Thompson [

Deep learning and deep architectures in general have been applied in an enormous number of research areas, with the majority being in computer vision [

The training problem remained until 2007 when Hinton proposed a two-stage learning scheme based on the Restricted Boltzmann Machine (RBM) [

The two main contributions of this work are as follows. First, to address the fundamental problem of the limited longitudinal data size, a massive

To achieve these contributions, we employ a Deep Belief Network (DBN) to predict the AAA shape in a regression framework. In section 2 of this study, the inscribed sphere method [

Diagram of overall methodology. A massive number of

Two kinds of data, including patient data and

The longitudinal patient dataset is collected from multiple CT images taken from 20 patients at the Seoul National University Hospital. The demography is shown in

Demographic data of patients.

P01 | 3 | Male | 71 | [0, 203, 733] |

P02 | 3 | Female | 64 | [0, 494, 1357] |

P03 | 5 | Male | 65 | [0, 182, 361, 538, 728] |

P04 | 5 | Male | 74 | [0, 347, 702, 1054, 1223] |

P05 | 5 | Male | 66 | [0, 374, 1074, 1438, 2136] |

P06 | 5 | Male | 54 | [0, 386, 757, 1121, 1290] |

P07 | 5 | Male | 62 | [0, 227, 674, 1049, 1403] |

P08 | 3 | Male | 73 | [0, 97, 564] |

P09 | 4 | Male | 59 | [0, 522, 922, 1344] |

P10 | 4 | Male | 54 | [0, 399, 774, 1152] |

P11 | 3 | Male | 78 | [0, 523, 873] |

P12 | 4 | Male | 68 | [0, 543, 691, 874] |

P13 | 3 | Male | 71 | [0, 349, 714] |

P14 | 5 | Male | 67 | [0, 183, 366, 534, 709] |

P15 | 3 | Male | 72 | [0, 189, 526] |

P16 | 5 | Male | 72 | [0, 246, 421, 587, 783] |

P17 | 4 | Female | 65 | [0, 613, 1048, 1515] |

P18 | 4 | Male | 78 | [0, 309, 1976, 2310] |

P19 | 4 | Male | 64 | [0, 729, 922, 2359] |

P20 | 3 | Male | 57 | [0, 440, 999] |

There are various ways of measuring the maximum diameter of an AAA [

As a result, the IMDC of an AAA image is obtained. The IMDC incorporates the maximum diameters (

Examples of 2D profile curves of maximal diameters over the centerline obtained from

We utilize a G&R computational model, which is based on the Finite Element Method (FEM), to generate

Elastin contributes resilience and elasticity to the aortic tissue, but it degenerates over time and is irreplaceable. The degeneration in elastin causes a localized dilation of the aorta, leading to the weakening of the aortic wall as well as the increase of aortic diameter and wall stress. This study utilizes an axisymmetric FEM model to generate

where μ_{d} represents the mean of the Gaussian function estimated from image data of each patient; σ_{d} defines the standard deviation of the Gaussian function, which controls the area of degraded elastin. In particular, σ_{d} determines the initial loss of elastin, which further affects the stress-stretch and the geometrical state of the AAA, causing various geometrical features of the AAA.

The collagen fiber family is suggested to be an important material in supporting the main aortic wall [

where ^{c}(^{c}(0) represents the mass density of collagen fiber in a healthy aorta at reference time 0; σ^{k}(^{k}(_{g} controls the magnitude of the stress-mediated mass production rate, so a larger _{g} implies that the aorta is able to produce more collagen fiber to maintain the stability of mechanical properties under elastin degeneration. Therefore, _{g} plays a decisive role in controlling the self-repairing and evolutionary process of an aneurysm.

Those three parameters {_{g}, σ_{d}, μ_{d}} directly affect the large time-scale enlargement of the aneurysm; thus, each unique group of the three parameters yields a unique outcome of the axisymmetric G&R code. One example is shown in

One common disadvantage of the G&R model is that it is time-consuming. Therefore, it is not the optimal option for generating a massive dataset for the deep structure. In this study, we utilize a PCM method to reproduce a massive dataset by approximating a small dataset from the G&R simulation outputs. The details of the PCM are discussed in section 3 and an example is shown in

The G&R model takes a group of parameters γ = {_{g}, σ_{d}, μ_{d}}^{1}

where η(.) represents the G&R computational code. Due to the high demand of computational resource and time during the simulation, we shall approximate η(.) by utilizing a set of _{i}(γ)}, with

where _{i} are the regression coefficients. Given a set of functions {_{i}(γ)}, the regression coefficients {β_{i}} can be solved as follow.

The residual between the truth and the approximation is defined as

By applying the ordinary least squares estimation to (4), the optimal set of coefficients β_{i} is formulated in

where

where _{j} are weights and

Note that the quadrature points _{i}} by running the model at

Suppose that the input γ is a random vector with a known probability density function (PDF) π(γ), (6) can be transformed into the probability space as

Similarly, with the proper choice of _{j} and _{i}(γ), (8) becomes

where

_{j} are weights and γ_{j} are abscissas. Given a weight function W^{α}(1 − ^{β}, an optimal choice of _{N + 1}(γ),

The detailed proof of Theorem 3.1 is provided in Chap 3 of Villadsen and Michelsen [

_{i}(γ)},

_{N + 1}(γ)

The proof of Corollary 3.2 is provided in _{i}(γ)} in a recursive manner as follows.

In practice, we define the initial conditions

and the orthogonal polynomials can be obtained recursively by solving the equations

However, solving for high order polynomials (13) is time-consuming and error-prone. Thus, we use Favard theorem to compute the set of basis functions more efficiently.

_{i}(γ)},

_{i} and _{i} are real numbers, then_{i}(γ)}

In this study, we utilized _{i} = 〈γ_{i − 1}, _{i − 1}〉 and

The algorithm of the PCM, which consisting of the computation of collation points (part 1), the realizations of physical-based model at collocation points (part 2), the computation of coefficients (part 3), and the generalization of approximated outputs (part 4).

In this section, we introduce the constructing and training of DBN. A standard structure of the DBN [

The deep architecture of the DBN. Two layers of RBM are trained in an unsupervised manner (pre-trained) using CD-1 algorithm. The top layer utilizes a neural network sigmoid regression for the prediction.

Assume that we have two types of variables: the

where θ is the collection of

where

The parameter θ is estimated using the maximum likelihood estimation. However, the energy-based PDF in this study requires sampling of two conditional probabilities:

The RBM is introduced by posting an additional restriction: _{j} ∈ {0, 1}. Thus, the modified energy function is defined as [

The conditional PDF of the visible units given the hidden units can be computed as

Note that

where

The likelihood gradient can be computed by taking the derivative of

The expectation _{P(h|x)}[.] is also called

Optimization of (19) involves sampling from

Given the approximation to the derivative of

After the pre-training, the DBN is unfolded into a Neural Network (NN), which is further trained in a supervised manner. This supervised learning, i.e., fine-tuning, is considered the second step of the two-stage learning scheme. Specifically, during the fine-tuning, the pre-trained weights of the unfolded DBN are properly adjusted for a better ability in capturing patient-specific features of aortic enlargement from the patient data.

Our proposed two-step training model could be interpreted as a deep learning version of the Bayesian approach, where computer-generated data act as a prior distribution and the patient data for fine-tuning can be viewed as new measurements to compute the posteriori distribution for prediction [

In this section, we introduce the data processing step and demonstrate the effectiveness of our proposed predictive model using observations of patient-specific CT images.

Given CT images of AAAs taken from a patient, we can obtain IMDCs with regular time intervals. Let _{t,i} be an IMDC of the _{i} as the collection of the three adjacent IMDCs, and the prediction target _{i} as the IMDC at the next time step in the future,

Following _{i}, _{i}} from 20 patients. Specifically, 6 sets of the patient data from 6 different patients are randomly selected as testing data while the others are employed for pre-training.

Given the multiple sets of G&R input parameters γ = [_{g}, σ_{d}, μ_{d}], the G&R model associated with the PCM approximation can produce a large number of longitudinal 2D profile curves to capture the enlargement of AAAs during a time span, which can be transformed into the _{i}, _{i}}. In this study, we focus on predicting the aneurysm growth; hence, only those _{i}, _{i}} are collected in the training dataset.

In order to assimilate patient data and _{i} and _{i} are 243 and 81, respectively. Additionally, it has been shown that it is much simpler to train the RBM by the data with a zero mean and unit variance [

As described in section 4, the 32,900 sets of normalized

Diagram of the model training. The DBN is pre-trained by

For a deep structure, parameters, such as the number of hidden units and the number of epochs^{2}

Effect of the number of nodes in a 2-layer DBN on the model testing.

1,000 | 50 | 31 | 0.264 |

500 | 100 | 21 | 0.186 |

300 | 300 | 24 | 0.180 |

100 | 500 | 18 | 0.192 |

50 | 1,000 | 23 | 0.2 |

As aforementioned, one of the problems in the DBN approach is that the pre-training generates a large

Effect of the number of epochs on the prediction error. The RMSE and the fine-tuning training time are plotted in solid blue and dashed red lines, respectively. The training time increases linearly with the number of epochs, while the RMSE rapidly decreases at the beginning but converges at around 400 epochs.

The performance of our proposed method is compared to the nonlinear mixed-effects model, which has been used extensively as a powerful growth hierarchical model over the decades [

where _{i,j} and _{i,j} are the diameter and the associated time at the _{1}, _{2}] is the random-effects terms and _{0}, ⋯ , α_{2}] is the parameters vector, and ϵ_{i,j} is the independent error term, i.e.,

As it is shown in

The absolute and relative prediction errors of 6 testing samples under DBN and Mix-effects model.

^{*} |
||||
---|---|---|---|---|

P1 | 0.224 | 0.393 | 4.3 | 7.6 |

P2 | 0.181 | 0.645 | 3.1 | 11.2 |

P3 | 0.168 | 0.535 | 2.8 | 8.8 |

P4 | 0.147 | 0.655 | 2.3 | 10.2 |

P5 | 0.197 | 0.406 | 2.9 | 6.2 |

P6 | 0.165 | 0.494 | 3.1 | 7.3 |

Mean value | 0.180 | 0.521 | 3.1% | 8.6% |

The true value, the DBN model prediction and mixed-effects model prediction of IMDCs are shown in dotted dashed black (“true”), solid red(“DL prediction”), and dashed green lines(“ME prediction”), respectively.

The deep learning model, which is implemented on the MATLAB, can be trained within 30 s on a PC with a 3.3 GHz 10-core CPU and a 64 GB RAM (

A Monte-Carlo cross-validation method is performed to show the robustness of the proposed deep learning model. As the first step, 13 sets of eligible testing data, i.e., {_{i}, _{i}}, are collected from 20 patients' CT images. There are two criteria for choosing eligible data from a patient. The first criterion is that the eligible {_{i}, _{i}} should be the last set of data of the patient. The second criterion is that the number of raw CT images of the patient is at least four. As the second step, the cross-validation trials are independently performed 100 times under the deep structure of 2-layers DBN with 300 nodes on each layer. In each trial, we randomly choose 3 sets of test data out of the whole eligible dataset and leave the other sets of eligible data as training data to be used in the fine-tuning step. As a result of the Monte-Carlo cross-validation, 100 prediction errors (RMSEs) are independently collected, of which the mean and the standard deviation are 0.196 cm and 0.051 cm. The standard deviation is so small relative to the mean that it guarantees the robustness of the proposed method. Moreover, as a comparison, the average testing RMSE (0.18 cm) falls into the range of the standard deviation of the cross-validation result, thus supporting the test results shown in

This study utilized a physical G&R computational model combined with follow-up image data from 20 patients to predict the shape evolution of AAAs represented by IMDCs. To our knowledge, this is the first study that utilizes the deep learning technique to predict the shapes of AAAs in an evolutionary scheme based on a small dataset of follow-up images. The main difficulty in applying deep learning to predict AAA enlargement is the limited size of the training dataset, i.e., follow-up images of AAAs. In this study, we overcame this difficulty by proposing a work-flow, in which the DBN is pre-trained by massive

Besides combining deep learning and AAA prediction, the proposed study also contributes by making fast and accurate predictions of AAA enlargement. Following

Additionally, due to the high complexity of physics in patient-specific predictions of AAA, even high-fidelity physical models cannot promise to make accurate predictions. To enable accurate predictions, in this paper, we proposed a two-stage training approach: first, we pre-train our deep learning model with a computationally generated sizable dataset; second, we fine-tune the deep structure with the patient data for patient-specific predictions. In this study, the average prediction error is 0.180 cm, which is significantly small compared to the AAA diameters (3–8.5 cm). The results shown in

In our previous paper [

The maximum diameter, the largest value in an IMDC, is one of the most important factors to help decide whether to perform surgery or not in clinical practice, e.g., performing surgery if the maximum diameter is larger than 5.5 cm [

Additionally, we also utilized another method, i.e., the drop out technique from Hinton et al. [

There are also limitations in this study. The first limitation is that because the patient-specific IMDCs are 2D profile curves, we choose to generate the simulation data using a simple axisymmetric G&R model which also gives 2D profile curves. The use of an the axisymmetric G&R model is acceptable because it can predict reasonable patient-specific growth of AAAs under normal conditions, but it cannot capture the special features caused by aortic bending, such as proximal neck bend. Fortunately, the bending has little effect on the center parts of AAA and the simulation of maximum diameter, so it does not affect the key prediction results. In the future, 2D profile curves can be replaced with 3D patient-specific geometries in order to provide more accurate patient-specific predictions.

The second limitation is that the G&R code does not take other factors into account, such as hemodynamics, thrombus and surrounding tissues [

The third limitation is that we have to interpolate patient CT images into the

In the future, we expect to further enrich the variability of the training data by improving the physical models or collecting more patient data to incorporate the marginal situations. Specifically, we generate a large amount of data by varying three parameters in the computational G&R simulations, which may not be enough to capture all the different features in actual AAA evolution, e.g., intraluminal thrombus [

The datasets generated for this study are available on request to the corresponding author.

The studies involving human participants were reviewed and approved by the Institutional Review Boards at Seoul National University Hospital and at Michigan State University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements because the data was collected for a retrospective study.

SB and JC designed the probabilistic model and the deep learning framework. ZJ and HD carried out the implementation and calculation. ZJ and HD wrote the manuscript with input from all authors. SB, JC, and WL conceived the study and were in charge of overall direction and planning.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

This work has been supported in part by the National Heart, Lung, and Blood Institute of the National Institutes of Health (R01HL115185 and R21HL113857), National Science Foundation CAREER Award (CMMI-1150376), the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2018R1A4A1025986) and the Vietnam Education Foundation. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Note that this study includes the content which first appeared in the dissertation from one of the authors [

The Supplementary Material for this article can be found online at:

^{1}For the sake of notational simplicity, we also denote {_{g}, σ_{d}, μ_{d}} as {γ^{[1]}, γ^{[2]}, γ^{[3]}}.

^{2}Epochs are the number of times that the model is trained through the whole training set.