Echocardiographic video-driven multi-task learning model for coronary artery disease diagnosis and severity grading

Introduction: Echocardiography is a first-line noninvasive test for diagnosing coronary artery disease (CAD), but it depends on time-consuming visual assessments by experts.

Methods: This study constructed an echocardiographic video-driven multi-task learning model, denoted Intelligent echo for CAD (IE-CAD), to facilitate CAD screening and stenosis grading. A 3DdeeplabV3+ backbone and multi-task learning were simultaneously incorporated into the core frame of the IE-CAD model to capture the dynamic myocardial contours. Multifarious features reflecting local semantic structures were extracted and integrated to yield echocardiographic metrics such as ejection fraction, strain, and myocardial work. For model training and testing, we used a total of 870 echocardiographic videos from 290 patients with clinically suspected CAD at Beijing Hospital (Beijing, China), split at an 8:2 ratio. To evaluate the model's generalizability, we used an external dataset comprising 450 echocardiographic videos from 150 patients at Fuwai Hospital (Beijing, China).

Results: The IE-CAD model achieved an AUC of 0.78 and a sensitivity of 0.85 for detecting significant or severe CAD, with a pearson correlation coefficient of 0.545 for predicting the Gensini score. When applied to the external dataset, the model achieved an AUC of 0.77 and a sensitivity of 0.78 for detecting significant or severe CAD.

Discussion: Thus, the IE-CAD model demonstrated effective CAD diagnosis and grading in patients with clinical suspicion.

Trial registration: This work was registered at ClinicalTrials.gov on 05 April 2019 (registration number: NCT03905200).

1 Introduction

Coronary artery disease (CAD), a common ischemic heart disease, is a leading cause of death worldwide. CAD is caused by the buildup of fats and cholesterol in the walls of the coronary arteries. This process leads to a narrowing or blockage of the coronary arteries, resulting in heart ischemia. Early detection of CAD is vital to avoid myocardial infarction and improve prognostic outcomes of patients. CAD is diagnosed through a combination of physical exams, patient medical history, and a variety of tests such as electrocardiography, echocardiography, and coronary angiography. Coronary angiography, an invasive procedure that involves injecting dye into the coronary arteries and taking X-ray images, is considered the gold standard and definitive diagnostic test for CAD. Echocardiography is an ultrasound of the heart that provides information about coronary artery wall motion and chamber sizes. It is a non-invasive, cost-effective, and easily accessible imaging technique that is commonly used for CAD screening and evaluation (Task Force et al., 2013). However, the diagnosis by echocardiography is mostly grounded on visual assessment and requires considerable expertise. Emerging myocardial strain analysis by speckle tracking echocardiography (STE) has allowed a quantification of the active myocardial deformation and thereby expanded the echocardiographic approach for CAD detection to a new level (Malagoli et al., 2020). Nonetheless, STE still requires extensive experience in imaging data acquisition and interpretation.

With the advancement of Artificial Intelligence (AI) technology, machine learning algorithms have been used to build predictive models for CAD based on imaging and non-imaging Data (Muhammad et al., 2021; Zhang et al., 2021; Kigka et al., 2022). Currently, most AI-assisted diagnostic workflows for CAD are built upon computed tomography (CT), which has a number of disadvantages including radiation exposure, the use of contrast agent, and high cost (Chu et al., 2023; Ihdayhid et al., 2023). There have been few reports on CAD evaluation by AI-assisted echocardiography. In 2021, Salte and colleagues (Salte et al., 2021) constructed an AI model that was used to assess ventricular global longitudinal strain (GLS) based on STE videos. This model successfully performed automatic segmentation, motion estimates, and GLS measurements across various cardiac pathologies. In 2021, Upton et al. (2022) developed an AI-assisted image processing system to extract novel geometric and kinematic features from stress echocardiograms. This automated system identified patients with severe CAD with a specificity of 92.7% and a sensitivity of 84.4%, although disease severity was not quantified. Given that stress echocardiogram is a time-consuming test that may also have issues such as patient intolerance and poor image quality, it may not be suitable for early CAD screening and evaluation. Other advanced echocardiography techniques such as myocardial work have not yet been integrated with AI.

Traditional CAD diagnosis mainly relies on the percentage of coronary stenosis and often overlooks the anatomical complexity of the coronary tree. Relying solely on the degree of luminal narrowing may lead to underdiagnosis of the disease in certain patient populations (Mangla et al., 2017). The Gensini score takes into account the impact of the location of coronary stenosis on heart function and the cumulative effect of multivessel narrowing. As such, the Gensini score is commonly used to evaluate the severity of CAD (Rampidis et al., 2019). In this study, we developed an echocardiographic video-driven intelligent deep learning model for coronary stenosis evaluation and severity grading, which we named IE-CAD. In this model, transthoracic echocardiogram (TTE) videos of the standard apical 4-chamber (A4C), 3-chamber (A3C), and 2-chamber (A2C) views were used to calculate echocardiographic metrics including ejection fraction, GLS, and myocardial work, as well as the Gensini score that indicates the severity of coronary stenosis. We used this model to detect and evaluate CAD in subjects with clinically suspected CAD.

2 Materials and methods

2.1 Patients

This prospective, single-center clinical trial was registered at ClinicalTrials.gov on 05 April 2019. It represents a post hoc analysis of data from an IRB-approved prospective clinical trial with the registration number NCT03905200. Subject recruitment started in April 2019 and is still ongoing. Up to the time of the preparation of this article, the trial screened 330 subjects with clinically suspected CAD in Beijing hospital, Beijing, China. The inclusion criteria included (1) the presence of typical myocardial ischemia-related symptoms (e.g., shortness of breath, chest tightness, chest pain, and palpitation) or positive physical examination or blood test results, (2) the presence of sinus rhythm, and (3) 18 years of age or older. The exclusion criteria were (1) obstruction or pressure gradient between the aorta and left ventricle, (2) severe valvular heart disease or arrhythmia, (3) other extremely severe organ illnesses, and (4) poor image quality for speckle tracking. Out of the 330 subjects who were included in the initial screening, 40 were excluded and 290 were included in the data collection. The 290 subjects received baseline echocardiography 1 day before coronary angiography. A workflow chart of data collection is illustrated in Figure 1. We also included 150 subjects at Fuwai Hospital, Beijing, China according to the inclusion and exclusion criteria for external validation. This study was approved by our hospital Ethics Committee (reference number: 2020BJYYEC-021-02) and conducted in accordance with the Declaration of Helsinki. All subjects provided written informed consent.

Figure 1

Figure 1. Workflow diagram of data collection.

2.2 Echocardiogram

Echocardiogram was performed in accordance with the Recommendations from the American Society of Echocardiography (Lang et al., 2015; Nagueh et al., 2016). Echocardiographic videos were captured on GE’s Vivid E9 and Vivid E95 ultrasound machines. The data were analyzed offline using the EchoPac software (EchoPAC v204, GE Vingmed Ultrasound, Norway). The cardiac images that showed the clearest visualization of the myocardium were selected for further analysis. Echocardiographic metrics were derived from the analysis of relevant features extracted from the videos. These metrics included left ventricular ejection fraction (LVEF) and factors related to strain and myocardial work. The Biplane Simpson’s method was used to calculate LVEF. The subjects were categorized into significant or non-significant CAD based on their extent of coronary stenosis. Significant CAD had a stenosis percentage of ≥50% for left main coronary artery or ≥70% for non-left main vessels (Lawton et al., 2022; Upton et al., 2022). All other subjects were categorized as “non-significant” CAD. The subjects were also categorized into severe or non-severe CAD based on their Gensini scores. The Gensini Scores were calculated using a modified method that incorporates lesion severity, location-specific multiplying factors, and collateral adjustments for total occlusions or near-total stenoses (Rampidis et al., 2019). The subjects with severe CAD had a Gensini score of >= 15 points, and the non-severe subjects had a Gensini score of <15 points. In accordance with prior studies (Wang et al., 2019; Yokokawa et al., 2020; Darand et al., 2023), the cutoff value of 15 was selected to balance sample sizes across groups and to avoid underestimating the condition of patients categorized as non-significant, favoring a lower cutoff to include more cases. The categorization criteria were determined by an adjudication committee made up of 3 blinded, unbiased experts, including 1 board-certified cardiologist.

2.3 Data set

All echocardiographic videos and images were stored in the core echocardiography laboratory of our hospital. The 290 individuals in the internal dataset were randomly divided into a training set of 234 subjects and a test set of 56 subjects (8:2). The test set matched the training set in terms of significant vs. non-significant and severe vs. non-severe CAD distribution (P > 0.1) so that both subsets accurately represented the overall distribution within the study cohort. The videos from the 150 subjects included in the external dataset were acquired on either a Philips EPIQ 7C or CVX ultrasound system—both different from the system used for the internal data—and were processed with institution-specific protocols. This external dataset was used to evaluate the model’s generalizability across clinical settings, equipment, and acquisition protocols.

2.4 Data pre-processing

The raw echocardiographic video data were cleaned, organized, and transformed into a format suitable for deep learning model training. The videos were first converted from a DICOM file into an AVI video file, containing the sector-shaped region only. Each frame and its corresponding annotated segmentation mask had 256 × 256 pixels with a native aspect ratio. Considering the variations in video frame rate and the subjects’ heart rate, we pre-processed video data of a complete cardiac cycle for each subject, including 10 frames from the end-diastole to end-systole and another 10 frames from the end-systole to end-diastole. This resulted in a 20 × 256 × 256 video block with corresponding segmentation masks for each subject.

2.5 Development of the IE-CAD model

A methodology flowchart of the IE-CAD model is illustrated in Figure 2. Firstly, low-level and deep semantic features were extracted from the echocardiographic videos through an encoder. The extracted image features were processed through a classification decoder, a segmentation decoder, an echocardiographic parameter decoder, and a CAD decoder to yield cross-sectional classification results, cardiac structure segmentation masks, GLS curves, LVEF curves, global work efficiency (GWE), Gensini scores, and significant or non-significant CAD classification results. Specifically, the low-level features were used for image classification, while the deep semantic features, in combination with the low-level features, were utilized for cardiac structure segmentation. The deep semantic features were also employed for the assessment of cardiac metrics including LVEF, GLS, and GWE. As a result, the extracted image features were enriched to represent local features of cardiac structures, as well as global features related to cardiac function. These local and global features were subsequently integrated for predictive tasks related to CAD detection and stenosis grading.

Figure 2

Figure 2. Methodology flowchart of the IE-CAD model.

The detailed architecture of the low-rank multi-task deep learning model, including the number of layers, the number of filters in each convolutional layer, and the specific operations performed in each block, is illustrated in Figure 3; Table 1. The echocardiographic video clips were input into a 3D-DeepLabV3+ encoder backbone based on 3D-ResNet50 to generate low-level features (represented in blue in Figure 3) and deep semantic features (represented in green in Figure 3). The low-level features were first processed through a down-sampling network to match the dimensions of the deep semantic features and then through a view classification decoder to yield the view classification results. The deep semantic features were processed through a convolution block to generate the backbone features of the image (represented in dark green in Figure 3). After up-sampling, the backbone features were integrated with the low-level features before down-sampling, and the combined features were processed through a segmentation decoder to generate the segmentation masks of the ventricle and myocardium. The low-rank convolution block, compared to regular convolution block, can better extract sparse features, which can be utilized in multi-task learning to complement the primary features (Yang et al., 2024). The deep semantic features were further processed through three specific low-rank convolution blocks to extract sparse complementary features associated with GLS, LVEF, and GWE (represented in light green in Figure 3). The backbone and sparse features were integrated and processed through specific decoders to generate GLS, LVEF (in the form of curves), and GWE (in numerical values). The deep semantic features were again processed through a different low-rank convolution block to extract sparse complementary features associated with the Gensini score (represented in light green in Figure 3), which were fused with the backbone features, specialized features for GLS, LVEF, and GWE, and the low-level features used for view classification. The integrated features were processed through a score decoder and a CAD classification decoder to yield the Gensini score and CAD classification results, respectively.

Figure 3

Figure 3. Architectural diagram of the multi-task learning model.

Table 1

Table 1. Specifications of all network module architectures of the IE-CAD model.

2.6 Model training, testing and external validation

This model was trained in three steps. The first step involved training on view classification and heart structure segmentation. The cross-entropy loss function was used for training on view classification, while both the cross-entropy loss and Dice loss functions were employed for training on heart structure segmentation. The second step involved training on regression models for estimating GLS, LVEF, and GWE using the mean squared error loss. The third step involved training on the network for CAD classification and Gensini score prediction while the backbone weight fractions of view classification, heart structure segmentation, and metrics regression remained constant. The cross-entropy loss and mean squared error loss were used for training on CAD classification and Gensini score prediction, respectively. The entire training process consisted of three sequential phases: 30 epochs in the first step, 30 epochs in the second step, and 90 epochs in the third step. The final training results were generated from the last epoch. The training was performed on two NVIDIA A6000 GPUs, using the Adam optimizer and following a cosine annealing schedule. Each time the training loss function was altered, the learning rate was re-warmed, starting with an initial rate of 2 × 10⁻⁴ and a weight decay of 1 × 10⁻⁵. The batch size was 16, with parameters updated after every three gradient accumulations.

During model testing, the A4C, A3C, and A2C views of each echocardiogram were tested separately. The final Gensini score was defined as the maximum predicted value from the three views. The case was classified as significant CAD if one or more views were classified as significant. This approach maximizes the model’s sensitivity to detect CAD and minimizes underdiagnosis due to variations between different views. The same methods were applied to the external dataset to validate the generalizability of the IE-CAD model.

2.7 Statistical analysis

Continuous variables with normal distribution are presented as the mean and standard deviation, whereas those not conforming to normal distribution are presented as the median and interquartile range. Categorical variables were compared using the Chi-square test or Fisher’s exact test. Comparative analysis of continuous variables was conducted utilizing the t-test or Mann-Whitney U test. The performance of diagnostic measures was indicated by 95% confidence intervals. This study used the Dice coefficient to quantify cardiac structure segmentation performance, a common metric in image segmentation that measures the overlap between binary masks. The Mean Absolute Error (MAE), a metric that calculates the average absolute errors between the predicted and actual values, was used to assess the performance of the regression models for EF, GLS, and GWE prediction, with lower scores reflecting lower prediction errors. The predictive performance for significant and/or severe CAD was evaluated in terms of accuracy, precision, sensitivity, specificity, and the F1 Score and through receiver operating characteristic (ROC) curve analysis. The intra- and inter-observer variability of the strain indices was gauged by intraclass correlation coefficients. Test outcomes with a P-value of less than 0.05 were considered significantly different. All statistical computations were executed utilizing SPSS software version 26.0.

3 Results

3.1 Clinical characteristics and echocardiographic metrics of CAD patients

The 290 subjects who were included in data collection had an average age of 62.9 ± 9.9 years. There were 45 (15.5%) women and 245 (84.5%) men. A total of 122 (42.1%) subjects were diagnosed with CAD by coronary angiography. Table 2 summarizes the demographic and clinical characteristics of the 122 CAD-positive and 168 CAD-negative subjects and their echocardiographic metrics estimated using the IE-CAD model. Compared with the CAD-negative subjects, the CAD-positive subjects had a substantially higher Gensini score (45.77 vs. 13.18, P < 0.001). They also had higher incidence rate of diabetes (51.6% vs. 38.7%, P = 0.028) and elevated plasma levels of HbA1c (6.66% vs. 6.31%, P = 0.004), brain natriuretic peptide (BNP) (92.14 pg/mL vs. 37.92 pg/mL, P = 0.004), and creatinine (88.63 μmol/L vs. 73.18 μmol/L, P = 0.037). There were no significant differences in other characteristics between the two groups, including age, gender, body mass index (BMI), family history of CAD, smoking, hyperlipidemia, systolic and diastolic blood pressures, suspected myocardial ischemia by electrocardiogram, fasting plasma glucose, uric acid and LDL cholesterol (P > 0.05).

Table 2

Table 2. Summary of demographic and clinical characteristics of the CAD-positive and CAD-negative subjects and their echocardiographic metrics predicted by the IE-CAD model.

There were significant differences in most echocardiographic metrics between the two groups (P < 0.05), which included left ventricular posterior wall end-diastolic thickness, LVEF, regional wall motion abnormality, GLS, peak strain dispersion, global work index, global constructive work, global wasted work, GWE, global positive work, and global systolic constructive work. GLS is an indicator of the left ventricle deformation (strain) in the longitudinal direction (Voigt et al., 2015). Myocardial work incorporates both left ventricle deformation (strain) and afterload (pressure the heart must work against), allowing for a better evaluation of cardiac performance (Edwards et al., 2019; Guo et al., 2022). In regard to strain and myocardial work metrics, absolute values of GLS, global work index and GWE were significantly higher in non-significant CAD patients when compared to significant CAD patients (P < 0.001). These results supported the inclusion of GLS, global work index and GWE as assistant tasks in informative feature extraction for CAD evaluation. There were no significant differences between the two groups in left atrial diameter, left ventricular end-diastolic volume, and Mitral E/e’ ratio (P > 0.05), which were conventional echocardiographic indices.

3.2 Predictive performance of the IE-CAD model

3.2.1 Gensini score prediction

The Gensini score prediction results of the IE-CAD model are shown in Figure 4; Table 3. The maximal predicted scores across the A4C, A3C, and A2C views showed a Pearson correlation coefficient of 0.545 (Table 3), while the mean predicted scores across the three apical chamber views showed a slightly lower Pearson correlation coefficient of 0.540. Thus, the maximal predicted scores were used for subsequent CAD classification.

Figure 4

Figure 4. Scatter plots of Gensini scores predicted by the IE-CAD model vs. those determined by coronary angiography. Gensini label, Gensini scores by coronary angiography; Gensini pred, Gensini scores predicted by the IE-CAD model; A4C, the apical 4-chamber view; A3C, the apical 3-chamber view; A2C, the apical 2-chamber view; mean, mean Gensini scores predicted by the IE-CAD model across the A4C, A3C, and A2C views; max, maximal Gensini scores predicted by the IE-CAD model across the A4C, A3C, and A2C views.

Table 3

Table 3. Performance of the IE-CAD model in Gensini score prediction.

3.2.2 Echocardiographic metrics prediction

The cardiac structure segmentation and echocardiographic metrics prediction performance results of the IE-CAD model are presented in Tables 4, 5. Figure 5 shows sample cardiac structure segmentation results. The model’s predictions exhibited strong agreement with the ground truth annotations and maintained good temporal continuity throughout the complete cardiac cycle. This demonstrates the model’s effective capture of the structural and motion characteristics of both the left myocardium and left ventricle, providing a solid foundation for subsequent CAD-assisted diagnostic predictions. The MAEs for predicting GLS, LVEF, and GWE were 1.88%, 5.11%, and 2.6%, respectively. Compared to AI-assisted regression black box models for echocardiography (Christensen et al., 2024; Kwan et al., 2024), the IE-CAD model was able to extract latent features related to GLS, LVEF, and GWE more effectively, thus achieving a better performance in capturing the temporal myocardium features. By integrating these features into cross-sectional classification, cardiac structure segmentation, and model-learned features pertinent to CAD diagnosis, the IE-CAD model was able to achieve high performance in Gensini score prediction and CAD classification.

Table 4

Table 4. Performance of the IE-CAD model in cardiac structure segmentation.

Table 5

Table 5. Performance of the IE-CAD model in echocardiographic metrics estimation.

Figure 5

Figure 5. Sample Image of Cardiac Structure Segmentation. “A2C,” “A3C,” and “A4C” denote the corresponding standard image planes. The “IE-CAD” column displays the model’s predicted segmentation results, while the “GT” column shows the ground truth segmentation annotations. In the segmentation results, the red region indicates the predicted left myocardium, and the blue region indicates the predicted left ventricle.

3.2.3 Detection of significant CAD and severe CAD

The ROC curves for the detection of significant CAD and/or severe CAD (predicted Gensini score >= 15) by the IE-CAD model are presented in Figure 6. The detection performance results are presented in Table 6. The IE-CAD model detected significant CAD with an AUC of 0.77 and severe CAD with an AUC of 0.76. It detected cases that were classified as either significant or severe CAD but not both with an AUC of 0.78 and a sensitivity of 0.85. Finally, the model detected cases that were classified as both significant and severe CAD with an AUC of 0.75.

Figure 6

Figure 6. ROC curves illustrating the performance of the IE-CAD model in detecting significant CAD and/or severe CAD within the internal cohort.

Table 6

Table 6. Performance of the IE-CAD model for detecting significant CAD and/or severe CAD.

3.2.4 External validation of model performance

In the external dataset, the ROC curves for detecting significant CAD and/or severe CAD are presented in Figure 7, and the corresponding performance metrics are summarized in Table 7. The IE-CAD model achieved an AUC of 0.71 for detecting significant CAD and 0.75 for severe CAD. For cases classified as either significant or severe CAD (but not both), the model achieved an AUC of 0.77. For cases classified as both significant and severe CAD, the AUC was 0.71.

Figure 7

Figure 7. ROC curves illustrating the performance of the IE-CAD model in detecting significant CAD and/or severe CAD within the external validation cohort.

Table 7

Table 7. Performance of the IE-CAD model in external validation.

4 Discussion

In this study, an echocardiographic video-driven AI model denoted IE-CAD was established using a deep learning-based 3DdeeplabV3+ network for automatic myocardial motion estimation and cardiac structure segmentation. This model integrated traditional LVEF, GLS, and myocardial work measurements for CAD diagnosis and stenosis severity assessment. In a prospective single-center trial, the IE-CAD model automatically and accurately classified cardiac views, tracked myocardial motion, and estimated echocardiographic metrics such as GLS, LVEF, and GWE in subjects with clinically suspected CAD. This model detected significant CAD with an AUC of 0.77 and severe CAD with an AUC of 0.76, allowing rapid, noninvasive, and cost-effective CAD diagnosis and evaluation. As shown in Table 7; Figure 7, IE-CAD demonstrated strong generalizability on the external dataset, achieving an AUC of 0.71 for significant CAD detection and 0.75 for severe CAD detection. These results highlight the model’s robust performance across different imaging equipment and clinical settings. This study presents a novel integration of myocardial work metrics with GLS and LVEF within a multi-task learning framework for CAD diagnosis and severity grading. This approach builds on previous work and represents a significant advancement in leveraging echocardiographic data for comprehensive and automated CAD assessment (Salte et al., 2021; Upton et al., 2022).

Among all imaging techniques for diagnosing CAD, echocardiography is a common, first-line test thanks to its convenience, cost-effectiveness, and diagnostic performance. Echocardiogram-derived strain and myocardial work metrics can provide better evaluation of cardiac performance than conventional echocardiography and other diagnostic methods, and as a result, they have emerged as innovative instruments for CAD detection and evaluation (Smiseth et al., 2021; Guo et al., 2022; Guo et al., 2023). Although the measurement of strain and myocardial work from echocardiographic videos has been made a partially automatic process (Smiseth et al., 2021; Guo et al., 2022; Guo et al., 2023), it remains time-consuming and requires considerable professional expertise and hence, it is not suitable for early CAD screening in clinical practice. The recent years have seen a number of studies exploring fully automated GLS measurement using AI methods (Salte et al., 2021; Deng et al., 2022; Salte et al., 2023). Nonetheless, there have been few reports on AI models for fully automated assessment of myocardial work. The strength of the IE-CAD model lies in its ability to integrate fully automated assessment of GLS and myocardial work metrics to achieve rapid and accurate CAD diagnosis in a clinical setting.

Stress echo, a widely used non-invasive diagnostic tool, has reported AUCs ranging from 0.84 to 0.92 for detecting significant CAD, depending on interpreter experience and image quality (Picano et al., 2008; Banerjee et al., 2012). In a recent study by Upton et al. (Upton et al., 2022), an AI-assisted stress echo model achieved a specificity of 92.7% and a sensitivity of 84.4%, and an AUC of 0.93, demonstrating the potential of AI to enhance diagnostic accuracy. Resting echo interpretation, primarily based on strain parameters, has reported AUCs ranging from 0.68 to 0.80 for CAD detection (Billehaug Norum et al., 2015). Stress echo is generally more effective than resting echo in inducing myocardial ischemia and segmental wall motion abnormalities, which can lead to higher diagnostic performance. Additionally, the sample size of our study is relatively small compared to previous studies on resting and stress echo, primarily due to the labor-intensive frame-by-frame annotation required for labeling. Moving forward, we plan to expand our dataset, streamline the annotation process, and further optimize the model to enhance its diagnostic performance.

Compared to the degree of coronary artery stenosis alone, the Gensini score takes into account all major coronary arteries and their branches, providing a more comprehensive evaluation of the entire coronary tree. GLS is an indicator of the function and mechanical properties of the entire myocardium, and it can reveal areas of myocardial ischemia or injury throughout the heart. Thus, the Gensini score is closely linked to GLS. In this study, the maximal Gensini scores predicted by the IE-CAD model across the three standard apical views positively correlated with the Gensini scores determined by coronary angiography, showing a Pearson correlation coefficient of 0.545. This moderate correlation implies some prediction uncertainty. The Gensini score, while widely used in research, is not commonly adopted in clinical practice due to the lack of standardized cutoff values for defining severe CAD. However, the continuous nature of the Gensini score provides a more nuanced and precise representation of disease severity, making it particularly valuable for research and machine learning applications. In this study, we utilized the Gensini score as a continuous “soft label,” allowing the model to learn a gradient of disease severity rather than relying on arbitrary categorical thresholds. This approach enables the model to associate varying levels of CAD with subtle features in echocardiographic images, potentially improving prediction accuracy. Our future work will aim to bridge the gap between research and clinical utility, by developing standardized cutoff values and further refining our model for clinical integration. Intriguingly, the IE-CAD model classified a group of subjects as non-significant CAD who had a Gensini score of >= 15 points. This group of patients could have been underdiagnosed for CAD and therefore, they should undergo further diagnostic tests.

In this study, most subjects classified as significant CAD by the IE-CAD model also had impaired resting myocardial function. The majority of the myocardial work and strain metrics were found to be valuable parameters for diagnosing significant CAD, particularly when there was >= 70% coronary artery stenosis. Consistent with earlier findings (Edwards et al., 2019), LVEF, regional wall motion abnormalities, GWE, and global work index were identified as notable factors for identifying significant CAD. Furthermore, diabetes mellitus and plasma levels of BNP, HbA1c, and creatinine were found to provide additional predictive value in the detection of CAD. No significant differences were detected between the significant and non-significant CAD cases in other major clinical indices such as BMI, hyperlipidemia, systolic and diastolic BPs, fasting blood-glucose, uric acid, and LDL cholesterol. Notably, many subjects included in this study were taking medications at the time of enrollment, which may have influenced the levels of these clinical indices.

The IE-CAD model was developed by employing several innovative approaches. A 3DdeeplabV3+ backbone and multi-task learning were simultaneously incorporated into the core frame of its image processing network, enabling automatic tracking of the endocardial border in over 90% of cases. Previous AI-assisted echocardiogram segmentation models typically annotated only two frames of the images that correspond to the end-diastole and end-systole of the heart (Batool et al., 2023), which were not well-suited for the assessment of GLS. The IE-CAD model annotated a total of 20 frames to capture myocardial motions during a complete cardiac cycle, including 10 frames from the end-diastole to end-systole and another 10 frames from the end-systole to end-diastole. This enabled the IE-CAD model to capture the dynamic myocardial contours during a complete cardiac cycle to accurately predict GLS. A novel decoder-focused method for multi-task dense prediction, called Mixture-of-Low-Rank-Experts (MLoRE) (Yang et al., 2024), was employed to configure global task relationships in the IE-CAD model. In particular, the sparse features extracted by low-rank convolution were utilized to complement the primary features in multi-task learning to enhance the model’s predictive performance.

Future work could explore multimodal input strategies to overcome the limitations inherent in single-modality data. Integrating clinical data, such as laboratory results and ECG findings, with echocardiographic data could provide a more comprehensive assessment. Techniques such as weighted feature fusion could facilitate this integration. Additionally, incorporating more echocardiographic views (e.g., short-axis views) and modalities (e.g., color and spectral Doppler) could provide richer insights into cardiac structure, function, and hemodynamics. From a modeling perspective, exploring different algorithms for multimodal data integration, such as combining CNNs for spatial image analysis with RNNs for temporal dynamics could better capture the complexity of cardiac motion. Advanced fusion techniques, such as attention mechanisms, may further enhance the integration of multimodal data and improve diagnostic performance.

4.1 Limitations

This research had a number of limitations. Firstly, this study involved only two centers with a relatively small sample size, which may limit the generalizability of the findings. Secondly, the raw data were obtained on a single ultrasound machine and processed using a single data-processing algorithm before they were loaded to the IE-CAD model. This might have caused a certain level of data bias that limits the generalizability of the model. To enhance the model’s generalizability, future research should involve data from multiple centers and incorporate an adaptive learning approach that allows the model to evolve with the introduction of new data. Thirdly, the classification of “significant CAD” in this study was based on the clinical doctors’ interpretation of coronary angiography, without quantitative measurements of coronary stenosis to confirm the severity of the disease. Thus, the reliability of coronary angiography interpretation could have potentially affected the results.

In future research, new network architectures such as transformer or mamba may be incorporated into the model to improve its predictive performance. In addition, the model could be improved by automatically incorporating patients’ other diagnostic reports for rapid extraction of CAD-related features.

5 Conclusion

A fully automated AI model denoted IE-CAD was developed to automatically “read” the echocardiographic videos for CAD diagnosis and stenosis severity assessment. This model extracts informative ultrasound video features to yield multiple echocardiographic metrics such as LVEF, myocardial work and GLS and incorporates these factors for CAD diagnosis and coronary stenosis assessment. This model demonstrated good sensitivity and accuracy in a single center prospective trial, and it may be used for early CAD screening in a clinical setting.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Ethics Committee of Beijing Hospital (reference number: 2020BJYYEC-021-02). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

YG: Formal Analysis, Writing – original draft. Y-HC: Writing – original draft. TX: Formal Analysis, Writing – review and editing. X-YS: Data curation, Writing – review and editing. H-XG: Data curation, Writing – review and editing. MD: Data curation, Writing – review and editing. DN: Writing – review and editing. HL: Data curation, Resources, Validation, Writing – review and editing. FW: Writing – review and editing. W-FX: Conceptualization, Supervision, Writing – review and editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This study was supported by the National High Level Hospital Clinical Research Funding (BJ-2024-150, BJ-2023-107) and the CAMS Innovation Fund for Medical Sciences (CIFMS) (2023-I2M-C&T-B-117, 2023-I2M-C&T-B-056).

Acknowledgments

The authors thank Mr. Zhi-Cheng Zhu and Mr. Jin-Chuan Wang at the GE Company for their technical support. These individuals did not influence the study direction or the writing of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fbioe.2025.1556748/full#supplementary-material

References

Banerjee, A., Newman, D. R., Van den Bruel, A., and Heneghan, C. J. I. j.o.c.p. (2012). Diagnostic accuracy of exercise stress testing for coronary artery disease: a systematic review and meta-analysis of prospective studies. Int. J. Clin. Pract. 66 (5), 477–492. doi:10.1111/j.1742-1241.2012.02900.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Batool, S., Taj, I. A., and Ghafoor, M. (2023). Ejection fraction estimation from echocardiograms using optimal left ventricle feature extraction based on clinical methods. Diagn. (Basel) 13 (13), 2155. doi:10.3390/diagnostics13132155

PubMed Abstract | CrossRef Full Text | Google Scholar

Billehaug Norum, I., Ruddox, V., Edvardsen, T., and Otterstad, J. E. (2015). Diagnostic accuracy of left ventricular longitudinal function by speckle tracking echocardiography to predict significant coronary artery stenosis. A systematic review. BMC Med. Imaging 15 (1), 25. doi:10.1186/s12880-015-0067-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Christensen, M., Vukadinovic, M., Yuan, N., and Ouyang, D. (2024). Vision-language foundation model for echocardiogram interpretation. Nat. Med. 30 (5), 1481–1488. doi:10.1038/s41591-024-02959-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Chu, M., Wu, P., Li, G., Yang, W., Gutiérrez-Chico, J. L., and Tu, S. (2023). Advances in diagnosis, therapy, and prognosis of coronary artery disease powered by deep learning algorithms. JACC Asia 3 (1), 1–14. doi:10.1016/j.jacasi.2022.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Darand, M., Salehi-Abargouei, A., Vahidi Mehrjardi, M. Y., Feizi, A., Seyedhossaini, S. M., and Askari, G. (2023). Joint effects of paraoxonase 1 rs662 polymorphism and vitamins C/E intake on coronary artery disease severity (gensini and SYNTAX scores) and lipid profile in patients undergoing coronary angiography. Front. Nutr. 9, 1097411. doi:10.3389/fnut.2022.1097411

PubMed Abstract | CrossRef Full Text | Google Scholar

Deng, Y., Cai, P., Zhang, L., Cao, X., Chen, Y., Jiang, S., et al. (2022). Myocardial strain analysis of echocardiography based on deep learning. Front. Cardiovasc Med. 9, 1067760. doi:10.3389/fcvm.2022.1067760

PubMed Abstract | CrossRef Full Text | Google Scholar

Edwards, N. F. A., Scalia, G. M., Shiino, K., Sabapathy, S., Anderson, B., Chamberlain, R., et al. (2019). Global myocardial work is superior to global longitudinal strain to predict significant coronary artery disease in patients with normal left ventricular function and wall motion. J. Am. Soc. Echocardiogr. 32 (8), 947–957. doi:10.1016/j.echo.2019.02.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Guo, Y., Xia, C., Zhong, Y., Wei, Y., Zhu, H., Ma, J., et al. (2023). Machine learning-enhanced echocardiography for screening coronary artery disease. Biomed. Eng. Online 22 (1), 44. doi:10.1186/s12938-023-01106-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Guo, Y., Yang, C., Wang, X., Pei, Z., Zhu, H., Meng, X., et al. (2022). Regional myocardial work measured by echocardiography for the detection of myocardial ischemic segments: a comparative study with invasive fractional flow reserve. Front. Cardiovasc Med. 9, 813710. doi:10.3389/fcvm.2022.813710

PubMed Abstract | CrossRef Full Text | Google Scholar

Ihdayhid, A. R., Lan, N. S. R., Williams, M., Newby, D., Flack, J., Kwok, S., et al. (2023). Evaluation of an artificial intelligence coronary artery calcium scoring model from computed tomography. Eur. Radiol. 33 (1), 321–329. doi:10.1007/s00330-022-09028-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Kigka, V. I., Georga, E., Tsakanikas, V., Kyriakidis, S., Tsompou, P., Siogkas, P., et al. (2022). Machine learning coronary artery disease prediction based on imaging and non-imaging data. Diagn. (Basel) 12 (6), 1466. doi:10.3390/diagnostics12061466

PubMed Abstract | CrossRef Full Text | Google Scholar

Kwan, A. C., Chang, E. W., Jain, I., Theurer, J., Tang, X., Francisco, N., et al. (2024). Deep learning-derived myocardial strain. JACC Cardiovasc Imaging 17 (7), 715–725. doi:10.1016/j.jcmg.2024.01.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Lang, R. M., Badano, L. P., Mor-Avi, V., Afilalo, J., Armstrong, A., Ernande, L., et al. (2015). Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American society of echocardiography and the european association of cardiovascular imaging. J. Am. Soc. Echocardiogr. 28 (1), 1–39 e14. doi:10.1016/j.echo.2014.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Lawton, J. S., Tamis-Holland, J. E., Bangalore, S., Bates, E. R., Beckie, T. M., Bischoff, J. M., et al. (2022). 2021 ACC/AHA/SCAI guideline for coronary artery revascularization: a report of the American college of cardiology/american heart association joint committee on clinical practice guidelines. J. Am. Coll. Cardiol. 79 (2), e21–e129. doi:10.1016/j.jacc.2021.09.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Malagoli, A., Fanti, D., Albini, A., Rossi, A., Ribichini, F. L., and Benfari, G. (2020). Echocardiographic strain imaging in coronary artery disease: the added value of a quantitative approach. Cardiol. Clin. 38 (4), 517–526. doi:10.1016/j.ccl.2020.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Mangla, A., Oliveros, E., Williams, K. A., and Kalra, D. K. (2017). Cardiac imaging in the diagnosis of coronary artery disease. Curr. Problems Cardiol. 42 (10), 316–366. doi:10.1016/j.cpcardiol.2017.04.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Muhammad, L. J., Al-Shourbaji, I., Haruna, A. A., Mohammed, I. A., Ahmad, A., and Jibrin, M. B. (2021). Machine learning predictive models for coronary artery disease. SN Comput. Sci. 2 (5), 350. doi:10.1007/s42979-021-00731-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Nagueh, S. F., Smiseth, O. A., Appleton, C. P., Byrd, B. F., Dokainish, H., Edvardsen, T., et al. (2016). Recommendations for the evaluation of left ventricular diastolic function by echocardiography: an update from the American society of echocardiography and the european association of cardiovascular imaging. Eur. Heart J. Cardiovasc Imaging 17 (12), 1321–1360. doi:10.1093/ehjci/jew082

PubMed Abstract | CrossRef Full Text | Google Scholar

Picano, E., Molinaro, S., and Pasanisi, E. J. C. u. (2008). The diagnostic accuracy of pharmacological stress echocardiography for the assessment of coronary artery disease: a meta-analysis. Cardiovasc Ultrasound 6, 30–39. doi:10.1186/1476-7120-6-30

PubMed Abstract | CrossRef Full Text | Google Scholar

Rampidis, G. P., Benetos, G., Benz, D. C., Giannopoulos, A. A., and Buechel, R. R. (2019). A guide for gensini score calculation. Atherosclerosis 287, 181–183. doi:10.1016/j.atherosclerosis.2019.05.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Salte, I. M., Østvik, A., Olaisen, S. H., Karlsen, S., Dahlslett, T., Smistad, E., et al. (2023). Deep learning for improved precision and reproducibility of left ventricular strain in echocardiography: a test-retest study. J. Am. Soc. Echocardiogr. 36 (7), 788–799. doi:10.1016/j.echo.2023.02.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Salte, I. M., Østvik, A., Smistad, E., Melichova, D., Nguyen, T. M., Karlsen, S., et al. (2021). Artificial intelligence for automatic measurement of left ventricular strain in echocardiography. JACC Cardiovasc Imaging 14 (10), 1918–1928. doi:10.1016/j.jcmg.2021.04.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Smiseth, O. A., Donal, E., Penicka, M., and Sletten, O. J. (2021). How to measure left ventricular myocardial work by pressure-strain loops. Eur. Heart J. Cardiovasc Imaging 22 (3), 259–261. doi:10.1093/ehjci/jeaa301

PubMed Abstract | CrossRef Full Text | Google Scholar

Task Force, M., Montalescot, G., Sechtem, U., Achenbach, S., Andreotti, F., Arden, C., et al. (2013). 2013 ESC guidelines on the management of stable coronary artery disease: the task force on the management of stable coronary artery disease of the european society of cardiology. Eur. Heart J. 34 (38), 2949–3003. doi:10.1093/eurheartj/eht296

PubMed Abstract | CrossRef Full Text | Google Scholar

Upton, R., Mumith, A., Beqiri, A., Parker, A., Hawkes, W., Gao, S., et al. (2022). Automated echocardiographic detection of severe coronary artery disease using artificial intelligence. JACC Cardiovasc Imaging 15 (5), 715–727. doi:10.1016/j.jcmg.2021.10.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Voigt, J. U., Pedrizzetti, G., Lysyansky, P., Marwick, T. H., Houle, H., Baumann, R., et al. (2015). Definitions for a common standard for 2D speckle tracking echocardiography: consensus document of the EACVI/ASE/Industry task force to standardize deformation imaging. Eur. Heart J. Cardiovasc Imaging 16 (1), 1–11. doi:10.1093/ehjci/jeu184

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, X. B., Cui, N. H., Zhang, S., Liu, Z. J., Ma, J. F., and Ming, L. (2019). Leukocyte telomere length, mitochondrial DNA copy number, and coronary artery disease risk and severity: a two-stage case-control study of 3064 Chinese subjects. Atherosclerosis 284, 165–172. doi:10.1016/j.atherosclerosis.2019.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Yang, Y., Jiang, P.-T., Hou, Q., Zhang, H., Chen, J., and Li, B. (2024). Multi-task dense prediction via mixture of low-rank experts. Ieee T Pattern Anal. 46, 27927–27937. doi:10.1109/cvpr52733.2024.02638

CrossRef Full Text | Google Scholar

Yokokawa, T., Yoshihisa, A., Kiko, T., Shimizu, T., Misaka, T., Yamaki, T., et al. (2020). Residual gensini score is associated with long-term cardiac mortality in patients with heart failure after percutaneous coronary intervention. Circ. Rep. 2 (2), 89–94. doi:10.1253/circrep.CR-19-0121

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, J., Zhu, H., Chen, Y., Yang, C., Cheng, H., Li, Y., et al. (2021). Ensemble machine learning approach for screening of coronary heart disease based on echocardiography and risk factors. BMC Med. Inf. Decis. Mak. 21 (1), 187. doi:10.1186/s12911-021-01535-5

PubMed Abstract | CrossRef Full Text | Google Scholar