MINI REVIEW article

Front. Cardiovasc. Med., 23 February 2021
Sec. Cardiovascular Imaging
https://doi.org/10.3389/fcvm.2021.648877

Routine Echocardiography and Artificial Intelligence Solutions

  • 1Amsterdam University Medical Centers -Location Academic Medical Center, Department of Cardiology, University of Amsterdam, Amsterdam, Netherlands
  • 2Amsterdam University Medical Centers -Location Academic Medical Center, Department of Biomedical Engineering and Physics, University of Amsterdam, Amsterdam, Netherlands
  • 3Amsterdam University Medical Centers -Location Academic Medical Center, Department of Radiology and Nuclear Medicine, University of Amsterdam, Amsterdam, Netherlands
  • 4Amsterdam Cardiovascular Sciences, Amsterdam University Medical Centers -Location Academic Medical Center, University of Amsterdam, Amsterdam, Netherlands
  • 5Department of Cardiology, University Hospital Brussel, Brussels, Belgium

Introduction: Echocardiography is widely used because of its portability, high temporal resolution, absence of radiation, and due to the low-costs. Over the past years, echocardiography has been recommended by the European Society of Cardiology in most cardiac diseases for both diagnostic and prognostic purposes. These recommendations have led to an increase in number of performed studies each requiring diligent processing and reviewing. The standard work pattern of image analysis including quantification and reporting has become highly resource intensive and time consuming. Existence of a large number of datasets with digital echocardiography images and recent advent of AI technology have created an environment in which artificial intelligence (AI) solutions can be developed successfully to automate current manual workflow.

Methods and Results: We report on published AI solutions for echocardiography analysis on methods' performance, characteristics of the used data and imaged population. Contemporary AI applications are available for automation and advent in the image acquisition, analysis, reporting and education. AI solutions have been developed for both diagnostic and predictive tasks in echocardiography. Left ventricular function assessment and quantification have been most often performed. Performance of automated image view classification, image quality enhancement, cardiac function assessment, disease classification, and cardiac event prediction was overall good but most studies lack external evaluation.

Conclusion: Contemporary AI solutions for image acquisition, analysis, reporting and education are developed for relevant tasks with promising performance. In the future major benefit of AI in echocardiography is expected from improvements in automated analysis and interpretation to reduce workload and improve clinical outcome. Some of the challenges have yet to be overcome, however, none of them are insurmountable.

Introduction

Echocardiography is the most commonly performed non-invasive cardiac procedure. It is the recommended imaging modality for most cardiac diseases for diagnostic and prognostic purposes by the European Society of Cardiology (17). Echocardiography has unique characteristics such as portability, high temporal resolution, absence of ionizing radiation and low-costs. Precise and reliable echocardiographic assessment is prerequisite for high-quality clinical decision-making (8).

Analysis of echocardiography is associated with numerous challenges. Given that it is recommended as first-line diagnostic tool, an ongoing growing worldwide challenge is to process millions of echocardiography clips and images obtained daily. The increasing workload in all echocardiographic laboratories and varying image quality makes thorough and timely interpretation challenging. Technicians acquire the clips and images, perform manual measurements, write the draft report, which is followed by approval of cardiologists making the total process rather complex, resource intensive and time consuming (9). Cardiologists in private practices and those in small hospitals often do not have technicians available. Hence, they are often overloaded with routine tasks inherent to echocardiographic exams and sometimes miss very specialized expertise. Moreover, it also takes years of education and experience for a technician or cardiologist to become an expert in detecting perceptual cues in echocardiography clips and automatically integrating this information into a clinical differentiation based upon pattern recognition without overt statistical reasoning. Echocardiography is also increasing in complexity, particularly strain imaging and three dimensional (3D) analysis (10). Furthermore, volume of exams is rising due to new diagnostic assessments and therapeutic options leading to a further increase in expert workload (1113).

Artificial intelligence (AI) is a rapidly emerging field and refers to the broad concept of simulating human logic and intelligence and covers any algorithm or model executed by a computer that mimics human intelligence, see Figure 1 (1416). Machine learning (ML) is a subfield of AI where the algorithms learn to perform a task based on expert engineered characteristics describing the data (17). Deep learning (DL) is a subfield of ML where the algorithms learn directly from the data themselves circumventing the feature engineering. ML and DL techniques are described in detail elsewhere (18). Handling high complexity, high dimensional data; particularly time series and machine generated data is a strength of many ML and DL algorithm (18).

FIGURE 1
www.frontiersin.org

Figure 1. Artificial intelligence, machine learning and deep learning.

During development of AI methods, the data sets are partitioned into training, validation and test sets. The training set often encompasses the bulk of all available data and together with a smaller validation set, it is used for the development of the AI method (19). The hold-out test set is used to evaluate overall performance. To evaluate generalizability of the AI solution with respect to e.g., image acquisition or imaged population diverse datasets are required. Application of these approaches for analysis of echocardiography clips and images creates opportunities for automation of expert analysis to advent the acquisition and analysis and thereby improve the clinical workflow (18, 20, 21).

Routine Echocardiography and Artificial Intelligence Solutions

Here, we provide an overview of the AI methods developed for analysis of routine echocardiography. Advanced solutions like those for fusion imaging and role of 3D augmented virtual reality are beyond the scope of this paper. Thus far developed methods are mostly focusing on automated image view selection and image segmentation. One of the important steps in echocardiography is selecting the best view for the subsequent analysis. This can be challenging, and hence time consuming, particularly for inexperienced operators. Subsequently, we discuss AI solutions for diagnosis and prognostication in various diseases. A graphical abstract is shown in Figure 2.

FIGURE 2
www.frontiersin.org

Figure 2. Graphical abstract on future expectations. AI will support doctors, not replace them.

Automated Image View Classification

Automated view classification can help standardizing views and measurements in echocardiography exams. Moreover, automated view classification can help non-experts to start learning understanding and using echocardiography (12). During training, automatic tools classifying cardiac views could recognize off-axis acquisition and incorrect views and provide guidance on how to move the probe in order to obtain the correct diagnostic images. Nowadays, the acquisition of ultrasound images is still usually performed in two-dimensional (2D) mode (22). Khamis et al. developed a method exploiting supervised dictionary learning for classification of apical two-chamber (AP2CH), four-chamber (AP4CH), and long-axis (AL) views using clips of 103 patients. The evaluation demonstrated a classification accuracy of more than 91% in all views (23). Gao et al. performed viewpoint classification using a convolutional neural network (CNN) and achieved 92.1% accuracy (24). Later, Madani et al. used a CNN to classify 15 standard views, based on labeled still images and videos from 267 transthoracic echocardiograms that captured a range of real-world clinical variation (25). To our knowledge, there are no studies on how well view classification works in real life. However, every clinician knows that there is a learning curve to learn to practice ultrasound, and every clinician has seen a foreshortened recording of an inexperienced user in clinical practice. Evaluation demonstrated an accuracy of 91.7% among 15 views. Cheema et al. studied guiding of beginners toward a technically correct image with a DL technique (EchoGPS™, Bay Labs) (26). In this study 28 users with no prior training in echocardiography were evaluated on their ability to obtain images of 10 routine echo views on a standardized subject after a 1.5-h familiarization with the software. The mean percent of auto-captures was 69% in physicians, 72% in advance practice providers, 83% in registered nurses, and 70% in certified medical assistants. All participants were able to use static and dynamic guidance to improve image quality while scanning. As a resume, AI solutions are well-suited for image view classification tasks and are increasingly availably. Automation of these tasks improves learning curve of students and has clinical impact due to the collection of higher quality images that improve interpretation.

Automated Function Assessment

Left ventricular (LV) function assessment and quantification have been most often performed because of the clinical importance (19, 25, 2732). Several studies have evaluated AI-driven echocardiography image analysis methods including automated contour-based segmentation, see Table 1. Asch et al. used commercial software (AutoEF, BayLabs) with NN to perform LV EF estimation automatically on a database of more than 50.000 echocardiographic studies, including multiple AP2CH and AP4CH views. The evaluation on a set of 99 patients shows that the method performs similar to measurements of cardiologists with more than 20 years of experience: r = 0.94, bias=1.4%, limits of agreement = ±13.4%, sensitivity 0.93, specificity 0.87 (33).

TABLE 1
www.frontiersin.org

Table 1. Artificial intelligence for image analysis and quantification.

Cannesson et al. performed an evaluation study using commercial software (AutoEF, Siemens) in 218 patients, including 165 patients with abnormal LV function (34). The AI solution was trained on more than 10,000 tracings by human experts to automatically locate and track the LV endocardium from routine grayscale digital loops and calculate EF. The AI solution correlated well with visual EF by expert readers (r = 0.96; p < 0.001) and performed analysis in 15 s per patient. However, less favorable results were found by Rahmouni et al. who evaluated the same AI algorithm and found discrepancies in EF estimates between AutoEF and manual tracing and between AutoEF and CMR (35). The authors recommended validation in a number of large, busy echocardiographic laboratories. Knackstedt et al. performed an external evaluation study using a commercial ML solution (AutoLV, TomTec) in 255 patients, of whom apical AP2CH and AP4CH views were collected from four centers that assessed EF using both visual estimation and manual tracing (27). ML was applied for calculating fully automated EF and longitudinal strain measurements. Interclass correlation coefficients and Bland-Altman analysis revealed good agreements among automated EF (ICC: 0.83, bias 0.7%, 95%), local center manual tracking, and reference center manual tracking, but not for visual EF assessments. 3D echocardiography, which can be obtained with more complex transducers, is increasingly available. In an evaluation study by Tsang et al. the commercial ML solution HeartModel was evaluated in 159 patients to quantify 3D echocardiography derived left atrial and LV volumes and LV EF (28). The AI technique strongly correlated with expert measurements (r = 0.87 to 0.96) and volumes and ejection fraction derived from magnetic resonance imaging (r = 0.84 to 0.95) using a 1.5-Tesla scanner (Achieva, Philips Healthcare). Medvedofsky et al. used the same ML tool on 3D echocardiographic images in 180 patients at six sites and demonstrated that LV EF and chamber volume were an accurate alternative to expert assessment (r: LVEDV: 0.99, LVESV: 0.99, LVEF: 0.94, LAV: 0.99) (36).

Evaluation of diastolic parameters is also important in assessing LV function of patients. Diastolic dysfunction is associated with increased myocardial fibrosis, increased ventricular stiffness and reduced prognosis (37, 38). Lancaster et al. used hierarchical clustering to discriminate between the different degrees of diastolic dysfunction, and improved prediction of event-free survival was found by clusters over conventional guideline-based classification for all-cause mortality and cardiac mortality (38). More recently, Hubert et al. reported on an AI solution for diastolic assessment in fifty patients (25 with amyloidosis, and 25 with heart failure with preserved ejection fraction) (39). This AI solution demonstrated a significant difference of the global area between both groups (37 vs. 72 mL%, respectively, P < 0.0001). Applying a linear discriminant analysis classifier, results showed a mean area under the curve (AUC) of 0.91 for the comparison between both groups. In this study classical indices of diastolic function were pathological in both groups with greater left atrial volume index, greater mitral average E/e' ratio, faster tricuspid regurgitation (P < 0.0001) compared to controls. Another study on AI and diastolic function was performed by Sabovčik et al. The authors applied an AI solution to detect early stages of cardiac remodeling and diastolic dysfunction in 1,407 participants (mean age, 51 years, 51% women) with an AUC curve with values between 86.2 and 88.1% (40). In conclusion, AI solutions might help to pre-select individuals in whom further echocardiographic examination, monitoring, and preventive measures are warranted.

The aforementioned studies show that AI solutions are increasingly developed for both systolic and diastolic LV function assessment and quantification. Use of these AI solutions is feasible. To conform the findings external evaluation and assessment of clinically relevant outcomes is required.

Automated Disease Classification

Analysis of echocardiographic images plays a crucial role in clinical routine to measure the cardiac morphology to reach a diagnosis (41). Such analysis is based on the interpretation of clinical parameters which are extracted through image analysis such as segmentation and tracking. For instance, diagnosis of LV hypertrophy requires accurate delineation of the LV endocardium in both end diastole and end systole. As a next step labeled echocardiographic views from a patient with known pathology can be used to train an AI solution, or to automate disease classification in a new sample (42). In that case, the AI solution recognizes a pattern similar to what a technician or cardiologist recognizes.

Valvular Heart Disease

AI solutions are rapidly emerging for valvular heart disease, see Table 1 (12). AI can help with sizing and modeling of minimally invasive structural heart interventional devices, where possible with real-time guidance (43). Studies limited to internal validation demonstrate good performance. Moghaddasi et al. used supervised ML classifiers for assessment of mitral regurgitation (MR) severity in 102 patients with an accuracy of 99% (44). Ghesu et al. introduced Marginal Space DL to perform automated valve detection and segmentation in 869 patients with superior accuracy in corner error measured in millimeter as compared to cardiac computed tomography (45). ML was used to determine LV responses during the progression of aortic stenosis by Casaclang et al. and the authors demonstrated precise recognition of the pattern of LV responses during the progression of AS (p < 0.0001) (46). Another study was performed to determine the interobserver variability of automated 3D mitral valve analysis using commercial software (eSie Valve software, Siemens) (47). The authors found a high reproducibility in this study with a small data set size (P < 0.0083).

A number of studies focused on the evaluation of AI tools. Automated quantification of aortic stenosis and regurgitation in 3D trans esophageal echocardiography with ML was developed by a commercial vendor (Auto Valve, Siemens). External evaluation at Ohio State University showed an excellent performance as compared to expert assessment (ICC 0.99) (48). Jin et al. used ML to support both experts and non-experts in localizing mitral valve prolapse by 3D transesophageal echocardiography (49). The authors reported significantly improved accuracy of non-experts using the ML application (from 83 to 89%, P = 0.003). Moreover, significantly less time for image analysis was needed using ML by both experts (1.9 ± 0.7 vs. 9.9 ± 3.5 min, P < 0.0001) and non-experts (5.0 ± 0.5 vs. 13 ± 1.5 min, P < 0.0001), especially for complex pathology (49).

Cardiomyopathies

AI tools have potential to be used to discriminate various cardiomyopathies. Unfortunately, large multicenter studies are lacking and current studies are limited to evaluation utilizing single center data. For example, a supervised ML classifier was used to discriminate between restrictive cardiomyopathy and constrictive pericarditis using clinical and echocardiographic data of 50 patients with constrictive pericarditis and 44 with restrictive cardiomyopathy. The results demonstrated an excellent AUC of 96.2% (50). A similar study evaluated an ensemble combining three supervised classifiers (support vector machine, random forest, artificial neural network) to discriminate between hypertrophic cardiomyopathy and physiological hypertrophy in athletes (51). The results demonstrated superior performance compared to individual echocardiographic indices early-to-late diastolic transmitral velocity ratio, e', and strain (p = 0.04). In another work, unsupervised clustering approach was used to automatically classify 156 patients with a heart failure who underwent stress echocardiography. The method demonstrated good correlation with expert assessment (κ = 72.6%) (52). Later, CNN was successfully deployed by Zhang et al. to discriminate between diagnosis of hypertrophic cardiomyopathy, cardiac amyloidosis, and pulmonary arterial hypertension (AUC > 0.85) (42). Madani et al. utilized DL and clustering analysis of image classification in a small set of labeled data (4%) and a large set of unlabeled data for LV hypertrophy classification. The method achieved an accuracy of 0.92 (31). Other authors used CNNs to diagnose transposition of the great arteries or congenitally corrected transposition of the great arteries with 98.0% accuracy (29). Furthermore, a combination of autoencoder and support vector machine classifier was used to diagnose aortic coarctation using echocardiography data with high accuracy (53).

Quality Assessment and Enhancement

Quality of echocardiograms is operator dependent, and can vary across patients and medical equipment. Patient characteristics such as fat, bone and air, breathing and patient movements may lead to reduced quality of clips and images and artifacts. AI can help technicians and cardiologists to support acquisitions and automate quality assessment and enhancement of echocardiograms (19, 54). Wu et al. demonstrated superiority of ML assisted echocardiogram enhancement over other image despeckling methods and video denoising methods as visually evaluated by experts (54). Abdi et al. performed a DL study in which 6,916 echo images were annotated by an expert on a five-point scale with a score between one (not acceptable) and five (excellent) (55). Internal evaluation demonstrated satisfactory accuracy (mean absolute error = 0.71) and <10 milliseconds computation time per frame, sufficient for real-time deployment. In summary, data on AI solutions for quality assessment and enhancement are limited but promising.

Event Prediction

Risk assessment and prediction of both survival and cardiac events are key tasks in management of cardiac patients (56, 57). Studies on AI and event prediction in the field of echocardiography are listed in Table 2. In a study on in 866 patients referred for echocardiographic assessment automated prediction of major adverse cardiovascular events (MACE) was performed with ML cluster analysis (38). This technique was superior to conventional prediction techniques (AIC 157, kappa = 0.619, p < 0.001). Berchialla et al. used stress echocardiography data integrated with LV functional and angiographic data to predict MACE (58). The authors demonstrated discrimination ability superior or comparable of a Bayesian network to other ML classifiers. Ghorbani et al. developed a DL solution (EchoNet) to predict cardiovascular risk in 2,850 patients (59). Internal evaluation demonstrated a high accuracy of the DL solution to detect systemic factors such as age and sex from echocardiogram images alone (AUC 0.88), which is impossible for human experts. Kwon et al. focuses on the echocardiography reports and developed a DL solution using deep neural networks with TensorFlow (the Google Brain Team, Mountain View, United States) as the backend to predict survival from these reports in a multicenter retrospective cohort study on 25,776 patients with 1,026 mortalities (60). Importantly, the authors used derivation data of hospital A and performed external evaluation using echocardiography reports of hospital B. The authors obtained superior performance as compared to conventional prediction models (AUC = 0.88). In another study that focused on echocardiography reports Samad et al. predicted survival in 171,510 unselected patients who underwent 331,317 echocardiograms (61). The authors achieved a significantly higher prediction accuracy with nonlinear ML over linear logistic regression models (AUC>0.82). A model including clinical variables, LV function and 57 echocardiographic measurements yielded the highest prediction accuracy (p < 0.01 across all models and survival durations). In conclusion, AI solutions for risk assessment and prediction of both survival and cardiac events are promising, but require further evaluation. Most data are obtained from retrospective studies and evaluation on external datasets is often lacking.

TABLE 2
www.frontiersin.org

Table 2. Artificial intelligence solutions for prediction of events.

Discussion

We summarize eleven studies on the development of AI in the field of routine echocardiography. Overall performance of the AI solutions was comparable to expert performance. However, these studies were virtually all hampered by lack of external validation of multi-center datasets. Eight studies evaluated performance of commercial software, all for diagnostic purposes.

To this end most successful AI methods are supervised, in other words they learn from labeled data. Hence, performance of supervised AI solutions depends on careful labeling of input data, which often relies on the technicians and cardiologists. Intra- and interobserver variability in data labeling may limit AI performance (30, 33). The availability of large, diverse and labeled data is a prerequisite for progress in the development and evaluation of AI solutions. Last year two large open datasets in the field of 2D echocardiography became publicly available; from Stanford University and University of Lyon (32, 41). This opens exciting possibilities. Nevertheless, the data are limited to echocardiography videos only and not accompanied by relevant clinical patient data and outcomes. Another challenge has been the lack of data standardization (62). Poor data standardization leads to incomplete and inaccurate data collection, patient matching issues, and slower workflows. Large sets with reference labels and standardized evaluation procedures would allow better comparison between the methods. An itemized checklist that highlights steps for ensuring correct application of AI models and the consistent reporting of model specifications and result might help (63). Another challenge are potential ethical problems derived from data sharing or de-identification to maintain patients' privacy. Consequently, these studies need additional local institutional review board authorizations to evaluate appropriate use of data.

In the future it is to be expected that AI solutions will increasingly support technicians and cardiologists in the field of digital care (64, 65) and echocardiography (66). AI can be incorporated into everyday practice and become a valuable aid for cardiologists and technicians dealing with cardiovascular disease (67, 68). AI will help to reduce workload, increase reproducibility and standardize data reporting. AI is also expected to improve study preparation by all related views retrieved automatically. This would save the technician or cardiologist time in searching through the complete study with sometimes hundreds of images by allowing the them to visualize all requested information quickly. AI is expected to improve echocardiography acquisition with support on automated probe adjustments and recording leading to advances in efficiency and overcome human limitations of both distraction and fatigue. Automated acquisitions will additionally contribute to increased standardization. Future AI solutions are also expected to extract information not directly apparent to humans (43). Improved prediction of events and mortality is expected with new data driven AI solutions, preferably in real time (43). On an educational level, much more can be expected from automated disease classification. As a beginner it can be difficult to distinguish between normal and abnormal structures, and with AI support that may become much easier. So far, most educational studies focused on automated view classification to recognize off-axis acquisition and incorrect views and provide guidance on how to move the probe in order to obtain diagnostic images.

Specifically, for echocardiography, there is a major challenge in the absence of standardization of the image sets and the varying image quality. Datasets with CT and MRI images are often obtained in a more standardized manner, but this researcher has been performed in fewer patients. Echocardiography is one of the basic researches in cardiology, and therefore challenge in image quality are certainly not insurmountable because of the extensive data volume.

As a resume, contemporary AI solutions for image acquisition, analysis, reporting and education are developed for relevant tasks with promising performance. Studies with external validation must show whether successful performance is sustained. In the future major benefit of AI in echocardiography is expected from improvements in automated interpretation and event prediction to reduce workload and improve clinical outcome. Some of the discussed challenges have yet to be overcome, however, none of them are insurmountable. Studies are also needed to acquire trust in new technologies, supported by efforts toward explainable models and transparency.

Author Contributions

MS, II, and BB drafted the manuscript, which was critically revised and edited by BC and SC. All authors agree to be accountable for all aspects of the work.

Conflict of Interest

II is cofounder and Scientific Lead of Quantib-U BV.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank Mitchel Molenaar for his support in creating an illustration.

References

1. Knuuti J, Wijns W, Saraste A, Capodanno D, Barbato E, Funck-Brentano C, et al. 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes. Eur Heart J. (2020) 41:407–77. doi: 10.1093/eurheartj/ehz425

CrossRef Full Text | Google Scholar

2. Baumgartner H, Falk V, Bax JJ, De Bonis M, Hamm C, Holm PJ, et al. 2017 ESC/EACTS Guidelines for the management of valvular heart disease. Eur Heart J. (2017) 38:2739–91. doi: 10.5603/KP.2018.0013

CrossRef Full Text | Google Scholar

3. Steeds RP, Garbi M, Cardim N, Kasprzak JD, Sade E, Nihoyannopoulos P, et al. EACVI appropriateness criteria for the use of transthoracic echocardiography in adults: a report of literature and current practice review. Eur Heart J Cardiovasc Imaging. (2017) 18:1191–204. doi: 10.1093/ehjci/jew333

CrossRef Full Text | Google Scholar

4. Bouma BJ, Riezenbos R, Voogel AJ, Veldhorst MH, Jaarsma W, Hrudova J, et al. Appropriate use criteria for echocardiography in the Netherlands. Neth Heart J. (2017) 25:330–4. doi: 10.1007/s12471-017-0960-9

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Sengupta PP, Adjeroh DA. Will artificial intelligence replace the human echocardiographer? Circulation. (2018) 138:1639–42. doi: 10.1161/CIRCULATIONAHA.118.037095

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Galderisi M, Cosyns B, Edvardsen T, Cardim N, Delgado V, Di Salvo G, et al. Standardization of adult transthoracic echocardiography reporting in agreement with recent chamber quantification, diastolic function, and heart valve disease recommendations: an expert consensus document of the European Association of Cardiovascular Imaging. Eur Heart J Cardiovasc Imaging. (2017) 18:1301–10. doi: 10.1093/ehjci/jex244

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Klem I, Shah DJ, White RD, Pennell DJ, van Rossum AC, Regenfus M, et al. Prognostic value of routine cardiac magnetic resonance assessment of left ventricular ejection fraction and myocardial damage: an international, multicenter study. Circ Cardiovasc Imaging. (2011) 4:610–9. doi: 10.1161/CIRCIMAGING.111.964965

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kusunose K, Haga A, Abe T, Sata M. Utilization of artificial intelligence in echocardiography. Circ J. (2019) 83:1623–9. doi: 10.1253/circj.CJ-19-0420

CrossRef Full Text | Google Scholar

9. Krittanawong C, Johnson KW, Rosenson RS, Wang Z, Aydar M, Baber U, et al. Deep learning for cardiovascular medicine: a practical primer. Eur Heart J. (2019) 40:2058–73. doi: 10.1093/eurheartj/ehz056

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Mitchell C, Rahko PS, Blauwet LA, Canaday B, Finstuen JA, Foster MC, et al. Guidelines for performing a comprehensive transthoracic echocardiographic examination in adults: recommendations from the American Society of Echocardiography. J Am Soc Echocardiogr. (2019) 32:1–64. doi: 10.1016/j.echo.2018.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Ranschaert E, Morozov S, Algra P. Artificial Intelligence in Medical Imaging: Opportunities, Applications and Risks. Cham: Springer Nature Switzerland AG (2019). doi: 10.1007/978-3-319-94878-2

CrossRef Full Text | Google Scholar

12. Davis A, Billick K, Horton K, Jankowski M, Knoll P, Marshall JE, et al. Artificial intelligence and echocardiography: a primer for cardiac sonographers. J Am Soc Echocardiogr. (2020) 33:1061–6. doi: 10.1016/j.echo.2020.04.025

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Omar AMS, Krittanawong C, Narula S, Narula J, Argulian E. Echocardiographic data in artificial intelligence research: primer on concepts of big data and latent states. JACC Cardiovasc Imaging. (2020) 13:170–2. doi: 10.1016/j.jcmg.2019.07.017

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Siegersma KR, Leiner T, Chew DP, Appelman Y, Hofstra L, Verjans JW. Artificial intelligence in cardiovascular imaging: state of the art and implications for the imaging cardiologist. Neth Heart J. (2019) 27:403–13. doi: 10.1007/s12471-019-01311-1

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Benjamins JW, Hendriks T, Knuuti J, Juarez-Orozco LE, van der Harst P. A primer in artificial intelligence in cardiovascular medicine. Neth Heart J. (2019) 27:392–402. doi: 10.1007/s12471-019-1286-6

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Seetharam K, Brito D, Farjo PD, Sengupta PP. The role of artificial intelligence in cardiovascular imaging: state of the art review. Front Cardiovasc Med. (2020) 7:618849. doi: 10.3389/fcvm.2020.618849

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Deo RC. Machine learning in medicine. Circulation. (2015) 132:1920–30. doi: 10.1161/CIRCULATIONAHA.115.001593

CrossRef Full Text | Google Scholar

18. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. (2015) 521:436–44. doi: 10.1038/nature14539

CrossRef Full Text | Google Scholar

19. Al'Aref SJ, Anchouche K, Singh G, Slomka PJ, Kolli KK, Kumar A, et al. Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging. Eur Heart J. (2019) 40:1975–86. doi: 10.1093/eurheartj/ehy404

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Wallis C. How artificial intelligence will change medicine. Nature. (2019) 576:S48. doi: 10.1038/d41586-019-03845-1

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Litjens G, Ciompi F, Wolterink JM, de Vos BD, Leiner T, Teuwen J, et al. State-of-the-art deep learning in cardiovascular image analysis. JACC Cardiovasc Imaging. (2019) 12:1549–65. doi: 10.1016/j.jcmg.2019.06.009

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Slomka PJ, Dey D, Sitek A, Motwani M, Berman DS, Germano G. Cardiac imaging: working towards fully-automated machine analysis & interpretation. Expert Rev Med Devices. (2017) 14:197–212. doi: 10.1080/17434440.2017.1300057

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Khamis H, Zurakhov G, Azar V, Raz A, Friedman Z, Adam D. Automatic apical view classification of echocardiograms using a discriminative learning dictionary. Med Image Anal. (2017) 36:15–21. doi: 10.1016/j.media.2016.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Gao X, Li W, Loomes M, Wang L. A fused deep learning architecture for viewpoint classification of echocardiography. Information Fusion. (2017) 36:103–113. doi: 10.1016/j.inffus.2016.11.007

CrossRef Full Text | Google Scholar

25. Madani A, Arnaout R, Mofrad M, Arnaout R. Fast and accurate view classification of echocardiograms using deep learning. NPJ Digit Med. (2018). doi: 10.1038/s41746-017-0013-1. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Baljash C, Ching H, David A, Akhil N, James DT. Abstract 15694: automated guidance and image capture of echocardiographic views using a deep learning-derived technology. Circulation. (2019) 140:A15694. doi: 10.1161/circ.140.suppl_1.15694

CrossRef Full Text | Google Scholar

27. Knackstedt C, Bekkers SCAM, Schummers G, Schreckenberg M, Muraru D, Badano LP, et al. Fully automated versus standard tracking of left ventricular ejection fraction and longitudinal strain: the FAST-EFs multicenter study. J Am Coll Cardiol. (2015) 66:1456–66. doi: 10.1016/j.jacc.2015.07.052

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Tsang W, Salgo IS, Medvedofsky D, Takeuchi M, Prater D, Weinert L, et al. Transthoracic 3D echocardiographic left heart chamber quantification using an automated adaptive analytics algorithm. JACC Cardiovasc Imaging. (2016) 9:769–82. doi: 10.1016/j.jcmg.2015.12.020

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Diller G-P, Babu-Narayan S, Li W, Radojevic J, Kempny A, Uebing A, et al. Utility of machine learning algorithms in assessing patients with a systemic right ventricle. Eur Heart J Cardiovasc Imaging. (2019) 20:925–31. doi: 10.1093/ehjci/jey211

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Gandhi S, Mosleh W, Shen J, Chow CM. Automation, machine learning, and artificial intelligence in echocardiography: a brave new world. Echocardiography. (2018) 35:1402–18. doi: 10.1111/echo.14086

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Madani A, Ong JR, Tibrewal A, Mofrad MRK. Deep echocardiography: data-efficient supervised and semi-supervised deep learning towards automated diagnosis of cardiac disease. NPJ Digit Med. (2018) 1:59. doi: 10.1038/s41746-018-0065-x

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Ouyang D, He B, Ghorbani A, Yuan N, Ebinger J, Langlotz CP, et al. Video-based AI for beat-to-beat assessment of cardiac function. Nature. (2020) 580:252–6. doi: 10.1038/s41586-020-2145-8

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Asch FM, Poilvert N, Abraham T, Jankowski M, Cleve J, Adams M, et al. Automated echocardiographic quantification of left ventricular ejection fraction without volume measurements using a machine learning algorithm mimicking a human expert. Circ Cardiovasc Imaging. (2019) 12:e009303. doi: 10.1161/CIRCIMAGING.119.009303

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Cannesson M, Tanabe M, Suffoletto MS, McNamara DM, Madan S, Lacomis JM, et al. A novel two-dimensional echocardiographic image analysis system using artificial intelligence-learned pattern recognition for rapid automated ejection fraction. J Am Coll Cardiol. (2007) 49:217–26. doi: 10.1016/j.jacc.2006.08.045

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Rahmouni HW, Ky B, Plappert T, Duffy K, Wiegers SE, Ferrari VA, et al. Clinical utility of automated assessment of left ventricular ejection fraction using artificial intelligence-assisted border detection. Am Heart J. (2008) 155:562–70. doi: 10.1016/j.ahj.2007.11.002

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Medvedofsky D, Mor-Avi V, Amzulescu M, Fernández-Golfín C, Hinojar R, Monaghan MJ, et al. Three-dimensional echocardiographic quantification of the left-heart chambers using an automated adaptive analytics algorithm: multicentre validation study. Eur Heart J Cardiovasc Imaging. (2018) 19:47–58. doi: 10.1093/ehjci/jew328

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Importance of Diastolic Function for the Prediction of Arrhythmic Death | Circulation: Arrhythmia and Electrophysiology. Available online at: https://www.ahajournals.org/

Google Scholar

38. Lancaster MC, Salem Omar AM, Narula S, Kulkarni H, Narula J, Sengupta PP. Phenotypic clustering of left ventricular diastolic function parameters: patterns and prognostic relevance. JACC Cardiovasc Imaging. (2019) 12:1149–61. doi: 10.1016/j.jcmg.2018.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Hubert A, Le Rolle V, Galli E, Bidaud A, Hernandez A, Donal E. New expectations for diastolic function assessment in transthoracic echocardiography based on a semi-automated computing of strain-volume loops. Eur Heart J Cardiovasc Imaging. (2020) 21:1366–71. doi: 10.1093/ehjci/jeaa123

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Sabovčik F, Cauwenberghs N, Kouznetsov D, Haddad F, Alonso-Betanzos A, Vens C, et al. Applying machine learning to detect early stages of cardiac remodelling and dysfunction. Eur Heart J Cardiovasc Imaging. (2020). doi: 10.1093/ehjci/jeaa135. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Leclerc S, Smistad E, Pedrosa J, Ostvik A, Cervenansky F, Espinosa F, et al. Deep Learning for Segmentation Using an Open Large-Scale Dataset in 2D Echocardiography. IEEE Trans Med Imaging. (2019) 38:2198–2210. doi: 10.1109/TMI.2019.2900516

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Zhang J, Gajjala S, Agrawal P, Tison GH, Hallock LA, Beussink-Nelson L, et al. Fully automated echocardiogram interpretation in clinical practice. Circulation. (2018) 138:1623–35. doi: 10.1161/CIRCULATIONAHA.118.034338

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Alsharqi M, Woodward WJ, Mumith JA, Markham DC, Upton R, Leeson P. Artificial intelligence and echocardiography. Echo Res Pract. (2018) 5:R115–25. doi: 10.1530/ERP-18-0056

CrossRef Full Text | Google Scholar

44. Moghaddasi H, Nourian S. Automatic assessment of mitral regurgitation severity based on extensive textural features on 2D echocardiography videos. Comput Biol Med. (2016) 73:47–55. doi: 10.1016/j.compbiomed.2016.03.026

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Ghesu FC, Krubasik E, Georgescu B, Singh V, Yefeng Zheng null, Hornegger J, et al. Marginal space deep learning: efficient architecture for volumetric image parsing. IEEE Trans Med Imaging. (2016) 35:1217–28. doi: 10.1109/TMI.2016.2538802

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Casaclang-Verzosa G, Shrestha S, Khalil MJ, Cho JS, Tokodi M, Balla S, et al. Network tomography for understanding phenotypic presentations in aortic stenosis. JACC Cardiovasc Imaging. (2019) 12:236–48. doi: 10.1016/j.jcmg.2018.11.025

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Jeganathan J, Knio Z, Amador Y, Hai T, Khamooshian A, Matyal R, et al. Artificial intelligence in mitral valve analysis. Ann Card Anaesth. (2017) 20:129–34. doi: 10.4103/aca.ACA_243_16

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Calleja A, Thavendiranathan P, Ionasec RI, Houle H, Liu S, Voigt I, et al. Automated quantitative 3-dimensional modeling of the aortic valve and root by 3-dimensional transesophageal echocardiography in normals, aortic regurgitation, and aortic stenosis: comparison to computed tomography in normals and clinical implications. Circ Cardiovasc Imaging. (2013) 6:99–108. doi: 10.1161/CIRCIMAGING.112.976993

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Jin C-N, Salgo IS, Schneider RJ, Kam KK-H, Chi W-K, So C-Y., et al. Using anatomic intelligence to localize mitral valve prolapse on three-dimensional echocardiography. J Am Soc Echocardiogr. (2016) 29:938–45. doi: 10.1016/j.echo.2016.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Sengupta PP, Huang Y-M, Bansal M, Ashrafi A, Fisher M, Shameer K, et al. Cognitive machine-learning algorithm for cardiac imaging: a pilot study for differentiating constrictive pericarditis from restrictive cardiomyopathy. Circ Cardiovasc Imaging. (2016) 9:e004330. doi: 10.1161/CIRCIMAGING.115.004330

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Narula S, Shameer K, Salem Omar AM, Dudley JT, Sengupta PP. Machine-learning algorithms to automate morphological and functional assessments in 2D echocardiography. J Am Coll Cardiol. (2016) 68:2287–95. doi: 10.1016/j.jacc.2016.08.062

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Sanchez-Martinez S, Duchateau N, Erdei T, Kunszt G, Aakhus S, Degiovanni A, et al. Machine learning analysis of left ventricular function to characterize heart failure with preserved ejection fraction. Circ Cardiovasc Imaging. (2018) 11:e007138. doi: 10.1161/CIRCIMAGING.117.007138

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Pereira F, Bueno A, Rodriguez A, Perrin D, Marx G, Cardinale M, et al. Automated detection of coarctation of aorta in neonates from two-dimensional echocardiograms. J Med Imaging (Bellingham). (2017) 4:014502. doi: 10.1117/1.JMI.4.1.014502

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Wu H, Huynh TT, Souvenir R. Echocardiogram enhancement using supervised manifold denoising. Med Image Anal. (2015) 24:41–51. doi: 10.1016/j.media.2015.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Abdi AH, Luong C, Tsang T, Allan G, Nouranian S, Jue J, et al. Automatic quality assessment of echocardiograms using convolutional neural networks: feasibility on the apical four-chamber view. IEEE Trans Med Imaging. (2017) 36:1221–30. doi: 10.1109/TMI.2017.2690836

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Schuuring MJ, Vis JC, van Dijk APJ, van Melle JP, Vliegen HW, Pieper PG, et al. Impact of bosentan on exercise capacity in adults after the Fontan procedure: a randomized controlled trial. Eur J Heart Fail. (2013) 15:690–8. doi: 10.1093/eurjhf/hft017

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Schuuring MJ, van Riel ACMJ, Vis JC, Duffels MG, van Dijk APJ, de Bruin-Bon RHACM, et al. New predictors of mortality in adults with congenital heart disease and pulmonary hypertension: midterm outcome of a prospective study. Int J Cardiol. (2015) 181:270–6. doi: 10.1016/j.ijcard.2014.11.222

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Berchialla P, Foltran F, Bigi R, Gregori D. Integrating stress-related ventricular functional and angiographic data in preventive cardiology: a unified approach implementing a Bayesian network. J Eval Clin Pract. (2012) 18:637–43. doi: 10.1111/j.1365-2753.2011.01651.x

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Ghorbani A, Ouyang D, Abid A, He B, Chen JH, Harrington RA, et al. Deep learning interpretation of echocardiograms. NPJ Digit Med. (2020) 3:10. doi: 10.1038/s41746-019-0216-8

CrossRef Full Text | Google Scholar

60. Kwon JM, Kim KH, Jeon KH, Park J. Deep learning for predicting in-hospital mortality among heart disease patients based on echocardiography. Echocardiography. (2019) 36:213–8. doi: 10.1111/echo.14220

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Samad MD, Ulloa A, Wehner GJ, Jing L, Hartzel D, Good CW, et al. Predicting survival from large echocardiography and electronic health record datasets: optimization with machine learning. JACC Cardiovasc Imaging. (2019) 12:681–9. doi: 10.1016/j.jcmg.2018.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Badano LP, Kolias TJ, Muraru D, Abraham TP, Aurigemma G, Edvardsen T, et al. Standardization of left atrial, right ventricular, and right atrial deformation imaging using two-dimensional speckle tracking echocardiography: a consensus document of the EACVI/ASE/Industry Task Force to standardize deformation imaging. Eur Heart J Cardiovasc Imaging. (2018) 19:591–600. doi: 10.1093/ehjci/jey042

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Sengupta PP, Shrestha S, Berthon B, Messas E, Donal E, Tison GH, et al. Proposed requirements for cardiovascular imaging-related machine learning evaluation (PRIME): a checklist: reviewed by the american college of cardiology healthcare innovation council. JACC Cardiovasc Imaging. (2020) 13:2017–35. doi: 10.1016/j.jcmg.2020.07.015

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Schuuring MJ, Kauw D, Bouma BJ. COVID-19 pandemic: practical considerations on rapid initiation of remote care in chronic cardiac patients. Eur Heart J Digital Health. (2020) 1:8–9. doi: 10.1093/ehjdh/ztaa007

CrossRef Full Text | Google Scholar

65. Schuuring MJ, Kauw D. How to initiate eHealth in congenital heart disease patients? Eur Heart J Digital Health. (2020) 1:83–6. doi: 10.1093/ehjdh/ztaa012

CrossRef Full Text | Google Scholar

66. Lang RM, Addetia K, Miyoshi T, Kebed K, Blitz A, Schreckenberg M, et al. Use of machine learning to improve echocardiographic image interpretation workflow: a disruptive paradigm change? J Am Soc Echocardiogr. (2020). doi: 10.1016/j.echo.2020.11.017. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Kusunose K. Steps to use artificial intelligence in echocardiography. J Echocardiogr. (2020). doi: 10.1007/s12574-020-00496-4. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Seetharam K, Raina S, Sengupta PP. The role of artificial intelligence in echocardiography. Curr Cardiol Rep. (2020) 22:99. doi: 10.1007/s11886-020-01329-7

CrossRef Full Text | Google Scholar

Keywords: echocardiography, cardiac imaging, artificial intelligence, image analysis, diagnosis, prediction

Citation: Schuuring MJ, Išgum I, Cosyns B, Chamuleau SAJ and Bouma BJ (2021) Routine Echocardiography and Artificial Intelligence Solutions. Front. Cardiovasc. Med. 8:648877. doi: 10.3389/fcvm.2021.648877

Received: 02 January 2021; Accepted: 05 February 2021;
Published: 23 February 2021.

Edited by:

Matteo Cameli, University of Siena, Italy

Reviewed by:

Giulia Elena Mandoli, University of Siena, Italy
Maria Concetta Pastore, Università del Piemonte Orientale, Italy

Copyright © 2021 Schuuring, Išgum, Cosyns, Chamuleau and Bouma. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Mark J. Schuuring, m.j.schuuring@amsterdamumc.nl; orcid.org/0000-0002-2843-1852

Download