Feasibility and Accuracy of a Fully Automated Right Ventricular Quantification Software With Three-Dimensional Echocardiography: Comparison With Cardiac Magnetic Resonance

Background: A novel, fully automated right ventricular (RV) software for three-dimensional quantification of RV volumes and function was developed. The direct comparison of the software performance with cardiac magnetic resonance (CMR) was limited. Therefore, the aim of this study was to test the feasibility, accuracy, and reproducibility of a fully automated RV quantification software against CMR imaging as a reference. Methods: A total of 170 patients who underwent both CMR and three-dimensional echocardiography were enrolled. RV end-diastolic volume (RVEDV), RV end-systolic volume (RVESV), and RV ejection fraction (RVEF) were obtained using fully automated three-dimensional RV quantification software and compared with a CMR reference. For inter-technical agreement, Spearman correlation and Bland–Altman analysis were used. Results: The fully automated RV quantification software was feasible in 149 patients. RVEDV and RVESV were underestimated, and RVEF was overestimated compared with CMR values. RV measurements obtained from the manual editing method correlated better with CMR values than that without manual editing (RVEDV, 0.924 vs. 0.794: RVESV, 0.955 vs. 0.854; RVEF, 0.941 vs. 0.781 respectively, all p < 0.0001) with less bias and narrower limit of agreement (LOA). The bias and LOA for RV volumes and EF using the automated software without and with manual editing were greater in patients with severely impaired RV function or low frame rate than those with normal and mild impaired RV function, or high frame rate. The fully automated RV three-dimensional measurements were highly reproducible. Conclusion: The novel fully automated RV software shows good feasibility and reproducibility, and the measurements had a high correlation with CMR values. These findings support the routine application of the novel 3D automated RV software in clinical practice.

Background: A novel, fully automated right ventricular (RV) software for three-dimensional quantification of RV volumes and function was developed. The direct comparison of the software performance with cardiac magnetic resonance (CMR) was limited. Therefore, the aim of this study was to test the feasibility, accuracy, and reproducibility of a fully automated RV quantification software against CMR imaging as a reference.

INTRODUCTION
Right ventricular (RV) function has been demonstrated to be independently associated with poor clinical outcomes in patients with a variety of cardiac and pulmonary pathologies (1,2). Although cardiac magnetic resonance (CMR) is considered to be the gold standard method for the quantification of RV volumes and function (3)(4)(5), it is not feasible for those patients with common contraindication of implanted magnetic device (left ventricular assist device) other than with non-MRI conditional devices like ferromagnetic material such as shrapnel, medical devices, claustrophobia, etc. Based on its availability, versatility, and lower cost, echocardiography is the first-line option for cardiac quantification. Given the complex anatomy of the right ventricle, three-dimensional echocardiography (3DE) provides more accurate RV function assessment than two-dimensional echocardiography (2DE) due to avoidance of geometric assumptions and foreshortened views (6). Numerous studies have demonstrated the accuracy of 3DE for RV measurements in comparison with CMR reference (7)(8)(9)(10) and have also confirmed its incremental prognostic value over conventional 2DE indices (11)(12)(13). Despite these aforementioned advantages, however, widespread application of 3DE for RV assessments has not been incorporated into the clinical practice. This may be owing to it being time consuming and requiring training to obtain accurate 3DE for RV measurements.
Recently, a new fully automated 3D RV quantification software using machine learning algorithms (MLA) was introduced. Limited studies had validated its accuracy against CMR imaging (14,15). However, the feasibility and accuracy of a fully automated 3D RV software in large subjects with various RV function has not been well established, and the effects of RV ejection fraction (RVEF) and frame rate (FR) on the accuracy of this novel software has not been investigated.
Therefore, the purposes of this study were (i) to validate the feasibility, accuracy, and reproducibility of the fully automated 3D software for RV volumes and function quantification against CMR imaging in a large population with a wide range of RV function; (ii) to determine the impact of severity of RVEF and FR on the accuracy of the fully automated 3D RV measurements.

Population and Study Design
A total of 170 patients who had undergone both CMR and 3DE examinations with a median of 1 day [interquartile range (IQR); 0-6) were prospectively enrolled in our study from March 2017 to June 2020. The patients with arrhythmia or poor image quality of 3DE were excluded. Twenty-one patients were excluded, and a total of 149 patients were included in our final analysis.
The patients were divided into four subgroups on the basis of CMR-derived RVEF to normal RV function, mildly impaired, moderately impaired, and severely impaired RV function (RVEF: normal above 51% in males and 52% in females, mildly impaired 41-51%, moderately impaired 31-40%, and severely impaired ≤30%) (16). According to the RV function assessed by 3DE, patients were divided into three subgroups (normal RV function, RVEF > 45%; mild to moderately impaired RV function, 30% < RVEF ≤ 45%; and severely impaired RV function, ≤30%) (17). Additionally, the entire study population were divided into two subgroups based on the median FR of 3DE (≤23 frames/s or >23 frames/s). The study protocol was approved by the institutional ethics board of Union Hospital, Tongji Medical College, Huazhong University of Science and Technology. All patients provided informed consents.

Echocardiography
The RV-focused 2DE and 3DE data set were obtained by using a Philip echocardiographic system (EPIQ 7C, X5-1 and S5-1 probes, Philips Healthcare Andover, MA, USA). All subjects were told to lie on the left semirecumbent position with electrocardiogram (ECG) leads attached properly. 2DE images were obtained in the RV-focused apical four-chamber (A4C) view and ensured to avoid the foreshortening of the right ventricle. In the RV-focused A4C view in full volume mode, 3DE images were obtained with a median FR of 23 (IQR [20][21][22][23][24][25][26][27][28][29][30][31][32]) frames/s, for four cardiac cycles with short breath hold. The acquisition quality would be considered inappropriate if endocardial boundary in one segment was not clear in the A4C view. 2D and 3D images were stored for offline study. The stored data set were transferred to a workstation where they were studied and analyzed for RV quantification with the fully automated software. The RVfocused A4C views with clear endocardial border were selected. Five landmarks were ensured in the apical, coronal, and basal short-axis views.

Three-Dimensional Echocardiography Auto Right Ventricular Analysis
The acquisition of a 3DE data set was subjectively assessed to ensure the complete visualization of the endocardial border in 3DE acquisition in three views (A4C, coronal, and basal shortaxis views) (Figure 1). The image quality of the 3D data set was judged subjectively considering the signal-to-noise ratio and complete visualization of the RV endocardium, and was  categorized on a scale from 1 to 4 (from poor to excellent) (10). The image quality was considered as poor if ultrasound dropout was present in the coronal view more than one half of RV free wall (18). RV full volume 3DE data sets were analyzed by a novel fully automated 3D RV quantification software (3D auto RV, Philips Healthcare). The software detected the RV endocardial border using artificial intelligence, which consists of initial RV orientation and global shape recognition, and then 3D endocardial speckle tracking was conducted throughout one cardiac cycle. Once a good image frame was focused, the automated software was started by one click to adjust a 3D endocardial tracking cast of the RV at end diastole, and 3D speckle tracking analysis was subsequently performed. Finally, the software automatically generated the RV volumetime curves, from which the RV end-diastolic volume (RVEDV), RV end-systolic volume (RVESV), and RVEF were obtained. When the requested contour adjustment was performed on 2D images in both long-and short-axis views at end diastole, a new speckle tracking dataset was obtained. If the software made an erroneous 3D RV cast, a revision analysis would be performed. The subject would be discarded if the new 3D cast was still erroneous.
Results (RVEDV, RVESV, and RVEF) with manual editing were also obtained through manual adjustment in the enddiastolic frame and end-systolic frame at both long-and shortaxis views. These data (with or without manual editing) were compared with those from CMR for accuracy validation.
Cardiac Magnetic Resonance Imaging CMR imaging was performed using a 1.5T scanner (MAGNETOM Area, Siemens Healthcare, Germany) with a phased array of cardiovascular coil. For the identification of the long axis of the heart, retrospective ECG-gated localizing echo spine sequences were used. Steady-state free precession (SSFP) dynamic gradient-echo cine loop was acquired using retrospective ECG gating and parallel imaging technique with 6-mm slice thickness of the imaging plane. The parameters were set as 340 × 255-mm field of view, 256 × 205 scan matrix, and 80 • flip angle. The median FR was 32 [IQR (28 to 36)] frames/s with 10-to 15-s breath holds.

Cardiac Magnetic Resonance Analysis
CMR imaging was analyzed with conventional CMR software (Argus Siemens Medical Solution) by a sophisticated technician who was blinded to echocardiographic findings. Disc summation method of multiple short-axis views was used for the calculation of RVEDV and RVESV. RVEF was calculated through the standard formula (RVEF = RVEDV -RVESV/RVEDV * 100%).

Reproducibility
Intraobserver, interobserver, and test-retest variability was used to assess the reproducibility of the fully automated software on RV measurements. For intraobserver variability, 20 patients were selected randomly, and the analysis was performed again 2 weeks later by the same observer, while for interobserver variability, the same 20 patients were examined 2 weeks later by another observer who was blinded to the initial results. Test-retest variability was assessed by two different observers acquiring and analyzing the data independently at different time points. Intraobserver, interobserver, and testretest variability was evaluated by coefficients of variation (CoV), quantified as the absolute differences between the repeated two measurements in percentages of their mean, and intraclass correlation coefficient (ICC).

Analysis Time
The analysis time for the software in both modes (fully automated without and with manual editing) was measured in 20 patients. For fully automated mode, the time was measured from clicking the analysis button to displaying the results. While for manual editing mode, the analysis time also included tracking revision and endocardial border adjustment in both end-diastolic and end-systolic frames.

Statistical Analysis
Continuous variables are presented as mean ± standard deviation (SD), or median and IQR, while categorical variables are presented as frequency (percentage). Shapiro-Wilk test was used to determine whether the data conforms to the normal distribution. Since these data were not normally distributed, the Friedman's analysis with Wilcoxon comparison was used to compare the results among the three groups. Spearman correlation analysis was used to assess the correlation of between variables. Bland-Altman analysis was used for the determination of bias and limit of agreement (LOA). For

Study Population
The fully automated RV quantification software was feasible in 149 (87.6%) subjects. Clinical characteristics of study subjects are shown in Table 1. Image quality by 3D data set was graded excellent in 18.1% (n = 27), good in 50.3% (n = 75), fair in 23.5 % (n = 35), and poor in 8.1% (n = 12). The mean age of the subjects was 46 ± 15 years, and 115 (77.18%) were men. The entire population consisted of 12 (8.05%) healthy subjects and 137 (91.95%) patients. Among the 137 patients, 19 patients were diagnosed with ischemic heart disease, 53 with dilated cardiomyopathy, 18 with heart transplantation recipients, 15 with valvular heart disease, 16 with hypertrophic cardiomyopathy, eight with hypertensive heart disease, three with multiple myeloma, two with perinatal cardiomyopathy and one with uremic cardiomyopathy, one with rheumatic heart disease, and one with viral myocarditis. Forty-eight percent of the patients [dilated cardiomyopathy (n = 52) and ischemic heart disease (n=19)] with end-stage heart failure were included in the waiting list of heart transplantation.

Right Ventricular Volumes and Ejection Fraction Analysis
The correlation and Bland-Altman analysis for RV volumes and EF using 3D auto RV software with/without manual editing against CMR as a reference are presented in Table 2 and In addition, RVEF using 3D auto RV software with manual editing had smaller biases and narrower LOAs against CMR values than that of 3D auto RV quantification (bias: 2.6 and 6.8, LOA: 10.2 and 19.2, respectively). A similar trend was also observed in RVEDV and RVESV. The effects of RV function on RV volumes and EF using 3D auto RV software with/without manual editing compared with CMR measurements are summarized in Table 3. For RV volumes and RVEF, biases and LOA with or without manual editing were larger in patients with severely impaired RV function compared with those with normal or mild impaired RV function. Likewise, RV volumes and RVEF obtained by the manual editing method had smaller biases and narrower LOA compared with 3D auto RV software without manual editing measurements in patients with normal or mildly, moderately, and severely impaired RV function.
When the effect of FR was evaluated, there was no significant impact on the RVEF. However, the biases and LOA for RV volumes were larger in patients with FR ≤23 frame/s regardless of whether the manual editing was performed. Additionally, the biases and LOA for RV volumes and RVEF derived from the manual editing method was smaller than those without manual editing irrespective of FR ( Table 5).

Reproducibility
The intraobserver, interobserver, and test-retest variability of the 3D Auto RV software with and without manual editing method

Time of Analysis
The median time of analysis for 3D fully automated RV quantification software was 12s (IQR 9-14 s), which is much shorter than that of the manual editing method (71s; IQR 62-76 s).  Data are expressed as median and interquartile range. LOA, limit of agreement.

DISCUSSION
The main findings of the present study can be summarized as (i) RV volumes and function using fully automated RV quantification software were strongly correlated with CMR values. (ii) The manual editing method improved measurement accuracy. (iii) The bias and LOAs for the RV volumes and EF using the 3D auto RV software were smaller in patients with normal or mild impaired RV function or FR >23 frame/s than those with severely impaired RV function or FR ≤23 frame/s regardless of whether the manual editing was performed. (iv) The fully automated software showed excellent reproducibility and reduced the duration of analysis.

Accuracy of the Fully Automated Right Ventricular Quantification Software With/Without Manual Editing Method Compared With Cardiac Magnetic Resonance
3D assessment is required for accurate analysis of the right ventricle due to its irregular shape and complex contract mode (6). Nowadays, 3DE is recommended for RV size and function assessment according to updated recommendation from the American Society of Echocardiography (ASE) as well as European Association of Cardiovascular Imaging (EACVI). (6) In 2010, a 3D RV quantification software was first introduced, and its accuracy was tested against CMR and multidetector CT (19). A subsequent study demonstrated that RV volumes obtained by this software were significantly underestimated in patients with congenital heart diseases or moderate to severe RV dilatation in comparison with CMR (20). Recently, a new and semi-automated 3D RV quantification software was developed, and it can provide excellent accuracy and reproducibility for the measurement of RV volumes and function compared with CMR (10). Nevertheless, the software still needs manual input, which may leads to higher variability. In order to overcome the variability caused by manual input, a novel, fully automated RV quantification software using MLA has been developed and validated against CMR. It is reported that the fully automated software showed excellent accuracy and reproducibility compared with CMR. This new software without manual editing worked in one-third of the population. However, manual editing was still required in the rest of the study population (14). A recent study performed by Otani et al. demonstrated that a novel fully automated RV quantification software underestimated RV volumes in comparison with CMR. However, there was no significant difference in RVEF between the fully automated method and CMR (15). Most recently, another research demonstrated that RVEF using the fully automated software was associated with adverse clinical events (17).
However, these two studies were limited by the relatively small number of study subjects (14, 15); the fully automated 3D software was not fully validated against CMR, whereas our study focused on a large population with a wide range of RV sizes and function. Consistent with previous 3D RV studies (10,11,14,15), our data showed that RV volumes by the novel 3D automated software strongly correlated with CMR measurements. Moreover, our results further indicated that the correlation became higher when we performed manual editing. Meanwhile, in our study, the fully automated software slightly underestimated the RV volumes, similar to the previous study (10,11,14,15,21). With manual editing, the biases for RV volumes were smaller than in previous reports (11,21).
The prior results regarding the accuracy of novel 3D automated RV quantification software for RVEF measurement in comparison with CMR are discordant. Genovese et al. demonstrated that RVEF by the 3D automated software was underestimated against CMR imaging (14), whereas other studies showed no significant difference in RVEF between the fully automated method and CMR (15,17). In the present study, our results showed that the 3D automated RV software overestimated RVEF compared with CMR in a large population, similar to previous reports (11,21). The 3D automated RV software appeared to be excellent accuracy in quantifying RV volumes and EF using CMR as a reference, as reflected by small biases and narrow LOA. Advantages provided by the 3D automated RV software in terms of superior accuracy and usability could be useful particularly for those patients requiring a close follow-up of RV function that cannot be assessed by serial CMR imaging.

Impact of Right Ventricular Ejection Fraction and Frame Rate on the Three-Dimensional Auto Right Ventricular Quantification Software in Comparison With Cardiac Magnetic Resonance
The degree of RVEF has a great impact on 3D auto RV measurements and is the major factor contributing to the difference in RV quantification against CMR. Our study demonstrated that the biases for RV volumes and EF using the 3D auto method were larger in patients with severely impaired RV function. The biases improved when manual editing was performed. These results are in keeping with the study of Tsang et al. which showed that the biases and LOA for LV volumes using the 3D auto LV quantification software were larger in patients with reduced LVEF (22). Previous studies reported that the extent of underestimation of RV volumes in 3DE was greater in the case of RV dilatation than that of normal RV size (9). As patients with severely impaired RV function usually present with an enlarged right ventricle and abundant RV trabeculae, the software tracks further inside due to incognizance of these trabeculae and RV free wall, thus, leading to greater underestimation of RV volumes by 3DE in such subjects.
Our study demonstrated that the 3D auto RV software performed worse in subjects with severely impaired RV function, as evidenced by larger bias and wider LOA. The accuracy become higher when we performed manual editing. Although the 3D auto RV software worked worse in patients with severely decreased RVEF, the 3DE with fully automated analysis also diagnosed that the patient had severely impaired RVEF regardless of whether the values were different.
The FR plays an important role in the accuracy of 3D auto RV measurements. Our study revealed that novel 3D auto analysis for RV volumes performed better in high FR (>23 frame/s) than those of low FR (≤23 frame/s) regardless of whether the manual editing was performed. These findings are similar to those in previous study, which demonstrated that the biases and LOA for LV volumes using the 3D auto LV quantification software were larger in patients with low volume rates (22). However, our study revealed that FR had no significant impact on the RVEF, which was consistent with the prior study regarding the effect of 3DE volume rate on the LVEF using the 3D auto LV quantification software (22).
In this study, we tested the performance of 3DE auto RV software against CMR for RV quantification. The impressive feature of this software is the more rapid analysis and excellent reproducibly. Nevertheless, our study revealed that 3D auto RV software performed worse in subjects with RV dilation. Moreover, the severe outlier may occur in patients with extremely dilated right ventricle. Therefore, the novel software is needed to learn a large amount of RV contour information with the diverse pathology-related geometric changes to enhance the accuracy in a pathology-dependent way. With future technological improvements and machine learning algorithmic refinements with additional training on larger data sets, the percentage of subjects with accurate fully automated RV measurements without manual edit will rise over time.

LIMITATIONS
Although our study population consisted of a large number of patients with a wide range of RV sizes and function, it was still a single-center study with relatively limited sample size, which cannot be generalized to the whole community. Future multicenter studies with larger sample sizes are required to confirm our results. Forty-eight percent of the patients were included in the waiting list of heart transplantation; therefore, our study population cannot reflect the general population. Since both CMR and 3DE investigations were done at different times, there may be real volume discrepancies that limit agreement and trend. In addition, 3DE has lower temporal and spatial resolution than 2DE, which may have effects on RV measurements. Another important limitation is that the evaluated technology may depend on vendors regarding format and availability of images. Therefore, our findings might not be extrapolated to other vendors and software packages. The fully automated RV software may not be appropriate for patients with poor image quality. Hence, the accuracy of this software in individuals with poor image quality cannot be determined. Moreover, 3D full volume technique is strongly dependent on heart rate, so it is challenging in patients with arrhythmia such as atrial fibrillation. Thus, our results are not applicable to patients with irregular rhythms.

CONCLUSION
The novel fully automated 3D software for RV volumes and EF quantification is highly feasible and reproducible. The values measured by novel fully automated RV software were strongly correlated with the CMR measurements in a large number of population with a diverse range of RV sizes and function. These findings support the routine use of the novel 3D auto RV software in daily clinical practice.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the Institutional Ethics Board of Union Hospital, Tongji Medical College, Huazhong University of Science and Technology. The patients/participants provided their written informed consent to participate in this study.