Cardiac catheterization is a common procedure used to diagnose and treat patients with a variety of conditions such as congenital heart disease (CHD) (1–3). Catheterization procedures are traditionally navigated under fluoroscopic guidance (4), which exposes patients to harmful ionizing radiation and the unwanted risk of developing cancer, particularly in children and young adults who undergo long and repeated procedures (5–7). Fluoroscopic guidance also suffers from poor soft tissue visualization, making it difficult to position catheters and balloons, with the potential increased risk of instrumental perforations in patients (5).

Magnetic Resonance Imaging (MRI)-guidance is an attractive alternative to fluoroscopic guidance as it involves no radiation risks to patients involved, has superior soft tissue visualization and superior hemodynamic data using MR flow imaging (8–18). MRI-compatible balloon-wedge catheters are used during these procedures. The catheter tip is visually tracked passively using dynamic real-time imaging (∼5–10 images/s) from its corresponding hyper- or hypo-intense signal when using diluted gadolinium or CO₂-filled balloons, respectively (18). Several techniques exist for improved positive contrast visualization of the catheter tip when using gadolinium-filled balloons. These include the application of saturation pre-pulses intermittently (i.e., switching between balloon or blood/cardiac anatomy visualization) (18) or using black-blood preparation for simultaneous balloon and cardiac anatomy visualization (19). The use of partial saturation (pSAT) pre-pulses (i.e., a non-selective saturation pulse with a reduced flip angle, typically 30–70°) (16) has the advantage over the other two methods of providing simultaneous high-contrast visualization of the catheter tip, soft tissue and blood (17). Despite the benefits of MRI-guidance, several major technical challenges remain to be addressed to ensure clinical acceptance of this method. Current imaging techniques require frequent manual updating of the imaging plane location to track the catheter tip during navigation to deal with the catheter tip frequently falling out-of-plane. This typically arises when navigating the catheter within cardiac chambers, large vessels, or complex anatomical vessels which cannot be fully sampled using a 2D imaging plane. One study reported an average balloon visibility of <70% of the scanning time (17). This limitation leads to the unnecessary complication and slowdown of catheterization procedures when relocating the catheter tip. It has been shown that by acquiring a heavily T1-weighted image of the catheter using a large slice thickness and overlaying this onto a high-resolution image of the anatomy of interest improves catheter visibility to ∼90% of the scanning time (20). However, the larger slice thickness may also reduce the value of MRI-guidance in narrow anatomical structures.

Automatic tracking of the catheter and imaging plane during navigation may potentially facilitate and shorten the procedure. In a recent proof-of-concept study, automatic slice tracking based on real-time image processing and estimation of the catheter tip location was proposed to enable continuous visualization of the catheter tip during navigation (21). This approach relies on the dynamic acquisition of three contiguous slices, typically using a bSSFP readout for improved signal-to-noise ratio and reduced flow artefacts. When the catheter is detected in one of the outer slices, the image plane is prospectively adjusted automatically to maintain the catheter in the central slice. Despite the potential of this framework to achieve near 100% visibility of the catheter during navigation, it depends on several user-definable parameters, such as a bounding box definition to contain expected catheter trajectories, image segmentation threshold values, pattern matching and catheter movement speed. These parameters not only rely on user experience but may reduce the framework's robustness in patients.

In this study, an automatic, parameter-free, deep-learning-based post-processing pipeline was developed for robust real-time image-based detection of the catheter tip coordinates. This method was evaluated retrospectively in seven patients who underwent MRI-guided right heart catheterization using the aforementioned prototype catheter tracking sequence.

All imaging was performed on a 1.5 T MRI scanner (MAGNETOM Aera, Siemens Healthineers, Erlangen, Germany).

Supplementary Figure S1 shows the training and validation dice loss plot over 200 epochs. The global minimum of the validation loss was determined at epoch 169.

Representative example predictions of the network alone, for each of the seven patients who underwent MRI-guided cardiac catheterization, are shown in Figure 2. A large variation of catheter-to-blood contrast, catheter balloon size/shape, and overall anatomy can be observed within this patient cohort. Despite these variations, accurate prediction of the catheter balloon was achieved in all subjects using the proposed network. One representative example of true negative detection is also shown (see the bottom right case in Figure 2). No catheter was present in any part of this image. Interestingly, despite the presence of hyper-intense signal, specifically from fatty tissue, the network was able to predict the absence of the catheter balloon. On an image basis, the mean accuracy, sensitivity and specificity of the network alone over all patients were 87.5 ± 10.9%, 93.4 ± 13.8% and 85.3 ± 13.8%, respectively.

Figure 3 depicts the complete reconstruction pipeline in two patient examples (patients #6 and #7) where the tracking of the catheter is depicted in the three dynamically acquired contiguous images. Significant bright fat tissue is also visible in these images but did not affect the output of the network. Furthermore, the catheter was initially detected in two adjacent slices for both patients using the network alone. The proposed post-processing strategy resolves this ambiguity and identifies the slice containing most of the catheter. Supplementary Video S1 shows the corresponding dynamic video for patient #6. Accurate catheter detection was achieved in the majority of dynamics.

The detailed accuracy, specificity and sensitivity measured for each patient are shown in Table 1. While the worst accuracy was 95.2% in patient #4, an accuracy of 100% was achieved in 4/7 patients. Over all patients, the mean accuracy, specificity and sensitivity of the complete catheter tracking pipeline were 98.4 ± 2.0%, 99.9 ± 0.2% and 95.4 ± 5.5%, respectively.

High repeatability and reproducibility of the manual segmentation of the catheter was observed, as denoted by an intra- and inter-operator percentage of agreement of 100% and 100%, respectively.

The computational time of this automatic tracking pipeline was ∼30 ms, with the network prediction accounting for ∼10 ms/image.

In this work, an automated, parameter-free, post-processing strategy was developed using deep-learning for image-based detection of gadolinium-filled balloon wedge catheters during cardiac catheterization procedures. The proposed method provides excellent accuracy, sensitivity and specificity, with computation times compatible with real-time constraints.

Semi-artificial data were used for the training process due to the limited availability of real data. The fact that semi-artificial data can be successfully used for the training of such a network is an important finding, especially when the scarcity of very large training datasets for artificial intelligence applications has been identified as a major obstacle, particularly in the field of cardiac magnetic resonance imaging (27). This also greatly reduces the time-consuming and laborious nature of manually annotating large volumes of data required for training. A potential limitation to placing artificial catheter signal on underlying images is that signal may not fully represent the variability (i.e., shape and intensity) observed in real-world data. Similarly, the choice made by a single operator of placing the artificial signal at specific regions in the cardiovascular anatomy could lead to bias in the training data. However, the high performance of the proposed approach observed in this study indicates these effects may be limited.

Furthermore, all semi-artificial data were based on images acquired in adults and two orientations only (transverse and coronal). The test patient cohort consisted predominately of paediatric patients with all images acquired in an unseen orientation during the training (i.e., sagittal). Imaging protocols used for the acquisition of the training and testing datasets also had slightly different imaging parameters. Finally, images from this test cohort had high variability in terms of anatomy as well as catheter balloon contrast/shape. Therefore, the high accuracy of the proposed tracking strategy in the test dataset suggests that the network was able to adapt to different scenarios unseen during the training process. The network adaptability is particularly important since large anatomical variations are expected in the patient population referred for MRI-guided cardiac catheterization and different slice orientations are also expected to be used during catheter navigation (17).

The simplicity and success of the U-net model and its variants have resulted in its extensive adoption as the primary tool for segmentation tasks within the medical imaging community (22, 28–31) and thus was the choice of network used in this study. As discussed in Section 2.1., the U-Net model was used incorporating a ResNet-34 encoder. This model proved to be highly effective in achieving high accuracy, specificity, and sensitivity in the detection of the presence or absence of balloon signal. It is important to note that ResNet-34 is one of several potential choices for the encoder in the U-Net architecture. Other popular encoder architectures include, for example, VGG-19 (32), Inception (33) and SE-Net (34) which have shown to provide good performance for semantic segmentation in medical imaging (29, 35–37). Such models could be explored in future work to further improve the performance of the network. Future work using segmentation models that leverage the temporal dimension of data to improve segmentation accuracy could also be investigated, such as the use of convolutional LSTMs (12).

Gadolinium-filled balloon catheters were used in this study. Gas-filled balloon catheters can also be used and are visualized through the signal voids they generate. Although gas-filled balloons were shown to be less conspicuous than Gadolinium-filled balloons, they offer improved buoyancy which may be beneficial for catheter navigation (13). Retraining of the network using images with negative contrast is expected to be required and could be facilitated using the existing simulated dataset which enables easy generation of varying catheter contrasts. The potential of the proposed tracking strategy for negative contrast catheter will be the focus of future work.

Although the proposed tracking strategy was applied for the detection of gadolinium-filled balloon wedge catheters during MRI-guide cardiac catheterization procedures, it may also benefit the tracking of other interventional devices. For example, the network could be retrained to learn specific signal signatures produced by an MR-compatible guidewire or needle.

This study has some limitations. First, the catheter tracking sequence used in this study relies on the dynamic acquisition of three contiguous slices, which results in a 3-fold reduction of the temporal resolution. Future work will include the integration of advanced acceleration techniques to improve the temporal resolution of the framework. Second, this is a retrospective study in a small patient cohort demonstrating the potential of automatic parameter-free image-based tracking of the catheter balloon. A further prospective study, including the online integration of the proposed approach in a larger patient cohort during an entire cardiac catheterization procedure, is now needed to assess its clinical impact, both in terms of catheter tip visualization, and prospective tracking during catheter navigation.

In conclusion, deep-learning-based catheter balloon tracking is feasible, accurate, parameter-free and compatible with real-time conditions. Online integration of the technique and its evaluation in a larger patient cohort are now warranted to determine its benefit during MRI-guided cardiac catheterization.