Sex-Specific Patterns of Mortality Predictors Among Patients Undergoing Cardiac Resynchronization Therapy: A Machine Learning Approach

Introduction

Despite the comparable overall lifetime risk of heart failure (HF) between sexes (1, 2), there are notable differences between males and females with HF across the entire spectrum of ejection fraction (EF) (3). In HF patients with reduced EF (HFrEF), several studies have highlighted sex-related differences that involve multiple aspects of the syndrome, such as epidemiology, pathophysiology, phenotyping, and prognosis (4). Nevertheless, females are under-represented in HFrEF trials questioning their generalizability and leaving significant gaps in knowledge (4, 5).

While women with HFrEF have better survival and lower hospitalization rates, they have a greater burden of symptoms and more impaired health-related quality of life than men (6). Although sex disparities are also remarkable in the accessibility to HF device therapy, including cardiac resynchronization therapy (CRT) (7–9), women are more likely to respond favorably and derive a greater survival benefit from CRT implantation (10–13). Nonetheless, the sex-related differences in both short- and long-term outcomes and the varying importance of different predictors are still scarcely explored in this patient population (14). One conceivable explanation could be the failure of the applied statistical methods to harness the potential prognostic value of complex interactions between several weaker, often unexpected risk factors and the outcome. However, this limitation might be circumvented by advanced data analytic techniques (15).

To improve predictive modeling and elucidate novel determinants of a specific outcome, machine learning (ML) has been increasingly utilized in cardiovascular research (16–20). ML represents a collection of algorithms that autonomously acquire knowledge by identifying patterns from complex, multi-dimensional datasets. ML models can account for interactions between myriads of predictors and their non-linear associations with the outcome; therefore, their utilization could potentially lead to improved explanatory models (21).

In the current study, we sought to implement and evaluate ML algorithms for the prediction of 1- and 3-year all-cause mortality among patients undergoing CRT implantation. We also aimed to explore the sex-specific differences and similarities in the predictors of mortality using advanced ML-based approaches.

Methods

Study Population and Protocol

We identified 2,412 patients with chronic HFrEF (NYHA functional class II-IV) who underwent successful CRT implantation at the Heart and Vascular Center of Semmelweis University (Budapest, Hungary) between September 2000 and September 2018. For each patient, pre-implant clinical characteristics (demographics, medical history, physical status, vitals, currently applied medical therapy, ECG-, echocardiographic- and laboratory parameters) and procedural parameters [type of the implanted device, left ventricular (LV) lead position] were collected retrospectively from paper-based or electronic medical records and entered to our structured database.

The study protocol complies with the Declaration of Helsinki, and it was approved by the Regional and Institutional Committee of Science and Research Ethics (Approval No. 161/2019).

Study Outcomes

Follow-up data [status (dead or alive), date of death] was obtained for all patients by querying the National Health Insurance Database of Hungary in September 2019. Accordingly, all patients included in our database were followed for at least 1 year or died within 1 year. In the entire study population, 2,116 patients also had 3-year outcome data available. The primary endpoint of our study was all-cause mortality.

Feature Selection and Data Pre-processing

The data analysis pipeline, including feature selection, data pre-processing, and ML model development and evaluation is illustrated in Figure 1.

FIGURE 1

Figure 1. The schematic outline of the data analysis pipeline. The data analysis pipeline included three major steps: (1) data pre-processing, (2) machine learning model development and evaluation, and (3) the calculation of feature importances. During data pre-processing, feature selection was performed, patients with a high proportion of missing data were excluded, missing values were imputed using MICE, and z-transformation was performed. Then, machine learning models were implemented in the 6 partially overlapping subsets of patients (in all patients, females, or males of the 1- and 3-year cohorts). Before model training, each patient subset was split into training and test cohorts (80:20 ratio). Hyperparameter tuning was performed with 10-fold CV in each training cohort. Models' discriminatory power was estimated using the area under the receiver-operating characteristic curves. Each of the 6 models was retrained in the given training cohort, and its performance was evaluated in the corresponding test cohort. Finally, to identify the most important predictors of mortality in each subset, permutation feature importances were computed from each of the 6 final models. See text for further details. AUC, area under the receiver operating characteristic curve; CRT, cardiac resynchronization therapy; CV, cross-validation; MICE, Multiple Imputation by Chained Equations.

Feature selection included two consecutive steps. First, any feature with ≥40% missing data was removed. Second, collinear variables (Spearman correlation coefficient ≥ 0.3 or ≤-0.3) were also excluded as variables containing redundant information might bias the further steps of the analysis (Supplementary Figure 1). The final set of input features comprised 30 pre-implant and procedural variables: baseline demographics and clinical characteristics (n = 10), comorbidities (n = 6), ECG- (n = 1), laboratory parameters (n = 3), and currently applied medications (n = 10). The list of candidate variables and the feature selection process are presented in Table 1.

TABLE 1

Table 1. Steps of feature selection and the list of clinical features included in the machine learning models.

Patients with more than 30% of missing values were excluded from further analyses. Missing values were imputed using Multiple Imputation by Chained Equations (MICE). As the range of different continuous features varied widely, Z-score transformation was applied after imputation to eliminate the possibility of model bias caused by the differing magnitude of the numerical values.

ML Model Development and Evaluation

We developed ML models to predict two separate outcomes: (1) 1-year all-cause mortality, and (2) 3-year all-cause mortality in the entire cohort, in males and females separately (a total of 6 separate binary classification tasks). To quantify a model's discriminatory power, receiver operating characteristic curve analysis was performed, and the area under the curve (AUC) was calculated. Model development included trials of several binary classifiers such as logistic regression, support vector machines, k-nearest neighbors classifier, gradient boosting classifier, traditional random forest (TRF), conditional inference random forest (CIRF), and multi-layer perceptron.

As the first step of model derivation, 20% of the given patient subset (all, males or females) was randomly selected as the holdout (test cohort). This split was performed in a stratified manner to ensure that the original ratio of outcomes is preserved in the training and test cohorts. Hyperparameter tuning was performed with stratified 10-fold cross-validation in the remaining data (80%, training cohort). The algorithm (with fine-tuned hyperparameters) exhibiting the highest AUC was then retrained in the entire training cohort, and its performance was evaluated in the test cohort in a statistically independent way. Finally, calibration of the ML models was assessed in the test cohort using Brier score (ranging from 0 to 1, with 0 representing the best possible calibration), which is defined as the mean squared difference between the observed outcomes and the predicted probabilities.

Feature Importances

To determine the major predictors of 1- and 3-year all-cause mortality in each patient subset, permutation feature importances were computed from each of the 6 final models. Briefly, the importance of an input feature is measured by calculating the increase in the model's prediction error after permuting its values while keeping other features the same as before. In the current study, permutation was performed 10 times for each feature. A feature is considered important if shuffling its values decreases the model's discriminatory power (AUC) as the model relies heavily on that feature for the prediction. On the other hand, a feature is unimportant if shuffling its values leaves the AUC unchanged because, in this case, the model ignores the feature while predicting the outcome. After calculating the importance of each feature, we divided it by the AUC measured in the dataset before shuffling any of its features to enable the comparison of feature importances between different models.

Results

Baseline Clinical Characteristics and All-Cause Mortality

The final 1- and 3-year cohorts included 2,191 (74.7% males, 56.7% CRT-D) and 1,900 patients (75.0% males, 54.1% CRT-D), respectively (Figure 2). In the 1-year cohort, 50.4% of the patients had ischemic etiology of HF, 57.8% had NYHA functional class III/IV, and the median left ventricular EF (LVEF) was 28 (24–32) %. In the 3-year cohort, ischemic etiology was reported in 51.5% of the patients, 61.0% presented with NYHA functional class III/IV, and the median LVEF was 28 (24–32) %. The baseline clinical characteristics of the patients are summarized in Tables 2, 3.

FIGURE 2

Figure 2. Flowchart illustrating the steps of patient selection. For each patient who underwent successful CRT implantation at our center, pre-implantation clinical characteristics and procedural parameters were collected retrospectively from paper-based or electronic medical records and entered to our structured database. After excluding patients with ≥30% missing values, machine learning models were implemented to predict 1- and 3-year all-cause mortality in the entire cohort, in males and females separately (altogether 6 separate binary classification tasks). CRT, cardiac resynchronization therapy.

TABLE 2

Table 2. Clinical characteristics of the 1-year cohort.

TABLE 3

Table 3. Clinical characteristics of the 3-year cohort.

In the 1-year cohort, 203 (12.4%) men and 49 (8.8%) women died during the 1-year follow-up period. Univariable Cox regression analysis revealed a significantly lower risk of all-cause mortality in women compared to men [Hazard Ratio (HR): 0.698, 95% Confidence Interval (CI): 0.511–0.954; p = 0.024]; however, after adjusting for age, etiology of HF, QRS morphology, type of implanted device, and type of atrial fibrillation (AF, history of or current), we could not observe a significant difference between sexes (HR: 0.803, 95% CI: 0.581–1.110; p = 0.183) (Figure 3A).

FIGURE 3

Figure 3. Kaplan-Meier curves for males and females in the 1- (A) and 3-year (B) cohorts. Kaplan-Meier curve analysis illustrates the difference in the survival of male and female CRT patients during 1- and 3-year follow-up. Cox proportional hazards models were used to compute hazard ratios with 95% confidence intervals. Hazard ratios were adjusted for age (at implantation), QRS morphology, etiology of heart failure, the type of the implanted device, and the type of atrial fibrillation. CI, confidence interval; CRT, cardiac resynchronization therapy; HR, hazard ratio.

As observed in the 1-year cohort, males exhibited significantly higher mortality rates compared to females in the 3-year cohort as well [502 (35.2%) vs. 113 (23.8%); p < 0.001]. The univariable Cox regression analysis also confirmed this finding as it showed a significantly lower risk of all-cause mortality in females compared to males (HR: 0.625, 95% CI: 0.510–0.767; p < 0.001) (Figure 3B). Moreover, this difference remained significant even after adjusting for the previously listed covariates (HR: 0.686, 95% CI: 0.555–0.848; p < 0.001).

Patients with ischemic etiology had a significantly increased risk of death in both sexes; however, this difference was more pronounced in females compared to males in the 1- and 3-year cohorts as well (Supplementary Figure 2).

ML for the Prediction of All-Cause Mortality

Among the evaluated ML classifiers, CIRF exhibited the best performance for discrimination between survival/all-cause death with an AUC of 0.717 (95% CI: 0.676–0.758) and 0.739 (95% CI: 0.715–0.762) in the 1- and 3-year training cohorts, respectively (Supplementary Tables 5, 6). When evaluating the models' discriminatory power in the test cohorts, we observed an AUC of 0.728 (95% CI: 0.645–0.802) and 0.732 (95% CI: 0.681–0.784) for the prediction of 1- and 3-year mortality, respectively. Models were also trained and tested separately in the female and male subsets of the 1- and 3-year cohorts. The AUCs ranged from 0.712 to 0.748 in the training sets and from 0.681 to 0.798 in the test sets suggesting a modest variability in the models' predictive capabilities across the different subsets of patients (Supplementary Tables 7, 8).

After sorting the patients in ascending order based on the predicted probability of death and plotting the distribution of probability values, the accumulation of patients who died during the given follow-up period could be observed in the higher risk regions of the plots (Supplementary Figure 3). These findings suggest that our models can perform risk stratification effectively.

The Brier score—measuring the accuracy of the probabilistic predictions—for the 1- and 3-year models were 0.197 and 0.201, indicating a sufficiently good calibration of our models. Supplementary Table 9 summarizes the Brier scores for the remainder of the CIRF models.

Most Important Predictors of Mortality as Assessed Using ML

Leading predictors of all-cause mortality are illustrated in Figure 4, and the comprehensive list of feature importances is provided as Supplementary Tables 10, 11.

FIGURE 4

Figure 4. The most important predictors of 1- and 3-year all-cause mortality in patients undergoing CRT implantation. The importance of each feature was quantified with the permutation feature importances method, which measures the importance of a feature by calculating the mean decrease in the model's performance (area under the receiver-operating characteristic curve) after permuting its values 10 times (see text for further details). To keep the data comparable between the different models, we identified the top 5 predictors in each model and took the union of these features; then, we plotted the results on radar charts. AF, atrial fibrillation; LBBB, left bundle branch block; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association.

Top Predictors of Mortality in the 1- and 3-Year Cohorts

In the overall study population (including both sexes), the most important predictor of 1-year mortality was serum sodium, which was followed by serum creatinine, hemoglobin concentration, age, and etiology of HF (Figure 4). These features were also found among the strongest predictors of 3-year mortality, however, in different order of importance (serum sodium, age at implantation, hemoglobin concentration, serum creatinine, and etiology). Digitalis and type of AF were found to show the most prominent change in their importance from 1 to 3 years (both p < 0.001).

Sex-Specific Patterns of Mortality Predictors at 1-Year Follow-Up

We observed several sex-specific differences during the subgroup analysis. In males, the top predictors of 1-year mortality were hemoglobin concentration, serum sodium, serum creatinine, LBBB morphology, and age, whereas, in females, the most important predictors were serum sodium, etiology, LVEF, age, and serum creatinine (Figure 4).

The comparison of predictors by sex at 1-year revealed that etiology (p < 0.001), LVEF (p < 0.001), and treatment with amiodarone (p < 0.01) were at least twice as important in females as in males. Moreover, age at implantation and NYHA functional class were also significantly more predictive for 1-year mortality in women compared to men (both p < 0.001). Whereas, in males, hemoglobin concentration, type of the implanted device, treatment with allopurinol had significantly higher predictive power than in females (all p < 0.001).

Sex-Specific Patterns of Mortality Predictors at 3-Year Follow-Up

In males, the strongest determinants of 3-year mortality were serum sodium, hemoglobin concentration, age at implantation, serum creatinine, and allopurinol, whereas, in females, these features were serum sodium, age at implantation, type of AF, NYHA functional class, and etiology in decreasing order (Figure 4).

Regarding females, NYHA functional class, etiology, LVEF, and type of AF exhibited significantly higher predictive power than in men (all p < 0.001). In males, features with at least a 2-fold higher importance were loop diuretics (p < 0.001), hemoglobin concentration (p = 0.021), allopurinol (p < 0.001), diabetes (p < 0.001), LV lead position (p < 0.001) and LBBB morphology (p < 0.001).

Longitudinal Changes in the Sex-Specific Patterns of Mortality Predictors

We also identified features with the most prominent changes in importance from 1 to 3 years of follow-up.

Among males, the most prominent increase of feature importance occurred in LV lead position, NYHA class, age, type of AF, hypertension, and digitalis (all p < 0.001). The importance of serum creatinine declined significantly (p = 0.026).

In females, we observed the greatest increase in the importance of NYHA functional class (p < 0.001), type of AF (p < 0.001), hypertension (p < 0.001), and age at implantation (p < 0.014). Among the top 10 predictors, the most considerable decrease from 1- to 3-year in feature importance was noted in the following factors: serum creatinine, LV end-diastolic diameter, QRS morphology, and amiodarone (all p < 0.001).

In-depth Analysis of the Associations Between Top Predictors and Outcomes

The association between the most important predictors and the predicted outcome is visually presented in Figures 5, 6. Older age, higher serum levels of creatinine, lower values of LVEF, serum sodium, hemoglobin concentration, ischemic etiology, non-LBBB morphology, higher NYHA classes, and the history of or current paroxysmal, persistent or permanent AF were associated with a higher predicted probability of 1- and 3-year all-cause mortality. Males exhibited higher values of predicted probability of all-cause death in all examined features compared to females. However, as ML models capture complex, high-level interactions among a multitude of variables, it is challenging to determine the effect of a single feature on the predicted probability of mortality, and the results of univariable analyses should be interpreted with caution.

FIGURE 5

Figure 5. Effect of the most important features on the predicted probability of 1-year all-cause mortality in the training cohorts. The probability of death was calculated for each patient in the training cohort with 10-fold cross-validation. The predicted probability is plotted for each patient, and second-order polynomial trendlines are fitted to their values. *p < 0.05 vs. non-ischemic/non-LBBB morphology/NYHA class II/no AF, unpaired Student's t-test or Mann-Whitney U test. Abbreviations as in Figure 4.

FIGURE 6

Figure 6. Effect of the most important features on the predicted probability of 3-year all-cause mortality in the training cohorts. The probability of death was calculated for each patient in the training cohort with 10-fold cross-validation. The predicted probabilities are plotted for each patient, and second-order polynomial trendlines are fitted to their values. *p < 0.05 vs. non-ischemic/non-LBBB morphology/ NYHA class II/no AF, unpaired Student's t-test or Mann-Whitney U test. Abbreviations as in Figure 4.

Discussion

Using data from a single-center cohort of HF patients undergoing CRT implantation, we developed and evaluated ML-based algorithms for the prediction of 1- and 3-year all-cause mortality. The resulting CIRF models demonstrated good discriminatory power in assessing the risk of mortality with an AUC over 0.700 at 1- and 3-year follow-up. Moreover, ML performed substantially well across patient subsets containing exclusively males or females (AUCs ranging from 0.681 to 0.798). Serum sodium, creatinine, hemoglobin, age, and HF etiology were among the most important determinants of short- and mid-term mortality; however, their relative importance varied over time. As expected, female sex was associated with significantly better survival rates in our cohort as well. Sex-specific patterns were also identified in the predictors of mortality. The role of HF etiology (ischemic or non-ischemic), NYHA functional class, and LVEF were more pronounced in females, whereas hemoglobin concentration, QRS morphology, and treatment with allopurinol were notably more predictive for all-cause mortality in males.

Risk Stratification of HF Patients Using ML

The personalized prediction of prognosis is fundamental to patient-centered care, both in optimizing treatment strategies and informing patients as part of shared decision making. For this purpose, an abundance of prediction models has been developed; however, most of them had achieved only modest success, particularly when they were applied in HF populations other than those from which the scores were derived (22, 23). The unsatisfactory results of previous HF risk scores are likely due to multiple causes, including the fact that most of them were created using conventional statistical methods that failed to capture high-dimensional interactions among predictors that bear relevant prognostic information.

In contrast to traditional statistics, ML was explicitly designed to reveal and harness these correlations. Several studies have proved that these advanced data analytic approaches can leverage the complex, higher-level interplay between predictors and outcomes to achieve better discrimination. ML can improve the care of HF patients in various ways, e.g., by augmenting the prediction of readmission after HF hospitalization or by predicting the risk of mortality (16, 17, 19). In HF patients undergoing CRT implantation, our research group has previously confirmed the superiority of ML over pre-existing risk scores (24), and similar results have been reported by others as well (25, 26). Underpinning these findings, we were able to predict the 1- and 3-year mortality of CRT patients with good discrimination and excellent calibration, even in subsets of patients divided by sex. In light of the promising results of our single-center study, we will endeavor to validate our models in external cohorts in a multi-centric manner.

In our analysis, CIRF exhibited the best discriminative ability for predicting both 1- and 3-year mortality. To understand the outstanding performance of tree-based approaches such as CIRF in outcome prediction, an important difference between conventional regression models and tree-based methods should be highlighted. The former favors variables that have a uniform effect across the entire patient population, whereas the latter can uncover variables that might act differently in different patient subgroups. This is essential for personalized prognostication as in an individual patient, the discriminatory power of a given feature may be significantly enhanced or overshadowed by others. Due to this attribute, tree-based methods such as TRF and CIRF are extremely suitable for application as clinical decision-making tools (27).

Sex-Specific Differences in Outcomes Following CRT Implantation

Sex is increasingly recognized as an important modulator of outcomes in CRT patients, and several studies such as the MADIT-CRT (10), the RAFT (28), or the MASCOT (29) trials have suggested a greater CRT benefit in women. Despite the expanding knowledge about sex-related differences in HFrEF, the reason women benefit more than men from CRT remains unclear (14). Numerous plausible explanations have been proposed, such as the dissimilarities between sexes in the frequency of ischemic cardiomyopathy (30), AF, and comorbidities (9), or the sex-related differences in body height, LV size, and QRS duration (31, 32). In addition, the impact of sex hormones on the pathophysiology of HF or the sex-specific characteristics of pharmacodynamics and pharmacokinetics are also considerable factors (4, 33).

The sex-specific effects of QRS prolongation and morphology on outcomes have been intensively investigated in CRT patients (30, 31, 34–37). Thus, the findings of these studies have prompted calls for sex-specific guideline recommendations regarding the selection of CRT recipients. As women have shorter QRS durations than men in the absence of any conduction delay, they are more likely to exhibit a true LBBB compared to men at shorter QRS duration (38, 39). It has also been reported that among patients with LBBB and non-ischemic etiology, women have electrical dyssynchrony more frequently compared to men at any given QRS duration, and consequently, they would exhibit a better response to CRT (35). According to the study conducted by Beela et al., the interaction between HF etiology and mechanical dyssynchrony seems to represent another important aspect: due to the lower rate of ischemic etiology and the lower extent of scarred myocardium, women have more frequently uncomplicated patterns of LBBB-like mechanical dyssynchrony which is better amendable by CRT (30).

The beneficial effects of CRT also depend on device programming and the percentage of effective biventricular pacing. Notably, that latter significantly varies by sex, and therefore, sex-specific CRT programming has attracted increased attention (40). According to the results of the SMART-AV trial, the optimization of atrioventricular delay intervals is associated with improved outcomes in women but not in men (41), which might be attributable to the inherent sex-related differences in atrial geometry and PR intervals. A higher percentage of biventricular pacing has also been reported in women (29, 41, 42), most probably due to the lower rate of atrial fibrillation compared to men (43, 44). This could also contribute to the observed differences in mortality between sexes as even a small increment in the biventricular pacing rate may improve outcomes (45).

Although there are still many open questions, it is clear that multiple intercorrelated factors contribute to this phenomenon. Therefore, during the search for answers, ML-based approaches may come in handy, as they are particularly helpful in uncovering hidden patterns in large datasets by simultaneously interpreting predictors even in the presence of complex, non-linear interactions.

Sex-Specific Patterns in Mortality Predictors

Given the sex-related differences in the anatomy and physiology of the cardiovascular system, encountering dissimilarities in the importance of prognostic predictors between males and females is to be expected in CRT patients. Nevertheless, there is only a limited number of publications dedicated to the thorough exploration of this topic. To the best of our knowledge, our study is the first that evaluated the sex-related differences and similarities in mortality predictors of CRT patients using ML. In our analysis, we observed significant variations in the importance of several predictors such as HF etiology, NYHA functional class, LVEF, and AF between sexes, to name a few.

Utilizing the tools of conventional statistics, the sex-specific prognostic value of HF etiology has been previously investigated in large cohorts of HFrEF patients. In the MAGGIC meta-analysis, the ischemic etiology appeared to attenuate the protective effect of female sex on prognosis (46). In addition, ischemic cardiomyopathy and the extent of myocardial scar were found to be significant predictors of mortality in females but not in males among CRT patients (30). In line with this evidence, the paramount importance of HF etiology in women was proved in our study as well.

When analyzing the interaction between sex and different covariates in the prediction of survival after CRT implantation, Beela et al. reported that NYHA class was a significant predictor in males only (30). Moreover, among HFrEF patients, NYHA class had a more prominent prognostic value in men than in women (3). Contrary to these findings, a stronger association of NYHA functional class with outcomes was observed in females in our current analysis and the BEST trial as well (47).

Another well-established prognostic factor is LVEF, whose interaction with sex in the prediction of all-cause death has been demonstrated in CRT patients (30). Complementing these findings and the results of the BEST trial (47), we have also demonstrated that LVEF is a stronger predictor of prognosis in women than in men.

In HFrEF patients, most studies agree on the prognostic value of AF; however, there is some inconsistency regarding its exact role as some investigations attribute more prognostic impact to AF in females (47), whereas others observed comparable predictive power in males and females (3, 30). Our results support the former as we found AF to have a more prominent effect on outcomes in females.

According to our analysis, the prognostic relevance of hyponatremia and renal function should also be emphasized in CRT patients. Our results are in accordance with the findings of Zusterzeel et al., who reported that despite being significant determinants in both sexes, serum creatinine and hyponatremia appeared to be stronger predictors in women than in men (34).

Lately, the interplay between sex and diabetes in HFrEF patients has attracted increased attention among researchers. Confirming the findings of the MAGGIC (46), the recently published analysis of the ASIAN-HF registry demonstrated that diabetes is coupled with a greater risk of adverse outcomes in women than in men (48). In contrast, diabetes was associated with a higher risk of all-cause death or HF hospitalization in males in the Swedish HF Registry (3), and it was proven to be a significant predictor only in men in the BEST trial (47). Interestingly, in our study, diabetes was not ranked among the top five predictors in any of the analyzed patient subsets, and we detected inter-sex differences in its importance only at 3-year follow-up.

Some of our findings coincide with those of previous studies, whereas some others may not. These apparent discrepancies might be partly attributable to the fact that most studies applied Cox proportional hazards regression, whereas we utilized an entirely different methodology that captures other aspects of associations between risk factors and outcomes. Although the exact reasons behind these contradicting results should be clarified in further investigations, our findings underscore the necessity of sex-specific approaches in the management of HFrEF patients.

Limitations

Despite the highlighted advantages, there are a few limitations to be acknowledged. First, our study represents results from a single center. As we were aware of this limitation, we performed hyperparameter tuning with 10-fold cross-validation in the training cohorts, and we also tested our models in statistically independent test cohorts to enhance generalizability. Nonetheless, as the next step, the robustness of our models should be tested in external populations as well. Second, the utilized database bears the inherent limitations of retrospective data collection, such as the higher proportion of missing data (compared to prospective trials) and the heterogeneity partly attributable to the changes in guideline recommendations over the years. However, the use of such real-world data holds the potential for better generalizability. Third, our models use baseline (pre-implant and procedural) variables without incorporating the time-varying values of these parameters. Although a dynamic model integrating values of the same parameter from multiple time points may be superior, in the present study, we aimed to predict 1- and 3-year mortality using clinical data that could be acquired at device implantation. Finally, there may remain additional domains of variables (e.g., imaging data, novel biomarkers, genetics, or quality of life questionnaires) that could further improve the predictive capability of our models. Future work should explore the addition of such features to enhance the models proposed in the present study.

Conclusions

Using advanced ML techniques in combination with easily obtainable clinical features, our models effectively predicted 1- and 3-year all-cause mortality in patients undergoing CRT implantation. ML also exhibited good discriminative ability in patient subsets containing males or females exclusively. Moreover, sex-specific patterns of mortality predictors were identified, which also changed over time. These models lay the foundation stone for future testing of their clinical utility as decision support tools to optimize candidate selection and to improve the prognostication of CRT patients.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Regional and Institutional Committee of Science and Research Ethics (Approval No. 161/2019). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author's Note

A part of the results was presented at the Ph.D. Scientific Days (2020) of the Semmelweis University and at the annual scientific congress of the European Society of Cardiology (ESC Congress 2020—The Digital Experience).

Author Contributions

MT participated in the conceptualization and designing of the study, implemented the machine learning models and analyzed the data, interpreted the results, and was a major contributor to writing the manuscript. AB was a major contributor to data collection, participated in the interpretation of results, reviewed the literature, and thoroughly reviewed the manuscript. EM participated in data collection, helped in reviewing the literature, and reviewed the manuscript. AKov participated in the conceptualization and designing of the study and critically reviewed the manuscript. ZT made a major contribution to the conceptualization of the study, supervised the machine learning model development, and participated in the interpretation of the results. AS and MC participated in the machine learning model development and helped in the interpretation of the results. BL participated in the interpretation of results, helped in the literature review, and critically reviewed the manuscript. WS contributed to data collection and critically reviewed the manuscript. AKos made a major contribution to the conceptualization of the study and supervised data collection and study execution. BM provided the institutional background of the research, supervised the study execution, and thoroughly reviewed the manuscript. All authors read and approved the final version of the manuscript.

Funding

This work was supported by the National Research, Development, and Innovation Office of Hungary (NKFIA; NVKP_16-1-2016-0017—National Heart Program), the New National Excellence Program (ÚNKP-19-3-I-SE-24), the Artificial Intelligence National Laboratory Program, and the Thematic Excellence Program (2020-4.1.1.-TKP2020) of the Ministry for Innovation and Technology in Hungary, within the framework of the Therapeutic Development and Bioimaging programs of the Semmelweis University. AKov was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences.

Conflict of Interest

BM receives lecture fees from Biotronik, Medtronic, and Abbott. ZT is a co-founder and CEO of Argus Cognitive, Inc., holds equity in the company, and receives financial compensation for his work. AS and MC are employees of Argus Cognitive, Inc., and receive compensation for their work.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2021.611055/full#supplementary-material

References

1. Lloyd-Jones DM, Larson MG, Leip EP, Beiser A, D'Agostino RB, Kannel WB, et al. Lifetime risk for developing congestive heart failure: the Framingham Heart Study. Circulation. (2002) 106:3068–72. doi: 10.1161/01.CIR.0000039105.49749.6F

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Bleumink GS, Knetsch AM, Sturkenboom MC, Straus SM, Hofman A, Deckers JW, et al. Quantifying the heart failure epidemic: prevalence, incidence rate, lifetime risk and prognosis of heart failure The Rotterdam Study. Eur Heart J. (2004) 25:1614–9. doi: 10.1016/j.ehj.2004.06.038

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Stolfo D, Uijl A, Vedin O, Stromberg A, Faxen UL, Rosano GMC, et al. Sex-based differences in heart failure across the ejection fraction spectrum: phenotyping, and prognostic and therapeutic implications. JACC Heart Fail. (2019) 7:505–15. doi: 10.1016/j.jchf.2019.03.011

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Lam CSP, Arnott C, Beale AL, Chandramouli C, Hilfiker-Kleiner D, Kaye DM, et al. Sex differences in heart failure. Eur Heart J. (2019) 40:3859–68c. doi: 10.1093/eurheartj/ehz835

CrossRef Full Text | Google Scholar

5. Tahhan AS, Vaduganathan M, Greene SJ, Fonarow GC, Fiuzat M, Jessup M, et al. Enrollment of older patients, women, and racial and ethnic minorities in contemporary heart failure clinical trials: a systematic review. JAMA Cardiol. (2018) 3:1011–9. doi: 10.1001/jamacardio.2018.2559

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Dewan P, Rorth R, Jhund PS, Shen L, Raparelli V, Petrie MC, et al. Differential impact of heart failure with reduced ejection fraction on men and women. J Am Coll Cardiol. (2019) 73:29–40. doi: 10.1016/j.jacc.2018.09.081

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sridhar AR, Yarlagadda V, Parasa S, Reddy YM, Patel D, Lakkireddy D, et al. Cardiac resynchronization therapy: US trends and disparities in utilization and outcomes. Circ Arrhythm Electrophysiol. (2016) 9:e003108. doi: 10.1161/CIRCEP.115.003108

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lund LH, Braunschweig F, Benson L, Stahlberg M, Dahlstrom U, Linde C. Association between demographic, organizational, clinical, and socio-economic characteristics and underutilization of cardiac resynchronization therapy: results from the Swedish Heart Failure Registry. Eur J Heart Fail. (2017) 19:1270–9. doi: 10.1002/ejhf.781

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Chatterjee NA, Borgquist R, Chang Y, Lewey J, Jackson VA, Singh JP, et al. Increasing sex differences in the use of cardiac resynchronization therapy with or without implantable cardioverter-defibrillator. Eur Heart J. (2017) 38:1485–94. doi: 10.1093/eurheartj/ehw598

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Arshad A, Moss AJ, Foster E, Padeletti L, Barsheshet A, Goldenberg I, et al. Cardiac resynchronization therapy is more effective in women than in men: the MADIT-CRT (multicenter automatic defibrillator implantation trial with cardiac resynchronization therapy) trial. J Am Coll Cardiol. (2011) 57:813–20. doi: 10.1016/j.jacc.2010.06.061

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Zusterzeel R, Spatz ES, Curtis JP, Sanders WE, Selzman KA, Pina IL, et al. Cardiac resynchronization therapy in women versus men: observational comparative effectiveness study from the National Cardiovascular Data Registry. Circ Cardiovasc Qual Outcomes. (2015) 8(Suppl. 1):S4–11. doi: 10.1161/CIRCOUTCOMES.114.001548

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Varma N, Mittal S, Prillinger JB, Snell J, Dalal N, Piccini JP. Survival in women versus men following implantation of pacemakers, defibrillators, and cardiac resynchronization therapy devices in a large, nationwide cohort. J Am Heart Assoc. (2017) 6:5031. doi: 10.1161/JAHA.116.005031

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Yin FH, Fan CL, Guo YY, Zhu H, Wang ZL. The impact of gender difference on clinical and echocardiographic outcomes in patients with heart failure after cardiac resynchronization therapy: a systematic review and meta-analysis. PLoS ONE. (2017) 12:e0176248. doi: 10.1371/journal.pone.0176248

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Hsich EM. Sex differences in advanced heart failure therapies. Circulation. (2019) 139:1080–93. doi: 10.1161/CIRCULATIONAHA.118.037369

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bzdok D, Krzywinski M, Altman N. Machine learning: a primer. Nat Methods. (2017) 14:1119–20. doi: 10.1038/nmeth.4526

CrossRef Full Text | Google Scholar

16. Mortazavi BJ, Downing NS, Bucholz EM, Dharmarajan K, Manhapra A, Li SX, et al. Analysis of machine learning techniques for heart failure readmissions. Circ Cardiovasc Qual Outcomes. (2016) 9:629–40. doi: 10.1161/CIRCOUTCOMES.116.003039

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Angraal S, Mortazavi BJ, Gupta A, Khera R, Ahmad T, Desai NR, et al. Machine learning prediction of mortality and hospitalization in heart failure with preserved ejection fraction. JACC Heart Fail. (2019) 8:12–21. doi: 10.1016/j.jchf.2019.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Al'Aref SJ, Singh G, van Rosendael AR, Kolli KK, Ma X, Maliakal G, et al. Determinants of in-hospital mortality after percutaneous coronary intervention: a machine learning approach. J Am Heart Assoc. (2019) 8:e011160. doi: 10.1161/JAHA.118.011160

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Adler ED, Voors AA, Klein L, Macheret F, Braun OO, Urey MA, et al. Improving risk prediction in heart failure using machine learning. Eur J Heart Fail. (2019) 22:139–47. doi: 10.1002/ejhf.1628

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Tokodi M, Shrestha S, Bianco C, Kagiyama N, Casaclang-Verzosa G, Narula J, et al. Interpatient similarities in cardiac function: a platform for personalized cardiovascular medicine. JACC Cardiovasc Imaging. (2020) 13:1119-32. doi: 10.1016/j.jcmg.2019.12.018

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Obermeyer Z, Emanuel EJ. Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med. (2016) 375:1216–9. doi: 10.1056/NEJMp1606181

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Ouwerkerk W, Voors AA, Zwinderman AH. Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure. JACC Heart Fail. (2014) 2:429–36. doi: 10.1016/j.jchf.2014.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Canepa M, Fonseca C, Chioncel O, Laroche C, Crespo-Leiro MG, Coats AJS, et al. Performance of prognostic risk scores in chronic heart failure patients enrolled in the European society of cardiology heart failure long-term registry. JACC Heart Fail. (2018) 6:452–62. doi: 10.1016/j.jchf.2018.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tokodi M, Schwertner WR, Kovacs A, Toser Z, Staub L, Sarkany A, et al. Machine learning-based mortality prediction of patients undergoing cardiac resynchronization therapy: the SEMMELWEIS-CRT score. Eur Heart J. (2020) 41:1747–56. doi: 10.1093/eurheartj/ehz902

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Feeny AK, Rickard J, Patel D, Toro S, Trulock KM, Park CJ, et al. Machine learning prediction of response to cardiac resynchronization therapy. Circ Arrhythm Electrophysiol. (2019) 12:e007316. doi: 10.1161/CIRCEP.119.007316

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kalscheur MM, Kipp RT, Tattersall MC, Mei C, Buhr KA, DeMets DL, et al. Machine learning algorithm predicts cardiac resynchronization therapy outcomes: lessons from the COMPANION trial. Circ Arrhythm Electrophysiol. (2018) 11:e005499. doi: 10.1161/CIRCEP.117.005499

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Banerjee M, Reynolds E, Andersson HB, Nallamothu BK. Tree-based analysis. Circ Cardiovas Qual Outcomes. (2019) 12:e004879. doi: 10.1161/CIRCOUTCOMES.118.004879

CrossRef Full Text | Google Scholar

28. de Waard D, Manlucu J, Gillis AM, Sapp J, Bernick J, Doucette S, et al. Cardiac resynchronization in women: a substudy of the resynchronization-defibrillation for ambulatory heart failure trial. JACC Clin Electrophysiol. (2019) 5:1036–44. doi: 10.1016/j.jacep.2019.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Schuchert A, Muto C, Maounis T, Frank R, Ella RO, Polauck A, et al. Gender-related safety and efficacy of cardiac resynchronization therapy. Clin Cardiol. (2013) 36:683–90. doi: 10.1002/clc.22203

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Beela AS, Duchenne J, Petrescu A, Ünlü S, Penicka M, Aakhus S, et al. Sex-specific difference in outcome after cardiac resynchronization therapy. Eur Heart J Cardiovasc Imaging. (2019) 20:504–11. doi: 10.1093/ehjci/jey231

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Varma N, Lappe J, He J, Niebauer M, Manne M, Tchou P. Sex-specific response to cardiac resynchronization therapy: effect of left ventricular size and QRS duration in left bundle branch block. JACC Clin Electrophysiol. (2017) 3:844–53. doi: 10.1016/j.jacep.2017.02.021

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Linde C, Cleland JGF, Gold MR, Claude Daubert J, Tang ASL, Young JB, et al. The interaction of sex, height, and QRS duration on the effects of cardiac resynchronization therapy on morbidity and mortality: an individual-patient data meta-analysis. Eur J Heart Fail. (2018) 20:780–91. doi: 10.1002/ejhf.1133

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Tamargo J, Rosano G, Walther T, Duarte J, Niessner A, Kaski J, et al. Gender differences in the effects of cardiovascular drugs. Eur Heart J Cardiovasc Pharmacother. (2017) 3:163–82. doi: 10.1093/ehjcvp/pvw042

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Zusterzeel R, Curtis JP, Caños DA, Sanders WE, Selzman KA, Piña IL, et al. Sex-specific mortality risk by QRS morphology and duration in patients receiving CRT: results from the NCDR. J Am Coll Cardiol. (2014) 64:887–94. doi: 10.1016/j.jacc.2014.06.1162

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Varma N, Manne M, Nguyen D, He J, Niebauer M, Tchou P. Probability and magnitude of response to cardiac resynchronization therapy according to QRS duration and gender in nonischemic cardiomyopathy and LBBB. Heart Rhythm. (2014) 11:1139–47. doi: 10.1016/j.hrthm.2014.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Loring Z, Caños DA, Selzman K, Herz ND, Silverman H, MaCurdy TE, et al. Left bundle branch block predicts better survival in women than men receiving cardiac resynchronization therapy: long-term follow-up of ~ 145,000 patients. JACC Heart Fail. (2013) 1:237–44. doi: 10.1016/j.jchf.2013.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Biton Y, Zareba W, Goldenberg I, Klein H, McNitt S, Polonsky B, et al. Sex differences in long-term outcomes with cardiac resynchronization therapy in mild heart failure patients with left bundle branch block. J Am Heart Assoc. (2015) 4:e002013. doi: 10.1161/JAHA.115.002013

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Strauss DG, Selvester RH, Wagner GS. Defining left bundle branch block in the era of cardiac resynchronization therapy. Am J Cardiol. (2011) 107:927–34. doi: 10.1016/j.amjcard.2010.11.010

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Linde C, Ståhlberg M, Benson L, Braunschweig F, Edner M, Dahlström U, et al. Gender, underutilization of cardiac resynchronization therapy, and prognostic impact of QRS prolongation and left bundle branch block in heart failure. EP Europace. (2014) 17:424–31. doi: 10.1093/europace/euu205

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Kloosterman M, Maass AH. Sex differences in optimal atrioventricular delay in patients receiving cardiac resynchronization therapy. Clin Res Cardiol. (2020) 109:124–7. doi: 10.1007/s00392-019-01492-0

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Cheng A, Gold MR, Waggoner AD, Meyer TE, Seth M, Rapkin J, et al. Potential mechanisms underlying the effect of gender on response to cardiac resynchronization therapy: insights from the SMART-AV multicenter trial. Heart Rhythm. (2012) 9:736–41. doi: 10.1016/j.hrthm.2011.12.013

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Xu YZ, Friedman PA, Webster T, Brooke K, Hodge DO, Wiste HJ, et al. Cardiac resynchronization therapy: do women benefit more than men? J Cardiovasc Electrophysiol. (2012) 23:172–8. doi: 10.1111/j.1540-8167.2011.02168.x

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Ousdigian KT, Borek PP, Koehler JL, Heywood JT, Ziegler PD, Wilkoff BL. The epidemic of inadequate biventricular pacing in patients with persistent or permanent atrial fibrillation and its association with mortality. Circ Arrhythmia Electrophysiol. (2014) 7:370–6. doi: 10.1161/CIRCEP.113.001212

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Auricchio A, Gasparini M, Linde C, Dobreanu D, Cano Ó, Sterlinski M, et al. Sex-related procedural aspects and complications in CRT survey II: a multicenter European experience in 11,088 patients. JACC Clin Electrophysiol. (2019) 5:1048–58. doi: 10.1016/j.jacep.2019.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Hayes DL, Boehmer JP, Day JD, Gilliam FR, 3rd, Heidenreich PA, Seth M, et al. Cardiac resynchronization therapy and the relationship of percent biventricular pacing to symptoms and survival. Heart Rhythm. (2011) 8:1469–75. doi: 10.1016/j.hrthm.2011.04.015

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Martínez-Sellés M, Doughty RN, Poppe K, Whalley GA, Earle N, Tribouilloy C, et al. Gender and survival in patients with heart failure: interactions with diabetes and aetiology. Results from the MAGGIC individual patient meta-analysis. Eur J Heart Fail. (2012) 14:473–9. doi: 10.1093/eurjhf/hfs026

PubMed Abstract | CrossRef Full Text

47. Ghali JK, Krause-Steinrauf HJ, Adams KF, Khan SS, Rosenberg YD, Yancy CW, et al. Gender differences in advanced heart failure: insights from the BEST study. J Am Coll Cardiol. (2003) 42:2128–34. doi: 10.1016/j.jacc.2003.05.012

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Chandramouli C, Teng TK, Tay WT, Yap J, MacDonald MR, Tromp J, et al. Impact of diabetes and sex in heart failure with reduced ejection fraction patients from the ASIAN-HF registry. Eur J Heart Fail. (2019) 21:297–307. doi: 10.1002/ejhf.1358

PubMed Abstract | CrossRef Full Text | Google Scholar