A machine learning evaluation of patient characteristics associated with prescribing of guideline-directed medical therapy for heart failure

Abstract

Introduction/background:

Patients with heart failure and reduced ejection fraction (HFrEF) are consistently underprescribed guideline-directed medications. Although many barriers to prescribing are known, identification of these barriers has relied on traditional a priori hypotheses or qualitative methods. Machine learning can overcome many limitations of traditional methods to capture complex relationships in data and lead to a more comprehensive understanding of the underpinnings driving underprescribing. Here, we used machine learning methods and routinely available electronic health record data to identify predictors of prescribing.

Methods:

We evaluated the predictive performance of machine learning algorithms to predict prescription of four types of medications for adults with HFrEF: angiotensin converting enzyme inhibitor/angiotensin receptor blocker (ACE/ARB), angiotensin receptor-neprilysin inhibitor (ARNI), evidence-based beta blocker (BB), or mineralocorticoid receptor antagonist (MRA). The models with the best predictive performance were used to identify the top 20 characteristics associated with prescribing each medication type. Shapley values were used to provide insight into the importance and direction of the predictor relationships with medication prescribing.

Results:

For 3,832 patients meeting the inclusion criteria, 70% were prescribed an ACE/ARB, 8% an ARNI, 75% a BB, and 40% an MRA. The best-predicting model for each medication type was a random forest (area under the curve: 0.788–0.821; Brier score: 0.063–0.185). Across all medications, top predictors of prescribing included prescription of other evidence-based medications and younger age. Unique to prescribing an ARNI, the top predictors included lack of diagnoses of chronic kidney disease, chronic obstructive pulmonary disease, or hypotension, as well as being in a relationship, nontobacco use, and alcohol use.

Discussion/conclusions:

We identified multiple predictors of prescribing for HFrEF medications that are being used to strategically design interventions to address barriers to prescribing and to inform further investigations. The machine learning approach used in this study to identify predictors of suboptimal prescribing can also be used by other health systems to identify and address locally relevant gaps and solutions to prescribing.

1. Introduction

Most patients with HFrEF have significant gaps in the prescribing of evidence-based medications (1–8). In the absence of intolerance or contraindications, many barriers to prescribing these medications have been described. Barriers include clinical inertia and patient characteristics such as certain thresholds for vitals or laboratory values, e.g., blood pressure and potassium (5, 8–11).

Given the large potential benefits of treatment (1–8), it is critical to identify all factors associated with underprescribing. Known barriers to HFrEF prescribing have been identified using traditional quantitative and qualitative methods. However, traditional quantitative methods require an a priori signal to evaluate, while qualitative methods require clinician awareness of factors that may influence their prescribing behaviors. These traditional methods are also limited in their ability to comprehensively consider the complexity of factors that can dynamically influence prescribing, leaving the possibility of some barriers being undiscovered. A comprehensive understanding of the underpinnings driving underprescribing can lead to the strategic design of interventions to address barriers (12).

Machine learning (ML) methods are a hypothesis-free means to identify factors that influence prescribing for HFrEF when the relationships between predictors and the outcome is unknown. ML may offer advantages over traditional approaches when applied to healthcare data, in particular mitigating the need to prespecify variable relationships and handling complex interactions and nonlinear variable effects (13–16). Applying such approaches to real-world electronic health record (EHR) data also facilitates a pragmatic evaluation of predictors in a real-world clinical context where such interactions and nonlinear variable effects are important to account for.

In this single-center study, we used ML and regression-based approaches to identify important predictors of prescribing evidence-based medications for HFrEF that are routinely available within the EHR. We compared penalized regression, decision trees, and random forests for predicting prescribing of evidence-based medications for HFrEF. For the best-performing model, we ascertained the most important predictors of prescribing that could then be used by the health system to design interventions or inform areas in need of further investigation.

2. Materials and methods

2.1. Study population

This study was conducted at the UCHealth system, which includes three large regional centers (North, Metro, and South) comprised of 12 hospitals, 900 clinics, and over 6,000 physicians who serve academic, rural, suburban, and community settings. The entire system is serviced by a single instance of the Epic EHR software program (Epic Systems, Verona, WI, United States). We collected data on eligible patients from Health Data Compass, our institutional enterprise data warehouse that extracts, integrates, and delivers data from the EHR and medical/medication claims data for patients within the UCHealth system. This protocol was approved by the University of Colorado Multiple Institutional Review Board using deidentified and uniquely encoded datasets, with waiver of informed consent.

Patients included non-deceased adults (age ≥18 years) with prevalent HFrEF documented at any patient visit within the health system on or before December 11, 2018. HFrEF was defined as a most recent ejection fraction value ≤40%. Patients were excluded if they did not have at least one outpatient visit with cardiology or primary care between December 11, 2016, and December 11, 2018. These patients were excluded to maximize representation of patients receiving longitudinal care at UCHealth.

2.2. Primary outcome

The outcomes of interest were four types of medications used to treat HFrEF: angiotensin converting enzyme inhibitor/angiotensin receptor blocker (ACE/ARB), angiotensin receptor-neprilysin inhibitor (ARNI), evidence-based beta blocker (BB), or mineralocorticoid receptor antagonists (MRA). Evidence-based BBs included metoprolol succinate, bisoprolol, and carvedilol. A patient was determined to be prescribed a given medication type (yes/no) if at least one medication in that category was ordered or dispensed between December 11, 2017, and December 11, 2019 (latest date of data available). The medication did not need to be prescribed after a patient met eligibility criteria, given that the medications of interest can be used for other indications. This time period was chosen (1) due to limited access to medication dispensing data [via SureScripts (17, 26) integration with the EHR]; (2) to account for the possibility of prescribing a medication with refills that would cover up to a 1-year supply on or before December 11, 2018 (date of study eligibility); and (3) to allow patients with a new diagnosis as of the study eligibility date sufficient time to safely be initiated on the medications (within 1-year post study eligibility). When dispensing data was available through SureScripts, dispenses were evaluated to account for medication orders initiated outside of the health system's EHR. A patient's index date associated with a particular medication category is defined as the most recent date that medication was prescribed on or before December 11, 2019. The index date for those who were not prescribed a given medication was defined as the most recent office visit on or before December 11, 2019.

2.3. Clinical predictors

Candidate variables were determined by study investigators representing advanced heart failure specialists, general cardiology, electrophysiology, clinical pharmacy, and epidemiology. A total of 126 candidate predictor variables were selected based on availability in structured fields in the EHR and because they are either generally recognized as barriers to prescribing or were determined to be novel predictors to explore. See Supplementary Appendix S1 for a detailed list of candidate variables, which includes patient demographics, social history, insurance type, vitals, diagnoses, procedures, medications, and laboratory values. In some cases, missing values were considered as candidate variables (e.g., missing social security number). The number of unique medications was defined based on 10-digit Generic Product Identifier (GPI) codes that specify the active ingredient including the base formulation. The timing of measuring candidate predictor variables was based on the patient-specific index dates for each medication type (not baseline date of study eligibility) and all available data up to the index date. Baseline characteristics were collected on or before December 11, 2018 (date of study eligibility). Values for predictors that were outside of possible physiologic ranges (e.g., ejection fraction of −99.6) were considered to be erroneous and set to be missing.

2.4. Statistical analysis

Patient characteristics were summarized overall and for each medication type. Continuous variables were normalized prior to inclusion in model training. Missing variables were imputed using random forest imputation (18). Four separate analyses were performed to identify top predictors associated with each of the medication outcomes of interest. We trained and evaluated the predictive performance of several predictive models, specifically decision tree, random forest, and logistic regression with lasso, ridge, and elastic net regularization (19–22). Figure 1 provides an overview of the model-building and training process. We performed a 100-fold Monte-Carlo cross-validation using an 80% training and 20% test split. Hyperparameter optimization was performed using a grid search within 5-fold cross-validation on the training set. A manual grid optimization approach was used to ensure that the grid contained the optimal hyperparameters (e.g., if a hyperparameter value was identified on the upper end of the grid range, the grid was expanded to ensure that the overall optimal hyperparameter was not beyond the bounds of the grid space). For outcomes with severe class imbalance (i.e., outcome prevalence of <10%), we oversampled the minority class using the data augmentation method of synthetic minority oversampling (23, 24) in the training set.

Figure 1

We compared the performance of the different predictive models in the test data set. Predictive performance was primarily assessed using area under the receiver operating characteristic curve (AUC), which is a measure of discrimination (i.e., ability to distinguish between high and low probability of prescribing). We compared the differences between the AUCs from the different methods using the DeLong test (25). We additionally report the Brier score, which is a measure of overall performance (26). In this framework, each feature is assigned an importance value for each observation. Shapley Additive Explanation (SHAP) values were reported for the best-performing predictive model to identify the top predictors for prescribing each medication type (27). The mean SHAP value across all observations indicates how much each predictor contributes to the target variable and in what direction. The purpose of developing an ML/regression model is to identify the top predictors of prescribing that jointly predict the outcome. This model is not developed for prospective use for obtaining predictions in clinical practice. The process of identifying the best-performing model based on predictive performance was used to select a model from which to describe the top predictors for prescribing used by that model. A similar approach has been used in other settings to identify clinical predictors of an outcome (28–30).

All analyses were performed in Python 3.7.5 using the packages imblearn 0.0, numpy 1.18.1, pandas 1.1.2, scikit-learn 0.24.2, researchpy 0.3.2, and shap 0.40.0.

3. Results

3.1. Patient characteristics

A total of 3,832 individuals met the inclusion criteria. The median age of the population was 69 years, 30% were female, and 87.7% were non-Hispanic. Table 1 describes the baseline characteristics of these individuals overall and stratified by medication type. Of these, 69.6% were prescribed an ACE/ARB, 7.5% an ARNI, 74.6% a BB, 39.7% an MRA, and 16.7% were prescribed none of these medications.

3.2. Model performance

Table 2 presents the performance metrics of the different models for each medication type.

The random forest was the best-performing model across all medication categories: ACE/ARB (AUC = 0.791, Brier score = 0.158), ARNI (AUC = 0.821, Brier score = 0.063), evidence-based BB (AUC = 0.797, Brier score = 0.140), and MRA (AUC = 0.788, Brier score = 0.185). For each of the medication categories, by Delong's test, there were not statistically significant differences at the 0.05 level between the AUCs for the random forest, elastic net, lasso, and ridge regression methods, although the best-performing models (random forests) consistently had significantly higher AUCs than the decision trees.

3.3. Top features for predicting prescribing

Figure 2 illustrates the top 20 features of importance (limited for visual clarity) for each medication outcome sorted by the mean absolute SHAP value. See Supplementary Appendix S2 for all features of importance for each of the medication outcomes.

Figure 2

3.4. Top predictors by medication outcome

Among the top 20 predictors for ACE/ARB prescribing were higher values of potassium and estimated glomerular filtration rate (eGFR) and lower values of serum creatinine and BUN. For prescription of ARNI, some of the strongest predictors were younger age and implantable cardioverter defibrillator (ICD). Other predictors among the top 20 for ARNI included being in a relationship (not single), absence of tobacco use, and no diagnosis of chronic kidney disease (CKD), chronic obstructive pulmonary disease (COPD), or hypotension. Among the top 20 predictors for evidence-based BB prescribing were lower values of heart rate, systolic blood pressure, and glucose values. Among the strongest predictors for MRA prescribing were younger age and lower systolic blood pressure values. Other predictors among the top 20 for MRA included lower values of diastolic blood pressure and serum creatinine.

3.5. Top predictors across all medication outcomes

Across all medication outcomes, prescribing of at least two of the other three medication types of interest was consistent among the top five predictors. All three other medications of interest (ACE/ARB, evidence-based BB, and MRA) were among the top five predictors for prescribing ARNI. Prescription of an ARNI was not among the top 10 predictors of prescribing an ACE/ARB or MRA, nor among the top 20 predictors of prescribing an evidence-based BB. For all medication outcomes, prescription of a loop diuretic and a statin were among the top 20 predictors. With the exception of evidence-based BB, younger age was among the top 10 predictors for all medication outcomes. Greater number of medications and lower ejection fraction values were also among the top 20 predictors for all medication outcomes, except ARNI.

4. Discussion

We applied ML methods to data readily available within most EHRs to identify predictors of prescribing evidence-based medications for HFrEF at our health system. We found that several ML methods performed similarly well for predicting prescribing of guideline-directed medical therapy (GDMT). From the random forest models, we identified multiple predictors of prescribing ACE/ARB, ARNI, evidence-based BB, and MRAs that can be used to guide the development of targeted strategies to improve prescribing for HFrEF and inform hypotheses for future study. Our application of machine learning approaches and the use of SHAP to identify predictors of prescribing are unique and strengths of this study. By exploring the use of ML algorithms to build predictive models for prescribing, we take advantage of their ability to account for possible interactions, collinearity, and nonlinear effects that commonly exist in complex healthcare data (40–43). By using SHAP, a model-agnostic approach to identify feature importance, we were able to quantify the directionality and importance of the relationship between the predictors and medication outcomes. SHAP additionally has an implementation to explain the modeling of local interaction and nonlinear predictor effects (31, 32), which can be explored in future studies to further investigate predictor behavior. Another strength of our study is the use of real-world EHR data, in contrast to data sources that are narrowly focused on a specific issue or setting or that rely on data generated from tightly controlled clinical trials (43, 33). We also included diverse social history data available within EHRs (e.g., substance use, primary language, relationship status), which facilitates the evaluation of more holistic factors influencing prescribing decisions. The use of EHR data allowed for evaluation of a large set of routinely collected data to detect predictor variables. In this setting, we found that a penalized logistic regression performed similarly to the best-performing ML approach; however, our approach for model-building and model-agnostic explanation allows for the evaluation of ML algorithms that may have superior predictive performance when applied to future explorations of complex EHR data. This model-building approach to defining predictors of prescribing is actively being used by our health system to strategically design interventions to overcome barriers to evidence-based prescribing. As the context and evidence change, this approach to using ML to define predictors of prescribing can be repeated to identify new issues and design corresponding solutions for them.

Some predictors of prescribing that we identified align with findings from past studies, which serves to triangulate our findings. For example, worse renal function is a well-known barrier to prescribing ACE/ARB, ARNI, and MRA (34–36). In our study, lower values of serum creatinine, which suggests better renal function, was among the top 20 predictors of prescribing for ACE/ARB and MRA. Although higher values of serum creatinine was among the top 20 predictors for ARNI, the absence of a CKD diagnosis was also one. Patients prescribed an ARNI may have higher serum creatinine values within the range of normal renal function. The finding of better renal function as a predictor of prescribing confirms the presence of this modifiable knowledge issue, for which our health system is actively designing interventions, such as tailored clinical decision support (CDS) tools and education to address.

Past studies also point to hyperkalemia or concerns of precipitating hyperkalemia as common barriers to prescribing ACE/ARB, ARNI, and MRA (37). Interestingly, in our study, the absence of a hyperkalemia diagnosis was not among the top 20 predictors for prescribing these medications, but potassium supplementation was for ACE/ARB, ARNI, and MRA. Furthermore, higher values of serum potassium was a top predictor for ACE/ARB. These discrepancies between our findings and past studies may suggest that concerns related to hyperkalemia are not as influential of a predictor after accounting for the complex interacting effects of other key predictors. Additional studies are needed to evaluate the generalizability of this finding in other health systems. Based on these findings, our health system is not currently prioritizing resources to design interventions to address any modifiable informational needs related to hyperkalemia.

Another common barrier to prescribing these medications are concerns related to hypotension across all medications and bradycardia for evidence-based BBs (38–40). This aligns well with our findings of lower blood pressures and heart rates being among the top predictors for prescribing evidence-based BBs (heart rate and blood pressure) and MRAs (blood pressure only). However, blood pressure was not among the top predictors of prescribing an ACE/ARB or ARNI. The absence of blood pressure as a top predictor for ACE/ARB or ARNI may suggest that clinicians are less concerned about hypotension with these specific medications, but requires further investigation to understand these findings within our health system and externally. In the case of ACE/ARBs specifically, there is a large therapeutic dose range and it is possible that the absence of blood pressure as a predictor is because lower doses are prescribed to these patients to mitigate clinician concerns.

There were also some notable findings when comparing predictors of prescribing ARNI to the other medication outcomes. We found that patients prescribed an ARNI were more likely to be in a relationship (not single) and be nontobacco users, which may suggest a stronger support system and choice of healthier lifestyle choices. A study conducted in Sweden also found that patients in a relationship were more likely to be prescribed ARNI (41). In our study, the absence of CKD, COPD, and hypotension were among the top predictors, while greater number of medications was not among the top predictors (it was the 70th most important predictor; see Supplementary Appendix B), which further supports the notion that these patients were healthier. Younger age was also the third strongest predictor of ARNI prescription. Other studies have also associated younger age (41–44) and fewer comorbidities (43) with ARNI prescription. Although we found the absence of a CKD diagnosis was a predictor of ARNI, other studies report conflicting findings regarding the association with CKD (42–44), which may be related to differences in the types (inpatient, outpatient, EHR, claims), volume, or variety of data used. At our institution, we are continuing to explore these findings and any potential solutions by conducting qualitative interviews.

The higher relative cost of ARNI (45) may, in part, explain some of the predictors uniquely associated with ARNI prescription in our study. Younger patients with greater support systems who have historically made healthier lifestyle choices (tobacco abstinence) may be more likely to choose to invest in more expensive medications. It is also possible that there is implicit or explicit clinician bias that results in clinicians being more likely to selectively recommend an ARNI to this demographic. Others have found that patients of higher socioeconomic status are more likely to be prescribed an ARNI (42). In our study, we did not find commercial insurance, race, or ethnicity predictive of ARNI, but others have found that non-Hispanic patients are more likely to be prescribed an ARNI (42).

Similar to other studies, patients prescribed an ARNI were more likely to be prescribed other guideline-concordant therapies (42–44); the three other evidence-based medications and an ICD were some of the strongest predictors. Although not a new finding (42–44), the fact that we were able to reproduce this using ML, which more rigorously considers the complex interactions of diverse variables, supports the validity of our other findings. ARNIs are more likely to be prescribed by cardiologists who are more focused on optimizing management of HFrEF and who may be more familiar with the evidence and comfortable applying it than other providers (42, 46).

This study does have several limitations. EHR data often do not include some important variables, such as social determinants of health or patient-reported outcomes, which are known to influence patient care (42, 47). Our study also does not allow for evaluation or interpretation of temporal relationships between our predictor variables and medication outcomes. For example, we were limited to dates of prescriptions and unable to identify which medications a patient was taking or actively prescribed at a specific point in time, which is an inherent limitation of EHR data in nonintegrated health systems. As such, ACE/ARB as a predictor of prescribing ARNI should be interpreted as prescription of an ACE/ARB in the past year is associated with prescribing ARNI. Furthermore, given the inherent limitations in accurately measuring medication doses with EHR data, we did not evaluate doses of target medications or duration of medication use, but these are important areas for future inquiry. We also did not exclude absolute contraindications to the medication outcomes of interest; however, such contraindications are infrequent, and thus, this is unlikely to have significant impact our findings.

Although our model has not been validated in an external data set, this approach is highly relevant for health systems seeking to improve prescribing for this high risk population. Using this approach in our health system, we identified salient predictors of suboptimal prescribing that will be used to strategically design tailored interventions to address barriers to prescribing evidence-based medications for HFrEF, notably educational initiatives and CDS embedded within the EHR. For some predictors, such as a patient's relationship status, qualitative evaluations are needed to shed light on why these predictors influence prescribing before tailored interventions can be implemented. Although not all predictors can be tangibly addressed (relationship status), many are modifiable (e.g., clinician misconceptions or knowledge gaps). It is possible that through additional qualitative exploration, we identify that certain predictors such as relationship status have underlying drivers that are modifiable. Future research could also explore subgroup analyses for certain patient groups to better understand why they were not prescribed evidence-based medications. Given the persistence of nonadherence to evidence-based prescribing over time despite many concerted efforts to close the gaps, reimagined and multimodal interventions may be required to significantly move the needle. This study can assist in identifying modifiable barriers (e.g., salient knowledge gaps or misconceptions) to design targeted interventions for HFrEF. Types of interventions may include traditional modes of education to address knowledge gaps, direct to patient activation strategies (18), or integration of emerging technologies and artificial intelligence to translate evidence-based recommendations into clinician workflows at the right time (48). For example, perhaps patient-facing CDS tools delivered via EHR patient portals in combination with clinician-facing CDS tools and audit and feedback with peer-benchmarking are needed to improve the representativeness of prescribing ARNIs to patients of diverse backgrounds. In this example, CDS tools empower patients and remind or guide clinicians, while clinicians are also held accountable through social norming.

References

• 1.

  AllenLATangFJonesPBreedingTPonirakisATurnerSJ. Signs, symptoms, and treatment patterns across serial ambulatory cardiology visits in patients with heart failure: insights from the NCDR PINNACLE® registry. BMC Cardiovasc Disord. (2018) 18(1):80. 10.1186/s12872-018-0808-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2.

  GreeneSJButlerJAlbertNMDeVoreADSharmaPPDuffyCIet alMedical therapy for heart failure with reduced ejection fraction: the CHAMP-HF registry. J Am Coll Cardiol. (2018) 72(4):351–66. 10.1016/j.jacc.2018.04.070

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3.

  TranRHAldemerdashAChangPSuetaCAKaufmanBAsafu-AdjeiJet alGuideline-directed medical therapy and survival following hospitalization in patients with heart failure. Pharmacotherapy. (2018) 38(4):406–16. 10.1002/phar.2091

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  Smith KVDunningJRFischerCMMacLeanTEBosque-HamiltonJWFeraLEet alEvaluation of the usage and dosing of guideline-directed medical therapy for heart failure with reduced ejection fraction patients in clinical practice. J Pharm Pract. (2021) 35(5):747–51. 10.1177/08971900211004840

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  CowieMRSchöpeJWagenpfeilSTavazziLBöhmMPonikowskiPet alPatient factors associated with titration of medical therapy in patients with heart failure with reduced ejection fraction: data from the QUALIFY international registry. ESC Heart Fail. (2021) 8(2):861–71. 10.1002/ehf2.13237

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  AllenLAVenechukGMcIlvennanCKPageRL2ndKnoepkeCEHelmkampLJet alAn electronically delivered, patient-activation tool for intensification of medications for chronic heart failure with reduced ejection fraction: the EPIC-HF trial. Circulation. (2021) 143(5):427–37. 10.1161/circulationaha.120.051863

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  SrivastavaPKDeVoreADHellkampASThomasLAlbertNMButlerJet alHeart failure hospitalization and guideline-directed prescribing patterns among heart failure with reduced ejection fraction patients. JACC Heart Fail. (2021) 9(1):28–38. 10.1016/j.jchf.2020.08.017

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8.

  JarjourMHenriCde DenusSFortierABouabdallaouiNNigamAet alCare gaps in adherence to heart failure guidelines: clinical inertia or physiological limitations?JACC Heart Fail. (2020) 8(9):725–38. 10.1016/j.jchf.2020.04.019

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9.

  FürthauerJFlammMSönnichsenA. Patient and physician related factors of adherence to evidence based guidelines in diabetes mellitus type 2, cardiovascular disease and prevention: a cross sectional study. BMC Fam Pract. (2013) 14(1):47. 10.1186/1471-2296-14-47

  • CrossRef
  • Google Scholar

• 10.

  SteinmanMAPatilSKamatPPetersonCKnightSJ. A taxonomy of reasons for not prescribing guideline-recommended medications for patients with heart failure. Am J Geriatr Pharmacother. (2010) 8(6):583–94. 10.1016/S1543-5946(10)80007-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11.

  SteinmanMADimaanoLPetersonCAHeidenreichPAKnightSJFungKZet alReasons for not prescribing guideline-recommended medications to adults with heart failure. Med Care. (2013) 51(10):901–7. 10.1097/MLR.0b013e3182a3e525

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12.

  GreeneSJTanXYehYCBernauerMZaidiOYangMet alFactors associated with non-use and sub-target dosing of medical therapy for heart failure with reduced ejection fraction. Heart Fail Rev. (2022) 27(3):741–53. 10.1007/s10741-021-10077-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13.

  Lo-CiganicWHHuangJLZhangHHWeissJCWuYKwohCKet alEvaluation of machine-learning algorithms for predicting opioid overdose risk among Medicare beneficiaries with opioid prescriptions. JAMA Netw Open. (2019) 2(3):e190968. 10.1001/jamanetworkopen.2019.0968

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  ThottakkaraPOzrazgat-BaslantiTHupfBBRashidiPPardalosPMomcilovicPet alApplication of machine learning techniques to high-dimensional clinical data to forecast postoperative complications. PLoS One. (2016) 11(5):e0155705. 10.1371/journal.pone.0155705

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  AngraalSMortazaviBJGuptaAKheraRAhmadTDesaiNRet alMachine learning prediction of mortality and hospitalization in heart failure with preserved ejection fraction. JACC Heart Fail. (2020) 8(1):12–21. 10.1016/j.jchf.2019.06.013

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  GottdienerJSFohnerAE. Risk prediction in heart failure: new methods, old problems. JACC Heart Fail. (2020) 8(1):22–4. 10.1016/j.jchf.2019.08.015

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  SureScripts. Surescripts transforms prescription decision process between physicians and patients (2017). Available at:https://surescripts.com/news-center/press-releases/detail/surescripts-transforms-prescription-decision-process-between-physicians-and-patients(Accessed June 11, 2023).

• 18.

  StekhovenDJBühlmannP. Missforest—non-parametric missing value imputation for mixed-type data. Bioinformatics. (2012) 28(1):112–8. 10.1093/BIOINFORMATICS/BTR597

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  GeronA. Hands-on machine learning with scikit-learn, keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. Sebastopol, CA: O’Reilly Media (2019).

  • Google Scholar

• 20.

  SimonSTMandairDTiwariPRosenbergMA. Prediction of drug-induced long QT syndrome using machine learning applied to harmonized electronic health record data. J Cardiovasc Pharmacol Ther. (2021) 26(4):335–40. 10.1177/1074248421995348

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  TiwariPColbornKLSmithDEXingFGhoshDRosenbergMA. Assessment of a machine learning model applied to harmonized electronic health record data for the prediction of incident atrial fibrillation. JAMA Netw Open. (2020) 3(1):e1919396. 10.1001/jamanetworkopen.2019.19396

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  MandairDTiwariPSimonSColbornKLRosenbergMA. Prediction of incident myocardial infarction using machine learning applied to harmonized electronic health record data. BMC Med Inform Decis Mak. (2020) 20(1):252. 10.1186/s12911-020-01268-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  BlagusRLusaL. SMOTE for high-dimensional class-imbalanced data. BMC Bioinformatics. (2013) 14:106. 10.1186/1471-2105-14-106

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  ChawlaNVBowyerKWHallLOKegelmeyerWP. SMOTE: synthetic minority over-sampling technique. J Artificial Intelligence Res. (2002) 16:321–57. 10.1613/jair.953

  • CrossRef
  • Google Scholar

• 25.

  DeLongERDeLongDMClarke-PearsonDL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44(3):837. 10.2307/2531595

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26.

  SteyerbergEWVickersAJCookNRGerdsTGonenMObuchowskiNet alAssessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. (2010) 21(1):128–38. 10.1097/EDE.0b013e3181c30fb2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  LundbergSMLeeSI. A unified approach to interpreting model predictions. In: Advances in neural information processing systems. La Jolla, CA. (2017). p. 4765–74.

  • Google Scholar

• 28.

  WangKTianJZhengCYangHRenJLiCet alImproving risk identification of adverse outcomes in chronic heart failure using smote + enn and machine learning. Risk Manag Healthc Policy. (2021) 14:2453–63. 10.2147/RMHP.S310295

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29.

  MadakkatelIZhouAMcDonnellMDHyppönenE. Combining machine learning and conventional statistical approaches for risk factor discovery in a large cohort study. Sci Rep. (2021) 11(1):1–11. 10.1038/s41598-021-02476-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30.

  HarunRSaneRYoshidaKSutariaDSJinJYLuJ. Risk factors of hyperglycemia after treatment with the AKT inhibitor ipatasertib in the prostate cancer setting: a machine learning-based investigation. JCO Clin Cancer Inform. (2023) 7(7):e2200168. 10.1200/CCI.22.00168

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  LiRShindeALiuAGlaserSLyouYYuhBet alMachine learning-based interpretation and visualization of nonlinear interactions in prostate cancer survival. JCO Clin Cancer Inform. (2020) 4(4):637–46. 10.1200/CCI.20.00002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  Moncada-TorresAvan MaarenMCHendriksMPSieslingSGeleijnseG. Explainable machine learning can outperform cox regression predictions and provide insights in breast cancer survival. Sci Rep. (2021) 11(1):1–13. 10.1038/s41598-021-86327-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  JingLUlloa CernaAEGoodCWSauersNMSchneiderGHartzelDNet alA machine learning approach to management of heart failure populations. JACC Heart Fail. (2020) 8(7):578–87. 10.1016/j.jchf.2020.01.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  CarrollRMudgeASunaJDenaroCAthertonJ. Prescribing and up-titration in recently hospitalized heart failure patients attending a disease management program. Int J Cardiol. (2016) 216:121–7. 10.1016/j.ijcard.2016.04.084

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35.

  JenkinsRMandaranoLGugathasSKaskiJCAndersonLBanerjeeD. Impaired renal function affects clinical outcomes and management of patients with heart failure. ESC Heart Fail. (2017) 4(4):576–84. 10.1002/ehf2.12185

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36.

  HeidenreichPAZhaoXHernandezAFYancyCWFonarowGC. Patient and hospital characteristics associated with traditional measures of inpatient quality of care for patients with heart failure. Am Heart J. (2012) 163(2):239–45.e3. 10.1016/j.ahj.2011.10.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  OpolskiGOzierańskiKLelonekMBalsamPWilkinsAPonikowskiP. Adherence to the guidelines on the management of systolic heart failure in ambulatory care in Poland. Pol Arch Intern Med. (2017) 127(10):657–65. 10.20452/pamw.4083

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  HirtMNMuttardiAHelmsTMvan den BusscheHEschenhagenT. General practitioners’ adherence to chronic heart failure guidelines regarding medication: the GP-HF study. Clin Res Cardiol. (2016) 105(5):441–50. 10.1007/s00392-015-0939-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  FuatAHunginAPSMurphyJJ. Barriers to accurate diagnosis and effective management of heart failure in primary care: qualitative study. Br Med J. (2003) 326(7382):196. 10.1136/bmjopen-2013-003866

  • CrossRef
  • Google Scholar

• 40.

  HobbsFD. Management of heart failure: evidence versus practice. Does current prescribing provide optimal treatment for heart failure patients?Br J Gen Pract. (2000) 50(458):735–42.

  • Pubmed Abstract
  • Google Scholar

• 41.

  FuMVedinOSvennbladBLampaEJohanssonDDahlströmUet alImplementation of sacubitril/valsartan in Sweden: clinical characteristics, titration patterns, and determinants. ESC Heart Fail. (2020) 7(6):3633–43. 10.1002/EHF2.12883

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42.

  DeVoreAHillLThomasLSharmaPPAlbertNMButlerJet alPatient, provider, and practice characteristics associated with sacubitril/valsartan use in the United States. Circ Heart Fail. (2018) 11(9):e005400. 10.1161/CIRCHEARTFAILURE.118.005400

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43.

  SangaralinghamLRSangaralinghamSJShahNDYaoXDunlaySM. Adoption of sacubitril/valsartan for the management of patients with heart failure. Circ Heart Fail. (2018) 11(2):e004302. 10.1161/CIRCHEARTFAILURE.117.004302

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  LuoNFonarowGCLippmannSJMiXHeidenreichPAYancyCWet alEarly adoption of sacubitril/valsartan for patients with heart failure with reduced ejection fraction: insights from get with the guidelines–heart failure (GWTG-HF). JACC Heart Fail. (2017) 5(4):305–9. 10.1016/J.JCHF.2016.12.018

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  KingJShahRBressANelsonRBellowsB. Cost-effectiveness of sacubitril-valsartan combination therapy compared with enalapril for the treatment of heart failure with reduced ejection fraction. JACC Heart Fail. (2016) 4(5):392–402. 10.1016/J.JCHF.2016.02.007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46.

  PackerMArmstrongWMRothsteinJMEmmettM. Sacubitril-valsartan in heart failure: why are more physicians not prescribing it?Ann Intern Med. (2016) 165(10):735–6. 10.7326/M16-1932

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47.

  PaulusJKKentDM. Race and ethnicity: a part of the equation for personalized clinical decision making?Circ Cardiovasc Qual Outcomes. (2017) 10(7):e003823. 10.1161/CIRCOUTCOMES.117.003823

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48.

  KaoDPTrinkleyKELinCT. Heart failure management innovation enabled by electronic health records. JACC Heart Fail. (2020) 8(3):223–33. 10.1016/j.jchf.2019.09.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar