Prediction models for major adverse cardiovascular events after percutaneous coronary intervention: a systematic review

Background The number of models developed for predicting major adverse cardiovascular events (MACE) in patients undergoing percutaneous coronary intervention (PCI) is increasing, but the performance of these models is unknown. The purpose of this systematic review is to evaluate, describe, and compare existing models and analyze the factors that can predict outcomes. Methods We adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 during the execution of this review. Databases including Embase, PubMed, The Cochrane Library, Web of Science, CNKI, Wanfang Data, VIP, and SINOMED were comprehensively searched for identifying studies published from 1977 to 19 May 2023. Model development studies specifically designed for assessing the occurrence of MACE after PCI with or without external validation were included. Bias and transparency were evaluated by the Prediction Model Risk Of Bias Assessment Tool (PROBAST) and Transparent Reporting of a multivariate Individual Prognosis Or Diagnosis (TRIPOD) statement. The key findings were narratively summarized and presented in tables. Results A total of 5,234 articles were retrieved, and after thorough screening, 23 studies that met the predefined inclusion criteria were ultimately included. The models were mainly constructed using data from individuals diagnosed with ST-segment elevation myocardial infarction (STEMI). The discrimination of the models, as measured by the area under the curve (AUC) or C-index, varied between 0.638 and 0.96. The commonly used predictor variables include LVEF, age, Killip classification, diabetes, and various others. All models were determined to have a high risk of bias, and their adherence to the TRIPOD items was reported to be over 60%. Conclusion The existing models show some predictive ability, but all have a high risk of bias due to methodological shortcomings. This suggests that investigators should follow guidelines to develop high-quality models for better clinical service and dissemination. Systematic Review Registration https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=400835, Identifier CRD42023400835.


Introduction
As global populations continue to grow and age, cardiovascular diseases, particularly coronary artery disease (CAD), have emerged as significant contributors to both mortality and disability (1).Since its first introduction in 1977, percutaneous coronary intervention (PCI) techniques have seen rapid advancements.Presently, PCI is recommended for various scenarios including single-vessel disease accompanied by symptoms and ischemia, as well as early invasive treatment for acute coronary syndrome (ACS), specifically in high-risk patients (2).Despite successful revascularization, patients still face an incidence rate of approximately 20% (3) for cardiovascular events or deaths, primarily occurring within the first year after PCI (4).This has a profound impact on their prognosis and quality of life.
Major adverse cardiovascular events (MACE), which are typically defined as a combination of cardiovascular mortality, non-fatal myocardial infarction, and non-fatal stroke (3-point MACE), represent a frequently utilized outcome in cardiovascular research.When hospitalization for heart failure is also considered, it is referred to as the 4-point MACE (5).In certain studies, the scope of the MACE definition may extend to encompass additional events such as unplanned coronary revascularization, hospitalization for chest pain, arrhythmia, all-cause mortality, and others (6,7).
Numerous studies are dedicated to pinpointing predictive factors associated with MACE occurrence.Some modifiable clinical parameters and laboratory markers have garnered attention.Indicators such as increased neutrophil-to-lymphocyte ratio (8), elevated Lp(a) level combined with heightened hs-CRP (9), the monocyte to high-density lipoprotein ratio, and Gensini score (10) are acknowledged as potential predictors of MACE in individuals undergoing PCI.
Models combined with multiple predictors may assist in identifying high-risk populations.Several predicting models (11)(12)(13)(14) have been developed, manifesting as risk score systems or nomograms, to forecast 30-day MACE, 1-year MACE, or longer-term outcomes following PCI.However, the model performance of these models remains uncertain.The purpose of this study is to identify, describe, and appraise existing models used to predict MACE among post-PCI patients.

Materials and methods
This study was conducted in adherence to the guidelines provided by the Preferred Reporting Items for Systematic reviews and Meta-analysis (PRISMA) 2020 (15).The study was registered at PROSPERO with CRD42023400835.Since that this review comprises published studies and publicly available data, ethical approval is not deemed necessary.

Eligibility criteria
All model development studies, whether validated or not, whether the patient underwent elective or emergency PCI, are encompassed within the scope of this review.These models should have postoperative MACE as their designated outcome, and they are eligible for inclusion irrespective of the duration of follow-up.It is imperative that these studies report on the performance of the models, including but not limited to discrimination and calibration.
Studies will be excluded if they fall within the following categories: (1) Conference abstracts, editorials, expert views, notes, or letters; (2) Review or meta-analysis articles; (3) Full-text articles were not available; (4) Studies that developed CPMs exclusively for specific populations, such as patients with diabetes, chronic kidney disease, atrial fibrillation, or the elderly or women; (5) Studies that applied an existing model to a new domain or evaluated the performance of known models adding one or more new predictors; (6) Studies focused on comparing models rather than assessing their predictive capability for MACE; and (7) External validation articles lacking corresponding development data.

Search strategy
We conducted searches across the following databases: Embase, PubMed, The Cochrane Library, Web of Science, CNKI, Wanfang Data, VIP and SINOMED on 19 May 2023.All studies published from 1977 in which year the first coronary intervention was reported up until the date of literature retrieval are entirely considered.In addition, we supplemented our initial search by manually reviewing the reference lists of identified studies, aiming to minimize the possibility of missing relevant data.The search strategy takes a combination of subject words (MeSH, ENTREE, and others) and free-text terms related to PCI, CPM, and MACE.The search terms encompass a range of expressions including "Percutaneous Coronary Intervention," "Coronary Revascularization," "major adverse cardiovascular event," "cardiovascular outcome," "MACE," "prediction model," "risk stratification," "risk score," and others.Endnote X9 software (Thomson Reuters, Philadelphia, Pennsylvania, USA) was used for document management and duplicate removal.In cases where multiple reports are derived from the same population, only the most recent study will be included.Detailed search strategies are provided in Supplementary Appendix S1.

Selection process
The eligibility of studies was assessed independently (YW and SL) with a third investigator (XW) available to resolve any disagreements.Initially, articles seemingly unrelated to the intended research objectives were excluded based on the screening of titles and abstracts.Then, the studies meeting the criteria were included after a comprehensive full-text reading.

Data extraction
As the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) Checklist (16) suggests, descriptive tables were used to extract information that encompasses the following items: (1) characteristics of studies (e.g., the authorship, publication year, country, study design, participants, outcome definition); (2) characteristics of the models (e.g., the predictors, sample size, algorithms used to select the predictors, model development method, internal validation method, model evaluation metrics such as the area under the curve(AUC)/C-index for discrimination, calibration, sensitivity, specificity; details of dealing missing data; Model presentation).Data were extracted independently by two viewers (YD and WD), and the viewers cross-checked the data before analysis.

Quality assessment
The assessment of the risk of bias and applicability was performed independently by the reviewers (WD and DW) using the PROBAST tool (17), which consists of four domains: participants, predictors, outcome, and analysis.To support the evaluation, a set of 20 questions (Supplementary Appendix S2) was answered with "yes," "probably yes," "no," "probably no," or "no information."

Data synthesis and analysis
The key findings were summarized in a narrative manner and presented in a tabular format or graphs.In addition, we analyzed the adherence to the TRIPOD (18) statement of each study.The 22 items (Supplementary Appendix S3) covered a range of aspects including the title, abstract, methods, results, and other information.No qualitative analysis was performed in this study.

Study selection
The initial search identified 5,234 potentially relevant articles, and an additional one was added through reference citation (19).After removing 945 duplicates, 4,820 articles remained.Subsequently, we screened the titles and abstracts, resulting in the exclusion of 4,146 studies that did not meet the inclusion criteria.In total, 114 articles were reviewed in full text.Following this comprehensive evaluation, we identified 23 studies for the final analysis.Among them, 11 studies were published in Chinese (20-30), while the remaining 12 studies were published in English (11)(12)(13)(31)(32)(33)(34)(35)(36)(37)(38)(39).The selection process is visually depicted in Figure 1.

Study and model characteristics
A total of 23 studies developed 28 new models for predicting MACE.Each study presented at least one model.Notably, four studies (24,26,32,34) created multiple predictive models, incorporating distinct variables.The models sharing identical predictor variables and stemming from the same study population, yet predicting MACE at varying time intervals, were categorized as one model.A total of 12 studies addressed prognostic models for patients with ST-segment elevation myocardial infarction (STEMI) (20, 26-29, 32-34, 36-39).Six studies were centered on patients with myocardial infarction (MI) (11,12,21,22,24,35).Meanwhile, three studies specifically focused on patients with acute coronary syndrome (13,23,25).Notably, Grayson et al. (31) recruited patients undergoing PCI across a spectrum of conditions including stable angina, unstable angina, acute myocardial infarction (AMI), and cardiogenic shock.He (30) did not explicitly mention the diagnostic information of the patients.The prognostic models were mostly developed within 3 years (n = 19, 82.6%).Among the 23 studies, 20 were retrospective cohort studies, while two were prospective studies (13,37), and one was a nested case-control study (12).Approximately 91.3% of the studies (n = 21) were constructed using Chinese populations, while one study (31) was based on England individuals, and another study (37) was based on Spain individuals.The models developed in the eight studies (21,24,25,30,33,37,39) did not undergo either internal or external validation.Three studies performed external validation by enrolling patients at different times (11,31,35).The study sample sizes ranged from 124 (22) to 23,718 (35).Events were reported in 21 studies (91.3%), ranging from 17 (34) to 2,615 (35).The main characteristics of the included studies are summarized in Table 1.

Risk of bias and applicability assessment
According to the PROBAST assessment results, it was determined that all studies had a high risk of bias (Figure 4) owing to deficiencies in their study design, execution, and analysis.The participants' domain mostly consisted of data (n = 20, 86.9%) derived from retrospective cohort studies or registry data that were not originally collected for the specific purpose of developing predictive models.A total of 16 studies directly excluded cases with incomplete data or subgroups that might have an impact on predictive outcomes.This could potentially introduce higher risk of bias.In the predictors' domain, none of the studies provided any information regarding the use of a blinding method.Consequently, they all received a response of "no information" when addressing the signaling question "Were predictor assessments made without knowledge of outcome data?"In the results' domain, a predominant issue emerged: the inappropriate definition of outcomes in all studies.The outcome definition did not exclude predictor variables in five studies (20,21,24,25,30), for instance, heart failure was included in the outcome, while LVEF or NT-proBNP were used as predictors.Moreover, certain studies (20,23,25,27) aimed to predict inhospital MACE, yet they were unable to specify the timing of parameter collection.This could result in an improper time interval between the outcomes.The analysis domain is a highrisk area for bias, primarily stemming from several factors.These include sample sizes that were insufficient to fulfill the requirement of having events per variable (EPV) of ≥20 (15/23, 65.2%), the inappropriate conversion of continuous variables into categorical variables (16/23, 69.6%), the improper handling of missing data (18/23, 78.3%), and the absence of calibration reporting (6/23, 26.1%), among other factors.Predictor selection was predominantly based on univariable analysis (12, 22, 24, 26-28, 30, 37); however, it is advisable to avoid using this method.All the studies included in this review had a low risk of applicability due to the primary focus on evaluating models and identifying potential prognostic factors, with less emphasis on participant and outcome heterogeneity.The PROBAST results for each study are shown in Supplementary Table S2.

Transparent reporting assessment
The study's adherence to transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) items reflects the completeness of reporting the included CPM studies.In this review, the median adherence rate stands at 53%, with an interquartile range (IQR) varying from 30.4% to 100% (Figure 5).Those poorly reported items were mainly concentrated within the Methods section.The omission of blinding was a significant issue in this review since neither the evaluation of outcomes (Item 6b) nor the evaluation of predictors (Item 7b) provided any details regarding blinding procedures or measures to prevent bias.Despite the provision of sample sizes, researchers never clarified how the study size was determined (Item 8).Moreover, no studies explained how they deal with predictors (Item 10a).Five items had adherence rates below 50%.Only two studies (2/23, 8.7%) reported on risk groups (32, 35) (Item 11) and participant selection flowcharts (35, 37) (Item 13a).Furthermore, only three studies (3/23, 13.0%) reported relevant treatment (12, 25, 28) (Item 5c).In addition, seven studies (7/23, 30.4%) adhered to the recommendation citing existing models when introducing the medical background in the Introduction section (11-13, 29, 31, 32, 35) (Item 3a).Nine studies (11-13, 32-34, 36, 37, 39) (9/23, 39.1%) provided supplementary information (Item 21).Four items were incompletely reported (50%-80% adherence).The PROBAST risk of bias assessment.eligibility criteria (Item 5b) were addressed in 18 studies (18/23, 78.3%).Ten studies (13/23, 56.5%) described how to use the model (Item 15ba).Fourteen studies (14/23, 60.9%) detailed the definition of the predictor (Item 7a).The funding information (Item 22) was reported in 18 out of 23 articles (78.3%)where this item was suitable.All studies completely described the following nine items (100% adherence).In the Abstract, all studies provided ample information (Item 2).Within the Methods section, the description of the study design (Item 4a) and outcome definition (Item 6a) were all presented.Studies also efficiently described the characteristics of the participants (Item 13b), model presentation (Item 15a), and performance measures (Item 16) in the Results section.Meanwhile, the Discussion section of all studies were fully organized around the study limitations (Item 18), overall interpretation (Item 19a), and potential clinical application (Item 20).The articles included in this review had 80% compliance with the remaining TRIPOD items (Supplementary Figure S1).

Discussion
Recently, there has been a noticeable increase in the number of prognostic model development studies that specifically focus on post-PCI patients.These studies frequently center on endpoints such as mortality (40)(41)(42), hemorrhage, renal injury, and atrial fibrillation (43)(44)(45).Another important endpoint that has garnered significant interest among researchers is MACE.The application of clinical prediction models can provide valuable information to patients and families and assist healthcare professionals in allocating hospital resources, potentially contributing to the improvement of healthcare quality.In addition, prediction models may aid in the clinical trial design, identifying patients with the required risk characteristics, thereby enhancing statistical power or reducing sample size and costs (46).
This systematic review provides an overview of the current landscape of models designed to predict MACE after PCI.In total, 23 articles were included in this study, presenting a total of 28 proposed models.The discriminatory power of these models varied, spanning from 0.638 to 0.96, where 66.7% of the models achieved a discrimination value exceeding 0.75.In terms of follow-up time, the AUC range for models predicting in-hospital MACE is 0.76-0.888,while the AUC range for the six models predicting 1-year MACE is 0.715-0.96.It is evident that the models explored in our study demonstrated commendable predictive powers in identifying high-risk patients.
The best-performing model (34) in this study achieved an AUC of 0.94 in the training set and an AUC of 0.90 in the validation set.This outstanding performance might be attributed to the model's reliance on a wide array of quantitative imaging parameters.Currently, polygenic risk scores, proteomics, lipidomics (47-49), and other data are gradually being introduced to predict cardiovascular events.Clearly, clinical predictive models based on easily accessible traditional risk factors may be more straightforward to apply and generalize.Collecting and utilizing such data may involve higher costs but hold the potential to improve model performance.Artificial intelligence (AI) and machine learning algorithms, including techniques such as Extreme Gradient Boosting, the Gaussian mixture model, Decision Tree, and Random Forest (50)(51)(52)(53) have become increasingly prevalent in addressing such challenges.Machine learning with its remarkable capability to analyze extensive volumes of intricate data (54) holds tremendous potential for enhancing predictive performance and should see wider adoption in the medical field.
This review has found that the most common variables for predicting post-PCI MACE include LVEF, age, Killip classification, diabetes, Cr, BNP, gender, troponin, smoking, hypertension, and CRP.These variables encompass unmodifiable factors such as demographic data like age and gender.The included studies consistently indicate that older age is associated with worse outcomes.The odds ratio (OR) for individuals over 60 years of age is 1.212 times higher than for those under 60 years (25), while individuals aged 80 and above have a higher risk of MACE compared with those aged 70-79 (31).Several studies suggest that female patients generally have worse prognoses, although Ma et al.'s study (26) arrived at the opposite conclusion.This could be attributed to the study's male-tofemale patient ratio, which was 3.5 times higher, with no adjustment made during the analysis.In the models developed for STEMI patients, the top three variables are LVEF, Killip classification, and age.For AMI patients, the most common variables include LVEF, diabetes, and age, while in ACS patients, the primary variables are LVEF, age, and BNP.There is a consistent demonstration of the value of LVEF and age for adverse outcomes of PCI patients.When analyzing these predictive factors collectively, it cannot be overlooked that the Killip classification assesses cardiac function in cases of AMI, but it is not applicable to patients with unstable angina within the ACS population.Variables assessing kidney function, such as a history of CKD, SCr, and eGFR, were also included in multiple studies.This emphasizes the need for clinical attention to both cardiac and renal function to identify individuals at risk of adverse outcomes early.In recent years, several studies have revealed the connection between inflammation and coronary heart disease.Inflammatory-related variables such as NLR, CRP, and hs-CRP were also incorporated into multiple models.Residual inflammatory risk (RIR), defined as when plasma LDL-C levels are below 1.8 mmol/L and plasma hs-CRP levels are ≥2 mg/L (55), was employed in a model constructed to predict in-hospital MACE in AMI patients after PCI (21).The model achieved an AUC value of 0.82, indicating a close correlation between RIR and recurrent cardiovascular events.
However, the reporting is not rigorous enough, and all studies were judged to be at high risk of bias.The primary factors contributing to this included retrospective study designs, the absence of blinding during the assessment of predictors or outcomes, unjustified categorization and definition of outcomes, failure to circumvent univariate analysis when screening variables, insufficient sample size for EPV, improper internal validation methods, lack of external validation, and improper handling of missing data.The PROBAST, released in 2019 (17), serves as a

Potential clinical applications
Although these models demonstrate good predictive performance, all studies were at high overall risk of bias.This review reveals that there is insufficient evidence to apply any of these models in clinical practice.However, clinical practitioners can pay closer attention to the common variables in the models and intervene appropriately with modifiable risk factors, which may help reduce the incidence of MACE.

Strength and limitations
This study marks a notable effort in directing our attention towards predicting models for MACE in post-PCI patients.We adopted convincing tools such as PROBAST and TRIPOD in this systematic review to provide more informative results.Certain procedural characteristics, such as access site, vessel dilation, type of PCI, initial and final TIMI flow grade, stent type (drug-eluting stents or bare metal stents), and stent length, can reflect the contemporariness of the treatment and indicate the complexity of the medical condition (56).They can also serve as a basis for identifying high-risk individuals.However, these factors have not been sufficiently considered in the current models.Future researchers should give greater consideration to these aspects in their studies.In addition, the application of a language filter limiting studies to Chinese and English during literature screening may result in the omission of valuable data and insights published in other languages.This could introduce some and potentially impact the conclusions of this systematic review.Due to the limited number of relevant studies, we incorporated prediction models designed for patients undergoing PCI in all clinical settings.Nevertheless, it is essential to recognize that patient prognosis and risk factors exhibit variations in different clinical settings.Moreover, these studies vary in their definitions of MACE and the duration of follow-up.Although we made efforts to describe them categorically, the inherent heterogeneity makes further quantitative analysis challenging.

Conclusion
In summary, despite the rising number of modeling studies, the practicality of many models remains uncertain due to a lack of external validation and methodological shortcomings.It is imperative for researchers to adhere to guidelines to enhance study designs and construct models with high clinical applicability using proper internal validation and external methods.

FIGURE 1 Flowchart
FIGURE 1Flowchart of study inclusion.

FIGURE 2 Forest
FIGURE 2 Forest plots in the discrimination of models.(A) Forest plot for the training set.(B) Forest plot for the training set-continued.(C) Forest plot for the validating set.STEMI, ST-segment elevation myocardial infarction; AMI, acute myocardial infarction; MI, myocardial infarction; ACS, acute coronary syndrome; MACE, major adverse cardiovascular events.

TABLE 1
Main characteristics of the included studies.

TABLE 2
Model methods.
Intriguingly, despite 22 out of 23 articles in this review being published post-2019, they still exhibited methodological shortcomings across various facets.To enhance the quality of clinical predictive modeling and maximize the value of such models in clinical applications, it is advisable for researchers to acquaint themselves with the PROBAST tool at an early stage to minimize possible biases in the study design or data sources.