Prediction models for cardiovascular disease risk among people living with HIV: A systematic review and meta-analysis

1. Introduction

HIV continues to be a major global health issue. At the end of 2020, there were approximately 37.7 million people living with HIV (PLWH), with 1.5 million people becoming newly infected with HIV (1). The widespread usage of highly active antiretroviral therapy (HAART) drastically reduced death rates and potential years of life lost (2). However, as life expectancy increases, non-AIDS-defining illnesses are becoming increasingly common causes of death, with cardiovascular disease (CVD) accounting for a sizable portion (3, 4). CVDs, a set of heart and blood vessel disorders, are the leading causes of mortality worldwide. An estimated 17.9 million individuals died from CVDs in 2019, accounting for 32% of all global fatalities (5). According to a meta-analysis that included 793,635 participants from 73 studies, the relative risk (RR) of CVD among PLWH was 2.16 [95% confidence interval (CI), 1.68–2.77] compared to non-HIV-infections (6). In addition, it was indicated that HIV infection is a risk-enhancing factor for CVD (7). Moreover, ART may have a negative association with an increased risk of CVD events, especially among PLWH with other risk factors for cardiovascular disease (8, 9). As a result, the prediction and treatment of CVD are becoming even more important issues in current HIV management (4).

Prediction models are useful tools for estimating the probability or risk of specific future occurrences based on the combination of multiple predictors (10, 11). They aid clinicians in making therapeutic decisions and determining subsequent steps in therapy (12). CVD risk prediction is crucial to treatment guidelines and CVD control (13). Using prediction models is beneficial to make patients aware of their condition and to encourage them to adopt a healthy lifestyle (12). A substantial number of prediction models for various cardiovascular outcomes have been established for the general population, such as the Framingham Risk Score (FRS) (14), the Systematic COronary Risk Evaluation (SCORE) (15), and the pooled cohort equations of the American Heart Society/American score (PCE) (16). The usefulness of most of the models remains unclear because of methodological shortcomings, incomplete presentation, and lack of external validation (17).

Considering other potential factors driving CVD risk among PLWH, HIV-specific CVD prediction models have been developed, such as the Data collection on Adverse Effects of Anti-HIV Drugs Study risk equation (D:A:D) (18). Several CVD prediction models developed for the general population have also been validated in PLWH. However, the findings of model performance conflicted. Delabays et al. indicated that general models were valid to predict CVD for PLWH. Adding HIV-specific factors to scores did not result in a clinically significant improvement (19). In contrast, Triant et al. recommended adding HIV-related factors because general algorithms consistently underestimate the risk of CVD for PLWH (20). Whether HIV-specific models perform better than general models remains unknown. There is no consensus on optional CVD risk models for PLWH.

Soares et al. conducted a systematic review to summarize the CVD prediction models used for PLWH (21). However, they only considering studies published before January 31, 2021. Additionally, Soares et al. did not perform a subgroup analysis to ensure that the CVD risk prediction model treated subgroups (such as geographic region, race, and age) fairly in PLWH. The fairness of prediction models is crucial for promoting health equity, as ignoring differences in social determinants of health can result in inaccurate risk stratification in vulnerable groups, further exacerbating existing inequities (22).

Therefore, we aimed to systematically summarize multivariable prediction models for CVD risk among PLWH. Specific objectives included, first and foremost, objectively appraising the risk of bias in papers. Second, we conducted a meta-analysis on discrimination to estimate and compare the models' performance quantitatively. This review was conducted in accordance with the guidelines of the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (23). The protocol was registered on PROSPERO (CRD42022322024). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used to guide the reporting of our review (24).

2. Methods

2.1. Search strategy

We conducted a three-step search to identify both published studies and gray literatures. First, keywords and search terms were captured from an initial limited search via PubMed/MEDLINE. This informed the development of the search strategy with the help of a librarian. Second, a comprehensive search was conducted using the following databases: PubMed/MEDLINE, MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCO), and Web of Science. Gray literatures were searched via Google Scholar and Baidu. In PubMed/MEDLINE, we searched for papers in English using MeSH terms ([“HIV” OR “Acquired Immunodeficiency Syndrome” OR “HIV infections”] AND [prognosis OR survival OR mortality OR risk]) combined with the title/abstract keywords ([“progn*” OR “predic*” OR “risk” OR “model*”] AND [“HIV” OR “AIDS” OR “Acquired Immunodeficiency Syndrome” OR “PLWH”] AND [“machine learning” OR “artificial intelligence” OR “algorithm”] AND [“cardiovascular disease” OR “CVD” OR “cardiovascular event*” OR“main cardiovascular adverse event”OR “MACE”]). The search was limited to studies from inception to January 2022. The full search strategies for each database are presented in Supplementary Appendix S1. Finally, references in all included studies were manually reviewed to supplement the database search.

2.2. Inclusion and exclusion criteria

The inclusion criteria were as follows: (1) studies that assessed risk models that predicted the short-term or long-term risk of cardiovascular disease among PLWH. Cardiovascular disease in our review included coronary heart disease (CHD), myocardial infarction (MI), stroke, and other heart and blood vessel disorders (5). (2) Studies that developed new models, conducted external validation of an existing model, and/or updated an existing model. (3) Studies published in English or Chinese. No restrictions were made on the setting (e.g., inpatients, outpatients), prediction horizon (how far ahead the model predicted), or predictors or outcomes. Cross-sectional studies were excluded. Studies that merely carried out risk factor analysis without modeling were further excluded.

2.3. Study screening and selection

All identified citations were imported into EndNote X8 (Clarivate Analytics, PA, United States) to remove duplicates. Two reviewers (JY & ZZ) independently carried out screening and selection. First, titles and abstracts were screened to ascertain potentially relevant studies. Next, we screened the full texts and identified studies that met the inclusion and exclusion criteria. Discrepancies were discussed between two reviewers until a consensus was reached after referring to the protocol.

2.4. Quality appraisal

We appraised the quality of the included articles by using the Prediction model Risk Of Bias ASsessment Tool (PROBAST), which is an assessment tool developed specifically for diagnostic and prognostic prediction model studies (25). Two reviewers (JY & ZZ) assessed the risk of bias and applicability respectively. For the assessment of risk of bias, four domains (participants, predictors, outcome, analysis) with a total of twenty signaling questions were judged. Each signaling question was answered as “yes”, “probably yes”, “probably no”, “no”, or “no information”. Based on the answers to the signaling questions, we used our own judgment to rate the domains as high (−), low (+), or unclear (?) risk of bias. We assessed applicability by using the same first three domains but without signaling questions. Finally, we rated the overall ROB and applicability as high (−), low (+), or unclear (?). Any disagreement was discussed between the two reviewers until a consensus was reached.

2.5. Data extraction

Two reviewers (JY & ZZ) independently extracted information from the included papers, including the authors, year, location, study design, study population, predicted outcomes, predictors, sample size, missing data, modeling method, method of validation, and predictive performance. We extracted data separately for each model when a study described multiple models. Any disagreement was discussed, and a consensus was reached.

2.6. Data synthesis

We performed descriptive analysis to summarize the characteristics of the models. If a model was validated in two or more studies, we applied a random effects meta-analysis model to evaluate discrimination performance (26). Since different studies reported different discrimination metrics, either by Harrell's C-statistic or by area under the curve (AUC), we further pooled them separately. Due to the variability in the validation population, differences in outcome and predictor definitions across studies, and the non-normal distribution of the C-statistic between studies, we transformed the C-statistic using a logit transformation (27). When confidence intervals were not available, we approximated them by using the standard normal distribution (28). We considered C-statistics in the range of 0.50–0.59 to indicate poor, 0.60–0.69 to indicate moderate, 0.70–0.79 to indicate acceptable, 0.80–0.89 to indicate very good, and 0.90 or greater to indicate excellent discrimination (29). The I² statistics were used to quantify the heterogeneity of the studies. In addition, subgroup analyses were performed in terms of sample size, age, and follow-up. The validation cohort was defined as “Mainly White” if more than 50% of the population was white. Since all the studies encompass both white and black individuals, the term “Mainly” was used instead of specifying either “White” or “Black”. Analyses were performed in Stata 17.0 (College Station, TX).

3. Results

3.1. Inclusion of studies

A total of 6,192 records were obtained from the databases. After removal of duplicates, the titles and abstracts of 3,901 articles were screened. The full texts of 76 articles were reviewed, and 63 articles were excluded because they did not meet the predefined inclusion criteria. Finally, 13 articles were included in our systematic review (Figure 1) (19, 20, 30–40).

FIGURE 1

Figure 1. Flow diagram of the selection process.

3.2. Characteristics of eligible studies

Table 1 shows the characteristics of the included studies. Overall, 17 different models were derived from 13 studies (19, 20, 30–40). There were six HIV-specific models, including the D:A:D model (2010), the D:A:D (2010) 10-year version model, the updated full D:A:D (2016) model, the updated reduced D:A:D (2016) model, the HIV MI-1, the HIV MI-2 models. The other 11 models were developed for the general population and validated in PLWH, including the FRS (2004), FRS for CVD risk (2008), FRS for CHD risk (1998), FRS adjusted for Colombia, FRS adaptation for the Spanish population (REGICOR), FRS developed specifically for the Italian population by Progetto CUORE, the Prospective Cardiovascular Munster Study (PROCAM), SCORE, SCORE 2, SCORE adjusted for national data (SCORE-NL), and PCE.

TABLE 1

Table 1. Characteristics of the included studies.

In total, four studies (20, 32–34) described the development process of prediction models, and 13 studies (19, 20, 30–40) conducted model external validation. Four studies were carried out in the United States (20, 30, 32, 39), followed by Italy (n = 2) (31, 37), Switzerland (n = 1) (19), Colombia (n = 1) (35), Spain (n = 1) (36), Germany (n = 1) (38), and Netherlands (n = 1) (40), with two being multicenter studies from several countries (33, 34). The sample size ranged from 369 (31) to 32,663 (34). The number of prediction events varied from 8 (35) to 1,393 (40). Twelve studies were mostly conducted on males, while one study (20) included only male participants.

Among four development studies, three (20, 33, 34) used backward selection to remove nonsignificant predictors, whereas the other one (32) did not report a specific strategy for variable selection. Strategies dealing with missing data varied, including excluding participants with missing data (34) and using single imputation (32); one study did not describe a strategy (33).

Among 13 validation studies, nine studies reported both the calibration and discrimination performance of the models (19, 20, 30–34, 39, 40). Two studies reported only discrimination (35, 38), one study reported only calibration (37), and one study did not report model performance (36). Discrimination was reported as Harrell's C-statistic in six studies (20, 30, 34, 39, 40) and AUC in five studies (19, 31, 33, 35, 38). A variety of methods were used to report the calibration performance of the models, including the observed-expected ratio (n = 5) (30, 31, 33, 39, 40), Hosmer–Lemeshow test (n = 3) (19, 34, 39), calibration plot (n = 3) (30, 32, 40), Greenwood-Nam-D'Agostino (GND) test (n = 3) (20, 32, 40), and Brier score (n = 1) (19).

3.3. Risk of bias

Figure 2 shows a summary of the risk of bias and applicability for all included studies. All of the included studies had a high risk of bias. Ten studies (76.9%) (19, 20, 30–32, 36, 37, 40) were considered to have a low risk of bias in the predictor domain, and all the included studies (19, 20, 30–40) were considered to have a low risk of bias in the outcome domain. The risk of bias related to the participant domain was high in seven studies (53.8%) (30, 33–35, 38–40), mainly because they excluded participants with missing data. Twelve studies were judged to have high risk of bias in the analysis domain (19, 20, 30–33, 35–40). Nine studies ignored or dealt with model overfitting in an inappropriate manner (18, 30–32, 35–39). Seven studies had lower-than-reasonable numbers of participants with the events (20, 30, 31, 35–38). Ten studies did not report the methodologies for continuous and categorical predictor transformation (19, 20, 31, 32, 35–40).

FIGURE 2

Figure 2. Risk of bias and applicability of the included studies.

3.4. HIV-specific models

Table 2 shows the characteristics of six HIV-specific models (32–34, 39, 40). In total, four models were developed in multicenter studies (33, 34, 39, 40), while the remaining two models were developed in the US (32). Four models could be used in all age groups (33, 34, 39, 40). Three models were developed to predict the 5-year risk of CVD, including the D:A:D (2010), updated full D:A:D (2016) and reduced models, whereas the other three models were for 10-year risk prediction. For the modeling methods, three models utilized Cox proportional hazards modeling (32, 34), two used Poisson regression modeling (33, 39, 40), and one used lasso and ridge regression (32).

TABLE 2

Table 2. Predictors and outcomes of six HIV-specific models.

In total, 24 predictors were considered. Age, sex, family history of CVD, systolic blood pressure, Smoking, total cholesterol, high-density liptein cholesterol, and diabetes mellitus (n = 4) (33, 34, 39, 40) were the most common predictors, followed by current abacavir use (n = 3) (34, 39, 40). HIV-specific risk factors were included in four models: three models included current abacavir usage (33, 39, 40), two models included CD4 count (34) and exposure to lopinavir/ritonavir and indinavir (33, 39, 40), and one model included cumulative protease inhibitor (PI) exposure (34) and cumulative nucleoside reverse transcriptase inhibitor (NRTI) exposure (34).

The definitions of CVD outcomes revealed considerable heterogeneity. All six models included myocardial infarction (MI) (32–34, 39, 40), followed by coronary heart disease (CHD) death, carotid artery endarterectomy, invasive coronary artery procedure, and stroke (n = 4) (33, 34, 39, 40).

3.5. Meta-analysis of prediction models

Figure 3 shows the results of the meta-analysis of the eight models. We conducted meta-analysis only on model discrimination. Calibrations were not synthesized due to inconsistent and inadequate data. In total, eight models were validated more than once and included in the meta-analysis. Two validation studies were removed from the meta-analysis because they did not report C-statistics or AUCs (36, 37). The FRS CVD model (2008) was the most widely validated model (n = 7) (20, 31, 33–35, 39, 40), followed by the PCE model (n = 6) (20, 32, 35, 38–40), the D:A:D model (2010) (n = 4) (19, 33, 35, 39), and the SCORE (n = 4) (31, 35, 38, 39). In the meta-analysis, the pooled estimated C-statistic/AUC was 0.76 (95% CI: 0.72–0.81, I² = 84.8%) for the D:A:D (2010), 0.75 (95% CI: 0.70–0.79, I² = 82.4%) for the D:A:D (2010) 10-year risk version, 0.77 (95% CI: 0.74–0.80, I² = 82.2%) for the full D:A:D (2016) model, 0.74 (95% CI: 0.68–0.79, I² = 86.2%) for the reduced D:A:D (2016) model, 0.71 (95% CI: 0.61–0.79, I² = 87.9%) for the FRS CHD model (1998), 0.74 (95% CI: 0.70–0.78, I² = 87.8%) for the FRS CVD model (2008), 0.72 (95% CI: 0.67–0.76, I² = 75.0%) for the PCE model, and 0.67 (95% CI: 0.56–0.77, I² = 51.3%) for the SCORE. However, there were no significant differences between any of the model pairings. Table 3 shows the results of the meta-analysis of C-statistics and AUC respectively. Overall, all of the models showed an acceptable discrimination.

FIGURE 3

Figure 3. Meta-analysis of the C-statistic for the CVD prediction models in PLWH.

TABLE 3

Table 3. Meta-analysis of the Harrel's C-statistic and AUC respectively of the CVD models for PLWH.

3.6. Subgroup analysis

To explore potential inequities in prediction models and sources of heterogeneity, we conducted subgroup analyses for sample size, age, and follow-up period (Table 4). The discrimination of PCE was significantly better in the group aged ≤40 years than in the groups aged 40–45 years (P = 0.024) and ≥45 years (P = 0.010). The discrimination of the full D:A:D (2016) model was significantly better in the group with sample size ≥10,000 than in the group with sample size 1,000–10,000 (P = 0.0076).

TABLE 4

Table 4. Subgroup analysis of the C-statistics of the CVD prediction models for PLWH.

We conducted subgroup analysis to compare different models in North America (20, 30, 39) and Mainly White (20, 33, 34, 39, 40) population separately because all models were validated in these two groups. No significant differences were found (P > 0.05). In both groups, only the SCORE performed poorer than the other models in terms of discrimination, with a C-statistic of 0.59 (0.45, 0.73). No models were developed or validated in Sub-Saharan Africa and the Asia region.

4. Discussion

This systematic review identified 17 prediction models for predicting CVD risk among PLWH. Among them, six were HIV-specific models. All included studies were rated as having a high risk of bias against the PROBAST checklist. Only eight models were externally validated at least once. These were included in the meta-analysis, and all of them showed acceptable discrimination. The FRS was the most widely validated model. Most prediction models had acceptable discrimination. The full D:A:D (2016) model performed the best in terms of discrimination, followed by D:A:D (2010) and PCE. However, there were no significant differences between any of the model pairings. We also found that differences in model discrimination existed when they were stratified by geographic region, sample size, and age.

The full D:A:D (2016) model had higher discrimination for PLWH. However, it remained at a high risk of bias as they only included participants with complete information on all predictors. It is inappropriate to exclude participants with missing data directly. Missing data could be divided into missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) (41). MCAR and MAR are less problematic, but they seldomly occur (24, 41). Consequently, complete case analysis could lead to selection bias, and the characteristics of participants may differ from those with missing data. A nonrandom sample subset could frequently generate considerable bias that cannot be overcome in estimating model parameters and yielding predictive performance (24, 41, 42). Currently, common strategies for handling missing data include zero imputation, mean imputation, and multiple imputation (43). Multiple imputation is often regarded as a preferred method for avoiding selection bias and statistical power loss (24, 44). Furthermore, it is advisable to consider combining unsupervised and supervised learning methods for imputation (42). Strategies for handling missing data should be chosen with caution depending on the intended application of the prediction model (41).

The D:A:D cohort was the largest HIV cohort in our study. A large sample size was one of the key points in developing robust models that would be more reliable for application in new datasets (45). In contrast, nearly half of the included studies (n = 7) (20, 30, 31, 35–38) had limited sample sizes and numbers of outcome events. For development studies, the sample size should ensure at least 10 events per candidate predictor parameter. For studies validating prediction models, a minimum of 100 events and 100 nonevents are suggested (24). A small sample size may result in model overfitting and inaccurate predictions (45, 46).

Current models applied for PLWH lack HIV-specific predictors. This might be attributed to the fact that the majority of prediction models are developed for the general population. According to our review, frequently used traditional CVD risk predictors included age, smoking status, systolic blood pressure, total cholesterol, and diabetes mellitus. Common HIV-specific predictors included abacavir usage, CD4 count, and exposure to lopinavir/ritonavir and indinavir. Previous studies presented multiple factors that drive the risk of CVD among PLWH (47). Decreased CD4 T-cell counts and increased HIV RNA have been demonstrated to be linked to increased CVD rates (47, 48). A persistently lower or inverted CD4/CD8 ratio has emerged as an independent predictor of CVD risk (48). Analysis from observational studies showed that most PIs, such as ritonavir-boosted darunavir/ritonavir, are associated with progressively increased CVD risk, and the effect is cumulative (49, 50). The results of D:A:D cohort indicated that the incidence rate ratio of using ritonavir-boosted darunavir was 1.59 (95% CI: 1.33–1.91, per 5 years of additional use) compared to non-use (50). Our review indicated that the full D:A:D model (2016) performed better than the reduced model (C-statistics: 0.775 vs. 0.739). This might be due to the reduced model leaving out ART covariates, such as PI and NRTI exposure. We recommend including CD4 counts and the CD4/CD8 ratio, as well as ART covariates (PI and NRTI exposure) as potential predictors in future studies that aim to develop HIV-specific models. In addition, it is critical to understand the role of HIV-specific inflammation and immune activation in conferring CVD risk (48, 51–53). Few models contained immunological-related factors, such as soluble markers of interleukin-1 (IL-1) (53). The feasibility and effectiveness of adding immunological-related factors into models still remain to be assessed. Further study is needed to add HIV-specific predictors to models.

Among published models, most used univariate or multivariate regression analysis and no validation process. The performance of the prediction model would be overestimated if there was no internal or external validation process (54). Internal validation is beneficial to provide a more accurate assessment of model performance in new subjects (54). External validation is necessary to determine a model's reproducibility and generalizability to populations with different characteristics (55). Models that have not been validated should not be recommended for clinical use. We suggest that future studies validate the multiple existing HIV-specific models and compare their performance head to head.

Calibration, one of the important aspects of validating model performance, has remained underreported in many published studies. Calibration refers to the accuracy of absolute risk estimates by comparing how similar the predicted risk to the true (observed) risk in different risk strata (56). It could be assessed in various ways, including the Hosmer–Lemeshow test (H–L test), calibration-in-the-large, Brier score, Cox intercept and slope, and integrated calibration index (57, 58). Although the H–L test was one of the most common proxies for calibration measures, we should consider its limitations, including its vulnerability when increasing the sample size and the arbitrary number of groups (55, 57). We recommend that future studies combine multiple calibration measures to assess the model calibration more comprehensively and to promote comparability of model calibration across studies.

Our subgroup analysis revealed differences in the model's discrimination when stratified by age. We found that the model's discrimination degraded with age. Age is an important independent risk factor for CVD (59). Aging is always accompanied by changes in the heart and vascular system. In addition, older individuals are more susceptible to drug interactions and side effects (60). Frailty, a common symptom in older people, may also be associated with an increased risk of CVD and CVD mortality (61, 62). These characteristics of older adults may lead to changes in the relationship between risk factors and CVD outcomes. Specific CVD risk models for older PLWH should be developed and validated in the future. In addition, the majority of models were developed and validated in America and Europe, whereas there was a lack of models from Asia and Africa. As global statistics revealed, Sub-Saharan Africa and the Asia have the highest burden of HIV-associated CVD (63). Models should be adapted to these regions and validated further in the future due to differences in race/ethnicity, healthcare systems, and lifestyle.

5. Limitations

There are several limitations of our study. First, we included only studies that were in English or Chinese. Articles in other languages should be summarized in future reviews. Second, most data for model development and validation came from developed countries. Caution should be used when applying our findings to individuals from different regions. Third, we conducted a meta-analysis only on discrimination rather than calibration. This is because of the inadequate reporting of calibration in the validation studies.

6. Implications for practice

The full D:A:D (2016) model had higher discrimination for PLWH. In addition, the findings indicated that model performance was associated with age. In clinical practice, health professionals should revalidate models if the population, region, or age to which they are applied is different from the original study. Moreover, only eight models were validated more than once. Models that have not been thoroughly validated are not useful for clinical practice. Health professionals should focus on developing, updating, and validating HIV-specific models and report them in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines. Specific CVD risk models for older PLWH, as well as models for Sub-Saharan Africa and the Asia region should be established that CVD risk and prevalence vary in different regions due to culture, local economy and health care policy.

7. Conclusion

Our systematic review summarized the prediction models for CVD in PLWH and conducted a meta-analysis on model discrimination performance. All of the models showed acceptable discrimination. The full D:A:D (2016) model performed the best in terms of discrimination, followed by D:A:D (2010) and PCE. However, all these models were assessed as having a high risk of bias. Future studies should adhere to the TRIPOD guidelines to ensure the quality and applicability of the models. Researchers should focus on developing and validating CVD models for PLWH. Specific CVD risk models for older PLWH, as well as models for Sub-Saharan Africa and the Asia region should be established.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Author contributions

JY, XL, ZZ, and HL developed the initial concept for the manuscript and the design of this systematic review. JY and ZZ led the study screening and selection, study's quality appraisal, and data extraction and synthesis. JY and XL led the writing of the manuscript. XL, ZZ, ZY, JH, LZ, and HL participated in quality appraisal, data extraction and revisions of manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Natural Science Foundation of China [grant numbers 72104051], Shanghai Sailing Program [grant numbers 20YF1401800], China Medical Board Open Competition Program [grant numbers #20-372], and Shenzhen Science and Technology Innovation committee project [grant numbers (2021) No. 255].

Acknowledgments

We would like to acknowledge the support of the PROSPERO Review Group and would also like to thank the peer referees who provided review comments to improve the protocol.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2023.1138234/full#supplementary-material.

References

1. World Health Organization. Data on the size of the HIV/AIDS epidemic. Available at: https://www.who.int/data/gho/data/themes/topics/topic-details/GHO/data-on-the-size-of-the-hiv-aids-epidemic?lang=en (Accessed March 30, 2022).

2. Antiretroviral Therapy Cohort Collaboration. Life expectancy of individuals on combination antiretroviral therapy in high-income countries: a collaborative analysis of 14 cohort studies. Lancet. (2008) 372:293–99. doi: 10.1016/S0140-6736(08)61113-7

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Farahani M, Mulinder H, Farahani A, Marlink R. Prevalence and distribution of non-AIDS causes of death among HIV-infected individuals receiving antiretroviral therapy: a systematic review and meta-analysis. Int J Std Aids. (2017) 28:636–50. doi: 10.1177/0956462416632428

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Ghosn J, Taiwo B, Seedat S, Autran B, Katlama C. HIV. Lancet. (2018) 392:685–97. doi: 10.1016/S0140-6736(18)31311-4

PubMed Abstract | CrossRef Full Text | Google Scholar

5. World Health Organization. Cardiovascular diseases (CVDs). Available at: https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds) (Accessed March 30, 2022).

6. Shah ASV, Stelzle D, Lee KK, Beck EJ, Alam S, Clifford S, et al. Global burden of atherosclerotic cardiovascular disease in people living with HIV. Circulation. (2018) 138:1100–12. doi: 10.1161/CIRCULATIONAHA.117.033369

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA guideline on the management of blood cholesterol: a report of the American college of cardiology/American heart association task force on clinical practice guidelines. Circulation. (2019) 139:e1082–143. doi: 10.1161/CIR.0000000000000625

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Islam FM, Wu J, Jansson J, Wilson DP. Relative risk of cardiovascular disease among people living with HIV: a systematic review and meta-analysis. HIV Med. (2012) 13:453–68. doi: 10.1111/j.1468-1293.2012.00996.x

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Bavinger C, Bendavid E, Niehaus K, Olshen RA, Olkin I, Sundaram V, et al. Risk of cardiovascular disease from antiretroviral therapy for HIV: a systematic review. PLoS One. (2013) 8:e59551. doi: 10.1371/journal.pone.0059551

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Moons KG, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerbery EW, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. (2015) 162:W1–73. doi: 10.7326/M14-0698

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Br Med J. (2015) 350:g7594. doi: 10.1136/bmj.g7594

CrossRef Full Text | Google Scholar

12. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? Br Med J. (2009) 338:b375. doi: 10.1136/bmj.b375

CrossRef Full Text | Google Scholar

13. Farzadfar F. Cardiovascular disease risk prediction models: challenges and perspectives. Lancet Glob Health. (2019) 7:e1288–9. doi: 10.1016/S2214-109X(19)30365-1

PubMed Abstract | CrossRef Full Text | Google Scholar

14. D'Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al. General cardiovascular risk profile for use in primary care. Circulation. (2008) 117:743–53. doi: 10.1161/CIRCULATIONAHA.107.699579

CrossRef Full Text | Google Scholar

15. Conroy R. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. (2003) 24:987–1003. doi: 10.1016/S0195-668X(03)00114-3

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Goff DC, Lloyd-Jones DM, Bennett G, Coady S, D'Agostino RB, Gibbons R, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American college of cardiology/American heart association task force on practice guidelines. Circulation. (2014) 129:S49–73. doi: 10.1161/01.cir.0000437741.48606.98

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Damen JAAG, Hooft L, Schuit E, Debray TPA, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. Br Med J. (2016) 353:i2416. doi: 10.1136/bmj.i2416

CrossRef Full Text | Google Scholar

18. Friis-Møller N, Sabin CA, Weber R, et al. Combination antiretroviral therapy and the risk of myocardial infarction. N Engl J Med. (2003) 349:1993–2003. doi: 10.1056/NEJMoa030218

CrossRef Full Text | Google Scholar

19. Delabays B, Cavassini M, Damas J, Beuret H, Calmy A, Hasse B, et al. Cardiovascular risk assessment in people living with HIV compared to the general population. Eur J Prev Cardiol. (2022) 29:689–99. doi: 10.1093/eurjpc/zwab201

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Triant VA, Perez J, Regan S, Massaro JM, Meigs JB, Grinspoon SK, et al. Cardiovascular risk prediction functions underestimate risk in HIV infection. Circulation. (2018) 137:2203–14. doi: 10.1161/CIRCULATIONAHA.117.028975

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Soares C, Kwok M, Boucher KA, Haji M, Echouffo-Tcheugui JB, Longenecker CT, et al. Performance of cardiovascular risk prediction models among people living with HIV: a systematic review and meta-analysis [published online ahead of print, 2022 Dec 28]. JAMA Cardiol. (2022) 8(2):139–49. doi: 10.1001/jamacardio.2022.4873

CrossRef Full Text | Google Scholar

22. Zou J, Schiebinger L. AI Can be sexist and racist—it's Time to make it fair. Nature. (2018) 559:324–26. doi: 10.1038/d41586-018-05707-8

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. (2014) 11:e1001744. doi: 10.1371/journal.pmed.1001744

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Br Med J. (2021) 372:n71. doi: 10.1136/bmj.n71

CrossRef Full Text | Google Scholar

25. Wolff RF, Moons KGM, Riley RD, Whiting RF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. (2019) 170:51–8. doi: 10.7326/M18-1376

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Tufanaru C, Munn Z, Stephenson M, Aromataris E. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. Int J Evid Based Healthc. (2015) 13:196–207. doi: 10.1097/XEB.0000000000000065

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Snell KI, Ensor J, Debray TP, Moons KG, Riley RD. Meta-analysis of prediction model performance across multiple studies: which scale helps ensure between-study normality for the C-statistic and calibration measures? Stat Methods Med Res. (2018) 27:3505–22. doi: 10.1177/0962280217705678

PubMed Abstract | CrossRef Full Text | Google Scholar

28. NCSS. Confidence Intervals for the Area Under an ROC Curve. Available at: https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/PASS/Confidence_Intervals_for_the_Area_Under_an_ROC_Curve.pdf (Accessed March 10, 2022).

29. Hanley JA, McNeil BJ. The meaning under a receiver characteristic and use of the area operating (ROC) curve. Radiology. (1982) 143:29–36. doi: 10.1148/radiology.143.1.7063747

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Anikpo I, Agovi AMA, Cvitanovich MJ, Lonergan F, Johnson M, Ojha RP. The data-collection on adverse effects of anti-HIV drugs (D:A:D) model for predicting cardiovascular events: external validation in a diverse cohort of people living with HIV. HIV Med. (2021) 22:936–43. doi: 10.1111/hiv.13147

PubMed Abstract | CrossRef Full Text | Google Scholar

31. De Socio GV, Pucci G, Baldelli F, Schillaci G. Observed versus predicted cardiovascular events and all-cause death in HIV infection: a longitudinal cohort study. BMC Infect Dis. (2017) 17:414. doi: 10.1186/s12879-017-2510-x

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Feinstein MJ, Nance RM, Drozd DR, Ning H, Delaney JA, Heckbert SR, et al. Assessing and refining myocardial infarction risk estimation among patients with human immunodeficiency virus. JAMA Cardiol. (2017) 2:155–62. doi: 10.1001/jamacardio.2016.4494

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Friis-Møller N, Thiébaut R, Reiss P, Weber R, Monforte AD, De Wit S, et al. Predicting the risk of cardiovascular disease in HIV-infected patients: the data collection on adverse effects of anti-HIV drugs study. Eur J Cardiovasc Prev Rehabil. (2010) 17:491–501. doi: 10.1097/HJR.0b013e328336a150

CrossRef Full Text | Google Scholar

34. Friis-Møller N, Ryom L, Smith C, Weber R, Reiss P, Dabis F, et al. An updated prediction model of the global risk of cardiovascular disease in HIV-positive persons: the data-collection on adverse effects of anti-HIV drugs (D:A:D) study. Eur J Prev Cardiol. (2016) 23:214–23. doi: 10.1177/2047487315579291

CrossRef Full Text | Google Scholar

35. García-Peña AA, De-Vries E, Aldana-Bitar J, Cáceres E, Botero J, Vásquez-Jiménez J, et al. Cardiovascular risk prediction models in people living with HIV in Colombia. Rev Invest Clin. (2022) 74:23–30. doi: 10.24875/RIC.21000251

CrossRef Full Text | Google Scholar

36. Herrera S, Guelar A, Sorli L, Vila J, Molas E, Grau M, et al. The Framingham function overestimates the risk of ischemic heart disease in HIV-infected patients from Barcelona. HIV Clin Trials. (2016) 17:131–9. doi: 10.1080/15284336.2016.1177266

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Raggi P, De Francesco D, Manicardi M, Zona S, Bellasi A, Stentarelli C, et al. Prediction of hard cardiovascular events in HIV patients. J Antimicrob Chemother. (2016) 71:3515–18. doi: 10.1093/jac/dkw346

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Schulz CA, Mavarani L, Reinsch N, Albayrak-Rena S, Potthoff A, Brockmeyer N, et al. Prediction of future cardiovascular events by Framingham, SCORE and ASCVD risk scores is less accurate in HIV-positive individuals from the HIV-HEART study compared with the general population. HIV Med. (2021) 22:732–41. doi: 10.1111/hiv.13124

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Thompson-Paul AM, Lichtenstein KA, Armon C, Palella FJ Jr, Skarbinski J, Chmiel JS, et al. Cardiovascular disease risk prediction in the HIV outpatient study. Clin Infect Dis. (2016) 63:1508–16. doi: 10.1093/cid/ciw615

PubMed Abstract | CrossRef Full Text | Google Scholar

40. van Zoest RA, Law M, Sabin CA, Vaartjes I, van der Valk M, Arends JE, et al. Predictive performance of cardiovascular disease risk prediction algorithms in people living with HIV. J Acquir Immune Defic Syndr. (2019) 81:562–71. doi: 10.1097/QAI.0000000000002069

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Sperrin M, Martin GP, Sisk R, Peek N. Missing data should be handled differently for prediction than for description or causal explanation. J Clin Epidemiol. (2020) 125:183–87. doi: 10.1016/j.jclinepi.2020.03.028

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Zhang X, Yan C, Gao C, Malin BA, Chen Y. Predicting missing values in medical data via XGBoost regression. J Healthc Inform Res. (2020) 4:383–94. doi: 10.1007/s41666-020-00077-1

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Mercaldo SF, Blume JD. Missing data and prediction: the pattern submodel. Biostatistics. (2020) 21:236–52. doi: 10.1093/biostatistics/kxy040

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Hoogland J, Barreveld M, Debray TPA, Reitsma JB, Verstraelen TE, Dijkgraaf MG, et al. Handling missing predictor values when validating and applying a prediction model to new patients. Stat Med. (2020) 39:3591–607. doi: 10.1002/sim.8682

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Riley RD, Ensor J, Snell KIE, Harrell FE Jr, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. Br Med J. (2020) 368:m441. doi: 10.1136/bmj.m441

CrossRef Full Text | Google Scholar

46. Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. (2016) 35:214–26. doi: 10.1002/sim.6787

PubMed Abstract | CrossRef Full Text | Google Scholar

47. McGettrick P, Mallon PWG, Sabin CA. Cardiovascular disease in HIV patients: recent advances in predicting and managing risk. Expert Rev Anti Infect Ther. (2020) 18:677–88. doi: 10.1080/14787210.2020.1757430

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Triant VA, Grinspoon SK. Epidemiology of ischemic heart disease in HIV. Curr Opin HIV AIDS. (2017) 12:540–47. doi: 10.1097/COH.0000000000000410

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Lundgren J, Mocroft A, Ryom L. Contemporary protease inhibitors and cardiovascular risk. Curr Opin Infect Dis. (2018) 31:8–13. doi: 10.1097/QCO.0000000000000425

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Ryom L, Lundgren JD, El-Sadr W, Reiss P, Kirk O, Law M, et al. Cardiovascular disease and use of contemporary protease inhibitors: the D:A:D international prospective multicohort study. Lancet HIV. (2018) 5:e291–300. doi: 10.1016/S2352-3018(18)30043-2

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Larson D, Won SH, Ganesan A, Maves RC, Kronmann K, Okulicz JF, et al. Statin usage and cardiovascular risk among people living with HIV in the U.S. military HIV natural history study. HIV Med. (2022) 23:249–58. doi: 10.1111/hiv.13195

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Boccara F, Lang S, Meuleman C, Ederhy S, Mary-Krause M, Costagliola D, et al. HIV and coronary heart disease: time for a better understanding. J Am Coll Cardiol. (2013) 61:511–23. doi: 10.1016/j.jacc.2012.06.063

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Hoel H, Ueland T, Knudsen A, Kjær A, Michelsen AE, Sagen EL, et al. Soluble markers of interleukin 1 activation as predictors of first-time myocardial infarction in HIV-infected individuals. J Infect Dis. (2019) 221:506–09. doi: 10.1093/infdis/jiz253

CrossRef Full Text | Google Scholar

54. Steyerberg EW, Harrell FJ, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. (2001) 54:774–81. doi: 10.1016/S0895-4356(01)00341-9

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M. External validation of prognostic models: what, why, how, when and where? Clin Kidney J. (2021) 14:49–58. doi: 10.1093/ckj/sfaa188

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. (2017) 318:1377–84. doi: 10.1001/jama.2017.12126

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Huang Y, Li W, Macheret F, Gabriel RA, Ohno-Machado L. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc. (2020) 27:621–33. doi: 10.1093/jamia/ocz228

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Stevens RJ, Poppe KK. Validation of clinical prediction models: what does the “calibration slope” really measure? J Clin Epidemiol. (2020) 118:93–9. doi: 10.1016/j.jclinepi.2019.09.016

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Costantino S, Paneni F, Cosentino F. Ageing, metabolism and cardiovascular disease. J Physiol. (2016) 594:2061–73. doi: 10.1113/JP270538

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Hines LE, Murphy JE. Potentially harmful drug–drug interactions in the elderly: a review. Am J Geriatr Pharmacother. (2011) 9:364–77. doi: 10.1016/j.amjopharm.2011.10.004

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Farooqi MAM, Gerstein H, Yusuf S, Leong DP. Accumulation of deficits as a key risk factor for cardiovascular morbidity and mortality: a pooled analysis of 154 000 individuals. J Am Heart Assoc. (2020) 9:e014686. doi: 10.1161/JAHA.119.014686

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Veronese N, Cereda E, Stubbs B, Solmi M, Luchini C, Manzato E, et al. Risk of cardiovascular disease morbidity and mortality in frail and pre-frail older adults: results from a meta-analysis and exploratory meta-regression analysis. Ageing Res Rev. (2017) 35:63–73. doi: 10.1016/j.arr.2017.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Feinstein MJ, Hsue PY, Benjamin LA, Bloomfield GS, Currier JS, Freiberg MS, et al. Characteristics, prevention, and management of cardiovascular disease in people living with HIV: a scientific statement from the American heart association. Circulation. (2019) 140:e98–124. doi: 10.1161/CIR.0000000000000695

PubMed Abstract | CrossRef Full Text | Google Scholar