Combining Charlson comorbidity and VACS indices improves prognostic accuracy for all-cause mortality for patients with and without HIV in the Veterans Health Administration

Introduction

With advances in antiretroviral treatment (ART), people with HIV (PWH) are typically able to achieve HIV RNA viral suppression and maintain high CD4 count. However, as PWH age, many non-HIV related comorbid diseases are more common than among age matched comparators without HIV (PWoH) (1–4), and disparities in life expectancy persist (5).

The observed shift from AIDS to non-AIDS morbidity prompted the development of the Veterans Aging Cohort Study (VACS) Index using data from the VACS-HIV cohort, which is comprised of Veterans Health Administration (VHA) electronic health record (EHR) data. The VACS Index incorporates general health (age, albumin, body mass index, hemoglobin, Fibrosis-4 Index for Liver Fibrosis [FIB-4], estimated glomerular filtration rate [eGFR], white blood count, hepatitis C virus [HCV]), and HIV-specific (HIV RNA and CD4 cell count) clinical data to characterize overall disease burden and reflect risk of mortality (6–8). Since its inception and validation, the VACS Index has consistently demonstrated its utility in discriminating mortality and other outcomes in various samples of PWH (7, 9–14). VACS Index risk estimates also generalize well to demographically similar people without HIV in VHA by assuming HIV RNA is undetectable and CD4 cell counts are normal (15). The VACS Index is available online at MDCalc¹ (14). Our long-term goal is to provide automated, individualized estimates from these risk index tools in real time and at the point of care to facilitate informed medical decision making in people with and without HIV.

Although the biomarkers included in the VACS Index reflect physiologic injury from comorbid disease, the only comorbidity directly included is HCV. Comorbidities such as diabetes, pulmonary disease, cancer, cardiovascular disease, and others are not directly reflected in the biomarkers included in the VACS Index. The Charlson Comorbidity Index (CCI) estimates mortality risk based on common comorbid conditions and uses International Classification of Diseases (ICD) codes readily available in administrative billing data (16). The CCI is widely used in clinical research with over ten thousand citations in Pub Med but its discrimination of mortality is only fair (16).

We aimed to determine whether adding comorbid disease groupings from the CCI would improve the accuracy of the VACS Index in people with and without HIV in care in the VHA.

Materials and methods

Data

The VHA includes longitudinal, paperless, national EHR data on ~13.5 million Veterans who received care at over 1,200 points-of-care since 1999 and we capture this information in the VACS-National. To identify a reasonably current patient sample with adequate follow up, we restricted to patients with VHA healthcare visits 2007 through 2017.

Development and validation samples from the general VHA population

To maximize our ability to model mortality among older age groups, we began with the general VHA population that does not include PWH. For the development and validation samples, we excluded Veterans diagnosed with HIV at any time so that we could evaluate the VACS Index in a general, non-HIV sample. We used a two-stage sampling design to randomly select a single index visit per patient. In the first stage, we identified primary care or other clinic visits (using stop codes, Supplementary Table S1), that included a routine blood pressure reading. For each year from 2007 to 2017 we randomly selected one outpatient visit date per person that was at least 18 months after the date of the first outpatient diagnosis in the EHR, giving preference to a primary care visit if one was available. In the second stage, to ensure a range of follow-up times and disease severity, we pooled the visits from each year for those patients that had complete VACS Index data. Compared to those with complete data, patients who had visits with missing data were of similar age and sex, but more likely to have unknown race (8.9% vs. 5.7%), no comorbidities (58.5% vs. 46.2%), and a visit from 2007 to 2011 (48.1% vs. 41.0%). We then randomly selected a single visit per person from all eligible visits. This index visit date was used as the baseline for lab values, comorbidities and start of follow-up.

PWH validation sample from the VHA population

Using data from the VACS-HIV for the PWH sample, we used similar methods to the two-staged sampling design described above. However, we randomly selected one outpatient visit date per person that was at least 1 year after the date of the first ART prescription. HIV status was determined from ICD-10 codes as previously described (17).

Mortality data

Mortality data are from a combination of sources available in the VHA (Social Security Administration, Center for Medicare and Medicaid Services, VHA inpatient deaths and the VA Death Beneficiary database) with accuracy comparable to the National Death Index (18, 19). Patients were followed until death, 10 years, or 12/31/2021. This study was approved by Yale University and VA Connecticut Healthcare System institutional review boards.

Veterans Aging Cohort Study (VACS) Index

The development and internal validity of VACS Index has been described in detail elsewhere (7, 8). As in prior work, for the general non-HIV sample, we assumed CD4 cell count was normal (>500 CD4 cells/mm3) and HIV RNA was undetectable for people without HIV infection (7, 20, 21). In addition to these HIV specific laboratory tests, VACS Index 2.0 includes age, BMI and the following routinely monitored laboratory tests: hemoglobin, platelets, alanine and aspartate transaminases (ALT, AST), creatinine, white blood count, albumin, and HCV status.

Composite markers of liver and renal injury were computed. FIB-4, composed of AST, ALT, platelets, and age, is a validated indicator of liver fibrosis (22). Estimated glomerular filtration rate (eGFR; using the CKD-EPI 2021 equation, composed of serum creatinine, age, and gender, but not race) is a validated indicator of impaired renal function (23).

HCV status was defined as positive if the patient ever had a detectable virus, detectable HCV genotype, treatment for HCV, antibody positive (unless RNA negative) or ICD code for chronic HCV before the index date. Of those deemed HCV positive, 75% had detectable virus. We extracted values from the EHR, restricting to those obtained in outpatient settings, using those closest to the index visit date within 18 months before to 14 days after to allow for labs ordered at the visit. Because full liver panels are not always obtained, we set values to normal when albumin (4 g/dL), or one of ALT (25 IU/L) or AST (20 IU/L) was missing, resulting in complete data for 81% of otherwise eligible sample.

Charlson Comorbidity Index (CCI)

Components of the CCI (Table 1) were based on conditions documented by ICD-9 and ICD-10 diagnosis codes in the 2 years prior to index visit date. Conditions were considered present based on one or more inpatient or two or more outpatient codes. The ICD codes used to identify CCI comorbidities (Supplementary Table S2) were based on a careful examination of prior studies (24–27). The codes identified in these three sources substantially overlapped. A few codes differed due to Canadian versus US codes, errors, and omissions but the difference in CCI score using these sources was negligible.

Table 1

Table 1. Characteristics of patients at randomly selected index visit, 2007–2017.

Statistical analyses

Development and validation in the general PWoH VHA patients

Noting important demographic differences between our general patient sample and our prior samples in PWH, we considered whether associations between predictors and all-cause mortality were different by reassessing variable weights and model calibration. To provide a test of the temporal generalizability, we included those with an index visit between 2007–2011 and 2013–2017 in the developmental sample, withholding those identified in 2012 for inclusion in the independent validation sample. Choosing 2012 for the holdout sample for temporal external validation allowed us to ensure similar average follow-up time and still provide a test of generalizability. We developed a series of prognostic models to evaluate associations between predictors and all-cause mortality.

We used Cox proportional hazard models to predict 10-year mortality with predictors as follows: (1) age, (2) CCI components plus age, (3) VACS Index 2.0 score as published, (4) VACS Index 2.0 components (to allow reweighting, categorical values), (5) CCI components and VACS Index components (continuous values with appropriate functional forms), (6) #5 plus age by CCI interaction, (7) #6 plus sex. Model performance was assessed using Akaike’s information criterion (AIC, lower is better) for model fit and Harrell’s c-statistic (range 0.5–1.0, higher is better) for discrimination. Because of the extremely large sample size c-statistics were generated by averaging results from 5 sets of random samples of 100,000 observations (standard error was the same for all samples). Calibration (details below) was assessed by comparing to observed mortality. For models using continuous numeric variables extreme values were replaced with 1st or 99th percentile to avoid undue influence and variables were centered at clinically meaningful values for interpretability and to avoid variance inflation when using square and cubic terms necessary for using continuous measures.

For final candidate models [#5–#7], we translated regression output to meaningful risk scores that could be compared across models. For each variable, we multiplied regression coefficients by patient’s individual values, then summed to create a linear predictor (Xbeta). Then we scaled to 0–100 by dividing each patient’s Xbeta by the difference of highest and lowest values across all patients in the development sample. We then predicted ten-year all-cause mortality using the newly derived risk score as the only predictor in a parametric (gamma) survival regression model. Observed mortality was estimated using the Kaplan–Meier (KM) method. For each five-point interval of score we calculated mortality and 95% confidence intervals (CI). Calibration was assessed by overlaying plots of observed mortality on predicted mortality and checking for systematic deviations.

We summarized risk scores derived from the final model [#7], dubbed VACS-CCI, and assessed performance in development and validation samples, overall and in subgroups (men: non-Hispanic white, non-Hispanic black, Hispanic, age < 65, age 65+; women; level of CCI score). Risk scores were summarized with mean, standard deviation, and coefficient of variation (COV), which measures the spread of values relative to the magnitude of the mean. We assessed discrimination and calibration as described above. Data points of observed mortality were plotted when there was a minimum of 10 deaths and 5 survivors for a given level of risk score.

Finally, using the full data set for maximum precision, we predicted one-, two-, five-, and ten-year mortality again using the gamma survival model with VACS-CCI score as the only predictor. Upon publication, the score calculation and mortality prediction will be available online via MDCalc. We will also collaborate with VHA leadership to provide automatic calculation in the VHA EHR.

VACS Index and VACS-CCI in PWH

VACS-CCI and VACS Index 2.0 were calculated for each person at their index date. We used Cox proportional hazard models to predict 10-year mortality with predictors as follows: (1) age, (2) CCI components plus age, (3) VACS Index 2.0 score as published (add reference), (4) VACS-CCI as created above, (5) VACS-CCI plus CD4 and HIV-RNA viral load (VL). Model performance was assessed using Akaike’s information criterion (AIC, lower is better) for model fit and Harrell’s C-statistic (range 0.5–1.0, higher is better) for discrimination. Additionally, 10-year mortality predictions based on VACS Index and VACS-CCI were compared graphically with observed mortality for CD4 (<200, 200–499, 500+) and HIV RNA-1 VL subgroups. Data points of observed mortality were plotted when there was a minimum of 10 deaths and 5 survivors for a given level of risk score. We also compared prevalence of comorbid disease groups from the CCI by CD4 groups (<200, 200–499, 500+) using tests for trend.

Results

Patient characteristics in the general PWoH and PWH VHA patients

We identified 6,062,932 PWoH for development (years 2007–2011 and 2013–2017), 525,756 for validation (year 2012), and 30,539 PWH (years 2007–2017) (Table 1). Demographics, laboratory values, and frequency of comorbidities comprising the CCI were similar between the development and validation samples. Compared to the sample of 6,588,688 PWoH in the development and validation samples, the PWH sample was younger (median age 55 vs. 65 years), less likely to be female (3% vs. 6%), more likely to be non-Hispanic Black (48% vs. 15%), had lower median VACS Index score (47 vs. 69), had a higher proportion with elevated FIB-4, HCV, other liver disease as defined by CCI, and lower proportion with diabetes. Over half of the randomly selected index dates were in 2013–2017 for the development and validation (52%) and PWH (55%) samples. Median observation time was 6.2 years (IQR = 4.4–8.8) in development, 9.4 years (IQR = 5.5–9.8) in validation, and 6.5 years (IQR = 4.6–9.5) in PWH. After a maximum of 10 years follow-up there were 2,082,928 deaths in development, 203,953 in validation, and 9,005 in PWH. Additional characteristics are also shown in Table 1.

Model development in the general PWoH VHA patients

Compared to age alone (C-statistic = 0.740), discrimination for predicting 10-year mortality in the development dataset was better for CCI components plus age (C-statistic = 0.782), the VACS Index 2.0 as published (C-statistic = 0.794), and VACS Index 2.0 components reweighted (C-statistic = 0.798) (Figure 1). Combining CCI and VACS index components (using either categorical or continuous values) improved discrimination further (C-statistic = 0.811). Plots of mortality vs. risk score revealed poor calibration at extreme scores in younger patients and women. This was resolved by adding an age x CCI score interaction and sex to the model. The final VACS-CCI model (Table 2, Model 7) had the highest C-statistic (C-statistic = 0.814). Similar patterns of improvement were seen with AIC which dropped from 61,790 for CCI plus age to 60,660 for VACS-CCI.

Figure 1

Figure 1. Model fit (AIC) and discrimination (c-statistic) from series of models for 10-year, all-cause mortality using Cox regression in development sample.

Table 2

Table 2. VACS-CCI prognostic model developed in 6,062,932 U.S veterans with index visit in 2007–2011 or 2013–2017.

VACS-CCI risk score summary, discrimination, and calibration in the general PWoH VHA patients

Using the VACS-CCI risk score derived from Model 7 as the only predictor (1 variable rather than individual components), we observed minimal degradation of discrimination in development (C-statistic = 0.813) or validation (C-statistic = 0.810) datasets compared to the original (Model #7, C-statistic = 0.814). Observed 1-, 5-, and 10-year mortality showed good concordance with predicted mortality in development and validation datasets (Supplementary Figure S1). The exception was 1-year and 5-year mortality in those with scores greater than 70 (<0.5%), where the model over-predicted mortality. For context the mean VACS-CCI was 37 (SD = 13) in both the development and validation datasets.

In subgroup analysis there were more than 22,000 observations and 4,000 deaths in all groups, even among the smallest groups (Hispanics and women) in the validation dataset (Table 3). The proportion that died during 10 years of follow-up ranged from 11 to 74%; and was lowest in women and those under age 65, and highest in those with CCI scores of 3 or greater. Variation within subgroups was highest among women (COV = 49%) and considerably lower in the age 65 and older subgroup (COV = 20%) and those with higher CCI scores. Within subgroups, C-statistics were similar in development and validation samples. As expected, C-statistics were smaller when there was less variation in a subgroup. For example, for CCI scores of 0, 1, 2 and ≥ 3, C-statistics were 0.819, 0.767, 0.736, and 0.714, respectively (Supplementary Figure S2). Observed mortality was congruent with predicted mortality (from VACS-CCI) among all subgroups examined for both the development and validation datasets (Figure 2).

Table 3

Table 3. VACS-CCI summary statistics, and discrimination of 10-year mortality using VACS-CCI score as only predictor in a Cox model, in development and validation samples, overall and by subgroup.

Figure 2

Figure 2. Observed (open circles) and predicted (solid line) 10-year, all-cause mortality as a function of VACS-CCI score. 95% confidence intervals for observed mortality are very narrow and may be difficult to discern.

Translating the VACS-CCI into mortality risk estimates in the general PWoH VHA patients

In the combined development and validation samples VACS-CCI score was approximately normally distributed with mean 36.6 (SD 13.2) and median 37 (IQR 28–46); 80% of scores were between 17 and 54; 90% of scores were between 14 and 58. Predicted mortality in this sample (index visit 2007–2017) ranged from 0 to 100% for one-, two-, five-, and ten-year mortality (Figure 3; Supplementary Table S3). For example, for VACS-CCI score of 30, predicted mortality was 1% at 1 year, 3% at 2 years, 8% at 5 years, and 18% at 10 years. For a score of 45, predicted mortality was 6% at 1 year, 14% at 2 years, 35% at 5 years, and 61% at 10 years. Around the median score, a one-point increment in VACS-CCI score translated to approximately 10% increase in predicted mortality.

Figure 3

Figure 3. VACS-CCI predicted mortality based on general (PWoH) sample, 2007–2017.

VACS-CCI validation in PWH

Model discrimination for predicting 10-year mortality in the sample of PWH was better for the CCI components plus age (C-statistic = 0.699), VACS Index 2.0 as published (C-statistic = 0.774), VACS-CCI (C-statistic = 0.772) and the VACS-CCI plus CD4 and HIV RNA-1 VL (C-statistic = 0.785) compared to age alone (C-statistic = 0.637).

Plots of observed and predicted 10-year mortality (using both VACS Index and VACS-CCI to predict mortality) overall and by CD4 and HIV-RNA VL groups are shown in Figure 4. Observed mortality was generally congruent with predicted mortality among subgroups, except that VACS Index underestimated mortality for PWH with CD4 < 200.

Figure 4

Figure 4. Among PWH, observed (open circles) and predicted (solid line) 10-year, all-cause mortality as a function of VACS-CCI and VACS Index 2.0 risk scores. 95% confidence intervals for observed mortality are very narrow and may be difficult to discern.

To better understand the underestimated mortality for PWH CD4 < 200 using VACS Index, we compared conditions included in the CCI by CD4 count groups (<200, 200–499, 500+). Those with CD4 < 200 were more likely than those with higher CD4 to have many comorbidities including liver disease, kidney disease, malignancy, congestive heart failure, and dementia (p-trend <0.0001 for all, Figure 5).

Figure 5

Figure 5. Charlson comorbidity components by CD4 level among PWH.

Discussion

Among PWH and PWoH in VHA care, age alone weakly discriminates risk of mortality, and both VACS Index and the newly developed VACS-CCI provide much better 10-year mortality discrimination. Additionally, the VACS-CCI, which includes components of the VACS Index plus comorbidities included in the CCI, was more consistently calibrated than the VACS Index among important subgroups.

The VACS-CCI may be an important improved version of the VACS Index which has been widely used in HIV clinical research as an estimate of overall disease burden and in clinical settings to prioritize health screening and medical management. For example, colon cancer screening and active treatment for non-metastatic prostate cancer are both indicated in those expected to live more than 10 years. In contrast, end-of-life planning should be considered among those with a shorter life expectancy (7).

There are notable limitations to both the CCI and VACS Index components. For CCI, comorbidities are considered to be present only if the corresponding diagnostic code is present. While this is an inclusive approach, it can result in under estimation of risk for those with fewer clinical encounters. For example, a recent study of CCI as a predictor of mortality after COVID-19 found that CCI based on ICD codes assigned during a single inpatient stay substantially under-estimated risk of mortality compared to also including codes assigned at outpatient visits within VHA (28). In contrast, VACS Index is a less inclusive approach as it requires biomarkers to be present with only two assumptions and allowances regarding missing data. Missing values are assumed normal when only albumin is missing, or if only ALT or AST are missing. In the general sample of people engaged in care, 19% of otherwise eligible patients were excluded due to missing laboratory data. Future work will explore developing and validating algorithms for more systematic estimation of other missing values to minimize the proportion of patients excluded.

In addition to the issue of missing laboratory data, there were several other noteworthy limitations. First, while the VACS-CCI is ready for application within the national VA and has demonstrated accuracy overall and among 979,166 patients who are Black, 407,215 women, and 319,826 patients of Hispanic ethnicity, it will require validation outside VA prior to external application. Our prior experience with validating VACS Index among external samples of people with HIV suggests that this is likely to be successful. Second, additional work will be needed to help clinicians understand the major drivers of an individual patient’s risk and which of these might be modifiable. Similarly, if VACS-CCI is to be used as an outcome, we will need to explore the degree to which changes in estimated risk correspond to actual changes in individual risk. This need to reflect responsiveness to change is supported by our use of a random visit baseline, thus ensuring a diversity of observations with respect to stage of disease and age. Third, mental health, alcohol and substance use, and socio-economic factors are not included in the VACS-CCI. Nevertheless, we believe that medical harm from these factors typically operates through variables already included in VACS-CCI. For example, we have previously demonstrated that people who have unhealthy alcohol use, use injection drugs, or smoke cigarettes have higher VACS Index scores (15, 29). Lastly, although the calibration of VACS-CCI for Hispanic patients appeared to over-estimate mortality risk, we have not included race or ethnicity as a variable in the VACS-CCI. This decision is based on recommendations from multiple sources regarding predictive tools (30, 31). and inclusion of race would complicate the use of VACS-CCI internationally.

This study has many strengths. We included only variables that would be widely available in clinical practice and research databases and could also be accurately and reliably measured. The very large development and validation sample sizes allowed us to refine VACS-CCI to perform well in important subgroups, including women. Although only 6% of the sample, 400,000 women provided adequate power for us to discern poor calibration, which was resolved by adjusting for female sex. We found that the discriminating power could be enhanced by including both CCI score and indicators for the individual conditions. We also discovered an interesting interaction between age and CCI score in which for those with low CCI scores (0 or 1), greater age increased the associated HR whereas for those with higher CCI scores (3 or 4), greater age decreased the associated HR. In other words, when you have few serious comorbidities, increasing age is an important driver of prognosis. When you have several serious comorbidities, the associated increased risk of mortality is greater when you are younger. VACS-CCI will be made accessible for clinical and research purposes in the following ways. We will post the programming to calculate the score on GitHub. We will also collaborate with MDCalc to post a web-based calculator as with VACS Index 2.0 (see Footnote 1). Further, MDCalc is developing a direct implementation in electronic health record-based decision support systems which would obviate the need to enter data.

Among PWH, by comparing CCI components by CD4 groups, we found that conditions not captured by the VACS Index, such as malignancy, congestive heart failure, and dementia, were more common among people with CD4 < 200. Additionally, liver disease and kidney disease were more common among people with CD4 < 200; while liver and kidney disease would be captured by FIB-4 and eGFR, symptom and disease severity not reflected in lab values may be further captured by ICD diagnoses and may be important additional predictors of mortality. The dose response association of common comorbid diseases with lower CD4 count underscores the likely role of HIV in non-AIDS conditions and warrants further exploration.

Age alone should not be used to determine health screening, treatment or other recommendations for patients with and without HIV. VACS-CCI provides excellent discrimination and calibration for 10-year all-cause mortality in a general sample of over six million patients and over 30,000 PWH in VHA care. VACS-CCI is ready for application in clinical research and patient care within the general VHA patient population but will require external validation prior to external use. Future work will also include adding CD4 count and HIV RNA to the VACS-CCI for PWH and validating it in independent data.

Data availability statement

The data that support the findings of this study are not permitted to leave the VA firewall without a Data Use Agreement due to VA regulations. However, VA data are made freely available to researchers with an approved VA study protocol. For more information, please contact the VA Information Resource Center (VIReC) at VIRec@va.gov or the corresponding author.

Ethics statement

The studies involving humans were approved by Yale University and VA Connecticut Healthcare System institutional review boards. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants’ legal guardians/next of kin because the study utilized electronic health record data; there was no direct patient contact.

Author contributions

KM: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Resources. AJ: Conceptualization, Funding acquisition, Investigation, Methodology, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing. VM: Conceptualization, Investigation, Methodology, Resources, Writing – review & editing. MR-B: Conceptualization, Investigation, Resources, Writing – review & editing. RH: Data curation, Investigation, Resources, Writing – review & editing. KO: Conceptualization, Investigation, Resources, Writing – review & editing. SB: Conceptualization, Investigation, Resources, Writing – review & editing. KB: Funding acquisition, Investigation, Resources, Writing – review & editing. JT: Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Validation, Visualization, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by National Institutes of Health grants (P01 AA029545, U24 AA020794, U10 AA013566, U01 AA020790, U01 AA020795). VM has received funding support from Emory Center for AIDS Research (P30AI050409) for work related to this manuscript.

Conflict of interest

VM has received investigator-initiated research grants (to the institution) and consultation fees from Eli Lilly, Bayer, Gilead Sciences, Merck, and ViiV.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The views and opinions expressed in this manuscript are those of the authors and do not necessarily represent those of the Department of Veterans Affairs or the United States government. This work uses data provided by patients and collected by the VA as part of their care and support.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2023.1342466/full#supplementary-material

Footnotes

1. ^ https://www.mdcalc.com/calc/10402/veterans-aging-cohort-study-vacs-2.0-index

References

1. Althoff, KN, McGinnis, KA, Wyatt, CM, Freiberg, MS, Gilbert, C, Oursler, KK, et al. Comparison of risk and age at diagnosis of myocardial infarction, end-stage renal disease, and non-AIDS-defining cancer in HIV-infected versus uninfected adults. Clin Infect Dis. (2015) 60:627–38. doi: 10.1093/cid/ciu869

PubMed Abstract | Crossref Full Text | Google Scholar

2. Wong, C, Gange, SJ, Moore, RD, Justice, AC, Buchacz, K, Abraham, AG, et al. Multimorbidity among persons living with human immunodeficiency virus in the United States. Clin Infect Dis. (2018) 66:1230–8. doi: 10.1093/cid/cix998

PubMed Abstract | Crossref Full Text | Google Scholar

3. Hogg, RS, Eyawo, O, Collins, AB, Zhang, W, Jabbari, S, Hull, MW, et al. Health-adjusted life expectancy in HIV-positive and HIV-negative men and women in British Columbia, Canada: a population-based observational cohort study. The measurement of health adjusted life expectancy (HALE) on a population-based observational cohort of HIV-positive and negative men and women in British Columbia. Lancet HIV. (2017) 4:e270–6. doi: 10.1016/S2352-3018(17)30029-2

Crossref Full Text | Google Scholar

4. Park, LS, Tate, JP, Sigel, K, Brown, ST, Crothers, K, Gibert, C, et al. Association of Viral Suppression with Lower AIDS-defining and non-AIDS-defining Cancer incidence in HIV-infected veterans: a prospective cohort study. Ann Intern Med. (2018) 169:87–96. doi: 10.7326/M16-2094

PubMed Abstract | Crossref Full Text | Google Scholar

5. Althoff, KN, Chandran, A, Zhang, J, Arevalo, WM, Gange, SJ, Sterling, TR, et al. Life-expectancy disparities among adults with HIV in the United States and Canada: the impact of a reduction in drug- and alcohol-related deaths using the lives saved simulation model. Am J Epidemiol. (2019) 188:2097–109. doi: 10.1093/aje/kwz232

PubMed Abstract | Crossref Full Text | Google Scholar

6. Justice, AC, McGinnis, KA, Skanderson, M, Chang, CC, Gibert, CL, Goetz, MB, et al. Towards a combined prognostic index for survival in HIV infection: the role of 'non-HIV' biomarkers. HIV Med. (2010) 11:143–51. doi: 10.1111/j.1468-1293.2009.00757.x

PubMed Abstract | Crossref Full Text | Google Scholar

7. Tate, JP, Sterne, JAC, and Justice, AC. Veterans aging cohort S, the antiretroviral therapy cohort C. Albumin, white blood cell count, and body mass index improve discrimination of mortality in HIV-positive individuals. AIDS. (2019) 33:903–12. doi: 10.1097/QAD.0000000000002140

PubMed Abstract | Crossref Full Text | Google Scholar

8. McGinnis, KA, Justice, AC, Moore, RD, Silverberg, MJ, Althoff, KN, Karris, M, et al. Discrimination and calibration of the veterans aging cohort study index 2.0 for predicting mortality among people with human immunodeficiency virus in North America. Clin Infect Dis. (2022) 75:297–304. doi: 10.1093/cid/ciab883

PubMed Abstract | Crossref Full Text | Google Scholar

9. Akgun, KM, Gordon, K, Pisani, M, Fried, T, McGinnis, KA, Tate, JP, et al. Risk factors for hospitalization and medical intensive care unit (MICU) admission among HIV-infected veterans. J Acquir Immune Defic Syndr. (2013) 62:52–9. doi: 10.1097/QAI.0b013e318278f3fa

PubMed Abstract | Crossref Full Text | Google Scholar

10. Salinas, JL, Rentsch, C, Marconi, VC, Tate, J, Budoff, M, Butt, AA, et al. Baseline, time-updated, and cumulative HIV care metrics for predicting acute myocardial infarction and all-cause mortality. Clin Infect Dis. (2016) 63:1423–30. doi: 10.1093/cid/ciw564

PubMed Abstract | Crossref Full Text | Google Scholar

11. Womack, JA, Goulet, JL, Gibert, C, Brandt, CA, Skanderson, M, Gulanski, B, et al. Physiologic frailty and fragility fracture in HIV-infected male veterans. Clin Infect Dis. (2013) 56:1498–504. doi: 10.1093/cid/cit056

PubMed Abstract | Crossref Full Text | Google Scholar

12. Marquine, MJ, Montoya, JL, Umlauf, A, Fazeli, PL, Gouaux, B, Heaton, RK, et al. The veterans aging cohort study (VACS) index and neurocognitive change: a longitudinal study. Clin Infect Dis. (2016) 63:694–702. doi: 10.1093/cid/ciw328

PubMed Abstract | Crossref Full Text | Google Scholar

13. Marquine, MJ, Umlauf, A, Rooney, AS, Fazeli, PL, Gouaux, BD, Paul Woods, S, et al. The veterans aging cohort study index is associated with concurrent risk for neurocognitive impairment. J Acquir Immune Defic Syndr. (2014) 65:190–7. doi: 10.1097/QAI.0000000000000008

PubMed Abstract | Crossref Full Text | Google Scholar

14. Justice, AC, Modur, SP, Tate, JP, Althoff, KN, Jacobson, LP, Gebo, KA, et al. Predictive accuracy of the veterans aging cohort study index for mortality with HIV infection: a north American cross cohort analysis. J Acquir Immune Defic Syndr. (2013) 62:149–63. doi: 10.1097/QAI.0b013e31827df36c

PubMed Abstract | Crossref Full Text | Google Scholar

15. Justice, AC, McGinnis, KA, Tate, JP, Braithwaite, RS, Bryant, KJ, Cook, RL, et al. Risk of mortality and physiologic injury evident with lower alcohol exposure among HIV infected compared with uninfected men. Drug Alcohol Depend. (2016) 161:95–103. doi: 10.1016/j.drugalcdep.2016.01.017

PubMed Abstract | Crossref Full Text | Google Scholar

16. Deyo, RA, Cherkin, DC, and Ciol, MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. (1992) 45:613–9. doi: 10.1016/0895-4356(92)90133-8

PubMed Abstract | Crossref Full Text | Google Scholar

17. Fultz, SL, Skanderson, M, Mole, LA, Gandhi, N, Bryant, K, Crystal, S, et al. Development and verification of a "virtual" cohort using the national VA health information system. Med Care. (2006) 44:S25–30. doi: 10.1097/01.mlr.0000223670.00890.74

PubMed Abstract | Crossref Full Text | Google Scholar

18. Sohn, MWAN, Maynard, C, and Hynes, DM. Accuracy and completeness of mortality data in the Department of Veterans Affairs. Popul Health Metrics. (2006) 4:7954. doi: 10.1186/1478-7954-4-2

PubMed Abstract | Crossref Full Text | Google Scholar

19. Maynard, C. (2021) Ascertaining Veterans’ Vital Status: VA Data Sources for Mortality Ascertainment and Cause of Death. Available at: https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/3544-notes.pdf (Accessed March 23, 2021)

Google Scholar

20. Wyse, JJ, Robbins, JL, McGinnis, KA, Edelman, EJ, Gordon, AJ, Manhapra, A, et al. Predictors of timely opioid agonist treatment initiation among veterans with and without HIV. Drug Alcohol Depend. (2019) 198:70–5. doi: 10.1016/j.drugalcdep.2019.01.038

PubMed Abstract | Crossref Full Text | Google Scholar

21. Akgun, KM, Tate, JP, Crothers, K, Crystal, S, Leaf, DA, Womack, J, et al. An adapted frailty-related phenotype and the VACS index as predictors of hospitalization and mortality in HIV-infected and uninfected individuals. J Acquir Immune Defic Syndr. (2014) 67:397–404. doi: 10.1097/QAI.0000000000000341

PubMed Abstract | Crossref Full Text | Google Scholar

22. Sterling, RK, Lissen, E, Clumeck, N, Sola, R, Correa, MC, Montaner, J, et al. Development of a simple noninvasive index to predict significant fibrosis in patients with HIV/HCV coinfection. Hepatology. (2006) 43:1317–25. doi: 10.1002/hep.21178

PubMed Abstract | Crossref Full Text | Google Scholar

23. Inker, LA, Eneanya, ND, Coresh, J, Tighiouart, H, Wang, D, Sang, Y, et al. New creatinine- and cystatin C-based equations to estimate GFR without race. N Engl J Med. (2021) 385:1737–49. doi: 10.1056/NEJMoa2102953

PubMed Abstract | Crossref Full Text | Google Scholar

24. Quan, H, Sundararajan, V, Halfon, P, Fong, A, Burnand, B, Luthi, JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. (2005) 43:1130–9. doi: 10.1097/01.mlr.0000182534.19832.83

PubMed Abstract | Crossref Full Text | Google Scholar

25. Glasheen, WP, Cordier, T, Gumpina, R, Haugh, G, Davis, J, and Renda, A. Charlson comorbidity index: ICD-9 update and ICD-10 translation. Am Health Drug Benefits. (2019) 12:188–97.

PubMed Abstract | Google Scholar

26. Beyrer, J, Manjelievskaia, J, Bonafede, M, Lenhart, G, Nolot, S, Haldane, D, et al. Validation of an international classification of disease, 10th revision coding adaptation for the Charlson comorbidity index in United States healthcare claims data. Pharmacoepidemiol Drug Saf. (2021) 30:582–93. doi: 10.1002/pds.5204

PubMed Abstract | Crossref Full Text | Google Scholar

27. Akgun, KM, Krishnan, S, Tate, J, Bryant, K, Pisani, MA, Lo Re, V 3rd, et al. Delirium among people aging with and without HIV: role of alcohol and Neurocognitively active medications. J Am Geriatr Soc. (2023) 71:1861–72. doi: 10.1111/jgs.18265

PubMed Abstract | Crossref Full Text | Google Scholar

28. King, JT Jr, Yoon, JS, Rentsch, CT, Tate, JP, Park, LS, Kidwai-Khan, F, et al. Development and validation of a 30-day mortality index based on pre-existing medical administrative data from 13,323 COVID-19 patients: the veterans health administration COVID-19 (VACO) index. PLoS One. (2020) 15:e0241825. doi: 10.1371/journal.pone.0241825

PubMed Abstract | Crossref Full Text | Google Scholar

29. Williams, EC, McGinnis, KA, Tate, JP, Matson, TE, Rubinsky, AD, Bobb, JF, et al. HIV disease severity is sensitive to temporal changes in alcohol use: a National Study of VA patients with HIV. J Acquir Immune Defic Syndr. (2019) 81:448–55. doi: 10.1097/QAI.0000000000002049

PubMed Abstract | Crossref Full Text | Google Scholar

30. Wang, HE, Landers, M, Adams, R, Subbaswamy, A, Kharrazi, H, Gaskin, DJ, et al. A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models. J Am Med Inform Assoc. (2022) 29:1323–33. doi: 10.1093/jamia/ocac065

PubMed Abstract | Crossref Full Text | Google Scholar

31. Levey, AS, Stevens, LA, Schmid, CH, Zhang, YL, Castro, AF 3rd, Feldman, HI, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med. (2009) 150:604–12. doi: 10.7326/0003-4819-150-9-200905050-00006

PubMed Abstract | Crossref Full Text | Google Scholar