This study is a substudy of the retrospective multicenter cohort Brazilian COVID-19 Registry, conducted in 32 Brazilian hospitals, in 14 cities from five Brazilian states (Minas Gerais, Pernambuco, Rio Grande do Sul, Santa Catarina and São Paulo), described in detail elsewhere (6). The study was approved by the National Commission for Research Ethics (CAAE 30350820.5.1001.0008) and the individual informed consent was waived due to the pandemic circumstances and analysis of unidentified data.

The cohort study included consecutive adult patients (≥18 years-old) with laboratory-confirmed COVID-19, according to World Health Organization guidance (7), admitted in one of the participating hospitals. For the assessment of the ABC₂-SPH score and the comparison with other scores, patients admitted from July 31, 2020, to March 31, 2022, were included. For ABC₂-SPH score recalibration and also comparison with other scores, patients admitted from January 1, 2021, to March 31, 2022, were enrolled. However, for recalibration, only patients younger than 80 years were included, since mortality is particularly high for mechanical ventilation at an older age. This supported recommendations for conservative treatment for elderly and/or frail patients (8–10).

Patients with at least one of the following conditions were excluded: (i) pregnant women; (ii) “do not resuscitate” order; (iii) patients who manifested COVID-19 while admitted for other conditions; (iv) those transferred to other hospitals who had no defined outcome (discharged or death); (v) patients who were already on IMV at hospital presentation; and (vi) exclusively for score recalibration, patients ≥80 years old (Figure 1).

Medical records were reviewed to collect data concerning the patients’ characteristics, including age, sex, pre-existing comorbid medical conditions and medications taken at home; COVID-19-associated symptoms at hospital presentation; clinical assessment upon hospital presentation; laboratory results; inpatient medication, treatment, and outcomes. The data collection instrument was designed with reference to COVID-19 guidelines from the World Health Organization and the Brazilian Ministry of Health, as previously described (6).

A detailed guidance manual for data collection was developed, containing the definitions used in the study (Supplementary material). It was provided to all participating centers, and online training was mandatory before local research personnel were allowed to start collecting study data.

Data was collected by trained researchers from the medical records, using Research Electronic Data Capture (REDCap®) (version 7.3.1) (11, 12), hosted at the Telehealth Center of the University Hospital, of the Universidade Federal de Minas Gerais (13). To ensure reliability and monitor data, a code was developed in the R software that periodically verified possible data entry errors. When detected, the analysts notified the participating center for correction.

The primary outcome was IMV during hospitalization.

Model validation followed guidance from the Transparent Reporting of a Multivariable Prediction Model for Individual Prediction or Diagnosis (TRIPOD) checklist (14, 15) and the Prediction model Risk Of Bias Assessment Tool (PROBAST) (16). TRIPOD checklist ideally recommends at least 100 events (as deaths) and 100 non-events as samples for score validation. In the present analysis, the sample size was not calculated, since all patients eligible by the inclusion criteria were enrolled.

Continuous variables were described by medians and interquartile ranges (IQR) and categorical variables were represented by absolute and relative frequencies. Data were analyzed with R software (version 4.0.2), using mice (function mice), pROC (functions roc, ci.auc, roc.test), glmnet (function cv.glmnet), tidyverse (dplyr functions), gtsummary (function tbl_summary) and ggplot2 packages. value of ps <0.05 were considered statistically significant.

The statistical analysis was divided into two stages: (i) evaluation of the ABC₂-SPH risk score for predicting IMV in COVID-19 patients and comparison with other available scores; and (ii) recalibration of the ABC₂-SPH score, as well as comparison with other scores.

Discrimination of the ABC₂-SPH score was compared to other existing scores, including CALL (17), COVID-IRS (18), CURB-65 (19), PREDI-CO (20), SOFA (21), STSS (22), SUM (23) and 4C Mortality Score (24). The scores were chosen based on the two conditions: (i) parameters available within the Registry’s database, and (ii) accessible methods for calculation.

The main characteristics of the scores are listed in Supplementary Table S1. The comparison of ABC₂-SPH (5) with other scores (17–24) was performed using the number of complete cases for each score (non-imputed database) through a procedure for unpaired receiving operator characteristic (ROC) curves that is an extension of Delong et al. recommendations (25). This procedure was implemented in the pROC package, function “roc.test.” Due to the multiple comparisons, alpha was corrected using the Bonferroni method.

The score was recalibrated, in an attempt to improve the prediction risk of IMV among patients with COVID-19. The sample of patients included COVID-19 patients under 80 years of age, divided into derivation (from January 1 to April 30, 2021) and validation cohorts (from May 1, 2021, to March 31, 2022), resulting in approximately 75 and 25% of the sample, respectively. This division guarantees the minimum of 100 events in the validation cohort, as recommended by the TRIPOD checklist (14, 15).

The recalibration methods consisted of fitting a logistic regression model [for more details, see Steyerberg et al. (26)] in the derivation sample and the evaluation of the method was done in the validation sample.

To handle missing values, multiple imputation by chained equations (MICE) was used, considering missing at random assumption. The imputation technique included all variables with up to 30% missing values. The prediction of missing values was performed using all variables included in the analysis. Invasive mechanical ventilation was not imputed, and was not used as a predictor in the MICE model in the validation dataset. The predictive mean matching (PMM) method was used for continuous predictors and polytomous regression for categorical variables. Ten imputed datasets were obtained with 10 iterations, and their results were combined following Rubin’s rules (27).

Model’s discrimination was assessed by the area under the ROC curve (AUROC), with 95% confidence interval (95% CI) calculated by bootstrap resampling, through 2,000 samples. A value of 0.5 indicates no predictive ability, 0.60 to 0.69 is considered poor, 0.70 to 0.89 good, and 0.90 to 1.0 excellent (28).

The accuracy of the predictive model was assessed using the Brier score, a measure that quantifies how close predictions are to the truth (29). The score ranges between 0 and 1, in which smaller values indicate superior model performance. Results were stratified by age groups (<60, 60–69, 70–79 and ≥ 80 years-old), sex and presence or absence of key comorbidities before recalibration, to assess score performance in different subgroups.

Calibration was assessed graphically by plotting the predicted IMV probabilities against the observed IMV, testing intercept equals zero and slope equals one, simultaneously.

Overall, 9,350 patients were included in the study, the median age was 58.5 (IQR 47.0–69.0) years old, and 45.4% were women. Of those, 33.5% were admitted to the ICU, 25.2% received IMV, and 17.8% died. Patients who received IMV were older; had a higher frequency of hypertension, diabetes, obesity, chronic kidney disease, rheumatologic disease and previous transplant; a higher number of comorbidities; and a higher frequency of ICU, dialysis, thromboembolism and mortality, when compared to those who did not receive IMV (Table 1). They also had a higher frequency dyspnea, cough, fever, nausea, and arthralgia; clinical findings such as fever, tachycardia and arterial hypotension (Supplementary Table S4) and laboratory findings such as neutrophilia, lymphopenia, thrombocytopenia and increased lactate, D-dimer and C-reactive protein, when compared to those who did not receive IMV (Supplementary Tables S4, S5).

The AUROC for the ABC₂-SPH 0.677 (0.661–0.694), and the Brier score 0.196. Subject-specific risks were calculated, and patients were classified according to ABC₂-SPH risk groups (Table 2). Score’s performance was worse among older patients, especially the octogenarians, and patients with chronic pulmonary obstructive disease (Supplementary Table S1).

For the comparison with other scores, the main characteristics of each score are shown in Supplementary Table S2. When compared with other scores in a complete case analysis, the ABC₂-SPH score achieved a significantly higher discriminatory capacity than CURB-65, STSS, and SUM scores (Table 3; Figure 2A). When assessing specifically the sample < 80 years, ABC₂-SPH score still achieved a significantly higher discriminatory capacity than CURB-65, STSS, SOFA and SUM scores (Supplementary Table S3).

The calibration curve indicates that the ABC₂-SPH underestimated IMV at lower ranges of the score and overestimated it at the higher ones, as observed in Figure 2B (slope = 0.557, intercept = −0.097, value of p < 0.001).

When assessing specifically the sample of patients used for score recalibration (<80 years-old admitted to hospital with COVID-19, from January 1, 2021, to March 31, 2022), patients from the validation cohort had a slightly lower age, frequency of hypertension and inotropic requirement; a slightly higher frequency of atrial fibrillation and COPD; a higher frequency of smoking and a lower frequency of outcomes than the derivation cohort (Table 4; Supplementary Table S6). As for laboratory findings, there were no clinically relevant differences (Supplementary Table S7).

When assessing score performance in this sample before calibration (Table 5), the AUROC for ABC₂-SPH was superior to the assessed scores. The recalibrated ABC₂-SPH score, named as ABC₂-SPHr score, obtained good overall performance (Brier score = 0.132) and calibration (slope = 1.048, intercept = 0.378, value of p < 0.001) (Figure 3) in the validation subsample.

Our study contributes to the literature because it is a multicenter study, with a large sample of patients from 32 Brazilian hospitals (including public, private, and philanthropic), from different regions and degrees of complexity, which validated and recalibrated the ABC₂-SPH score for prediction of IMV in COVID-19 patients under the age of 80. Additionally, we included comparisons with existing risk stratification scores, ensuring superior performance to the CURB-65 (19), SOFA (21), STSS (22), and SUM (23) scores.

Even with these multiple strengths, The present study presents limitations that should Be addressed. Despite The fact that All hospitals referred there Was adequate supply of IMV during The study period, We cannot assure that All patients Who required IMV In fact received IMV. Therefore, The outcome for this analysis Was receipt of IMV, Not IMV requirement. That Is also Why We opted To recalibrate The score excluding The sample of octogenarians, As frequently doctors have conservative treatment for elderly and/or frail patients, which includes avoiding intubation, and this could Be observed By a worse AUROC curve In this stratum. Additionally, The scores were calculated based On data from a retrospective, observational, and non-randomized study, with data collected from medical records. Therefore, some variables were Not found uniformly, generating missing data. However, In order To reduce this impact, Our data were collected By researchers with extensive training and accompanied closely By a professional with important experience In research. Furthermore, information that depends On a more accurate clinical history, such As The description of comorbidities and details of symptoms, may Not have been obtained.

Like other viruses, SARS-CoV-2 evolves over time. The majority of mutations in the SARS-CoV-2 genome have no impact on viral function, but certain variants have garnered widespread attention because of their rapid emergence within populations and evidence for transmission or clinical implications. These are considered variants of concern. The World Health Organization (WHO) has also designated labels for notable variants based on the Greek alphabet: Alpha, Beta Gamma Delta and Omicron (46). The omicron variant and its sublineage have been increasing in prevalence worldwide (47). In August 2023, the World Health Organization classified the EG.5 coronavirus strain as a “variant of interest,” although it did not seem to add public health risks relative to the other currently circulating Omicron descendent lineages (48). So, the current global epidemiology of SARS-CoV-2 is characterized by the continued spread of the Omicon variant. These findings underscore the importance of vaccination to prevent both moderate and severe COVID-19 and to reduce the circulating variant (49). Currently, in the world, around 70% of persons are vaccinated with at least one dose, of a total of 13.3 billion doses administered globally, but there is still great vaccine inequality between countries (30, 50). Therefore, the future severity of the pandemic is not yet known.

As COVID-19 is a dynamic disease, further assessments in the model are required. The outbreak of COVID-19 was accompanied by an unprecedented explosion of scientific evidence, and a living review has found almost 600 prognostic models to predict diverse outcomes in patients with confirmed COVID-19 (51). In the aforementioned systematic review on the methodology of prediction models, Binuya et al. (45) have discussed that the incessant de novo derivation of models instead of refinement of an existing one is a widely recognized issue, and a huge waste of information from previous modeling studies (and we could infer, also a waste of time and money). If a reasonable prediction model is available and produces accurate estimates, the consensus is to build upon such a model and check whether some adjustments (“model updating”) may improve its fit or performance in new data, for example, with recalibration or incorporating a novel marker into the model (45). Thus, further studies should take this into account.