Machine learning-based approach to guide the choice between baricitinib and tocilizumab in critical COVID-19 pneumonia treatment: a retrospective cohort study

Abstract

Introduction:

Clear guidance on choosing baricitinib (BCT) or tocilizumab (TCZ) for critical COVID-19 pneumonia remains limited. We developed machine-learning (ML) models to inform immunomodulator selection.

Methods:

We curated clinical data from patients with critical COVID-19 pneumonia admitted between January 2020 and June 2024. Development cohort (n = 390) was split 4:1 into training and validation sets, with Day-90 mortality as endpoint. For each therapy, patients were labeled high risk when model-predicted mortality exceeded F1-optimized thresholds. External validation used a test cohort (n = 95). A combinatorial risk stratification assigned patients to four groups: I (low risk for both), II (low risk for TCZ, high risk for BCT), III (high risk for TCZ, low risk for BCT), and IV (high risk for both). Survival was compared for TCZ- and BCT-treated patients within each group.

Results:

TCZ and BCT models achieved ROC-AUCs of 0.81 and 0.84, with test accuracies of 0.67 and 0.77, respectively. In test cohort, survival differed significantly between high- and low-risk strata for each agent. In Group II, mortality was significantly higher with BCT than TCZ (hazard ratio (HR) 2.32, p = 0.032); in Group III, mortality was significantly higher with TCZ than BCT (HR 3.34, p < 0.001). Model-guided selection would have changed therapy in 13.2% (65/492) of patients; as the models are prognostic rather than causal, any survival benefit from the alternative agent remains hypothetical.

Conclusion:

ML models may support treatment selection between BCT and TCZ in patients with critical COVID-19 pneumonia. Prospective studies are warranted to assess whether model-guided choices improve survival and to validate generalizability across clinical settings.

1 Introduction

Coronavirus disease 2019 (COVID-19) remains one of the leading causes of global morbidity and mortality, accounting for an estimated 420 million disability-adjusted life years in 2020-2021 (1). More than seven million deaths were reported worldwide between December 2019 and December 2024 (2). Although disease severity has generally declined with successive variants, COVID-19 continues to cause substantial mortality among the elderly and immunocompromised patients (3, 4). Mortality associated with COVID-19 is primarily attributed to critical COVID-19 pneumonia, which often requires oxygen supplementation through high-flow nasal cannula (HFNC) devices, noninvasive ventilation (NIV), mechanical ventilation (MV), or extracorporeal membrane oxygenation (ECMO) (5). Consequently, effective treatment for critical COVID-19 patients is essential to reduce mortality and improve global health outcomes.

The progression of critical COVID-19 pneumonia is driven by excessive secretion of inflammatory cytokines along with viral replication (6). Immunomodulatory agents such as baricitinib (BCT) and tocilizumab (TCZ), in conjunction with dexamethasone and remdesivir, are recommended for treating critically ill COVID-19 patients (5). Both BCT and TCZ alleviate hyperinflammation processes during COVID-19 pneumonia by inhibiting Janus kinase (JAK) 1/2 and interleukin-6 (IL-6) receptors, which drive and modulate inflammatory signaling pathways (7, 8). Several randomized controlled trials have demonstrated that BCT and TCZ can significantly reduce disease progression and mortality in critically ill COVID-19 patients (9–11). Thus, the combination of immunomodulators such as BCT and TCZ with dexamethasone is considered superior to relying solely on antiviral agents like remdesivir for treating critical COVID-19 pneumonia.

However, no clear guidelines currently exist for selecting between BCT and TCZ when treating critical COVID-19 pneumonia. Although previous studies have compared their efficacy, no significant differences between the two have been reported (12–14). One recent retrospective study found that BCT was associated with lower risks of mortality and intubation after adjustment for confounders, but randomized trials are still needed to confirm these findings (15). Consequently, the selection of immunomodulators often relies on clinicians’ experience, patient comorbidities, and current clinical status. Although this approach is valuable, it remains uncertain whether this method effectively improves survival outcomes in patients with critical COVID-19 pneumonia. Personalized therapeutic approaches are therefore needed to guide the selection of either BCT or TCZ in a manner that optimizes survival.

Machine learning (ML) offers a promising approach for personalizing treatment by analyzing not only physicians’ clinical experience and patient data but also by identifying complex, high-dimensional patterns that may not be immediately apparent to physicians (16). Several studies have demonstrated the utility of ML in developing personalized treatment strategies across various clinical settings (17, 18). Additionally, ML has been employed to accurately diagnose COVID-19 infections and to predict their severity and progression (19, 20). However, few studies have explored the use of ML approaches in guiding the selection of immunomodulators for treating critically ill COVID-19 patients.

This study aims to develop an ML model to assist in selecting between BCT and TCZ in patients with critical COVID-19 pneumonia. By predicting treatment responses and outcomes, this model aims to improve survival outcomes through personalized treatment recommendations. Using the ML-based approach, we estimate survival rates when patients were treated with BCT or TCZ. We also apply ML-based risk stratification to group patients, predict the most appropriate immunomodulator for each group, and assess improvements in survival rates.

2 Methods

2.1 Patient selection and treatments

This retrospective study included adults aged 18-80 admitted to Asan Medical Center (AMC) in the Republic of Korea from January 2020 to June 2024, with critical COVID-19 pneumonia requiring HFNC, NIV, MV, or ECMO (5). Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection was confirmed by nasopharyngeal polymerase chain reaction or rapid antigen testing. Patients received either BCT or TCZ in addition to standard of care (SOC) and were followed for 90 days after initiation of immunomodulators.

During the study period, three variants of concern successively predominated in the Republic of Korea: Alpha (January 2020–July 2021), Delta (August 2021–January 2022), and Omicron (February 2022–June 2024). Treatments were administered in a non-randomized manner and were dependent on guidelines and drug availability (5). SOC for critical COVID-19 pneumonia included appropriate respiratory support based on disease severity, remdesivir, dexamethasone, low-molecular-weight heparin, and immunomodulators such as BCT or TCZ. Remdesivir became available in the Republic of Korea in July 2020 and dexamethasone was introduced into SOC for patients requiring oxygen support in July 2020, following results from a large-scale randomized controlled trial (21).

BCT or TCZ was administered to patients with rapidly deteriorating severe or critical COVID-19 pneumonia. BCT was approved for the treatment of COVID-19 pneumonia in April 2021, while TCZ was approved in March 2022. Immunomodulator choice was individualized based on clinical status and physician preference; TCZ was favored for chronic kidney disease or dysphagia, whereas BCT was preferred when bacterial/fungal coinfection was suspected, given long half-life of TCZ. Contraindications for each agent are listed in Supplementary Table 1 (22, 23).

2.2 Development of machine-learning models

Clinical data from the development cohort were extracted from the Asan Biomedical Research Environment (ABLE) system at AMC. A total of 16 underlying diseases and 38 clinical characteristics (Supplementary Methods) obtained around admission and immunomodulator initiation were included as features for the ML models predicting outcomes following BCT or TCZ treatment. The corresponding dataset for the independent test cohort was obtained from Hanyang University Medical Center (HYUMC) in the Republic of Korea. This study was approved by the Institutional Review Boards (IRB) of AMC (IRB No. 2021-0024) and HYUMC (IRB No. 2023-09-037), with an informed consent waived.

The datasets comprised both categorical and numerical variables. Within the development cohort, the dataset, consisting of input variables and treatment outcomes, was randomly split into a training set and an internal validation set in a 4:1 ratio, in a stratified manner with respect to Day-90 mortality. This internal validation set was used for final model performance assessment before external validation. The HYUMC cohort was reserved as an independent external test set. Categorical variables were transformed into numerical values through label encoding, while numerical variables were preprocessed using the MinMax scaler (24). Missing values (proportions described in Supplementary Table 2) were imputed using the median of the non-missing values in the training set for each variable. The approach was chosen after repeating the entire model-development pipeline under alternative imputation strategies—minimum, maximum, kNN, and MICE (25, 26).

The ML models were trained to predict Day-90 mortality under BCT or TCZ treatment. Model development, including feature selection and hyperparameter tuning, was performed exclusively within the primary training set. A minimal feature subset was first identified using permutation feature importance with an artificial neural network (27, 28). Subsequently, to robustly evaluate candidate feature sets and optimize model hyperparameters, we employed recursive feature elimination. This process utilized repeated stratified 5-fold cross-validation (100 resampling iterations per combination of features/hyperparameters), and the combination yielding the highest mean F1-score across the cross-validation folds was selected (29). Final models were then refitted on the entire training set using the selected features and hyperparameters and subsequently evaluated on the previously unseen HYUMC test cohort.

2.3 Treatment-specific risk stratification based on machine-learning models

Machine-learning models were employed to stratify patients into treatment-specific risk groups. TCZ-treated patients were stratified by predicted Day-90 mortality using both the TCZ-specific model (actual treatment) and the BCT-specific model (had they received baricitinib instead); the same dual stratification was applied to BCT-treated patients. The probabilities computed by the trained models were calibrated (Supplementary Figure 1) using the method that employs the Brier score decomposition (30). The threshold probability for each stratification was determined by maximizing the F1-score. Patients with a computed probability of Day-90 mortality under a specific immunomodulator exceeding the threshold were classified as high risk for the corresponding treatment.

Using this methodology, we classified the entire AMC critical COVID-19 cohort, including the development cohort and those excluded due to missing follow-up data, into four combinatorial risk subgroups: Group I (TCZ^LRBCT^LR) comprised patients with low risk for both immunomodulators; Group II (TCZ^LRBCT^HR) included patients at low risk for TCZ and high risk for BCT; Group III (TCZ^HRBCT^LR) contained patients at high risk for TCZ and low risk for BCT; and Group IV (TCZ^HRBCT^HR) comprised patients at high risk for both treatments. Next, we compared overall survival (OS) between BCT- and TCZ-treated patients within each group. Additionally, we classified the entire AMC critical COVID-19 cohort into two subgroups based solely on mortality risk for BCT or TCZ. Additionally, OS was compared between BCT and TCZ in the BCT^LR and TCZ^LR subgroups, respectively.

2.4 External validation of the models’ predictive performance

We evaluated model performance on the HYUMC test cohort, which included critical COVID-19 pneumonia patients treated with BCT or TCZ and with available Day-90 follow-up data. Predictive accuracy for Day-90 mortality was calculated using decision thresholds derived from the development cohort, optimized for the F1-score during cross-validation. Discrimination was assessed using the area under the receiver operating characteristic curve (ROC–AUC) and the area under the precision–recall curve (AUPRC), given the imbalance between survivors and non-survivors. For ROC–AUC, AUPRC, and accuracy, 95% confidence intervals (CIs) were estimated by stratified bootstrap resampling (1,000 replicates). Using the entire HYUMC critical COVID-19 cohort—including those without follow-up data, treated as right-censored—we estimated hazard ratios (HRs) to compare OS between BCT- and TCZ-treated patients within each risk subgroup. For these analyses, each patient was classified as low- or high-risk for each treatment (BCT and TCZ) according to the model-predicted probability of Day-90 mortality.

2.5 Statistical analysis

Survival analysis was performed using the Kaplan–Meier method, with differences in survival rates assessed via the log-rank test. Unlike model development, right-censored patient data were included in Kaplan–Meier analyses. The Cox proportional hazards model was employed to estimate HRs in multivariate analysis, with p-values calculated using the likelihood method. We verified the proportional hazards assumption using Schoenfeld residuals and found no violations. The lifelines package was used for Cox proportional hazards modeling. For subgroup comparisons, categorical and continuous variables were compared using chi-squared, Fisher’s exact tests, t-tests, or Mann–Whitney U tests, as appropriate. Statistical significance was defined as p < 0.05. All statistical analyses were conducted using the SciPy package, while Python (version 3.11.9) was used for implementation. For model performance, ROC–AUC, AUPRC, and accuracy were accompanied by 95% CIs derived from stratified bootstrap resampling with 1,000 replicates.

3 Results

3.1 Patient characteristics

Records of 6,388 COVID-19 patients admitted to AMC were reviewed; 492 critically ill patients treated with BCT or TCZ comprised the entire AMC critical COVID-19 cohort. After excluding 102 without baseline or follow-up data up to Day 90 post-initiation of immunomodulators, the development cohort included 390 patients (TCZ 236; BCT 154). The development cohort was further stratified into training and validation sets in a 4:1 ratio (Figure 1a). At HYUMC, 159 critically ill patients treated with BCT or TCZ were identified from 337 admissions and comprised the entire HYUMC critical COVID-19 cohort; excluding 64 without Day-90 data yielded a 95-patient test cohort (TCZ 12; BCT 83) (Figure 1b).

Figure 1

The demographic and clinical characteristics of the development and test cohorts are presented in Table 1. The test cohort includes only variables used in the model. The test cohort was older (median age 71), whereas the development cohort had more patients with solid malignancies or solid-organ transplants. Diastolic blood pressure (DBP), respiratory rate, and most laboratory values were comparable between the two cohorts. Within the development cohort, leukocyte count, hemoglobin, and creatinine did not differ between TCZ and BCT groups despite BCT contraindications in neutropenia, severe anemia, and chronic kidney disease (Supplementary Table 1).

Mortality was higher in the test than in the development cohort and was consistently higher in TCZ- than BCT-treated patients (development: 28.4% vs. 23.4%; test: 50.0% vs. 34.1%). The proportion of patients receiving remdesivir was similar in the entire AMC (n = 492) and HYUMC (n = 159) critical COVID-19 cohorts (88.3 and 85.7%, respectively). Similarly, corticosteroid use, such as dexamethasone and methylprednisolone, was comparable between the two centers (87.6% vs. 86.4%, respectively).

3.2 ML-based mortality prediction and risk stratification

The development cohort (n = 390) was split into training (n = 311) and internal validation (n = 79) sets in a 4:1 ratio. Separate ML models were trained to predict Day-90 mortality for TCZ-treated (n = 188) and BCT-treated (n = 123) patients using the training set. Recursive feature elimination with cross-validation (RFECV) identified eight features with the highest predicted performances in both TCZ and BCT ML models (Supplementary Figure 2). For TCZ model, key covariates were age, weight, DBP, blood urea nitrogen (BUN), creatinine, hemoglobin, platelets, and albumin to predict Day-90 mortality in TCZ-treated patients, achieving an ROC-AUC of 0.81 (95% CI, 0.74–0.87) during internal validation (Figure 2a). The accuracy was 0.67 when applied to the TCZ-treated patients (n = 48) in the internal validation set. The AUPRC in the internal validation set was 0.65 (95% CI, 0.52–0.77), and the precision–recall curve is shown in Supplementary Figure 3. Similarly, for BCT model, age, weight, heart rate, respiratory rate, potassium, serum creatinine, hemoglobin, and albumin were selected to predict Day-90 mortality in BCT-treated patients, achieving an ROC-AUC of 0.84 (95% CI, 0.75–93) and AUPRC of 0.74 (95% CI, 0.57–0.87) during internal validation (Figure 2b). Accuracy was 0.77 when applied to the BCT-treated patients (n = 31) in the internal validation set. Model performance did not vary significantly with the way missing values were handled (Supplementary Table 3).

Figure 2

In the development cohort, baseline clinical characteristics and differences between TCZ^HR and TCZ^LR (among TCZ-treated patients) and between BCT^HR and BCT^LR (among BCT-treated patients) are summarized in Tables 2, 3. In the TCZ^HR subgroup, the median age, proportions of patients with chronic kidney disease or malignancies, and levels of procalcitonin, lactic acid, and BUN were significantly higher compared to the TCZ^LR subgroup. Conversely, the median weight, DBP, and hemoglobin, platelets, albumin, and bicarbonate levels were all significantly lower in the TCZ^HR subgroup compared to the TCZ^LR subgroup.

Similarly, in the BCT^HR subgroup, the median age, proportions of patients with cardiovascular disease or solid malignancies, heart rate, respiratory rate, and C-reactive protein levels were significantly higher than in the BCT^LR subgroup, while median weight and levels of hemoglobin, platelets, and albumin were significantly lower. Severe comorbidities, including malignancies and solid organ transplantation, are well known to significantly impact overall mortality (31). In multivariable analyses, the ML-estimated risk was the strongest predictor of OS, compared to other features, including severe comorbidities (Supplementary Figure 4).

Applying stratification based on the TCZ- and BCT-specific model–estimated survival probabilities to the entire AMC cohort, we classified TCZ-treated patients (n = 301) as TCZ^HR (n = 103) or TCZ^LR (n = 198) and BCT-treated patients (n = 191) as BCT^HR (n = 61) or BCT^LR (n = 130). Overall mortality was significantly higher in the high-risk groups than in their corresponding low-risk groups (TCZ^HR vs. TCZ^LR: HR 3.28, 95% CI 2.41–4.48, p < 0.001; BCT^HR vs. BCT^LR: HR 3.58, 95% CI 2.39–5.37, p < 0.001). In the HYUMC cohort, BCT-treated patients (n = 143) showed an HR of 2.09 (95% CI 1.06–4.12; p = 0.016) between high- and low-risk groups. Among TCZ-treated patients (n = 16), the HR was 6.32 (95% CI, 3.18–12.60; p = 0.018), but the small sample size limits the interpretability of this result (Figure 2c).

3.3 Combinatorial risk stratification

Using combinatorial risk stratification, we classified the entire AMC critical COVID-19 cohort (n = 492) into the four groups: Group I (n = 264), Group II (n = 50), Group III (n = 79), and Group IV (n = 99) (Figure 3a). Overall mortality did not differ between TCZ- and BCT-treated patients in Group I (HR 1.15, 95% CI 0.79–1.66; p = 0.490) or Group IV (HR 1.21, 95% CI 0.78–1.89; p = 0.407). In Group II, BCT-treated patients had significantly higher mortality than TCZ-treated patients (HR 2.32, 95% CI 1.05–5.13; p = 0.032), whereas in Group III, TCZ-treated patients had significantly higher mortality than BCT-treated patients (HR 3.34, 95% CI 1.94–5.76; p < 0.001) (Figure 3b). Because Group II had relatively few patients and an imbalance in the number of TCZ- and BCT-treated cases, the observed excess mortality with BCT should be interpreted cautiously. The HYUMC cohort (n = 159) showed similar directions of effect, although Group II was underpowered due to small sample size (Supplementary Figure 5).

Figure 3

Although ML models are prognostic only and cannot definitively determine counterfactual outcomes with the alternative immunomodulator, these findings suggest that model-guided choice might have improved the outcome in 65/492 patients (13.2%): 16 BCT-treated patients in Group II and 49 TCZ-treated patients in Group III. Among those who received an immunomodulator despite being at high risk for that specific agent, 49/103 TCZ^HR patients treated with TCZ (47.6%) and 16/61 BCT^HR patients treated with BCT (26.2%) might have had different outcomes with the alternative agent.

Additionally, using risk stratification based solely on mortality risk for BCT or TCZ, 70% of patients in the entire AMC critical COVID-19 cohort (n = 492) were categorized into the BCT^LR subgroup (n = 343) and 64% into the TCZ^LR subgroup (n = 314). In the BCT^LR subgroup, TCZ-treated patients had significantly higher mortality than BCT-treated patients (HR 1.61, 95% CI 1.18–2.18; p = 0.007) (Supplementary Figure 6A). In the TCZ^LR subgroup, mortality was similar (HR 1.05, 95% CI 0.74–1.49; p = 0.786). Similar trends were observed in the BCT^LR subgroup (n = 135) and the TCZ^LR subgroup (n = 111) from the HYUMC critical COVID-19 cohort (n = 159), although no significant difference in overall mortality was detected in the BCT^LR or TCZ^LR subgroups due to the limited number of TCZ-treated patients (Supplementary Figure 6B).

4 Discussion

In this study, we developed and externally validated two separate machine-learning models to predict Day-90 mortality in critically ill COVID-19 patients treated with BCT or TCZ. Using only readily available clinical variables, the models achieved acceptable discrimination (ROC-AUC 0.81–0.84 in internal validation) and identified distinct risk profiles for each immunomodulator. When combinatorial risk stratification was applied to the entire AMC cohort (n = 492), 13.2% of patients received an immunomodulator for which they were classified as high-risk while being low-risk for the other agent. Survival analyses suggested that model-guided selection of the lower-risk agent in these patients could have altered outcomes, although these models are prognostic rather than causal and counterfactual outcomes under the alternative therapy cannot be definitively established. These findings provide the first treatment-specific prognostic framework for choosing between BCT and TCZ in critically ill patients requiring HFNC, NIV, MV, or ECMO.

Previous randomized and observational studies comparing BCT and TCZ have reported conflicting results, with some showing similar efficacy, others favoring BCT in invasively ventilated patients, and still others demonstrating non-inferiority of BCT (12, 13, 15, 32). However, these studies compared overall treatment effects and focused on specific subgroups of patients with severe or critical COVID-19 pneumonia (e.g., MV/ECMO or HFNC/NIV patients). In contrast, our treatment-specific modeling approach, applied across the full spectrum of critically ill COVID-19 pneumonia patients, reveals that the relative benefit of BCT versus TCZ is dependent on baseline clinical characteristics. This framework explains much of the apparent discrepancy across prior studies and provides a practical method for personalizing immunomodulator selection.

The clinical features selected by our ML models were highly consistent with established prognostic factors in critical illness. Both TCZ and BCT models included age, weight, creatinine, hemoglobin, and albumin as clinical variables. In the TCZ model, older age, elevated BUN, and thrombocytopenia were significantly associated with higher mortality. In the BCT model, older age, tachycardia, tachypnea, anemia, and hypoalbuminemia were significantly associated with higher mortality (Supplementary Figure 7). These features align closely with components of validated severity scores [e.g., SOFA platelet component (33); NEWS/NEWS2 heart and respiratory rates (34, 35)] and known prognostic markers such as age, BUN, anemia, and hypoalbuminemia (36–39). The observed differences between the two models likely reflect their distinct pharmacologic mechanisms of action and the heterogeneous inflammatory phenotypes encountered in critical COVID-19.

Notably, BCT conferred a significant survival benefit only in patients classified as low-risk for BCT (BCT^LR subgroup), whereas no corresponding benefit was observed for TCZ in the TCZ^LR subgroup. In the external test cohort, differences in overall survival between the two low-risk subgroups did not reach statistical significance, likely because of the very small number of TCZ-treated patients (n = 12). These observations do not imply overall superiority of BCT but rather underscore the presence of heterogeneous treatment effects that can be captured by treatment-specific risk models.

Treatment assignment was non-randomized and confounding by contraindications was inevitable. Patients with active bacterial or fungal co-infection, profound neutropenia, inability to swallow tablets, or advanced chronic kidney disease were systematically directed toward TCZ owing to BCT contraindications or practical considerations, whereas suspected secondary infection favored BCT, because TCZ is relatively contraindicated in that setting. This bidirectional selection bias almost certainly contributed to the higher crude mortality observed in the TCZ-treated patients than in the BCT-treated patients.

Mechanistically, the broader cytokine-suppressive profile of BCT may offer theoretical advantages in certain inflammatory phenotypes. TCZ selectively blocks the IL-6 receptor (40), whereas BCT inhibits JAK1/JAK2 signaling and thereby interferes with multiple cytokine pathways, including IL-6, IL-2, IL-15, and interferon signaling (41). This wider spectrum could, in principle, sustain anti-inflammatory effects more effectively over several days of treatment. Supporting this hypothesis, a previous study in mechanically ventilated or ECMO-supported patients reported lower Day-30 mortality with BCT than with TCZ (32), and other retrospective analyses observed greater respiratory improvement at Day 7 or survival with BCT in severe-to-critical COVID-19 (14, 15). However, these observational comparisons are subject to the same confounding by indication and randomized data directly comparing the two agents remain limited.

Our goal was to build ML models that predict mortality using a minimal, clinically practical set of covariates. ML can capture complex, non-intuitive relations from existing clinical data, making it more feasible than biomarker-heavy, multi-omic approaches that demand substantial time and resources (42). Although ROC–AUC can rise with more weakly correlated features, we intentionally selected only the most predictive variables for Day-90 mortality in critical COVID-19, and we show how these features contribute to model probabilities for each therapy (Supplementary Figure 8). Using too many features hampers interpretability, increases data-collection burden and cost, and heightens overfitting risk, reducing generalizability (43). Because there is no single gold standard for feature selection, the number and composition of features should be guided by predictive performance and dataset size (44). We trained on the largest available development cohort for each immunomodulator, and external validation in the test cohort showed similar performance.

Despite these strengths, our study has limitations. First, the models were developed and validated in single-center cohorts, and temporal shifts in circulating variants (Alpha → Delta → Omicron), as well as staggered approvals of BCT/TCZ, may limit generalizability. Therefore, multi-center validation across various settings and pandemic phases is needed. Second, there was substantial treatment-arm imbalance—especially in the test cohort (TCZ n = 12 vs. BCT n = 83)—which reduces power for subgroup/survival comparisons and may bias effect estimates; results should be interpreted with caution and larger, more balanced cohorts are warranted. Third, the retrospective design entails potential selection bias and uneven baseline severity between treatment arms. Additionally, we analyzed only critically ill patients receiving immunomodulators, limiting applicability to patients with moderate/severe disease not receiving these agents. Fourth, SARS-CoV-2 vaccination status was not included. While vaccination significantly influences the severity of COVID-19, its impact on the effectiveness of immunomodulators in critically ill patients remains uncertain. Fifth, our ML models used static (not time-series) features due to missing temporal data in the test cohort; despite this, discrimination was moderate and static-feature models are more practical for bedside use. Finally, we do not treat between-model differences for the same patient as counterfactual effects because treatment was non-randomized and models were trained separately; isotonic regression was used only for within-arm calibration. Accordingly, findings are hypothesis-generating and require confirmation in prospective randomized trials or rigorously adjusted observational studies under a shared covariate structure.

5 Conclusion

Despite these limitations, to the best of our knowledge, this study is the first to develop and externally validate two treatment-specific prognostic machine-learning models for critically ill patients receiving either baricitinib or tocilizumab. These trained ML models can be a useful adjunct in selecting immunomodulators for critically ill COVID-19 pneumonia patients. Further studies are needed to investigate the potential application of these models in enhancing survival outcomes.

References

• 1.

  World Health Organization . Global Health Estimates: Life expectancy and leading causes of death and disability. World Health Organization (2024) https://www.who.int/data/gho/data/themes/mortality-and-global-health-estimates (Accessed December 21, 2024)

  • Google Scholar

• 2.

  World Health Organization . WHO COVID-19 dashboard. World Health Organization (2024) https://data.who.int/dashboards/covid19/deaths (Accessed December 21, 2024)

  • Google Scholar

• 3.

  Jeong S-J An M Jang M Choi SY Choi Y-J Jang J et al . Severity of COVID-19 associated with SARS-CoV-2 variants dominant period in the Republic of Korea. Public Health Weekly Rep. (2023) 16:1464–87. doi: 10.56786/phwr.2023.16.43.2,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  Nakamura K Goto T Shiraishi K Yonekawa A Eriguchi Y Akashi K et al . Clinical and virological features of SARS-CoV-2 omicron variant-infected immunocompromised patients receiving immunosuppressive medications. BMC Infect Dis. (2024) 24:736. doi: 10.1186/s12879-024-09633-1,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  Gulick RM Pau AK Daar E Evans L Gandhi RT Tebas P et al . National Institutes of Health COVID-19 treatment guidelines panel: perspectives and lessons learned. Ann Intern Med. (2024) 177:1547–57. doi: 10.7326/ANNALS-24-00464,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  Del Valle DM Kim-Schulze S Huang HH Beckmann ND Nirenberg S Wang B et al . An inflammatory cytokine signature predicts COVID-19 severity and survival. Nat Med. (2020) 26:1636–43. doi: 10.1038/s41591-020-1051-9,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  Abidi E El Nekidy WS Alefishat E Rahman N Petroianu GA El-Lababidi R et al . Tocilizumab and COVID-19: timing of administration and efficacy. Front Pharmacol. (2022) 13:825749. doi: 10.3389/fphar.2022.825749,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8.

  Ying B Albeituni S Liu G Owlia P Bagheri N . COVID-19 and the potential of Janus family kinase (JAK) pathway inhibition: a novel treatment strategy. Front Med. (2019). 9:961027. doi: 10.3389/fmed.2022.961027

  • CrossRef
  • Google Scholar

• 9.

  Marconi VC Ramanan AV de Bono S Kartman CE Krishnan V Liao R et al . Efficacy and safety of baricitinib for the treatment of hospitalised adults with COVID-19 (COV-BARRIER): a randomised, double-blind, parallel-group, placebo-controlled phase 3 trial. Lancet Respir Med. (2021) 9:1407–18. doi: 10.1016/S2213-2600(21)00331-3,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10.

  Ely EW Ramanan AV Kartman CE de Bono S Liao R Piruzeli MLB et al . Efficacy and safety of baricitinib plus standard of care for the treatment of critically ill hospitalised adults with COVID-19 on invasive mechanical ventilation or extracorporeal membrane oxygenation: an exploratory, randomised, placebo-controlled trial. Lancet Respir Med. (2022) 10:327–36. doi: 10.1016/S2213-2600(22)00006-6,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11.

  Abani O Abbas A Abbas F Abbas M Abbasi S Abbass H et al . Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. (2021) 397:1637–45. doi: 10.1016/S0140-6736(21)00676-0,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12.

  Lakatos B Szabó BG Bobek I Kiss-Dala N Gáspár Z Riczu A et al . Baricitinib vs tocilizumab treatment for hospitalized adult patients with severe COVID-19 and associated cytokine storm: a prospective, investigational, real-world study. Int J Infect Dis. (2022) 125:233–40. doi: 10.1016/j.ijid.2022.10.037,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13.

  Karampitsakos T Papaioannou O Tsiri P Katsaras M Katsimpris A Kalogeropoulos AP et al . Tocilizumab versus baricitinib in hospitalized patients with severe COVID-19: an open label, randomized controlled trial. Clin Microbiol Infect. (2023) 29:372–8. doi: 10.1016/j.cmi.2022.10.015,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  Sunny S Tran A Lee J Abdallah M Chaudhry N Quale J . Comparison of tocilizumab vs baricitinib in clinical outcomes among hospitalized patients with COVID-19: experience from a public hospital system in new York City. Open Forum Infect Dis. (2023) 10:ofad426. doi: 10.1093/ofid/ofad426,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  Tomos I Grigoropoulos I Kosti C Chrysikos S Digalaki A Thomas K et al . Comparison of effectiveness and safety between baricitinib and tocilizumab in severe COVID-19: a retrospective study. Expert Rev Respir Med. (2025) 19:389–97. doi: 10.1080/17476348.2025.2473486,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  Peng J Jury EC Dönnes P Ciurtin C . Machine learning techniques for personalised medicine approaches in immune-mediated chronic inflammatory diseases: applications and challenges. Front Pharmacol. (2021) 12:720694. doi: 10.3389/fphar.2021.720694,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  Park SS Lee JC Byun JM Choi G Kim KH Lim S et al . ML-based sequential analysis to assist selection between VMP and RD for newly diagnosed multiple myeloma. NPJ Precis Oncol. (2023) 7:46. doi: 10.1038/s41698-023-00385-w,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18.

  Buell KG Spicer AB Casey JD Seitz KP Qian ET Graham Linck EJ et al . Individualized treatment effects of oxygen targets in mechanically ventilated critically ill adults. JAMA. (2024) 331:1195–204. doi: 10.1001/jama.2024.2933,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  Bayat V Phelps S Ryono R Lee C Parekh H Mewton J et al . A severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) prediction model from standard laboratory tests. Clin Infect Dis. (2021) 73:E2901–7. doi: 10.1093/cid/ciaa1175,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  Wang K Zuo P Liu Y Zhang M Zhao X Xie S et al . Clinical and laboratory predictors of in-hospital mortality in patients with COVID-19: a cohort study in Wuhan, China. Clin Infect Dis. (2020) 71:2079–88. doi: 10.1093/cid/ciaa538

  • CrossRef
  • Google Scholar

• 21.

  RECOVERY Collaborative Group . Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. (2021) 384:693–704. doi: 10.1056/NEJMoa2021436,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  Ahmad A Zaheer M Balis FJ . Baricitinib. StatPearls (2024) Available online at: https://www.ncbi.nlm.nih.gov/books/NBK572064/ (Accessed January 18, 2025)

  • Google Scholar

• 23.

  Preuss C V Anjum F . Tocilizumab. StatPearls (2024) Available online at: https://www.ncbi.nlm.nih.gov/sites/books/NBK570644/ (Accessed January 18, 2025)

  • Google Scholar

• 24.

  Pedregosa F Varoquaux G Gramfort A Thirion B Grisel O Dubourg V et al . Scikit-learn: machine learning in Python Gaël Varoquaux Bertrand Thirion Vincent Dubourg Alexandre Passos PEDREGOSA, VAROQUAUX, GRAMFORT ET AL. Matthieu Perrot. J Mach Learn Res. (2011) 12:2825–30. doi: 10.5555/1953048.2078195

  • CrossRef
  • Google Scholar

• 25.

  Beretta L Santaniello A . Nearest neighbor imputation algorithms: a critical evaluation. BMC Med Inform Decis Mak. (2016) 16:74. doi: 10.1186/s12911-016-0318-z,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26.

  White IR Royston P Wood AM . Multiple imputation using chained equations: issues and guidance for practice. Stat Med. (2011) 30:377–99. doi: 10.1002/sim.4067,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  Isaac Abiodun O Jantan A Esther Omolara A Victoria Dada K AbdElatif Mohamed N Arshad H . State-of-the-art in artificial neural network applications: a survey. Heliyon. (2018) 4:938. doi: 10.1016/j.heliyon.2018

  • CrossRef
  • Google Scholar

• 28.

  Fisher A Rudin C Dominici F . Model class reliance: variable importance measures for any machine learning model class, from the “Rashomon” perspective. [Epub ahed of print] (2018) doi: doi: 10.48550/arXiv.1801.01489

  • CrossRef
  • Google Scholar

• 29.

  Awad M Fraihat S . Recursive feature elimination with cross-validation with decision tree: feature selection method for machine learning-based intrusion detection systems. J Sens Actuator Netw. (2023) 12:67. doi: 10.3390/jsan12050067

  • CrossRef
  • Google Scholar

• 30.

  Murphy AH . A new vector partition of the probability score. J Appl Meteorol Climatol. (1973) 12:595–600. doi: 10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2

  • CrossRef
  • Google Scholar

• 31.

  Thakur B Dubey P Benitez J Torres JP Reddy S Shokar N et al . A systematic review and meta-analysis of geographic differences in comorbidities and associated severity and mortality among individuals with COVID-19. Sci Rep. (2021) 11:8562. doi: 10.1038/s41598-021-88130-w,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  You SH Baek MS Kim TW Jung SY Kim WY . Baricitinib versus tocilizumab in mechanically ventilated patients with COVID-19: a nationwide cohort study. Crit Care. (2024) 28:282. doi: 10.1186/s13054-024-05063-2,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  Singer M Deutschman CS Seymour CW Shankar-Hari M Annane D Bauer M et al . The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. (2016) 315:801–10. doi: 10.1001/jama.2016.0287,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  Prytherch DR Smith GB Schmidt PE Featherstone PI . ViEWS-towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. (2010) 81:932–7. doi: 10.1016/j.resuscitation.2010.04.014,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35.

  Royal College of Physicians . National Early Warning Score (NEWS) 2 Standardising the assessment of acute-illness severity in the NHS. Royal College of Physicians. (2017). Available online at: www.rcplondon.ac.uk (Accessed October 14, 2025).

  • Google Scholar

• 36.

  Ok F Erdogan O Durmus E Carkci S Canik A . Predictive values of blood urea nitrogen/creatinine ratio and other routine blood parameters on disease severity and survival of COVID-19 patients. J Med Virol. (2021) 93:786–93. doi: 10.1002/jmv.26300,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  Veronese N Segala FV Carruba L La Carrubba A Pollicino F Di Franco G et al . Anemia as a risk factor for disease progression in patients admitted for COVID-19: data from a large, multicenter cohort study. Sci Rep. (2023) 13:9035. doi: 10.1038/s41598-023-36208-y,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  Arnau-Barrés I Pascual-Dapena A López-Montesinos I Gómez-Zorrilla S Sorlí L Herrero M et al . Severe hypoalbuminemia at admission is strongly associated with worse prognosis in older adults with sars-cov-2 infection. J Clin Med. (2021) 10:5134. doi: 10.3390/jcm10215134,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  Kang S-J Jung SI . Age-related morbidity and mortality among patients with COVID-19. Infect Chemother. (2020) 52:154–64. doi: 10.3947/ic.2020.52.2.154,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  Scott LJ . Tocilizumab: a review in rheumatoid arthritis. Drugs. (2017) 77:1865–79. doi: 10.1007/s40265-017-0829-7,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41.

  Bronte V Ugel S Tinazzi E Vella A De Sanctis F Canè S et al . Baricitinib restrains the immune dysregulation in patients with severe COVID-19. J Clin Invest. (2020) 130:6409–16. doi: 10.1172/JCI141772,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42.

  Odenkirk MT Reif DM Baker ES . Multiomic big data analysis challenges: increasing confidence in the interpretation of artificial intelligence assessments. Anal Chem. (2021) 93:7763–73. doi: 10.1021/acs.analchem.0c04850,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43.

  Kourou K Exarchos KP Papaloukas C Sakaloglou P Exarchos T Fotiadis DI . Applied machine learning in cancer research: a systematic review for patient diagnosis, classification and prognosis. Comput Struct Biotechnol J. (2021) 19:5546–55. doi: 10.1016/j.csbj.2021.10.006,

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  Ahmed SF Alam MSB Hassan M Rozbu MR Ishtiak T Rafa N et al . Deep learning modelling techniques: current progress, applications, advantages, and challenges. Artif Intell Rev. (2023) 56:13521–617. doi: 10.1007/s10462-023-10466-8

  • CrossRef
  • Google Scholar