Efficacy of COVID-19 Treatments: A Bayesian Network Meta-Analysis of Randomized Controlled Trials

Background: We provided a comprehensive evaluation of efficacy of available treatments for coronavirus disease 2019 (COVID-19). Methods: We searched for candidate COVID-19 studies in WHO COVID-19 Global Research Database up to August 19, 2021. Randomized controlled trials for suspected or confirmed COVID-19 patients published on peer-reviewed journals were included, regardless of demographic characteristics. Outcome measures included mortality, mechanical ventilation, hospital discharge and viral clearance. Bayesian network meta-analysis with fixed effects was conducted to estimate the effect sizes using posterior means and 95% equal-tailed credible intervals (CrIs). Odds ratio (OR) was used as the summary measure for treatment effect. Bayesian hierarchical models were used to estimate effect sizes of treatments grouped by the treatment classifications. Results: We identified 222 eligible studies with a total of 102,950 patients. Compared with the standard of care, imatinib, intravenous immunoglobulin and tocilizumab led to lower risk of death; baricitinib plus remdesivir, colchicine, dexamethasone, recombinant human granulocyte colony stimulating factor and tocilizumab indicated lower occurrence of mechanical ventilation; tofacitinib, sarilumab, remdesivir, tocilizumab and baricitinib plus remdesivir increased the hospital discharge rate; convalescent plasma, ivermectin, ivermectin plus doxycycline, hydroxychloroquine, nitazoxanide and proxalutamide resulted in better viral clearance. From the treatment class level, we found that the use of antineoplastic agents was associated with fewer mortality cases, immunostimulants could reduce the risk of mechanical ventilation and immunosuppressants led to higher discharge rates. Conclusions: This network meta-analysis identified superiority of several COVID-19 treatments over the standard of care in terms of mortality, mechanical ventilation, hospital discharge and viral clearance. Tocilizumab showed its superiority compared with SOC on preventing severe outcomes such as death and mechanical ventilation as well as increasing the discharge rate, which might be an appropriate treatment for patients with severe or mild/moderate illness. We also found the clinical efficacy of antineoplastic agents, immunostimulants and immunosuppressants with respect to the endpoints of mortality, mechanical ventilation and discharge, which provides valuable information for the discovery of potential COVID-19 treatments.


INTRODUCTION
The pandemic of novel coronavirus disease 2019 (COVID- 19) has become a global threat to public health. By August 27, 2021, over 214 million confirmed cases including 4.47 million deaths have been reported (1). Faced with such a global crisis, identifying effective treatments for COVID-19 is of urgent need and paramount importance for clinical researchers. Development of novel drugs typically takes years of concerted efforts and thus most of the research in COVID-19 treatment has been focused on drug repositioning, i.e., investigating the effectiveness of drugs approved for other diseases on COVID-19 patients. By August 18, 2021, over 11,000 clinical trials related to COVID-19 have been registered worldwide (2), while only dexamethasone (3,4) and remdesivir (5,6) were proven to be clinically effective.
With global efforts on pursuing effective treatments during the pandemic, a large number of short-term randomized controlled trials (RCTs) of small size were conducted and published at a high rate, and some trials were carried out in a rather rush manner which might cause deterioration of trial quality. Timely summaries and analyses of existing clinical trial results can help researchers to better understand various treatments, early terminate investigation on ineffective treatments and provide necessary guidelines for further research and discovery of new treatments. However, the conventional pairwise meta-analysis is limited in simultaneous comparisons among multiple trials and it often fails to capture indirect evidence for treatments that have not been tested in head-to-head comparisons. A network meta-analysis (NMA) which combines both direct and indirect information would be more appropriate to accommodate such a complex situation. Several NMA publications provided useful information on the comparative effectiveness of repurposed drugs for patients with COVID-19 (7,8).
During the drug repurposing process, clinicians identify candidate drugs by estimating drug-disease or drug-drug similarities. Drugs with shared chemical structures and mechanisms of action are expected to deliver similar therapeutic applications (9). Not only should research focus on individual treatment for COVID-19, but it is also of great interest to evaluate a class of treatments with shared clinical properties and biochemical structures. For example, glucocorticoids including methylprednisolone, dexamethasone and hydrocortisone were reported to be associated with reduced 28-day mortality for critical COVID-19 patients (10).
This NMA aimed to provide a comprehensive evaluation of the efficacy of available treatments for patients with COVID- 19. Not only does our NMA evaluate treatments at the drug level, but it also provides an overall estimated effect at the class level which may contain several drugs of similar types via a Bayesian hierarchical model using fixed-effects. Such class levels of treatment evaluation have not been explored in the literature.

MATERIALS AND METHODS
This systematic review and NMA were conducted and reported in accordance with the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses for NMAs (11). A prespecified protocol can be found in Supplementary Materials.

Information Sources and Eligibility Criteria
We performed an exhaustive online search for eligible studies in the WHO COVID-19 Global Research Database (12). The WHO COVID-19 Global Research Database is a global multilingual literature database which gathers the latest COVID-19 related studies as a composite of other databases (e.g., Medline, Global Health, PubMed Central, PsycInfo, Scopus, ProQuest Central, Embase, Web of Science and others). Supplementary Table S1 presents the detailed searching strategy. We updated the literature search weekly to include newly published trials in peer-reviewed journals. The current version of our manuscript includes studies from January 1, 2020 to August 19, 2021. Original articles investigating treatment effects for suspected or confirmed COVID-19 were included. We considered appropriate COVID-19 treatments while excluding (i) herbal medicine; (ii) preventive interventions (e.g., vaccination and mask wearing); (iii) non-drug supportive care; (iv) exercise, psychological and educational treatments. We included studies that compared one intervention with other interventions or the standard of care (SOC).
The outcomes of interest in the NMA included overall mortality, requirement for mechanical ventilation (MV), discharge from hospital on day 14 or the day closest to that, and viral clearance on day 7 or the day closest to that. We evaluated only binary outcomes since most COVID-19 trials had <1-month follow-ups (7) and for such short-term studies, continuous or survival outcomes might not provide a clinically meaningful summary for treatment effect (13). In addition, clinical definitions of several continuous outcomes, e.g., time to clinical improvement or deterioration, were not consistent across trials. Different reporting patterns of point and interval estimates for continuous outcomes may also cause additional difficulties and biases in the NMA.
We only included RCTs in this NMA because nonrandomized trials and observational studies were considered of low certainty from the causal inference perspective (14). We included trials published in peer-reviewed journals in English and Chinese regardless of ways of randomization (double-blind, single-blind or open-label) or demographic characteristics.

Study Selection
Two reviewers independently screened the titles and abstracts using the inclusion criteria. Full texts of potentially eligible studies were further assessed for eligibility. Discrepancies were resolved by discussion and, if necessary, a third investigator was consulted.

Data Collection Process
Data extraction was conducted by two investigators independently. For each eligible study, we collected trial characteristics, interventions, demographic characteristics and outcomes of interest. For binary outcomes of interest, numbers of events and overall numbers of patients were collected. Two reviewers resolved discrepancies via discussion and a third party adjudicated if any conflict arose. For multiple reports on the same trial, we adopted the latest peer-reviewed publication.

Risk of Bias of Individual Studies
For each eligible trial, we used a revision (7) of version 2 of the Cochrane risk of bias tool (RoB 2.0) (15) to assess risk of bias in RCTs. Detailed RoB judgments were listed in Supplementary Materials. Two reviewers separately completed the RoB assessment and, in presence of any disagreement, a third party made the final decision.

Data Synthesis and Statistical Analysis
In the network, each node represents a treatment, regardless of the dose or duration of administration. For studies involving different doses or durations of the same drug, we aggregated data of the same drug into one arm. Each multi-arm trial was treated as a single study in the network analysis, instead of being split into multiple two-arm sub-trials. Interventions comprising more than one drug (i.e., combination therapy) were treated as separate treatment nodes. For each clinical outcome, we excluded the treatments appearing in only one trial with fewer than 100 patients to alleviate potential risk caused by inadequate information. We plotted the network for each outcome of interest using the igraph (16) package of R version 4.0.3 (RStudio, Boston, MA).
We considered a hierarchical model structure for investigated interventions where the relative effects compared with SOC were nested within drug classifications. Based on the Anatomical Therapeutic Chemical Classification System with Defined Daily Doses (ATC/DDD) (17) published by WHO, we classified included drugs by the second level of their ATC/DDD codes. For investigational drugs without ATC/DDD codes, we determined their classifications according to the pharmacological mechanism and therapeutic use. The detailed Bayesian hierarchical model structure for the NMA is shown in Supplementary Materials.
We fitted the Bayesian NMA model and generated posterior samples of parameters using the Markov chain Monte Carlo (MCMC) algorithm. The treatment effects of eligible drugs were evaluated in terms of the odds ratio (OR) estimated by the posterior mean and corresponding equal-tailed 95% credible interval (CrI). To obtain direct and indirect estimates for treatment effects and assess local inconsistency in the network, we considered the node-splitting method (18). The MCMC sampling was performed using the jagsUI (19,20) package, and further network analyses were performed using the gemtc (21) package of R.

Certainty of the Evidence
The grading of recommendations assessment, development and evaluation (GRADE) approach for NMA (14) was used to rate the certainty of evidence of NMA estimates. Two investigators rated the certainty of each treatment comparison independently and resolved discrepancies by discussions and, if necessary, consulted with a third party. Detailed ratings and rationales for GRADE were provided in Supplementary Materials.

Subgroup and Sensitivity Analysis
Planned subgroup analyses were conducted for mild/moderate vs. severe/critical COVID-19 patients. In addition to Bayesian fixed-effects NMA, we also performed Bayesian random-effects NMA. Several RCTs which were designed to be multi-arm trials but reported results for different interventions vs. SOC as if they has been compared in separated two-arm studies. In the primary analysis, we treated these RCTs as multi-arm, and the SOC group with the largest number of participants was used if the periods of patient enrolment of specified interventions had overlaps, otherwise we considered a new SOC group which combined the SOC groups of all studies for the same RCT. In addition, we performed a sensitivity analysis by treating these multi-arm RCTs as separated two-arm trials.

RESULTS
According to the prespecified inclusion and exclusion criteria, we identified and screened titles and abstracts of 11,626 studies. Among these, 402 studies were further reviewed for full text and 222 eligible studies were included in the systematic review.  Out of the 222 studies, 31 studies were not considered in the meta-analysis. Among them, nine studies investigated different doses or durations of administration of the same intervention without comparisons with other interventions or SOC, 19 trials did not report outcomes of interest, and treatments in three trials were not connected in the network.
Among the 191 studies included in the quantitative synthesis, 179 unique RCTs were reported, which evaluated the efficacy of 94 different COVID-19 treatments from 41 classes. The RECOVERY trial (NCT04381936) was reported in six studies (3,(22)(23)(24)(25)(26). According to an early version of the protocol of the RECOVERY trial (27), the main randomization consisted of two parts: (A) lopinavir/ritonavir, hydroxychloroquine, azithromycin and dexamethasone vs. SOC; (B) convalescent plasma vs. SOC. Patients after the main randomization but with progressive COVID-19 would undergo a second randomization to either tocilizumab or SOC groups. Therefore, in the primary analysis we treated the studies of convalescent plasma (25) and tocilizumab (24) as separated two-arm trials and the four studies of lopinavir/ritonavir (22), hydroxychloroquine (26), azithromycin (23) and dexamethasone (3) vs. SOC as a five-arm trial. The SOC group with the largest number of patients (23) was used. Clinical results of the PRINCIPLE trial (ISRCTN86534580) were shown in three studies (28-30) comparing azithromycin, budesonide and doxycycline with SOC, respectively. Due to no overlap between the enrolment periods of the azithromycin and budesonide studies, we created a new SOC group by combining the two SOC groups and the PRINCIPLE trial was considered as a four-arm trial in the primary analysis. The REMAP-CAP trial (NCT02735707) was reported in three studies which investigated tocilizumab vs. sarilumab vs. SOC (31), hydrocortisone vs. SOC (32) and hydroxychloroquine vs. lopinavir/ritonavir vs. hydroxychloroquine plus lopinavir/ritonavir vs. SOC (33) with patients overlapped in the SOC arms. Thus, we treated it as a seven-arm trial and used the SOC group including the most patients (31). The DISCOVERY trial (34) was a participant of the WHO SOLIDARITY trial (35) while it reported additional endpoints of interest. The observed outcomes of the SOLIDARITY trial (35) were used in the NMA if existed, otherwise we considered those in the DISCOVERY trial (34). The phases II and III of the BLAZE-1 trial (NCT04427501) (36,37) were reported in two separated articles and we simply merged results of the two stages as one RCT in the primary analysis. The REMAP-CAP, ACTIV-4a, and ATTACC Investigators examined the clinical effect of therapeutic-dose anticoagulation for patients with COVID-19 and presented their results for critically ill and non-critical patients in two publications, respectively (38, 39). In the primary analysis, we combined results of these two articles and in the subgroup analysis on baseline illness severity, they were treated separately.
Under the random-effects model, the estimated treatment effects relative to SOC were similar to those under the fixed-effects model but with wider credible intervals, e.g., tocilizumab with an OR of 0.91 (95% CrI [0. 74,1.16]). In the sensitivity analysis by treating part A in the main randomization of RECOVERY (3,22,23,26), PRINCIPLE (28-30), REMAP-CAP (31, 32), BLAZE-1 and two therapeuticdose anticoagulation trials for critical and noncritical patients (36,37) as separated trials, all estimates were close to those in the primary analysis except for dexamethasone, which reported an OR of 0. 97 (Figure 3). Immunostimulants (interferon beta and recombinant human GCSF) showed significant benefit on the reduction of mechanical ventilation   (Figure 4).

Summary of Findings
In this systematic review and NMA, we provided a detailed summary of trial characteristics of published RCTs for confirmed COVID-19 patients up to August 19, 2021 and reported effectiveness of treatments at both the drug and class levels in terms of mortality, mechanical ventilation, hospital discharge and viral clearance. Compared with SOC, imatinib, intravenous immunoglobulin and tocilizumab were shown to reduce the risk of mortality; baricitinib plus remdesivir, colchicine, dexamethasone, recombinant human GCSF and tocilizumab resulted in fewer events of mechanical ventilation; patients who received convalescent plasma, ivermectin, ivermectin plus doxycycline, hydroxychloroquine, nitazoxanide and proxalutamide had a higher viral elimination rate; tofacitinib, sarilumab, remdesivir, tocilizumab and baricitinib plus remdesivir demonstrated their effectiveness with significantly higher 14-day hospital discharge rates.
At the class level of treatments, antineoplastic agents including bamlanivimab, imatinib and INM005 could reduce mortality; immunostimulants containing interferon beta and recombinant human GCSF showed clinical benefit over SOC in reducing mechanical ventilation; immunosuppressants consisting of canakinumab, sarilumab, tocilizumab and tofacitinib led to higher hospital discharge rates around 14 days, and the use of anthelmintics (ivermectin), anthelmintics plus antibacterials for systemic use (ivermectin plus doxycycline), endocrine therapy (proxalutamide) increased the rate of viral clearance on day 7. For other classes and outcomes, we observed no significant difference from SOC.
With an urgent need to identify effective treatments for COVID-19, researchers desired to aggregate information from individual trials investigating various interventions and, toward this goal, several NMAs for pharmacological interventions of COVID-19 have been published. Siemieniuk et al. (7) conducted a living systematic review and NMA for RCTs up to March 1, 2021 to evaluate the efficacy of potential COVID-19 treatments. They found corticosteroids (budesonide, dexamethasone, hydrocortisone and methylprednisolone) and Janus kinase inhibitors (baricitinib and ruxolitinib) could reduce death, mechanical ventilation, and increase the number of days free from mechanical ventilation; interleukin-6 inhibitors (tocilizumab and sarilumab) reduced mechanical ventilation and lengths of hospital stay. Kim et al. (167) reported improved outcomes for patients receiving anti-inflammatory agents (corticosteroids, tocilizumab, anakinra, and intravenous immunoglobulin), convalescent plasma, and remdesivir in their NMA including both RCTs and observational studies up to August 24, 2020. The majority of our findings are consistent with previous research except for the significant treatment effects of dexamethasone and corticosteroids due to the discrepancies among different SOC arms when we treated RECOVERY (3,22,23,26) as one multi-arm trial. More treatments with clinical effectiveness against COVID-19 have been identified by including recently published studies in our NMA.
On the other hand, pairwise meta-analyses for a single drug vs. SOC have also revealed clinical benefits of potential COVID-19 treatments with accumulated evidence from completed studies. For example, tocilizumab led to reduction in mortality (168,169), ventilation (170) and biomarkers of the COVID-19 infection (171); patients receiving ivermectin had a lower risk of death as well as an increase in the viral clearance rate (172,173); the administration of colchicine resulted in a lower risk of mortality and improvements of clinical outcomes (174); remdesivir showed its superiority over SOC with faster recovery, shorter time to clinical improvement and reduction in mortality (175,176).

Strength and Limitations
Not only was this NMA timely conducted, but it also included a wide range of RCTs, which contained a large number of common drugs as well as interferons, blood products, mineral and vitamin supplementations. Treatment effects were evaluated in a network at both the individual drug level and class level. Using a hierarchical Bayesian model based on the WHO ATC/DDD classification rule, we grouped the treatments from a scientific and pharmacological perspective and provided a further guideline for discovery of new treatments on COVID-19.
This study has several limitations. One is the low certainty of evidence for many NMA estimates. At the early stage of COVID-19 pandemic, with limited clinical resources and urgent need to obtain trial results, many RCTs were conducted with simplified procedures, e.g., no placebo prepared, leading to downgrading of evidence due to study limitations (177). Over time, the situation has gradually improved and many double-blind RCTs have been conducted and published recently. Moreover, networks of treatments were sparse because most of the included studies evaluated interventions vs. SOC and there were few direct comparisons among interventions. As we considered COVID-19 RCTs regardless of demographic characteristics, intransitivity existed in many indirect comparisons. For example, hydroxychloroquine trials usually investigated patients with mild/moderate COVID-19, while patients treated by convalescent plasma were mainly of severe/critical illness. Detailed subgroup analysis might help to resolve such problems (see Supplementary Materials).
Another limitation of this study arises from the evaluation of NMA estimates at the class level. Many investigated classes contained only one treatment, leading to large variation and thus insignificant results. To confirm the superiority of a class of drugs, one should present evidence of stronger strengths. More treatments could have been included in the NMA if the exclusion criteria of treatment nodes were relaxed, while it would inevitably introduce additional bias due to treatments tested with small sample sizes.
In the primary analysis, we only included peer-reviewed studies to maintain the credibility of evidence. However, among such a large number of completed COVID-19 trials, studies reporting positive results or with large sample size were more likely to be published, leading to possible publication bias (178). To alleviate the potential publication bias, we conducted an exploratory analysis including both peer-reviewed papers and preprints from unpublished studies, for which results were shown in the Supplementary Figures S14-S17. While caution should be taken on the evidence implied by only preprints since clinical results without peer-reviews should not be trusted equally as those published.
In addition, we mainly focused on the efficacy of interventions in this NMA and did not evaluate the corresponding safety profiles. Evidence from other NMAs (7,179) showed that most of investigated treatments in this NMA did not lead to increased adverse events, and remdesivir and lopinavir/ritonavir were associated with fewer occurrences of adverse events and serious adverse events, respectively.
Different approaches to dealing with the RECOVERY, REMAP-CAP, PRINCIPLE, BLAZE-1 and therapeutic anticoagulation trials led to discrepancies between results of the primary and sensitivity analyses, especially for dexamethasone. The RECOVERY trial was designed as a multi-arm trial (27) while the numbers of patients randomized to SOC and event rates of outcomes of interest were different across different reports (3,22,23,26). Although we observed no clinical benefit on the reduction of mortality and increase of the hospital discharge rate for dexamethasone vs. SOC, the sensitivity analysis drew an opposite conclusion and credibility of this finding warrants extra caution.

Conclusion
This systematic review and NMA showed that imatinib, intravenous immunoglobulin and tocilizumab could reduce the mortality. Patients receiving baricitinib plus remdesivir, colchicine, dexamethasone, recombinant human GCSF and tocilizumab had a lower risk of mechanical ventilation. Administration of tofacitinib, sarilumab, remdesivir, tocilizumab and baricitinib plus remdesivir led to higher hospital discharge rates. Convalescent plasma, ivermectin, ivermectin plus doxycycline, hydroxychloroquine, nitazoxanide and proxalutamide could improve the viral elimination.
At the treatment class level, compared with SOC, patients receiving antineoplastic agents had a lower risk of death; immunostimulants tended to reduce the need of mechanical ventilation; the use of immunosuppressants led to an increased hospital discharge rate; anthelmintics, anthelmintics plus antibacterials for systemic use and endocrine therapy showed clinical improvements on viral clearance, while these three classes contained only one treatment, for which the evidence might not be sound.
The clinical benefits of several treatments on confirmed COVID-19 patients have been reported in this study. The endpoints of mortality and mechanical ventilation can be viewed as the deterioration of COVID-19 illness, and for clinicians and patients with severe COVID-19, these effective treatments (e.g., tocilizumab, imatinib, intravenous immunoglobulin, dexamethasone) can prevent or alleviate the progression of disease. Hospital discharge and viral clearance represent the recovery from COVID-19, and patients with mild or moderate illness might suffer less from the SARS-COV2 infection. Overall, tocilizumab performed the best against COVID-19 compared with SOC, which showed its superiority in terms of lower mortality and mechanical ventilation rates as well as a higher hosptical discharge rate.
On the other hand, we found the significance of classes of treatments on each investigated endpoint. The discovery of effective treatments on COVID-19 is still an essential issue, especially after the occurrence of more infective and fatal variants. The efficacy of antineoplastic agents, immunostimulants and immunosuppressants on reduced risk of death, mechanical ventilation and increased hosptical discharge, respectively, was shown by our NMA with sound statistical evidence, which shed new light on further research and discovery of potential COVID-19 treatments. Further large clinical trials are still needed to confirm these results.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

AUTHOR CONTRIBUTIONS
CZ and GY contributed to the conception and design of the work. CZ, HJ, and YW collected information and analyzed data used in the systematic review and meta-analysis. CZ and HJ drafted the work. YW and GY substantively revised it. All authors read and approved the submitted version and agreed to be personally accountable for the authors' own contributions and to ensure the accuracy and integrity of the work.

FUNDING
This study was supported by the Research Grants Council of Hong Kong (17308420).

ACKNOWLEDGMENTS
We thank the two referees and Editor for their many constructive and insightful comments that have led to significant improvements in the article.