No Efficacy of the Combination of Lopinavir/Ritonavir Plus Hydroxychloroquine Versus Standard of Care in Patients Hospitalized With COVID-19: A Non-Randomized Comparison

Objectives: No specific treatment has been approved for COVID-19. Lopinavir/ritonavir (LPV/r) and hydroxychloroquine (HCQ) have been used with poor results, and a trial showed advantages of combined antiviral therapy vs. single antivirals. The aim of the study was to assess the effectiveness of the combination of antivirals (LPV/r and HCQ) or their single use in COVID-19 hospitalized patients vs. standard of care (SoC). Methods: Patients ≥18 years with SARS-CoV-2 infection, defined as positive RT-PCR from nasal/oropharyngeal (NP/OP) swab or positive serology, admitted at L. Spallanzani Institute (Italy) were included. Primary endpoint: time to invasive ventilation/death. Secondary endpoint: time to two consecutive negative SARS-CoV-2 PCRs in NP/OP swabs. In order to control for measured confounders, a marginal Cox regression model with inverse probability weights was used. Results: A total of 590 patients were included in the analysis: 36.3% female, 64 years (IQR 51–76), and 91% with pneumonia. Cumulative probability of invasive ventilation/death at 14 days was 21.2% (95% CI 17.6, 24.7), without difference between SOC, LPV/r, hydroxychloroquine, HCQ + LPV/r, and SoC. The risk of invasive ventilation/death in the groups appeared to vary by baseline ratio of arterial oxygen partial pressure to fractional inspired oxygen (PaO2/FiO2). Overall cumulative probability of confirmed negative nasopharyngeal swabs at 14 days was 44.4% (95% CI 38.9, 49.9), without difference between groups. Conclusion: In this retrospective analysis, we found no difference in the rate of invasive ventilation/death or viral shedding by different strategies, as in randomized trials performed to date. Moreover, even the combination HCQ + LPV/r did not show advantages vs. SoC.


INTRODUCTION
In December 2019, an outbreak of viral pneumonia cases of unknown cause was identified in Wuhan, China. A novel coronavirus was quickly identified in some of these patients and it has been designated as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (European Centre for Disease Prevention and Control, 2020). Currently, there are no approved therapeutic agents available for SARS-CoV-2, and great efforts have been unfolded for the discovery of possible treatment strategies. Many repurposed drugs have shown some preclinical activity against SARS-CoV-2 and have been experimented in vivo (Tobaiqy et al., 2020). Prompt identification and implementation of life support therapies are pivotal steps in order to prevent the spreading of the infection and improve patient's clinical outcome. Some data about treatment from observational studies, compassionate use programs, and few RCT results are available up-to-date.
Among the antiviral strategies, the antiretroviral drug lopinavir/ritonavir (LPV/r) had already demonstrated activity against SARS-CoV-2 and Middle East respiratory syndrome (MERS)-CoV (Ford et al., 2020). Two initial RCTs about lopinavir/ritonavir for treatment of SARS-Cov-2 showed inconclusive results. A small randomized, controlled, openlabel trial conducted in China did not observe any benefit of lopinavir/ritonavir treatment vs. standard care in reducing the time to clinical improvement in hospitalized adult patients with severe COVID-19. However, the study appeared to be underpowered and post hoc analyses showed accelerated clinical recovery (16.0 vs. 17.0 days) and reduced mortality (19.0 vs. 27.1%) in the subgroup of patients treated within 12 days after the onset of symptoms . Another very small RCT comparing lopinavir/ritonavir vs. arbidol for treating patients with mild/moderate COVID-19 showed no differences in term of viral clearance, symptoms resolution, and radiological improvement between the arms .
Follow-up retrospective studies did not show evidence of effectiveness of lopinavir/ritonavir and of other antiretrovirals, as recently systematically reviewed (Ford et al., 2020).
A possible antiviral activity against SARS-CoV-2 of hydroxychloroquine (HCQ) has been supposed. This drug inhibits the glycosylation of ACE II, the receptor used by SARS-CoV-2 to enter the cells, and could result in a reduced ligand recognition and internalization of the virus (Vincent et al., 2005). This activity, together with the best known immunomodulatory and anti-inflammatory effects, yielded HCQ an interesting drug in this contest, but the most recent results showed lack of efficacy.
A review of seven clinical trials has shown contrasting results, but the analyzed studies posed significant risk for bias in the randomization process, in measurement of outcomes, or in deviations from planned interventions (Chowdhury et al., 2020). Recent data from the RECOVERY trial, a large randomized study, showed no evidence of benefit for mortality or other outcomes (duration of hospitalization and need for invasive ventilation) of HCQ treatment in hospitalized patients with COVID-19. Indeed, day-28 mortality was reported as 25.7% with HCQ and 23.5% with comparator (hazard ratio 1.11, 95% CI 0.98-1.26, p 0.10), so the investigators announced closure of the HCQ arm due to lack of effectiveness (Recovery Randomised Evaluation of COVID-19 Therapy, 2020a). Similarly, also the LPV/r arm in RECOVERY was halted for the same reasons (World Health Organization, 2020).
Even the ORCHID study, a clinical trial evaluating the safety and effectiveness of hydroxychloroquine for the treatment of hospitalized adults with COVID-19, has been halted by NIH (NIH, 2020b). Solidarity trial showed no effect of hydroxychloroquine or lopinavir/ritonavir on hospitalized patients with COVID-19, as indicated by overall mortality, initiation of ventilation, and duration of hospital stay (WHO Solidarity Trial Consortium et al., 2020). Even in terms of antivirals' effectiveness on ending SARS-COV-2 shedding, conflicting results about the clinical role of HCQ have been published. In fact, one report observed a positive impact on viral shedding (Huang et al., 2020), but a large RCT showed no difference in probability of negative conversion . Currently, NIH guidelines recommend against the use of HCQ or LPV/r for treatment of COVID-19 because of lack of effectiveness (NIH, 2020a).
However, triple combination of interferon beta-1b, lopinavir/ ritonavir, and ribavirin resulted to be superior to lopinavir/ ritonavir alone in alleviating symptoms and shortening the duration of viral shedding and hospital stay in patients with mild-to-moderate COVID-19 (Hung et al., 2020). Thus, these data provided a proof of concept for the possible synergic effect of using a combination of two or more antivirals to improve effectiveness like that seen for other infections such as HIV. More recently, an in silico approach proposed a possible synergistic effect of 16 compounds with independent mechanism of action in SARS-CoV-2 (Bobrowski et al., 2021).
The purpose of this study was to explore the difference in effectiveness of single antivirals (lopinavir/ritonavir and HCQ) and their combination when compared to current standard of care (SoC) in COVID-19 hospitalized patients, by emulating a RCT using the observational retrospective data of the INMI COVID-19 database.

MATERIALS AND METHODS
We conducted a retrospective cohort study on the INMI COVID-19 database of L. Spallanzani Institute in Rome (Italy) that contains data from consecutive hospitalized patients (≥18 years of age) who had a positive test result for the SARS-CoV-2 virus at any time during their hospitalization from January 29 to June 13, 2020. Participants' follow-up of those not yet discharged was administratively censored on July 1st , 2020. INMI COVID-19 database was approved by the local INMI, Rome Ethical Committee and patients provided written informed consent. The study was performed in accordance with the Declaration of Helsinki. INMI COVID-19 database retrieves epidemiological, demographic, clinical, and laboratory data of patients, as well as therapy prescribed (antiviral, immunomodulatory drugs, oxygen therapy, and need for ventilation) for COVID-19 patients.
Patients were included in this study if the following inclusion criteria were satisfied: ≥18 years of age, a diagnosis of SARS-CoV-2 infection, defined as positive RT-PCR from nasal/ oropharyngeal (NP/OP) swab or positive serology, and admitted at INMI L. Spallanzani Institute. This is a retrospective study which was conducted in exceptional conditions during the first wave of COVID-19 pandemic in Italy, so sample size was not preplanned.
In all included patients, diagnosis of SARS-CoV-2 infection was confirmed by the detection of SARS-CoV-2 RNA through real-time polymerase chain reaction (RT-PCR) targeting the E and RdRp viral genes on NP/OP swab. Subsequently, during the hospitalization, all patients underwent follow-up NP/OP swab to assess the clearance of viral RNA. The timing of follow-up NP/OP swab was variable, according to treating physician's judgment.
We compared four treatment strategies initiated after hospital admission: 1) starting hydroxychloroquine; 2) starting lopinavir/ ritonavir; 3) starting hydroxychloroquine plus lopinavir/ ritonavir; and 4) a control group receiving none of the previous drugs (standard of care). Standard of care included any supportive therapy: fluids, antibiotics, oxygen supplementation, and any concomitant therapy except for HCQ and LPV/r. Concomitant use of therapy with immunomodulants (e.g., anti-IL6 and anti-JAK), corticosteroids, heparin, and antibiotics (including azithromycin) was controlled for in the analysis. The decision of whether to treat patients with off-label hydroxychloroquine or lopinavir/ritonavir or other drugs was based on local medical consensus, guidelines, and the clinicians' own opinion.
The most commonly prescribed dosage of HCQ was 400 mg orally bid in the first day, followed by 200 mg bid for a total of 10 days and of lopinavir/ritonavir was 400/100 mg orally bid for 14 days.
The start of follow-up (baseline) for each patient was the first start of any therapy. All patients were followed up from baseline until death, discharge, last available visit, or the administrative censored date of July 1st , 2020, whichever occurred first.
The ratio of arterial oxygen partial pressure (PaO2) to fractional inspired oxygen (FiO2) between ≤300 mmHg was used as marker of severe respiratory disease, according to NIH guidelines (NIH, 2020a), for the stratified analysis.
The primary endpoint of this study was the evaluation of time from starting of therapy to invasive ventilation or death (whichever occurred first).
The secondary endpoints were 1) the evaluation of time from treatment initiation to two consecutive negative SARS-CoV-2 PCRs in nasal/oropharyngeal swabs, without an in-hospital relapse; 2) the evaluation of time from starting of therapy to noninvasive or invasive ventilation or death (whichever occurred first). Noninvasive ventilation includes CPAP or NIV.
For the secondary endpoint of evaluation of viral shedding, patients with diagnosis of COVID-19 infection made by SARS-CoV-2 serology were excluded from the analysis.
For the statistical analysis, chi-square or nonparametric Kruskal-Wallis tests were used to compare categorical or continuous variables in descriptive analysis, respectively. Besides age and PaO2/FiO2, which showed approximately symmetric distribution, all other continuous variables showed skewed distributions and are therefore expressed as median values with interquartile ranges (IQR). Comparison of age and PaO2/FiO2 by the parametric ANOVA test led to identical conclusions (data not shown).
Unweighted Kaplan-Meier curves were used to compare cumulative probabilities of invasive ventilation/death and of confirmed negative NP/OP swabs by treatment group. Stratification for baseline PaO2/FiO2 (0-300 mmHg vs. > 300 mmHg) and interaction test were used to test whether response to treatment groups might differ in subsets of participants.
A marginal Cox regression model with inverse probability of treatment weighting approach (IPW) was used to balance the differences in baseline and time-varying variables between treatment groups. Propensity scores to construct the weights were based on a vector of potential confounders identified a priori on the basis of axiomatic knowledge and previously published results. These included time-fixed variables measured at entry (i.e., gender, age, extent of comorbidity, and duration of symptoms), as well as time-varying confounders affected by initial treatment choice such as intensification by use of azithromycin, anticoagulants, steroids, and immunomodulatory drugs. A double-robust estimator was used, controlling also for potential informative censoring. The assumption of no positivity was checked by inspecting the distribution of the standardized combined weights. In a subset of participants with available data, we further controlled for baseline levels of inflammation and coagulation (CRP, ferritin, and d-dimer). In an additional sensitivity analysis, severity of disease at baseline was controlled using a diagnosis of pneumonia at entry in the study instead of the PaO2/FiO2 level.
An intention-to-treat approach was used. For endpoints including individual components of the composite endpoint (e.g., separately only invasive ventilation or death), the first event occurred was counted.
All statistical analyses were performed with SAS statistical package version 9.4 (Carey NC, United States).
Among the 590 patients included in the analysis, 109 received hydroxychloroquine, 124 received lopinavir/ritonavir, 244 received hydroxychloroquine plus lopinavir/ritonavir, and 113 did not receive any of them (standard of care). The latter group consisted of 44 people who did not start any drug, 55 who started anticoagulants, 25 steroids, 25 azithromycin, and 3 immunomodulatory drugs. The different treatment groups were not homogenous for sex, timing of hospitalization, pneumonia, baseline PaO2/FiO2, and some inflammatory biomarkers. In particular, patients in the SoC group had a higher PaO2/FiO2 at baseline (median 381, IQR 300-467 mmHg) and were less frequently diagnosed with pneumonia (68.1% of them); had lower LDH, AST, CRP, and ferritin; and higher neutrophils and lymphocytes count than the other three groups. Overall, 132 (22%) were treated also with azithromycin, 196 (33%) with corticosteroids at various dosage, 92 (16%) with immunomodulatory drugs, and 277 (47%) received heparin at various dosage. Concomitant use of azithromycin was most prevalent in the lopinavir/r group (n 45, 36 vs. 22% in SoC), while immunomodulatory drugs were most frequently used in the dual antiviral combination (n 51, 21% vs. 3% in SoC) ( Table 3).
Even considering only the strata of moderate patients (PaO2/ FiO 2 > 300 mmHg at baseline), no difference among the groups was detected (log rank p 0.43, Figure 2B).
Unadjusted and adjusted marginal relative hazards of invasive ventilation/death from fitting a marginal Cox regression model are shown in Table 4. This model was adjusted for age, gender, presence of comorbidities, duration of symptoms, time-varying use of immunomodulatory drugs, heparin and azithromycin, and censoring using IPW. There was no evidence of a difference in risk of invasive ventilation/death in the three treatment groups when compared to standard of care. The aHR was 1.09 (95% CI 0.60, 1.98) with LPV/r + HCQ, 0.81 (95% CI 0.38, 1.72) with HCQ, and 1.55 (95% CI 0.82, 2.93) with LPV vs. SoC.
The risk of invasive ventilation/death in the three groups appeared to vary by PaO2/FiO2, driven by the use of LPV/r or HCQ alone. In fact, for HCQ vs. SoC, the aHR resulted 1.48 (95% CI 0.35, 6.18) in the strata of patients with PaO2/FiO2 < 300 mmHg at baseline and 0.83 (95% CI 0.22, 3.18) in the strata of patients with PaO2/FiO2 > 300 mmHg at baseline, suggesting a more beneficial effect of HCQ in people with less severe disease. Results were similar for LPV/r vs. SoC, with an aHR of 2.48 (95% CI 0.65, 9.43) in the strata of patients with PaO2/FiO2 < 300 mmHg at baseline and 0.73 (95% CI 0.14, 3.95) in the strata of patients with PaO2/FiO2 > 300 mmHg at baseline (p-value for interaction <0.001) ( Table 4).
Results were similar in two additional sensitivity analyses. First, after we further controlled for baseline level of inflammation and coagulation in a subset of participants with available values of these markers (ferritin, CRP, and d-dimer), the aHR was 0.84 (95% CI 0.41, 1.70) with LPV/r + HCQ, 0.63 (95% CI 0.26, 1.54) with HCQ, and 1.15 (95% CI 0.54, 2.47) with LPV vs. SoC (Supplementary Table S1). The second sensitivity analysis was done after controlling for pneumonia instead of baseline levels of PaO2/FiO2, to try to remove bias due to imbalance in the severity of disease at entry. The HR in this

DISCUSSION
Herein, we report a retrospective study with real-world data collected from routine care to assess the clinical and virological efficacy of hydroxychloroquine, lopinavir/ritonavir, or the combination of hydroxychloroquine plus lopinavir/ ritonavir vs. SoC in a population of 590 patients admitted to our hospital for COVID-19 infection. We found that none of the antivirals investigated or their combination were associated with a reduction of invasive ventilation or death 14 days after starting of therapy compared with standard of care alone. Additionally, a reduction in estimated probability of any ventilation (noninvasive or invasive ventilation) or death was not demonstrated. Even in terms of ending of the viral shedding, in the subgroup of patients with positivity to SARS-CoV-2 in nasal/oropharyngeal swabs, we found no benefit with hydroxychloroquine, lopinavir/ritonavir, or the combination of hydroxychloroquine plus lopinavir/ ritonavir in comparison to the standard of care.
Our population is well-characterized and clinical features of our patients were consistent with many other reports, with a predominance of men, mean aged in the seventh decade of life. In contrast with other reports, we found a lower prevalence of comorbidities in our population (Docherty et al., 2020;Richardson et al., 2020), but higher number of symptomatic patients with fever (Richardson et al., 2020).
As expected, results of the comparison between HCQ or LPV/r monotherapy and SoC were similar to those of randomized trials, where no advantage in the use of HCQ or LPV/r was observed (Cavalcanti et al., 2020;NIH, 2020b; Recovery Randomised Evaluation of COVID-19 Therapy 2020a; Recovery Randomised Evaluation of COVID-19 Therapy, 2020b; World Health Organization, 2020). All our analyses took into account heparin use and corticosteroids, and this is particularly relevant in light of the possible association between anticoagulant treatment and decreased mortality in severe COVID-19 and above all in Frontiers in Pharmacology | www.frontiersin.org April 2021 | Volume 12 | Article 621676 7 light of recently published data from randomized and observational studies about benefits of corticosteroids in terms of clinical evolution (Group TRC, 2020;Salton et al., 2020) and mortality (Fadel et al., 2020;Fernández Cruz et al., 2020;Group TRC, 2020;Salton et al., 2020;Wu et al., 2020).
The risk of invasive ventilation/death in the groups appeared to vary by PaO2/FiO2 at baseline; its putative mechanism was not clear and could deserve further investigation. Antivirals were administered approximately 9 days after symptoms' initiation, similarly to other studies Goldman et al., 2020). It is possible that therapies that may limit viral replication may be more effective earlier in the course of the disease, so this could explain our signal for a greater potential benefit of HCQ and LPV/ r vs. SoC in the subset of people with less severe disease. According to Hung et al. (Hung et al., 2020), antivirals could also potentially have a role in reducing viral shedding, but we found no such evidence in our cohort. The analysis has been prompted by a recent RCT showing the beneficial effect of combining antivirals for the treatment of COVID-19 patients (Hung et al., 2020). Indeed, these strategies have been seldom compared in randomized studies and more research is needed. Of note, the effectiveness of the triple combination in that study was mainly ascribed to the use of interferon beta, which has potential to prevent SARS-CoV-2 from shutting down the host innate immunity in the first few days from infection (Richardson et al., 2020). Interferon beta was not used in this case, so this might explain why we could not replicate these earlier results. In general, COVID-19 is a complex disease from the standpoint of pathogenesis with different stages, so broad comparisons of drug A vs. B might not be as useful as trials designed to compare interventions tailored to patients following specific pathogenic pathways (e.g., cytokines storm as opposed to microcirculatory platelet aggregation, etc.). Indeed, the casemix of our study population was quite heterogeneous. The ending of viral shedding seemed to be possibly favored by HCQ, but only in patients with moderate COVID-19 and not confirmed in a further analysis adjusted for PaO2/FiO2 at baseline and stratified for duration of symptoms. Anyway, this is a subset analysis, so p-values should be considered with caution. Even previously published data showed conflicting results, but considering all the evidence cumulated to date on the lack of efficacy of HCQ, its beneficial role on SARS-COV-2 shedding should be considered unlikely (NIH, 2020a;Tang et al., 2020;Huang et al., 2020). Our study presents some limitations. First of all because it is retrospective and observational, we cannot rule out unmeasured confounding. The analysis also relies on specific assumptions regarding the underlying causal structure of the data (time-fixed and time-varying confounding factors) and the linear predictor of the model, with or without interaction terms, to be correctly specified. Thus, it is possible that a key variable was missing in our propensity scores; however, after further controlling for markers of inflammation/coagulation, results were similar. Also, although there was a standard treatment protocol in place, untreated patients in the SoC group might have been a selected population in which treatment was withheld because of predicted poor prognosis or conversely because of initial better evaluation. In fact, the SoC group showed healthier profiles at baseline, in terms of inflammation biomarkers and of leukocyte count and a rate of admission to the ICU which was comparable with the overall mortality (10.3% vs. 9.7%), suggesting that all participants had been equally considered for critical care. Further, other sub-analyses have been performed, taking into account a diagnosis of pneumonia and the baseline difference in biomarkers among groups and similar results were obtained. Indeed, in our population, patients allocated to SoC showed a much less COVID-19 disease severity and our propensity score adjustment should have minimized this imbalance. Also, because time zero of the survival analyses was the date of starting treatment, immortal bias also cannot be completely ruled out. However, the average time from hospital admission to therapy initiation was <1 day for 75% of the study population. Finally, safety data (i.e., risk of arrhythmia in people receiving HCQ) have not been analyzed in this work.
On the other hand, key strengths of this work were the detailed characterization of the study population, including the coadministered drugs, the possibility of comparing combination treatment strategies seldom investigated in randomized studies, and the use of a sophisticated counterfactual prediction framework to appropriately control for time-fixed and time-vary confounding factors.

CONCLUSION
In our retrospective analysis of real-life data of hospitalized patients with mild-to-severe COVID-19, we did not find a significant difference in clinical and virological outcome among lopinavir/ ritonavir, hydroxychloroquine, and lopinavir/ritonavir plus hydroxychloroquine or standard of care. Our results are consistent with those of randomized trials comparing monoantiviral treatment arms vs. placebo which led to the recommendation against the use of these antivirals for treatment of COVID-19 patients in national and international guidelines. Indeed, some of the early RCTs were of poor quality and risk of bias was high, but larger more recent and reliable studies confirmed these results. Additional RCTs specifically addressing the timing of initiation of these and other interventions according to patients' disease course and specific pathogenic pathways as well as the use a combination of approaches are further needed.