Development and internal validation of simplified predictive scoring (ICU-SEPSA score) for mortality in patients with multidrug resistant infection

Abstract

Background: Mortality from multidrug-resistant (MDR) pathogens is an urgent healthcare crisis worldwide. At present we do not have any simplified screening tools to predict the risk of mortality associated with MDR infections. The aim of this study was to develop a screening tool to predict mortality in patients with multidrug-resistant organisms.

Methods: A retrospective cohort study to evaluate mortality risks in patients with MDR infections was conducted at Phrae Hospital. Univariable and multivariable analyses were used to classify possible risk factors. The model performance was internally validated utilizing the mean of three measures of discrimination corrected by the optimism using a 1000-bootstrap procedure. The coefficients were transformed into item scores by dividing each coefficient with the lowest coefficient and then rounding to the most adjacent number. The area under the receiver operating characteristic curve (AuROC) was used to determine the performance of the model.

Results: Between 1 October 2018 and 30 September 2020, a total of 504 patients with MDR infections were enrolled. The ICU-SEPSA score composed of eight clinical risk factors: 1) immunocompromised host, 2) chronic obstructive pulmonary disease, 3) urinary tract infection, 4) sepsis, 5) placement of endotracheal tube, 6) pneumonia, 7) septic shock, and 8) use of antibiotics within the past 3 months. The model showed good calibration (Hosmer-Lemeshow χ2 = 19.27; p-value = 0.50) and good discrimination after optimism correction (AuROC 84.6%, 95% confidence interval [Cl]: 81.0%–88.0%). The positive likelihood ratio of low risk (score ≤ 5) and high risk (score ≥ 8) were 2.07 (95% CI: 1.74–2.46) and 12.35 (95% CI: 4.90–31.13), respectively.

Conclusion: A simplified predictive scoring tool wad developed to predict mortality in patients with MDR infections. Due to a single-study design of this study, external validation of the results before applying in other clinical practice settings is warranted.

Introduction

The burden of infections caused by multidrug-resistant organisms (MDROs) is increasing worldwide, while the available antibiotics become less effective over time. (Morrison and Zembower, 2020) Fifteen new antibiotics have been developed and launched in the market in a recent year, but only two of them can be effectively used to treat multidrug resistant gram-negative bacterial infections. (Carlet et al., 2012) Consequently, bacterial antimicrobial resistance is a critical and urgent problem that needs to be addressed globally. In 2019, it was estimated that the highest rate of antimicrobial-resistant burden was found in sub-Saharan Africa. Approximately 27.3 deaths per 100,000 population had been reported in the region. Other affected areas with high incidences included South Asia, Europe, Latin America, and Oceania. Most common organisms leading to deaths associated with resistance were 1) Pseudomonas aeruginosa, 2) Acinetobacter baumannii, 3) Escherichia coli, 4) Staphylococcus aureus, 5) Streptococcus pneumoniae, and 6) Klebsiella pneumoniae. Predominantly, respiratory infections, bloodstream infections, and intra-abdominal infections were among the most common infectious diseases associated with antimicrobial-resistance. (Murray et al., 2022) According to collective evidence from 1,023 hospitals in Thailand in 2010, almost one-third of hospitalized patients diagnosed with infectious diseases had multidrug-resistant (MDR) infections. The previous report revealed that A. baumannii (57.6%) and P. aeruginosa (20%–30%) were the leading organisms of MDR infections, respectively. (Anudit et al., 2016) Additionally, infections with gram-negative bacteria (i.e., Enterobacteriaceae) are crucial challenges in any health-care settings. It is widely recognized that they have become difficult to treat and often require a complex treatment plan. Despite optimal and aggressive therapy, they frequently lead to increased morbidity and mortality, resulted in heightening anxiety worldwide. (Kanchanasurakit et al., 2020).

MDR infections are associated with many problems including prolonged length of hospital stays, increased cost of treatment and mortality rates. (Cosgrove, 2006; Sunenshine et al., 2007; Tanwar et al., 2014; Lim et al., 2016; Founou et al., 2017) The previous studies have shown a relationship between higher rates of MDROs and mortality. (Lim et al., 2016; Mave et al., 2017; Teerawattanapong et al., 2018) The actual causes for this association are not well defined. Comorbidities and altered immune response of patients may also contribute to the increased risk of mortality. Nevertheless, other factors including the patient’s clinical status and source or severity of infection may have played a pivotal role. (Gandra et al., 2019) At present, we do not have any tools to predict the risk of mortality in these patients.

To the extent of our knowledge, we believe that a tool used for prediction of mortality in patients with MDR infections has not been established. The previous study has identified a risk scoring system to predict risk of acquiring MDR infection, however mortality outcome was not addressed. (Tseng et al., 2017) While there were tools to predict mortality in patients diagnosed with bacteremia, those tools did not specifically focus on patients with MDROs. (Shapiro et al., 2003, 2007; Taniguchi et al., 2017) The objective of this study was to develop a risk scoring model to predict mortality for patients with MDR infections.

Materials and methods

Setting

This study was carried out at Phrae Hospital, a 500-bed medical center, located in the northern part of Thailand.This work followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement (Supplementary Table S1). (Moons et al., 2015).

Study design and study population

We conducted a single-center, retrospective cohort study of hospitalized adults (aged ≥ 18 years) with at least one positive culture of MDROs during the admission at Phrae Hospital between 1 October 2018, and 30 September 2020. A total of 504 patients were enrolled in the study. For individuals with multiple positive culture results within the same admission, we only used the data from their first result for analysis. Patients with negative culture results as well as those with positive culture results with mono resistance or susceptible to antibiotics were excluded. In addition, patients with missing data in the medical record were excluded.

Multidrug resistant bacteria was defined as acquired non-susceptibility to at least one agent in three or more antimicrobial categories. (Falagas and Karageorgopoulos, 2008; Magiorakos et al., 2012) Antibiotics are classified into one of the following classes: aminoglycosides, penicillins, penicillins plus beta-lactamase inhibitors, cephalosporins, fluoroquinolones, tetracyclines, macrolides, carbapenems, folate pathway inhibitors, glycopeptides, polymyxins, and lincosamides. (Magiorakos et al., 2012) In this study, the infectious disease physician diagnosed and registered patients with MDR infections depending on susceptibility tests and clinical manifestations.

Data collection and definition

Demographic characteristics including age, sex, status of patients, receipt of intensive care unit (ICU) admission, duration of hospitalization, comorbidities, source of infection, risk factors for MDR, disease severity, and use of medical devices were collected. Comorbidities were identified according to the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10) code version 2016. These were: essential hypertension (I10), diabetes mellitus type 2 (E11.0-E11.9), dyslipidemia (E78.5), heart failure (I50.0-I50.9), chronic kidney disease (N18.3-N18.9), atrial fibrillation (I48.0-I48.9), chronic obstructive pulmonary disease (J44.9), gout (M10.9), liver cirrhosis (K74.6), coronary artery disease (I20.0, I21.3-I21.4, I25.2), epilepsy (G40.9), Alzheimer’s disease (G30.9), Parkinson’s disease (G20), cerebrovascular disease including stroke (I63.0-I63.9), depression (F32.9), anxiety (F41.1-41.2, 41.9), and schizophrenia (F20.9).

Immunocompromised hosts were defined as patients with active cancer, current use of chemotherapy or immunosuppressant drugs, Acquired Immune Deficiency Syndrome (AIDS), chronic alcohol use, dialysis including hemodialysis and continuous ambulatory peritoneal dialysis, and uncontrolled diabetes mellitus with HbA1C >7%. (Delves and Roitt, 2000) Sepsis and septic shock were identified according to the Sepsis-3 criteria. (Singer et al., 2016) Sepsis was defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. For clinical operationalization, organ dysfunction can increase the Sequential Organ Failure Assessment (SOFA) score of 2 points or more. Septic shock can be clinically identified if patients were in need of vasopressor to maintain a mean arterial pressure of 65 mmHg or greater and serum lactate level greater than 2 mmol/L (>18 mg/dl) in the absence of hypovolemia (Kanchanasurakit et al., 2021). In this study, mortality was defined as an in-hospital mortality secondary to infections. The cause of death was confirmed by physicians specialized in infectious disease.

Statistical analysis

Baseline characteristics are reported using frequency (percentage), mean (SD), or median (interquartile range) as appropriate. To analyze mortality associated with MDROs-infected patients, we conducted univariable analysis using chi-square or Fisher’s exact test. Odds ratios (ORs) and 95% confidence intervals (95%CI) were evaluated for each factor. Variables with p-value ≤ 0.1 were then selected into the multivariable model using logistic regression. Any risk factors with p-value < 0.05 were integrated in the process of developing the risk prediction model for mortality. Additionally, Akaike information criterion (AIC) and Bayesian information criteria (BIC) were used to help identify variables that would be incorporated in the predictive model. The model with the lowest AIC and BIC values was chosen. (Chakrabarti and Ghosh, 2011) Model performance in the original sample was evaluated using the C-statistic, Hosmer-Lemeshow statistic, calibration slope, and Brier score (Huang et al., 2020).

Sample size

We calculated sample size by applying 20 outcome events per predictor variable (EPV). (Austin and Steyerberg, 2017) Based on the previous study that used five binary variables (Zhou et al., 2019), at least 100 patients with an outcome are required.

Model internal validation

Internal validation was performed using a bootstrap procedure with 1,000 bootstrapped samples. We calculated the bootstrap performance (i.e., calibration slope, calibration-in-the-large [CITL] and C-statistic) of the final model to derive an optimism adjusting factor to correct the model for overfitting. To develop a scoring system, each variable was assessed and turned into a point using their weighted coefficients. These were then rounded to the most adjacent number to be used as a score.

Score development

To derive the point-based system, optimism adjusted coefficient and the intercept from the final multivariable model were used. We weighted and transformed each coefficient by dividing it by the lowest coefficient in the model and rounding the results to the most adjacent number. (Harrell, 1996; Moons et al., 2002) To examine the discriminative power of the predictive score we developed, a receiver operating characteristic (ROC) curve was plotted and a Hosmer-Lemeshow goodness-of-fit test was performed. (Steyerberg et al., 2010) The cut-off score was chosen to classify patients into three groups based on the risk of mortality: low (≤ 49.54%), moderate (56.98%–81.42%), and high (≥ 85.53%), respectively. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, positive likelihood ratio (LR+), and negative likelihood ratio (LR-) were assessed. (Leisenring and Pepe, 1998; Janssens et al., 2005) The multi-collinearity was obtained using the variance inflation factor (VIF).

Score validation

The score was internally validated using bootstrapping. A total of 1,000 bootstrap samples were drawn and the final model containing all included variables was fitted in each bootstrap sample. This process was followed by re-simplification of the score and point assignment using the logistic regression model in each bootstrap sample. Performance of the re-fitted, simplified score was appraised in each bootstrap sample and in the original sample using the performance measures described above. Mean bootstrap and bootstrap test performances were assessed. Subsequently, optimism and optimism-corrected performance estimates were calculated.

Results

Characteristics of patients

From, 1 October 2018, through 30 September 2020, a total of 4,217 patients with culture results were identified during the study period. After excluding 3,713 patients with negative culture results, positive culture results with mono-resistance to antibiotics, missing data, or susceptible of antibiotics, 504 patients were included in the final analysis. There were 172 deaths (35%) during the follow-up (Figure 1). The mean age of the patients were 65 years, and the majority of patients (58%) were male. The mean duration of hospitalization was 18.39 days. Compared between the survival and non-survivor groups, there was no difference in bedridden status and comorbid conditions, except for heart failure, atrial fibrillation, and chronic obstructive pulmonary disease. Patients that died had recent admission at other facilities and later transferred to Phrae Hospital, admitted in the ICU, known immunocompromised status, and had recent antibiotic use with the past 3 months. Both groups had similar reports of wound infection. The survival group had less severe infections as well as fewer uses of medical devices. Baseline characteristics of both groups are shown in Table 1.

FIGURE 1

Predictors for mortality in patients with multidrug-resistant organisms

Univariate analysis for mortality showed a significant association with hospitalization more than 14 days (OR 1.88, 95% CI: 1.27–2.78, p-value = 0.001), admission at ICU (OR 1.70, 95% CI: 1.10–2.61 p-value = 0.014), immunocompromised host (OR 1.76, 95% CI: 1.19–2.60, p-value = 0.004), heart failure (OR 3.48, 95% CI: 1.23–10.62, p-value = 0.012), atrial fibrillation (OR 3.31, 95% CI: 1.48–7.67, p-value = 0.002), chronic obstructive pulmonary disease (OR 2.78, 95% CI: 1.44–5.42, p-value = 0.002), bacteremia (OR 0.32, 95% CI: 0.21–0.48, p-value< 0.001), urinary tract infection (OR 2.16, 95% CI: 1.33–3.49, p-value = 0.001), pneumonia (OR 7.11, 95% CI: 3.63–14.56, p-value< 0.001), use of antibiotics within the last 3 months (OR 1.77, 95% CI: 1.19–2.61, p-value = 0.003), sepsis (OR 2.98, 95% CI: 1.99–4.46, p-value< 0.001), septic shock (OR 4.51, 95% CI: 2.83–7.23, p-value< 0.001), placement of foley catheter (OR 4.24, 95% CI: 2.75–6.61, p-value< 0.001), and placement of endotracheal tube (OR 6.98, 95% CI: 4.54–10.73, p-value< 0.001) (Table 2).

Model development

In the multivariable analysis, some variables were independent predictors of mortality. These were chronic obstructive pulmonary disease (OR 3.59, 95% CI:1.61–7.99, p-value = 0.002), pneumonia (4.86, 95% CI: 2.29–10.27, p-value<0.001), urinary tract infection (OR 3.24, 95% CI: 1.85–5.66, p-value<0.001), use of antibiotics within the past 3 months (OR 1.89, 95% CI: 1.18–3.01, p-value = 0.008), immunocompromised host (OR 2.24, 95% CI: 1.39–3.62, p-value = 0.001), sepsis (OR 2.24, 95% CI: 1.31–3.82, p-value = 0.003), septic shock (OR 2.02, 95% CI: 1.09–3.74, p-value = 0.026), and placement of endotracheal tube (OR 5.30, 95% CI: 3.27–8.59, p-value<0.001) (Table 3 and Figure 2). Since the VIF values of each variable were less than 5 (averaged value was 1.17), multi-collinearity was not a major identified in our study. We have analyzed the AIC and BIC scores to help identify variables that would be incorporated in the predictive model. The model with the lowest AIC and BIC values was chosen as the best model for predicting mortality in patients with MDR infection (Supplementary Table S2).

FIGURE 2

This scoring system was created by setting the cut-off point according to the differentiation plot and performance of diagnostic parameters to identify patients at risk of increased mortality. Cut-off scores of 5 and 8 were chosen to classify patient into 3 groups (Table 4). Patients with a score ≥ 8 were placed in the high-risk group. Patients in this group were correctly predicted for the mortality outcome 86.49% (32/37) of the time. Patients with a score ≤ 5 were placed in the low-risk group and correctly predicted for the mortality outcome 79.95% (299/374) of the time. Correct prediction of survival or death occurred 80.54% [(299 + 32)/(374 + 37)] of the time. The incorrect prediction rate was 19.46% [(75 + 5)/(374 + 37)]. Based on this scoring system and one cut-off point, the original performance model showed good discrimination (AUC = 84.60%, 95%Cl: 81%–88%; Hosmer-Lemeshow χ2 = 19.27; p-value = 0.50) and calibration (calibration slope, 1.000; Brier score, 0.149; product moment correlation between observed and predicted probability, 1.000). Internally validated performance that gave an AUC similar original performance model and calibration (calibration slope, 1.061; Brier score, 0.149; product moment correlation between observed and predicted probability, 0.983) (Figures 3, 4A,B). (Marcin and Romano, 2007) The probability of mortality is presented in Figure 5.

FIGURE 3

FIGURE 4

FIGURE 5

Discussion

Due to lack of predictive scoring for mortality in patients with multidrug resistant infection, we aimed to develop a risk scoring model to predict mortality in this population. Our study revealed that a clinical scoring system could be used to screen and predict the mortality in patients with MDROs. Our newly formulated tool, named “ICU-SEPSA score”, was able to classify patients at risk of mortality.

During our study, we collected data of 504 patients from October 2018 to September 2020. Using newly identified eight factors, we were able to categorize patients into three groups in relation to mortality risk: low, moderate, and high. These factors were 1) Immunocompromised host, 2) chronic obstructive pulmonary disease, 3) urinary tract infection, 4) sepsis, 5) placement of endotracheal tube, 6) pneumonia, 7) septic shock, and 8) use of antibiotics within the past 3 months. We observed similar findings compared to previous studies. (Miravitlles et al., 1999; Johnson et al., 2011; Lee et al., 2016; Moreau et al., 2018; Capsoni et al., 2019; Cillóniz et al., 2019; Du et al., 2019) While the exact relationship between immunocompromised hosts and mortality is unknown, the study by Moreau et al. (2018) found that immunocompromised hosts were linked to recent use of antibiotics. This is a well-known risk factor for developing MDROs and increased risk of mortality. In our study, we found that sepsis and septic shock were associated with higher mortality rates due to multiple organ dysfunction. (Capsoni et al., 2019) Use of any medical devices that posed an increased risk of bacteria growth either through lower bacteria clearance or epithelial damage that allowed formation of biofilm, can result in higher risk of mortality. In our study, we found that endotracheal tube was associated with mortality in MDROs infections. (Nseir et al., 2006) In addition, a previous systematic review and meta-analysis showed that intubation was considered as an invasive techniques associated with adverse outcomes. (Du et al., 2019) It has been established that some comorbidities are associated with mortality, particularly chronic obstructive pulmonary disease, but the mechanism is unclear. A previous study suggested that patients with acute exacerbation of chronic obstructive pulmonary disease required intubation or ventilators more often. (Nseir et al., 2006) The sputum culture of patients with severe pulmonary function impairment, indicated by forced expiratory volume in one second (FEV-1) <50%, has shown to contain more potentially pathogenic microorganisms, such as Haemophilus influenzae or P. aeruginosa. (Miravitlles et al., 1999) In our study, we found that certain types of infections (urinary tract infection and pneumonia) led to increased risks of mortality. This may be the result of receiving insufficient empiric treatment leading to the development of MDROs infections. (Lee et al., 2016; Cillóniz et al., 2019) The last factor correlated to mortality in patients with MDR pathogens was the use of antibiotics within the past 3 months. We hypothesized that prior antibiotic use led to higher likelihood of antimicrobial resistance. (Johnson et al., 2011)

Our study has a number of strengths. To the extent of our knowledge, this was the first time that risk scoring tool to predict mortality secondary to any MDROs infections was identified. Our data analysis was not restricted to a particular type of infections. We evaluated various sources of cultures, including blood, urine, sputum, catheter, ascitic fluid and pus. Our tool was user friendly. It was simple and did not require any specific or expensive laboratory investigation. The newly established tool, named “ICU-SEPSA score” consisted of eight predictors, is easy to remember for clinicians. We foresee a great impact of this tool to help assess mortality risks in MDRO-infected patients. Lastly, the power of this model was sufficient to detect a difference between the two groups.

There are several limitations to this study. First, the data was collected retrospectively from medical records and electronic database. There was a potential absence of information that could impact data analysis and outcome such as type of organism, smoking status, history of alcohol use, and pertinent past medical history. In addition, during medical record review, we were unable to assess the severity of some variables such as urinary tract infection and pneumonia. Third, this study was designed as a single-center study, therefore, the ability to generalize our results may be limited. Lastly, it is worth noting that our model has not been conducted in the external validation cohort.

Our newly formulated tool may be incorporated into clinical practice to help healthcare providers estimate the risk of mortality in this population. A high score corresponds to a greater risk of mortality in patients with MDROs infections that should assist clinicians to promptly identify the appropriate therapy. To implement our findings in clinical setting, we suggested that patients who classified into the high-risk group (score ≥ 8) should be considered for admission in the ICU for close monitoring. These patients should receive combination antibiotic therapy for synergistic effect and reduced risk of modified risk factors. Frequent vital sign monitoring is warranted for moderate-risk group (score 5.5–7.5) as well as the high-risk group. At Phrae Hospital, vital sign monitoring as frequently as every 2 h had been suggested. In addition, every effort to minimize prolonged use of any catheters should be considered. Patients in the low-risk group (score ≤ 5) should still have vital signs monitored regularly according to the standard of care at that clinical setting. Regardless of which group patients are categorized into, MDROs infections should be immediately assessed. Aggressive therapy with close monitoring is key to lower risks of mortality and optimize overall outcome.

Conclusion

A simplified predictive scoring tool wad developed to predict mortality in patients with MDROs infections. The score was composed of eight factors including 1) immunocompromised host, 2) chronic obstructive pulmonary disease, 3) urinary tract infection, 4) sepsis, 5) placement of endotracheal tube, 6) pneumonia, 7) septic shock, and 8) use of antibiotics within the past 3 months. Due to a single-study design of this study, external validation of the results before applying in other clinical practice settings is warranted.

References

• 1

  AnuditC.KooltheatN.PotupP.Pankla SranujitR.UsuwanthimK. (2016). Nosocomial infection of multidrug-resistant Acinetobacter baumannii in Thailand. Am. J. Infect. Control44, 1161–1163. 10.1016/j.ajic.2016.03.068

  • CrossRef
  • Google Scholar

• 2

  AustinP. C.SteyerbergE. W. (2017). Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Stat. Methods Med. Res.26, 796–808. 10.1177/0962280214558972

  • CrossRef
  • Google Scholar

• 3

  CapsoniN.BelloneP.AlibertiS.SotgiuG.PavanelloD.VisintinB.et al (2019). Prevalence, risk factors and outcomes of patients coming from the community with sepsis due to multidrug resistant bacteria. Multidiscip. Respir. Med.14, 23. 10.1186/s40248-019-0185-4

  • CrossRef
  • Google Scholar

• 4

  CarletJ.JarlierV.HarbarthS.VossA.GoossensH.PittetD.et al (2012). Ready for a world without antibiotics? The pensières antibiotic resistance call to action. Antimicrob. Resist. Infect. Control1, 11. 10.1186/2047-2994-1-11

  • CrossRef
  • Google Scholar

• 5

  ChakrabartiA.GhoshJ. K. (2011). “AIC, BIC and recent advances in model selection,” in Handbook of the philosophy of science. Editors BandyopadhyayP. S.ForsterM. R. B. T.-P. S. (Amsterdam: North-Holland), 583–605. 10.1016/B978-0-444-51862-0.50018-6

  • CrossRef
  • Google Scholar

• 6

  CillónizC.DominedòC.TorresA. (2019). Multidrug resistant gram-negative bacteria in community-acquired pneumonia. Annu. Updat. Intensive Care Emerg. Med.2019, 459–475. 10.1007/978-3-030-06067-1_36

  • CrossRef
  • Google Scholar

• 7

  CosgroveS. E. (2006). The relationship between antimicrobial resistance and patient outcomes: mortality, length of hospital stay, and health care costs. Clin. Infect. Dis.42, S82–S89. 10.1086/499406

  • CrossRef
  • Google Scholar

• 8

  DelvesP. J.RoittI. M. (2000). The immune system. First of two parts. N. Engl. J. Med.343, 37–49. 10.1056/NEJM200007063430107

  • CrossRef
  • Google Scholar

• 9

  DuX.XuX.YaoJ.DengK.ChenS.ShenZ.et al (2019). Predictors of mortality in patients infected with carbapenem-resistant acinetobacter baumannii: A systematic review and meta-analysis. Am. J. Infect. Control47, 1140–1145. 10.1016/j.ajic.2019.03.003

  • CrossRef
  • Google Scholar

• 10

  FalagasM. E.KarageorgopoulosD. E. (2008). Pandrug resistance (PDR), extensive drug resistance (XDR), and multidrug resistance (MDR) among gram-negative bacilli: need for international harmonization in terminology. Clin. Infect. Dis.46, 1121–1122. 10.1086/528867

  • CrossRef
  • Google Scholar

• 11

  FounouR. C.FounouL. L.EssackS. Y. (2017). Clinical and economic impact of antibiotic resistance in developing countries: A systematic review and meta-analysis. PLoS One12, e0189621. 10.1371/journal.pone.0189621

  • CrossRef
  • Google Scholar

• 12

  GandraS.TsengK. K.AroraA.BhowmikB.RobinsonM. L.PanigrahiB.et al (2019). The mortality burden of multidrug-resistant pathogens in India: A retrospective, observational study. Clin. Infect. Dis.69, 563–570. 10.1093/cid/ciy955

  • CrossRef
  • Google Scholar

• 13

  HarrellF. (1996). Regression coefficients and scoring rules. J. Clin. Epidemiol.49, 819. 10.1016/0895-4356(95)00068-2

  • CrossRef
  • Google Scholar

• 14

  HuangY.LiW.MacheretF.GabrielR. A.Ohno-MachadoL. (2020). A tutorial on calibration measurements and calibration models for clinical prediction models. J. Am. Med. Inf. Assoc.27, 621–633. 10.1093/jamia/ocz228

  • CrossRef
  • Google Scholar

• 15

  JanssensA. C. J. W.DengY.BorsboomG. J. J. M.EijkemansM. J. C.HabbemaJ. D. F.SteyerbergE. W. (2005). A new logistic regression approach for the evaluation of diagnostic test results. Med. Decis. Mak.25, 168–177. 10.1177/0272989X05275154

  • CrossRef
  • Google Scholar

• 16

  JohnsonM. T.ReichleyR.Hoppe-BauerJ.DunneW. M.MicekS.KollefM. (2011). Impact of previous antibiotic therapy on outcome of Gram-negative severe sepsis. Crit. Care Med.39, 1859–1865. 10.1097/CCM.0b013e31821b85f4

  • CrossRef
  • Google Scholar

• 17

  KanchanasurakitS.SantimaleeworagunW.McPhersonC. E.PiriyachananusornN.BoonsongB.KatwilatP.et al (2020). Fosfomycin dosing regimens based on Monte Carlo simulation for treated carbapenem-resistant enterobacteriaceae infection. Infect. Chemother.5, 516–529. 10.3947/ic.2020.52.4.516

  • CrossRef
  • Google Scholar

• 18

  KanchanasurakitS.SuthumpoungP.SantimaleeworagunW.NakaranurackC.HuynhN. S.SrisawatC.et al (2021). Effectiveness of thiamine therapy in mortality rate in patients with septic shock: A systematic review and meta-analysis. Int. J. Crit. Illn. Inj. Sci.11, 86–94. 10.4103/IJCIIS.IJCIIS_159_20

  • CrossRef
  • Google Scholar

• 19

  LeeY.-C.HsiaoC.-Y.HungM.-C.HungS.-C.WangH.-P.HuangY.-J.et al (2016). Bacteremic urinary tract infection caused by multidrug-resistant enterobacteriaceae are associated with severe sepsis at admission: Implication for empirical therapy. Med. Baltim.95, e3694. 10.1097/MD.0000000000003694

  • CrossRef
  • Google Scholar

• 20

  LeisenringW.PepeM. S. (1998). Regression modelling of diagnostic likelihood ratios for the evaluation of medical diagnostic tests. Biometrics54, 444–452. 10.2307/3109754

  • CrossRef
  • Google Scholar

• 21

  LimC.TakahashiE.HongsuwanM.WuthiekanunV.ThamlikitkulV.HinjoyS.et al (2016). Epidemiology and burden of multidrug-resistant bacterial infection in a developing country. Elife5, e18082. 10.7554/eLife.18082

  • CrossRef
  • Google Scholar

• 22

  MagiorakosA.-P.SrinivasanA.CareyR. B.CarmeliY.FalagasM. E.GiskeC. G.et al (2012). Multidrug-resistant, extensively drug-resistant and pandrug-resistant bacteria: an international expert proposal for interim standard definitions for acquired resistance. Clin. Microbiol. Infect.18, 268–281. 10.1111/j.1469-0691.2011.03570.x

  • CrossRef
  • Google Scholar

• 23

  MarcinJ. P.RomanoP. S. (2007). Size matters to a model’s fit. Crit. Care Med.35, 2212–2213. 10.1097/01.CCM.0000281522.70992.EF

  • CrossRef
  • Google Scholar

• 24

  MaveV.ChandanwaleA.KagalA.KhadseS.KadamD.BharadwajR.et al (2017). High burden of antimicrobial resistance and mortality among adults and children with community-onset bacterial infections in India. J. Infect. Dis.215, 1312–1320. 10.1093/infdis/jix114

  • CrossRef
  • Google Scholar

• 25

  MiravitllesM.EspinosaC.Fernández-LasoE.MartosJ. A.MaldonadoJ. A.GallegoM. (1999). Relationship between bacterial flora in sputum and functional impairment in patients with acute exacerbations of COPD. Study Group of Bacterial Infection in COPD. Chest116, 40–46. 10.1378/chest.116.1.40

  • CrossRef
  • Google Scholar

• 26

  MoonsK. G. M.AltmanD. G.ReitsmaJ. B.IoannidisJ. P. A.MacaskillP.SteyerbergE. W.et al (2015). Transparent reporting of a multivariable prediction model for individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann. Intern. Med.162, W1–W73. 10.7326/M14-0698

  • CrossRef
  • Google Scholar

• 27

  MoonsK. G. M.HarrellF. E.SteyerbergE. W. (2002). Should scoring rules be based on odds ratios or regression coefficients?J. Clin. Epidemiol.55, 1054–1055. 10.1016/s0895-4356(02)00453-5

  • CrossRef
  • Google Scholar

• 28

  MoreauA.-S.Martin-LoechesI.PovoaP.SalluhJ.RodriguezA.ThilleA. W.et al (2018). Impact of immunosuppression on incidence, aetiology and outcome of ventilator-associated lower respiratory tract infections. Eur. Respir. J.51, 1701656. 10.1183/13993003.01656-2017

  • CrossRef
  • Google Scholar

• 29

  MorrisonL.ZembowerT. R. (2020). Antimicrobial resistance. Gastrointest. Endosc. Clin. N. Am.30, 619–635. 10.1016/j.giec.2020.06.004

  • CrossRef
  • Google Scholar

• 30

  MurrayC. J.IkutaK. S.ShararaF.SwetschinskiL.Robles AguilarG.GrayA.et al (2022). Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet399, 629–655. 10.1016/S0140-6736(21)02724-0

  • CrossRef
  • Google Scholar

• 31

  NseirS.Di PompeoC.CavestriB.JozefowiczE.NyungaM.SoubrierS.et al (2006). Multiple-drug-resistant bacteria in patients with severe acute exacerbation of chronic obstructive pulmonary disease: Prevalence, risk factors, and outcome. Crit. Care Med.34, 2959–2966. 10.1097/01.CCM.0000245666.28867.C6

  • CrossRef
  • Google Scholar

• 32

  ShapiroN. I.HowellM. D.TalmorD.DonninoM.NgoL.BatesD. W. (2007). Mortality in emergency department sepsis (MEDS) score predicts 1-year mortality. Crit. Care Med.35, 192–198. 10.1097/01.CCM.0000251508.12555.3E

  • CrossRef
  • Google Scholar

• 33

  ShapiroN. I.WolfeR. E.MooreR. B.SmithE.BurdickE.BatesD. W. (2003). Mortality in emergency department sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit. Care Med.31, 670–675. 10.1097/01.CCM.0000054867.01688.D1

  • CrossRef
  • Google Scholar

• 34

  SingerM.DeutschmanC. S.SeymourC. W.Shankar-HariM.AnnaneD.BauerM.et al (2016). The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA315, 801–810. 10.1001/jama.2016.0287

  • CrossRef
  • Google Scholar

• 35

  SteyerbergE. W.VickersA. J.CookN. R.GerdsT.GonenM.ObuchowskiN.et al (2010). Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology21, 128–138. 10.1097/EDE.0b013e3181c30fb2

  • CrossRef
  • Google Scholar

• 36

  SunenshineR. H.WrightM.-O.MaragakisL. L.HarrisA. D.SongX.HebdenJ.et al (2007). Multidrug-resistant Acinetobacter infection mortality rate and length of hospitalization. Emerg. Infect. Dis.13, 97–103. 10.3201/eid1301.060716

  • CrossRef
  • Google Scholar

• 37

  TaniguchiL. U.PiresE. M. C.VieiraJ. M. J.AzevedoL. C. P. de (2017). Systemic inflammatory response syndrome criteria and the prediction of hospital mortality in critically ill patients: a retrospective cohort study. Rev. Bras. Ter. intensiva29, 317–324. 10.5935/0103-507X.20170047

  • CrossRef
  • Google Scholar

• 38

  TanwarJ.DasS.FatimaZ.HameedS. (2014). Multidrug resistance: an emerging crisis. Interdiscip. Perspect. Infect. Dis.2014, 541340. 10.1155/2014/541340

  • CrossRef
  • Google Scholar

• 39

  TeerawattanapongN.PanichP.KulpokinD.Na RanongS.KongpakwattanaK.SaksinanonA.et al (2018). A systematic review of the burden of multidrug-resistant healthcare-associated infections among intensive care unit patients in southeast Asia: The rise of multidrug-resistant acinetobacter baumannii. Infect. Control Hosp. Epidemiol.39, 525–533. 10.1017/ice.2018.58

  • CrossRef
  • Google Scholar

• 40

  TsengW.-P.ChenY.-C.YangB.-J.ChenS.-Y.LinJ.-J.HuangY.-H.et al (2017). Predicting multidrug-resistant gram-negative bacterial colonization and associated infection on hospital admission. Infect. Control Hosp. Epidemiol.38, 1216–1225. 10.1017/ice.2017.178

  • CrossRef
  • Google Scholar

• 41

  ZhouH.YaoY.ZhuB.RenD.YangQ.FuY.et al (2019). Risk factors for acquisition and mortality of multidrug-resistant acinetobacter baumannii bacteremia: A retrospective study from a Chinese hospital. Med. Baltim.98, e14937. 10.1097/MD.0000000000014937

  • CrossRef
  • Google Scholar