Development and Validation of a Nomogram for the Prediction of Hospital Mortality of Patients With Encephalopathy Caused by Microbial Infection: A Retrospective Cohort Study

Background Hospital mortality is high for patients with encephalopathy caused by microbial infection. Microbial infections often induce sepsis. The damage to the central nervous system (CNS) is defined as sepsis-associated encephalopathy (SAE). However, the relationship between pathogenic microorganisms and the prognosis of SAE patients is still unclear, especially gut microbiota, and there is no clinical tool to predict hospital mortality for SAE patients. The study aimed to explore the relationship between pathogenic microorganisms and the hospital mortality of SAE patients and develop a nomogram for the prediction of hospital mortality in SAE patients. Methods The study is a retrospective cohort study. The lasso regression model was used for data dimension reduction and feature selection. Model of hospital mortality of SAE patients was developed by multivariable Cox regression analysis. Calibration and discrimination were used to assess the performance of the nomogram. Decision curve analysis (DCA) to evaluate the clinical utility of the model. Results Unfortunately, the results of our study did not find intestinal infection and microorganisms of the gastrointestinal (such as: Escherichia coli) that are related to the prognosis of SAE. Lasso regression and multivariate Cox regression indicated that factors including respiratory failure, lactate, international normalized ratio (INR), albumin, SpO2, temperature, and renal replacement therapy were significantly correlated with hospital mortality. The AUC of 0.812 under the nomogram was more than that of the Simplified Acute Physiology Score (0.745), indicating excellent discrimination. DCA demonstrated that using the nomogram or including the prognostic signature score status was better than without the nomogram or using the SAPS II at predicting hospital mortality. Conclusion The prognosis of SAE patients has nothing to do with intestinal and microbial infections. We developed a nomogram that predicts hospital mortality in patients with SAE according to clinical data. The nomogram exhibited excellent discrimination and calibration capacity, favoring its clinical utility.


INTRODUCTION
Sepsis is defined as a life-threatening organ dysfunction with host response imbalance caused by infection (Singer et al., 2016). Sepsis-associated encephalopathy (SAE) is defined as diffuse brain dysfunction without the central nervous system (CNS) infection in sepsis patients. Metabolic encephalopathy, drug intoxication, structural brain lesions, cerebrovascular events, encephalitis, meningitis, and non-convulsive status epilepticus need to be ruled out in sepsis patients before a diagnosis of SAE (Eidelman et al., 1996). SAE develops in up to 70% of septic patients (Gofton and Young, 2012;Fraser et al., 2014).
SAE is related to increased mortality, extensive costs, prolonged hospitalization, followed by persistent cognitive impairment (Iwashyna et al., 2010;Sonneville et al., 2017). The mortality rates of SAE patients over 60% in sepsis patients (Eidelman et al., 1996;Schuler et al., 2018). At hospital discharge, 45% of patients are related to the development of dementia (Annane and Sharshar, 2015). Early recognition of brain injury and prompt management are of great importance for the survival and prognosis of septic patients. Intestinal microbial infection is one of the important sites of infection in patients with sepsis. Intestinal microbes are not only related to infections. Studies have found that can have an impact on the brain through the microbiota-gut-brain axis, included depression, anxiety, dementia, and other diseases (Grochowska et al., 2018). Li et al. (2018) found that intestinal flora can affect SAE through the vagus nerve. The relationship between intestinal flora and the prognosis of SAE patients is still unclear.
Therefore, further studies for identifying the relationship between intestinal flora and the prognosis of SAE patients, and the predictors of the prognosis of SAE patients, especially accurate and measurable prediction models for hospital mortality, are pivotal for risk-optimized therapeutic strategies and to improve the prognosis of sepsis patients. This study aimed to investigate the predictors associated with hospital mortality in patients with SAE and establish a comprehensive visual predictive nomogram of hospital mortality, calculating a probabilistic estimate that could be of use to clinicians these patients.

Data Source
Data were obtained from the Medical Information Mart for Intensive Care (MIMIC-III, Version 1.4), which contains 46,520 patients admitted to the Beth Israel Deaconess Medical Center (Boston, MA, United States) from 2001 to 2012 (Johnson et al., 2016). The database documents included charted events such as demographics, vital signs, microbiology events, medication prescriptions, laboratory tests, etc. International Classification of Diseases, Ninth Revision (ICD-9) codes were also documented by hospital staff on patient discharge. The following CITI program course was completed: CITI 33690380. The raw data were extracted using a structure query language (SQL) using Navicat and further processed with R software.

Patient Population
Inclusion criteria were as follows: Patients with (1) sepsis 3.0. (2) age ≥ 18 years-old. (3) at least 24 h stay in the ICU. Sepsis was defined as an infected patient on discharge according to ICD-9 codes and microbial culture positive. According to the definition of sepsis 3.0, we included patients with SOFA score ≥ 2.

Sepsis-Associated Encephalopathy
Sepsis-associated encephalopathy was defined as (1) patients with GCS < 15. (2) The patient was diagnosed with delirium, cognitive impairment, altered mental status according to the ICD-9 code. (3) The patient was treated with haloperidol during hospitalization. (4) Exclude consciousness disorders caused by other reasons. Many studies use GCS score as an essential tool for evaluating SAE patients (Iwashyna et al., 2010).

Statistical Analysis
The Shapiro-Wilk test for the sample distribution was used. Continuous variables with normal distribution were expressed as the mean ± standard deviation (SD), and continuous nonnormal distributed variables were expressed as the median (interquartile range, IQR), categorical variables were expressed as frequency and percentage, as appropriate. A non-parametric test (Mann-Whitney U test or Kruskal-Wallis test) was applied for data with non-normal distribution or heterogeneity of variances. Pearson Chi-squared test was applied to categorize variables.
Patients were randomly assigned to either the training cohort (80%) or the validation cohort (20%). The selection of predictive features of the nomogram used the least absolute shrinkage and selection operator (Lasso) regression model (Sauerbrei et al., 2007;Sun et al., 2013;Wang et al., 2020). A multivariate COX regression analysis was performed on the selected variables, and a nomogram was constructed based on the results of the multivariate COX regression analysis (P < 0.05). We applied a bootstrapped resample with 1,000 iterations to verify the accuracy of the nomogram. The C-index was employed as an indicator to determine the discrimination ability of the nomogram through receiver operating characteristic (ROC) curve analysis and area under the curve (AUC). The calibration was performed by plotting the calibration curve to analyze the association between the observed incidence and the predicted probability. We evaluated the clinical usefulness and net benefit of the new predictive models by using decision curve analysis (DCA).
Statistical analysis was conducted with R software (version 3.4.3). Statistical significance was defined as p < 0.05.

Demographic Baseline Characteristics
1,055 patients with SAE were identified from the MIMIC database after applying the inclusion and exclusion criteria. We randomly assigned 80% and 20% of the patients to the training (n = 844) and validation (n = 211) cohorts. The recruitment process is illustrated in Figure 1. Table 1 shows the patient characteristics in the primary and validation cohorts. SAE patients who were older, had urinary tract infection or yeast infection were more likely to die. Circulatory failure was more common in non-survivors [Heartrate, Table 2 shows the outcomes for the survival group and nonsurvival group. Among non-survivors, there was a higher incidence of multiple organ failure including respiratory failure (63.6 vs. 32.9%), renal failure (69.4 vs. 57.3%), hepatic failure (10.5 vs. 3.3%), cardiovascular failure (58.1 vs. 8.5%), and hematological failure (26.7 vs. 21.2%). This led to a higher rate of mechanical ventilation (52.7 vs. 35.2%) and renal replacement therapy (12.0 vs. 2.0%) among non-survivors.

Feature Selection
Using the LASSO regression model, among the non-survivors of SAE, we identified 89 features which reduced to 13 potential predictors. They include SAPS II, renal replacement therapy, temperature, SpO 2 , albumin, INR, lactate, respiratory failure, urinary tract infection, anemia, systolic blood pressure (sysbp), partial thromboplastin time (Supplementary Material 13: Data supplement) (Figures 2A,B).

Multivariate Cox Regression
Furthermore, we performed a univariate, and multivariate cox regression analysis of these 13 potential predictors, sex, and admission type. According to our results, SAPS II, renal replacement therapy, temperature, SpO 2 , albumin, international normalized ratio (INR), lactate, and respiratory failure were independent prognostic factors for SAE patients (p < 0.01 or p < 0.05) ( Table 3).

Predictive Nomogram Development
A Lasso regression model and multivariate cox regression analysis identified SAPS II, renal replacement therapy, temperature, SpO 2 , albumin, INR, lactate, and respiratory failure as independent prognostic factors for SAE patients in the training cohort. These factors can be used to predict the hospital mortality of patients with SAE (Table 3), which was presented as the visualization nomogram (Figure 3). The hazard ratio values of these risk factors were established and scored for each level of prognostication. By adding up the scores associated with each variable to assess the hospital mortality of SAE patients.

Discrimination and Calibration
The AUC for the hospital mortality prediction nomogram was 0.812 (95% CI, 0.780-0.843) in the training cohort, which is greater than the SAPS II score of 0.745 (95% CI, 0.708-0.783) (Figure 4). The predictive accuracy of the nomogram was shown with a sensitivity of 0.601 and a specificity of 0.867. Our study employed the bootstrap resampling method for internal validation of the model. The calibration plot of hospital mortality of SAE patients revealed good agreement between the observed and predicted values ( Figure 5).

Clinical Utility
The DCA of the nomograms and SAPS II for the hospital mortality of patients with SAE are illustrated in Figure 6. The results showed that the nomogram provided a greater net benefit in predicting hospital mortality compared to that of SAPS II.

DISCUSSION
In our study, patients with SAE have a hospitalized mortality rate of 30.5%. Intestinal infections and microbial infections were not found to be related to the prognosis of SAE patients. We identified independent factors for the prognosis of SAE patients, which included SAPS II, renal replacement therapy, albumin, INR, lactate, temperature, SpO 2 , and respiratory failure. We further developed and validated a comprehensive visual nomogram to predict the prognosis of SAE patients. The nomogram showed a high degree of validity, discrimination, and clinical utility.
Microorganisms in the body are related to many diseases. Li-Hong Peng andLihong Peng et al. (2018, 2020a) established a model to predict the association of microorganisms with various diseases through microorganisms, and the model showed excellent performance. Probiotics can change the types of intestinal microflora and affect patients' mood and memory function (Bagga et al., 2018). Xia et al. (2018) found that probiotics can improve the cognitive function of Frontiers in Microbiology | www.frontiersin.org   patients with hepatic encephalopathy. Wei et al. (2020) found that Enterobacteriaceae can improve patients' mild cognitive impairment. Although the study of Li et al. (2018) has proved that the intestinal flora could affect SAE through the vagus nerve. Unfortunately, our study found that intestinal infections and microbes have nothing to do with the prognosis of SAE patients. It may require further experimental study in the future.
The SAPS II was developed from a European/North American study. Patients included in that study were from medical and surgical wards, as well as ICUs, in ten European and two North American countries. The authors showed that SAPS II demonstrated a high level of predictivity on the death of hospitalized patients (Le Gall et al., 1993). Although later studies have suggested better predictive tools than SAPS II (Norrie,  2015), our cohort study showed that the SAPS II score of nonsurviving patients was significantly higher than that of patients in the survival group, which further supports the accuracy of SAPS II as an independent predictive factor for hospital mortality in SAE patients. Our cohort study demonstrated that the incidence of renal replacement therapy in the non-survival group was significantly higher than that in the survival group. After LASSO and multivariate Cox regression analyses, it was found to be an independent risk factor for the death of SAE. However, the use of renal replacement therapy cannot be assumed to be an independent factor for death. Patients who were given renal replacement therapy were more likely to be severely ill with worse kidney function, more serious infection, and a higher incidence of multiple organ dysfunction, and internal environmental disorders (Palevsky, 2008;Bagshaw and Wald, 2018;Tandukar and Palevsky, 2018). This, in turn, leads to a higher mortality rate. Our cohort study also showed that SAE patients with respiratory failure, worse coagulation function, and lower albumin levels were more likely to die. The mechanism of multiple organ dysfunction in patients with SAE is consistent with sepsis patients, and it may be attributed to the immune response to sepsis (Nolt et al., 2018); circulatory abnormalities Finfer et al., 2013), organ ischemia; hypoxia endothelial permeability increases (Kopterides et al., 2011;Opal and van der Poll, 2015); cell death (Pinheiro da Silva and Nizet, 2009); and mitochondrial dysfunction (Yang et al., 2015;Sun et al., 2019). We should promptly correct the respiratory failure, give component blood transfusions, correct coagulation function, supplement albumin, and reduce the mortality of SAE patients.
Lactate is a vital laboratory indicator that affects the prognosis of patients with sepsis. It is widely known, the higher the lactate level, the worse the patient's prognosis (Suetrong and Walley, 2016;Liu et al., 2019). Serum lactate is also an independent risk factor for the prognosis of SAE patients in our cohort study. In patients with septic shock, fluid resuscitation guided by monitoring the serum lactate is still the most effective method for reducing the mortality of septic shock (Hernández et al., 2019). Serum lactate is used to evaluate disease severity, guide treatment plan, and predict patient prognosis (Suetrong and Walley, 2016). Lower serum lactate levels are associated with reduced patient mortality (Puskarich et al., 2013;Vincent et al., 2016). Therefore, serum lactate is an important indicator for evaluating the prognosis of patients with sepsis and SAE. The results of previous studies further support our conclusion. Patients with lactate acidosis and hyperlactic acidosis, we should timely rehydration and other treatments to reduce lactate levels and improve the survival rate of SAE patients.
There is currently a lack of effective tools for predicting hospital mortality in SAE patients. By exploring the clinical indicators for evaluating the prognosis of SAE patients,  through Lasso and Cox regression analysis, eight potential predictors, including SAPS II, renal replacement therapy, albumin, INR, lactate, body temperature, SpO 2 , respiratory FIGURE 4 | Discriminatory accuracy for predicting the incidence of SAE assessed by receiver operator characteristics (ROC) analysis calculating area under the curve (AUC). SAPS II, simplified acute physiology score.
failure were identified and used to establish a comprehensive visual nomogram for predicting hospital mortality of SAE patient. The nomogram demonstrated excellent discrimination (AUC, 0.812; 95%CI: 0.780-0.843) that was better than SAPS II (AUC, 0.745; 95%CI: 0.708-0.783) in the primary cohort. The validation cohort is used to verify the calibration function of the nomogram and has good consistency with the model (Figure 5).
In terms of clinical application, the net benefit of patients using nomogram is better than that of SAPS II (Figure 6), and the nomogram shows good performance in predicting hospital mortality of SAE patients. For the evaluation of nomograms, in addition to the above-mentioned AUC value and other methods, some new methods may be needed to evaluate in the future (Zhou et al., 2019;. Several limitations must be acknowledged. Firstly, our study is retrospective based on the MIMIC database, which has its inherent limitations. For instance, our study identified septic patients using the definition from the ICD-9 diagnostic code, which may be different from the Sepsis-3 definition. However, this small discrepancy does not deny the clinical application value of our study. Although our nomogram has excellent performance, our data is older and we need new data to verify in the future. Secondly, we included ICU patients for analysis, which enhanced the heterogeneity of the study population, and thus our results may not be suitable for patients outside the ICU. Third, there are a lot of more widely used methods in feature selection and classification than Lasso, such as elastic net, random forest, and deep neural network (Huang et al., 2017;He et al., 2020a,b;Liang et al., 2020;Liu C. et al., 2020;. Model development only uses the general FIGURE 5 | Calibration curves of a nomogram estimating the hospital mortality of SAE patients. The x-axis represents the predicted risk of hospital death in patients with SAE. The y-axis represents the actual risk of hospital death in patients with SAE. The dotted line represents the perfect prediction of the ideal model. The closer the red solid line is to the dotted line, the better the performance of this nomogram. linear regression method, fusing various biological information by multi-information fusion (Peng et al., 2017), bipartite local model (Peng et al., 2020b), and the KATZ method (Zhou et al., 2020) should be further studied in the future. We will apply these methods to further improve the performance of our model. Finally, the Model establishment was only verified internally, and further external verification is required in the future to illustrate its extrapolation.

CONCLUSION
A nomogram was established for predicting hospital mortality of SAE patients, which was accurate and clinically useful. The nomogram also performed better than the SAPS II with a higher net benefit.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by the establishment of the database was approved by the Massachusetts Institute of Technology (Cambridge, MA, United States) and the Institutional Review Boards of Beth Israel Deaconess Medical Center (Boston, MA, United States). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

AUTHOR CONTRIBUTIONS
LZ, YL, XS, and YL developed this manuscript central ideas. YW and ZG collected the data regarding the manuscript. LZ wrote the first draft of the manuscript. YL and XS revised the manuscript, worked on the English, and made the final version of the manuscript. All authors read and approved the final manuscript.