Antibiotic Discontinuation 24 h After Neonatal Late-Onset Sepsis Work-Up—A Validated Decision Tree Model

Objectives: Neonatal late-onset sepsis work-up is a frequent occurrence in every neonatal department. Blood cultures are the diagnostic gold standard, however, a negative culture prior to 48–72 h is often considered insufficient to exclude sepsis. We aimed to develop a decision tree which would enable exclusion of late-onset sepsis within 24 h using clinical and laboratory variables. Study Design: Infants evaluated for late-onset sepsis during the years 2016–2019, without major malformations, in a tertiary neonatal center were eligible for inclusion. Blood cultures and clinical and laboratory data were extracted at 0 and 24 h after sepsis work-up. Infants with bacteriologically confirmed late-onset sepsis were compared to matched control infants. Univariate logistic regression identified potential risk factors. A decision tree based on Chi-square automatic interaction detection methodology was developed and validated. Results: The study cohort was divided to a development cohort (105 patients) and a validation cohort (60 patients). At 24 h after initial evaluation, the best variables to identify sepsis were C-reactive protein > 0.75 mg/dl, neutrophil-to-lymphocyte ratio > 1.5 and sick-appearance at 24 h. Use of these 3 variables together with blood culture status at 24 h, enabled identification of all infants that eventually developed sepsis through the decision tree model. Our decision tree has an area under the receiver operating characteristic curve of 0.94 (95% CI: 0.90–0.98). Conclusions: In non-sick appearing infants with a negative blood culture at 24 h and normal laboratory values, sepsis is highly unlikely and discontinuing antibiotics after 24 h is a viable option.


BACKGROUND
Neonatal late-onset sepsis (LOS) is an important worldwide health care issue commonly occurring in Neonatal Intensive Care Units (NICU's) (1)(2)(3). Exclusion of neonatal sepsis is challenging as symptoms are often non-specific, diagnostic tests lack sufficient sensitivity and specificity and blood cultures, which are the diagnostic gold standard, often require 48-72 h until becoming positive in septic infants. As a result, infants suspected of having sepsis often receive antibiotics for 48-72 h until sepsis is excluded.
Many infants at risk of LOS are treated with antibiotics as the consequences of untreated sepsis are severe compared to the side effects of short-term antibiotic therapy (2). Thus, physicians often prefer to err on the side of caution and to continue antibiotics until there is virtually no risk of sepsis. The direct result is increased hospital length of stay, increased health care costs, increased antibiotic use and increased development of resistance to antibiotics. Short-term effects have been shown in preterm infants and result in an increase in adverse outcomes such as necrotizing enterocolitis and sepsis (4). However, longterm effects may be more important and should be considered such as the effect of antibiotics on infants' gut microbiota (5) and subsequent altered risk for obesity, asthma and allergies (6,7).
Markers of neonatal LOS such as clinical signs and assessments, C-reactive protein (CRP), procalcitonin or complete blood count (CBC) are used for diagnosis of LOS, however none have shown sufficient diagnostic accuracy to be used as a single marker for early and accurate diagnosis of LOS (8-10), or even more so for exclusion of LOS. Our aim was to create a decision tree using clinical findings and laboratory tests that can be used to safely exclude sepsis in low-risk infants within 24 h after initial evaluation.

Study Population and Data Collection
Our retrospective case-control study was conducted in the NICU of Schneider Children's Medical Center of Israel. The NICU is a university-affiliated tertiary care pediatric medical center, which admits 900 newborns annually and serves about 9000 deliveries per year. The study was approved by the local Research Ethics Board (Study number RMC-18-142).
The NICU database routinely collects data on all admitted infants. A retrospective search of the database was performed for all infants diagnosed with primary culture-proven LOS between January 2016 and December 2019. We validated the diagnosis of LOS by chart review and by comparison with all infants identified by the microbiology laboratory as having positive blood culture results meeting the definition of LOS. During the study period, repeated blood cultures were taken when clinical deterioration occurred or to ensure bacterial eradication when blood cultures Abbreviations: NICU, Neonatal Intensive Care Unit; LOS, Late-onset sepsis; CBC, complete blood count; CRP, C-reactive protein; NLR, Neutrophil-to-Lymphocyte ratio; CI, Confidence interval; OR, Odds ratio; SD, standard deviation; PPV, Positive predictive value; NPV, negative predictive value. were positive. If infants had multiple episodes of LOS, only the first episode was included. Only infants hospitalized in the NICU from birth were included. There was no corrected gestational age cutoff limit. Excluded were infants with any major congenital malformation or any episode of early-onset sepsis. A control group of infants evaluated tor sepsis with negative blood cultures was chosen matching for prematurity and corrected gestational week in a 1:2 ratio. Infants were categorized as either preterm (below 34 weeks gestation) or near-term to full-term (equal or above 34 weeks gestation).
Data were extracted from the medical files as follows: demographic data (gestational age, birth weight, and gender), hospitalization characteristics at sepsis work-up (presence of central line, parenteral nutrition, day of life at time of sepsis work-up, antibiotic treatment, diagnoses of sepsis or necrotizing enterocolitis). Initial and repeated clinical and laboratory data measurements were collected during the time period starting 6 h prior to sepsis work-up and until 24 h after work-up and included clinical data (temperature, blood pressure, apneic or bradycardic episodes, ventilation requirement) laboratory data [automated CBC, neutrophil to lymphocyte ratio (NLR), CRP, blood glucose] and microbiological data (blood culture result, time to positivity of blood culture). Immature to mature neutrophil ratio is not routinely used in our institution and was not assessed.
For every patient, a health evaluation score was assigned using a scoring sheet by two senior neonatologists, who blindly evaluated the health record at time of sepsis workup and 24 h after work-up. Disagreement between evaluating neonatologists was resolved by discussion or consulting a third neonatologist until consensus was achieved. The heath evaluation score (Appendix 1) was graded into three groups as follows: healthy, equivocal, or sick appearing (11) and is similar to the classification of clinical signs used routinely to evaluate earlyonset sepsis (12,13).

Definitions
The study definitions have been previously described in detail (11). LOS was defined by a single positive blood culture by a causative pathogen 72 h or later after birth, with the exception of sepsis caused by coagulase negative staphylococci, which required two positive blood cultures. Each isolate was identified using the VITEK 2 system (bioMérieux, SA, France) and/or MALDI Biotyper System (Bruker Daltonics, Bremen, Germany), in accordance with the manufacturers' instructions for bacterial identification. Temperature abnormality was defined as below 36 • C or above 37.5 • C. Tachycardia was defined as a heart rate above 186 (bpm) (14). Hyperglycemia was defined as a glucose value above 108 mg/dl (14). CBC abnormalities (15)(16)(17)(18) were defined as follows: leukocyte count below 5,000/mm 3 or above 20,000/mm 3 , neutrophil count below 1,800/mm 3 or above 5,400/mm 3 . Lymphopenia (18) was defined as lymphocyte count below 2,000/mm 3 . Thrombocytopenia was defined as a thrombocyte count below 150,000/mm 3 . Abnormal CRP (>0.75 mg/dl) and abnormal NLR (>1.5) were identified and calculated using receiver operating characteristic (ROC) curve, as described previously in a near identical cohort (11).

Statistical Analysis
The entire study cohort was divided into a development cohort (about two-thirds of the entire cohort) and a validation cohort. The development cohort was used to build our model and decision tree and based on data from January 2016 through July 2017. The temporal validation cohort was used to evaluate our findings and based on data from August 2017 through December 2019. All data are expressed as mean and standard deviation (SD), median and interquartile range, or frequency and percentage. Unless otherwise indicated SD is presented in parenthesis. Categorical variables were compared using chisquare test or Fisher's exact test, as appropriate. Ordinal variables were compared using the Mann-Whitney test, and continuous variables were analyzed by the Student's t-test. Univariate logistic regression using variables available within 24 h after initial work-up was performed to identify factors associated with risk of LOS. Odds ratio (OR) and 95% confidence intervals (CI) were computed. A two-tailed p-value < 0.05 was accepted as statistically significant. A decision tree based on Chi-square automatic interaction detection methodology (19) was created using variables identified in the univariate analysis with a pvalue < 0.2. Our decision tree performance was assessed by comparing the area under the curve (AUC) of the ROC curve of the development and temporal validation cohorts. Exclusion of LOS based on the decision tree was compared to previous predictors using AUC's, sensitivity, specificity, PPV and NPV, similar to previously described methodology (20).
Statistical analyses conformed to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) criteria (21). All statistical analyses were performed using the IBM SPSS statistical package (SPSS 24, Chicago, IL, USA).

RESULTS
During the study period, culture positive LOS was identified in 140 of 3263 infants (4.3%) that were admitted to our NICU, representing 145 episodes. Multiple episodes of sepsis occurred in 3 infants. Of the 140 infants with culture positive sepsis, 50 were excluded because of major malformations, 36 due to a prior episode of early-onset sepsis and 1 patient due to missing data. The remaining 53 infants with LOS were matched to 112 control infants.
The entire study cohort was divided in a 2:1 ratio, to a development cohort (33 infants with LOS, 72 control infants) and a temporal validation cohort (20 infants with LOS, 40 control infants).
Gram-negative bacteria were the primary pathogens identified in 25 (47.2%) infants, followed by coagulase negative staphylococci found in 18 Table 1. There were no significant differences in the corrected mean gestation age and birth weight of the LOS vs. control groups, however, central line use was significantly higher in the LOS group. Comparison of clinical and laboratory predictors of infants with vs. without LOS within 24 h of sepsis work-up is presented in Table 2.
Use of a decision tree enabled identification of all infants with LOS at the 24-h mark. The combination of blood culture, clinical assessment (sick appearance) with CRP > 0.75 mg/dl or NLR > 1.5 identified 33 of 33 infants with sepsis (100%) with only 18 of 72 (25.0%) control infants considered as possible sepsis beyond 24 h. Figure 1 presents a decision tree summarizing  the approach to antibiotic discontinuation 24 h after neonatal LOS work-up. Positive blood culture or sick appearance were selected as variables for the first node. The blood culture and sick appearance identified 64% of infants with LOS, with the remaining 36% identified by CRP and NLR. The AUC for our decision tree was 0.94 (95% CI, 0.90-0.98). We assessed our decision tree performance by use of a temporal validation cohort with an AUC of 0.92 (95% CI, 0.84-0.99). Comparisons between our decision tree to previously identified sepsis markers in both the development and temporal validation cohorts are presented in Table 3. The AUC using our decision tree was higher than previously reported sepsis markers.

DISCUSSION
LOS is a common issue in any NICU, however despite implementation of antibiotic stewardship there is limited data on early antibiotic discontinuation in infants suspected of LOS and even less data when the time-frame for antibiotic discontinuation is 24 h after initiation. Our aim was to create a method that would safely exclude LOS within 24 h after initial evaluation. Use of a decision tree model, which included blood culture status, clinical sick appearance, CRP and NLR, to stop antibiotics at 24 h during the neonatal period, helped identify all culture-positive septic newborns while enabling antibiotic discontinuation in about two-thirds of non-septic infants. Our research presents a novel approach using a decision tree to minimize antibiotic duration while maintaining a very low risk of inappropriately discontinuing antibiotics in infants with sepsis.
In 2002, Kaiser et al. (22) posed the question of whether antibiotics should be discontinued 48 h after LOS work-up. Their conclusion was that this should be the standard of care. Since their publication various attempts have been made to further decrease the duration of antibiotic therapy. The motivation for decreased antibiotic duration is obvious. Increased antibiotic use not only directly impacts the health care system by increasing NICU length of stay, health care costs and resistance to antibiotics, but also has direct long-term adverse effects on the infant such as altered gut microbiota (5,23) and obesity (6). Early cessation of antibiotic therapy enables recovery of the normal gut microbiota (23). However, discontinuing antibiotics after 24 h entails the risk of not continuing appropriate therapy in an infant with sepsis.
Use of various sepsis markers for LOS evaluation is ubiquitous. CBC for sepsis evaluation commonly includes a leukocyte count and the immature to mature neutrophil ratio. These markers lack the adequate sensitivity needed to rule-out sepsis as single markers (15,24,25). A less known, but readily available marker is the NLR (16). During sepsis, neutrophil counts increase due to increased release of immature neutrophils and delayed apoptosis, whereas the lymphocyte count decreases (26). The combined effect of increased neutrophils and decreased lymphocytes is an increased NLR. Because NLR takes into account both lymphocyte and neutrophil changes, it is potentially a more accurate marker for sepsis than using only lymphocyte or neutrophil counts. To date no single marker has been found to be accurate enough for rapid identification or exclusion of sepsis within 24 h. For this reason, we used a combination of sepsis markers that are readily available in the NICU. We sought to include a simple FIGURE 1 | Decision tree for antibiotic discontinuation 24 h after neonatal late-onset sepsis work-up. All infants with positive blood culture or sick appearance will continue antibiotics beyond 24 h (node 2). Infants with elevated CRP (c-reactive protein) or elevated NLR (neutrophil to lymphocyte ratio) will also continue antibiotics beyond 24 h (node 3). Antibiotic therapy may be discontinued in all remaining infants (node 4).
clinical evaluation as we deemed it unlikely that any treating physician would rapidly discontinue antibiotics in any sickappearing infant.
Use of blood cultures is the gold standard for diagnosis of sepsis. In previous studies the time to positivity of cultures varied. Two studies (27,28) including neonatal blood cultures from early and late onset sepsis showed 24-h positivity rates of 56.1 and 77%. For LOS a 24-h positivity rate of 51.7% was reported by Abdelhamid (29). Recently, Ur Rehman Durrani et al. (30) reported a 24-h positivity rate of 71.9%. The authors suggested discontinuing antibiotic therapy for Gram negative bacteria at 24 h, however recommended continuing therapy for Gram positive bacteria for which time to positivity was often longer than 24 h. In our study the 24-h positivity rate was only about 40%. In contrast to previous studies (27)(28)(29), we calculated the real time to positivity based on the time the blood culture was obtained to the time the result was communicated to the NICU and not on the minimum theoretical time indicated by the computerized blood culture system. While a positive blood culture mandates antibiotic therapy, a negative culture at 24 h without additional support is not sufficiently conclusive for antibiotic discontinuation.
Use of the combination of CRP and CBC for discontinuation of antibiotics at 36 h after LOS work-up has previously been suggested by Beltempo et al. (31). CBC abnormalities in this study were abnormal white blood cell count, thrombocytopenia or a high immature neutrophil count. However, considering the price of inappropriate antibiotic discontinuation in a septic infant, exclusive use of laboratory markers, when below cut-off values, was not accurate enough for a categorical recommendation to discontinue antibiotics at 24 h. Thus, addition clinical markers need to be used for decision making at 24 h. Our decision tree model used a similar approach but incorporated clinical evaluation, blood culture status as well as maximal CRP and NLR values at 24 h after workup. Because both sick appearance and positive cultures clinical circumstances in which early antibiotic discontinuation would not be considered by most if not all treating physicians, we chose to group them together at the beginning of our decision tree.
The study cohort included a relatively large cohort of infants, obtained from a single large NICU. The study findings were validated using a temporal validation cohort. Splitting the cohort enables a more accurate evaluation of the suggested model, assists in avoiding overfitting of the model and is the preferred statistical approach (21). During the study period no changes were made in antibiotic policy. Infants with congenital anomalies or infants with prior early-onset sepsis that were possibly at increased risk for sepsis were excluded. The study was performed in a tertiary referral center accounting for the relatively high number of major malformations in our cohort. The study used a strict definition of sepsis based only on blood cultures. Sepsis markers readily available in any NICU were used. Despite the study strengths, some limitations should be acknowledged. The study's retrospective design has known limitations. Clinical evaluation is subjective, may be abnormal due to disease states other than LOS and was not always assigned at time of LOS. The limitations of clinical evaluation were mitigated by use of a standardized scoring sheet and by its blind performance by 2 neonatologists. Although the study was of sufficient statistical power to identify multiple risk factors for LOS, a larger multicenter cohort would strengthen the generalizability and validity of the study results, as would inclusion of more full-term infants.
In conclusion, the study presents a simple approach, feasible in any NICU, enabling identification of infants at low-risk of LOS, only 24 h after initial evaluation. In nonsick appearing infants with a negative blood culture at 24 h and normal laboratory values, sepsis is highly unlikely and antibiotics may be discontinued. The suggested approach can reduce the duration of antibiotic therapy in low-risk infants without inappropriately discontinuing antibiotics in septic infants. Additional research is needed to confirm our study results.

DATA AVAILABILITY STATEMENT
The datasets presented in this article are not readily available because research is still being conducted, however, data will be made available for collaboration purposes. Requests to access the datasets should be directed to Gil Klinger, gilkl@tauex.tau.ac.il.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Rabin Medical Center Research Ethics Board, Rabin Medical Center, Petah Tikva, Israel. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

AUTHOR'S NOTE
The study was presented in part in the annual Pediatric Academic Society 2019 meeting.

AUTHOR CONTRIBUTIONS
OG and GK contributed to leading role in all study aspects including: conception and design of study, data acquisition, analysis and interpretation, writing of manuscript, and final approval of the submitted manuscript. NA and HB-Z contributed to study conception and design, data acquisition, critical revision of manuscript, and approved the submitted manuscript. NS contributed to data acquisition and critical revision of manuscript and approved the submitted manuscript. RB contributed to study conception and design, data acquisition and interpretation, and critical revision of manuscript and approved the submitted manuscript. GC contributed to study conception and design, data analysis and interpretation, critical revision of manuscript, and approved the submitted manuscript. OS contributed to study conception and design, data interpretation, critical revision of manuscript, and approved the submitted manuscript. All authors agree to be accountable for all aspects of the work.