Validation of the Computerized Pediatric Triage Tool, pediaTRI, in the Pediatric Emergency Department of Lenval Children's Hospital in Nice: A Cross-Sectional Observational Study

Introduction

The number of visits to emergency departments (ED) has been rising steadily for both adult and pediatric patients over the past decades (1) resulting in an increase in waiting and care times. Each ED manages a wide variety of pathologies ranging from a simple general consultation to a life-threatening emergency. However, overcrowding in the ED as well as difficulties in monitoring patients waiting for clinical examination, can endanger patient safety (2). Patients require prioritization and triaging as soon as they reach the ED and cannot be seen purely in the order of arrival. An ideal triage system should be able to identify those who require immediate care (high-level emergency) from those who can wait or those who will not require emergency care (intermediate- to low-level emergency) (3). This triage is mostly carried out by a nurse at the triage zone (4) who must quickly identify high-emergency patients requiring immediate care and organize their care pathway. The triage nurse uses a decision support tool known as a triage tool.

There are many triage tools around the world, all of which have five levels of triage ranging from a non-urgent consultation (Level 5) to a life-threatening emergency (Level 1) (5–9). However, these tools are not suitable in all EDs because they rely on the diagnostic skills of a “clinical” nurse, a qualification that does not exist in all countries, including France. In addition, these tools were initially designed for adult patients, and then adapted to the pediatric patient, while a child cannot be considered as “a miniature adult.”

Regardless of the triage algorithm, the identification of life-threatening emergencies is a common and implicit denominator. In a pediatric ED (PED) setting, a high-level emergency corresponds to a child presenting an immediate life-threatening risk that could lead to cardio-respiratory arrest or a related emergency, and thus requires rapid intervention. These patients, for whom a Level 1 or 2 is usually assigned by commonly used pediatric triage tools (10–13), can also be screened using warning scores (14–16) that are predictive of clinical deterioration within 24 hours after visiting the PED. Among them, the Pediatric Early Warning System (PEWS) system, created in 2001, is considered to be efficient, easy to use, and reliable (17–21).

In France, there is no gold standard in pediatric triage and each hospital uses their own “home-made” triage system. Portas et al. (22) aimed to validate their pediatric triage tool correlating treatment times with severity and the hospitalization prediction rate. However, this was a 3-level scale that excluded newborns and trauma complaints, and the validity study was based on the correlation of resource utilization rather than comparing the triage level against a reference standard.

In 2000, the PED of the University Hospital of Nice (France) created a 5-level pediatric triage tool – the pediaTRI – based on clinical items of inspection, interview, and analysis of vital signs (Appendix 1). The items were initially selected by a panel of local experts based on the most life-threatening conditions (i.e., purpura fulminans, meningitis, severe pediatric trauma) and the most frequent diseases (i.e., bronchiolitis, gastroenteritis, traumatic brain injury). In 2011, the tool was integrated into the pediatric triage module of the Terminal Urgences® (TU®) (23), a software developed by the Regional Emergency Observatory of the South East of France (ORU Provence Alpes Côte d'Azur, France) under the supervision of the Regional Health Agency (ARS). However, this computerized version of pediaTRI has never been evaluated.

Our objective was to validate pediaTRI integrated into the TU® software against a reference standard, the PEWS system, in screening high-level emergencies.

Methodology

Design

We conducted a cross-sectional observational study in the PED of the Lenval Children's Hospital in Nice, a tertiary level, University Teaching Children's Hospital. This PED is the only one in the southeast of France (1.140.000 inhabitants aged 0 to 19 in 2017) and is the 4th biggest PED in France with an average number of 60.000 visits per year (before the Covid 19 Pandemic) of patients from zero to 17 years of age. The hospital is a reference center for specific diseases including cystic fibrosis, neuromuscular diseases, sudden infant death syndrome, and psycho-trauma, and provides a complete medico-surgical and technical platform delivering 24/7 care for any emergency (medical, surgical, psychiatric) with two operating rooms managed by pediatric anesthesiologists and surgeons. It has 180 beds including 10 belonging to the Intensive Care Unit (ICU) and 60 for outpatient care. The patient recruitment pool covers a radius of 90 miles from Nice. Although our PED is referenced as one of the two Level 1 pediatric trauma centers in the area, potentially life- threatening conditions remain rare and represent about 1% of the visits per year.

Participants

All patients under the age of 18 who visited the PED of Lenval Children's Hospital between January 1, 2016 and December 31, 2017 (open 24/7) were included. Patients who were called back, patients treated in prehospital care, and patients who left without an assigned triage level were not included. Patients visiting the PED for psychological or psychiatric conditions, patients who left without being seen by a physician, whose recorded vital signs were considered as uninterpretable (not reliable or error measures), or for whom there was no information about outcome, were excluded. The study was approved by the French Data Protection Authority (CNIL) (registration number: 2157640v0) in accordance with the laws which govern “non-interventional clinical research” in France (namely articles L.1121-1 and R.1121-2 of the public health code). Patient data were anonymized using a specific patient numbering procedure for the study. Informed written consent or ethics committee authorization was not necessary for this no-effect observational study collecting anonymized data on patient management.

The Pediatric Triage Tool Integrated Into the Terminal Urgences® Software

The Terminal Urgences® software was created in 2002 under the supervision of the Regional Health Agency, and is now used in more than 65 EDs (for both adult and pediatric patients) in the southeast of France (24). This software contains many modules – a triage module, medical file module (medical and paramedical), nursing care prescription module (procedures and care), and a therapeutic prescription module – which are accessed and completed in real time by various health professionals.

In pediatric triage, the nurse follows a pre-established evaluation algorithm for triaging patients. Firstly, the nurse must identify whether the patient is in a critical condition (e.g., in respiratory or cardiac failure) or judged to be at high risk (e.g., newborns, immunosuppressed). In those cases, the triage process is stopped at this point so that lifesaving care can be initiated. For the other children, the nurse collects details about the chief complaint covering all the triage items, and selects all the complaints that the child presents or that are reported by the parent(s). Each item is assigned to a triage level, although the triage can be moderated by risky situations/conditions and out-of-range values for vital signs. If several items are selected, the one with the highest level of severity determines the final triage level. If the nurse disagrees with the level assigned by the tool, they can change the triage level after consulting with the attending PED physician. The triage module is a user-friendly interface allowing the nurse to perform a quick triage within an optimal duration of 5 min.

The pediatric triage tool, pediaTRI, is structured into five levels ranging from the most severe (Level 1) to the least severe (Level 5). Levels 1 (life-threatening emergency) and 2 (very urgent) are mainly assigned to patients for whom the vital and / or functional prognosis has already been engaged or will probably be engaged very quickly if no care is administered. These patients are defined as high-level emergencies. Low-level emergencies correspond to Levels 3 (urgent), 4 (consultation), and 5 (non-urgent) for which a vital or functional prognosis may not be engaged either in the short or medium term. However, according to resource utilization and to the architecture of the PED at Lenval Children's Hospital, some risky situations (i.e., newborns) or risky profiles (i.e., immunosuppressed patients) may be systematically attributed Level 2. Each triage level determines the optimal time (in min) for the patient to see a physician (fractile response) (25) according to the organization of the PED: 0, 15, 60, 120, or 240 min, respectively, for triage levels 1 to 5.

All the nurses must have worked in the PED for at least 6 months and then undergone a 3-month supervision period by an experienced nurse before performing a triage alone.

Justification of the Reference Standard

We could not validate pediaTRI against other triage tools developed in this setting as the quality of evidence (robust validation) to support their use is poor.

Among the existing scores for predicting mortality and morbidity in pediatrics, we opted for the PEWS system described by Akre et al. (26) as our reference standard due to its simplicity of use based on clinical signs for triage. PEWS was created in the UK in 2001 specifically for children (rather than an adult tool adapted to pediatrics) by a panel of experts to rapidly detect clinical deterioration including cardiopulmonary arrest (CPA) by the use of a simple and objective clinical tool. Studies have shown that PEWS identifies pediatric patients who require intensive care (16, 27). The PEWS system is based on three main components each given a 3-point rating as follows: (a) behavior and early signs of shock, recognizable and assessable by the parents; (b) skin tone and capillary refill time to assess the cardiovascular system; (c) and respiratory rate and oxygen dependence to assess the respiratory system. According to the literature (26–28), the optimal cutoff level to calculate the sensitivity and specificity for admission to an ICU, defined as a high-level emergency, is ≥ 4/9 with a sensitivity ranging from 61.3 to 94.4% and specificity from 25.2 to 86.7% (16).

Sample Size

Based on data from previous years, approximately 200 patients classified as Level 1 (life-threatening emergency) visit our PED each year. Thus, we conducted a large cross-sectional study to allow for sufficient statistical power and detailed evaluation of specific categories of patients by analyzing patients visiting our PED over 2 successive years (2016–17).

Endpoints

The primary endpoint was the evaluation of the performance of pediaTRI in classifying patients as a high-level emergency (pediaTRI Levels 1 or 2) or low-level emergency (Levels 3, 4, and 5). In the same way, a PEWS score ≥ 4/9 defined a high-level emergency vs. a PEWS score ≤ 3/9 as a low-level emergency. Postoperative vomiting and aerosol administration were not assessed. The secondary endpoint was the performance of pediaTRI according to gender, age (divided into subgroups), type of complaint (medical or surgical), and the main diagnosis. The final endpoint was the evaluation of the markers of severity such as length of stay (LOS) and hospitalization rates.

Data

Data from 2016–17 constituted the main dataset, and data from 2018–19 was used as an independent validation cohort. Patient data were collected prospectively in the TU® software: gender (male/female), age (recorded as age groups: 0 – 27 days, 28 days – 3 months, 3 months – 1 year, 1 – 3 years, 3 – 7 years, 7 – 12 years, and 12 – 18 years), data from pediaTRI with the level of triage (from 1 to 5), the PEWS score calculation, medical complaints (categorized as: ENT, pulmonary, cardiovascular, neurology, digestive, urology-nephrology, gynecology, dermatology, endocrinology-metabolism, infectious diseases, rheumatology and pain, hematology, poisoning, and others), surgical complaints (categorized as head and neck trauma, upper and lower limb trauma, trauma of the trunk-pelvis-urogenital apparatus, burns, and others), final diagnosis and LOS in mins.

The vital signs such as the pediatric Glasgow score (/15), heart rate (in number of beats per minute), respiratory rate (in number of breaths per minute), pulsed oxygen saturation (in %) with details of oxygen input or not (in ambient air or in L/min depending on the case) were collected into the triage module. The patient's outcome (hospitalization in ICU or another ward, transfer to another establishment, discharged home) was collected in the medical file module. The diagnoses were entered in the diagnosis and procedures rating module. We considered the vital signs to be normal if they were not entered by the triage nurse.

Statistical Analysis

The data, compiled exclusively as categorical comparisons, were evaluated using the chi-squared test. Over-triage was defined as a higher pediaTRI triage level compared with PEWS. Conversely, under-triage corresponded to a lower triage level.

The validity of pediaTRI was evaluated vs. PEWS using sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios. The results are expressed in numbers and percentages with 95% confidence intervals. Taking into account our sample size, the significance level “p” was set at 0.001 to avoid random statistical significance (29) with readjustment during multiple comparisons according to the Bonferroni method. One-way analysis of variance (ANOVA) was used to determine mean differences among patients for LOS in the PED, and post hoc comparisons were conducted using Scheffe's post hoc tests. Missing data were not analyzed as they represented 6% of the study sample only with negligible impact on our final outcomes. Statistical analyses were performed using R Studio version 4.0.2 for Macintosh® software from the Lenval Children's Hospital.

Validation Cohort

We decided to conduct a single-center study due to a lack of training in pediatric triage and the use of pediaTRI in other centers. In 2015, the steering committee of the ORUPACA recommended evaluating the performance of pediaTRI at the Lenval Children's Hospital before introducing it in other PEDs of the region subject to prior training in pediatric triage. As external validation was not possible, the 2018–2019 dataset was used as an independent validation cohort (the statistical analyses for the two periods were identical and are provided in Appendix 2).

Results

Among the 121,358 patients who visited the PED -between January 1, 2016 and December 31, 2017, 100,506 children were included in the analyses (82.8%), 70,330 of whom [70.0% (95% CI 69.7–70.3)] presented medical complaints (p < 0.001) (Figure 1).

FIGURE 1

Figure 1. STARD flow diagram presenting patients triaged by both pediaTRI (index test) and PEWS (reference standard). TP, true positive; TN, true negative; FP, false positive; FN, false negative.

Patients triaged as Level 4 (simple consultation) and Level 3 (urgent) were the most prevalent (Table 1): 41.1% (40.8–41.4) and 34.7% (34.4–35.0), respectively (p < 0.001) (Table 1). Boys were rated significantly more severe than girls: 59.9% (53.1–66.5), 59.4% (58.6–60.1) and 55.1 (54.6–55.7), respectively, for Levels 1, 2, and 3 (p < 0.001). Patients evaluated at Level 1 were more likely to be from 1 to 3 years old (29.0% (23.1–35.6) while those evaluated at Levels 2, 3, 4, or 5 were more likely to be 3 to 7 years old [22.2% (21.5–22.8), 29.7% (29.2–30.2), 27.2% (26.8–27.7) and 26.9% (25.9–27.9), respectively] (p < 0.001).

TABLE 1

Table 1. Characteristics of the sample by level of triage.

The most common presentations for a medical complaint (Table 2) were:

Level 1: Pulmonary [42.9% (36.2–49.7)] and neurological [18.0% (13.1–23.7)] conditions

Level 2: Pulmonary [31.0 (30.3–31.7)] and digestive [21.2% (20.6–21.8)] conditions

Level 3: Digestive (23.8% (23.3–24.2) and pulmonary conditions [18.1% (17.7–18.5)], and infectious diseases [15.4 (15.0–15.8)]

Levels 4 and 5: Dermatological [12.3% (11.9–12.6) and 20.6% (19.7–21.6)], ENT conditions [14.8% (14.4–15.1) and 14.6% (13.8–15.4)], and infectious diseases [12.2% (11.9–12.6)] and 15.3% (14.5–16.1)] for Levels 4 and 5, respectively (p < 0.001).

TABLE 2

Table 2. Characteristics of complaints by level of triage.

Finally, 60.9% (60.6–61.2) of the patients presented with at least one of the 11 most frequently encountered diagnoses in our study (Table 1). The most prevalent medical diagnoses were ENT infections and acute gastroenteritis with 15.9% (15.7–16.1) and 9.0% (8.8–9.1), respectively, while patients with upper and lower limb trauma were the most prevalent surgical cases with an overall prevalence of 18.4% (18.1–18.6).

Severity Outcome

Overall, 94.0% of patients were discharged. The hospitalization rate significantly decreased with triage level, from 53.5% in Level 1 to 0.6% in Level 5 (p < 0.001). The ICU admission rates among patients defined as a “high-level emergency” (pediaTRI Levels 1 and 2) were significantly higher than those defined as “low-level.” PED LOS was also strongly associated with triage level: LOS decreased from 176 min in Level 1 to 96.6 min in Level 5 (p < 0.001). No deaths were reported in the PED during the study period.

Validity

The high- and low-level emergencies assigned by pediaTRI were consistent with those defined by the PEWS score with an agreement of 84.5% (84.2–84.7) (Table 3). pediaTRI over-triaged 15,082 patients [15.0% (14.8–15.2), and under-triaged 528 (0.5% (0.5–0.6)].

TABLE 3

Table 3. Agreement in triage between pediaTRI vs. PEWS in screening “high-level emergency” and “low-level emergency.”

pediaTRI had a sensitivity of 76.4 (74.6–78.2) and a specificity of 84.7 (84.4–84.9) for identifying high-level emergencies (Table 4). The likelihood ratios were 5.0 (4.8–5.1) for high-level emergencies and 0.3 (0.3–0.3) for low-level emergencies. However, pediaTRI was more sensitive for infants under 3 months – 100.0% (66.4–100.0) for newborns (under 28 days of age) and 100.0% (75.3–100.0) for infants from 28 days to 3 months – but less specific at 14.8% (12.8–16.9) and 46.8% (44.8–48.8), respectively, for the same age groups. The negative predictive value was high in the overall sample (99.4) and their values remained high whatever the subgroup (from 90.5 to 100.0). The positive likelihood ratio values were mostly higher in the overall sample and in the subgroups than the negative likelihood ratios (0–32.9 vs. 0–1.1, respectively).

TABLE 4

Table 4. Sensitivity, specificity, predictive values and likelihood ratios of pediaTRI vs. PEWS (Continued on next page).

The performance values differed according to the category of chief complaint. Specificity values for medical complaints, were moderate for pulmonary, cardiovascular system, neurology, digestive, urology-nephrology, and rheumatology conditions, and low for endocrinology-metabolism disorders, hematology, and poisoning. Specificity values for surgical complaints were mainly high contrasting with a lower sensitivity except for head and neck trauma and burns. The most frequent medical reason assessed by PEWS as a high-level emergency was pulmonary related (17.2%; 16.5–17.9), followed by cardiovascular disease (3.3%; 2.5–4.3), endocrinology-metabolism disorders (2.1%; 0.7–4.8), and finally neurology (2.0%; 1.5–2.6). The corresponding assessments by pediaTRI were 45.2% (44.3–46.1), 32.2% (29.8–34.5), 69.5% (63.2–75.2) and 36.1% (34.2–38.0), respectively. High-level emergency cases for surgical reasons ranged from 0.0% (0.0–0.1) to 1.0% (0.4–2.2) for PEWS and from 2.0% (1.8–2.4) to 38.6% (34.6–42.6) for pediaTRI.

For the main medical diagnoses, pediaTRI had higher specificity (73.3–97.0) than sensitivity (41.8–56.7) for ENT infectious diseases, acute gastroenteritis, Flu' and unknown fever. Sensitivity was higher than specificity for asthma and bronchiolitis (89.4 and 84.9 vs. 56.6 and 35.9, respectively). For surgical diagnoses, as few patients were classified by PEWS as high-level emergency, values of specificity varied from 59.4 to 97.0 while those of sensitivity were not systematically interpretable.

Validation Cohort

The patient characteristics per triage level were similar between the periods 2016–17 and 2018–19. Although LOS and the level of emergency followed the same downward trends, we reported significantly longer PED LOS during 2018–19 with 205 min, 146 min, 134.6 min, 118.6 min, and 107.2 min respectively from Level 1 to 5. Prevalence of high- and low-level emergencies between pediaTRI and PEWS between the two periods were also similar with an agreement of 84.9% (84.7–85.1), an over-triage of 14,522 patients [14.5% (14.3–14.8)], and an under-triage 528 patients (0.6% (0.6–0.7). We reported similar values of performance of pediaTRI between the two periods except for lower sensitivities for surgical reasons (46.2 vs.78.6%) and for digestive diseases (60.7 vs. 82.6) during the 2018–19 period.

Discussion

Principal Findings and Interpretation

The pediaTRI triage tool has an overall moderate validity compared to the PEWS system with a sensitivity of 76.4% and specificity of 84.7%. The agreement with PEWS was 84.5%, with over-triage in 15.0% of the patients and under-triage in 0.5% (mostly in one category).

However, the performance of pediaTRI varies according to the patient profile. In children under 3 months, sensitivity was high (100% for those under 28 days and 100% for 1 to 3 months) contrasting with a poor specificity (respectively, 14.8 and 46.8%) due to potential “risky” situations in this young population. The sensitivity for children with a medical history or existing chronic disease (for example, a cardiovascular, neurological, endocrinological, or hematological condition) was much higher than the specificity. For these particular cases, over-triage is necessary whatever the chief complaint so that these children do not wait too long before being seen by a physician as their clinical condition could deteriorate rapidly.

The performance of pediaTRI also varies according to the subcategory of the chief complaint. Each subcategory contains many items, some of which are prevalent – such as fever (n = 11,827), and vomiting (n = 6,685) – whereas these items are not accurate enough to define a severity level. Thus, it would be necessary to do more in-depth analysis to assess the performance of each item and evaluate its relevance, or to reclassify or delete insufficiently relevant items.

Severity outcomes (ICU admission, hospitalization, PED LOS) correlated inversely with the pediaTRI triage level: the severity profile decreased from Level 1 to 5.

The variability of the performance values of pediaTRI underlines the need to provide a framework to mitigate the risk of failing to identify critically ill patients. Consequently, we compiled a guide to ensure that patients are triaged safely. Firstly, the duration of the triage should be long enough to assess the patient adequately without unnecessarily delaying access to care. We recommend that the triage should initially focus on the recognition and management of critically ill children based on the ABCDE approach of the Pediatric Life Support of the European Resuscitation Council Guidelines (30). In the absence of a life-threatening condition, the trained nurse then performs a structured clinical interview to detect the presence or absence of signs system by system (i.e., cardiovascular, pulmonary,..) to capture any significant signs that could upgrade the triage level. The computerized version of the pediaTRI tool includes pop-up messages to inform the nurse if essential data (vital signs) are missing for the triage process. The nurse can upgrade the triage level if they disagree with the one suggested by pediaTRI, but not downgrade the level. Finally, the nurses are encouraged to ask the on-call pediatrician to confirm the final triage level of a patient at any time if in doubt.

Comparison With Other Studies

Most triage tool validation studies are based on correlations with resource utilization rather than comparing performance against a gold standard (6) (Appendix 4), mainly the Manchester Triage Scale (MTS). Our performance values seem better than those of the Australasian Triage Scale, the Canadian Emergency Department Triage and Acuity Scale, the Emergency Severity Index (5), and similar to those of MTS (31–33) with sensitivity values ranging from 57 to 83% and specificity values from 69 to 93.7% depending on the study. However, the prevalence of pediaTRI for over-triage and under-triage appears to be better than those estimated for MTS. Indeed, while 84.5% of the patients in our study were correctly triaged, 15% were over-triaged and only 0.5% under-triaged.

Except for patients under 3 months old, for whom pediaTRI was more sensitive than MTS (100% vs. 50 – 79%) due to the over-triage setting by default as explained previously, the performance values of pediaTRI are in the range of those reported in the literature (32, 34): i.e., with a higher specificity (pediaTRI: 78.5 – 88.2% vs. others: 69 – 88%) than sensitivity (pediaTRI: 75.6 – 84.4% vs. others: 65 – 67%). Unfortunately, we are not able to interpret our results concerning the performance of pediaTRI by subcategory of chief complaint or specific diagnosis in relation to other works, as these outcomes are missing in the literature.

However, our reference standard differed from previous studies that used a benchmark triage tool for identifying high-level emergencies. The benchmarks were mainly created in situ by a committee of experts and differed from one study to another due to different practices. Although the performance of our reference standard (PEWS) may not be perfect, it has nonetheless been shown that a high PEWS score is correlated with a higher risk of patient morbidity and mortality that defines high-level emergency (16, 18, 19, 28, 35). Nevertheless, we also report a strong association between triage level and markers of severity, namely hospitalization rates and PED LOS. As reported in previous studies, hospitalization (especially ICU admission) and a longer PED LOS are associated with high-level emergencies. Our hospitalization rates are in the lower range of those reported in the literature (11, 13, 31, 36–40) with 53.5–100%, 28.6–46%, 8–17%, 1.2–6%, and 0–2%, respectively, for triage levels 1 to 5. Similarly, our PED LOS are also slightly lower than those reported in the literature (11, 13, 36, 37, 40, 41) with 102–334 min, 221–309 min, 191–262 min, 96 −186 min, and 66–160 min, respectively, for triage levels 1 to 5: as expected, high-acuity patients present conditions that require more procedures, time for stabilization, and work-up. Thus, hospitalization rates and PED LOS would appear to be useful markers to validate triage tools.

Strengths and Limitations of the Study

A Large-Scale Population Study

Our study analyzed data from a large number of patients (> 1,00,000) over 2 successive years (2016–2017), thereby avoiding any bias related to temporality (season, week, day, time of day/night) and suggesting that our results are representative of all the pathologies encountered in a large-scale PED in France. Statistical analyses carried out on a large control cohort (> 100,000 patients from 2018–2019) served as validation for the sample cohort.

The prevalence of our five severity levels were in line with those reported in previous studies, with prevalence ranging from 0.3–16.5%, 4.5–26.4%, 20.0–55.0% 17.2–52.1%, and 0.1–21.2% for Levels 1 to 5, respectively (13, 34, 41, 42) (Appendix 3).

The prevalence of medical diagnoses in our study were also similar to that of previous studies, from 1.1–5.4% for bronchiolitis (43, 44), 2.5–16.1% for asthma (36, 44), and 4.9–6.4% for acute gastroenteritis (42). However, we had more cases of limb and head trauma compared with other studies (11, 42, 43). Limb trauma in our study comprised all types of diagnosis, such as contusion and sprain, wound, and dislocation which explains our higher prevalence compared to Gravel et al. (36) and Acworth et al. (42). Likewise, head injuries in our study were more frequent compared with the study by Christoffel et al. (43) because our definition of head injuries included mild contusion with or without scalp wound as well as severe head injuries (including skull bone fracture and post-traumatic intracranial hemorrhage) (Appendix 5).

A Tool Specifically Used for Triaging Pediatric Patients

In the absence of a standardized pediatric triage tool in France, our pediaTRI tool shares the characteristics of the other large-scale tools: it is a computerized tool with five levels of severity that analyzes symptoms and vital signs. However, pediaTRI was specifically designed for evaluating pediatric patients while the existing pediatric triage tools are adaptations of tools designed for adult patients. Finally, pediaTRI can be used by any trained nurse without specific clinical skills.

The Reference Standard

PEWS can be applied to any PED to identify high-level emergencies. It is a simple and objective tool that scores the severity of patients based on three parameters: the neurologic, cardiovascular, and respiratory systems. No lab workup or complementary tests are required which means that the nurse can quickly triage their pediatric patients. The PEWS has been validated with good reproducibility (18, 35, 45–47) due to its simplicity, and allows both initial assessment and continuous monitoring irrespective of the hospital unit. Although it was designed to detect changes over time, its implementation in EDs to detect early clinical deterioration of patients has been studied with a score that changes little over time as the patients are mostly in the ED for a relatively limited time. (19, 27, 45). Thus, a high PEWS score (≥4/9), defined as a predictor of ICU admission, assigned in a PED setting can be an indicator of high-level emergency. However, authors agree that PEWS alone is insufficient for assessing or triaging patients.

There are some other limitations to our study. First, this was a single-center study which somewhat limits the generalizability of our results despite the large sample size.

Furthermore, data were missing for around 6 % of the children, especially for patients with unreliable vital signs (2%). These were considered as abnormal or alarming because of difficult conditions during the measurement as the children were crying or scared. The PED environment is not the ideal setting to record reliable vital signs (48) which is why we considered the vital signs as normal by default. Although we included a large patient sample, we observed a very low rate of Level 1 cases (n = 217, 0.2%). Level 1 (immediate emergency requiring resuscitation and emergency care) is rare in pediatrics and our prevalence is in the lower range of those in the literature (from 0.3 to 16.5%) (Appendix 3). Differences in the prevalence of such cases may be partly explained by the organization of the healthcare system according to the country. In France, mobile emergency and intensive care service units (Service Mobile d'urgence et de Réanimation-SMUR) (49, 50) are triggered before hospital involvement to manage patients in a critical condition and these patients are admitted directly in the ICU without being assessed in the PED.

We compared our 5-level pediaTRI tool with PEWS, a reference standard based on identifying patients at risk of clinical deterioration by classifying them into two groups: high-level vs. low-level emergencies. However, the PEWS screens for signs of severity and does not include analysis of the chief complaint suggesting that it might not be as efficient as pediaTRI for analyzing medical vs. surgical complaints. It would be interesting to use a modified PEWS that integrates pain and disability, for example. However, we considered that PEWS was an appropriate reference standard because of its practicability (standardized evaluation), and feasibility (using a simple score for triaging patients) even though its validity has been challenged in the literature (51). Furthermore, Seiger et al. found that the ability of the PEWS for predicting hospitalization was low to moderate (16), but moderate to good for ICU admission. In a prospective study, Breslin et al. (45) demonstrated an area under the curve of 0.68 to detect the need for hospitalization (but 0.80 in the subpopulation of respiratory pathologies). Mandell et al. (52) concluded that no threshold was of sufficient sensitivity or specificity to be clinically useful for identifying with certainty the need for ICU admission, although a high PEWS was statistically associated with an increased risk of admission to intensive care. However, setting the threshold too high would result in failure to identify some seriously ill children, while setting it too low would unnecessarily utilize precious time and resources. In our study, we defined a PEWS ≥ 4 (26–28) as denoting a high-level of emergency as this threshold is significantly related to ICU admission. Finally, the performance of pediaTRI was compared with PEWS as a 2-level pediatric triage tool to screen for high- and low-level emergencies rather than a 5-level tool as recommended in the literature (5, 31, 32, 34, 53, 54). Meanwhile, our results based on the markers of severity (ICU admission, hospitalization, PED LOS) are in agreement with those reported in the literature.

Conclusion

Our pediaTRI tool used by trained nurses showed a moderate to good validity to screen high-level emergencies in pediatric emergency care but its performance differs according to the subcategory of the chief complaint and the patient profile. Multicenter studies are needed to assess its validity and reliability as a 5-level triage tool. Experienced nurses with clinical skills at the triage zone could improve the triage process and, in the future, we could aim to introduce the “clinical nurse” status in France.

Strengths and Limitations of This Study

• This is the first study to validate a French pediatric triage tool, pediaTRI, compared with a reference standard.

• We analyzed a large study sample over 2 successive years in the 4^th largest 24/7 PED in France.

• Statistical analyses carried out on a large control cohort (>100,00 patients from 2018–2019) served as validation for the sample cohort.

• The performance of pediaTRI was comprehensively assessed according to the age of the patients, the type of complaint (medical vs. surgical), and the category of complaint.

• This was a single-center study limiting its applicability in emergency departments across France.

• The PEWS, used as the reference standard, did not allow effective comparison of performance of pediaTRI with previous studies.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements. Verbal informed consent was provided.

Author Contributions

AT was responsible for protocol development, analyses, writing of the paper and the main author of the paper. PV supervised the protocol development, supervised the analyses, writing of the paper and had a main contribution in editing the paper. CR supervised the protocol development, data collection, and writing of the paper. ER was responsible for data collection. EF, EB, and OH supervised the writing of the paper. HH supervised the protocol development, and writing of the paper. CP and SG supervised the protocol development, analyses, and writing of the paper. All the authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We are grateful to PV for revising the manuscript, and to Felicity Neilson for her editing services.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2022.840181/full#supplementary-material

References

1. Urgences. Direction De La Recherche, Des Études, De L'évaluation Et Des Statistiques. Available online at: https://drees.solidarites-sante.gouv.fr/urgences (accessed September 3, 2021).

Google Scholar

2. American Academy of Pediatrics Committee on Pediatric Emergency Medicine. Overcrowding crisis in our nation's emergency departments: is our safety net unraveling? Pediatrics. (2004). 114:878–88. doi: 10.1542/peds.2004-1287

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Jiménez JG, Murray MJ, Beveridge R, Pons JP, Cortés EA, Garrigós JBF, et al. Implementation of the Canadian emergency department triage and acuity scale (CTAS) in the principality of andorra: can triage parameters serve as emergency department quality indicators? CJEM. (2003) 5:315–22. doi: 10.1017/S1481803500015827

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Référentiels SFMU / Société Française de Médecine d'Urgence - SFMU. Available online at: https://www.sfmu.org/fr/vie-professionnelle/outils-professionnels/referentiels-sfmu/referentiel-ioa-infirmiere-organisatrice-de-l-accueil/ref_id/11 (accessed July 28, 2021).

Google Scholar

5. Aeimchanbanjong K, Pandee U. Validation of different pediatric triage systems in the emergency department. World J Emerg Med. (2017) 8:223–7. doi: 10.5847/wjem.j.1920-8642.2017.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

6. de Magalhães-Barbosa MC, Robaina JR, Prata-Barbosa A, Lopes C de S. Validity of triage systems for paediatric emergency care: a systematic review. Emerg Med J EMJ. (2017) 34:711–9. doi: 10.1136/emermed-2016-206058

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Magalhães-Barbosa MC, Robaina JR, Prata-Barbosa A, Lopes C de S. Reliability of triage systems for paediatric emergency care: a systematic review. Emerg Med J EMJ. (2019) 36:231–8. doi: 10.1136/emermed-2018-207781

PubMed Abstract | CrossRef Full Text | Google Scholar

8. van Veen M, Moll HA. Reliability and validity of triage systems in paediatric emergency care. Scand J Trauma Resusc Emerg Med. (2009) 17:38. doi: 10.1186/1757-7241-17-38

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open. (2019) 9:e026471. doi: 10.1136/bmjopen-2018-026471

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Ebrahimi M, Heydari A, Mazlom R, Mirhaghi A. The reliability of the Australasian Triage Scale: a meta-analysis. World J Emerg Med. (2015) 6:94–9. doi: 10.5847/wjem.j.1920-8642.2015.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Gravel J, Fitzpatrick E, Gouin S, Millar K, Curtis S, Joubert G, et al. Performance of the Canadian triage and acuity scale for children: a multicenter database study. Ann Emerg Med. (2013). 61:27–32.e3. doi: 10.1016/j.annemergmed.2012.05.024

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Seiger N, van Veen M, Almeida H, Steyerberg EW, van Meurs AHJ, Carneiro R, et al. Improving the manchester triage system for pediatric emergency care: an international multicenter study. PLoS ONE. (2014) 9:e83267. doi: 10.1371/journal.pone.0083267

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Green NA, Durani Y, Brecher D, DePiero A, Loiselle J, Attia M. Emergency Severity Index version 4: a valid and reliable tool in pediatric emergency department triage. Pediatr Emerg Care. (2012) 28:753–7. doi: 10.1097/PEC.0b013e3182621813

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Chamberlain JM, Patel KM, Pollack MM. The pediatric risk of hospital admission score: a second-generation severity-of-illness score for pediatric emergency patients. Pediatrics. (2005) 115:388–95. doi: 10.1542/peds.2004-0586

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Pollack MM, Patel KM, Ruttimann UE, PRI SM III. an updated pediatric risk of mortality score. Crit Care Med. (1996) 24:743–52. doi: 10.1097/00003246-199605000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Seiger N, Maconochie I, Oostenbrink R, Moll HA. Validity of different pediatric early warning scores in the emergency department. Pediatrics. (2013) 132:e841–850. doi: 10.1542/peds.2012-3594

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Brown SR, Martinez Garcia D, Agulnik A. Scoping review of pediatric early warning systems (PEWS) in resource-limited and humanitarian settings. Front Pediatr. (2018) 6:410. doi: 10.3389/fped.2018.00410

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Duncan H, Hutchison J, Parshuram CS. The pediatric early warning system score: a severity of illness score to predict urgent medical need in hospitalized children. J Crit Care. (2006) 21:271–8. doi: 10.1016/j.jcrc.2006.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Gold DL, Mihalov LK, Cohen DM. Evaluating the pediatric early warning score (PEWS) system for admitted patients in the pediatric emergency department. Acad Emerg Med Off J Soc Acad Emerg Med. (2014) 21:1249–56. doi: 10.1111/acem.12514

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lambert V, Matthews A, MacDonell R, Fitzsimons J. Paediatric early warning systems for detecting and responding to clinical deterioration in children: a systematic review. BMJ Open. (2017) 7:e014497. doi: 10.1136/bmjopen-2016-014497

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Lillitos PJ, Hadley G, Maconochie I. Can paediatric early warning scores (PEWS) be used to guide the need for hospital admission and predict significant illness in children presenting to the emergency department? An assessment of PEWS diagnostic accuracy using sensitivity and specificity. Emerg Med J. (2016) 33:329–37. doi: 10.1136/emermed-2014-204355

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Portas M, Firoloni J-D, Brémond V, Giraud P, Coste M-E, Lescure P, et al. Impact of a triage scale in a pediatric emergency department. Arch Pediatr Organe Off Soc Francaise Pediatr. (2006) 13:1507–13. doi: 10.1016/j.arcped.2006.08.019

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Présentation, TU. Terminal Urgences. Available online at: https://tgs.ies-sud.fr/presentation-tu/ (accessed December 17, 2020).

Google Scholar

24. Sites, Equipés. Terminal Urgences. Available online at: https://tgs.ies-sud.fr/sites/ (accessed November 22, 2021).

Google Scholar

25. Haas H. Triage scales in paediatric emergency departments. Arch Pediatr Organe Off Soc Francaise Pediatr. (2005) 12:703–5. doi: 10.1016/j.arcped.2005.04.048

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Akre M, Finkelstein M, Erickson M, Liu M, Vanderbilt L, Billman G. Sensitivity of the pediatric early warning score to identify patient deterioration. Pediatrics. (2010) 125:e763–769. doi: 10.1542/peds.2009-0338

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Bradman K, Borland M, Pascoe E. Predicting patient disposition in a paediatric emergency department. J Paediatr Child Health. (2014) 50:E39–44. doi: 10.1111/jpc.12011

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Monaghan A. Detecting and managing deterioration in children. Paediatr Nurs. (2005) 17:32–5. doi: 10.7748/paed.17.1.32.s27

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Wayant C, Scott J. Vassar M. Evaluation of lowering the P-value threshold for statistical significance from 05 to 005 in previously published randomized clinical trials in major medical journals. JAMA. (2018) 320:1813–5. doi: 10.1001/jama.2018.12288

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Van de Voorde P, Turner NM, Djakow J, de Lucas N, Martinez-Mejias A, Biarent D, et al. European resuscitation council guidelines 2021: paediatric life support. Resuscitation. (2021) 161:327–87. doi: 10.1016/j.resuscitation.2021.02.015

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Roukema J, Steyerberg EW, van Meurs A, Ruige M, van der Lei J, Moll HA. Validity of the manchester triage System in paediatric emergency care. Emerg Med J EMJ. (2006) 23:906–10. doi: 10.1136/emj.2006.038877

PubMed Abstract | CrossRef Full Text | Google Scholar

32. van Veen M, Steyerberg EW, Ruige M, van Meurs AHJ, Roukema J, van der Lei J, et al. Manchester triage system in paediatric emergency care: prospective observational study. BMJ. (2008) 337:a1501. doi: 10.1136/bmj.a1501

PubMed Abstract | CrossRef Full Text | Google Scholar

33. van der Wulp I, van Baar ME, Schrijvers AJP. Reliability and validity of the manchester triage system in a general emergency department patient population in the Netherlands: results of a simulation study. Emerg Med J EMJ. (2008) 25:431–4. doi: 10.1136/emj.2007.055228

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Seiger N, van Veen M, Steyerberg EW, van der Lei J, Moll HA. Accuracy of triage for children with chronic illness and infectious symptoms. Pediatrics. (2013) 132:e1602–1608. doi: 10.1542/peds.2013-1076

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Tucker KM, Brewer TL, Baker RB, Demeritt B, Vossmeyer MT. Prospective evaluation of a pediatric inpatient early warning scoring system. J Spec Pediatr Nurs JSPN. (2009) 14:79–85. doi: 10.1111/j.1744-6155.2008.00178.x

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Gravel J, Gouin S, Goldman RD, Osmond MH, Fitzpatrick E, Boutis K, et al. The Canadian triage and acuity scale for children: a prospective multicenter evaluation. Ann Emerg Med. (2012). 60:71–77.e3. doi: 10.1016/j.annemergmed.2011.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Gravel J, Manzano S, Arsenault M. Validity of the Canadian paediatric triage and acuity scale in a tertiary care hospital. CJEM. (2009) 11:23–8. doi: 10.1017/S1481803500010885

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Gouin S, Gravel J, Amre DK, Bergeron S. Evaluation of the paediatric canadian triage and acuity scale in a pediatric ED. Am J Emerg Med. (2005) 23:243–7. doi: 10.1016/j.ajem.2004.02.046

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Travers DA, Waller AE, Bowling JM, Flowers D, Tintinalli J. Five-level triage system more effective than three-level in tertiary emergency department. J Emerg Nurs. (2002) 28:395–400. doi: 10.1067/men.2002.127184

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Baumann MR, Strout TD. Evaluation of the emergency severity index (version 3) triage algorithm in pediatric patients. Acad Emerg Med Off J Soc Acad Emerg Med. (2005) 12:219–24. doi: 10.1197/j.aem.2004.09.023

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Travers DA, Waller AE, Katznelson J, Agans R. Reliability and validity of the emergency severity index for pediatric triage. Acad Emerg Med Off J Soc Acad Emerg Med. (2009) 16:843–9. doi: 10.1111/j.1553-2712.2009.00494.x

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Acworth J, Babl F, Borland M, Ngo P, Krieser D, Schutz J, et al. Patterns of presentation to the Australian and New Zealand paediatric emergency research network. Emerg Med Australas EMA. (2009) 21:59–66. doi: 10.1111/j.1742-6723.2009.01154.x

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Christoffel KK, Garside D, Tokich T. Pediatric emergency department utilization in the (1970s). Am J Emerg Med. (1985) 3:177–81. doi: 10.1016/0735-6757(85)90083-X

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Thakker Y, Sheldon TA, Long R, MacFaul R. Paediatric inpatient utilisation in a district general hospital. Arch Dis Child. (1994) 70:488–92. doi: 10.1136/adc.70.6.488

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Breslin K, Marx J, Hoffman H, McBeth R, Pavuluri P. Pediatric early warning score at time of emergency department disposition is associated with level of care. Pediatr Emerg Care. (2014) 30:97–103. doi: 10.1097/PEC.0000000000000063

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Miranda J de OF, Camargo CL de, Nascimento CL, Portela DS, Monaghan A. Accuracy of a pediatric early warning score in the recognition of clinical deterioration. Rev Lat Am Enfermagem. (2017) 25:e2912. doi: 10.1590/1518-8345.1733.2912

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Skaletzky SM, Raszynski A, Totapally BR. Validation of a modified pediatric early warning system score: a retrospective case-control study. Clin Pediatr (Phila). (2012) 51:431–5. doi: 10.1177/0009922811430342

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Thompson T, Stanford K, Dick R, Graham J. Triage assessment in pediatric emergency departments: a national survey. Pediatr Emerg Care. (2010) 26:544–8. doi: 10.1097/PEC.0b013e3181ea718e

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Barrier G. Emergency calls in the Samu system. C R Acad Sci III. (2001) 324:663–6. doi: 10.1016/S0764-4469(01)01334-8

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Julliand S, Lodé N. Paediatric mobile emergency and intensive care services, objectives and missions. Soins Pediatr Pueric. (2016) 37:17–21. doi: 10.1016/j.spp.2016.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Parshuram CS, Dryden-Palmer K, Farrell C, Gottesman R, Gray M, Hutchison JS, et al. Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: the EPOCH randomized clinical trial. JAMA. (2018) 319:1002–12. doi: 10.1001/jama.2018.0948

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Mandell IM, Bynum F, Marshall L, Bart R, Gold JI, Rubin S. Pediatric early warning score and unplanned readmission to the pediatric intensive care unit. J Crit Care. (2015) 30:1090–5. doi: 10.1016/j.jcrc.2015.06.019

PubMed Abstract | CrossRef Full Text | Google Scholar

53. van Veen M, Steyerberg EW., Van't Klooster M, Ruige M, van Meurs AHJ, van der Lei J, et al. The Manchester triage system: improvements for paediatric emergency care. Emerg Med J EMJ. (2012) 29:654–9. doi: 10.1136/emermed-2011-200562

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Branes H, Solevåg AL, Solberg MT. Pediatric early warning score vs. a paediatric triage tool in the emergency department: a reliability study. Nurs Open. (2021) 8:702–8. doi: 10.1002/nop2.675

PubMed Abstract | CrossRef Full Text | Google Scholar