East London Modified-Broset as Decision-Making Tool to Predict Seclusion in Psychiatric Intensive Care Units

Seclusion is a last resort intervention for management of aggressive behavior in psychiatric settings. There is no current objective and practical decision-making instrument for seclusion use on psychiatric wards. Our aim was to test the predictive and discriminatory characteristics of the East London Modified-Broset (ELMB), to delineate its decision-making profile for seclusion of adult psychiatric patients, and second to benchmark it against the psychometric properties of the Broset Violence Checklist (BVC). ELMB, an 8-item modified version of the 6-item BVC, was retrospectively employed to evaluate the seclusion decision-making process on two Psychiatric Intensive Care Units (patients n = 201; incidents n = 2,187). Data analyses were carried out using multivariate regression and Receiver Operating Characteristic (ROC) curves. Predictors of seclusion were: physical violence toward staff/patients OR = 24.2; non-compliance with PRN (pro re nata) medications OR = 9.8; and damage to hospital property OR = 2.9. ROC analyses indicated that ELMB was significantly more accurate that BVC, with higher sensitivity, specificity, and positive likelihood ratio. Results were similar across gender. The ELMB is a sensitive and specific instrument that can be used to guide the decision-making process when implementing seclusion.

approach to violent patients based on degree of risks is identified as one of the major flaws by its detractors.
In spite of the fact that assessment of risk of violence on psychiatric wards has become much easier and more reliable thanks to the implementation of validated instruments like the Broset Violence Checklist (BVC) (24), or the Staff Observation Aggression Scale Revised (25), the decision to seclude a patient remains mostly an ad hoc process. This is not surprising considering the dynamic nature of a decision-making process (26) that reflects a complex interplay of multiple patient-related (5,(27)(28)(29), environmental-related (9,(30)(31)(32), and team-related factors (33)(34)(35).
Indeed, assessment tools that may predict seclusion as part of a therapeutic care plan have become available (36,37), but they are research-oriented, and far too impractical to be used in every day clinical practice, as they depend on prior collection of detailed risk information regarding demographic, historical, and behavioral factors. The complexity and length of these tools means they cannot meet the requirements of a quick decisionmaking process by the frontline staff on the ward, particularly needed when dealing with rapidly unfolding situations (38).
Based on our clinical experience of Psychiatric Intensive Care Units (PICUs), and supported by evidence from the literature on seclusion, we extended on our previous exploratory work (39) on the development of a modified version of the BVC, the East London Modified-Broset (ELMB), taking into consideration two factors affecting staff 's seclusion decision-making when containing high acuity behaviors, that is, patient's degree of compliance with PRN (pro re nata) medications, per os (P.O.) or intramuscular (I.M.), and response to de-escalation (verbal and/or physical) or lack thereof.
In this case-control study, we retrospectively employed ELMB in a sample of patients consecutively admitted to two PICUs over a 3-year period. The aims of this study were to analyze the predictive characteristics of ELMB with respect to secluding adult psychiatric inpatients at high risk of violence; to describe ELMB's accuracy levels; its ability to discriminate patients' degrees of risk; to delineate ELMB as a seclusion decision-making tool. In the absence of a gold-standard seclusion tool, the BVC was used as a benchmark instrument to compare the discriminatory characteristics of ELMB.

Participants
The sample consisted of 252 inpatients (males n = 130; females n = 122) consecutively admitted to Millharbour (male) and Rosebank (female) PICUs at Tower Hamlets Centre for Mental Health, Mile End Hospital (London, UK), between June 2012 and May 2015. The two PICUs serve the London Borough of Tower Hamlet catchment area with an estimated population of about 280,000, whereas Rosebank PICU covers two additional London boroughs, with an aggregated overall population of over 550,000. The male PICU, with a bed capacity of 14, and the female PICU of 11, serve primarily Mile End Hospital in Tower Hamlets with an overall capacity of 76 general adult beds. Patients were included in the study if they had a confirmed or suspected diagnosis of psychiatric disorder meeting the criteria of DSM-IV-TR (40), and were 18-65 years old. Two of the individual participants in the female sample were older than 65 (female age range = 18-69; male age range = 18-53). Although individual participants with age >65 would usually be treated on an Old Age Psychiatry ward, in selected circumstances, these patients would be admitted to the general adult PICU based on severity of violence and physical fitness. Patients with behavioral disturbance secondary to general medical condition were excluded. Only one individual diagnosed with Limbic Encephalitis with N-methyl-d-aspartate receptor antibodies was thus excluded from the analyses. Because of the "revolving door nature" of patients admitted to PICUs (41), in order to minimize confounders on the seclusion decision-making process of patients already known to each PICU (42), only the first admission to PICU of patients' multiple admissions were included for ratings. Finally, 51 individual participants (20%) were excluded from the analyses due to inadequate information reported in the clinical records.

Psychiatric assessment
A specialist in General Adult Psychiatry (Felice Loi) was responsible for the retrospective ratings of the participants' clinical entries in the electronic medical records. For the purpose of this study, an incident was defined as any clinical event that led to an interaction between a patient and one or more members of the clinical team and which involved the use of de-escalation techniques (verbal and/or physical), and/or the administration of PRN sedative medications (P.O. and/or I.M.-see below).

Psychometric Tool characteristics and scoring system
East London Modified-Broset (Figure 1) is an 8-item checklist based on the 6-item BVC, of which follows similar compilation rules (24), albeit not for all items. In our setting, a patient was deemed "confused" as synonym for severely thought-disordered, where the level of thought disorder interfered considerably with the ability to process information provided during interaction with staff (for instance, significant thought blocking, illogicality, racing thoughts with overinclusive thinking, and loosening of associations) which could be associated or not with cognitive disorientation to time, place, or person. The items "irritable" (significant affect reactivity disproportionate to the level of stimuli available); "boisterous" (being loud, banging on items in order to attract someone's attention without intention to damage hospital property); "verbal threats" (articulating verbal insults with or without menacing words which threaten the victim's perceived sense of safety); "physical threats" (threatened or actual violence); and "attacking objects" (hitting objects with the intention to cause damage to property), were rated as in the original BVC, that is, 0 if the behavior was not present, 1 if the behavior was present. The main difference between the two tools is the inclusion of two additional items in the ELMB: (1) "response to de-escalation" and (2) "PRN compliance" (i.e., whether the patient has accepted or not any PRN medications offered). The item "response to de-escalation" deserves further specification. A patient whose physically aggressive behavior led to members of staff employing measures of physical control over his behavior (i.e., any type of physical contact between a member of staff and a patient) was generally accompanied by measures of verbal de-escalation which could, within a variable length of time, lead to the patient regaining full control over his own behavior with resolution of the behavioral crisis. In this case, the patient was deemed to have responded to de-escalation and his behavior rated on the ELMB as "0" as opposed to the inability to respond to staff 's interventions and the consequent escalation of behavior, an outcome that would be rated as "1". The number of positive items was added up to reach a total possible score of 8/8 on the ELMB, and of 6/6 on the BVC with higher scores indicating higher levels of behavioral disturbance (implicit for the ELMB, of reduced ability to cooperate with staff). The inclusion of these two factors in the ELMB is supported by evidence indicating that staff 's interaction with patients, represents the most important trigger to violence in almost 40% of cases (43) often with patients perceiving some form of imposed restriction. Overall, the BVC is a risk prediction tool of violence occurring within 24 h following an assessment, while the ELMB is a decision-making tool to be used at the time an incident is unfolding.

Data analysis
Statistical analyses were carried out using Stata SE 14.1 for Mac (44) and MedCalc Statistical Software for Windows version 16.2 (45). Sample size required was calculated with G*Power Version 3.1.9.2 for Mac (46). A minimum sample size of 158 individual participants was required to achieve a pre-specified power of 85% for a one-tailed probability. To minimize bias of the logistic regression model (47), a minimum number of 10 events per variable was calculated by multiplying the number of predictors (8) by a constant K (10 seclusion events), thus yielding a minimum total number of 80 seclusion events.
Parametric and non-parametric analyses along with effect size (ES) were calculated to identify differences in demographic and clinical variables among secluded and non-secluded patients. In order to improve accuracy of estimates, bootstrap test statistics were implemented with bias corrected accelerated (BCa) computation of confidence intervals. Prior to running bootstrap analyses, a seed number was set in order to ensure reproducibility of the findings (48) generating a random numeric sequence taken from a pound sterling note (set seed number 78484608). Number of bootstrap repetitions was set to 1,000. Analyses were carried out for the whole sample and, separately, for the male and female individual participants to detect a main effect of gender.
Model accuracy for each of the two tools was estimated using Receiver Operating Characteristics (ROC) curves with overall performance expressed by the area under the curve (AUC) index. The Youden Index (YI) was calculated in order to identify the optimum threshold point on the ROC curve correspondent to best compromise between true positive and false positive rates (49). Pairwise difference of the AUCs for the two tools was calculated and ES of the differences reported. In addition, positive (LHR + ) and negative likelihood (LHR − ) ratios were reported to quantitatively reflect the accuracy of the tools to confirm the presence of a true positivity or a true negativity compared to participants that, respectively, needed to be secluded or that did not need to be secluded.

Model Building
In order to identify ELMB psychopathological items that were predictive of the use of seclusion, logistic regression analyses were carried out. Chi-square analyses were initially performed to identify psychopathological items that were significantly associated with the use of seclusion. All eight ELMB predictors achieved significance level. Therefore, they were forced-entered into a binary regression model (BRM) as independent variables, with seclusion as dependent variable. In preliminary analyses, "response to de-escalation" disclosed very large odds ratios (ORs) = 451, SE = 327.2, p = 0.000, 95% CI (108.8, 1,869.8) suggesting a problem of separation (50). In conditions of separation, one or more predictors exert a very high predictive effect, which leads to instability in the BRM whose maximum likelihood estimates cannot be calculated. In such conditions, a penalized likelihood estimate can be more appropriately calculated with Firthlogit method (51) implemented in Stata with firthlogit command (52). However, since Firthlogit algorithm did not appreciably correct the bias, albeit not desirable due its strong predictive effect (53), "response to de-escalation" was constrained in the model (i.e., retained to exert its overall effect but without estimating its predictive value). Using a hierarchical method, two constrained models were thus developed based on prior findings from seclusion literature, our own clinical experience, and statistical measures of fit, including Aikake's Information Criterion (AIC). Both models incorporated physical threats, attacking objects, and PRN compliance, with the difference of verbal threats, which was present only in model 1. This is because verbal threatening behavior can shift staff 's decision to seclude a patient based on findings from previous studies (27,54,55   In the sample of patients excluded from the analyses, a statistically significant difference was found among secluded and non-secluded in type of primary diagnosis only, two-sided Fisher's Exact: p = 0.001, with most of individuals diagnosed with paranoid schizophrenia (n = 19 or 59.4%).

Psychometric Tools
East London Modified-Broset Analyses were repeated to compare male and female patients on levels of behavioral disturbance as measured by ELMB and the BVC for all seclusion and non-seclusion incidents. No statistically significant difference was found (results not reported).

Regression Model
The results of Firthlogit (constrained) model for the whole sample of participants, as well as the female and male groups are reported in Tables 2-4, respectively. The analyses show that the most important predictors of seclusion were acts of physical threats (males OR = 18.9; females OR = 33.1; combined OR = 24.2), attacking objects (males OR = 2; females OR = 3.6; combined OR = 2.9), PRN compliance (males OR = 18.4; females OR = 7.5; combined OR = 9.85).
Although only three of the eight behavioral items (physical threats, attacking objects, PRN compliance) listed in the ELMB achieved significance level, all predictors were included in the final ROC analyses as these factors were judged to aid team discussion around the appropriateness of using seclusion. This choice was based on the clinical ward experience that the interaction between a member of staff and a patient is not motivated only by the occurrence of a sudden episode of physical aggression, as most incidents are dealt with prior to an escalation to an aggressive episode. The early identification of behavioral elements such as significant thought disorder, irritability, boisterousness, verbal threats, associated or not with aggression toward items, offer an immediate opportunity for early therapeutic engagement (through the offer of de-escalation approaches and/or PRN mediations), which prevents, in many cases, the occurrence of incidents of actual violence. In support of this approach, the bootstrap logistic regression analyses indicated that higher scores on ELMB were more positively predictive of the use of seclusion ( Table 5) female OR = 7.2; male OR = 4.5; combined OR = 5.7, with similar trends, albeit with lower ES of the model, described by BVC scores (bottom half of Table 5). In order to identify which scores on the ELMB were associated with the highest predictive value of seclusion, incidents were stratified into two clusters, one with ELMB scores ≤4 and one with ELMB scores >4 based on Youden index associated criterion >4. Bootstrap analyses ( Table 6) showed     Figure 2 plots a comparison of ROC curves for the whole sample for ELMB and BVC. The difference between the two AUCs ( Table 7) was highly significant, Δ AUCs = 0.10, SE = 0.0079, p < 0.0001, BCa 95% CI (0.085, 0.117), with large ES: z = 12.6 indicating that ELMB's overall discriminatory characteristics outperformed the BVC by a large margin. The Youden Index and the associated criterion for the BVC and ELMB are reported in Table 7. The best trade-off between true positive and false positive rates as identified by the Youden Index lies at a score >4 for both tools. Table 7 also shows that at the Youden Index associated criterion >4, the sensitivity and specificity are higher for ELMB as compared to the BVC. Accuracy measures for the whole sample are reported in Table 8. From a close examination of Table 8, two clear trends emerge: (1) at Youden Index associated criterion >4 and (2) with ELMB score >6 (respectively indicating patients who achieved an ELMB score >4 and those with a score above 6). In the first case, the classification system indicated that ELMB was superior to the BVC (similar LHR + but LHR − and NPV significantly greater for ELMB) across all of the parameters, but for specificity (BVC = 87.7% > ELMB = 83.7%) and FPR (BVC = 12% < ELMB = 16%). At cut-off > 6, ELMB significantly outperformed BVC with an extremely high specificity (99.3%), low FPR (0.7%), and high PPV (96.4%), and LHR + significantly higher (66.5) than what normally expected (≥20) of tests' requirements (56). Results were similar for the female and male groups (Tables 9 and 10).

DiscUssiOn
The results of this study support ELMB as a promising seclusion decision-making and prediction tool for hic et nunc use on acute psychiatric wards. Three factors significantly predicted the occurrence of seclusion in a sample of acutely disturbed patients, namely attacking objects, physical threats, and PRN compliance. The effect size of these findings was moderate for the first factor, with a large effect for the second and the third factors. In addition, results were consistent across the male and female samples. This can be partly explained on one hand, by the involvement in emergency situations of composite teams with members responding from both PICUs (as well as from other acute wards within the hospital), who, therefore, share common approaches to violent behavior, on the other, by the characteristics common to patients generally admitted to these highly specialized wards. These results are in line with findings from previous studies (5,27,28,55). Lack of response to de-escalation was statistically significant and has been associated in previous studies with violent behavior leading to seclusion (54,55). Mann-Poll et al. (57) found that patient's uncooperativeness played a paramount role in the decision-making process over seclusion. In our study, however, the true magnitude of the effect of response to de-escalation on the final odds of being secluded could not be fully estimated due to issues of separation. Problems of separation in logistic regression are common in situations in which one or more factors in the model perfectly predict the outcome (58). We believe that perfect prediction was at play in this case, as the preliminary assumptions of the BRM were fully met. In addition, results were consistent across the male and female samples. This indicates that response to de-escalation had a disproportionate weight on the decision to seclude a patient and is largely independent of the combined effects of the other behavioral predictors (attacking objects, physical threats, and PRN compliance), because the inclusion in a non-constrained BRM of response to de-escalation alone did not significantly change the final odd ratios. This primacy of response to de-escalation over other factors in driving the decision-making process is a likely reflection of the strict compliance with the Mental Health Act Code of Practice in England (59) and the view that seclusion is a last-resort approach to violent behavior. This is also supported by the large discrepancy between ELMB's identified optimal threshold to seclusion (Youden Index associated criterion > 4) and staff 's perception of patients' risk levels appropriate to seclusion (ELMB>6). The items of confused, irritable, boisterous, and verbal threats did not predict the occurrence of seclusion, in line with our clinical experience that patients exhibiting these behaviors do not generally meet the threshold to being secluded. Patients with history of seclusion were twice as likely to be secluded during subsequent admissions compared to never secluded patients, although the small to moderate effect size of the odds, indicated that a positive history of seclusion had a negligible influence on the decision to seclude an acutely unwell patient. Although this association has not so far been fully investigated, these results are broadly in line with those reported by other investigators. For instance, while Swett (60) described a non-significant trend, Thompson (61) indicated that patients with history of seclusion were more likely to be secluded during future re-admissions, between 43% and 81% of the cases. Georgieva et al. (55) also found that patients subjected to measures of coercion (seclusion and/or restraint) were 2.4 times more likely to have had a history of previous exposure to coercive measures. In the latter study, coerced patients were more frequently male, younger, more severely unwell, and uncooperative. It could be that, as proposed by Thompson (61), known violent patients might make staff less tolerant to aggressive behaviors, which may thus lead more easily to seclusion. However, Thompson did not employ measures of current violent behavior, which makes it difficult to draw a full comparison with the attitude of staff in the present study. Our findings showed that patients who were secluded scored significantly higher on ELMB (>6) than the best estimated threshold to seclusion (Youden Index associated criterion >4), indicating that staff tolerated very high levels of disturbed behavior before using seclusion to manage aggression. Unlike Thompson's study, our sample included only the first admission to PICU of patients with multiple admissions, which minimized the chances that staff had secluded the same patient during a previous admission and thus reducing the influence of this confounder on the decision-making process. In addition, the effect of this confounder was minimized by the collective nature of the decision-making process that took place within a composite team of staff, several of whom generally did not have any prior knowledge of the patients' circumstances at the time of the incident.
Substance abuse, common among psychiatric patients (62), and an important contributor to aggression (63)(64)(65), has been linked to violent behavior leading to seclusion (27,66). In our study, no significant difference was found among secluded and nonsecluded patients in substance abuse consumption. There was no other difference in demographic and other clinical characteristics among secluded and non-secluded individuals. These results are not surprising and echo the weak and variable associations between violence, use of seclusion, and demographic and clinical     characteristics described in other studies (67,68). However, these results could also be explained by the homogenous characteristics of patients admitted to PICUs. This arguably indicates that the decision to seclude a patient was mainly motivated by the degree of aggressive behavior regardless of other patients' clinical or demographic background. East London Modified-Broset and BVC scores were significantly higher among patients who were secluded as compared to non-secluded patients with large ES indicating that both tools were very good at discriminating degrees of aggressive behavior, with similar results in the male and female groups. Despite these similarities, we believe that ELMB has an important additional advantage over the BVC in that it allows a dynamic assessment of the patient's response to staff 's interventions (last resort component), and allows staff to review behaviors in order to decide on initiating seclusion. Interestingly, these elements are also rehearsed in some of the major international psychiatric guidelines on the use of seclusion (10, 69,70).
Receiver operating characteristic analyses indicated that the discriminatory power of ELMB was over 12 SDs higher than the BVC's. The inclusion of patient's response to staff 's pharmacological and behavioral interventions are the two key-factors setting it apart from the benchmarked BVC for the purpose of seclusion assessment. Regression models indicated that higher scores were more predictive of seclusion outcome, with almost a sixfold increase in the probability of being secluded with ELMB score > 6. The choice to consistently seclude a patient at a higher (>6) than the best identified threshold (Youden Index associated criterion > 4) speculatively delineates a minimum expected standard that is likely to have acted upon by staff to use seclusion as the very last resort, potentially at the cost of increased risks toward staff. Because of the important implications that the adoption of this course of action can have in terms of ward safety, it is not immediately apparent the rationale behind this decision. Beside the strict compliance with the Mental Health Act Code of Practice England (59), an additional reason might be found in the higher therapeutic cost (FPR = 16%) at the lower threshold > 4, which would have meant that up to 16 in 100 incidents could have been potentially misled seclusion events. Because of the inverse relationship between sensitivity and specificity, the choice of a lower threshold would have driven up sensitivity and FPR to the detriment of specificity (FPR = 1-specificity). The FPR is much higher at ELMB > 4 than that at a cut-off > 6 (FPR = 0.7%), where less than 1 incident in 100 was dealt with seclusion. This might have represented a far better cost-benefit option for the staff decision-making. It is the view of these authors, however, that an increase in the FPR would not automatically reflect an inappropriate increase of seclusion events. This is indicated by the ROC model, which identifies as a "true positive" any seclusion incident that has achieved a high score on the ELMB, compared to nonseclusion incidents that were scored low on the ELMB (i.e., "true negatives"). The seclusion of a patient with a low ELMB score (for instance, a patient secluded following an actual physical assault might score as low as 2 or 3) would be identified by the classification model as a false positive when it would indeed be a "true positive" (i.e., the patient needed to be secluded as immediate precaution for staff and patient safety).
An additional limitation lays in the retrospective nature of the study, which limits the generalizability of the findings. However, the sample size was generous and the results consistent across the male and female samples. Employing the BVC as standard of reference against the ELMB might be regarded as controversial given that the BVC was not designed as seclusion prediction tool. This limitation notwithstanding, the BVC displayed reasonable discriminatory properties non-dissimilar from that of other specific, albeit lengthy, seclusion instruments, like the Resident Assessment Instrument-Mental Health (71). Despite these limitations, our study presents several strengths. ELMB isolates well recognized behavioral components of violence and, unlike many other seclusion assessment tools, is short and focuses on how patients respond to staff 's interventions regardless of other demographic and clinical characteristics. As such, our tool is novel and has the potential to fill an important gap in seclusion clinical practice where it can standardize a consistent approach to the implementation of seclusion on the ward.
In the future, a prospective case-control study looking into the validation of ELMB as seclusion prediction and decision-making tool will be required in order to examine its discriminatory properties in real-time analysis. Finally, in a clinical panorama characterized by the absence of any seclusion discontinuation tools, it would be equally important to test ELMB in this aspect of the clinical assessment in light of the important repercussions that such an assessment can have on the safety of the ward and the patient's care plan.

eThics sTaTeMenT
This study was carried out in accordance with the recommendations of the Joint Research Management Office of Queen Mary Innovation Centre and East London NHS Foundation Trust who gave Ethical and R&D approval. Since the present research was limited to secondary use of information previously collected in the course of normal care (without an intention to use it for research at the time of collection), it was excluded from Research Ethics Committee (REC) review and approval. As such, the present research required only R&D approval by East London NHS Foundation Trust (ELFT). The study was sponsored by East London NHS Foundation Trust: protocol reference KO1308/1.

aUThOr cOnTriBUTiOns
FL and KM conceptualized the work, interpreted the data, and wrote the manuscript. FL designed the research and acquired and analyzed the data. All authors approved the final version of the present manuscript to be published.