Integrated behavioral health services in pediatric primary care and emergency department utilization for suicide risk

Introduction Universal screening for suicide risk in primary care settings is a promising avenue for preventing self-harm and improving health outcomes. Triaging youth to an appropriate level of care, including diverting lower-risk patients from the emergency department (ED) is a meaningful goal. Previous research indicates integrated behavioral health (IBH) may prevent unnecessary admission to the ED on the day of suicide risk screening. We hypothesized that youth who received an IBH consultation the same day as suicide risk screening would be less likely to be admitted to the ED, but more likely to contact IBH services and utilize primary care in the following month. Methods We conducted a retrospective chart review of 3,649 youth aged 10-18 years who were screened with the Ask Suicide-Screening Questions (ASQ) in two pediatric primary care practices. We collected demographic data, ASQ and Patient Health Questionnaire-9 (PHQ-9) scores, as well as patient contacts with IBH, the ED, and medical primary care the day of screening and the following 31 days. We conducted a series of logistic regressions and chi-square analyses to determine whether contact with IBH on the same day as positive suicide risk screenings predicted same-day admission to the ED, IBH contact, and medical primary care utilization. Results Among the 7,982 ASQ scores, 1,380 (18%) were non-acute and 87 ASQs (1%) screened acutely positive. Over 90% of positive screens were diverted from the ED regardless of IBH contact. None of the patients died from suicide. Same-day IBH was associated with higher likelihood of general ED visits for all positive screens (acute and non-acute together). None of the positive screens that received an IBH consultation on the same day as screening were admitted to the ED in the subsequent month. Contact with IBH the same day as screening positively predicted utilization of IBH and medical primary care services in the subsequent month, especially for youth with minority race and ethnicity identities. Discussion In the context of clinics with IBH and systematic risk assessment processes, most youth who screen positive for suicide risk are diverted from the ED. However, contrary to our hypothesis, our study showed that youth who received same-day IBH consultations were more likely to be admitted to the ED compared to peers who did not receive IBH consultations. These findings suggest that systematic suicide screening combined with IBH consultations in pediatric primary care can effectively identify risk levels and triage patients to appropriate care.


Introduction
Youth suicide rates in the United States (U.S.) have risen over the past decade; in 2021, 22% of high school students reported seriously considering attempting suicide, 18% made a plan, and 10% made at least one suicide attempt (1).Universal screening for suicide risk in primary care settings is a promising avenue for identifying at-risk youth and improving health outcomes (2,3); however, screenings identify varying levels of risk that necessitate a range of clinical responses.Evidence-based based guidelines for assessing and responding to suicide risk in pediatric settings exist (4), but primary care providers (PCPs) report significant barriers, including time pressures, lack of training, and limited resources for managing positive screens (5,6).When patients screen positive in primary care settings and providers do not have sufficient time or expertise to fully assess risk, youth with a wide range of risk may be referred to an emergency department (ED) out of an abundance of caution (7).
Although appropriate for those with imminent risk, ED environments may not be wellsuited for assessing and supporting the mental health needs of most patients with suicidality due to a focus on managing acute medical conditions and a lack of mental health professionals onsite.Youth and their families report experiencing long wait times and insufficient care (8)(9)(10) and may leave before being evaluated (11).Beyond suboptimal patient experience, responding to BH concerns in the ED setting is costly and inefficient.The average duration of BH encounters in the ED is significantly longer than other ED visits [nearly twice as likely to be over 4 h (12)].One study estimated the cost of caring for one BH patient to be $219 per hour, with little apparent value to the patient (8).Further, many youth who present to the ED with behavioral health (BH) concerns do not receive a referral to follow-up services (13).
Given the apparent mismatch between the goals and resources available in the ED and the needs of youth with less than imminent suicide risk, accurately diverting lower-risk patients from the ED, while assuring an appropriate level of care, is a meaningful goal.In the case of primary care-based suicide risk screening, integrated behavioral health (IBH) services that incorporate mental health professionals into the risk assessment process have been proposed as a method to accomplish this goal (7,14).One promising study found that 93% of patients who screened positive for suicide risk were diverted from the ED following a consultation with a BH specialist, resulting in a considerable estimated cost-savings (7); however, that study was limited by lack of a comparison group and use of a depression screener with suboptimal sensitivity for detecting suicide risk (15).Further, that study focused only on ED visits that occurred the same day as suicide risk screening, so subsequent utilization of the ED and other healthcare, including follow-up mental health supports, was unknown.Understanding the impact of IBH on healthcare besides ED visits is important, because while all patients who screen positive for suicide risk do not require emergency care, they may benefit from ongoing monitoring or intervention.Continuity of care is a core function of primary care (16), as well as an important aspect of suicide prevention (17), so ongoing contact with at-risk patients is desirable.
To address this gap in the literature, we conducted a retrospective analysis of youth patients who screened positive for suicide risk in pediatric primary care, comparing those who received a same-day IBH consultation with those who did not.In addition to same-day ED visits, we assessed outcomes from the subsequent 31 days, including ED visits, IBH contacts, and primary care utilization.We hypothesized that youth who received an IBH consultation the same day as screening would be less likely to be admitted to the ED, but more likely to contact IBH services and to utilize primary care in the following 31 days.

Method Participants
Participants were patients ages 10-18 years who were screened for suicide risk in two academic medical centeraffiliated pediatric primary care practices in the Portland, Oregon metropolitan area.One clinic was hospital-based, and the other was community-based.Screening data from December 2019-July 2022 were collected via a retrospective chart review and extracted from the electronic health record using Epic Data Warehouse.Before this period, suicide screening results were not entered into the electronic health record in a manner that could be reliably extracted.

Setting and integration model
Both clinics were staffed by attending and resident physicians.IBH staffing consisted of licensed clinical social workers, clinical social work associates, psychologist postdoctoral fellows, psychology interns, and psychiatry consultants.Exact levels of IBH staffing and availability fluctuated, and in-person consultation was limited during portions of 2020 and 2021 due to COVID-19 precautions.The integration model at both clinics approximated Level 5 of the Substance Abuse and Mental Health Services Administration-Health Resources and Services Administration Standard Framework for Levels of Integrated Healthcare (18), with clinical and care management services provided both concurrent with medical care, as well as separate from medical care in a "colocated" fashion.Telehealth IBH services were available starting in April 2020, with IBH staff available for both "on demand" and scheduled consultations with patients.
Global screening for suicide risk at both clinics using the Ask about Suicide Questions (ASQ) tool was first implemented in February 2019, with a goal of screening every patient aged 10 years or older at every medical visit.The approach to screening and secondary assessment was consistent with the Zero Suicide Model (19) and informed by established recommendations for managing suicide risk in pediatric settings (4, 20).The framework for assessing and responding to risk was meant to provide a flexible model with a range of outcomes that could be followed by both medical and IBH providers, ideally in a team-based fashion.Social workers were considered first-call for consulting on suicide risk with other members of the IBH team available to assist as needed.Physicians were encouraged to facilitate IBH involvement

Measures Demographics
Demographic data included age, race, ethnicity, biological sex assigned at birth, and type of insurance.We also attempted to collect gender identity for all participants, but it was available in < 10% of cases.Insurance was categorized as either commercial or public and was the most relevant proxy available for socioeconomic status.

Suicide risk screening
The Ask Suicide-Screening Questions (ASQ) tool is a brief suicide risk screening instrument designed for use in medical settings with strong sensitivity and specificity for youth aged 10-21 years (21).The ASQ consists of 4 initial yes/no items.Any score ≥ 1 is considered a positive screen.In the event of a positive screen, a fifth item ("Are you having thoughts of killing yourself right now?") assesses acute ideation (acute positive screen).The ASQ has been widely used in previous studies and is a reliable and valid measure of suicide risk in pediatric settings (22).

Depressed mood
The Patient Health Questionnaire-9 (PHQ-9) is a commonly used self-report screening instrument for symptoms of depression (23).The PHQ-9 consists of nine items that correspond to the nine diagnostic criteria for major depressive disorder, and the language has been slightly adapted for use in pediatric settings.The PHQ-9 and its adaptation for adolescents possess strong psychometric properties and has demonstrated utility in pediatric settings (23)(24)(25)(26).The adolescent version (25) was used in the current study, and was intended to be administered at the same visits as ASQ.We were unable to analyze responses to individual PHQ-9 items, as only total scores were archived in the medical record.

Healthcare utilization
We recorded all ED visits, IBH encounters, and primary care physician (PCP) encounters that occurred within 31 days of an ASQ screening.Encounters that occurred the same date as ASQ screening were deemed "same-day, " and subsequent utilizations were deemed "31-day."For ED visits, we also categorized the reason for each admission into either non-suicide related or general mental health challenges (e.g., panic attack), or suiciderelated concerns.Two of the authors independently reviewed the admission documentation in the medical record and determined the reason for admission.Interrater agreement after initial coding was strong (Cohen's ℵ = 0.93).Discrepancies were discussed and resolved.Given the ultimate number of recorded visits, we dichotomized reason for admission into "suicide-related" or "other."

Mortality
We queried the data warehouse for any instances of mortality in the sample.We additionally cross-referenced each patient with state-level surveillance data from the Oregon Health Authority to identify any deaths that occurred outside of the medical system.

Analytic plan
We used descriptive statistics to characterize the sample.We determined a set of binomial logistic regression analyses to test the study hypotheses and test for other relationships of interest.For the planned logistic regressions for which there were fewer than five events per condition, we described the results in lieu of reporting the inferential statistics due to potential for bias (27).The primary predictor variable was whether the patient  received an IBH consultation the same day as a positive screen on the ASQ (same-day IBH encounter).The primary outcome variables were same-day and 31-day ED visits.Additional outcome variables included same-month IBH utilization and same-month PCP utilization.We included a number of potential covariates, including age, biological sex assigned at birth, race/ethnicity, PHQ-9 score, and insurance type.To select covariates for the final regressions, we conducted a series of univariate regressions with each potential covariate as the sole predictor (28,29).Predictors with p < 0.2 were retained as covariates.When possible, chi-square analyses were conducted if the nature of the data precluded regression analysis.

Sample characteristics
Table 2 reports the characteristics for three samples: (1) all patients who were screened, (2) patients who screened positive (non-acute or acute) on the ASQ, and (3) patients who screened acute positive on the ASQ.Age in years at the time of the ASQ, biological sex assigned at birth, and insurance plan were available for all participants.Race and ethnicity information that was reported as "unknown" or "declined" was categorized as missing (rates reported in Table 2 for each sample).

Emergency department utilization
During the study period, 129 patients had 163 documented ED visits (mode = 1, range = 1-12), 18 of which were related to suicidality.Table 3 displays the results of binomial logistic regressions with same-day IBH predicting same-day ED visits for participants with ASQ ≥ 1 (acute and non-acute positive screens together).Among all positive ASQ screens, same-day IBH was significantly associated with increased odds for all same-day ED visits and suicide-related same-day ED visits (p < 0.05).We were not able to run logistic regressions for the acute positive sample or with same-month ED visits as the outcome for the ASQ ≥ 1 sample due to insufficient events per condition.Table 4 outlines descriptive breakdowns of ED visits by ASQ result and receipt of IBH consultation.In all cases, a larger percentage of patients who had had same-day IBH contact had subsequent ED visits, but statistical significance could not be tested in most cases (although whether the differences are statistically significant is unknown in most cases).Among the 56 patients with an acute positive ASQ who did not receive an IBH consultation, one (2%) was admitted to the ED for suicidality in the following 31 days.Among the 31 who did receive a same-day IBH consultation, none were admitted to the ED for suicidality in the following 31 days.

Integrated behavioral health utilization
Among the 1,467 positive/acute ASQ screens, 327 (22%) received at least one additional contact with IBH within the next 31 days.Table 5 displays the results of a binomial regression with same-day IBH predicting 31-day IBH utilization for patients with positive ASQ screenings.Same-day IBH significantly and positively predicted subsequent 31-day IBH contacts, as did higher PHQ-9 score and race/ethnicity (racial and/or ethnic minority-identifying more likely to have a 31-day IBH visit).

Medical primary care utilization
Among patients who screened positive on the ASQ, 358 (24%) had a contact with a PCP within the next 31 days.Table 6 displays the results of a binomial regression with same-day IBH predicting 31-day PCP utilization for patients with positive ASQ screenings.Same-day IBH positively predicted at least one visit with the PCP in the following 31 days, as did race/ethnicity (racial and/or ethnicminority identifying patients were more likely to have a 31-day PCP visit).

Mortality
There was a single recorded death in the sample due to nonsuicide causes.

Discussion
Integration of behavioral health services into pediatric primary care has grown substantially in recent years in recognition of its promise for meeting the increasing behavioral health needs of the population (30, 31).We sought to assess how IBH consultations in the context positive suicide screenings in pediatric primary care relate to subsequent use of services to address suicidality, including ED visits, follow-up IBH, and primary care.Nearly onein-five patients screened non-acute or acute positive in our sample.Despite the high incidence of suicidality, less than 1% of screenings resulted in an ED visit and no patients died from suicide.These findings counter common concerns from providers and families that suicide screening may be iatrogenic or lead to marked increases in ED utilization (32)(33)(34).This study replicated a previous finding that over 90% of screen-positive youth who received a same-day IBH consult were diverted from the ED on both the day of their positive screen and during the subsequent month (7).However, we found that youth with positive screens who did not receive sameday IBH were also diverted from the ED at high rates.Indeed, contrary to our hypothesis, youth who received same-day IBH   consultation after a positive suicide screen were more likely to be admitted to the ED that same day than their peers who did not receive an IBH consultation.These results should be interpreted in the context of the broader roll-out of ASQ screening and systematic risk determination in the clinics under study.In contrast to Mancini and colleagues (7), who compared rates of IBH-associated ED diversion to a pre-IBH norm of referring all youth with positive screens to the ED, medical providers in our study did not refer all positive screens to the ED as a matter of course.Consistent with the U.S. National Strategy for Suicide Prevention (35), medical personnel in our clinic were trained in a secondary assessment process as part of implementing the ASQ.Beyond direct training, the presence of IBH in primary care is associated with PCP's increased confidence in managing mental health issues independently.Further, IBH-to-PCP "curbside" consultations may have occurred for some of the patients who did not receive face-to-face IBH consultations (36).We believe it is likely that medical providers prioritized patients they perceived as at higher-risk IBH consults, which is supported by the positive relationship between PHQ-9 scores and receipt of IBH consults (37).
The results suggest a unique value-add of IBH may be linking patients with suicide risk to short-term follow-up services.Patients in this study were more likely to engage with IBH in the month following their positive screen when they received a same-day IBH consultation.This finding is consistent with a recent study of same-day IBH responses to an array of behavioral health concerns in adolescent primary care, which found that patients who received a same-day consult were more likely to return for follow-up behavioral health in that clinic (38).Further, these results are consistent with the theoretical framework that IBH can address inequalities in care, reduce stigma, build rapport, and increase engagement (39-41).The relatively quick connection to follow-up in our study is consistent with 2012 Office of the Surgeon General's National Strategy for Suicide Prevention objective to provide timely access to care (35), particularly in the context of significant waitlists for longer-term care in the community at the time of the study, which has only increased in the post-pandemic surge in demand for behavioral health resources (42).
Patients who received a same-day IBH consultation were also more likely to contact a PCP in the subsequent month, even when controlling for depression severity.To our knowledge, the relationship between IBH and primary care utilization in the context of suicide risk has not previously been examined in pediatric populations, but other research has demonstrated better short-term attendance at medical appointments following sameday IBH consultations (i.e., "warm handoffs") more generally (43).While longer-term reduction in utilization of unnecessary or inappropriate care would be a positive outcome of IBH, in the context of our at-risk sample, short-term increases in utilization may represent an appropriate level of care.Follow-up with a PCP in the month after a positive suicide screen allows for additional caring contacts (44), updates to safety plans, identification of changes in risk, and management of psychiatric medications or medical comorbidities, which are more likely among youth with suicidal ideation and attempts (45).Previous research suggests that PCPs' follow-up care for suicide risk is inconsistent and clinical guidelines are needed (45).
Importantly, we found that the relationship between sameday IBH and subsequent-month IBH and PCP follow-up was stronger for patients who self-identified as racial and/or ethnic minorities.While the reasons for this relationship are unclear, it aligns with the aim of IBH to increase accessibility and quality of care for historically underserved youth (40).We dichotomized race and ethnicity given the low incidence of our outcomes of interest.However, we acknowledge there is great variation in the social and material experiences of individuals from different identity groups that warrant exploration.For example, suicide rates among Black youth increased over the past 20 years faster than any other group, such that they are twice as likely to by suicide (46,47).Our results are consistent with other findings indicating that higher-integration IBH models may hold particular benefits for people from traditionally under-served groups (48).Future research should explore possible explanations of this dynamic, including familiarity with the primary care setting, reduction of stigma, provision of culturally responsive care, and logistical accessibility.
This study highlights some of the challenges of IBH implementation and areas for refinement of this model.Just over one third of acute-positive screens were followed by a same-day face-to-face IBH consult, considerably lower than expected even considering the apparent impact of COVID-19 social distancing restrictions.For comparison, Pereira and colleagues reported sameday consultations with 69% of acute-positive patients (14), whereas Mancini and colleagues reported 100% (7).We observed high variation in the proportion of acute-positive screens that received IBH consult across time, with the lowest proportions generally following the onset of the COVID-19 pandemic.Disruptions to medical systems during the peak of the pandemic may have impacted successful IBH integration at multiple levels (e.g., clinic distancing and space restrictions, reluctance to engage the ED) that took time to remediate.Established methods for increasing warm handoffs may be useful in such circumstances (49).Further study of the implementation process and quality improvement efforts to increase the rate of contact with IBH for youth with positive suicide screens is merited.

Limitations and future directions
This study possesses notable limitations inherent to retrospective research.Group membership on the primary independent variable, receipt of same-day IBH consult, was non-random and subject to a number of potential confounds.
Barriers to same-day IBH consultations were not formally assessed but may be attributable to limitations in IBH provider availability, high medical providers self-competence, or other unknown factors, such as existing mental health care.Further, it remains unclear whether and how the risk assessment and response process varied across IBH and medical providers.Future research should examine the specific content of suicide-risk responses to determine whether particular strategies impact outcomes.Data were limited to one healthcare system in large metro area under study.Future studies should compare clinics with IBH services to those without, to assess the indirect impact of IBH services on PCP behavior and subsequent ED admissions.Additionally, it is possible that patients presented to another ED or utilized other healthcare in the month following screening.When considering generalizability, it is important to note that the clinics under study are part of an academic medical center in a metropolitan area.Future work should explore how IBH may affect suicide screening and follow-up in clinics in non-academic settings and rural areas.

FIGURE
FIGURENumber of positive acute suicide risk screenings and receipt of same-day integrated behavioral health (IBH) consultations during the study period.

TABLE Participant -
level characteristics.
TABLE Regression of same-day emergency department admission on same-day integrated behavioral health encounters for patients screening positive for suicide risk.
TABLE Emergency department visits by suiciding screening result and receipt of same-day integrated behavioral health consultation.
TABLE Regression of integrated behavioral health encounters within days of positive suicide risk screening on integrated behavioral health encounters the same day as screening.