Implementation of a fully virtual enterprise-wide clinical evidence-based suicide prevention program in the U. S. Department of Veterans Affairs: the suicide prevention 2.0 clinical telehealth initiative

Introduction: Veteran death by suicide is a complex issue made up of many factors. Despite the high need for mental health treatment, and treatments that specifically target suicide, evidence-based psychotherapies (EBPs) are difficult to access, even more so in rural areas. In concordance with the 2018 National Strategy for Preventing Veteran Suicide, VA suicide prevention leadership developed Suicide Prevention 2.0 (SP 2.0) to implement a public health model that includes community-based prevention strategies and improves clinical interventions within VA. The Suicide Prevention 2.0 Clinical Telehealth program was implemented in each of VA’s 18 regional Clinical Resource Hubs and expanded clinical intervention strategies within VA by implementing four EBPs for Suicide Prevention (EBP-SP) via telehealth: the Safety Planning Intervention, Problem-Solving Therapy for Suicide Prevention, Cognitive Behavioral Therapy for Suicide Prevention, and Dialectical Behavior Therapy.

Methods: A wide variety of implementation strategies were used (e.g., access new funding, training, consultation, create new clinical teams). The primary inclusion criterion for veteran referral to SP 2.0 Clinical Telehealth is a recent history of suicidal self-directed violence. Implementation was guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework and RE-AIM was used as an evaluation framework.

Results: By April 2023, SP 2.0 Clinical Telehealth services were available in all 18 regions and in 139 of 139 (100%) VA health care systems in the U.S. By the end of September 2024, the program had hired 137 therapists and retained 78.10% in their role, and 100% were trained in two or more EBP-SPs. By the end of September 2024, the program received 23,628 referrals nationwide. Increasing referral rates year over year suggests ongoing sustained reach.

Discussion: SP 2.0 Clinical Telehealth represents the first and only enterprise-wide fully virtual evidence-based treatment program for veterans with a recent history of suicidal self-directed violence. The program’s implementation was successful in reaching all VISNs and all VA health care systems in the U.S. The SP 2.0 Clinical Telehealth program can be used as a model for other large health care systems looking to improve provision of evidence-based interventions for suicide prevention.

Introduction

Suicide is a public health crisis

Suicide is an urgent public health crisis, with rates on the rise since 2000 (1). Suicide is an even greater issue for veterans. In 2022, the age-adjusted suicide rate for male and female veterans was 44% and 92% greater than that of non-veteran male and female adults respectively (2). In 2022, suicide was the 12th-leading cause of death for veterans overall and the second-leading cause of death among veterans age 45 and younger (3). That same year, 6,407 veterans died by suicide (2). Veteran death by suicide is a complex issue made up of many factors, and this tragic loss of life cannot be solved by a single intervention or initiative (4). Mental health and substance use disorder diagnoses put veterans at an increased risk for suicide (5), along with other societal issues such as housing insecurity and legal system involvement (6, 7).

Difficulty getting treatment for mental health and suicide

Despite the high need for mental health treatment, and treatments that specifically target suicide, evidence-based psychotherapies (EBPs) are difficult to access. In general, it is well documented that even though EBPs exist, most therapists do not receive training in or use EBPs in routine practice (8–11). Access is even more limited for those who live in rural and/or medically underserved areas (12, 13).

The critical importance of expanding access to EBPs

To address these issues for veterans, in 2006 the U.S. Department of Veterans Affairs (VA) initiated an organization-wide dissemination and implementation effort to train therapists in EBPs for posttraumatic stress disorder (PTSD) (14). In 2008, this was followed by the development of an Evidence-Based Psychotherapy Program to disseminate and train therapists in additional EBPs nationally (14, 15). As such, the EBP Program has a longstanding history of successful competency-based training in EBPs, which involves multi-component didactic and experiential training followed by structured case consultation (15). This model of training in EBPs has demonstrated significant, positive therapist training outcomes, including increased clinical competencies (16–18), enhanced self-efficacy (19), and improved knowledge and attitudes (20). VA EBP program evaluation results also indicate that veterans served by EBP-trained therapists demonstrate symptom reductions with effect sizes in the medium-to-large or large range among primary patient outcomes, as well as significant improvements in quality of life (16, 18–23). VA has supported national dissemination of EBP training in interventions that address PTSD, major depressive disorder, substance use disorders, insomnia, chronic pain, serious mental illness, motivation and engagement, and relationship distress. Since 2007, more than 15,000 unique therapists have been trained to competency in one or more EBPs via national training and dissemination efforts (24).

As an early adopter of system-wide implementation methods, VA’s traditional EBP training program model has demonstrated effectiveness in achieving therapist competency to deliver EBPs and improving known provider-level facilitators of implementation. Despite VA’s successes, barriers to sustained adoption and implementation of the interventions post-training include staff turnover, scheduling difficulties, patient volume, clinician workload (25), inconsistent institutional support, and lack of an EBP-focused clinical mission (26). Of note, these barriers are not unique to VA and are common across settings implementing EBPs (27–29). Because of the significant variability in these factors across VA medical centers, ensuring national access to an EBP with the VA’s existing dissemination and training models has been a challenge, resulting in substantial variability in rates of veteran reach and therapist adoption between VA facilities and clinics (25, 30). As a learning healthcare system, VA has used a variety of strategies to improve implementation outcomes, including EBP Coordinators to serve as internal facilitators at VA facilities, national policy requiring EBP availability, and performance metrics and monitoring systems to support EBP tracking and accountability (15). Rates of EBP implementation are relatively low in many regions and facilities across VA (31). EBP adoption and reach tend to be highest in specialty mental health settings where therapists experience fewer competing time demands, have frequent access to veterans with the target condition(s) for the EBP, and have consistent leadership and structural supports to deliver EBPs (32). VA continues to use this information to improve upon methods by which to implement EBPs.

Implementation science can aid program implementation

Implementation science is the “scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services.” (33) As described above, there are challenges to implementing EBPs into routine care. Establishing the effectiveness of an intervention does not guarantee its uptake into routine care (34). Within the field of implementation science there are a variety of theories, models, and frameworks to guide both implementation planning and evaluation (35). The field has also specified implementation strategies to “enhance the adoption, implementation, and sustainability of a clinical program or practice.” (36) Use of a theory, model, or framework and implementation strategies that target barriers and facilitators to implementation can enhance the likelihood that an intervention is successfully implemented (37). VA has a long history of both supporting and benefiting from implementation research (38–42), as operational leaders within VA frequently partner with implementation science experts.

Current paper

In 2018, the VA released a new national strategy for preventing veteran suicide that called for both enhancing care for individuals at risk within the VA health care system and adding a public health approach to reach all veterans, both those inside and outside of the VA system (43). To move forward a full public health approach to veteran suicide prevention, VA suicide prevention leadership developed Suicide Prevention 2.0 (SP 2.0) to implement community-based prevention strategies, embarking to reach all veterans, while simultaneously ensuring expansion of evidence-based clinical strategies (4, 44). This work built on VA’s experience and knowledge of implementing EBPs and used implementation science strategies to ensure program implementation success. The goal of the present manuscript is to describe the implementation of the Suicide Prevention 2.0 Clinical Telehealth program and report on initial reach, adoption, and maintenance outcomes.

Methods

Setting

The Veterans Healthcare Administration (VHA) is the arm of VA that provides health care to veterans; it is the largest integrated health care system in the U.S. VHA is comprised of 18 regional networks, referred to as Veteran Integrated Service Networks (VISNs). Care is provided at 1,380 health care facilities that include 170 medical centers and 1,193 outpatient sites. Over 9 million veterans are enrolled in VHA care. VHA facilities offer a range of health and mental health services (https://www.va.gov/health/aboutvha.asp). Each VISN is home to a VISN Clinical Resource Hub (CRH), which provides a telehealth model of care (45). Every CRH offers virtual primary care and mental health services and may offer specialty care services based on the needs of their VISN (e.g., dermatology, sleep medicine, etc.). Methods and data ascertainment for analyses were considered non-research and did not require institutional review board (IRB) approval per the Department of Veterans Affairs Office of Research and Development Program Guide 1200.21. In addition, the VA Connecticut Healthcare System Research Department designated this as a non-research quality improvement project and confirmed IRB review was not required.

Implementation framework

Implementation and evaluation were guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) (46, 47) framework. The EPIS framework was developed based on implementation science literature in public sector social and allied health service systems. EPIS defines four phases that align with the implementation process:

1. Exploration (awareness of health need to be addressed, identification of practices to be implemented),

2. Preparation (selection of practice(s) to implement, assessment of barriers and facilitators to implementation, development of an implementation plan, selection of implementation strategies),

3. Implementation (practice use is initiated, monitoring of implementation process, adjusting strategies as needed) and,

4. Sustainment (structures, processes, and supports are ongoing so the practice continues being offered to realize an impact).

In addition to phases of the implementation process, EPIS defines constructs that impact implementation, including outer context, inner context, innovation factors (e.g., characteristics of the EBP), and bridging factors (i.e., interconnections between these constructs) (47). More detail can be found at www.episframework.com. See Figure 1 for activities that occurred during each phase of implementation and Figure 2 for an overall timeline of implementation activities that also includes when different outcomes will become available.

Figure 1

Figure 1. Select activities that occurred during each EPIS phase.

Figure 2

Figure 2. Overall timeline of implementation activities.

Context

VA suicide prevention leaders engaged stakeholders across the VA system to develop SP 2.0. Stakeholders included medical center directors, VISN Network Directors, VISN Chief Mental Health Officers (CMHOs), CRH leadership, and different program offices like the Office of Primary Care. Initial engagement with stakeholders also included briefings with external partners such as Veteran Service Organizations, Congressional staff, and the White House to respond to their questions and obtain their input on implementation of a full public health approach to suicide prevention. This stakeholder engagement (48) was needed for both program design and the culture shift needed at senior levels of leadership to agree to such an innovative approach to suicide prevention that included engaging community partners to reach veterans not enrolled in VA health care.

As the community-based prevention portion of SP 2.0 moved through the initial approval processes, VA suicide prevention leaders also engaged implementation scientists and suicide prevention national research experts in identifying the most promising, evidence-based clinical approaches to include in the SP 2.0 model. The release of the 2019 revised clinical practice guideline (CPG) for assessment and management of patients at risk for suicide occurred during the development of SP 2.0, which served as the foundation of evidence to inform the decision making on which clinical treatments to promote in this new initiative (49). Given the limited availability of EBPs (15, 50) and the release of a new guideline, SP 2.0 provided an opportunity to expand clinical intervention strategies within VA by identifying EBPs with the best outcomes to expand nationwide access.

Those EBPs for Suicide Prevention (EBP-SP; described in detail below) included the Safety Planning Intervention (SPI), Problem-Solving Therapy for Suicide Prevention (PST-SP), Cognitive Behavioral Therapy for Suicide Prevention (CBT-SP), and Dialectical Behavior Therapy (DBT). However, at that time, there were limited VA therapists trained in these EBP-SPs and there was concern about how to ensure veteran access across the country to trained providers. Program development efforts addressed these barriers by obtaining a financial commitment to fund therapists whose time would be fully dedicated to deployment of EBP-SPs and to develop EBP-SP training programs. In addition, leadership decided to deliver these EBP-SPs using a telehealth delivery method to maximize veteran access across the nation. VA suicide prevention leaders met with the VA’s EBP program to learn from their experience in training VA therapists, which highlighted the importance of competency-based training in EBP. Collaborative work was also done with VA’s CRHs to establish a partnership such that SP 2.0 Clinical therapists and supervisors would work within existing VISN-based CRH infrastructure. Collaborations with CRH included creating a framework for patient flow, building clinics specifically for telehealth, and partnering with the Office of Connected Care (VA’s office that brings VA digital technology to veterans and health care professionals) for utilization of the “digital divide” consult to connect veterans to internet capable devices, therefore bridging the digital divide. CRH partnership also supported and facilitated relationship building with local facility leaders and stakeholders. SP 2.0 Clinical established standards for therapists, particularly to protect their time for focused delivery of EBP-SPs, a lesson learned from prior EBP implementation. VA suicide prevention leadership also engaged additional experts in suicide prevention and program evaluation to plan the program, its implementation, and its evaluation.

VA suicide prevention leadership worked through different levels of governance, resolved the financial details with VHA finance, and obtained final approval for SP 2.0 from the Under Secretary for Health. After approval was obtained, suicide prevention leadership created workgroups that developed implementation and evaluation plans. Two work groups were established in March 2020: 1) the SP 2.0 Clinically-Based Interventions Work Group and 2) the Evaluation and Implementation Work Group. The work groups were tasked with developing logic models, identifying project outcomes to assess, creating an implementation and data collection plan, and developing an implementation checklist.

To inform SP 2.0 Clinical Telehealth Initiative program development, a telehealth CBT-SP program pilot was conducted in two VISNs from 2019 to 2021. The pilot trained 10 therapists as CBT-SP experts and developed and implemented substantial programmatic infrastructure, including: a telehealth CBT-SP therapy manual with incorporated clinical measures (e.g., session check-in questions regarding suicidal ideation and behaviors, standardized measures of depression and suicide), a patient therapy workbook, referral and care documentation templates, etc. The pilot accepted new veteran patients between February 2019 and March 2021, at which time the program was subsumed under the SP 2.0 Clinical Telehealth initiative. There were 436 veterans referred to the pilot, 223 (51.14%) of whom completed at least one CBT-SP therapy session. Of those 223 who initiated CBT-SP therapy, 140 (62.78%) completed a full treatment course (12–14 sessions). Manual electronic health record (EHR) abstraction and data analysis found no significant demographic or mental health diagnostic differences between those who refused CBT-SP, those who left treatment early, and those who received a full course of treatment. Veterans who received at least 1 session of CBT-SP reported a significant improvement in depression as measured by the Patient Health Questionnaire (PHQ-9) (51), with those receiving a full course of CBT-SP reporting a larger reduction in symptoms. Across all levels of participation, there were significant reductions in self-reported suicidal ideation and suicidal behavior. Findings generally supported the feasibility and acceptability of this telehealth program model. Ultimately, this pilot provided proof of concept and information regarding implementation strategies, logistics, potential barriers and facilitators, and best practices for a national initiative. Of note, because the CBT-SP pilot and SP 2.0 Clinical Telehealth initiative were based on a virtual model, program implementation was well-suited to navigate the uncertainties of the COVID-19 pandemic that began midway through the CBT-SP pilot.

The SP 2.0 Clinical Telehealth Program was conceptualized to maximize facilitators of EBP implementation while reducing implementation barriers common within the traditional mental health continuum of care in VHA. See Table 1 for challenges of the traditional facility-based EBP training program model and how the SP 2.0 Clinical Telehealth initiative was designed to address those challenges.

Table 1

Table 1. Challenges of the traditional facility-based EBP training program model and how those challenges are addressed in the SP 2.0 Clinical Telehealth model.

Select clinical interventions

As described above, SP 2.0 Clinical Telehealth implemented four EBP-SPs, all recommended by the 2019 CPG (49). The Safety Planning Intervention (SPI) is a one-time 45 to 60 minute evidence-based clinical intervention designed to mitigate suicide risk by providing an individual experiencing suicidality with a written, personalized safety plan that is to be used in the event of a suicidal crisis (54). Stanley et al. (55) administered SPI in emergency departments to over 1,600 patients who were experiencing a suicidal crisis but not requiring hospitalization. They found that completing the SPI with follow up phone contact was associated with 45% fewer suicidal behaviors over 6 months (55). Of note, the CPG recommendation was for “completing a crisis response plan” and the guideline notes that the crisis response plan and the SPI share similar components. The evidence was also reviewed for SPI in the guideline. SPI was implemented versus the crisis response plan given the existing training program and health care system infrastructure (e.g., safety plan form template in the EHR) already available for safety planning in the VA.

Problem-Solving Therapy for Suicide Prevention (PST-SP) is a cognitive-behavioral treatment aimed at preventing or mitigating suicidal crises by improving an individual’s ability to cope with stressful life experiences and problems that contribute to suicidal thoughts and behaviors (56). The manualized treatment protocol is delivered in 6–12 sessions in an individual format. The overarching goal of PST-SP is to reduce suicide risk by teaching adaptive problem-solving strategies and emotion regulation skills and does so by employing an emotion-centered PST approach to address four common barriers to effective problem-solving: cognitive overload, feelings of hopelessness, intense negative emotions, and ineffective problem-solving. There is a strong evidence base regarding the effectiveness of PST for reducing suicide and death ideation and for a variety of other physical, cognitive, and mental health problems across multiple settings, age groups, and diagnoses (57–65). Similar to protocols employed with veterans PST-SP is recommended for individuals who would benefit from a practical, brief, skills-based intervention to facilitate coping with life stressors and problems associated with their suicide risk.

Cognitive Behavioral Therapy for Suicide Prevention (CBT-SP) is a treatment that uses cognitive and behavioral strategies to facilitate a reduction in the likelihood of future suicidal behavior or attempts (66). CBT-SP is a manualized treatment protocol consisting of 12 to 14 sessions, delivered in an individual format. The objectives of CBT-SP are to 1) build a sense of hope, 2) increase awareness of reasons for living, 3) develop alternative ways of thinking and behaving via skill-building, imagery, and rehearsal techniques, and 4) increase coping skills and self-efficacy to manage crises. CBT-SP has been found to significantly reduce risk of suicidal behaviors for up to 24-months following treatment in community and military samples (67–69).

Dialectical Behavior Therapy (DBT) (70, 71) is an evidence-based cognitive behavioral psychotherapy for treating emotional dysregulation and behavioral dyscontrol, including suicidal and self-harm behavior. DBT’s efficacy and effectiveness have been demonstrated in more than 50 randomized controlled trials (72–76), two of which were conducted with veterans (77, 78). DBT is recommended for individuals with repeated suicidal behavior and Borderline Personality Disorder (BPD). DBT is completed in six to twelve months and includes weekly individual therapy, weekly group skills training, weekly therapist consultation team, and as needed phone coaching. The skills taught in DBT include mindfulness, emotion regulation, interpersonal effectiveness, and distress tolerance. Of note, prior to the COVID-19 pandemic, it was not common practice to provide DBT via telehealth. During the pandemic, most DBT teams in the VA transitioned to telehealth. A survey was conducted of DBT teams across VA to allow their experience to inform implementation of DBT in SP 2.0 Clinical Telehealth (79). DBT was implemented first as a telehealth pilot to allow for iterative changes before expanding to other VISNs.

Participants

Each facility in a VISN can refer eligible, consenting clinically appropriate veterans to their respective VISN CRH. The primary inclusion criterion for patient referral to SP 2.0 Clinical Telehealth is a recent history of suicidal self-directed violence (SSDV), including a suicide attempt or preparatory behavior in the past 12 months. Preparatory behavior may include any acts or preparation towards engaging in SSDV, but before potential for injury has begun (i.e., action beyond the verbalization of a thought, such as buying a gun, collecting pills, writing a suicide note). For DBT, veterans must also have a documented diagnosis of BPD. The program aligned inclusion criteria as closely as reasonable to the populations for which each EBP-SP was recommended in the CPG (e.g., DBT was recommended for those with BPD and SSDV). Of note, an updated CPG was released in May 2024 (80). During SP 2.0 Clinical Telehealth intake, therapists use a shared decision making approach to collaboratively work with veterans to determine the most appropriate treatment.

Therapists participating in SP 2.0 Clinical Telehealth are licensed independent providers hired by the CRHs to provide EBP-SPs. They primarily include psychologists, social workers, and licensed counselors. Historical SSDV and geographical location trend data informed the staffing model for each VISN CRH location. Based on that data, the virtual therapist team sizes range from 3–14 in each VISN, in addition to size-appropriate teams of administrative support (e.g., scheduling assistants).

Implementation strategies

Given the scope of implementing a national suicide prevention clinical telehealth initiative, a variety of implementation strategies were used, some of which were described in Context. See Table 2 for a list of implementation strategies used, operational definition, and the targets of each strategy.

Table 2

Table 2. Implementation strategy name, operational definition, and targets of each strategy using Powell et al. (81) naming convention listed in alphabetical order.

Measures

We used RE-AIM (84) as our evaluation framework. RE-AIM examines Reach into the target population, Effectiveness of the intervention, Adoption by the setting, Implementation consistency or fidelity, and Maintenance over time. Incorporating stakeholder input, we identified key measures; see Table 3 for how we defined each RE-AIM dimension. Of note, this paper will not report results for all measures, and will focus on preliminary reach, adoption, and maintenance outcomes. Future reports will include implementation fidelity and effectiveness outcomes.

Table 3

Table 3. RE-AIM dimensions measured with their definition and data source.

Two dashboards collected the data of interest: (1) the EBP Training Data Portal for elements related to tracking therapist training requirements (e.g., adherence and competency to EBP treatment fidelity) and (2) the SP 2.0 Clinical Telehealth Implementation dashboard, which was specifically built to monitor and evaluate the SP 2.0 Clinical Telehealth Program. Monitoring and evaluation were conducted via data collection and visualization tools created to report outcomes consistent with the RE-AIM framework.

The EBP Training Data Portal is an internal web-based tool developed to support all VA EBP training programs and their training participants, consultants, and program administration staff. The main function of the portal is to provide a consistent way to enter and track training cases (e.g., training case demographics, sessions, measurement-based care assessments), track therapist progress toward meeting training requirements, and facilitate discussion for consultation calls. With informed consent, veteran patients may choose to allow their therapy sessions to be audio-recorded as part of their therapist’s training process. Recordings are stored and reviewed in the portal by consultants for adherence and competency evaluation before being deleted. With respect to the RE-AIM framework, the EBP Training Portal focuses most specifically on the elements of Implementation Fidelity.

The SP 2.0 Clinical Telehealth Implementation Dashboard was designed to monitor enterprise-wide outcomes consistent with elements of the RE-AIM framework. For reach, the dashboard monitors the number of veterans referred (and from what facility, what region in the country, and from what kind of treatment setting) as well as the program and treatment engagement as the veteran journeys through the program. For adoption, the dashboard monitors hired staff in alignment with the special purpose funding model and the volume of EBP-SP note templates in the EHR in alignment with the treatment provided. For implementation fidelity, the dashboard monitors treatment completion across the EBP-SPs and examines the intake assessment appointment to notice trends in the safety planning addressed during the first appointment in the program.

The dashboard has been used to provide implementation snapshots on a regular basis; these snapshots are graphical depictions of progress towards the current implementation goal (e.g., hiring, staff trained, volume of referrals). There has been select major implementation foci per year and the snapshots have evolved to showcase the focus. At the beginning of program implementation, the focus was on staff hired and hired staff trained in EBP-SPs. Once the referral consult was accessible at the national level for providers to refer patients, the focus evolved to reach every health care system in the country. To demonstrate that access was available, and the referral process was working, the goal was for each of the 139 VA health care systems¹ in the U.S. to submit a referral for a patient that is received by a CRH team. Once that goal was achieved, the focus shifted to examining the volume of referrals and the volume of intake appointments (see program engagement in the table above). Finally, the current focus has transitioned to program and treatment engagement, looking beyond the first appointment to continuity of care within therapy protocols.

The implementation snapshots assessed progress on national priorities and served as a messaging strategy to the field. Snapshots presented data by VISN so that all stakeholders could see progress across and within VISNs. VISN leaders could use the data to inform how to promote SP 2.0 Clinical within their VISN. For example, once the snapshots included information about work settings making referrals that attended an intake appointment (e.g., primary care mental health integration, residential care), program managers could do localized targeted messaging to increase referrals.

A SP 2.0 Clinical facility-level performance metric was established in FY23 to monitor and incentivize every facility to refer eligible consenting clinically appropriate veterans to the program. This was also displayed on the dashboard. The metric consists of the percentage of patients with suicide behavior event days in the past four quarters that resulted in a referral to SP 2.0 Clinical Telehealth. A suicide behavior event day is defined as a unique day within a facility in which a patient reported one or more suicide behaviors, including preparatory behavior. This metric was highlighted by the Under Secretary for Health’s Priority to Action initiative (90), which set facility level goals for expected improvement in FY24.

Analysis

Summary statistics of the reach, adoption and associated maintenance measures under the RE-AIM framework were conducted to assess implementation using VA administrative data and analyzed with SAS 9.4. Unless otherwise noted, data from April 1, 2021 through September 30, 2024, representing three and a half years of implementation. Except for reach data (number of VISNs and health care systems, number of referrals) and therapist data (number hired, percent retained, and training completion data), administrative data were available from 134 (of 139) health care systems due to transition to a new EHR during the project period.

Results

Reach

By April 2023, SP 2.0 Clinical Telehealth services were available in 18 of 18 (100%) VISNs and in 139 of 139 (100%) health care systems in the U.S. By the end of September 2024, 23,632 patients were referred for care. See Figure 3 for the number of referrals and intakes each month and the totals per fiscal year (FY).

Figure 3

Figure 3. Number of SP 2.0 Clinical Telehealth referrals and intakes from April 2021 through September 2024. *Based on timing of program implementation start date, FY21 does not represent an entire fiscal year.

Regarding the SP 2.0 Clinical facility-level performance metric that examined the percentage of patients with suicide behavior event days that resulted in a referral to SP 2.0 Clinical Telehealth, the mean percentage of eligible patients receiving referrals showed an increase from 10.5% in the first quarter of FY23 to 21.7% in the fourth quarter of FY24 (see Figure 4).

Figure 4

Figure 4. Measures of dispersion (mean, median, interquartile range, minimum and maximum values, and outliers) of SP 2.0 Clinical Telehealth referral percentages by FY and quarter.

To examine program engagement, we reviewed referrals received through the end of September 2024. Of the 23,632 referrals placed, 12,701 (53.75%) resulted in a completed intake. Of the completed intakes, 11,960 (94.17%) received further intervention, defined as a safety plan either created, updated, or reviewed and/or attended one or more EBP-SP session.

Tracking of the provision of the full safety planning intervention began on July 1, 2023. Of the 6,318 intake appointments that occurred from July 1, 2023 to September 30, 2024, providers addressed safety plans in 5,815 (92.04%). This included reviewing an existing safety plan (4,471; 76.89%) with no changes, updating an existing safety plan (792; 13.62%), and completing a new safety plan (504; 8.67%). Among those who developed a new safety plan, 55.95% (282) were developed in the context of completion of the full SPI.

For treatment engagement, we examined the number of patients who attended an intake and completed at least one session of an EBP-SP psychotherapy. Of those who completed an intake (12,701), 8,758 (68.96%) completed at least one session. Among patients who received at least one session, 36.07% received PST-SP only, 48.80% received CBT-SP only, 4.22% received DBT only, and 10.90% received more than one type of treatment. Of those who initiated treatment, 4,222 (48.21%) completed a full protocol of treatment by the close of June 30, 2025, as denoted by their therapist in a discharge EHR note template. Among patients who completed treatment by the end of September, 2024 (8,363) received an average of 6.26 sessions of PST-SP (n=2,856) and 8.86 sessions of CBT-SP (n=3,747) (DBT data still pending), indicating that a significant percentage of patients reached at least the middle stages of treatment.

Adoption

As of September 30, 2024, the program hired 137 therapists and retained 107 (78.10%) in their role within SP 2.0. Of those 107 retained therapists, at the close of September 2024, 100% were trained in two or more EBP-SPs, 91.59% were trained in three or more EBP-SPs, and 42.99% were trained in all four EBP-SPs. We define trained as 1) completed all pre-requisites for a training, 2) completed workshop portion that includes didactics and experiential role plays, and 3) currently seeing cases and in weekly consultation or having a certificate of completion. Further, in FY24 (October 1, 2023-September 30, 2024), 32,946 (70.08%) of clinical appointments within the SP 2.0 Clinical Telehealth program (N = 47,012) included provision of EBP-SP psychotherapy (PST-SP, CBT-SP, DBT) based on use of standardized EHR note templates.

Maintenance

By the end of FY24 the program received 23,632 referrals nationwide. Increasing referral rates (see Figure 5) year over year (45.23% increase from FY22-23, 46.92% increase from FY23-24) suggests ongoing sustained reach (Maintenance). By the close of FY22, SP 2.0 Clinical Telehealth teams had reached 89.57% staffing of the number of recommended therapist positions. Even with the percent of approved staff positions increasing by 1.74% from FY22-24, staffing levels increased to 91.45% in FY24, demonstrating the program’s ability to manage turnover over time and sustained adoption.

Figure 5

Figure 5. Implementation foci over time.

Discussion

The SP 2.0 Clinical Telehealth program is the first and only national, enterprise-wide, fully virtual clinical infrastructure creating access for veterans to receive evidence-based suicide prevention clinical treatment. The program addressed challenges to VA’s traditional EBP training model, as well as known barriers to care for veterans, by providing care nationally via telehealth using the existing CRH structure. The program used a host of implementation strategies and engaged stakeholders at all levels of VA to put this portion of the national strategy for suicide prevention into practice.

The program’s implementation was successful in reaching all VISNs and all VA health care systems in the U.S. In the first three and a half years, over 23,000 patients were referred for care and a notable proportion of those referred were successfully evaluated for and engaged in treatment. Over half of those referred attended an intake appointment and 94.16% of individuals attending the intake received further intervention (safety plan addressed and/or EBP-SP). For patients who completed an intake, 68.96% engaged in treatment by completing at least one session of an EBP-SP psychotherapy and of those, 48.21% completed a full course of treatment.

Patients who completed PST-SP and CBT-SP reached the middle to late stages of treatment, receiving an average of 6.26 and 8.86 sessions respectively. Importantly, this “real-world” length of care is comparable to length of treatment received in several randomized controlled trials (RCT) for these treatments that demonstrate clinical improvement. For example, in a RCT of PST that compared in-person PST, PST via telehealth (tele-PST), and telephone support (control condition), PST was offered for 6 sessions. Results indicated that the impact of tele-PST on depression and disability outcomes were sustained significantly longer than in-person PST (91). They also found that tele-PST (but not in-person PST) was more effective than the control in reducing death/suicidal ideation (92). In one RCT of CBT-SP patients received an average of 8.92 (SD = 5.97) sessions. Those who received CBT-SP versus control had a significantly better outcomes (i.e., lower suicide reattempt rate, lower self-reported depression, and less hopelessness) (67). Therefore, results regarding number of EBP-SP psychotherapy sessions completed are promising. Evaluation is ongoing to determine the impact of that dose of therapy on patient outcomes in SP 2.0 Clinical Telehealth.

In terms of adoption, most SP 2.0 Clinical Telehealth therapists have been trained in three of the four EBP-SPs. Further, despite a slower DBT implementation to allow for iterative change to the training program based on telehealth pilot results, over 40% of therapists have been trained in all four EBP-SPs. Once trained, available data also confirms that therapists are dedicating their clinical time to provision of approved EBP-SPs as intended, with 70.08% of clinical appointments documented using EBP-SP EHR note templates.

There is evidence of sustained reach and adoption when examining the maintenance of implementation of the SP 2.0 Clinical Telehealth Initiative. Sustained reach was demonstrated with increasing referral rates over the years of program implementation. A notable contributor to this sustainment was the establishment of the SP 2.0 Clinical Telehealth facility-level performance metric, which demonstrated an increase in the percentage of potentially eligible patients referred nationally each quarter. Implementation of this metric allowed VA suicide prevention leadership to successfully monitor and measure an EBP-SP referral system (at the enterprise level) that held VA health care systems accountable (at the facility level) focusing on veteran care (at the patient level). Finally, the program reached 89.57% staffing, and that number slowly increased over the next two fiscal years. Despite some turnover, the program was able to sustain adoption and increase staffing.

Several factors supported VA’s success in this implementation. First, during program development, VA suicide prevention leadership engaged stakeholders at all levels of VA and VHA finance to support this program’s approval and funding. Second, the SP 2.0 Clinical Telehealth program hired over 100 psychotherapists with teleworking agreements and tours of duty that allowed for provision of care across multiple time zones. As federal employees working for VA, the Supremacy Clause 38 C.F.R § 17.419 (2025) and the Anywhere to Anywhere regulation 38 C.F.R. § 17 (2017) allows mental health providers licensed in one state to serve veterans in other states. Therefore, VA could cast a wide net to hire the most competitive applicants without limiting the search to geographic restrictions or requiring applicants to move. Third, once hired, all therapists were embedded into a CRH infrastructure, made up of regionally-based telehealth teams, allowing for nationwide veteran reach of this fully virtual model. Fourth, the therapists received gold standard, competency-based training in EBP-SPs based on recommendations from the 2019 CPG for the assessment and management of patients at risk for suicide. Training programs were piloted, and iterative changes were made to improve therapists’ experience and training outcomes.

Finally, the development and utilization of multiple dashboard reports allowed VA suicide prevention leadership to monitor and provide feedback on the reach, effectiveness, adoption, implementation, and maintenance of this program, driving continuous quality improvement. The dashboard reports also evolved to focus on program engagement and treatment engagement. This allowed for examination of referrals, intakes, percentage of veterans consenting to come back and engage in more EBP-SP and ultimately look at percentage of treatment completion.

Next steps

VA suicide prevention leadership continues its focus on program improvement. For example, EBP-SP training programs are assessing and adapting the training programs based on training participant progress and feedback and site visits offer the opportunity to identify other areas for improvement. The program was implemented based on recommendations from the 2019 CPG. A revised CPG was published in 2024, and the update included changes in methodology and focus that resulted in changes in the ratings of evidence-based practices since the 2019 version. Like with the updated CPG, VA suicide prevention leadership is continually assessing how the program can evolve in response to new research as it becomes available.

As the program continues and serves more veterans, additional data will become available. Regarding effectiveness, evaluation of patient outcomes (i.e., changes in symptoms as demonstrated by standardized measures) is ongoing. Evaluation of self-directed violence and suicide behaviors including death by suicide are planned. Each training program is evaluating the impact of their respective trainings on therapist adherence and on implementation fidelity.

Implications

Implementation of the SP 2.0 Clinical Telehealth program demonstrated how strategic use of implementation science strategies supported successful implementation of a new national approach to providing evidence-based practices for suicide prevention via telehealth to veterans across the U.S. These implementation science strategies included engaging leadership across multiple levels of the health care system, accessing new dedicated suicide prevention funding, and engaging implementation science experts and program evaluation partners from the onset. Telehealth was thoughtfully selected as a treatment modality to enhance reach to veterans, especially for those in rural areas (93). Telework was used for hiring to attract the most qualified applicants regardless of geographic region. The partnership with a telehealth infrastructure (CRH) facilitated enterprise-wide reach and hiring of therapists. Training programs were developed to provide appropriate training and support to therapists and their supervisory chain of command, so the entire team had the tools and skills to navigate implementation steps. This all resulted in improved access to evidence-based suicide prevention clinical care for veterans within VA. The SP 2.0 Clinical Telehealth program can be used as a model for other large health care systems looking to improve provision of evidence-based interventions for suicide prevention.

Conclusion

SP 2.0 Clinical Telehealth represents the first and only enterprise-wide fully virtual evidence-based treatment program for veterans with a recent history of suicidal self-directed violence. The model uses a fully virtual telehealth team-based approach, addressing access barriers of face-to-face care (e.g., transportation time and cost, childcare, limited clinic hours). Program funding is specific purpose funded to support the SP 2.0 Initiative, as outlined within the VHA Presidential budget. As such, therapists are fully dedicated to the program and have supervisory support to engage in EBP-SPs. This gives SP 2.0 Clinical Telehealth therapists the time and tools they need to provide EBP-SPs according to their respective protocols. Notably, perceived institutional support for providing EBP has been linked to less burnout and more job satisfaction for VA providers (24). Within two years, all VISNs and all VA health care systems in the U.S. had access to these services. In the first three and a half years, over 23,000 veterans were referred to care and over half completed an intake. Almost all of those who completed an intake had a safety plan completed, updated, or reviewed, in the first clinical contact. After intake, 68.96% completed at least one session of an EBP-SP psychotherapy and nearly half of those patients completed a full course of treatment. The program continues to evaluate outcomes and make iterative changes to improve suicide prevention intervention for veterans.

Data availability statement

The datasets presented in this article are not readily available because they include PHI from VA healthcare systems. Requests to access the datasets should be directed to the corresponding author.

Ethics statement

Ethical approval was not required for the study involving humans in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and the institutional requirements.

Author contributions

SL: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. JW: Conceptualization, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing. NB: Project administration, Writing – original draft, Writing – review & editing. MK: Writing – original draft, Writing – review & editing. SL: Writing – original draft, Writing – review & editing. RH: Conceptualization, Methodology, Writing – review & editing. LE: Data curation, Formal analysis, Writing – original draft, Writing – review & editing. LB: Investigation, Project administration, Writing – original draft, Writing – review & editing. SB: Writing – original draft, Writing – review & editing. WB: Project administration, Writing – original draft, Writing – review & editing. LK: Conceptualization, Funding acquisition, Project administration, Resources, Writing – original draft, Writing – review & editing. MM: Conceptualization, Funding acquisition, Resources, Writing – review & editing. JP: Formal analysis, Writing – original draft, Writing – review & editing. MS: Formal analysis, Investigation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declared that financial support was received for this work and/or its publication. This project and its evaluation were supported by funding from the VA Office of Suicide Prevention and a VA Quality Enhancement Research Initiative (QUERI) grant (QUE 20-026) awarded to the first author.

Acknowledgments

The authors would like to thank the following individuals who were involved in the successful implementation of Suicide Prevention 2.0 (SP 2.0) Clinical Telehealth: Catherine Rotolo, Andrew Moon, Nicole Stromberg, Matt Moore, Laurie Lamonde, Bruce Crow, Lisa Brenner, Chris Crowe, Charles Surber, Elizabeth Fassig, Bridget Matarazzo, Kendra Weaver, Kristine Day, Claire Collie, Corey Terhune, Christina Gerstel, Karen McGoff-Yost, Caryn DiLandro, Edgar Villarreal, Jesse Burgard, Lawrence Wahlberg, Jennifer George, Mark Ilgen, Jennifer Olson-Madden, Erin Goldman, Aaron Eagan, Matt Rogers, Kay Burnett, Stacy Swanson, Jessica Barber, Jeff Hoerle, Nancy Furst, Kim Ba, Jon Calvert, Diana M. Mendez, and Kiesha Ross. The authors are deeply grateful to the SP 2.0 Clinical Telehealth Therapists and Program Managers; Evidence Based Psychotherapies and Interventions for Suicide Prevention Subject Matter Experts, Trainers, and Consultants; and Veterans. The authors would like to thank Ciara Oliver and Sonia Singh for their support in preparing this manuscript. This work was authored as part of the Contributor’s official duties as an Employee of the United States Government and is therefore a work of the United States Government. In accordance with 17 U.S.C. 105, no copyright protection is available for such works under U.S. Law. The views expressed in this paper are those of the authors and do not necessarily reflect the position or policy of the United States Department of Veterans Affairs (VA), Veterans Health Administration (VHA), or the United States Government.

Conflict of interest

The authors declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author SL declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2025.1668417/full#supplementary-material

Footnotes

1. ^ VA’s 170 medical centers and 1,193 outpatient sites described in Setting are housed within 140 health care systems, 139 are within the U.S. and one is in Manilla, Philippines. VA is not allowed to provide telehealth internationally.

References

1. Centers for Disease Control and Prevention. Suicide Data and Statistics (2023). Available online at: https://www.cdc.gov/suicide/suicide-data-statistics.html (Accessed May 2, 2025).

Google Scholar

2. U.S. Department of Veterans Affairs. 2024 National Veteran Suicide Prevention Annual Report (2024). Available online at: https://www.mentalhealth.va.gov/docs/data-sheets/2024/2024-Annual-Report-Part-1-of-2_508.pdf (Accessed December 31, 2024).

Google Scholar

3. U.S. Department of Veterans Affairs. 2023 National Veteran Suicide Prevention Annual Report (2023). Available online at: https://www.mentalhealth.va.gov/docs/data-sheets/2023/2023-National-Veteran-Suicide-Prevention-Annual-Report-FINAL-508.pdf (Accessed December 31, 2024).

Google Scholar

4. Carroll D, Kearney LK, and Miller MA. Addressing suicide in the veteran population: engaging a public health approach. Front Psychiatry. (2020) 11:569069. doi: 10.3389/fpsyt.2020.569069

PubMed Abstract | Crossref Full Text | Google Scholar

5. Ilgen MA, Bohnert ASB, Ignacio RV, McCarthy JF, Valenstein MM, Kim HM, et al. Psychiatric diagnoses and risk of suicide in veterans. Arch Gen Psychiatry. (2010) 67:1152. doi: 10.1001/archgenpsychiatry.2010.129

PubMed Abstract | Crossref Full Text | Google Scholar

6. Hawton K and Van Heeringen K. Suicide. Lancet. (2009) 373:1372–81. doi: 10.1016/S0140-6736(09)60372-X

PubMed Abstract | Crossref Full Text | Google Scholar

7. Tsai J and Cao X. Association between suicide attempts and homelessness in a population-based sample of US veterans and non-veterans. J Epidemiol Community Health. (2019) 73:346–52. doi: 10.1136/jech-2018-211065

PubMed Abstract | Crossref Full Text | Google Scholar

8. Garland AF, Bickman L, and Chorpita BF. Change what? Identifying quality improvement targets by investigating usual mental health care. Administration Policy Ment Health Ment Health Serv Res. (2010) 37:15–26. doi: 10.1007/s10488-010-0279-y

PubMed Abstract | Crossref Full Text | Google Scholar

9. Gyani A, Shafran R, Myles P, and Rose S. The gap between science and practice: How therapists make their clinical decisions. Behav Ther. (2014) 45:199–211. doi: 10.1016/j.beth.2013.10.004

PubMed Abstract | Crossref Full Text | Google Scholar

10. Shiner B, D’Avolio LW, Nguyen TM, Zayed MH, Young-Xu Y, Desai RA, et al. Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Administration Policy Ment Health Ment Health Serv Res. (2013) 40:311–8. doi: 10.1007/s10488-012-0421-0

PubMed Abstract | Crossref Full Text | Google Scholar

11. Frank HE, Becker-Haimes EM, and Kendall PC. Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes. Clin Psychology: Sci Pract. (2020) 27:e12330. doi: 10.1111/cpsp.12330

PubMed Abstract | Crossref Full Text | Google Scholar

12. Mott JM, Grubbs KM, Sansgiry S, Fortney JC, and Cully JA. Psychotherapy utilization among rural and urban veterans From 2007 to 2010. J Rural Health. (2015) 31:235–43. doi: 10.1111/jrh.12099

PubMed Abstract | Crossref Full Text | Google Scholar

13. Cully JA, Jameson JP, Phillips LL, Kunik ME, and Fortney JC. Use of psychotherapy by rural and urban veterans. J Rural Health. (2010) 26:225–33. doi: 10.1111/j.1748-0361.2010.00294.x

PubMed Abstract | Crossref Full Text | Google Scholar

14. Rosen CS, Matthieu MM, Wiltsey Stirman S, Cook JM, Landes S, Bernardy NC, et al. A review of studies on the system-wide implementation of evidence-based psychotherapies for posttraumatic stress disorder in the Veterans Health Administration. Adm Policy Ment Health. (2016) 43:957–77. doi: 10.1007/s10488-016-0755-0

PubMed Abstract | Crossref Full Text | Google Scholar

15. Karlin BE and Cross G. From the laboratory to the therapy room: National dissemination and implementation of evidence-based psychotherapies in the U.S. Department of Veterans Affairs Health Care System. Am Psychol. (2014) 69:19–33. doi: 10.1037/a0033888

PubMed Abstract | Crossref Full Text | Google Scholar

16. Karlin BE, Brown GK, Trockel M, Cunning D, Zeiss AM, Taylor CB, et al. National dissemination of cognitive behavioral therapy for depression in the department of veterans affairs health care system: Therapist and patient-level outcomes. J Consulting Clin Psychol. (2012) 80:707–18. doi: 10.1037/a0029328

PubMed Abstract | Crossref Full Text | Google Scholar

17. Brown GK, Batdorf WH, Dedert EA, Kumpula MJ, Day KT, Stanley B, et al. National implementation of advanced training in the safety planning intervention in the Department of Veterans Affairs health care system. psychol Serv. (2024) 22(3):465–76. doi: 10.1037/ser0000880

PubMed Abstract | Crossref Full Text | Google Scholar

18. Karlin BE, Trockel M, Taylor CB, Gimeno J, and Manber R. National dissemination of cognitive behavioral therapy for insomnia in veterans: Therapist- and patient-level outcomes. J Consulting Clin Psychol. (2013) 81:912–7. doi: 10.1037/a0032554

PubMed Abstract | Crossref Full Text | Google Scholar

19. Karlin BE, Ruzek JI, Chard KM, Eftekhari A, Monson CM, Hembree EA, et al. Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the Veterans Health Administration. J Traumatic Stress. (2010) 23:663–73. doi: 10.1002/jts.20588

PubMed Abstract | Crossref Full Text | Google Scholar

20. Chard KM, Ricksecker EG, Healy ET, Karlin BE, and Resick PA. Dissemination and experience with cognitive processing therapy. J Rehabil Res Dev. (2012) 49:667. doi: 10.1682/JRRD.2011.10.0198

PubMed Abstract | Crossref Full Text | Google Scholar

21. Eftekhari A, Crowley JJ, Ruzek JI, Garvert DW, Karlin BE, and Rosen CS. Training in the implementation of prolonged exposure therapy: provider correlates of treatment outcome: training and implementation of PE. J Traumatic Stress. (2015) 28:65–8. doi: 10.1002/jts.21980

PubMed Abstract | Crossref Full Text | Google Scholar

22. Murphy JL, Cordova MJ, and Dedert EA. Cognitive behavioral therapy for chronic pain in veterans: Evidence for clinical effectiveness in a model program. psychol Serv. (2022) 19:95–102. doi: 10.1037/ser0000506

PubMed Abstract | Crossref Full Text | Google Scholar

23. DeMarce JM, Gnys M, Raffa SD, Kumpula M, and Karlin BE. Dissemination of cognitive behavioral therapy for substance use disorders in the Department of Veterans Affairs health care system: Description and evaluation of veteran outcomes. Subst Abuse. (2021) 42:168–74. doi: 10.1080/08897077.2019.1674238

PubMed Abstract | Crossref Full Text | Google Scholar

24. Sripada RK, Grau PP, Porath BR, Burgess J, Van T, Kim HM, et al. Role of institutional support for evidence-based psychotherapy in satisfaction and burnout among veterans affairs therapists. PS. (2024) 75:206–13. doi: 10.1176/appi.ps.20230086

PubMed Abstract | Crossref Full Text | Google Scholar

25. Ackland PE, Cutting A, Spoont MR, Nugent S, Clothier BA, Hudson EM, et al. Was training enough? Examining the implementation of evidence-based psychotherapies for depression in Veterans Health Administration. psychol Serv. (2024) 22(3):432–9. doi: 10.1037/ser0000910

PubMed Abstract | Crossref Full Text | Google Scholar

26. Sayer NA, Rosen CS, Bernardy NC, Cook JM, Orazem RJ, Chard KM, et al. Context matters: Team and organizational factors associated with reach of evidence-based psychotherapies for PTSD in the Veterans Health Administration. Adm Policy Ment Health. (2017) 44:904–18. doi: 10.1007/s10488-017-0809-y

PubMed Abstract | Crossref Full Text | Google Scholar

27. Woltmann EM, Whitley R, McHugo GJ, Brunette M, Torrey WC, Coots L, et al. The role of staff turnover in the implementation of evidence-based practices in mental health care. PS. (2008) 59:732–7. doi: 10.1176/ps.2008.59.7.732

PubMed Abstract | Crossref Full Text | Google Scholar

28. Harding KE, Porter J, Horne-Thompson A, Donley E, and Taylor NF. Not enough time or a low priority? Barriers to evidence-based practice for allied health clinicians. J Continuing Educ Health Professions. (2014) 34:224–31. doi: 10.1002/chp.21255

PubMed Abstract | Crossref Full Text | Google Scholar

29. Ayoubian A, Nasiripour AA, Tabibi SJ, and Bahadori M. Evaluation of facilitators and barriers to implementing evidence-based practice in the health services: A systematic review. Galen Med J. (2020) 9:e1645. doi: 10.31661/gmj.v9i0.1645

PubMed Abstract | Crossref Full Text | Google Scholar

30. Sripada RK, Bohnert KM, Ganoczy D, and Pfeiffer PN. Documentation of evidence-based psychotherapy and care quality for PTSD in the Department of Veterans Affairs. Adm Policy Ment Health. (2018) 45:353–61. doi: 10.1007/s10488-017-0828-8

PubMed Abstract | Crossref Full Text | Google Scholar

31. Program Evaluation Resource Center. VA Mental Health EBP Templates Summary Report. Washington, D.C.: U.S. Department of Veterans Affairs. (2025). https://dvagov.sharepoint.com/sites/VHAPERC/Reports/SitePages/EBP_home.aspx (Accessed 2025-05-16).

Google Scholar

32. Sayer NA, Bernardy NC, Yoder M, Hamblen JL, Rosen CS, Ackland PE, et al. Evaluation of an implementation intervention to increase reach of evidence-based psychotherapies for PTSD in US Veterans Health Administration PTSD clinics. Adm Policy Ment Health. (2021) 48:450–63. doi: 10.1007/s10488-020-01086-3

PubMed Abstract | Crossref Full Text | Google Scholar

33. Eccles MP and Mittman BS. Welcome to implementation science. Implementation Sci. (2006) 1:1, 1748–5908–1–1. doi: 10.1186/1748-5908-1-1

Crossref Full Text | Google Scholar

34. Bauer MS and Kirchner J. Implementation science: What is it and why should I care? Psychiatry Res. (2020) 283:112376. doi: 10.1016/j.psychres.2019.04.025

PubMed Abstract | Crossref Full Text | Google Scholar

35. Tabak RG, Khoong EC, Chambers DA, and Brownson RC. Bridging research and practice: Models for dissemination and implementation research. Am J Prev Med. (2012) 43:337–50. doi: 10.1016/j.amepre.2012.05.024

PubMed Abstract | Crossref Full Text | Google Scholar

36. Proctor EK, Powell BJ, and McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation Sci. (2013) 8:1–11. doi: 10.1186/1748-5908-8-139

PubMed Abstract | Crossref Full Text | Google Scholar

37. Waltz TJ, Powell BJ, Fernández ME, Abadie B, and Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implementation Sci. (2019) 14:42. doi: 10.1186/s13012-019-0892-4

PubMed Abstract | Crossref Full Text | Google Scholar

38. Stetler CB, Mittman BS, and Francis J. Overview of the VA Quality Enhancement Research Initiative (QUERI) and QUERI theme articles: QUERI Series. Implementation Sci. (2008) 3:8. doi: 10.1186/1748-5908-3-8

PubMed Abstract | Crossref Full Text | Google Scholar

39. Kilbourne AM, Abraham KM, Goodrich DE, Bowersox NW, Almirall D, Lai Z, et al. Cluster randomized adaptive implementation trial comparing a standard versus enhanced implementation intervention to improve uptake of an effective re-engagement program for patients with serious mental illness. Implementation Sci. (2013) 8:1–14. doi: 10.1186/1748-5908-8-136

PubMed Abstract | Crossref Full Text | Google Scholar

40. Bauer MS, Miller CJ, Kim B, Lew R, Stolzmann K, Sullivan J, et al. Effectiveness of implementing a collaborative chronic care model for clinician teams on patient outcomes and health status in mental health: A randomized clinical trial. JAMA Netw Open. (2019) 2:e190230. doi: 10.1001/jamanetworkopen.2019.0230

PubMed Abstract | Crossref Full Text | Google Scholar

41. Damschroder LJ, Hamilton A, Farmer MM, Bean‐Mayberry B, Richardson C, Chanfreau C, et al. Real-world impacts from a decade of Quality Enhancement Research Initiative-partnered projects to translate the Diabetes Prevention Program in the Veterans Health Administration. Health Serv Res. (2024) 59:e14349. doi: 10.1111/1475-6773.14349

PubMed Abstract | Crossref Full Text | Google Scholar

42. Landes SJ, Matarazzo BB, Pitcock JA, Drummond KL, Smith BN, Kirchner JE, et al. Impact of implementation facilitation on the REACH VET clinical program for veterans at risk for suicide. PS. (2024) 75:726–32. doi: 10.1176/appi.ps.20230277

PubMed Abstract | Crossref Full Text | Google Scholar

43. Office of Mental Health and Suicide Prevention. National strategy for Preventing Veteran Suicide 2018–2028 (2018). Available online at: https://www.mentalhealth.va.gov/suicide_prevention/docs/Office-of-Mental-Health-and-Suicide-Prevention-National-Strategy-for-Preventing-Veterans-Suicide.pdf (Accessed August 19, 2024).

Google Scholar

44. Mackowiak C, Eagan AE, Gerstel-Santucci CM, Barnes KM, Brubaker CE, Fredritz KL, et al. Expanding veteran suicide prevention: The role of community engagement and partnership coordinators. psychol Serv. (2025). doi: 10.1037/ser0000944

PubMed Abstract | Crossref Full Text | Google Scholar

45. Burnett K, Stockdale SE, Yoon J, Ragan A, Rogers M, Rubenstein LV, et al. The clinical resource hub initiative: first-year implementation of the veterans health administration regional telehealth contingency staffing program. J Ambulatory Care Manage. (2023) 46:228–39. doi: 10.1097/JAC.0000000000000468

PubMed Abstract | Crossref Full Text | Google Scholar

46. Aarons GA, Hurlburt M, and Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration Policy Ment Health Ment Health Serv Res. (2011) 38:4–23. doi: 10.1007/s10488-010-0327-7

PubMed Abstract | Crossref Full Text | Google Scholar

47. Moullin JC, Dickson KS, Stadnick NA, Rabin B, and Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implementation Sci. (2019) 14:1. doi: 10.1186/s13012-018-0842-6

PubMed Abstract | Crossref Full Text | Google Scholar

48. Potthoff S, Finch T, Bührmann L, Etzelmüller A, Van Genugten CR, Girling M, et al. Towards an implementation-stakeholder engagement model (I-STEM) for improving health and social care services. Health Expectations. (2023) 26:1997–2012. doi: 10.1111/hex.13808

PubMed Abstract | Crossref Full Text | Google Scholar

49. The Assessment and Management of Suicide Risk Work Group. VA/DoD Clinical Practice Guideline for the Assessment and Management of Patients at Risk for Suicide (2019). Available online at: https://www.healthquality.va.gov/guidelines/MH/srb/VADoDSuicideRiskFullCPGFinal5088919.pdf (Accessed August 19, 2024).

Google Scholar

50. Kazdin AE. Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. Behav Res Ther. (2017) 88:7–18. doi: 10.1016/j.brat.2016.06.004

PubMed Abstract | Crossref Full Text | Google Scholar

51. Kroenke K, Spitzer RL, and Williams JBW. The PHQ-9. J Gen Intern Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | Crossref Full Text | Google Scholar

52. Ecker AH, O’Leary K, Fletcher TL, Hundt NE, York-Ward KM, Kauth MR, et al. Training and supporting mental health providers to implement evidence-based psychotherapies in frontline practice. Trans Behav Med. (2022) 12:ibab084. doi: 10.1093/tbm/ibab084

PubMed Abstract | Crossref Full Text | Google Scholar

53. Rosen RC, Ruzek JI, and Karlin BE. Evidence-based training in the era of evidence-based practice: Challenges and opportunities for training of PTSD providers. Behav Res Ther. (2017) 88:37–48. doi: 10.1016/j.brat.2016.07.009

PubMed Abstract | Crossref Full Text | Google Scholar

54. Stanley B, Brown GK, MacRae F, Rotolo CA, Hughes G, Mina L, et al. VA Safety Planning Intervention Manual. (2021) Washington, D.C.: U.S. Department of Veterans Affairs.

Google Scholar

55. Stanley B, Brown GK, Brenner LA, Galfalvy HC, Currier GW, Knox KL, et al. Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiatry. (2018) 75:894. doi: 10.1001/jamapsychiatry.2018.1776

PubMed Abstract | Crossref Full Text | Google Scholar

56. Beaudreau SA, Brown GK, Chaison A, Green KL, Sisco S, Padilla-Martinez V, et al. Problem-Solving Therapy for Suicide Prevention for Veterans Provider Manual. (2023) Washington, D.C.: U.S. Department of Veterans Affairs.

Google Scholar

57. Beaudreau SA, Gould CE, Mashal NM, Terri Huh JW, and Kaci Fairchild J. Application of problem solving therapy for late-life anxiety. Cogn Behav Pract. (2019) 26:381–94. doi: 10.1016/j.cbpra.2018.05.003

Crossref Full Text | Google Scholar

58. Cuijpers P, De Wit L, Kleiboer A, Karyotaki E, and Ebert DD. Problem-solving therapy for adult depression: An updated meta-analysis. Eur Psychiatr. (2018) 48:27–37. doi: 10.1016/j.eurpsy.2017.11.006

PubMed Abstract | Crossref Full Text | Google Scholar

59. Hopko DR, Funderburk JS, Shorey RC, McIndoo CC, Ryba MM, File AA, et al. Behavioral activation and problem-solving therapy for depressed breast cancer patients: preliminary support for decreased suicidal ideation. Behav Modif. (2013) 37:747–67. doi: 10.1177/0145445513501512

PubMed Abstract | Crossref Full Text | Google Scholar

60. Toobert DJ, Hampson SE, and Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care. (2000) 23:943–50. doi: 10.2337/diacare.23.7.943

PubMed Abstract | Crossref Full Text | Google Scholar

61. García-Vera MP, Labrador FJ, and Sanz J. Stress-management training for essential hypertension: A controlled study. Appl Psychophysiol Biofeed. (1997) 22:261–83. doi: 10.1023/A:1022248029463

Crossref Full Text | Google Scholar

62. Tenhula WN, Nezu AM, Nezu CM, Stewart MO, Miller SA, Steele J, et al. Moving forward: A problem-solving training program to foster veteran resilience. Prof Psychology: Res Pract. (2014) 45:416–24. doi: 10.1037/a0037150

Crossref Full Text | Google Scholar

63. Beaudreau SA, Karel MJ, Funderburk JS, Nezu AM, Nezu CM, Aspnes A, et al. Problem-solving training for Veterans in home based primary care: an evaluation of intervention effectiveness. Int Psychogeriatrics. (2022) 34:165–76. doi: 10.1017/S104161022000397X

PubMed Abstract | Crossref Full Text | Google Scholar

64. Funderburk JS, Dollar KM, King PR, Pomerantz AS, Bernstein L, Aspnes A, et al. Problem-solving training for VA integrated primary care providers: Real-world outcomes. Prof Psychology: Res Pract. (2021) 52:58–66. doi: 10.1037/pro0000340

Crossref Full Text | Google Scholar

65. Otero MC, Walker JA, Kumpula MJ, Hernandez B, Funderburk JS, Wetherell JL, et al. Negative problem orientation is associated With mental health outcomes for veterans enrolled in problem-solving training. Cogn Behav Pract. (2024) 31:203–14. doi: 10.1016/j.cbpra.2022.11.001

Crossref Full Text | Google Scholar

66. SP 2.0 Clinical Telehealth. Cognitive Behavioral Therapy for Suicide Prevention with Veterans: Therapist Manual. (2023) Washington, D.C.: U.S. Department of Veterans Affairs.

Google Scholar

67. Brown GK, Ten Have T, Henriques GR, Xie SX, Hollander JE, and Beck AT. Cognitive therapy for the prevention of suicide attempts: A randomized controlled trial. JAMA. (2005) 294:563. doi: 10.1001/jama.294.5.563

PubMed Abstract | Crossref Full Text | Google Scholar

68. Slee N, Garnefski N, van der Leeden R, Arensman E, and Spinhoven P. Cognitive-behavioural intervention for self-harm: randomised controlled trial. Br J Psychiatry. (2008) 192:202–11. doi: 10.1192/bjp.bp.107.037564

PubMed Abstract | Crossref Full Text | Google Scholar

69. Rudd MD, Bryan CJ, Wertenberger EG, Peterson AL, Young-McCaughan S, Mintz J, et al. Brief cognitive-behavioral therapy effects on post-treatment suicide attempts in a military sample: results of a randomized clinical trial with 2-year follow-up. Am J Psychiatry. (2015) 172(5):441–49. doi: 10.1176/appi.ajp.2014.14070843

PubMed Abstract | Crossref Full Text | Google Scholar

70. Linehan MM. Cognitive-Behavioral Treatment of Borderline Personality Disorder. New York, NY: Guilford (1993).

Google Scholar

71. Linehan MM. DBT Skills Training Manual (Second Edition). New York, NY: Guilford Publication (2015).

Google Scholar

72. DeCou CR, Comtois KA, and Landes SJ. Dialectical behavior therapy is effective for the treatment of suicidal behavior: A meta-analysis. Behav Ther. (2019) 50:60–72. doi: 10.1016/j.beth.2018.03.009

PubMed Abstract | Crossref Full Text | Google Scholar

73. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. (2006) 63:757–66. doi: 10.1001/archpsyc.63.7.757

PubMed Abstract | Crossref Full Text | Google Scholar

74. Linehan MM, Korslund KE, Harned MS, Gallop RJ, Lungu A, Neacsiu AD, et al. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: A randomized clinical trial and component analysis. JAMA Psychiatry. (2015) 72(5):475–82. doi: 10.1001/jamapsychiatry.2014.3039

PubMed Abstract | Crossref Full Text | Google Scholar

75. McMain SF, Chapman AL, Kuo JR, Dixon-Gordon KL, Guimond TH, Labrish C, et al. The effectiveness of 6 versus 12 months of dialectical behavior therapy for borderline personality disorder: a noninferiority randomized clinical trial. Psychother Psychosom. (2022) 91:382–97. doi: 10.1159/000525102

PubMed Abstract | Crossref Full Text | Google Scholar

76. Hu F-H, Xu J, Jia YJ, Ge MW, Zhang WQ, Tang W, et al. Non-pharmacological interventions for preventing suicide attempts: A systematic review and network meta-analysis. Asian J Psychiatry. (2024) 93:103913. doi: 10.1016/j.ajp.2024.103913

PubMed Abstract | Crossref Full Text | Google Scholar

77. Koons CR, Robins CJ, Lindsey Tweed J, Lynch TR, Gonzalez AM, Morse JQ, et al. Efficacy of dialectical behavior therapy in women veterans with borderline personality disorder. Behav Ther. (2001) 32:371–90. doi: 10.1016/S0005-7894(01)80009-5

Crossref Full Text | Google Scholar

78. Goodman M, Banthin D, Blair NJ, Mascitelli KA, Wilsnack J, Chen J, et al. A randomized trial of dialectical behavior therapy in high-risk suicidal veterans. J Clin Psychiatry. (2016). doi: 10.4088/JCP.15m10235

PubMed Abstract | Crossref Full Text | Google Scholar

79. Landes SJ, Pitcock JA, Harned MS, Connolly SL, Meyers LL, and Oliver CM. Provider perspectives on delivering dialectical behavior therapy via telehealth during COVID-19 in the Department of Veterans Affairs. psychol Serv. (2021) 19(3):562–72. doi: 10.1037/ser0000571

PubMed Abstract | Crossref Full Text | Google Scholar

80. Assessment and Management of Patients at Risk for Suicide Work Group. VA/DoD Clinical Practice Guideline for Assessment and Management of Patients At Risk for Suicide (2024). Available online at: https://www.healthquality.va.gov/guidelines/MH/srb/VADoD-CPG-Suicide-Risk-Full-CPG-2024_Final_508.pdf (Accessed August 19, 2024).

Google Scholar

81. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Sci. (2015) 10. doi: 10.1186/s13012-015-0209-1

PubMed Abstract | Crossref Full Text | Google Scholar

82. Oslin DW, Ross J, Sayers S, Murphy J, Kane V, and Katz IR. Screening, assessment, and management of depression in VA primary care clinics: The behavioral health laboratory. J Gen Intern Med. (2006) 21:46–50. doi: 10.1111/j.1525-1497.2005.0267.x

PubMed Abstract | Crossref Full Text | Google Scholar

83. Tew J, Klaus J, and Oslin DW. The behavioral health laboratory: Building a stronger foundation for the patient-centered medical home. Families Systems Health. (2010) 28:130–45. doi: 10.1037/a0020249

PubMed Abstract | Crossref Full Text | Google Scholar

84. Glasgow RE, McKay HG, Piette JD, and Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. (2001) 44:119–27. doi: 10.1016/s0738-3991(00)00186-5

PubMed Abstract | Crossref Full Text | Google Scholar

85. Stanley B, Green KL, Ghahramanlou-Holloway M, Brenner LA, and Brown GK. The construct and measurement of suicide-related coping. Psychiatry Res. (2017) 258:189–93. doi: 10.1016/j.psychres.2017.08.008

PubMed Abstract | Crossref Full Text | Google Scholar

86. Bryan CJ, May AM, Thomsen CJ, Allen MH, Cunningham CA, Wine MD, et al. Psychometric evaluation of the suicide cognitions scale-revised (SCS-R). Military Psychol. (2022) 34:269–79. doi: 10.1080/08995605.2021.1897498

PubMed Abstract | Crossref Full Text | Google Scholar

87. Robichaud M and Dugas MJ. Negative problem orientation (Part II): construct validity and specificity to worry. Behav Res Ther. (2005) 43:403–12. doi: 10.1016/j.brat.2004.02.008

PubMed Abstract | Crossref Full Text | Google Scholar

88. Bjureberg J, Ljótsson B, Tull MT, Hedman E, Sahlin H, Lundh LG, et al. Development and validation of a brief version of the Difficulties in Emotion Regulation Scale: The DERS-16. J Psychopathol Behav Assess. (2016) 38:284–96. doi: 10.1007/s10862-015-9514-x

PubMed Abstract | Crossref Full Text | Google Scholar

89. Bohus M, Limberger MF, Frank U, Chapman AL, Kuhler T, and Stieglitz RD. Psychometric properties of the borderline symptom list (BSL). Psychopathology. (2007) 40:126–32. doi: 10.1159/000098493

PubMed Abstract | Crossref Full Text | Google Scholar

90. U.S. Department of Veterans Affairs. VHA Long-Range Plan FY 2022-2025 (2021). Available online at: https://www.va.gov/HEALTH/docs/FY-2022-2025-VHA-Long-Range-Plan.pdf (Accessed April 14, 2024).

Google Scholar

91. Choi NG, Marti CN, Bruce ML, Hegel MT, Wilson NL, and Kunik ME. Six-month postintervention depression and disability outcomes of in-home telehealth problem-solving therapy for depressed, low-income homebound older adults. Depress Anxiety. (2014) 31:653–61. doi: 10.1002/da.22242

PubMed Abstract | Crossref Full Text | Google Scholar

92. Choi NG, Marti CN, and Conwell Y. Effect of problem-solving therapy on depressed low-income homebound older adults’ death/suicidal ideation and hopelessness. Suicide Life Threat Behav. (2016) 46:323–36. doi: 10.1111/sltb.12195

PubMed Abstract | Crossref Full Text | Google Scholar

93. Myers US, Birks A, Grubaugh AL, and Axon RN. Flattening the curve by getting ahead of it: How the VA healthcare system Is leveraging telehealth to provide continued access to care for rural veterans. J Rural Health. (2021) 37:194–6. doi: 10.1111/jrh.12449

PubMed Abstract | Crossref Full Text | Google Scholar