Education and training of telemental health providers: a systematic review

Objective To conduct a systematic literature review of education and training (E&T) programs for telemental health (TMH) providers in the past 10 years to qualitatively clarify field offerings and methodologies, as well as identify areas for future growth. Methods We searched five major electronic databases: PubMed, PsycINFO, Scopus, CINAHL, and Web of Science for original publications on TMH E&T from January 2013 to May 2023. We extracted information from each publication and summarized key features of training programs including setting, target group, study aims, training modality, methods of assessing quality, and outcomes. Results A total of 20 articles were selected for the final review. Articles meeting inclusionary criteria were predominantly comprised of case studies and commentaries, focused on a TMH service/practice for a specific region/population, and were performed after 2020. All of the selected studies demonstrated a significant increase in the measured knowledge, skills, and abilities of the participants after TMH training. Nevertheless, there remains a lack of standardization of training methodologies, limited sample sizes and demographics, variability in study methodologies, and inconsistency of competency targets across studies. Conclusion This systematic review highlighted the diversity of methods for TMH E&T. Future research on this topic could include more varied and larger-scale studies to further validate and extend current findings, as well as explore potential long-term effects of TMH training programs on both provider attitudes and patient outcomes.


Introduction
Telemental health (TMH) is used to refer to mental health services that are provided via telecommunications technologies (e.g., email, video, telephone, messaging programs).This field has demonstrated rapid development over the past three decades (1,2).Advances in technology have enabled real-time distant consultations, assessments, therapy, and training opportunities.The distinction between traditional mental health providers and TMH lies in the mode of service delivery.Traditional mental health providers usually supply in-person services while TMH providers utilize telecommunication technologies for remote mental health service.While TMH continues to gain wider acceptance and adoption, driven by factors such as the need for increased access to mental health services, advancements in technology, and recognition of its efficacy and effectiveness (3), it has also faced criticism, especially on its use in psychotherapy and psychodynamic procedures (4,5).
The COVID-19 pandemic further accelerated the implementation and adoption of TMH as an essential means of delivering mental healthcare.Many studies showed that when properly adapted from in-person practices, TMH approaches effectively promoted the mental health and well-being of participants with various mental problems (6).To ensure proper adaptation, a growing number of research studies detailed how TMH has been implemented, adopted, and perceived by diverse individuals and providers (7)(8)(9).
Many studies have demonstrated that education and training (E&T) in TMH is needed, as a lack of education can lead to suboptimal care and a lack of knowledge about how to address technological-related issues.Contrastingly, proper E&T has been suggested as positively impacting provider attitudes and knowledge, and subsequently patient outcomes (10)(11)(12).However, there is a lack of standardization in TMH E&T (13,14).Some researchers have attempted to propose core competencies for TMH (12,(15)(16)(17)(18) to include, but not necessarily be limited to, knowledge of the literature and evidence-based practices for different demographic characteristics, TMH modalities, and settings; evidence-informed methods of adapting in-person techniques (e.g., clinical work, interpersonal skills, rapport, communication) for digital administration; ethics of TMH practice; legality of TMH practice (e.g., cross-jurisdiction practice, data security); professionalism in practice, selection and troubleshooting of technology; diversity considerations of unique populations; administrative tasks (e.g., documentation, billing); and interdisciplinary collaboration.Nevertheless, to date, there has been little success in establishing standardized competencies, let alone efficient, scalable strategies for training existing TMH providers in said competencies.Additionally, very few E&T programs are dedicated to long-term career development for future TMH providers (19).For example, limited TMH education has been suggested as available in graduate education, affecting a provider's knowledge and future use of TMH technologies in clinical care.This lack of training may be partially due to the relative youth of the field though has been suggested as an issue in need of addressing (20,21).
To meet the increasing demand for evidence-based TMH providers, it is important to develop evidence-informed methods of E&T.The evidence-informed usage of TMH can guide ethical, legal, evidence-informed, and safe provider usage to not only maximize outcomes, but to reduce challenges that arise from the use of technology in clinical practice (22)(23)(24).Unfortunately, no known review has consolidated literature to clarify current field offerings and future directions.Therefore, there is a need for a systematic review concerning E&T of TMH providers to identify currently available programs, strengths and weaknesses of said programs, and the ongoing needs of the field.This information will help inform future adjustments in training design to better reflect the rapid changes in the field of TMH.
The objective of this study was to review and aggregate recent literature on TMH E&T to clarify field offerings and methodologies, as well as identify areas for further growth.Several research questions were derived to guide analyses: (a) who provided TMH E&T (i.e., setting), (b) who was the TMH E&T provided to, (c) what was the aim of TMH E&T, (d) how was the TMH E&T provided, (e) how was the quality of the TMH E&T assessed, and (f) what were the outcomes of the TMH E&T.Given limited prior literature to guide research questions, the current study was viewed as exploratory.

Study type
This study was conducted as a systematic review.The method of a systematic review was chosen over other methods, such as a scoping review, due to its rigorous and comprehensive approach in synthesizing existing literature to address focused research questions rather than summarize a broader and more diverse field of study.

Search strategy
This review was conducted according to the PRISMA standard with the involvement of every member of the research team (25).The screening process for determining which papers were to be analyzed in this study is depicted in Figure 1.We used the PubMed, PsycINFO, Scopus, CINAHL, and Web of Science databases to search TMH related English language articles published from January 2013 to May 2023.Keywords utilized were telemental health, TMH, telepsychology, telepsychiatry, telebehavioral health, TBH, and mobile mental health; all of these keywords were also separately paired with "education" and "training" to further expand the search.The full search strategies for each database, including complete keyword combinations, Boolean operator usage, and filtering criteria, can be found in the Supplementary material.Initially, title and abstract screening were conducted to assess relevance of the articles in the search results.Then, full-text screening was performed for eligible articles.Every article reviewed in this process was screened independently by at least two study members with any disagreements resolved through a group consensus.The Systemic Review Accelerator (SRA) was a computer tool that was utilized to facilitate this process.

Inclusion and exclusion criteria
The inclusion and exclusion criteria were based on relevance to education, training and competency development of TMH providers.Studies providing a description of education or training programs for mental health-focused telehealth providers were included.Contrastingly, articles focusing on only providers' experiences or perspectives of TMH E&T were excluded.Papers that utilize remote or online technology to train providers in the fields of mental health or psychiatric informatics were also excluded, unless they were sufficient to cover all necessary training content and requirements of TMH itself.Papers examining the educational needs or the level of

Quality of studies
Each article meeting inclusionary criteria was assessed for relevance, quality, and contribution to the topic of TMH providers' education, training, and competency development.The quality of studies was assessed using the Joanna Briggs Institute Qualitative Appraisal and Review Instrument (JBI QARI) Critical Appraisal checklist for Interpretive and Critical Research (Attachment).This quality assessment tool was modified for this systematic review; less relevant elements were removed as based upon consensus by the authors.The papers in this study were subsequently qualitatively analyzed through this tool to identify factors relevant to address research questions (See Tables 1, 2).

Review composition
The electronic searches identified 971 articles.An additional 4 were identified by a manual search from references.The abstracts of all 971 papers were reviewed with 184 meeting the inclusion criteria and 787 excluded.Based on full-text reading of the remaining 184, a   • A process improvement evaluation.
• Primary Care Centers or Mental Health Clinics.
• A pilot/feasibility study.
• To describe a TMH network that uses a store-and-forward platform (asynchronous communication).
• Psychology trainees and staff psychologists.
• Social Workers practicing in a variety of clinical settings.
• Limitations: There was a low response rate for online events/follow-up surveys.
• Interdisciplinary mental health providers including psychologists, social workers, psychiatrists, nurses, and others not specified.
• Evaluation of program development and implementation.
• To assess the effectiveness of the program and implementation with interdisciplinary mental health providers in the VA system.
• A mixed-methods approach: • All psychology trainees in the system.
• To provide guidance and recommendations for mental health training programs during public health crises (including COVID-19).
• Commentary article.(Continued)  further 168 papers were excluded, adding 4 additional records searching for manually and leaving a final set of 20 studies for inclusion in this review (Figure 1).

Characteristics of studies
A summary of the remaining 20 study's objective, target group, training modality, outcomes and other characteristics can be seen in Table 3.
Despite the analysis spanning ten years, review suggested that a majority (i.e., n = 14, 70%) of articles meeting inclusionary criteria were published following 2020.

Providers of TMH E&T
The TMH training programs were held in various settings, including, but not limited to university counseling centers, community clinics, private practices, VA hospitals, and prisons/jails.A significant portion of the settings were affiliated with academic institutions.Many clinical service settings, such as VA, expanded TMH E&T during the COVID-19 pandemic due to the urgent need for TMH at that time.While beneficial to all patient demographics, the option for TMH was especially useful for certain populations, such as children and adolescents (29), people with social anxiety disorder (30), and individual experiencing schizophrenia (44).

Users of TMH E&T
The target trainees were diverse and included military personnel, doctoral students, healthcare providers, interdisciplinary practitioners, and a mix of students and qualified professionals.The majority, however, were psychology/social worker students or clinicians.Of note, each program had a training focus area and goals unique to the training program.

Aims of TMH E&T
Almost all training programs included in the study focused on enhancing the skills and knowledge of TMH providers.Study objectives collectively aimed to improve the reach of mental health professionals, especially in the context of public health crises and underserved communities, while leveraging technology for future training and service delivery.While some studies described specific training activities among specific trainee populations (e.g., psychology practicum training for specific interventions, telepsychiatry) (27,37), others assessed the feasibility and efficacy of TMH E&T more broadly among different cultural (30), and clinical settings (34).A few studies focused on promoting access to mental health services through telehealth, addressing barriers (31), and examining participant experiences with online training tools (35).Additionally, some studies focused on comparing different training modalities, and exploring factors influencing telepsychology training from the perspective of doctoral students (40).

Methods of providing TMH E&T
While the training modalities all included virtual components such as web-based courses, live sessions or asynchronous audio-video communications, there were some notable differences.For example, some programs incorporated specialized in-person training (36), or diverse formats (e.g., training to use a specific online platform) to meet the specific needs of the training (37).A mixed-methods approach, combining various training components, was evident in some studies to provide a comprehensive learning experience (38).Many training programs involved interactive elements such as group supervision, role-plays, case discussions, and hands-on training to engage participants (34,37,39) which were delivered via digital methods including videoconferencing, online platforms, and other digital resources (See Figure 2).

Quality assessment of TMH E&T
Most evaluations of the effectiveness, feasibility, and impact of TMH training programs were done through surveys.Those included competency-based evaluations (29), feedback (34,37), programmatic evaluations (28, 33), and sustainability assessments (43).Most assessments focused on using feedback and data to improve training programs, guidelines, and processes (36,40,43); however, some assessments, particularly in programmatic evaluations, aimed to provide resource savings in training delivery (42).One study used both quantitative and qualitative data to evaluate the effectiveness of training programs (40).

Outcomes of TMH E&T
All training programs were found to be effective in achieving their goals, including expanding access to services, improvement of competencies, facilitating global collaboration among international institutions, and facilitating rapid adaptation during time-sensitive public health crises.User satisfaction was a  common theme, indicating that participants found the training programs to be helpful, timely, and relevant (34,36,38).The results also highlighted the adaptability of these programs in response to challenges, such as the rapid adoption of telepsychology during the COVID-19 pandemic, and the importance of collaboration, outreach, and ongoing support in reaching diverse audiences and expanding the positive impact of TMH training efforts (29).

Discussion
The current systematic review of TMH E&T summarized the setting, target group, study aims, training modality, quality of training assessment, and outcomes of studies over the last 10 years.
While present to some degree prior, the study of TMH E&T demonstrated the most notable growth since 2020.While definitive reasons for this finding are unclear, as they were often not directly addressed in each study's methodology, it is hypothesized that the growing recognition of the importance of TMH E&T arose in response to the exponential increase in clinical usage following the advent of COVID-19 (i.e., providers experienced a rapid and unanticipated transition to telehealth, despite having limited prior training).As a result, providers, training programs, and researchers recognized the gap in both literature and training related to TMH competencies, prompting the increase in study.

Current field offerings
Analysis suggested that while still in its relative infancy, literature on TMH E&T is developing.Data indicated that trainings have occurred across a wide variety of locations (e.g., university counseling centers, community clinics, hospitals), and for healthcare providers across professional developmental levels (e.g., doctoral level, licensed providers).While professional emphasis varied, the majority of identified literature focused on psychology or social worker students and licensed professionals rather than other healthcare specialties.Related to aims, training programs predominantly focused on developing both knowledge and skill domains, two equally important, but different, According to the quality score, 6 studies were rated as 8, and 2 as 3,2, which are descriptive studies.The average score is 6.15, which indicates that the overall quality is high.Out of 20 studies, 16 studies have clearly stated the research question or aim about the TMH E&T, because 3 studies are not a research study with a specific research question or aim (28,31,43) and also 2 studies do not explicitly state the research approach for the TMH E&T program (30, 43).Instead, they provide an overview of the TMH E&T program (30) or focus on describing the methodology and outcomes of the program (17,27).The data collection methods of 15 studies align with the research question and approach, and the other 5 studies do not provide detailed information on the specific data collection methods used (27,31,37,39,42).There is evidence of ethical considerations in 16 studies, while 4 studies do not provide detailed information on the ethical considerations of the research itself (31,32,36,39), they do acknowledge the importance of ethical guidelines and considerations in the provision of telepsychology services.In terms of findings and transferability, 17 studies adequately discussed the findings in the context of existing literature, and there are enough details to judge if the findings could apply to other contexts.However, out of 20 papers, only 6 studies acknowledged the influence of the researcher on the research process, findings, and limitations, while most of the studies did not.16 studies addressed the research question adequately.

Key factors and themes
As based upon primary findings, key factors and themes for the studies can be summarized as follows: • Supervision and manpower: availability of flexible on-ground supervisors is crucial for effective TMH E&T.This includes ensuring there is adequate support, follow-up, and ongoing assistance for trainees (28, 39).While collective results are promising, findings highlight the need for additional and targeted study on TMH E&T in terms of defining necessary competencies for providers, greater diversity of study location and demographic considerations, and long-term follow-up to clarify provider ongoing use and outreach activities (e.g., whether TMH E&T fostered a desire for additional outreach efforts than what the provider offered prior to the training).Future studies should seek to remedy noted limitations.

Application of findings
With consideration of both strengths and areas of improvement of current literature, several recommendations for future application of E&T are presented.Most importantly, field experts, training programs, and guiding organizations must work to consolidate literature to identify essential and universal TMH competencies, both knowledgeand skill-based.Additionally, optimal means of conveying such information (e.g., in-person training, web-based self-guided, hybrid approach) is necessary.As part of this training (or following), supervision of TMH usage is vital (46).Supervision of TMH allows for the trainee to receive targeted feedback of both strengths and areas for improvement, allows for advancement of knowledge and skills, and gatekeeps the profession.To track trainee progress, evaluate training program efficacy and effectiveness, and allow for iterative development of the programs, comprehensive and consistent objective outcome measures are required.Utilization of similar measures across studies will allow for cross-site comparison.Targets of evaluation can include satisfaction, as well as the program's ability to foster greater TMH knowledge, greater TMH skills, and outreach efforts to rural and underserved communities.Additionally, evaluation should consider cost effectiveness of the programs to optimize both training methodologies and costs to create sustainable and scalable programs that can provide a template for TMH education, while allowing the ability to tailor to different regions, states, provinces, territories, or countries to accommodate different languages, cultures, and contexts.Of important note, E&T is not a one-time endeavor.Similar to other aspects of healthcare, continuing education is a lifelong activity.As such, E&T efforts should be designed around different professional developmental stages.More specifically, E&T should be created and adopted by graduate-level training programs to provide novice providers with education and supervision of TMH practices.Such practices can be augmented and supplemented by additional E&T on internships, fellowships, and residencies.Finally, continuing education activities and programs should be tailored to those more senior providers who already have experiences with TMH, but require updates to ensure adherence to ever-changing evidence-based standards and regulations surrounding the use of technology in healthcare services.To formalize training activities, policy makers (e.g., American Psychological Association, American Psychiatric Association, American Medical Association) who guide and/or accredit healthcare training programs, and license regulators (e.g., state licensing boards) should encourage consistent TMH accreditation standards for training and continuing education requirements for maintenance of licensure.

Limitations of the current study
While the current study is believed to address an identified gap in the literature, it is not without limitations that should be considered in the finding's interpretation.First, the study pulled from five databases.While broad databases were chosen, it is recognized that additional relevant literature may be available in other databases.Similarly, only peer-reviewed manuscripts were considered for inclusion, potentially excluding relevant literature that has yet to be published, or included in non-peer-reviewed outlets (e.g., books).In addition, this review focused on TMH E&T broadly and did not include discussion of other technologies such as AI, a rapidly emerging technology revolutionizing today's health care.Further, the study aimed to qualitatively describe current field offerings and guide future directions.Nevertheless, it is recognized that future work should consider quantitively defining and analyzing data to further understand current strengths and limitations.Such future study should also seek to address identified field limitations, such as small sample sizes, limited external validity, poor standardization of competencies across studies, and different methodologies that each create challenges for direct comparison.

Conclusion
Telemental health has rapidly expanded in recent years, subsequently producing a myriad of different training programs, each with unique strengths and limitations.While meaningful progress has been made related to TMH E&T, additional work is required.As the use of TMH is projected to continue (47), it remains prudent for E&T programs to work towards standardization of training foci and knowledge and skills of providers in TMH were also excluded if they were not in the context of a specific training program.The current study elected to review the literature from only the past decade, as other reviews have covered July 2003 to March 2013 (3).Due to the rapid advances, a review of the last ten years is believed to better reflect the current status of TMH E&T than prior research.

FIGURE 1 PRISMA
FIGURE 1 PRISMA diagram of search process.
To describe and evaluate a program of therapists training to deliver internetbased CBT for social anxiety disorder (CT-SAD).•After initial face-to-face case Overview of the VA's TMH training program, to improve access to TMH by addressing barriers to training and implementation.
Describing how a School of Social Work rapidly deployed a free Continuing Education (CE) training program on the basics of TMH practice for the response to COVID-19, program's usage, and user feedback. of a control group, long-term impact of the training was not assessed and potential for selection bias among participants.Future directions: Necessary to have a more rigorous evaluation of the effectiveness of TMH services and the development of best practices for TMH delivery.Malhotra et al. sample size limits the generalizability of the findings; the study did not assess the long-term impact of the training on patient outcomes and poor internet connectivity and variable quality of the video/ audio impeded training.

•
Pre-and post-training survey.• To examine the effectiveness of a TMH training initiative to rapidly train mental health providers to provide TMH in response to the COVID-19 pandemic.• Self-paced with 4 sets of training materials including pre-recorded videos, PowerPoint slides, and written materials.• Pre-training and post-training surveys.Presenting preliminary process and outcome data that compares the reach and impact of support before and during COVID-19-related restrictions.• Through in-person and online training, workshops,

Future
of impact and seeking to identify which training formats are associated with the most positive outcomes.
outcome-based study.• To evaluate and define an ATP training model for primary care physician (PCP)integrated behavioral health clinicians.

Criteria Yes ( 1 )
Not clear (0) No (−1) 1. Research question or aim: is the research question or aim clearly stated?2. Methodological congruence: is the methodology congruent with the research question and approach?3. Data collection: do the data collection methods align with the research question and approach?4. Ethics: is there evidence of ethical considerations in the research?5. Findings: are the findings adequately discussed in the context of existing literature? 6. Transferability: are there enough details to judge if the findings could apply to other contexts?7. Reflexivity: is the influence of the researcher on the research process, findings, and limitations acknowledged?8. Overall appraisal: does the study address the research question adequately?Overall appraisal: does the study address the research question adequately?10.3389/fpubh.2024.1385532Frontiers in Public Health 15 frontiersin.org

TABLE 1
Summation of articles included for review.

TABLE 2
Joanna Briggs Institute Qualitative Appraisal and Review Instrument (JBI QARI) critical appraisal checklist for interpretive and critical research.

TABLE 3
Assessment of the quality of the 20 studies.
(27)ewed studies collectively had limitations that may affect their generalizability.First, a majority of studies were case examples of local/regional training programs.While serving as a template for other locations, direct application may not be possible to other /locations without adaptation.Similarly, most existing studies were based on small sample sizes and specific target groups, and therefore limit the generalizability of the findings.Larger-scale research in more diverse settings is needed to confirm the generalizability of the results.Additionally, many studies relied on one-time self-reported surveys as opposed to more objective data collection of specific operationally-defined targets, such as changes in attitude towards TMH, usage of TMH, how providers adapted (or did not adapt) practices following TMH E&T, locations served via TMH, and long-term use of TMH.Objective evaluation is believed necessary, as just because a provider gains new knowledge and skills does not necessarily mean that they will integrate such information into their practice in the immediate or long-term, especially if barriers present (e.g., lack of time, financial challenges).As a result, external validation of assessments currently utilized, as well as longitudinal data collection are required (30).Related to the training itself, there remains a lack of uniformity in terms of how the information is presented (e.g., length, methodology)(27).More concerning, many studies failed to define what was actually being taught in terms of TMH modality (e.g., video vs. email) or competencies.Finally, a wider range of provider demographic characteristics is needed to determine the influence of location (i.e., rural, urban), race, ethnicity, socioeconomic status, treated population, and technology modalities (e.g., video, telephone, email) on training outcomes.Towards this end, additional evaluation should evaluate provider current practices, and directly inquire about what is most needed to tailor the E&T to unique populations of providers (e.g., underserved or remote areas). settings 10.3389/fpubh.2024.1385532Frontiers in Public Health 18 frontiersin.orgmethodologies in line with evidence-based literature and developing literature and policies.