# DIGITAL INTERVENTIONS IN MENTAL HEALTH: CURRENT STATUS AND FUTURE DIRECTIONS

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-635-8 DOI 10.3389/978-2-88963-635-8

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# DIGITAL INTERVENTIONS IN MENTAL HEALTH: CURRENT STATUS AND FUTURE DIRECTIONS

Topic Editors: Elias Aboujaoude, Stanford University, United States Lina Gega, University of York, United Kingdom Michelle Burke Parish, University of California, Davis, United States Donald M. Hilty, UC Davis Health, United States

Citation: Aboujaoude, E., Gega, L., Parish, M. B., Hilty, D. M., eds. (2020). Digital Interventions in Mental Health: Current Status and Future Directions. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-635-8

# Table of Contents

*04 Editorial: Digital Interventions in Mental Health: Current Status and Future Directions*

Elias Aboujaoude, Lina Gega, Michelle B. Parish and Donald M. Hilty

*07 Improving Reading in Adolescents and Adults With High-Functioning Autism Through an Assistive Technology Tool: A Cross-Over Multinational Study*

Arlinda Cerga-Pashoja, Jorge Gaete, Antoneta Shishkova and Vesna Jordanova


Gillian Strudwick, Anthony Yeung and David Gratzer

*93 Internet-Delivered Cognitive Behavior Therapy as a Prequel to Face-To-Face Therapy for Depression and Anxiety: A Naturalistic Observation*

Daniel Duffy, Angel Enrique, Sarah Connell, Conor Connolly and Derek Richards

*108 From Digital Mental Health Interventions to Digital "Addiction": Where the Two Fields Converge*

Elias Aboujaoude and Lina Gega

# Editorial: Digital Interventions in Mental Health: Current Status and Future Directions

Elias Aboujaoude1\*, Lina Gega<sup>2</sup> , Michelle B. Parish<sup>3</sup> and Donald M. Hilty <sup>3</sup>

<sup>1</sup> Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, United States, <sup>2</sup> Department of Health Sciences & Hull York Medicine School, University of York, York, United Kingdom, <sup>3</sup> Department of Psychiatry and Behavioral Sciences, University of California, Davis, Davis, CA, United States

Keywords: telemedicine, digital mental health interventions, virtual reality therapy, artificial intelligence, computerized cognitive behavior therapy, telepsychiatry, electronic health record - (EHR), internet addiction

Editorial on the Research Topic

#### Digital Interventions in Mental Health: Current Status and Future Directions

An honest supply-and-demand assessment of mental health services in 2020 leads one to conclude that treatment needs would be impossible to meet without increased leveraging of technology. Several inherent factors make mental health interventions particularly well suited to digital platforms: pronounced provider shortages; reduced reliance on examinations and testing; the stigma still attached to mental illness; and diagnosis-specific obstacles to visiting mental health clinics (1). As such, various technology-enabled platforms have been tested to support mental health treatment delivery, from internet-mediated video-based psychotherapy to virtual reality (VR) and artificial intelligence (AI) enabled programs. Yet the reach of digital mental health interventions (DMHIs) still falls short of its touted potential (2).

Internet-delivered cognitive behavioral therapy (ICBT) may be among the best-studied DMHIs (3). It has been used for nearly 20 years, during which it has been subjected to several efficacy trials. To determine more reliable response rates, Andersson et al. conducted an individual patient data meta-analysis, comprising 29 Swedish trials enrolling 2866 participants, and covering three categories of conditions: anxiety disorders, depression, and other. Overall, 65.6% of all clients receiving ICBT responded, and about a third achieved remission. More symptoms and female sex increased the likelihood of improvement, and having an anxiety disorder seemed to decrease it.

Children, adolescents, and young adults have received particular attention with respect to DMHIs. As "digital natives," might they engage with, and benefit from, technology-enabled treatment more than those who did not grow up with technology? Garrido et al.'s meta-analysis focusing on young people with anxiety and depression suggests that DMHIs—in particular supervised DMHIs rather than standalone self-help—outperform "no intervention" but do no better than active alternatives (e.g., face-to-face therapy). Surprisingly, adherence of young people with DMHIs was generally low, mirroring reviews on DMHIs in adults (4, 5).

Other sub-populations that have received research attention include army veterans, older adults, and patients in Latin America. Boykin et al. examined the use of video-based therapy in the naturalistic setting of a clinic serving US veterans with post-traumatic stress disorder. In 74 veterans receiving at least one session of video-delivered cognitive processing therapy (CPT) or prolonged exposure (PE), the completion rate was higher for CPT, but attrition by session 7 was 50% (similar to in-person treatment in this population).

#### Edited and reviewed by:

Wulf Rössler, Charité Medical University of Berlin, Germany

> \*Correspondence: Elias Aboujaoude eaboujaoude@stanford.edu

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 27 January 2020 Accepted: 10 February 2020 Published: 27 February 2020

#### Citation:

Aboujaoude E, Gega L, Parish MB and Hilty DM (2020) Editorial: Digital Interventions in Mental Health: Current Status and Future Directions. Front. Psychiatry 11:111. doi: 10.3389/fpsyt.2020.00111

Among older adults, the use of DMHIs comes with unique obstacles and advantages. Seifert et al. dissect those in an opinion article that focuses on healthcare inequalities and how, depending on awareness, training and public health priorities, new technologies can serve to either mitigate or magnify the disparities that already exist across the age spectrum.

As a region, Latin America suffers from unique access-to-care challenges, but the territory also enjoys relatively good internet and smartphone penetrance (6), making DMHIs a possible solution. In a scoping review of 22 studies that cover prevention, treatment, education, and symptom self-management, Jiménez-Molina et al. explored the potential for leveraging internet-based interventions in mental health in Latin America. Results from the three RCTs identified were mixed, and, while most feasibility and pilot studies showed reasonable acceptability, participant retention was challenging, follow-ups were short, and data on costs and outcomes were limited. The authors conclude that more evidence is required before DMHIs can be considered a realistic remedy to access and delivery problems in Latin America.

But DMHIs do not have to replace traditional care altogether; they can also play a role alongside or preceding traditional treatment. In a UK study, Duffy et al. tested a stepped care model for treating 124 patients with severe depression and anxiety using ICBT as a prequel for a high-intensity face-toface intervention. Significant reductions were noted across primary outcome measures from baseline to ICBT treatment exit, and from ICBT exit to service exit. Results support the use of ICBT as a means to reduce frustrating waiting times and enhance efficiency.

Another example of DMHIs working "with" traditional treatments may be the use of assistive technologies to address specific deficits within a disorder that contribute to functional disability. In a multinational study involving 243 participants, Cerga-Pashoja et al. tested an open-source text simplification tool designed to help people with autism spectrum disorder better comprehend complex texts, metaphors and idioms—a common challenge in this condition. The tool significantly enhanced functioning.

The future of technology-enabled treatment in mental health is probably best captured in the VR and AI "revolutions" unfolding within the larger space of DMHIs. VR has been tested for its possible therapeutic value in mental health for almost as long as it has been exploited for gaming purposes—a quarter century (7)—but research into VR treatments has accelerated considerably in recent years, particularly for anxiety disorders. Yet VR's reach remains limited among patients and therapists (2), a fact that Boeldt et al. dissect in their opinion piece, suggesting educational, practice-based and research steps to take full advantage of what VR therapeutics have to offer.

AI and machine learning are seen as transforming fields like radiology and pathology (8, 9), and, in medicine, more broadly, they hold the promise of bringing individualized low-cost interventions that can be easily scaled. But what is their potential niche and pitfalls in mental health, specifically? In their viewpoint article, Miner et al. discuss how conversational AI may impact psychological and psychiatric care at the level of diagnosis, information gathering and treatment, and propose four possible approaches as guideposts that inform future research and policy.

Mental health interventions, digital or not, have to be documented in patient records held by clinical professionals and health services. In their opinion article, Strudwick et al. argue for opening up these records to patients on the grounds of empowerment and autonomy. The electronic health record now in common use greatly facilitates this process, yet patient portals still often limit access to mental health notes for reasons and controversies elucidated by the authors.

Finally, DMHIs represent potentially beneficial uses of technology in mental health. A parallel and similarly rich body of research has focused on the negative aspects of technology, including addiction, gaming, cyberbullying, and online impulsivity. However, these two areas of scholarship have grown in mutually insular ways with little cross-fertilization, despite some shared commonalities that Aboujaoude and Gega explore in a perspective piece. Collaboration between researchers in both camps is essential if we are to reach a more complete understanding of the issues lying at the technologypsychology intersection.

Overall, the studies and articles included in this special issue suggest that outcomes with DMHIs are comparable with traditional offerings; that some under-researched populations are receiving much needed attention; and that enthusiastic interest animates clinicians and industry professionals to apply the latest digital developments, including VR and AI, to mental health diagnosis, symptom-tracking and treatment. However, plenty remains to be done to address some basic shortcomings that are borne out in the present issue, including: adherence and engagement challenges; access to technology obstacles; lack of cost effectiveness data; lack of long-term research; and the paradoxical process by which digital tools can sometimes serve to fortify rather than diminish healthcare inequalities. Given the relatively capped supply of traditional mental health treatments, and the ethical imperative to meet the ever-increasing demand, DMHIs are likely to be an unavoidable part of any solution to treatment access issues. This special issue highlights the dizzying diversity and richness propelling the field of DMHIs as well as the limitations still holding it back. Further rigorous, yet pragmatic, research—across platforms, populations, diagnoses and interventions—is needed to arrive at a more realistic assessment of the true potential of DMHIs.

#### AUTHOR CONTRIBUTIONS

EA, LG, MP, and DH contributed to the design, review and editing of the Research Topic and to the editorial summarizing it.

## REFERENCES


treatment of acrophobia. Am J Psychiatry (1995) 152(4):626–8. doi: 10.1176/ ajp.152.4.626


Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Aboujaoude, Gega, Parish and Hilty. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Improving Reading in Adolescents and Adults With High-Functioning Autism Through an Assistive Technology Tool: A Cross-Over Multinational Study

*Arlinda Cerga-Pashoja1,2, Jorge Gaete3,4, Antoneta Shishkova5 and Vesna Jordanova6,7*

*1 Faculty of Population Health, London School of Hygiene and Tropical Medicine, London, United Kingdom, 2 Central and North West London NHS Foundation Trust, London, United Kingdom, 3 Department of Public Health and Epidemiology, Faculty of Medicine, Universidad de los Andes, Santiago, Chile, 4 Millennium Nucleus to Improve the Mental Health of Adolescents and Youths (Imhay), Santiago, Chile, 5 Parallel World Association, Plovdiv, Bulgaria, 6 Institute of Psychiatry, Neurology and Neurosciences, King's College London, London, United Kingdom, 7 Neurodevelopmental Services, South London and Maudsley NHS Foundation Trust, London, United Kingdom*

#### *Edited by:*

*Elias Aboujaoude, Stanford University, United States*

#### *Reviewed by:*

*Christi Carnahan, University of Cincinnati, United States John W. Yuen, Hong Kong Polytechnic University, Hong Kong*

> *Correspondence: Vesna Jordanova vesna.jordanova@kcl.ac.uk*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 02 January 2019 Accepted: 15 July 2019 Published: 07 August 2019*

#### *Citation:*

*Cerga-Pashoja A, Gaete J, Shishkova A and Jordanova V (2019) Improving Reading in Adolescents and Adults With High-Functioning Autism Through an Assistive Technology Tool: A Cross-Over Multinational Study. Front. Psychiatry 10:546. doi: 10.3389/fpsyt.2019.00546*

People with autism spectrum disorder (ASD) experience reading comprehension difficulties, often misinterpreting complex texts, metaphors, and idioms. We have developed and tested a new assistive technology tool for adaptive, personalized text simplification, called Open Book. This tool is an open-sourced, online platform that uses Natural Language Processing with the specific aim of assisting reading and aiding understanding of written text for people with ASD. The accessibility and effectiveness of Open Book was tested by examining the differences in text comprehension scores between the original texts and texts that were simplified by Open Book tool, randomly allocated to study participants. Two hundred fortythree participants (153 adults and 90 adolescents) with high-functioning ASD were recruited in the UK, Spain, and Bulgaria. Regarding the primary outcome, results showed that both adults and adolescents with ASD gave more correct answers for the simplified (*M* = 11.2, SD = 4.1) than original texts (*M* = 10, SD = 4.1; *p* < 0.001). This finding was consistent across age groups and countries. Regarding the secondary outcome, when participants were asked to blindly rate how easy was to understand each text, simplified texts were rated as easier (*M* = 7.6, SD = 2.4) to understand than the original texts (*M* = 8.7, SD = 2.6; *p* < 0.001). The Open Book software seems to have the potential to be a useful tool in assisting reading among people with ASD. Our findings support our primary hypothesis that texts simplified through Open Book were easier to comprehend compared to original texts.

Keywords: autism spectrum disorder, adults, adolescents, Natural Language Processing, reading

### INTRODUCTION

The autism spectrum disorder (ASD) has been recognized as the fastest growing developmental disability with 1 in 88 children diagnosed having ASD (1). People with ASD experience a range of language deficits, which have a life-long impact on their psychosocial functioning (2). These deficits include difficulties in comprehension of speech and writing, especially misinterpreting

and understanding complex instructions (3). Although individuals at the higher end of the autistic spectrum appear to have good reading abilities, several studies have shown that these individuals have difficulties in different components of written language comprehension. For instance, they fail to make inferences about social scripts and understand metaphors, which interfere with successful social communication (4). Many individuals with ASD are unable to derive the gist or meaning of written documents (5–7). Studies show that people with high-functioning ASD have excellent phonetic decoding (ability to capture the meaning of unfamiliar words by translating groups of letters back into the sounds that they represent, link them to one's verbal vocabulary, and access their meaning) but poor comprehension (6, 8, 9). Similar results were reported by Huemer and Mann (10) who compared reading accuracy with reading comprehension in a population with ASD. This study found that error patterns observed in the participants suggested that children with ASD are more focused on accurately decoding text than on preserving the meaning of the passage. This was supported in another study where readers with ASD were good at decoding sounds but had poor comprehension (11). These findings also support the evidence that the skill in both decoding and linguistic comprehension is necessary if skill in reading is to advance (12). In addition, people with ASD are not able to use their background knowledge to construct an understanding of text (13).

Traditionally, the difficulty with reading comprehension has been related to the cognitive profile of these readers especially with their problems to comprehend the perspectives of others (14). Saldana and Frith (15) have found that people with ASD have difficulty with inferences, which appear to be greater in text with social content and suggest that these difficulties may be related to mentalizing deficits and could also influence other reading processes such as referential inferences or attributions of authors' aims. Furthermore, comprehension difficulties have been associated with differences in linguistic information processing causing a negative impact in the metaphor comprehension (16).

Several problems with the pragmatic aspects of language have been found among people with ASD (16, 17). For instance, Dennis et al. (4) studied the different ability to understand pragmatic inferences about given or presupposed knowledge in mental state words. This study confirmed that children with highfunctioning ASD struggle to understand metaphors and make inferences about social scripts. These results are also consistent with those of Beversdorf et al. (18) who showed that people with high-functioning ASD recall less of emotional sentences than nonemotional ones. On the other hand, recent evidence suggest that the risk for reading comprehension difficulties is a specific characteristic of the social-communication phenotype of many high functioning ASD children and adolescents (19–22).

Although there is an abundance of research on reading difficulties for children with autism, there seems to be a considerable gap in investigation of this issue beyond adolescence. Nevertheless, a few studies that address language disorders in adults with autism indicate that, although reading accuracy improves with age in high functioning children with autism, they continue to struggle with many linguistic

phenomena such as homographs, multiple meaning words, phrases, and metaphors (10).

To the knowledge of the authors, there are no reading comprehension interventions tested among adults with autism, and there are very few studies involving adolescents. In a recent review about reading comprehension interventions for school-aged children and adolescents with ASD (23), 12 studies were identified, 3 using treatment comparison designs and 8 using single-case designs. These interventions included strategy instruction (24–27), explicit instruction (28–30), and anaphoric cueing (6, 31). None of these interventions have been tested using an experimental design or including a large sample. However, these interventions were time consuming and required a facilitator, which increased the cost of the intervention (23). The field of reading interventions for people with ASD had followed the research involving students with reading difficulties in general (32, 33), and most of the interventions tested for students with ASD have included reading expert recommendations (34). However, it seems that there is high need for research-based knowledge to enhance reading comprehension performance in people with ASD, especially among older adolescents and adults (23).

Assistive technology has been used to enhance communication and academic skills for children with disabilities (35, 36). The use of technology to teach several academic and social skills to students with ASD has a long history, since the first study reporting the use of a computer to increase understanding of how letters and sounds form words, and how texts can form expressions (37, 38). However, very few studies have explored or tested the use of assistive technology to facilitate reading comprehension among ASD subjects (39).

The assistive tool tested in this study was developed in the project FIRST (Flexible Interactive Reading Support Tool) by a multinational group of interdisciplinary researchers that involved collaboration between clinical, machine-learning, and Natural Language Processing (NLP) experts in the UK, Spain, and Bulgaria. We adapted Language Technologies resources to design a system called Open Book in three languages—English, Spanish, and Bulgarian. Further details of this project can be found in previous publications (40–42).

Open Book is a noncommercial electronic platform that can be personalized to meet and support the specific reading needs of people with autism. It uses Natural Language Processing (NLP) to make documents for people with autism more accessible. Some of the processes utilized by Open Book include the following: detection of language obstacles in the text; adding definition to terms or infrequent (rare) words; adding images to words in order to aid word visualization; providing synonyms for infrequent words; providing options to change text format (e.g., background color, text color); and "magnify" feature which highlights particular sentence to ease focusing users' attention and support when following specific text sections. This approach is supported by several studies saying that text comprehension depends on understanding words and integrating their meaning into a mental model of the text (43–45).

Open Book can convert a standard document into a personalized and simplified version, which was hypothesized that it would be easier to understand. Another feature of the platform is that it encompasses two different interfaces—for independent users with autism and for caregivers such as parents or teachers. The Open Book independent user can benefit from assistive elements using features such as "Explain word," "Explain with image," "Provide summary," or "Ask caregiver" to make the text clearer. The program also simplifies complex text structures by shortening long sentences and clarifies ambiguities. Not relying purely on textual changes, the conversion software also provides illustrative pictures to selective words and offers concise document summaries.

The interface designed for caregivers provides them with a semiautomatic program where they cannot only convert text using the NLP technologies implemented in the software but can also make their own editions to the text. They can upload images, review texts from their user's library, suggest other support if needed, and/or create new documents. All the documents are collected in the user's personal library, which can be arranged with different folders and labels. A privacy function allows the user to keep select documents private and not share them with their caregiver.

The initial software prototype was produced in English, Spanish, and Bulgarian.

The aim of this study was to assess the accessibility, utility, and the effectiveness of Open Book in simplifying complex texts by making them easier to understand for adolescents and adults with high-functioning ASD in UK, Bulgaria, and Spain.

The hypothesis was that texts simplified through Open Book would be easier to comprehend compared to original texts for participants with ASD. It was expected that, when participants were tested about written texts' comprehension, they would give more correct responses on the simplified texts compared to original (not-simplified) documents. It was also hypothesized that participants would blindly rate simplified texts as easier to comprehend compared to original texts.

By improving access of people with autism to written information, we ultimately aim to facilitate their empowerment and social inclusion. Open Book is expected to help individuals with autism to increase their independence by improving access to the wealth of textual information that is available in the information society.

#### MATERIAL AND METHODS

#### Study Design

Crossover design was used to test (46–48) the effectiveness of Open Book to improve reading comprehension among adolescents and adults with autism spectrum disorder.

#### Participants

All participants who met the following criteria were included in the study: a) a formal ICD-10 diagnosis of ASD based on diagnostic clinical interview conducted by psychiatrists or clinical psychologists; b) 12–17 years old in the adolescents branch of the study undertake in Spain and Bulgaria, and ≥18 years old in the adult branch of the study carried out in the UK and Spain; and

c) a score of ≥70 in a measure of an intelligence test confirmed by clinical records. The study exclusion criteria were as follows: a) not native speakers of the respective languages, i.e., English, Spanish, and Bulgarian; b) documented history of learning disabilities; c) additional diagnosis of dementia or other organic brain disorder that could affect memory; and d) presence of a sensory impairment that could prevent reading, writing, or hearing.

#### Ethical Approval

All study procedures were in accordance with the ethical standards of the respective institutional and/or national research committees and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Full ethical approval for the project was sought and received from each center separately.

In the UK, full ethical approval was sought and received by East of Scotland Research Ethics Service (ref: 13/ES/0059). Separate ethical approvals were also received by local Research and Development teams from each NHS site that participated in recruitment.

In Bulgaria, Parallel World received approval from the Ethical Commission of Plovdiv University St. Paisii Hilendarski. In addition, for the control group, permissions were received from the school management where the tests were conducted. Parallel World is a Registered Administrator of Personal Data according to the Bulgarian Law for Protection of the Personal Data.

In Spain, consultations were conducted following internationally accepted ethical regulations, the legal normative applicable, and the Good Clinical Practice standards (CPMP/ ICH/135/95). The guidelines of investigation compatible with those suggested by the American Psychological Association for investigations involving human participants were also followed.

The process for obtaining participant informed consent was in accordance with the REC guidance and GCP. All participants provided written informed consent. The decision regarding participation in the project was entirely voluntary. The research worker emphasized to participants that consent regarding project participation could be withdrawn at any time without penalty or affecting the quality or quantity of their future medical care, or loss of benefits to which the participant was otherwise entitled. No project-specific testing was done before informed consent had been obtained.

The informed consent forms were signed and dated by all potential participants/parents before they entered the project. The research worker explained the details of the project and provided a participant information sheet, then allowed participants to consider whether they liked to be involved in the project. The research worker encouraged the participant to ask any questions that could help them make a decision on their potential involvement in the project.

Informed consent was collected from each participant before they underwent the reading comprehension test, including history taking related to the project. One copy of the informed consent form was kept by the participant, while the other was kept by the research worker and was retained in the project Master File.

The study was granted by the FP7 EU Grant for Social Inclusion.

#### Sample Size

The sample size calculation was based on the precision with which we will be able to estimate the proportion of participants who prefer the simplified text. Based on a clinical assumption that 80% of people with ASD would prefer the simplified text, and using a confidence level of 95%, a sample of 100 participants would allow us to have 80% power to estimate the true proportion that prefer the simplified text of between 72 and 88%.

#### Recruitment

Recruitment involved active collaboration between the clinicians in the specialist clinical centers and service user and carers.

The recruitment in the UK was expanded at a national scale including several important urban areas such as Greater London, Leicester, Sheffield, and Plymouth. Majority of the participants were recruited from the National Health Service (NHS). Voluntary and charity organizations also played a very important role in reaching recruitment targets. Thus, the National Autistic Society played a major role in recruitment activity in the UK.

In Spain, the recruitment was focused in the whole province of Madrid, and it involved specialized diagnostic and treatment centers, public and private schools, centers for work mediation for people with ASD, and leisure facilities for people with ASD.

Although the autism diagnostic assessment provision in Bulgaria is sporadic, we have developed a successful collaborative work with clinical centers who have autism expertise in Sofia and Stara Zagora and Parallel World Association (charity organization) in Plovdiv.

All participant services across the three countries used identical recruitment strategy.

A researcher arranged to see the adults and the parents of children with ASD who expressed an interest in participating in the project. Consent was given by adult participants, and for children, it was obtained by their parents.

A total of 243 people who met the inclusion criteria completed the study. A detailed description of the participants is provided in **Table 2**.

#### Randomization

Reading comprehension testing that was conducted in a controlled environment under exam conditions.

One hundred fifty-three participants set reading tests in groups of 20 participants. Each participant received three simplified and three original documents. Participants were blind to text conditions. Both participants and researchers were blinded to text allocation sequence, which was block randomized by an independent researcher in the UK using a 1:1 ratio.

#### Materials

The reading comprehension tests for adults used documents that covered a range of topics: education about general and mental health, sexual health issues, newspapers articles, chapters from electronic novels, and general knowledge articles. The texts for adolescents were selected through children and young books, school material, and the Internet.

**Text selection**. Each clinical center in UK and Bulgaria identified 12 texts that were appropriate to reading abilities and interests of respective age groups (adolescents and adults). The research team in Spain identified 24 texts in total: 12 for adolescents and 12 for adults. Texts for adults were selected from comprehension test batteries used to examine reading comprehension in language proficiency, e.g., International English Language Testing System (IELTS) and Cambridge English Proficiency. All texts identified by clinical teams were inspected and analyzed by Natural Language Processing (NLP) specialists, partners in the FIRST project. NLP specialists selected 6 out of 12 texts in each language, which were matched between languages for word length, complexity, and number of obstacles. Thus, each adult text used in Spain was matched for word length and complexity with the texts used in the UK. The same was done for adolescent texts in Spain and Bulgaria.

**Text simplifications**. The original texts were forwarded to the technical teams who uploaded them into Open Book and simplified them automatically. The outcome was postedited by the clinical teams through Open Book caregiver platform. Reading obstacles and their resolutions are described in **Table 1**.

#### Measures

#### Primary Outcome: Comprehension Score

The study participants undertook a reading comprehension test under exam conditions. Multiple choice questions (MCQs)

TABLE 1 | Reading obstacles and resolutions.


were generated by each clinical team for their respective texts, with the help of technical partners' input. MCQs were selected based on the original texts so that they could tap into the general comprehension of the text's content, especially parts of the text with identified obstacles. The MCQs were the same for both original and simplified texts, and an example of two text versions followed by the MCQ is provided in **Figure 1**.

Each adult text was followed by six MCQs, and each adolescents' texts had four MCQs. This selection was done to accommodate adolescents' performance within the same timeframe as the adults.

Each center created a library of 12 texts, 6 original and 6 modified (simplified) version of original texts, while the MCQs were the same for each corresponding text. The test battery was comprised of three original and three simplified texts randomly selected for each participant. Both adolescents and adult participants were given 10 min to read each text and answer all MCQs per text.

The primary outcome was the comprehension score calculated by adding the text scores for each question. Scores from the simplified texts were compared with scores from the original texts. Adult texts were followed by six questions each. Every right answer was scored as 1, and each wrong answer was scored as 0. Therefore, each text score could range from 0 (no correct answer) to 6 (all correct answers) for adults, and 0–4 for adolescents. The overall score for original and simplified texts was calculated separately by adding the score for each of the three corresponding texts. The overall range of scoring values are 0–18 for adults (6 questions × 3 texts) and 0–12 for adolescents (4 questions × 3 texts).

#### *Secondary Outcome: Self-Reported Text Complexity*

The secondary outcome was self-reported text complexity that was measured on a Likert-type scale, where participants were asked to blindly rate how easy it was to understand each text. The scores ranged from 1 (very easy) to 5 (very difficult). Therefore, the range of scores for each text was 1–5, and overall (for three texts) 3–15. Higher subjective scores indicated self-reported higher level of comprehension difficulty, while lower scores indicated that the texts were easier to understand.

### Data Analysis

**General features**. Descriptive statistics are presented as numbers and percentages for categorical variables and means with standard deviations for continuous data.

**Primary analyses for primary and secondary outcomes**. The primary analyses tested the effectiveness of the tool using repeated measures *t*-tests for primary and secondary outcomes. The effect size using the Cohen's *d* was also calculated (49).

**Secondary analyses**. Correlation analyses were performed to assess the association between original and simplified text scores and subjective rating to test if participants were able to identify which text was original and which one was simplified. The scores of the MCQ tests were compared between the original and simplified versions of each text and between the individuals. Paired *t*-tests for analyses of comparisons between the original and simplified texts and independent sample *t*-tests for comparisons between individuals were used. Finally, univariable and adjusted regression analyses were performed to



FIGURE 1 | Example of two text versions followed by a multiple-choice question (MCQ).

assess the association between participants´ characteristics and simplified text scores.

All data were stored electronically and analyzed with SPSS.

### RESULTS

#### General Features

We invited 445 people to participate in the evaluation task, 140 of whom were excluded because they did not meet inclusion criteria, declined to participate, or did not respond to our invitation. Three hundred five people consented to participate; 11 of them dropped out and did not carry out the reading test. The main reason for the drop out was poor health on the day of the test. Two hundred ninety-four people completed the test, and all their data were analyzed. For detailed information, see **Figure 2**.

A total of 243 subjects (29%, female) participated in this study. Overall age ranged from 12 to 70 years old [adolescents, mean = 14.0 years old (SD = 2.1); adults, mean = 35.3 years old (SD = 13.1)]. The sample was predominantly male. Considering the moderately homogenic ethnic composition of Bulgaria and Spain, the sample was principally (93%) of white ethnic background.

Adult participants had higher IQ scores [109.25 ± 21.4 (75–168)] than adolescent participants [85.97 ± 13.2 (70–127)] *p* < 0.001.

A prominent characteristic of our adult participants sample is that they were well educated with just one person educated to elementary level (see **Table 2**). More than half of the sample were educated to secondary school level (55.7%) followed by graduates (35.57%), and MSc and PhD holders (4.03%, respectively). Nevertheless, although adult participants are very well educated, high percentages are unemployed, single, and do not live independently (see **Table 2**).

Psychiatric comorbidities were prevalent in our adult sample, especially depression (25.5%) and anxiety (23.5%), but no psychiatric comorbidities were identified among adolescents.

### Primary Analyses Results

#### Primary Outcome: Comprehension Score

The scores in **Table 3** indicate the summary of the results of correct answers to the MCQs for the original and simplified texts. The scores ranged from 0 to 18 for adults' texts and 0–12



#### TABLE 3 | Text score analysis.


for adolescents' texts, with each score meaning a correct answer to a question related to text comprehension. The two sets of text scores were compared through related *t*-tests. All participants had a higher score on the simplified texts than on the original texts, meaning that overall both adults and adolescents gave more correct responses for simplified texts compared to original texts. This difference was statistically significant in all groups, with the exception among adolescents in Bulgaria. When all participants were included in the analysis, difference in the scores for simplified texts (*M* = 11.2, SD = 4.1) and original texts (*M* = 10, SD = 4.1) conditions was statistically significant (*p* < 0.001, effect size = 0.3). Among different groups, the effect sizes were of medium magnitude (*d* = 0.3–0.7). These findings were also consistent across age groups. Examining age groups separately, adults performed better on questions about simplified texts (*M* = 13.3, SD = 3.3) compared to original texts (*M* = 12, SD = 3.5; *p* < 0.001, *N* = 153). Adolescents also gave more correct responses on questions about simplified content (*M* = 7.8, SD = 2.8) compared to questions about original texts (*M* = 6.6, SD = 2.6; *p* < 0.001, *N* = 90).

#### Secondary Outcome: Self-Reported Text Complexity

A similar set of analyses were performed for the participants' blind rating about text complexity. Overall, all participants blindly rated simplified texts as easier to understand than the original texts. This difference was statistically significant in all groups, with the exception among adolescents in Spain. When all participants were included in the analysis, the original text was considered more difficult to understand (*M* = 7.6, SD = 2.4) than the simplified text (*M* = 8.7, SD = 2.6; *p* < 0.001, *N* = 243). The findings were consistent for our subgroups of adults and adolescents. See **Table 4**.

#### Secondary Analyses Results

#### Association Between Text and Self-Reported Text Complexity Scores

The correlation coefficients and *p* values between the original text and subjective scores was 0.03 (*p* = 0.56) and between the simplified text and subjective scores was 0.03 (*p* = 0.67).

#### Association Between Participants' Characteristics and Simplified Text Scores

The univariable and adjusted regression analyses between participants' characteristics and simplified text scores are presented in **Table 5**. The majority of variables examined were associated with the simplified text scores in the univariable analyses. The exception was occupation and ADHD, which were not found to be significant. Female participants scored higher than male participants, with scores 1.6 units higher. Participants with higher IQ values achieved higher text scores on simplified texts. A 10-unit increase in IQ was associated with a 0.9-unit increase in text score. A higher level of education was also associated with higher outcome values. Those with university education had scores that were 6.6 units higher, on average, than those with no or only elementary education. There was little difference in scores between married and divorced/widowed participants. However, single participants had the highest scores.

In the multivariable analyses, the results suggested that higher education was significantly associated with the text scores.

### DISCUSSION

The study provides the first clinical evaluation of novel assistive technology, Open Book, that aims to assist reading comprehension

TABLE 4 | Analysis of subjective scoring.


#### TABLE 5 | Univariable and multivariable regression models.


*(\*) Regression coefficient given for a 10-unit increase in IQ; (†) Data not applicable for Spanish adolescents.*

of written texts in adults and adolescents with ASD. While this is not a reading comprehension intervention *per se*, we have found that Open Book can help convert written texts into simpler forms, which are easier to understand by people with ASD. Open Book can be used either autonomously or with the online aid of a carer or teacher, which makes the tool adaptable to different ages and levels of comprehension. Open Book is available in English, Spanish, and Bulgarian. It automatically simplifies written text by splitting long sentences; replacing metaphors, slangs, and idioms with commonly used synonyms; resolving anaphors, etc. It also has the option of replacing some complex words with pictures, which was especially used by adolescents and their teachers.

Open Book was evaluated by adults and adolescents in UK, Spain, and Bulgaria. Significant work went towards developing reading comprehension testing methodology and materials that were age specific and matched for the level of complexity across three languages.

The evaluation of Open Book indicates that adult and young people with ASD benefit from automatic text simplification. Participants in our study achieved significantly better tests' results when they processed simplified than original texts, which indicates that their understanding of the text content was enhanced when the written information was modified by the assistive technology.

The effect sizes were of medium magnitude overall, and for the adolescent sample in Spain, the effect size was large. The subjective, blind ratings of self-reported text complexity indicated in all instances that simplified versions were deemed as easier to comprehend compared to original texts.

Advanced education (university studies vs. lower education) was associated with higher text scores. We may hypothesize that reading skills improve with education, but it may be explained by having better cognitive abilities. However, other findings support the idea that individuals with ASD continue to struggle with complex linguistic phenomena (10).

There are some limitations in this study. Even though Open Books seems to have a positive impact in immediate reading comprehension of written texts, we are not able to determine if there is a longer-term effect in the reading abilities of our target group. Furthermore, we could not evaluate the effect of the use of this assistive technology in the functionality of our participants and their quality of life. Although we have demonstrated the potential benefits for high-functioning individuals, the results may not be generalizable to other people on the autistic spectrum.

### CONCLUSIONS

The study indicates that assistive technologies could be useful in supporting understanding of written text for people with ASD. The written texts simplified by the Open Book platform were significantly easier to understand by both adults and adolescents with high functioning ASD. This demonstrates a novel direction in translational autism research that opens the doors of interdisciplinary collaboration and innovation to benefit people with this disabling condition.

The next step would be to assess the feasibility of Open Book, its uptake and utility by both people with ASD and their carers in real-life conditions.

### ETHICS STATEMENT

All study procedures were in accordance with the ethical standards of the respective institutional and/or national research committees and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Full ethical approval for the project was sought and received from each centre separately. In the UK full ethical approval was sought and received by East of Scotland Research Ethics Service (ref: 13/ES/0059). Separate ethical approvals were also received by local Research and Development teams from each NHS site that participated in recruitment. In Bulgaria Parallel World received approval from the Ethical Commission of Plovdiv University St. Paisii Hilendarski. Also, for the control group, permissions were received from the School management where the tests were conducted. Parallel World is a Registered Administrator of Personal Data according to the Bulgarian Law for Protection of the Personal Data. In Spain consultations were conducted following internationally accepted ethical regulations, the legal normative applicable and the Good Clinical Practice standards (CPMP/ ICH/ 135/95). The guidelines of investigation compatible with those suggested by the American Psychological Association for investigations involving human participants were also followed.

### AUTHOR CONTRIBUTIONS

AC-P, AS and VJ designed and executed the study, assisted with the data analyses and wrote de paper. JG assisted with the data analyses and collaborated with the writing of the results and the whole paper.

### REFERENCES


### FUNDING

The study was granted by the FP7 EU Grant for Social Inclusion. This study was supported by the Central and North West London NHS Foundation Trust, with assistance from M. Keats and A. Bela. The writing contribution of the author JG was supported by Millennium Science Initiative of the Ministry of Economy, Development and Tourism, grant "Millennium Nucleus to Improve the Mental Health of Adolescents and Youths, Imhay."

### ACKNOWLEDGMENTS

We gratefully acknowledge the contribution of all the partners of the FIRST study from Wolverhampton University (Prof. R. Mitkov, Dr. C. Orasan, and R. Evans); Deletrea Spain (Sandra Freire Prudencio, Juan Martos, A. Gonzalez, and D. Gill); University of Jaen (M. T. Valdivia, E. M. Cámara, E. Barbu, and A. Ureña López); University of Alicante (P. Moreda and E. Lloret); iWeb Technologies Ltd; Kodar OOD, Bulgaria; Autism Europe (A. Baranger).


*J Autism Dev Disord* (2015) 45(5):1213–29. doi: 10.1007/s10803-014- 2280-x


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Cerga-Pashoja, Gaete, Shishkova and Jordanova. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Designing and Using Digital Mental Health Interventions for Older Adults: Being Aware of Digital Inequality

*Alexander Seifert1,2,3\*, Dominique Alexandra Reinwand3 and Anna Schlomann3*

*1 University Research Priority Program "Dynamics of Healthy Aging", University of Zurich, Zurich, Switzerland, 2 Center of Gerontology, University of Zurich, Zurich, Switzerland, 3 Rehabilitative Gerontology, Faculty of Human Sciences, University of Cologne, Cologne, Germany*

Keywords: mobile intervention, gerontechnology, seniors, digital divide, mental health

## BACKGROUND

Worldwide, approximately 15% of the people older than 60 suffer from a mental disorder, such as depression and anxiety disorders (1). A total of 322 million people worldwide are affected. The prevalence rates vary by age and peaks in older adults (2). Modern devices, such as smartphones and tablets, can be used to provide digital interventions for various health-related issues. Digital interventions are promising in their ability to provide researchers, medical practitioners, and patients with a dynamic and individualizable tool for assessing behavior and behavioral change, consultation, treatment, and integrated care. These digital interventions can help patients manage their diseases or their general health as a form of disease prevention. This is particularly important in older people, as individuals often have to deal with highly complex interactions involved in managing their daily lives along with the consequences of a multitude of chronic diseases (3). Digital interventions can, for example, assess, control, and positively influence mental health and well-being among older patients (4). Particularly with regard to mental health, digital mental health interventions (DMHIs) appear to close a gap in healthcare provision. Many patients with mental health problems have to wait a long time to get an appointment for initial counseling or therapy; in rural areas, older patients may face long travel distances, and many people are still afraid of stigmatization and avoid therapy completely (5). Reviews and meta-analyses have shown the benefits of DMHIs, for example, for people with depression and anxiety (6–8).

While DMHIs are becoming more important and popular, there exists a danger that older people will be excluded. When not comfortable with new technologies, older adults can experience barriers to accessing DMHIs, which might result in larger healthcare inequalities (9). This can happen if only already-advantaged populations use and benefit from these interventions. Therefore, this article will outline and discuss the problems in this field and make recommendations for future developments.

## DIGITAL DIVIDE ACROSS AGE GROUPS

Internet access and the usage of Internet-connected devices, such as smartphones, are becoming more widespread globally, which paves the way for DMHIs. Nevertheless, a digital gap between generations remains; older adults make less frequent use of the Internet or smartphones than do younger adults (10, 11). For example, in the United States, 67% of people 65 and older have access the Internet, whereas nearly all younger people are online (12). A representative survey across Switzerland and 16 European Union countries showed that only 49% of people aged 50 and older use the Internet (13). The study indicated that Internet use among older adults is predicted by personal factors such as age, gender, education, income, health, prior experience with technology,

#### *Edited by:*

*Elias Aboujaoude, Stanford University, United States*

*Reviewed by: Lucia Romo, Université Paris Nanterre, France*

> *\*Correspondence: Alexander Seifert alexander.seifert@uzh.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 01 April 2019 Accepted: 22 July 2019 Published: 09 August 2019*

#### *Citation:*

*Seifert A, Reinwand DA and Schlomann A (2019) Designing and Using Digital Mental Health Interventions for Older Adults: Being Aware of Digital Inequality. Front. Psychiatry 10:568. doi: 10.3389/fpsyt.2019.00568*

social salience (Internet use among the members of one's social network), and contextual factors, such as country-specific wealth and communication technology infrastructure.

Against this background, older adults are at risk of being excluded from DMHIs for the following five reasons: first, from the perspective of environmental gerontology, new technologies may contribute to a stimulating environment for successful aging (14), but older people often lack experience, skills, social support, and access to digital tools, and they may face numerous barriers to the effective use of these technologies (15). This could increase the risk that older adults perceive their digital environment as exclusionary rather than stimulating (16). Second, as mentioned above, a considerable number of older adults do not use digital technologies. Third, older, retired adults do not need to use new technologies as part of their jobs, which might reduce their motivation to adopt new technologies in their spare time. Fourth, besides an individual's age, it is also important to consider his or her level of technological socialization (17); the baby-boom generation, for instance, did not grow up with digital technologies and, therefore, they have not been socialized to use them. Finally, from a developmental perspective, people become more vulnerable as they grow older. Therefore, they have to make a greater effort to learn to use new technologies and often have to overcome the barriers arising from having fewer cognitive, physical, financial, and social resources (18).

#### CAN ALL OLDER ADULTS BENEFIT FROM DMHIS?

Despite the fact that older people use the Internet or mobile devices, such as smartphones, less frequently than the rest of the population, there is an interest among older adults in integrating new technologies into their healthcare (19). Reviews have found that DMHIs are effective in reducing depressive symptoms and stress experiences (3, 20). Nevertheless, in a study of 121 people from community mental health services, Ennis et al. (21) found that lack of skills is the reason older patients do not engage with computers or mobile devices and are, therefore, less frequent users of DMHIs. The available digital interventions are often not designed for frail and technologically unskilled older adults or for their age-related learning and usability needs (22).

While it has been reported that older participants appreciated the intervention (23), it must be mentioned that older people who dropped out of the intervention program were not included in such evaluations. Dropout rates are known to be high in Web-based interventions (24), but it has been shown that older people are more motivated to use digital health interventions than younger people (25). However, there is a lack of evaluation regarding systematical noninclusion of technical unskilled older adults within DMHIs. We assume a bias regarding inclusion of technically skilled older adults and noninclusion of technically unskilled older adults among empirical evaluation studies of digital interventions.

Research has noted that, next to the difference of access or non-access (which is known as the fist digital divide), Internet skills vary within the older age group with the oldest less skilled than the youngest (the second level digital divide) (10). Furthermore, there is a third digital divide that affects the outcomes and benefits of digital health intervention usage (26). Due to less access and less usage, older adults gain fewer benefits of those interventions. It seems that older adults have a higher risk of being more disadvantaged on all three levels than younger digital health intervention users.

Furthermore, only a few studies have focused on older participants' satisfaction and perceived usefulness of digital interventions. It is known that only a minority of interventions (8%) considered the specific needs of older people during the development and design process (23), so it can be assumed that most digital interventions are designed for advanced users and neglect unskilled older people. These tendencies have been described as the innovativeness-needs paradox, which means that individuals who objectively need a given innovation the most are the ones least likely to adopt it (27).

#### RECOMMENDATIONS

Given the rapid expansion of digital interventions, it seems worthwhile to educate older adults in how to use DMHIs that could be useful in their daily lives. It would be helpful to offer support and training to these people to increase their self-efficacy and digital literacy skills. Learning new technological skills can even foster a certain sense of competence and autonomy (28) that can encourage the efficient use of a digital intervention. These learning tools can be generally provided by adult educational services, such as senior universities or adult education centers, but should especially be offered prior to intervention participation through the provider of the intervention. The special learning needs of older adults need to be considered in these educational services (29), with attention paid to things such as the tempo of the learning session and the technological skill background of the older participants.

In the literature, there is an assumption that learning tools will alone suffice to increase the use of DMHI, but this is hard to prove because there are only a few studies available that have tested the effect of training on intervention usage and outcomes (30). A scoping review, however, suggests that usability and technical problems, lack of value of an intervention, and insufficient training are among the most important barriers facing older adults in using digital interventions (19). These findings suggest that in the future, in order to reduce age-related inequalities in digital interventions, more should be invested in training, education, and support to increase participation among older adults and decrease dropout rates.

Older adults often use technology selectively and in unexpected ways. They often develop their own digital skills and strategies. Research on aging an technology should consider this usage behavior as legitimate and not as mistakes or wrong usage because it helps to understand the role of technology in older adults' everyday lives (31). Therefore, it is crucial to motivate developers and professional users (e.g., researchers, medical practitioners, and companies within the health sector) of DMHIs to take a closer look at how different designs and content can be tailored in a way that encourages trust and facilitates use among older people. This requires a development of DMHIs tools (software and hardware) that include the needs of older users (15, 29). It is known that taking end users into account increases the usage and effectiveness of interventions (32). For example, Darvishy et al. (33) developed a brochure for age-appropriate mobile applications that is grounded in the recommendations of the W3C Web Accessibility Initiative (34) and focuses on older adults' needs for an accessible and useful application. The applications should, for example, be presented in a clear, intuitive, self-explanatory, transparent, and consistent way, and navigation within the application should be logical, with steps communicated clearly and kept to a minimum.

Therefore, it is beneficial to involve older adults as part of participative research before developing a new digital intervention. After developing such an intervention, it is also crucial to invest time in educating the participants regarding all aspects of the intervention before they begin to use it. Furthermore, during the intervention, a reachable support hotline and contact partner can be used to assist the older participants when needed. Finally, after an intervention, the daily challenges in using it as reported by the participants can be used as important input for further development and indicator for sustained long-term use.

#### REFERENCES


### CONCLUSIONS

DMHIs offer unique and innovative opportunities for older adults with mental disorders, such as depression or anxiety. Developers and medical practitioners in the mental health field can use the advantages of digital tools to provide older adults with a helpful instrument. However, the approach also brings challenges, especially when technologically unskilled older adults cannot benefit from DMHIs because of their lack of access or digital skills. On the one hand, researchers must be aware that findings of the usefulness, acceptance, and effectiveness of DMHIs might be very limited and biased when certain groups of older adults are not included. On the other hand, older adults are not generally technology-averse and already use a number of technologies as part of their everyday life; although sometimes differently than younger generations. Developers, practitioners, and researchers in this field must be aware of this digital inequality and provide training tools and support services, in addition to developing digital interventions that consider the background knowledge and needs of older people, who are not "digital natives" (35).

### AUTHOR CONTRIBUTIONS

All authors provided substantial contributions to this article from conception to final approval and share the same opinion.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Seifert, Reinwand and Schlomann. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Internet-Based Interventions for the Prevention and Treatment of Mental Disorders in Latin America: A Scoping Review

*Álvaro Jiménez-Molina1,2,3, Pamela Franco4, Vania Martínez1,2,5, Pablo Martínez1,2,6,7, Graciela Rojas1,2,3,7 and Ricardo Araya1,8\**

*1 Millennium Nucleus to Improve the Mental Health of Adolescents and Youths (Imhay), Santiago, Chile, 2 Millennium Institute for Research in Depression and Personality (MIDAP), Santiago, Chile, 3 Millennium Nucleus in Social Development (DESOC), Santiago, Chile, 4 Programa de Doctorado en Psicoterapia, Facultad de Ciencias Sociales and Facultad de Medicina, Pontificia Universidad Católica de Chile and Universidad de Chile, Santiago, Chile, 5 CEMERA, Facultad de Medicina, Universidad de Chile, Santiago, Chile, 6 Escuela de Psicología, Facultad de Humanidades, Universidad de Santiago de Chile, Santiago, Chile, 7 Hospital Clínico, Universidad de Chile, Santiago, Chile, 8 Centre for Global Mental Health, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom*

#### *Edited by:*

*Elias Aboujaoude, Stanford University, United States*

#### *Reviewed by:*

*Stephan T. Egger, University of Zurich, Switzerland Michaela Pascoe, Victoria University, Australia*

*\*Correspondence: Ricardo Araya ricardo.araya@kcl.ac.uk*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 10 June 2019 Accepted: 19 August 2019 Published: 13 September 2019*

#### *Citation:*

*Jiménez-Molina Á, Franco P, Martínez V, Martínez P, Rojas G and Araya R (2019) Internet-Based Interventions for the Prevention and Treatment of Mental Disorders in Latin America: A Scoping Review. Front. Psychiatry 10:664. doi: 10.3389/fpsyt.2019.00664*

Background: There is a huge gap in the treatment of mental disorders in Latin America, especially among socioeconomically disadvantaged groups. Given the sharp increase in Internet access and the rapid penetration of smartphones in the region, the use of Internet-based technologies might potentially contribute to overcoming this gap and to provide more widely distributed and low-cost mental health care in a variety of contexts.

Methods: We conducted a scoping review of the literature in order to systematically map the existing evidence on use of Internet-based interventions for prevention, treatment, and management of mental disorders across Latin American countries, as well as to identify existing gaps in knowledge. Six electronic databases were searched for published papers (PubMed, Embase, CINAHL, Web of Science, SciELO, and CENTRAL).

Results: After the eligibility assessment, we identified 22 Internet-based studies carried out in Latin America for prevention, treatment, education, or facilitating self-management of mental disorders. Included studies mainly targeted depression (*n* = 11), substance misuse (*n* = 6), anxiety (*n* = 3), and mental health literacy for education and health professionals (*n* = 2). Most studies were undertaken in Brazil (*n* = 6), Mexico (*n* = 5), and Chile (*n* = 4). Only 3 studies were randomized controlled trials (RCTs), 4 were pilot RCTs, and 15 were naturalistic, acceptability, or feasibility studies. The three RCTs identified showed disparate results, but overall, there are challenges to face. Better results are seen in the short-term (postintervention or after 3 months), but most studies do not explore outcomes for long enough (follow-up after 6 or 12 months). Most of the feasibility and pilot studies showed reasonably good acceptability for a wide range of strategies but difficulties to engage and retain participants for long enough or adhering to established protocols.

Conclusion: This study shows that Internet-based interventions for the prevention and treatment of mental disorders are growing rapidly in Latin America, but there are few

studies on effectiveness and cost effectiveness, making it difficult to provide the evidence needed to justify scaling up these interventions.

Keywords: Internet, technology, telepsychiatry, mental disorders, prevention, treatment, Latin America

#### INTRODUCTION

Mental disorders are common in Latin America. For instance, depression is the largest cause of disability in the region, and mental health problems represent one-third of all years lost for disability in Latin American countries (1). A striking imbalance exists between government spending on mental health and the related disease burden (2), and the treatment gap for mental disorders in Latin America is huge with ~80% of those suffering from depression receiving no help at all (1). This is even more marked among socioeconomically disadvantaged groups (3, 4).

Among many reasons for this situation, the lack of trained personnel figures prominently. To address this problem, the strategy of task shifting/sharing has been implemented, in which nonmedical health workers deliver most, if not all, of the treatment for mental disorders (5, 6). A model developed in Latin America became the first task-shifted treatment program to show its cost effectiveness from low-middle income countries (LMICs) (7, 8), and it was subsequently successfully adapted and replicated in other LMICs (9, 10). Although this is a promising strategy, it has become increasingly clear that nonmedical health workers have an increasing number of competing demands, and other barriers, such as access to health care clinics and stigma, still pose major hurdles to improve coverage in the region and elsewhere. Expanding task shifting to include the use of technology to either complement or even replace health workers might potentially contribute to overcoming some of these obstacles and reducing the treatment gap.

There has been a rapid growth in the development of Internetbased interventions for preventing and treating mental health problems in developed countries (11). For instance, approximately one-third of all health-related apps are on mental health topics (12). Nonetheless, there are still a number of challenges to be faced by this rapidly growing industry, in particular those related to the evaluation of these products and interventions affecting countries at all levels of development (12, 13).

Given the sharp increase in Internet access and the rapid penetration of smartphones in Latin America and elsewhere (12), the integration of Internet-based interventions may offer a potential to provide more widely distributed and low-cost mental health care (12–14). Users have the opportunity to access help, often based on proven effective clinical guidelines, at a convenient time and wherever they are, saving precious time and unnecessary transport costs. Internet-based programs and digital technologies have also the potential to be important tools for disseminating sustainable training programs and to support and supervise mental health workers (11, 14). In spite of all these merits, there has been limited use of digital advances in mental health in LMICs, and some questions remain about their effectiveness in less-developed countries and settings with restrained resources (11). Previous reviews have found virtually no studies on Internet-based solutions in Latin America that can be scaled up in real health-care contexts (15–17).

We conducted a scoping review of the literature in order to systematically map the existing evidence on use of Internet-based interventions for mental disorders in Latin America, as well as to identify existing gaps in knowledge. The following research question was formulated: What is known from the literature about Internet-based interventions for the prevention, treatment, or management of mental disorders across Latin American countries? We consider how Internet-based interventions could overcome barriers to improving access to mental health care in these countries, and we evaluate the main strengths and limitations of the interventions developed in the region.

#### METHOD

We carried out a systematic search of the literature on Internetbased studies in mental health undertaken in Latin America. While our initial intention was to do a systematic review, we decided to do a scoping review because potentially eligible studies used diverse study designs (many studies were pilot, acceptability, or feasibility studies), recruited diverse population groups with a wide range of mental disorders, and reported heterogeneous outcomes. We use the PRISMA Extension for Scoping Reviews (PRISMA-ScR) to conduct the review and report the results (18). A systematic review would have required greater consideration of intervention effectiveness (19), whereas scoping reviews are useful for answering much broader questions. Likewise, a scoping review allowed for greater discussion of important areas in which we believe that Internet and digital technologies could yield considerable gains towards addressing mental disorders in Latin America.

#### Eligibility Criteria

We searched for published papers in peer-reviewed journals, written in English or Spanish, indexed from inception to April 5, 2019. The following study eligibility criteria were applied, according to the Population, Intervention, Comparison, Outcomes, and Study design (PICOS) framework (19):

(P) Individuals living in Latin America, without distinction of age, sex, or ethnicity. The scope of this review allowed for the inclusion of subjects experiencing varying degrees of psychopathology, determined either through validated self-reported scales, observerrated questionnaires, or clinical interviews. For instance, in the case of interventions aimed at the promotion of mental health and/or prevention of mental disorders, individuals may have been identified as healthy or at-risk of mental ill health. We did not exclude any type of mental health problem, as such, we included common mental disorders/symptoms (e.g., depression and/or anxiety), serious mental illnesses (e.g., schizophrenia or bipolar disorder), and alcohol and other substance use disorders. Moreover, we also considered for inclusion those studies in which participants were mental health care providers, whether they were lay health care-workers, primary health-care personnel, or specialized mental health-care professionals.

(I) Internet-based interventions aimed at the promotion of mental health and the prevention and/or treatment of mental disorders, targeted at healthy, at-risk, or mentally ill individuals. Additionally, we included supporting and/or training Internetbased interventions for mental health-care providers. For purposes of this scoping review, interventions must have been based on or supported by Internet and digital technologies (i.e., web and mobile based), with varying degrees of human support. Importantly, these interventions may have been provided alongside other treatment components, regardless of their technological status (e.g., as in the case of blended interventions, which combines face-to-face and Internet-based approaches).

(C) Any intervention that is not based on Internet or digital technologies was considered as a comparator or control. However, studies without comparison/control groups were also considered.

(O) To be included, studies must have used validated selfreported scales, observer-rated questionnaires, or clinical interviews and must have provided postintervention assessments or follow-up evaluations. We do not consider specific restrictions regarding the length of follow-up periods. Outcomes considered included measures of positive mental health, mental disorders or symptoms, information or attitudes (e.g., knowledge) related to mental illness, and use, adherence, or satisfaction with health services.

(S) We did not exclude studies by their design; rather, the rationale for the inclusion of studies was based on studies' aims. Thus, we identified two study types of interest to our scoping review: feasibility studies, dealing with issues such as the acceptability, demand, and practicality of interventions (i.e., "Can it work?"); and efficacy/effectiveness studies (which addresses the question: "Does the intervention work?"). This meant that we expected to include quantitative, qualitative, and mixed methods studies in order to consider different aspects of Internet-based interventions.

Papers were excluded if they did not fit into the conceptual framework of the study.

#### Information Sources and Search

To identify potentially relevant documents, the following bibliographic databases were searched: PubMed, Embase, CINAHL, Web of Science, SciELO, and CENTRAL. The appendix shows the complete search strategy used. The search results were exported into EndNote, and duplicates were removed. In addition, we searched reference lists of included studies and other systematic reviews.

Included studies evaluated Internet-based interventions targeting depression, including symptoms and disorders; serious mental illnesses, including schizophrenia or bipolar disorder; alcohol and other substance-misuse disorders; and other common mental health conditions, including anxiety disorders and posttraumatic stress disorder. The included studies considered intervention involving the use of Internet and digital technologies, such as telepsychiatry, webbased, wearable devices, and mobile applications.

We excluded discussion articles, program descriptions, study protocols, and conference abstracts.

#### Selection of Sources of Evidence

To increase consistency among reviewers, three reviewers (ÁJ-M, PF, and PM) carried out study selection and data collection in an independent manner. These three reviewers evaluated the titles, abstracts, and then full text of all publications identified. The study selection process was documented using the PRISMA-ScR flow diagram (18).

Data were extracted and synthetized for country of origin, sample description, study design, intervention description, and primary outcome. Three authors (RA, VM, and GR) reviewed the list of included studies and the data included in the summary tables to confirm accuracy in data extraction. All authors reviewed the final tables.

#### Risk of Bias Across Studies

A key difference between systematic and scoping reviews is that the latest are generally conducted to provide an overview of the existing evidence regardless of methodological quality or risk of bias. However, we performed a risk of bias assessment of included randomized controlled trials (RCTs) according to the criteria of the Cochrane Collaboration tool for assessing risk of bias (19). Two reviewers (ÁJ-M and PF) carried out risk of bias assessment in an independent manner.

#### Synthesis of Results

We grouped the studies according to the type of design: randomized controlled trials, pilot randomized controlled trials, and naturalistic, acceptability or feasibility studies. Narrative techniques were the selected approach for data analysis and synthesis, with due emphasis on study characteristics and risk of bias assessment to interpret study results. We also use comparative tables to facilitate the analysis of study characteristics and results.

### RESULTS

After duplicates were removed, a total of 1,375 records were screened. Based on the title and the abstract, 1,283 were excluded, with 92 full text to be retrieved and assessed for eligibility. Of these, 69 were excluded for the following reasons: 18 did not directly evaluate an intervention, 16 did not specifically address mental health issues, 20 were not Internet-based interventions, 6 were not implemented in Latin America, and 3 reported duplicate results in other publications. We also excluded six studies because they were reported in conference abstracts. After the eligibility assessment, we identified 22 Internet-based studies carried out in Latin American countries for the prevention, treatment, education, or facilitating self-management of mental disorders considered eligible for this review (see **Figure 1**).

### Characteristics of Sources of Evidence

Most included studies were undertaken in Brazil (*n* = 6), Mexico (*n* = 5), and Chile (*n* = 4). We found four multisite studies conducted in several countries in South and Central America. Included studies mainly targeted depression (*n* = 11), substance misuse (*n* = 6), anxiety (*n* = 3), and mental health literacy for education and health professionals (*n* = 2).

The studies included in this review were classified and grouped according to the research design (RCTs, pilot RCTs, and other studies) and the type of intervention: telepsychiatry/ psychology, guided Internet-based self-help programs, unguided Internet-based self-help programs, and Internet-based programs for education and training of health/education workers.

Telepsychiatry/psychology interventions reported here use the Internet to allow simultaneous and/or time-delayed communication between a patient or primary health-care professional and a mental health specialist (psychiatrist/ psychologist) *via* video conferencing, chat, or other electronic platform. These interventions can enable mental health specialists to provide remote services or supervise nonspecialized care teams. Guided Internet-based self-help programs were automated interventions that included some kind of human support but considerably less than in face-to-face therapy. Unguided Internet-based self-help programs were automated interventions with no human interaction at all. Finally, Internetbased programs for education and training of health/education workers were interventions using Internet to access mental health knowledge (e.g., e-learning) to professionals who may have a role in preventing, diagnosing, or managing mental disorders.

We identified three RCTs testing the effectiveness of Internetbased interventions: a web-based mental health training program for school teachers (20), an unguided web-based alcohol intervention (21, 22), and a remote collaborative care program (23). Four included studies were pilot RCTs. One evaluated the efficacy of a videoconference for mental health care delivery in clinical settings (24), and the others studied the use of selfhelp Internet-based programs for the prevention or treatment of mental disorders, with or without support (25–27).

We also identified 15 naturalistic, acceptability, or feasibility studies. Three studies used Internet-based technologies to support mental health care delivery in clinical settings (28–30) and six studies that used guided self-help Internetbased programs to reach individuals with mental disorders outside clinical settings, promote treatment, or medication adherence, and provide ongoing encouragement and targeted psychosocial support (31–36). Likewise, we found four unguided Internet-based self-help programs for mental disorders and alcohol misuse (37–40) and two interventions oriented to provide basic mental health education to healthcare workers (41, 42).

The main characteristics of each study are presented in **Tables 1**, **2**, and **3**, including population setting, aims of interventions, digital technology used, and main outcomes.

#### Results of Individual Sources of Evidence Randomized Controlled Trials

In Brazil, a web-based educational program was designed to educate primary school teachers on the recognition and classroom management of children with mental health problems through interactive tutorials, educational videos, and an online discussion forum with mental health experts (20). Teachers receiving this web-based program had greater gains in knowledge about mental disorders such as depression and conduct disorder, and less stigmatized views on mental disorders, than did those trained with a text- and video-based program and those receiving no training. These results suggest that web-based interactive tools can be more effective than traditional educational tools in increasing knowledge and reducing stigma. However, in spite of increased knowledge, the impact on improving recognition and referral could not materialize.

In addition, in Brazil, a web-based intervention aimed at decreasing alcohol consumption and/or preventing alcohol abuse among nightclub patrons demonstrated significant reductions in reducing scores on the Alcohol Use Disorders Identification Test (AUDIT) and the prevalence of binge drinking over time in a "high-risk" group (21, 22). However, in a "low-risk" group, AUDIT scores increased in those receiving the intervention and a control group, and there were no differences in the prevalence of binge drinking across groups or compared with baseline levels.

In Chile, an online platform facilitated collaborative care by giving primary care providers remote access to a psychiatrist who supported the treatment of adolescent depression (23). The intervention proved to be feasible and well accepted by both patients and primary care clinicians. Satisfaction with psychological care, in both groups, was related to a significant change in depressive symptomatology at 12 weeks follow-up. However, at follow-up, the intervention did appear to have equivalent effectiveness to reduce depressive symptoms compared with enhanced usual care.

#### Pilot Randomized Controlled Trials

In Brazil, monthly telepsychiatry consultations through videoconferencing for the care of people with depression seem to achieve similar clinical outcomes as standard face-to-face treatment regarding mental health status, satisfaction with treatment, treatment adherence, or medication compliance (24). An interesting result is that participants in the videoconferencing group were able to establish an equivalent therapeutic relationship as those treated in person. However, all these results need to be taken with caution, as this was a feasibility study not statistically powered to test any of these differences.

In Mexico, two studies used guided self-help Internetbased programs. One study explored the feasibility of an Internet-based psychoeducation program for social anxiety (25). A module of the program incorporated cognitive– behavioral techniques such as cognitive restructuring and exposure to scenarios depicting real audiences in publicspeaking situations through videos. The results supported the feasibility of this program and showed a tentative clinical improvement in measures of anxiety. A relevant aspect to consider is the difference in attrition rate across groups, with the group assisted by therapist showing the lowest (27%) while the group without treatment the highest (45%). In the self-applied group, it seems that some of the attrition can be explained by problems with accessing Internet and achieve the necessary immersion in the exposure scenarios.

Another study developed a web-based cognitive– behavioral intervention (CBT) for the reduction in substance use and depressive symptomatology (PAADD) and compared it with two other CBT brief interventions: an in-person session with an addiction therapist combined with ASSIST Self-help Strategy Guide and the treatment ordinarily offered at participating treatment centers (26). The results suggest that CBT strategies were successfully translated to web format in the PAADD, with possibilities similar to those of brief interventions carried out in person and with printed materials. This intervention appears to be feasible and shows a potential reduction of participants' depressive symptomatology, although it was not demonstrated to be more effective than the other interventions. Regrettably, the intervention presented problems associated with the high dropout rate.

Finally, a multisite study implemented an unguided Internetbased self-help program for the prevention of postpartum depression in different Latin American countries and the USA (27). Although this study showed that online self-help programs could be delivered and promote mental health among individuals in low-resource settings, it showed high dropout rates and failed to suggest that results could potentially be beneficial. In this respect, it seemed that benefits of receiving the intervention were greater for pregnant women reporting high (vs. low) levels of prenatal depression symptoms.

#### Naturalistic, Acceptability, or Feasibility Studies

#### Telepsychiatry/Psychology

In Chile, an online platform facilitated collaborative care by giving primary care providers remote access to a psychiatrist (28). The intervention was feasible and achieved higher user satisfaction. This program also shows better treatment adherence rates at 6 months compared to usual care, suggesting that technology-assisted interventions may help rural primary care teams in the management of depressive patients. While this is not a randomized clinical trial, the potential effectiveness of this intervention seems comparable to the usual care delivered by a national depression treatment program.

In Mexico, a 16-session telepsychology treatment for depression (via chat, audio, or videoconference) seems potentially beneficial in decreasing depressive symptomatology, with gains remaining at 6-month follow-up (29). However, the quality of the study and the small sample size (*n* = 8) do not allow many conclusions to be drawn from this intervention.

In Colombia, a telepsychiatry intervention for depression management among prison inmates was compared with TABLE 1 | Description of randomized controlled trial (RCT) studies of Internet-based interventions for mental health in Latin America.


mental disorders.

#### TABLE 2 | Description of pilot randomized controlled trial (RCT) studies of Internet-based interventions for mental health in Latin America.


showed high attrition rate.

TABLE 3 | Description of naturalistic, acceptability, and feasibility studies of Internet-based interventions for mental health in Latin America.


Interventions for Mental Disorders in Latin America

*(Continued)*

Frontiers in Psychiatry

 |

www.frontiersin.org


Jiménez-Molina et al.

*(Continued)*

Interventions for Mental Disorders in Latin America


*(Continued)*


asynchronous teleconsulting with a psychiatrist through a webbased platform where the primary care professional present clinical information to the psychiatrist (30). Both interventions showed depression symptomatology reduction, but the asynchronous teleconsulting intervention proved to be less costly than the telepsychiatry intervention.

#### Guided Self-Help Internet-Based Programs

Two pilot studies from Chile showed acceptable online depression treatment programs, including an online video game for supporting depression treatment among adolescents (31) and an online program for symptom monitoring in patients receiving treatment for major depression (32). The online adventure video game is a psychotherapeutic tool for female adolescents in psychotherapy with mild to moderate depression (31). In the game, players follow the story of a female adolescent, who gets involved in interpersonal situations that require psychosocial reasoning. The majority of participants (patients and therapist) valued the game and considered that they could obtain mental health-related benefits from playing it. However, the time of use of the game was very low. Therapists also suggest that it was possible in postgame sessions to relate elements of the game to aspects of the patients' real lives. Nevertheless, the design of the study and the small number of participants make it impossible to draw conclusions regarding the effectiveness of the game.

Espinosa et al. (32) studied the feasibility and acceptability of the Chilean version of the Supportive Monitoring and Disease Management over the Internet program (SUMMIT) (43), which was called in Spanish ASCENSO. This program aims to monitor and support patients after being discharged from depression treatment. Most of the participants displayed a good level of acceptance and generally regarded the program as a source of support and as beneficial; however, only half of them actively used the program and/or the online chat available. In Colombia, where the ASCENSO program was replicated, the results showed also a favorable acceptability from the participants, but there were important problems with usability and attrition (33). One of the main limitations of its implementation was associated with the difficulty of involving mental health institutions in the use of the program.

In Mexico, a study aimed to explore the feasibility of a program aimed to people who were victims of assaults, kidnappings, and criminal violence and suffered from posttraumatic stress disorder (PTSD) and/or acute stress disorder (ASD) (34). Participants were exposed to Virtual Reality scenarios and asked to talk about the traumatic event in the first person. The results showed that treatment was well received and potentially useful in reducing PTSD and ASD symptoms from pre- to posttreatment. Regarding the satisfaction of participants with the intervention, it is interesting to note that they did not manifest a preference between virtual reality exposure techniques and traditional exposure therapy (*in vivo* or imagined).

A study conducted in Nicaragua, Guatemala, and Costa Rica developed an Internet-based education program to promote adolescent health, integrating the program in existing organizational curricula and activities (35). This program was delivered in an interactive (requiring feedback) and/or noninteractive fashion (information only). The program proved to be feasible and provided a confidential way for youth and family development organizations to gather information about teenagers' health behaviors and needs. Nevertheless, access and integration of materials and methods into existing curricula was a major problem, possibly associated with limited access to computers and the Internet in general.

A study conducted in Brazil and Peru developed a psychoeducational technological platform delivered *via* mobile phones to patients and assisted by a nurse holding a tablet dashboard to monitor progress in the reduction of depressive symptoms among individuals with comorbid diabetes or hypertension (36). The platform (CONEMO) aims at increasing daily life activities and motivation, as well as providing further information and health self-care messages. This tool demonstrated to be feasible in both settings, where three pilot studies were conducted, and participants were satisfied with the intervention. The samples were small to test the efficacy of the intervention. However, there was a trend in all pilot studies for a reduction in depressive symptoms over time. A fully powered RCT is in progress.

#### Unguided Internet-Based Self-Help Programs

In Mexico, a study collected data on individuals who entered a Mexican open access free web-based psycho-education and CBT intervention for depression (37). Data showed that adherence dropped considerably as individuals progressed through the intervention modules. However, all modules were rated very high for helpfulness/usefulness.

In Brazil, a program seemed beneficial for reducing alcohol consumption among harmful users and those with probable dependence, but program adherence was low (38). In Uruguay, a web-based substance misuse prevention intervention was implemented in 10 private schools (39). This study shows that sending participants periodic reminders *via* e-mail and SMS text messages has a positive impact in engagement with the program. Despite the adolescent-friendly design and the provision of social networking tools and interactive dynamics, low participation and high attrition were observed during the intervention. In Colombia, participants with a flying phobia that went through a program for treating their phobia with psychoeducation and exposure techniques reported high satisfaction with the intervention (40).

#### Internet-Based Programs for Education and Training of Health/Educational Workers

In Brazil, a tele-education program consisted of weekly web conference seminars and moderated discussion forums to provide education to family health-care teams about mental health (41). While participants felt that the seminars contributed to their professional development, the implementation of the tele-education program faced significant obstacles. Two major challenges were associated with Internet connectivity and the insertion of new technologies into the daily lives of health professionals, especially physicians.

In addition, in Brazil, an online course was oriented to enhance primary care professionals' knowledge about the clinical management of alcohol misuse (42). In this study, health-care workers expressed satisfaction with the course, and a significant improvement in their knowledge about the clinical management of alcohol-related problems was observed. This indicates that e-learning is a useful medium for teaching mental health issues. However, this intervention also showed a number of difficulties. First, there was no reduction in stigma and prejudice related to alcohol problems. Second, the comparison between pre- and postcourse scores suggests that general knowledge about alcohol addiction did not improve over time. Finally, the lack of a control group did not allow the performance of e-learning to be compared with traditional face-to-face teaching.

#### Risk of Bias Across Studies

**Table 4** presents a synthesis of the risk of bias assessment of the three included RCTs.

The first included RCT (20) describes the use of random sequences to generate the allocation and adequate concealment of allocation prior to assignment. In this type of intervention, it does not seem feasible to blind participants and personnel. Thus, the risk of bias for this item is described as "high." The publication does not describe the measures used to blind outcome assessors to treatment allocation, so a high risk of detection bias can be inferred. This study describes the outcome data and the number of participants in each group. While this study included replacing missing data through imputation methods, there is a lack of information regarding the specific reasons for attrition. Thus, we assess the risk of attrition bias as "unclear." The publication includes all prespecified results.

The second included RCT (21, 22) describes the use of random sequences to generate the allocation and adequate


*Low risk of bias:* ✓ *| High risk of bias: × | Unclear risk of bias: ?*

concealment of allocation prior to assignment. Participants were probably aware of allocation when they received feedback on their level of risk, so performance bias was identified during the conduct of the study. The outcomes were evaluated through self-report scales, eliminating observer bias. However, lack of information made it difficult to assess the blinding of outcome assessment, resulting in an "unclear" judgment on the risk of detection bias. This study describes the outcome data, reporting the reasons for attrition and exclusions, as well as the number of participants in each group. The publication includes all prespecified results.

The third RCT identified (23) describes that randomization was performed using computer generated random numbers with adequate allocation concealment. However, the participants knew the allocation. A trained consultant, who was blinded to the treatment allocation, assessed patient baseline data and outcomes at 12 weeks of follow-up. The study reports the reasons for attrition and exclusions, as well as the number of participants in each group. The publication includes all the prespecified results.

#### Synthesis

Overall, the studies identified in this review are heterogeneous in terms of participants (e.g., adolescents, adults, patients, professionals), contexts (rural and urban, clinical and community settings), mental problems addressed (depression, anxiety, alcohol and substance misuse), and methods used to deliver the interventions (e.g., teleconsulting, online selfhelp programs, education and training of health/education workers, psychoeducation).

Within the 22 studies identified, only 3 studies were RCTs, 4 were pilot RCTs, and 15 were naturalistic, acceptability, or feasibility studies. Most of the feasibility and pilot studies showed reasonably good acceptability for a wide range of strategies but difficulties to engage and retain participants for long enough or adhering to established protocols. We found no large-scale effectiveness studies and no cost-effectiveness study. The methodological quality of RCTs studies identified was reasonably good but showed disparate results, and there are challenges to face. Overall, better results are seen in the short term (postintervention or after 3 months of follow-up), but most studies do not explore outcomes for long enough (follow-up after 6 or 12 months).

#### DISCUSSION

Internet-based interventions for mental disorders have already shown their potential benefits in high-income countries. The use of the Internet and digital technologies may improve access, enhance the flexibility of conventional treatments, facilitate the monitoring of treatment progress and fidelity with which interventions are delivered, and may improve the integration of different levels of care, obtaining results comparable to face-to-face care (11, 14, 44). However, much less is known about the feasibility and potential benefits of these interventions in LMIC. Although the penetration of technology and its application to health has been swift in all countries regardless of their level of development (12), there are many issues that remain to be resolved if these technologies want to be disseminated at scale. Among these, the relative lack of studies showing effectiveness of Internetbased interventions for mental disorders in LMICs (15, 16). Some questions still remain about the effectiveness of these interventions in settings with constrained resources.

In this scoping review, we identified 22 primary studies addressing Internet-based interventions for mental disorders across Latin American countries published until April 2019. Our findings indicate that there is a growing number of studies testing the feasibility and acceptability of Internet-based interventions for prevention, treatment, education, or facilitating self-management of mental disorders across various settings. However, very few tested the effectiveness of these interventions through RCT designs. This is not an issue just affecting Latin America but the field in general (12).

The studies identified in this review are heterogeneous, demonstrating the flexibility of Internet-based interventions to adapt to different population, contexts, and formats. Many of the studies have prioritized the use of online technologies to assist vulnerable groups in low-resource settings without considering the challenges involved in their use and in future implementation. As a reviewed study shows (34), in some regions of Latin America, especially Central American countries, there is still limited access to computers and connectivity with the Internet, let alone a chronic shortage of supervisory human resources, all of which can become major barriers to the successful implementation of Internet-based interventions. Similarly, a study conducted in Brazil showed significant obstacles to implementing at larger scale a teleeducation program not only because of low connectivity but also because of the challenge of introducing new technologies in the daily work of busy practitioners in public health contexts (41). Future studies should pay more attention to these potential challenges when it comes to bring innovation to scale; otherwise, the solutions developed and tested at high cost will never be scaled up.

Most studies had methodological limitations such as poorly defined samples, unclear comparison groups, and lack of randomization methods, and interventions of short duration. One of the most important problems identified is that most of these studies compared results across groups in terms of efficacy, even though none of them was statistically powered to estimate differences across groups. Notwithstanding this, most studies showed a trend suggesting increased benefits when compared to control groups, whenever these had been included in the study design. Caution must be exercised when presenting results of feasibility or pilot studies. A common finding in mental health interventions is that early gains tend to fade away unless there is continuous intervention over time; most studies included were short-term. There is a need for studies that explore outcomes beyond six months; however, it is acknowledged that funding for longer lasting feasibility studies is hard to get.

Some interventions were unguided, but most of them included guidance or support that varied from low-intensity support (feedback and motivation) to using the technology as a means to deliver synchronous assessments, therapy, or supervision (telepsychiatry, telepsychology, or teleconsultation through videoconference). Most interventions adopted evidence-based techniques of traditional face-to-face treatments, which indicate that there is still ample scope for innovation in Internet-supported techniques. There is growing awareness that guided intervention tends to achieve better results, but there is still not enough knowledge in terms of the most cost-effective options to provide this guidance and the options to innovate in this respect are endless. In the case of Latin America, it is possible to explore various options to adopt the "blended approach," since there is still a large supply of reasonably trained health workers who can assist in this process. However, resources are finite, and there are competing duties that need to be accounted for when planning for human guidance and support. Guided Internet-therapies have the potential to improve effectiveness and reach of psychological support and treatment for mental disorders in developing countries (16), without disproportionately increasing costs for health services.

The lack of specialized human resources is a critical issue in delivering mental health care in LMIC (5, 6). The training of nonspecialist professionals (primary health-care providers, or community health-care workers) with tools to diagnose and detect mental disorders is an important area of development that aims to reduce the treatment gap. As some reviewed studies show, the use of the Internet provides an option to overcome this problem, by delivering online training on mental health for primary healthcare professionals and/or providing remote communication between them and mental health specialists. This can be fundamental in task-shifting strategies from specialist medical providers to less welltrained personnel. However, as this review shows, further research is needed in this area, especially on how to change attitudes towards mental disorders.

Internet-based interventions for mental health identified in this review go beyond clinical settings. A few identified studies targeting school populations, both students and teachers. The results of a web-based educational program included in this review (20) suggest that web-based interactive tools can be more effective than traditional educational tools in increasing knowledge and reducing stigma. This is an interesting result, considering that stigma and misconceptions related to mental health are commonly seen in the educational sector (45). Schools have become relevant spaces for mental health promotion, indicated prevention and early detection of mental health problems. A reduction in stigma related to mental health problems might improve the referral process from the educational sector. To improve teachers' knowledge on mental health, online training appears as a promising strategy, with benefits like flexible usage not constrained by time and place and easy dissemination through national territories.

High attrition in Internet-based interventions where there is none or low-intensity coach/therapist support is a problem in Latin America as well as in developed countries (11, 15). Future studies should explore in detail how important is the human support in Latin America for increasing engagement and adherence as well as improving outcomes. In addition, other strategies for increasing adherence, like the inclusion of persuasive systems or user-centered designs (46), need to be evaluated. Other aspect less addressed by the included studies is how the integration of these interventions to the existing health networks in which the patient is already inserted could increase their acceptability, adherence, and efficacy.

Overall, this study shows that Internet-based interventions for the prevention and treatment of mental disorders are at an early stage of development in Latin America. The accumulating evidence shows promising results but with important challenges that are not different to those found elsewhere in the world (11–16). There is an urgent need to agree on evaluation methodologies and frameworks to assess the growing number of emerging interventions in this field (13). It is also necessary to produce methodological developments in large-scale effectiveness studies, as well as cost effectiveness and implementation studies, especially in primary care services. Future studies should also place greater emphasis on comparing online interventions with traditional face-to-face interventions, either alone or in combination. Progress in these areas is a necessary condition for scaling up these interventions and obtaining funding from health insurers.

#### Strengths and Limitations

As far as we know, this is the first review of Internet-based interventions for mental disorders that specifically addresses developments in this area in Latin America. However, our scoping review has some limitations. An important limitation is related to the small number of studies and publications found, in particular, the scarcity of randomized clinical trials. There are many feasibility studies of commercial applications and products that are not published anywhere. Likewise, pilot studies for a range of mental disorders have been carried out, and there are many which are developed for commercial purposes, but there is no reporting of its quality or outcomes. There is little consistency in the methodologies used in feasibility and pilot studies, which interferes further when comparing studies. Another important limitation is that four authors of this article (RA, GR, VM, and PM) are also responsible for some of the studies reviewed, which can be a source of many biases in the interpretation of the results. Nevertheless, the risk of bias assessment of RCT studies was carried out by ÁJ-M and PF, who did not participate in any of the studies reviewed. Besides, it shows that there is a group of local researchers in the field that is eager to learn from the experiences of others.

### CONCLUSION

Internet-based interventions of mental disorders are growing rapidly in countries at all levels of development. The aim of this scoping review was to systematically map the existing evidence on use of Internet-based interventions for prevention, treatment, education, or facilitating selfmanagement of mental disorders in Latin America, as well as to identify existing gaps in the literature. The results show that there are a growing number of studies testing the feasibility and acceptability of interventions, but there are few studies on effectiveness and cost effectiveness. Furthermore, there are few studies comparing the efficacy of Internet-based interventions with traditional face-to-face interventions. The relative lack of evidence conspires against efforts to disseminate and scale up digital interventions in the region. These results lead us to advocate for increasing the number of studies and, more importantly, improving the quality of e-mental health research in Latin America to produce better evidence to guide mental health policies.

### DATA AVAILABILITY

All datasets generated for this study are included in the manuscript/**Supplementary files**.

### AUTHOR CONTRIBUTIONS

ÁJ-M, RA, GR, VM, and PM contributed conception and design of the study; PM and ÁJ-M organized the database; ÁJ-M, PF, VM, and RA wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

### FUNDING

This study was supported by the Millennium Science Initiative of the Ministry of Economy, Development and Tourism, grant "Millennium Nucleus to Improve the Mental Health of Adolescents and Youths, Imhay," grant "Millennium Nucleus in Social Development, Desoc," and the Fund for Innovation and Competitiveness (FIC) of the Chilean Ministry of Economy, Development and Tourism, through the Millennium Science Initiative, Grant No. IS130005. PF received funding from CONICYT PFCHA/DOCTORADO NACIONAL/2019-21190745.

### ACKNOWLEDGMENTS

This article is part of the work of the e-mental health network of the University of Chile, VID U-REDES-C\_2018\_07.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00664/ full#supplementary-material

### REFERENCES


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Jiménez-Molina, Franco, Martínez, Martínez, Rojas and Araya. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Key Considerations for Incorporating Conversational AI in Psychotherapy

*Adam S. Miner1,2,3\*, Nigam Shah4, Kim D. Bullock1, Bruce A. Arnow1, Jeremy Bailenson3 and Jeff Hancock3*

*1 Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, United States, 2 Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, United States, 3 Department of Communication, Stanford University, Stanford, CA, United States, 4 Stanford Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, CA, United States* 

#### *Edited by:*

*Michelle Burke Parish, University of California, Davis, United States*

#### *Reviewed by:*

*Stefanie Kristiane Gairing, University Psychiatric Clinic Basel, Switzerland Donald M. Hilty, UC Davis Health, United States Peter Yellowlees, University of California,Davis United States*

#### *\*Correspondence:*

*Adam S. Miner aminer@stanford.edu*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 09 December 2018 Accepted: 17 September 2019 Published: 18 October 2019*

#### *Citation:*

*Miner AS, Shah N, Bullock KD, Arnow BA, Bailenson J and Hancock J (2019) Key Considerations for Incorporating Conversational AI in Psychotherapy. Front. Psychiatry 10:746. doi: 10.3389/fpsyt.2019.00746*

Conversational artificial intelligence (AI) is changing the way mental health care is delivered. By gathering diagnostic information, facilitating treatment, and reviewing clinician behavior, conversational AI is poised to impact traditional approaches to delivering psychotherapy. While this transition is not disconnected from existing professional services, specific formulations of clinician-AI collaboration and migration paths between forms remain vague. In this viewpoint, we introduce four approaches to AI-human integration in mental health service delivery. To inform future research and policy, these four approaches are addressed through four dimensions of impact: access to care, quality, clinician-patient relationship, and patient self-disclosure and sharing. Although many research questions are yet to be investigated, we view safety, trust, and oversight as crucial first steps. If conversational AI isn't safe it should not be used, and if it isn't trusted, it won't be. In order to assess safety, trust, interfaces, procedures, and system level workflows, oversight and collaboration is needed between AI systems, patients, clinicians, and administrators.

Keywords: natural language processing, artificial intelligence, expert systems, psychotherapy, conversational AI, chatbot, digital assistant, human–computer interaction

### INTRODUCTION

Clinicians engage in conversations with patients to establish a patient-therapist relationship (i.e., alliance), make diagnoses, and provide treatment. In traditional psychotherapy, this conversation typically involves a single patient and a single clinician (1). This model of psychotherapy is being modified because software programs that talk like people (i.e., conversational artificial intelligence, chatbots, digital assistants) are now beginning to provide mental health care (2). Conversational artificial intelligence (AI) is gathering diagnostic information (3, 4) and delivering evidence-based psychological interventions (5–7). Additionally, conversational AI is providing clinicians with feedback on their psychotherapy (8) and talking to young people about suicide, sex, and drug use (9, 10).

Conversational AI appears unlikely to achieve enough technical sophistication to replace human therapists anytime soon. However, it does not need to pass the Turing Test (i.e., able to hold human seeming conversations) to have a significant impact on mental health care (2). A more proximal challenge is to plan and execute collaborative tasks between relatively simple AI systems and human practitioners (11–13). Although AI in mental health has been discussed broadly (for a review

see 14), specific formulations of clinician-AI collaboration and migration paths between forms remain vague.

Articulating different forms of collaboration is important, because the deployment of conversational AI into mental health diagnosis and treatment will be embedded within existing professional services. Conversational AI will likely interact with traditional workers (i.e., clinicians), but how these roles and responsibilities will be allocated between them has not been defined. To guide future research, we outline four approaches and dimensions of care that AI will affect.

Within the four approaches of AI-human integration in mental health service delivery, one extreme is a view that any involvement by conversational AI is unreasonable, putting both patients and providers at risk of harmful unintended consequences. At the other extreme, we explore how conversational AI might uniquely serve a patient's needs and surpass the capacity of even the most experienced and caring clinician by overcoming entrenched barriers to access. Although embodiment (e.g., virtual avatars or robots) can have a significant impact on interactions with virtual systems, we focus exclusively on the potential benefits and challenges of verbal and written language-based conversation and ignore the implications of embodiment or presence (15). **Table 1** summarizes the four approaches and our related assumptions.

#### CARE DELIVERY APPROACHES

It is unclear whether the path forward will involve simultaneous experimentation with all four degrees of digitization, or progression through these approaches. We first briefly describe how these compare to the way individual psychotherapy is most often delivered today. Perhaps surprisingly, laws, norms and the ethics of data sharing represent a nonobvious but critical factor in how these alternative approaches can operate now or develop in the future.

Currently, psychotherapy sessions are rarely recorded except in training institutions for supervision. When they are, for example during training or to assess clinician fidelity during clinical trials, trained human clinicians with prescribed roles and responsibilities are the listeners and provide oversight. With few exceptions, such as immediate risk of serious harm to the patient or others, clinicians need explicit permission to share identifiable patient information. When one of these exceptions is invoked, there is an obligation to limit the sharing strictly to the extent needed to provide effective treatment and ensure safety (16, 17). Against this backdrop, having conversational AI listen to psychotherapy sessions or talk directly with patients represents a departure from established practice.

In the "humans only" approach, psychotherapy remains unchanged. Most psychotherapy sessions are heard only by the patient and clinician who are in the room. If a session were recorded, the labor intensiveness of human review would ensure most sessions would never be analyzed (8). The second approach, "human delivered, AI informed," introduces into the room a listening device connected to software that detects clinically relevant information (18) such as symptoms or interventions (19), and relays this information back to the patient or clinician. Quantitative analysis of recorded psychotherapy is in its early stages, but it shifts to software programs the burden of extracting relevant information from audio or text. In the third approach, "AI delivered, human supervised," patients speak directly to a conversational AI with the goal of establishing diagnoses or providing treatment (20). A human clinician would either screen patients and hand off specific tasks to conversational AI or supervise conversations between front-line conversational AI and patients. The fourth approach, "AI only," would have patients talk to a conversational AI with no expectation of supervision by a human clinician.

One of the less developed but more alluring ideas of AI psychotherapy is "AI delivered, human supervised." Even the most ardent supporters of AI will acknowledge that there are certain things humans do better than computers. Combining people and algorithms may potentially build on the best of both approaches, and AI–human collaboration has been suggested as a way to address limitations in planning treatment in other medical areas such as oncology (21). Indeed, the prevailing opinion of expert systems researchers in the 1980s argued that computer–human collaboration would outperform either people or computers alone (for a review see 22).

In assessing any system to augment the practice of psychotherapy the first consideration of its impact should be that it will ensure patients and clinicians are helped and not harmed (23, 24). In the discussion below, we consider salient issues that impact the potential value and harm of different delivery mechanisms by focusing on four dimensions of impact: access to care, quality, clinician-patient relationship, and patient self-disclosure.


*aBy "disrupt" we do not mean to signal that the result will be necessarily good or bad.*

### DIMENSIONS OF IMPACT

### Access to Care

Limited access to mental health treatment creates a demand for scalable and non-consumable interventions (25, 26). Despite the high costs and disease burden associated with mental illness (27), we have a decreasing number of clinicians per capita available to provide treatment in the US (28). Increasing the number of human clinicians is not currently feasible, in part because of the decline from 2008 to 2013 per capita for both psychologists (from a ratio of 1:3,642 to 1:3,802) and psychiatrists (from a ratio of 1:7,825 to 1:8,476) (28). Conversational AI has the potential to help address insufficient clinician availability because it is not inherently limited by human clinician time or attention. Conversational AI could also bridge one of the current tensions in care delivery: although clinicians value patient conversations, they have no financial incentive to engage in meaningful but lengthy conversations (29).

The decreasing amount of time spent in meaningful conversations exacerbates the shortage of psychiatrists and psychologists. Psychiatrists' use of talk therapy has been consistently and steadily declining, meaning fewer patients are receiving talk therapy during psychiatric visits (30). In contrast to a human clinician's time and attention, conversational AI is relatively non-consumable, making it an attractive alternative to delivery of care by a human. If conversational AI is effective and acceptable to both patients and clinicians, it may address longstanding challenges to mental health access. These include the ability to accommodate rural populations and to facilitate increased engagement from people who may experience traditional talk therapy as stigmatizing (31).

### Quality

Technology has been highlighted as a way to better understand and disseminate high quality psychotherapy (32, 33). Clinicians are already using texting services to deliver mental health interventions (34), which demonstrates a willingness by patients and clinicians to test new approaches to patient-clinician interaction. These new approaches facilitate novel measures of intervention quality. For example, innovations in computer science (e.g., natural language processing and machine learning) are being used to assess language patterns of successful crisis interventions in text-based mental health conversations (18, 35). Computational analysis of psychotherapy is encouraging researchers and companies to identify patterns of patient symptomology and therapist intervention (36, 37). This approach may improve psychotherapy quality by better understanding what effective clinicians actually do. This assessment has historically occurred through clinicians' self-reports or time intensive human audits (e.g., 38).

Although its efficacy is not definitively established, there are reasons to expect that conversational AI could constructively enhance mental health diagnosis and treatment delivery (39, 40). A diagnostic interview aids the patient and clinician in understanding the patient's presenting problem and provides a working model of how problems are being maintained. Approaches vary from highly structured diagnostic interviews [e.g., Structured Clinical Interview for DSM-5 (41)] to unstructured interviews in which the conversation develops based on the clinician's expertise, training, and the patient's features. Conversational AIs have interviewed patients about symptoms for PTSD with a high level of patient acceptance (20). Conversational AI has been piloted across numerous clinically relevant groups such as clinical depression (6) and adolescent stress (42). In a study in which students believed they were speaking with a conversational AI, the students reported feeling better after talking about their problems following the encounter (43). Although these early findings point to potential benefits, there is a lack of rigorous clinical trial data and uncertainty about regulatory oversight (2).

Yet while there is reason for optimism, inflated or unsubstantiated expectations may frustrate patients and weaken their trust in psychotherapeutic interventions (44, 45). Many current computation methods can be used to search for specific dialogue acts, but additional work is needed to map theoretically important constructs (e.g., therapeutic alliance) to causal relationships between language patterns and clinically relevant outcomes. Psychotherapy quality will be difficult to assess without disentangling causal inferences and confounding factors. Beyond computation, patients' attitudes matter in psychotherapy because those who have a negative experience compared with their expectations have worse clinical outcomes (46). If a patient loses trust in a conversational AI, they may be less likely to trust human clinicians as well. As conversational AI becomes more sophisticated and expectations of benefit increase, there are growing concerns that users will transition from feeling let down to feeling betrayed (47). These factors suggest that careful experimentation about sub-processes in AI-mediated communication merits research attention.

### Clinician–Patient Relationship

Modern medicine views the patient–clinician relationship as critical to patient health (48), and provider wellness (49). Indeed, appreciation of the importance of the patient–clinician relationship in modern medicine can be traced back to the influence of clinical psychology (50). Therapeutic alliance develops from clinicians' collaborative engagement with patients and reflects agreement on treatment goals, the tasks necessary to achieve such goals, and the affective bond between patient and provider (51). Therapeutic alliance is consistently associated with symptom improvement in psychotherapy (52–54). Numerous approaches exist to create alliance during psychotherapy, including the use of supportive language, mirroring emotions, and projecting warmth. Although originally conceptualized for human-to-human conversations, users have reported experiencing a sense of therapeutic alliance when speaking directly with conversational AI, suggesting this bond may not necessarily be restricted to human-human relationships (3). If conversational AI can create and maintain a therapeutic alliance, the provision of psychotherapy will not be necessarily limited by human clinicians' time and attention.

Establishing therapeutic alliance with conversational AI may benefit both patients and providers. By allowing conversational AI to take over repetitive, time-consuming tasks, clinicians' attention and skill could be deployed more judiciously (55). Allowing clinicians to do less of the work that contributes to burnout, such as repetitive tasks performed with little autonomy, may improve clinicians' job satisfaction (56). Clinician burnout is associated with worse patient outcomes and is increasingly recognized as a problem which must be more adequately addressed (57, 58).

At the same time, software that augments clinical duties has been criticized for distancing clinicians from patient care (59). In mental health, this risk is especially salient because the content of therapy is often quite intimate. Some of the repetitive, timeconsuming tasks clinicians engage in with patients, such as reviewing symptoms or taking their history, are precisely the vehicles by which clinicians connect with and understand their patients' experiences and develop rapport. It is unknown whether having a conversational AI listen in on psychotherapy will significantly impact patients' and clinicians' sense of therapeutic alliance. This area merits further research.

#### Patient Self-Disclosure and Sharing

Patient self-disclosure of personal information is crucial for successful therapy, including sensitive topics such as trauma, substance use, sexual history, forensic history, and thoughts of selfharm. Patient self-disclosures during psychotherapy are legally and ethically protected (24) and professional norms and laws have been established to set boundaries for what a clinician can share (60). Unauthorized sharing of identifiable patient information can result in fines, loss of license, or even incarceration. Moreover, because of the natural limitations of human memory, patients are unlikely to expect a human clinician to remember entire conversations perfectly in perpetuity. This capacity is in stark contrast to conversational AI, which has near-limitless capacity to hear, remember, share, and analyze conversations as long as desired. Because humans and machines have such different capacities, patient expectations of AI capabilities may impact treatment decisions and consent to data sharing (23).

In mental health, conversational AI has been shown to both facilitate and impede disclosure in different contexts. For example, users were more open with a conversational AI than with a human listener in reporting mental health symptoms (20), and have been successfully used to treat persecutory delusions for people with psychosis (61). Conversely, users were more reluctant to disclose sensitive information such as binge drinking behavior to a conversational AI compared to a non-responsive questionnaire (62). Because personal disclosures are central to diagnosis and treatment in psychotherapy, users' expectations and behavior towards technology-mediated conversations merit further assessment (63, 64, 65).

Certain disclosures in a psychotherapy context carry specific ethical and legal mandates, such as reporting suicidal or homicidal ideation. In 1969, a therapist at the University of California did not share the homicidal ideation of a patient with the intended victim. The patient subsequently killed the named victim, and the victim's family sued. This case (Tarasoff v. Regents of the University of California, 1974) established clinicians' duty not only to protect the confidentiality of their patients but also to notify individuals their patient might harm. A failure to warn leaves a clinician liable to civil judgment (66). Most case law and norms have been established on the premise of a dyadic relationship between patient and clinician. The extent to which conversational AI inherits liability for harm is untested. As conversational AI takes on clinical duties and informs clinical judgment, expectations must be clarified about how and when these systems will respond to issues related to confidentiality, safety, and liability.

### DISCUSSION

Experts in AI, clinicians, administrators, and other stakeholders recognize a need to more fully consider safety and trust in the design and deployment of new AI-based technologies (67, 68). A recent Lancet commission on global mental health states that "technology-based approaches might improve the reach of mental health services but could lose key human ingredients and, possibly, lower effectiveness of mental health care" (33). To inform future research directions, we have presented four approaches to integrating conversational AI into mental health delivery and discussed the dimensions of their impact.

Because conversational AI may augment the work of psychotherapy, we seek to encourage product designers, clinicians, and researchers to assess the impact of new practices on both patients and clinicians. Other areas of medicine have seen success with AI, such as lung cancer imaging and building diagnostic or prognostic models (69–73), and conversational AI for health is an emerging field with limited research on efficacy and safety (40, 63, 74).

Before we deploy AI-mediated treatment, workflow changes must be considered in the context of other demands on clinician time and training. Clinicians are already being asked to be familiar with telehealth (75) social media (76), and mobile health (77), while simultaneously being reminded of the need for selfcare in light of clinician burnout (58). Before we insert new devices into clinical care, it will be crucial to engage clinicians and design evaluation strategies that appreciate the skills, attitudes, and knowledge of affected workers. Just as we can't expect technology companies to easily understand healthcare, we can't expect medical professionals to intuit or work in harmony with new technology without thoughtful design and training.

A limitation of this work is that we do not set out a specific research agenda, and some important considerations are beyond the scope of this work (e.g., the cost and feasibility of each approach). We propose instead that initiatives using conversational AI anticipate challenges and leverage lessons learned from existing approaches to deploying new technology in clinical settings that involve clinician training and patient protections from the start (32, 77). We instead encourage those proposing to put AI into care settings to directly consider and measure impact on access, quality, relationships, and data sharing.

The potential benefits are clear for mental health. If diagnosis or treatment can be done by conversational AI, the societal burden of treating mental health could be diminished. Additionally, conversational AI could have a more long-term relationship with a patient than clinicians who rotate out of training centers. Despite these potential benefits, technology carries risks related to privacy, bias, coercion, liability, and data sharing that could harm patients in expected (e.g., denial of health insurance) and unintended ways (33, 44, 74, 78, 79–81). Conversations are valuable for patients and clinicians, and it is crucial to make sure they are delivered safely and effectively, regardless of who or what does the talking.

### AUTHOR CONTRIBUTIONS

ASM and JH contributed to the initial conceptualization and design of the manuscript. AM wrote the first draft. NS, KDB, BAA, and JB contributed to manuscript revision, read and approved the submitted version.

### REFERENCES


### ACKNOWLEDGMENTS

This work was supported by grants from the National Institutes of Health, National Center for Advancing Translational Science, Clinical and Translational Science Award (KL2TR001083 and UL1TR001085), the Stanford Department of Psychiatry Innovator Grant Program, and the Stanford Institute for Human-Centered Artificial Intelligence. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. We thank Nicole Martinez-Martin JD PhD, Victor Henderson MD MS, and Stan Fisher for their valuable feedback. Reference formatting assisted by Charlesworth Author Services.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Miner, Shah, Bullock, Arnow, Bailenson and Hancock. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Using Virtual Reality Exposure Therapy to Enhance Treatment of Anxiety Disorders: Identifying Areas of Clinical Adoption and Potential Obstacles

#### *Debra Boeldt1\*, Elizabeth McMahon2, Mimi McFaul1 and Walter Greenleaf3*

*1 National Mental Health Innovation Center, University of Colorado Anschutz Medical Campus, Aurora, CO, United States, 2 Independent Researcher, San Francisco, CA, United States, 3 Department of Communication, Stanford University, Stanford, CA, United States*

#### *Edited by:*

*Lina Gega, University of York, United Kingdom*

#### *Reviewed by:*

*Raluca Sfetcu, Spiru Haret University, Romania Pawel Izdebski, Kazimierz Wielki University of Bydgoszcz, Poland*

*\*Correspondence: Debra Boeldt debra.boeldt@cuanschutz.edu*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 05 June 2019 Accepted: 25 September 2019 Published: 25 October 2019*

#### *Citation:*

*Boeldt D, McMahon E, McFaul M and Greenleaf W (2019) Using Virtual Reality Exposure Therapy to Enhance Treatment of Anxiety Disorders: Identifying Areas of Clinical Adoption and Potential Obstacles. Front. Psychiatry 10:773. doi: 10.3389/fpsyt.2019.00773*

Despite strong evidence of effectiveness, exposure therapy is an underutilized treatment for anxiety disorders at a time when effective treatment for anxiety is greatly needed. The significant worldwide prevalence and negative impact of anxiety are documented and highlight the importance of increasing therapist and patient use of effective treatment. Obstacles to the use of exposure therapy are explored and steps to lessen these obstacles are proposed. In particular, virtual reality (VR) technology is discussed as a way to increase the availability of exposure therapy. Incorporating VR in therapy can increase the ease, acceptability, and effectiveness of treatment for anxiety. VR exposure therapy (VRET) permits individualized, gradual, controlled, immersive exposure that is easy for therapists to implement and often more acceptable to patients than *in vivo* or imaginal exposure. VR is presented as a scalable tool that can augment access to and effectiveness of exposure therapy thus improving treatment of anxiety disorders. VR also has the potential to help with assessment and with therapist training standardization. The authors advocate for providing continuing education in VRET to practicing clinicians and including training in exposure therapy and VRET in training programs. Ongoing development of VR applications for clinical use is encouraged, especially when developed in collaboration with software developers, clinical users, therapists who are experienced in VRET, and researchers.

Keywords: virtual reality, anxiety, exposure therapy, virtual reality exposure therapy, technology, mental health

### INTRODUCTION

Anxiety disorders are among the most common of mental disorders affecting nearly 18.1% of adults (1). The estimated prevalence of anxiety disorders worldwide is 7.3% (2), and they cause a high proportion of the global burden of disease (3). Anxiety symptoms can cause significant distress, impair quality of life, and increase stress. Anxiety increases risk for a range of co-occurring physical conditions, including chronic pain (4). Given the pervasiveness of anxiety and its impact on mental and physical health, effective treatment is clearly needed, yet a majority of affected individuals remain untreated (3, 5). These data highlight the importance for individuals and for society at large of increasing access to effective treatment. This perspective paper presents the ways that incorporating virtual reality (VR) in therapy can improve treatment for anxiety.

VR consists of a fully immersive, 3-D environment that transports people to engaging, interactive environments that can promote new learning. VR technology also has the potential to assist in training, evaluation, delivery, and supervision of psychotherapy skills (6), and can provide patients with a physiologically and emotionally evocative experience which can make VR a valuable tool for mental health treatment (7) VR exposure therapy (VRET) permits individualized, gradual, controlled, immersive exposure that is easy for therapists to implement and often more acceptable to patients than *in vivo* or imaginal exposure (8).This can allow users to practice behavioral skills in a safe environment through the support of a therapist. VRET has been used for the treatment of a range of conditions including social anxiety, Post-Traumatic Stress Disorder (PTSD), and panic disorder (9). The authors present several methods for incorporating VR technology into the therapeutic process and review how VRET can improve the ease, acceptability, and effectiveness of treatments for anxiety.

A patient can encounter difficulties finding or completing evidence-based anxiety treatment. These barriers can include failure to recognize and diagnose anxiety disorders, lack of access to treatment, and less-than-optimal quality of care (10). Since anxiety causes somatic symptoms, anxious patients often seek treatment through their primary care doctor (11). Anxiety disorders may go underrecognized and untreated. Access to effective care is further hampered by a shortage of therapists trained in evidence-based anxiety treatment (12). Limited access to effective treatment results in a large treatment gap allowing millions to suffer from anxiety even though evidence-based treatment exists (5). Obstacles to effective therapy need to be addressed.

Exposure therapy for anxiety disorders has a strong evidence base, yet few therapists utilize this method and patient and therapist misconceptions limit its availability (8, 13). Exposure therapy involves gradual and repeated exposure to feared stimuli with resultant changes in cognitions, behaviors, and emotional and physical responses. Feared stimuli can be almost anything: living organisms, inanimate objects, situations, activities, thoughts, mental images, physical symptoms, and/or affective experiences. Extensive research demonstrates the efficacy of exposure-based therapy for various anxiety disorders, especially phobias (14–16). Exposure facilitates extinction of the fear response and helps change dysfunctional assessments of threat and unhelpful responses, reducing the conditioned anxiety associated with feared stimuli (17, 18). Gradual exposure allows habituation and re-evaluation of the threat to occur.

Unfortunately, although exposure therapy can relieve anxiety symptoms ranging from mild to severe, therapists may not offer it and patients may be reluctant to try it, contributing to the treatment gap for anxiety (19). What are some obstacles to offering this proven effective treatment and how might VR help overcome these obstacles?

#### Obstacles to Exposure Therapy

Several studies have explored why exposure therapy is offered less often than its proven effectiveness would warrant (20–23). Both patient and therapist concerns were uncovered. One barrier is patient fears of exposure therapy. Even when appropriately diagnosed and referred to a clinician offering exposure therapy, patients may refuse treatment or drop out if therapy requires prolonged exposure to their most feared stimuli.

A second barrier may be difficulties arranging exposure. The non-VR options for exposure are imaginal exposure (the patient confronts the feared stimulus in imagination) and *in vivo*  exposure (the patient confronts the feared stimulus in actuality). Difficulties exist in both. In imaginal exposure, therapists cannot know or control what the patient imagines and the ability to create vivid mental images declines with age (24). *In vivo* exposure is often difficult or impossible to arrange inside the office and usually impractical to do outside the office. These above issues all interfere with making effective treatment for anxiety as widely available as is needed (20).

Therapist concerns about exposure therapy are a third barrier. Therapists often worry that exposure will be distressing for both therapist and patient and will increase patient drop out (8, 13, 21–23). When therapists have no way to control exposure, there is a risk that exposure may sensitize the patient and actually worsen anxiety. VR ameliorates this risk. Therapists can control different aspects of the patients' experience during exposure, thus permitting gradual, repeatable, individualized exposure. This minimizes the risk of patient distress and maximizes the chance of patient success.

A fourth barrier is that relatively few mental health providers are trained in exposure therapy. For example, research supports imaginal exposure as a therapy for PTSD, yet a broad survey of community therapists regarding PTSD treatment found that the majority of the licensed psychologists in clinical practice were not using this evidence-based intervention (12). Given projections that in United States the current shortage of behavioral health providers will worsen by 2025, with approximately 40 million individuals not having access to behavioral healthcare (25), it is imperative that the available mental health providers offer the most effective treatments. How can this issue be addressed?

Formal clinical training is one way to correct therapist misconceptions of exposure treatment. Deacon et al. (26) reported that therapists who attended a day-long didactic workshop about exposure therapy showed a decrease in negative beliefs about the treatment approach and an increase in using it. Doctorate-level therapists report fewer reservations about exposure therapy compared to other mental health professionals, perhaps as a result of having more training opportunities (26). Disseminating evidence-based anxiety treatments is essential to ensure that the future mental health workforce has adequate training. Directly addressing clinicians' concerns increases adoption as does training that incorporates motivational enhancement and/or offers a supportive learning community (22, 27). Even therapists trained and interested in exposure, such as imaginal exposure for PTSD, still underutilize the treatment (12, 28).

A final obstacle to offering exposure therapy may be time and/or difficulty involved, especially for *in vivo* exposure. *In vivo* exposure may be prohibited by clinic policy, can be difficult to arrange, take too long, and present confidentiality risks. These factors limit the availability of exposure therapy (28) despite its effectiveness. VR technology can help overcome these obstacles and support patient access to and acceptance of exposure therapy.

#### Applications and Benefits of Virtual Reality

Motivation to integrate technology and behavioral health exists, as highlighted by the *All of Us* research initiative funded by the NIH (29). Presently available VR technology is being used to enhance treatment of anxiety and VR has the potential to improve clinical training (6). VR could create an engaging, controllable, repeatable, and safe training environment (30). Advances in VR allow users to enter a fully immersive environment and have simulated interactions with virtual humans (31). Practice with virtual humans has also demonstrated effectiveness as a way to train for difficult conversations (32). A recent study found that virtual patient simulations can be an effective tool for developing brief clinical interviewing skills among behavioral health providers (33).

In the near future, VR may help provide standardized clinical training in exposure therapy, making training easier and more accessible. A VR training environment would allow therapists to repeatedly practice with virtual patients while mastering clinical assessment and exposure therapy skills. VR might allow clinical supervisors to vary the scenarios and customize the virtual patients within the training environment. Practicing exposure therapy within VR could increase skills and decrease fears about delivering exposure therapy. Increased therapist comfort with and competence in exposure therapy will help therapists provide evidence-based treatment and counter patients' fears about the therapy.

Researchers are also beginning to explore the potential of VR to help assess mental health conditions (34). VR-facilitated clinical assessment may improve the speed and accuracy of diagnosing patients' anxiety disorders, allowing fast referral to appropriate treatment. Improved assessment and treatment should improve patient outcomes (35).

Clinically, VRET is a practical, empirically-based treatment that makes exposure therapy easier and more acceptable for therapists and patients (9). It can help patients learn and practice anxiety management skills and permits controlled, gradual exposure, which minimizes distress and optimizes treatment success. Practically, VR is increasingly affordable. The cost of VR software and hardware continues to decline, while the quantity and quality of VR content increases. VRET has been shown to reduce anxiety and phobia symptoms (36). Research on VRET has proliferated exploring its use with a range of mental health disorders, particularly anxiety disorders such as phobias (9). Over two decades of research exist documenting the effectiveness of VRET for anxiety disorders. Meta-analyses demonstrate a large effect size compared to a control or waitlist condition, and no significant difference in effect size or attrition rates when compared to the gold standard of *in vivo* exposure therapy (9, 36–39). To summarize, VRET is an acceptable and effective alternative to *in vivo* exposure.

VRET allows the therapist to see what the patient sees in the virtual environment. This addresses four limitations of imaginal exposure: not every patient imagines well; the ability to form mental images declines with age; the patient's imagery may be too frightening; and the therapist neither knows nor controls what is being imagined. With VRET, the therapist can choose the VR content and personalize it for the patient. Therapists can guide patients through exposure in the office while monitoring and supporting the patient. Patients feel engaged, and the experience feels "real" but is a safe way for a patient to practice before facing a feared stimuli on their own in a real-world setting (34, 40). Therapists can monitor whether patients are attending to the content. Guiding patients to look at specific content and manage any anxiety that arises helps improve the efficiency of the treatment, ultimately improving patient outcomes. VRET also offers exposure therapy to patients who are resistant to other methods of exposure therapy (41). In the author's (EM) clinical experience, VR has additional uses in psychotherapy beyond exposure. VR helps engage patients in treatment. Relaxing virtual environments (VEs) can help patients learn and practice anxiety management skills and can be used to reinforce patient involvement in treatment and increase positive therapeutic alliance.

The authors further propose that VRET might help manage clinic throughput and can lower costs associated with intensive anxiety treatment. VRET has demonstrated costeffectiveness when compared to treatment as usual for PTSD (42). VRET offers the benefits of an *in vivo* experience in the office without requiring the burdensome time commitment required for therapists to guide, support, and monitor patients during real-world exposure. Such efficiency in providing effective, tailored exposure therapy can help therapists offer evidence-based treatment while handling the demands of a full caseload. Botella et al. (43) reported that successfully treating one phobia using VRET resulted in symptom relief of other, untreated, phobic behaviors. Effective treatment can be efficient treatment.

Lastly, this technology provides an opportunity to increase scalability. In addition to VR's potential uses to improve therapist training, thus increasing the number of therapists with adequate training in exposure therapy. VR self-help interventions for people with distressing, but subclinical, anxiety may ease suffering and may prevent progression to more severe symptoms. A recent study investigated the efficacy of using a consumer, off-the-shelf VR hardware and software to conduct exposure therapy for fear of public speaking (44). The authors compared traditional therapist-led exposure to self-led exposure at home and found that improvements were comparable across both delivery methods. A single-blind, parallel-group randomized controlled trial showed that a low-cost, scalable intervention for fear of heights delivered by a virtual coach resulted in reduced symptoms (45).

#### The Impact on Clinical Practice

The authors do not suggest that VRET will replace the need for trained therapists. The therapist's clinical skills are the key factor to the effective use and implementation of VR (46). As research advances, people may use evidence-based VR self-help programs to prevent subclinical or mild to moderate anxiety from worsening, prepare for therapy when clinical anxiety is present, and use as an adjunct to therapy to get maximum benefit from every session thus saving patient, clinic, and therapist time. Such advances may help therapists manage their caseloads and allow them to focus limited resources on individuals with clinical anxiety. Treatments for specific phobia domains will also increase as has been demonstrated by an automated immersive therapy treatment for acrophobia (45). Such advances will help therapists maintain a manageable caseload and allow them to focus limited resources on individuals with clinical anxiety.

VR and VRET does have limitations. Virtual environments (VEs) are not commercially available for all anxiogenic stimuli. Cybersickness (nausea or discomfort in response to VR) can also limit VR's use with some patients and limit the immersive effects of VR (47). Despite these limitations, VRET is an empirically supported treatment that makes exposure therapy easier and more acceptable. VR is increasingly affordable. The cost of VR software and hardware has declined, while the quantity and quality of VR content has increased.

### Additional Barriers to Adoption and Needs to Be Addressed

Several broader barriers exist to fully exploiting the potential of VR technology. First, the pace of new technology adoption by therapists is generally slow. Behavioral health interventions historically emphasize face-to-face delivery (29) relying on human judgment and assessment (6). This barrier may be reduced by increasing therapist knowledge of VRET's research support, addressing therapist concerns, and offering training in VRET.

Second, therapists wishing to incorporate VR in treatment must decide which software and equipment best suits their needs and may struggle to find information about choosing and using VR. The growing number of VR products and apps can make it difficult to decide what to purchase. Administrators may worry about cost-effectiveness and benefit. This is likely to become less of a barrier as cost for VR equipment and products drops and as more patients request VRET. Publicizing the benefits of VRET to lay as well as professional audiences will increase patient demand, motivating therapists and administrators to offer VR.

Third, since new technology can affect delivery models and require more research and provider input in the process (48), ongoing research on the implementation of VR in a range of clinical settings is strongly recommended. Presently, not all patients become immersed in the currently available virtual environments (37). This may reflect differences in patient ability, therapist skills, and a need for more clinical VR content. Since patients present with complex problems, an expansive library of content is ideally needed for personalized, effective treatment. To date, research has primarily focused on applications of VR for phobias, social anxiety, obsessive–compulsive disorder, PTSD, eating disorders, psychosis, and substance use. Academicindustry partnerships are advancing the development of technology for mental health (32); however, inclusion criteria for studies is often restrictive. Input from a range of therapy providers in diverse clinical settings can help expand the use cases for VR.

Lastly, therapist interest in using VR in therapy is increasing, but training in VRET is generally unavailable. Normalizing exposure therapy approaches and minimizing clinician concerns about the safety and tolerability of exposure therapy are essential for successful dissemination (27). For technology to be effectively adopted, training needs to be available, clinically relevant, and focused on answering therapists' concerns and needs. Resources should be developed that help therapists learn VRET and other therapeutic uses of VR. The authors advocate for the development of accredited continuing education classes and offering training in VRET to students training to become licensed mental health practitioners.

### DISCUSSION

This paper reviews the need for evidence-based treatment of anxiety, the importance of training therapists in exposure therapy, the uses and benefits of VR technology to improve anxiety treatment and explores future applications of VR for training. VR may help reduce current obstacles to wider adoption of exposure therapy by making it more acceptable to patients and easier for therapists. VR is presented as a scalable tool that can augment access to and effectiveness of evidencebased exposure therapy.

In conclusion, research on VR applications and collaborative development of clinical content are recommended to move this tool forward in the field. Therapists experienced in VRET can serve as subject matter experts in the development of content. Training in VRET and other uses of VR should be offered in graduate school and through continuing education. Ongoing expansion of training in VRET for therapists is one way essential to address the vast need for anxiety treatment.

## AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

### FUNDING

The work of this group is grant-funded from Anschutz Foundation, Award #: N/A.

## REFERENCES


Therapist Beliefs about Exposure Scale. *J Anxiety Disord* (2013a) 27(8):772– 80. doi: 10.1016/j.janxdis.2013.04.006


controlled trial. *Lancet Psychiatry* (2018) 5(8):625–32. doi: 10.1016/ S2215-0366(18)30226-8


**Conflict of Interest:** EM and WG are on the Scientific Advisory Board of Limbix, a company developing virtual reality content for mental health. NMHIC (DB and MM) provides consultation and has a partnership with Limbix.

*Copyright © 2019 Boeldt, McMahon, McFaul and Greenleaf. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Response and Remission Rates in Internet-Based Cognitive Behavior Therapy: An Individual Patient Data Meta-Analysis

#### *Gerhard Andersson1,2\*, Per Carlbring3,4 and Alexander Rozental2,5*

*1 Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden, 2 Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden, 3 Department of Psychology, Stockholm University, Stockholm, Sweden, 4 Department of Psychology, University of Southern Denmark, Odense, Denmark, 5 Institute of Child Health, University College London, London, United Kingdom*

#### *Edited by:*

*Lina Gega, University of York, United Kingdom*

#### *Reviewed by:*

*Eva-Maria Pichler, University of Zurich, Switzerland Kazuki Ide, Kyoto University, Japan*

*\*Correspondence: Gerhard Andersson Gerhard.andersson@liu.se*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 10 June 2019 Accepted: 18 September 2019 Published: 25 October 2019*

#### *Citation:*

*Andersson G, Carlbring P and Rozental A (2019) Response and Remission Rates in Internet-Based Cognitive Behavior Therapy: An Individual Patient Data Meta-Analysis. Front. Psychiatry 10:749. doi: 10.3389/fpsyt.2019.00749*

Background: Internet-delivered cognitive behavior therapy (ICBT) was developed over 20 years ago and has since undergone a number of controlled trials, as well as several systematic reviews and meta-analyses. However, the crucial question of response rates remains to be systematically investigated. The aim of this individual patient meta-analysis (IPDMA) was to use a large dataset of trials conducted in Sweden to determine reliable change and recovery rates across trials for a range of conditions.

Methods: We used previously collected and aggregated data from 2,866 patients in 29 Swedish clinical trials of ICBT for three categories of conditions: anxiety disorders, depression, and others. Raw scores at pre-treatment and post-treatment were used in an IPDMA to determine the rate of reliable change and recovery. Jacobson and Truax's, (1991) reliable change index (RCI) was calculated for each primary outcome measure in the trials as well as the recovery rates for each patient, with the additional requirement of having improved substantially. We subsequently explored potential predictors using binomial logistic regression.

Results: In applying an RCI of *z* = 1.96, 1,162 (65.6%) of the patients receiving treatment were classified as achieving recovery, and 620 (35.0%) were classified as reaching remission. In terms of predictors, patients with higher symptom severity on the primary outcome measure at baseline [odds ratio (OR) = 1.36] and being female (OR = 2.22) increased the odds of responding to treatment. Having an anxiety disorder was found to decrease the response to treatment (OR = 0.51). Remission was predicted by diagnosis in the same direction (OR = 0.28), whereas symptom severity was inversely predictive of worse outcome (OR = 0.81). Conclusions: Response seems to occur among approximately half of all clients administered ICBT, whereas about a third reach remission. This indicates that the efficacy of ICBT is in line with that of CBT based in prior trials, with a possible caveat being the lower remission rates. Having more symptoms and being female might increase the chances of improvement, and a small negative effect of having anxiety disorder *versus* depression and other conditions may also exist. A limitation of the IPDMA was that only studies conducted in Sweden were included.

Keywords: response rates, recovery, predictors, individual patient data meta-analysis, internet-based cognitive behavior therapy

#### INTRODUCTION

Internet-delivered cognitive behavior therapy (ICBT) has existed for more than 20 years (1), and treatment programs have been developed for a wide range of clinical and non-clinical conditions. Most forms of ICBT are administered in the form of guided selfhelp, with text and video presentations boosted by email support in a secure online platform resembling Internet banking (2). One way to describe ICBT is to pinpoint the similarities with online education, even if the treatment is largely based on self-help texts and cognitive behavioral therapy (CBT) manuals (3). Thus, ICBT programs tend to rely on psychoeducation and instructions for how to change thoughts, emotions, and behaviors in everyday life, and, like CBT, programs usually last 5 to 15 weeks with homework assignments and therapist feedback provided on a weekly basis (3).

Research suggests that therapist-supported ICBT—in contrast to unguided treatments (4)—can be as effective as face-to-face cognitive behavior therapy (5), yield long-term results (6), and work under clinically representative conditions (7). ICBT has also been tested for different target groups—for example, young persons (8), adults (9), older persons (10), and immigrants (11). Treatment programs have focused on specific problems, such as procrastination (12); diagnoses like post-traumatic stress disorder (PTSD) (13); or tailored according to a client's specific problem profile (14). Another approach has been to deliver transdiagnostic treatments in which one treatment is used to target underlying common processes, such as avoidance (15). There are also studies on other psychotherapy forms, including psychodynamic psychotherapy (e.g., 16), interpersonal psychotherapy (17), versions of CBT such as acceptance and commitment therapies (18), and attention training (19). Finally, there are also programs based on physical exercise (20), mindfulness (21), and relaxation (22), even if the latter two are often incorporated into ICBT protocols (23). In addition to the controlled trials, several studies exist on moderators and mediators of outcome (e.g., 24), as well as some qualitative studies on the client's experience during ICBT (25).

While it is common to report effects in clinical trials, there are also studies and reviews reporting negative effects and nonresponse to ICBT. In two previous individual patient data metaanalyses (IPDMA), we studied these two outcomes (26, 27). With regards to negative effects in the form of deterioration, 5.8% of treated research participants showed such effects (26), and non-response was present among 26.8% of participants (27). When completing these two reviews based on our dataset of controlled trials, a question emerged regarding response rates in our ICBT studies as this was not reported in the previous ones given uncertainties regarding definitions and scope of the two previous reviews. In contrast to the standard of reporting mean standardized differences with metrics like Cohen's *d*, there is far less agreement on how to define response in psychological treatments studies in general, and specifically in CBT (28). Several questions emerge when defining what constitutes a "response" to treatment. In a review, 26, p. 73-74) mentioned several issues, such as (a) number of measures used to define response, (b) number of modalities (e.g., self-report *versus* observed behavior), (c) blinding of assessors, (d) degree of change from baseline, and (e) use of a clinical cut-off to determine if a client has reached a non-clinical state (sometimes referred to as high end-state or remission). In a seminal paper, Jacobson and Truax (29) outlined guidelines for defining change from baseline (reliable change index-RCI) and different ways to define what constitutes being within a non-clinical range or having reached high endstate function/remission. In the present review, we will use the term "remission" to refer to what can be counted as high endstate function, which allows us to be consistent with a previous IPDMA on depression by Karyotaki et al. (30).

In their IPDMA on guided ICBT for depression focusing on response and remission, Karyotaki et al. (30) included 24 RCTs (4,889 participants) and compared guided ICBT with a control group. The mean pooled response rate (based on RCI) at posttreatment was 56.19%, and the mean remission rate at posttreatment in the treatment groups (N = 26) was 38.51% based on the RCI criteria (1.96)—two standard deviations improvement from baseline for each measure. Given the dataset we coded based on our own trials, we decided to conduct a new IPDMA (31) knowing that we could use original data across studies to investigate reliable change and remission (high end-state function). As in our previous two IPDMAs (26, 27), we used data from 29 clinical trials. The final dataset with complete data consisted of 1,535 patients who had received ICBT, and trials were categorized into three groups: anxiety disorders, depression/ mood disorders, and other conditions (i.e., erectile dysfunction, relationship problems, and gambling disorder). The aim of the current study was to determine the rates of treatment response and remission in clients who had received ICBT in clinical trials conducted by our group in Sweden. As a secondary exploratory aim, we examined potential predictors of response.

#### MATERIALS AND METHODS

#### Individual Patient Data Meta-Analysis

As described in our two previous IPDMAs (26, 27), we used the scores for individual client and outcome variable across studies (32). In IPDMAs, it is study factors that might be predictive of treatment outcome using the raw data instead of group means, as has been done in previous IPDMAs—for example, on low-intensity psychological treatments (33) and Internet interventions for problem drinking (34). As in the previous IPDMAs, we aggregated available data from clinical trials that we conducted. A complete description surrounding our data collection procedure is presented in Rozental et al. (26). An obvious limitation of using this method is that we cannot assess the risk of bias, which is a common practice in systematic reviews (32). However, by including trials from our own group, we were able to obtain an overall view of response and remission, which we assume could be representative, particularly for Swedish settings. For a complete description of the inclusion and exclusion criteria used, see 26, p.163).

The raw scores from each client in the included trials were entered into one data matrix, and codes for background variables were aligned. Given the complexity of modeling reasons for missing data in such a heterogeneous group of research participants and the fact that we were focused on binary outcomes in this review, we decided to use a complete case approach instead of multiple imputation, as we were convinced that data were not missing completely at random (35). Moreover, complete case analysis has been recommended as the first approach when conducting meta-analyses, even if this approach is followed by sensitivity analyses to detect possible bias in estimates (36).

Sociodemographic variables were occasionally collapsed to facilitate comparisons and to obtain consistency across trials (see 26, 27). Trials were categorized into three categories: (1) anxiety disorders, (2) depression and mood disorders, and (3) other (i.e., erectile dysfunction, relationship problems, and gambling disorder). In **Table 1**, we present an overview of the clients' sociodemographic variables in the trials included (the table overlaps with **Table 3** in 26, p. 169 but does not include the control groups). We present data from baseline in the trials and also the amount of missing data for the full sample.

#### Statistical Analysis

The RCI was chosen based on its widespread use for assessing reliable change (29, 37). As is common practice, the RCI was calculated by taking each individual change score and then dividing this score by the standard error of the difference, i.e., *SE*diff = *SD*1√2√1-*r*. In the formula, *SD*1 corresponds to the standard deviation of a condition at pre-treatment, and *r* is the reliability estimate (38). We calculated RCIs for the primary outcome measure of each included trial and used the test–retest reliability for that specific trial measure (see **Table 2**; also reported in 27). Basically, the RCI sets the limit for when a change score is unlikely to be real (*p* = .05). Following the usual standards, we calculated RCIs for which a change equal to *z* = 1.96 on the basis of a standard deviation unit was used. Following this, each participant in the trials could be classified as either a responder or a non-responder to treatment, with the definition of response being specific to each study and measure used, a similar method to that used by Karyotaki et al. (30). Heterogeneity was tested by entering response rates into the program comprehensive metaanalysis (version 2.2.021; CMA).

We also followed the methods of Karyotaki et al. (30) when calculating remission, again using criteria set by Jacobson and Truax (29). Participants were classified as remitters if they moved two standard deviations below the mean of the clinical group to which they belonged in each study. The resulting cut-off scores indicates remission, which is a hard criterion of remission, often being equivalent to a symptom-free state. For six of the studies, it was not possible to use the two standard deviation criteria due to floor effects, so instead, we used one standard deviation as the criterion in these cases.

In order to investigate possible predictors, we applied binomial logistic regression and used either response or recovery rates as dependent variables. All variables were entered into the model simultaneously. In terms of the variables used, we selected a few clinically relevant demographic and clinical predictors (54) (31). We selected the same variables as in our previous IPDMAs on deterioration (26) and non-response (27). The predictors were (a) symptom severity at baseline, (b) civil status, (c) age, (d) sick leave, (e) previous psychological treatment, (f) previous or ongoing psychotropic medication, (g) educational level, (h) diagnosis, and (i) gender. We also added a separate analysis of the association between publication year and the two outcomes response and recovery.

We present odds ratios (OR) with 95% confidence intervals (CI) in order to reflect an increase or decrease in odds of response and remission in relation to a reference category. In the case of dichotomous predictors (such as gender), the OR reflects the odds of response or remission when a client is female *versus* male (reference). A positive OR thus means better response in women. For continuous predictors (symptom severity at baseline and age), the OR instead represents an increase of one standard deviation above their respective mean. The statistical analyses on predictors were performed using jamovi version 0.9.2.9 (55), with the proportions of response and remission performed on a complete case basis (see online **Supplementary Material**).

#### Ethics

The data in the current IPMA were derived from several clinical trials, all of which had received ethical approval from the respective regional ethical review boards at each study location.

### RESULTS

#### Study Characteristics

The 29 clinical trials were coded according to the previously predefined inclusion and exclusion criteria and deemed eligible for the current IPDMA (see 26, 27 for further details). Raw scores from all clients were entered into one spread sheet. The exception was 46 clients who had received either psychodynamic psychotherapy or interpersonal psychotherapy *via* the Internet as a control condition. In total, 1,535 clients were included in this IPDMA. The following diagnoses were included (clinical trials, *k*): social anxiety disorder (9), depression (with/without TABLE 1 | Sociodemographic characteristics of the participants included in the analysis (based on Table 3 in 26).


*n.a., not applicable; NOS, not otherwise specified.*

*aValid percent, i.e., percent of available data, excluding missing data.*

*bPercent, i.e., percent of complete dataset, including missing data.*

*cSelf-rated 0–5.*

*dSelf-rated 0–10.*

*eBased on patients receiving treatment.*

*f Weighted mean and standard deviation.*

*gNumber of hours per week.*

dysthymia) (5), generalized anxiety disorder (3), anxiety disorder (with/without depression) (3), mixed anxiety disorders (e.g., panic disorder as well as social anxiety disorder) (2), specific phobia (2), posttraumatic stress disorder (1), panic disorder (with/without agoraphobia) (1), gambling disorder (1), erectile dysfunction (1), and relationship problems (1) (see 26, p.6). Briefly, most participants in the trials had been recruited from the general population based on self-referral (n = 27). A common practice in the trials was to use structured telephone interviews such as the Structured Clinical Interview for DSM-IV-Axis I Disorders (56) or the MINI-International Neuropsychiatric Interview (57). Some studies had used diagnosis-specific instruments such as the Clinician-Administered PTSD Scale (58). For a description of treatment content original studies, (see 26, 27). The amount of missing data for the primary outcome measures at post-treatment was 12.9%. A complete overview of the clinical trials is presented in **Table 3** (which to some extent overlaps with **Table 3** in 27 but with different results presented).

#### Response and Remission Rates

Using the RCI criteria for detecting response, 1,027 (69.9%; 95% CI: 67.61–72.19) of the 1,535 participants receiving treatment were categorized as treatment responders when using an RCI



*NORC, a National Organization for Research at the University of Chicago.*

*a Reversed scales, higher scores indicate less problems.*

*b Information regarding the time period was unavailable.*

*c Separate estimates for the two subscales.*

*d Lifetime test statistic and past year test statistic.*

of *z* = 1.96. The lowest rates were found in the trials on erectile dysfunction (12.1%) and older adults with anxiety (27.3%), whereas the highest rates were found in the trials on gambling (93.6%) and spider phobia (92.3%). As seen in **Table 3**, the proportions varied, which was confirmed by the CMA program showing a significant heterogeneity (*I2 =* 87.5%; Q = 223, *p* < .001).

Remission was achieved in 540 participants (35.2%; 95% CI: 32.81–37.59) when adjusting for floor effects (the unadjusted proportion was 31.9%). There was a large variation with ranging from 0% (erectile dysfunction, depression, and bias modification for social anxiety disorder) to 82% (gambling) and 69% (spider phobia). As with the response rates, a significant heterogeneity was found (*I2 =* 91.6%; Q = 334, *p* < .001).

#### Predictors of Response

Binomial logistic regressions were calculated with the predefined variables entered as predictors of response. Results are presented in **Table 4** with OR and 95% CI for each predictor. The results that indicate a higher symptom severity on the primary outcome measure at baseline was predictive of better outcome. The odds for responding to treatment decreased when having an anxiety disorder as compared to depression/mood disorder and other diagnoses (i.e., erectile dysfunction, relationship problems, and gambling disorder), and the odds increased if the subject was female. The other variables were not predictive of response.

We also repeated the analyses with remission as outcome (see **Table 5**). In this analysis, symptom severity at pre-treatment was marginally associated with less remission (OR = 0.81). As with the analysis for response, the odds for remission were lower if the subject suffered from anxiety. Gender and the other variables did not reach statistical significance. Publication year was unrelated to response and remission.

### DISCUSSION

The aim of this IPDMA was to obtain estimates of response and remission rates for ICBT with a range of conditions categorized into three groups (anxiety, depression, and other). In line with a previous IPDMA on depression by Karyotaki et al. (30) which included trials from different countries, we found that 65.6% of the treated research participants could be classified as treatment responders. This is slightly higher than the 56.19% reported by Karyotaki et al. (30). While Karyotaki et al. (30) imputed missing data, they also reported that the estimates between complete case analysis and the imputed dataset were minor. In addition, we found that 35.0% of participants could be classified as having TABLE 3 | Characteristics and rates of improvement and recovery for the clinical trials included in the individual patient data meta-analysis.


(*Continued*)

Remission and Response in ICBT


(*Continued*)


**57**

Remission and Response in ICBT


*SCID-I, structural clinical interview for DSM-IV axis I disorders; NOS, not otherwise specified; FAQ, frequently asked questions; BAI, beck anxiety inventory; BDI, beck depression inventory; QOLI, quality of life inventory; ISI, insomnia severity index; n.a., not applicable; ACT, acceptance and commitment herapy; LSAS-SR, Liebowitz social anxiety scale—self-report; PDSS-SR, panic disorder severity scale—self-report; PHQ-9, patient health questionnaire—nine items; GAD-7, generalized anxiety disorder—seven items; BA, behavioral activation; MADRS-S, montgomery–åsberg depression rating scale—self-report; IIEF-5, International Index of erectile functioning—five items; CBT, cognitive behavior therapy; IIEF, International index of erectile functioning; RAS, Relationship assessment scale; MINI, the MINI-International neuropsychiatric interview; BBQ, Brunnsviken brief quality of life inventory; Mini-SPIN, mini-social phobia inventory; HAM-D, Hamilton rating scale for depression; WAI, Working alliance inventory; CAPS, Clinician-administered PTSD scale for DSM-IV; IES-R, Impact of event scale—revised; PDS, Posttraumatic diagnostic scale; PSWQ, Penn ptate worry questionnaire; GAD-Q-IV, Generalized anxiety disorder questionnaire; CORE-OM, Clinical outcome in routine evaluation—outcome measure; STAI, State-trait anxiety inventory; CIDI, Composite international diagnostic interview; ACQ, Agoraphobic cognitions questionnaire; BSQ, Body sensations questionnaire; MI, Mobility inventory; DAS, Dyadic adjustment scale; MSI, Marital status inventory; SNAQ, Snake anxiety questionnaire; ADIS, anxiety disorders interview schedule; FSS, Fear Survey schedule; CBM, Cognitive bias modification; SIAS, Social interaction anxiety scale; SPS, Social phobia scale; SPSQ, Social Phobia screening questionnaire; AR, applied relaxation; SPQ, Spider phobia questionnaire; NODS, the NORC diagnostic screen for gambling problems.*

*\*Based on complete case analysis, i.e., only complete data.*

*aSeparate analyses of deterioration were conducted for the two primary outcome measures depending on the diagnosis of the patient.*

*bFour treatment conditions were included in the study, with/without treatment rationale, respectively, but are pooled in the current analysis.*

*cAn additional outcome measure, International Physical Activity Questionnaire, IPAQ, was used in the study but is not included in the current analysis.*

*dPassive control with the possibility to contact the research team if needed.*

*ePatients were able to choose 10 out of 16 modules to be completed during 10 weeks.*

*f Patients were able to choose 10 out of 19 modules to be completed during 10 weeks.*

*gAn additional treatment group, guided psychodynamic therapy, was also used in the study but is not included in the current analysis.*

*hOne brief orientation session and one session of 3-h prolonged exposure.*

*i In addition to 2 weeks of CBM.*

*j An additional treatment group, interpersonal psychotherapy, was also used in the study but is not included in the current analysis.*

*k Additional outcome measures were included in the original study but lost in the raw data file.*

*l Patients were able to complete up to eight modules selected by the therapist.*

*1Andersson G, Burman M, Norlander A-K, Magnusson K, Stalby M, Svensk M, et al. Internet-delivered couples therapy: a randomized controlled trial. (2019). Unpublished manuscript.*

October 2019 | Volume 10 | Article 749



*OR, odds ratio; CI, 95% confidence interval; p, p-value.*

TABLE 5 | Significance levels, odds ratios, and 95% confidence intervals (CI) for predictors of recovery.


*OR, odds ratio; CI, 95% confidence interval; p, p-value.*

remitted, which is slightly lower than the 38.51% remission rate reported by Karyotaki et al. (30). Moreover, we had a problem with floor effects and if not considering that our estimate is even lower (31.9%). Overall, the results are rather similar to Karyotaki et al. (30) in that roughly half of clients showed improvement following ICBT and remission was achieved by a third.

Given these estimates, the outstanding question is how well this compares against face-to-face CBT. As previously mentioned, there are a few studies in which clients have been randomly assigned either face-to-face CBT or therapist-guided CBT. In the most recent of such study, Carlbring et al. (5) found no differences in effect. The response rates in CBT across different disorders and conditions are difficult to estimate; as to our knowledge, there is no similar IPDMA on response rates in face-to-face CBT. The closest we can get is a meta-analysis on anxiety disorders based on published data (28). In that review, 31% of the studies had defined response using RCI, and the results were similar to the range found in this analysis (44.5–51.1%). However, Loerinc et al. (28) also reported that the use of RCI in combination with a clinical cut-off (as was done in the present study) was associated with a 28% lower response rate (or, as expressed in this study, the remission rate was 28% lower than the RCI response rate). The difference we found was somewhat larger, as about half of the responders also showed remission. From a more naturalistic perspective, Gyani et al. (82) reported that 63.7% of participants in their clinical sample showed reliable improvement following evidence-based face-to-face treatment. In a recent meta-analysis, Springer et al. (83) reported a remission estimate as high as 51% (compared to our estimate of 35%). As stated by Loerinc et al. (28), there is a large variation in how to define both response and remission in CBT trials; thus, one advantage of the approach taken here is that we can use the same approach across studies. However, the definition of remission used (two SDs below the pre-treatment mean) was impossible in some studies (leading us to correct that estimate) and unrealistic in others. Another disadvantage of a statistical definition of response and remission is that it is heavily dependent on the sample upon which it is calculated. This pertains to both RCI and recovery as outlined by Jacobson and Truax (29). Another approach would have been to determine criteria for response and remission independently of the study, which is possible when there is data on non-clinical samples. For example, on the Beck Depression Inventory (44), a 10-point reduction could be seen as indicative of response, and 13-point reduction indicative of recovery. Applying these criteria for the Stella trial (see **Table 3**; 63), in which 57.5% of participants responded according to the RCI and 0% remitted, the corresponding figures of using a 10-point reduction (53.1%) and a score of 13 or below on the BDI (59.4%) paints a slightly different picture (although the results are similar between the RCI criteria and the 10-point reduction). In particular, the difference between the 0% classified as remitters *versus* the 59.4% having a score indicating minimal depression (44) shows the importance of response definitions (for a detailed discussion, see 37). However, in the present IPDMA, we found it to be of value to use similar criteria across trials. Future research could focus more on the external criteria of improvement instead of study-specific criteria. Unfortunately, the BDI is an exception, being widely used in many trials, and for some conditions and measures included in this IPDMA, cut-off scores and nonclinical norms have not been established.

In the present IPDMA, we conducted exploratory analyses to see if response could be predicted. It is important to note that there was no firm theoretical basis for our selection of predictors and, therefore, our findings must be interpreted with caution, as the identification of a significant predictor could have practical implications for future treatment recommendations. This is particularly the case when findings are based on samples larger than is commonly used in psychotherapy trials. However, the finding that symptom severity was not a negative predictor, but rather the opposite is in line with a previous IPDMA on lowintensity interventions for depression (33). From a clinical point of view, this makes sense, as having symptoms makes treatment more relevant than if subclinical or even different symptoms are present in the client. However, it also suggests that our ICBT trials probably included participants without severe symptoms. It is therefore possible that this finding is based on selection criteria used in trials and not necessarily relevant for clinical practice when treatment is offered with fewer restrictions. In contrast to the finding for response, we found a small negative effect of pre-treatment severity for the prediction of remission. These two conflicting results may indicate that large improvements are less likely if the client has more symptoms. However, as remission requires a low level of symptoms, it is less likely that a client with many symptoms will reach that low level. As in treatment research in general, it could be that the search for predictors is best pursued in ordinary clinical settings rather than in wellcontrolled trials. On a promising note, large ICBT effectiveness studies are being reported (84), which may provide clearer insight into the efficacy of different approaches for different populations.

The odds ratio in favor of female participants was surprisingly high (OR = 2.22), which is hard to explain as this is not a consistent result from previous individual trials. One possible explanation is the fact that the gender proportions in trials are nested with the condition treated. For example, in a typical depression trial, there is a majority of women, whereas in other conditions, there are more or less equal proportions of men and women, while some studies have only men (e.g., erectile dysfunction). The overall takeaway message here is that this finding needs further investigation in future trials to confirm its veracity. Gender was not found to be predictive of remission.

Interestingly, while the IPDMA by Karyotaki et al. (30) found old age to be weakly associated with better response (OR = 1.01), there was no such effect in this study suggesting that age is not a predictor of outcome. In Karyotaki et al.'s (30) complete case analyses, baseline severity (OR = 1.16) was found to predict better outcome, which was in line with our findings (but not in their intention to treat analyses). Gender did not predict outcome in Karyotaki et al.'s (30) IPDMA, but, again, their review focused on only depression which means a larger portion of the population was likely female.

In this IPDMA, we found decreased odds for responding to treatment when having an anxiety disorder as compared to depression/mood disorders or other (erectile dysfunction, relationship problems, and gambling disorder). This was a small effect, but still puzzling and hard to understand given that the overall picture is the opposite—indicating a higher response to ICBT in clients with anxiety disorders. Anxiety disorders were also found to be predictive of lower rates of remission.

While non-significant findings cannot be viewed as proof of absence of an effect, it is still interesting that ICBT has had marginal success in finding predictors of change as well as moderators and mediators of outcome (1). There are several possible reasons for this. First, participants in trials are not selected for their differences. Rather, they are chosen based on diagnostic criteria, access to the Internet, willingness to be a research participant, etc., all of which likely reduce the chance of identifying accurate predictors. Second, predictors of dropout (85) may ultimately be more illuminating, even if a given IPDMA was on self-guided ICBT for depressive symptoms. It is, however, interesting to note that they found that male gender (RR = 1.08) and co-morbid anxiety symptoms

(RR = 1.18) significantly increased the risk of dropping out of the study, which is in line with our findings with regards to gender and anxiety disorders. In spite of the overall inconsistency in findings and lack of findings, we believe that IPDMA is a powerful and reliable tool for answering questions regarding predictors of response (86). However, this will require concerted efforts to align both outcome measures as well as data on predictors in order to make studies comparable. Even if we were co-workers and principal investigators in the trials included in this IPDMA, the data were not consistent with regards to background variables etc. It becomes even more problematic when attempting to combine datasets from different research groups.

This study had several limitations. We focus on four, knowing that there are further objections that could be raised. First, we only included our own trials, which were conducted in Sweden. IPDMAs are often selective, as original data sometimes cannot be obtained from authors, but in this case, we cannot generalize the results outside of our own culture and setting. In addition to not including trials from international groups, we also did not include the most recent and unpublished trials from our own groups, and there are other groups in Sweden with trials that were also not included. Furthermore, with the focus on our own studies conducted over a period close to 20 years, there is a possibility of time trends, which we did not focus on in this study. Technological changes may not influence effects, but, to give an example, early studies relied more on printing out text materials (87), whereas recent studies are typically delivered *via* online platforms (e.g., responsive to the presentation format, such as smartphones or computers) (2). Second, even if this limitation is not unique for ICBT, outcome measures were largely based on self-report. Such measures are useful and generally have good psychometric properties but still present a possible risk that selfreported changes do not correspond with actual behavior changes and do not conform to the findings of an interview. Third, as commented on by Loerinc et al. (28), treatment response can be defined by several measures, but here, we focused only on the primary outcomes. Our dataset on IPDMA could also be used to analyze effects on other secondary measures of constructs, like quality of life, as several of our studies have used the same measure for this construct (e.g., 88), and it has been reported that the effects of ICBT might be lower on that construct (89). The fourth limitation relates to the statistical methods used. We decided to report on a complete case basis, but we could not exclude bias, as missing data was not considered.

### CONCLUSIONS

In spite of the limitations, the present IPDMA suggests that ICBT can lead to major reductions in symptoms and that more than half of clients, on average, respond to treatment. A lower proportion remits; thus, there is room for improvement. It is possible that women benefit more from ICBT based on our findings, and symptom severity seems to be predictive of outcome, but these findings should be interpreted with caution. Our findings that studies on clients with anxiety disorders were associated with less response is also notable but should be regarded with care. Future IPDMAs should include more trials and should also consider secondary outcomes, such as quality of life.

#### DATA AVAILABILITY STATEMENT

The datasets analyzed in this manuscript are not publicly available. Requests to access the datasets should be directed to gerhard.andersson@liu.se.

### AUTHOR CONTRIBUTIONS

All of the authors contributed in the process of completing the current study and writing the final manuscript. AR conducted the aggregation of raw scores into a single data matrix and completed the statistical analysis with input from GA and PC. GA drafted the first version of the text and did additional calculations,

### REFERENCES

(\* References marked with an asterisk indicate studies included in the meta-analysis.)


while AR and PC provided feedback and reviewed and revised the manuscript.

### ACKNOWLEDGMENTS

The authors would like to thank all of the students, researchers, clinicians, and administrative staff who, over the years, have been involved in the clinical trials that were included in the individual patient data meta-analysis. GA also thanks Linköping University for support in the form of a professor contract. The code to analyse the data using jamovi can be obtained found in the online supplement.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00749/ full#supplementary-material


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Andersson, Carlbring and Rozental. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# What Works and What Doesn't Work? A Systematic Review of Digital Mental Health Interventions for Depression and Anxiety in Young People

*Sandra Garrido1,2,3\*, Chris Millington1, Daniel Cheers1,4, Katherine Boydell5, Emery Schubert6, Tanya Meade3 and Quang Vinh Nguyen7*

1 MARCS Institute for Brain, Behaviour & Development, Western Sydney University, Milperra, NSW, Australia, 2 Translational Health Institute, Western Sydney University, Campbelltown, NSW, Australia, 3 School of Social Sciences & Psychology, Western Sydney University, Milperra, NSW, Australia, 4 NSW Health, Sydney, NSW, Australia, 5 Blackdog Institute, Randwick, NSW, Australia, 6 School of Arts & Humanities, University of New South Wales, Kensington, NSW, Australia, 7 School of Computing & Engineering, Western Sydney University, Parramatta, NSW, Australia

#### Edited by:

Lina Gega, University of York, United Kingdom

#### Reviewed by:

Fahad Riaz Choudhry, International Islamic University Malaysia, Selayang, Malaysia Peter Knapp, University of York, United Kingdom

\*Correspondence: Sandra Garrido s.garrido@westernsydney.edu.au

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 07 March 2019 Accepted: 20 September 2019 Published: 13 November 2019

#### Citation:

Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T and Nguyen QV (2019) What Works and What Doesn't Work? A Systematic Review of Digital Mental Health Interventions for Depression and Anxiety in Young People. Front. Psychiatry 10:759. doi: 10.3389/fpsyt.2019.00759

Background: A major challenge in providing mental health interventions for young people is making such interventions accessible and appealing to those most in need. Online and app-based forms of therapy for mental health are burgeoning. It is therefore crucial to identify features that are most effective and engaging for young users.

Objectives: This study reports a systematic review and meta-analysis of digital mental health interventions and their effectiveness in addressing anxiety and depression in young people to determine factors that relate to outcomes, adherence, and engagement with such interventions.

Methods: A mixed methods approach was taken, including a meta-analysis of 9 randomized controlled trials that compared use of a digital intervention for depression in young people to a no-intervention control group, and 6 comparing the intervention to an active control condition. A thematic analysis and narrative synthesis of 41 studies was also performed.

Results: The pooled effect size of digital mental health interventions on depression in comparison to a no-intervention control was small (Cohen's d = 0.33, 95% CI 0.11 to 0.55), while the pooled effect size of studies comparing an intervention group to an active control showed no significant differences (Cohen's d = 0.14, 95% CI -.04 to 0.31). Pooled effect sizes were higher when supervision was involved (studies with no-intervention controls: Cohen's d = 0.52, 95% CI 0.23 to 0.80; studies with active control: Cohen's d = 0.49, 95% CI -0.11, 1.01). Engagement and adherence rates were low. Qualitative analysis revealed that users liked interventions with a game-like feel and relatable, interactive content. Educational materials were perceived as boring, and users were put off by non-appealing interfaces and technical glitches.

Conclusions: Digital interventions work better than no intervention to improve depression in young people when results of different studies are pooled together. However, these interventions may only be of clinical significance when use is highly supervised. Digital interventions do not work better than active alternatives regardless of the level of support. Future interventions need to move beyond the use of digital educational materials, considering other ways to attract and engage young people and to ensure relevance and appeal.

Keywords: children, adolescents, unguided self-help, self-management, low mood, prevention

### INTRODUCTION

In Australia, approximately 8% of young people between 11–17 years of age meet the DSM criteria for major depressive disorder (MDD), while about 20% report high levels of psychological distress (1). The rates of MDD may be as high as 11% in youths in the U.S. (2). In fact, suicide is the second leading cause of death among 15–29-year-olds globally (3). In addition, depression is highly under-diagnosed and thousands who fall outside these statistics experience its debilitating effects on functioning at an important developmental stage. Depression and other mental illnesses affect the social and intellectual development of young people, reducing engagement with education, and if untreated, can become lifelong disabilities (4).

Despite the importance of addressing mental illness early only 20–40% of youths in need in Australia (1) and 25% of youths in the U.K. receive professional help (5). This low engagement with mental health services appears to occur for a variety of reasons: the lack of motivation inherent in conditions such as depression (6), low rates of mental health literacy (7), and the stigma, discrimination and embarrassment surrounding mental illness (8). Young people are also still developing skills in executive functioning such as self-monitoring and organization, which are necessary to identify a mental health problem and obtain support (9).

Although few young people seek professional help, during episodes of depression consumption of media such as music, internet, and television increases (10). Thus Digital Mental Health Interventions (DMHIs) are increasingly of interest as a solution to the low help-seeking and uptake rates of professional mental health services. Studies tend to support the effectiveness of self-help mental health programs whether digital or otherwise, which can be as effective if not more effective than face-toface delivery (11). Numerous studies have demonstrated the usefulness of web-based programs (12, 13). Young people report feeling more comfortable discussing sensitive and personal issues in the relative anonymity of an online context and use the internet as a major source of mental health information (14, 15). Mobile "apps" are proving particularly useful for administering DMHIs because of the widespread ownership of mobile phones, with the majority of young people in the U.S. reporting almost constant usage of smartphones (16). Several reviews of smartphone applications for mental health across age groups have reported positive benefits (13, 17).

Notably, however, many apps for depression and anxiety that are currently available are not evidence-based and may thus actually be harmful to people with mental illness (18). Even among those purporting to be drawn from evidence-based therapies such as cognitive behavioural therapy, only a small percentage actually contain the core principles of those therapeutic traditions (19). Furthermore, Hollis and colleagues (20) in their meta-review reported that while there is some evidence in support of the effectiveness of DMHIs for depression and anxiety in young people, studies are methodologically limited making it difficult to draw clear conclusions. Furthermore, they suggest the need for identification of the components that make DMHIs effective such as human interaction.

In fact, human interaction has been identified as an important factor influencing effectiveness and engagement with DMHIs (21), but it may also detract from the cost-effectiveness of DMHIs in comparison to face-to-face treatment (20). Furthermore, other disadvantages to the inclusion of social features exist, such as unhelpful advice from peers, and the possibility that some youths may feel afraid to share personal problems even anonymously.

The aim of the current review, therefore, is to examine the literature about DMHIs to address mental health in young people. We have focused on depression and anxiety as these are among the most prevalent mental health conditions experienced by young people and often co-occur with many other disorders (22–24). Specifically we aimed to investigate:


### METHOD

#### Study Design

Given the focus in the current review on both effectiveness and engagement, it was expected that the literature reviewed could include both quantitative and qualitative data. Therefore, a mixed methods approach was selected. Mixed methods reviews attempt to combine looking at 'what works' with 'how and why it works', combining varied research methods in the analysis or in the types of studies reviewed (25). Mixed method reviews can provide a more holistic understanding than a meta-analysis alone since they are able to integrate a wider variety of studies and provide insights into mechanisms and processes.

### Identification and Selection of Studies

A systematic search was conducted in PsychInfo, PubMed, Proquest, and Web of Science using the following terms in the title, abstract and subject descriptors: "mobile", "application", "smartphone", "mobile phone", "cell phone", "text message", "internet-administered therapy", "computer-aided therapy", "online" AND "depression", "anxiety" AND "youth", "young person", "adolescent". The initial search returned 4,828 articles. With duplicates deleted this number was reduced to 3,352.

Identified references were screened according to the following inclusion criteria: (1) participants aged 12 to 25, (ii) interventions targeting depression or anxiety, (iii) interventions delivered by computer, on smartphones, or online, (iv) studies published between 2007 and 2017. We also excluded reviews, opinion, or discussion pieces and unpublished works. While quality assessment was part of the review process, studies were not excluded on the basis of study type or quality since the mixed methods approach taken allowed the inclusion of varied methodologies.

After title screening by 1 researcher 184 abstracts were uploaded to Covidence, an online platform for conducting systematic reviews (http://www.covidence.org). These were scrutinised by 2 researchers. Once agreement was reached on eligibility, 68 articles remained. Full text appraisals were then conducted by 2 researchers and a further 27 articles not meeting the inclusion criteria were excluded, leaving 41 in the review (**Figure 1** and **Supplementary Materials**).

## Quality Assessment and Data Extraction

Two researchers independently conducted the data extraction. Information about the characteristics of the studies, participants, interventions, and final outcomes were entered into a

pre-established template in Covidence. Qualitative data from the studies relating to usability and appeal of the interventions was also extracted. The quality of studies was assessed using the Joanna Briggs Institute's (26) critical appraisal tools and the CONSORT-EHEALTH Checklist (V.1.6.1) (27). Studies were considered methodologically sound if they had a matched control group, pre-post data, and randomization, and were only included in the meta-analyses if they met these criteria. However, since this is a mixed-methods review, studies not meeting the criteria for inclusion in the meta-analysis were retained in the study to form part of the narrative synthesis. Risk of bias was assessed using the standard Cochrane Risk of Bias tool (28) in Covidence, which poses questions about aspects of trial design, conduct, and reporting which the user rates as 'Low' or 'High' risk of bias for each study. Single cohort studies and RCTs were assessed using different indicators of bias under the 5 categories of possible bias as outlined in the *Cochrane Handbook of Systematic Reviews* (29), and a overall assessment of bias as 'Low' or 'High' given for each study. Where not enough details were reported in the study to assess risk of bias, this was labeled 'Unclear'. Again, however, studies with some risk of bias were not excluded from the analyses but sub-group comparisons were conducted to assess whether results differed depending on bias.

## Analysis

#### Meta-Analysis

Due to a lack of comparable randomized controlled trials (RCTs) for anxiety, the meta-analysis focused on DMHIs targeting depression. For each RCT of a DMHI for depression the effect size indicating the difference between the intervention group and the control group at post-test was calculated in OpenMeta Analyst (30). Effect size calculations (Cohen's *d* or standardized mean differences) were conducted using the means and standard deviations of post-test scores on instruments measuring symptoms of depression such as the depression subscale of the Depression, Anxiety, Stress, Scale (DASS, 31) or the Centre for Epidemiological Studies Depression Scale (CES-D, 32). Scores on these measures immediately after the study were used rather than longitudinal follow-up scores due to a lack of consistent reporting across the studies. Sample sizes used were the number of participants with complete data in each group rather than intention-to-treat numbers. If effect sizes could not be calculated due to a lack of information, the study was excluded from the effectiveness analysis. Pooled mean effect sizes were calculated using the random effects model as considerable heterogeneity between studies was expected. These were calculated separately for studies using a no-intervention control group and those using an active control group. Subgroup analyses were conducted to investigate the impact of other variables on effectiveness by testing for significant differences between subgroups using a fixed effects model. In particular a sub-group analysis according to risk of bias was conducted to assess how study design contributed to outcomes. Since one area of interest in this review was to investigate whether DMHIs are useful for the high percentages of young people not obtaining professional help, we also conducted a sub-group analysis according to the level of supervision or interaction involved in the treatment condition. FIGURE 1 | PRISMA flowchart of article selection.

#### Narrative Synthesis

All studies including those not in the meta-analysis were evaluated using a narrative synthesis model (33). Qualitative data included both results of interviews with participants, and descriptions of survey results relating to the appeal of DMHIs. For example, some studies reported direct comments from participants about their experiences using the DMHI, while others reported levels of agreement with statements about appeal and quality. These data were coded by 2 researchers using standard techniques for thematic analysis to generate an understanding of features that are appealing and aspects that promoted engagement and adherence (34). Codes were grouped to detect patterns, and themes were identified and defined. Once consensus was reached the lead author prepared a narrative analysis, which was checked independently by the other authors.

### RESULTS

### Study and Participant Characteristics

The 41 studies included 11 from Australia, 10 from the U.S., 12 from other English speaking countries, 5 from Northern Europe, 2 from Asia, and one from South America. The majority of studies were RCTs (n = 27) (**Table 1**), while 13 were single cohort studies (including 4 with pre-post designs), and one used a case study methodology. Most of the studies used participants recruited from educational institutions (n = 20), 9 were conducted in mental health care settings, 4 with the general community, 4 in primary care settings, 2 in hospitals, and one in a youth organisation. One study did not report recruitment methods or participant information in enough detail to determine the setting.

Studies included participants with no specific mental health symptoms at baseline (n = 12), some with varying levels of depression (mild to moderate n = 7, moderate to severe n = 3, severe = 1, all levels n = 6), others with diagnosed MDD (n = 2) or suicidal risk (n = 5). Participants with varying levels of anxiety were also a focus in some studies (mild n = 1, mild-mod n = 2, mod to severe n = 1, severe n = 1, all levels n = 1). Two studies looked at people with a variety of mental illnesses, and another focused on mood issues in people with traumatic brain injury (**Table 1**).

### Intervention Characteristics

Overall, 32 different DMHIs were investigated across the 41 papers (**Table 2**). Several DMHIs were evaluated in multiple studies including Bite Back (n = 2), CATCH-IT (n = 2), Master Your Mood (n = 2), MoodGym (n = 3), Reframe-IT (n = 3), SPARX (n = 2), and MEMO (n = 2). Several of these papers reported results from the same data sets (62, 63, 73, 74), but

TABLE 1 | Study and Participant Characteristics.


(Continued)


#### TABLE 2 | Intervention Characteristics.




reported on different aspects of the study and therefore both papers were included in the review. Most of the DMHIs drew on established therapeutic models, primarily Cognitive Behavioural Therapy (CBT) (n = 28) or a combination of CBT with other models.

The technologies utilized in the various DMHIs included some phone-based interventions such as text-messages (n = 4) and smartphone applications containing assessment tools and/ or psychoeducational materials (n = 3). The majority of DMHIs were web-based (n = 30), including many with online modules, learning materials or activities (n = 24), group chats or courses (n = 2), online forums (n = 2), and online chat facilities with a mental health professional (n = 2). Others were computer-based but not online, including games (n = 5) and psychoeducational computer programs (n = 2).

Many DMHIs included learning modules (n = 18), interactive learning activities (n = 6), psychoeducational materials in a variety of formats including text and video (n = 7), or game-based learning activities (n = 4). Additional features included regular inspiring messages (n = 1), challenges (n = 3), mood tracking (n = 3), or diary/journals (n = 2). Four studies included DMHIs with an accompanying workbook for participants or their parents. Only 11 of the studies reviewed included DMHIs that were entirely self-help and were completed in the participant's own time. The rest of the studies involved interaction with a mental health professional or completion of the intervention in some kind of supervised setting. Two of these were completed in hospitals, one with some minimal supervision from a health professional (36). Others were completed in a school setting (n = 10). Some of the studies completed at school involved a high level of supervision (n = 5) such as in studies where the intervention was presented by the school wellbeing staff (47), the classroom teacher (39), in the presence of the research team (62, 63), or in class groups (65). Other school-based studies involved lower levels of interaction with a therapist or the research team (n = 3) such as moderation of an online discussion board by a therapist (38), initial training completed at school but the intervention otherwise used in the students' own time (44), and where the intervention was accessed at school with minimal supervision by the school counselor (68). Studies outside of school settings included DMHIs that could be completed at home in the participant's own time, but included interactions with a therapist such as sending reminders or text messages (n = 4), or participating in online group courses or chats (n = 9).

### Effectiveness of DMHIs

The effectiveness of various DMHIs in treating symptoms of depression was compared to a control group in 15 studies (**Table 3**). Nine of these studies compared DMHIs to no intervention (a waitlist control group), while five of the studies compared DMHIs to an active control in which some alternative online materials were used, including one with some psycho-educational content (68), and one contained a Treatment As Usual (TAU) comparison group in which face-to-face counseling was offered (56). Since the TAU group in this case included active treatment it was combined in analysis with the active control groups. The pooled effect size of studies comparing the intervention group to a no-intervention group (n = 9) was 0.33 (95% CI 0.11 to 0.55) (**Figure 2**), suggesting that DMHIs have a small effect size when compared to a no intervention control group, while the pooled effect size of studies comparing the intervention group to an active control group (n = 6) was 0.14 (95% CI -0.04 to 0.31). Thus this review did not find a difference in outcomes between DMHIs and active controls, including a mixture of usual care for depression and non-depression specific interventions (**Figure 3**). Heterogeneity was relatively high (*I2 =* 70%) and statistically significant (p < .001). Two studies had negative effect sizes indicating that the control group had lower depression scores at post-test than the intervention group (59, 74). Reid and colleagues (59) evaluated the effectiveness in comparison to a waitlist control group of a smartphone application that allowed self-assessment on 8 domains of mood and functioning, referring this information to general practitioners for medical review. No significant effect on depression was found at post-test, but increased emotional selfawareness was reported. Whittaker and colleagues (74) similarly used a phone-based approach, delivering multimedia messages based on CBT and comparing this to use of similar multimedia messages with no focus on depression. The authors concluded that there was no evidence of benefits superior to the active comparison program with content about healthy behaviours.

Sub-group analyses were conducted to investigate the effect of therapist interactions and study completion settings on the outcomes in the 9 studies that compared the intervention to no intervention (**Table 3**), and the 6 studies comparing the intervention to an active control group. Studies were categorised as having High levels of human interaction (H) if they involved direct contact with a therapist or were completed in supervised settings such as a lab, clinic or school (47, 51, 55, 65, 66, 68, 70). They were categorised as Low interaction (L) (43, 49, 53) if they had some limited interaction such as regular emails, text messages, or optional opportunities to contact a therapist, or No interaction (N) (52, 54, 56, 59, 74) if they did not involve any

#### TABLE 3 | RCTs included in meta-analysis.


PHQ, Patient Health Questionnaire; DASS, Depression, Anxiety, Stress Scale, RADS, Reynolds Adolescent Depression Scale; CDRS-R, Children's Depression Rating Scale Revised; K6, Kessler 6; CES-D, Centre for Epidemiological Studies Depression Scale.

\* The study included active comparison groups as well, but only the comparison to the waitlist control group was included in this analysis.

C, control group; H, High level of interaction; involved direct contact with a therapist or were completed in supervised settings; I, Intervention group; L, Low level of interaction; limited interaction such as regular emails, text messages or optional opportunities to contact a therapist; N, No interaction; did not involve any interaction with a mental health professional and were completed unsupervised in personal time.

FIGURE 2 | Forest plot of meta-analysis of randomised controlled comparisons between DMHIs and no intervention for depression in adolescents.

interaction with a mental health professional and were completed unsupervised in personal time.

For studies comparing the DMHI to no intervention, the pooled effect size was smallest in the No interaction group (*d* = 0.04), and also small in the Low interaction group (*d* = 0.16), while the High interaction group returned a medium effect size (*d* = 0.52) (**Figure 4**). This indicates that DMHIs were mostly effective when they involved high levels of human interaction. The DMHIs in the No interaction group that did have a positive effect size were highly interactive, containing multimedia lessons (52), interactive online exercises (54), and game-based challenges and puzzles aiming to improve mental health literacy (56). Levin and colleagues (52) did not include direct conversations with mental health professionals. However, the system did send automatically generated emails that were customized based on participants' earlier input. This could have given the illusion of human interaction, thereby increasing the effectiveness of the program despite there being no direct personal communication. No significant differences were found according to intervention type or severity of symptoms at baseline.

Similarly, for the studies with active comparison groups, the pooled effect size was again smallest in the No interaction group (*d* = 0.08), and also small in the Low interaction group (*d* = 0.21), while the High interaction group returned a medium effect size (*d* = 0.49) (**Figure 5**). Both of the studies in the High interaction group were completed in school classroom settings (55, 68). Thus, across both active and no intervention control groups, effect sizes reached a moderate size only when there was a high level of therapist interaction or supervision in the study design.

### Risk of Bias

The proportion of studies at high/unclear risk of bias was: 34% selection bias (e.g. randomization or allocation concealment), 63% detection bias (e.g. blinding of outcome assessment), 41% attrition bias, and 31% selective reporting (**Table 4**). Overall 76% of the studies included in the review as a whole, and all but 4 studies (43, 66, 68, 74) included in the meta-analysis had some risk of bias. Sub-group analyses were performed on the 15 studies included in the meta-analyses according to type of bias. Significant differences were found for selection bias with the low

FIGURE 4 | Forest plot of sub-group analysis for randomised controlled comparisons between DMHIs with high, low or no support compared to no intervention for depression in adolescents.

FIGURE 5 | Forest plot of sub-group analysis of Interaction level for randomised controlled comparisons between DMHIs with high, low and no support compared to active control groups for depression in adolescents.

risk of bias group having a lower pooled effect size (*d =* 0.01, 95% CI -0.05 to 0.24) than the unclear or high-risk group (*d =* 0.44, 95% CI 0.22 to 0.69). Studies with low risk of detection bias also had a lower pooled effect size (*d* = .09, 95% CI -0.08 to 0.26) than the unclear or high-risk group (*d* = .40, 95% CI 0.18 to 0.62).

#### Attrition, Adherence, and Engagement

Attrition rates were defined as the number of participants who completed the study as a percentage of the participants who commenced the intervention (**Table 5**).Information about adherence and engagement (how much those who completed the study engaged with the intervention), tended to be reported differently across papers. For example, some papers reported module completion rates (36). Others reported time spent on a website (43). Overall 16 (39%) of the studies had attrition rates over 20%, the level broadly considered indicative of possible attrition bias (76) (**Table 5**). In several of these studies, while attrition rates were high, they were equal between groups (for e.g. 50) suggesting that drop out rates related more to recruitment methods than to non-engagement. However, this was not always the case and in many studies, even among those with low study attrition, engagement tended to be low,

#### TABLE 4 | Assessment of bias across all studies


with participants completing less than half of the intervention components (36, 45, 49, 51, 53, 73). The majority of these studies were again ones that involved completion in one's own time. For example, Ip and colleagues (2016) reported low drop out rates and a small effect size. However, participants only completed roughly three of 10 modules and spent about 39 minutes on the website over 4 months, suggesting relatively low engagement.

Qualitative data from all papers were categorized according to features of the DMHIs that were liked by participants, features that were disliked (**Table 6**), and features predicting adherence. There were four key categories of data in relation to liked features. The first category related to *social support.* Several studies reported that participants had found it useful to be in contact with professionals. Participants in one study who had access to a trained supporter in addition to regular text messages reported that they "liked talking to someone who was friendly" (35, p. 101). Participants in one study of a game-based CBT intervention called Pesky gNATs (41) reported liking the fact that doing it on a computer was "not as full on as face-to-face" (p. 15). However this was not perceived by all to be preferable to in-person contact. For example, two studies that evaluated an interactive CBT-based fantasy game called SPARX, both found that some participants preferred face-to-face support from a therapist (36, 56). Similarly, some participants in an online CBT-based group course reported a preference for a face-to-face version of the course (45).

For other participants, it was the opportunity to connect with peers who were experiencing similar difficulties that was helpful. Gerrits and colleagues (56) reported that participants "found chatting to be a pleasant and positive way to talk about being down and their feelings of depression" (p. 6). Similarly, Horgan and colleagues (48) studied the impact of an online forum and reported that participants found it "good to say what was going on aloud (albeit in writing)"

#### TABLE 5 | Attrition rates, sample sizes and indicators of adherence and engagement.


(Continued)


C, Comparator group; I, Intervention group; NR, Not reported.

(p. 87). One participant in the same study stated: "Its about empathy and the realization that you're not alone. That others are feeling the same way you do and are having trouble coping" (p. 87).

For some participants the primary attraction of DMHIs were their *online or computer-based* nature. In contrast to users noted above who reported a preference for face-to-face contact, for many users a key benefit of DMHIs was the privacy they afforded. One participant in an online self-help program, Crystal, stated: "You can kind of do it in a secluded area where nobody is watching you … the privacy is kind of like a really big appeal" (37, p. 28). In one study of an online peer-support website, anonymity allowed participants to share details that they had never shared before, and in fact had "put a lot of effort into hiding" (48, p. 87). DMHIs also had the advantage of fitting into the daily routines of users, connecting with current interests (40), and helping "to bring back a sense of normality" (36, p. 290). A participant named Rob stated: "Most teens are always on the internet … while you're on say Facebook or something, you can just open up another tab" (37, p. 28) It can also be accessed from a variety of locations such as school, home or in a clinic and participants could "learn by myself and at my own pace" (56, p. 7). Participants reported that it was "fun to be able to do it on a computer" (41, p. 13).

Other participants commented on particularly *useful content*. Participants reported that the DMHIs "showed me things I didn't know" (56, p. 7), and helped them learn more about mental health (58). Appreciation was expressed for content that helped participants to learn specific techniques such as problem solving and anger control (71), or challenging negative thoughts (37).

Another major category of the data related to the *look and feel* of the DMHIs. Participants preferred situations, characters, or avatars that were relatable. For example, participants reported that it was helpful when the focus was "situations any teenager goes through" such as school and interpersonal relationships (37, p. 27). Conversely, several studies reported that drop-outs occurred when the content "did not seem relevant for them" (54, p. 7). Other features that participants reported liking included interactive activities (58, 62), and video components (62, 73). Similar comments were made in

#### Garrido et al. Digital Mental Health for Young People

#### TABLE 6 | Liked and disliked features of DMHIs.


relation to DMHIs with a game-like feel. Participants stated that this made engaging with the DMHIs fun (36, 41). It was also important to the users that DMHIs were aesthetically appealing and easy to use and navigate.

One of the most prominent features that participants reported disliking was the *educational content* of many DMHIs. For example in their case study of use of a smartphone app for anxiety, Whiteside and colleagues (72) reported that the participant "appeared less engaged and interested in the background educational content" (p. 86). Multiple other studies reported similar comments by participants, particularly non-completers (36, 45, 54, 56, 58, 73). Educational modules were viewed as too long (37), "tough and sometimes quite tiring" (45, p. 6), "tedious and laborious" (38, p. 6). Some participants argued that it would be more convenient to be able to tailor modules to one's own needs: "I didn't like that you couldn't skip out of something if you already understood the concept"(58, p. 163). Burckhardt and colleagues (2015) suggested that more structured settings and dose effects may have contributed to their negative results, since other studies have found that the number of activities participants are required to do can reach a saturation point (77).

A noteworthy point was that numerous participants reported that the DMHIs often felt *too juvenile or patronising.* Participants did not enjoy using DMHIs that seemed like they were designed for younger children (41, 68). One participant suggested: "make it more grown up" (41, p. 14).

*Technical glitches and difficulties navigating sites* were also frequently cited as reasons for low adherence and engagement (37, 54, 68, 73). Participants stated that DMHIs should be improved to make them "comparable to commercially available games" (36, p. 290). Others reported disliking particular *aesthetic features* such as the colour scheme and a lack of variety of icons, cartoons, and diagrams (37).

*Factors predicting adherence.* Only four studies reported *predictors* of adherence to the DMHIs. Neil and colleagues (57) compared a school-based completion setting for MoodGYm to a community setting, finding that a school-based setting predicted greater adherence. Gender was also a consistent predictor of adherence, with females being more likely to complete compared to males (51, 57, 70). Mental health also played a role, with higher pre-test scores in depression (57), a longer history of mood disorders (51), or low scores in anxiety at pre-test (70) predicting greater adherence.

### DISCUSSIONS

This review aimed to determine the types of DMHIs that are effective in treating depression and anxiety in young people and the components of these interventions most associated with positive outcomes and engagement. Overall, studies in relation to depression demonstrated a small effect size in favour of DMHIs when interventions were compared to no intervention. While this might not always reflect a clinically significant level of change, it suggests that such DMHIs may be of value in the context of public health and preventative interventions. On the other hand, studies comparing DMHIs to active control conditions were not effective. In fact, in two studies the control group actually had lower depression levels at posttest than the intervention group. Both studies included phone based interventions: an app that referred users for medical review, and a program of multimedia mobile phone messages. However, given that there was a risk of bias in many studies included in the metaanalyses, these results should be interpreted with caution. In fact, the two studies in the meta-analysis that reported negative effect sizes were two of the only three studies assessed as having low risk of bias. Studies which did not involve blinding of either group allocation or of outcome assessment tended to have higher effect sizes than studies with low risk of bias, indicating that methodological limitations of the studies reviewed likely inflated the larger effect sizes.

Of further importance in our findings was the fact that only DMHIs involving regular interactions with a therapist or that were completed in a supervised setting reached a moderate effect size in comparison to a no-intervention control group, while DMHIs that involved educational programs completed in the participant's own time were not found to be effective in this study. This suggests that currently available DMHIs may not be effective in causing clinically detectable levels of change unless they involve a high level of supervised use or therapist involvement. These results reflect the overarching significance of human interaction in psychological interventions (78). However, the preference among some participants for human contact revealed in this review existed in tension with the need for privacy and anonymity, suggesting that there is a need for more effective design of DMHIs to fill a gap that traditional face-to-face therapies do not.

Despite this, adherence and engagement rates tended to be low in many studies particularly those where interventions were completed in their own time. This reinforces the idea that many DMHIs are most likely to be useful for people already receiving mental health support or at least those not averse to doing so. However, DMHIs completed in settings such as schools, labs, or clinics cannot reliably indicate their effectiveness in reaching young people outside of these settings. Where DMHIs were completed in their own time or did not include interaction with a mental health professional, effectiveness was much reduced. Nevertheless, for many users it is the anonymity and privacy afforded by the online context that holds the greatest appeal. Therefore, there is a need to balance the competing advantages of anonymity and social support. Other studies have similarly concluded that social networking features in DMHIs are a "gamble" due to the potential for both negative and positive effects (79).

These results indicate two distinct needs in DMHI development. Firstly, a dire need exists to increase the appeal of DMHIs so as to reach the 80% of young people who are not already obtaining professional help. These young people may not understand that their symptoms indicate the need for mental health assistance. Other barriers such as a lack of energy or motivation to engage with complex tasks, or a fear of the stigma of mental illness may prevent them from accessing even DMHIs where such are overtly about mental health or are educational in nature. Research indicates that young men are particularly unlikely to receive professional help (80), which makes the findings in this review that males are less likely to engage with DMHIs than females particularly disturbing. The development of DMHIs that build on the existing interests of young people in non-confronting ways may be of more appeal to this group. DMHIs that particularly cater to the interests of young men are especially needed. Only highly interactive DMHIs involving multimedia materials or game-based activities were successful in studies with low levels of human interaction, suggesting that these types of features are highly appealing.

Secondly, for DMHIs designed to help young people who are already seeking or willing to seek professional assistance but prefer to do so in the relative anonymity of digital settings, it is clear that the help of schools and mental health professionals is a crucial part of the roll out of such interventions. Similarly, some scholars have recommended a model of 'supportive accountability', in which accountability to a supporter or coach can enhance adherence to eHealth interventions (81).

The need for further refinement to available DMHIs was confirmed by our qualitative analysis which revealed factors involved in high drop out rates and low engagement rates in numerous studies. In general, participants liked the online or computer based formats and game-like feel of some DMHIs, particularly when the content was interactive, had relatable situations or characters, and had appealing aesthetic features. However, feedback by study participants or people who withdrew from the studies referred to the boring nature and hard work involved in the online learning modules. Participants found DMHIs with non-appealing interfaces, frequent technical glitches, or material that seemed too juvenile to be off-putting. Other studies have similarly found that making DMHIs easy to use and to

navigate is important to users (82), and in fact criteria like this are commonly used to evaluate usability and appeal (83).

Although some participants seemed to appreciate the opportunity to learn various psychological skills to improve their wellbeing, many disliked the high educational focus of the interventions, the fact that they seemed designed for children much younger, and that they did not match commercially available programs in quality. This highlights the need for future DMHIs to consider the opinions of young people closely in their design. There is a need for feedback from young people and co-design methods to ensure that both content and aesthetics are appealing to the target audience. There is also a need for DMHI developers to ensure that materials are not 'dumbed down' and that they are presented in a way that does not feel like hard work but that builds on the natural interests of young people. This again presents a challenge for developers, to balance the need for simplicity of use with ageappropriate content. This is especially important for DMHIs designed to address depression and anxiety since these conditions are associated with both a lack of motivation (84) and impaired concentration (85), making such users a particular challenge to engage.

The current study was limited by the search terms used. Future reviews should also include search terms such as "internet-delivered", "computer", or "computerised", since this could have picked up a broader range of studies in the current review. Nevertheless, this review demonstrates that while somewhat effective for those who use them, DMHIs fail to appeal to a large proportion of young people. In fact, when compared to active comparison groups including online materials with no psycho-educational content, DMHIs had only minimally better effects. As yet there seems to be a dearth of DMHIs that are likely to attract the large numbers of young people with mental illness who are not already open to receiving professional help. There is thus a need for urgent attention to developing high quality DMHIs that address the weaknesses and focus on the strengths identified, to help young people currently in the shadows to access appealing and accessible tools for managing their mental health. There is also a need for methodologically robust double-blinded RCTs to be designed to provide more stringent testing of the effectiveness of such interventions.

### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

## AUTHOR CONTRIBUTIONS

All authors contributed to the conception and design of the review. SG, CM, and DC were involved in searches, quality assessment and data extraction. SG and CM conducted the analyses. All authors contributed to manuscript revision and read and approved the submitted version.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00759/ full#supplementary-material

### REFERENCES


school-based population. (Report). *J Affective Disord* (2013) 147(1-3):338. doi: 10.1016/j.jad.2012.11.036


adolescents seeking help for depression: randomised controlled noninferiority trial. *Br Med J (Online).* (2012) 344:e2598. doi: 10.1136/bmj.e2598


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Garrido, Millington, Cheers, Boydell, Schubert, Meade and Nguyen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Video to Home Delivery of Evidence-Based Psychotherapy to Veterans With Posttraumatic Stress Disorder

Derrecka M. Boykin1,2,3\*, Fallon Keegan<sup>4</sup> , Karin E. Thompson1,3,5, Emily Voelkel 3,5, Jan A. Lindsay 1,2,3 and Terri L. Fletcher 1,2,3

<sup>1</sup> VA South Central Mental Illness Research, Education and Clinical Center, Houston, TX, United States, <sup>2</sup> Houston VA HSR&D Center for Innovations in Quality, Effectiveness, and Safety, Michael E. DeBakey VA Medical Center, Houston, TX, United States, <sup>3</sup> Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, United States, <sup>4</sup> Department of Psychology, University of North Texas, Denton, TX, United States, <sup>5</sup> Michael E. DeBakey VA Medical Center, Houston, TX, United States

### Edited by:

Lina Gega, University of York, United Kingdom

#### Reviewed by:

Sally Brabyn, University of York, United Kingdom Gemma Wilson, Northumbria University, United Kingdom

\*Correspondence: Derrecka M. Boykin Derrecka.Boykin@va.gov

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 02 May 2019 Accepted: 13 November 2019 Published: 06 December 2019

#### Citation:

Boykin DM, Keegan F, Thompson KE, Voelkel E, Lindsay JA and Fletcher TL (2019) Video to Home Delivery of Evidence-Based Psychotherapy to Veterans With Posttraumatic Stress Disorder. Front. Psychiatry 10:893. doi: 10.3389/fpsyt.2019.00893 Background: The Veterans Health Administration (VHA) has pioneered the implementation of video to home (VTH) technology to increase access to mental health treatments for Veterans facing barriers to receiving in-person care, particularly for posttraumatic stress disorder (PTSD). Randomized controlled trials have established the noninferiority of evidence-based psychotherapies (EBPs) for PTSD delivered through VTH, compared to in-person delivery. Less is known about the use of VTH to deliver EBPs for PTSD in routine clinical practice.

Objective: We examined the provision of EBPs for PTSD delivered via VTH at a large Southwestern VHA PTSD outpatient clinic.

Methods: Data were obtained from chart review of the electronic medical records of Veterans receiving at least one session of Cognitive Processing Therapy or Prolonged Exposure via VTH in the VHA PTSD clinic during the study time frame.

Results: Fourteen providers (including six psychology trainees) delivered EBPs for PTSD via VTH between 2016 and 2018. Providers treated 74 Veterans (33.8% women) from diverse sociocultural backgrounds who ranged in age from 25 to 79. Each provider treated about 3.08 (± 2.18) Veterans using VTH, not including one provider who saw more than 30. A hybrid approach, in which VTH-delivery was coupled with in-person delivery, was used with 70.3% of Veterans across treatment (including sessions completed before initiation and after termination of the EBP). This demonstrates the versatility of VTH for meeting individual patient needs. Most EBP sessions (85.4%) were conducted over VTH. Despite Veterans attending an average of 6.85 (± 4.88) EBP sessions, 50% terminated before session 7. This dropout rate is consistent with national and local EBP completion averages within the VHA. Veterans receiving Cognitive Processing Therapy via VTH were more likely to complete treatment than those receiving Prolonged Exposure. No other patient factors predicted attrition.

Conclusions: This study highlights the use of VTH as "tool in the toolbox" that expands the scope of practice for providers and increases opportunities for Veterans to receive EBPs for PTSD. We describe other potential advantages of using VTH to deliver EBPs for PTSD.

Keywords: telehealth, telemental health, posttraumatic stress disorder, evidence-based psychotherapy, veterans

### INTRODUCTION

Evidence-based psychotherapies (EBPs), including Cognitive Processing Therapy and Prolonged Exposure, are considered first-line interventions for posttraumatic stress disorder (PTSD) and have strong support in Veteran samples (1–3). Despite the resources the Veterans Health Administration (VHA) has invested in these two therapies, Veteran access to and engagement in EBPs for PTSD remain a challenge (4). Relatively few Veterans with PTSD (9%) receive these EBPs (5–7), and approximately half who initiate these treatments terminate prematurely and do not receive a full course of therapy (8, 9). Notable barriers to accessing and engaging in EBPs include logistic issues such as taking time off work or school, arranging child or elder care, physical limitations, and transportation and travel costs (10, 11). Additional barriers include stigma around seeking mental health care (12) and clinical symptoms such as anxiety (13) that may interfere in seeking PTSD care at the VHA. Some women Veterans and Veterans who experienced military sexual trauma (MST) also express discomfort in VHA clinics (14, 15).

The provision of EBPs via video to home (VTH) technology is one promising option for overcoming barriers to care and increasing access to and retention in EBPs for PTSD. VTH is a safe and convenient alternative to in-person care and offers patients the opportunity to receive PTSD treatment in the comfort of their home, work, or other private location. Delivery of EBPs via VTH allows patients and providers to synchronously connect for live, interactive therapy sessions using a secure web-based videoconferencing feature on a personal computer, laptop, tablet, or other device. The content of the VTH-delivered EBPs for PTSD is identical to traditional inperson delivery.

Patients report high satisfaction with VTH-delivered care (16), and noninferiority trials demonstrate the clinical effectiveness of EBPs for PTSD delivered via VTH compared to in-person treatment, though sample sizes were relatively small (17, 18). Although these randomized controlled trials found no differences in dropout between VTH and in-person treatment, this may be because patients were randomized to a condition and were not able to select which delivery option they preferred. Nonetheless, among patients who terminated therapy prematurely, patients receiving care via VTH engaged in more sessions prior to dropout than patients who received in-person care (19).

Although these studies demonstrate the effectiveness of VTH-delivered PTSD treatment and suggest promise in increasing treatment retention, little is known about the current use of VTH to deliver EBPs for PTSD in routine clinical practice. A recent study found that, although VTH use is increasing across the VHA, growth is slow; and the number of Veterans receiving mental health care through VTH is still small (13). One successful model for implementing VTH for EBPs is to integrate VTH across multiple providers in a clinic, rather than designating certain providers (often in remote telehealth centers) to provide VTH. This integrated model allows patients and providers to collaboratively decide whether to conduct all EBP sessions via VTH, or whether a hybrid approach of both VTH and in-person sessions is preferable. This Personalized Implementation of Video Telehealth approach has demonstrated success in increasing both provider and patient adoption of VTH for mental health treatment (13, 20).

The goal of this study was to examine the provision of EBPs for PTSD delivered via VTH within an outpatient PTSD clinic at the Houston Veterans Affairs Medical Center that uses an integrated model of VTH implementation. Specific aims were to: 1) describe demographic and clinical characteristics of Veterans receiving VTH-delivered EBPs for PTSD, 2) report the number of PTSD clinic providers delivering these EBPs via VTH during the study period, 3) report the average number of Veterans each provider treated via VTH, 4) report the percentage of Veterans who engaged in the EBP for PTSD exclusively via VTH vs. both VTH and in-person, and 5) report rates of VTH EBP for PTSD completion. We also examined known predictors of in-person EBP for PTSD completion including demographic characteristics (e.g., gender, race/ethnic minority), symptom severity, comorbid substance use disorder, service-connection for a disability, and military service era (6, 21, 22) to determine whether these variables predicted VTH EBP for PTSD completion. Given noted differences in completion of Cognitive Processing Therapy compared to Prolonged Exposure, we also included EBP treatment type as a potential predictor (8).

### METHODS

#### Treatment Setting

The VHA PTSD outpatient clinic is a subspecialty clinic located within the General Mental Health Clinic. The VHA PTSD Clinic at the Houston Veterans Affairs Medical Center had 9 full-time VHA providers and 1 half-time provider (7.5 psychologists and 2 social workers) and employed two to three psychology interns and fellows at a time (their cases are included here). VHA PTSD clinic staff and trainees treat Veterans with a primary diagnosis of PTSD or Other Trauma/Stressor-Related Disorder. Of note, a diagnosis of military-related PTSD is not required for enrollment and Veterans are not excluded for having comorbid psychiatric diagnoses. The VHA PTSD clinic routinely offers Cognitive Processing Therapy and Prolonged Exposure as first-line, gold standard EBPs for PTSD. The VHA PTSD clinic staff were introduced to and gradually trained in VTH starting in 2015 as part of on-going implementation efforts to increase adoption of this clinical innovation (13).

#### Data Collection

We reviewed the electronic medical records of all Veterans enrolled in the VHA PTSD clinic who completed at least one individual EBP for PTSD session via VTH between January 1, 2016, and December 31, 2018. The study protocol was reviewed and approved by the Institutional Review Board at Baylor College of Medicine and Affiliated Hospitals in addition to the Research and Development committee at the Houston Veterans Affairs Medical Center. Because this study involved a retrospective review of patient medical records, a waiver of informed consent/HIPAA authorization was required and granted. De-identified data (described below) were collected in a SPSS database securely stored behind a VHA firewall only accessible by the principal investigator and approved research staff. Identifiable information was only used to collect aggregate data from the medical records and no identifiable data was downloaded, copied, or printed.

#### Veteran Demographic and Clinical Characteristics

Demographic data were age (in years), gender, race, ethnicity, marital status, academic status (i.e., college student vs. nonstudent), employment, rural or urban residence, military service era, deployment status, and VHA service-connected disability status for PTSD. Veterans' residency zip codes were classified as rural or urban according to the Rural-Urban Commuting Area Codes (23). Information regarding comorbid medical and psychiatric diagnoses was taken from Veterans' problem lists. Medical diagnoses were weighted and summed to compute the Charlson Comorbidity Index, an indicator of disease burden and mortality risk (24). We also tracked whether Veterans had been prescribed recommended medications for PTSD (25).

#### Treatment History

Veterans' treatment history was gathered from two time intervals —the year preceding admission into the VHA PTSD clinic and during the most recent episode of care. The most recent episode of care was defined as the period during which the Veteran completed at least one EBP for PTSD session via VTH. Key variables collected at both time intervals were: use of mental health services within this or other VHA clinics (e.g., primary care), medication management visits, use of emergency services (including crisis line calls), and inpatient hospitalizations. Veterans who received psychotherapy (either group or individual) in the VHA PTSD clinic prior to their most recent episode of care were distinguished from Veterans who were newly admitted or never received treatment after admission into the clinic.

#### EBP Treatment Delivered via VTH

We extracted treatment information about the most recent episode of care. Provider information, EBP type (Prolonged Exposure or Cognitive Processing Therapy), and nature of primary or index trauma were recorded. Other treatment variables were number of individual sessions (including sessions completed before initiation or completion of EBP), number of EBP sessions, number of missed and cancelled sessions, and notable VTH connectivity issues (e.g., "session terminated early due to poor audio quality"). We differentiated between sessions delivered in-person vs. those delivered through VTH. A 7-session benchmark was used to define EBP completion based on existing definitions of EBP for PTSD completion (6, 26–28). Initial and last recorded PTSD Checklist for DSM-5 severity scores (29) were collected. Unfortunately, the last recorded PTSD severity scores were available for 24 of 74 Veterans and, thus, we omitted them from data analysis.

### Data Integrity

Deidentified chart review data were entered into a shared project database by two research assistants, who had experience conducting VHA chart review research and had obtained thorough training for the current study. The first author (D.B.) oversaw chart review and met regularly with the research assistants to resolve any emergent problems. For any issues, the first author reviewed the individual charts, and the group reached a consensus.

#### Data Analyses

Descriptive statistics of Veteran characteristics and treatment history, provider characteristics, and clinical use of VTH relative to in-person service delivery of EBPs were reviewed. We used logistic regression analysis to examine predictors of EBP completion. Predictors were Veteran demographic and clinical characteristics (e.g., race/ethnicity, service era, and medical/ psychiatric comorbidities), EBP type, index trauma type, and prior treatment history. Due to concerns about adequate power, predictors for the logistic regression analysis were purposefully selected by first conducting a univariate analysis of each predictor to determine which variables should be included in the final multivariate model. Predictors that had a significant univariate test at p ≤ 0.25 process were included in the final model (30).

### RESULTS

#### Veteran Characteristics and Treatment History

A total of 74 Veterans (49 men, 25 women) received an EBP for PTSD via VTH within the VHA PTSD clinic during the study time fame. Veteran demographic and clinical characteristics are described in Table 1. Veterans ranged in age from 26 to 79 (M = 42.01 ± 11.71). Most Veterans were Caucasian or African American, non-Hispanic/Latino, married, employed, receiving


N, 74; VHA, Veterans Health Administration; OIF/OEF, Operation Iraqi Freedom/Operation Enduring Freedom; PTSD, Posttraumatic Stress Disorder.

a n = 73. <sup>b</sup> n = 64. <sup>c</sup> n = 63.

VHA service connection for a PTSD disability, and living at an urban residence at the time of admission into the clinic. Most Veterans had been deployed to a combat zone with many serving during the Operation Iraqi Freedom/Operation Enduring Freedom, or Gulf War eras. Almost all Veterans (73 of 74) had co-occurring psychiatric diagnoses, most often depression (52.7%) or a substance/alcohol use disorder (24.3%). Fewer Veterans (23 of 74) had significant medical comorbidities. Overall, this sample had low Charlson Comorbidity Index scores indicating low overall disease burden and mortality risk. Combat (54.1%) was the most commonly identified index trauma, followed by MST (20.3%); other military, noncombat trauma (16.2%; e.g., training accidents); and nonmilitary, civilian trauma (9.5%).

Table 2 describes Veterans' treatment history as it relates to use of outpatient psychotherapy, medical management, emergency services, and inpatient hospitalization prior to and during individual treatment in the VHA PTSD clinic. Most Veterans had received outpatient psychotherapy (including group therapy) prior to their most recent episode of care in the VHA PTSD clinic. Notably fewer Veterans were participating in concurrent outpatient psychotherapy once they started an EBP for PTSD. The majority of Veterans (82.4%) were taking TABLE 2 | Percent of Veterans receiving VHA Services before and during VTH Administration in VHA PTSD Clinic.


N, 74; VHA, Veterans Health Administration; PTSD, Posttraumatic Stress Disorder.

psychotropic medications for PTSD; however, only a quarter used medication management services during the study time frame. Less than 10% of Veterans sought emergency services (including contacting the crisis line) or were hospitalized for urgent mental health crises.

#### Clinical Utilization of VTH

The number of clinic providers delivering EBPs for PTSD via VTH steadily increased from 2016 to 2018. In 2016, four clinic providers (including one trainee) used this treatment modality with at least one patient. Seven additional providers (including four trainees) delivered EBPs via VTH in 2017, followed by three more providers (including one trainees) in 2018. Altogether, 14 providers (8 of 9.5 clinical staff and six trainees) administered EBPs for PTSD using VTH during the study period. On average, each provider delivered care to approximately 3.08 (± 2.18) Veterans via VTH, not including one provider who saw more than 30 Veterans. Between 24 and 26 Veterans were seen each year during the study time frame.

Including sessions completed before initiation and after termination of an EBP for PTSD, Veterans were seen for 11.08 (±11.19) total sessions. A minority of Veterans (29.7%) completed individual sessions that were delivered exclusively through VTH. Most often, a hybrid approach in which VTHdelivery was coupled with in-person delivery was used. Of those receiving this hybrid approach (n = 52), 67.8% of sessions were delivered via VTH. Fifty-seven of 74 (77%) Veterans missed or cancelled an average of 2.77 (±2.59) scheduled sessions. Technical difficulties associated with VTH ranged in severity from temporary audio or video loss to major broadband issues (e.g., lost wireless connection). Significant technical difficulties that disrupted the quality of sessions (e.g., persistent loss of audio or video) or led to early termination of a session were experienced once or twice during treatment by 54.0% of the sample.

### EBP Completion During VTH Delivery

Cognitive Processing Therapy was administered more often than Prolonged Exposure (75.7% vs. 24.3%, respectively). Due to the small number of Veterans receiving Prolonged Exposure (n = 18), Prolonged Exposure and Cognitive Processing Therapy were examined together for the remaining analyses. Veterans attended an average of 6.85 (±4.88) EBP for PTSD sessions, with 85.4% of these sessions conducted over VTH. Thirty-seven (50%) Veterans completed treatment (i.e., attended seven or more EBP sessions).

Univariate and multivariate logistic regression results are shown in Table 3. Predictors of treatment completion that had significant univariate logistic regression results were gender, service-connected PTSD disability, service era, EBP type, index trauma type, prior psychotherapy in the VHA PTSD clinic, and prior outpatient psychotherapy (see Table 2). When these variables were placed into a multivariate logistic regression model, only EBP type remained a significant predictor at p < .05 (see Table 2). This finding suggests that Veterans receiving Cognitive Processing Therapy via VTH were more likely to complete treatment than Veterans receiving Prolonged Exposure.

#### DISCUSSION

The current study highlights the broad reach of VTH technology in improving access to EBPs for Veterans with PTSD in routine clinical practice. Our findings support a growing emphasis on clinical expertise and competence in delivery of EBPs via VTH as opposed to predetermined criteria to identify which patients might benefit from this treatment modality (31). The present sample was fairly heterogeneous in terms of clinical complexity, yet representative of Veterans who seek VHA care (32). In the past, patients presenting with complex, chronic comorbidities might have been deemed ineligible for VTH services due to concerns about clinical appropriateness and safety. Now, telemental health services are considered a topmost strategy for engaging difficult-to-reach populations, such as Veterans with PTSD (33). VTH eliminates many practical barriers (e.g., transportation issues, long distance from facility, inflexible work schedule) and VHA-system-related barriers (e.g., inflexible clinic scheduling, discomfort with physical VHA environment, overcrowding) that impact utilization of EBPs for PTSD. There are also potential advantages to seeing patients in their home environments, where they may feel safer discussing trauma-related issues. It is, thus, important not to limit use of VTH to straightforward clinical cases (e.g., physical limitations or disabilities) and potentially miss critical opportunities to reach patients who might otherwise not receive care.

Patients from diverse demographic and clinical backgrounds consider home-based videoconferencing technology, such as

TABLE 3 | Logistic regression predicting completion of EBPs for PTSD delivered using VTH.


N, 74; OIF/OEF, Operation Iraqi Freedom/Operation Enduring Freedom; PTSD, Posttraumatic Stress Disorder; EBP, Evidence-based Psychotherapy; CPT, Cognitive Processing Therapy; PE, Prolonged Exposure; MST, Military sexual trauma.

Frontiers in Psychiatry | www.frontiersin.org

VTH, highly satisfactory and acceptable (31, 34). The current sample included a mixture of younger and older Veterans from a range of racial/ethnic backgrounds, service eras, etc. There was also a high percentage of women (33.7%), a historically underserved group within the VHA (14.1%; 35). These data challenge preconceived notions about which patients may be interested in receiving technology-based services. Therefore, providers who are or will be integrating VTH into their clinical practice should consider routinely offering this service to patients as an option for care delivery.

The Houston Veterans Affairs Medical Center adopted an integrated model of VTH delivery in which most providers (84.2% of clinical staff and six psychology trainees) offered VTH as one delivery option, rather than designating a smaller number of providers who exclusively offered telehealth services (13). The inclusion of trainees as VTH providers further illustrates efforts to integrate VTH as a standard care option available to all patients by all providers within this clinic. Having multiple providers who can offer VTH decreases the burden of trying to meet diverse patient needs with one or two designated VTH providers. Clinical settings with more than one VTH provider are also better equipped to manage whenever patient needs change (e.g., unexpected changes in work schedule or family obligations). This approach also ensures continuity of care by allowing patients to receive in-person and VTH care from a single provider. As providers in this study demonstrated, integrating VTH-delivery with in-person delivery capitalizes on the flexibility of VTH as an as-needed treatment delivery option that truly promotes a patient-centered model of care that optimizes patient choice (31).

The Houston VHA PTSD clinic, as part of a multisite implementation project (13, 33), received external guidance and support that, in collaboration with the clinic director as a local advocate, facilitated provider adoption of VTH. Another influential factor was the occurrence of Hurricane Harvey in 2017 and provider concerns about Veterans experiencing major disruptions in care in the aftermath of this natural disaster. Providers who had not yet used VTH were eager to learn how to integrate this treatment modality into their practice to minimize undue delays in care. One major advantage to working in VHA and other large healthcare systems is that providers have access to trainings, equipment, and other resources that support the uptake of VTH. For providers in other mental health settings, private practice, or rural communities, establishing a telehealth practice may require additional effort and careful attention to federal and state policies about reimbursement models for treatment delivered remotely [see Campbell et al. (36), see also Yellowless et al. (37)].

Attrition in this study (50%) was on par with local and national in-person EBP completion averages for Veterans with PTSD (6, 28, 38). Unfortunately, present data were insufficient for understanding what contributed to these attrition rates. Treatment response varies and, in the absence of treatment outcome data, we cannot conclude whether noncompleters represent early responders or Veterans who were nonresponsive to treatment (26). Attrition could have also been due to other unknown factors, such as patient relocation. Our data showed an association between EBP type and treatment completion. However, it cannot be discerned whether this finding is attributable to differences between the two treatments or an artifact of selection bias since patients are not randomized into a treatment condition in clinical practice.

Despite attrition rates comparable with in-person EBP delivery, VTH resolves many major deterrents (e.g., lack of timely access, discomfort with VHA facilities) that dissuade Veterans with PTSD from initiating EBPs (39). Put differently, VTH is reaching a broader audience of patients who might otherwise not seek in-person care. Providers can leverage the flexibility of VTH as a resolution for temporary and persistent logistical barriers to in-person care. Patients feel as though their individual needs are at the forefront of clinical care, which, in turn, promotes engagement. This may partly explain our finding showing a possible dose advantage for VTH. Dose advantage refers to the tendency for patients receiving treatment remotely to complete more sessions than those receiving in-person care (19). Veterans in our study completed at least six EBP for PTSD sessions, whereas dropout among Veterans receiving these EBPs in-person typically occurs before session 5 (8, 19, 40, 41). Offering VTH as-needed may, thus, promote increased retention in EBPs.

A relative strength of this study is that it provides a longitudinal view of VTH utilization for EBPs in a single setting from 2016 to 2018. Nevertheless, our findings should be interpreted in light of study limitations. First, our data were derived from a single VHA PTSD clinic within a large VHA medical center and may not generalize to other VHA settings, such as smaller or more rural facilities. Of note, our sample size was relatively small, but mirrors the slow growth of VTH use within VHA nationally (13). Second, data were extracted from medical records that are limited by the validity of Veterans' reports, accuracy of clinicians' treatment records, and data integrity of the chart review. Finally, there was no comparison group of Veterans with PTSD receiving EBPs in-person in this specific clinic. Having the comparison group might have better elucidated differences between Veterans who used VTH and those who did not.

### GENERAL CONCLUSIONS AND FUTURE DIRECTIONS

Home-based telemental health services, such as VTH, are revolutionizing mental healthcare delivery. VTH affords a high degree of flexibility that optimizes patient choice and eliminates major disruptions in care due to practical or logistical issues. With VTH, providers can remotely connect with a broad range of patients to deliver patient-centered, evidence-based care that might have otherwise been inaccessible. Despite these advantages, widescale adoption of VTH into clinical practice remains slow (13). More research on uptake and utilization of VTH in routine clinical practice as well as factors that impact its adoption (e.g., provider attitudes, clinic structure/workforce size) is needed to facilitate implementation efforts. Future work should also address provider concerns and increase their comfort with VTH since provider attitudes have influenced the uptake of this innovation (42).

As VTH becomes more integrated into routine clinical practice, it will be important to gain a richer understanding of issues related to patient engagement and EBP completion. Our study indicated a potential dose advantage for VTH, although attrition rates were comparable to in-person delivery of EBPs. Future studies should consider mixed method research designs to better understand factors related to EBP engagement, especially factors that are unique to VTH administration. Knowing this information will inform best practice standards for VTH-delivery of EBPs and enhance the quality of care delivered through this modality.

#### DATA AVAILABILITY STATEMENT

The datasets for this study will not be made publicly available because the dataset contains Veteran personal health information. Privacy laws prevent us from sharing personal health information outside the VHA. The Michael E. DeBakey Veterans Affairs Medical Center will not provide unrestricted, open public access to large scale health related datasets because of re-identification concerns and the obligation to protect Veterans' private information.

#### ETHICS STATEMENT

The study protocol was reviewed and approved by the Institutional Review Board at Baylor College of Medicine and Affiliated Hospitals in addition to the Research and Development committee at the Houston Veterans Affairs Medical Center. Because this study involved a retrospective review of patient medical records, a waiver of informed consent/HIPAA authorization was required and granted. De-identified data were collected in a SPSS database securely stored behind a

#### REFERENCES


VHA firewall only accessible by the principal investigator and approved research staff. Identifiable information was only used to collect aggregate data from the medical records and no identifiable data was downloaded, copied, or printed.

#### AUTHOR CONTRIBUTIONS

All authors contributed to conception of the study. TF, DB, JL, and FK designed the study. DB and FK organized the database. DB completed data collection and performed statistical analyses. DB, FK, and TF wrote sections of the manuscript. All authors contributed to manuscript revision, read and approved the submitted version.

#### FUNDING

This work was partly supported by the use of facilities and resources of the Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIn13-413), the Office of Academic Affiliations, Advanced Fellowship Program in Mental Illness Research and Treatment, and the VA South Central Mental Illness Research, Education and Clinical Center. The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs, the U.S. government, or Baylor College of Medicine. None of these bodies played a role in study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.

#### ACKNOWLEDGMENTS

We would like to thank Armandina (Dina) Garza and Matt Escamilla for their contributions to data collection. We would also like to thank Amber (Amy) Amspoker for her guidance with statistical analyses.

time: a case example from a large Veterans Affairs posttraumatic stress disorder clinic. Psychol Serv (2018) 4:429–36. doi: 10.1037/ser0000138


VA outpatient clinic. Psychol Trauma (2016) 1:107–14. doi: 10.1037/ tra0000065


Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Boykin, Keegan, Thompson, Voelkel, Lindsay and Fletcher. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Easy Access, Difficult Consequences? Providing Psychiatric Patients With Access to Their Health Records Electronically

Gillian Strudwick 1,2, Anthony Yeung3 and David Gratzer 1,3\*

<sup>1</sup> Centre for Addiction and Mental Health, Toronto, ON, Canada, <sup>2</sup> Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada, <sup>3</sup> Department of Psychiatry, University of Toronto, Toronto, ON, Canada

#### Keywords: psychiatry, OpenNotes, medical records, electronic health record, mental health

#### Edited by:

Elias Aboujaoude, Stanford University, United States

#### Reviewed by:

Peter Yellowlees, University of California, Davis, United States

\*Correspondence: David Gratzer david.gratzer@camh.ca

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 03 May 2019 Accepted: 18 November 2019 Published: 16 December 2019

#### Citation:

Strudwick G, Yeung A and Gratzer D (2019) Easy Access, Difficult Consequences? Providing Psychiatric Patients With Access to Their Health Records Electronically. Front. Psychiatry 10:917. doi: 10.3389/fpsyt.2019.00917 Should psychiatric patients have access to their clinical notes (often referred to as OpenNotes)? For decades, mental health providers have debated this. Some have argued that open access to notes will undermine the therapeutic rapport, and possibly undermine care itself (1). Others have suggested that involving patients in their care decisions must include access to their records; in 2014, an editorial called on providers to "show patients their mental health records" (2). Following this publication, several psychiatrists opened up their clinical notes to patients electronically through patient portals and have been providing access ever since (3). Despite initial concerns from providers about what opening up their notes might to do therapy and care, these concerns have generally not been realized (4, 5). We believe that more effort needs to be made to support providers in feeling more comfortable opening up their notes.

Although patient portals are increasingly found in other areas of medicine, they are not available to the same extent in psychiatry. Progress has been made both in terms of: 1) the number of organizations that have opened up their psychiatric notes (6); and, 2) our understanding of the impact of opening up notes to both patients and providers (7, 8). Yet, in our opinion, the uptake of "opening up" mental health clinical notes remains slow. In large part, we believe that this is due to the ongoing discomfort and concerns that providers have had (9).

A recent study has shown that working in psychiatry is a predictor of discomfort in providing clinical notes to patients, with psychiatrists and those working in acute settings being particularly concerned (9). This does not come as a surprise as we are aware of the sensitive topics discussed during clinical encounters, and the stigma often associated with mental illness. In our experience talking to providers about this topic, we have received mixed feedback from those that have not yet opened up their notes. There are providers who see the merit in opening up clinical notes to support patient empowerment, and providers who are opposed to the idea (often adamantly) due to a number of clinical concerns. These concerns include possible documentation changes that influence the clinical utility of the notes (note "sanitization"); patients reading their notes in unsupported environments and becoming upset; providers answering more questions after hours; patients not understanding the content of their notes; clinical interactions taking more time as questions related to note content are discussed; concerns about providers being subjected to violence; and, the list goes on (5, 8–10).

Since there have now been implementations of psychiatric patient portals with OpenNotes in numerous organizations, the question is now: what have the experiences been of providers? Generally speaking the literature has shown that the concerns of providers who are less

comfortable with the idea of opening up their notes, have not been realized (4, 5). Although providers are concerned about a number of topics pre-implementation, the post-implementation reality seems incongruent with these initial concerns. One study showed that instead of the extensive time that providers worried about explaining the content of notes to patients, only about 15% of the time did patients even mention if they read their notes. In one study, patients who had access to their notes showed improvement in their self-reported mental health recovery scores, attended more appointments, and requested information less often from the health records department (possibly because they had access to the information they needed through the portal) (11). In this study, patients were provided a portion of their notes, and requested of the study researchers, increased access to the remaining notes in their record.

Although there is growing evidence that suggests that the concerns about opening up clinical notes to mental health patients may be unwarranted (and acknowledge that there may be some scenarios where granting access may not be as appropriate), we do believe that providers should be given more support with regards to how to make the transition to opening up their clinical notes. We believe it is important to validate concerns, yet identify opportunities to support providers in opening up their notes as comfortably as they can. We recommend that organizations use a toolkit developed on this topic by the OpenNotes team in Boston (12). In this toolkit, suggestions and vignettes for documenting sensitive topics are provided and could be utilized during training sessions. Organizations may wish to use a phased approach to open up notes. This could be done in a few ways. Certain note types (e.g.,

#### REFERENCES


outpatient areas first) could be made available right away, and the remaining notes types made available at a later time. Another solution could be enrolling a small number of patients initially to ease providers into the process. As well, there have been initial successes documented in the literature when veterans have been provided web-based education related to reading their mental health notes online (7). Support for patients that does not rely on providers, could be considered.

Although privacy laws, consent norms and the specific legal considerations vary from country to country, patients in numerous countries have had the right to access their health record for a number of years now. However, in many organizations it's been challenging and sometimes expensive to do so, and therefore providers aren't accustomed to having patients read their notes. These same consent and legal implications that are present in most organizations today remain when electronic access is granted.

As healthcare moves toward patient empowerment and autonomy, the OpenNotes movement is here to stay. Although mental health records do have unique concerns that should be considered, there are tools and strategies to help providers navigate this. It is time to make open medical records a reality for all patients, and to support providers in this transition.

#### AUTHOR CONTRIBUTIONS

GS and DG conceptualized the commentary. The initial draft of the opinion was written by GS with substantive editing and inputs from DG and AY.


Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Strudwick, Yeung and Gratzer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Internet-Delivered Cognitive Behavior Therapy as a Prequel to Face-To-Face Therapy for Depression and Anxiety: A Naturalistic Observation

Daniel Duffy 1,2\*† , Angel Enrique1,2\*† , Sarah Connell <sup>1</sup> , Conor Connolly <sup>1</sup> and Derek Richards 1,2

<sup>1</sup> Clinical Research and Innovation, SilverCloud Health, Dublin, Ireland, <sup>2</sup> E-Mental Health Research Group, School of Psychology, Trinity College, Dublin, Ireland

#### Edited by: Lina Gega,

University of York, United Kingdom

#### Reviewed by:

François Borgeat, Université de Montréal, Canada Pawel Izdebski, Kazimierz Wielki University of Bydgoszcz, Poland

#### \*Correspondence:

Daniel Duffy duffyd8@tcd.ie Angel Enrique enriquea@tcd.ie

† These authors share first authorship

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 03 May 2019 Accepted: 14 November 2019 Published: 09 January 2020

#### Citation:

Duffy D, Enrique A, Connell S, Connolly C and Richards D (2020) Internet-Delivered Cognitive Behavior Therapy as a Prequel to Face-To-Face Therapy for Depression and Anxiety: A Naturalistic Observation. Front. Psychiatry 10:902. doi: 10.3389/fpsyt.2019.00902 Background: The UK's Improving Access to Psychological Therapies (IAPT) program is a stepped-care model treating individuals with depression and anxiety disorders. Internetdelivered cognitive behavioral therapy (iCBT) is routinely offered to individuals with mild to moderate symptoms, but its applicability to individuals with severe clinical symptoms and requiring a high-intensity intervention is relatively unknown. The current study sought to investigate the potential impacts of using iCBT as a prequel for patients requiring highintensity treatment (HIT; face-to-face) for depression and anxiety in IAPT.

Methods: The study utilized an open study design. One hundred and twenty-four participants who were on a waiting list for high-intensity, face-to-face psychological treatment were offered iCBT. Psychometric data on symptoms of depression, anxiety, and functioning were collected from participants before starting and on finishing iCBT and at the point of service exit. Therapeutic alliance data were collected from patients and clinicians during treatment. Patient pathway data, such as number of treatment sessions and time in treatment, was also collected and incorporated into the analysis.

Results: Significant reductions across primary outcome measures of depression and anxiety, as well as improved functioning, were observed from baseline to iCBT treatment exit, and from iCBT exit to service exit. Analysis of the therapeutic alliance data for patients and clinicians illustrated differences in outcome for those who dropped out and those who completed treatment.

Discussion: This study illustrates the potential for using iCBT as a prequel to highintensity therapy for depression and anxiety disorders and is the first of its kind to do so within IAPT stepped care. The results show that iCBT is a valuable option reducing waiting times and enhancing clinical efficiency. The study contributes to the well-established evidence on online psychological treatments worldwide, but further clinical and service development research is necessary to scale these treatments appropriately.

Keywords: depression, anxiety, internet-delivered interventions, internet-delivered cognitive behavioral therapy (iCBT), Improving Access to Psychological Therapies (IAPT)

### INTRODUCTION

According to the Global Burden of Disease study, depression and anxiety disorders contribute the greatest degree of disability amongst all mental and substance abuse disorders (1). Psychological therapies have been shown to be both clinically and cost effective in the treatment and management of depression and anxiety disorders (2) and preferred by some over pharmacological interventions (3). In recent years, technology has facilitated the dissemination of psychological therapy, particularly cognitive behavior therapy, through internet-delivered interventions for depression and anxiety (4). Based on the available evidence for the efficacy of internetdelivered cognitive behavioral therapy (iCBT) for depression and anxiety across multiple meta-analyses (5–7), these interventions are currently deployed in a supported manner as part of routine care in mental health clinics in several countries (8). The literature shows that clients value the helpfulness of supporters, as they encourage and motivate clients to keep using the intervention as well as provide helpful guidance and feedback, which contribute to enhanced outcomes (9, 10, 11) Furthermore, iCBT is included as a treatment option in the UK clinical guidelines for the treatment of depression and anxiety disorders (12, 13).

Different authors advocate for the inclusion of iCBT interventions within stepped mental healthcare models (14, 15). Stepped-care models seek to match treatment intensity to client needs by providing the least intrusive and most effective intervention for the client upon entering services (16). The Improving Access to Psychological Therapies (IAPT) program is a stepped-care approach to psychological care for people with depression and anxiety within the National Health Service (NHS) in the UK (17). Specifically, the IAPT model delivers low-intensity psychological interventions (i.e. iCBT, guided selfhelp) (18) at step 2 (mild to moderate presentations of depression and anxiety disorders) and delivers high- intensity psychological interventions at step 3 (severe presentations of depression and anxiety disorders).

In the IAPT model, supporters are primarily psychological wellbeing practitioners (PWPs), graduate workers who are trained to provide low-intensity mental health services in primary care (18). PWPs are trained to identify and assess common presentations of mental health difficulties and work collaboratively with their clients to develop a treatment plan that best suits their needs (19) PWPs, as they encourage and motivate clients to keep using the intervention as well as provide helpful guidance and feedback, which contribute to enhanced outcomes (15–17). The work of the PWP is principally guided by the Reach Out Curriculum and advocates competencies in six core areas, namely, information gathering, information giving, shared decision-making, low-intensity treatment interventions, supervision and values, culture, diversity, and policy (19).

The deployment of iCBT as part of both routine care in mental health clinics and step 2 of IAPT is similar in that these interventions are offered to individuals with mild to moderate symptoms of depression and anxiety and generally exclude those with more severe presentations. Thus, most studies testing the efficacy of iCBT have focused on individuals with mild to moderate symptoms of depression and anxiety, leading to a less established evidence base for the use of iCBT with more severe presentations (20). Despite this, studies that have explored the effects of iCBT on those with severe presentations have illustrated positive effects (21, 22), even showing maintenance of improvements at follow-up (22, 23). A recent meta-analysis of studies of iCBT robustly demonstrates the relevance of iCBT for severe depression presentations; the authors argue that their findings should lay to rest the notion that iCBT should be limited to clients with milder depressions (7). This raises the possibility of using iCBT as a frontline intervention alongside high-intensity therapy (HIT) service provision for individuals with severe symptoms of depression and anxiety (22). In the IAPT stepped-care context, iCBT could be offered while service users wait for high-intensity treatment resources (face-to-face) to become available. To date, however, there are no data on the utility of iCBT as part of HIT service delivery in IAPT.

Therapeutic alliance refers to the collaborative nature of the interaction between therapist and client that emerges due to the affective bond between them, the agreement on the specific tasks in treatment, and the agreement on therapeutic goals (24, 25). Psychotherapy research findings indicate that alliance is related to therapeutic change (26–28) and overall satisfaction with care (29). In the field of internet interventions, research on therapeutic alliance with online supporters is scarce (30). A recent narrative review concluded that a positive therapeutic alliance can be formed in guided iCBT, with client ratings similar to those found in face-to-face treatment (31). This review also suggests that while an alliance may be formed online, the bond may be less important for the alliance to develop through this medium than in face-to-face therapy. Another review concluded that the therapeutic alliance in iCBT, as traditionally measured, has shown mixed results (32). There is opportunity to explore further the nature of the therapeutic alliance online and its relative contribution to outcomes. Given the established importance of the relationship between client and supporter in alliance and outcomes, it could be valuable to consider both client and clinician ratings to provide a comprehensive picture and inform the development and accurate deployment of iCBT as part of stepped care (30, 33–35). Also, some authors suggest that there is a need for studies that measure therapeutic alliance ratings at different points of treatment, as this could provide important information about alliance development (36).

The current study sought to examine clinical outcomes and the alliance online in delivering the SilverCloud iCBT for depression and anxiety disorders as a prequel to face-to-face therapy for clients with severe symptoms of depression and anxiety. Based on previous research of the SilverCloud platform and programs (37–39), the following research questions were developed:

1. Can clinical symptoms of severe depression/anxiety improve pre–post iCBT?


### MATERIALS AND METHODS

### Design

The study followed an uncontrolled feasibility design, On the one hand, it aimed at examining quantitatively clinical outcomes on depression and anxiety, functional outcomes in terms of work and social functioning, waiting time reduction, and therapeutic alliance between client and clinician in regards to step 3 services. Qualitatively, clinicians' experiences about the acceptability of the online intervention as a prequel to high-intensity therapy at step 3 of IAPT were assessed.

### Setting

Recruitment took place within one IAPT mental healthcare service of a National Health Service Trust in England over a 9 month period, between September 2016 and June 2017. Like most other IAPT providers, the service experiences high levels of demand on their service and, consequently, they struggle with waiting lists due to a shortage of trained professionals who can provide HIT. HIT service provision at the site has undergone a transformation in its delivery of evidence-based treatments. The service introduced a therapeutic package for those requiring HIT. Service users were offered iCBT before commencing faceto-face treatment. Clients were monitored throughout the intervention, and any deterioration in their symptoms was responded to. Alternatively, if an appointment for face-to-face therapy (high-intensity therapy) became available, they were offered this treatment to begin with. Options available at step 3 for escalation include individual CBT (face-to-face and some online counseling), face-to-face delivered primary care counseling, or interpersonal therapy.

### Participants

Our participants included both clinicians who worked at the IAPT service and clients who were referred to the service and for whom HIT was indicated as suitable. Clients who were deemed suitable for a typical HIT intervention and subsequently selected iCBT as part of their therapeutic package were invited to the study by their clinician. Criteria for receiving HIT in IAPT consists of clients with more severe presentations of anxiety and/or depression, client presentations that did not subside with a low intensity (step 2) treatment, or having a disorder that is not typically seen at the low intensity level (e.g. social anxiety). As with all IAPT services, substance abuse that is actively contributing to symptoms is an exclusion criterion, and these clients are referred on to specialist services for treatment.

All clinicians and PWPs supporting clients on SilverCloud as a prequel to HIT were eligible to participate in the study. Therefore, clinician participants consisted of a combination of clinical psychologists, counseling psychologists, and PWPs employed by the service.

### Procedure

#### Client Procedure

Eligible participants were informed of the study through their clinician and invited to take part during their assessment appointment. On sign-up, participants were presented with information sheets for the study, and were also invited to discuss their participation with their clinician. Those willing to participate were then required to digitally sign to give their informed consent for participation. Those who declined consent or decided to withdraw from the study upon commencing treatment were requested to contact their clinician at the healthcare service, who would re-assess the client and assign them to SilverCloud treatment-as-normal or another intervention before their face-to-face appointment became available. Once participants finished their course of SilverCloud treatment or the waiting period came to an end, they progressed to either group therapy, face-to-face counseling, face-to-face CBT, or CBT delivered by a clinician via the internet.

Throughout their use of services, participants were asked to complete the minimum data set, as per the national requirements regarding IAPT services. In addition, they were asked to complete the Scale to Assess the Therapeutic Relationship— Patient Version (STAR-P), a measure of therapeutic alliance from the client's perspective, during the 8-week supported period of the SilverCloud intervention.

#### Clinician Procedure

Clinicians were presented with a notification on their user accounts of the iCBT intervention that alerted them to the opportunity to participate in the research study. After reading the information sheet, they gave their consent to participate through their digital signature. Clinicians using the SilverCloud dashboard were able to monitor their client's progress throughout the 8-week supported period of the intervention, and they regularly gave them feedback and responded to the work they had completed. Following each of these iCBT review sessions, clinicians were invited to complete the Scale to Assess the Therapeutic Relationship—Clinician Version (STAR-C) through their user accounts on the platform, which assesses the therapeutic relationship from the clinician's perspective. Clinicians were also invited to participate in a qualitative semistructured interview pertaining to acceptability of iCBT as a prequel to HIT.

### Risk Management

At initial assessment and throughout treatment, clients were assessed for risk in line with routine clinical practice. The initial assessment for entry into services included questions of whether clients could maintain their safety while on the waiting list. Those who exceeded the cutoff score for risk in terms of selfharm on the screening questions were not eligible to participate in the study and were referred for additional support. Integrated risk measures in the SilverCloud platform allowed for the monitoring of any changes in risk for clients throughout the program. For example, if the client scored above 0 on the selfharm item of the PHQ-9, an alert would be sent through to their clinician, who could then escalate it appropriately within the established clinical governance structure. It is important to note that SilverCloud was not presented to clients as a program capable of providing crisis support, and this was further emphasized through informed consent, the client information sheet, and the user contract. Significant adverse events (SAEs) were handled in-service by the clinical team and were escalated appropriately.

### Completers vs. Dropout

Treatment dropout was defined using the IAPT Care Spell End Code (40) which collects the reason for service exit as determined by the clinician. In this study, clients were categorized into two categories: completers comprising the service exit reasons "completed scheduled treatment" and "referred to other service" and dropouts comprising "dropped out of treatment (unscheduled discontinuation)." Other service exit reasons were not observed in this study population. As 10 clients were still in treatment by the study end point, their dropout status was classed as missing data.

#### Medication Status

Medication status was defined using the patient's medication status at assessment as recorded in the patient management system. The options could be: "prescribed and taking," "prescribed and not taking," and "not prescribed." Prescribed and not taking and not prescribed were combined within the same group (no medication), since both groups were not taking medication.

#### Intervention

SilverCloud delivers CBT-based online interventions for anxiety disorders, depression, and also comorbid depression and anxiety. Each program is compliant with National Institute for Health and Care Excellence (NICE) guidelines for the use of CBT in treatment, and is composed of eight modules that follow evidence-based CBT principles. They include tools such as selfmonitoring and thought recording, behavioral activation, cognitive restructuring, and challenging core beliefs; all of which are central to the learning goals of the program. Research to date on the SilverCloud interventions has yielded significant positive clinical outcomes (41, 42).

Within the IAPT program, SilverCloud is typically delivered as a stand-alone low-intensity (step 2) intervention, or as an adjunct to high-intensity therapy. Specifically, for this study, participants were signed up at point of assessment and received reviews every 10–14 days via the platform. Once a participant is registered and assigned to a program on the iCBT platform, they receive a message from their clinician at their first login. This message welcomes them to the program, highlights its numerous aspects, and encourages them in the use of the program. Every fortnight, a clinician logs on and review participants' progress, leaving feedback for them and responding to the work they have completed.

#### Data Collection

The following data were collected (Table 1):

#### Participant Self-Reported Outcomes

Routinely Collected Data (The IAPT Minimum Data Set). This is a battery of psychometric measures common to all IAPT services collected as part of treatment-as-usual in all services. This battery primarily consists of the Patient Health Questionnaire-9 (PHQ-9), the Generalized Anxiety Disorder-7 (GAD-7), and the Work and Social Adjustment (WSAS). These questionnaires are administered to participants in all interventions at specific points in the patient pathway, for example at assessment, during treatment, and at post-treatment.

Patient Health Questionnaire-9 (43, 44) is a self-report measure of depression that has been widely used in screening, in primary care, and for research. The PHQ-9 items reflect the diagnostic criteria for depression outlined by the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition—Text Revision (DSM-IV-TR) (45). Summary scores range from 0 to 27, where larger scores reflect a greater severity of depressive symptoms. Cutoff scores for the PHQ-9 include none: 0–4, mild: 5–9, moderate: 10–14, moderately severe: 15–19, and


iCBT exit, end of iCBT treatment; service exit, end of step 3 Improving Access to Psychological Therapies (IAPT) treatment.

severe: 20–27. The PHQ-9 has been found to discriminate well between depressed and non-depressed individuals using the clinical cutoff of total score ≥10, with good sensitivity (88.0%), specificity (88.0%), and reliability (.89) (43, 44).

Generalized Anxiety Disorder-7 (46); GAD-7 comprises seven items measuring symptoms and severity of GAD based on the DSM-IV diagnostic criteria for GAD. Scores on the GAD-7 range from 0 to 21, where a higher score reflects greater severity of anxiety symptoms. The GAD-7 has good internal consistency (a = .92) and good convergent validity with other anxiety scales (46). Higher scores indicate greater severity of symptoms. Cutoff scores for the GAD-7 include minimal: 0–4, mild: 5–9, moderate: 10–14, and severe: 15–21. The GAD-7 has increasingly been used in large-scale studies as a generic measure of change in anxiety symptomatology, using a cutoff score of 8 (47–49).

Work and Social Adjustment Scale is a simple, reliable (a > .75), and valid measure of impaired functioning (50). The measure contains five self-report items that look at how the disorder impairs the client's ability to function day to day on five dimensions: work, social life, home life, private life, and close relationships. A score of 0–9 on the measure indicates subclinical symptoms, 10–19 is indicative of significant functional impairment but lower clinical symptoms, and above 20 suggests both severe clinical symptoms and functional impairment (41).

Scale to Assess the Therapeutic Relationship—Patient Version (51) is the client version of the scale to assess the therapeutic relationship in community mental healthcare. It was developed to be used in adult patients with mental health problems in psychiatric care. It is composed of 12 items and three subscales. The first subscale, positive collaboration, explores the general quality of the therapeutic relationship between patient and clinician. The second subscale, positive clinician input, illustrates the extent to which the client perceives their clinician positively regarding them. The third subscale, non-supportive clinician input, illustrate clients' perceived problems within the therapeutic relationship. The range of scores for the STAR-P is 0 to 48, with a higher score suggesting a better therapeutic relationship. The measure has shown excellent psychometric properties.

#### Access Data

Several types of treatment access data were collected as part of this research project. These data contained the following variables:


### Clinician-Reported Outcomes

Scale to Assess the Therapeutic Relationship—Clinician Version (51) is the clinician version of the scale to assess therapeutic relationship in routine scale. It was developed to be used in adult patients with mental health problems in psychiatric care. Similar to the client counterpart (STAR-P), the measure is composed of 12 items and three subscales. Firstly, positive collaboration, which explores the general quality of the relationship and the overall degree to which the relationship works. Secondly, positive clinician input measures to what extent clinicians encourage, understand, and support the patient. Lastly, emotional difficulties assesses the clinician's feeling that they cannot empathize with and are not accepted by the patient. The range of scores for the STAR is 0 to 48, with a higher score suggesting a better therapeutic relationship. The measure has shown excellent psychometric properties.

Semi-structured interview. The semi-structured interview schedule for clinicians was developed ad hoc for the present study. It aimed to explore clinicians' perceptions regarding the content of the program, their experience offering the reviews, and the clinical utility of the intervention being offered during the waiting period and before accessing HIT.

### Ethics

This study was carried out in accordance with the recommendations of the United Kingdom's Research Ethics Service (52) with written informed consent from all participants (see section 2.4.1, Client Procedure, for procedure). All participants gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by Wales Rec 7 (reference number: 16/WA/0257).

For participants that opted out of the SilverCloud intervention or the research, it is important to note that they were made aware that their place on the waiting list for services would not be jeopardized by their non-participation. Furthermore, for those who chose to partake in the intervention, their choice did not prolong their stay on the waiting list or their access to HIT.

#### Statistical Analyses

Estimates of caseness, reliable change, and recovery were calculated using IAPT recovery criteria. A client is at "caseness" if they score above the clinical caseness threshold on the PHQ-9 (≥ 10) or GAD-7 (≥ 8). A client is classed as recovered if they move from caseness at baseline to below the clinical caseness threshold post-treatment. Reliable improvement is defined as a decrease in either or both measures greater than or equal to the reliable change index (PHQ-9 RCI = 6, GAD-7 RCI = 4) with no parallel increase in score on either measure greater than or equal to the RCI. Clients that both recover and show a reliable improvement in their score(s) are said to be in reliable recovery.

Treatment effects on primary outcome measures from baseline to iCBT treatment exit to service exit were assessed using linear mixed models fit with restricted maximum likelihood (REML) in the R (53) package lme4 v1.1-13 (54). This analysis was conducted separately for each measure, and missing data were assumed to be missing at random (MAR). Dropout status, defined according to the IAPT service exit reason

as detailed in Materials and Methods, was included as a factor in each model. All participants attended part or all of the scheduled iCBT intervention. However, many participants did not thereafter attend any high-intensity treatment. To investigate whether this factor was associated with more positive or negative outcomes, clients were classified according to whether they attended at least one high-intensity treatment appointment (High Intensity Yes) or no high-intensity treatment appointments (High Intensity No). Medication status was also included in the model, where participants who were prescribed and not taking and those who were not prescribed were combined within the same group (medstatus No), and the other group referred to people who were prescribed and taking medication (medstatus Yes). Model selection was carried out by first creating a full model including all fixed factors (time point, dropout status, medication status, high-intensity attendance, and all interactions between them) and then performing backward elimination of effects from this full model. Backward elimination and model evaluation were completed using the step function from R package lmerTest v3.0-0 (55), which determined the optimum model. Post-hoc comparisons of least-square mean predictions from the optimum models were carried out using the R package lsmeans v2.27 (56).

Progression of the therapeutic relationship throughout iCBT treatment was modeled separately from the client's perspective (with STAR-P scores) and clinician's perspective (with STAR-C scores). For each measure, scores were modeled using a linear mixed model with fixed factors of time (a continuous variable: number of days after iCBT treatment start date), dropout status, and the interaction between them and client as a random factor. Post-hoc comparisons of least-square mean predictions at the predicted treatment exit day were carried out using the R package lsmeans v2.27.

Lastly, due to the low response rate from clinicians for the qualitative interviews, a descriptive–interpretative qualitative approach was used (57). This method was used to identify clinicians' viewpoints regarding acceptability of the iCBT intervention as a prequel to step 3 services. Acceptability was defined as the degree to which the intervention, from the clinician's viewpoint, was acceptable in terms of content, relevancy, and ability to engage the user in a therapeutic relationship. For the analysis of the interviews, these were transcribed, and thereafter, meaning units were extracted from the responses of the clinicians. Meaning units are parts of data that even standing out of context provide a piece of meaning to the reader. These meaning units were clustered together to form categories around acceptability. A more detailed description of these categories, including quotes that mirror each of them, is depicted in Table 2.

### RESULTS

### Overview

One hundred and twenty-four (N = 124) clients were recruited to the study (Figure 1). One client did not engage after baseline, did not complete any post-assessment questionnaires, and therefore could not be included in analyses, leaving an effective sample size of N = 123. Sociodemographic characteristics of the population at baseline are presented in Table 3. Baseline scores were unavailable for 13 clients as the authors were unable to guarantee the integrity of their data at this time point due to a lack of clarity around previous treatment received and instances of a double assessment. Service exit scoreswere unavailablefor 10 clients whowere stillin treatment by the end of the study. All 123 clients were included in analyses of primary outcome measures where models were fit with REML. However, the 23 clients with incomplete data could not be included in estimation of reliable change and recovery rates. All 123 clients were included in analyses of therapeutic alliance. Eleven clinicians were recruited and participated in the study.

TABLE 2 | Definition of the categories used in the qualitative analyses of Clinician views on acceptability.


#### Treatment Adherence: Completers and Dropouts

One hundred and thirteen (n = 113) clients' IAPT treatment episode had terminated by the study end point—68 completers and 45 dropouts as defined by their IAPT Care Spell End Code explained in the Materials and Methods. Ten clients were still in treatment by the study end point, so their dropout status was classed as missing data.

Completers attended on average 1.2 more iCBT appointments than dropouts [completers mean = 3.8, 95% CI (3.3, 4.3); dropouts mean = 2.6, 95% CI (2.1, 3.1); t (108) = 3.5, p < .001). There was no significant difference in the number of unattended iCBT appointments between completers and dropouts [completers mean = 1.9, 95% CI (1.5, 2.3); dropouts mean = 2.4, 95% CI (2.0, 2.7); t (109) = −1.9, p = .067]. There was no difference in the average amount of time in days spent in iCBT treatment [mean = 46.8, 95% CI (43.8, 49.7); t (91) = 1.3, p = .219].

Completers also attended on average 8.1 more high-intensity treatment appointments than dropouts [completers mean = 9.9, 95% CI (8.5, 11.2); dropouts mean = 1.8, 95% CI (0.6, 3.0); t (110) = 9.0, p < .001]. There was no significant difference in the number of unattended high-intensity treatment appointments between completers and dropouts [completers mean = 2.5, 95% CI (1.7, 3.2); dropouts mean = 2.1, 95% CI (1.4, 2.7); t (111) = 0.8, p = .437]. On average, completers spent almost twice the amount of days in high-intensity treatment than dropouts [completers mean = 147.2, 95% CI (122.9, 171.5); dropouts mean = 74.5, 95% CI (41.4, 107.6); t (63) = 3.6, p < .001].

#### Reliable Change, Recovery, and Reliable Recovery

One hundred (n = 100) clients had a full set of questionnaire scores from all time points (baseline, iCBT exit, and service exit) and could be included in estimation of reliable improvement rates. Fifty-eight percent (58%; n = 58) exhibited reliable improvement from baseline to iCBT exit, and 70% (70 clients) exhibited reliable improvement from baseline to service exit.

Ninety-nine (n = 99) clients were above the clinical caseness threshold at baseline and could be included in estimation of recovery and reliable recovery rates. Recovery and reliable recovery rates for these 99 clients are detailed in Table 4. Twenty-two (n = 22) clients had achieved recovery by the time of iCBT exit and 20, of these had reliably recovered. Thirty-three (n = 33) clients were in recovery at the point of service exit, all of which had reliably recovered. The recovery rate was significantly higher for those 56 clients who completed treatment (46%)



compared to the 43 clients who did not complete treatment (16%) (X-squared = 10.7, df = 2, p = .005).

#### Primary Outcome Treatment Effects

For all three primary outcome measures, the optimum linear mixed model included only time point, dropout status, and the interaction of time point and dropout status as fixed effects and client as a random effect. High-intensity treatment attendance did not add significantly to any of the models indicating that dropout is a more accurate predictor of treatment outcome for all three measures. Medication status was found to be a significant variable in the model for PHQ-9 and not for GAD-7 and WSAS. Post-hoc comparisons within the PHQ-9 model did not find any interaction with any of the other variables and, therefore, was discarded for being included in further analyses (Supplementary Table S1).

Post-hoc analyses of the optimum linear mixed models showed a significant reduction of severity scores from baseline to iCBT exit and again from iCBT exit to service exit for all three primary outcome measures (Table 5). For all measures, a large reduction in severity score took place in the period from baseline to iCBT exit for both completers and dropouts. A further reduction was observed from iCBT exit to service exit only for completers. Dropouts showed no significant change.

For the PHQ-9, scores were reduced by an average of 3.6 points from baseline to iCBT exit [Cohen's d = 0.61, 95% CI (0.34, 0.88)], with no significant difference between completers and dropouts at either the baseline or iCBT exit time point. Completers had a further reduction of on average 4.0 points from iCBT exit to service exit [Cohen's d = 0.64, 95% CI (0.29, 0.99)], whereas those who disengaged with the service showed no further change in score at service exit [Cohen'sd= −0.05, CI (−0.47, 0.37)].

Similarly, baseline GAD-7 scores were reduced by on average 3.2 points at iCBT exit time point [Cohen's d = 0.69, 95% CI (0.42, 0.97)] and a further 4.0 points at service exit for completers [Cohen's d = 0.83, 95% CI (0.47, 1.18)]. Dropouts did not show this further reduction in scores at service exit [Cohen's d = 0.03, 95% CI (−0.39, 0.45)].

Change in WSAS score, which estimates severity of work and social adjustment impairment, followed a similar pattern with a substantial reduction (mean = 2.4 points) in severity score at iCBT exit time point for the whole population [Cohen's d = 0.31, 95% CI (0.04, 0.58)]. Completers showed a further reduction of 4.1 points [Cohen's d = 0.60, 95% CI (0.25, 0.94)], whereas dropouts did not have any significant change in score from iCBT exit to service exit [Cohen's d = 0.15, 95% CI (−0.27, 0.57)].

#### Therapeutic Alliance—Client's Perspective

In total, 134 STAR-P questionnaires were collected, representing the therapeutic relationship of 56 clients from the client's perspective. The total amount of time in days spent in iCBT treatment was similar for completers (mean = 47.279, SD = 15.757) and dropouts (mean = 46.145, SD = 17.274). No significant change in average STAR-P scores were seen for dropouts. For completers, a significant increase in STAR-P scores of on average 3.9 points was observed from baseline (day 0) to average end of treatment (day 46) (Table 6).


df, degrees of freedom.


TABLE 5 | Differences in least-square mean questionnaire scores between time points.

iCBT, internet-delivered cognitive behavioral therapy; SE, standard error.

TABLE 6 | Difference in least-square mean STAR-P total scores and sub-scores from baseline to iCBT treatment exit for completers and dropouts.


STARP PC, Positive Collaboration subscale of the STAR; STARP PCI, Positive Clinician Input subscale of the STAR; STARP NCI, Non-supportive Clinician Input subscale of the STAR.

### Therapeutic Alliance—Clinician's Perspective

In total, 371 STAR-C questionnaires were collected, representing the therapeutic relationship of 82 clients from their clinician's perspective. For completers, no significant change in STAR-C scores was observed from baseline (day 0) to end of treatment (day 47). For dropout clients, the STAR-C scores declined significantly by on average 5.4 points from baseline to end of treatment (day 46) (Table 7).

### Practitioner Feedback

Of the 11 clinicians, 2 consented to a follow-up interview to further explore their experiences about acceptability. One male CBT therapist and one female PWP constituted the interview sample. Table 2 describes each of the four identified categories and supporting quotes from both participants.

## DISCUSSION

The current study examined an innovativemodel of service delivery, integrating digital interventions as a frontline intervention before accessing HIT. On average, clients were on the moderate-severe range for depressive symptoms and on the severe range for anxiety symptoms at baseline. Regarding treatment outcomes, there was a decrease in symptoms upon completion of both iCBT and HIT interventions with effect sizes ranging from moderate to large, and around 20% of the sample achieved reliable recovery in advance of starting face-to-face therapy. These results corroborate previous results demonstrating the benefits of iCBT in reducing symptoms of depression and anxiety in more severe presentations of depression and anxiety (22). The results also indicate that iCBT can be beneficial as a prequel to high-intensity therapy in IAPT and in doing so facilitate clients to gain access to a frontline evidence-based intervention while waiting for face-to-face therapy, which is in line


TABLE 7 | Difference in STAR-C total scores and sub-scores from baseline to iCBT treatment exit for completers and dropouts.

with the findings of other studies in blended interventions where internet interventions where offered before face-to-face therapy.

Despite the majority of clients transitioning from severe to mild presentations by point of service exit, some patients were still at caseness and, by definition, not recovered at this point. One potential explanation for these results is that, although current iCBT treatments are capable of producing improvements in symptoms, they may not be sufficient for producing lasting effects in severely anxious and depressed clients. Further treatments or booster sessions could therefore be necessary to achieve higher recovery rates (58, 59). This was especially noticeable in the dropout group, whose numbers achieved significantly less recovery rates at point of service exit. Completers received an adequate amount of HIT sessions to alleviate their symptoms (60), whereas those who dropped out completed only two HIT sessions on average and therefore any improvements made during iCBT were not further enhanced due to dropout from HIT. Understanding exactly what works for whom when integrating digital interventions into care pathways is an area of growing knowledge that would help to understand, and predict, future dropouts (61).

Clients spent on average 47 days in iCBT treatment and had 3.4 reviews on average during this period, meaning one review every 10 to 12 days. The observed administration of the intervention was below the intended use, that is, to be administered over a period of 8 weeks and for clients to receive six supporter reviews during this time (22, 62). This finding illustrates the importance of investigating the implementation of iCBT in novel contexts, since the ideal standard of intervention delivery was not adhered to in this study, but still produced clinical benefits. Further research regarding implementation of these interventions can illustrate and account for the changes that occur when interventions are implemented in novel contexts, as well as the barriers to implementation that are encountered (63). The value of the iCBT intervention is further reflected in the long waiting times clients experienced to accessing treatment. Average wait time was 158 days, and implementing iCBT can provide clients in these pathways with a frontline intervention in contexts where resources are scarce. In this instance specifically, waiting times for clients were reduced by 30%.

The dropout rate observed in this study (37%) is lower than the 45% rate of individuals not ending their course of therapy reported by IAPT (64). Reasons for dropout were not collected, but one study in this context found that waiting for an excessive period and the lack of contact during the wait lead people to dropout from services (65). In the current study, the long waiting list could have influenced people to disengage from services, and it might also have had an effect on improvement rates, as they have been shown to play a detrimental effect on outcomes (66). In regards to the wider literature on blended care interventions, a review conducted by Erbe et al. (67), found some studies where iCBT was also offered as a prequel to face-to-face therapy. The studies included in this review found dropout rates in the ranges of 30% to 59% across their samples (68–70), which is in line with what we observed in this study.

Therapeutic alliance from the client perspective increased significantly from pre to post for those who completed the scheduled intervention, while no change was observed in the case of those who dropped out before finishing their scheduled sessions. The positive alliance outcomes for completers showed that a strong therapeutic alliance can be established and maintained online (30, 31). These results further align with client perceptions of the role of support in iCBT, where different studies have shown that clients found it helpful and motivating during the intervention period (71, 72). The lack of change in perception of alliance for dropouts could be explained by the group receiving significantly less iCBT reviews and not getting enough contact to perceive an increase in alliance, which may take more time to develop in iCBT (73).

Therapeutic alliance from the clinician perspective showed the opposite pattern. Clinicians perceived no change in therapeutic alliance with clients who remained in treatment and perceived a significant decrease in alliance with those who dropped out from the services. The maintenance of alliance ratings perceived by the therapists over time on completers is similar to another study where these ratings were perceived as high, but no significant change was observed from pre- to posttreatment (34). The decrease in alliance with clients who dropped out could be explained by the fact that these clinicians conduct reviews for many clients in one scheduled review period and that they mostly base their perception of alliance on the usage of the platform while receiving little direct feedback from their client. In this sense, if they noticed some clients were engaging less with the platform, they may potentially feel a decrease in their perceptions of alliance over the course of the iCBT treatment. This interpretation is supported by the qualitative interviews, where clinicians said they put less effort in the reviews (i.e. write less and use more standard responses) if clients were not engaging, which could influence the levels of therapeutic alliance. Furthermore, these results could also be reflecting clinician bias toward relationship formation online, a result observed in other studies where clinicians also found iCBT as impersonal and thought that it was not feasible to create a therapeutic relationship in iCBT (74). However, more studies are needed to explore therapists' perceptions of alliance in iCBT, and specially within regular clinical settings, in order to determine if these patterns are also observed in different settings and countries.

Although we gathered only a small amount offeedback from a small number of clinicians and therefore caution is advised in interpreting the comments, some interesting points arise. For starters, a positive aspect of the review process highlighted by clinicians was the asynchronous nature of the clinician–client contact, which allowed for them to reflect and provide more insightful feedback to their client. However, they reported that once they sent the message, there was no way for them to know how the client would interpret it. This uncertainty was further emphasized by the perceived removed nature of the therapeutic relationship, which they believed had a negative impact and made it difficult to establish a therapeutic alliance. Both clinicians agreed that the intervention would work best for clients who are self-motivated to work through the content with minimal guidance. These reports are in line with findings about the importance of the readiness to engage in therapy and the sense of self-directedness for iCBT (74), which have been found to be predictors of adherence (75). They also agreed that for step 3 service provision, an iCBT intervention should not be offered as a stand-alone solution but as an adjunct (prequel) to face-to-face therapy. This finding is also in line with the feedback obtained from therapists in other studies, who consider iCBT as an adjunct to face-to-face therapy more than a replacement and are more open to the possibility of offering blended interventions (74, 76, 77).

#### Implications for Psychological Therapy Services

Some studies have explored the effects of internet and computerized CBT interventions as an adjunct to face-to-face therapy or as a treatment alternative in similar settings, showing mixed results. Thus, some studies have explored the beneficial effects of iCBT interventions when deployed at specialized care settings (78, 79), whereas others have not found any value in adding iCBT as an adjunct to face-to-face therapy (69). These inconsistencies in the findings may well be attributed to differences in implementation strategies and contextual factors, such as training with supporters, clinician's attitudes toward the interventions, and level of integration of iCBT interventions into services and treatment pathways (80). Future studies should explore and consider the impact of different implementation factors on the uptake and effects of these interventions.

The findings observed in this study also have some implications for IAPT, where the national target for recovery is 50%, but there is no extant benchmark to untangle the recovery rates across different steps in IAPT. For the total sample, clients who received iCBT as a frontline intervention achieved a 22.2% recovery rate upon iCBT completion. Thereafter, with face-toface therapy, this rate of recovery rose to 33.3%. Of importance, those who completed their entire course of treatment achieved a 46% recovery rate. These results provide initial supporting evidence for the use of iCBT as an adjunct to face-to-face therapies for more severe presentations. They do, however, stand in contrast to current guidance and practice in IAPT (12, 13) for the treatment of depression and anxiety, where digital interventions are recommended for clients with mild-moderate symptoms. As a consequence, these guidelines can also limit the scope and applicability of iCBT interventions to other contexts, such as high-intensity services for clients with more complex needs. Building on the work of prior research in the field (7, 21, 22) and the results from the current study, the authors acknowledge a need for further controlled research to robustly articulate the impacts of digital interventions in order to transform high-intensity service delivery. This will contribute to the development of clinical guidelines that incorporate digital interventions alongside current modalities of treatment for individuals with more severe presentations of depression and anxiety.

#### Limitations

Naturalistic observation studies, such as this one, have some limitations due to their uncontrolled nature. In future studies, the magnitude of the effect produced by the intervention could be more clearly characterized through comparison with a control group with no access to the iCBT intervention. Primary outcomes were not collected before starting high-intensity therapy, and there was a gap between iCBT exit and the start of high-intensity therapy, so the true severity of symptoms when starting high-intensity therapy was unknown. Information about when the clients dropped out from services was unclear, making it difficult to establish whether effects could be attributed to iCBT or the HIT therapy. The current study recorded no SAEs for any participant, but it is important for the field of internet interventions to explore adverse events as they occur within natural service. For example, dropout from service is not rigorously followed up within IAPT, which limited the researchers in expanding further on those in the "dropout" group. In regards to the interviews with clinicians, the low response rate might have biased the feedback, where perhaps only those with a positive attitude toward digital interventions accepted to participate. The low response rate is likely to be due to the busy schedule the clinicians work under; however, future studies should confirm this hypothesis. Finally, date of referral was not collected, so we cannot ensure how long clients waited to access the iCBT program from referral. However, from IAPT reports and our experience with different services, the delay between referral and access to iCBT use to be minimal, and this was also probably the case in this study.

### CONCLUSION

The current research has illustrated the potential effectiveness and benefit of implementing an iCBT intervention as a prequel to face-to-face therapy in individuals with severe presentations of depression and anxiety in naturalistic settings. As indicated by Andersson and colleagues (81), open studies with no control group play a key role in clinical effectiveness research and, as such, the present paper constitutes a relevant contribution to the advancement of the field of iCBT. The results showed that iCBT was a valuable option regarding waiting time reduction and clinical efficiency. As stated by the clinicians participating in this research, communicating the foundations of CBT through an internet-delivered intervention can be of high value to a service regarding time, cost, and clinical efficiency. Although generalizability of the findings is limited by the uncontrolled nature of the research, future investigations into this area are warranted to further validate the potential benefits of incorporating iCBT as a frontline intervention to highintensity services. The authors also acknowledge the need for research to be conducted on the implementation of these types of interventions into these contexts, using evidence-based methodologies, in order to determine how iCBT interventions can be best implemented into client pathways while accounting for levels of clinical risk. If implemented correctly within step 3 pathways, the clinical and cost benefits observed within mild-moderate clinical ranges and step 2 (lowintensity) services in IAPT could be successfully replicated and illustrated.

### KNOWLEDGE CONTRIBUTION

Depression and anxiety disorders are a leading cause of disability worldwide. Internet-delivered cognitive behavior therapy (iCBT) has been successfully used to treat mild-moderate depression and anxiety disorders, but its applicability to more severe presentations is relatively undocumented. Typically severe presentations of depression and anxiety are treated using highintensity interventions (HIT) such as face-to-face therapy. This study explored the potential benefits of iCBT as a prequel to faceto-face therapy for service users with depression/anxiety in a high-intensity therapy pathway for more severe presentations of depression and anxiety. Results from this preliminary research are promising; positive changes in clinical outcomes were observed for those completing iCBT, which further improved post-HIT. The relationship established between clinicians and patients was important and influenced by patient dropout. The results illustrate the potential utility of iCBT when used as a prequel to HIT. The research also presents a novel knowledge contribution to iCBT and the wider field of psychological therapies and paves the way for future investigations.

### DATA AVAILABILITY STATEMENT

The datasets generated for this study are available on request to the corresponding author.

## ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the United Kingdom's Research Ethics Service with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by Wales Rec 7 [reference number: 16/WA/0257]. For participants that opted out of the SilverCloud intervention or the research, it is important to note that they were made aware that their place on the waiting list for services would not be jeopardised by their non-participation. Furthermore, for those who chose to partake in the intervention, their choice did not prolong their stay on the waiting list or their access to step 3 interventions.

## AUTHOR CONTRIBUTIONS

DD and DR conceptualized the initial design of the study. AE led on the development of the first and subsequent drafts of the manuscript, with significant contributions from both DD and SC. DD, AE, and SC designed the data analytic plan, and SC implemented it. CC conducted and analyzed the qualitative interviews. DR reviewed the manuscript and provided feedback for each draft.

### ACKNOWLEDGMENTS

Thanks to all of the participants who contributed to this research study. Thanks to the host NHS Trust and their Research & Development department for hosting this study. Thanks to Colette Clyne for assisting with qualitative data collection.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019. 00902/full#supplementary-material

### REFERENCES


81. Andersson G, Titov N, Dear BF, Rozental A, Carlbring P. Internet-delivered psychological treatments: from innovation to implementation. World Psychiatry (2019). doi: 10.1002/wps.20610

Conflict of Interest: DD, AE, SC, CC, and DR are all employees of SilverCloud Health. DD, AE, and DR are also researchers with the E-Mental Health Research Group of Trinity College, Dublin, Ireland. This research was funded by SilverCloud, the commercial company marketing the iCBT system studied here.

As principal funder for this research, SilverCloud Health provided the necessary research resource to design the study, collect, and analyze the data and write the report. The researchers adhere to the ethical standards of research and reporting of the National Health Service (NHS) and Trinity College Dublin (https://www.tcd.ie/ research/dean/ethics/).

Copyright © 2020 Duffy, Enrique, Connell, Connolly and Richards. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# From Digital Mental Health Interventions to Digital "Addiction": Where the Two Fields Converge

Elias Aboujaoude1\* and Lina Gega<sup>2</sup>

<sup>1</sup> Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, United States, <sup>2</sup> Mental Health and Addiction Research Group, Department of Health Sciences & Hull York Medical School, University of York, York, United Kingdom

Scientific literature from the last two decades indicates that, when it comes to mental health, technology is presented either as panacea or anathema. This is partly because researchers, too frequently, have planted themselves either in the field of digital mental health interventions (variably called "telepsychiatry", "digital therapeutics", "computerized therapy", etc.), or in that of the problems arising from technology, with little crossfertilization between the two. Yet, a closer look at the two fields reveals unifying themes that underpin both the advantages and dangers of technology in mental health. This article discusses five such themes. First, the breakneck pace of technology evolution keeps digital mental health interventions updated and creates more potentially problematic activities, leaving researchers perennially behind, so new technologies become outdated by the time they are studied. Second, the freedom of creating and using technologies in a regulatory vacuum has led to proliferation and choice, but also to a Wild-West online environment. Third, technology is an open window to access information, but also to compromise privacy, with serious implications for online psychology and digital mental health interventions. Fourth, weak bonds characterize online interactions, including those between therapists and patients, contributing to high attrition from digital interventions. Finally, economic analyses of technology-enabled care may show good value for money, but often fail to capture the true costs of technology, a fact that is mirrored in other online activities. The article ends with a call for collaborations between two interrelated fields that have been—till now—mutually insular.

Keywords: telepsychiatry, telemental health, telemedicine, problematic internet use, internet addiction, privacy

## INTRODUCTION

In Aesop's ancient fable "Man and Satyr1 ", the satyr saw the man blow on his hands on a cold winter's day and asked why. The man responded: "so that I can warm them up; can't you see how cold it is?" When they sat down to eat, the man cut a piece of his roast and blew on it before he ate it. The satyr asked why. The man responded: "so that it cools down; can't you see how hot it is?" The

1 Satyr: Ancient Greek creature, half-man and half-goat, associated with hedonistic pursuits and the God Dionysus.

#### Edited by:

Jochen Mutschler, Private Clinic Meiringen, Switzerland

#### Reviewed by:

Agnes Von Wyl, Zurich University of Applied Sciences, Switzerland Roser Granero, Autonomous University of Barcelona, Spain

> \*Correspondence: Elias Aboujaoude eaboujaoude@stanford.edu

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 02 September 2019 Accepted: 23 December 2019 Published: 21 January 2020

#### Citation:

Aboujaoude E and Gega L (2020) From Digital Mental Health Interventions to Digital "Addiction": Where the Two Fields Converge. Front. Psychiatry 10:1017. doi: 10.3389/fpsyt.2019.01017 satyr, indignantly, walked away. "That's enough," he said. "I can no longer be your friend since I see you blowing hot and cold with the same breath." The allegory helps illustrate how digital technology in mental health can be therapeutic and problematic at the same time. This dissonance can cause confusion and mistrust, which leads clinicians, patients and the general public to either disengage from digital mental health interventions, or to underestimate the problems and risks associated with extensive technology use.

As a case in point, a US telemedicine company recently announced positive results in a study of an action video game designed to treat attention deficit and hyperactivity disorder (ADHD), with plans to market it as a prescription digital therapy for ADHD (1). Meanwhile, several studies, including systematic reviews of comorbidity studies [(e.g., (2–4)], have shown a strong link between ADHD and Internet Gaming Disorder (IGD), suggesting that online gaming and other online activities may actually be a causal or contributing factor to developing ADHD. Similarly, a meta-analytic study showed that digital therapies for children and young people's mental health were promising (5), but also fueled anxieties in parents who could not reconcile encouraging their children to use the internet and games for treatment when they were constantly pressing them to reduce screen use, due to concern about the negative impact of digital media on children's emotional and social wellbeing (6, 7).

As this data illustrates, the field of digital mental health interventions is increasingly in collision with that of digital mental health problems and risks. On the one hand, empirical findings into the adverse effects of digital technologies can dissuade adoption of digital mental health interventions such as "serious games", out of fear that they may fuel a "gaming disorder" (8) or have other negative psychological impact, such as incite violence as result of possible violent content (9, 10), invite bullying by facilitating access to users (11, 12), or encourage social withdrawal (13). On the other hand, overzealous evidence about the efficacy and cost-effectiveness of digital interventions is rarely accompanied by reasonable caution about the uncertainty of positive outcomes with technology, or its risks–financial, legal, ethical and therapeutic —until a brick wall is hit when trying to transfer digital interventions from scholarly ivory towers to real world clinics.

Between the field of digital mental health interventions and that of the risks and adverse effects of digital technologies, rarely has one research group cited or built upon the work of the other (14). Yet reciprocal recognition and collaboration could prove very beneficial: research from both camps over the last two decades suggests shared observations and challenges. In this article, we aim to highlight common themes that have defined research into both the benefits and problems of technology in mental health, and to make recommendations for crosscollaborations between two interrelated fields of research that have traditionally been mutually insular. Rather than review the benefits and harms of technology when it comes to mental health as revealed in research—a very different, much more ambitious endeavor— we instead will focus on how research into both fields has revealed common issues and challenges.

### Speed of Technology: Rapidly Evolving But Easily Outdated

In 1965, Gordon Moore, an engineer and the cofounder of Intel, predicted that the number of transistors that can be fitted on a microchip would double every two years (15). What became known as "Moore's Law" has held up surprisingly well, as has its corollary: that computer processing speed would grow at a similar rate. Digital technology has evolved exponentially, its breakneck speed far outpacing that of mental health research into it.

Slowness is inherent to a robust research process: an investigation question has to be identified, a protocol conceived and approved, participants recruited, an intervention tested, data analyzed, an article written, and peer review performed, before a manuscript can finally be accepted and published. If investigators were testing an antidepressant, there would be little reason to fear the obsoleteness by publication time of the hard earned results. However, when investigating the positive health uses of technology or its negative psychological effects, there is a "moving target" aspect to the process that makes it so that the platform being investigated risks becoming less relevant by the study's end, as users move to ever newer, faster and more engaging platforms. As such, digital technology is, almost by definition, always ahead of the science investigating it, and the research community is perennially "playing catch up", whether it is weighing in on the merits of the latest digital therapeutic tool being marketed or on the ills of the most recent all-consuming video-game overtaking culture. One result is that clinical researchers often feel ill equipped to address the latest technologies, leaving it to investors, startup executives and engineers to take the lead and steer the field.

### Proliferation of Technology: Freedom But Also a "Wild-West"

The proliferation of digital products means that developers have the freedom to create and distribute diverse technologies, whereas consumers have the freedom to choose and use different products. This "laissez-faire" approach may have been the cornerstone of the classical economic liberalism that propelled innovation and trade in 18th c. Europe. But, in the 21st c. World Wide Web, societies have a duty of care towards technology's end users, especially if these users turn to technology because they seek psychological, social, and emotional support.

Legislative and regulatory bodies have struggled to understand, and respond to, this proliferation of technology. Delayed oversight and lack of consistent and comprehensive protective frameworks have led to negative consequences. In the US, for much of the internet's life, online communications were governed by the 1996 Communications Decency Act, Section 230, which essentially immunized sites from liability for problematic behavior by their users, including bullying, misinformation and sexual predation (16). While Europe has often led the way in internet regulation as a means of protecting consumers, significant safety protections such as the Right To Be Forgotten (17) and the General Data Protection Regulation (18) arrived relatively late (2012 and 2018, respectively). This contributed to a "Wild West"-like online environment that nurtured negative personality traits from narcissism (19, 20) to aggression (20), and may have contributed to the radicalization (20–22), terrorism (20–22) and counter-democratic shifts that have been blamed on the internet (20).

Similarly, despite the field of digital mental health interventions being three decades old (23), it is only in 2017 that, in the US, the Food and Drug Administration initiated a dedicated program for the testing and scrutiny of digital health tools and innovations (24). In the UK, the National Health Service (NHS) introduced a digital, data and technology standards framework as recently as late 2018 (25), forming the basis for the March 2019 Evidence Standards Framework for Digital Health Technologies by the National Institute for Health and Care Excellence (NICE) (25). The legal and regulatory vacuum in which digital mental health interventions have evolved has had deleterious effects on the field's credibility and, one fears, on public health. This is in part because it has allowed to go largely unchallenged false advertising and unfounded scientific claims of the therapeutic value of some "health products" (26). Everyone stands to benefit from comprehensive regulations and legislations that offer broader protection against a digital "Wild West", without compromising the freedom of using online media and digital mental health interventions.

### Visibility Through Technology: Improved Access But Compromised Privacy

Technology is an open window through which people can access and distribute information, share experiences and communicate with each other. In the past, clinical knowledge was the privilege of a few professionals who communicated it to their patients during face-to-face meetings, if locally available. Technology has improved patient access to specialist knowledge via standardized therapy programs, which allow users to learn therapy skills and manage their own care, and via remote digital mental health platforms, which allow visits with geographically removed professionals. Technology has also enabled users with mental health problems to share their stories and form peer support networks. Further, it has allowed information exchange between professionals via digital media in a way that expedites risk assessment and peer consultations and ensures continuity of care. The open window afforded by technology has come with a price, though: the threat of compromised privacy.

Issues of privacy are at the core of how research into both digital mental health interventions and technology-related problems has evolved. With all too frequent news of hacks into supposedly secure networks, there is growing distrust of digital systems as repositories of health information (27). Besides concerns around electronic medical records, this has meant hesitation on the part of some providers and patients to adopt digital platforms whose confidentiality cannot be guaranteed, even when efficacy data suggests benefit. Legislative actions to try to protect health information in the digital medical record and on telemedicine platforms have, again, lagged behind the increasingly sophisticated modes of violation. As such, regulations, such as the US Health Insurance Portability and Accountability Act (HIPAA) (1996) (28), the Health Information Technology for Economic and Clinical Health Act (HITECH Act) (2009) (29) and the Omnibus Rule (2013) (30), as well as the UK's Data Protection Act (2018) (31), have only been partially successful.

Beyond compromising health information, the post-privacy age ushered in by internet-related technologies has had important effects on psychology. Pre-internet psychological literature delineated several privacy components (32, 33), all of which would seem impacted by our heavily technology-reliant lifestyle (20, 34). Components that, together, constitute privacy include reserve, or the ability to control disclosures; isolation, or the use of geographic distance to separate oneself from others; solitude, or the freedom to place oneself where one cannot be seen or heard; selective intimacy, or the ability to be with an individual or group to the exclusion of others; and anonymity. These privacy components have been shown to mediate psychological functions that are crucial to wellbeing, including contemplation, autonomy, rejuvenation, catharsis, and recovery (33, 34). If the building blocks of privacy are under digital assault, including by facial recognition and Artificial Intelligence (AI) tools now being applied to massive social media databases, then the psychological processes that rely on them may be negatively impacted, with potentially serious consequences (35). As such, privacy violations may be contributing to internet-related psychopathology, just as they threaten the evolution and adoption of digital mental health interventions.

### Attraction of Technology: Strong Pull But Loose Ties

In June 2019, there were 4,422,494,622 internet users, 57.3% of the global population (36). There are over 10,000 mental healthrelated smartphone apps (37), without counting computerized programs, websites, virtual reality systems and wearables. Designers often approach online users as fickle consumers who are on constant lookout for new digital opportunities—easily drawn in, but just as easily distracted away. Much of the research in online "user experience", for example, focuses on maximizing "dwell time" (the average time a user spends engaged with a site's content), extending "scroll depth" (how far down the page a user gets when reading content), minimizing "bounce rate" (the percentage of users who navigate away from a site after viewing only one page), and decreasing "time between visits" (38). In Internet psychology, this has been blamed for moving the internet in more extreme directions of representation, including radicalization and narcissism, as desperate page owners vie to attract users at all costs (20).

Like the tenuous commitment of internet users to content, the online definition of a social media "friend" or romantic interest (the app-driven evolution from "dating" to "hooking up" [(20, 39, 40)]) speaks to a similarly "shaky" commitment to relationships developed online. More generally, the virtualization of relationships across digital platforms means looser ties to sites and individuals. This seems true across social media and digital mental health delivery platforms.

Aboujaoude and Gega Technology Benefits and Risks in Mental Health

The ease of "unfriending" an acquaintance on Facebook or blocking someone on Instagram or Twitter may not differ in fundamental ways from the premature termination of therapy with an e-counselor over a digital mental health portal (41). The provider-patient relationship across many digital platforms is often nonexistent or limited, mirroring digital relationships in the broader sense. The thousands of mental health apps do not appear to have led to measurable population-level mental health benefits. Could it be because, for the most part, there is no trusted provider to recommend or guide their use, or to incorporate them within a more traditional delivery model that encourages patient engagement through supportive accountability?

The attrition problem in digital mental health interventions has been borne out by scientific data emanating from welldesigned research studies [(e.g., (42, 43)]. Perhaps because of the lack of a visible, knowable interlocutor on the other side of many online exchanges, engagement with digital experiences tends to be superficial and to lack anchoring. There is ample evidence, though, to suggest that some clinician support is better than none when it comes to outcomes with digital interventions (44–48); in fact, the greater the therapist input, the more effective the interventions seem to be (49, 50). Will fully automated AI platforms that simulate human decision-making and adaptability be able to sustain patients' engagement beyond an initial curiosity-driven stage? Until we find out, we need to invest in human support that strengthens ties and engagement—and ultimately improves outcomes—with digital interventions.

### Economic Value of Technology: Cost-Effective But Costly

There is an assumption that digital mental health interventions offer "good value for money". By encouraging patient self-management, allowing remote delivery, enabling a less specialist workforce to deliver complex interventions, and reducing waiting lists, digital mental health interventions would be expected to save clinician time and make clinical work more efficient. This assumption comes with several problems. First, we may not be able to forego the traditional intervention for ethical, clinical or practical reasons; e.g., we cannot prohibit patients from seeing their family doctor in favor of following self-management at home. Second, spending for technology is often frontloaded (e.g., cost of software and hardware), whereas savings or improved outcomes are accrued in the longer term, and payers may not have the money to invest upfront. Third, costs may be incurred in one sector and benefits or savings in another, even if their budgets are not linked (e.g., costs for digital therapies are paid by the health clinic or the user, but savings are accrued in the employment sector in the form of less absenteeism). Fourth, the per-patient treatment cost may decrease, but, due to technology's greater reach, the overall number of people treated may go up, thereby increasing total healthcare costs. In the end, the overall economic incentives to adopt digital therapies may be weak, even if they are proven cost-effective.

It is similarly easy to explain away the overuse of internetrelated platforms as a means of enhancing one's productivity and, therefore, living standards. "Multi-tasking", as allowed by a powerful smartphone or by simultaneously opening windows on one's desktop, can feed the illusion that one has cloned himself or developed an extra pair of hands and can now do the work of more than one individual. However, economists still debate whether a "productivity miracle" has been sustained through the successive waves of internet-related technology evolution (51). It turns out that a lot of what people do online —from mindless surfing to online gaming to catching up on celebrity gossip or social media updates—may not necessarily add to their material wealth or the gross domestic product (51). Any benefits from technology-enabled activity have to also be weighed against the costs of treating the population of distracted or otherwise psychologically affected individuals. Whether assessing online distractibility or technologydelivered treatments, there is more to the cost debate than a surface economic reading might suggest.

### CONCLUSION

Technology blows hot and cold with the same breath when it comes to mental health. Speed, proliferation, visibility, attraction, and economic value are some of the attributes that underlie both its benefits and problems. To our knowledge, no paper has addressed the challenges and themes common to the field of digital mental health interventions and that of the problematic use of digital technology. Research at the intersection of digital technology and mental health has typically focused either on the benefits or the problems, in mutually exclusive fields of enquiry. The resulting literature makes digital technology look like either a panacea or an anathema. The truth, of course, is more complex and more likely to be revealed via a "global", collaborative approach between researchers in the arena of digital mental health interventions and those exploring the risks and negative consequences of technology. In this paper, we made an attempt to bring closer two disparate areas of scholarship. The fact that, as we discussed, similar forces appear to have partially defined both research fields makes such collaborations particularly promising. Joint efforts would empower the research community to understand the psychological, societal, ethical and economic forces at play—and to suggest solutions.

### AUTHOR CONTRIBUTIONS

Both authors contributed to the conceptualization, researching, and writing of the article.

### ACKNOWLEDGMENTS

The authors received no external help in conceptualizing, researching, and writing the article.

### REFERENCES


Conflict of Interest: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Aboujaoude and Gega. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author (s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

digital media

of impactful research

article's readership