# Computers and Games for Mental Health and Well-Being

edited by : Yasser Khazaal, Jérôme Favrod, Anna Sort, François Borgeat and Stéphane Bouchard published in : Frontiers in Psychiatry and Frontiers in Public Health

#### Frontiers Copyright Statement

© Copyright 2007-2018 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-496-9 DOI 10.3389/978-2-88945-496-9

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# Computers and Games for Mental Health and Well-Being

Topic Editors:

Yasser Khazaal, Université de Genève, Switzerland Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland Anna Sort, Universitat de Barcelona, Spain François Borgeat, Université de Montréal, Canada Stéphane Bouchard, Université du Québec en Outaouais, Canada

Image: GrandeDuc/Shutterstock.com

Citation: Khazaal, K., Favrod, J., Sort, A., Borgeat, F., Bouchard, S., eds. (2018). Computers and Games for Mental Health and Well-Being. Lausanne: Frontiers Media. doi: 10.3389/ 978-2-88945-496-9

# Table of Contents

#### Editorial

*07 Editorial: Computers and Games for Mental Health and Well-Being* Yasser Khazaal, Jérôme Favrod, Anna Sort, François Borgeat and Stéphane Bouchard

#### Opportunities and Challenges

*13 Maximizing the Impact of e-Therapy and Serious Gaming: Time for a Paradigm Shift*

Theresa M. Fleming, Derek de Beurs, Yasser Khazaal, Andrea Gaggioli, Giuseppe Riva, Cristina Botella, Rosa M. Baños, Filippo Aschieri, Lynda M. Bavin, Annet Kleiboer, Sally Merry, Ho Ming Lau and Heleen Riper


#### Prevention of Mental Disorders

*28 Prevention of Mental Health Disorders Using Internet- and Mobile-Based Interventions: A Narrative Review and Recommendations for Future Research*

David Daniel Ebert, Pim Cuijpers, Ricardo F. Muñoz and Harald Baumeister

*44 Online Positive Interventions to Promote Well-Being and Resilience in the Adolescent Population: A Narrative Review* Rosa M. Baños, Ernestina Etchemendy, Adriana Mira, Giuseppe Riva, Andrea Gaggioli and Cristina Botella

#### Serious Games


Theresa M. Fleming, Lynda Bavin, Karolina Stasiak, Eve Hermansson-Webb, Sally N. Merry, Colleen Cheek, Mathijs Lucassen, Ho Ming Lau, Britta Pollmuller and Sarah Hetrick


*82 Six Tips on How to Bring Epic Wins to Health Care* Anna Sort and Yasser Khazaal

#### Sensor-Related Technologies

*88 An Objective Screening Method for Major Depressive Disorder Using Logistic Regression Analysis of Heart Rate Variability Data Obtained in a Mental Task Paradigm*

Guanghao Sun, Toshikazu Shinba, Tetsuo Kirimoto and Takemi Matsui

*95 Changes in Heart Rate Variability Recorded in Natural Situation with T-Shirt Integrated Sensors and Level of Observed Behavioral Excitation: A Pilot Study of Patients with Intellectual Disabilities and Psychiatric Disorders*

Julie Palix, Michel Akselrod, Charly Cungi, Fabienne Giuliani and Jérôme Favrod

*102 Development and Clinical Evaluation of an mHealth Application for Stress Management*

Brent D. Winslow, George L. Chadderdon, Sara J. Dechmerowski, David L. Jones, Solomon Kalkstein, Jennifer L. Greene and Philip Gehrman

#### Smartphone Apps

*110 Ecological Momentary Assessment and Smartphone Application Intervention in Adolescents with Substance Use and Comorbid Severe Psychiatric Disorders: Study Protocol*

Xavier Benarous, Yves Edel, Angèle Consoli, Julie Brunelle, Jean-François Etter, David Cohen and Yasser Khazaal


#### Computer-Supported Treatments


#### Virtual Reality and Augmented Reality

*160 DJINNI: A Novel Technology Supported Exposure Therapy Paradigm for SAD Combining Virtual Reality and Augmented Reality* Maher Ben-Moussa, Marius Rubo, Coralie Debracque and Wolf-Gero Lange

Frontiers in Psychiatry and Frontiers in Public Health | www.frontiersin.org


Marie-Eve St-Pierre-Delorme and Kieron O'Connor


#### Schizophrenia and Related Disorders

*224 A Serious Game to Improve Cognitive Functions in Schizophrenia: A Pilot Study*

Isabelle Amado, Lindsay Brénugat-Herné, Eric Orriols, Colombe Desombre, Maxine Dos Santos, Zelda Prost, Marie-Odile Krebs and Pascale Piolino

*235 Improving Facial Emotion Recognition in Schizophrenia: A Controlled Study Comparing Specific and Attentional Focused Cognitive Remediation*

Baptiste Gaudelus, Jefferson Virgile, Sabrina Geliot, The GAÏA/RECOS Study Team and Nicolas Franck


Michael Kyrios and Neil Thomas *270 The Impact of a Videogame-Based Pilot Physical Activity Program in Older Adults with Schizophrenia on Subjectively and Objectively* 

*Measured Physical Activity* Heather Leutwyler, Erin Hubbard, Bruce Cooper and Glenna Dowling

#### *275 Reducing Delusional Conviction through a Cognitive-Based Group Training Game: A Multicentre Randomized Controlled Trial*

Yasser Khazaal, Anne Chatton, Karen Dieben, Philippe Huguelet, Maria Boucherie, Gregoire Monney, Laurent Lecardeur, Virginie Salamin, Fethi Bretel, Silke Azoulay, Elodie Pesenti, Raoul Krychowski, Andreia Costa Prata, Javier Bartolomei, Perrine Brazo, Alexei Traian, Thomas Charpeaud, Elodie Murys, Florent Poupart, Serge Rouvière, Daniele Zullino, Alberto Parabiaghi, Mohamed Saoud and Jérôme Favrod


Elodie Peyroux and Nicolas Franck

# Editorial: Computers and Games for Mental Health and Well-Being

Yasser Khazaal 1,2,3 \*, Jérôme Favrod<sup>4</sup> , Anna Sort 5,6, François Borgeat 3,7 and Stéphane Bouchard8,9

<sup>1</sup> Department of Psychiatry, University of Geneva, Geneva, Switzerland, <sup>2</sup> Geneva University Hospitals, Geneva, Switzerland, <sup>3</sup> Research Centre of the Montreal University Institute of Mental Health, Montreal, QC, Canada, <sup>4</sup> La Source, School of Nursing Sciences, HES-SO University of Applied Sciences and Arts of Western Switzerland, Lausanne, Switzerland, <sup>5</sup> University of Barcelona, Barcelona, Spain, <sup>6</sup> PlayBenefit, Barcelona, Spain, <sup>7</sup> Department of Psychiatry, University of Montréal, Montreal, QC, Canada, <sup>8</sup> Department of Psychoeducation and Psychology, Université du Québec en Outaouais, Gatineau, QC, Canada, <sup>9</sup> Centre Intégré de Santé et de Services Sociaux de l'Outaouais, Gatineau, QC, Canada

Keywords: internet treatment, virtual reality, serious games, augmented reality, smartphone app, cognitive remediation, cognitive behavior therapy

**Editorial on the Research Topic**

**Computers and Games for Mental Health and Well-Being**

#### INTRODUCTION

Recent years have seen major developments in the computer and game industry. Tools such as games, virtual reality, and applications for smartphones may foster learning, enhance motivation, promote cognitive and behavioral change, support psychotherapy, favor empowerment, and improve cognitive functions. Games and computer design share an opportunity for creativity and innovation in helping to specifically build and assess preventive or therapeutic tools.

The research topic computers and games for mental health and well-being illustrates some of the developments in the field across 34 papers written by 188 authors around the world. Overall, close to 30% of the papers consist of clinical studies, including two controlled trials; 20% are reviews that describe the potential of various technologies and applications; and about 50% express scientists' opinions, innovative perspectives, and thoughts about the use of computers and games. A scan of the research interests covered by this Research Topic reveals an important focus on clinical populations (at least 24 papers), especially schizophrenia and related psychotic disorders (nine papers), as well as significant coverage of serious gaming (seven papers), augmented or virtual reality (six papers), apps and smartphones (five papers), web-based applications (three papers), and much more. The present editorial aims to briefly summarize the contribution of the published papers.

Edited and reviewed by: Wulf Rössler, Universität Zürich, Switzerland

> \*Correspondence: Yasser Khazaal yasser.khazaal@hcuge.ch

#### Specialty section:

This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry

Received: 10 March 2018 Accepted: 29 March 2018 Published: 13 April 2018

#### Citation:

Khazaal Y, Favrod J, Sort A, Borgeat F and Bouchard S (2018) Editorial: Computers and Games for Mental Health and Well-Being. Front. Psychiatry 9:141. doi: 10.3389/fpsyt.2018.00141 PREVENTION OF MENTAL HEALTH DISORDERS

Prevention of mental health disorders with e-health- and m-health-related tools was explored by some of the topic authors (Ebert et al.; Baños et al.). Ebert et al. reviewed the evidence for the effectiveness of such Internet and mobile interventions in preventing the onset of several mental health disorders. In consideration of the limited number of randomized controlled trials related to such a question, it was not possible to draw any definite conclusion from the review. However, reports related to the prevention of depression showed promising findings.

Baños et al. reviewed the evidence specifically related to the impact of positive online interventions that aim to promote well-being and resilience in the adolescent population. The paper discusses the potential of such interventions for mental health promotion. Despite the promising rationale of these interventions, the authors concluded that more controlled studies are needed with long-term follow-up (Ebert et al.; Baños et al.).

## SERIOUS GAMES

In a perspective article, Desseilles discussed some of the paradoxes (i.e., using a game for a serious issue), opportunities (i.e., for learning and communication), and challenges (i.e., misuse and abuse-related risks) offered by games in relation to mental health. Fleming et al. explored the current status and promising directions for the development of games and gamification for mental health.

Some authors argued that games have the potential to increase the impact of Internet-based interventions by improving their reach and engagement potential. In their reviews, they detailed some of the main engagement mechanisms from games reported in the health literature, as well as from studies related to Internet gaming and Internet gaming disorders [1, 2]. They make the point that the ubiquity and variability of the possible game designs (exergame biofeedback, cognitive training, etc.) would help to train or foster specific change mechanisms adapted from traditional evidence-based interventions. According to these authors, the number of trials in the field is still limited. In particular, only a few trials included comparisons between gamebased and non-game-based interventions [3].

Lau et al. however, conducted a systematic review and meta-analysis of randomized controlled trials related to the effectiveness of serious games for the treatment of mental disorders. The review included studies that involved a total of 674 participants treated for different mental conditions (depression, alcohol use disorder, etc.). A meta-analysis of nine studies revealed a positive moderate effect on symptoms favoring games over no-intervention control groups.

#### THE NATURALISTIC UPTAKE CHALLENGE

In a paper issued by an international Collaboration On Maximizing the impact of E-Therapy and Serious Gaming (COMETS), Fleming et al. highlighted a gap between the positive results of controlled studies and difficulties in uptake and adherence outside of trials. In response to the reach and engagement challenge, the COMETS group proposed a paradigm change in e-health that focuses on four pillars: (1) increasing user involvement and user-centered approaches in the field, (2) increasing emphasis on engagement (including via gaming), (3) increasing collaboration and data sharing, and (4) rapid testing and implementation.

De Beurs et al. described and discussed different methods that, consistent with the proposed COMETS paradigm, aim to increase end-user uptake in the development of webbased mental health interventions. Regarding improved uptake, Thorens et al. discussed the possible lessons learned from the success of massively multiplayer online role-playing games (MMORPGs) and some of the MMORPG mechanisms that could be extrapolated in designing games for health interventions. Finally, Sort and Khazaal proposed six tips that aim to improve game design processes and possibly increase the success of games for health in terms of reach, engagement, and clinical outcomes.

## SENSOR-RELATED TECHNOLOGIES

Several authors reported on the integration of sensor-related technologies as possible tools to predict further emotional or behavioral modifications. One may assume that such tools would be helpful in increasing the rate and efficacy of early-delivered interventions.

Sun et al. compared heart rate variability (HRV) during a mental task in drug-naïve patients who had a major depressive disorder (MDD) with HRV in healthy controls. They concluded that HRV indices measured during a mental task are a promising tool for screening patients with MDD. In another pilot study, this time with patients who have intellectual disabilities and psychiatric disorders, Palix et al. showed promising results on the possibility of predicting clinical agitation from a change in HRV. Winslow et al. described the development of a classifier of real-time physiological stress (based on electrodermal and cardiovascular inputs) in a healthy population. In a pilot trial, they compared the addition of the classifier associated with an mHealth app intervention for stress management to the use of cognitive behavioral therapy (CBT) alone. The addition of the classifier to the app intervention was associated with greater improvement in measures of stress and anxiety, showing promising results that are to be confirmed in further studies.

#### SMARTPHONE APPS

Benarous et al. reported a study protocol that aims to offer ecological momentary assessment and interventions through a smartphone app to adolescents with substance use and comorbid severe psychiatric disorders. The patients will also be assessed through clinical interviews at a 1-year follow-up. On the one hand, the study will increase our knowledge about the acceptability of such apps, and on the other, it may provide important inputs after 1 year of evolution in psychiatric and substance use-related symptoms among adolescents.

Monney et al. described the theoretical background of a smartphone app for cannabis users ("stop-cannabis"). They further reported the results of a satisfaction survey carried out among ∼500 app users, showing high involvement, a good level of satisfaction, and a high level of perceived usefulness. The results are promising, considering the moderate-to-high level of cannabis dependence reported by the participants. Previous studies showed the ability of e-health interventions to appeal to people with moderate or severe dependence [4]. The selfselected [5] and non-randomized nature of the sample in the survey cannot, however, allow more conclusive statements about the app's efficacy. Benarous et al. planned to use the "stopcannabis app" in their study, which may provide more evidence about such interventions. Similarly, Zhang and Ho illustrated in their perspective paper that recent developments in smartphone

app-related technologies will open new prevention and treatment avenues for addictive disorders.

Van Singer et al. assessed the content quality, interactivity, and self-help features of the panic disorder smartphone apps available at Google Play Store. The authors adapted the tools used for the assessment of health-related websites to the smartphone apps [6–8]. They further developed a specific self-help assessment tool that was based on the adaptation of a panic disorder CBT into app format. As shown in other studies on the assessment of apps for different mental health disorders using the same [9] or different methodologies [10], the results led the authors to conclude that most of the assessed apps were of poor quality.

The findings on smartphone apps, as shown by other authors [9–12], highlight a gap between the wide availability of healthrelated smartphone apps in the market and lower levels of evidence-based conceptions of such tools.

#### COMPUTER-SUPPORTED TREATMENTS

Zidani et al. reported on the effects of a computerized, masked, priming-based intervention called Augmentation of Psychotherapy through Alternative Preconscious Priming (APAP) [13]. The intervention was offered to eight treatmentresistant patients with social anxiety disorder or generalized anxiety disorder. In this case series, the authors found promising improvements in quality of life and anxiety symptoms and related beliefs.

Demily et al. described "Cognitus & Moi," a computer-based cognitive remediation program for children with intellectual disability. Cognitus & Moi is based on a metacognitive strategy specifically dedicated to the training of attentional and visuospatial areas. The tool includes a variety of exercises and may offer individualized training according to the specific needs of the patient.

#### VIRTUAL REALITY AND AUGMENTED REALITY

Six of the papers included in the research topic focused on virtual therapy. In their conceptual design paper, Ben-Moussa et al. proposed the DJINNI solution. This technology- and gamification-supported exposure therapy paradigm combines virtual reality and wearable augmented reality to complement traditional in vivo exposure in the treatment of social anxiety disorder. One aspect of significant interest in the proposed system is the possibility of an augmented reality system that interprets and guides the user during in vivo exposure exercises in the absence of a therapist. The data collected with this system could be used to improve and adapt virtual reality scenarios for use in exposure exercises.

The work of Urech et al. was also about the use of virtual reality for patients with social anxiety disorder. The paper described a proof-of-concept and pilot study on a new use of virtual reality. The originality of the tool lies in its ability to modify attentional biases, instead of using virtual reality, to expose users until they reach extinction of the feared response. The preliminary results suggest sufficient improvements in attention bias and social anxiety to argue for further assessments of such a promising technique with more robust clinical trials.

Addressing another severe mental disorder, compulsive hoarding, St-Pierre-Delorme et al. showed in a pilot control trial the potential benefits of adding a virtual reality component in which objects belonging to the patients can be integrated in the virtual environment. They recruited 14 adults with severe problems in accumulating objects without being able to throw them away and randomly assigned them to a virtual realitybased treatment in which they would discard common household objects that belonged to them (experimental treatment) or did not belong to them (control condition). The results were promising, encouraging the authors to pursue this line of research with a larger sample.

Working with a patient with obsessive-compulsive disorder, Laforest et al. also provided preliminary support for the use of a modified CBT treatment, focusing on the use of virtual reality to conduct exposure exercises related to fear of contamination. They used a single-case design with multiple baselines across subjects, combined with time-series analyses, a classic approach for demonstrating the impact of a new treatment before conducting randomized control trials.

Bouchard et al. presented another innovation related to the use of virtual reality, this time for the treatment of gambling disorder. The paper reported three successive studies about the development, usefulness, and safety of virtual reality environments designed to induce the desire to gamble. The authors considered these preliminary steps requirements before dedicating several therapy sessions to immersion in virtual reality in which psychotherapists would help pathological gamblers by applying therapy techniques while the gamblers were emotional and in a state of craving.

Contrasting with other virtual reality studies focusing on the treatment of a specific disorder, Riva et al. explored a transforming experience paradigm supported by augmented and virtual reality. From a solid understanding of the science behind their ideas and the observation that people are not always able to implement desired behavior change, the authors argue in favor of the use of augmented and virtual reality technologies, together with the feeling of presence and emotional engagement. They propose fostering change and transformative experiences by increasing perceived self-efficacy and self-reflectiveness and by structuring or altering bodily self-consciousness.

## SCHIZOPHRENIA AND RELATED DISORDERS

The large number of papers (9) on schizophrenia and related disorders is a good illustration of how clinicians and researchers in the field can improve the use of computers and games in the treatments being offered to patients.

Four papers concerned the development and evaluation of computerized programs for the remediation of cognitive deficits in schizophrenia. Amado et al. presented a pre-post pilot study of a virtual reality game to train attention, memory, and planning. The game simulates a town in which the participants navigate and plan actions that they have difficulty carrying out in real life. The pilot study of 12 interactive 90-min weekly sessions with this virtual reality game showed improvement in participants' attention and memory. Demily et al. presented a partially computerized program to improve impairments in social cognitive processes. The computerized program allows participants to develop social cognitive abilities in a simulated environment. The authors tested their program through singlecase studies. Their results showed an improvement in the targeted processes in theory of mind and attributional styles. Gaudelus et al. measured the efficacy of a computerized program called GAÏA, which focuses on facial emotion recognition processes. Their randomized controlled trial compared GAÏA to another cognitive remediation program. The study showed a significant improvement in facial emotion recognition performance in both groups, with a significantly larger effect in the GAÏA arm. Vianin underlined the importance of not limiting cognitive remediation to the single use of computerized exercises. According to this author, cognitive remediation should involve the training of metacognitive skills throughout the intervention, as well as going back and forth between the computerized remediation training of cognitive deficits to real-life exercises to promote the generalization of learning.

Two papers examined online interventions. Thomas et al. developed a website that can be used on a tablet computer by mental health workers to structure therapeutic discussions about personal recovery. They tested the feasibility and acceptability of a low-intensity intervention with 10 participants. All participants improved their personal recovery, as measured by the Questionnaire for the Process of Recovery, and stated that they would recommend the interventions to others. Rehm et al. reviewed the literature on the use of avatars to facilitate online communication between clients and therapists. Their narrative review suggests that avatars serve several functions in facilitating treatment engagement through a virtual therapeutic alliance, which reduces communication barriers, promotes treatment seeking, promotes expression and exploration of self, and enables therapists to control treatment stimuli.

Three papers focused on the use of games or playful interventions to reduce symptoms by improving physical activity. Leutwyler et al. used a video game to improve physical activity in older adults with schizophrenia. Twenty participants played various fitness video games for 30 min once a week for 6 weeks. The results indicate a significant increase in frequency of selfreported vigorous physical activity, as well as a non-significant increase in number of steps taken and a reduction in sedentary hours. Khazaal et al. compared Michael's game with treatment as usual in a randomized controlled trial. The study included 172 patients. Michael's game is a collaborative game that trains the ability to generate alternative hypotheses to the erroneous conclusions that "Michael" draws from common everyday life situations. Results indicated that conviction measured with the 21-item Peters et al. Delusions Inventory improved at the end of the intervention. At 6-month follow-up, the results showed a sustained effect on conviction and a delayed effect on distress and preoccupation for participants in Michael's game compared with participants in the treatment-as-usual condition. Nguyen et al. presented the development of a short, easy-to-use, group-based intervention to improve pleasure and motivation in individuals with schizophrenia, called "Positive Emotion Program for Schizophrenia" (PEPS). A literature review led to the identification of the components of the program, and different beta tests enabled its refinement.

## CONCLUSION

The 34 papers published on the research topic addressed here highlight the important developments in the field. The variety of technology- and gaming-based solutions proposed across different disorders and sometimes for the same disorders illustrates the high level of creativity that characterizes the field. Such developments are opening avenues for innovation in the assessment, prevention, and treatment of mental disorders.

The first wave of computerized and Internet-based therapies reproduced, in electronic format, previously validated treatments of specific disorders such as anxiety and depression. This first wave tried to overcome the gaps in service provision and use among people with mental disorders [14]. A number of meta-analyses of randomized-controlled trials reported positive outcomes of such treatments for different disorders [15, 16]. Translation of such results in naturalistic settings was, however, compromised by low engagement [17, 18], arguing for a paradigm shift in this area (Fleming et al.).

The present topic provides an overview of possible options and methods to overcome such challenges, including game-based approaches. An important trend is further highlighted: a new generation of computer- and game-based approaches not only reproduces validated face-to-face treatments, but also uses the interaction potential of the technologies to enhance, modify, combine, or create new treatments (Winslow et al.; Zidani et al.; Ben-Moussa et al.; Urech et al.; Riva et al.). Some of these proposed tools include training emerging from developments in cognitive neuroscience (Zidani et al.; Demily et al.; Gaudelus et al.; Vianin). The topic includes articles on schizophrenia and other treatment-resistant conditions. Such papers try to enhance or improve treatments as usual. But again, this observation underlines a new trend. The field initially focused on the service provision gap, but is now working on the improvement of existing treatments or on the development of new treatments.

Some aspects of the developments in the field are still missing in the present topic, particularly those related to the potential addition of machine learning [19, 20]. According to Zhang and Ho, the full potential of e-health, m-health, machine learning, and gaming in psychiatry still needs to be investigated.

In conclusion, this research topic highlights some of the main developments in the field of computers and games that open the horizon for future improvements in tools to enhance mental health and well-being. This growing field offers new intervention tools for the prevention and treatment of a wide range of psychiatric conditions. Although the efficacy of each tool needs to be considered separately, one may expect common factors to specifically contribute to the retention and/or efficacy of some types of interventions. Identifying such common factors will help guide the design of future interventions. Further studies will be needed to assess not only the outcomes of these interventions, but also the processes related to user engagement and associated changes in behavior.

#### REFERENCES


## AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.


**Conflict of Interest Statement:** SB is now consultant for and owns equities in Cliniques & Développement In Virtuo, which distributes virtual environments.

The other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Khazaal, Favrod, Sort, Borgeat and Bouchard. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## GLOSSARY

**Augmented reality**: The addition of digital components (possibly multiple sensory modalities) to a real-world experience.

**e-health:** The use of the Internet and related technologies for health- and public health-related purposes.

**m-health**: M-health is a component of e-health. The m (for mobility) indicates the use of mobile Internet and related devices (e.g., mobile phones, global positioning systems, patient monitoring devices) for health- and public health-related purposes.

**Serious games**: Games specifically developed to engage users in behaviors with the aim of improving health-related outcomes.

**Virtual reality:** An immersive lifelike experience provided by a computer-simulated interactive environment in which the participant's actions to some extent influence the scenario and the experience.

# Maximizing the Impact of e-Therapy and Serious Gaming: Time for a Paradigm Shift

*Theresa M. Fleming1,2\*, Derek de Beurs3 , Yasser Khazaal4 , Andrea Gaggioli5,6 , Giuseppe Riva5,6 , Cristina Botella7,8,9 , Rosa M. Baños8,9 , Filippo Aschieri5 , Lynda M. Bavin1 , Annet Kleiboer10 , Sally Merry1 , Ho Ming Lau11 and Heleen Riper10*

*1Department of Psychological Medicine, University of Auckland, Auckland, New Zealand, 2Department of Paediatrics: Child and Youth Health, University of Auckland, Auckland, New Zealand, 3Netherlands Institute for Health Services Research (NIVEL), Utrecht, Netherlands, 4Department of Psychiatry, University of Geneva, Geneva, Switzerland, 5Department of Psychology, Catholic University of Sacred Heart, Milan, Italy, 6Applied Technology for NeuroPsychology Laboratory, Istituto Auxologico Italiano, Milan, Italy, 7Department of Psicología Básica, Clínica y Psicobiología, Castellón, Spain, 8Department of Personalidad, Evaluación y Tratamiento Psicológicos, Valencia, Spain, 9CIBER Fisiopatología Obesidad y Nutrición (CIBERon), Instituto Salud Carlos III, Valencia, Spain, 10Department of Clinical Psychology, Faculty of Behaviour and Movement Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands, 11Department of Psychiatry, VU University Medical Center, Amsterdam, Netherlands*

#### *Edited by:*

*Fotios C. Papadopoulos, Uppsala University, Sweden*

#### *Reviewed by:*

*Ma Angeles Gomez Martínez, Universidad Pontificia de Salamanca, Spain Stefano Manzo, Anima Research Istitute, Italy*

> *\*Correspondence: Theresa M. Fleming t.fleming@auckland.ac.nz*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 16 February 2016 Accepted: 01 April 2016 Published: 18 April 2016*

#### *Citation:*

*Fleming TM, de Beurs D, Khazaal Y, Gaggioli A, Riva G, Botella C, Baños RM, Aschieri F, Bavin LM, Kleiboer A, Merry S, Lau HM and Riper H (2016) Maximizing the Impact of e-Therapy and Serious Gaming: Time for a Paradigm Shift. Front. Psychiatry 7:65. doi: 10.3389/fpsyt.2016.00065*

Internet interventions for mental health, including serious games, online programs, and apps, hold promise for increasing access to evidence-based treatments and prevention. Many such interventions have been shown to be effective and acceptable in trials; however, uptake and adherence outside of trials is seldom reported, and where it is, adherence at least, generally appears to be underwhelming. In response, an international Collaboration On Maximizing the impact of E-Therapy and Serious Gaming (COMETS) was formed. In this perspectives' paper, we call for a paradigm shift to increase the impact of internet interventions toward the ultimate goal of improved population mental health. We propose four pillars for change: (1) increased focus on user-centered approaches, including both user-centered design of programs and greater individualization within programs, with the latter perhaps utilizing increased modularization; (2) Increased emphasis on engagement utilizing processes such as gaming, gamification, telepresence, and persuasive technology; (3) Increased collaboration in program development, testing, and data sharing, across both sectors and regions, in order to achieve higher quality, more sustainable outcomes with greater reach; and (4) Rapid testing and implementation, including the measurement of reach, engagement, and effectiveness, and timely implementation. We suggest it is time for researchers, clinicians, developers, and end-users to collaborate on these aspects in order to maximize the impact of e-therapies and serious gaming.

Keywords: computerized therapy, serious games, implementation, cCBT

## INTRODUCTION

The rationale for internet interventions for mental health is commonly centered on the following premises:


Systematic reviews and meta-analyses of randomized controlled trials of internet interventions and computerized therapies delivered off-line (e.g., via CDRom) for anxiety and/or depression have reported good evidence of effectiveness (5–7), with adherence rates from 26 to 76% (8).

Despite the robust rationale and promising evidence, relatively few evidence-based interventions have been implemented or made publicly available. Among those that have been implemented in naturalistic or "real-world" settings (i.e., outside of traditional trials), limited data regarding implementation, uptake, and impact have been published. Available data suggest that significant numbers of people may be interested in online mental health support. For example, the publicly available self-help program, MoodGYM, attracted approximately 38,000 registrants over a 14-month period (9), and the mental health app, Happify, had been downloaded between 100,000 and 500,000 times on Google Play as at 16 December 2015. Moreover, a high proportion of people who complete the self-assessments in such interventions have substantial symptoms (10, 11). Despite this, naturalistic use of internet interventions for mental health appears to be associated with high attrition (non-adherence or drop out from the intervention); with notably higher rates of attrition in implemented programs than in randomized controlled trials, even when the same program is used. For example, in field studies of MoodGYM, only 3.9% of public registrants completed at least three of the five modules, in contrast to 53.8% of participants in a controlled trial of the same intervention (12). Less than 7% of public registrants continued past two modules in a newer version of the program (9), and similar results were found with adolescents (13). In another example, only 23% of users of a PTSD coach app used it during the first month after download, and the median time spent per session was less than one minute (14).

Increasing human support has been a core strategy for enhancing adherence to online interventions (8, 15, 16). This is promising with many trials finding higher rates of adherence to supported interventions than pure self-help interventions (17), although this is not always the case (18). Regardless of the comparison to pure self-help, attrition is still a challenge for supported internet interventions. For example, in a recent independent study by Gilbody and others (19), even with weekly telephone support, non-adherence was such that there was no treatment gain for patients accessing computerized therapies (MoodGYM and Beating the Blues) over those allocated to primary health care alone. Moreover, given that part of the rationale for computerized therapies is their scalability and low barriers to helpseeking, alternative approaches to increasing engagement should also be considered.

Together, findings from naturalistic or implementation studies suggest that potential users, including those with significant symptoms, are interested in internet interventions for mental health (20–23); however, implementation and engagement require improvement. We contend that the current paradigm or approach for the development of evidence-based internet interventions for mental health is typically researcher led; that the programs are often designed to replicate tested face-to-face therapies online; and that these are usually tested using classic validation designs (randomized controlled trials), with a focus on the efficacy of stand-alone interventions. Many of these components are critical to demonstrate that internet interventions can be effective under trial conditions as shown in **Figure 1**. However, to increase the impact of serious games and e-therapies, further developments are required.

An international group initiated by the last author (Heleen Riper) met in Amsterdam and Valencia with students and game designers. The group included authors of the present paper and others involved in e-health topics, including the fields of serious gaming (SPARX, Michael's Game) and virtual reality (EMMA, BUTLER), apps for depression (MOODBUSTER) and substance use (Stop-cannabis), and online interventions for suicide prevention (PITSTOP suicide). An international Collaboration on Maximizing the impact of E-Therapy and Serious Gaming (COMETS) was established. We identified four pillars that need stronger emphasis for advancing research and increasing the impact of e-therapies and serious gaming. As shown in **Figure 1**, these are:


The present paper outlines these potential approaches for increasing the real-world impact of internet interventions. New emphases or approaches should enhance, not replace, rigorous research-based approaches.

#### TOWARD A NEW PARADIGM

#### Increased Focus on User-Centered Approaches

We propose that one of the key ways of increasing the impact of internet interventions is through increasing the focus on usercentered approaches. This would include user-centered design processes and greater individualization within programs, with the latter perhaps utilizing increased modularization.

#### User-Centered Design

While some computerized therapies and serious games have been designed with significant user input, we contend that uptake and

adherence to internet interventions can be enhanced with greater involvement and understanding of users. This is not merely about consulting users on drafts but also about a deep understanding of user needs and preferences, and actively involving users in design processes from the outset. For example, a traditional research-centered process might begin with reviewing published evidence and subsequently planning an internet intervention containing six to ten modules to be completed at a rate of about one per week, hereby approximating evidence-based face-to-face therapies. By contrast, a user-centered design would begin with users, to understand issues such as how and when they would be willing to use the internet for mental wellbeing, and to explore their current behavior, needs, and preferences. Such an approach might suggest alternative processes or content. For example, alternative frequencies and durations of use might be proposed in order to reflect how people actually utilize the internet for psychological needs. Alternative processes for engagement or therapeutic change could also be identified for investigation (such as the use of sharable content or the opportunity to help others). The evidence for each of these components should, of course, be investigated. The point being not to replace research with usercentered design, but to utilize user-centered processes alongside scientific research.

#### Increasing Individualization

Alongside an increased focus on user-centered design *of* interventions, increased individualization, responsiveness, and choice *within* interventions may be important for engagement. To date, most evidence-based internet interventions for mental health are not very individualized. That is, all people using the same program generally all receive the same content, albeit sometimes with some optional modules. This is out of keeping with contemporary personalized experiences of the internet, which is very choice based. Engaging clients in defining their goals may result in better compliance than a more clinical and generalized aim of treating their diagnostic condition (24, 25). It should also be simple for users to select, from effective alternatives, their own preferred options or approaches for meeting goals. Modularization of interventions is promising in this regard.

#### Exploration of Modular Approaches

Limitations of current disorder-focused approaches to psychological therapies have been increasingly recognized (26). Briefly, most evidence-based psychological therapies have been developed for single disorders, such as depression, anxiety, and so on. However, in clinical practice, co-morbidity is the rule. In addition, there is an overlap in the techniques used to treat different disorders. In order to make treatments more efficient and to deal with clinical realities, a modular approach for face-to-face therapy has been developed, and early clinical trials show promise with better clinical outcomes delivered in less time (26–29). Should modularization prove effective, this approach would be feasible online, and could facilitate increased user choice and increased collaboration between groups.

#### Increasing Engagement

A second key area for increasing the impact of internet interventions will involve the use of approaches that motivate continued usage (adherence) and improve user engagement (30, 31). The use of serious gaming and gamification, enhanced telepresence, and increased use of persuasive technology are promising in this regard. Moreover, the routine assessment of engagement may help to further develop the field and monitor progress toward this goal.

#### Serious Gaming and Gamification

"Serious games" are interventions that are games, or that utilize elements of gaming, as an integral and primary method for achieving a serious purpose, such as a health or educational goal (32). Gamification refers to the addition of gaming elements (such as challenges, reward, and experiences of exploration) to a non-game environment. The inclusion of gaming elements within computerized psychotherapies, and games or game-like environments with embedded therapeutic content, has been tested in several trials (32, 33). This approach is at an early stage, with few (if any) trials performed independent of developers (32, 34). However, there is promising evidence for serious games in other areas of health and behavior change (35–38). The potential for mental health has been identified (10, 39–44), and is supported by relatively low attrition rates in initial trials of mental health interventions utilizing these strategies (10, 45, 46).

#### Enhanced Telepresence

A therapeutic relationship is arguably a critical "active ingredient" of therapy (47). Increasing human support for users of internet interventions, via telephone, text, email, or face-to-face contact, appears to be helpful (8, 15, 16). Increasing social "telepresence," or the feeling of connections with others within the computer program itself (48), may also hold promise. This can be achieved by using thoughtful design processes; for example, in the SPARX computerized CBT program, the "Guide" was designed as a virtual therapist, with warm welcoming wording, a carefully selected voice actor, and active rapport building. In interviews, young people reported feeling that the guide and other virtual characters in the computer program cared about them, and that this enhanced their experience of the intervention (49).

#### Increased Use of Persuasive Technology

The science of persuasive technology refers to the use of technology to influence human behavior, motivation, and attitudes through human–computer interaction or computer-mediated communication (50). Examples of persuasive technology include the use of automated support to increase primary task completion, such as automated SMS or mobile phone messages, email prompts, continued feedback by the program, and built-in explanations of why the program might help (51). Ethical issues related to the use of persuasive technologies must be carefully taken into account. Persuasion should be based on consent (52), and should help people to change the behavior they would like to change. Furthermore, not all participants require or prefer the same amount (or the same kind) of support, and this assistance may only be needed at critical times during the treatment program (53). Nevertheless, persuasive technology has been significant in promoting engagement and behavior change in other areas, and has arguably been underused in the development of internet interventions for mental health.

#### Measuring Engagement

A critical variable for improving the impact of computerized therapy interventions is patient or user engagement. According to Patient Health Engagement (PHE) models (54), making patients highly engaged in their care sustains them in attributing full meaning to the therapy, and in enacting self-management behaviors effectively, even when life contexts change. When effectively engaged, patients also develop a sustainable perspective about their actual and possible conditions, which can be better integrated into action (55). In this way, patient engagement can be considered as a compass to help developers customize their interventions. Patient engagement also has another advantage for developers and researchers: it can be easily measured using validated instruments (see **Table 1**).

#### Rapid Testing and Implementation Rapid Prototyping and Testing

Many health care interventions are developed after first consulting the literature, and a small number of experts and consumers. They are often then piloted within small groups, after which minor adjustments may be made before the intervention is tested in a larger trial. It can, therefore, be a number of years from initial development until the results of a controlled trial are published. Replication may be required, and only then does implementation become a priority. This drawn-out process is problematic when testing e-health interventions due to the speed of technology change. Alternative models of rapid development and iterative testing should be considered (58); for example, using agile software design principles, such as the lean start-up method (59) or scrum (60).

In agile development processes, the product is tested with users from the outset using rapid development and testing feedback loops. An important component involves the development of a minimal viable product (MVP). An MVP is a barely finished

Table 1 | Available validated tools for assessing Patient Engagement.


product that contains an essential element, but is missing details, and is provided to end-users to gage their reactions and inform the next steps in development. Reponses to the product are measured and used to inform next steps that are rapidly developed and tested in the same way. This iterative process involves close collaboration between designers, software developers, and endusers. Larger scale testing gradually replaces small opportunistic samples as progressively more complex features are tested. When a near-finished version is ready, more traditional testing can be carried out, for example, via a randomized controlled trial. As described by Mohr et al. (61), internet interventions can utilize approaches that focus on evaluating the working mechanisms, rather than a locked-down version of the intervention. Such a framework allows for improvements in functionality to be made during a trial, subsequently resulting in a more generalizable and durable intervention.

#### A Planned Focus on Implementation

Many internet interventions for mental health have been developed and shown positive results in trials, but are not publicly available, or, are available, but have limited uptake or adherence. This highlights two points. First, that efficacy alone is not sufficient to indicate an intervention will have a significant impact on health. Reach (or exposure to and uptake of the intervention) and adherence must also be evaluated, and findings used to improve programs and their implementation (62). Second, the necessary conditions for the sustainable implementation of interventions, including consideration of the project's future ownership and the identification of possible revenue streams for ongoing hosting and updates, must be considered from the outset. Implementation may, therefore, necessitate non-traditional collaborations.

#### Increased Collaboration Intersectorial

A key opportunity for improving the impact of computerized therapies and serious gaming is increased use of diverse knowledge and skills. The design of internet interventions for mental health should involve input from different fields, including – but not limited to – users, therapists, computer engineers, game designers, behavior change experts, and human factors specialists. Such collaboration requires researchers to move beyond their discipline and consider new knowledge, methods, and techniques. Current approaches for evidence-based internet interventions for mental health are often initiated and led by researchers. Alternative approaches, for example, where researchers join projects that are led by users, game developers, or internet and software experts, should also be considered.

#### International Collaborations

International collaboration is a further key opportunity for improving the long-term sustainability and growth of e-therapy. Many internet interventions are developed within a specific country or jurisdiction, and fail to make use of the international nature of the internet, with subsequent limitations in terms of funding and the impact of the proposed intervention. These stand-alone programs are often supported by modest financial resources, resulting in small-scale clinical trials or case studies. A promising strategy is to develop stronger multinational teams and projects.

#### An Ambitious Future Vision

Should increased collaboration and modular user-centered approaches be pursued, greater gains in population mental health may be realized. This could be achieved, for example, by creating a user-centered online platform or ecosystem that allows users to select the components that most appeal to them or that are recommended based on their self-assessments. Should such a model become a reality, components could be continuously developed and additional ones added over time. A well-designed system could allow uptake, adherence and effectiveness of components to be routinely measured and compared. We propose that, in the future, such systems could safely invite user-generated content and input from other researchers and developers within agreed guidelines. Should this vision be achieved, the input and energy of diverse groups could be harnessed to facilitate development in the field.

#### CONCLUSION

Evidence-based mental health interventions are promising; however, uptake and adherence outside of trial settings have not yet met hopes and projections. We have proposed that it is time for a paradigm shift in order to maximize the impact of evidence-based e-therapies and serious gaming. Promising directions include a greater focus on user-centered approaches (including user-centered design, individualization within interventions, and exploration of modularized programs), increased emphasis on engagement (utilizing processes such as gaming, telepresence and persuasive technology, and measuring engagement), increased international and intersectorial collaboration, and rapid testing and implementation. We propose that, in the future, such systems could safely invite both user-generated content and input from other researchers and developers. In each case, input should be within agreed guidelines.

#### AUTHOR CONTRIBUTIONS

HR initiated the COMETS group. CB, RB, GR, AG, YK, TF, DB, AK, HL, and SM developed or contributed to the concepts of the COMETS group, which are represented in this paper. TF, DB, YK, AG, GR, CB, RB, FA, and LB drafted the paper. AK, SM, HL, and HR contributed substantial content to the paper. All co-authors approved the paper.

#### FUNDING

No funding was received for the development of this paper. SM, TF, and LB are funded by the University of Auckland. GR and AG are funded by Ministero della Salute.

#### REFERENCES


depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. *BMJ* (2015) **351**:h5627. doi:10.1136/bmj.h5627


**Conflict of Interest Statement:** TF and SM are co-developers of SPARX computerized therapy for depression and can benefit from any commercialization of it outside of New Zealand. The remaining authors have no conflict of interest to declare.

*Copyright © 2016 Fleming, de Beurs, Khazaal, Gaggioli, Riva, Botella, Baños, Aschieri, Bavin, Kleiboer, Merry, Lau and Riper. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Active Involvement of End Users When Developing Web-Based Mental Health Interventions

*Derek de Beurs1 \*, Inge van Bruinessen1 , Janneke Noordman1 , Roland Friele1,2 and Sandra van Dulmen1,3,4*

*1NIVEL, Netherlands Institute for Health Services Research, Utrecht, Netherlands, 2 Tranzo, Scientific Centre for Care and Welfare, Tilburg, Netherlands, 3Department of Primary and Community Care, Radboud University Medical Center, Nijmegen, Netherlands, 4 Faculty of Health Sciences, University College of Southeast Norway, Drammen, Norway*

Background: Although many web-based mental health interventions are being released, the actual uptake by end users is limited. The marginal level of engagement of end users when developing these interventions is recognized as an important cause for uptake problems. In this paper, we offer our perceptive on how to improve user engagement. By doing so, we aim to stimulate a discourse on user involvement within the field of online mental health interventions.

#### *Edited by:*

*Jérôme Favrod, University of Applied Sciences Western Switzerland, Switzerland*

#### *Reviewed by:*

*Peter Olutunde Onifade, Neuropsychiatric Hospital, Aro, Abeokuta, Nigeria Diane Elliot, Oregon Health & Science University, USA*

> *\*Correspondence: Derek de Beurs derekdebeurs@gmail.com*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 25 November 2016 Accepted: 13 April 2017 Published: 03 May 2017*

#### *Citation:*

*de Beurs D, van Bruinessen I, Noordman J, Friele R and van Dulmen S (2017) Active Involvement of End Users When Developing Web-Based Mental Health Interventions. Front. Psychiatry 8:72. doi: 10.3389/fpsyt.2017.00072*

Methods: We shortly describe three different methods (the expert-driven method, intervention mapping, and scrum) that were currently used to develop web-based health interventions. We will focus to what extent the end user was involved in the developmental phase, and what the additional challenges were. In the final paragraph, lessons learned are summarized, and recommendations provided.

Results: Every method seems to have its trade-off: if end users are highly involved, availability of end users and means become problematic. If end users are less actively involved, the product may be less appropriate for the end user. Other challenges to consider are the funding of the more active role of technological companies, and the time it takes to process the results of shorter development cycles.

Conclusion: Thinking about user-centered design and carefully planning, the involvement of end users should become standard in the field of web-based (mental) health. When deciding on the level of user involvement, one should balance the need for input from users with the availability of resources such as time and funding.

Keywords: user-centered design, online intervention, adolescent health services, design strategies, mental health

#### INTRODUCTION

No start-up company today would be around for long if only 5% of its registered customers actually used the product. Also, no start-up would develop a product mainly based on expert knowledge and literature, and then test that product in a 4-year trial. Still, that is the reality of research and practice in the field of web-based mental health interventions (WMHIs) (1, 2). In western countries, mental health disorders such as depression and anxiety are highly prevalent (3). Evidence-based psychological therapies are available on an individual basis, but as this form of treatment is time consuming and expensive, many mental health patients do not receive treatment. WMHIs are

thought to improve the accessibility of evidence-based psychological treatment for mental disorders and are becoming increasingly popular (4).

A challenging feature of WMHIs is the high drop-out and non-usage rate (5, 6). A second challenge concerns the implementation and uptake after the initial trial is finished. Often, there is no implementation plan for this last phase, and if the implementation is prepared, the uptake in daily practice seems generally disappointing (2, 7–10). For example, a pragmatic trial examining the implementation of two evidence-based WMHIs [MoodGym (11) and Beating the Blues (12)] found that most participants never used the module or only completed one of the proposed six to eight modules (7). The lack of adherence and poor dissemination does not only hold for WMHIs, but for web-based health interventions in general. Whether in the field of oncology, diabetes (13), or physiotherapy (14), the findings are similar: there is a wide gap between the steady increase in web-based interventions that are being released and their actual uptake. Several factors for the low uptake, such as financial and managerial, have been noted. Another important factor is argued to be the marginal level of engagement of end users during the development phase (10). The more top-down interventions are developed, the more likely they will not match with the needs of the users. As a result, even after research confirmed the effectiveness of web-based health interventions, patients are unlikely to ever use them.

The active involvement of the proposed end user is considered to be an important chance to improve the success rate. With active involvement we mean the collection and usage of input from end users from the start (the design phase) to the end phase (the implementation). Involving end users during the design phase is referred to as user-centered design (UCD). UCD is defined by Preece et al. (15) as "an approach, which views knowledge about users and their involvement in the design process as a central concern."

The potential benefits of UCD are widely accepted (16). There are a variety of toolboxes available, with methods, tips, and tricks how to involve patients (e.g., *via* http://www.participatiekompas.nl/). The active involvement of patients in the design phase is argued to improve usability and credibility of an eHealth intervention. Usability is defined as the ease a user can use the intervention, and a necessary element to bind end users to an online intervention. Credibility deals with the face-validity and persuasive character of the intervention. It increases the change of acceptance and adherence to an online intervention (17). In practice, however, the actual operationalization, that is, how and when end users are involved, is rarely reported. Therefore, it is also not known how to best involve end users or to what extent adherence is improved when end users are indeed involved.

In this perspective article, we describe the lessons learned from three different web-based health interventions projects that were recently conducted. The projects were selected based on their diversity in development approach and on the experience and knowledge of the authors of this article. The authors do not offer a systematic review or quantitative evaluation but rather share recent insights to stimulate a discourse on user involvement. In paragraph three, lessons learned are summarized, and recommendations provided.

## DIFFERENT METHODS TO DEVELOP WEB-BASED INTERVENTIONS

In this section, we describe three recent eHealth projects. We start with a short description of the applied method. Next, a health intervention that was developed according to the described method is outlined. Finally, we focus on how the end users were involved.

#### Expert-Driven Method

In an expert-driven or top-down method, a small group of specialists usually develops an intervention. They identify a problem or have been asked to develop an intervention for a specific problem by others. After they reach consensus on the problem and the proposed intervention, a first draft of the intervention is developed. This draft is then discussed within the group of experts. If accepted, the intervention is further developed and tested among a few patients or other experts. Based on the feedback, the intervention is adjusted and finalized.

#### Application of Expert-Driven Method: An E-Learning Module for Suicide Prevention

The first author (Derek de Beurs) recently developed and tested an e-learning supported Train-the-Trainer program to implement the suicide prevention guideline in Dutch mental health care (18). In this study, experts in suicide prevention trained senior professionals. Next, these professionals trained their own peer-colleagues in a 1-day face-to-face training session (19). The face-to-face training was supplemented with an e-learning module, as multifaceted interventions have been found to be more effective when compared with single interventions (20, 21). The e-learning module was developed according to an expert-driven method: the research team developed scenarios for six videos to cover the content of the guideline (22). Two experienced nurses and two experienced psychiatrists were approached to act as a role model in the module. After the videos were tapped, one of the researchers edited the videos into 5-min scenes, selecting those parts that reflected the guideline recommendations best. The link to the first version of the e-learning module was sent to the research team and the clinical experts that were role models in the videos. They all commented on the selected videos and specifically on the text that accompanied the scenarios. When all experts agreed to the final version, the module was piloted among a small group of mental health professionals. A short translated demo is available *via* https://www.youtube.com/ watch?v=tj9kFrYzynw.

#### Level of User Involvement

The scenarios were developed by the researchers themselves and not presented to professionals before the actual videos were made. One researcher edited the material, deciding himself which scenes were relevant to the end users. The final version was piloted among proposed end users, but only minor adjustments were possible. By not involving end users earlier it was not anticipated that, for example, end users could not view the movies on the workplace because of firewall restrictions, and the professionals interpreted the content of the different scenes ambiguously. As a result, only 23% (122 of the 518 professionals) of the mental health professionals used the e-learning module (18).

#### Intervention Mapping (IM)

A method developed to structure the integration of evidence-based information, theory, and practice in the field of health-promoting programs is the IM Framework (23). One of the aims of the IM framework is to develop interventions or programs that are compatible with the targeted population. The framework comprises six fundamental steps and systematically guides the planning and decision-making process. Each step comprises several tasks, and the completion of these tasks guides the subsequent step. It has been used successfully in developing a range of eHealth programs (24–26). The framework starts with a needs assessment or problem analysis (step 1). In step 2, the needs assessment is used to develop the frame of the intervention by defining its objectives. Accordingly, theory-based intervention methods and practical strategies are selected (step 3). By translating the method and strategies, the intervention is than developed in step 4. Step 5 considers the adoption, implementation, and sustainability of the intervention, and in the last step an evaluation plan is developed. Although the process is presented linear and works cumulative, iterative actions are possible.

#### Application of IM: PatientTIME

PatientTIME.nl is a web-based platform aiming to support patients with malignant lymphoma in how to gain more control over the communication with their health-care provider (17, 27). The intervention makes use of different evidence-based methods; video modeling, tailoring information, pre-visit goal setting, and listening back to one's own audio-recorded visits. Registered patients get a personal secured account with information tailored to their personal needs and characteristics.

The IM framework was used as theoretical backbone of the protocol applied to develop the intervention with corresponding evaluation and implementation plan. IM mapping was chosen as a guideline because it links decisions, final materials, and activities to theory.

In the PatientTIME project, the applied protocol was altered to realize a more patient-driven protocol (17). Practical patient participatory methods were integrated in the theoretical IM framework and used to inspire when and how patients could be involved. A preparatory step was added to the IM framework to plan and prepare the patient participation throughout the entire protocol.

#### Level of User Involvement

The user involvement in PatientTIME (i.e., patients diagnosed with malignant lymphoma) was operationalized in three ways and started early in the project. First, a close collaboration with the patient association for malignant lymphoma was set up. Members of the patient association requested the researchers to develop an intervention for the purpose mentioned above. During the course of the project, the patient organization informed and supported patients and championed patient interests. They were also involved from the start in the development of the implementation plan. Second, two patients were included as research partners who were involved throughout the entire project. They were equal partners next to the researchers and clinicians by having an agenda setting and decision-making role, and their involvement ensured a continuous patient-centered view. Third, 37 patient service users (28 patients and 9 spouses) contributed to this needs assessment. Finally, the usability and credibility of the intervention was thoroughly tested with two patients and two healthy people. They were asked to test the major functionality of the intervention and were encouraged to verbalize their thoughts while testing (17).

In line with other studies, more actively involving patients during the developmental phase resulted in many changes of the intervention that otherwise would not have been made (17). As a concrete example, one of the identified issues was that the presentation of the video clips was unclear. This led the developers and researchers to improve the video diaries. When the intervention was tested in a randomized trial, the patient–program interaction showed that the core element (the video fragments) were well used (28). It seems highly likely that the involvement of patients during the design phase resulted in higher adherence. However, it was not tested if and in what way the more active involvement of patients improved the adherence to eHealth interventions.

#### Scrum

A relatively new development method in the field of online health interventions is derived from the agile working principle, originally used in software development (29). Working agile became synonym for an iterative, dynamic, and flexible way of working, involving all stakeholders during the process. One popular agile framework is called scrum, after an element of rugby that involves the team players to pack closely together with their head down in order to get possession over the ball (30). Scrum resembles a difficult task performed by many players working closely together. It is defined as "A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value" (30). Scrum shares with science that it is based on empirical data; data are gathered, and decisions are made based on that data.

#### Application of Scrum: Listeningtime

Recently, a web-based intervention for older patients with cancer and their caregivers is developed called "Listeningtime".1 The aim of the intervention is to support patients and their caregivers in reaching effective communication during clinical encounters. The content of Listeningtime is based on the expectations, needs, and

<sup>1</sup>Noordman J, Driesenaar JA, van Bruinessen IR, van Dulmen S. ListeningTime; participatory development of a web-based preparatory communication tool for elderly cancer patients and their healthcare providers. *Under review* (2017).

experiences of elderly patients2 and their caregivers and builds on the earlier discussed PatientTIME intervention. Listeningtime also makes use of short "tailored" video fragments of simulated doctor–patient interactions and an audio-facility to listen back to recorded encounters.

The scrum framework has been used to develop the intervention. Six 2-week sprints with regular meetings between researchers (in this case the product owners), developers, (ex)patients, oncological health-care professionals, and their representatives (i.e., one of a patient and one of a provider organization) were planned. The first sprint started with a "brown paper session," where the user requirements of end users (i.e., researchers, patients, and oncological health professionals) were mapped. In the following sprints, first features of the platform were built by the developers, and researchers inquired input from potential end users. After a few sprints, it appeared that a sprint session every 2 weeks was not feasibly for the researchers and end users. Sprint frequency was adapted to once every 3 weeks.

#### Level of User Involvement

At the start of the Listeningtime project, potential end users (i.e., patients and health professionals) were interviewed by the researchers (see text footnote 1). Information from this preliminary needs assessment provided input for the first product backlog that was realized during the brown paper session. Parallel to the development of the web-based platform, scripts for the video fragments were written and shared with the end users before any actual video-recordings were made. Between 1 and 14 end users were involved during the sprints. Most end users participated during one sprint, some during several sprint. Their role during the sprints was advising or/and decisional. During the different sprints, the involvement of end users was planned, but their actual attendance was not that easy to realize. To start with, the end users (oncology nurses, oncologists, and older patients diagnosed with cancer) are a difficult group to continually involve due to their high work load or the burden of the disease. Next, the short feedback loops challenged the researchers to interpret the data and adjust the intervention accordingly. More details about Listeningtime and user involvement can be found elsewhere (see text footnote 1).

#### CHALLENGES AND CONSIDERATIONS WHEN INVOLVING END USERS

User involvement during the development process is an important aspect that may influence the successful uptake of a web-based intervention. As we have seen, there are several ways to involve end users. Bellow, we highlight the challenges and considerations when involving end users.

#### When to Stop?

Working actively with end users requires that smaller hypothesis are tested in fast feedback loops. As reported in Listeningtime, the sample size for short sprints was mostly small ranging from 1 to 14 people and homogeneous (as mainly male patients with prostate cancer were involved). The major challenge is how to determine if a hypothesis is false or not. UCD is mainly assessed *via* qualitative (interview) methods (31). There are no psychometrically sound assessment tools. When can we say that enough users have given feedback on the product, and the product is ready (enough) to be implemented?

#### Involving (Mental Health) Patients and Health-care Professionals

End users for more commercial application, such as a new supermarket app, are quite easy to find, as most people tend to visit supermarkets. When developing a web-based tool for, for instance, depressed patients, it is a challenge to find several groups of 5–10 depressed patients to test prototypes of a web-based intervention. Depending on their mental health condition, it can be quite difficult to actively involve them in the project. When developing products for (mental) health professionals, the time pressure and loss of production limit the availability of end users. Therefore, involvement of mental health patients needs to be carefully planned, which might make short-term development cycles (sprints) less feasible. In PatientTIME, flexibility in terms of planning and setup was experienced as a precondition to get seriously ill patients involved (17).

#### Difference in Working Processes

Adjusting a design or changing login procedures is relatively easy to do by experienced developers. Developing, adjusting, and verifying the content of the intervention with researchers and targeted end users are a much slower process. At the start of Listeningtime (2.3), the software developers proposed to meet every week. As this was not feasible for the scientists and the targeted end users, it was agreed to organize the sprints every 2 weeks. Soon it was realized that this interval was also not manageable. Before the first sprint session, time should be invested to understand the different ways of working. In the end, this will save time and frustration.

#### Research Proposals As a Starting Point

Traditionally, when applying for a grant to develop a new WMHI, a detailed development, evaluation, and implementation plan is required. This challenges the researcher to let end users influence the development process. Additionally, often only a small portion of the budget is reserved for the actual development phase in the research proposal. We have seen that the involvement of end users takes time. When more feedback loops are incorporated, the budget reserved for the technological development becomes larger. These aspects require a different mind-set from funders, developers, and researchers.

#### CONCLUSION, LIMITATIONS, AND FUTURE RESEARCH

Thinking about user-centered design techniques and carefully planning the involvement of end users should become standard in the

<sup>2</sup>Noordman J, Driesenaar JA, Henselmans I, Verboom J, Heijmans M, Dulmen SV. Patient participation during oncological encounters: barriers and need for supportive interventions experienced by elderly cancer patients. (2016).

field of web-based (mental) health. As this article is based on only three cases and the experiences of the authors, we have not provided an extensive overview of user-centered methods for web-based health interventions. When deciding on the level of user involvement, one should balance the need for input from users with the availability of resources such as time and funding. Further research should provide guidance how to select the best user-centered design strategies for the development of web-based (mental) health intervention.

#### REFERENCES


## AUTHOR CONTRIBUTIONS

DB and RF initiated the idea. DB and IvB drafted the first version. All authors contributed to the following versions and approved of the final version.

#### FUNDING

No funding was obtained for this study.

Rationale, content and dissemination in Dutch mental health care. *Internet Interv* (2015). doi:10.1016/j.invent.2015.04.004


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 de Beurs, van Bruinessen, Noordman, Friele and van Dulmen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Enabling Psychiatrists to Explore the Full Potential of E-Health

*Melvyn W. B. Zhang1 \*† and Roger C. M. Ho2†*

*1 National Addictions Management Service, Institute of Mental Health, Singapore, 2 Department of Psychological Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore*

Keywords: eHealth, mHealth, smartphone, psychiatry, mental disorders

#### DEFINITION OF E-HEALTH

The World Health Organization in its recent statement has highlighted the potential of the evolving field of E-Health (1). It defines E-health as the process in which health resources and health care are being communicated and transferred by electronic medium (1). It states explicitly that with the implementation of E-health, it is to be expected that there will be more efficient usage of healthcare resources in the future. Indeed, there has been major advances in the technological field over the past decade, especially so, with the introduction of Smartphone and their associated applications. Smartphones represent a new modality of technology that offers more than what conventional mobile technology could offer. Given that they are equipped with tremendous computing power, they now allow individuals and healthcare workers to access not only information but also work on the go. The authors acknowledge that E-health also encompasses other modalities of technologies, such as telephone-delivered therapy, virtual reality, and text messaging technology. In this manuscript, we will focus mainly on the smartphone aspect of E-health.

#### *Edited by:*

*Stéphane Bouchard, Université du Québec en Outaouais, Canada*

#### *Reviewed by:*

*Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland*

*\*Correspondence:*

*Melvyn W. B. Zhang melvynzhangweibin@gmail.com*

*† Melvyn W. B. Zhang and Roger C. M. Ho have contributed equally to this work.*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 11 September 2015 Accepted: 01 December 2015 Published: 15 December 2015*

#### *Citation:*

*Zhang MWB and Ho RCM (2015) Enabling Psychiatrists to Explore the Full Potential of E-Health. Front. Psychiatry 6:177. doi: 10.3389/fpsyt.2015.00177*

CURRENT STATE OF TECHNOLOGICAL ADOPTION BY OTHER DISCIPLINES

The adoption of technology by healthcare professionals is not something new, as a previous systematic review (2) has highlighted that healthcare professionals are more receptive toward technology. The current technological advances implies that healthcare professionals are no longer confined to use individual workstations or computers, but they could have access to hospital records, laboratory results, and also latest updates in medical information at any location and at any time (2). This would indeed fulfill the World Health Organization's vision of using technology to make the utilization of healthcare resources more efficient. Several disciplines have been early adopters of the evolving technologies, and they have utilized it for several key areas, such as in helping individuals manage their chronic diseases (3), in facilitating rehabilitation programs for patients (4), in helping clinicians with their medical diagnosis and lastly, helping to facilitate outreach efforts in developing countries (5). The utility of the current advances in technology would seemed unlimited, given that there have been other advances in this field, which include that of the development of robots, wearable computing, and wearable devices for monitoring and further advances in virtual reality technologies.

#### CURRENT STATE OF TECHNOLOGICAL ADOPTION IN PSYCHIATRY WITH REGARDS TO SMARTPHONE APPLICATIONS

Given the current advances in E-health and technologies, it is thus key in determining whether psychiatry has embraced these technologies developments and to what extent psychiatry has utilized these technological advances. A review of the existing published literature revealed that in psychiatry, these technologies advances have been utilized in several areas, such as the application of text-messaging services in reinforcing the medication compliance for schizophrenic patients (6), and also the usage of smartphone-based software to facilitate selfreporting of symptoms in schizophrenia patients (7). The current application of technologies is not limited to just patients with schizophrenia, but similar applications have been developed and implemented for patients with bipolar disorder, in helping them to monitor their mood states (8) as well as for assessment for depressive symptoms in patients (9). There has been previous research documenting the clinical efficacy of technologies in augmenting conventional psychotherapy in helping patients with substancerelated disorders (9). Hence, it is beyond doubts that psychiatry, as a discipline, has tapped onto these advances in technologies and has embraced the technologies that E-health has introduced. In the recent edition of the British Journal of Psychiatry (10), an in-depth analysis has been given with regards to how psychiatrists could harness the best of this technological revolution. The authors of the paper that was published (10), pointed out the core issues of confidentiality and privacy issues, as well as the limitations in terms of the evidence base with regards to existing mobile phone and smartphone applications on the application store. The authors of the current paper do concur and acknowledge that these are crucial issues that need to be addressed first, prior to there being further utilization of technology for future advances in this field. It is the aim of the authors, to make use of this paper to address one of the main and core issues pertaining to the lack of evidence base with regards to existing published mobile applications and illustrate how psychiatrists could make use of simple methodologies to tap onto advances in this field, but yet ensure the evidence base of the products created, whether for education, research, or patient care. By doing so, it is thus hoped that psychiatrist would be able to explore the full potential of E-health.

#### METHODOLOGIES FOR PSYCHIATRISTS TO TAP ONTO DEVELOPMENTS IN THIS EVOLVING ARENA

#### Validation of Existing Applications Using Standardized Scales

Given that one of the core limitations identified pertaining to the implementation of E-health, especially so for smartphone applications for patient care, is the concern with regards to the safety of the application and the evidence base of the application, it is the aim of the authors to address this issue. One of the solutions might be getting psychiatrists to be involved in the identification of a collection of safe healthcare applications using systematic self-certification model. There have been previous studies conducted (11) that have proposed the application and the utilization of a systematic self-certification model for the peer review of applications. It is believed that such a methodology would be able to provide a systematic solution for healthcare workers in determining whether an application is safe and applicable for heir practice. One proposed solution involves the utilization of the self-certification model, which is based on the Health on the Net foundation (HON) guidelines. It is believed that this model would help in determining the reliability and credibility of the information presented on both medical and general health information websites. **Table 1** illustrates in detail the aspects of this model.

Another model, known as the Silberg scale (12), has also been used for the evaluation of the information quality of current applications (13, 14). More recently, a team of researchers in Australia has also proposed another scale, known as the mobile application rating scale (15). The scale has been developed after careful analysis of existing criteria for app evaluation, and the proposed scale has demonstrated good internal consistency and inter-rater reliability. Five core areas of an application could be evaluated by the scale, which ranged from that of engagement, functionality, esthetics, information quality, and a subjective quality scale (15). The development of such scales would greatly help psychiatrists in determining the evidence base pertaining to existing applications. Hence, it could facilitate a compilation of good evidence-based application that could then be recommended to patients.

Through the introduction of these scales, the authors hope that healthcare professionals could appreciate that there are methods that could be used to determine whether an application is evidence based or not.

#### Enabling Psychiatrists to be Expert Developers

By providing psychiatrists with the above criteria, it is thus hoped that more psychiatrists could make use of the above-mentioned criteria to identify more evidenced-based applications. However, psychiatrists could tap upon the full potential of E-health if they could embrace methodologies of developing applications catered to the areas to which they could use smartphone applications in, for example, in education, research, and even in clinical care.

#### Table 1 | Self-certification model.

#### Description of criteria assessed by the HON guidelines

(a) Nature of information – it is recommended that all information included within the application should be attributed to an author. The qualifications of the author should be specified

(b) Purpose of the application – it is recommended that there should be a statement indicating the main purpose of the application

(c) Confidentiality – it is mandatory that the respective application documented how information is collected, and the confidentiality and privacy associated with the information collected

(d) Information – it is recommended that all medical information included should have a date of creation as well as a date documenting when the most recent update has been made

(e) Justification of claims – it is recommended that any information included pertaining to the benefits or the performance of any treatment should be supported by concrete scientific evidence

(f) Contact details – it is recommended that appropriate contact information should be included so that the respective developer or authors could be contacted

(g) Disclosures – all sources of funding should be disclosed

(h) Advertising policies – all advertisement included within the application should have a disclaimer indicating that they are advertisement and not scientific evidence

Previous published research has encouraged clinicians to take upon more ownership of publishing application by means of educating them how best to make use of an Internet browser as well as a text editor to create applications (16). Zhang et al. (17, 18) in their recent published papers have further commented on how psychiatrists could become app developers, without the need to know programing skills. In their paper, online application builders were introduced to help facilitate this process. Apart from online application builders, they have also introduced responsive microblogging sites that could help create mobile friendly websites for instant reference on the go. In addition, they have highlighted how some of these builders could further facilitate the publication of applications onto the respective app stores, thus helping to minimize barriers with regards to the access of the applications.

#### REFERENCES


#### CONCLUSION

In conclusion, given the advances in technology in today's day and age, it is essential for clinicians to tap upon technology and harness the full potential of today's technologies. In order to empower psychiatrists, it is the perspective of the authors that psychiatrists could either make use of evidence-based methodologies to identify applications that are suitable for their usage, or empower themselves to be web-based and smartphone developers using simple methodologies shared previously. Given the simplicity and the low-cost involved using the methods introduced (17, 18), it is hoped that psychiatrists would be willing to put their ideas into action and help in the formulation of more evidence-based applications.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer Jérôme Favrod and handling Editor Stéphane Bouchard declared a current collaboration (working on the current Research Topic). The handling Editor states that the process nevertheless met the standards of a fair and objective review**.**

*Copyright © 2015 Zhang and Ho. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Prevention of Mental Health Disorders Using Internet- and Mobile-Based Interventions: A Narrative Review and Recommendations for Future Research

#### *David Daniel Ebert1 \*, Pim Cuijpers2 , Ricardo F. Muñoz 3,4 and Harald Baumeister <sup>5</sup>*

*1Clinical Psychology and Psychotherapy, Friedrich-Alexander University of Erlangen-Nürnberg, Erlangen, Germany, 2Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands, 3Palo Alto University, Palo Alto, CA, United States, 4University of California, San Francisco, San Francisco, CA, United States, 5Department of Clinical Psychology and Psychotherapy, University of Ulm, Ulm, Germany*

#### *Edited by:*

*Yasser Khazaal, Université de Genève, Switzerland*

#### *Reviewed by:*

*Gianluca Castelnuovo, Università Cattolica del Sacro Cuore, Italy Katarzyna Wac, Université de Genève, Switzerland*

*\*Correspondence:*

*David Daniel Ebert david.ebert@fau.de*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 21 September 2016 Accepted: 15 June 2017 Published: 10 August 2017*

#### *Citation:*

*Ebert DD, Cuijpers P, Muñoz RF and Baumeister H (2017) Prevention of Mental Health Disorders Using Internet- and Mobile-Based Interventions: A Narrative Review and Recommendations for Future Research. Front. Psychiatry 8:116. doi: 10.3389/fpsyt.2017.00116*

Although psychological interventions might have a tremendous potential for the prevention of mental health disorders (MHD), their current impact on the reduction of disease burden is questionable. Possible reasons include that it is not practical to deliver those interventions to the community *en masse* due to limited health care resources and the limited availability of evidence-based interventions and clinicians in routine practice, especially in rural areas. Therefore, new approaches are needed to maximize the impact of psychological preventive interventions. Limitations of traditional prevention programs could potentially be overcome by providing Internet- and mobile-based interventions (IMIs). This relatively new medium for promoting mental health and preventing MHD introduces a fresh array of possibilities, including the provision of evidence-based psychological interventions that are free from the restraints of travel and time and allow reaching participants for whom traditional opportunities are not an option. This article provides an introduction to the subject and narratively reviews the available evidence for the effectiveness of IMIs with regard to the prevention of MHD onsets. The number of randomized controlled trials that have been conducted to date is very limited and so far it is not possible to draw definite conclusions about the potential of IMIs for the prevention of MHD for specific disorders. Only for the indicated prevention of depression there is consistent evidence across four different randomized trial trials. The only trial on the prevention of general anxiety did not result in positive findings in terms of eating disorders (EDs), effects were only found in *post hoc* subgroup analyses, indicating that it might be possible to prevent ED onset for subpopulations of people at risk of developing EDs. Future studies need to identify those subpopulations likely to profit from preventive. Disorders not examined so far include substance use disorders, bipolar disorders, stress-related disorders, phobic disorders and panic disorder, obsessive–compulsive disorder, impulse-control disorders, somatic symptom disorder, and insomnia. In summary, there is a need for more rigorously conducted large scale randomized controlled trials using standard clinical diagnostic instruments for the selection of participants without MHD at baseline and the assessment of MHD onset. Subsequently, we discuss future directions for the field in order to fully exploit the potential of IMI for the prevention of MHD.

Keywords: mental health, self-help, e-health, m-Health, Internet interventions, depression, anxiety, prevention

#### INTRODUCTION

Mental health disorders (MHD) are highly prevalent, with estimated lifetime and 12 month-prevalence rates, ranging across countries between 18.1–36.1 and 9.8–19.1%, respectively (1). MHD are one of the leading causes of disability (2) and associated with an immense disease burden such as poorer quality of life of sufferers and their loved ones, an increased risk of developing chronic physical conditions and related mortality (3–5). The economic burden of these disorders is enormous, including substantial economic costs, reduced workforce participation, occupational impairment, and lost productivity (6–8).

In the past decades, a variety of interventions have been developed to treat MHD for which efficacy has been demonstrated in a large number of randomized trials (9, 10). However, even assuming the hypothetical scenario of 100% coverage and compliance to evidence-based treatments, approximately only 28% of the disease burden attributable to MHD could be averted (11). In fact, less than half of the individuals with a MHD are recognized and treated (12). Therefore, attention has increasingly been focused on the prevention of MHD.

Preventive interventions can be classified as universal interventions, directed at the whole population; selective interventions, directed at individuals with specific risk factors for the development of a MHD; or as indicated preventive interventions, directed at individuals in the prodromal stage of a disorder, who do not yet fulfill the criteria for a full blown disorder but experience subclinical symptoms (13).

Emerging evidence indicates the potential of psychological interventions for the prevention of MHD. For example, in a recent meta-analysis, van Zoonen and colleagues found psychological interventions aiming to prevent major depressive disorders (MDDs) to reduce the incidence by approximately 22% (14). Results from another review found that cognitive behavioral indicated preventive interventions reduced the transition to psychosis with a risk ratio of 0.54 (95%-CI: 0.34–0.86) (15); encouraging evidence from a limited number of randomized controlled trials is also available, for example, for the prevention of eating disorders (EDs) (16) and tobacco use (17), whereas the efficacy for other disorders such as interventions to prevent anxiety (18) is not yet established.

Although psychological interventions might have a tremendous potential for the prevention of MHD, their current impact on the reduction of disease burden is questionable. Possible reasons include that it is not practical to deliver those interventions to the community *en masse* due to limited health care resources and the limited availability of evidence-based interventions and clinicians in routine practice, especially in rural areas. Therefore, new approaches are needed to maximize the impact of psychological preventive interventions.

Limitations of traditional prevention programs could potentially be overcome by providing Internet- and mobile-based interventions (IMIs). This relatively new medium for promoting mental health and preventing MHD introduces a fresh array of possibilities, including the provision of evidence-based psychological interventions that are free from the restraints of travel and time and allow reaching participants for whom traditional opportunities are not an option.

Internet- and mobile-based interventions have been shown to be effective in clinical populations, including the treatment of depression (19–21), anxiety (20, 22, 23), alcohol use (24), and sleep disorders (25). However, evidence for their effectiveness in preventing the incidence of MHD is much less documented.

This article provides an introduction to the subject and narratively reviews the available evidence for the effectiveness of IMIs with regard to the prevention of MHD onsets. Subsequently, we will offer some suggestions regarding the direction of future research in this field.

#### CHARACTERIZING IMIs

The possibilities to use IMIs for the prevention of MHD range from mobile-based apps for the monitoring of health behavior and stand-alone self-help interventions to supplemental elements integrated in conventional on-site psychological interventions (blended concepts). One common element of such interventions is that emotional, cognitive, and behavioral processes are modified and their generalizations to users' daily lives promoted using established psychological techniques (26). IMIs can be categorized in regard to their use of technology, the extent of human support, the theoretical basis, and with respect to their areas of applications and indications (**Figure 1**).

#### Technical Implementation

For the implementation of IMIs, numerous technical possibilities are applicable. These range from (1) the provision of evidencebased strategies as interactive self-help lessons; (2) e-mail, chat, or video-based sessions (27); (3) virtual reality for exposure interventions (28); (4) serious-games, in which psychological strategies are trained in the context of a computer game (29); (5) the use of automated memory, feedback, and reinforcement interventions, for example, through apps, e-mails, text messages, or short prompts, which support the participant in incorporating intervention content into everyday life; to (6) sensors and apps for

monitoring health behavior such as physical activity, which can be used to support the learning process (30).

#### Theoretical Basis

Due to their distinctive structured nature, standardization, and focus on the training of strategies and specific behavior, IMIs are particularly suited for techniques that target changes in thoughts and behaviors (26). These include well-researched cognitive, behavioral, and interpersonal interventions. Approaches such as mindfulness-based methods, acceptance and commitment therapy, or psychodynamic approaches, which are already used within some clinical IMIs (31–35), also have the potential for application to preventive IMIs.

#### Human Support

As a basic principle, IMIs can be implemented with varying degrees of human support. The current most commonly used method is the so-called "guided self-help," in which evidence-based content is usually provided as self-help material so that the participants can perform most tasks independently. An accompanying coach then regularly gives feedback on the completed exercises. Fostering adherence to the content of the intervention is usually the main aim of human support, rather than the delivery of new therapeutic techniques that go beyond the content of the current lesson (36, 37). The main task of the coach is to clarify any comprehension questions, provide feedback on solved problems and progress, and to encourage participants to continue to work on themselves. For this to happen, communication can happen either synchronously (per chat or video) or asynchronously (for example, per e-mail), the latter of which is more commonly used, and normally takes a few minutes to a few hours (1–3 h) per participant and intervention. For the participant, the processing of self-help material, execution and repetition of exercises, as well as correspondence with a coach can, however, be very intense and require a much greater time investment than that of the supporting coach (38). The combination of self-help material with minimal human support *via* the Internet thereby increases empowerment of the participants and the degree of self-directed coping while maximizing the efficiency of the accompanying coach. Irrespective of location, asynchronous contact and timeindependent communication results in increased flexibility and autonomy for both participants and prevention workers.

An "Efficiency Model of Support" (39) has been recently put forward to contribute to the development of a taxonomy of elements involved in guided interventions, such as type, quantity, timing, quality, and cost of human support. A commentary on that article (40) suggests that it may be useful to categorize contemporary interventions into four major types: traditional face-to-face (FTF) interventions, FTF interventions augmented by behavior intervention technologies (BITs), BITs augmented by human support, and BITs that are fully automated.

#### Areas of Application

Applications of IMIs range from mental health promotion and mental disorder prevention to full treatment of mental disorders and interventions to reduce relapse or recurrence. In the fields of psychological health promotion and prevention, IMIs are considered a promising approach for increasing the accessibility of evidence-based psychological techniques to people on a larger scale due to their low threshold for accessibility, location and time independence, and anonymous usability (41). IMIs can be used in the prevention of mental disorders either as a stand-alone approach, as part of a stepped-care approach or as an integrated element of a preventive intervention consisting of online and conventional on-site sessions (blended).

As a *stand-alone measure*, IMIs increase the reach of effective psychological interventions. Telehealth interventions (live therapy online) can transcend space. IMIs can transcend space and time. For example, the temporal and spatial independence of IMIs facilitates access to evidence-based interventions for individuals with limited mobility or those who live in areas with low access to preventive interventions. Populations who are not able to attend appointments during usual visiting hours and, therefore, are not able to attend other on-site FTF options, would then also be able to participate in interventions in the evenings or on the weekend at their own pace. Persons who would have not sought to participate in a preventive intervention due to other individual reasons, such as fear of stigma, would also have access to IMIs. Despite increasing social acceptance of psychological interventions, everything that might be associated with mental problems produce for some individuals a sense of shame, which in itself creates a barrier to the actual use of preventive interventions (39, 40). Moreover, a general problem with preventive interventions lies in the fact that, by definition, the impairment people appropriate for preventive interventions are experiencing is low or not noticeable to them, diminishing their willingness to "invest" into mental health interventions. Hence, the lower the threshold and effort associated with participating in a preventive intervention, the likelier it is that the target group makes actual use of it.

In the combination of IMIs and personal FTF interventions, the so-called *Blended-Concept*, IMIs take over areas that need not necessarily be mediated by a prevention worker, allowing more time during the sessions for FTF psychological process work (26). Prevention workers could, for example, delegate time-consuming routine aspects of the intervention, such as the delivery of psychoeducation to digital tools. In principle, IMIs could also be used to improve FTF interventions by providing exercises for the participant to work on in between the intervention sessions, thereby increasing intervention intensity. Another way in which IMIs could be used to improve the outcome of FTF interventions is by supporting the integration of behavior changes or training of techniques into routine life, thus extending the reach of the psychological intervention into the daily lives of participants. This can be achieved through methods, such as smartphone-based behavioral diaries, sending of messages with ultra-short prompts aimed at training-specific strategies in daily life, or smartphone-based coaches that lead patients through potential anxiety-provoking or other difficult situations.

Furthermore, the objective of most psychological interventions is that participants actively try to integrate new behavior into their daily life and maintain these changes in the long term. IMIs emphasize the active role of the person concerned in this process, thus promoting a sense of empowerment through encouraging them to use their own resources to solve problems. IMIs could be used by people much before there is any need to seek FTF mental health services (and hopefully prevent the need to do so), during times when they are receiving treatment, as adjuncts to FTF care, and after treatment ends, to maintain gains, continue to progress in terms of reaching ever sturdier mental health levels, and to reduce relapse and recurrence.

Within *stepped-care approaches*, the degree of support participants receive are designed according to their actual individual need. In step-up interventions (guided) preventive approaches can be offered, for example, to individuals in the prodromal disease stage (indicated prevention) as a first element in the chain of treatment in order to prevent the transition to the full blown disorder. Further intensive therapeutic support, such as outpatient psychotherapy, then occurs should the patient respond insufficiently to the self-help intervention. Similarly, step-down interventions supplement more intensive therapeutic measures with lower intensity support. For example, IMI-relapse prevention concepts could be offered to patients following an acute treatment in order to stabilize acute treatment effects and thereby prevent relapse and recurrence (42–46).

#### EFFECTIVENESS OF IMI IN PREVENTING THE ONSET OF MHD

While there are currently well over 100 randomized controlled studies on Internet- and mobile-based concepts fostering mental health, only few studies to date have investigated their potential in preventing the incidence or onset of a MHD. In the following section, we will review these studies and thereby only focus on studies that assessed the effects of an IMI on MHD onset (assessed according to diagnostic and statistical manual of mental disorders/ICD criteria) in a sample of adults, adolescents, or children, who is free from an acute mental disorder at baseline. There are further studies on mental health IMIs evaluated in samples with subthreshold mental health conditions. However, these studies were often planned for other purposes than examining disorder onset and limited in their methodological quality (47). **Table 1** gives an overview of the reviewed studies, and **Tables 2** and **3** are about the characteristics of the reviewed studies and interventions.

#### Indicated Prevention

#### Depression

Three studies have evaluated the effects of Internet-based approaches with regard to the primary prevention of depression. Buntrock et al. recently published a trial on the effects of an Internet-based indicated prevention stand-alone intervention (41, 48, 60). They randomized 406 adults with subclinical symptoms of depression who did not fulfill the criteria for MDD in the last 6 months to either a 6-week guided Internet-based cognitive behavioral intervention or to an online passive psychoeducation intervention. The intervention group (IG) included behavioral activation and problem solving as core intervention components; in addition, participants were able to choose among several different optional modules (e.g., sleep hygiene/sleep restriction, progressive muscle relaxation, and rumination techniques). In addition to the Internet-based self-help module, a text message coach sent a set of "tiny tasks" to the participant's mobile phone in order to foster the application of intervention techniques in daily life. In the IG, 32% of the participants experienced a MDD during the 12 months of follow up, whereas 47% in the control group (CG) did. Cox regression analyses controlling for baseline depressive symptom severity showed a hazard ratio (HR) of 0.59 indicating a 41% reduction in the risk for developing a MDD with a number need to treat to avoid one new case of MDD of 5.9. It is important to note that, in this trial, it was not assessed whether participants had a prior history of MDD. Hence, future studies are needed to investigate whether the effects count both for first incidence and subsequent onset of MDD.

In a randomized crossover trial, Thompson and colleagues evaluated an 8-week Internet or telephone-delivered mindfulnessbased stand-alone intervention in 64 adult epilepsy patients with subthreshold depressive symptoms. They found that the incidence of MDD episodes, assessed *via* self-report from baseline to interim assessment was significantly lower in the intervention condition (0.0%) than in treatment as usual (TAU) (10.7%) condition 8 weeks following randomization. Half of the participants were assigned to receive the intervention *via* web, half *via* telephone, but the authors did not find any differences between the two different forms. Although future studies with longer follow-up periods, larger sample sizes and observer-based clinical interviews are clearly needed to determine the potential of the approach for the prevention of depression, this trial is an example of the potential of Internet-based approaches.

Imamura and colleagues (51) evaluated an Internet-based indicated prevention program with workers who self-identified as having depressive symptoms but not fulfilling the diagnostic criteria for MDD. They randomized 822 workers either to a 6-week, Internet-based cognitive behavioral program delivered in a comic-form or to a wait list CG. The CBT components of the program included self-monitoring, cognitive restructuring, assertiveness, problem-solving, and relaxation. Results showed significantly lower incidence of MDE at the 12-month follow-up, with 0.8 and 3.9% of the experimental and control participants, respectively, experiencing a MDE. This corresponds to a HR of 0.22 and a numbers needed to be treated of 32 in order to prevent one case of major depression. However, the results need to be interpreted with caution as the diagnosis of MDD had been established only on the basis of a self-report instrument [World Mental Health Composite International Diagnostic Interview (WHO-CIDI) self-administered] and not using standard clinician/expert-based diagnostic instruments.

#### Anxiety Disorders

The only trial we are aware of that evaluated the effects of an IMI on anxiety disorder onset is an indicated prevention of general anxiety disorder trial conducted by Christensen et al. (50). They evaluated three different versions of iChill, a 10-week Internetbased cognitive behavioral intervention for anxiety symptoms (without any reminders, phone reminders, and e-mail reminders) compared to two attention placebo CG (interactive, attentionmatched, Internet-based placebo control program "Healthwatch" with/without phone reminders) in 558 young adults (age 18–30) with general anxiety symptoms who did not meet the criteria for an anxiety disorder.

Generalized anxiety disorder onset at 6-month follow up was 6.7% across all IGs, and 4.5% across both CGs, a difference that was not statistically significant.

#### Selective Prevention Depression

Christensen and colleagues evaluated the effects of a 6-week unguided fully automated Internet-based intervention for sleeping problems (SHUTi) with regard to the prevention of major depressive episodes (49, 61). They randomized 1,149 adults with primary insomnia and depressive complaints who did not fulfill the criteria for a major depression to either SHUTi or to Healthwatch. Although large effects in the SHUTi group on insomnia complaints and a lower depression symptoms on the patient health questionnaire-9 at 6 weeks and 6 months compared with Healthwatch were found, the intervention was not superior with regard to the effect on diagnosis of MDD assessed with the Mini-International Neuropsychiatric Interview. However, only ~4% percent of the total sample developed a major depression during the 6-month follow up, making it difficult to detect any preventive effects, even at such a large sample size. Hence, future studies on the longer term follow-up data of this trial (12/18 months) are needed in order to conclude whether this approach is indeed promising with regard to the prevention of major depressive episodes.

#### Eating Disorders

At least two randomized trials have been conducted to date that focused on the selective prevention of ED onset. Taylor and colleagues evaluated the effects of an online cognitive behavioral selective preventive intervention (StudentBodies) on ED incidence compared with a wait list control condition over 3 years in a sample of women with a body mass index >24 and any baseline compensatory behaviors (53). Although they found Table 1 | Summary of included randomized controlled trials that assessed MHD onset using categorical ICD/DSM diagnostic criteria.



*Universal—universal prevention. Interventions directed at the whole population; selective—selective prevention. Interventions directed at individuals with specific risk factors for the development of a MHD; indicated—indicated prevention. Interventions directed at individuals in the prodromal stage of a disorder; relapse prevention—interventions aiming to reduce relapse and recurrences after first onset.*

*aTwo waves (wave 1: N* = *896, wave 2: N* = *771).*

*CES-D, Center for Epidemiological Studies Depression Scale; CBT, cognitive behavioral therapy; IG, intervention group; CG, control group; SCID, structured clinical interview for DSM disorders; MDD, major depressive disorder; RRR, relative risk reduction; HR, hazard ratio; p, level of significance; NNT, number needed to treat; MHD, mental health disorders; MINI, Mini-International Neuropsychiatric Interview; PHQ, patient health questionnaire; GAD, generalized anxiety disorder; PD, panic disease; SP, social phobia; BDP, borderline personality disorder; WHO-CIDI, World Mental Health Composite international Diagnostic Interview; EDs, eating disorders; LIFE, longitudinal interval follow-up evaluation; TAU, treatment as usual; DSM, diagnostic and statistical manual of mental disorders.*

1


0 0 0 0

1

0 0

Table 2 | Target conditions addressed by studies investigating the effectiveness of Internet- and mobile-based interventions on mental health disorders onset.

August 2017


Article 116

Total number of studies

5 0 3

Table 3 | Characteristics of studies and interventions that investigated the effectiveness of Internet- and mobile-based interventions on mental health disorders onset.

0

0


the intervention to be efficacious in reducing high weight/shape concerns over a period of 24 months and lower ED incidence rates in the intervention (5%) group compared to controls (10%), the difference in ED onset was only significant in a subgroup of individuals, recruited and treated at one particular trial site. However, they identified in subsequent analyses specific risk factors associated with ED onset [i.e., comments/teasing about eating from a teacher, coach, or sibling and lifetime depression (62)]. Based on these findings, they adapted the IMI to target these specific risk factors and subsequently evaluated this intervention (image and mood) in 185 young adult women (age 18–25) with elevated weight/body shape concerns, eating-related teasing, depression and compensatory behavior compared to a wait list-CG. Although ED onset rates within a 24-month follow up was 27% lower in the intervention compared to the CG, this did not reach the level of statistical significance. Significant lower onset rates were, however, found in a subgroup with the highest body shape concerns, onset (20 vs. 42%, number needed to treat = 5), indicating that it is possible to prevent EDs in very high-risk samples. Another study on StudentBodies showed promising effects on subthreshold ED onset, but did not investigate the effects on full blown disorder onset (63).

#### Universal Prevention

To the best of our knowledge, only one study has been published to date evaluating a universal prevention approach with regard to MHD onset. Lindenberg and Kordy (52) evaluated a universal approach for the prevention of EDs in secondary education students (age 13–16) with no prior ED diagnosis and not undergoing treatment for any ED. Young E[s]sprit is a stepped program tailored to the individual risk of the participant, with elements ranging from screening and tailored risk feedback plus recommendations for specific self-help modules, through monitoring of risk behavior and symptoms and synchronous group and individual online chats up to individual FTF counseling. Schools including a total of 1,667 adolescents were clusterrandomized in two waves to receive either Young E[s]sprit or an online-psychoeducation intervention. Results showed significantly reduced ED onset rates in the IG compared to control schools in the first wave (intervention: 5.6%, controls: 9.6%) but the second wave (intervention: 5.6%, controls: 4.8%) did not yield significant differences in the overall analyses.

#### Relapse Prevention

By now, quite a few studies have investigated the effects of IMI as a relapse prevention intervention (42–45, 64, 65), and at least two of these also investigated the effects of an IMI on the prevention of MHD relapse.

Holländare and colleagues (55, 56) evaluated a 10-week guided self-help cognitive behavioral intervention for the prevention of relapse in 84 partially remitted depressed adults with at least one previous depressive episode in the past 5 years compared to a no-treatment CG. Six (intervention 5%, controls: 37.8%) and 24 months (intervention: 13.7%, controls: 60.9%) following randomization they found lower rates of relapse in the IG compared to the CG with a HR for time to relapse within 24 months of HR 0.16 in favor of the IG.

Bauer and colleagues (57) investigated the effects of a synchronous transdiagnostic non-manualized Internet-chat group as a stepped-care intervention following inpatient psychotherapy of MHD compared to TAU in 152 adults. They found the chat group to significantly reduce the risk for relapse with 46.5 and 22.2% of the participants experiencing a relapse within 1 year of inpatient discharge in the control and IG, respectively.

#### FUTURE DIRECTIONS FOR THE FIELD

Although significant strides have been made in recent years regarding the development of effective IMIs for the prevention of MHD, there are several important directions for future research.

#### THE NEED FOR MORE RIGOROUSLY CONDUCTED LARGE-SCALE RANDOMIZED CONTROLLED TRIALS

First, the number of randomized controlled trials that have been conducted to date is very limited and so far it is not possible to draw definite conclusions about the potential of IMIs for the prevention of MHD for specific disorders. Only for the indicated prevention of depression there is consistent evidence across four different trials (three primary prevention trials aiming to reduce first incidence, one relapse prevention trial). However, only one primary prevention trial (48) and the relapse prevention trial (56) used standard diagnostic procedures, while the other two trials relied only on self-report questionnaires for onset identification. The only trial on the prevention of general anxiety did not result in positive findings. In terms of EDs, effects were only found in *post hoc* subgroup analyses, indicating that it might be possible to prevent ED onset for subpopulations of people at risk of developing EDs. Future studies need to identify those subpopulations likely to profit from preventive IMIs. It should be noted that two of the five successful prevention trials with positive findings (54, 56) were based on very small sample sizes, whereas several much larger trials did not find positive results. Disorders not examined so far include substance use disorders, bipolar disorders, stress-related disorders, phobic disorders and panic disorder, obsessive–compulsive disorder, impulse-control disorders, somatic symptom disorder and insomnia. However, it is of note that there is quite substantial evidence for the effectiveness of health behavior change IMIs regarding the reduction of problematic alcohol consumption (24), improving sleep (66–68), reducing work-related stress (36, 69–71), all of which might be useful as MHD prevention IMIs as well. In summary, there is a need for more rigorously conducted large-scale randomized controlled trials using standard clinical diagnostic instruments for the selection of participants without MHD at baseline and the assessment of MHD onset.

#### ASSESSING DIAGNOSTIC STATUS AT BASELINE AND FOLLOW-UP

Second, one general problem of prevention trials is that one needs very large sample sizes in order to be able to detect existing differences between groups, as transition rates to full blown disorders tend to be low during follow-ups of typical length of controlled studies even in high-risk groups. This problem is (could be) true of many of the studies reviewed above (49–51, 53, 54, 58). Given the low chance of positive findings in a trial that is not primarily designed nor powered to detect such findings, it is understandable that many prevention researchers abstain from assessing diagnostic status. However, even without positive findings such information would be very valuable for the field and we would like to encourage researchers to assess these data. Diagnostic status data could help to obtain the necessary information on the transition rates of the target population and generate hypotheses on the size of the effect on MHD onset, which are both necessary to design and power subsequent trials adequately. Moreover, if several of such smaller trials would be conducted, the data of these trials could be combined using individual participant data meta-analytic techniques (72, 73). By collecting and pooling the primary data of individual trials, multiple underpowered trials can contribute to a large enough pooled sample size with sufficient power to examine effects on the incidence of MHD as well as analyzing patient subgroups and other effect modifying variables (74). If conducting observer-based clinical interviews is too expensive, web-based self-administered version of instruments such as the WHO-CIDI could be a low-cost alternative (75).

#### EXAMINING DIFFERENT PREVENTION SETTINGS AND APPROACHES

Provided that preventive IMIs are transferable to clinical praxis, we further need to establish the best way to implement preventive IMIs into our health care systems. Integrating IMIs as a first step of stepped-care approaches might be one promising way. Prevention IMIs for MHDs might also easily be integrated in already existing prevention programs (blended MHD prevention). That this would be worthwhile is supported by recent meta-analytic findings, indicating that traditional FTF interventions profit from providing additional IMI-components (76). Lindhiem and colleagues showed in their systematic review on 10 RCTs that a mobile component as a supplemental element in psychological interventions (e.g., SMS to support behavior changes between therapy sessions) considerably increase the effectiveness of these interventions compared to the respective strictly on-site interventions (SMD = 0.27) (76). IMIs also offer the unique potential of reaching people who would not access preventive mental health care *via* the established channels delivery, either because they do not utilize the available offers or because they do not feel comfortable discussing their mental health issues with their general practitioners and mental health specialists. Thus, we should also think about implementing preventive MHD–IMIs in a way that allows people to self-refer to IMIs [10]. This might, for example, be achievable by providing preventive IMIs *via* websites of established associations or as a direct prevention offer from health insurance companies, which would probably make it necessary for several countries to think about alternative financing health care models to integrate preventive IMIs in the best possible way. However, an approach just aimed at the GP and health insurance companies might be too narrow to exploit the potential of preventive services, as the majority of the target population might not use them. Preventive IMIs should be, therefore, I think that preventive interventions should be delivered through multiple channels that have "natural" possibilities to engage people, such as schools (they can reach all students), universities and colleges, pregnant women (because they all receive prenatal care), patients with general medical disorders, but also, for example, companies.

#### STUDIES IN CHILDREN, ADOLESCENTS, AND YOUNG ADULTS

Most of the trials that have been conducted up to this point have included only adults as participants. Although the general potential of IMIs to foster mental health in children and adolescents has been documented (20), only one study has investigated the effects of a prevention IMI, delivered as a universal preventive approach, on MHD onset (52). Given that ~75% of all MHD have their onset before the age of 25 (77), future studies should explore the potential of IMIs for preventing the first incidence in children, adolescents and young adults. That this is possible using a psychological intervention has been shown, for example, for depression (78). An interesting development in this field is a shift away from traditional computerized and browser-based interventions to mobile-based smart phones interventions. Using the most up-to-date technology and/or access paths for providing mental health prevention might increase the interventions attractiveness and user-friendliness in this modern technology oriented age group. However, evidence from mobile-based interventions and other mode of deliveries is yet scarce and future studies are needed to explore their potential. One of these other possibilities are serious games or even augmented reality interventions, that go beyond what can be done in typically "talking" interventions likely to attract children and adolescents. The potential of serious games to foster mental health has been shown, for example, in the field of depression (29, 79), but yet there is no evidence regarding the prevention of MHD onset, with one trial currently being conducted (80). The use of augmented reality has to the best of our knowledge not yet been explored. One general problem with psychological interventions for the prevention of MHD in children and adolescents is that one needs parental consent, which is difficult to obtain online in a reliable manner, and which can be seen as barrier to reach these high-risk groups.

Although many MHD already have their initial incidence before college matriculation (81), college entry might nevertheless be a very promising point in time to deliver preventive psychological interventions. College entry allows screening of the whole college student population, identification of those at risk for development of mental health problems, and subsequent offers of targeted preventive interventions. These might be disorder-specific prevention IMIs or trainings focusing on missing skills and competencies (e.g., procrastination, limited social competencies, and low self-efficacy) as well as other known risk factors for developing mental disorders. Similarly, entering vocational schools and the working environment might be a possibility to establish mental health screening and subsequent mental health trainings. In this context, where freshmen might be anxious about disclosing mental health issues, the possibility of IMIs being provided anonymously can be regarded as one substantial advantage over other occupational and college mental health management programs.

#### EVALUATING THE ROLE OF HUMAN SUPPORT IN PREVENTIVE INTERVENTIONS

After development of an IMI, ongoing costs are directly related to guidance time. Hence, evaluating and comparing the effectiveness of interventions with different guidance formats is of particular importance. In the present review, only three of ten trials examined an unguided intervention, of which only one found significant, but very small effects on disorder onset. This is in line with previous findings from the clinical field that indicate that IMIs with human support have a significantly greater success than IMIs without therapeutic support (82–84). However, unguided IMIs might still produce larger effects at population level with regard to the reduction of disease burden, as more individuals could be reached at a given budget (85). A recent review on the cost-effectiveness of IMIs for improving mental health problems suggested that guided IMIs might be more costeffective than unguided IMIs despite their higher intervention costs per participants (86). Hence, there is a need for studies that compare not only the cost-effectiveness of guidance vs. no guidance in randomized trials but also different intensities and forms of guidance [e.g., guidance concept with individual feedback on the completed exercises, i.e., content feedback, vs. feedback aiming only to increase the adherence to the intervention (37)]. Moreover, the type of human support might not only have an impact on the effectiveness of interventions but also on the willingness to use such interventions. Given that the effects of interventions on a population level also depend on the acceptance and the reach of the target population, studies should address both effectiveness and reach of IMIs with different forms of guidance.

#### COST-EFFECTIVENESS STUDIES

Implementing preventive IMIs into our health care system might be a promising strategy regarding the cost-effectiveness of the health care system as has been recently estimated on the basis of a Markov-model study (87). However, evidence from randomized controlled trials is still scarce when it comes to IMIs for the prevention of MHD Studies indicated that guided Internet interventions for depression, anxiety, sleeping problems, smoking cessation, and alcohol consumption have favorable probabilities of being more cost-effective when compared to controls (86, 88). However, these studies were mainly directed at the treatment of mental health problems, and only one study has to the best of our knowledge been published so far, that investigated the cost-effectiveness of an IMI with regard to the prevention of MHD onset (59). Two ongoing studies might provide first results in the near future (41, 46).

#### POSSIBLE ADVERSE EFFECTS OF MHD PREVENTION IMIs

As with any other method, it is important to take into account the limitations and risks are involved with IMIs alongside all of the potential benefits of the procedure. At this stage, however, reliable empirical information showing negative effects of preventive IMIs has been very limited (89) and also in the present review, none of the identified studies reported results on potential negative effects.

Potential risks and negative effects include, depending on the concept, the following points, among others: (1) limited ability to timely identify patients prone to self-injury, for example, in relapse prevention interventions; (2) the development of reduced health-related self-efficacy if participants are not successful with a stand-alone IMI; and (3) the development of negative attitudes of non-responders toward psychological interventions in general and as a result a reduced willingness to utilize mental health care in case of MHD onset. Possible negative effects of such interventions cannot be ruled out at present, which counts to a similar degree also to classical FTF psychological interventions. There is an urgent need for further research.

One "cautionary tale" that has been published involves a secondary analysis of the impact of a mood management intervention embedded in an online smoking cessation randomized control trial. Participants came to the online site in order to stop smoking. Half of the sample was randomly assigned to a cognitive behavioral mood management intervention. The authors wondered if smokers at risk for major depressive episodes (defined as having subthreshold levels of major depression symptoms at baseline) who had been randomly assigned to receive the mood management intervention had lower incidence of major depressive episodes at follow up, thus showing a preventive effect. The results showed that the incidence was actually significantly greater for the group assigned to the mood management condition (90). The authors speculate that being assigned to a mood management intervention when one is not looking for such might make a participant more aware of depressive symptoms, thus increasing their self-report scores at follow-up assessments. They then examined a subsequent study in which participants in a similar online smoking cessation trial could freely choose (rather than being randomly assigned to) the elements of the interventions provided, including the same mood management intervention. In this study, participants screening positive for major depression were more likely to choose the mood management intervention, and, if they did so, were more likely to quit smoking (91).

These studies suggest that providing participants with a choice of interventions may be preferable to assigning them interventions that they did not choose and that participants are likely to choose appropriate interventions for themselves.

#### MAKE USE OF THE TECHNOLOGICAL POTENTIAL OF IMIs

All of investigated interventions were delivered over the Internet, only one study additionally used the mobile phone to facilitate the transfer of learned skills in daily life routine (48). None of the studies were delivered mobile only and none of the interventions' used smartphone sensors, wearables, or artificial intelligence algorithms. It seems that the field so far focused mainly on delivering psychological intervention as a (guided) self-help format through the Internet, without making full use of the technological potential of such approaches. Artificial intelligence algorithms based on user behavior might bare a great potential for supporting participants to change behavior, for example, in form of just-in-time adaptive intervention (92). However, their potential with regard to preventing MHD onset still needs to be proven and should be investigated in future studies.

#### DEVELOPMENT OF MULTIVARIATE PREDICTION ALGORITHMS TO IDENTIFY PEOPLE AT RISK AND MATCH PARTICIPANTS TO INTERVENTIONS

One general problem in the field of prevention is identifying the right people to target with specific preventive interventions. Many, if not most, individuals displaying single risk factors stay disease free without intervening, which has also occurred in most of the trials reviewed above. The combination of several specific risk factors might be a promising strategy toward overcoming this problem (93), and has been utilized in some first trials (58). Recent advantages in the field of precision medicine and machine-learning techniques might further help to identify the people at highest risk who might profit from a preventive intervention. There have been first studies in the mental health field predicting MDD onset (94, 95) in the general population, general anxiety and panic onset in general practice attendees (96) panic recurrence (97), or recurrence of suicidal ideation (97). A next necessary step in the field is to develop and validate such algorithms further for different populations and then, subsequently, test whether applying these risk prediction algorithms with subsequent preventive interventions is effective in reducing incidence of MHD in these high-risk groups.

The use of supervised machine-learning methods could also be used to explain the heterogeneity in intervention response (98, 99) in order to finally match individual to specific interventions. The necessary large sample sizes to develop such prediction equations are feasible to obtain with scalable IMIs.

A high-risk group that has been noticeably absent from preventive IMI trials is the group of individuals with genetic markers associated with mental disorders. A recent study (100) suggests a method that could combine genetic research with IMI depression prevention research. A massive sample collected by the genetic screening company was used to identify 15 genetic loci associated with risk of major depression: 75,607 individuals reporting clinical diagnosis of depression were compared to 231,747 individuals reporting no history of depression. A replication data set (45,773 cases and 106,354 controls) was then used to confirm the findings.

This very large sample could be the basis for an interesting set of depression prevention studies. Subgroups of individuals who have no history of depression but have the genetic markers associated with risk could be recruited for randomized trials testing promising IMIs. Such studies could contribute to prevention science in several ways by helping to confirm whether individuals with the markers actually have higher risk for developing major depressive episodes, particularly when confronted with stressful life events. They could also determine prospectively which of the 15 loci yield the greatest risk and help determine whether interventions that can be conducted online can reduce incidence in individuals with genetic loading for depression.

## TARGETING UNDERLYING RISK AND PROTECTIVE FACTORS IN INDIVIDUALLY TAILORED INTERVENTIONS

The use of IMIs allows the provision of tailored interventions on large scale to a degree unlikely ever to be implementable using traditional FTF approaches. So far, most studies evaluated standardized interventions based on CBT, including standard packages (e.g., cognitive restructuring, behavioral activation, relaxation in a depression prevention intervention, etc.) for all participants regardless of the individual specific risk or protective factors, as well as intervention preferences. One potential next step for the field would be to develop and provide intervention modules that target specific underlying risk or protective factors (e.g., rumination, emotion regulation, social skills, experiential avoidance, compensatory behavior anxiety sensitivity, and physical activity) and tailor the combination of these modules based on the individual risk and need profiles of the participant. However, whether such an approach is superior to far more simple to develop and maintain standardized interventions that provide non-tailored to all participants is an empirical question that has yet to be adequately addressed.

#### STRATEGIES TO INCREASE REACH AND UTILIZATION OF AVAILABLE INTERVENTIONS

The impact of evidence-based preventive intervention on population level incidence heavily depends on the acceptance and use of such interventions in the target population (101). As the field of MHD prevention continues to make progress in identifying programs that yield positive effects, it will be important to develop and evaluate effective and cost-effective strategies to disseminate evidence-based programs. Hence, there is a need for studies that investigate potential obstacles in order to subsequently develop and evaluate strategies for overcoming them. For example, Ebert, Baumeister, and colleagues evaluated acceptance—facilitating interventions that address potential barriers for acceptance of IMIs in different target populations in a series of randomized controlled trials (102–104). Moessner and colleagues evaluated the effectiveness and cost-effectiveness of different school-based dissemination strategies for the prevention and early intervention in EDs (105). However, such experimentally examined strategies to increase the reach of preventive interventions are scarce and need to be conducted more often in order to fully exploit the potential of preventive interventions on the population level.

#### CONCLUSION

Internet- and mobile-based interventions are flexible, technically diverse methods which lend themselves to a variety of application areas. Such approaches have an ability to reach target groups in a way not yet achieved by classical FTF activities. A number of studies have shown that such interventions can be effective in preventing mental disorders. But clearly much more research is needed in order to fully determine the potential of IMIs for substantially reducing the immense disease burden of MHD at the population level.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

DE drafted the first draft of the manuscript, all authors contributed to the further writing of the manuscript.

#### FUNDING

The preparation of this manuscript was funded by the European Commission (ICare: Integrating Technology in Mental Health Care in Europe; 634757).


in rehabilitation]. *Rehabilitation (Stuttg)* (2013) 52:155–63. doi:10.1055/ s-0033-1343491


controlled trial. *Diabetes Res Clin Pract* (2014) 105:30–9. doi:10.1016/j. diabres.2014.04.031


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer KW and handling editor declared their shared affiliation, and the handling editor states that the process met the standards of a fair and objective review.

*Copyright © 2017 Ebert, Cuijpers, Muñoz and Baumeister. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Online Positive Interventions to Promote Well-being and Resilience in the Adolescent Population: A Narrative Review

*Rosa M. Baños1,2,3\*, Ernestina Etchemendy1,2,4, Adriana Mira2,5, Giuseppe Riva6,7, Andrea Gaggioli6,7 and Cristina Botella1,2,5*

*1CiberObn ISCIII, Valencia, Spain, 2Red de Excelencia PROMOSAM (PSI2014-56303-REDT), Madrid, Spain, 3Universidad de Valencia, Valencia, Spain, 4University of Zaragoza, Teruel, Spain, 5Universitat Jaume I, Castelló, Spain, 6 Interactive Communication and Ergonomics of NEw Technologies – ICE-NET Lab, Milano, Italy, 7Applied Technology for Neuro-Psychology Lab, Milano, Italy*

#### *Edited by:*

*Stéphane Bouchard, Université du Québec en Outaouais, Canada*

#### *Reviewed by:*

*Caroline Vandeleur, Centre hospitalier universitaire vaudois, Switzerland Claudie Loranger, Centre intégré de santé et de services sociaux de l'Outaouais, Canada*

*\*Correspondence:*

*Rosa M. Baños banos@uv.es*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 29 July 2016 Accepted: 13 January 2017 Published: 30 January 2017*

#### *Citation:*

*Baños RM, Etchemendy E, Mira A, Riva G, Gaggioli A and Botella C (2017) Online Positive Interventions to Promote Well-being and Resilience in the Adolescent Population: A Narrative Review. Front. Psychiatry 8:10. doi: 10.3389/fpsyt.2017.00010*

Numerous studies have shown an alarming prevalence of depression, anxiety, and behavior disorders in youth. Thus, prevention of psychological problems in this population becomes crucial. According to the World Health Organization (1), prevention should also include the promotion and development of the individual's strengths in order to reduce vulnerability to suffering from mental disorders. In addition, other key elements of prevention are the reach, adoption, implementation, and maintenance of interventions. The information and communication technologies, especially the Internet, have much to offer in terms of the prevention and promotion of positive mental health in adolescents. This paper reviews these fields of research—prevention, positive psychology, Internet, and adolescents—and discusses the potential of positive interventions delivered over the Internet as effective and sustainable health promotion tools. The paper provides a brief description of the systems developed so far and a summary of selected features of the studies detected in the literature review. The overall conclusions are that there is a need for more controlled studies with long-term follow-ups, the interventions should be designed considering the specific features of the target users and the specific contexts where the interventions will be delivered, and they could be enhanced by the use of other technologies, such as smartphones, sensors, or social networks.

Keywords: adolescents, mental health, prevention, positive psychology, online interventions, Internet

#### INTRODUCTION

Mental health disorders represent a large proportion of the disease burden (2), decreasing quality of life, and increasing the vulnerability to developing severe disabling diseases. Moreover, they have an important economic impact on society (career, family cost, low productivity, etc.) (3, 4). For the adolescent population, these problems take on special importance. Numerous studies have shown an alarming prevalence of depression, anxiety, and behavior disorders in youth (5, 6), and both retrospective and prospective research has shown that most adulthood psychiatric disorders begin in childhood and adolescence (7–10).

Estimates indicate that about 10–20% of children and adolescents suffer from mental health problems worldwide, but large differences in prevalence estimates across countries have also been noted (1). Thus, prevention of psychological problems in the youth population is crucial, and the promotion of young people's mental health and well-being is considered a key priority in the *EU Framework for Action* and *H2020* (11). Evidence shows that mental illness can be prevented in both adult and youth populations (12–15), and that intervening during childhood and adolescence maximizes the benefits of prevention tasks (16, 17).

According to the World Health Organization (1), prevention involves different actions aimed to reduce risk factors, interrupt the disease's progress, and reduce its consequences. However, mental health is more than just the absence of mental illness; instead, it is "*a state of well-being in which the individual realizes his or her own abilities, can cope with the normal stresses of life, can work productively, and is able to make a contribution to his or her community* …" [(18), p.1]. Mental health also implies a positive emotional state, positive expectations for the future, and an adaptive way of interpreting reality (19). Therefore, prevention should also include the promotion and development of the individual's strengths, encouraging protective variables that empower the person and act as barriers and shields that can reduce vulnerability to suffering from mental disorders (1).

Evidence points out that some human strengths act as buffers against mental illness (20). Nevertheless, literature on adolescents and a prevention approach addressed to fostering individual strengths and protective variables, such as resilience, optimism, and positive emotions, is scarce (21). Although support for interventions designed to enhance these variables in adolescence is steadily growing, more research is needed (22).

Another key element of prevention involves reaching as many people in need as possible (23). The information and communication technologies (ICTs), especially the Internet and mobile devices, offer important advantages for reaching different target groups (24). ICTs offer accessibility with an attractive costeffective relationship for the challenges of mental disorders and prevention.

In the case of adolescents, these advantages have even greater potential because they are considered "*digital natives*." Therefore, this population is not as hindered by potential obstacles and barriers, such as acceptability of technology, self-concordance, digital format, etc. Most adolescents are fully immersed in digital worlds, and their activities, relationships, and concerns are being defined by technologies.

In sum, ICTs can offer several advantages for fostering individual competencies, resources, and psychological strengths during adolescence. However, this area of research has hardly been considered. Thus, this paper seeks to contribute to this research gap and clarify the current state of the art. The paper starts with an explanation of the role of positive psychology (PP) in preventing mental health problems and promoting well-being and resilience in the adolescent population. Then, it presents an overview of the advantages of ICTs, specifically focusing on delivering mental health interventions for youth *via* the Internet. Finally, the results of a literature review combining the three fields of research—PP and Internet interventions in youth—are presented, providing a brief description of the systems developed so far and a summary of selected features of these studies.

#### The Role of PP

Positive psychology is the study of the conditions and processes that contribute to the flourishing or optimal functioning of people, groups, and institutions (25). Its main areas of study can be grouped in four categories (26): (a) positive emotions (e.g., happiness, flow, etc.) and their effects on psychological and physical functioning; (b) positive individual traits (e.g., values, talents, etc.) and their protective role in different physical and psychological disorders; (c) positive interpersonal relationships (e.g., friendship, marriage, etc.); and (d) positive institutions (e.g., family, school, business, etc.).

Positive psychology has also been interested in the development of positive psychology interventions (PPIs). Sin and Lyubomirsky (27) defined them as intentional activities specifically addressed to cultivate positive feelings, cognition, and behaviors. PPIs are exercises (e.g., counting your blessings, practicing kindness, expressing gratitude, using personal strengths, etc.), which have demonstrated empirically to increase positive emotions, satisfaction with life, or other positive states. In this sense, PPIs are not activities focused on remedying or healing pathological or negative aspects, but on contributing to well-being and health through activities aimed to enhance positive affect, cognitions, and behaviors, and they may be considered as a complementary strategy in mental health promotion and treatment (28).

Currently, there is substantial evidence about their effectiveness in improving well-being, reducing depressive symptoms (27, 28), and increasing the effectiveness of available psychological treatments (29). PPIs have also emerged as promising tools in the promotion and prevention of mental health (30–33).

Due to the high prevalence of emotional problems in young people worldwide (7), skills for improving well-being and happiness should be taught to teenagers as well (34, 35). Adolescence is an optimal stage to do this because it is a crucial period in human personal and emotional development (identity construction, social relationships, etc.). PP tries to create a fresh conceptual framework that allows the development of interventions focused on human strengths and individuals' potential, rather than on their deficits or the troubled image traditionally associated with adolescence (34). However, it is necessary to further investigate the development and application of PP interventions in young people because the evidence is still scarce (36–38). Research with PPIs has mostly been carried out with adult populations (28), and although some studies have shown the effectiveness of PPIs in increasing well-being in adolescents (39, 22), the need for more specific knowledge about the application of PPIs in the young population is evident (22).

#### Technology and Interventions

The use of ICTs to enhance health services is increasing due to numerous advantages they offer to the health-care system. Increased accessibility to interventions, reliability (efficacy/ effectiveness), and financial efficiency in health-care systems have been identified as relevant benefits of e-health in different studies (40–43). Its contributions to the five dimensions postulated by the RE-AIM model (reach, efficacy/effectiveness, adoption, implementation, and maintenance), a framework extensively used to assess the public health impact of interventions (44), are supported by a large body of literature. Moreover, in the case of adolescents, ICTs facilitate access to their habits, culture, communications, social connection, etc. (45, 46). A comprehensive report elaborated in 2010 detected that young people consume technological devices for at least the same amount of time an adult spends daily at his/her work place, they use ICTs 7 days a week, and several devices simultaneously (47). Therefore, using ICTs to approach adolescents could facilitate the implementation of preventive interventions. Recent studies highlight the advantages of technologies to engage this population, monitor their behaviors, and provide them with information, interventions, etc., although literature on effectiveness is limited (48–50).

#### Positive Technology/Internet-Based Positive Technology

The brief duration of PPIs makes them good candidates for administration over the Internet, and online administration improves their dissemination. Proposals have been made to capture the fruitful relationship between PPIs and ICTs, and different terms have been suggested. Mitchell et al. (51) offered the term *online positive psychological intervention* and defended the potential of these interventions as an effective and sustainable health promotion tool within a comprehensive approach to mental health care. Riva et al. (52) and Botella et al. (53) proposed the term *Positive Technologies* to refer to the "scientific and applied approach that uses the technology for improving the quality of our personal experience with the goal of increasing wellness, and generating strengths and resilience in individuals, organizations, and society" [(53), pp. 1]. This field tries to combine and enhance the objectives of PP using all the possibilities offered by ICTs. Currently, there is already evidence of the effectiveness of these positive technologies in adult populations (28, 51), For example, Baños et al. (54) reported positive results of a selfguided Internet intervention to induce positive emotions and reinforce psychological resources. Shapira and Mongrain (55) also tested positively the effectiveness of two online exercises to help individuals experience self-compassion and optimism. Gander et al. (56) reported the impact of nine strengths-based positive interventions on well-being and depression in an Internet-based randomized placebo-controlled study, and these authors also offered data about the efficacy of PPIs *via* the Internet for older age groups (57). However, little work has been done in adolescence. Whereas there is a range of Internet-based well-being programs based on PP available to the general public (28, 51), there are not many Internet interventions specifically designed to promote well-being and resilience in adolescents and youth.

The present study conducts a narrative review that describes and discusses the state of the art of this field. Specifically, the purpose of this paper is to review the scientific literature on PPIs delivered through technological systems for promoting health and well-being in adolescent and youth populations.

#### MATERIALS AND METHODS

In April 2016, a bibliographic search was conducted on the PsychINFO and PubMed databases. In addition, experts in the field were contacted for information on relevant material. Studies were eligible if: they have been focused on prove the efficacy of an online PPI to promote well-being and resilience in adolescent population; they were published until April 2016; they were English studies, and they were randomized controlled trials (RCTs). Studies were excluded if they were review articles. The search was limited to RCTs because a recent meta-analysis suggested that effectiveness of PPIs could be overestimated in not controlled studies (28). The key terms used were: "Positive Psychology" OR well-being OR wellbeing OR resilience OR happiness AND adolescent OR youth OR teenager AND "Internet intervention" OR online OR "Internet-based" OR "web-based" AND prevention OR "mental health." Two independent evaluators reviewed and selected the studies. Those who were finally selected were reviewed by a third expert evaluator. Finally seven studies were identified about Internet-based PP programs for adolescents and youth. Consultation with international experts in the area identified one additional study (58), and one unpublished doctoral thesis (59) (**Figure 1**).

## RESULTS

The literature review identified seven technological systems that include PPIs to promote well-being in adolescent and youth populations, which efficacy was tested in nine studies. All studies used a website to deliver the PPIs, one study also used social network (Facebook) and email (60), and another one also used text messages reminders (61). Regarding the efficacy of the interventions, in general all studies showed positive results, decreasing anxiety and depression scores, and increasing well-being.

A brief description of the systems and a summary of these studies are presented in **Table 1**.

#### DISCUSSION

Only seven ICT-based PPI for promoting health and well-being in adolescent and youth populations were identified in the literature review, and only nine studies testing their efficacy. Therefore, our first conclusion is that more controlled studies are needed. Furthermore, no longitudinal studies were identified, and these ICT-based prevention interventions should also prove their effectiveness through longitudinal studies that can clearly demonstrate whether the protective factors (virtues, strengths, etc.) are maintained in adulthood. Only one intervention (Mother–Daughter Prevention Program) reported data from a 2-year follow-up (64).

Another conclusion has to do with the design of the ICT-based PPIs. They should be designed considering the specific features of the target users. Both the contents (language, instructions, images, etc.) and other format and design elements are crucial in engaging teens. For example, InJoy was modified to improve its effectiveness and enhance its ability to engage. Compared to the initial version, the recent one was rated higher by students on the targeted social learning domains and found to be significantly

more engaging (helpful, interesting, and fun) (58). Another example of adapting the systems to young people's characteristics is the use of Internet social networks. E-health4Uth uses social networks (Facebook). The use of these resources could increase the attractiveness of these programs for young people and their adherence to them (47).

This review also identifies the importance of considering the contexts where the interventions will be delivered. For example, Bite Black was not more effective than the control task when used in schools, perhaps because teens perceived the application as an additional "school task" (63). This finding suggests the need to explore which technological systems and which PPIs best fit the different settings (school, family, leisure, etc.). Schools offer the possibility of reaching a large number of users, but the adaptation of interventions to increase their effectiveness should be considered.

Another relevant issue is the role of different technologies to deliver PPIs. Technological advances, especially related to smartphones, sensors, and virtual/augmented reality in daily life, could be very useful in promoting well-being. In our revision, it draws attention the limited number of ICTs used. It is important to explore this field because young people use ICTs every day through several mobile devices simultaneously (47) and this trend is expected that increases in the coming years. Currently, smartphones are main channels of communication, socialization, and entertainment for adolescents, and therefore to use them as devices to deliver PPIs could be appropriate. In this review, no system uses mobile phone resources to carry out the interventions and nurture interpersonal relationships. However, since smartphones or similars were not considered as keywords it is possible that some systems were not detected. This is a limitation in our search that should be considered.

This paper focuses on the benefits of using ICT-based PPIs to promote wellbeing in adolescents and youth. However, it should be highlighted that these interventions could also have some negative effects and potential dangers, if misused. It is necessary

#### Table 1 | Summary of findings.


(*Continued*)

#### table 1 | Continued


(*Continued*) Online Positive Interventions for Adolescents

#### table 1 | Continued


to be cautious when designing a prevention program, as people with more severe mental disorders might consider these programs to be substitutes to psychological treatments. In addition, other important aspects should also be taken into account in the e-mental health field (67), including confidentiality, privacy, and rigor of contents. Further work needs to consider all these issues, because despite limitations and potential dangers, ICT-based PPIs offer new mental health opportunities, especially for adolescents, and important benefits in terms of sustainability and accessibility. This review shows that some controlled studies have already been conducted, but much more research is required to establish the efficacy of these preventive interventions, including long-term follow-up studies. Successful prevention of mental disorders and mental health promotion are priorities worldwide, and ICTs can be a great ally, especially among young people.

#### REFERENCES


## AUTHOR CONTRIBUTIONS

All authors (RB, EK, AM, GR, AG, and CB) jointly contributed to write the manuscript and all of them approved the final version for submission. Literature search has been done by EK and AM. Studies selected were reviewed by a third expert evaluator (RB). All authors contribute to the discussion of the literature review.

#### FUNDING

This study was funded in part by the Excellence in Research Program PROMETEO II (Generalitat Valenciana, Conselleria de Educación, 2013/003) and CIBER of Physiopathology of Obesity Nutrition, an initiative of ISCIII (ISC III CB06 03/0052).

psychological interventions. *Int J Epidemiol* (2014) 43(2):318–29. doi:10.1093/ ije/dyt175


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Baños, Etchemendy, Mira, Riva, Gaggioli and Botella. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Perspectives on Games, computers, and Mental Health: Questions about Paradoxes, evidences, and challenges

#### *Martin Desseilles1,2\**

*1Department of Psychology, University of Namur Medical School, Namur, Belgium, 2Clinique psychiatrique des frères Alexiens, Henri-Chapelle, Belgium*

In the field of mental health, games and computerized games present questions about paradoxes, evidences, and challenges. This perspective article offers perspectives and personal opinion about these questions, evidences, and challenges with an objective of presenting several ideas and issues in this rapidly developing field. First, games raise some questions in the sense of the paradox between a game and an issue, as well as the paradox of using an amusing game to treat a serious pathology. Second, games also present evidence in the sense that they involve relationships with others, as well as learning, communication, language, emotional regulation, and hedonism. Third, games present challenges, such as the risk of abuse, the critical temporal period that may be limited to childhood, their important influence on sociocognitive learning and the establishment of social norms, and the risk of misuse of games.

#### *Edited by:*

*Yasser Khazaal, Geneva University Hospitals, Switzerland*

#### *Reviewed by:*

*Eric Brunet-Gouet, Centre Hospitalier de Versailles, France Andrea Gaggioli, Università Cattolica del Sacro Cuore, Italy*

*\*Correspondence: Martin Desseilles martin.desseilles@unamur.be*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 14 February 2016 Accepted: 22 June 2016 Published: 04 July 2016*

#### *Citation:*

*Desseilles M (2016) Perspectives on Games, Computers, and Mental Health: Questions about Paradoxes, Evidences, and Challenges. Front. Psychiatry 7:122. doi: 10.3389/fpsyt.2016.00122*

Keywords: games, computers, mental health, playing, learning, serious game, emotional regulation, critical period

#### INTRODUCTION

In the field of mental health, the electronic revolution, through the use of computers and the Internet, questions the concept of games. While the game is part of the psychological development of every individual, the electronic context leads to a redefinition of its use in mental health. This redefinition involves the identification of paradoxes regarding the use of games to deal with complex and serious problems. The identification of potential mechanisms of action underlying the use of games in mental health is also important to better identify targets for therapeutic intervention. Finally, the challenges, including variable concept of games, depending on gamers (kid games or adult games), and the possible misuse of information that can be made on the grounds that it is a game, also seem important to question. We will develop several ideas, including place of games in mental health and the idea of games both in the development of the child's psyche and in the evaluation of adult relationships.

## PARADOXES

Questions arise from the paradox that playing a game should mean that there are no issues. Amusingly, in French, the word for "I" (*je*) is pronounced the same as the word for "game" (*jeu*), and the word for "issue" (*enjeu*) sounds very similar. When *issues* become too prominent, *I* am no longer playing a *game*, because there are risks, and risks come with consequences.

In addition, can we play with something as serious as mental health? A depression is no fun, and schizophrenia is even less fun. In this case, illness is certainly not a game, but games can be used Desseilles Games, Computers, and Mental Health

as tools for communicating and regulating emotions in mental illness. However, although humor is a useful approach in mental health (1), it should not be confused with irony or cynicism. Cyberpsychology research is sometimes too "detached" from the concrete reality of mental illnesses and should not discredit the concrete stigma that follows mental illnesses when proposing computer games in the treatment of such serious illnesses.

If our main focus in this perspective is computer games, our reflections encompass certain aspects of games involving people. As such, it may appear an absence of a delimitation of "game" concepts. A taxonomy and description of games and their use in mental health has been proposed by recent researchers such as Miller (2) and would be far beyond the scope of our perspectives. Far from wanting to present the specific needs and applications of games in mental health, we wish to question some issues that this area can raise.

#### EVIDENCE

First, it is widely held that games are central to relationships between individuals and the "other," to communication, and to the notion of ex-istence itself (to *ex-sist:* from Latin *ex-sisterer*, to "stand apart"). From Latin, *Ludus* (game in action) replaced *jocus* (game in speech) and absorbed its value (3). By doing so, we tend to forget that play and games are made of language and communication. Through games (for example, "peek-a-boo," where the other is first hidden and then revealed, or the "bobbin" toy, which is set loose and then wound back in, or the game of "*fort/ da*," described by Freud as he watched his grandson make objects "disappear"), children experience disorientating surprises when something or someone becomes absent, followed by marvelous delight when the once absent other suddenly returns. Thus, games allow children to discover the permanence of connections with the other, or the continuity of the other as an object. The prominent psychotherapist Mélanie Klein and others later developed theories of the application of games and play to child psychanalysis (4). Importantly, in his book *Playing and Reality*, Winnicott states that "Playing is an experience, always a creative experience, and it is an experience in the space-time continuum, a basic form of living" [Ref. (5), p. 54]. Winnicott beautifully expresses the importance of play as an area that allows the interweaving movement between inner reality and the outside. Freud also stated that "The opposite of play is not seriousness but rather reality," recognizing the seriousness of children's play, creating an alternative to reality.

Second, games and play facilitate learning [e.g., Ref. (6), and also https://news.stanford.edu/news/2013/march/games-educationtool-030113.html] that is progressive, through trial and error, and without excessive guilt, because "it's only a game." When one plays a game, there is a desire to improve and make progress, there is no judgment, and the rules allow a group of players to assume delineated roles. The abovementioned homophony of the French words for "game" (*jeu*) and "I" (*je*) along with the closely resembling verb "play" (*joue*) reminds us that the "I" (or self) is at the center of the game: "*I play*" (*je joue*), therefore I learn.

Third, computers enable people to play games, among other things, but more to the point, they allow the environment to be redesigned and transformed. For example, the tempo of the presented stimuli can be changed to positively bias certain behaviors or images in order to correct biases of thought and memory [e.g., Ref. (7)]. If games are a primary mode of communication that existed before computers, the computer enables the addition of a network dimension (so that groups that are dispersed around the globe can play together), continuity (the game can be paused and picked up again later), singularity (individuals can play alone on a computer), diversity (a vast range of games are possible), precision (a variety of parameters can be measured, ranging from behaviors to players' personalities), and so forth. The computer is therefore a tool that allows people to communicate with others (in real time or not) or with sets of users in a network (in real time or not). An interesting development of network gaming is crowdsourcing games that bridge the limits between games and issues, since they use the playing motivation and intelligence of people to solve real important scientific problems (8).

Fourth, the game is a *primary and primordial* form of communication among individuals, because even the youngest humans, babies, can interact with adults through play. They can not only play games like "catch me" and "hide and seek" but also engage in forms of informal play like tickling, attempting to do something again and again, trying out new words, and trying to achieve something. By learning how to verbalize, by using language and playing word games, babies become children (9). Games facilitate learning because they contain the critical elements of *repetition* as well as *surprise*. Thus, even when the players know a game very well, they can still be surprised. This feeling of surprise occurs during moments of full awareness and cognitive reevaluation, so that things can be seen in a new light, with or without interpretation. This helps children establish the bases for *emotional regulation* and fosters the acquisition of *emotional competencies* (10). Hence, the game is above all a means, or mode, of communication, which in metonymic terms means the object that enables communication. Adapting the classic model of Gross (11), we may propose that games involve (1) a choice or an intention to place oneself in a certain situation; (2) a direction of one's attention or attentional deployment (what I am doing, what the other is doing); (3) cognitive reevaluation when questions arise or when the game changes as other players make progress; and (4) modulation of emotional responses, along with learning about the reactions of others when they are sad or happy for you or for themselves. As the game develops, these various factors are deployed in turn, and all of them contribute to mental health and well-being. Specifically, an European study has elegantly built a platform offering three mini-games that impact different components of emotion regulation as a complementary therapy tool in people with an impulsive disorder (12).

Fifth, further to the idea of emotional regulation, a hypothesis on the function of emotional regulation in dreams was proposed by Revonsuo, who suggests that dreams are a kind of *safe laboratory* where we can learn how to react appropriately to frightening stimulations (Threat Simulation Theory) (13). In games, as in dreams, even though appearances may closely resemble reality (and the computer enables pushing that resemblance much farther than ever before), the fact remains that there are no consequences. At least, this should be the case, or else it's no longer a "game." The game is therefore a safe laboratory for guessing what the other is thinking, to interact with the other, and so on. In this sense, the game is symbolic, and it fosters both *representation and mentalization*. Referring back to Freud, it facilitates a thicker preconscious. Beyond the metacognitive impact of game, concrete behavioral adaptation might occur since it is observed in animal games, which are without negative consequences and moreover deal with important aspect of their social and survival behaviors [e.g., Ref. (14)].

Sixth, a hedonistic approach [e.g., Ref. (15)] is beneficial for mental health insofar as it supports several temporally defined dimensions (past, present, and future) of memory retrieval, pleasure, and anticipation of shared times, without involving risk. Therefore, when applied to games, a hedonistic time perspective would also be beneficial for mental health. We stress this approach that as to be distinguished from pure rewarding approach, in which television serial or games with multiple level are especially experts, and that could induce more addictive stress than euphoric, serene and peaceful pleasure.

## CHALLENGES

First, although video games can contribute to the well-being (16), spending long hours playing has been associated with a number of *harmful consequences* [e.g., Ref. (17)]: social retreat and withdrawal ("The only thing I do is play"; "I don't care about anything else"), dependence ("I can't stop playing"), depression ("I play all the time"; "That's all I do, because nothing else interests me"), nervousness or anxiety ("I want to be able to do it"; "Can I do it?"; "I don't know how to do anything else"), disrupted sleep patterns ("The computer screen keeps me awake"; "I'm chronically sleep-deprived"), and so on. But this is not new. Long before the computer, there were card-playing fanatics. However, the permanent availability (24/7) and the option of playing alone on a machine are certainly factors that distinguish the games of today from those of yesteryear, and that have turned gaming into a formidable challenge.

Second, for children, almost anything can be a "game," because unlike adults, children are largely unaware of the issues associated with objects and events. Children can choose and ask for the toys that they want, and they willingly follow the rules that are imposed. And even if they ask "why" from time to time, the fact remains that games hardly ever inhibit their learning, in the non-pathological sense. At this developmental stage, disinhibition is not even a relevant term, because inhibition is only in the process of being established. However, compared with children, it is usually more difficult to persuade adults to play together. The inhibition that adults have established makes it harder for them to learn in this manner. Repeating a new word over and over is a game for a child, and possibly, a game for the adult who teaches it, but for most adults, word repetitions would be regarded more as performance than play. There appears to be a *critical temporal window* (i.e., childhood) during which play enables learning, whereas adults play less often, in the sense of child's play. When adults reappropriate play in the form of serious gaming or educational games (gamification), they tend to be ambivalent about using a childish means for adult learning. In the expression, "Let's stop playing and get serious," the two terms "playing" and "serious" are opposed. Nevertheless, these terms are not necessarily antithetical, because games can be powerful learning tools. They use a multimodal approach to knowledge that incorporates combined sensoralities, emotions, and rational thinking. And, there is practically no inhibition involved, because the risks of the game are not the same as the risks that are incurred in serious work. Therefore, the conceptualization of certain developmental stages as critical stages that can be reactivated later on has implications for the learning process and for the conceptualization of mental health [e.g., Ref. (18)]. In addition, we have seen previously that "the opposite of play is not seriousness but rather reality" as Freud stated and further theorized by Klein and Winnicott. Thus, the oxymoron-like semantic construction "serious gaming" is questionable in more than one aspect. In such manner, the distinction drawn between games as tool for mental health and games as a potential risk for mental health might be considered cautiously since the concept of serious games is often misused or even abused. Nevertheless, the potential of serious games as mental health treatment has been very elegantly reviewed by Miller (2). Miller proposes a taxonomy that allows considering the many ways to include games in mental health care (2). We encourage the reader to refer to this review, since the aim of our perspective is more to question the notion of seriousness and reality, beyond the usefulness of recent and promising approaches using games for mental health.

Third, sociocognitive theories of learning by *imitation and identification*, first proposed by Bandura (19), contend that we identify with games, images, behaviors, and words. It is not that we see a model and then become that model. Instead, we learn about norms, standards, and status, which we then internalize, and which become important guidelines for distinguishing between the normal and the abnormal. Playing also involves thinking about "identities" and what we consider "normal" or "abnormal." In this line of concern, there is a very important educational and social challenge to have games that reflect social and democratic values as well as pacific and respectful interactions, to cite only a few important components.

Fourth, at times, games involving people can be misused, in that they become overly *professionalized*. This has happened in music, tennis, football, and many other fields. Adults strive to coach their children, to push the game into the realm of professionalism. This can create discrepancies between the language of the child (play with no issues) and the language of the adult (issues with no play), if we may borrow the words of Ferenczi (20) and his "confusion of language." When and how this transition occurs are some of the questions that we might raise here.

Fifth, in a further misuse of the playful spirit, games have also been repurposed as team-building exercises, whereby adults are brought together to create and nurture bonds. However, we know very well that these "games" are designed for observing the interactions that occur among the group, with the aim of improving these interactions and ultimately improving productivity. These games involve calculated monitoring by adult observers of other adults, who are aware that they are being observed, and who know that these observations may subsequently be used in their favor or disfavor. These are games "with issues," largely used to investigate or generate group dynamics and their effects. This raises questions about the uses of imposed, monitored, and calculated games. Even if these uses of games involving people are accepted ways to work on professional bonds and even if people are aware of the consequences, it seems to us that spontaneity and unintentional curiosity seem to be far behind the scene of children's gaming and constitute some specific characteristics of adult's games.

#### CONCLUSION

In conclusion, we wish to draw readers' attention to the paradoxes, the issues, and challenges of the use of games in mental health, among others, in the context of the development of

#### REFERENCES


computers and the Internet. At the same time, important for the mental development of everyone, the games are a wonderful communication and emotion regulation tool, but must be framed in a critical way in order not to misuse it.

## AUTHOR CONTRIBUTION

The author confirms being the sole contributor of this work and approved it for publication.

#### ACKNOWLEDGMENTS

We thank both reviewers for their contribution through review process.


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Desseilles. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Serious Games and Gamification for Mental Health: Current Status and Promising Directions

*Theresa M. Fleming1,2\*, Lynda Bavin1 , Karolina Stasiak1 , Eve Hermansson-Webb1 , Sally N. Merry1 , Colleen Cheek3 , Mathijs Lucassen1,4, Ho Ming Lau5 , Britta Pollmuller6 and Sarah Hetrick7*

*1Department of Psychological Medicine, University of Auckland, Auckland, New Zealand, 2Department of Paediatrics: Child and Youth Health, University of Auckland, Auckland, New Zealand, 3University of Tasmania Rural Clinical School, Burnie, TAS, Australia, 4Department of Health, Wellbeing and Social Care, The Open University, Milton Keynes, UK, 5Department of Psychiatry, VU University Medical Center, Amsterdam, Netherlands, 6School of Art and Design, Auckland University of Technology, Auckland, New Zealand, 7Orygen Youth Health Research Centre, Centre for Youth Mental Health, University of Melbourne, Parkville, VIC, Australia*

Computer games are ubiquitous and can be utilized for serious purposes such as health

#### *Edited by:*

*Fotios C. Papadopoulos, Uppsala University, Sweden*

#### *Reviewed by:*

*Craig Speelman, Edith Cowan University, Australia Anna Sort, University of Barcelona, Spain*

> *\*Correspondence: Theresa M. Fleming t.fleming@auckland.ac.nz*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 21 September 2016 Accepted: 28 December 2016 Published: 10 January 2017*

#### *Citation:*

*Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, Lucassen M, Lau HM, Pollmuller B and Hetrick S (2017) Serious Games and Gamification for Mental Health: Current Status and Promising Directions. Front. Psychiatry 7:215. doi: 10.3389/fpsyt.2016.00215*

and education. "Applied games" including serious games (in brief, computerized games for serious purposes) and gamification (gaming elements used outside of games) have the potential to increase the impact of mental health internet interventions *via* three processes. First, by extending the reach of online programs to those who might not otherwise use them. Second, by improving engagement through both game-based and "serious" motivational dynamics. Third, by utilizing varied mechanisms for change, including therapeutic processes and gaming features. In this scoping review, we aim to advance the field by exploring the potential and opportunities available in this area. We review engagement factors which may be exploited and demonstrate that there is promising evidence of effectiveness for serious games for depression from contemporary systematic reviews. We illustrate six major categories of tested applied games for mental health (exergames, virtual reality, cognitive behavior therapy-based games, entertainment games, biofeedback, and cognitive training games) and demonstrate that it is feasible to translate traditional evidence-based interventions into computer gaming formats and to exploit features of computer games for therapeutic change. Applied games have considerable potential for increasing the impact of online interventions for mental health. However, there are few independent trials, and direct comparisons of game-based and non-game-based interventions are lacking. Further research, faster iterations, rapid testing, non-traditional collaborations, and user-centered approaches are needed to respond to diverse user needs and preferences in rapidly changing environments.

Keywords: serious gaming, games for health, gamification, computerized CBT, e-therapy, engagement

## INTRODUCTION

Computer games are played by millions of adolescents (1) and adults (2) around the world, with over 40% of the United States population playing computer games for 3 or more hours per week in 2015 (2). Computer games vary enormously along dimensions such goals, interaction, and involved technologies. They include fast mini-games, as simple as lining up dots in a row, through to augmented reality (AR), and intricate shared worlds. Quality computer games have been shown to enhance concentration (3), improve retention of information (4), facilitate deep learning (5), and bring about behavior change (6). Over recent decades, computerized game-based approaches, both "serious games" and "gamification" have been developed for "serious" purposes: to educate, motivate, and/or persuade users, in educational, health, and other settings (7, 8). "Serious games" and "gamification" can be defined variously. However, both seek to employ games (or substantial game elements) in an effort to educate and change patterns of experience and/or behavior. Serious games utilize gaming as a central and primary medium (9). In contrast, gamification refers to the addition of game elements to non-game contexts (10). A gamified intervention may not operate as a full game experience but contains gaming elements, such as the scoring of points, in-game rewards, or engaging in quests.

Game-based approaches for mental health are in their infancy. However, initial studies, mainly of serious games, suggest potential benefits for psychological and behavioral changes, or symptom relief (11–17). Alongside these scientific developments, there has been significant growth in smartphone apps for mental health (18–20). Some of these use gaming or gamification, but most have not been scientifically tested (18).

To date, the potential of serious games and gamification (together "applied games" or "applied gaming") in mental health has been understudied. Where applied games have been researched, interventions are often poorly described, and diverse approaches are treated as homogeneous. In this perspective paper, we aim to advance the field by highlighting the scope of applied games for mental health. We consider the potential for applied gaming and motivational features that may be utilized. We examine evidence of effectiveness from systematic reviews and demonstrate major types of tested applied games for mental health. Finally, we highlight promising directions for development.

#### THE POTENTIAL OF APPLIED GAMES

Applied games are intriguing for mental health for three reasons:

 First, applied gaming offers "*appealing potential*," as suggested by the popularity of computer games (21). Applied gaming approaches might increase the reach of mental health interventions to some who might not otherwise access help. This is important given the large numbers of people who experience mental distress and yet receive no treatment (the mental health treatment gap) (22).

 Second, applied gaming has "*engaging potential*." Users might experience gaming approaches as enjoyable, want to "win" the game, or see how the story unfolds. Such dynamics may contribute to reducing high attrition rates in naturalistically implemented internet-based interventions (23, 24).

 Third, applied gaming has "*effectiveness potential*," because it provides opportunities for both conventional and nontraditional processes for behavior change and learning. For instance, applied gaming can offer immersive experiences where a state of "flow" can be achieved, provide rich sensory environments to support learning, allow behavioral modeling and social learning, allow users to try new skills in a safe yet reactive environment, and facilitate repeated rehearsal of new behavior (9, 25–27).

## ENGAGEMENT

Games exploit varied processes for engagement. Hamari and Tuunanen (28) carried out a meta-synthesis of 12 studies and identified key motivational orientations that support engagement. These were achievement, exploration, sociability, domination, and immersion (for examples, see **Table 2**). A further motivation of escape (where a user plays to escape real-life problems) has also been reported (29). A user may have several or all of these motivations for playing a game, and predominant motivations may vary across demographic groups, contexts, and types of game (28). This model could be extended to reflect the proposition that users of applied games might also have a further dimension along which their motivation varies, that of the serious purpose itself, in the present case, interest in improving mental health. For users who are motivated in this way, it may be that just a few gaming elements might enhance engagement or even that gaming elements are off-putting (30). For those less motivated to improve their mental health, stronger and more extensive gaming features may be critical.

#### EXISTING RESEARCH: A SCOPING REVIEW

In order to provide an overview of evidence for applied games, we carried out a review of systematic reviews. Our inclusion criteria were systematic review of serious games and/or gamified interventions for mental health (treatment or prevention), published in the peer-reviewed literature from 2010 to June 2016 (reflecting the recent development of the field), and available in English. We searched PsycINFO and Medline using the terms (systematic review) AND (mental health OR mental illness OR depression OR anxiety) AND (treatment OR prevention) AND (computer OR internet OR digital OR online) AND (game OR gaming OR gamification OR play). After duplicates were removed, a total of 18 papers were identified. A Google search and check of citations yielded no further papers for inclusion. Titles and abstracts were independently scanned by two authors (TF and LB). From these, three systematic reviews, two of which also included metaanalyses, fitted the inclusion criteria (see **Table 1**).

Each of the included papers was focused specifically on depression. Two reviews (9, 32) examined serious games for depression, while Li et al. (31) focused specifically on a subset of serious games, exergames. Fleming et al. (9) identified nine studies of six computerized interventions that utilized gaming as a major or primary component to treat or prevent depression. Li et al. (32) identified a higher number of studies, although exactly how many studies of how many interventions was not clearly reported and the search terms and inclusion criteria were not specified. As shown in **Table 1**, both Li et al. (32) and Fleming et al. (9) conclude that the utilization of serious games for depression is promising, but that further research is needed. Given the heterogeneity of included studies and the nature of many of these being small trials, some not randomized or controlled, stronger

#### Table 1 | Systematic reviews of applied games for mental health.


conclusions would be premature. In the more narrowly focused review (31), only three Randomised Controlled Trials (RCTs) were identified; however, the authors reported a significant effect of exergames on depressive symptoms. As noted in each review, there was a lack of direct comparison of game-based to nongaming interventions, and most studies were not independent of the developers. The research is at an early stage.

#### TYPES OF APPLIED GAMES

The three systematic reviews included six main types of applied games. Each of these categories is outlined below. An example of each, and potential mechanisms for therapeutic change and engagement, is given in **Table 2**.

#### Exergames

Exergames are sport or movement-based games. Nine of the included studies (across the three reviews) tested the use of exergames for depressive symptoms, primarily among older adults (31). Eight studies used "repurposed games," games developed for entertainment or commercial purposes and now tested for mental health. One was purpose-built. Li et al. (31) reported a significant effect of exergames on depressive symptoms, with this being higher among more playful games, over those that included less game elements. However, these results should be interpreted with caution, given that only three of the studies were RCTs, and the sample sizes were small.

#### Virtual Reality Games

Virtual Reality (VR) and Augmented Reality (AR) can offer immersive interactivity in a virtual or augmented world, with visual, audio, and sometimes other sensory stimuli, to increase user engagement and possibly therapeutic impact (38, 39). Li et al. (32) identified six studies of VR gaming interventions. They reported that these had positive results. However, only 2 had over 10 participants, and most of the original papers provide

#### Table 2 | Examples of major types of tested applied games for mental health.


little detail regarding the game. Hence an alternative example, that of Virtual Iraq for PTSD (34) is given in **Table 2**. Promising findings from VR interventions, including non-game-based VR interventions (40) along with the popularity of commercial AR games, suggest promise in this area.

#### Cognitive Behavior Therapy (CBT)-Based Serious Games and Gamification

Five of the interventions identified in the systematic reviews were multilevel CBT-based programs, often utilizing a fantasy environment, and designed to be completed at a rate of approximately one level per week on a personal computer. Each of these programs was aimed at children or young people. Each reported positive or promising results, except for ReachOutCentral, which had mixed findings and has since been retired. Of these, SPARX is described in the greatest depth and is outlined in **Table 2**.

A further example, SuperBetter, has been tested since the publication of the systematic reviews; however, it is included here (**Table 2**), as it illustrates new opportunities for applied games. SuperBetter is a positive psychology program in which players earn points and "level-up," as they progress through activities. Rather than being a narrative-based serious game, SuperBetter offers a more gamified approach, with scoring and rewards. In another point of difference, SuperBetter allows "snacktivity," frequent, brief activities that can be done a few minutes at a time, every day or more often (therefore like "snacking" behavior). This pattern of use is common in contemporary online apps but is not common in tested online mental health tools to date, many of which follow traditional clinical therapeutic models (e.g., weekly sessions of 30 minutes or more). In a recent RCT, participants who were asked to play SuperBetter for 10 minutes daily over 30 days experienced significantly greater reductions in depressive symptoms and anxiety compared to a waitlist control group; however, attrition was high (35).

#### Entertainment Computer Games

This category of interventions is quite different from those that translate an evidence-based mental health therapy (such as CBT or exposure therapy) into a game format. In this grouping, entertainment video games were tested for effects on mood. Several studies of this nature were included in Li et al. (32). In the first of these, students were given a "frustrating task" and then 45 minutes of violent video game play or a control condition. Those playing the violent game reported lower symptoms of depression immediately following the intervention in one study (41). However, in a second study, no effect for depression was reported (42). In a further study, Rossoniello et al. (43) reported subjects had improved mood immediately following playing a casual video game Bejeweled II. These entertainment games were proposed to affect mood *via* emotional regulation, stress release, or social support pathways.

A different form of using commercial games for mental health is the use of the puzzle game Tetris for therapeutic purposes. In Tetris, players strategically move, rotate, and drop "Tertriminos" to complete horizontal lines. Engaging in this visuospatial cognitive activity when memories are activated is proposed to help impede traumatic flashbacks in PTSD by interfering with memory consolidation (37). Preliminary findings have also shown promise for using Tetris to reduce cravings (44).

#### Biofeedback-Based Games

The included reviews described two biofeedback-based games: the Journey to the Wild Divine and Freeze-Framer 2.0. In each of these, users rehearse relaxation skills while receiving visual feedback on physiological indicators (measured using a sensor attached to the ear lobe or the fingertips). In a small trial, youth receiving the intervention had significantly lower post-intervention levels of depression and anxiety compared to those in a waitlist control group (36).

#### Cognitive Training Games

The reviews included one study using cognitive training games. Alvarez et al. (45) tested number and letter sequence training games to reduce cognitive impairment in 31 depressed students. In this study, the game had positive results on cognitive impairment, but direct effects on mood were not tested.

#### FUTURE DIRECTIONS

It is demonstrably feasible to translate traditional evidence-based interventions, such as CBT and exposure therapies, to computer gaming formats. Included interventions have shown that it is also possible to exploit features of computer games for therapeutic change using mechanisms that are not traditionally salient in psychological therapies, such as in the example of Tetris for PTSD. Further, the potential for positive mental health outcomes from casual play of entertainment games is worthy of exploration, as this might offer opportunities to reach large numbers of people. Each of these approaches appears to be promising. However, this evidence is at an early stage, and independent, larger robust studies are needed. Further, there is a lack of data regarding whether gaming-based approaches might be more appealing than non-gaming mental health interventions for users with different motivations, including both those who do and those who do not want to access help for distress. This is a question for future research. Similarly, findings from trials and user reviews of some commercially available programs suggest that applied gaming approaches can be engaging; however, many analyses do not report engagement or ongoing use. The assertion that quality gaming dynamics will increase engagement, at least for some users, should also be tested in future research.

We have highlighted that applied gaming interventions vary widely, in terms of types of games and in terms of features that might be appealing and motivating. It would be valuable to explore popular engaging game types for target groups and compare features in those games with those used in games for mental health. For example, highly accessed games currently include smart phone-based mini-games, massive multiplayer games where millions of players interact, games that allow usergenerated content, and games that are linked to popular social media platforms (2, 46, 47). These approaches did not feature strongly in the included interventions.

Despite the potential for applied gaming, there are challenges in proposing such approaches, including costs, speed of implementation, issues of face validity, and user preferences. Many gamers will be familiar with commercially produced games, which often involve development budgets in the tens of millions of dollars (48, 49). Even simple mini-games undergo rapid advances, with new versions regularly released. Funding limitations make it challenging to develop and maintain comparable scientific initiatives.

Speed of implementation is also critical (49). Traditional scientific methods of sequential development, piloting, refinement, testing in a RCT, followed by publication, and independent replication, prior to real-world implementation, will ensure that evidence-based applied gaming lags behind the rapidly shifting commercial hardware and software environments. Newer methodological approaches are available (24).

Rather different challenges include those of face validity and diverse user motivations. People often do not seek help for mental health issues until these are relatively serious (22), in which case a game may be perceived as trivializing or inappropriate (Fleming et al., manuscript in preparation). Conversely, those who do not want therapeutic help might be irked that their game has a mental health agenda as suggested by the gamergate phenomena (50). These factors should be investigated in future research. The diversity of gaming approaches and user motivations also pose challenges. Not all games are successful. Quality game development requires specialist skills, and meeting user preferences may necessitate the creation of a range of interventions.

We have previously proposed four key ways of maximizing the impact of E-therapies and serious games in mental health. Utilizing this framework, as identified by the Collaboration on Maximizing the impact of E-therapy and Serious Gaming (COMETS) (24), we believe that to achieve significant mental health benefits, the field requires:


#### REFERENCES


uptake are diverse and go beyond many science or clinically focused teams. Further, the costs of developing interventions may be more easily borne across sectors and jurisdictions.

– Rapid testing and implementation. User expectations in technology-driven approaches and gaming evolve rapidly. Innovative, rapid research designs and planning for implementation are needed to ensure that interventions are still appealing when they are ready for implementation.

#### CONCLUSION

In this perspective paper, we have illustrated the potential of serious games and gamification for mental health and highlighted that there is serious work still to be done. The field is ready for further development, as the feasibility and range of possible approaches has been shown, and as there is an urgent need for engaging, appealing effective mental health interventions which reach large numbers of people. Future research should include independent trials and direct comparisons of game-based and non-game-based options for varied user groups.

#### AUTHOR CONTRIBUTIONS

TF drafted the paper, coordinated the input from other authors, and was responsible for the full submission. LB assisted with drafting and completing all aspects of the paper. All other authors contributed substantially to the draft and approved final submission.

#### FUNDING

No specific funds were received for the preparation of this paper. All authors are staff or PhD candidates, and the manuscript was prepared as part of these roles.


on subjectively and objectively measured physical activity. *Front Psychiatry* (2015) 6:180. doi:10.3389/fpsyt.2015.00180


**Conflict of Interest Statement:** The intellectual property for SPARX is held by UniServices at the University of Auckland. Any proceeds from licensing or selling SPARX outside of New Zealand will be shared in part with UniServices and SM, KS, TF, M. Shepherd, and ML.

*Copyright © 2017 Fleming, Bavin, Stasiak, Hermansson-Webb, Merry, Cheek, Lucassen, Lau, Pollmuller and Hetrick. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Serious Games for Mental Health: Are They Accessible, Feasible, and Effective? A Systematic Review and Meta-analysis

*Ho Ming Lau1 \*, Johannes H. Smit1 , Theresa M. Fleming2 and Heleen Riper1,3*

*1Department of Psychiatry, The EMGO Institute for Health and Care Research, VU University Medical Center Amsterdam, GGZ inGeest, Amsterdam, Netherlands, 2Department of Psychological Medicine, University of Auckland, Auckland, New Zealand, 3Department of Clinical, Neuro and Developmental Psychology, Section of Clinical Psychology, VU University Amsterdam, Amsterdam, Netherlands*

Introduction: The development and use of serious games for mental health disorders are on the rise. Yet, little is known about the impact of these games on clinical mental health symptoms. We conducted a systematic review and meta-analysis of randomized controlled trials that evaluated the effectiveness of serious games on symptoms of mental disorder.

#### *Edited by:*

*Stéphane Bouchard, Université du Québec en Outaouais, Canada*

#### *Reviewed by:*

*Martin Desseilles, University of Namur, Belgium Erika Comasco, Uppsala University, Sweden*

> *\*Correspondence: Ho Ming Lau hm@lau.im*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 17 August 2016 Accepted: 21 December 2016 Published: 18 January 2017*

#### *Citation:*

*Lau HM, Smit JH, Fleming TM and Riper H (2017) Serious Games for Mental Health: Are They Accessible, Feasible, and Effective? A Systematic Review and Meta-analysis. Front. Psychiatry 7:209. doi: 10.3389/fpsyt.2016.00209*

Method: We conducted a systematic search in the PubMed, PsycINFO, and Embase databases, using mental health and serious games-related keywords. Ten studies met the inclusion criteria and were included in the review, and nine studies were included in the meta-analysis.

Results: All of the serious games were provided *via* personal computer, mostly on CD-ROM without the need for an internet connection. The studies targeted age groups ranging from 7 to 80 years old. The serious games focused on symptoms of depression (*n* = 2), post-traumatic stress disorder (*n* = 2), autism spectrum disorder (*n* = 2), attention deficit hyperactivity disorder (*n* = 1), cognitive functioning (*n* = 2), and alcohol use disorder (*n* = 1). The studies used goal-oriented (*n* = 4) and cognitive training games (*n* = 6). A total of 674 participants were included in the meta-analysis (380 in experimental and 294 in control groups). A meta-analysis of 9 studies comprising 10 comparisons, using a random effects model, showed a moderate effect on improvement of symptoms [*g* = 0.55 (95% confidence interval 0.28–0.83); *P* < 0.001], favoring serious games over no intervention controls.

Discussion/conclusion: Though the number of comparisons in the meta-analysis was small, these findings suggest that serious gaming interventions may be effective for reducing disorder-related symptoms. More studies are needed in order to attain deeper knowledge of the efficacy for specific mental disorders and the longer term effects of this new type of treatment for mental disorders.

Keywords: serious games, gamification, game-based intervention, depression, anxiety, post-traumatic stress disorder, alcohol, attention

#### INTRODUCTION

Serious games are "games that do not have entertainment, enjoyment or fun as their primary purpose" (1). Primary purposes of serious games can be, but are not limited to, education, training, human resource management, and health improvement (2). The term "serious games" was introduced more than a decade ago (3). Since then, the development and use of serious games have grown (4). A Google search for the term "serious games" shows approximately 3.4 million entries in 2016 (search conducted by us on August 8, 2016), compared to some 1.1 million entries that was found using the same search string in 2007 (5). The definition of serious games is, however, still evolving. Various definitions of serious games can be found in the literature (1, 3, 6–9). Some definitions focus on the technological aspect of the games (3, 8), while others focus more on training (7) or educational purposes (6). In the current study, we used the following definition of serious games: games that are designed to educate, train, or change behavior as they entertain players (10). Serious games can be non-digital (11); however, most serious games in the peer-reviewed literature are delivered online or *via* stand-alone computer technology.

Serious games have found their way into health care (12, 13) as shown by the increase in releases in this sector from 4.7% in 2002 to 8.2% in 2011 of the total serious games market (4). For example, the serious game *Re-Mission* was developed in order to actively involve young people with cancer in their own treatment by educating them on cancer and its treatment (14). In recent years, the potential use of serious games in mental health care has also been explored. For example, a web-based social network electronic game designed to enhance mental health literacy in young people was developed and evaluated in a pre- and posttest design. This gaming approach was found to be effective for this purpose (*d* = 0.65) (15). Also, a review of the potential of using games to improve mental health professionals' knowledge as a teaching strategy found that those allocated to educational games performed considerably better on a mental health nursing test than those health professionals who were not (16). In addition, the deployment and design of serious games for psychotherapy has recently been studied (17, 18). In a review of literature on the use of video games in psychotherapy, Ceranoglu (17) concluded that games will be likely to be used in psychotherapy as therapists gain familiarity with gaming equipment. Further, games have been developed to treat impulse-related conditions such as eating disorders (19, 20). Initial results of an evaluation study showed that patients with eating disorders feel comfortable using a serious game in treatment (20).

Other forms of digital interventions for mental disorders already exist and have been studied more extensively. Internet and computerized interventions are found to be effective for the prevention and treatment of adult common mental disorders, such as depression, anxiety, and alcohol use disorders (AUDs) (21–24). There is also evidence, albeit limited, that computerized cognitive behavioral therapy (CBT) is effective for the treatment of anxiety and depressive symptoms among young people (25–27).

Digital serious games may enrich the array of digital interventions due to their specific characteristics such as the provision of an alternative world in which learning and exploration is encouraged (2, 28). Serious games may make learning more meaningful, engaging, and challenging than traditional teaching by using the interactive, visual, and immersive characteristics available in video games (29). They may help children and adults alike to develop simpler solutions and become more creative in solving problems (30). Looking at these characteristics and benefits of video games, it seems attractive that these are being explored for their potential use in the prevention and treatment of mental disorders (31). Whether it is using games to serve a serious purpose or gamify a serious purpose, the goal is to help individuals reduce mental health complaints or improve their mental wellbeing. But how do these mental health games perform on improving mental health? Are there evidence-based serious games for the treatment or prevention of mental health symptoms?

To date, few serious games have been tested and reported in the scientific literature. A pilot study was conducted to investigate the effectiveness of a brain–computer interface in the treatment of attention deficit hyperactivity disorder (ADHD) (32). The results showed improvement in inattentive symptoms and hyperactive– impulsive symptoms after playing on the attention training game system. Furthermore, a case study has been conducted using a serious game in the treatment of specific phobia (33), indicating that serious games are helpful in reducing fear and avoidance. Results of these randomized controlled trials (RCTs) show that serious games have the potential to be used as a whole or part of treatment for mental health disorders (34, 35). Recent reviews offer a broad view on serious games or game-based digital interventions within the mental health field. Li et al. (36) have focused on game-based digital interventions for depression, including serious games, but also simulations without game elements such as virtual reality exposure therapy (VRET). VRET is exposure therapy that makes use of virtual reality (VR) to simulate a realworld situation in order to treat a specific phobia. Support for the effectiveness of game-based interventions for depression was found. In another review of serious games and mental health conducted by Van der Krieke et al. (37), the scope also comprised simulations without game elements (38, 39), VRET (40, 41), and interactive computerized interventions (42). Games and simulations are, however, different conceptual entities (43). Simulations do not necessarily contain a competitive or conflict element, in contrast to games where users try to win or cope with certain problems bounded by rules. Although some video games are based on simulations, simulations without game elements were not considered as (serious) games for this study. VRET, simulations, and interactive computerized interventions do not necessarily contain elements that make a game *a game* and thus should not be categorized as serious games (43). Also, since the available reviews (36, 37) included studies that are not all RCTs, questions about effectiveness could not be answered optimally. Thus while a few reviews (36, 37) have been conducted on the potential impact of serious games for common mental disorders, none of these are robust. Therefore, a contemporary update is called for, given the rapidly evolving field; and an evaluation of trials to date by means of a meta-analytic review is still lacking.

The current study aims to systematically evaluate studies that have assessed the effectiveness of serious games in treatment outcomes for mental disorder-related symptoms by means of a systematic review including a meta-analysis.

#### MATERIALS AND METHODS

"The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)" statement (44) was used as guideline to conduct this study.

#### Search and Study Selection

A literature search of the PubMed, PsycINFO, and Embase databases was conducted. The search string was a combination of serious games-related terms, such as "serious games," "game-based," "videogames," "computer-assisted therapy," "virtual reality intervention," "gamification," "gaming simulation," and mental health-related terms, such as "mental health," "depression," "anxiety," "problem drinking," "schizophrenia," and "obsessive-compulsive disorder." Duplicate items were removed from the records that were identified through the literature search. The remaining items were screened on basis of title, abstract, and keywords by two independent raters (Ho Ming Lau and Ka Wai Ma). Items were included if the following inclusion criteria were met (a) the intervention used a digital game delivered on any technical platform including personal computers (PCs), consoles, cell phones, and handheld devices, which means that non-digital games were excluded; (b) the intervention targeted mental disorders such as those mentioned above; and (c) the study conducted an RCT. For the purpose of our study, simulations, VR interventions, and interactive programs without game elements were not considered serious games and were therefore excluded. The remaining records were assessed for eligibility by two independent raters (Ho Ming Lau and Jan Smit Jr.). Differences in ratings were resolved by discussion till a consensus was reached. If no consensus was reached, the coauthors of this paper were consulted to make the final decision.

#### Data Extraction and Synthesis

A data extraction sheet was developed and pretested on two studies. The variables that were extracted from the articles were divided into two categories: (1) participant and study characteristics and (2) game characteristics.

Participant and study characteristics included variables, such as target group, recruitment, treatment type (single- or multicomponent intervention), primary outcome measures, how and how much guidance during intervention was given, setting of intervention, study conditions, attrition, and results.

Game characteristics comprised variables, such as title of the game used in the study, serious game type, game genre, and purpose of the game.

#### Quality Assessment

The validity of the included studies was assessed using the Risk of Bias Assessment tool of the Cochrane Collaboration (45). We used the following six criteria: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessors, incomplete outcome data, and selective outcome reporting. Each type of risk bias was rated low, high, or unclear.

#### Meta-analysis

Analysis of the data was done using the program Comprehensive Meta-Analysis (CMA) (46). Cohen's *d* is an effect size that can be used to indicate the standardized difference between the two means divided by the pooled standard deviation at posttest. Hedges' *g*, a variation of Cohen's *d*, can be used to correct for potential bias as a result of small sample sizes. In this study, Hedges' *g* was applied due to a number of studies with small sample sizes. An effect size of 0.2 indicates a small effect, 0.5 indicates a moderate effect, and 0.8 to infinity indicates a large effect (47). For studies that did not provide means or standard deviations, statistics such as *F* value or *t* value were used to calculate the effect sizes according to formulae of CMA. It should be noted that one study conducted two experiments (48). Both experiments were included as separate comparisons in the meta-analysis. The random effects model was used to calculate the mean effect sizes, as we expected heterogeneity among the included studies. It assumes that the calculated effect sizes differ because of true variation in effect size from one study to the next and not only of the random error within studies. The *Q*-statistic was calculated to assess the presence versus absence of heterogeneity, but only significance was reported. The *I* 2 -statistic was also calculated to test the homogeneity of effect sizes. A value of 0% indicates no observed heterogeneity, while larger values indicate increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high (49). Furthermore, we estimated 95% confidence intervals (CIs) around *I* 2 (50) using the non-central chi square-based approach within the Deducer GUI (51) of R (52).

The numbers needed to treat (NNTs) were calculated, using the formulae provided by Kraemer and Kupfer (53). By calculating the NNT, we obtained an estimation of the number of patients who need to be treated in order to have one who would benefit. The lower the NNT, the more effective the treatment is.

In studies where multiple primary outcomes were used, the effect sizes were averaged to produce a single summary effect size for use in the meta-analysis (54). Data were extracted by two independent reviewers (Ho Ming Lau and Jan Smit Jr.). Differences in extractions were resolved by consulting the coauthors.

Furthermore, two subgroup analyses were conducted according to the random effects model comparing (1) younger (≤18) versus older (>18) participants and (2) clinical versus nonclinical participants. We conducted additional subgroup analyses for the studies on the disorders, such as autism spectrum disorder (ASD), cognitive functioning, and post-traumatic stress disorder (PTSD) separately.

We examined the funnel plots visually in order to detect possible publication bias. A funnel plot is a simple scatterplot of the intervention effect against a measure of each study's size. A symmetrical inverted appearance of the funnel plot indicates low publication bias, whereas an asymmetrical funnel plot indicates potential publication bias which may lead to an overestimation of the intervention effect (55). To verify an unbiased estimate of the pooled effect size, the Duval and Tweedie trim-and-fill analysis was performed (56). Moreover, Egger's linear regression method was applied on the intercept to quantify the possible publication bias captured by the funnel plot and its significance.

#### RESULTS

#### Study Selection

The PubMed, PsycINFO, and Embase search returned 4,130 items. After removal of duplicates, 3,693 records remained. Initial screening of title and abstract excluded a further 3,612 articles, leaving 81 items. After applying the exclusion criteria, 10 studies were included in the review and 9 in the meta-analysis. See **Figure 1** for a flow chart of the study selection.

#### Results of the Review

We start by reviewing some characteristics of the serious games at stake, including their game design features per disorder (**Table 1**). We then present the results of our meta-analysis on the impact of these games on psychiatric-related disorders.

#### Game Characteristics

An overview of the game characteristics can be found in **Table 2**. Three types of serious games in terms of design processes can be identified, namely, designed, purpose-shifted, and modified games (4). Designed serious games are games that are designed with a "serious" purpose from the beginning. Purpose-shifted serious games are games that were not designed as a serious game but are being used for a serious purpose. An example is the use of the game Tetris as part of an intervention to reduce the number flashbacks in PTSD research (48). Modified serious games are similar to purpose-shifted ones, but while purpose-shifted games are left intact, modified ones can differ from the original in terms

of gameplay and characters. An example is modifying the engine of the first-person shooter game "Unreal" into a firemen training program (57). All three types (designed, purpose-shifted, and modified) of serious games were of interest for the current study. However, we found no modified games in our study. Eight studies used serious games that were designed as such (34, 35, 58–63). Two studies used an existing entertainment game for a serious purpose (purpose-shifted) (48, 64).

We divided the variable game genre into goal-oriented, problem-solving, cognition training, and so-called exergames. Goal-oriented games focus on tasks and the end results of those tasks. Problem-solving games challenge players to find solutions for problems. Cognition training games train the players' working memories by a series of similar brief challenges that usually have to be tackled within time constraints. Exergames are games that combine physical exercises with game elements. No exergames were identified in the current study. Six studies used serious games in the cognition training genre (48, 58, 59, 61, 63, 65). Three studies used serious games that can be categorized in two genres, namely, goal-oriented and problem-solving (34, 35, 60). One study used a serious game that was goal-oriented only (62).

All the studies used serious games for training purposes. Three of these 10 studies also had psychoeducation as a purpose (34, 35, 60).

#### Depression

Two RCTs targeted depression, both with the same serious game SPARX (34, 60), aimed at adolescents (aged from 12 to 19 years). In both studies, this goal-oriented and problem-solving game was used in order to reduce depression-related symptoms. This game is based on CBT (66). The version of *SPARX* used in the studies can be played on a PC without the need for an internet connection (however, an online version is now available at https://www. sparx.org.nz/); it can be used free of charge by collaborators in New Zealand only currently. In this game, the player controls a personalized character who has to restore the balance in a fantasy world, for instance, by solving problems and shooting negative thoughts, all components of CBT. The player is guided by a virtual character who speaks about dealing with depression and gives instructions and objectives for the seven levels in the game. Each level (or module) has a duration of approximately 30 min. The game was played under minimal supervision from educational service providers in Fleming et al. (34). The Children's Depression Rating Scale Revised (67) was used as primary outcome measure for both studies. In Merry et al. (60), SPARX was played on primary health-care or school guidance center/health locations.

#### Post-traumatic Stress Disorder

Post-traumatic stress disorder was the focus in two serious game studies (48, 64). A study by Holmes et al. (48) was conducted in an attempt to deal with the limitations in an earlier study (64). *Tetris* was used in both studies in order to reduce PTSD-related symptoms. The theory of using *Tetris* is based on findings in cognitive science and neurobiology of memory. Flashbacks of traumatic events are assumed to consist of sensory-perceptual and visuospatial mental images. When visuospatial tasks are performed after a traumatic event and also within the time window

#### Table 1 | Characteristics of the randomized controlled trials that are included in the review.


(*Continued*)

Serious Games for Mental Health

#### table 1 | Continued


*ac, active control; ADHD, attention deficit hyperactivity disorder; ASD, autism spectrum disorder; AUQ, Alcohol Urge Questionnaire; BRIEF, Behavior Rating Inventory of Executive Function Questionnaire; CBTT, Corsi block tapping task; CDRS-R, Child Depression Rating Scale Revised; CG, control group; DBDRS, Disruptive Behavior Disorder Rating Scale; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, fourth edition; DSM-IV-TR, Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision; ERSSQ, Emotion Regulation and Social Skills Questionnaire; ESP, Educational Service Provider; Exp., experiment; fa, full-active condition; fu, follow-up; HSQ, The Home Situations Questionnaire; mth, month (after start intervention); n.i., no information; PedsQL, Pediatric Quality of Life Inventory (parent and child versions); OCDS, Obsessive Compulsive Drinking Scale; pa, partially active condition; pt, posttreatment; SG, serious games condition; SPSRQ-C, Sensitivity to Punishment and Sensitivity to Reward Questionnaire for children; SSQ, Social Skills Questionnaire (parent and teacher forms); TAU, treatment as usual; TMT, Trail Making Test; TSSE-RP, Task-Specific Self-Efficacy for Relapse Prevention Questionnaire; WISC-III, Wechsler Intelligence Scale for Children, third edition; wk, week (after start intervention); WL, waitlist.*


Table 2 | Game characteristics of the randomized controlled trials that are included in the review.

of memory consolidation, competition for the same resources will occur, causing interference with and reduction of flashbacks. *Tetris* (68) was purpose-shifted, since it was originally developed for entertainment. Nowadays, *Tetris* is available on different devices and platforms (e.g., mobile, tablet, and game consoles). In both studies, the game was played on a PC in the research lab at the university. No internet connection was needed to play *Tetris*. In this game, a random sequence of geometric shapes consisting of four square blocks each (Tetriminos) fall down the playing field. The player has to try to make horizontal lines without gaps with the Tetriminos. When a full line is created, this line of blocks will disappear and the blocks on top of the line will fall. When the blocks reach the top of the playing field and thus no new Tetriminos are able to enter, the game ends. Within both studies by Holmes et al. (48, 64), this game was played for 10 min with little instruction needed, after watching a film containing traumatic scenes.

#### Autism Spectrum Disorder

Two studies used serious games to address symptoms of ASD (35, 63). Beaumont and Sofronoff (35) and Tanaka et al. (63) had facial expression as common primary outcome measure. The games are based on the theory of enhancement of emotional understanding and social skills (35) through training. The goaloriented and problem-solving game *Junior Detective Training Program* (*JDTP*) was used in order to reduce social skills impairment (35). *JDTP* can be played on a PC without the need for an internet connection. In this game, the player is a junior detective in the year 2030 who is specialized in decoding suspects' thoughts and feelings. The player plays three levels with different missions including decoding suspects' feelings through facial expressions and body postures, deciphering cartoon character's feelings in different situations from non-verbal and environmental clues, dealing with bullying, and playing with others. After completing the three levels, the player graduates from the "Detective Academy." The games group was asked to play for the first hour in the first two sessions and 45 min per session in the third and fourth session. The seven sessions in total also comprised training time for parents. As primary outcome measures, one study used emotion recognition (facial expression and body posture) and emotion management (*Dylan is being teased*, a coping with bullying test and *James and the Maths Test*, a coping with anxiety test) (35).

The games of Tanaka et al. (63) are based on the theory of enhancement of recognition skills (63). The cognition training game *Let's Face It!* was used in order to reduce ASD-related symptoms such as poor facial recognition skills (63). *Let's Face It!* is available to the public and can be downloaded free of charge from the website of the University of Victoria (http://web.uvic. ca/~letsface/letsfaceit/); however, supervision of the player is recommended. The game can be played on the PC (or Mac) without internet connection after downloading. In *Let's Face It!*, the player plays face and object recognition games that target face processing skills, e.g., matching faces and connecting faces of the same identity. Participants were instructed to play the games for at least 100 min/week at home until intervention time reached 20 h. The parents received advice from the researchers about which games their children should play based on the data collected on compliance and the child's game play. *Let's Face It!* used face subtests (face dimensions, immediate memory for faces, matching identity, masked features, expression, and parts/whole identity) and object subtests (house dimensions and immediate memory for cars) (63). Data collection was done pre- and post-intervention. The intervention had a duration of 19.1 weeks average (63).

#### Attention Deficit Hyperactivity Disorder

Symptoms related to ADHD were targeted in one study (59). The serious game is based on ADHD theories which argue that deficits in executive functioning are related to impulsivity, hyperactivity, and attention (69–77). The cognition training PC game *Braingame Brian* (78) was used in order to reduce ADHD-related symptoms. In the game, the participant plays as Brian, a young inventor who helps and befriends in-game characters by creating inventions. The games consisted of a working memory task, a cognitive flexibility task, and an inhibition task. The game was played for 25 sessions of 35–50 min each.

#### Cognitive Functioning

Two studies targeted cognitive functioning symptoms (58, 61), meaning limited attention capacity due to impairment (61) and age-related decline in cognitive performances including working memory, speed of processing, and cognitive control (58).

The serious games studied by Rezaiyan et al. (61) are based on the finding that playing video games based on internal motivation can be a source of increasing attention power (61). A cognition training computer games program that focused on path-finding (proceeding from easy to hard) was used in order to reduce cognitive decline symptoms (61). The cognition training games were played for 35 sessions of 20–30 min each in one study (61).

Ballesteros et al. (58) targeted children with cognitive decline symptoms, determined by the Toulouse Pierson Scale. The serious games are based on theories of neuroplasticity (58). Ballesteros et al. (58) pursued a reduction in cognitive decline symptoms by using the commercially available cognition training PC platform Lumosity (79). This game is also available on mobile devices. The serious games were played for 20 sessions of 1 h each (58).

#### Alcohol Use Disorder

Treatment of AUD symptoms was of interest in one study (62). The participants were recruited at a veterans' outpatient medical center. The goal-oriented PC game *Guardian Angel* (80) is based on cognitive behavioral approaches (81). *Guardian Angel* was designed and used to reduce AUD-related symptoms. The participants played the game on a laptop at the medical center. In this game, the player acts as a "guardian angel" that needs to guide a character in early recovery of AUD to make daily decisions in support of recovery and continued abstinence. Players have to recognize and remove relapse risk factors. *Guardian Angel* emphasizes relapse prevention intervention techniques including identification of high-risk situations, drink-refusal skills, stimulus control, and craving-management techniques. The game was to be played during eight sessions over the course of 12 weeks. Participants in the game condition played 1 h per session, with the opportunity to play up to 8 h per session.

## Results of the Meta-analysis

#### Study Characteristics

The participants and study characteristics of the included studies are presented in **Table 1**. The 10 included studies were conducted in various geographical regions ranging from Europe [Sweden (58), the United Kingdom (48, 64), the Netherlands (59)], to Australia (35), New Zealand (34, 60), Asia [Iran (61)] and the United States (62, 63). A total of 674 participants were included in the meta-analysis (380 in experimental and 294 in control group). Total sample sizes ranged from 32 to 89 participants. Two studies targeted depression (34, 60). One study focused on ADHD (59). AUD was targeted in one study (62). PTSD was the subject of two studies (48, 64). Cognitive functioning was targeted in two studies (58, 61). ASD was the focus of two studies (35, 63). Four studies were aimed at children aged between 7 and 12 years (35, 59, 61, 63), two studies focused on teens to young adults (aged between 12 and 18), three studies were aimed at adults (18+) (34, 48, 60, 62, 64), and one study was aimed at older adults aged between 57 and 80 years (58). Six studies compared serious games to no intervention (34, 35, 58, 61, 63, 64); three studies compared them to active controls, such as training cognitive tasks, playing a quiz, and watching slides (48, 59, 62); and one study compared gaming to treatment as usual (TAU) (60).

Two studies used serious games in adjunct to TAU (35, 62). The serious games were played in different settings: three studies were conducted in a research laboratory (48, 58, 64), two at an educational institution (34, 35), two at home (59, 63), two at a health-care center (60), and one remained unclear (61).

The total number of comparisons that could be included in the analysis was *n* = 9. The study of Verduin et al. (62) on AUD did not provide amenable data for inclusion in the meta-analysis and is therefore excluded (**Table 3**).

These studies evaluated the effectiveness of serious games across a broad range of mental disorders and outcomes. The common ground between the included studies is, however, the evaluation through RTCs of possible improvement to mental health symptoms using serious games. Though the studies varied in the use of outcome measures for behavior change, its construct may be measured validly using meta-analytic methods by taking an average of outcome measures of the same construct (82). Firstly, an overall meta-analysis was conducted, which means that we looked at whether participants in the serious game conditions improved over the control conditions. Secondly, a meta-analysis was conducted per disorder using a similar approach.

#### Risk of Bias

**Figure 2** shows the results of the methodological assessment of the included studies in the meta-analysis. Five studies reported adequate random sequence generation (34, 59, 60, 62, 63). Two studies reported allocation to be concealed (59, 60). Two studies reported that both participants and personnel were blinded (59, 60). Three studies reported that research assistants were blinded for the outcome assessment (35, 59, 60). Four studies had a high likelihood of incomplete outcome data (35, 58, 60, 63). Several domains were rated unclear—the reason for this was incomplete reporting.

#### Publication Bias

Visual inspection of the funnel plot indicated possible publication bias. Performance of the Duval and Tweedie trim-and-fill procedure also indicated possible publication bias. After adjustment for missing studies, the effect size for serious games interventions changed from *g* = 0.63 to 0.48 (95% CI 0.17–0.78; trimmed studies = 2). The Egger's test did not indicate an asymmetrical funnel plot (*P* > 0.10).

#### Meta-analyses Outcomes

The overall outcome of the nine studies (*n* = 10 comparisons) showed a moderately significant effect size of *g* = 0.55 (95% CI 0.28–0.83, *P* < 0.001) for improvement on mental disorder symptoms at posttest (see **Figure 3**). Heterogeneity was substantial and significant (*I* <sup>2</sup> = 58.53, 95% CI 1.31–34.15, *P* < 0.05) (**Table 3**).



*CI, confidence interval; n comp, number of comparisons; NNT, number needed to treat; n/a, not available, could not be calculated due to the number of comparisons.*

*a The P values in this column indicate whether the Q-statistic is significant (I2 -statistics do not include a test of significance).*

*bThe P values in this column indicate whether the difference between the effect sizes in the subgroups is significant.*

*\*P* ≤ *0.001.*

*\*\*P* ≤ *0.05.*

One study (60) had a control group that received TAU in contrast to the other studies (no treatment or waitlist). After excluding this study, a significant moderate almost similar effect size remained, *g* = 0.63 [95% CI 0.36–0.90, *P* < 0.001, NNT = 2.91] (**Figure 4**). Excluding the study with the highest (34) and lowest (63) effect size showed similar results (see **Table 3**).

We then conducted meta-analyses on psychiatric disorderrelated symptoms for those where two or more studies were available (**Table 3**). This was the case for ASD (35, 63), cognitive functioning (58, 61), and PTSD (48, 64). The group that aimed at ASD-related symptoms such as lowered ability of recognition skills (35, 63) was shown to have a moderate non-significant (*P* > 0.05) effect size [*g* = 0.46 (95% CI 0.10–0.81, NNT = 3.91)]. The group targeting cognitive functioning-related symptoms such as lower attentional ability (58, 61) showed a large significant effect [*g* = 0.79 (95% CI 0.36–1.21, *P* < 0.05, NNT = 2.36)]. The group targeting PTSD-related symptoms such as flashbacks (48, 64) showed a moderate significant effect size [*g* = 0.59 (95% CI 0.20–0.99, *P* < 0.05, NNT = 3.09)].

We also conducted two subgroup analyses, grouping the studies by age and participant type. The group targeting youth (≤18) (34, 35, 59–61, 63) was shown to have a moderate effect size [*g* = 0.70 (95% CI 0.32–1.07, NNT = 2.63)] differing nonsignificantly from the adult group (18+) (48, 58, 64) [*g* = 0.53 (95% CI 0.07–0.99, NNT = 3.42)]. The group targeting clinical participants (with diagnosed mental disorder) (35, 59, 61, 63) showed a moderate effect size of [*g* = 0.59 (95% CI 0.18–0.99, NNT = 3.09)] differing non-significantly from the non-clinical group (34, 48, 58, 64) [*g* = 0.68 (95% CI 0.26–1.09, NNT = 2.70)].

#### DISCUSSION

#### Summary of Results

This study aimed to give an overview of serious games for mental health-related symptoms that were evaluated with RCTs.

There were eight different games in our study. One of the games, SPARX, is both a goal-oriented and problem-solving game. The games Guardian Angel and JDTP are goal-oriented games. Five games (Tetris, Let's Face It!, Braingame Brian, Lumosity, and "path-finding") were categorized as cognition training games. In order to ascertain which game genre works best in targeting specific mental disorder symptoms more exploration is needed and studies need to be replicated.

usual as control group removed from analysis.

All of the serious games were PC applications that required no internet connection to play. However, trends show that PC sales are declining, tablet sales are increasing, more time is spent on the smartphone and that most time spent on the smartphone is in playing games (in the US). It seems that the development and/ or validation of serious games for mental health on this technical platform is lagging behind. As Fleming et al. (83) remarked, many mental health apps are already available. It is clear that considerable opportunities lie in this area. Accessibility and feasibility can be improved if this area is utilized. Software can be conveniently downloaded or distributed in app stores already available on smartphones.

We included 10 studies in the review that comprised 8 different serious games. The studies targeted depression-related symptoms, ASD, PTSD, ADHD, cognitive functioning, and AUD. The results of the meta-analysis showed a mean moderate effect size of these serious games for reducing psychiatric disorder-related symptoms. This finding is similar to that of a recently conducted meta-analysis of game-based interventions by Li et al. (36) for depression only. The effect sizes found in this study correspond to NNTs of approximately 3, indicating that three patients have to be treated in order to have one who would benefit, slightly lower than Li and colleagues. However, Li and colleagues did not confine their meta-analysis to serious games—they included simulation studies as well. The clinical impact for the youth group (≤18, *g* = 0.70, *n* = 5) is comparable to the adult group (*g* = 0.53), as in the Li et al.'s study (36).

These findings indicate that serious games for mental healthrelated symptoms have potential for various age groups. Our results compare favorably to those found in the review of serious games for depression only by Fleming et al. (83). This review was not confined to RCTs. Comparing our results with internet interventions that are not serious game based is more complex due to the differences in setting, study designs, and outcomes applied. A very generic comparison, with similar results could be made with the meta-analysis of Ebert et al. (27) on youth (2015) who likewise found evidence for the moderate efficacy of internet and computer-based CBT in the treatment of depression and anxiety (*g* = 0.76). Regarding adults, another meta-analysis on the impact of internet-based depression interventions (18 RCTs) of Cowpertwait and Clarke (84) likewise showed that web-based interventions targeting adult depression reduced symptoms moderately significantly compared to controls (*g* = 0.43). This study also showed that reminders and guidance are important moderators for treatment outcome.

The majority of studies focused on youth and young adults especially for ASD, ADHD, and depression (34, 35, 59–61, 63). Games for children with ASD-related symptoms showed comparable moderate effect as found by Grynszpan et al. (85) for technology-based (non-serious gaming) interventions for children with autism spectrum-related symptoms (*d* = 0.47). Studies using serious games targeting cognitive functioning were available for both adults and youth.

No serious games primarily targeting anxiety were found in this study. This finding may not be surprising, because this can be expected of a relatively new field of research. Another explanation could be that game elements do not add much more to VRET, which is found to have potential to combat anxiety (86–88).

#### Strengths and Limitations

The strength of this study is that it is the first to provide insight into the potential effectiveness of serious games on psychiatric disorder-related symptoms based on RCTs only and on a strict definition of what is regarded to be a "serious game." This is important as RCTs are considered the gold standard of research to measure effectiveness of interventions and to make clear what a serious game is or is not. This study has some limitations as well. First, the number of studies that were eligible for inclusion in this review was small and the number of participants included in

#### REFERENCES


those studies likewise. The risk of bias in these studies was unclear in many cases due to incomplete reporting and may have been a problem (selection, performance, and detection bias) indicating that the methodological quality of studies could be improved and effect sizes could be overestimated. Furthermore, given the small number of studies, several mental health disorders and clinical outcomes were collated as psychiatric-related disorders instead of focusing on single ones. This means the findings can only indicate improvement instead of clear symptom reduction (89). Such an approach is, however, not uncommon in an evolving new domain of academic endeavor [see, e.g., Ref. (89)]. As a consequence, we decided not to conduct subgroup analyses of distinct mental disorders as the number of studies and power for doing so was too low.

#### Conclusion

Using serious games as interventions for reducing mental health problems appears feasible. Due to the limited number of RCTs that we have been able to include in this analysis, this review can only give an idea of the potential of serious games for treating mental disorders in the future. More RCTs are needed to determine the effectiveness of serious games. Future studies should not lose the technological development out of sight. Smartphone-based serious games for mental health need more exploration. Further, the effect and use of serious games for mental health that let players connect with other players using an internet connection need to be investigated.

#### AUTHOR CONTRIBUTIONS

HL: substantial contribution to every aspect of the manuscript, critical revision, and final approval and agrees to be accountable. JS: substantial contribution to design/conception, analysis, critical revision, and final approval and agrees to be accountable. TF: substantial contribution to interpretation of data, critical revision, and final approval and agrees to be accountable. HR: substantial contribution to design/conception, analysis, interpretation of data, critical revision, and final approval and agrees to be accountable.

#### ACKNOWLEDGMENTS

The authors would like to express their sincere thanks to Ms. Ka Wai Ma for assisting with the study selection and Mr. Jan Smit Jr. for helping with the data extraction.


**Conflict of Interest Statement:** TF is a codeveloper of SPARX computerized therapy for depression and can benefit from any commercialization of it outside of New Zealand. The remaining authors have no conflict of interest to declare.

*Copyright © 2017 Lau, Smit, Fleming and Riper. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Capitalizing upon the Attractive and Addictive Properties of Massively Multiplayer Online Role-Playing Games to Promote Wellbeing

*Gabriel Thorens1 \*, Joel Billieux2 , Pierre Megevand3 , Daniele Zullino1 , Stéphane Rothen1 , Sophia Achab1 and Yasser Khazaal1*

*1Addictology Division, Mental Health and Psychiatry Department, Geneva University Hospitals, Geneva, Switzerland, <sup>2</sup> Laboratory for Experimental Psychopathology, Psychological Sciences Research Institute, Catholic University of Louvain, Louvain-La-Neuve, Belgium, 3 Neurology Division, Clinical Neuroscience Department, Geneva University Hospitals, Geneva, Switzerland*

Keywords: game, serious game, gamification, health, video games, MMORPG

## INTRODUCTION

#### *Edited by:*

*Alexandre Andrade Loch, University of São Paulo, Brazil*

#### *Reviewed by:*

*Martin Desseilles, University of Namur, Belgium Andrew James Greenshaw, University of Alberta, Canada*

*\*Correspondence:*

*Gabriel Thorens gabriel.thorens@hcuge.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 29 July 2016 Accepted: 23 September 2016 Published: 17 October 2016*

#### *Citation:*

*Thorens G, Billieux J, Megevand P, Zullino D, Rothen S, Achab S and Khazaal Y (2016) Capitalizing upon the Attractive and Addictive Properties of Massively Multiplayer Online Role-Playing Games to Promote Wellbeing. Front. Psychiatry 7:167. doi: 10.3389/fpsyt.2016.00167*

Forty years after they were first introduced to the general public, video games have become firmly established into the mainstream of entertainment media. In parallel, early adopters and enthusiasts have become active scientists. As a result, interest in video gaming practices has shifted from an exclusive niche leisure activity to extensive social, commercial, cultural, medical, and psychiatric issues. An increasing number of studies on game-based treatments for various mental health disorders suggest that these approaches have the clear potential to promote cognitive and behavioral changes as well as symptom relief (1–10).

Game-based approaches for mental health disorders are still in an early stage of development and validation. Some interesting results have emerged, however. The inclusion of gaming tools within digital therapies has been assessed in several trials that studied participants with various sociodemographical profiles and disorders (11–13), including promising preliminary work in participants suffering from psychotic disorders (8, 12). Controlled studies on the assessment of digital serious games targeting depression (2, 11) and cognitive training in older adults (4) found promising findings, as did controlled studies on non-digital serious games developed to reduce delusional convictions in psychotic patients (1) or to promote smoking cessation (5). Also, two randomized, controlled studies showed that playing casual video games 30 min per day during 1 month (*Tetris*, *Bejeweled*) had positive effects on depressive and anxious symptoms (14–16).

It was furthermore shown (in spite of controversy) that game and video game practice may have a positive impact through the transfer of cognitive skills acquired in the game to functional competences (17, 18), although see Ref. (19, 20). A meta-analysis (21) on the effect of video game play on information processing showed minor effects on executive functions and small to moderate effects on visual, auditory, spatial imagery, and motor skills.

Another meta-analysis on the effect of brain-training games in older adults (22) showed a significant, if very small, improvement in cognitive testing after training, suggesting that some transfer of competence had occurred (albeit in a limited fashion).

Data on naturalistic use (in milieu use outside the scope of specific research programs) of internet-based therapies suggest that these programs are appealing for many users, particularly so for people who could benefit significantly from such treatments (23). Naturalistic use, however, appears associated with high levels of attrition. For instance, in a study of an internet treatment program for depression, less than 4% of community users completed at least three modules out of five compared with 53.8% of participants in the controlled trial (24–26). Attrition issues were also observed across digital games-related studies. Less than one-third of the 1622 adolescents allocated to a computerized game for binge drinking prevention returned to the second session and none of them completed all five sessions (27).

One of the ways to tackle the problem of attrition would be to develop interventions that succeed in increasing the commitment of the participants, possibly *via* the use of serious games tools (28). The important retention problems encountered in internet treatments, particularly among spontaneous community users, contrasts with the great success of games played for leisure, and, especially, massively multiplayer online role-playing games (MMORPGs) (29), which are sometimes associated with addictive use (30, 31). This opinion paper discusses how the attractive properties of video games, particularly those of MMORPGs, given their success and online setting, could be mobilized to promote wellbeing.

#### SPECIFIC POTENTIALLY ATTRACTIVE AND ADDICTIVE COMPONENTS IN VIDEO GAMES AND THEIR POTENTIAL USE FOR WELLBEING

Considering the attractive properties of video games as a single entity is reductive. There are indeed multiple types of video games, and an exhaustive listing of game types and their specific attractive properties is beyond the scope of this article. We will focus on attractive components common to all video games, with a specific focus on MMORPGs, since they have specific structural characteristics that could be useful for developing health-oriented video games. In the next sections, we will provide a (not necessarily exhaustive) list of video game components that have often been related to addictive use in past research (32), but which will be discussed here in terms of their potential to optimize interventions that promote wellbeing.

#### REWARD CONDITIONING SCHEME

Video games offer countless opportunities to optimize the distribution of reinforcers. The pioneering work of Schultz and colleagues on rewards (33) has shown that the rate of reward distribution may condition the intensity of dopamine release in the ventral tegmental area (a group of neurons in the midbrain that is a critical component of the reward system) and that the expectation of a reward can be as incentive as the reward itself. It is worth emphasizing that in commercial video games, in contrast to "real life" (one can strive all one's life for an ultimately unattainable goal), rewards are generally distributed in an optimized manner. Most video games are conceptualized to reward players in a fair manner, and the reward progression is harmonious and relatively predictable. When a player starts a game, the challenges are easy and the progression is fast. Thus, almost all MMORPGs are designed so that the time and effort required to "level up" (a major in-game reward whereby the player's avatar improves their abilities) increase progressively (i.e., 5 min of play are necessary to progress from level 1 to level 2, whereas 3 h are necessary to improve from level 60 to level 61) (34). In real life, many of the skills that people have or desire to acquire follow the opposite rule; for instance, learning to play the violin is unrewarding during the first few years and becomes easier and more enjoyable the more one's skill improves. Furthermore, in MMORPGs, the rate of progression is always in a positive direction: the avatar's strength and power grow steadily during the game, and developers do not usually integrate avatars that age or develop chronic diseases or cognitive decline.

In the next sections, we provide illustrations of how MMORPG-based reward distribution could be used in games designed to increase wellbeing. In the case of video games that aim to improve physical condition (exergaming, for example a video game coupled with a running treadmill), a solution to compensate for the frustration of the difficult first steps of practicing (as in the violin example) would be to increase the amount of rewards at the beginning of the game. Users could initially gain in-game rewards and levels after running for short distances and, therefore, experience the subjective impression that the task is rapidly rewarding. Such a view is supported by a recent review on exergaming (35).

A more tentative proposition would be a game designed to overcome social phobia, which could take the form of a multiplayer game where players are encouraged to have and rewarded for having face-to-face video conferences. In such a game, every social contact should be rewarded according to the reinforcement schedule depicted earlier, so that the participant fully appreciates his or her progression. Using such a game is susceptible to optimize the learning process of social exposure, in comparison to real-life situations where no direct positive feedback follows an exposure exercise (the shopkeeper will rarely congratulate the patient for sustaining their gaze and speaking clearly).

These rewards mechanisms can also have downfalls. The loss of the novelty effect is one: the generalization of the so-called gamification of everyday activities tends to reduce the effect of virtual rewards, thus the incentives to receive virtual rewards could progressively decrease. It is also important to bear in mind that in-game rewards remain intangible; strategies to increase behavioral modification could link virtual rewards to real ones to increase their attractiveness (for instance, winning a real pair of running shoes could be linked to an in-game achievement). Similar strategies have been applied for a long time in contingency management for the treatment of substance use disorders with positive outcomes overall (36).

Rewards should be conceptualized as a means to promote wellbeing and not as an end in themselves to avoid the development of an addiction to the game mechanisms. Take the case of *Zombies, Run!,* an audio/video game that couples running exercises to missions where the player must escape zombies chasing them. Running with headphones, the player hears where zombies are lurking and how to escape them. The more they run effectively, the more they gain levels and bonuses. A problematic use of this game could appear if the player's purpose becomes not to enhance their running ability, but rather to progress in the game as fast as possible. As they become addicted to the in-game rewards, they will start to play the game while driving their motorcycle or car to be more effective. This supposition might sound unrealistic, but it is in fact the core mechanism of the so-called "free-to-play" games, where the player has the choice to either play a game for free, but must play for very long to get a reward, or pay to progress much faster and avoid the long wait (37).

#### NEVER-ENDING GAMES

Massively multiplayer online role-playing games, as opposed to single-player role-playing games, are endless. The developers constantly update the game to implement new features and provide every player new tasks to accomplish, often on a daily or weekly basis. The game will stop only when its popularity decreases and the developers stop maintaining the dedicated servers. Even then, sometimes, private communities continue to maintain the game on their own and provide continuity in the game environment for the most committed players.

It is obviously difficult to provide the same continuity in so-called serious games, whose development and validation are generally tied to studies that themselves depend on grants with a fixed and limited amount of time and money. Commercial video games are extensively tested before they are eventually released. After release, constant updates and improvements have to be implemented. Players directly contribute to these ameliorations through constant feedback on their gaming experience.

Open-source games for health could represent a potential solution (38). In open-source software, the computer code of the software is available for anyone to modify; thus, users could modify themselves the game's content and continue the development and maintenance of validated games, i.e., games whose potential for health improvement was demonstrated in controlled trials. The risk with open-source software, however, is that games could evolve in an undesirable direction. If any player is free to modify or implement new game features, ill-intentioned players or hackers could intentionally introduce inappropriate content or threaten other players (cyberbullying).

#### SOCIAL INTERACTION BETWEEN PLAYERS

The more a game is popular, the more players will be attracted. In 2009, the MMORPG *World of Warcraft* reached more than 12 million players worldwide. This high number of players obviously offers the possibility for everyone to find affinities with other players and progress together. Game developers are aware of the key role of social interactions as a factor for the success of a video game (39), and MMORPGs are thus designed to allow people to regroup in social virtual networks (often called "guilds") or to communicate in real time while playing. If the same principle is applied to a game that addresses social phobia, the large amount of people playing this game will help to alleviate the stigma of the diagnosis and make the players feel less alone and better understood regarding their specific difficulties. Furthermore, the peer reinforcing effect would be beneficial for users.

Another example of peer reinforcing effect is a Swiss initiative proposing a self-help group on Facebook for tobacco addiction. The objective was to set a common date for participants to quit smoking; the members of the group would then help each other to keep motivated.1 This initiative was more successful than expected, and developers of games for health could be inspired by this type of experience.

One of the negative counterparts of the social aspect of games is the risk of recreating within the game communities that are stigmatized or excluded in real life. Clinical practice shows that people with social interaction problems also tend to have difficulties in virtual life and could reproduce their difficulties and poor coping strategies in games (40). Video game companies have already had to deal with antisocial in-game behaviors. Their employees must constantly monitor the interactions between players and sanction or ban problematic players. It would, therefore, be fundamental to avoid cyberbullying by potentially antisocial individuals (41).

## APPEALING CONTENT

Developing a video game requires a large multidisciplinary team of programmers, artists, scenarists, 3D motion specialists, etc. Obviously, a task generated with the tools traditionally used in experimental psychology and psychiatry (software such as *E-Prime* or *Psychopy*) will not have the same visual or emotional impact as software developed by such a multidisciplinary professional team. As an illustration, a video game to treat anxiety *via* biofeedback, called *Mindlight*, 2 was developed in collaboration with a video game company to combine the video game industry's *savoir faire* with scientific knowledge. The result is designed to propose the appealing visuals of a state-of-the-art video game with a scientific rationale.

#### COMPETITION

Along with social interaction, competition is a key component in the attractiveness of MMORPGs and of video games more generally (42). The net democracy or "equal chances for all" is an important incentive and a rewarding possibility. Someone who is 1.5 m tall (4 ft 11″) has very low chances of becoming an NBA basketball star, no matter their skill. Video games offer a chance for everyone to start with the same characteristics; as such, the balance between avatars is a main concern for game developers. Competition is described as an important motivation for MMORPG players (43), and most games are designed to match individuals with similar skill levels together. Thus, the impression of having a fair competition enhances the motivation to perform a task. In exergaming for example, the level of the player's performances could be monitored, and the possibility of competing against virtual avatars or other individuals constantly matched to their level is expected to enhance their motivation to play. Thus, if someone aged 33 has the running capacity of a 70-year-old, they will not be able to compete with runners their age, but will likely not be interested or allowed to join the senior category in a

<sup>1</sup>https://www.facebook.com/j.arrete.de.fumer.ch

<sup>2</sup>http://www.gainplaystudio.com/mindlight/

#### Table 1 | Summary of specific potentially attractive components in MMORPGs and their potential use for wellbeing.


traditional running competition. If, on the other hand, the same runners are represented by their avatar performance without any mention of age, they could have the impression to progress and achieve good performances in their own avatar category.

Competition could also become counterproductive in wellbeing-oriented games, and, once again, if the end goal becomes the competition by itself, it could divert the game from its original objective and lead to negative outcomes.

#### GAMES ARE VIRTUAL

Video games give opportunities to experience sensations otherwise too dangerous, forbidden or inaccessible. *The virtual-reality headset technologies (i.e., the "Oculus Rift" system purchased by Facebook) become affordable and broadly available and could greatly enhance immersion*. For example, people who are afraid of driving or flying could play realistic simulations to have a first experience, and eventually gain knowledge and confidence in these activities (44). This could then lead to a better motivation

#### REFERENCES


to try these activities in real life. As described before, the generally established efficacy of virtual-reality treatments of anxiety disorders (45) (e.g., exposing individuals to virtual spiders for arachnophobia) sustains this argument. The main advantage of this approach is to provide a realistic but safe environment, free from any real threat, that makes the patients more inclined to accept the treatment (there are no real spiders during the treatment or, in the case of acrophobia, no real cliffs), but that still triggers real manifestations of fear. On the opposite, it could be argued that because there are no real consequences to a virtual action, dangerous behaviors devoid of any concrete negative consequence could be promoted and applied to the real life, as in the example of reckless driving in video games (there are no real consequences to a virtual car crash) (46). *The use of virtual avatars could also be helpful in psychiatric treatments, as, for example, evidenced by a recent study that demonstrated how using avatar-based therapy is useful to decrease persecutory auditory hallucinations* (47).

#### CONCLUSION

The popularity of video games has been established in the past four decades and is still growing. Nowadays, successful video games are designed for entertainment, but as researchers are analyzing the mechanisms of this success, they are more and more confident on the possibilities of designing video games for wellbeing. In this article, we formulated ideas to suggest how to take advantage of the core attractive mechanisms of MMORPGs, such as conditioning reward schemes enhanced by players' social interactions (**Table 1**).

We also pointed out some obstacles to the realization of video games for wellbeing, such as the lack of academic resources compared to the game industry and the risk of encouraging problematic or addictive video game use. Technological progress in the field is rapidly evolving, and virtual or augmented reality will soon provide new innovative tools to further the development of video games for wellbeing.

#### AUTHOR CONTRIBUTIONS

All authors listed have made substantial, direct, and intellectual contribution to the work and approved it for publication.


PRISMA guidelines of the current findings and the associated risk of bias. *Front Psychiatry* (2015) 6:151. doi:10.3389/fpsyt.2015.00151


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Thorens, Billieux, Megevand, Zullino, Rothen, Achab and Khazaal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Six Tips on How to Bring Epic Wins to Health Care

*Anna Sort1,2 and Yasser Khazaal 3,4,5\**

*1University of Barcelona, Barcelona, Spain, 2PlayBenefit, Barcelona, Spain, 3Research Centre of the Montreal University Institute of Mental Health, Montreal, QC, Canada, 4University of Geneva, Geneva, Switzerland, 5Geneva University Hospitals, Geneva, Switzerland*

Keywords: games for health, gamification, Internet treatment, mental disorders, Internet

#### INTRODUCTION

Gamification and game-based treatment for mental health disorders are growing topics in the scientific literature. In health care, gamification is the use of various game processes in non-game contexts to engage users in behaviors that aim to improve health-related outcomes (1). In other words, gamification seeks to create motivational experiences that engage users in gameful actions and fulfill the human need for fun.

#### *Edited by:*

*Shervin Assari, University of Michigan, United States*

#### *Reviewed by:*

*Mohammadreza Peyman, University of Michigan, United States Marilyn Rose Lennon, University of Strathclyde, United Kingdom*

#### *\*Correspondence:*

*Yasser Khazaal yasser.khazaal@hcuge.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 30 March 2017 Accepted: 15 November 2017 Published: 30 November 2017*

#### *Citation:*

*Sort A and Khazaal Y (2017) Six Tips on How to Bring Epic Wins to Health Care. Front. Psychiatry 8:264. doi: 10.3389/fpsyt.2017.00264*

Frontiers in Psychiatry | www.frontiersin.org

Several benefits are expected from the development of games for mental health, such as an increase in the reach of mental health interventions (appealing effect) and a reduction in the attrition rates of Internet-based interventions (engaging effect) (2, 3). Furthermore, games are expected to help behavioral change *via* situated learning (4)—which derives from achievements in participatory knowledge in immersive contexts—possibly involving activity, training, and social practice (2, 3, 5–7). Over the past 5 years, a number of studies on board games and digital games have suggested that approaches such as the gamification of therapeutic processes or casual games used for therapeutic purposes have the potential to promote cognitive and behavioral change and improvements in symptoms (8–21). A well-designed recent meta-analysis (22) that included nine randomized controlled trials (i.e., 674 participants) assessed the effectiveness of digital serious games (SGs) for the symptoms of mental disorder. The authors found a moderate effect on the improvement of symptoms that favored SGs over controls. They concluded that SGs may be effective in reducing the symptoms of mental disorder and asked for additional studies in this promising field.

The assessed games varied widely in their aims (e.g., psychoeducation, cognition training, goal-oriented, problem solving), format (e.g., involving avatars, fantasy world, personalization, exergames) and mechanisms. Some of these games were designed for treatment, while others were casual games (e.g., "Tetris") used in expectation of a therapeutic effect (22). Unsurprisingly, therapeutically designed games vary widely in their theoretical background depending on their specific aims. For example, the game called "Let's Face It!" is based on the theory of enhancement of recognition skills and aims to increase facial recognition abilities (23), the game called "Sparx" is based on cognitive behavioral therapy (CBT) and seeks to treat depressive symptoms (10), and "Michael's Game," which is based on the principles of CBT for psychotic disorders, aims to reduce delusional conviction (8).

Some reviews have reported, however, that most health-related games lack several typical immersive gamification features (24) and have a limited theoretical foundation (25). In addition, despite promising results and the *a priori* engaging nature of games, high rates of attrition and a decrease in game use over time have been previously reported (26, 27). For instance, none of the 1,622 adolescents participating in a study on a computerized game designed for reducing binge drinking completed the five sessions of the game (28). Thus, although game design provides flexible opportunities to promote behavior change, target goals need to be reached in order for games to be successfully adopted by users.

Despite the attractiveness (29) and potential of games, the processes underlying game design have not been reported in detail in most game-related studies. This weakness has commonly been reported in the literature for behavior change interventions (30) and has led to a lack of guidance on how to practically design such interventions, particularly in health-related games (31).

The present opinion paper aims to outline possible steps to follow in order to implement gamification in health care by using adequate solutions. SGs must demonstrate the transfer of learning (to be "serious") while, at the same time, remaining engaging and entertaining (to be "games"). The balance between fun and educational measures should be targeted throughout game development, starting from the design phase. Yet, despite the potential of digital games in terms of interactivity, immersion, and engagement, further studies are needed to understand how to better design, administer, and assess such games across different learning contexts and targets (31, 32). To date, the insufficient integration of behavior change and game design principles is one of the biggest issues with SGs (27).

The proposed tips for gamification are as follows.

#### KNOW WHAT YOUR GOALS ARE

Translating goals into game design mechanisms and processes is the pinnacle of gamification (31). In other words, one of the most important steps to ensure a successful outcome of the engagement design is to know what your goals are (i.e., the game goals and the real-life outcome goals). Accordingly, users' engagement is expected to increase when their expectations match the goal of the intervention (5). To move from general, non-specific goals to more specific goals and to refine goal setting (33), it might be useful to involve designers, clinicians, and end users in collaborative workshops.

#### PUT THE USER FIRST

As obvious as it may sound, this second tip is too frequently disregarded. Giving priority to the client's or the patient's experience is a necessary condition for meeting end users' needs effectively (3). Designers often do a great job in thinking about how to get the best results, but then completely forget to consider how to interest older adults in using a tablet and a medical device, how to teach them to use the device, and how to warn them that they even have to use a device. Therefore, the project might be doomed to failure before it even begins. By putting the users' experience first, you will understand exactly what you want the users to go through, as well as what they are going through. As a result, you will ensure that game processes are designed to suit the users' needs and abilities.

This is a crucial step in developing, planning, and performing actions that succeed in promoting goal attainment in the game (33). It involves considering the main determinants of the given behavior, possible short- and long-term concurrent goals, practical obstacles, and the correspondence between the goal and the users' self-image, self-compassion, and values (34–37). When you meet with users, adopting a detailed approach may help you precisely describe the steps they take and their level of stress with each step. It is important to ask them to describe the steps that they expect next for their benefit, as well as any "crazy ideas" they may have. In the end, you might gain valuable insights into what users are doing, what they would like to happen and what they would love to happen.

Furthermore, game designs must be tailored to the needs and capabilities of the target group in order to enhance motivation and fun (38), self-efficacy (39), and adherence (38). For instance, recently published design recommendations for older adults' SGs have taken age-related cognitive change into account (40). Giving priority to the user's experience could be achieved by using participatory, user-centered design, i.e., continuously involving users at every stage of development or formative research steps and collecting information from the target group to ensure that game processes are in tune with their needs and capabilities (41). Despite its theoretical importance, the impact of participatory design (PD) on game engagement and effectiveness is nevertheless inconclusive and is likely to depend on the type of PD and focus. Nonetheless, involving users in the development of key game components may be more effective than involving them in esthetic appreciation (41, 42).

#### USE BEHAVIOR CHANGE THEORIES TO INSPIRE YOUR GAMIFICATION

When thinking about "gamifying" something, bear in mind what behavior you want to change. Do you want people to be using something they are not using now? Do you want them to increase the number of times a day they use something—something they currently use only once a day? Do you want to introduce something new into their routine? Are the people you want to help facing specific competing impulses or inhibitions related to the game's main pursued goal? Are knowledge, specific beliefs, expectations, or barriers influencing the given behavior? Is the game addressing motivation, self-regulatory capacity, or any other specific skill? These are concerns that must be addressed to define what should be achieved in terms of behavior and to reach your goal. To do so adequately, using a given behavior change theory (30, 43) provides helpful guidance in the game design process (44).

Considering the variety of behavior change theories and their differences in focus and construct—such as the Health Belief Model, the Theory of Planned Behavior, and self-control and willpower-related models (43, 45, 46)—it is important to choose the most appropriate guidance in line with the game's goals.

Some studies found that explicit reliance on behavior change theory and its related techniques (47, 48) was associated with better outcomes (49, 50). Unfortunately, most of the available health-related digital interventions did not explicitly rely on such theory (51–55). Moreover, specific behavior change techniques may vary across goals and topics (47, 48, 56, 57). However, some behavior change theories such as the Self-determination Theory (58), Fogg's Behavior Model (59), and Social Cognitive Theory (SCT) (60) may provide guidance about game development across health-care sectors.

Self-determination Theory (58) highlights the importance of competence (the need to control the outcome, to experience mastery), relatedness (being connected with others), and autonomy in the change process, as well as of a coherent framework in game design (5). Fogg's Behavior Model states that behavior change is a function of three fundamental elements: motivation, ability, and trigger (e.g., a motivating element, a facilitator, a signal) (59). SCT (60) focuses on how to change self-efficacy—a main determinant of behavior change—when facing barriers in achieving goals and suggests different solutions such as vicarious experiences. A smoking cessation game, leading to an increase in self-efficacy based on game processes and learning situations related to the barriers usually faced by people trying to change their smoking behavior, provides a valuable illustration of that process (11).

To improve the articulation between the different components of game design for behavior change, Starks proposed a cognitive behavioral game design framework (44). This framework involves the SCT, the Theory of Multiple Intelligences (MI), and gamification processes (44). MI tackles the multiple ways of achieving learning and experiences (music, mathematics, logic, humor, narratives) and highlights the game's ability to use a multimodal approach, possibly combining sensoriality, emotions, and rational thinking (61). The author (44) summarized the challenge of game design in the question, "How do I express one or more social cognitive elements through the mechanism of one or more intelligences in a way that facilitates the enjoyment process?" thereby referring to the involvement of game theories in the design of SGs.

#### USE PLAY, FUN, AND GAMES THEORIES

It makes little sense to talk about gamification and then create something utterly boring. But how exactly does one create fun? The following three features are particularly helpful in developing a game process:


The player's journey is essential to gamification for structuring vision, content, and progression. Have you ever been on a website where you were overloaded with choice and yet had no idea how to find what you were looking for? Conversely, have you ever been on a website with little choice and without a search function, which did not help you to find what you were looking for? These examples both address the issues of purpose, autonomy, and mastery. Accordingly, if people have greater autonomy, but have no idea where they are heading—in other words, if they have little mastery—they find themselves in the first example. If they have a high level of mastery but little autonomy, they find themselves in the second example. The purpose in finding something needed on a web page is essential for a satisfying experience. Indeed, without purpose, there is no action (62).

When setting up and developing a solution, consideration of the gamer's journey will help structure what you want users to find first. This will help you understand the basic features of the system that users might need to proceed to the next level, so that they do not feel over-challenged, but confident enough in their ability to take up the next challenge with mastery. These three features are closely related to one another for distinguishing what is boring from what is engaging: too much mastery with little autonomy is as boring as too much autonomy and a low level of mastery of what one is doing or supposed to do or achieve (6). According to Flow Theory and Motivation (63), activities that provide an optimal balance between challenge level and skill acquisition level create a motivational state of flow. The flow enhances immersion, creativity, and performance.

Building an attractive journey for the player with optimal flow further requires a sound knowledge of the audience's perceived health-related needs, as well as of possible gamer types—e.g., an achiever is motivated by mastery, a socializer by social interactions, a philanthropist by purpose, and a free spirit by autonomy (64, 65).

## BE MINDFUL OF THE SOCIAL LINK

According to Self-determination Theory (66), relatedness (e.g., the social link) is one of the keystones in behavior change processes. Many studies have shown the benefit of having a buddy, a community, or some kind of social support when one engages in a process, be it about improving wellness and fitness or dealing with a disease (39).

In games, it is easy to understand how these varied social interactions develop in a fun way. Amy Jo Kim—a game designer known for her career and work on games such as "The Sims"—has developed a social interaction chart (67) that shows the different ways in which people engage socially, some of which might seem more meaningful than others, depending on the type of player.

Social interactions may bring meaning into the system that you are implementing and may encourage bonding and a sense of belonging, which can be powerful for driving behavior change. In the game design process, social interactions can be divided into three categories corresponding to the following questions:


#### IT IS NOT JUST ABOUT GAME MECHANICS AND GAME DYNAMICS

Gamification is about games and about making games appealing. Indeed, it is critical to keep in mind that the system has to be attractive enough for gamers to continue using it, even if points, badges and leader boards are removed (1). One should favor mechanics such as meaningful feedback and timely rewards that help gamers to improve their standing or to proceed to the next level more effectively (68). This will ensure that, besides appealing to users who favor extrinsic motivators (e.g., a Smart Box, tablets, etc.), your system will appeal to those who favor intrinsic motivators (e.g., learning) (69, 70) and may secure users' engagement in the game for a longer time.

When designing games, one should also keep in mind that static things, although exciting at first, can easily lose their appeal. By way of illustration, the Fun Theory piano stair (71) is a cool way of making people choose regular stairs rather than the escalator; but would people feel the same way after 5 days? 10 days? Providing a game experience that is difficult enough for users to feel challenged (where they must make an effort to succeed), but easy enough for them to remain confident that they can succeed, may be the condition for keeping users engaged in the game. Conversely, providing a game experience where users repeat the same actions over and over may cause them to lose interest.

#### REFERENCES


#### FURTHER STEPS

Because further improvements in health-related games are needed, we encourage further studies to be done to explicitly describe the game design process, possibly by using the six tips described in the present paper. This may help to further refine game design taxonomy in the field. Assessing health-related outcomes that have already been achieved in a number of studies, as well as linking such outcomes to intra-game processes and gamers' profiles, might also prove valuable.

#### AUTHOR CONTRIBUTIONS

We confirm that all authors contributed equally to the writing of this manuscript.

#### ACKNOWLEDGMENTS

We thank Barbara Every, ELS, of BioMedical Editor, for English language editing.


promotion involving serious digital games. *Games Health J* (2015) 4(2):78–90. doi:10.1089/g4h.2014.0110


**Conflict of Interest Statement:** AS is CEO of PlayBenefit and YK is one of the authors of the following games: "Michael's Game" and "Pick-Klop."

The reviewer MP and handling editor declared their shared affiliation.

*Copyright © 2017 Sort and Khazaal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# An Objective Screening Method for Major Depressive Disorder Using Logistic Regression Analysis of Heart Rate Variability Data Obtained in a Mental Task Paradigm

#### *Guanghao Sun1 \*, Toshikazu Shinba2 , Tetsuo Kirimoto1 and Takemi Matsui3*

#### *Edited by:*

*Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland*

#### *Reviewed by:*

*Erich Flammer, ZfP Suedwuerttemberg, Germany Stephane Rothen, University of Geneva, Switzerland*

#### *\*Correspondence:*

*Guanghao Sun guanghao.sun@ieee.org, guanghao.sun@uec.ac.jp*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 29 August 2016 Accepted: 20 October 2016 Published: 04 November 2016*

#### *Citation:*

*Sun G, Shinba T, Kirimoto T and Matsui T (2016) An Objective Screening Method for Major Depressive Disorder Using Logistic Regression Analysis of Heart Rate Variability Data Obtained in a Mental Task Paradigm. Front. Psychiatry 7:180. doi: 10.3389/fpsyt.2016.00180*

*1Graduate School of Informatics and Engineering, The University of Electro-Communications, Tokyo, Japan, 2Department of Psychiatry, Shizuoka Saiseikai General Hospital, Shizuoka, Japan, 3Graduate School of System Design, Tokyo Metropolitan University, Tokyo, Japan*

Background and objectives: Heart rate variability (HRV) has been intensively studied as a promising biological marker of major depressive disorder (MDD). Our previous study confirmed that autonomic activity and reactivity in depression revealed by HRV during rest and mental task (MT) conditions can be used as diagnostic measures and in clinical evaluation. In this study, logistic regression analysis (LRA) was utilized for the classification and prediction of MDD based on HRV data obtained in an MT paradigm.

Methods: Power spectral analysis of HRV on R–R intervals before, during, and after an MT (random number generation) was performed in 44 drug-naïve patients with MDD and 47 healthy control subjects at Department of Psychiatry in Shizuoka Saiseikai General Hospital. Logit scores of LRA determined by HRV indices and heart rates discriminated patients with MDD from healthy subjects. The high frequency (HF) component of HRV and the ratio of the low frequency (LF) component to the HF component (LF/HF) correspond to parasympathetic and sympathovagal balance, respectively.

Results: The LRA achieved a sensitivity and specificity of 80.0 and 79.0%, respectively, at an optimum cutoff logit score (0.28). Misclassifications occurred only when the logit score was close to the cutoff score. Logit scores also correlated significantly with subjective self-rating depression scale scores (*p* < 0.05).

Conclusion: HRV indices recorded during a MT may be an objective tool for screening patients with MDD in psychiatric practice. The proposed method appears promising for not only objective and rapid MDD screening but also evaluation of its severity.

Keywords: heart rate variability, major depressive disorder, self-rating depression scale, logistic regression analysis, autonomic activity

#### INTRODUCTION

Major depressive disorder (MDD) is a critical public health concern, with an estimated 350 million people having this disease worldwide (1). Therefore, rapid and accurate detection of patients with MDD during the early stages can facilitate the decision-making process of psychiatric clinicians in the mental health treatment of patients. Unlike other areas of medicine, objective diagnostic biomarkers for psychiatric diseases are not readily available (2, 3). Psychiatric clinicians diagnose outpatients subjectively based on clinical interview and diagnostic criteria. Thus, the diagnosis of MDD tends to depend on the knowledge and experience of the clinician (4). In this study, we propose a novel, non-invasive, objective, and physiological method for screening outpatients with MDD that uses quantitative estimates of altered autonomic nervous system activity and reactivity based on heart rate variability (HRV) indices *via* logistic regression analysis (LRA).

Heart rate variability, which refers to beat-to-beat alterations in the heart rate (HR), can be used to assess autonomic nervous system function (5). Low frequency (LF: 0.04–0.15 Hz), high frequency (HF: 0.15–0.4 Hz), and LF/HF HRV indices provide quantitative estimates of sympathetic and parasympathetic activities (6). Changes in HRV have been found in various disorders presenting both somatic and psychological symptoms, including eating disorder, pain, anxiety, stress disorder, and depression (7–9). Although HRV changes appear non-specific, frequent comorbidity of these disorders may underlie the prevalence. Recent studies have shown a strong association between MDD and HRV abnormalities (10–12). A recent review investigated the impact of depression on HRV indices in patients with MDD without cardiovascular disease (13). High HR and LF/HF and low HF among patients with MDD reflect a baseline shift of autonomic balance toward sympathetic activation (14–16).

In addition to altered autonomic baseline activity, we further assessed the reactivity of HRV measurements during rest and mental task (MT) conditions in patients with MDD and healthy control subjects (17, 18). We found that both HRV at rest and its response to MT conditions were different in patients with MDD. Specifically, HF, LF/HF, and HR were less reactive in patients with MDD compared with healthy subjects during the MT condition. Differences in the reactive of HRV indices during rest following the MT condition were also observed. These multiple autonomic alterations in MDD are possibly related to disturbances in arousal control. These findings suggest that HRV indices may be a useful screening tool for MDD in clinical practice. One such method involves the use of HF, LF/HF, and HR before, during, and after an MT in LRA.

In the present study, a prediction model for binary classification of multidimensional data (i.e., HF, LF/HF, and HR indices) was constructed using multiple LRAs. LRA, which is a probabilistic statistical classification model, is a simple but powerful linear algorithm for analyzing multidimensional data and generating predictions of clinical results (19). The aim of this study was to evaluate the efficacy of a proposed screening method for risk stratification of patients with MDD in clinical settings.

#### MATERIALS AND METHODS

#### Participants

We tested this method with clinical data from 44 patients, thereby performing a qualitative assessment of the proposed screening method based on LRA for risk stratification of patients with MDD. The patients (43 ± 12 years old; 23 men and 21 women) from Shizuoka Saiseikai General Hospital were drug-naïve patients and had a diagnosis of MDD according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) criteria published by the American Psychiatric Association. Forty-seven age- and sex-matched healthy control subjects (41 ± 12 years old; 21 men and 26 women) who had never been diagnosed with cardiac, neurological, or psychiatric disorders were also included. There was no difference in the proportion of tobacco users between patients and healthy subjects.

Figure 1 | A schematic diagram of heart rate variability (HRV) monitoring. Electrocardiograms (ECGs) were recorded before, during, and after a mental task (MT).

Participants were instructed not to consume alcohol or coffee for 24 h prior to the study and not to consume tobacco on the day of the study. The Zung's self-rating depression scale (SDS) score was used to assess the severity of symptomatology for both groups (20). This study was approved by the Ethics Committee of Shizuoka Saiseikai General Hospital. All subjects provided written informed consent.

#### Study Protocol

The study protocol has previously been described (18). Briefly, each subject was seated in a chair with electrodes attached to their chest (**Figure 1**). Electrocardiograms (ECGs) were recorded before, during, and after the MT. Prior to the MT, subjects were instructed to relax for 100 s. Following this rest period, subjects completed random number generation for 100 s as the MT. During the MT, subjects were instructed to verbalize single-digit numbers (0–9) in a random order at a rate of 1 Hz with the assistance of a metronome. Following the MT, subjects were instructed to relax for 120 s. ECG signals were transferred to a computer *via* an A/D converter for offline analysis. Heartbeat intervals and HR were derived from R peak intervals. Power spectra of the time series of the heartbeat intervals were detected every 2 s for the preceding 30 s data using a maximum entropy method with the MemCalc software (GMS, Tokyo, Japan). To avoid the influence of preceding states, HF, LF/HF, and HR were averaged in the interim from 30 to 60 s after the onset of each new state.

Figure 2 | Heart rate variability (HRV) and heart rate (HR) indices before, during, and after a mental task (MT) in a representative healthy control subject (left panels) and a patient with major depressive disorder (MDD) (right panels). The low frequency to high frequency ratio (LF/HF) (sympathetic index) and HR increased and HF (parasympathetic index) decreased during the MT condition in healthy control subjects, but not in patients with MDD.

#### LRA Development Using Multivariate Data

Logistic regression analysis, which is a probabilistic statistical classification model, is a simple but powerful linear algorithm for analyzing multidimensional data and generating predictions of clinical outcomes. Multivariate selection is an important step in the development of a multivariate model that can effectively separate patients with MDD from the healthy control subjects. In the present study, we included nine HRV and HR indices in the LRA: HFbeforeMT, HFMT, HFafterMT, LF/HFbeforeMT, LF/HFMT, LF/ HFafterMT, HRbeforeMT, HRMT, and HRafterMT.

We defined the above selected nine variables as *x*1−*x*9 in a nine-dimensional vector *X* = (*x*1, *x*2, …, *x*9). LRA is a linear discriminative approach that attempts to optimize a logistic sigmoid function to calculate the probability *p* that vector *X* is representative of patients with MDD. The present study used binary classification; therefore, the probability of healthy control subjects is 1 − *p*. Thus, the LRA is expressed as follows:

$$\log\left(\frac{p}{1-p}\right) = b\_0 + b\_1\varkappa\_1 + b\_2\varkappa\_2 + \dots + b\_9\varkappa\_9\tag{1}$$

where log *<sup>p</sup>* 1− *p* is the predicted logit score, *b*0 is a constant,

and *b*1, *b*2, …, *b*9 are regression coefficients estimated by maximum likelihood criterion. Probability *p* is written as the logistic sigmoid function:

$$\mathcal{P} = \frac{1}{1 + \mathcal{e}^{-(b\_0 + b\_1 x\_1 + b\_2 x\_2 + \dots + b\_b x\_b)}} \tag{2}$$

determine the optimum cutoff point for classifying the patients with MDD and healthy subjects. According to the ROC, the area under the curve for this screening test was 0.891 (95% confidence interval, 0.828–0.954); the sensitivity of 80.0% and specificity of 79.0% were obtained from the optimum cutoff point of 0.28.

#### Statistical Analysis

We evaluated the ability of the LRA to discriminate patients with MDD by calculating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The performance of the LRA was evaluated using Nagelkerke's *R*<sup>2</sup> and Cox and Snell's *R*<sup>2</sup> tests (21). Moreover, to find the optimal cutoff point with the highest sensitivity and specificity of LRA, a receiver operating characteristic (ROC) curve was adopted. The area under the curve (AUC) of the ROC is considered an effective measure of the performance of a diagnostic test, with results ranging from 0.5 to 1.0, where larger values are indicative of better performance (22). The HRV and HR indices were compared between the classified MDD group and the healthy group *via* Student's *t*-test. A *p*-value of less than 0.05 was considered to indicate statistical significance. The LRA and ROC were performed using SPSS 23.0 (IBM, Armonk, NY, USA).

#### RESULTS

Sample data from a patient with MDD and a healthy control subject are shown in **Figure 2**. In healthy control subjects, the LF/ HF and HR increase, and HF decreases during the MT condition and return to baseline levels following the MT. In contrast, these changes are not observed in patients with MDD.

To discriminate patients with MDD from healthy control subjects in our dataset, the logistic sigmoid function log *<sup>p</sup>* 1− *p* was calculated from the HRV and HR variables with the following

estimated regression coefficients:

$$\begin{aligned} \log\left(\frac{p}{1-p}\right) &= -5.062 + 0.201 \text{HR}\_{\text{beforeMT}} - 0.053 \text{HR}\_{\text{MT}} \\ &- 0.086 \text{HR}\_{\text{afterMT}} - 0.019 \text{HF}\_{\text{beforeMT}} + 0.012 \text{HF}\_{\text{MT}} \\ &+ 0.007 \text{HF}\_{\text{afterMT}} + 0.094 \text{LF} / \text{HF}\_{\text{beforeMT}} \\ &- 0.087 \text{LF} / \text{HF}\_{\text{MT}} + 0.312 \text{LF} / \text{HF}\_{\text{afterMT}} \end{aligned} \tag{3}$$

The derived logistic sigmoid function log *<sup>p</sup>* 1− *p* was sta-

tistically significant (*p* < 0.001). The Nagelkerke's *R*<sup>2</sup> and Cox and Snell's *R*<sup>2</sup> values were 0.614 and 0.461, respectively, which indicate that the derived logistic sigmoid function was useful in discriminating patients with MDD. The ROC analysis was performed to find the optimal cutoff point of predicted LRA logit score to discriminate the patients with MDD from healthy subjects (**Figure 3**). According to the ROC, the AUC for this screening test was 0.891 (95% confidence interval, 0.828–0.954), which indicated that the LRA was able to significantly distinguish between patients with MDD and the healthy group. In addition, a sensitivity of 80.0% and specificity of 79.0% were obtained from an optimum cutoff point of LRA logit score (0.28). Furthermore, a scatter plot was used to compare the results of our HRV-based objective screening method with subjective SDS scores (**Figure 4**). With this cutoff point, a total of 35 patients with MDD were predicted as patients with MDD by the LRA; 9 patients with MDD were misdiagnosed as healthy. Similarly, a total of 37 healthy control subjects were predicted as healthy control subjects; 10 healthy control subjects were misdiagnosed as patients with MDD. The corresponding PPV and NPV of the LRA were 78.0 and 80.4%, respectively. The gray area highlights the boundary region between patients with MDD and healthy control subjects, which demonstrates that the 9 false negatives and 10 false positives occur at the boundary region. In contrast, the sensitivity, specificity, PPV, and NPV were 72.0, 85.1, 82.0, and 76.9%, respectively, for the SDS score using a cutoff of 50. Thus, our HRV-based objective method using LRA significantly improved sensitivity and NPV compared with the SDS score alone. Logit scores also correlated significantly with subjective SDS score (*r* = 0.4, *p* < 0.05). Moreover, the HRV and HR indices were compared with the MDD and healthy groups. **Figure 5A** shows that the HR of before MT differed significantly among MDD and healthy groups (*p* < 0.05), whereas no significant difference are found on MT and after MT conditions. **Figure 5B** shows that the HF of before MT and after MT conditions differed significantly among MDD and healthy groups (*p* < 0.05), but no difference was present on MT conditions. **Figure 5C** shows that the LF/HF of before MT and after MT conditions differed significantly among MDD and healthy groups (*p* < 0.05), but no difference on MT conditions. **Table 1** summarizes a more detailed comparison between our HRV-based objective method and patient-reported subjective SDS scores in part. In order to

better understand the data, we sorted the table in descending order on logit scores. We found that the misclassified individuals gathered at close to the cutoff on both the patient-reported subjective SDS scores and HRV-based objective logit scores (gray areas). Moreover, the HRV-based screening method reduced both false negative and false positive error rate, comparing to the SDS method.

#### DISCUSSION

We proposed an objective, non-invasive, HRV-based screening method for clinical risk stratification of patients with MDD using LRA. This method performed better than a subjective patientreported screening method to discriminate patients with MDD. Existing diagnosis methods rely solely on the psychiatric clinician's impression of the clinical presentation, which has minimal objectivity (23). A recent review reported that the accuracy of patient-reported and symptom-based diagnosis of MDD was approximately 47% (24). In this study, we obtained a similar result using patient-reported subjective SDS scores only. In contrast, our HRV-based objective method significantly improved screening accuracy.

Heart rate variability, which is a biomarker that reflects alterations in autonomic reactivity, is related to disturbed

MDD by the logistic regression analysis (LRA); 9 patients (red triangles) with MDD were misdiagnosed as healthy. Similarly, a total of 37 healthy control subjects (green circles) were predicted as healthy control subjects; 10 healthy control subjects (red circles) were misdiagnosed as patients with MDD. The gray area highlights the boundary region between patients with MDD and healthy control subjects, which demonstrates that the 9 false negatives and 10 false positives occur at the boundary region.

before, during, and after a mental task (MT) were compared within the MDD and healthy groups. (A) HR of before MT differed significantly among MDD and healthy groups (*p* < 0.05), whereas no significant difference are found on MT and after MT conditions. (B) HF of before MT and after MT conditions differed significantly among MDD and healthy groups (*p* < 0.05), but no difference was present on MT conditions. (C) LF/HF of before MT and after MT conditions differed significantly among MDD and healthy groups (*p* < 0.05), but no difference on MT conditions.

arousal function and can be used as a diagnostic measure and in clinical evaluation. In our previous studies, individual HRV and HR were shown to vary between depressed and non-depressed subjects (17, 18). However, there were significant data overlap between the two groups. The need to differentiate between patients with MDD and healthy control subjects led to the present study, which combined information from individual HRV and HR variables in LRA to improve screening accuracy in an automated way. The logit score output of this LRA can help psychiatric clinicians to make better diagnostic judgments. The logit score of the LRA can also be used to evaluate the severity of depression, and thereby aid the clinical risk stratification of patients with MDD.

The main limitation of this study was that our HRV dataset contains a total of 91 subjects, which is small compared to typical datasets from the field of classification of medical data and information. This is a limiting factor to the accuracy of Table 1 | Objective logit scores based on heart rate variability (HRV) compared with patient-reported subjective self-rating depression scale (SDS) scores.


our proposed MDD screening method. In statistical viewpoint, the dichotomization may result in the lost of information, so the performance to detect the relation between the variables and output will also be limited (25). However, in this study from the clinical viewpoint, dichotomization may be preferred because which offers a simple risk classification in setting screening criteria. In order to address these limitations, we plan to extend the current study to include a larger sample of clinical data for creating a more accurate logistic regression-based discriminant function. It is also important in the future study to apply the regression coefficients in the LRA on the present dataset to the data from newly recruited control subjects and patients with depression in order to consolidate the findings.

In addition, the MT paradigm is a provocative procedure used to differentiate between MDD and healthy subjects. The subject generates his or her own numbers in a random way for 100 s. This subject-generated method is a limitation that might create unintended subject-to-subject variability based on individual motivation, cooperation, and perhaps even current mood state. Therefore, further research should be done to investigate the subject-to-subject variation in this MT and apply other biomarkers, such as functional near-infrared spectroscopy, to measure the brain activity during subject generating random numbers. Moreover, in this study, we measured HRV and HR indices *via* ECG that requires electrodes to be attached to patients and can cause discomfort. We will expand this screening method to incorporate less invasive methods for HRV measurement, such as photoplethysmogram, microwave radar, or smart phone monitoring (26–28). These bio-measurement technologies, which can be combined with our proposed screening method without the need for electrode placement, allow for fully unconstrained HRV measurements to rapidly screen patients with MDD.

## CONCLUSION

In summary, the objective, non-invasive, HRV-based screening method developed in this study is effective in discriminating patients with MDD, which enables a higher screening sensitivity of 80.0% compared to the conventional subjective patient-reported screening method. Overall, the proposed method, which is based on a comprehensive analysis of HRV indices, has high potential for use as a novel objective tool in screening patients with MDD.

#### REFERENCES


## AUTHOR CONTRIBUTIONS

Study concept and design: GS, TS, TM, and TK. Acquisition of data: TS. Analysis and interpretation of data: GS, TS, and TM. Drafting of the manuscript: GS, TS, and TM. All the authors gave final approval of the version to be submitted.

## FUNDING

This study was supported by a Grant-in-Aid for Young Scientists (16K16363) founded by the Ministry of Education, Culture, Sports, Science and Technology of Japan.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Sun, Shinba, Kirimoto and Matsui. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Changes in Heart Rate Variability Recorded in Natural Situation with T-Shirt Integrated Sensors and Level of Observed Behavioral Excitation: A Pilot Study of Patients with Intellectual Disabilities and Psychiatric Disorders

*Julie Palix1,2, Michel Akselrod3 , Charly Cungi4 , Fabienne Giuliani2 and Jérôme Favrod1,2\**

*1School of Nursing Sciences, La Source, University of Applied Sciences and Arts of Western Switzerland, Lausanne, Switzerland, 2Department of Psychiatry, Centre Hospitalier Universitaire Vaudois, Lausanne, Switzerland, 3Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, 4Clinique Belmont, Geneva, Switzerland*

#### *Edited by:*

*Wolfram Kawohl, University of Zurich, Switzerland*

#### *Reviewed by:*

*Till Beiwinkel, Lüneburg University, Germany Sebastian Olbrich, University of Zurich, Switzerland*

> *\*Correspondence: Jérôme Favrod j.favrod@ecolelasource.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 31 May 2016 Accepted: 11 January 2017 Published: 01 February 2017*

#### *Citation:*

*Palix J, Akselrod M, Cungi C, Giuliani F and Favrod J (2017) Changes in Heart Rate Variability Recorded in Natural Situation with T-Shirt Integrated Sensors and Level of Observed Behavioral Excitation: A Pilot Study of Patients with Intellectual Disabilities and Psychiatric Disorders. Front. Psychiatry 8:4. doi: 10.3389/fpsyt.2017.00004*

Background: The present study investigates the possibilities of using heart rate variability (HRV) parameters as physiological markers that precede increase in observed behavioral excitation of intellectually disabled individuals. The ability to recognize or predict such patterns, especially in patients showing unpredictable reactions and language deficiencies, might be a major step forward in clinical research.

Method: Thirteen volunteers with intellectual disabilities, who had suffered of at least one event of overt aggression in the preceding 3 months, participated to the study. The protocol consists in the acquisition of continuous electrocardiogram (ECG) throughout approximately two times of 8 h in natural situation, using a T-shirt integrated with sensors. Simultaneously, an observer evaluates the patient's level of overt excitation from calm (level 1) to extremely tense (level 5) and send online *via* Bluetooth these triggers into the ECG signals. The HRV indexes were then estimated offline on the basis of the inter-beat intervals recorded by the ECG, independently for the 30 min preceding each behavioral tension marking point, averaged, and compared through non-parametric Wilcoxon matched-pairs test. Of these, the RMSSD and LF/HF calculations were used to observe the fluctuations of inhibitory activity and cardiovagal balance through different tension states.

Results: Seven individuals have sufficient reliable data for analysis. They have reached at least a level 3 of behavioral excitation (moderately tense) or more (very to extremely tense, level 4 and 5) and have been retained for further analysis. In sum, a total of 197 periods of tension were kept, made up of 46 periods of slight excitation (level 2), 18 of moderate excitation (level 3), 10 of high excitation (level 4), and 5 of extreme agitation (level 5). Variations in the HRV as a function of degree of excitation are observed for RMSSD index only (inhibitory parasympathetic activity). The changes from calm to increasing levels of excitation are characterized by a significant downfall in RMSSD index when patients were evaluated to be in a very high level of tension (level 4).

Conclusion: The presence of precursors to agitation, reflected in the falling-off of parasympathetic activity, offers potentially interesting prospects for therapeutic development.

Keywords: heart rate variability, parasympathetic nervous system, sympathetic nervous system, intellectual disability, psychological stressors

## INTRODUCTION

The present study is particularly aimed at a population combining mental retardation and psychiatric co-morbidity, with relatively low intelligence quotient (1, 2) and maladaptive behavioral reactions (3). Indeed, such characteristics can lead to inadequate responses to changes in the individual's environment. The frequency of aggression and abnormal episodes (e.g., selfmutilation, anger, verbal or physical aggression toward others, or property damage) can be a challenge with this population, with a prevalence of 4–10% depending on the country and the duration of observations (4–7). The unpredictability of these behaviors and the deficiencies in individuals' abilities to communicate their feelings before these episodes take place make them particularly dangerous to themselves and the others.

A person's ability to deal with situational changes is made possible through the dynamics of his autonomic nervous system (ANS). The ANS either activates or inhibits the body's system of vigilance through the innervations of the myocardium. This occurs *via* the sympathetic nervous system (SNS) when activating and *via* the parasympathetic nervous system (PNS) when inhibiting. Depending on the psychiatric, psychological, or emotional conditions that an individual is facing, the ANS responses to the demands of their environment could be expressed by in an increase in SNS activity, a decrease of PNS control, or a combination of both (8–11). These autonomic functional differences are described as representative of different emotional state and social position, anxious, and submissive on the one hand with a key behavioral inhibition through the parasympathetic branch (12–16), and hostile and dominant on the other hand, requiring a rapid influx of blood to muscle and the brain through sympatheticadrenaline activity, which may of may not occur with a lowering of parasympathetic inhibition (17, 18). The ability to recognize or predict of such patterns by examining a patient's cardiovagal balance, especially in patients showing unpredictable reactions and language deficiencies, might be a major step forward in clinical research to help them. In addition, this type of data might help to clarify the concept of intentionality in the unexpected explosive behavior exhibited by this population.

Educational monitoring such as operant conditioning or applied behavior analysis (19, 20) and breathing control (21, 22) have shown encouraging results in attempts to defuse antisocial behavior and mood fluctuations in long term therapies. A means of recognizing the imminence of aggressive reactions and exploiting that recognition in order to avoid such unpleasant interventions as the seclusion, restraint, or antipsychotic medication of people with an intellectual disability, appears not only essential but also rational and ethical (23–26). Indeed, some heart rate and heart rate variability (HRV) fluctuations have been described to be predictive of reactive aggression (18, 27, 28). This study's primary goal was to identify markers of increasing physiological agitation—the indicators of imminent aggressive behavior.

#### MATERIALS AND METHODS

#### Participants and Methods

Study participants were recruited from patients followed by the mental development psychiatry mobile team of the Community Psychiatry Service at the University Hospital Center of Lausanne, Switzerland, between March and November 2012.

The inclusion criteria for participation in this study was at least one recorded episode of aggressive behavior as defined by the Retrospective Modified Overt Aggression Scale (5, 29, 30) in the three months prior to the study (verbal or physical aggression against others, oneself, or property). Each participant wore a T-shirt with integrated sensors developed to record electrocardiograms (Smartex s.r.l., Giuntini 13L Navacchio, Pisa 56023, Italy). This apparatus was designed specifically for the assessment of physiological information in an everyday environment, and it has been successfully tested for the management and study of pathologies such as heart diseases, diabetes, or bipolar disorders (21, 31). A T-shirt was put on after the participant's morning shower (approximately 9:00 a.m.) and was taken off in the evening (approximately 6:00 p.m.). The T-shirt was worn on two non-consecutive days within 1 week, in order to validate the reliability of the data gathered. Data from the high resolution electrocardiogram (ECG) signal, with sampling every 4 ms (32), were recorded in a device held in the T-shirt's pocket (CSEM, Neuchâtel, Switzerland). While the ECG signal was recorded continuously, each participant's "level of excitation" was estimated every half hour by a staff member. Three potential participants refused to be included and two abandoned the study by declining to participate on the second day (see flowchart on **Figure 1**). Of the remaining 11 subjects, those with a reliable ECG signal for more than 85% of the day were retained, giving a final total of seven participants. The main reason for ECG signal failure was identified as lack of shirt adhesion on the participant's chest (e.g., because of a BMI below 18, skin diseases, agitation). The final participant group comprised of three women and four men. The average age was 32 years old (SD ± 7.3, min. 28,

max. 47). Six participants lived in sheltered accommodation and one lived with parents, three worked in sheltered workshops, six participants took antipsychotic medication, and three took mood stabilizers. As psychoactive substances are recognized as being unrelated to systematic effects on the autonomic system (33, 34), no major recurrence was expected in HRV of our participants as a result of drugs intake. None of the patients were taking a betablocker, but one was taking medication for high blood pressure. The seven participants had IQ levels between 35 and 40 to 50 and 55. This study was carried out in accordance with the recommendations of the Human Research Ethics Committee of the Canton de Vaud, Switzerland, and the Declaration of Helsinki. All subjects gave informed written consent to their participation in the study, with the agreement of their legal representative and their family.

## Excitation Levels

During the 8 h of ECG recording, the participant's level of excitation was estimated by a member of staff every 30 min. A ringtone on a smart phone linked to the recording device *via* Bluetooth reminded the staff member to evaluate the patient. Each participant was evaluated by a single, different evaluator on each day, providing observations of noticeable behaviors representative of different levels of excitation. An observation grid was simplified into five highly distinguishable excitation levels in order to limit inter-evaluator discrepancies: 0, the patient is asleep; 1, the patient is in a calm state; 2, the patient is slightly excited; 3, the patient is moderately excited; 4, the patient is very excited; and 5, the patient is extremely excited. The ringtone acted as a time-based marking point, integrated to the participant's continuous ECG recording *via* Bluetooth, which could then be matched to each subject's clinical manifestations of excitation in order to select and analyze of their individual physiological signals corresponding to the clinical manifestations of each subject.

#### HRV Measurement

Heart rate variability was estimated independently for the 30 min preceding each behavioral excitation marking point, using the Kubios computing program (Biosignal Analysis and Medical Imaging Group, Department of Physics, University of Kuopio, Finland (35, 36); for an analysis of beat-to-beat variations, and Matlab (MATLAB version 7.10.0, Natick, MA, USA: The MathWorks Inc., 2010) to compute averages for each segment. The variations in beat-to-beat intervals (NN intervals) were extracted and re-sampled using spline interpolation and analyzed in accordance with international guidelines (9, 36–39). Calculations included (1) the square root of the mean of the sum of the squares of the successive differences between adjacent NNs (RMSSD, in milliseconds) and (2) the power of the low frequencies (LF: 0.04–0.15 Hz) and the high frequencies (HF: 0.15–0.4 Hz) after a fast Fourier transform and normalized using logarithmic transformation to calculate the ratio of the low over the high frequencies (LF/HF). The RMSSD index is representative of the inhibitory parasympathetic component of the ANS, whereas the LF/HF ratio is considered to be an indicator of the sympathovagal balance representing an increased in activity in the SNS and/or a reduction in activity in the PNS (9). These two HRV indices were chosen according to previous studies in psychiatric disorders or fragile psychological and emotional states with significant variations (40–43). Furthermore, these indices are frequently used in applied research as shown for mood disorders and emotionalstate recognition (21, 22).

#### Statistical Analysis

The HRV indices RMSSD and LF/HF have been averaged (mean ± SE) and normalized for each individual (*z*-scores procedure of Statistica, independent across individual data). This procedure is used to shade off individual baseline differences in physiological arousal and thus to improve the examination of fluctuations between tension levels.

#### RESULTS

Seven individuals reached at least a level 3 of overt excitation (moderately tense) or more (very to extremely tense, levels 4 and 5). Among all the marked time periods picked up from the ECG signals, a total of 197 was retained after removing noisy or sleep episodes. In sum, the data of overt tension levels are made up of 46 periods of slight excitation (level 2), 18 of moderate excitation (level 3), 10 of high excitation (level 4), and 5 of extreme agitation (level 5). The detailed contribution of each subject for every tension level is provided in **Figure 2**.

The physiological values in the calm state (over the seven participants) were measured at 83 b.p.m. (±2 SE) for heart rate frequency, 2.5 (±0.2 SE) for LF/HF ratio, and 68 (±14 SE) for RMSSD index. These estimations are normal to above the expected standard (32, 44).

#### HRV Fluctuations and Excitation Levels

**Figure 3** shows the variations in the RMSDD variables as a function of degree of excitation. Differences between tension levels are observed for RMSSD index only (inhibitory parasympathetic activity). The change from calm to increasing levels of excitation is characterized by a gradual downfall in RMSSD index with a

maximum reduction before the extreme level of tension. Only five occurrences of extreme excitation concerning three participants were observed in this pilot study. The extreme occurrence was measured after outburst. The LF/HF remains unchanged through the different levels of excitation.

## DISCUSSION

This study's primary goal was to identify markers of increasing physiological agitation that might reveal imminent outburst, particularly in a group of patients with intellectual disabilities and associated behavioral disturbances. By recording continuous ECGs over long periods of time and evaluating the patient's level of excitation every 30 min, it seems that we identify fluctuations in HRV that might be specific to the moments prior to an increase in the level of excitation. The results of this small pilot study require additional, more in-depth study before they can be confirmed. Nevertheless, we were able to show that the LF/ HF ratio did not increase proportionally with excitation. Instead, the index of parasympathetic activity, RMSSD, decreased as the patient's state changed from slightly tense (level 2) and very tense (level 4), and increased just when reaching extreme excitation (level 5). The increase of RMSSD at level 5 could be explained by the fact that level 5 was utilized when outburst occurred for the three participants concerned. The resolution phase of the outburst might lead to this increase. Thus, this pilot study demonstrated that fluctuations in HRV do not necessarily follow a linear relationship with the observed level of excitation. It also showed that this simple, non-invasive method of collecting ECG data may offer many advantages in efforts to identify changes in physiological arousal prior to potentially aggressive behavior.

Our results show that the RMSSD index measured for the tested population is normal to above the expected standard (32, 44). This normal-to-over-activation of parasympathetic activity discards suggestions that such population exhibit functional parasympathetic failure (12, 13, 45, 46). The deficiencies in behavioral adaptation observed in intellectually impaired subjects could thus be the result of a heightened general state of excitation, with a final breakdown in control when stress reaches an extreme level (30).

Overall, our results seem to suggest that the dominance of the parasympathetic branch of the ANS might demonstrate a position of subordination and a fear of losing control (16, 47, 48). This is similar to recent descriptions related to aggression in fragile X syndrome (49), or brain oscillations of this intellectually disabled population when confronted with emotionally negative images (50). The inhibitory parasympathetic branch appeared late in mammalian evolution in order to suppress the strong fight-or-flight reactions resulting from SNS accelerator inputs (9). Controlling the inhibitory parasympathetic branch is described as promoting interpersonal exchanges and developing social connections (51). According to this pilot study's results, this nervous inhibitory pathway is mainly activated in individuals with intellectual disabilities and psychiatric conditions, offering new perspectives for therapies that put the emphasis on dealing with anxiety rather than aggressiveness (52). Specific intervention aiming at improving heart rate coherence might be useful. A pilot study using training in heart coherence was tested with mild intellectual disability volunteers working in sheltered workshops. Training in cardiac coherence was implemented during two consecutive weeks. Results indicated that participants could benefit physiologically, psychologically of both (53). However, these results are preliminary and need more controlled validation.

The main limitations of this study are the small sample of participants and the loss of participants due to failure in signal acquisition. The small number of participants prevent from finding a statistically significant result. The ideal design would require a continuous video recording of the behavior of participants to segment signals in realistic fitting periods, but this is difficult to achieve in natural conditions without limiting the freedom of the participants.

In conclusion, the data gathered in this study of a group of patients with intellectual disabilities and psychiatric co-morbidity

#### REFERENCES


showing overt aggressive behavior suggests that anxiety-based reactions predominate above hostile aggressive behavior. Furthermore, a decreased in parasympathetic activity during tense condition emerge to be a marker to imminence of agitation.

#### ETHICS STATEMENT

Commission cantonale d'éthique de la recherche sur l'être humain. Secrétariat administrative, Avenue de Chailly 23, 1012 Lausanne. This study was carried out in accordance with the recommendations of the Human Research Ethics Committee of the Canton de Vaud, Switzerland, and the Declaration of Helsinki. All subjects gave informed written consent to their participation in the study, with the agreement of their legal representative and their family.

#### AUTHOR CONTRIBUTIONS

JF, CC, FG, and JP contributed substantially to the conception and design of the study. JF and FG contributed substantially to data acquisition; JF, MA, and JP contributed substantially to the analysis and interpretation of data. JF and JP drafted the article. MA, CC, and FG revised it critically for important intellectual content. All authors approved the version to be published and agree to be held accountable for all aspects of the work, ensuring that questions related to the accuracy or integrity of any part of it are appropriately investigated and resolved.

#### FUNDING

The study was supported by a grant from Réseau d'études appliquées des pratiques de santé, de réadaptation/réinsertion (Ré-Sa-R) and a donation from Dr. Alexander Engelhorn.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer SO and handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

*Copyright © 2017 Palix, Akselrod, Cungi, Giuliani and Favrod. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Development and Clinical Evaluation of an mHealth Application for Stress Management

*Brent D. Winslow1 \*, George L. Chadderdon1 , Sara J. Dechmerowski1 , David L. Jones2 , Solomon Kalkstein3 , Jennifer L. Greene3 and Philip Gehrman3*

*1Design Interactive, Inc., Orlando, FL, USA, 2Quantified Design Solutions LLC, Orlando, FL, USA, 3Philadelphia VA Medical Center, University of Pennsylvania, Philadelphia, PA, USA*

A large number of individuals experience mental health disorders, with cognitive behavioral therapy (CBT) emerging as a standard practice for reduction in psychiatric symptoms, including stress, anger, anxiety, and depression. However, CBT is associated with significant patient dropout and lacks the means to provide objective data regarding a patient's experience and symptoms between sessions. Emerging wearables and mobile health (mHealth) applications represent an approach that may provide objective data to the patient and provider between CBT sessions. Here, we describe the development of a classifier of real-time physiological stress in a healthy population (*n* = 35) and apply it in a controlled clinical evaluation for armed forces veterans undergoing CBT for stress and anger management (*n* = 16). Using cardiovascular and electrodermal inputs from a wearable device, the classifier was able to detect physiological stress in a non-clinical sample with accuracy greater than 90%. In a small clinical sample, patients who used the classifier and an associated mHealth application were less likely to discontinue therapy (*p* = 0.016, *d* = 1.34) and significantly improved on measures of stress (*p* = 0.032, *d* = 1.61), anxiety (*p* = 0.050, *d* = 1.26), and anger (*p* = 0.046, *d* = 1.41) compared to controls undergoing CBT alone. Given the large number of individuals that experience mental health disorders and the unmet need for treatment, especially in developing nations, such mHealth approaches have the potential to provide or augment treatment at low cost in the absence of in-person care.

Keywords: stress, electrodermal response, heart rate, mobile applications, wearable devices, cognitive behavioral therapy, telemedicine

#### INTRODUCTION

Recent estimates suggest that approximately one-third of individuals globally will experience mental health disorders in their lifetime (1). Individuals in developing nations are particularly vulnerable (2, 3). In developed nations, an especially vulnerable population is military veterans. For example, approximately one-third of US military veterans suffer from some type of psychological distress, including post-traumatic stress disorder (PTSD), major depressive disorder (MDD), and suicidal ideation (4). Diagnostic and subthreshold levels of PTSD are associated with poor quality of life, including anger, stress, alcoholism, depression, poor physical health, and increased suicidality (5, 6), and cause impaired ability to function in social, educational, and work environments (7).

#### *Edited by:*

*Yasser Khazaal, University of Geneva, Switzerland*

#### *Reviewed by:*

*Ma Angeles Gomez Martínez, Pontifical University of Salamanca, Spain Renzo Bianchi, University of Neuchâtel, Switzerland*

*\*Correspondence:*

*Brent D. Winslow brent.winslow@designinteractive.net*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 13 June 2016 Accepted: 12 July 2016 Published: 26 July 2016*

#### *Citation:*

*Winslow BD, Chadderdon GL, Dechmerowski SJ, Jones DL, Kalkstein S, Greene JL and Gehrman P (2016) Development and Clinical Evaluation of an mHealth Application for Stress Management. Front. Psychiatry 7:130. doi: 10.3389/fpsyt.2016.00130*

Among various interventions to treat depression and anxiety, cognitive behavioral therapy (CBT) has emerged as standard practice for reduction of psychiatric symptoms (8), with previous studies indicating that CBT has similar therapeutic effects as anti-depressant medication (9). CBT is generally administered by mental health professionals and consists of a structured, collaborative process that helps individuals consider and alter their thought processes and behaviors associated with stress or anxiety, usually administered weekly over several months (10). However, standard CBT for stress and anxiety does not offer the provider information regarding therapeutic efficacy outside of office visits nor does it provide objective information about individuals' triggers, such as location, time, or severity (11). In addition, high dropout rates from CBT programs have been reported to span from 25% (12) to as high as 40% (13) for individuals suffering from depression. The limitations of CBT, including lack of objective data available for providers and high patient dropout rates, could be mitigated with emerging technologies. To support real-time objective stress monitoring in mental health treatment, wearable physiological sensors and associated mobile health (mHealth) applications (14) have the potential to quantify biological metrics associated with stress (15), support remote monitoring, and alert the wearer or provider to real-time changes in emotional state.

Existing approaches to stress detection use a wide array of features calculated from sensor data measuring various aspects of heartbeat, including pulse photoplethysmography (PPG) or ECG (15–17), skin conductance measurement (18–20), and measurement of respiration, all of which are responsive to increased sympathetic nervous system activity associated with stress (21). Standard supervised machine learning methods have been used previously to develop stress classifiers, which require subjects to engage in tasks known to induce stress so that stress or non-stress labels can be assigned to the input features. Previous work has emphasized the difficulties imposed on stress classification by individual subject variability in physiological responses to stress (16, 19). Another concern is the physical activity of subjects that triggers similar cardiovascular and electrodermal physiological signals as stress, leading to masking and confounds of stress detection (15, 19). The major challenge in using mobile physiological sensors to quantify stress is the lack of robust and clinically tested algorithms to classify stress in a mobile environment in real time (22).

Previous stress monitoring algorithms have been built with traditional laboratory physiological sensor suites that do not translate well to operational settings (17, 19), such as mental health treatment. New wearable devices with clinical grade sensors and associated mobile applications have the potential to take real-time stress monitoring outside of the laboratory. There is an opportunity to combine foundational mobile stress monitoring algorithm research methods with new mobile physiological sensor suites to create an accurate, quantitative classifier for continuous and objective real-time stress assessment. In the current study, we develop a physiological classifier of stress and apply it in a clinical evaluation of patients undergoing CBT for stress/anger management, following military deployment. It is hypothesized that stress induced using standard methods can be classified with high accuracy using a machine learning algorithm and that the use of such an algorithm in an mHealth application can reduce post-deployment psychiatric symptoms, including stress, anger, and anxiety, in a clinical population undergoing CBT.

#### MATERIALS AND METHODS

#### Classifier Study Participants

All methods involving participants were approved by a series of Institutional Review Boards [Copernicus Group IRB, Durham, NC, USA; Human Studies Subcommittee, Department of Veteran's Affairs, Philadelphia, PA, USA; US Army Medical Research and Material Command Human Research Protection Office (HRPO), Fort Detrick, MD, USA].

Thirty-five participants (24 males; average age 25.7 ± 6.2 years) were recruited for the initial classifier-development study, which lasted approximately 1.5 h. Participants were recruited using recruitment flyers posted online and through recruitment fairs at local universities.

#### Experimental Procedure

Upon arrival, participants provided written-informed consent and completed a series of questionnaires including: demographics; the subjective units of distress scale (SUDS); the depression, anxiety, and stress scale (DASS); and the patient-reported outcomes measurement information scale (PROMIS) anger scale. Wireless physiological sensors were then placed on the participants, followed by a 5-min recording of baseline physiological activity, while participants remained seated. Participants then completed the Trier Social Stress Test [TSST; Ref. (23)]. The TSST was used to elicit physiological stress, consisting of 5 min each of: preparatory anticipatory stress (TSST-P); oral speech (TSST-S); and mental arithmetic (TSST-A). Following data acquisition, participants were debriefed and thanked for their participation. A subset of participants (*n* = 7) also provided a saliva sample *via* passive drool for cortisol assessment at baseline and following the TSST.

#### Qualitative Measures

Participants in the classifier study responded to the SUDS, in which they reported their current level of stress on a scale of 0–100, with 0 indicating that they were completely relaxed and 100 indicating that they were experiencing severe stress (24). Participants then completed the DASS, designed to assess current depression, anxiety, and stress using responses to 21 statements (25). Respondents indicated the degree to which each statement has been true for them over the preceding week on a 4-point Likert scale. Participants also responded to the PROMIS anger scale, which consists of an 8-item measure on which respondents indicate the frequency of each item from the past week on a 5-point Likert scale (26).

#### Physiological Measures

The Biopac MP-150 system (Goleta, CA, USA) was used for collection of physiological data. Participants were fitted with PPG at the non-dominant thumb and electrodermal activity (EDA) on the fourth and fifth fingers of the non-dominant hand, with band limits set between DC and 10 Hz. All physiological data were sampled at 1000 Hz and wirelessly sent to an MP-150 system running AcqKnowledge software (Biopac Systems, Goleta, CA, USA). Following data collection, PPG data were downsampled to 64 Hz and EDA was downsampled to 4 Hz. Heart rate (HR) was calculated from the R–R interval from the PPG signal, with intervals <40 and >180 bpm excluded from the analysis. Salivary cortisol was measured by standard ELISA (Salimetrics, Carlsbad, CA, USA; intra-assay CV = 4.5%, inter-assay CV = 5.8%).

In a subset of participants (*n* = 8), the Empatica E3 sensor was also used for physiological data collection. A second system was used to ensure the stress algorithm was compatible across multiple hardware solutions and to provide for mobile stress classification in future studies. Physiological data, consisting of PPG (64 Hz) and EDA (4 Hz), were transmitted *via* Bluetooth 4.0 to a custom mobile application for data collection implemented in the Android OS on a Samsung Galaxy S4 phone.

#### Classifier Development

Event times and physiological data were stored in Biopac.acq files. All data were read into Python analysis scripts running under the Enthought Canopy environment. The numpy, scipy, pandas, and matplotlib libraries were used for feature extraction and data analysis (27), and the scikit-learn library (28, 29) was used for classifier development. Visual inspection of the raw data in the Biopac software and the interactive Python environment was used to discard physiologically noisy or missing participant data.

From the raw data, non-overlapping 1-min windows were analyzed to yield feature vectors for the minute blocks. Inter-beat intervals (IBI) were extracted from the PPG data using a signal derivative-based algorithm (30). For minutes with less than 40 valid IBI samples, the block of data was discarded; for remaining blocks, the mean IBI was calculated. For each valid IBI block, the mean HR was estimated by dividing 60 by the IBI mean. For the EDA data, the mean was taken over the minute's raw data. The HR and EDA means were then normalized separately for each participant by subtracting the average of the 5-min baseline.

Matplotlib boxplots and scatterplots were used to explore the distributions of the task-specific patterns (e.g., baseline and TSST-S) in feature space. A stress vs. non-stress classifier was trained using baseline vs. TSST-S, using baseline-normalized mean HR and EDA features. A 2-feature linear model classifier was trained and tested on the E3 dataset using stochastic gradient descent. The train:test (75:25%) set consisted of data feature vectors taken from the baseline and TSST-S minutes of the participants with E3 data. Five-fold cross-validation was implemented to evaluate the average performance of the algorithm on the train set: this set was divided into fifths and, iteratively, one of the five blocks was left out for testing and the other four were used to train the classifier. While the cross-validation measures were used to compare performance of different learning algorithms (e.g., stochastic gradient descent vs. support vector machines), the training and testing accuracies were calculated from performance from training on the full train set. Performance of the E3 data-trained model was also measured using the entire Biopac data set with good HR and EDA baseline and TSST-S minute data as a test set.

#### Data Analysis and Statistics

Classifier training accuracy was defined by signal detection theory (31) (hit = classifier correctly identified spike stress state; miss = classifier missed a spike in stress state; false alarm = classifier identified a spike in stress state when one did not actually occur; and correct rejection = classifier did not identify a spike in stress state when one did not occur). Accuracy was defined as the ratio of the sum of hits and correct rejections over the sum of minute blocks classified as either stressed or non-stressed. The hit rate was defined as the number of hit minute blocks divided by the total number true stress blocks (TSST-S), and the false-alarm rate was defined as the number of false-alarm blocks divided by the total number of true non-stress blocks (baseline). SUDS scores between baseline and the TSST were analyzed using paired samples *t*-tests.

## Clinical Evaluation

#### Participants

Following development and evaluation of the stress classifier, 16 participants [13 males; average age 39.8 ± 10.5 (SD)] were enrolled for participation in the clinical evaluation study of the classifier and associated mHealth application, which lasted 8–10 weeks for each individual. Participants were recruited from patients at the Philadelphia VA Medical Center who reported significant difficulties with anger and/or stress and indicated a willingness to participate in a research study. Exclusion criteria consisted of: currently active duty military; moderate or severe TBI; severe mental impairment as assessed in their electronic medical record; and/or functional limitations preventing use of a mobile device.

#### Experimental Procedure

Following written-informed consent, participants were randomly assigned to the experimental or control group and completed the DASS, PROMIS-Anger, and PTSD Checklist-Military (PCL-M) questionnaires. The control group (*n* = 6) received standard CBT; the experimental group (*n*= 10) received standard CBT integrated with the stress classifier and mobile application (see Physiological Measures). Following initial assessment, an appointment was made to begin treatment within 1–2 weeks. All treatment was administered in an individual format by the study clinicians, who are licensed mental health professionals. The study clinicians were directed to use standard CBT treatment manuals (32) as a foundation for CBT while using clinical judgment to determine what content to cover in each session and how many sessions to schedule. Typical treatment following this approach was expected to last 8–10 weeks. This approach was chosen rather than utilizing a fixed protocol in order to represent routine clinical practice.

Sessions involved a weekly, in-person meeting, which lasted 60 min. Patients were asked to keep a log of daily activities, summarizing key stress/anger events that occurred, and present this written report to therapists during their session. Weekly sessions continued until: (a) the participant and clinician jointly determined that there was significant clinical improvement; (b) it was judged by the therapist that no further improvement was likely to occur; or (c) the participant discontinued therapy. Compliance in the experimental group was quantified by use of the mobile application. One month following the completion of treatment, participants were asked to return for a followup visit to complete the DASS, PROMIS Anger, and PCL-M questionnaires.

#### Qualitative Measures

Participants completed the DASS and PROMIS anger scale at their initial assessment and following therapy completion; the DASS-Stress and PROMIS anger scales were considered primary outcome metrics. Participants in the clinical study also completed the PCL-M, which is a 17-item continuous severity measure that corresponds to the 17 DSM-IV criteria for PTSD (33). Respondents indicated the extent to which they had experienced each symptom described in the past month using a 5-point scale, from 1 (not at all) to 5 (very often). The PCL-M was considered a secondary outcome metric in the clinical study.

#### Physiological Measures

An mHealth application and stress classifier were used for data collection in the clinical study. The mHealth application was implemented in Android on a Samsung Galaxy S4 phone and received data from the E3 band (Empatica, Milan, Italy), classified stress using the algorithm developed in the classifier study, alerted the user when stress was detected, and presented stress mitigation techniques to the user, such as breathing exercises. The E3 band sent PPG, EDA, temperature, and accelerometer information to the mobile application *via* Bluetooth 4.0. A web-based provider portal that resided on a secure cloud server was also implemented and allowed the provider to view physiological data for individual patients and enter reminders (e.g., complete your cognitive restructuring homework) or focus points (e.g., practice breathing), which were sent to the mobile application.

#### Data Analysis and Statistics

Non-parametric statistical analysis was used to compare within groups measures across the two timepoints (initial and final assessment) and consisted of Wilcoxon signed-ranks tests with significance set to 0.05. Between groups differences were assessed using Mann–Whitney *U* tests with significance set to 0.05. All statistical testing was done in SPSS software version 18.

#### RESULTS

#### Classifier Study

The sociodemographic factors in the initial classifier study are listed in **Table 1**. The average age of the participants was 25.7 ± 6.2 (SD) years, and participants had an average of 3.4 ± 2.0 (SD) years of post-secondary education.

Subjective Units of Distress Scale scores are shown in **Figure 1**. As compared to baseline, the TSST elicited a significant increase in perceived stress (*p* < 0.001, *d* = 1.44). Baseline cortisol in the Table 1 | List of sociodemographic factors in the classifier study sample.


Mean + SD shown. \**p* < 0.001.

Table 2 | Mean (SD) DASS scores in the classifier-development group.


study subset was 0.39 ± 0.33 (SD) μg/dL, which did not differ following the TSST at a level of 0.44 ± 0.39 (SD) μg/dL (*p* = 0.29, *d* = 0.16) *via* paired samples *t*-test.

Depression, anxiety, and stress scale scores are shown in **Table 2**. Stress, depression, and anxiety scores were considered normal (25). The PROMIS anger score was 50.1 (7.2).

Noise or data loss affected 4/35 participants' physiological data, which were removed from analysis. Each task phase in the experiment was regarded as having distinct ground truth values for whether the participant would be considered stressed or not stressed. The TSST-S and TSST-A phases of the TSST were considered to be psychological stress phases. The baseline resting task was considered to be a non-psychological stress phase. For the TSST-P task no assumption of stress vs. non-stress was made. **Figure 2** shows the distributions of (non-normalized) HR (left) and skin conductance (right) data vs. task for all participants.

Based on the distributions, baseline-normalized HR and EDA means were used for stress vs. non-stress classification. **Figure 3** shows the classification results of training the stress vs. non-stress classifier using 75% of the E3 physiological data. For the E3 data,

Figure 2 | Task-dependent heart rate and skin conductance measures across participants. (Left) Includes heart-rate estimate distributions; TSST-S, TSST-A, have notably high HR distributions, whereas baseline tends to be low. (Right) includes electrodermal activity estimate distributions; the baseline conductance is relatively low, whereas TSST-S and TSST-A, are relatively high. Shown are group means ± SD.

to the left of this boundary were classified as non-stress.

the training accuracy was 97.1%. Test set accuracy on the remaining 25% of the data were 91.7%. The hit rate on the test set was 100%, and the false-alarm rate was 12.5%. For the Biopac data, the testing accuracy was 95.1%, the hit rate was 89.1%, and the false alarm rate was 1.7%.

#### Results from Clinical Assessment

The sociodemographic factors in the clinical evaluation are listed in **Table 3**. The average age of the participants was 39.7 ± 10.5 (SD) years and most were US Army veterans.

Nine individuals in the study dropped out prior to completion of therapy and follow-up visit. A Mann–Whitney test indicated that individuals in the experimental group completed a significantly greater number of therapy sessions (*p* = 0.016, *d* = 1.34) at an average of 7.2 ± 1.6 (SD) sessions as compared to 3.4 ± 2.4 (SD) in the control group. The remaining participants that completed the study included five in the experimental group and two in the control group. One participant in the experimental group that completed the study did not use the mHealth application but completed standard CBT and was reassigned to the control group.

Depression, anxiety, and stress scale scores are shown in **Table 4**. For the initial assessment, stress and depression for the Table 3 | List of sociodemographic factors of study sample.


participants was in the 96th percentile, and anxiety was in the 99th percentile as compared to a normative sample (34). Anxiety scores were considered extremely severe, while stress and depression scores were in the severe range (25). No differences between

#### Table 4 | Mean (SD) DASS assessment scores.


*a indicates significant difference between groups.*

the control and experimental group were observed during the initial assessment for stress scores (*p* = 0.616, *d* = 0.21), depression (*p* = 0.964, *d* = 0.09), or anxiety (*p* = 0.682, *d* = 0.29) as assessed by Mann–Whitney testing.

The follow-up assessment was completed by four participants in the experimental group and three participants in the control group. A significant reduction in stress (*p* = 0.032, *d* = 1.61) and anxiety (*p* = 0.050, *d* = 1.26) was observed between the experimental and the control group but not for depression (*p* = 0.719, *d* = 0.29) per Mann–Whitney testing. Within groups, the control group showed no significant changes in DASS scores for stress (*p* = 0.593), anxiety (*p* = 0.109), or depression (*p* = 1.000) between the initial assessment and follow-up as assessed by Wilcoxon signed-ranks tests. The experimental group trended toward a significant decrease in stress (*p* = 0.068) and depression (*p* = 0.068) between timepoints but not anxiety (*p* = 0.144) as assessed by Wilcoxon signed-rank tests.

Patient-reported outcomes measurement information scale anger scores are shown in **Table 5**. No differences between the control and experimental group were observed at the initial assessment (*p* = 0.703, *d* = 0.20), but follow-up scores indicated a significant reduction in anger for the experimental group (*p* = 0.046, *d* = 1.41) as assessed by Mann–Whitney testing. Within groups, no difference in anger was observed for the control group (*p* = 0.715) or the experimental group (*p* = 0.109) as assessed by Wilcoxon signed-ranks testing.

PTSD checklist-military scores are shown in **Table 6**. No differences between the control and experimental group were observed at the initial assessment (*p* = 0.639, *d* = 0.16) or at follow-up (*p* = 0.480, *d* = 0.57) as assessed using Mann– Whitney testing. Within groups, no difference in PCL-M scores was observed for the control group (*p* = 0.285) or the experimental group (*p* = 0.144) as assessed by Wilcoxon signed-ranks testing.

#### DISCUSSION

The current series of studies shows the feasibility of creating an individualized, physiological classifier of stress with a high degree of accuracy compatible with different sensor suites. The use of such an algorithm in an mHealth application (35) may reduce symptoms of stress and anger in a small clinical population, increase the number of CBT sessions individuals will attend, and decrease their dropout rate. Given the large number of individuals that experience mental health disorders and the unmet need for treatment, especially in developing nations, such mobile approaches have the potential to provide or augment treatment in the absence of standard, in-person care (36). However, most

#### Table 5 | Mean (SD) PROMIS Anger scores.


*a indicates significant difference between groups.*

#### Table 6 | Mean (SD) PCL-M Scores.


commercially available apps targeting mental health remain untested (22, 37).

Classification of stress was based on features gathered from a large user group undergoing the TSST, which has one of the highest effect sizes for eliciting stress and associated cortisol responses in laboratory settings (38). Stress classification was based on cardiovascular and electrodermal inputs (3), which showed high variance due to individual differences (19), and was addressed by individual baseline normalization. The psychoendocrine reaction to life stressors, or stressors outside of the laboratory setting, such as bereavement, declining health, or flashbacks in PTSD, are likely of higher duration and intensity than laboratory stressors (39). Therefore, the algorithm and decision boundary developed using acute socio-evaluative stress in the current work may underperform for more severe stressors associated with MDD, PTSD, or other forms of mental health disorders. For instance, the DASS and PROMIS-anger scores from the classifier-development study sample indicated a relatively low burden of stress, depression, anxiety, and anger as opposed to a relatively high burden of mental health symptoms in the clinical evaluation study sample. In addition, veteran post-traumatic stress is often comorbid with depression, which has recently been shown to be associated with intensified anger (2). Anger has been acknowledged as the most prevalent veteran readjustment problem (14). Interestingly, the use of an mHealth application focused on stress identification and reduction in conjunction with CBT reduced metrics of anger and anxiety in addition to stress in a small clinical sample.

The overall dropout rate from CBT has been reported to be between 25–40% for depression (12, 13). In the current study, over 50% of the participants discontinued therapy, early. This higher dropout rate may be due to characteristics of the veteran population or the high burden of mental health symptoms in the study sample. For example, previous research has indicated that medication compliance among veterans is relatively low (40). The high dropout rate likely also reflects that many of the participants were experiencing periods of acute stress and were often preoccupied with these stressors. The availability of validated mHealth applications that individuals could use within the context of their daily lives would help to address this issue. Within the sample, those who used the mobile application and stress algorithm were more likely to complete the study and demonstrated reduced stress and anger as compared to the control group. This reduced stress may result from an increased awareness of their stressors due to the alerts provided through the mobile application to the user (41), or the use of the guided breathing exercises within the application (42).

Future research will include further accuracy refinement through reduction in environmental noise, and a method to learn individual user stress thresholds (19). Additional operational testing to reduce environmental noise is being conducted in order to determine the changes in classifier false alarms and misses when collecting data in different temperatures and while performing different physical activities (43), ranging from typing on a keyboard to walking or running. The 2-feature linear model trained with stochastic gradient descent employed in this effort has the advantage of including a bias term to tune the decision boundary threshold on the stress vs. non-stress classifier to allow adjusting the tradeoff between hit and false-alarm rates, ultimately generating an individualized threshold for each user.

The low sample size in the clinical evaluation and the high dropout rate represent a limitation in the current study. Even though there have been an estimated 180,000 cases of US military veterans with PTSD over the past two decades (44), many do not seek care (45). Additional challenges include long wait times experienced in the VA medical system (46), low participation rates in clinical studies (47), and a high dropout rate during CBT. Further data and objective outcome measures are needed to validate the observed reductions in stress, anger, and anxiety symptoms in the study sample.

#### REFERENCES


The capability to classify individual physiological stress in a mobile environment has additional uses outside of veteran mental health treatment, including military or medical training (48). For example, training instructors could remotely and simultaneously monitor objective stress status for individual trainees during live training sessions and act on the information in real time (49). In addition, instructors could identify individual trainees that tend to have more intense stress responses than others during training scenarios to provide targeted coping and resilience training interventions (50). Beyond training, additional applications for this capability include stress research for laboratory and field settings, chronic disease monitoring for tracking outpatient health and long-term data capture to inform care, and objective, real-time user experience evaluations. mHealth applications and wearable physiological sensors have the potential to analyze and present meaningful data to better manage and optimize general health and specific health conditions. However, high quality, wearable devices and robust, validated algorithms remain a necessary component to realizing the potential of this technology.

## AUTHOR CONTRIBUTIONS

BW analyzed data and wrote the manuscript. GC and BW developed the algorithm for stress. SD and DJ developed the mobile application and managed the experiments. PG designed the clinical evaluation. SK and JG conducted the clinical evaluation.

#### ACKNOWLEDGMENTS

Portions of the classifier study were reported at the 49th Annual Meeting of the Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), as SD, BW, GC, T. Schmidt-Daly, DJ (2014) Classifying stress in a mobile environment (14.195). This work was funded by the US Army Medical Research and Materiel Command (USAMRMC) grant W81XWH-12-C-0071.


**Conflict of Interest Statement:** GC, BW, SD, DJ (2014) System, method, and computer program for the real-time mobile evaluation of physiological stress. US Patent Pending #10878-023.

*Copyright © 2016 Winslow, Chadderdon, Dechmerowski, Jones, Kalkstein, Greene and Gehrman. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Ecological Momentary Assessment and Smartphone Application Intervention in Adolescents with Substance Use and Comorbid Severe Psychiatric Disorders: Study Protocol

*Xavier Benarous1 \*, Yves Edel2 , Angèle Consoli1 , Julie Brunelle1 , Jean-François Etter3 , David Cohen1,4 and Yasser Khazaal5,6*

*1Department of Child and Adolescent Psychiatry, Hôpital Pitié-Salpêtrière, Paris, France, 2Department of Addiction, Hôpital Pitié-Salpêtrière, Paris, France, 3 Institute of Global Health, University of Geneva, Geneva, Switzerland, 4UMR 7222, Institute of Intelligent Systems and Robotics, Université Pierre et Marie Curie, Paris, France, 5 Faculty of Medicine, Geneva University Hospitals, University of Geneva, Geneva, Switzerland, 6Geneva University Hospitals, Geneva, Switzerland*

#### *Edited by:*

*Matthias Jaeger, University Hospital of Psychiatry Zurich, Switzerland*

#### *Reviewed by:*

*Caroline Vandeleur, Centre Hospitalier Universitaire Vaudois, Switzerland Niklaus Stulz, Psychiatric Services Aargau AG, Switzerland Sebastian Olbrich, University of Zurich, Switzerland*

> *\*Correspondence: Xavier Benarous xavierbenarous@gmail.com*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 02 June 2016 Accepted: 30 August 2016 Published: 20 September 2016*

#### *Citation:*

*Benarous X, Edel Y, Consoli A, Brunelle J, Etter J-F, Cohen D and Khazaal Y (2016) Ecological Momentary Assessment and Smartphone Application Intervention in Adolescents with Substance Use and Comorbid Severe Psychiatric Disorders: Study Protocol. Front. Psychiatry 7:157. doi: 10.3389/fpsyt.2016.00157*

Context: Substance use disorders (SUDs) are highly prevalent among inpatient adolescents with psychiatric disorders. In this population, substance use and other psychiatric outcomes can reinforce one another. Despite the need for integrated interventions in youths with dual diagnoses, few specific instruments are available. App-based technologies have shown promising results to help reduce substance use in adolescents, but their applicability in youths with associated severe psychiatric disorders is poorly documented. We aim to evaluate the feasibility of an ecological momentary assessment (EMA) intervention for all substance users, and of a smartphone application for cannabis users (Stop-Cannabis), for outpatient treatment after hospital discharge.

Methods and analysis: All inpatient adolescents with psychiatric disorders hospitalized between 2016 and 2018 in a university hospital will be systematically screened for SUD and, if positive, will be assessed by an independent specialist addiction team. Participants with confirmed SUDs will be invited and helped to download an EMA app and, if required, the Stop-Cannabis app, the week preceding hospital discharge. Information about the acceptability and use of both apps and the validity of EMA data in comparison to clinical assessments will be assessed after 6 months and 1 year.

Discussion: This research has been designed to raise specific issues for consideration regarding the sequence between substance use, contextual factors, and other psychiatric symptoms among adolescents with comorbid severe psychiatric disorders. A better understanding of the mechanisms involved will inform the development of integrated treatment for dual disorders at that age.

Ethics and dissemination: The study has already been approved and granted. Dissemination will include presentations at international congresses as well as publications in peer-reviewed journals.

Trial registration: European Clinical Trials Database: Number 2016-001999-30.

Keywords: substance use disorder, cannabis addiction, prevention, adolescents, ecological momentary assessment, ecological momentary intervention, mHealth app, smartphone app

#### INTRODUCTION

#### Background

#### Substance Use Disorders and Psychiatric Disorders in Adolescents

Psychiatric comorbidity in adolescents who abuse substances is the rule rather than the exception, with 61–88% having a dual diagnosis (1, 2). The coexistence of both disorders is associated with important disruptions in a range of functional domains, including academic attainment, social and family relationships (2, 3), and accidents (4). In addition, initiation of substance use at a younger age (5) and a faster relapse after treatment (6) are reported in youths with preexisting mental health problems. In turn, the natural course of psychiatric disorders may be affected by substance use, either due to negative psychosocial consequences associated with substance involvement (7) and/or to the direct effect of the substance on the central nervous system (8). In youths with mood or psychotic disorders, for example, daily use of cannabis is associated with a more severe clinical presentation, impaired cognitive and interpersonal abilities, and poorer responses to pharmacological treatments (9). The development of integrative treatment for substance use disorders (SUDs) and psychiatric disorders is therefore a priority (10, 11).

Among adolescents hospitalized for a psychiatric problem, 17–50% meet criteria for one or more SUDs (12–17). In a previous study of adolescents hospitalized in three psychiatric units in France, most subjects met criteria for a non-nicotine SUD: around 70% for cannabis, 60% for alcohol, and 20% for other substances (18). Unsurprisingly, those with dual disorders exhibited more severe clinical characteristics (e.g., more suicide attempts) and a higher level of functional impairment (e.g., more school absenteeism) than patients without SUDs. They also received more diagnoses of axis 2 comorbidities and were exposed to more severe psychological contexts (e.g., maltreatment or loss of a first relative) (18). Given the high prevalence of SUDs among adolescents who are hospitalized, and the poor clinical outcomes due to the negative interplay between both disorders, specific preventive interventions for addiction should be developed and evaluated in this population.

#### Addiction Preventive Interventions in Inpatient Adolescents with Psychiatric Disorders

It is well established that treatments for SUDs and psychiatric disorders are less effective when provided separately (1, 6). Although the efficacy of integrated approaches for the management of dual disorders has been demonstrated in adolescents (10, 11), there is some concern that – in practice – the so-called "integrated approach" often consists of the addition of evidence-based treatments for both problems, with too few specific dual diagnostic or therapeutic tools. For example, in order to choose the optimal treatment, the sequence between substance use and other psychiatric symptoms in patients' daily life should be explored, but we are not aware of any tool that would measure this pattern. Furthermore, treatment compliance is a key prognostic factor in addiction. For most teenagers, SUDs are commonly underrecognized as a health issue (19). The development of tools that promote better recognition of patients' problems with substances, and enhance behavior change, would therefore be worthwhile. Digital approaches may be a promising avenue for health-related interventions among adolescents given their propensity to use digital technologies (20–24).

#### Digital Approaches

The uptake of smartphones in the general population has encouraged the development of apps used to facilitate real-time assessment of the patient in his/her natural environment [Ecological Momentary Assessment (EMA)] and allowed delivery of smartphone app interventions in this context.

Ecological momentary assessment consists of using computers or other devices to collect self-assessments repeatedly in realtime, in various situations of the participant's daily life, either for research purposes or to provide participants with timely feedback and advice (25). Different terms have been used for this methodology since the 1980s, including "ambulatory assessment," "experience sampling method," or "real-time data capture." Even though the terms differ, these approaches have the data collection in common (e.g., symptoms, behaviors, physiological processes, or daily life circumstances) while the participant undergoes normal activities. Therefore, this allows the interplay between contextual stressors, psychological distress, and maladaptive behaviors to be modeled (26). Compared to traditional measures using paper–pencil questionnaires, EMA is regarded as having many benefits for addiction research, including decreased recall bias, decreased contextual bias (25, 27), and increased validity to model the temporal sequence of events between risky situations, craving, and substance use (28, 29). Although such tools have been available for over 30 years, the emergence of smartphone applications as a platform for EMA greatly improved the acceptability and usability of this technology (30). In the last decade, EMA systems have been developed for adolescents with addictions (31) and applied to the assessment of psychiatric symptoms in outpatients with internalizing/externalizing disorders (32–36). In contrast with adults [e.g., Ref. (37)], EMA has not yet been assessed, to our knowledge, for studying the relationship between psychiatric symptoms and patterns of substance use in adolescents. In addition, the feasibility of using EMA in adolescents with severe psychiatric symptoms remains understudied.

The smartphone app Stop-Cannabis was developed in 2013 in Geneva, Switzerland with financial support from the local Health Department.1 The app aims to help users reduce or stop their cannabis use and prevent relapse. Stop-Cannabis consists of several modules based on cognitive behavioral therapy strategies, motivational interviewing, and relapse prevention programs. Particular effort has been made to include the principles of selfdetermination (autonomy, competence, and relatedness) (38) into the app and promote its customization (39). As in other studies (40–42), participant interaction and rewards support the playful aspect of the intervention. Personalization of the users' goals and the app main screen may further contribute to increase adoption by user (43). The contents of the application are summarized below [and described in detail in Ref. (39)]. Since February 2013, the Stop-Cannabis app [available on Google Play (Android) and

<sup>1</sup>https://itunes.apple.com/ch/app/stop-cannabis/id532498307?l=fr&mt=8

(iOS) App Store] has been downloaded more than 13,000 times, and the app is currently used by 1,000 people every month. The app was reported to have a good level of acceptability among community users in an online satisfaction survey (39).

This study is part of a general collaborative research program between the child and adolescent psychiatric department team and the addiction unit. The purpose of this wider program is to inform the difference in the trajectories of inpatients adolescents with SUDs compared to other and to help developing a better clinical practice (i.e., early detection, clinical assessment, and therapeutics).

#### Objectives

The primary objective of our pilot study is to evaluate the feasibility and acceptability of two different digital tools (an EMA tool and Stop-Cannabis) based on smartphone apps in adolescents with dual disorders during a 1-year follow-up period after hospital discharge. We aim to determine the feasibility of the EMA, adherence to recommendations, and the quality of data collected using an EMA digital assessment tool. In addition, we aim to evaluate the feasibility, acceptability, and use of the Stop-Cannabis app, a digital intervention tool, during outpatient aftercare. The authors expect to obtain from this pilot study data on the feasibility and acceptability of two kinds of digital tools (one related to assessment and the other one to an intervention) possibly useful for adolescents with SUDs and comorbid severe psychiatric disorders.

Secondary objectives are (1) to estimate the prevalence of SUDs in adolescents with psychiatric disorders hospitalized in the Department of Child and Adolescent Psychiatry at the Pitié-Salpêtrière University Hospital, Paris, France (this department provides one-third of all child and adolescent inpatient beds in the Paris area); (2) to describe the clinical and psychosocial characteristics of substance users compared with non-users; (3) to determine the risk factors for the onset of SUDs during follow-up in participants without dual diagnoses at baseline; (4) to determine the risk factors for the persistence of noncannabis SUDs during follow-up; (5) to document the longitudinal interplay between substance use and psychopathology in adolescents with dual diagnoses at baseline; and (6) to examine the association between the use of the Stop-Cannabis app and the patterns of substance use at 1-year follow-up in naturalistic settings to inform the potentiality for future research.

#### METHODS AND ANALYSIS

## Selection of Participants

#### Inclusion and Exclusion Criteria

Participants will be adolescents (11–18 years old) who are hospitalized in the Department of Child and Adolescent Psychiatry at the Pitié-Salpêtrière University Hospital between September 2016 and September 2018. The participants will be assessed at baseline. According to the presence or absence of comorbid SUDs, they will be included in either the SUDs group or the control group. The SUDs group will receive the mobile apps, whereas the control group will not. Exclusive tobacco consumption is not considered to be a criterion for inclusion in the SUDs group. Inclusion to the non-SUD control group will be made according to a match with adolescents from the SUDs group (for age, gender, inpatient unit, and time of admission). Exclusion criteria include non-French speaking, those without a personal mobile phone, lack of informed consent from parents or from the adolescents, and any mental or physical problems that interfere with participation in the study (e.g., autism with very low communication abilities, severe to profound intellectual disability, severe motor disability).

#### Sample Size

A conservative approach was adopted for this feasibility study, which emphasizes descriptive and qualitative feedback. Consequently, based on the recommendations for pilot studies (44), we plan to include 80 participants (40 cases, 40 controls) over a 2-year inclusion period. As there are around 280 inpatient admissions per year in the Department of Child and Adolescent Psychiatry, i.e., *n* = 560 potential participants contacted at baseline during the 2 years, this will represent a 14% inclusion rate. We predict that the prevalence of SUDs will be similar to that previously reported (18). A good level of compliance with EMA and Stop-Cannabis is expected, consistent with previous studies that reported high levels of adherence (up to 80%) to similar tools in adolescents with comorbid psychiatric disorders (31, 45), and in adults with associated severe psychiatric disorders (32, 46). An acceptance rate of 80% is expected in the SUD group and 50% for the control group. A 50% retention rate at 1-year follow-up is anticipated based on previous longitudinal studies among adolescents (47). In case of a lower enrollment rate, we will either continue to enroll participants during an additional year (2019) in order to reach the intended sample size or use incentives (e.g., free participation in leisure and sport activities in groups) to encourage participant enrollment.

#### EMA and Stop-Cannabis App Tools Ecological Momentary Assessment

The EMA was designed as an app for mobile phones. This application allows several daily self-assessments (e.g., five times a day for 2 weeks at different times of the study/monthly). Data are collected in response to a signal from the EMA app that occurs at various times of the day. The app has been programed to send signals (i) a fixed number of times per day at predetermined times (e.g., at bedtime) and (ii) at random (with a balance between week, weekends, and holidays, with different time intervals in daytime). Participants are also asked to complete self-assessment immediately following substance use.

The same questions are presented regardless of assessment type. There are seven questions in total:

	- Daily use of each substance in list.
	- Level of withdrawal symptoms during participants' most recent period of abstinence from 0 (not at all) to 5 (severe).
	- Level of craving from 0 (no urge) to 5 (extreme urge).
	- Level of negative affect rate using a brief version of the Positive and Negative momentary Affect Scale (PANAS) (48) adapted for EMA (32). Participants provide ratings of

the extent to which they felt each emotion on a 1–5 point Likert-type scale.

	- Stressful events.
	- Activities: physical activity, planned activity, lone activity, social activity with peers, family activity, and online activity.

Participants will be trained on EMA use during the week prior to hospital discharge. They will be asked to respond to any signals within 1 h if possible. Consistent with other EMA protocols, participants will complete 2 days of practice data (not used for analyses), after which they will receive feedback on adequate use and compliance. Participants will then be invited to complete EMA assessments for 2 weeks, as this timeframe appears sufficient to monitor substance use (45). To limit attrition, participants will be paid 30€ for completing the baseline assessment and 10€ for each week of complete EMA data, up to a maximum possible payment of 60€ over 1 year.

#### The Stop-Cannabis App

Launched in 2013, Stop-Cannabis is available free on iOS and Android. It has been continuously updated and improved over the past 3 years, in response to users' suggestions. When people use the app for the first time, they are asked to document their objectives and choose a quitting date. The app has several modules. First, it allows users to assess their cannabis usage profile (including motivations for use and severity of cannabis use) *via* different assessments (49). Users then receive an individually tailored feedback message that includes links to online psychoeducation and complementary resources. Second, motivational messages are regularly delivered using push-notification technology at different stages of the quitting process, on the basis of updated information regarding current substance use, the selected quitting date, and exposure to situations associated with a high risk of relapse documented by high levels of craving or irritability. Third, the app includes messages on how to cope with craving symptoms or emotional difficulties to enhance participants' insight into contextual factors for substance use. Fourth, positive reinforcers, such as the number of days since quitting the substance and the amount of money saved, are displayed on the main screen and can be personalized (**Figure 1**). Fifth, a discussion forum called "The Tribe," moderated by a psychologist, is available for all users to encourage discussion, mutual support, and information sharing. Sixth, the app includes direct access to a website2 for additional information about substance use and online motivational interviewing training.

The app will automatically monitor the number of times each participant opens the app and records the number of times participants looked at or used each element. A Unique Device Identifier (UDI) is used to connect the participants' device to the study database and will be replaced in the data set with a random ID for anonymity. After study completion, the UDIs will be automatically deleted from users' devices. The app will be available for participants in the intervention group for 12 months. After this period, the app will automatically stop data collection. However, participants are free to use the app for as long as they want.

## Design

#### Screening

The study flowchart is shown in **Figure 2**. Each patient admitted to the Department of Child and Adolescent Psychiatry at the Pitié-Salpêtrière University Hospital during the study recruitment phase will be systematically screened for SUDs and eligibility at admission. The DEP-ADO questionnaire (50) will be used in the study to document substance use in the previous 12 months. The complete questionnaire is available free online3 . The screening question will be "During the last twelve months, how often have you [has X] used one of the following substances: alcohol, cannabis, cocaine, inhalant/solvent, stimulant, hallucinogen, or heroin" (examples and trivial names are provided for each substance). For each substance, participants and their families will have the options of answering: "Never," "Occasionally," "Once per month," "Once or twice per week," "More than three times per week," or "Daily." The clinician version of the DEP-ADO will be completed by the clinician involved in usual care after an interview with the adolescent and his/her family. The self-report version will be completed by the adolescent, with assistance from a specially trained member of the paramedical staff. Assessments will be carried out within 3 days following admission and repeated after 1 week as an inpatient, and again if necessary (especially if the clinical condition is not compatible with self-assessment).

Urinary drug tests will be performed on all adolescents hospitalized in the Department of Child and Adolescent Psychiatry during the study recruitment phase. The tests screen for the presence of cannabis, cocaine, opiates, and amphetamines. In the interest of having as broad as possible a representation of the target population, we will include as "screen positive" those who answer at least "occasionally" to use of cocaine, inhalant/solvent, stimulant, hallucinogen, or heroin, and/or at least monthly use of cannabis or alcohol, and/or when their urine test indicates the presence of one of these substances.

#### Assessment to Determine Eligibility

All youths screened positively for substance use will be reported to the hospital's liaison addiction unit (ECIMUD, a French acronym for *coordination and medical care team for addictive disorders)* for proper diagnosis and care. Systematic evaluation of the substance use and habits will be conducted *via* a semi-structured interview, which allows for scoring the RECAP index, the clinician version of the DEP-ADO, and the Cannabis Use Screening Test (CAST), a self-reported scale that focus on use of cannabis in the previous 6 months (51). Final diagnoses of SUDs will be made by one of the ECIMUD senior psychiatrists using DSM-5 criteria (52).

#### Initial Clinical Assessment and Follow-Up

Participants (i.e., adolescents with SUDs and control subjects) will be approached by a researcher involved in the project (either a clinician specialized in child psychiatry or a master's level psychologist) who will explain the study and obtain written

<sup>2</sup>http://www.stop-cannabis.ch

<sup>3</sup>https://oraprdnt.uqtr.uquebec.ca/pls/public/docs/GSC3472/F463443489\_DEP\_ ADO\_ang\_V3\_2.pdf

benefits of change. Adapted from: Monney et al. (39).

informed consent from parents or guardian and assent from the adolescent. Participants will be assessed in the week prior to hospital discharge for sociodemographic data, psychosocial information, psychopathology, and substance use (see Variables and Instruments). All youths will be assessed in face-to-face interviews by a specially trained master's level psychologist at 6-month and 1-year follow-up.

#### Data Analysis

#### Variables and Instruments

**Table 1** lists the questionnaires and other assessments that will be used at baseline and at follow-up. The reading level of the questionnaires is grade 6. The questionnaires we plan to use to measure addiction are described below. A combination of self- and clinician-report measures with different time scales (i.e., DEP-ADO, 12 months; CAST, 6 months; RECAP, 1 month) will be used to examine whether EMA data are consistent with information collected in the traditional way. The second type of measures relate to psychopathology with the aim of examining the interplay between substance use and psychiatric disorders.

#### *Substance Use*

• The RECAP index is a clinician-reported questionnaire systematically used for standardized collection of information

#### Table 1 | Variables examined in the study.


*DEP-ADO, RECAP Index, CAST, Cannabis Use Screening Test; ICSU, Compulsive Internet Use Scale; K-SADS-PL, Kiddie-SADS-Present and Lifetime Version; Ado DIB, Adolescent Diagnostic Interview for Borderline Patients; ARI, Affective Reactivity Index; UPPS-P, Impulsive Behavior Scale; C-GAF, Children's Global Assessment of Functioning; ACE, Adverse Childhood Experiences Questionnaire; CSSRI-EU, Client Sociodemographic and Service Receipt Inventory-European Union.*

regarding substance use during the last month in outpatient addiction centers in France, recommended by the EMCDDA (European Monitoring Centre for Drugs and Drug Addiction) (available at http://www.ofdt.fr/enquetes-et-dispositifs/recap/ presentation/).


#### *Psychopathology*


widely used measures of the overall severity of adolescent disturbance (61).

#### *Other Measures*


#### Outcomes

Information will be collected from both EMA data and from questionnaires administered during hospitalization and follow-up.

#### *Primary Outcomes*

The first parameter is whether the level of adherence and the acceptability of the EMA and the Stop-Cannabis app are satisfactory in this sample. Acceptability will be assessed by measuring the dropout rate after 6 and 12 months, and usability and satisfaction *via* questionnaire in all participants and *via* face-to-face interviews with a subgroup of 10 participants. Compliance will be assessed *via* mean percentage of random prompts and predetermined assessments completed per participant. The second parameter is whether information collected using EMA *via* a mobile app is valid and reliable. Information obtained by EMA tools using the mobile app will be compared to those collected by traditional paper–pencil and clinical interview methods during the initial and follow-up assessments using correlations analysis.

#### *Secondary Outcomes*


#### Statistical Methods

#### *Primary Outcomes*

Pearson correlations will be used to determine the strength of associations between daily substance use assessed by either EMA, or self-report questionnaire (e.g., CAST), or by clinician-report questionnaire (e.g., RECAP) during the clinical assessment at 6 months. Clinical reliability of the EMA will be endorsed if the correlations are statistically significantly higher than 0.70. Further analyses would be performed to estimate different aspects of reliability of the data collected in line with Perrez et al. (64). The severity of SUDs at baseline should predict the severity of substance use and the level of functional impairment at 1 year at least to comparable levels while using EMA data at baseline compared with traditional methods. Two simple linear regression models will be performed. For both the models, the dependent variable will be the severity of SUDs at follow-up; in one model, the independent variable will be the severity of SUDs based on EMA data (Model A), whereas in the other model, the variable will be severity of SUDs based on self- and clinician-report (Model B). The estimate of the independent variable is expected not to be lower in the Model A than in the Model B.

The dropout rate is expected to be less than 40% at 6 months and less than 50% at 12 months for both the EMA and traditional method, with no significant difference between methods. The results of the satisfaction survey will be estimated by analyzing the percentage of answers and the differences among the different categories using Chi-square analysis.

#### *Secondary Outcomes*


#### Calendar

March 2016: training the medical team to use the screening tools (i.e., DEP-ADO).

December 2016: start of participant inclusion (t0).

December 2017: end of data collection for participants enrolled at t0.

November 2018: end of inclusion.

December 2018 to January 2019: start of analysis of baseline data, i.e., prevalence of SUDs.

November 2019: end of data collection for final participants included.

December 2019 to January 2020: analysis of final data.

February 2020: preparation of an abstract for submission to psychiatric congress.

April 2020: writing one to three articles for psychiatric and addiction journals.

#### DISCUSSION

#### Anticipated Results

#### Primary Objectives

A degree of correlation higher than 0.70 is expected between EMA data and data collected using other methods (particularly frequency of use), in line with prior studies (46). EMA should lead to predictions regarding the course of SUDs over the 1-year follow-up (e.g., in term of persistence, severity, and functional impairment) at least as good as other methods; that would be of interest for the clinicians involved in usual care. A high level of compliance with EMA is expected, despite the relative severity of the study sample. Based on previous studies in adolescents (45, 67), participants are expected to complete a mean of more than 70% of random signals, 50% of end-of-day assessments, and 40% of both random and end-of-day assessments. Indeed, there is good evidence that EMA can be used by patients with severe psychiatric disorders (e.g., schizophrenia) or cognitive difficulties (32, 46).

#### Secondary Objectives

This study will measure the prevalence and the characteristics associated with the presence of SUDs in a sample of adolescents hospitalized in a psychiatric setting. Findings are expected to be in a similar range as those previously reported by Daudin et al. (18). We will then examine the possible relationship between SUDs and specific psychosocial and clinical features of inpatient adolescents. We expect this study to confirm that both internalized and externalized disorders are significantly associated with SUDs and that the presence of SUDs is associated with a higher level of impulsivity, mood dysregulation/borderline traits, and sensation-seeking traits (68–70). We predict that a bidirectional positive relationship between the level of substance use and the severity of other psychiatric symptoms will be noted at a daily level, and – if confirmed – that this association would be strengthened in a context of interpersonal relationship (with family and with peers) compared to other situations. A significant negative association between the level of the Stop-Cannabis app use and the participant's clinical outcome at follow-up is expected. Such additional exploratory analysis would provide preliminary information about the possible benefit of the app in naturalistic settings.

#### Limitations

Several limitations may preclude interpretation of our findings. As no control group for the Stop-Cannabis app is defined (i.e., subjects with cannabis use disorders without the intervention), the effectiveness of the Stop-Cannabis app to reduce or quit cannabis cannot be directly examined in our study. Furthermore, one other limitation is related to the lack of EMA assessments for the control group. The groups cannot be then compared with the EMA data. The authors choose to not use EMA assessment for the control group in order to avoid possible impact of the assessments on the behaviors [for a discussion about the reactivity effect of EMA, see Shiffman (25)]. The groups will receive, however at different time, a number of clinical assessments useful for the planned comparisons. As our study participants will be recruited during hospitalization in a university teaching hospital, the sample is likely to comprise youths with severe or resistant clinical profiles, which could limit the generalizability of our findings. In addition, the study design requires that only participants who own a smartphone are included, which is a risk for selection bias. Attrition biases may result from high dropout rates and low use of the app, but incentives and reminder phone calls will be employed to improve adherence. However, if important differences in baseline characteristics between adherent and non-adherent participants are detected, this will be further controlled by regression analyses. Finally, no firm conclusion about the benefit of the Stop-Cannabis application can be drawn from this study with regards to the lack of controlled randomized group.

#### Clinical and Research Implications

The utility of app tools for adolescents with dual disorders would be of clinical interest, considering the need for evidence-based treatments and interventions in this population (71). Longitudinal data and mediation analyses would provide preliminary data on the mechanisms involved in the relationship between dual disorders (**Figure 3**). A better understanding of this mechanism is essential to focus on specific subpopulations in pursuit of more integrated treatment and support for their mental health and addiction problems. The originality of this study stems from the combination of tool-based measures with distinct temporal scale: standardized semi-structured interviews at baseline and end-point, self-reported non-ecological data obtained every 6 months, and self-measurement in the individual's natural environment using EMA tools.

Ecological momentary assessment data provide insights on the sequence between substance use, contextual factors, and psychiatric symptoms that could not have been obtained using more traditional self-report measures. This is particularly important for adolescents where psychopathology may be more fluctuating and context-dependent than in adults. In contrast with previous EMA data research projects conducted in children and adolescents, our sample encompasses a large range of psychiatric disorders with a more severe profile. Finally, in line with other studies (72), the insights obtained from EMA may not only serve researchers and clinicians but might also benefit patients directly as patients become increasingly aware of their symptoms.

Finally, this research will provide information that ultimately helps determine whether a randomized controlled trial could be conducted in the future to test the benefit of a digital intervention based on the Stop-Cannabis app in adolescents with severe comorbid psychiatric disorders.

#### ETHICS AND DISSEMINATION

#### Informed Consent and Institutional Review Boards

Regarding patient safety, all participants will receive detailed information about local help in case of crisis, emergency, or suicidal ideation. Participants will also be informed about, and trained how to use, the apps and told what can be expected from their use. They will also be advised to consult their doctor and share concerns with their family and relatives according to their needs. Participants will be instructed to not use the apps when it is inconvenient (e.g., in class) or unsafe (e.g., on bikes).

Informed consent will be obtained from the adolescent and his/her family or legal representatives after they have received information about the study objectives and procedures. Agreement has already been provided by the scientific board of the Institute of Research in Public Health; however, data collection will only start when the protocol will be approved by the Ethics Committee (EC) of the Pitié-Salpêtrière Hospital. In the case of protocol changes, an amendment will be submitted to the concerning EC. The project has already received financial support from French institutions, i.e., la Direction General de la Santé (DGS), la Caisse Nationale de l'Assurance Maladie des Travailleurs Salariés (CNAMTS), la Mission interministérielle de lutte contre les drogues et les conduites addictives (MILDECA), and l'Observatoire national des Jeux (ODJ) on the basis of an open invitation to tender from the IreSP in 2015 (reference "IReSP-15-Prevention-11").

The study protocol has been recorded on the European Clinical Trials Database (EudraCT Number 2016-001999-30).

#### Confidentiality

In order to ensure patient confidentiality during the study and transmission of personal data, a security protocol using a digital identification number will be employed.

The collected data will be only accessible to the principal investigator and study staff as well as the monitors.

#### Dissemination

After study completion, the results of the primary and secondary analyses will be published in international peer-reviewed journals (at least one specialized in addiction and another in adolescent psychiatry).

If shown to provide valid and reliable information in addition to traditional measures, findings would be presented in international symposiums to consider ways to develop new research protocols on dual disorders in adolescents, which involved these applications.

Findings regarding the usefulness and possible benefits of the Stop-Cannabis app will be transmitted to the team from the University of Geneva who programed the app to suggest possible areas for improvement.

#### AUTHOR CONTRIBUTIONS

XB, YE, DC, and YK: substantial contributions to the conception and design of the article; XB, YE, AC, JB, J-FE, DC, and YK: drafting the article or revising it critically for important intellectual content, final approval of the version to be published, and agreement to be accountable for all aspects of the article in ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved.

#### REFERENCES


#### ACKNOWLEDGMENTS

The authors are grateful to Dr Tremblay who gave advices on the design of the study.

#### FUNDING

The project has already received financial support from French institutions, i.e., la Direction General de la Santé (DGS), la Caisse Nationale de l'Assurance Maladie des Travailleurs Salariés (CNAMTS), la Mission interministérielle de lutte contre les drogues et les conduites addictives (MILDECA), and l'Observatoire national des Jeux (ODJ) on the basis of an open invitation to tender from the IreSP in 2015 (reference "IReSP-15-Prevention-11").


**Conflict of Interest Statement:** The research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. YK was involved in the development of Stop-Cannabis website and app.

*Copyright © 2016 Benarous, Edel, Consoli, Brunelle, Etter, Cohen and Khazaal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **mHealth app for cannabis users: satisfaction and perceived usefulness**

*Grégoire Monney 1,2 , Louise Penzenstadler <sup>1</sup> , Olivia Dupraz <sup>2</sup> , Jean-François Etter 2,3 and Yasser Khazaal 1,3 \**

*<sup>1</sup> Department of Mental Health and Psychiatry, Geneva University Hospitals, Geneva, Switzerland, <sup>2</sup> Faculty of Medicine, Institute of Global Health, University of Geneva, Geneva, Switzerland, <sup>3</sup> Department of Psychiatry, University of Geneva, Geneva, Switzerland*

**Objective:** The aim of this study was to describe the characteristics of cannabis users and their levels of satisfaction with Stop-cannabis, an app intended for cannabis users who want to stop or reduce their cannabis use or prevent relapse.

**Methods:** A cross-sectional online survey was administered to users of Stop-cannabis, a French-language app for iOS and Android devices. All app users were invited to participate in the survey via a message sent to the app.

*Edited by: Shervin Assari, University of Michigan, USA*

#### *Reviewed by:*

*Mohammadreza Mokhtari, Hartford Hospital, USA Masoumeh Dejman, University of Social Welfare and Rehabilitation Sciences, Iran*

#### *\*Correspondence:*

*Yasser Khazaal, Department of Mental Health and Psychiatry, Geneva University Hospitals, Grand pré, Geneva 70C-1206, Switzerland yasser.khazaal@hcuge.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 20 April 2015 Accepted: 11 August 2015 Published: 27 August 2015*

#### *Citation:*

*Monney G, Penzenstadler L, Dupraz O, Etter J-F and Khazaal Y (2015) mHealth app for cannabis users: satisfaction and perceived usefulness. Front. Psychiatry 6:120. doi: 10.3389/fpsyt.2015.00120* **Results:** For hundred and eighty-two users answered the survey. The app was used daily by 348 of the participants (around 70%). More than 80% of participants (397) considered the app to have helped them "a little" or "a lot" to stop or reduce cannabis consumption. Most of the users' suggestions were related to the number or the quality of the messages sent by, or displayed in, the app.

**Conclusion:** This pilot study supports the feasibility of such an app and its perceived usefulness. A self-selection bias, however, limits the conclusions of the study. The efficacy of the app should be evaluated in a randomized controlled trial.

**Keywords: cannabis, addiction, app, mhealth, smartphone, cognitive behavior therapy**

## **Background**

Cannabis is a widely used substance associated with harms, addiction, and possible psychiatric disorders in some users (1–8), including young adults and adolescents (9, 10).

During the last years, an increasing number of Internet-based self-help and treatments were developed as an attempt to give information, assessment, support, or treatment for people with substance use disorders or behavioral addictions (11–23)**.**

Interventions, such as motivational interviewing and cognitive behavioral therapy, including those used in web-based treatment formats, have been shown to have a favorable impact on a number of lifestyle and health-related behavior (24, 25) and cannabis use (20, 26–28). Unfortunately, the number of users who seek help for cannabis addiction remains very low (29). This may be due to a perceived stigma and limited access to treatment (17, 30), or to expectations of care ineffectiveness (31). With the launch of mobile applications software ("apps") and the wide dissemination of smartphones, however, a new opportunity for e-health and for the development of interactive tools has emerged (11, 32–35).

Particularly, apps combine mobile communication and computation in a handheld-sized tool, allowing new ways for clinicians to help people in real-time in order to promote positive change (36).

Several behavioral change techniques were considered by expert consensus as possibly useful for the design of apps aiming to help people with substance use disorders, such as self monitoring, goal setting, action planning, and feedback in relation to goals (37).

This potential recently led to the emergence of App-based technologies in order to help people with alcohol use disorders (37–40). Particularly (40), it was found that the smartphone delivered addiction-comprehensive health enhancement support system (A-CHESS) reduced the number of risky drinking days in comparison with treatment as usual controls in people with Alcohol use disorder. A-CHESS provides monitoring, information and real-time text messaging, communication and support.

Smartphone apps may therefore increase access to services and professional services for cannabis users who would otherwise not use them, or would use them only after the occurrence of serious consequences of their addiction. Furthermore, health-related smartphone apps allow people to integrate such technological support or "treatment" into their everyday lives, by way of ecological momentary assessments (EMAs) and ecological momentary interventions (EMIs) (41–43).

Nonetheless, several concerns have been reported in relation to such tools. In particular, several studies reported that the lack of concordance between the content of health-related apps available on the stores, including for smoking and substance use, and evidence-based recommended treatments, low emphasis on psychological needs related to the self-determination theory (i.e., low emphasis on autonomy, competence, and relatedness), and rarity of apps that include EMAs or EMIs (42, 44–47). Furthermore, an important gap still exists between the considerable number of apps for health available on the market and the limited number of scientific publications related to the field (48, 49).

One app for cannabis addiction has been previously described (50). This app was based on cognitive behavioral therapy and motivation enhancement therapy. Reactions of 10 cannabis users after a 2-h testing session (50) showed good overall satisfaction with the app by the participants. We recently developed a new app, Stop-cannabis, to help cannabis users stop or reduce their use of this substance.

The aim of this study is to describe the app<sup>1</sup> and to describe the characteristics of cannabis users and their levels of satisfaction with the Stop-cannabis app. The app is intended for cannabis users who want to stop or reduce their cannabis use or prevent relapse.

#### 1 http://www.stop-cannabis.ch/les-app-gratuites-stop-cannabis-ch-pour-iphoneandroid

## **App Description**

The app is associated with an Internet website<sup>2</sup> and is available on Google Play (Android) and the App Store (iOS) at no charge without any commercial advertising or costly upgrading tools. The development of this app was funded by the local Health Department in Geneva, Switzerland. As in other self-help and web-based treatments for addictive disorders [Ref. (20, 42, 50–53), the Stop-cannabis app is based on screening and brief intervention (54–56), motivational interviewing (57)], and principles of relapse prevention in the treatment of addictive disorders (58, 59). Furthermore, and in accordance with self-determination theory (60), the app particularly emphasizes competence, relatedness, and autonomy (60), important features that should be included in all apps related to addictive behaviors (45). The App was described according to this conceptual framework as shown in **Table 1**.

Launched in February 2013, the app was downloaded by 13,734 users at the time of the study, involving around 700 active users/week (people with at least one section during a week) and 2000 active users/month (people with at least one active session during a month).

## **App Components**

#### **Brief Intervention**

The brief intervention is accessible to all users, without specific registration, and is based on a brief questionnaire on cannabis use followed by a set of brief individually tailored feedback messages (**Figure 1**). The questionnaire includes the French version of the cannabis-related questions from the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) (61–63). The feedback messages are written on the basis of each participant's ASSIST scores. A feedback message is also given on the basis of the comparison between the frequency of reported cannabis use and the frequency of cannabis use by Swiss people from a similar age and sex group (percentage of people with less cannabis use or more cannabis use than the app user).

#### **Discussion Forum**

In the iOS app only, users can access a discussion forum called "La Tribu" (The Tribe), allowing written interactions with other users. Prior research suggests that this type of service is an important user demand (33). This discussion forum is moderated by a psychologist who also encourages discussions between users.

http://www.stop-cannabis.ch

#### **TABLE 1 | Access to main app components and possible links with competence, relatedness, and autonomy**.


2

**FIGURE 1 | Screenshots of the welcome page, the Feedback page, and the main screen with the benefits of change**.

#### **Personalization of the App**

To facilitate the appropriation of the tool, users can personalize the app with pictures. Such pictures may be related to personal motives to change (i.e., expected benefits, gifts, goals). These pictures can be seen the by user on the main screen at any time.

#### **Positive Reinforcement**

The main screen shows two metrics, the number of days since stopping cannabis use and the amount of money saved. Positive reinforcement is further offered in the form of awards that can be earned when reaching a given number of days since stopping cannabis use, a given number of times accessing the app, or a given number of times seeking advice.

#### **EMAs and EMIs**

The app includes some patterns of EMAs and EMIs. Users can access automatic tailored advice or encouragement at the moment of exposure to potential relapse situations, such as craving, irritability, insomnia, appetite problems, and anxiety. He or she can also indicate if a lapse occurs: the app then delivers encouraging messages in order to reduce the risk of full relapse or the negative consequences of excessive guilt associated with lapse events.

#### **Text Messaging**

Automatized personalized text messages and emails that focus on helping users give up the consumption of cannabis are sent over a period of several months. Text messaging is activated upon the user's selection of a cannabis quitting date.

#### **Association with an Internet Website<sup>2</sup>**

In addition to the app tools, at the website, users can access additional forms of help, such as information pages (psychoeducation), automated online motivational interviewing, and addresses of local clinics.

## **Materials and Methods**

A cross-sectional online survey, in a specific format that is readable from smartphones, was posted on the Stop-cannabis website. A link to the survey was included in all app messages from February 2013 to December 2014 (up to twenty messages/users during this time-frame). After reading a description of the study on their smartphone, users accessed the online survey.

The study is part of the annual reports of quality monitoring of the app. This report is part of the requirements of the funding source (Canton of Geneva) and was approved by its board. Assessments and data were reported anonymously.

The questions were specifically designed for the survey and covered sociodemographic characteristics (gender, age), frequency of cannabis use, number of joints smoked daily before stopping (**Table 2**), frequency of the app use, and satisfaction with the app (**Table 3**). Participants also provided written comments on the app (**Table 4**). The ASSIST questionnaire is not included in the assessments of the study at hand. It is a part of the brief intervention offered to the app users.

After exclusion of incomplete answers and duplicate answers emerging from the same IP address, we performed a descriptive analysis of the answers. Free comments were grouped according to categories related to the idea expressed by the participants.

Descriptive statistics were used to summarize the participants' characteristics and answers. Means and SDs as well as number and percent were reported.

#### **TABLE 2 | User characteristics**.


#### **Results**

#### **User Characteristics**

In total, 482 users aged 14–59 years were included in the analysis. Most participants (70%) were men, more than 40% were 15–25 years old, and 15% were under 20 years old. About 85%

#### **TABLE 3 | Use and satisfaction relative to the application**.


*N* = *478.*

#### **TABLE 4 | User suggestions related to the app**.


of the respondents were under 40 years old. Most participants (60%) were former cannabis users (described themselves as past cannabis users at the time of the survey). Daily cannabis use was reported by 20% of the participants, whereas about 30% reported at least weekly cannabis use and <1% had never smoked cannabis (**Table 2**). The users who never smoked cannabis were excluded from the analyses related to app's use and satisfaction (**Table 3**). Former users smoked about 5 joints/day on average before quitting (minimum: 0; maximum: 9).

#### **User Views on the App**

The app was used at least weekly by most participants, daily by more than 70%, and several times a day by half of the participants (**Table 3**). About 80% of the users reported a good overall level of satisfaction with the app ("reasonably or very satisfactory"; **Table 3**). More than 80% of users considered the app to have helped them "a little" or "a lot" to stop or reduce cannabis consumption. Similarly, more than 80% of the participants found the app messages "helpful" or "very helpful" (**Table 3**).

We collected 190 free text comments posted by 150 users. Most of them, 126, were related to the app in general (**Table 4**) and 64 comments specifically related to the discussion forum "La Tribu" (**Table 5**). Half of the comments referred to the notification messages (**Table 4**, suggestions 1 and 2). Concerning these messages, several comments recommended improving the content of the messages (e.g., "more personalized") and increasing the number of messages (especially after 4 months of abstinence, as the messages were least frequent in this period). The other suggestions were related to the following points: more personalization of tools (**Table 4**, suggestions 7, 9, 11, and 15); more information, more tailored feedback, and more rewards and training (**Table 4**, suggestions 3, 5, 10, 12, 14, and 20); and options related to the observation of one's own cannabis use and advice on the reduction rather than the cessation of cannabis use (**Table 4**, suggestions 13 and 16). Only 3% of the suggestions were related to the possibility of speaking with a health professional.

About 40% of the suggestions on the discussion forum were related to the improvement of the system used to manage messages and to options for mutual support between users (**Table 5**,

**TABLE 5 | Specific user suggestions related to the social network "La Tribu."**


suggestions 2, 3, 4, 5, and 7). Other suggestions were related to moderation of the forum, including the possibility of excluding a participant (**Table 5**, suggestions 6 and 9). The comments related to the social network mostly asked for improvement of "La Tribu" by adding tools similar to those used in other social networks, such as notification options or being able to see if a user is online.

#### **Discussion and Conclusion**

#### **Principal Results**

This study suggests that the Stop-cannabis mobile app is acceptable and perceived as useful by the users. The app was appreciated by most of the participants and was furthermore considered helpful in either stopping or reducing cannabis use. More than 80% of users considered the app to have helped them "a little" or "a lot" to stop or reduce cannabis consumption. The study was not, however, designed to assess the effectiveness of the app and cannot provide an assessment of the effect of the app on cannabis use.

About 60% of respondents reported that they no longer smoked cannabis. However, it is unclear how many of them stopped smoking cannabis after using the app or whether they had stopped before downloading it, because we did not ask this question. About one-third of the user group comprised daily cannabis users and about one-third comprised weekly cannabis users. The usefulness of the app in current cannabis users is of high interest in consideration of the generally low rate of users among this group who seek medical or psychological help (64).

Most of the open comments by users were related to the text content of the app, asking for more personalized messages, longer support time, and more personalized content. Such aspects should be considered for future developments of this app. Perceived support from text messaging seems important for users and may be reinforced by support on the discussion forum. Most of the suggestions about the social network were also related to the text messaging options, highlighting the possible importance of this kind of support between users. Few open text comments asked for more interactions with health professionals, possibly because the users do not expect such support on the app.

## **Limitations**

It is likely that there was a self-selection bias in our study and that we enrolled a disproportionate number of users who were satisfied by the app or who were more involved in its use, as suggested by other studies on selection bias in Internet-based studies (65).

## **Conclusion**

As shown in other studies related to app for substance use (13, 40, 50, 66), «stop-cannabis»was appreciated by most of the participants and seen as useful. This is in accordance with the rapid spread of e-health (67–72).

The usefulness of the app in current cannabis users is of high interest in consideration of the generally low rate of users among this group who seek medical or psychological help (64).

Text messages were considered as supportive. Users however asked for more messages with more personalized contents. This is in concordance with other studies related to user's views and preferences on text messaging (73, 74). Further studies linking messages content, the moment of delivery, interactive features, the level of tailoring and user satisfaction and preference may help to further improvements of such kind of interactions.

The app could be used for relapse prevention as well as a tool helping people who would like to prepare a quit attempt. Further studies in preparation may help to better understand how cannabis smokers use the app and what they expect from its use. Intra-app navigation analyses as well as further user' satisfaction surveys would be helpful.

The app impact on cannabis use will be assessed. Naturalistic and randomized controlled trials linking clinical characteristics of the users with outcomes related to cannabis use are warranted.

Nevertheless, it seems that tools, such as the app presented here are probably useful for at least some cannabis users, possibly

#### **References**


as a "primary intervention" in the community, as a screening tool to detect problematic cases, or as a complement to clinical interventions between medical visits.

Other studies are needed to assess the impact of such tools on cannabis use, as well as their possible use in different settings (i.e., as an adjunct to specialized treatments or general practitioner advice, or in non-clinical settings) or with different populations in terms of age, social status, or comorbid conditions.

Tools, such as the app presented here, are acceptable and probably useful for at least some cannabis users from the community.

## **Acknowledgments**

The development of this app was funded by the local Health Department in Geneva, Switzerland.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Monney, Penzenstadler, Dupraz, Etter and Khazaal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Tapping onto the Potential of Smartphone Applications for Psycho-Education and Early Intervention in Addictions

*Melvyn W. B. Zhang1 \* and Roger C. M. Ho2*

*1 Biomedical Global Institute of Healthcare Research & Technology (BIGHEART), National University of Singapore, Singapore, Singapore, 2 Department of Psychological Medicine, National University Healthcare Systems (NUHS), Singapore, Singapore*

#### *Edited by:*

*Stéphane Bouchard, Université du Québec en Outaouais, Canada*

#### *Reviewed by:*

*Taiwo Lateef Sheikh, Federal Neuropsychiatric Hospital, Nigeria Isabelle Giroux, Universite Laval, Canada Claudie Loranger, Centre intégré de santé et de services sociaux de l'Outaouais, Canada*

#### *\*Correspondence:*

*Melvyn W. B. Zhang melvynzhangweibin@gmail.com*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 29 December 2015 Accepted: 04 March 2016 Published: 17 March 2016*

#### *Citation:*

*Zhang MWB and Ho RCM (2016) Tapping onto the Potential of Smartphone Applications for Psycho-Education and Early Intervention in Addictions. Front. Psychiatry 7:40. doi: 10.3389/fpsyt.2016.00040*

E-health, and in particular smartphone-based technology, is increasingly becoming commonplace in healthcare. While psychiatry has tapped onto these innovations for conditions, such as affective disorders, and schizophrenia and psychosis, the usage of these technologies in addiction is limited. Addiction psychiatry could harness the potential of smartphone technologies. Given the increasing incidences of substance-related problems globally, and along with the normalization of the general public's perspectives toward substances, and also in consideration of unwillingness for at-risk individuals in seeking help, the authors hope to illustrate how these issues could potentially be solved using E-health and technological innovations. The objectives of the current perspective article are to illustrate how recent advances in smartphone-based technologies could help in terms of psycho-education, as well as in helping individuals who are at-risk users in seeking help earlier. The authors aim to illustrate how the above are possible, building on existing theory-driven framework that has been extensively reviewed in the previous literature. Limitations with regard to the implementation of such technologies will also be discussed.

Keywords: mental health, addiction, psychiatry, smartphone, mobile phone, E-health, m-health

## PERSPECTIVES

Zhang et al. (1) have highlighted in their recent article, how psychiatrists could be empowered in harnessing the full potential of E-health. Globally, E-health has affected the way healthcare professionals work. With the implementation and the integration of E-health into the existing healthcare framework, it has certainly improved the efficacy of work processes. E-health in itself encompasses a wide variety of technological advances, which include that of telemedicine, mobile devices, and their accompanying applications, as well as wearable devices. Over the last decade, there has been major development and growth in terms of the development of healthcare-related smartphone applications. With the launch of these smartphone applications, individuals are now able to be in control and are now able to self-manage their own conditions using their smartphone. In the field of psychiatry, there are smartphone innovations such as smartphone applications that could help to enforce medication compliance for schizophrenic patients (2). Other smartphone applications could also facilitate the self-reporting of symptoms among schizophrenia patients (3). This would enable psychiatrists to be better aware of how their patients are progressing and the stressors that their patients might be experiencing that might trigger a relapse of their underlying conditions (3). The application of technology is not limited to schizophrenia in psychiatry, but it does extend to other mood and affective disorders (4) and also substance disorders (5). Zhang et al. (1) have previously recommended two simple modalities to which psychiatrists could tap onto and harness the potential that technology has brought into healthcare. It is also important for psychiatrists to recognize the limitations of psychiatry-related application out there in the smartphone application marketplace and be able to recommend evidence-based applications to their patients. Hence, Zhang et al. (1) recommended that psychiatrists ought to use validated scales, such as the Silberg scale (6), to help them to evaluate the information quality of the existing applications. In addition, the authors (1) have also pointed out another scale, known as the mobile application rating scale (7) that has just been developed, that might be suitable for identification of applications with good evidence base. On knowing the current limitations of psychiatry-related smartphone applications in the marketplace, psychiatrists could then play a part in the conceptualization and development of smartphone innovations that not only help to mitigate the current gaps in the existing applications but also help in the conceptualization of applications that are in-line with the needs of their profile of patients (1). More importantly, psychiatrists could help in the conceptualization and development of applications that are grounded in theory and in accordance to established behavioral change models. Subhi et al. (8) have previously shared how smartphone applications could be built using just a text editor alone. However, Zhang et al. (9, 10) have highlighted the limitations of that particular methodology, and have shared their methodologies of overcoming those limitations. In particular, they have highlighted how the usage of an online application builder could help in the development of low-cost interactive applications, which could eventually be deployable to the various smartphone application stores, for ease and convenience of download.

To date, one of the most challenging issues to deal with in psychiatry currently has been the increasing incidence of substance and drug abuse and dependence. The increasing incidences of such abuse and dependency are of concern, as these disorders do predispose vulnerable individuals to other comorbid psychiatric disorders, such as affective disorders and psychosis. There are a multitude of reasons for the increasing incidences globally, but one of the likely causative factors might be due to the normalization of the general public's perspective toward drugs of abuse. Following the legalization of marijuana or cannabis in Colorado (11), it was found that there have been increases in the incidences of individuals in Colorado who have presented themselves to the emergency departments due to an acute intoxication of commonly abused substances. The availability of marijuana in the forms of edible products, such as muffin, would serve to further reinforce the idea that marijuana is safe for once-off usage and consumption. The availability of these products does have its accompanying dangers as well, not just in terms of the risk of abuse and addiction. Vulnerable individuals, such as children, might consume them unknowingly. Of note, in Colorado (12), there has been a growing incidence of children who have presented to the emergency services following an acute intoxication of these substances. It does not help that with the legalization of marijuana, perspectives toward substances have shifted. There are on-going clinical trials involving ketamine and ecstasy (MDMA), and these trials have purported that ketamine does have inherent rapid antidepressant efficacy, while MDMA would facilitate psychotherapy for individuals with post-traumatic stress disorder (13, 14). Researchers have previously highlighted how the mass media has hyped up the clinical efficacy of these drugs, such as ketamine (15), and this will further cause a mass media effect and lead to normalization of perception toward drugs and substance usage. Despite the promising trials conducted initially, there has been extensive evidence released recently by the Cochrane collaboration (16, 17) of the limited efficacy of ketamine in the treatment of depression.

Addiction specialists or even psychiatrists who have worked with individuals with dual diagnosis (with addiction as a codiagnosis) know that dealing with substance-related issue is particularly tough. Most individuals with substance-related issues as their primary problems do not routinely seek help. There has been much literature supporting this locally and overseas. Rehm et al. (18) in recent cross-sectional study highlighted that, in comparison to other mental health disorders, substance use disorder has the lowest treatment rates, despite them having the highest burden in terms of morbidity and mortality. There have been a multitude of reasons accounting for rates. The World Health Organization purported that it was in part due to the failure of primary care physician in terms of recognition of these disorders. Stigmatization mediates the help-seeking relationship and the last reason being that these individuals tend to prefer not to seek help unless they have hit rock-bottom (18).

With the advances in smartphone technologies, there have since been a lot of applications that have been developed for addiction. Previous content analysis done for Marijuana-related applications (19) has highlighted the concern about the evidence base and the information quality of the existing applications on the store. Most of the applications are for entertainment purposes, and most of these applications do not provide accurate information about the dangers associated with the usage of Marijuana (19). Another prior content analysis (20) done for alcohol-related application highlighted the fact that most of the current alcohol applications made available in the store attempt to track the amount that an individual has drunk using the blood alcohol concentration method (20). The content analysis highlighted that such methodology might results in individuals drinking even more, as an attempt to challenge their pre-set limits (20). A keyword search of the existing research literature online reveals that there are to date several published literature about how technology could help in substance-related disorders. Prior review done by Keoleian et al. (21) has looked at the efficacy of the utilization of text messaging for addiction. The 11 studies (which included smoking, alcohol, and marijuana and methamphetamine addiction) that they have looked at have highlighted that text messaging is effective in helping individuals in achieving abstinence. For alcohol addiction, prior research has demonstrated that smartphone-based brief interventions could help targets drinking choices amongst University students (Gajecki et al.) (22). ACHESS, another smartphone application to support patients in their recovery from alcoholism has its basis on self-determination theory (McTavish et al.) (23). The ACHESS smartphone application helps to prevent users from heavy drinking post treatment, using features such as geo-location services to locate individuals when they are close to geographically high-risk locations. Other functionality implemented includes that of a mood and a withdrawal questionnaire and charting. LBMI-A (24) is another application for intervention for alcohol issues and provides individuals with assessment, information, and intervention.

The objectives of the current perspective article are to illustrate how recent advances in smartphone-based technologies could help in terms of psycho-education, as well as in helping individuals who are at-risk users in seeking help earlier. The authors aim to illustrate how the above ones are possible, building on existing theory-driven framework that has been extensively reviewed in previous literature. Limitations with regard to the implementation of such technologies will also be discussed.

#### SMARTPHONE AS PSYCHO-EDUCATIONAL TOOLS

Prior research has described how smartphones could educate individuals who are diagnosed with HIV in risk reduction (25). More recent research has demonstrated the utility of web-based alcohol screening and brief intervention tools for University students (26). The literature review above demonstrated that there are existing applications that provide information about substances. While there might be existing interventions designed to educate the general public about substances, it is concerning that most of the other applications on the application stores do not provide the similar quality of information, in accordance with the findings of the previous content analysis of existing applications. Addiction psychiatrists could, thus, play a further role in the conceptualization and the development of new applications that aims to provide further information about substances, much like what the Royal College of Psychiatrists in the United Kingdom has done. Rosenbaum (27) have proposed that in order for education about drugs to be successful in today's world, taking into consideration the recent drive toward legalization, it is thus of importance that programs developed have a scientific basis, and that these programs would encourage moderation of usage. In addition, apart from the provision of scientific information, it is pertinent now for applications to include relevant legal statues and laws to deter further experimentation.

Prior to the introduction of the web as well as smartphone technologies, the mass media has been utilized for various substance-related problems. The Cochrane collaboration (28) previously evaluated mass media interventions for smoking cessation and has concluded that these interventions are deemed to be effective in smoking cessation. Mass media interventions include that of the television, radio, newspapers, billboards, and also posters and leaflets. Other studies (29) have also demonstrated the utility of videos in encouraging individuals to adopt healthy behaviors, such as going for HIV testing. The Royal College of Psychiatrists in the United Kingdom has always been a strong advocate for psycho-education of the general public to increase public's literacy about common mental health disorders. The previous mechanism of delivery of such materials was via their online website as well as printed brochures. Recently, the college has tapped onto smartphone technologies and has launched the RCPsych Mental Health Application to provide members of the general public with easier access to a smartphone friendly version of this information. The conceptualization and the development of the application are led by the first author, making use of the methodologies previously described by Zhang et al. (9, 10). The newly developed version of the smartphone application has not only included the traditional print materials but the smartphone application has also included animations and podcasts in order to provide users with alternatives to gather the information that they require.

Thus, taking into consideration the limitations of existing applications on the store, and the recent recommendations by Rosenbaum (27), and also taking into consideration how prior mass media interventions have been deemed to be useful in promoting health behaviors, the authors hope to share their conceptualization of a smartphone-based intervention that is based on existing theories. The methodologies described previously by Zhang et al. (9, 10) are limited in the sense that those methodologies described do not enable the addition of interactive features into smartphone applications. In addition, there are inherent difficulties with the publication of these applications to the application stores. It is of importance for these applications to be featured on the stores, so that it is more accessible for the general public to download. With further advances in technology, there are more strategies that could enable psychiatrists to create evidence-based application that are in-line with what they feel patients and the general public might benefit from. Apache Cordova (30) is one such potential method, which could enable psychiatrists to create cross-platform compatible application. Given the inherent effectiveness of videos, it is thus envisioned that in the current application, there will be integration of animation, info graphics as well as videos to better communicate key messages about drugs to individuals. There should also be other interactive features, such as that of a geo-location service that would be able to locate the precise location of the user. If users are intoxicated after using drugs, they would be able to tap onto the application to look for the nearest medical facility to receive treatment from. This conceptualization is based on the theory-driven model of identification of high-risk locations for alcohol addiction, as previously described. **Figure 1** illustrates the core features of an evidence-based drug psycho-education smartphone application.

#### SMARTPHONE INTERVENTIONS FOR AT-RISK USERS

As discussed previously, one of the key challenges in addiction psychiatry is that the vast majority of individuals do not seek help for their issues. Various studies have highlighted various reasons to account for this. Harm reduction has been widely looked into and is currently one of the intervention strategies in addiction psychiatry. Logan and Marlatt (31) have proposed

harm reduction as one of the intervention strategy for youths who are at-risk for addictive behaviors, such as alcohol addiction. With regard to the problem of at-risk drinking, prior studies have also demonstrated the superiority of controlled drinking over abstinence approaches (Marlatt and Witkiewitz) (32). With the recent legalization of substances, such as marijuana, and with the push for the usage of illicit substances, such as ketamine and MDMA for psychiatric treatment, there might be normalization of the perspectives held by the general public toward drugs. It is important to target those who are at-risk users early, as the earlier an intervention is implemented, the better the resultant prognosis of the underlying condition.

As reviewed previously, most of the current literature showed that to date most of the interventions are geared toward supporting the recovery process of patients with addiction issues. While there is a wide variety of smartphone-based applications out there in the application stores, the previous content analysis have clearly highlighted the drawbacks of such applications, in that the majority of them are lacking in terms of evidence base. Given that the objective of this article is to highlight how smartphone technologies could be applied in the domain of addiction psychiatry and in particular, applied for at-risk individuals, using theorydriven framework, the authors would describe two smartphone applications that they have conceptualized and developed. The theoretical framework underpinning these interventions is that of harm reduction.

The alcohol tracker application has been previous described by Zhang et al. (34). The currently available alcohol trackers on the application store attempts to minimize harm among at-risk drinkers through enabling them to self-monitor their blood alcohol concentration. Some applications do not seemed to help reduce drinking as it promotes and encourages users to challenge their pre-defined limits. Thus, instead of tracking the absolute amount of drinks that an individual has had and converting it to a blood alcohol concentration, and advising the individual when it would be safe for them to engage in their routine activities, such as driving again, the conceptualization by Zhang et al. (34) attempts to empower individuals in helping them to track the amount that they have drank and warning them if they are over the limits as stipulated by the respective guidelines. The alcohol tracker application conceptualized (34) makes no recommendation of the timing to which individuals are deemed safe to return to their original activities. In addition, in order to ensure that individuals who are deemed to be at-risk would be able to receive help, the alcohol tracker application incorporates information not only about alcohol abuse and dependence but has also links to a direct line to which individuals could call (if they are in Singapore) and they will be advised about how best to receive help. Despite the fact that the conceptualization of the application is done in Singapore, much of the application is relevant for an international audience. There are not much variant in terms of the recommended guidelines with regard to

the number of units individuals are to consume in a day and a week, and the information of alcohol abuse and dependence and adverse medical risks associated with alcohol also does not differ globally. Collaborators from the University of British Columbia were, thus, invited to help evaluate the conceptualized applications and to gather user perspectives with regard to the new conceptualization, that has a theoretical basis (34). A group of 100 Canadians were recruited in order to gather user perspectives. The results obtained showed that notification and information were regarded as the most useful functions of the innovative application (34).

The alcohol tracker application is just one of the two smartphone innovations that the authors wish to highlight that could potentially help individuals who are at-risk. The only smartphonebased intervention for gamblers seemed to be limited to an online gambling self-help workbook currently, which has its theoretical basis on harm reduction and behavioral strategies (35). Users are asked to identify high-risk situations and provided with personalized feedback with regard to dealing with these situations; and, in addition, they are also able to compute their financial risks (35). In order to help at-risk gamblers, the authors have jointly conceptualized the eGambling Intervention application (36). The eGambling Intervention smartphone application might seem that it is only capable of providing general information about gambling addiction and how best to seek help and treatment. However, the main developer (MWBZ) has actually made use of the build in global positioning sensor in the smartphone to track the position of the individual. The locality and the position of the individual would be tracked and with the coordinates updated once every 20 s and feedback to a server. Once the individual is within the proximity of the 25 gambling locations in Singapore, a notification is sent to them. The number of notifications sent and the duration will be tracked and sent back to an Apache Server. In addition, there is the enhanced functionality of allowing individuals to phone their family members or even their counselors to get help. If they chose not to phone, they could also get help by means of an email notification. Thus, by means of the application, the authors are able to determine the approximate number of visits individuals make to the casinos to gamble. One of the potential limitations is that the authors have not tapped onto the other sensors, in order to understand more about the individual's behavior prior to the gambling episode. The integration of other sensors to track the individual's behavior prior to the activity would open up opportunities for ecological momentary analysis. While the application does not allow for ecological monetary analysis currently, the theoretical framework underpinning this intervention is once again that of harm reduction, in preventing users from frequenting casinos; as well as behavioral approaches that have been previously described in the literature, by enabling users to identify high-risk locations and providing interventions accordingly.

**Figure 2** illustrates the core features of each of the application.

#### POTENTIAL LIMITATIONS AND ETHICAL CONCERNS

Smartphone applications might seemingly be the answer and solution for addiction-related psychiatric problems. However, it is still key to recognize that they are certainly not a replacement for a consultation with a mental health specialist. The assessment that applications could do is limited. In addition, smartphone applications might not reach out to those who do not have access to smartphones and their accompanying applications, or who have no knowledge with regard to how best to use smartphone applications. While psychiatrist could help in the conceptualization and development of the applications, it is still key for applications to be continuously kept updated, even after publication.

#### REFERENCES


In addition, gathering patient information on smartphone devices raises concerns about ethics and security.

## CONCLUSION

E-health, and in particular smartphone applications, is increasingly becoming commonplace in healthcare. While psychiatry has tapped onto these innovations for conditions, such as affective disorders, and schizophrenia and psychosis, the usage of these technologies in addiction is limited. Addiction psychiatry could harness the potential of smartphone technologies in educating the masses about the harmful effects of drugs. This is particularly important given the changing perception held by individuals toward commonly abused drugs, as more drugs are being legalized or might be legalized for medical usage. Smartphone technologies incorporating theory-driven framework could be harnessed and used as interventional tool for those who are at-risk for the development of addiction. However, there remain limitations to the usage of such technologies that should be carefully considered.

#### AUTHOR CONTRIBUTIONS

All authors have contributed equally to the manuscript.


students: a randomized controlled study. *Addict Sci Clin Pract* (2014) **9**:11. doi:10.1186/1940-0640-9-11


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Zhang and Ho. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **Quality of smartphone apps related to panic disorder**

*Mathias Van Singer <sup>1</sup> , Anne Chatton<sup>2</sup> and Yasser Khazaal 1,2 \**

*<sup>1</sup> Geneva University, Geneva, Switzerland, <sup>2</sup> Geneva University Hospitals, Geneva, Switzerland*

Quality of smartphone apps related to panic: smartphone apps have a growing role in health care. This study assessed the quality of English-language apps for panic disorder (PD) and compared paid and free apps. Keywords related to PD were entered into the Google Play Store search engine. Apps were assessed using the following quality indicators: accountability, interactivity, self-help score (the potential of smartphone apps to help users in daily life), and evidence-based content quality. The Brief DISCERN score and the criteria of the "Health on the Net" label were also used as content quality indicators as well as the number of downloads. Of 247 apps identified, 52 met all inclusion criteria. The content quality and self-help scores of these PD apps were poor. None of the assessed indicators were associated with payment status or number of downloads. Multiple linear regressions showed that the Brief DISCERN score significantly predicted the content quality and self-help scores. Poor content quality and self-help scores of PD smartphone apps highlight the gap between their technological potential and the overall quality of available products.

#### *Edited by:*

*Shervin Assari, University of Michigan, USA*

#### *Reviewed by:*

*Erika Comasco, Uppsala University, Sweden Theodoros Michelakos, University of Athens, Greece*

#### *\*Correspondence:*

*Yasser Khazaal, Geneva University Hospitals, Grand Pré 70 C, Geneva 1202, Switzerland yasser.khazaal@hcuge.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 06 March 2015 Accepted: 15 June 2015 Published: 14 July 2015*

#### *Citation:*

*Van Singer M, Chatton A and Khazaal Y (2015) Quality of smartphone apps related to panic disorder. Front. Psychiatry 6:96. doi: 10.3389/fpsyt.2015.00096* **Keywords: Internet, apps, smartphone, panic disorder, anxiety disorder, cognitive behavior therapy**

## **Introduction**

Panic disorder (PD) is a common anxiety disorder associated with an important social and economic burden (1, 2). Available treatments include pharmacotherapy and cognitive–behavioral therapy (CBT) (3). Such treatments have, however, been insufficiently disseminated in clinical settings (4).

Smartphones are widely used worldwide (5, 6) and have a growing role in health care (7– 10). Concerns about the regulation of medical smartphone apps are, at the same time, rising among public administrations and the scientific community (9, 11, 12). The U.S. Food and Drug Administration regulates some health-related apps, as it does medical devices. Smartphone apps related to psychiatric conditions do not yet fall under this regulation (13).

A number of recent studies have assessed the quality of medically oriented apps in various fields, such as smoking cessation, weight management, sleep, cancer, and diabetes (14–36). While acknowledging the potential opportunity offered by apps-related technologies, these studies concluded that the apps available from different stores, with few exceptions, were of overall poor quality. A gap was furthermore found between the considerable number of apps related to medical conditions available in stores and the low number of peer-reviewed papers about them (37). In particular, despite their potential to improve health care, mental health apps currently available in stores lack scientific evidence about their efficacy (38). With few exceptions (39–41), preliminary findings reported for health apps were similar to previous findings on the poor quality of health information websites (42–46).

One may hypothesize that persons with PD could benefit from the development of well-conceived apps. Indeed, Internetbased CBT has previously been shown to offer some efficacy in PD (47–50), and several studies are under way to evaluate apps designed specifically for the treatment of PD (51, 52). Nonetheless, developers have not waited for scientific evidence to create apps for PD, as many are already available on smartphone stores. To our knowledge, no studies have yet been performed to rate these apps.

In the present study, we aimed to assess the quality of Englishlanguage PD-related apps available on the Google Play Store like any layperson searching for an app related to PD on the Google Play Store. It is a descriptive and exploratory study of what it is possible to find. Furthermore, the study aimed to compare free and paid apps. We furthermore assessed the factors associated with the main quality indicators, as well as the links between the quality indicators and users' ratings (star ratings, as reported on the Google Play page) and downloads.

#### **Materials and Methods**

#### **Selection of Apps**

A keyword search was performed between February and March 2014 to produce a comprehensive list of PD-related apps that were accessible in English on the Google Play Store. The Google play account was set to English United Kingdom language and linked to a mobile phone, which was registered on a Swiss mobile network.

Google is the developer of the Android operating system, the most widely used smartphone operating system in the world (53). The following queries were entered into the Google Play search engine: "stop panic," "stop panic attack," "panic attack," "PD," "anxiety attacks," and "anxiety disorder."

Studies of Internet users have shown that most people rarely search beyond the first 20 retrieved results (54). However, we extended the coverage of the present study to the first 50 free apps and to the first 30 paid apps for each tag to obtain the most comprehensive list of apps. Apps were included if they were related to PD. Exclusion criteria were as follows: the app could not be downloaded after more than three attempts, the app was not in English, or the app was a book or an article.

#### **Evaluation of Apps**

Apps were reviewed on an HTC One Android 4.3. They were assessed by using tools reported in previous studies, tools adapted from quality evaluation studies of websites (55–59), and tools described in other studies on the quality of smartphone apps (15, 16, 23, 30, 31). The assessment instruments are described below.

#### Google Play's Page and Functionalities of Apps

As reported by other investigators (15, 30), we extracted a number of items from the Google Play page, such as number of downloads and ratings of the apps.

#### Self-help Model

A self-help model assessment tool for PD was used (**Table 1**). The model was based on the potential of smartphone apps to help users in daily life and on the second edition of the *Practice Guideline for the Treatment of Patients with Panic Disorder* by the American Psychiatric Association (3).

#### Content Quality

As in other studies on Internet websites related to mental health disorders (39–42), evidence-based content quality was assessed according to the availability of information related to the following questions that a patient could search for:


Answers found on the apps were assessed on the basis of the American Psychiatric Association practice guideline (3). For every request, the coverage (the extent to which the question was addressed) and correctness (the extent to which the answer was right) of the answer were comprehensively scored on a 3-point scale (0 = absent; 1 = partially incorrect or incomplete; 2 = correct and complete). A total content quality score, ranging from 0 to 14, was calculated by combining the scores.


#### Interactivity

Interactivity (**Table 2**) was measured with an adaptation of the Abbott scale (58). Three items were added to the scale: presence of a gamification module, possibility of personalizing the user's profile (avatar, color, sound), and tailoring of the app upon use.

#### Health on the Net Code

The health on the net code (HON) label (55) was created for websites that focus on ethical standards in online publishing. Usually, a website requests evaluation, after which the label is awarded. In the absence of the common use of this label by the apps, we assessed whether they respected the HON criteria (**Table 2**).

#### Brief DISCERN

The Brief DISCERN (56) is a six-item (**Table 2**) assessment tool adapted from the DISCERN instrument (60). It is used as a potential indicator to estimate the quality of the information about the choice of treatment in websites. The Brief DISCERN includes six items on a five-point scale (1 = not at all; 5 = completely). The first two items identify the transparency of information sources; the other four items estimate the quality of the information regarding treatment. A cutoff score of *≥*16 has previously been associated with good content quality scores of health-related websites (56).

#### Accountability

Accountability was estimated with the Silberg scale (57), which includes authorship (names of authors, affiliation, and references), attribution (sources and references), disclosure (property of site, sponsorship, and advertising), and currency (date of creation, modification of site, and updating in the last 6 months). A total score ranging from 0 to 9 (1 point for each item if present) was calculated for each app (**Table 2**).

#### **Statistical Analyses**

Statistical analyses were performed with SPSS software (version 18.0, Chicago, IL, USA). An initial exploratory analysis involved the calculation of proportions, as well as means and SDs, of the above-mentioned outcome measures. Next, we compared paid apps with free apps in bivariate analyses by using parametric tests (*t*-test, chi-square, or Fisher's exact test) or non-parametric tests (median test) when appropriate. Finally, we computed prediction models by using multiple linear regressions for two variables of interest.

Applying the principle of model parsimony and statistical relevance, the selection of the independent variables was driven by the objective to find the simplest model (i.e., contain a small number of variables):

#### (1) that adequately fits the data and

(2) that explains most of the variance in the dependent variable (the highest value of R2).

Beforehand, the relevance of the independent variables for the prediction of each dependent variable was discussed among the authors. Taking into account multicollinearity and using the «Enter»method, several subsets of these independent variables were regressed and the model with the least number of variables

#### **TABLE 2 | Assessment scales**.


that still explain a percentage of variance in the dependent variable that is comparable to the percentage explained with all the variables in the equation was retained.

On the one hand, the prediction of content quality was fitted by using the Abbot interactivity scale, the Brief DISCERN score, the number of installs, and the link to paid content (yes vs. no) as the independent variables, and by controlling for payment status (free apps vs. paid apps). On the other hand, the prediction of self-help was assessed with the Brief DISCERN score, whether the app was recommended (yes vs. no), the link to paid content (yes vs. no), and the Silberg accountability scale as the independent variables, controlling for payment status (free apps vs. paid apps). For all analyses, a significance level of *p ≤* 0.05 was used.

Payment status was hypothesized to matter as a potential confounder on the ground that it could moderate the impact of the

independent variables on the dependent variable. Using the 10% change rule of thumb, confounding was considered present if the measures of association changed by 10% or more and absent otherwise. Measuring this change implied running the model twice: without and with the confounder.

#### **Results**

A total of 480 apps were found (50 free apps and 30 paid apps for each of the six keywords). The search with these keywords highlighted several duplicates among the apps identified. After their removal, of the 247 remaining unique apps, 52 were retained for analysis and 195 were excluded (**Figure 1**).

One app offered only a self-assessment tool aiming to help user to screen for a possible PD. The other apps are designed to help user manage their symptoms (via information, assessments, and techniques to deal with PD). Most of the apps (58.1% of free apps and 61.9% of paid apps) recommend the user to consult a medical doctor if suffering from symptoms of PD.

There is furthermore a wide variability on the contents of the apps. For example, one of the most downloaded app offers features like psychoeducation, audio's for relaxation, and mindfullness as well as a diary tool to record panic events. Some apps offer interactive modules to face panic attacks. For example, one app tells the user to control his breathing while giving instruction on the screen with pictograms. Other apps offer discussion forum. Some applications are selling various products, such as books or medicinal herbs.

The characteristics of paid and free unique apps are reported in **Table 3**.

The two subgroups, paid and free apps, were similar, although we found several differences. For instance, the variable publicity appeared to be present in 61.3% of the free apps group but in none of the paid apps group, with a *p*-value of *<*0.0005. In addition, the HON criteria were more fulfilled in the paid apps group than in the free apps group (*p*-value 0.002). As shown in **Table 3**, the overall adherence to HON criteria was low: none of the assessed apps fulfilled the eight criteria. Significantly more apps that were designed to be stand-alone (no additional app content) were found in the paid apps group than in the free apps group (*p*value 0.001). Both paid and free apps groups had low content quality and low self-help scores, with no differences between the two groups.

#### **Regression Results**

Content quality was regressed on a set of independent variables, namely the Abbot interactivity scale, the Brief DISCERN score, the number of downloads, the link to paid content controlling for payment status. We found that the Abbot interactivity scale and the Brief DISCERN score significantly predicted the content quality (*p* = 0.01 and *p <* 0.0005, respectively), but not the number of downloads (*p* = 0.9), or the link to paid content (*p* = 0.1). After careful examination of the regression coefficients without and with payment status, no confounding effect could be imputed to this variable. The full model performed well with an adjusted *R* 2 of about 70%.

Another regression model predicted the self-help score with the Silberg accountability scale, the Brief DISCERN score, whether the app was recommended, and the link to paid content as the independent variables, controlling for payment status. This model performed less well than the preceding model, as shown by an adjusted *R* 2 of 54.4%. We found that the Silberg accountability scale (*p <* 0.0005) and the Brief DISCERN score (*p* = 0.03) significantly predicted the self-help score, but not whether the app was recommended (*p* = 0.4), or the link to paid content (*p* = 0.5). After due consideration of the regression coefficients without and with payment status, we did not detect a confounding effect of this variable.

#### **Discussion**

In this study, we aimed to assess the quality of English-language smartphone apps for PD. In particular, we evaluated the content quality and self-help scores with instruments that were adapted from other studies on the content quality of medical websites and from previous assessment studies on medical smartphone apps.

In consideration of the lack of specific studies with similar purpose to our study, and of the potentially high interactivity of apps in daily life, we adapted some assessment tools for the study herein. Abbott's scale of interactivity (58) was adapted to match the specificity of smartphone apps, as described in the Section "Materials and Methods." Another important adaptation was the




*Summary statistics report variable mean and standard deviation (SD), or percentage, as appropriate.*

self-help tool. It may be a helpful indicator for apps assessment and development, especially for those based on CBT treatment models. The low scores obtained with the tool may underline the gap between the clinical potential offered by apps technology and its rather low level of clinical development. The self-help model was based on CBT for two reasons: first, its easy translation into eHealth, as shown by important developments related to Internetbased therapy for mental health disorders (48, 61–64); second, its validity for the treatment of PD (3). The self-help model score is probably also useful for other CBT treatment apps. Further studies on apps for mental health and about the instruments proposed here are warranted.

Similar to the results reported in other studies on health-related apps, the mean content quality (14–16, 19, 23, 30, 31, 34, 35) and self-help scores were low in our study. Most apps were insufficiently evidence based; furthermore, the technological capabilities were underused in most of the available PD-related apps. As shown in website studies (40, 41, 56), the Brief DISCERN score (56) is linked to content quality scores, as well as to the self-help score specifically developed for CBT-based app assessment. In the present study, measures such as accountability and interactivity were associated with the main quality indicators, such as content quality and self-help scores, as was previously found in some (39, 41, 65), but not all (66), studies on health-related websites.

Factors related to the community success of a given app, such as the number of downloads and whether the app was recommended, as well as factors linked to the economic model, such as payment status or a link to paid content, were not associated with content quality or self-help scores. This is somewhat surprising, particularly in regard to the number of downloads. One might expect better quality for the most downloaded apps. The results are possibly limited by the assessments of apps found only on the Google Play Store as well as by the small number of apps with a high amount of downloads (only three apps with more than 5000 downloads).

The number of active users (unavailable on the Google Play Store) would, however, probably be more informative for the sustained success of a given app after the initial download.

Payment status was not associated with the quality indicators assessed. Further studies may assess in more details the commercial strategy linked to the development model related to healthrelated apps.

The link found between payment status and publicity reflects some differences in the commercial model. Other aspects should, however, be included in further assessments (i.e., marketing strategies, interaction with users.).

Our study contains several limitations. We assessed only apps from the Google Play Store and not the Apple Appstore or others. This aspect limits the generalization of the study findings.

In addition, the keywords for the search used in this study might be different from those used by people with PD, and we may have

#### **References**


missed PD-related apps on the Google Play Store. Furthermore, the results may differ depending on the country and the language setting of the Google Play store.

Nonetheless, the study suggests possible modifications to medical eHealth assessments through the proposal of adaptations to apps. Despite expectations about the potential of PD apps to improve treatments (51, 52), the apps available to users from stores to date need to be improved and to include more patterns of evidence-based information, more interactive assessments, such as ecological momentary assessments (67), and more self-help options.

#### **Acknowledgments**

The authors thank Barbara Every, ELS, of BioMedical Editor, for English language editing.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Van Singer, Chatton and Khazaal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Augmentation of Psychotherapy through Alternative Preconscious Priming: A Case Series Exploring Effects on Residual Symptoms

*Melha Zidani1,2, Jean-Sébastien Audet1,2, François Borgeat1,2,3\*, Frederick Aardema1,2, Kieron Philip O'Connor1,2 and Yasser Khazaal2,4,5*

*1Department of Psychiatry, University of Montreal, Montreal, Canada, 2Research Center of the Montreal University Institute of Mental Health, Montreal, Canada, 3Montreal University Institute of Mental Health, Montreal, Canada, 4University of Geneva, Geneva, Switzerland, 5Geneva University Hospitals, Geneva, Switzerland*

#### *Edited by:*

*Matthias Jaeger, University Hospital of Psychiatry Zurich, Switzerland*

#### *Reviewed by:*

*Andres Ricardo Schneeberger, Psychiatrische Dienste Graubuenden, Switzerland Niklaus Stulz, Psychiatric Services Aargau AG, Switzerland*

*\*Correspondence: François Borgeat francois.borgeat@umontreal.ca*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 27 September 2016 Accepted: 11 January 2017 Published: 31 January 2017*

#### *Citation:*

*Zidani M, Audet J-S, Borgeat F, Aardema F, O'Connor KP and Khazaal Y (2017) Augmentation of Psychotherapy through Alternative Preconscious Priming: A Case Series Exploring Effects on Residual Symptoms. Front. Psychiatry 8:8. doi: 10.3389/fpsyt.2017.00008*

The current paper describes a case series using a new strategy for facilitating change based on Augmentation of Psychotherapy through Alternative Preconscious Priming (APAP) (1) in the treatment of eight treatment-resistant patients suffering from social phobia or generalized anxiety disorder. The patients had previously only shown a partial response to cognitive behavioral therapy (CBT) despite good treatment adherence. The patients completed APAP using a computerized program, which consisted of three steps during which alternative, more functional thoughts and beliefs relevant to the idiosyncratic difficulties experienced by the patients were formulated. Subsequently, these formulations were recorded and mixed with masking relaxing music, which the patient listened to in a passive state twice daily for 20 min for a period of 8 weeks. This case series aimed to assess the effect and acceptability of APAP using quantitative and qualitative measures administered before, after, and 16 weeks' posttreatment. Results showed a reduction in dysfunctional idiosyncratic thoughts reported by most patients, as well as mild improvements in anxiety and important improvements in quality of life. APAP could be a valuable addition to CBT by facilitating or enhancing cognitive and symptom change. Further studies are needed to confirm these promising results.

Keywords: priming, preconcious, social anxiety disorder/social phobia, generalized anxiety disorder, psychotherapy research

#### INTRODUCTION

Anxiety disorders, as defined by the fifth edition of the Diagnostic and Statistical Manual of the American Psychiatric Association (2), represent the most frequent mental health problems (3). According to the public health agency of Canada, anxiety disorders currently affect about 12% of the population and create light to severe impairment (4). Their lifetime prevalence in the general population varies between 16 and 29% (5). Anxiety disorders also represent a major economic burden for society in terms of direct costs, such as psychiatric or general health services, and indirect costs, such as lost of productivity and unemployment (6).

#### Treatment of Anxiety Disorders

Treatment for anxiety disorder usually consists of antidepressant and cognitive behavioral therapy (CBT) (7). CBT stipulates that cognitions are modifiable through reasoning and that these modifications produce change in emotions and behaviors, which are central in psychiatric disorders (8). CBT is the most studied form of psychotherapy and seems to be the most effective psychotherapy for the treatment of anxiety disorders (3, 9). Yet, many patients completing individual or group CBT have residual symptoms and they, or their therapist, still feel that there is room for improvement (10). Old habits of thinking might resist CBT in those with high anxiety and limited or slow change in psychotherapy can occur even in the presence of a positive patient–therapist alliance and good adherence to treatment (11, 12). One of the factors involved in resistance to change could be that psychotherapists face the very difficult task of helping patients solve long standing emotional and irrational problems with treatment methods that are almost always based on rational and conscious communication.

#### Preconscious Priming

People suffering from anxiety disorder have difficulties generating positive scenarios compared to control participants (13), which might account for part of the limited effectiveness of treatments where alternative cognitions or scenarios must be generated rationally by the patient, such as CBT. Priming is a technique by which a stimulus previously presented makes a related but new stimulus more readily available. This should be a potential candidate for helping participants generating or accessing different cognitions. This paradigm has been used mainly in cognitive research, particularly for word recognition. Typically, a word (e.g., cars) is presented, followed by either a word of the same semantic category (e.g., Ferrari) or a different semantic category (e.g., cattle). Results show that the word of the same semantic category is then recognized faster because it has been primed, i.e., is more readily accessible to consciousness (14). If priming can make words more readily available in laboratory word recognition tasks, it could logically produce similar effects in natural settings (15). Hence, the results of priming studies suggest that a repeated activation or priming of cognitions, attitudes, and related emotions can also render a set of positive cognitions and self-statements more readily available to patients.

Priming is postulated to be mostly a preconscious phenomenon (15). Research concerning preconscious processes mainly involves the effect of stimuli presented long enough to be perceived, but not long enough to enter consciousness. For example, Ohman and Soares (16) successfully induced phobic responses by presenting pictures of fearful faces under the threshold of conscious recognition. Masked priming (i.e., unconscious priming) is also more effective than non-masked priming at inducing affective effects (17).

#### Alternative Preconscious Priming

The procedure presented in this study is termed APAP for Augmentation of Psychotherapy through Alternative Preconscious Priming. The APAP strategy is based on the notion that priming and masked priming phenomena could render a set of positive cognitions, attitudes, and emotions more available to patients undergoing CBT. A previous randomized controlled study (1) has shown that the addition of APAP to CBT facilitated cognitive change and was associated with increased positive cognitions and decreased negative cognition in social phobia. This study also suggested that patients with important residual anxiety following CBT could benefit more from APAP than patients with less residual anxiety. A recent case report also demonstrated the effectiveness of APAP in a patient with severe social phobia (18).

#### Aims

The current case series adds to the previous randomized controlled trial in two ways: first, the subjects were treatment-resistant social anxiety disorder (SAD) or generalized anxiety disorder (GAD) patients still showing important residual symptoms and, second, a new computerized program available through the internet and linking all the steps of the procedure was employed. The following questions were explored: Were residual symptoms of SAD or GAD improved by the addition of APAP? Was the use of the APAP program available through internet viewed as user friendly? Was the possibility of individual work at home with the possibility of relying on a therapist through internet judged positively?

#### METHOD

#### Subjects

Nine SAD or GAD patients were recruited at a specialized anxiety clinic. This clinic receives patients deemed treatment resistant after two unsuccessful treatment attempts, either with drugs or psychotherapy, but more frequently following two unsuccessful pharmacological attempts. Patients were referred to the clinic by their physician and received individual or group CBT. The format, duration, and intensity of the treatment were personalized to the patient. Participants were referred to the study by their therapist (psychologist or psychiatrist) based on important residual symptoms.

Of the nine participants entering treatment, one participant desisted because of a lengthy stay outside of the county, leaving eight completers (age range: 22–71 years old; five women; four SAD, and four GAD). Diagnostics were obtained through clinical judgment of professionals and were confirmed through previous diagnostics.

Inclusion criteria were (1) being over the age of 18 years old; (2) having a primary diagnostic of SAD or GAD; (3) presenting residual symptoms after CBT despite good treatment adherence; and (4) no evidence of substance or alcohol abuse within the past 6 months. Participants were excluded if they had (1) a current primary mood disorder, except for premorbid dysthymia or (2) a history of psychotic disorder or eating disorder.

Participants gave consent prior to the beginning of the study and were assessed by the first and third author at baseline. Participants' age and diagnostics are presented in **Table 1**. All names have been changed to preserve anonymity.

#### Table 1 | Participants' age and diagnostics.


*GAD, generalized anxiety disorder; SAD, social anxiety disorder; OCD, obsessive– compulsive disorder; ADD, attention-deficit disorder.*

#### Measures

#### Beck Anxiety Inventory (BAI)

The BAI (19) is a 21-item questionnaire measuring the intensity of anxiety symptoms in the past week. As with the English version, the French version present a good internal consistency (α = 0.84) and a good test–retest fidelity (*r* = 0.63) (20).

#### Hamilton Anxiety Rating Scale (HARS)

The HARS (21) measures state anxiety using 14 items. These items cover the following dimensions: psychic anxiety, somatization, cognitive impairments, sleep difficulties, and depressive mood. The interrater reliability is excellent (*r* = 0.89) (21) and convergent validity with other measures of anxiety is good (*r* = 0.51). The French validation was used in this study (22).

#### Quality of Life Systematic Inventory (QLSI)

The QLSI (23) measures participant's quality of life by measuring their capacity to reach their personal goals in nine different life spheres. The QLSI presents a good test–retest reliability (*r* = 0.84) and good divergent validity with social desirability (*r* between 0.11 and −0.13) (23).

#### Idiosyncratic Measure of Convictions in Thoughts

This measure was designed specifically for the study and was inspired by the postulate of cognitive theory according to which change in cognition brings change in emotions and behaviors (8). Many other studies have used this methodology (24, 25) and this type of measure has been previously validated (26). The idiosyncratic measure asks participants to name the thoughts that would be targeted by APAP and to rate the degree of conviction in each thought in percentage. Thoughts targeted by APAP were decided by the patient and the therapist (see Procedure), but the ratings were provided by the patients independently of therapist. The mean of the ratings was used as a total score to effectively compare participants.

#### Procedure

The study was approved by the local ethics and research committees. Participants with residual symptoms still causing disturbances were approached at the end of their CBT and invited to participate in APAP. Participants were informed that APAP could help to free them from a cognitive rut by making different cognitions more available to them. If the participants agreed to participate, individual sessions with FB and the participant's therapist were organized to explain the procedure. The participants received at least two sessions face to face explaining APAP, its rationale, and how the procedure could be adapted to their situation. Afterward, participants had the option of continuing the therapeutic work by themselves over the internet through the APAP website1 or by doing it partly or totally face to face with the therapist using the web interface together. Information was exchanged *via* the APAP website's interface for patients choosing to work from the internet to help specify the targets of change. The therapeutic work proceeded as follows: (1) recap on difficulties regarding change in cognition and inner discourse and explanation of APAP rationale (this step occurred in person); (2) exploring the residual problematic cognitions and related emotions and mental imagery (case formulation with the patient and her/his therapist using the APAP website); (3) making a list of plausible alternative and desired personal cognitions (formulated by the patient with the input of her/his therapist using the APAP website); and (4) explaining the listening modalities, which were: passive, effortless listening, and at least 20–30 min a day for 8 weeks. The listening setting could be during a relaxing moment, for instance at bedtime, but could also be used before or during stressful situations.

The list of plausible alternatives created by the patient were mixed with relaxing music chosen by the patient. The music was 20–30 dB stronger than the plausible alternatives to actively mask it. Participants were given the option to change the music using the APAP website to prevent boredom and increase motivation. They were also required to hold a diary monitoring their adherence to the treatment. Participants were contacted weekly by the first author to verify their adherence and to respond to any queries. Questions concerning the website as well as information about life event of participants were collected during the weekly contacts and at end point (8 and 24 weeks). Also, 4–6 weeks after the beginning of the procedure, patients were evaluated and changes were made to the plausible alternatives, if judged necessary. The trial ended after 8 weeks and further follow-up was made at 24 weeks (16 weeks posttreatment).

#### RESULTS

#### Change in Dysfunctional Thoughts and Anxiety Symptoms

**Table 2** presents the mean degree of conviction in the different dysfunctional beliefs held by each APAP participants before the beginning of the procedure and at 24 weeks. Substantial decrease from pre to 16 weeks posttreatment in the mean conviction were observed and only one participant (Sharon) failed to improve

<sup>1</sup>The program is available in French freely to psychotherapist upon request to the corresponding author.

#### Table 2 | Conviction in dysfunctional thoughts pre-APAP and at 24 weeks.


#### Table 3 | Change in quality of life and anxiety symptoms during APAP.


*QLSI, Quality of Life Systematic Inventory; BAI, Beck Anxiety Inventory; HARS, Hamilton Anxiety Rating Scale.*

in regard to cognitions, but remained stable. All participants showed an increase in quality of life as well as an overall decrease in anxiety symptoms (see **Table 3**). Decrease in anxiety symptoms was modest, with some participants greatly decreasing (e.g., Caroline on the BAI) and others showing increase in symptoms (e.g., Pierre on the HARS). No participants showed an increase on both the BAI and the HARS suggesting that there was at least some improvement with all participants. There was also no relapse during follow-up.

#### Patient's Report and Their Perception of APAP and Its Website

Participants reported further gains that could be attributable to APAP. Sharon and Geneviève started doing some volunteer work. Isabelle returned to work while Sharon and Genevieve joined a return to work group after APAP. Claude decided to go back to work for the first time in 6 years and started a traineeship for people reintegrating the market place during the follow-up. Caroline made new friends and kept regular contact with them, as well as exposing her paintings. Finally, Haroun started reusing the phone.

There were also events not related to APAP that could have negatively impacted participants. Natalia moved, Pierre had some health problems, and his wife was hospitalized just before the end of the eighth week (he went on to have worries about health which were not targeted during his APAP trial) and Haroun was accused of sexual harassment.

Participants' testimony revealed that they greatly appreciated the possibility of working autonomously with APAP while being able to rely on their therapist, if needed. They appreciated being listened to when they felt the need to change their therapeutic target and they underlined the importance of a fluid program that allows to make such changes according to the patient's needs.

#### DISCUSSION

The present study sought to investigate the effectiveness of an augmentation of psychotherapy through alternative preconscious priming (APAP) intervention following CBT in treatment-resistant patients with residual symptoms. The procedure was supported by a patient–clinician collaborative website specifically designed for this purpose. Results showed improvement in targeted cognitions, quality of life, and anxiety symptoms during APAP that were maintained at follow-up, indicating that APAP could be used to help diminish residual symptoms of SAD and GAD following CBT. The users also positively appraised the procedure and the website. This study suggests that APAP could be a useful addition to CBT, particularly for those affected by resistant problematic cognition and internal discourse. Conceptually, APAP rationale is similar to CBT as it aims to change problematic cognitions postulated to be related to negative emotion and behaviors. To our knowledge, no other study has been reported using a rationale of preconscious priming or using repeated preconscious listening in the context of psychotherapy augmentation.

As previously mentioned by Tulving and Schacter (15), priming can have practical therapeutic uses. Change in cognitions targeted by APAP suggests sound internal validity of the intervention, APAP being the only intervention received by participants which targeted these thoughts. The results replicate the change in cognition found in a previous validation of APAP (1). While the reduction in anxiety was modest, the improvement in quality of life was substantial for some participants and gains in functioning were achieved by all participants and maintained at follow-up. This suggest that 8 weeks of passive repeated preconscious listening could have lasting effects beyond the use of a simple listening period. Results also suggest that comorbidities (particularly OCD and personality traits/disorder) or life events could diminish the effectiveness of APAP, as they do with any therapy. Yet even those with comorbidities or encountering adverse life events improved in some way.

Like APAP, other internet based treatments have also shown effectiveness (27), even compared with the usual treatments (28). These results are encouraging for internet intervention and suggest that some people might respond better when the setting is different then traditional psychotherapy.

Preconscious processes are also implicated in other anxiety disorders. People with obsessive–compulsive disorder show preconscious learning difficulties using a procedural task compared to healthy controls.2 Change in preconscious processes (in this case, attention processes) can effectively reduce the risk of developing subsequent posttraumatic stress disorder (29). This tends to show the importance of preconscious processes in anxiety disorders as an area to investigate, particularly for those who are treatment resistant.

#### Limits

In this case series, it is hard to distinguish the gains from APAP from the influence of other factors, for example, the residual effect of the participants' previous CBT. However, APAP has been previously validated as an effective procedure beyond residual effect of CBT and bogus priming (1). The small sample size and the nature of the study prevent us from generalizing the observation reported. A future replication should thus use a larger sample. The reliance of self-report assessment prevents us from distinguishing theses results from participants' biases, since they were not blind to the study hypothesis. Future studies should address this issue.

#### Strengths and Clinical Implications

This is the first time APAP has been used to treat GAD, the previous study focusing only on SAD (1), implying that APAP can be used to target various anxiety symptoms. This was also the first time that APAP was delivered over the internet, which was appreciated by the participants and allowed flexibility in the program. The previous utilization of APAP necessitated the use of a recording studio, limiting the capacity to adapt APAP to individual needs over time.

#### Future Recommendations

Future studies could include randomized controlled trial of APAP with a larger sample size, which would also aim to understand which clinical characteristic predicts meaningful response to APAP. Other psychiatry disorders presenting problematic internal discourse could potentially benefit from a similar procedure. APAP could also be applied in other formats (e.g., as a self-help procedure or combined with medication).

## CONCLUSION

Alternative Preconscious Priming could be a new and useful strategy to facilitate cognitive change and augment the effects of psychotherapy. In the present study, it helped to diminish participants' anxiety and improved their quality of life. Patients presenting comorbid OCD and personality disorder might not benefit from APAP as much as other patients. The use of a website was judged appropriate to facilitate access and autonomy, but improvements are still needed to increase program friendliness. Further studies, especially controlled studies, are warranted to assess APAP usefulness and its possible applications.

#### ETHICS STATEMENT

The study was approved by the comité d'éthique de la recherche de l'institut universitaire en santé mentale de Montréal. This ethics committee has been dissolved due to recent changes in the administrative structure. It is now called: comité d'éthique de la recherche du centre intégré universitaire de santé et des services

<sup>2</sup>Dulude G, Bédard M-A, O'Connor K, Audet J-S. Over facilitation of unadapted cognitive processes in obsessive compulsive disorder as assessed with the computerized mirror pointing task. *J Psychiatr Res* (In revision).

sociaux de l'est-de-Montréal. Participants were approached at the end of their psychotherapy and they were invited to participate in the current study. They were given the consent form and were told to read it thoroughly before agreeing. The research team answered any question concerning the consent form, and all was done to ensure that participants understood that they were free to disengage from the study at any points and that there would be no consequences to their disengagement.

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

MZ, FB, FA, and YK contributed to the design of the work. MZ and FB collected the data. MZ analyzed the data. MZ, J-SA, FB, FA, KO, and YK participated in the interpretation of the data. J-SA drafted the manuscript. MZ, FB, FA, KO, and YK revised the manuscript critically. All the authors approved the final version of the manuscript.

*de thérapie comportementale et cognitive* (2015) 25:85–92. doi:10.1016/ j.jtcc.2015.06.001


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Zidani, Audet, Borgeat, Aardema, O'Connor and Khazaal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# «Cognitus & Moi»: A Computer-Based Cognitive Remediation Program for Children with Intellectual Disability

*Caroline Demily1,2\*, Caroline Rigard1,2 , Elodie Peyroux1,2,3 , Gabrielle Chesnoy-Servanin4 , Aurore Morel1,2 and Nicolas Franck2,3*

*1GénoPsy, Center for the Diagnosis and Management of Genetic Psychiatric Disorders, CH Le Vinatier, Bron, France, 2UMR 5229, EDR-Psy, Center of Cognitive Neuroscience, CNRS, University Lyon 1, Lyon, France, 3Centre référent lyonnais en réhabilitation psychosociale et en remédiation cognitive (CL3R), CH Le Vinatier, Bron, France, 4Unité de Psychoéducation Polaire, Centre Hospitalier le Vinatier, Bron, France*

#### *Edited by:*

*Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland*

#### *Reviewed by:*

*Laurent Lecardeur, CHU de Caen, France Caroline Vandeleur, Lausanne University Hospital, Switzerland*

*\*Correspondence:*

*Caroline Demily caroline.demily@ch-le-vinatier.fr*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 03 December 2015 Accepted: 18 January 2016 Published: 03 February 2016*

#### *Citation:*

*Demily C, Rigard C, Peyroux E, Chesnoy-Servanin G, Morel A and Franck N (2016) «Cognitus & Moi»: A Computer-Based Cognitive Remediation Program for Children with Intellectual Disability. Front. Psychiatry 7:10. doi: 10.3389/fpsyt.2016.00010*

Attentional, visuospatial, and social cognition deficits have a negative impact on children's adaptative and social competences and, as a result, on their ability to achieve a normal functioning and behavior. Until now and despite the frequency of those deficits, there is a lack of children's specific cognitive remediation tools specifically dedicated to attentional and visuospatial areas. The «Cognitus & Moi» program involves a variety of exercises in a paper and/or pencil (*n* = 30) or a computerized format (*n* = 29) and a strategy coaching approach. Each module of «Cognitus & Moi» targets a single impaired cognitive area, within the limits of cognitive domains' overlapping. The little cartoon character named Cognitus, who illustrates the program, is supposed to be very friendly and kind toward children. Cognitus will accompany them throughout the program for an effective and positive reinforcement. The main goal of «Cognitus & Moi» is to adjust to children's difficulties in daily life. Moreover, since the cognitive remediation benefit is complex to apply in daily life, the program is based on a metacognitive strategy. After a complete neuropsychological assessment and a psychoeducational session (with the child and the parents), 16 1-h-sessions of cognitive remediation with the therapist are proposed. Each session is composed of three parts: (1) computerized tasks focusing on specific attentional or visuospatial components (20 min). The attentional module targets hearing, visual, and divided attention. A double attention task is also proposed. The visuospatial module targets eye tracking and gaze direction, spatial orientation, visuospatial memory and construction, and mental imagery; (2) pen and paper tasks focusing on the same processes (20 min) and a facial emotion recognition task; (3) a proposal of a home-based task (during 20 min). Weekly, specific attentional and visuospatial home tasks are proposed to the child and analyzed with the parents and the therapist. Indeed, home exercises are useful to promote the transfer of strategies to daily life and their subsequent automation. The heterogeneity of cognitive deficits in intellectual deficiency necessitates an individualized cognitive remediation therapy. In this regard, «Cognitus & Moi» seems to be a promising tool.

Keywords: attention, visuospatial functions, social cognition, cognitive remediation, intellectual disability, behavior, hyperactivity

#### Demily et al. Cognitus & Moi

#### WHY SHOULD WE CONSIDER COGNITIVE REMEDIATION FOR PEOPLE WITH INTELLECTUAL DISABILITY?

#### Definition and Functional Outcome

In most classifications, such as the DSM-5 (1), intellectual disability is defined as a deficit of cognitive abilities such as reasoning, problem solving, planning, abstract thinking, judging, academic learning, and learning by experience. However, other cognitive dysfunctions are associated with intellectual disability. Actually, attentional (2), memory (3), visuospatial (4), and executive (5–7) alterations are also found.

All these can be associated with an impairment of adaptive functioning. This impairment prevents people from being independent and socially responsible and appears during the development stages. Research on the functional and concrete impact of these cognitive deficits is currently being structured and developed. As an example, Berg (8) and Bull and Scerif (9) have shown that the mathematical difficulties encountered by children with intellectual disabilities are due to their low working memory capacities and to an executive dysfunction. In effect, working memory and short-term memory play a major role in a number of daily activities (10) because they help maintain and control information in memory (11). Thus, we may conclude that cognitive deficits are closely related to the functional outcome.

The care of people with intellectual disability is a public health issue, 1–3% of the population being concerned (12). Speech therapy and psychomotor care are well-developed for children with intellectual disability (13). These techniques mainly help these people develop their communication and motor skills. Since cognitive functions other than language (14, 15) and psychomotor abilities are impaired in children, as well as adults, it seems interesting to consider the development of specific cognitive remediation programs.

#### Definition of Cognitive Remediation

Cognitive remediation is a behavioral training aiming at minimizing the daily impact of cognitive deficits by optimizing and improving cognitive functioning (16). In this regard, the notion of functional adaptation should be highlighted. In this connection, cognitive remediation has been defined as psychological treatment aimed at "increasing the general cognitive efficiency to improve global adaptation, independence, and well-being."

The development of new strategies and tasks can be used to improve cognitive functioning. In the field of intellectual disability, two theories explain the effectiveness of cognitive remediation.


observed. Differences depend on the origin of the deficit. For example, diverse cognitive profiles can be observed according to the genetic conditions (Williams syndrome, Down syndrome, or X-Fragile syndrome…) (5, 19). Those data have mainly been shown in the field of memory (20). In this case, cognitive remediation can help develop strategies to improve impaired cognitive processes. The «Cognitus & Moi» program is based on this conception.

#### COGNITIVE REMEDIATION OF CHILDREN AFFECTED BY INTELLECTUAL DISABILITIES: THE DIFFERENT PROGRAMS

Concerning intellectual disability, cognitive remediation programs must be based on some general principles. They should be clearly defined before beginning the treatment. The therapist ought to set objectives according to the child's practical needs. Improvement of cognitive functions will enable the child to reach his/her goals.

#### Memory Programs

Cognitive remediation focused on short-term memory has been well studied. Hulme and Mackenzie (21) worked on articulatory recapitulation strategy using a cumulative repetition method. It consisted of ten 10-min sessions. This program was carried out with teenagers (13–18 years old) affected by intellectual disabilities. The authors showed a significant improvement of the memory span. Then, Comblain (22) enhanced this program by proposing 30-min individual weekly training sessions (growing difficulty) with subjects (*n* = 12) affected by Down syndrome. This study showed a significant improvement of the average memory span. Moreover, long-term effects were also observed 18 months later. With this strategic approach, Bussy et al. (23) demonstrated an extension of the verbal span and a growing passive vocabulary. Finally, improvements were observed in the verbal short-term memory of 33 children suffering from fetal alcohol syndrome (24).

In the field of meta-memory, Kendall et al. (25) have shown that children with intellectual disability are able to learn an interrogative strategy encouraging them to find links between different stimuli. This group showed higher scores than the control one as far as recalling items were concerned.

#### Metacognitive Program

Découvrez vos capacités, rEalisez vos possibilités, pLanifiez votre démarche, soyez créatiFs (DELF) is a metacognitive program, which aims at discovering the subject's skills and highlighting the person's abilities. This program is used in groups and aims at teaching metacognitive strategies (anticipation, planning, and control) and more specific strategies helping, for example: to better use working memory so as not to overload memory. DELF proved to be effective with teenagers suffering from intellectual disability when associated with a regular teaching program (26). Within this experimental group, inductive thinking abilities were improved.

#### Virtual Reality

Few studies focus on the use of virtual reality for the treatment of people with intellectual disabilities (27). Rose et al. (28) pointed out the relevance of an active exploration in a virtual environment with the use of a joystick rather than a passive exploration thanks to a mere observation. Handling a joystick enabled the subjects to better memorize the environment visuospatial data. Thus, development of virtual reality programs would enable patients to improve their visuospatial abilities.

#### Attentional Program

Galbiati et al. (29) proposed treatment for children and teenagers aged 6–18 years, presenting traumatic brain injury combined with a mild intellectual disability and attentional difficulties. The program used targeted attentional abilities by using metacognitive strategies. Treatment lasted for 6 months, including four 45-min individual weekly sessions led by a therapist. During the sessions, 30 min were dedicated to computerized exercises and 15 min to paper/pencil exercises. The participants' attention resources and adaptive skills improved on a daily basis.

#### VISUOSPATIAL SKILLS AND SOCIAL COGNITION IN INTELLECTUAL DISABILITY

Visuospatial ability refers to the capacity to identify visual and spatial relationships among objects. How subjects imagine objects, perceive global shapes, and how they locate small components or understand the similarities and differences among objects are major cognitive functions.

Visuospatial and visuoperceptual skills play a key role in everyday life. Visual information and complex visual stimuli are analyzed with a complete unawareness of the visuoperceptual process or the complexities of the stimuli involved. This process becomes conscious in a context of learning. Repetition and familiarity enable a more spontaneous approach and turn the conscious and effortful process into an automatic one.

If this ability is impaired, many types of deficits can occur, ranking from a failure to process the basic elements of a visual stimulus (i.e., colors, lines, orientation) to more complex and integrative features such as object identification, faces, or familiar scenes. These deficits can include social cognition defects, especially in the area of facial emotion recognition.

Social cognition is a psychological construction referring to the understanding of others' thoughts and including several components such as empathy, attribution bias, theory of mind, and emotion processing. Impairments in this field may largely underlie social dysfunctions and reduce adaptive skills. Moreover, social cognitive disabilities contribute more or less directly to behavioral disturbances and psychiatric symptoms (e.g., depression, anxiety) (30). Yet, depressive and anxious symptoms are often found in children with intellectual disabilities (31, 32). In this regard, a link is clearly established between children's language and behavioral impairments when it comes to intellectual disability (33). Behavioral and psychiatric symptoms are also correlated to intellectual disability in adults, as shown by Deb et al. (34).

Children with externalizing behavioral problems provide aggressive responses to hypothetical vignettes more spontaneously than children with intellectual disability without any behavioral problems (35). Cognitive dysfunctions explain the hardship children encounter in the treatment of social information (36). Indeed, social cognition dysfunction appears like a core symptom (37). The theory of mind and facial emotion recognition seem to be central for social adaptation to the environment (38).

As far as social cognition is concerned, the facial emotion recognition is well documented. Children with intellectual disability fail to recognize and match emotional facial expressions from a series of photographs depicting various facial expressions (39). This deficit is correlated with an abnormal behavior (40) and underlain by visuospatial and attentional deficits (41).

In conclusion, the improvement of attentional and visuospatial deficits thanks to a specific cognitive remediation program could have a positive impact on the children's social cognition and behavior. This approach would complete the methods already available.

#### THE «COGNITUS & MOI» PROGRAM

Attention, visuospatial, and social cognition deficits have a negative impact on children's adaptive and social competences and, as a result, on their ability to achieve normal functioning and behavior. Until now and despite the frequency of those deficits, there is a lack of children's specific cognitive remediation tools specifically dedicated to attentional and visuospatial areas.

The «Cognitus & Moi» program targets attentional and visuospatial functions (**Table 1**). Cognitive goals are embedded in two different modules (attention and visuospatial) and the level of these modules is chosen according to the child's key difficulties. Each exercice of «Cognitus & Moi» targets a single-impaired cognitive area.

«Cognitus & Moi» was developed in France through the collaboration between the GénoPsy center (*Center for the Detection and Management of Psychiatric Genetic Disorders*) in Lyon, the EDR-Psy research team (CNRS & Lyon 1 University, headed by Pr. Nicolas Franck) and the SBT Company (headed by Pr. F. Tarpin-Bernard). The Génopsy team examines patients who suffer from psychiatric genetic disorders. Thus, neurocognition, social cognition, as well as metacognition are routinely evaluated. The SBT Company and the EDR-Psy team have already developed various cognitive remediation programs (RECOS, GAIA, RC2S) (42, 43).



All the developers implicated in «Cognitus & Moi» are trained therapists for cognitive remediation. The exercises of the attentional and visuospatial modules were selected in the SBT software database. First, the selected exercises were proposed to children with normal cognitive functioning in order to appreciate the technical feasibility. Second, each exercise was adapted (levels one to nine) in order to be proposed to children with intellectual disability. The third step was the development of a facial emotion recognition task. Each Cognitus' facial expression was based on the Ekman pictures (*Appendix 4* in Supplementary Material) with three levels of intensity (44).

«Cognitus & Moi» The «Cognitus & Moi» program involves a variety of exercises in a paper and/or pencil (*n* = 30) or a computerized format (*n* = 29) and a strategy coaching approach. Therapists make use of techniques known to benefit the rehabilitation of cognitive syndromes.

This new cognitive remediation program was elaborated with two main goals in mind. The first one aimed at adjusting to the children's difficulties in daily life. In order to achieve this purpose, we developed an individualized and flexible program meant to improve both specific attentional and visuospatial impairments and to reach each child's concrete objectives. Since the cognitive remediation benefit is complex to apply in daily life, a program based on collaboration with parents was elaborated (**Table 2**).

«Cognitus & Moi» is designed for 5–13-year-old children with or without intellectual deficiency. The little cartoon character named Cognitus, who illustrates the program, is supposed to be very friendly and kind toward children. Cognitus will accompany them throughout the program for an effective and positive reinforcement.

#### Cognitive Assessment

A complete and detailed neuropsychological assessment prior to the cognitive remediation treatment seems necessary (45). This evaluation will also determine the severity of the impairments and their impact on everyday life. Currently, consensus lacks concerning attentional and visuospatial evaluations, so they reflect the heterogeneity of the performances.

This assessment must help establish the global degree of intellectual disability by using standardized tools such as the Wechsler scales. Then it is important to assess the functional level of the person by using more specific tools such as the Vineland Adaptative Behavior Scale, second Edition [VABS-II (46)], the EFI [Functional Intervention Scale (47)] or the AAPEP [Psycho-educative profile for adolescents and adults (48)]. In intellectual disability, the IQ is most often evaluated without a thorough evaluation of the different cognitive domains. However, a detailed cognitive assessment is necessary to establish a detailed neuropsychological profile by identifying the cognitive impairments and also the preserved abilities of the person.

The following assessment is recommended (before the «Cognitus & Moi» program): (**Table 3**).

#### Preliminary Session: Psychoeducation

In the first part of this session, the therapist, the parents and the child go through the previously administered assessment together. By explaining the child's cognitive assessment to the

#### Table 2 | Keys of the program.

#### «COGNITUS & MOI»: main principles


#### Table 3 | Neuropsychological assessment.


*CMS, Children's Memory Scale; NEPSY II, Developmental NEuroPSYchological Assessment (second edition); TAP/kiTAP, test battery of attention performance; WISC IV, Wechsler Intelligence Scale for Children (fourth edition); WPPSI III, Wechsler Preschool and Primary Scale of Intelligence (third edition).*

*"Looking for":* selective attention, visual exploration and discrimination Several identical symbols are hided in a photography. The child has to observe the image and to locate the symbols. The complexity of the task increases according to the determined level (one

to nine).

*"The differences"*: selective attention, visual exploration and discrimination A child, with the help of a therapist, will determine the difference between the two sides of the presentation. He/She will have to find which image appears on the right but not on the left. The child should use a specific strategy to answer properly.

parents helps them better understand his/her profile, and which of his/her cognitive components are impaired and which are preserved. That is the reason why this session lays emphasis upon the specific impairment in the child's daily life.

The last session of the preparation phase provides psychoeducation via two specific documents. A specific comic strip has been elaborated for children explaining the main goals of attentional and visuospatial functions (*Appendix 1* in Supplementary Material). Moreover, a didactic document «Parent handbook» is proposed to the parents in order to detail the attentional and visuospatial functions and the «Cognitus & Moi» program (*Appendix 2* in Supplementary Material).

The aim of the psychoeducational session is to allow the child and his/her parents to understand the specific terms used in the field of neurocognition, their implication in daily life, to finally increase motivation.

#### Cognitive Remediation Sessions with the Therapist

Then, 16 1-h-sessions of cognitive remediation with the therapist are proposed, each session is composed of three parts:


Each of the 16 sessions deals with a specific attentional and visuospatial stimulation. The computer-based tasks are ranked in increasing order of difficulty, with nine levels of complexity. This allows the therapist to adapt difficulty of exercises to the child's abilities.

#### Design of the Computer-Based Modules (*n* **=** 29) (Table 4)

#### Design of the Pen-and-Paper Based Modules (Table 5)

Similarly, pen-and-paper exercises will help the child to develop compensatory strategies when it comes to visuospatial or attentional tasks. Adapting the rhythm of the treatment to the child's abilities helps prevent him/her being faced to failures. The rhythm is progressive, as must be the difficulty of each exercise.

Finally, the child will have to recognize Cognitus' facial emotion (*Appendix 4* in Supplementary Material) with the help of the therapist. This exercise was developed in order to establish a concrete link between visuospatial functions and facial emotion recognition. The drawings are designed in order to encourage the person to focus on the eyes. People with intellectual disabilities often study the mouth area to recognize emotions. The strategy developed in this tool will help children determine an emotion by using relevant information. By highlighting the eye area, the strategy can be automated by the child. He/She has to choose the correct emotional label by verbalizing the name of the emotion. Six universal emotions are represented (without neutral condition): happiness, fear, anger, disgust, surprise, and disgust.

#### Table 4 | Computer-based modules.

#### For example: visual attention

#### Table 5 | Pen-and-paper based modules.

The child has to follow and color the link between the characters and the

# balloons

#### Therapist's Strategies in «Cognitus & Moi»

Mediation by a therapist is one of the key factors to the success of psychological treatment. Individual care helps maintain the collaboration over time between the child and the therapist. Verbalization plays an active part in the training of a new strategy. The therapist helps the child find sense when a problematic situation arises. The development of the new skills will be transferred to daily life. The therapist should encourage the child to actively take part in the treatment by choosing his/her own program of exercises and goals.

The structuration of the sessions is required in order to help the child deal with the notion of delay and waiting. This can be done by presenting the program of each session at the beginning and by estimating the time to be spent on each task. After each task, the therapist will remark on the time spent on the exercise to improve the perception of chronological indicators.

This technique is detailed in different steps in «Cognitus & Moi», in order to automatize and generalize the new strategies.


Children with an intellectual disability will find it easier to transfer memory strategies to daily life when it has been explicitly verbalized during the cognitive remediation sessions (49).

#### Home Task with the Parents

The cognitive remediation session with the therapist finishes with a proposal of a home task. Exercises take into account both the level of performance of the child, the collaboration of the parents, and are related to the concrete objectives defined at the beginning of the therapy.

The outstanding idea of the program is to establish a weekly link between cognitive remediation and daily life. Moreover, we will help the child understand his/her environment using colors and time landmarks. Tasks are adapted to the child's needs and take into account his/her daily reality. For example, at the beginning of the therapy, it could be suggested that the child should shop at the local supermarket and find a given item in a department.

Later, specific attentional and visuospatial home tasks can be proposed to the child and analyzed with the parents and the therapist. Finally, tasks such as the planning of a daily route can be suggested. Home exercises are useful to promote the transfer of strategies to daily life and their subsequent automation. This is consistent with learning theories. Indeed when cognitive remediation is reinforced in real-world settings, the learning process is facilitated and then generalization and transfer are promoted (50). The child and the therapist will go through the home task together at the beginning of the next remediation session. In order to promote motivation during the home task, the use of a personalized exercise book allows a concrete weekly evaluation of the degree of achievement of the task. The child can self-evaluate his/ her performances with a smiley, in the same way, his/her parents can also evaluate their child by answering four concrete questions:


#### METACOGNITIVE ASPECTS AND BENEFITS

People with intellectual disabilities have difficulties in tasks requiring a voluntary effort and a conscious analysis of the cognitive demands of the task. This difficulty prevents the subjects from spontaneously engaging themselves in a task when this one requires a strategic active treatment.

Metacognition corresponds to the knowledge of one's own functioning. The metacognitive processes are procedural aptitudes that have an impact on the cognitive aims of the person and on the tasks carried out. In this regard, positive reinforcement represents a key to success.

In our program, Cognitus the little character, kindly supports the child (*Appendix 3* in Supplementary Material). Moreover, the parents' cooperation plays a key role. That is the reason why, «Cognitus & Moi» is clearly distinguished from the daily homework to avoid conflicts.

Only few patients with intellectual disabilities manage to find work or live alone; however, families hope that close relatives will be able to be more independent (27). This is why the ultimate aim of a cognitive remediation program is to improve the subject's quality of life. This has to be done by acquiring new strategies that will help the patient achieve the exercises and then generalize the strategies and benefits. The idea is to apply a learned strategy to a more global context, distant from the initial one. People with intellectual disabilities are described as cognitively passive (51). This is characterized by a lack of strategy transfer (52). It is particularly complicated to obtain transfer; each strategy taught has a different adaptive value depending on the environment and on the needs of the person (53). It is important to first acquire the skills before trying to contextualize them in more complex environments. The therapist will help systematically to establish the link between the skills acquired during the sessions and situations of daily life by using, for example, home tasks. These tasks should first be defined with the patient during the sessions and a feed-back will then be given during the following session.

In general, children with intellectual disabilities have a sense of failure and the feeling of being incompetent. Cognitive remediation helps the child escape this negative spiral. Positive reinforcement and mediation by a therapist in the «Cognitus & Moi» program have an impact on the child's confidence in his/ her abilities and thus on self-esteem. In this regard, the child is the winner of an award after the cognitive remediation training (*Appendix 3* in Supplementary Material).

Self-determination is the possibility for the person to choose his/her own activities and behaviors (54). The therapist plays an important part in the improvement of self-determination. By reinforcing the child's feeling of competence (positive feedback) and by choosing adapted exercises (nor too easy, nor too difficult) the therapist improves self-confidence (55).

#### VALIDATION STUDY

The cognitive deficits presented by patients with intellectual disability are very diverse. Yet, most studies on the effectiveness of cognitive remediation treatment are randomized controlled trials, especially in the area of mental health and education. Randomized controlled trials have been used to evaluate a number of educational interventions but remain less frequent in the cognitive remediation area.

We are currently conducting (preliminary steps) a randomized controlled trial in order to establish the validity of the «Cognitus & Moi» program. All the ethical approval (French legislation) is in process (CPP, CCTIRS, CNIL, and Clinical trials.gov registration). Our first objective is to evaluate the impact of the program on behavioral disorders. A thorough assessment is proposed including a complete evaluation of the components of neurocognition, social cognition, and of social functioning and behavior. These measures and the complete assessment are repeated at the end of the intervention to highlight the impact of the «Cognitus and Me» program, and 6 months later to investigate the possible long-lasting effects of the benefits. Currently, the first children are following the therapy with the «Cognitus and Me» program in a context of usual care. The exercises appear very close to daily life and very pleasant. Another point is the parents' wishes to be involved in their child's therapy. In this regard, the program meets their expectations.

However, our program presents several limits. First, only the attentional and visuospatial deficits as well as emotion recognition are targeted. Other cognitive dysfunctions are associated with intellectual disability, especially memory, and executive functions. Indeed, the development of these specific modules will be the next step of the «Cognitus & Moi» program. Second, the involvement of other components of social behavior (social skills training and management of emotions) plays a key role in the adaptative skills. In that regard, cognitive behavioral therapy should be a useful treatment in addition to the «Cognitus & Moi» program. Third, to our knowledge, the long-term effects of cognitive remediation in intellectual deficiency have not been investigated in previous studies.

#### CONCLUSION

This article opens the path for new thoughts about the development of specific cognitive remediation program in intellectual disability. A new modular conception of cognitive remediation in intellectual deficiency should include neurocognition, social cognition and metacognition training programs. Cognitive remediation tools, based on visuospatial and attentional functions, constitute promising tools to improve social cognition in patients with intellectual disability and behavioral disorder. The aim of this work is to enable these people to have access to cognitive remediation treatments that are specifically adapted to their abilities and needs.

#### REFERENCES


Intellectual disability is a public health issue and it is essential to consider all the possible solutions to help improve these people's everyday life. Cognitive remediation is thus one of the most promising tools that can be used. However, it must be used with a very strict methodology in order to respect the person's limits and preserved skills. Last but not least, cognitive remediation must be included in a global care of the person and cannot be substituted to other psychological treatments. This treatment depends on the participation of different actors (e.g., family, therapist, educator, etc.). The interaction between the different medical and social actors will help establish a complementarity among the different treatments carried out (medical treatment, psychomotor, speech therapy…). To conclude, the «Cognitus & Moi» program, by using transfer strategies to daily life, is close to the real world. This allows the child to practice skills in specific personal contexts, which is a major challenge to improve mental resources.

#### AUTHOR CONTRIBUTIONS

CD, EP, CR, AM, and GC-S were implicated in the concept and the design of «Cognitus & Moi». CD, CR, and EP wrote the paper. GC-S and NF were involved in the review of the literature. All authors read and approved the final manuscript.

#### ACKNOWLEDGMENTS

The authors would like to thank the *AFRC* (financial support), Sylvia Silberman-Tessier (English editing assistance), Franck Tarpin, Roland Hildebrandt and Lionel Paolini (technical assistance-SBT Company), and le Conseil Scientifique du Vinatier.

#### FUNDING

«Cognitus & Moi» was financially supported by *AFRC* (Association Francophone de Remédiation Cognitive, FRANCE) and SBT Company.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://journal.frontiersin.org//article/10.3389/fpsyt.2016.00010

Williams syndrome and Down syndrome. *Res Dev Disabil* (2013) **34**:1770–80. doi:10.1016/j.ridd.2013.01.024


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Demily, Rigard, Peyroux, Chesnoy-Servanin, Morel and Franck. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# DJINNI: A Novel Technology Supported Exposure Therapy Paradigm for SAD Combining Virtual Reality and Augmented Reality

*Maher Ben-Moussa1 \*, Marius Rubo2 , Coralie Debracque3 and Wolf-Gero Lange4*

*<sup>1</sup> Information Science Institute, Computer Science Centre, University of Geneva, Geneva, Switzerland, 2Experimental Clinical Psychology, Department of Psychology, University of Würzburg, Würzburg, Germany, 3Neuroscience of Emotion and Affective Dynamics Lab, Swiss Center for Affective Sciences, Department of Psychology, University of Geneva, Geneva, Switzerland, 4Department of Clinical Psychology, Behavioural Science Institute, Radboud University Nijmegen, Nijmegen, Netherlands*

#### *Edited by:*

*Anna Sort, University of Barcelona, Spain*

#### *Reviewed by:*

*Mª Angeles Gomez Martínez, Pontifical University of Salamanca, Spain Bulent Coskun, Kocaeli University, Turkey*

> *\*Correspondence: Maher Ben-Moussa maher.benmoussa@unige.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 20 September 2016 Accepted: 01 February 2017 Published: 28 April 2017*

#### *Citation:*

*Ben-Moussa M, Rubo M, Debracque C and Lange W-G (2017) DJINNI: A Novel Technology Supported Exposure Therapy Paradigm for SAD Combining Virtual Reality and Augmented Reality. Front. Psychiatry 8:26. doi: 10.3389/fpsyt.2017.00026*

The present paper explores the benefits and the capabilities of various emerging stateof-the-art interactive 3D and Internet of Things technologies and investigates how these technologies can be exploited to develop a more effective technology supported exposure therapy solution for social anxiety disorder. "DJINNI" is a conceptual design of an *in vivo* augmented reality (AR) exposure therapy mobile support system that exploits several capturing technologies and integrates the patient's state and situation by visionbased, audio-based, and physiology-based analysis as well as by indoor/outdoor localization techniques. DJINNI also comprises an innovative virtual reality exposure therapy system that is adaptive and customizable to the demands of the *in vivo* experience and therapeutic progress. DJINNI follows a gamification approach where rewards and achievements are utilized to motivate the patient to progress in her/his treatment. The current paper reviews the state of the art of technologies needed for such a solution and recommends how these technologies could be integrated in the development of an individually tailored and yet feasible and effective AR/virtual reality-based exposure therapy. Finally, the paper outlines how DJINNI could be part of classical cognitive behavioral treatment and how to validate such a setup.

Keywords: social anxiety disorder, social phobia, VRET, exposure therapy, virtual reality therapy, augmented reality

## INTRODUCTION

In the well-known movie Amélie (1), the eponymous heroine has trouble managing social situations. When a friend of hers is publicly embarrassed by his boss, being called a vegetable, she intends to help, but does not know how to react. However, her wishful thinking of having someone whispering the perfect response to her at the right time magically comes true, and she sovereignly masters the situation by replying: "You'll never be a vegetable. Even artichokes have hearts!." While the dream of being helped out unobtrusively when caught off-guard or feeling insecure in social situation is usually not fulfilled and may even seem naïve, the aim of this paper is to explore the technical possibilities in achieving just this. By using the metaphor of a "DJINNI," an Arabian mythological spirit that can be summoned to help it's "Master" and has the ability to hide itself to others, we convey our core idea behind a set of technological solutions for helping individuals fearing social situations.

The last two decades saw the emergence of virtual reality (VR) as a therapeutic tool for treatment of various mental disorders (2). Especially in the field of anxiety disorders, the immersive experience that VR offers has made it a useful tool for exposure therapy, the gold standard in the treatment of these conditions. In fact, in situations where the confrontation with the feared object or situation is expensive (e.g., fear of flying) or difficult to provide (e.g., high buildings in rural areas when treating fear of heights; fear of open spaces; audiences for fear of speaking in public, etc.), virtual reality exposure therapy (VRET) has become a tangible solution (3). Most challenging in that respect, however, is social anxiety disorder (SAD). The nature of social situations that SAD patients fear is heterogeneous. Simulating various *in vivo* situations in VR where patients can experience possible negative evaluation by others is quite difficult to achieve technologically. A lot of efforts and resources would be required to build virtual environments that would simulate different situations experienced by the patient. In fact, the heterogeneous nature of situations experienced by SAD patients is quite difficult to achieve also for traditional approaches of exposure therapy. While the presence of the therapist in traditional exposure therapy for first guidance or modeling purposes is usual in the treatment of other anxiety disorders (e.g., specific phobia or agoraphobia), it seems rather odd to accompany a patient to speeches they have to give, or to parties, or dates. As a mental disorder, SAD is desperately in need of novel technological therapeutic solutions that can overcome the current limitations. The aim of the present manuscript is to explore the potential of new emerging, alternative technologies, that have advanced a lot in the last years [e.g., augmented reality (AR) glasses and various Internet of Things (IoT) devices such as smartwatches, sporting sensors, etc.], as support tools for exposure therapy. With "DJINNI," we propose a technological solution integrating VR, AR, and IoT technologies to deliver a complete support solution for patients suffering from SAD as well as for their therapists.

#### SAD AS A MENTAL DISORDER

Social anxiety disorder (or social phobia) is a mental disorder accumulating in frequency (4, 5) with lifetime prevalence rates of 7–13% in Europe (6), and it belongs to the most prevalent mental disorders after depression and substance abuse (7, 8). It is a debilitating condition characterized by marked fear and embarrassment in social and performance situations and when under scrutiny by others (9). SAD typically has a chronic course if untreated (6), and becomes increasingly associated with comorbid mental problems such as depression, or alcohol abuse. In the Netherlands, mental health services for SAD cost 11,952€ per capita. Even the costs for sub-threshold SAD sum up to 4,687€, substantially more than for healthy individuals (10, 11). But, most importantly, it severely disrupts social and occupational functioning (6, 12) of individuals suffering from SAD.

Cognitive models of SAD (13, 14) suggest that socially anxious individuals (SAs) tend to interpret their own performance in social contexts and ambiguous social information surrounding them, in a threat-confirming way (15). In addition, they are believed to quickly attend to negative social stimuli, predominantly faces (16). Both of these cognitive biases are believed to fuel SAs' subjective experiences of anxiety and reconfirm the fearful convictions they enter a situation with thereby maintaining the disorder (12, 13). Interestingly, measurable relationships with observable behavior (14, 17–20) or physiology (21, 22) have been indecisive. This is intriguing as cumulative evidence suggests that SAs *are* experienced as somewhat odd in interactions—although no systematic behavioral patterns have been identified (23, 24). With SAs being experts in looking for signs of devaluation, they will most likely sense negative signals validating their fear of negative evaluation (24). Consequently, along with attentional biases, and interpretational biases, measurement of psychophysiological indices and assessment as well as retraining of "true" social behavior in real social contexts should become part of therapeutic approaches in the future.

In sum, while evidence supports the notion that social anxiety is related to a negative interpretive and attentional biases, and elevated subjective anxiety levels (15), deviations in behavioral and physiological indices have been difficult to substantiate. One reason may be that fear is, according to Lang's model of emotion (25, 26), reflected in three independent but interrelated systems: verbal report, fear related behavior, and patterns of somatic activation. He stresses that these systems are not necessarily synchronized. In fact, it is plausible to assume that, e.g., continuous reporting (on one's thoughts, behavior, or physiology) will disrupt their natural patterns. Additionally, human (anxious) behavior is often guided by reflexive, automatic behavioral responses inaccessible to introspection (27, 28). When social anxiety is considered, self-report could be compromised by SAs' proneness to social desirability effects (29), or again cognitive biases (15). Furthermore, SAs' self-reported deficits in, e.g., social skills appear to be quite accurate in one context (social interaction) but not in another (public speaking) (30).

Another reason for these inconsistencies may lie in the fact that social anxiety is quite heterogeneous. Its specific form knows many facets such as fear of trembling, blushing, or sweating, and speaking, writing, eating/drinking, and urinating in public restrooms. Often, research participants are screened on general aspects of social anxiety, which they all share. However, stimuli or social scenarios might not be specific enough to tap into their particular fear.

Although effective treatment regimens exist, the very same heterogeneity is thought responsible for the smaller treatment outcomes and higher relapse rates when compared to exposure therapy in other anxiety disorders: the effect sizes of SAD treatments (1.16) lag behind those of the other anxiety disorders (>1.74) (31, 32) and of the 60% that remit due to treatment 40% relapse (33). In addition, cognitive behavior therapy (CBT), the gold standard in anxiety treatment, focuses on distorted cognitions and exposure to frequently avoided social situations, but readily ignores shortcomings in subtle interpersonal behavior.

Finally, replicable scientific investigation of observable social behavior in real life is hampered by the necessity to incorporate numbers of confederates. Their (non-)verbal reactions, even after extensive training, are neither completely controllable nor predictable. Physiological assessment with mobile hardware makes "normal" interaction awkward. In addition, a reliable behavioral observation demands at least two observers/film angles capable of registering, e.g., frequency and length of eye contact or physical distance between interlocutors, and a team of observers/evaluators blind to the conditions (of the participant). High-resolution measurement is impossible in real social environments, even under laboratory conditions.

With regard to exposure treatment, SAD is one of the few anxiety disorders where the therapist is usually absent during the *in vivo* exposure sessions. He or she can only rely on the subjective record of the patient's description of the situation. They cannot assess the anxiety levels or behaviors "*in vivo*" to evaluate if the session was successful. In addition, gradual exposure is difficult to accomplish and the therapist cannot serve either as model or as helping hand, should an exposure task seem too difficult.

#### CURRENT STATE-OF-THE-ART SOLUTIONS

#### Virtual Reality Exposure Therapy

To overcome the abovementioned difficulties in creating anxiety evoking, highly controllable, and replicable situations in real life, immersive virtual reality (IVR) technology and VRET has gained considerable attention in research and assessment of anxiety disorders over the last decennia (34). As analog to *in vivo* exposure, VRET to threat-evoking stimuli has proven to be an equally effective way for provoking (reflexive) threat responses in close-to-real situations and initiating habituation as prerequisite of treatment (35).

However, the difficulty to assess and address the complex pattern of SAD symptoms in VRET is reflected in a relative scarcity of studies in the field. Two meta-analyses by Opriş et al. (36) and a more recent one by Kampmann and colleagues (37) explored the efficacy of technology-assisted interventions for SAD since 1985. They list only a handful of high-quality studies on VRET in SAD. These studies yield generally positive results. In a study by Klinger and colleagues (38), patients participated in virtual conversations in a meeting room and at a dinner table, were scrutinized by, and needed to assert themselves against virtual agents. This treatment was found to be similarly effective as group CBT. Wallach and colleagues (39) found effects comparable to CBT in a public speaking task in a VR scenario while at the same time achieving smaller dropout rates. Similarly, Anderson and colleagues (40) also report significant improvement in a public speaking task in VR and no difference between virtual and *in vivo* exposure therapy. Moreover, they found the effect to be stable throughout a period of 1 year after treatment. Finally, Kampmann et al. (37) found VRET treatment effects by exposing patients to virtual speech situations, small-talk with strangers, job interviews, dining in a restaurant, having a blind date, or returning bought products to a shop. Most interestingly in this study, the therapist could adjust the number, gender, and gestures of avatars, the friendliness, and to a certain degree content of the semistructures dialogs depending on the patients' needs, anxiety, and treatment progress. Yet, treatment evaluation was heavily based on subjective self-report questionnaires and improvements in speech performance as the only behavioral measure.

While the studies conducted so far document a promising future for VR applications in SAD treatment, they fall short on exploiting its full potential. Besides often being unpretentious in its audiovisual quality, they put, except of the Kampmann's study, a strong emphasis on the fear of public speaking, largely neglecting the complex structure of other contexts in which SAD symptoms occur. More importantly, they fail to properly address the dissimilarities that exist between SAD patients with regard to attentional and behavioral indices. Therefore, it is advised to establish a VR treatment program that covers the complex pattern of SAD symptoms while allowing for patient-specific adaptations.

#### The Potential of AR As a Complementary or an Alternative Tool to VR

Although numerous virtual scenarios exist for treatment purposes, they are generally not individually tailored and very few utilize behavioral measures available through VR technology such as amount of mimicry (41), interpersonal distance and movement speed (42), or gaze direction (43), nor combine them with physiological measures (44). In addition, AR devices would allow therapists to get a first-hand impression of their patient's behavior in a social situation in order to individually tailor the VRET scenarios. AR could also incorporate physiological data from, e.g., smartwatches to analyze indices of anxiety in a situation, hint on friendly faces in a crowd *via* emotion recognition, and provide helpful sentences to the patient (invisible for others) should he/she be stuck in a conversation.

Taken together, utilizing IVR or AR means a great step forward in investigating the interplay between self-report, actual behavior, and physiology in social anxiety as well as their response to different aspects of the treatment or their predictive value for treatment outcome. While circumventing the methodological difficulties of assessment in real life, IVR allows investigating differences in behavior such as gaze direction, gaze duration, distance from, and movement speed toward audience/interlocutor, psychophysiology, and voice properties between high and low SAs, in a high ecologically valid setting. In addition, e.g., wearable AR glasses could be used to assess actual social behavior of SADs in real social contexts. But they could also provide cues for positive social interaction *via* emotion recognition and help out in conversations or provide soothing comments, should the heartbeat registered by peripheral devices indicate anxiety. In sum, such an integrative approach will yield the potentially most reliable source of assessment for all possible indices of social anxiety, will fill the gaps in our comprehension of their correlations and of SAD, and will lead to better treatment in the future.

#### Application of Wearable AR Glasses in Mental Health

Similar to VR, AR has also been employed in mental disorders for the last two decades, however, in much smaller scale due to AR's complexity and limitations (45). With the more recent emergence of modern AR wearable glasses, such as Google Glass™, and the maturation of this technology, AR started becoming a serious potential tool for treatment of various mental disorders. AR has been employed as assistive technology for social interaction by assisting users to identify/remember people and acquire more information about these people (46, 47). Swan and colleagues (48) proposed a system for brain training using wearable AR glasses. AR has also been used in treatment of small animal phobia (49, 50) with some promising results. McNaney and colleagues investigate the employment of Google Glass™ wearable glasses as an assistive everyday device for people suffering from Parkinson's disease (51). Wearable AR glasses are recently also being utilized as tool for supporting and teaching children suffering from autism spectrum disorder in recognizing emotions and social signals in Stanford's Autism Glass project (52, 53) and as commercial products of the start-up company Brain Power (54).

Although AR has been used in exposure therapy, to date, no serious attempts have been published to use AR in SAD treatment, though its potential use has been speculated (45). However, early attempts at simulating patients in other fields of health sciences using wearable AR glasses can be taken as a demonstration of its feasibility (55).

#### PROPOSED SOLUTION

#### Enhancing CBT with VRET and Augmented Reality Exposure Therapy (ARET)

To address the challenges and limitations of contemporary SAD exposure therapy approaches as described above, DJINNI is proposed as a software and hardware solution that integrates AR and VR and various sensing technologies to support SAD patient's CBT treatment. In a nutshell, DJINNI would offer (1) an ARET that would compensate for the absence of the therapists during the *in vivo* exposure experiences and would automatically interpret various events occurring during these experiences and guide and support the patient; and (2) a VRET that would simulate exposure experiences in a safe 3D environment while incorporating the interaction and behavioral data collected from the ARET experiences. The goals and the scenarios of both ARET and VRET will be influenced by the CBT. The experiences and progress statistics collected by both ARET and VRET can influence the course of the whole SAD treatment (**Figure 1**).

#### Augmented Reality Exposure Therapy

In the ARET situation, DJINNI will be experienced as an intelligent assistant that guides and supports the patient during *in vivo* real-life exposure experiences. As illustrated in **Figure 2**, DJINNI

will be experienced by the patient through her/his wearable AR glass as a system that "understands" the patient's environment, interprets the state of the people in the environment (location, activity, conversational state, emotions, etc.), interprets the social context, assesses the state of the patient/user (location, activity, conversational state, emotions, etc.), and provides support and advices in the *in vivo* exposure experience by providing cues, advices, and soothing comments. As illustrated in **Figure 2**, the system's interaction with the patient follows a gamification approach (56) where progress is measured in scores/rewards and the patient is encouraged to progress in her/his exposure treatment. The gamification approach will be partly based on previous work where techniques for visualization of rewards and progress were employed to motivate eating disorder patients (57–59).

Let us say we have a patient suffering from anxiety of being in crowded places with high probability of scrutiny by others. For the *in vivo* exposure therapy, the patient will be equipped with wearable AR glasses, a smartphone, and a wristband with physiological sensors and will be sent to, for example, a post office. For such system to function properly in an unpredictable environment, it needs to be able to reliably detect and interpret various events occurring in the environment (e.g., spatial information; people in the environment, their roles, and their current actions/ behaviors; the patient, her/his actions, and her/his physiological state, etc.). With the help of the sensors, the patient is connected to and/or sensors installed in the environment, and based on the predefined information entered by the therapist about the environment, the system should be able to capture and detect various events reliably if we acknowledge some constrains and limitations as will be discussed later.

In practice, the system will understand that the persons behind the counter or person with specific shirt color and name tags are post office employees. The other persons are clients also waiting in the queue. Upon entering the post office, the system should monitor the physiological state of the patient and how she/he behaves in the environment. When hesitating, the system can display some instructional or motivational messages on the AR glasses on what to do (e.g., DJINNI could say: "*You are in the post office. Please stand in the queue and wait for your turn*"). When getting stressed the system can attempt to provide messages to calm down the patient (e.g., "*Don't panic. Everything is ok. Please keep standing in the queue*"). When detecting the facial expressions and the gaze of others in the environment the system would be able to have a better understanding of the situation (e.g., "*The employee just greeted you and smiles at you. Try to smile back*"). The system will also be able to detect if anxiety has caused the patient toward performing inappropriate behaviors such as jumping the queue, anxiously looking people right in the eyes, or especially avoiding eye contact (e.g., "*Relax. No reason to panic. Don't stare at her/Try to look into her eyes once a while*," or "*Call a friend on the phone to feel better*").

#### Virtual Reality Exposure Therapy

The DJINNI VRET system will complement the ARET system by allowing the patient to re-experience similar exposure events in the safe 3D VR environment. The novelty in DJINNI's VRET system in comparison with traditional VRET solutions lies in

Figure 2 | Proposed user interface of DJINNI in *in vivo* augmented reality exposure therapy situation. (Use of Djinni by courtesy of Gesa Kappen©; use of post-office photo by courtesy of RosaIreneBetancourt 6/Alamy Stock Photo).

its incorporation of data collected by the ARET system, and, by doing so, it is possible to automatically generate and adapt VR elements to simulate the events similarly to how they occurred in the *in vivo* ARET experience. Thus, it gives the patient a platform to, first, re-experience (replay) previous experiences recorded by the ARET system. Consequently, they can learn to objectively analyze/evaluate others' and one's own behavior to prevent negative rumination/post-event processing. Finally, they eventually learn how to deal with these experiences in the safe environment of VR as the ARET feeds right into the VRET platform. Similar to the ARET system, the VRET system will also follow a gamification approach to motivate the patient during her/his therapy by feeding back the actual achievements but also the progress that is already accomplished.

Considering the "post-office scenario" as described in Section "Augmented Reality Exposure Therapy," DJINNI's VRET system would utilize the data related to the events and the people in the post office environment to automatically adapt the VRET virtual world by simulating similar density of crowd in the post office, specific events that occurred (various people starting looking at the patient, someone in the queue smiled at the patient, etc.), the timing and duration of these events, etc.

#### Technical Specifications of DJINNI

The DJINNI solution consists of two separate systems that complement each other and exchange data between each other to improve the treatment. As illustrated in **Figure 3**, DJINNI's ARET system will rely on several intelligent software components that together determine the functioning and behavior of the system.

In general, DJINNI's ARET platform consists of the following:

• *Wearable AR glasses* for displaying DJINNI's messages to the user and capturing the events in the environment through the built-in camera.


Implementation of indoor localization may require equipping popular exposure therapy venues in the city (e.g., postal office, supermarket) with localization sensors that help the system in determining the exact location of the patient in an indoor environment.

• *Scenario workflows* are predefined workflows/programs defining, in a detailed way, how the system should behave during each situation in a specific exposure scenario/environment. Scenario workflows should be considered as complex but general definitions created for all the patients, while *individualized customization* are simple adaptations to these general workflows considering specific patient's needs.

As is the case with the ARET system, the VRET system as illustrated in **Figure 4** will also rely on several intelligent software components that together determine the functioning and behavior of the system. However, all the components that analyze and interpret the *in vivo* situation events are excluded as they are rather simulated instead of captured in the VR environment.

Next to the components already described above for the ARET systems, the VRET system includes the following components:


#### Workflows and Interpretations

DJINNI's scenario workflows will define step by step what actions the system should take at each possible step/situation of a specific exposure experience. Based on previous work on interactive virtual environments (60), the workflows will be implemented in artificial intelligence condition–action rules defining what actions to take given certain derived or predefined data. **Table 1** illustrates the types of data that will be derived by the system or predefined by the therapist/programmer that will serve as conditions for determining the system's actions.

#### TECHNOLOGICAL FEASIBILITY

The development of the DJINNI solution would require the integration of various technologies of which some are mature and others are experimental. The following sub-sections will review these technologies and determine their current level of development and will provide recommendations on how these technologies should be integrated and utilized in the DJINNI solution.

#### Wearable AR Glasses

Although, the idea of having wearable AR glasses as a communication interface for guiding and supporting the patient is innovative, it can also be seen as pitfall. The currently disconnected Google Glass™ wearable glasses (61) have been for a long time seen as the standard in wearable AR glasses. However, due to their cyborg style design, they would not be suitable for patients

#### Table 1 | What should DJINNI understand about the patient and the environment?


suffering from SAD to wear them as they would attract undesirable attention. However, the latest years also saw the development of various new wearable AR glasses that may attract less undesirable attention than Google Glass™. Currently available wearable AR glasses such as Laforge Shima™ (62) and Vuzix VidWear™ B3000 (63) will enable DJINNI to exploits the advantages of AR glasses without the negative impact risked if the technology is visible. Furthermore, the emerging waveguide optical technologies [e.g., Trulife Optics™ (64) and Dispelix™ (65)], which is also being used in Laforge and Vuzix, will enable many wearable AR glasses manufacturers to produce more fashionable and less attention-attracting glasses in the near future.

#### Consequence for DJINNI

For the first prototype of DJINNI, it is the intention to employ Laforge Shima™ or Vuzix VidWear™ B3000 AR glasses.

#### Audio- and Vision-Based Perception and Interpretation Technologies

Although our perception capabilities as humans is quite advanced and we are able to easily distinguish between the different objects in our human vision and to understand the characteristics and affordances of these objects, yet, for a computer system nowadays, it is impossible to reliably detect and recognize all the objects in its view, let alone to derive their characteristics and affordances.

*a Only for ARET. In VRET, this information is simulated.*

However, for each type of object, specific algorithms have been developed and some algorithms started to reach a level of sufficient maturity (66). Using machine learning techniques, computers are enabled to emulate human cognitive abilities such as sound detection, speech recognition, image recognition, emotion recognition, and behavior analysis.

#### Speech Recognition

Although speech recognition has significant success in research using techniques such as hidden Markov models (67), practical experiments have shown that currently available speech technologies are still not reliable enough for natural human–computer interaction as most algorithms are still too sensitive to noise and accents (68). In addition, it appears that the most reliable speech recognition stems from the English language (60).

#### *Consequence for DJINNI*

Because of its unreliableness, speech recognition technologies will only be used in DJINNI VRET system and only if really needed. Use of speech recognition in public places will be avoided.

#### Object Recognition/Tracking

Object recognition/tracking has seen some progress in the past decades due to innovations in learning algorithms, image processing techniques, and feature extraction (69, 70). However, most object recognition and tracking algorithms only work well with objects that are near the camera view or objects that are large enough (represented by a large number of pixels in the image).

#### *Consequence for DJINNI*

As the camera is part of the wearable glasses, object recognition/ tracking algorithms will only be able to detect objects that are in the view of the camera and close to the patient. Small objects and objects far away will not be detected by the system. Therefore, use of object recognition and tracking should be limited in the scenario workflows to large or clearly visible objects and only at certain states of the workflows (e.g., for detecting the post office counter).

#### Facial Expression Recognition

Facial expression recognition research has been ongoing for some time and has reached a convincing level of maturity. For instance, the facial action coding system, originally designed to allow human users to objectively describe facial states (71), can now be automatically processed and classified into prototypical emotional expressions using computer algorithms (72). Various research and commercial software prototypes have been designed and have reached a sufficient level of recognition.

#### *Consequence for DJINNI*

When speaking about emotional expressions, one has to be aware that there is no common definition of what emotions are (73) and workflows should be limited to utilizing prototypical Ekmanian emotions—the basic emotions anger, fear, sadness, disgust, surprise, and happiness, which are believed to be the most universal/ stable across cultures and contexts (74). In addition, emotion detection should only occur at states of the workflows where emotions are expected to be detected and relevant. Limiting the scenario will increase the reliability. Furthermore, the system should also be able to differentiate whether someone is talking to the patient or not.

#### Sound Detection

Sound detection algorithms have reached a very convincing level of maturity in the last years. Reliable commercial and opensource products that have been released can reliably distinguish between a large variety of sounds [e.g., ROAR™ (75) and Audio Analytics™ Ltd. (76)].

#### *Consequence for DJINNI*

One of the reliable sound detection products can be used to detect environmental sounds that are relevant to identify certain social environments.

#### VR Technologies

The last years saw a rapid emergence of high-quality and affordable commercial VR head-mounted devices [e.g., HTC Vive™ (77), Oculus Rift™ (78), Sony PlayStation VR™ (79), Samsung Gear VR™ (80), and Microsoft HoloLens™ (81)].

#### Consequence for DJINNI

All the existing products have reached sufficient quality and can be directly deployed for DJINNI VRET system.

#### Physiological and Affective State Interpretation Technologies

Over the last years, various fitness wristband and chest straps have been developed and employed to track one's fitness performance by integrating peripheral sensors for tracking of motions and heart rate [Mio LINK™ (82), Fitbit Charge HR™ (83), and Garmin Soft Strap Premium HR Monitor™ (84)]. Information derived from these affordable fitness trackers can be used to extract some information about the patient physiological state. The more professional wristband Empatica E4™ (85) has also been gaining prominence the last years in research due to its reliability and the number of sensors that it embeds. Detected sensor signals and signals changes can be directly incorporated into the workflows and can lead to system actions. Furthermore, research has also shown that heart rate variability and galvanic skin response can also be used as a good indicator of stress (86, 87). However, interpretation of psychological signals is also prune to errors due to noise and to the other causes of the same physiological changes.

#### Consequence for DJINNI

As indicated with other components, it is possible to reach a certain level of reliability if limited to certain scenario. When a change in physiological signals is preceded by events than can cause stress, there is a high chance that the patients feel stressed. By considering these indicators, DJINNI can reach a good level of reliability.

#### Indoor/Outdoor Localization

Localization can refer to either indoor or outdoor localization, and it refers to locating an object or oneself in a building or outside. Various technologies for outdoor localization already exist. However, it is mainly GPS, sometimes complemented by WIFI router-based localization, that has been the most reliable until now. Indoor localization is enabled by various technologies (e.g., iBeacons, RFID, sound, infrared, ultra-wideband, etc.) (88). However, the most accurate solutions for indoor localization are the ones that combine various technologies rather than using only one (89, 90). Several mature commercial systems exist that also employ various technologies [e.g., IndoorAtlas™ (91) and AccuWare™ (92)].

#### Consequence for DJINNI

For DJINNI, it is also intended to combine various technologies for indoor localization and create a hybrid system based on previous work (93). However, indoor localization is not the most crucial component of the DJINNI solution. Although the example of the post office may require some indoor localization, many other exposure scenarios do not (e.g., public speaking anxiety, dating anxiety, etc.). For several scenarios where indoor localization is needed, local health institutes can install localization devices in some public buildings if these buildings can be used for exposure therapy (e.g., iBeacons).

#### VALIDATION STUDY

As shown above, VR and AR are certainly the keys to improve actual exposure therapies. To validate DJINNI and its ARET and VRET components, the following validation study needs to be conducted. The study aims to evaluate the impact that new technology has on therapeutic progress in a population with, e.g., SAD. The proposed design consists of three consecutive steps.

#### Diagnostic Phase and Anxiety Evaluation

Naturally, the first step consists of thorough diagnostics of patients, by trained psychologist/therapists, by means of (structured) clinical interviews [e.g., Mini-International Neuropsychiatric Interview (94) or Structured Clinical Interview for DSM-5 (95)] and trait questionnaires [e.g., Social Phobia Scale (96, 97) or Fear of Negative Evaluation Scale (98)]. After the diagnostics, the therapist should repeatedly assess subjective reports of state anxiety in anxiety evoking situations by traditional scales [e.g., State Anxiety Inventory (99), Subjective Units of Discomfort Scale (100), or visual analog scale (101)] and/or its physiological equivalents (e.g., heart rate, skin conductance). These parameters at baseline should be assessed again in the same (formerly) anxiety evoking situations at the end of the treatment (after 12 weeks) and at follow-up (3 months after treatment) to eventually analyze treatment effects.

#### ARET/VRET/Traditional Exposure Therapies

Assuming that traditional CBT as well as VRET are effective treatments of SAD (37, 102), the experimental design aims to evaluate the *added* value of adaptive VRET and ARET. Accordingly, six groups of patients, with a principle diagnosis of SAD would be necessary to combine all possible exposure treatments (see **Table 2**) with ARET.

#### Table 2 | Experimental conditions of the validation study.


Based on this semi-experimental design and short-term and long-term changes in subjective experienced anxiety, behavioral and physiological indices can be used to evaluate the added value of (adaptive) VR and AR in exposure therapy. Furthermore, it will also help to assess the importance of technology-based individual tailoring/customization of treatment and evaluate the role of positive or negative feedback on the patient's behavior.

#### CONCLUSION AND DISCUSSION

The present article presented a conceptual design of DJINNI, a system that combines AR and VR to provide more effective exposure therapy solutions for patients suffering from SAD. Due to the meager effect sizes of traditional exposure therapies, heterogeneity of the patients' difficulties in social interaction and the difficulty of personalization in current state-of-the-art VRET, an approach that exploits the benefits of wearable AR glasses is desperately needed.

Yet, the use of AR technology as well as various experimental technologies for environment analysis should not lead to the development of an unreliable system that will not be beneficial or even suitable for use by SAD patients in the end. However, by taking the limitations of various technologies carefully into consideration and incorporating various contextual information regarding the place, situation, and phases of the interaction when developing the solution, it would be possible to achieve a reliable levels of accuracy with the proposed technologies. A thorough evaluation of various technologies is needed before integration into analytical and therapeutic workflows. After doing so, the paper has proposed technologies that have reached a sufficient level of reliability to be considered mature enough for building an effective ARET system.

Another innovation of DJINNI is the adaptive nature of its VRET system. By incorporating data collected from *in vivo* exposure experience or even everyday social encounters, it is possible to improve and personalize VRET scenarios. Data collected during anxiety evoking events, as well as the behavior of the people in the patients' environment, can be used to inform the behavior of the virtual environment and the people in it. It is expected that this solution can be more effective than traditional VRET solutions as it is very personalized: It simulates events that have been recently experienced by the patient in *in vivo* situations, and, yet, can take place in a "safe" therapy setting.

The final deployment of DJINNI would require the development of detailed workflows that define how the system should behave in ARET and VRET situations. This may require considerable work as each possible situation that may occur and possible actions to take by the system need to be defined in the workflows. However, by distinguishing between the general workflows and the individualized customizations, personalizing a system for each patient should not be too time consuming.

Ideally, by providing advanced, high-end technical support solutions, DJINNI could substantially improve the efficacy of CBT (for SAD) and thereby ameliorate the individuals' suffering and societal burden considerably.

#### AUTHOR CONTRIBUTIONS

All authors listed have made substantial, direct, and intellectual contribution to the work and approved it for publication.

#### REFERENCES


#### ACKNOWLEDGMENT

We would like to give special thanks to Mrs. Gesa Kappen for providing us with her lovely Djinni-drawing on such short notice.

#### FUNDING

This work was partly supported by the European research projects Miraculous Life (Grant No. 616421), Vizier (Grant No. AAL-2015-2-145), GrowMeUp (Grant No. 643647), and GEO-SAFE (Grant No. 691161).


to the CIDI. *Eur Psychiatry* (1997) 12:224–31. doi:10.1016/S0924-9338(97) 83296-8


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Ben-Moussa, Rubo, Debracque and Lange. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Virtual reality-based attention bias modification training for social anxiety: a feasibility and proof of concept study

*Antoine Urech\*, Tobias Krieger, Alvin Chesham, Fred W. Mast and Thomas Berger*

*Department of Psychology, University of Bern, Bern, Switzerland*

Attention bias modification (ABM) programs have been considered as a promising new approach for the treatment of various disorders, including social anxiety disorder (SAD). However, previous studies yielded ambiguous results regarding the efficacy of ABM in SAD. The present proof-of-concept study investigates the feasibility of a newly developed virtual reality (VR)-based dot-probe training paradigm. It was designed to facilitate attentional disengagement from threatening stimuli in socially anxious individuals (*N* = 15). The following outcomes were examined: (a) self-reports of enjoyment, motivation, flow, and presence; (b) attentional bias for social stimuli; and (c) social anxiety symptoms. Results showed that ABM training is associated with high scores in enjoyment, motivation, flow, and presence. Furthermore, significant improvements in terms of attention bias and social anxiety symptoms were observed from pre- to follow-up assessment. The study suggests that VR is a feasible and presumably a promising new medium for ABM trainings. Controlled studies will need to be carried out.

Keywords: attention bias modification, dot-probe paradigm, attention bias, virtual reality, social anxiety disorders, social phobia

## INTRODUCTION

Social anxiety disorder (SAD) is characterized by an intense fear of being criticized, judged, or rejected by others (1). SAD ranges among the most common mental disorders, with an estimated lifetime prevalence of 12.1% (2) and leads to personal, economic, and societal costs as well as comorbidity with other disorders (e.g., depression) (3).

Cognitive models of SAD suggest that socially anxious individuals are prone to biases at specific stages of information processing (4). In SAD, the attentional system is abnormally sensitive to threatrelated stimuli, and affected individuals tend to direct their attention toward threatening information during early, automatic stages of processing (5). Accordingly, reflecting the proposed hypervigilant mode toward threat in SAD, a meta-analysis showed that anxious individuals detect threat-related stimuli significantly faster than neutral ones (6). In contrast, alternative models highlight the avoidance mechanism and posit that threatening information is avoided or inhibited (7), and that anxiety has less impact on the initial detection of threat, but rather a stronger effect in modulating the maintenance of attention on the source of threat (8). Furthermore, individuals suffering from SAD showed prolonged disengagement from threat (9). In summary, there is evidence that social anxious

#### *Edited by:*

*Yasser Khazaal, Geneva University Hospitals, Switzerland*

#### *Reviewed by:*

*Bruno Herbelin, École Polytechnique Fédérale de Lausanne, Switzerland Stéphane Bouchard, Université du Québec en Outaouais, Canada*

#### *\*Correspondence:*

*Antoine Urech antoine.urech@psy.unibe.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 03 August 2015 Accepted: 15 October 2015 Published: 28 October 2015*

#### *Citation:*

*Urech A, Krieger T, Chesham A, Mast FW and Berger T (2015) Virtual reality-based attention bias modification training for social anxiety: a feasibility and proof of concept study. Front. Psychiatry 6:154. doi: 10.3389/fpsyt.2015.00154*

individuals differ from non-anxious individuals in their attention regarding their detection, disengagement, and avoidance of social threat information.

As a consequence, attention bias modification (ABM) studies have emerged to modify the attention bias and thus reducing anxiety in SAD (6, 10). ABM trainings aim at directly modifying the attentional system and patterns of neural activation in social anxious individuals in the context of the dot-probe paradigm (10, 11). The majority of attention trainings manipulated the attention bias away from threat and onto a neutral stimulus, because this approach proved efficient in some clinical trials (4, 12, 13). However, it is still not clear whether this procedure is indeed the most potent approach available (10).

Findings from different ABM training studies remained inconclusive (13–18). One possibility is that the lack of ecological validity and incomplete immersion impeded the success of some of the earlier studies, e.g., due to the fact that all ABM trainings were conducted on desktop computers or smartphones. Besides the lack of ecological validity, the use of multiple experimental manipulations in different studies (e.g., presentation length, stimulus type, or the study population) may have led to mixed results (15). Here, we propose the use of virtual reality (VR) in ABM training.

Virtual reality applications are defined by allowing the user to navigate through and interact with an environment that is close to its natural counterpart (19–21). VR enables to perform real physical actions (e.g., motor tasks) and the manipulation of virtual objects (22). Such enactments of bodily movements in VR might strengthen approach behaviors, which have been found to be crucial in SAD (23, 24). Furthermore, these possibilities lead to an improvement of the user experience compared to other media (e.g., desktop computers) (22). Moreover, it has been postulated that a high level of presence is positively associated with task performance (25), enjoyment (26), flow (27), and motivation (28). In general, it has been found that VR elicits stronger ratings on presence than desktop computers (29). To date, VR has successfully been adapted to exposure therapy, and studies have repeatedly found good long-term follow-up for several anxiety disorders (30–33). Consequently, the use of VR may have several advantages in ABM trainings.

The main goal of this study was to test the feasibility of a VR-based modified dot-probe paradigm in students with increased social anxiety.

#### METHODS

#### Participants

Fifteen undergraduate students (12 females and 3 males) between 19 and 24 years (*M* = 20.2 years, SD = 1.42) with normal or corrected-to-normal vision participated in this study. All participants were recruited through advertising at the University of Bern (e.g., pin board and lectures). The study was explicitly advertised for students suffering from increased social anxiety. For their participation, they received course credits. All participants provided written informed consent before the inclusion in the present study. The study was approved by the Local Ethics Committee of the University of Bern.

#### Apparatus

The VR-based modified dot-probe task was designed and rendered using the Python/OpenGL-based VR toolkit Vizard (WorldViz LLC, Santa Barbara, CA, USA). The virtual environment (VE) was modeled and textured using the open-source three-dimensional (3D) graphic software Blender (Blender Foundation, 2013). Participants were wearing a stereoscopic nVisor SX60 head-mounted display, which rendered the VE at 1,280 × 1,024 resolutions with a 60° diagonal field of view for each eye. The participant's position and body movement was tracked using the Microsoft Kinect Xbox 360, 250 GB (Microsoft, Redmond, USA). The whole body tracking device's depth-camera (Microsoft Kinect) was calibrated using the Flexible Action and Articulated Skeleton Toolkit (FAAST). The FAAST driver is an interface allowing for streaming the participant's skeleton to the VR engine over a VR peripheral network (VRPN) server.

#### Intervention

We created a VR-based modified version of the dot-probe task used in the previous studies to change the attention bias (15). The aim of this intervention was to associate a probe to a neutral cue, hence turning the attention away from the simultaneously presented negative cue.

Each dot-probe trial began with a fixation cross (+) presented in the center of a 3D model of a video conference wall for 500 ms. Directly after the fixation cross, two faces of the same individual were presented for 500 ms in two 3D picture frames on the 3D video conference wall, one on the left-hand side and one on the right-hand side. We used face stimuli from the NimStim set (34) and selected faces of eight individuals (four male and four female). After presenting the faces, a 3D model of a letter (E or F) appeared in front of the location of one of the faces. Participants were asked to hit the letter with their arms. If a 3D letter was presented on the left side, participants were told to hit the 3D letter with their left hand and vice versa for letters on the righthand side. After the virtual hand and the 3D letter collided, the next trial began. **Figure 1** shows an example trial of the VR-based modified dot-probe task. The instruction was to react (by hitting the letter) as quickly and accurately as possible. Before the actual dot-probe training session started, participants completed practice trials with pictures of fruits and houses instead of faces. The training session consisted of 160 dot-probe trials. The distance between the tracking system and the participants was kept constant (1.5 m) with a mark on the floor. The distance was based on the recommendations of Microsoft to achieve optimal resolution (e.g., range between 1.2 and 3.5 m). Participants were instructed not to move their feet during the experiment.

Furthermore, the VE as well as the NimStim set was preloaded in the cache to avoid further latencies. Typically, latency times are 106 ms for the Microsoft Kinect (35) and 5 ms for the VRPN server.

#### Self-Report Measures

Upon completion of the training, participants filled out the flow questionnaire (36) (example: "I feel perfectly claimed" one = disagree; seven = agree) and the eight-item presence

scale (37) (example: "the VR-training created a new world for me, which immediately disappeared when the training ended" one = not at all; seven = very strong). Additionally, they answered a single item to assess motivation ("How motivated were you to play the VR-training?" one = not at all; five = very much) and enjoyment ("Did you enjoy the VR-training?" one = not at all; five = very much) (38, 39). For measuring social anxiety symptoms, we used the following scales: the Liebowitz Social Anxiety Scale (LSAS-SR) (40), the Social Phobia Scale (SPS), and the Social Interaction Anxiety Scale (SIAS) (41).

#### Attention Bias Assessment

We used a modified version of the Posner task (42, 43). In order not to assess stimulus specific effects, we used words rather than faces as stimuli in the attention bias assessment. Similar to a previous study (15), we chose eight social threat words (e.g., "embarrassed") and eight neutral words (e.g., "original") out of a standardized set of words (44). The presented words were matched for length and frequency in the German language. The modified Posner task began with the presentation of a fixation cross, which was centered between two rectangles. After the fixation cross, a neutral or a social threat word was displayed in one of the two rectangles (left: −36.87°; right: +36.87°) for 600 ms. Following the presentation of the word, a cue (\*) appeared either at the location of the word (valid trial) or opposite of it (invalid trial). Participants were instructed to react to the cue as quickly as possible. The attention bias assessment consisted of 192 trials (128 trials valid trials, 32 invalid trials, and 32 uncued trials). The attention bias was assessed at pre-, post-, and follow-up assessment. The individual bias score was calculated by subtracting reaction times of invalid social threat trials from reaction times of invalid neutral trials. The greater the absolute value of this bias score, the more pronounced the bias (15). We eliminated 1.34% of the trials because of response latencies <50 ms or >1,200 ms (13).

#### Procedure

All participants read a standardized information and instruction sheet. After this, the participants filled out the SAD outcome measures online, followed by the assessment of the attention bias. Then the participants started the VR-based modified dot-probe task. The training took about 10 min. Afterwards, the self-report measures (presence, motivation, flow, and enjoyment) were administered in a paper–pencil version. After the training, attention bias was assessed a second time. Six weeks after the training session, participants were invited by e-mail to complete the whole assessment online (follow-up).

#### Statistical Analyses

Data were analyzed with SPSS 21. All data were analyzed for normality using the Kolmogorov–Smirnov test as well as the Shapiro–Wilk test; all tests were not significant, indicating normal distribution. Furthermore, all analyses were calculated based on the completed sample (14/15 participated in the postassessment and 13/15 in the follow-up assessment). In addition to the descriptive analyses, dependent *t-*tests (one tailed) were conducted to examine changes in the attention bias and SAD outcome measures. Additionally, the relationship between the process measures and social anxiety residualized gain scores (pre–follow-up measurements controlled for premeasurements) was calculated based on Spearman correlations.

#### RESULTS

Descriptive analyses showed that participants rated flow (*M* = 4.87; SD = 1.11), motivation (*M* = 4.13; SD = 0.63), presence (*M* = 3.46; SD = 0.96), and enjoyment (*M* = 3.79; SD = 0.79) above the midpoints of the scales. These ratings are relatively high when compared to the previous studies using the same scales (38, 39). There was no significant change in attention bias from pre to post (*P* = 0.132). Interestingly, however, there was a significant decrease of the attention bias from pretest to follow-up (*P* = 0.026).

From pre- to follow-up assessment, dependent *t*-tests showed a significant reduction of social anxiety measured with the LSAS (*P* = 0.04) and marginally significant results for the SPS score (*P* = 0.07) and the SIAS score (*P* = 0.06) (see **Table 1**). However, the observed statistical significant results would not be significant, if corrected for multiple testing.

The relationship between social anxiety residualized gain scores on the SPS score and presence showed a significant negative correlation (*r*=−0.534, *P*= 0.049), all other correlations were not significant.

#### DISCUSSION

The aim of this study was to test the feasibility of using ABM training via VR in a single session as a new approach for individuals with social anxiety symptoms. Most importantly, the present study shows that ABM training can be implemented successfully in VR. Furthermore, with respect to the technical side of the VR-based ABM training, it is possible to track an arm movement in real time without expansive motion tracking systems.

It is noteworthy that all participants finished the VR-based ABM training and that the training was associated with high

#### REFERENCES


#### Table 1 | Means and SDs for social anxiety measures.


*Pre: N* = *15 and Follow-up: N* = *14.*

*\*P* < *0.05.*

scores in enjoyment, flow, presence, and motivation, indicating good acceptance and feasibility of the intervention. Despite the short duration of the training (10 min), the attention bias and scores on the LSAS decreased significantly from preassessment to the 6 weeks follow-up assessment. Boettcher et al. (15) showed no change in attention bias over time in social anxious individuals using the same assessment procedure, as described in this study, in combination with a placebo control dot-probe training. ABM training in our study targeted the modification of reaction times to social threat stimuli. In this respect, our data indicate that reaction times to social threat stimuli changed significantly from pre- to follow-up assessment, whereas reaction times in trials with neutral stimuli did not change over time. This suggests that the present VR training showed specific effects on the target variables in the expected direction.

However, the results of the present study should be interpreted with caution. This feasibility study included no control group. Furthermore, the sample in the present study consisted of students with self-assessed increased social anxiety. Nevertheless, several authors showed that threat-related bias was similar in clinically anxious and non-clinical high-anxious participants (45, 46).

To evaluate the efficacy of the intervention, a sufficiently powered randomized controlled trial is needed. Moreover, future studies should examine whether ABM trainings could be used either as a stand-alone treatment or in combination with psychotherapy. In addition, it should be considered that as a first step to implement ABM trainings into VR, we used two-dimensional pictures of faces. More ecologically valid and realistic stimuli such as 3D stimuli of faces or meeting autonomous virtual characters in VR might improve the outcome.

In conclusion, the current study demonstrated the feasibility of a complex (e.g., body tracking and motor task) VR-based ABM training.


nonanxious individuals: a meta-analytic study. *Psychol Bull* (2007) **133**(1):1. doi:10.1037/0033-2909.133.1.1


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Urech, Krieger, Chesham, Mast and Berger. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Using Virtual Reality in the Inference-Based Treatment of Compulsive Hoarding

*Marie-Eve St-Pierre-Delorme and Kieron O'Connor\**

*Institut universitaire en santé mentale de Montréal, Montreal, QC, Canada*

The present study evaluated the efficacy of adding a virtual reality (VR) component to the treatment of compulsive hoarding (CH), following inference-based therapy (IBT). Participants were randomly assigned to either an experimental or a control condition. Seven participants received the experimental and seven received the control condition. Five sessions of 1 h were administered weekly. A significant difference indicated that the level of clutter in the bedroom tended to diminish more in the experimental group as compared to the control group *F*(2,24) = 2.28, *p* = 0.10. In addition, the results demonstrated that both groups were immersed and present in the environment. The results on posttreatment measures of CH (*Saving Inventory revised, Saving Cognition Inventory and Clutter Image Rating scale)* demonstrate the efficacy of IBT in terms of symptom reduction. Overall, these results suggest that the creation of a virtual environment may be effective in the treatment of CH by helping the compulsive hoarders take action over their clutter.

#### *Edited by:*

*Yasser Khazaal, Geneva University Hospitals, Switzerland*

#### *Reviewed by:*

*Lucia Romo, Université Paris Ouest Nanterre La Défense, France Thomas Berger, University of Bern, Switzerland*

*\*Correspondence:*

*Kieron O'Connor kieron.oconnor@umontreal.ca*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Public Health*

*Received: 29 January 2016 Accepted: 04 July 2016 Published: 19 July 2016*

#### *Citation:*

*St-Pierre-Delorme M-E and O'Connor K (2016) Using Virtual Reality in the Inference-Based Treatment of Compulsive Hoarding. Front. Public Health 4:149. doi: 10.3389/fpubh.2016.00149*

Keywords: compulsive hoarding, virtual reality, treatment, cognitive therapy, inference-based therapy

#### INTRODUCTION

Though classified in DSM-V as a distinct disorder, compulsive hoarding (CH) was long considered a subtype of OCD. Also new in DSM-V is that OCD is no longer considered an anxiety disorder. CH and OCD are now considered separate disorders, part of the Obsessive Compulsive and Related Disorders category. Regardless, the experience of intense anxiety when required to get rid of personal objects and difficulty in taking action are part of the diagnostic criteria for CH (1). The majority of studies investigating CH and OCD were conducted when CH was still considered as anxiety disorder. Virtual reality (VR) is a rapidly growing area of technology that is being used more and more as an adjunct to treatment for various mental health problems. VR facilitates exposure in that almost any context or situation can be simulated (2). Studies have found that VR is effective for the treatment of panic disorder (3), social phobia (4), obsessive–compulsive disorder (OCD) (5, 6), post-traumatic stress disorder (7, 8), specific phobia (9, 10), generalized anxiety disorder (11), and eating disorders (12). To our knowledge, no study has yet investigated the treatment of CH with VR.

A meta-analysis that evaluated the use of VR for anxiety disorders found that the effect sizes were quite large for the 21 included studies, with the average effect size being 0.95 (13). Another meta-analysis that exclusively looked at the use of VR for anxiety disorders (*n* = 13) corroborates the results of the other meta-analysis and reports an average effect size of 1.11 for VR, as compared to a control condition (14). When considering the effect size for VR in comparison to an *in vivo* condition, the authors describe an effect size of 0.35 in favor of VR. Furthermore, in several controlled trials, VR was found to be as effective as *in vivo* exposure (15).

A few studies have also examined environments known as "non-immersive," which are generated using a standard computer. These studies found that the use of non-immersive environments can allow for a state of presence in the environment, elicit an emotional response equivalent to an immersive environment, and can lead to significant amelioration of clinical symptoms (4, 16). It is, therefore, possible to infer that the creation of an environment simulating CH conditions can provoke anxiety as well as encourage sorting items and uncluttering in the therapeutic process.

#### Compulsive Hoarding

Compulsive hoarding is characterized by a number of behaviors, such as cluttered rooms and difficulty or refusal to get rid of unnecessary objects. These symptoms cause an important level of distress, which interferes with everyday life (1). Individuals with CH also tend to be indecisive, perfectionistic, and disorganized. They also often procrastinate and have an urgent need to save and acquire a diverse array of objects (17). It has also been observed that hoarders take significantly more time to sort, create more piles, and experience more anxiety than non-psychiatric controls, but this is only true when dealing with their own personal objects (18).

The efficacy of existing treatments for CH is limited (19). In addition, studies cite certain issues with using exposure, as hoarders tend to drop out of treatment at this step, and many others are reluctant to let the therapist enter their homes (19). As they have difficulty in making decisions, it is not easy for hoarders to take action once it is time for them to organize their environment or remove clutter. They also have difficulty in completing homework, and their lack of motivation is often an issue in the therapeutic process (20, 21). That said, VR represents an avenue to explore in the treatment of CH, as it circumvents the aforementioned difficulties that usually represent important obstacles in therapy.

#### Context of the Problem

As VR allows for precise control over what is presented, it is possible to create environments tailored to the individual, based on the needs of the client (6). Also, VR can be helpful for CH as it is controlled, predictable, and reliable (22). In addition to these advantages, VR can also be administered in the clinical context (without having to leave the office) and costs less (15). This tool can also be useful for individuals, like CH, who have difficulties with visualizing everyday scenes when using mental imagery techniques (23). Furthermore, the clutter in the homes of the hoarders is very visual, and VR is a technology that primarily uses this aspect of the sensory system. All of these reasons support VR as a viable alternative to at-home visits to facilitate action taking in hoarders.

The objectives of the present article are to evaluate whether VR is an efficacious component in helping hoarders to take action toward reducing clutter and to validate the efficacy of a group version of inference-based therapy (IBT) validated in an individual format (Blais et al. forthcoming)1 .

#### The hypotheses:


#### MATERIALS AND METHODS

#### Participants

Participants were recruited using advertisements posted in universities, hospitals, CLSC's, and community organizations in the region of Montréal, as well as Fernand Seguin research centre's website. Twenty-five participants were evaluated in the context of this project. Of these, nine were excluded following the initial evaluation, and two dropped out over the course of treatment, one in each group. Of the 14 participants who took part in the project, 2 were males and 12 were females. Demographic data are reported in **Table 1**. Inclusion criteria were as follows: (a) a primary diagnosis of OCD with CH as described in DSM-IV-TR as well as the criteria proposed for DSM-V; (b) stable medication for at 12 weeks; (c) accept to keep medication stable throughout



<sup>1</sup>Blais MT, Bodryzlova Y, Aardema F, O'Connor K. Open trial of inference-based therapy in the treatment of compulsive hoarding. *J Behav Addict* (Forthcoming).

participation in the study; (d) no evidence of current suicidal ideation; (e) no evidence of current alcohol or drug abuse; (f) no evidence or diagnosis of schizophrenia past or present, bipolar disorder, or organic mental disorder; and (g) accept to not receive any other treatment for CH during the course of the study. Excluded participants were referred to the most appropriate resources, given their situation. Comorbid symptoms, such as obsessive–compulsive disorder traits and other subtype of obsessive–compulsive disorder, were observed for seven participants. Participants with mild comorbidity were included, three participants had mild depressive symptoms, six had other subtypes of OCD symptoms and four had obsessive–compulsive personality traits.

#### Equipment

The environments were generated by a PC with the following specifications: PC Pentium 4®, 1.98 GHz 3.48 GB of RAM, with 256 MB of video memory. The environments were projected on a 21″-monitor. The *neuroVR* 2.0 platform (www.neurovr.org) was used to create the environments. A digital camera was used to take photographs of the CH's home environments and *Corel Paint Shop Pro Photo X2* software was used to treat the images. This procedure was validated at the Université du Québec en Outaouais (UQO) in a sample of six participants (24).

#### Virtual Reality

#### Experimental Environment

All of the virtual environments were created using the neuroVR 2.0 platform (www.neurovr.org), which is freely accessible online. In the experimental condition, objects that belong to the hoarders were inserted into the apartment environment created using this platform. In the control condition, images depicting objects were selected from the internet and inserted into the apartment environment. To create the experimental virtual environments, approximately 30 photos taken by participants of their homes were selected. The objects in these images were cut out using *Corel Paint Shop Pro Photo X2* and inserted in the apartment environment. The objective was to recreate the participant's apartment, mainly their living room, kitchen, and bedroom. The objects were placed in piles as represented in their pictures. The objects that were eventually selected to be used in VR procedure were cut out individually so that the participant could select them.

#### Control Environment

An active control condition (see **Figure 1**) was chosen as opposed to a passive control condition. This active type of condition was preferred because it allows for an equivalence of parameters, such as the number of sessions and contact with a therapist. For participants in the control group, the virtual environment was created in the same way as for the participants in the experimental group. The only difference was that the objects did not belong to the participants. Common household objects were selected like shoes, books, magazines, frames, etc. The selected objects came from the internet, the homes of the therapists, and the research center.

#### Clinical Evaluations

A battery of questionnaires and semi-structured interviews were administered to all participants who took part in the project. The purpose of these measures was to obtain information regarding the presenting problem of the individual, to establish the presence of depression and anxiety, and to measure the state the participant was in, following each VR task. The self-report measures were administered pretreatment (IBT), post-IBT treatment, and post-VR. The clinical interviews were administered by an evaluator trained in the administration of CH measures. The evaluation lasted for 4 h.

The *Structured Clinical Interview for DSM-IV* for Axis I disorders [SCID-I; French version; Ref. (25)] was used to establish a differential diagnosis for Axis I disorders according to the

diagnostic criteria of the DSM-IV-TR. These measures have very good psychometric properties (26).

The *Yale–Brown Obsessive Compulsive Scale* [YBOCS; Ref. (27); French translation; Ref. (28)] was used in the clinical evaluation of obsessive–compulsive symptoms and their severity. The YBOCS can also be used to evaluate overt and covert neutralization behaviors, separately. Studies have found support for the validity and reliability of these subscales (ICC = 0.01–0.94, *r*<sup>s</sup> = 0.90) (29).

The *Overvalued Ideas Scale* (OVIS) (30) is an 11-item semistructured interview evaluating overvalued ideas across several dimensions (e.g., efficacy of compulsions, degree of belief held by others, etc.). This measure is often used to measure the degree of introspection of individuals suffering from different obsessional disorders, such as OCD. The OVIS has satisfactory internal consistency (α = 0.88), test–retest reliability (*r* = 0.86), and interrater reliability (*r* = 0.88).

The *Evaluation of Primary Inferences Scale* (EPIS) (31) was developed to measure the strength of the primary obsessional doubt. This measure is complementary to and more specific than the OVIS, with regards to primary doubts. With the help of a psychologist, the participant must identify their primary inferences regarding their obsessions and determine for each their level of conviction (%) in terms of the probability that this belief is real in the "here and now." For a hoarder, the primary inference is often formulated as: maybe I can repair this object or maybe I can save some money.

The *Evaluation of Secondary Inferences Scale* (EPIS) (31). With the help of a psychologist, the participant must identify their secondary inferences (anticipated consequences of the primary inferences) and answer for each the following question: "please evaluate to what extent (%) inferences described here are realistic if you do not perform your compulsions."

#### Symptom Measures

Three measures of hoarding were used to evaluate the severity of hoarding based on the formal definition of the problem. The first questionnaire, the *Saving Inventory-Revised* (SIR) (32), is comprised of 23 items scored on a scale from 0 to 4. The subscales are (a) compulsive acquisition, (b) difficulty discarding objects, and (c) clutter in the home environment. These subscales have been found to have good reliability.

The *Saving Cognitions Inventory* (SCI) (33) is a questionnaire comprised of 24 items measuring beliefs related to CH symptoms. There are four main subscales: (a) emotional attachment, (b) preoccupation regarding memory, (c) need for control, and (d) responsibility regarding possessions. The internal consistency of these subscales is good and varies between 0.86 and 0.95.

The *Clutter Image Rating* (CIR) (34) is a series of nine images that correspond to different degrees of severity of clutter. The rooms depicted are the kitchen, living room, and bedroom. This measure has good internal consistency (α = 0.84) and intercorrelations between 0.56 and 0.71. A score of 4 or more reflects the presence of CH symptoms, and each room receives its own score. There is no total score.

The *Beck Depression Inventory* (BDI-II) (35) is a 21-item selfreport questionnaire, which aims to evaluate affective, cognitive, motivational, and physiological symptoms of depression during the last 2 weeks. Like the original English version, the French translation has good psychometric qualities, such as internal consistency (α = 0.92–0.93) and test–retest reliability (*r* = 0.93).

The *Beck Anxiety Inventory* (BAI) (36) contains 21 items designed to evaluate the intensity of anxiety symptoms during the last week. Like the original English version, the French translation has acceptable internal consistency (α = 0.84) and test–retest reliability (*r* = 0.63) (37).

The *Inferential Confusion Questionnaire – Expanded Version* (ICQ-EV) (38) is comprised of 30 items measuring inferential confusion, a construct referring to how an individual accords a certain degree of probability to imaginary possibilities (39). This measure has excellent internal consistency (α = 0.97) and test–retest reliability (*r* = 0.90).

#### *VR Measures*

The Canadian adaptation of the *Immersive Tendencies Questionnaire* (ITQ) (40) is comprised of 18 items divided into four subscales: (a) focus, (b) implication, (c) emotions, and (d) game. A study by Witmer and Singer (40) demonstrated good psychometric qualities, and the French translation validation study reported a Cronbach's alpha of 0.78 (41).

The French Canadian adaptation of the *Presence Questionnaire* (PQ-F) (40) consists of 24 items and seven subscales: (a) realism, (b) possibility to take action, (c) quality of the interface, (d) possibility to examine, (e) self-evaluation of performance, (f) auditory, and (g) tactile. The authors report good internal consistency. It is also worth noting that the last two subscales were not used in the present study, as they are optional and no auditory or tactile element was used.

The French Canadian translation of the Simulator Sickness Questionnaire (SSQ-F) (42) was also administered. The two main subscales are nausea and oculomotor problems. A study evaluating this French-language version reports an excellent Cronbach's alpha of 0.87 (43).

Finally, the therapist asked the participants to rate their level of anxiety and discomfort during the VR tasks. For example, when the client had to discard a virtual object, the therapist asked the client to rate their level of anxiety and discomfort on a scale from 0 to 10 and took note of the rating. This allows for measure of the difficulty of the task and also to verify if the VR elicited the expected emotions.

#### Procedure

Fourteen participants with a diagnosis of CH took part in this project. The participants were randomly assigned to one of two of the following conditions: active control condition and experimental condition.

#### Treatment Protocol

#### *Inference-Based Therapy*

All of the participants received IBT administered by psychologists trained in this approach. The treatment protocol used was developed by O'Connor et al. (31) and adapted for CH clients (44). Participants received 24 group format sessions, each lasting for 1½ h. The sessions were audio recorded and verified by an independent person involved with the project to ensure the integrity of the steps of the treatment was respected. The 10 steps of IBT adapted for CH are the following: (i) distinguish normal from CH obsessional doubt; (ii) establish the logic of the CH doubt; (iii) the CH doubt is 100% imaginary; (iv) how CH becomes a "lived" experience; (v) crossing the border of reality; (vi) and (vii) reasoning devices in CH; (viii) establish the selective nature of the imaginary doubt; (ix) vulnerability of these in the CH stories; and (x) awareness of reality and tolerating the void. Treatment was the same in both conditions.

#### *Virtual Reality*

After receiving IBT, five sessions of 1 h were given to participants in both conditions. These sessions were administered by psychologists trained in the use of neuroVR 2.0 and the established treatment protocol. These sessions were also audio recorded and verified by an independent person involved with the project to ensure the integrity of the steps of the VR sessions was respected. In addition, the participants had no contact with one another between VR sessions. The first session allowed participants in the experimental condition to familiarize themselves with the virtual environment and to change elements of the environment to make them resemble, as much as possible, their actual environment at home. The second session was aimed at helping participants begin to sort through their homes by establishing a plan of action based on the elements present in the virtual environment. In the last three sessions, participants took action virtually by disposing of objects already selected based on the degree of subjective distress they reported. Once an object was selected, the participants were supposed to put it in a virtual, 3D garbage can. They were then asked to re-evaluate their degree of anxiety and discomfort on a scale of 0 to 10. At the beginning of each session, the therapist evaluated if some objects were sorted or taken out from home by the participant. The last session was also used to discuss relapse prevention and to establish a plan of action for the coming months. The client was asked to identify at-risk situations for relapse and to write a plan of action with the therapist as a way to ensure the client had all of the tools that may be needed in the future. Finally, measures of presence and VR sickness were completed at the end of each VR session.

For control participants, the steps of treatment were the same. The only difference was that the objects selected when sorting and discarding did not belong to them. Their level of anxiety and discomfort on a scale from 0 to 10 was still evaluated to determine the presence of any emotions toward these objects. With regards to the task completed at home, participants were asked to discard one object, without specifying which one. The point of this exercise was to see if they would dispose of an object that was similar to the one discarded in the virtual environment.

#### Statistical Analysis

Normality was verified for all variables and the analyses were conducted by taking into consideration whether the assumptions were met or not. Analysis was conducted on completers. Repeated measures ANOVAs and *T*-test were conducted on all normal variables. Non-parametric analyses were conducted when variables did not meet the assumptions of normality or had a small sample size. The variables for which non-parametric analyses were conducted are the following: the subscale "control" of the SCIR and the "acquisition" and clutter subscales of the SIR. The non-parametric analyses of these variables will be presented following the presentation of the parametric analyses. Additionally, a significance level of 0.10 was chosen because of the small sample size. Even though we used a cut-off point of 0.10, we nonetheless corrected for multiple comparison analysis.

#### RESULTS

#### Pretreatment Analyses

Univariate (age) and chi-square (sex, civil status, level of education, medication, and revenue) analyses did not find any significant differences between the groups (*p* > 0.05). Also, there were no significant differences between the groups at pretreatment on clinical measures.

#### Pre-VR and Post-VR Analyses

#### Main Measures

Repeated measures ANOVAs were conducted on the main measure of clutter, the CIR. With regards to the images of the bedroom, a significant interaction was observed *F*(2,24) = 2.28, *p* = 0.10. It is possible to qualify this interaction because of the significance level. Contrasts indicated a linear interaction between the two groups for the pre-VR and post-VR *F*(1,12) = 7.80, *p* < 0.001. This interaction demonstrates that the participants' results for the series of images of the bedroom in the experimental group had a tendency to decrease over time as compared to the control group. There were no interactions observed for the other rooms depicted in the CIR, the kitchen *F*(2,24) = 0.16, *p* = 0.85 and the living room *F*(2,24) = 0.90, *p* = 0.42.

Paired sample *t*-tests were conducted in order to determine if there was a difference between the mean level of anxiety before and after action was taken in the two groups. A significant difference was observed for the experimental group *t*(6) = 17.67, *p* < 0.001 as well as for the control group *t*(6) = 8.00, *p* < 0.001. An independent *t*-test was conducted, and a significant difference was found between the groups before action was taken in VR *t*(12) = 3.36, *p* < 0.05 and after the VR task *t*(12) = 3.35, *p* < 0.05.

#### Secondary Measures

A repeated measures ANOVA demonstrated a main effect of time *F*(2,24) = 5.32, *p* < 0.05, but no interaction effect, for the "emotional" subscale of the SCIR *F*(2,24) = 0.97, *p* = 0.39. There was also a main effect of time for the "responsibility" subscale of the SCIR *F*(2,24) = 4.20, *p* < 0.05 and the "memory" subscale *F*(2,24) = 7.76, *p* < 0.05, but no significant interaction effects between groups (*p* > 0.05). A Greenhouse–Geisser correction was applied to these variables. Finally, a main effect of time was observed for the SCIR total score *F*(2,24) = 9.46, *p* < 0.001, but there were no interaction effects.

As previously mentioned, a main effect of time from pre-IBT to post-RV was found for the SIR total score and its three subscales. A repeated measures ANOVA did not find an interaction between the groups for the "discarding objects" subscale *F*(1,12) = 2.38, *p* = 0.15 or for the SIR total score *F*(1,12) = 1.68, *p* = 0.22. As the "clutter" and "acquisition" subscales did not meet the assumptions of normality, non-parametric analyses were conducted. There is no test equivalent to a 2 × 2 repeated measures analysis. As such, a composite score between post-RV and pre-IBT was calculated and a Mann–Whitney *U* test was conducted in order to determine if there was a significant difference between groups. No difference was observed for the "clutter" subscale for the experimental group (Md = 0.00, *n* = 7) and the control group (Md = 2.00, *n* = 7), *U* = 20.00, *z* = −0.58, *p* = 0.56, *r* = 0.14. Also, there was no difference for the "acquisition" subscale for the experimental group (Md = 12.28, *n* = 7) and control group (Md = 3.00, *n* = 7), *U* = 14.00, *z* = −1.35, *p* = 0.18, *r* = 0.36.

#### Other Clinical Questionnaires

A repeated measures ANOVA found a main effect of time for the total score on the YBOCS between pre-IBT, post-IBT, and post-RV *F*(2,24) = 3.56, *p* < 0.05, and no interaction effect *F*(2,24) = 0.16 *p*= 0.85. A main effect of time was also observed for the "obsessions" subscale *F*(2,24) = 4.19, *p* < 0.05 but not for the "compulsions" subscale *F*(2,24) = 1.13, *p* = 0.31. No interaction was observed between the groups for either subscale. A repeated measures ANOVA found no main effect *F*(2,24) = 2.39, *p* = 0.11 or interaction *F*(2,24) = 0.24, *p* = 0.24 for the OVIS. There was, however, a linear trend across time for both groups. Essentially, the total score demonstrated a tendency to decrease for the two groups. The results do not demonstrate a main effect for the BDI *F*(2,22) = 0.02, *p* = 0.98 or the BAI *F*(2,22) = 0.22, *p* = 0.80. There was also no interaction observed for the BDI *F*(2,22) = 1.81, *p* = 0.19. There was, however, an interaction found for the BAI *F*(2,22) = 3.17, *p* < 0.10. The BAI scores for the control group had


a tendency to decrease over time, while that of the experimental group increased from pre-IBT to post-IBT, and decreased at post-VR. A Mann–Whitney *U* test did not find any difference between the groups on the ICQ-EV at post-VR: experimental group (Md = 85, *n* = 6) and control group (Md = 82, *n* = 7), *U* = 19.5, *z* = −0.22, *p* = 0.83, *r* = 0.06. See **Table 2** for a synthesis of these results. **Table 2** also shows differences in pre and post-IBT as well as interactions between the two groups post-VR.

#### VR Measures

Independent *t*-tests found no significant difference between groups on any of the subscales of the presence questionnaire. The scores demonstrated a high degree of presence for both groups according to the French-language norms of the questionnaire; experimental group M = 90.96, SD = 3.07; control group M = 86.98, SD = 20.94. The SD is much larger for the control group than for the experimental group. An independent *t*-test did not find any difference between the two groups for the Simulator Sickness Questionnaire. Furthermore, each group obtained scores that were below the average reported in the norms for the questionnaire (experimental: M = 4.6, SD = 6.10; control: M = 7.27, SD = 2.95).

#### Comparison with a Non-Concurrent Control Group

We also compared the results of this study with a non-concurrent data collected from another group. Five participants were recruited in the same way as the participants in the present study's sample and participated in the first therapy group given at the Fernand Seguin research center, before the virtual environments were


*\*p* < *0.10.*

*\*\*p* < *0.001.*

*YBOCS, Yale–Brown obsessive compulsive scale; OVIS, overvalued ideation scale; BAI, Beck anxiety inventory; BDI, Beck depression inventory; SIR, saving inventory revised; SCIR, saving cognition inventory revised; ICQ-EV, inferential confusion questionnaire extended version; CIR, Clutter Image Rating.*

created. They received the same treatment protocol (IBT), and it was the same therapist who animated the sessions.

Measures were taken at pretreatment, posttreatment, and at 6 months following completion of the treatment. The results demonstrate no clinical or statistical difference on measures of clutter (CIR) from pre-IBT to post-IBT; bedroom: *z* = 0.00, *p* = 1.0; kitchen: *z* = −1.00, *p* = 0.32; living room: *z* = 0.00, *p* = 1.00. This was also the case for measures from post-IBT to 6-month followup: bedroom: *z* = 0.00, *p* = 1.0; kitchen: *z* = −1.41, *p* = 0.20; living room: *z* = 0.00, *p* = 1.00. The results for the three rooms decreased.

With regards to the secondary measures, scores on the YBOCS decreased significantly from pre-IBT to post-IBT *z* = −1.6, *p* < 0.10, with a large effect size (*r* = 0.57). The median score decreased from pre-IBT (Md = 21.50) to post-IBT (Md = 17.00). The "obsessions" subscale decreased significantly *z* = −1.83, *p* < 0.10, with a large effect size (*r* = 0.65). The median score decreased from pre-IBT (Md = 11.00) to post-IBT (Md = 8.50). No change, however, was observed for the "compulsions" subscale *z* = −0.38, *p* = 0.71. There was no significant difference for the total YBOCS score from post-IBT to 6-month follow-up *z* = −0.00, *p* = 1.00.

There was no significant difference from pre-IBT to post-IBT on the OVIS: *z* = −0.37, *p* = 0.72 and BDI: *z* = −0.92, *p* = 0.36. There was a significant difference on the ICQ-EV *z* = −1.60, *p* < 0.10, with a large effect size (*r* = 0.60). The median score decreased from pre-IBT (Md = 76.00) to post-IBT (Md = 65.50).

A significant difference on the BAI was also observed *z*=−1.83, *p* = 0.06, with a large effect size (*r* = 0.65). The median score decreased from pre-IBT (Md = 16.50) to post-IBT (Md = 7.00). There was no significant difference on any of the questionnaires from post-IBT to 6-month follow-up.

With regards to the secondary CH measures, no significant decrease on the SIR total score and the "clutter" and "discarding objects" subscales was observed between pre-IBT and 6-month follow-up. A significant difference was observed on the "acquisition" subscale *z* = −1.63, *p* < 0.10, with a large effect size (*r*= 0.62). The median score decreased from pre-IBT (Md = 2.00) to 6-month follow-up (Md = 0.00). No measures were taken at post-IBT in this sample, as they were in the present study.

With regards to the SCIR, no significant difference was observed from pre-IBT to post-IBT for the total score or for its three subscales: "emotional," "responsibility," and "memory." A significant difference was observed for the "control" subscale from pre-IBT to post-IBT *z* = −1.63, *p* < 0.10, with a large effect size (*r* = 0.62). The median score decreased from pre-IBT (Md = 2.00) to post-IBT (Md = 0.00). No significant difference was observed from post-IBT to the 6-month follow-up for the SCIR and its subscales.

## Pretreatment IBT and Posttreatment IBT Analyses

#### CH Questionnaires

Paired sample *t*-tests were conducted to measure change on measure of CH from pre-IBT to post-IBT. Unfortunately, the SIR was not completed at post-IBT because of an omission in the questionnaire package given to participants. As such, the only measures available for the SIR are at pre-IBT and post-VR. A significant difference was observed from pre-IBT to post-VR for the SIR total score *t*(13) = 4.79, *p* < 0.001 and the "discarding objects" subscale *t*(13) = 5.44, *p* < 0.001. A Wilcoxon test of signed-rank showed a statistically significant decrease on the "acquisition" subscale from pre-IBT to post-VR *z* = −2.64, *p* < 0.005, with a large effect size (*r* = 0.50). The median score on the "acquisition" subscale decreased between pre-IBT (Md = 21) and post-VR (Md = 15). No significant difference was observed for the "clutter" subscale between pre-IBT and post-VR *z* = −1.03, *p* = 0.31.

A significant difference was found for the SCIR total score from pre-IBT to post-IBT, *t*(13) = 4.59, *p* < 0.001, and for its three subscales: "emotional" *t*(13) = 2.82, *p* < 0.05; "responsibility" *t*(13) = 3.07, *p* < 0.05; and "memory" *t*(13) = 6.07, *p* < 0.001.

A Wilcoxon signed-rank test demonstrated a statistically significant decrease for the subscale "control" of the SCIR from pre-IBT to post-IBT *z* = −2.80, *p* < 0.005, with a large effect size (*r* = 0.53). The median score decreased between pre-IBT and post-TBI (Md = 15.46).

No significant difference was found between the different images on the CIR between pre-IBT and post-IBT (bedroom: *t*(13) = 1.14, *p* = 0.27; kitchen: *t*(13) = 1.30, *p* = 2.22; living room: *t*(13) = 0.54, *p* = 0.60).

A repeated measures ANOVA was conducted comparing the two groups (experimental and control) for all measures of CH from pre-IBT to post-IBT and no significant interaction was observed between the two groups (*p* > 0.05).

#### Other Clinical Measures

Paired sample *t*-tests were conducted and demonstrated a significant decrease in YBOCS total score from pre-IBT to post-IBT *t*(13) = 3.21, *p* < 0.05, and for the "obsessions" subscale *t*(13) = 2.77, *p* < 0.05. Also, a significant difference was observed for the "compulsions" subscale *t*(13) = 3.21, *p* < 0.06. There was no significant difference on the OVIS *t*(13) = 1.37, *p* = 0.20, BDI *t*(13) = 1.00, *p* = 0.34, or BAI *t*(13) = −0.09, *p* = 0.93. To better understand these results, a repeated measures ANOVA was conducted and the results demonstrated that for the BAI, an interaction effect was present between the two groups *F*(1,12) = 8.64, *p* < 0.05. The scores of the experimental group increased, whereas those of the control group decreased. As the assumption of sphericity was not met, the Greenhouse–Geisser correction was applied. As the ICQ-EV was not normally distributed and also not transformable, a Wilcoxon signed-ranked test was conducted. There was no significant difference from pre-IBT to post-IBT for the ICQ-EV *z* = −0.72, *p* = 0.47.

#### DISCUSSION

As expected, the results on measures of VR demonstrated a good state of presence during the experimentation, good immersion, and very little VR sickness. These results confirm the hypothesis that non-immersive virtual environments allow for the creation of a feeling of presence and immersion, which can contribute to the eliciting of emotions, when using the virtual environment.

It was observed that both groups experienced anxiety during the VR sessions, a finding that is contrary to the initial hypothesis that stipulated that the control group would not experience significant anxiety. A significant difference, however, was found between the groups. Specifically, the participants in the experimental group experienced significantly more anxiety than the control group during the action taking task completed in the virtual environment. Following the VR task, the control group was significantly less anxious than the experimental group. This may be explained by the fact that the virtual environment in the experimental condition used participants' personal objects, whereas the virtual environments in the control condition did not. For this reason, the feeling of attachment toward the objects was not the same for participants in each condition.

Following VR sessions, there was an interaction between the groups on the main measure of clutter, that is, for the images of the bedroom. The level of clutter in the experimental group showed a tendency to decrease, as compared to a tendency to increase in the control group. This may signify that taking action in VR allowed participants in the experimental group to discard significantly more objects in their bedroom. It does not seem that action taken in the control group allowed participants to discard objects in the same room, as their scores actually increased. It should be noted, however, that for participants in the experimental group, the majority of selected pictures were of their bedroom.

Following VR, there was also a decrease over time in cognitions related to the CH symptoms "responsibility," "emotional," and "memory" as well as for the total score. The SIR total score as well as the "acquisition" and "discarding objects" subscales also decreased over time, as well as the "obsessions" subscale of the YBOCS. Overvalued ideas also evinced a tendency to decrease over the three time points, pre-IBT, post-IBT, and post-VR. Finally, a significant interaction was observed between the groups on a measure of anxiety. The results of the experimental group increased slightly at post-IBT and decreased at post-VR, whereas anxiety in the control group consistently decreased over time. This may be explained by the fact that individuals in the experimental group had to submit pictures of their home so that their virtual environment could be created. This may have caused anxiety as many of the participants in this group reported worries about what was going to happen in the VR sessions.

Finally, the comparison of the experimental group with the control group and non-concurrent comparison group suggest that the VR condition is an accessible and interesting tool that may help the participant take action at home. The results also demonstrate that IBT is a promising approach in the treatment of CH as it was possible to observe improvements in CH symptoms in the participants. It may also be possible to use VR as a preventative measure to impede the development of compulsive behavior (45).

After receiving IBT, participants in all three groups evinced a significant improvement in their CH-related cognitions. The same can be said for the measures of obsessions and compulsions, as these scores significantly decreased as well for both groups included in the present study. No differences were observed, however, on measures of depression or inferential processes. Also, no differences were observed on the measure of clutter (CIR). These results are in line with other studies that have reported that hoarders have difficulty in taking action when it comes to their clutter. With regards to the measure of depression, the participants expressed that they would have appreciated additional sessions. The majority of them reported feeling discouraged regarding the clutter of their homes and did not see any progress in this respect. With regards to overvalued ideas, even though the scores on the OVIS did not significantly decrease, the clinical scales demonstrated a decrease of 40% in the primary doubt. This pattern of results may be attributable to the fact that the OVIS considers only one overvalued idea, whereas the clinical scales are more exhaustive, taking into account all of the overvalued ideas reported by the participant.

The limitations of the present study are mainly the small sample size and the absence of a passive control group. Also, it would have been preferable to have additional time points, such as at 3- and 1-year follow-up. Following the clinical evaluations post-VR, some participants indicated they had experienced difficulties during the IBT and the VR. As such, it would have been of interest to have included qualitative measures and analyses. For example, participants reported problems, such as bankruptcy, sickness or death of a loved one, eviction, and loss of employment. These elements were also corroborated by the participants' psychologists. The participants as well as the psychologists identified these events as obstacles to progress in treatment. It is, therefore, important to consider these life events as they can have a considerable impact on the success of treatment.

Other elements should also be considered for future research. Indeed, it would be interesting to compare the experimental condition to a passive control condition or a wait list. Though the results were compared to a non-concurrent control group that did not receive VR, these participants were not randomly assigned to this condition. Also, as many studies include home visits, it would be of interest to have an active control condition comparing VR to home visits. In addition, it may be that five sessions of VR is insufficient to observe significant results on CH measures. As the home environments are extremely cluttered, they require a lot of work to achieve satisfactory results.

In conclusion, it seems that non-immersive VR is accessible and elicits a state of presence and immersion in participants suffering from CH. VR also elicits emotions during sorting tasks and when virtually discarding objects. Personalization of the virtual environment seems to help hoarders clean out their environment, as was the case in the bedroom in the context of this study. Only two participants dropped out of treatment, which is very little compared to the majority of studies conducted with this population. It may be that VR is less overwhelming. Finally, the participants reported that virtually sorting and discarding objects helped them to take action and experience less distress and anxiety at home. They were also generally satisfied with the IBT and VR.

#### AUTHOR CONTRIBUTIONS

The present article was part of M-E St-P-D's thesis. Dr. KO was supervising the entire research.

## REFERENCES


participants in therapeutic virtual environments derived from computer games. *Cyberpsychol Behav* (2003) 6(5):467–76. doi:10.1089/109493103769710497


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 St-Pierre-Delorme and O'Connor. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Effectiveness of *In Virtuo* Exposure and Response Prevention Treatment Using Cognitive–Behavioral Therapy for Obsessive–Compulsive Disorder: A Study Based on a Single-Case Study Protocol

#### *Mylène Laforest1 , Stéphane Bouchard1,2\*, Jessie Bossé2 and Olivier Mesly2,3*

*1University of Ottawa, Ottawa, ON, Canada, 2Université du Québec en Outaouais, Gatineau, QC, Canada, 3Université Sainte-Anne, Pointe-de-l'Église, NS, Canada*

#### *Edited by:*

*Fotios C. Papadopoulos, Uppsala University, Sweden*

#### *Reviewed by:*

*Georgios D. Floros, Aristotle University of Thessaloniki, Greece Sebastian Olbrich, University of Zurich, Switzerland*

> *\*Correspondence: Stéphane Bouchard stephane.bouchard@uqo.ca*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 19 February 2016 Accepted: 25 May 2016 Published: 13 June 2016*

#### *Citation:*

*Laforest M, Bouchard S, Bossé J and Mesly O (2016) Effectiveness of In Virtuo Exposure and Response Prevention Treatment Using Cognitive–Behavioral Therapy for Obsessive–Compulsive Disorder: A Study Based on a Single-Case Study Protocol. Front. Psychiatry 7:99. doi: 10.3389/fpsyt.2016.00099*

Obsessive–compulsive disorder (OCD) is characterized by the presence of distressing, recurrent and intrusive thoughts, impulses, or doubts as well as behavioral or mental rituals. OCD has various subtypes, including the fear of contamination in which individuals fear bacteria, germs, disease, or bodily secretions, and engage in clinically significant cleaning and avoidance rituals. Cognitive–behavioral therapy (CBT) is an effective treatment for OCD and involves, among other therapeutic strategies, exposing patients to feared stimuli while preventing them to engage in compulsive behaviors. In recent years, virtual reality (VR) has shown the potential of *in virtuo* exposure with people suffering from anxiety disorders and OCD. The objective of this pilot study is to examine the effectiveness of a CBT program where exposure in conducted *in virtuo*. Three adults suffering from OCD with a dominant subtype of contamination were enrolled in a single-case design with multiple baselines across participants. The presence and intensity of obsessions and compulsions were assessed daily during baselines of 3-, 4-, or 5-week, and a 12-session treatment. Follow-up information was gathered after 4 and 8 months. Treatment outcome is assessed with visual inspection of the graphs and ARMA time-series analyses. Clinical information, self-reports, and details of the treatment are provided for each patient. Statistical analyses for the time-series data revealed a statistically significant improvement in all three participants, but global improvement is considered positive for only two. This study innovates in proving preliminary support for the usefulness of VR in the CBT of OCD with contamination features.

#### Keywords: virtual reality, obsessive–compulsive disorder, CBT, response prevention, exposure

#### INTRODUCTION

Obsessive–compulsive disorder (OCD) is a severe and debilitating mental disorder characterized by recurrent and intrusive thoughts, impulses, or doubts that cause marked anxiety in individual, as well as behavioral or mental rituals that are performed in order to reduce distress caused by the obsessions. Ninety percent of the general population shares similar concerns as people diagnosed with OCD (1). The difference between the two populations is the importance given to the presence of thoughts; people with OCD attribute much more importance to their thoughts and their ability to control them than the general population. Despite the fact that most people suffering from OCD are aware of the irrational aspect of their thoughts, they are unable to stop the unwanted thoughts from reoccurring. Compulsions or rituals therefore develop in an effort to control thoughts and bring temporary relief from their anxiety.

In terms of obsessions, fears may be related to contamination, doubts, orderliness, religion, morality, aggression, or sexuality. When it comes to compulsions, the most common are cleaning/ washing, checking, ordering/symmetry, and accumulating. More precisely, people with a fear of contamination are absorbed by worries such as fear of microbes, bacteria, diseases, bodily fluids, and chemicals. To ease their fears, they feel the urge to wash excessively (hands, body, teeth, clothes), clean household items or personal property, and avoid situations in which there are risks of contamination. OCD symptoms and avoidance behaviors are heterogeneous and led to the development of various subtypes (2), such as patients predominantly concerned by contamination and washing, checking and verifying, or hoarding. Subtyping OCD symptoms does not necessarily imply the need to develop different theoretical models and treatments for each subtype (2, 3), but is very important in terms of clarifying the stimuli that will be targeted in psychotherapy.

Various theoretical models explaining the occurrence of OCD have been proposed, referring in particular to various factors including biological (4, 5), genetic (6), and psychological (1, 7–9). In relation to these theoretical models, most researchers agree that OCD develops and evolves because of both cognitive and behavioral factors. On the one hand, thoughts are interpreted as catastrophic. Imminent danger is associated with thoughts stemming from dysfunctional associations with threat and danger. These associations with perceived threat can be linked to an inflated sense of responsibility, over importance given to thoughts or the need to control them, overestimation of danger, intolerance of uncertainty, perfectionism, or pathological doubt (1, 10, 11).

On the other hand, behavioral factors are also present in the development of OCD since individuals try to relieve distress by accomplishing rituals or avoidance behaviors, which have shortterm effects in reducing suffering. By resorting to compulsions, people with OCD fall into an avoidance trap; that is, avoidance behaviors prevent the possibility of confronting fears in order to learn that there is no danger and that they can cope with the stimuli and tolerate discomfort. In other words, by avoiding at all costs to confront or to tolerate feared situations without resorting to compulsions, individuals remain convinced that their compulsions prevent feared consequences. Fear can thereafter extend and generalize to other objects or situations, leading to further compulsions.

Cognitive–behavioral therapy (CBT) is an effective treatment for OCD, especially due to its technique of exposure and response prevention (12, 13). This treatment involves several components (e.g., case formulation, psychoeducation, cognitive restructuring, and relapse prevention), including repeated and prolonged exposure to stimuli and situations causing distress. Exposure usually takes place gradually, in collaboration with the patient who is invited to confront moderate fears at the beginning and progressively move on to fears triggering higher levels of anxiety. During or after exposures, the participant is asked to refrain from performing rituals by tolerating anxiety, despite the strong urge to resort to compulsions. Up until now, exposure and response prevention have been conducted mostly in reality (*in vivo*) or in imagination. For example, people suffering from OCD with contamination subtype are exposed to contaminated stimuli (i.e., going into a public restroom) and then asked to tolerate anxiety without resorting to washing rituals. However, research suggests that traditional exposure may have some limitations (i.e., participants may find it difficult to imagine situations and therapists may struggle in gaging the intensity of the anxiety-provoking stimuli). Despite evidence demonstrating its effectiveness in the treatment of OCD, some clinicians are reluctant to use *in vivo* exposure because they feel less knowledgeable or lack training as well as equipment for its application (14). Finally, participants often express strong apprehension about conducting exposure in a real situation (15).

Using virtual reality (VR) in therapy is a new approach that has been introduced in order to address some inherent limitations of traditional exposure techniques involved in the treatment of anxiety disorders. VR is defined as an application that allows users to navigate and interact in real time with a three-dimensional environment generated by computers (16) and can be used to conduct exposure and response prevention. *In virtuo* exposure allows user to be exposed to anxiety-provoking stimuli (while following the same principles as *in vivo* exposure), but relies on computergenerated situations. Recent studies have demonstrated the effectiveness of VR in the treatment of various anxiety disorders (13, 17–21). VR offers several advantages for exposure (21), such as easy access to stimuli, increased control over exposure stimuli, reassuring and flexible context of exposure, bypassing problems associated with mental imagery, and make exposure more enticing to for patients.

Only a few studies have been published on VR and OCD, these having focused on the checking and verification subtype [for a review, see Ref. (22)]. Results showed that people suffering from OCD reported significantly higher anxiety levels compared with the control group (23), which suggest that VR can be used as a tool for inducing anxiety in this clinical population and consequently supports its usefulness as a medium for exposure as part of CBT. VR can also be used a behavioral measure of verification with OCD patients (24).

To the best of our knowledge, there is no treatment study focusing on the contribution of VR in CBT for OCD with the contamination subtype. Laforest and Bouchard (25) (in revision) validated the potential of immersions in VR for conducting *in virtuo* exposure with participants suffering from OCD with a predominant subtype of contamination. They compared subjective and physiological reactions of a control group of 20 non-OCD adults to a group of 12 adults suffering from OCD with contamination subtype when immersed in a "neutral" VR environment and a "contaminated" public toilet. Results showed that people suffering from OCD reported a significantly higher level of anxiety during the immersion in the contaminated virtual environment when compared with the control group on both state anxiety and heart rate. Having shown the effectiveness of VR to elicit anxiety when confronted with stimuli relevant for the treatment of OCD, the next step is to evaluate its potential as a tool used to conduct *in virtuo* exposure and response prevention.

The main objective of the current study was to examine the efficacy of a therapeutic treatment using VR for the contamination subtype of OCD. Four variables were examined: (a) the presence of obsessions, (b) the presence of compulsions, (c) the intensity of obsessions, and (d) the intensity of compulsions. The hypotheses were that presence and intensity of obsessions and compulsions would decrease following the introduction of CBT using *in virtuo* exposure.

#### MATERIALS AND METHODS

This study uses an experimental protocol with multiple baselines across participants (26–28) in order to test the effectiveness of a VR treatment in reducing OCD symptoms. In order to demonstrate the effectiveness of the intervention, participants' self-reported symptoms are expected to be "stable" (i.e., not in a decreasing trend) at baseline level and decrease when treatment was introduced. This experimental design allows for maximization of internal validity by introducing treatment at different points in time for different individuals and provides experimental control over the possible effects of maturation, historical factors, and the impact of life events (26–28) by demonstrating that each participant improves when treatment is introduced. Participants were *a priori* randomly assigned to a baseline of 3, 4, or 5 weeks of "stability" before starting treatment. They were informed of the protocol being used and how they had to self-monitor the presence and intensity of their obsessions and compulsions in a daily diary during the duration of the entire study. After the baseline period, participants received a manualized cognitive–behavioral therapy with VR exposure during a 12-week period [based on Foa and Kozak (29)]. Pre- and post-questionnaires were added to the protocol in order to obtain additional descriptive data in regard to the therapeutic progress of each participant.

#### Participants

There are no rules specifying the required number of participants in multiple baselines protocols, but several case studies have been carried out with three or four participants (7, 8, 30, 31). Participants were recruited from ads in university and local newspapers inviting men and women between the ages of 18 and 65 years suffering from OCD with a primary subtype of contamination to contact the researchers. All participants were involved in a previous validation study of the VR environment [Ref. (25), in revision].

The following exclusion criteria were set *a priori*: (a) a main diagnosis other than OCD; (b) a secondary diagnosis of schizophrenia, bipolar disorder, organic brain disorder, intellectual disability, substance abuse or dependence, or suicidal ideations; (c) a primary subtype of OCD other than one related to obsessions of contamination and cleaning compulsions; (d) the presence of a physical condition contraindicating participation in the study (i.e., epilepsy, visual disturbances); (e) a duration of OCD of <12 months. Additionally, participants who had been taking anxiolytic medication to treat their OCD had to be symptomatic and pharmacotherapy must have started at least 6 months prior in order to ensure the effectiveness of the dosage. The dosage and medication type could not change following entry in the study. Finally, participants could not undergo parallel treatments, whether pharmacological or psychological, other than the one being offered by the study.

Three adults participated in the study (*n* = 3). Following assessment, they were randomly assigned to a baseline period of either 3, 4, or 5 weeks of self-monitoring of target OCD symptoms, while symptomatology had to remain stable before beginning treatment. More details about the participants and their treatment are found in the Section "Results."

#### Procedure

A telephone interview was conducted before the first appointment in order to assess whether or not candidate's symptoms were consistent with OCD and whether the current primary obsession was contamination. Prospective participants took part in a semi-structured diagnostic interview in order to establish the presence of a diagnosis of OCD and to assess for other potential comorbid disorders as defined by the DSM-IV-TR (32). The project was approved by the Ethics Research Boards of UQO and Ottawa University. Every participant was met individually in order to complete the consent form and questionnaires. Afterward, therapy took place during a 12-week period. CBT included daily self-monitoring of obsessions and compulsions for each participant, as well as *in virtuo* exposure during therapy sessions. Two follow-ups were conducted by mail in order to determine if therapeutic gains were maintained 4 and 8 months posttreatment.

#### Material

#### Equipment

The experimentation was conducted in a CAVE-like system [i.e., similar to Cruz-Neira et al. (33), but not built by the owners of the trademark] located in the Cyberpsychology Laboratory of the Université du Québec en Outaouais. This immersive system is made up of a 10 ft × 10 ft × 10 ft with stereoscopic images projected on all surfaces: four walls, floor, and ceiling [see Bouchard et al. (34) for a detailed description of the hardware]. The experimenter operated the system from outside the closed cube.

Two virtual environments were used, a training environment ("neutral") and an experimental ("contaminated") environment. The training environment consisted of an empty room with three windows, a glass door, and a cat resting on a table. The participant could hear the calm sound of a breeze and birds singing. The purpose of this environment was to allow participants to familiarize themselves with the immersion in the CAVE-like system. The training environment was only used once, more specifically at the beginning of the fourth session. The therapeutic environment depicted a public washroom with various degrees of filthiness (see **Figure 1**) and nothing allowing to eliminate germs (i.e., cleaning products, soap, hand sanitizer).

Figure 1 | Screenshots of the virtual environment used for exposure and response prevention in a "contaminated" public toilet.

#### Instruments

#### Diagnostic Assessment

The prescreening phone interview was conducted using a questionnaire overviewing different OCD symptoms, including those related to the contamination subtype. Diagnosis was established using the Structured Clinical Interview for DSM Disorders [SCID; (35)]. This semi-structured interview was conducted by two therapists trained in administering the SCID and supervised by a licensed psychologist. The establishment of primary and comorbid diagnosis was based on clarity and causal sequence of the clinical presentation. If there was ambiguity with respect to the primary/secondary diagnosis, a consensus was reached between the interviewers and supervisor.

#### Measuring Therapeutic Progress: Self-Monitoring

The primary measure of treatment efficacy was participants' selfmonitoring of obsessions and compulsions. Participants' specific impulsive thoughts and compulsive behaviors were operationalized during the first assessment session. Daily self-monitoring based on forms using a similar design (36) were provided to participants who had to rate the presence (i.e., the proportion of the day during which the intrusive thoughts or impulses were present) and intensity (i.e., the strength of intrusive thoughts and compulsive behaviors) of their primary obsession and primary compulsion. These were recorded on a self-rating form that used category partitioning (37) according to the following scale: "none," 0; "minimal," 1–20; "few," 21–40; "average," 41–60; "a lot," 61–80, and; "extreme," 81–100. The literature on OCD suggests that percentages should be preferred over hours since, in some cases, intrusive thoughts or compulsive behaviors are numerous but short in duration, whereas in others, obsessions are less frequent but last longer (38). Participants were informed of the importance of completing the self-monitoring every night in order to increase therapy's success.

#### Symptom Assessment

In order to provide a more detailed measure of frequency, severity, and diversity of OCD symptoms, therapists administered the YBOCS [(39, 40); French translation by Mollard et al. (41)]. The YBOCS is a 10-item scale administered by therapist in the form of a semi-structured interview to measure the severity of obsessions and compulsions. Each item was rated on a five-point scale ranging from 0 to 4; 0 reflecting no symptoms and 4 reflecting extremely severe symptoms. In order to include participants with fear of contamination as their main subtype, participants had score higher on this subscale compared with other subtypes.

#### Measure of Global Functioning

The Trait-Anxiety subscale of the State-Trait-Anxiety Inventory [TAI-Y2; (42); validated in French by Gauthier and Bouchard (43)] was used to document the general level of anxiety. Daily functioning was assessed with the Evaluation of Actual Life Functioning [EALF; (44)] measuring seven different life domains, namely, (a) occupation or employment, (b) education, (c) social life, (d) hobbies, (e) entertainment, (f) holiday, and (g) everyday activities (cleaning, shopping, etc.). For each life domain, participant had to indicate on a Likert-type scale ranging from 1 (no problem) to 9 (severe difficulties) the extent to which the OCD symptoms had influenced each life domain (1 item per life domain). An average was then calculated.

#### Measures Related to Treatment Using VR

Before the first immersion, participants completed the Immersive Tendencies Questionnaire [ITQ; (45)]. This questionnaire measures participants' immersive susceptibility in a virtual environment by assessing their immersive tendency when performing other activities (i.e., reading a book, watching a movie). The Simulator Sickness Questionnaire [SSQ; (46); validated in French by Bouchard et al. (47)] was used after each immersion to measure the extent to which participants experienced side effects induced by the immersion in VR (i.e., nausea, eye fatigue, dizziness). Following a description of the CBT, participants were asked to complete the French adaptation by Borkovec and Nau (48) Client Satisfaction Questionnaire (CSQ). This measures participant's perceived credibility in regard to proposed treatment using 5 items assessed on a scale ranging from 0 to 10.

#### Measure of Therapeutic Alliance

Therapeutic alliance was measured using the French version of the Working Alliance Inventory [WAI; (49, 50)]. The WAI is a self-administered questionnaire, containing 36 items that measures participant's perspective on three components of the therapeutic alliance between participant and therapist, as defined by Bordin (51). It uses Likert-type rating scales, ranging from 1 (not true at all) to 7 (completely true) with a maximum score of 252.

#### Treatment

Cognitive–behavioral therapy unfolded according to a standardized treatment protocol using a guided manual for therapists (29). Individual weekly therapy sessions lasted 60 min. Typically, each session would begin with a discussion on previous week's homework and self-monitoring of obsessions and compulsions, followed by exposure in VR (i.e., touching walls and toilet bowls with varying degrees of filthiness) and reviewing the exposure session, performing cognitive restructuring of dysfunctional thoughts, and discussing upcoming homework assignments (52). Homework usually included a review of the didactic material viewed in session, the occasional practice of exposure (see below), and self-monitoring (29).

The first three sessions aimed at case conceptualization and introducing treatment planning. During these sessions, therapist gathered information about obsessive fears and rituals and developed an exposure hierarchy of anxiety-provoking situations. Furthermore, the cognitive–behavioral model of OCD and the rationale for exposure and response prevention were discussed. Principles of exposure in VR were also briefly discussed. Sessions 4–11 consisted of exposure and response prevention in VR (*in virtuo* exposure). The first exposure sessions were devoted to mild anxiety-provoking situations, which eventually progressed to situations causing greater distress. It is important to note that participants were systematically asked to refrain from practicing exposure outside the therapist's office during sessions 4–7 (i.e., no exposure homework). Homework assignments with *in vivo* exposure outside the therapist's office were assigned from sessions 8–11. Contextual information relevant to contamination (i.e., "You are in the public toilet of a hospital") was provided verbally before every exposure session. Following *in virtuo* exposure, therapist would draw participant's attention toward their dysfunctional thoughts. Response prevention consisted of instructing participants to refrain from any compulsive behavior. Session 12 was devoted to relapse prevention. Self-monitoring was used between sessions not only to assess outcome but also to increase awareness of situations triggering urges to ritualize.

#### Therapists

Two therapists (Ph.D. candidates) with prior experience in the treatment of anxiety disorders with CBT and with previous experience in conducting therapy in research protocols administered all therapy sessions. The therapists had more experience with traditional exposure than *in virtuo*, but had already carried out therapy using VR. They received training and hands-on experience in treating OCD using the CBT manual. Continuous supervision by the senior author, a licensed psychologist with 18 years of experience with CBT for anxiety disorders and 9 years of experience in the treatment of anxiety disorders with VR, ensured proper monitoring and standardized and uniform application of treatment.

#### Adherence to Treatment Protocol

Assessing how treatment is delivered is important in clinical trials (53, 54) and following a clear treatment plan, establishing a good therapeutic alliance, explaining treatment rationale, and setting explicit goals are expected to increase the likelihood of treatment adherence among participants. Each therapy session was recorded. Three tapes were randomly selected for each participant (25%), one from the initial third of therapy, one from the middle, and one from the end of therapy. Two trained independent experimenters blind to patient's status in the study listened to the tapes and rated them. Recordings were assessed with the help of a checklist adapted from the Competency Checklist for Cognitive Therapist (55) and the Cognitive Therapy Checklist of Therapist Competency (56, 57). Results showed that treatment protocol was respected. More specifically, 93.33% of items were *very well* respected by therapists. Only two items were rated as *fairly respected* at times during therapy (proposed exposure exercises not allowing avoidance, therapist adequately reviewed exercises completed at home).

## RESULTS

#### Overall Clinical Impressions

Daily data collected from recordings of target OCD symptoms (i.e., presence and intensity of obsessions and compulsions) are reported in **Figures 2**–**5**. The moment at which intervention was introduced is indicated with a vertical line.

The first participant is a woman in her mid-20s with a primary diagnosis of OCD with contamination subtype, and a secondary diagnosis of social anxiety disorder, generalized anxiety disorder, and posttraumatic stress disorder. She also displayed symptoms from the symmetry and order subtypes, but they were deemed of a lesser severity than that of contamination. Her Yale–Brown Obsessive–Compulsive Scale [YBOCS; (39, 40)] was in the moderate range (see **Table 1**). She reported having never received CBT for OCD before. Her main obsessions were related to the contamination of food (cross-contamination) and contamination of her hands and objects in her surroundings. The case conceptualization explored the specific factors contributing to the maintenance of her OCD. They consisted of a difficulty to be aware of direct or subtle avoidance behavior and a tendency to deny her disorder. The eight VR sessions were focused on her daily difficulties (i.e., touching a soap dispenser and the door handle of a public restroom's toilet stall). Touching the actual floor and walls of the CAVE-like system while immersed in the virtual environment was encouraged in order to increase her sense of presence. Assigned home practice for this participant focused on meal preparation as well as exposure to contaminated objects, while, at the same time, attempting to reduce or avoid rituals.

The second participant was a woman in her mid-30s with a primary diagnosis of OCD with contamination subtype and secondary diagnoses of generalized anxiety disorder as well as social anxiety disorder. On the YBOCS, Participant 2 scored within the range of severe OCD symptoms (see **Table 1**). She had been receiving treatment with a hypnotherapist for about a year. Following a discussion with the hypnotherapist, it was clarified that no therapeutic efforts had been made to treat her OCD and that it was not included in their treatment plan. During treatment, it became clear that her obsessions were mainly triggered by factors such as fear of self-contamination or fear of causing

contamination due to "not having washed properly." Exposure in the virtual environment focused on the theme of contamination within the context of a filthy public restroom described to the patient as "located in a university where students were suffering from flu and cold symptoms." The participant completed eight VR exposure sessions, followed by cognitive restructuring on exposure experience (i.e., sticky substances, contaminated air, garbage bins). OCDs maintaining factors were also addressed in therapy using cognitive restructuring technique. They included low self-esteem, depressed mood, social isolation, and a tendency to self-criticize. Assigned homework focused on themes addressed during therapy sessions that proved to be difficult to recreate using VR, such as eating without performing compulsions, preparing a meal, and response prevention of compulsions at home, which she identified as a "safe zone" and where compulsions were most significant.

The third participant was a woman in her late 20s with a primary diagnosis of OCD with contamination subtype. This participant also reported obsessive doubts and compulsions of verification, but they appeared to be secondary to the fear of contamination. Her YBOCS score placed her in the severe range of OCD symptoms (see **Table 1**). She had never received CBT for her OCD. She had also been suffering from a depressed mood in recent weeks. It should be noted that the last session lasted 120 min as sessions 11 and 12 were combined due to time constraints. Her OCD symptoms related to the fear of infecting others, especially through a sexually transmitted infection (i.e., HIV, herpes). The virtual environment depicting filthy public restrooms was presented as "being on a university campus and in a hospital." Throughout the eight exposure sessions conducted in VR, tasks involved touching unknown sticky surfaces (i.e., floors, walls cabinets, toilet bowls), garbage bins, and a used needle. Following VR exposure, cognitive restructuring was conducted based

on discussed topics, such as the possibility of contracting a sexually transmitted infection. Despite having successfully completed VR exposure, Participant 3 showed difficulty with respect to homework (i.e., using public restrooms, preparing dinner for a friend). Maintaining factors were explored and included low self-esteem, difficulty in risk-taking behaviors, high personal standards and the presence of an irritable mood in regard to her general dissatisfaction with her life (employment and relationship).

Traditional visual inspection of graphs was performed for all three participants (see **Figures 2**–**5**). Results suggest that interventions had an immediate effect on obsessions and compulsions in the case of Participant 2, as inferred by an apparent change in symptom's level*.* For Participant 1 and Participant 3, results are more difficult to interpret on the basis of visual inspection, but suggest that the effect takes place progressively during the course of the treatment. Toward the end of therapy, Participant 1 reported a decrease in OCD symptoms (YBOCS

results in **Table 1** are in the range of mild symptoms) and the ability to be exposed to anxiety-provoking situations without performing rituals. A decrease in OCD symptoms was also noted for Participant 2. She reported being able to take risks in regard to contamination and to tolerate the associated discomfort. She also mentioned she wanted to keep practicing this technique and apply exposure to objects associated with residual symptoms of OCD. At posttreatment assessment, this participants' YBOCS score was in the range of mild OCD symptoms; scores remained stable 4 months after treatment (see **Table 1**) as revealed in follow-up evaluation. As for Participant 3, the candidate reported less avoidance, was able to restructure her unrealistic thoughts and to challenge her fears, as well as give herself better self-appraisal and take greater risks at the end of therapy. Her residual symptoms resided mainly in the practice of *in vivo* behaviors (i.e., using a public restroom); a practice that now generates less anxiety compared with the beginning of treatment. Her posttreatment score on the YBOCS (see **Table 1**) scaled down to the moderate symptoms range. Scores at followups remained in the same range.


#### Table 1 | Results for measures of anxiety and daily functioning completed pre- and posttreatments as well as at the fourth and eighth month follow-ups.

*YBOCS, Yale–Brown Obsessive–Compulsive Scale; TAI-Y2, Trait-Anxiety Inventory (Form Y2 of the STAI); EALF, evaluation of actual life functioning.*

Table 2 | Results for time-series analyses performed on the presence and the intensity of obsessions and compulsions in the three participants.


*AR, autoregression model; MA, moving average model.*

*\*\*p* < *0.01, \*\*\*p* < *0.001.* 

#### Time-Series Analyses

To compensate for the subjective nature of visual examination comparing symptom's during baseline and following the introduction of the intervention, researchers can use more powerful, rigorous, and reliable statistical methods that allow description of events based on mathematical models in order to assess the impact of an intervention (27, 28, 30, 58–62). Time-series analyses [or ARMA (63, 64)] were used as the primary tool to test whether the impact of the CBT using VR was statistically significant. Time-series analyses are statistical procedure that tests the influence of an intervention on series of observations collected at regular intervals, while controlling for autocorrelation in the data. Such analyses allow to test changes in the level and slope of the series (65) after the introduction of the intervention (i.e., compared with baseline self-monitoring of presence and intensity of obsessions and compulsions). The final ARMA intervention models are reported in **Table 2**.

The ARMA time-series analyses (63, 64) confirmed our hypotheses that the impact of the intervention is statistically significant for all participants. This effect was manifested clearly immediately after the introduction of treatment in the case of presence and intensity of obsessions for Participant 2, and after a few sessions of exposure (between sessions 4 and 11) in regard to other variables and other participants. The magnitude of the impact of the intervention remains greater for Participant 1 and 2 than for Participant 3.

#### Self-Reported Measures at Pre- and Posttreatment and Follow-Ups

**Table 1** summarizes results in terms of anxiety and general daily functioning of the three participants. These data were not used to demonstrate the effectiveness of the treatment, but rather as descriptive clinical add-on to the ARMA analyses. With results obtained on the YBOCS, it is possible to observe that obsessive and compulsive symptoms showed clinical improvement during the course of treatment for all three participants. Results vary among participants at follow-ups, with Participants 2 showing lasting improvements at follow-ups and Participant 1 showing relapse on the Y-BOCS at the 8-month follow-up. Participant 3 did not show much improvement at the last follow-up.

#### Treatment and VR Experience-Related Questionnaires

Participants considered the combined VR with CBT treatment as highly credible (CSQ total score: median = 49.67 on a maximum of 50). The therapeutic alliance established between participants and therapists was strong (WAI total score: median = 237 on a maximum of 252). Participants' reported a high propensity to be immersed in a virtual environment (ITQ total score: median = 81.67 on a maximum of 126). In terms of unwanted negative side effects experienced after immersion in VR, participants reported low levels of unwanted negative side effects, with slight variations from one session to the next.

#### DISCUSSION

The objective of this study was to examine the potential efficacy of using VR to conduct exposure with people suffering from OCD with contamination fears. To this end, the presence and intensity of obsessions and compulsions were documented in a single-case study where patients are used as their own control with treatment introduced at different moments in order to evaluate if its impact follows the introduction of the intervention. After analysis, results provide preliminary evidence for the effectiveness and usefulness of VR in the treatment of OCD.

Based on visual inspection of data and time-series analyses, it is possible to believe the intervention significantly improved OCD symptoms for each participant. The intervention's effect is clearer in regard to obsessions reported by Participant 2, but this effect seems to appear later in exposure sessions for the other two participants, both for obsessions and compulsions. It is interesting to note that the "contaminated" virtual environment was more akin to Participant 2's specific obsessions and compulsions (i.e., fear of contact with germs that could then contaminate someone else), compared with Participant 1 (i.e., contamination of food) and Participant 3 (i.e., contracting a sexually transmitted infection).

At the time of assessment, YBOCS score for all participants was contained in the moderate to severe range. Following treatment, Participants 1 and 2 reported a score in the mild range of the scale, thus indicating a clear reduction of obsessive–compulsive symptoms. These gains were maintained up to 4 months (followup) after the end of treatment. As for Participant 3, her YBOCS score was reduced from the severe to moderate range following treatment and was maintained until 4-month follow-up. For all three participants, there was a slight loss in therapeutic gains as indicated by results obtained on the YBOCS at 8-month follow-up. As with OCD symptoms, trait anxiety improved for Participant 2, but this improvement was less pronounced for Participant 1 and would have increased at 8-month follow-up for Participant 3. A similar pattern is observed with participants' daily functioning.

Given these results, we are questioning the duration of the treatment used in this pilot study, as well as whether booster sessions would have been useful in the maintenance of therapeutic gains. It is possible that 12 sessions of CBT, including 8 sessions of *in virtuo* exposure, are insufficient for the treatment of a chronic and complex disorder such as OCD. Indeed, length of CBT can vary between 5 and 22 sessions of CBT (66). In addition, we cannot deny the impact of each participant's individual factors on treatment. To this effect, a study led by Keijsers et al. (67) highlighted the detrimental impact of variables such as initial severity of OCD, comorbid depression, chronicity of OCD as well as client's motivation on the prognosis of patients suffering from OCD. In the case of our study, Participant 3 had been suffering from severe OCD for 5 years, showed depressed mood and had difficulty completing exposure outside of the therapist's office. This reluctance appeared to be caused by her depressed mood, which negatively influenced her motivation, and fear of exposure homework. Fear of exposure was present but easier to manage in VR. Following these considerations, a treatment of longer duration including more VR exposure sessions would have possibly promoted better outcome and encouraged generalization of treatment gains.

In regard to patient's perception of the treatment, participants positively assessed CBT with *in virtuo* exposure. All three participants exhibited high susceptibility to immersion in a virtual environment and felt little unwanted negative side effects during exposure. A strong therapeutic alliance was created by therapists. Following these considerations, our results support the potential of VR in the treatment of OCD.

This pilot study is not without limitations. Since the study is based on an individual protocol with three participants, generalization of the results is limited. In addition, our sample was composed entirely of women. In this same perspective, differences in symptom's severity may also blur the results. The duration of the treatment may need to be adjusted and adapted to other CBT protocols [e.g., Ref. (1)]. Given the promising results of this study, it would be important to reproduce the methodology with a larger and more diverse sample (gender, age, and ethnicity) as part of a controlled study protocol with random assignment by comparing *in virtuo* treatment with traditional *in vivo* treatment. It would also be interesting to explore other subtypes of OCD (i.e., symmetry, moral and religion, verification) as well as its impact on chronicity and severity of symptoms. In this regards, VR environments are being developed for other subtypes of OCD [e.g., Ref. (68)].

Participants in this study all reported anecdotes about how they were able to expose themselves to some particular fears in VR with moderate to severe anxiety (i.e., touching the floor of a public restroom), but felt they would not have been able expose themselves to such stimuli *in vivo* at that moment in their treatment. It is possible that VR allows participants to practice exposure technique during session with the assistance of a therapist and help them be better able to implement exposure *in vivo* or as homework.

In conclusion, we wish to emphasize that VR has proven itself in the field of psychological research, but appears in its infancy in regard to its use in clinical practice. Recently, some researchers have focused their attention on the implementation mechanisms of this new technology to understand the reasons for its underutilization amongst clinicians. A survey of 262 therapists by Schwartzman et al. (69) found that clinicians are reluctant to use VR due to beliefs that using such technology requires training, equipment, and high financial costs. In addition, most therapists reported not knowing the benefits and applications of VR. Another study led by Bertrand and Bouchard (70) demonstrated that the intention to use VR is mostly predicted by the therapists' perception of its usefulness as a part of treatment. To ensure wider and more efficient use of this technology in clinical practice, the dissemination of knowledge about the various applications and the numerous benefits of VR is essential to correct erroneous beliefs held by clinicians. Schwartzman and colleagues discussed the importance of informing clinicians through conferences and scientific journals, and especially to correct myths about costs. Dissemination of information related to VR and is applications may encourage greater use in clinical settings and consequently improve access of VR for patients.

#### AUTHOR CONTRIBUTIONS

This study is part of the doctoral thesis of ML, Ph.D. Candidate. The second author was the thesis supervisor. All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

#### FUNDING

The study was funded by grants from the Canada Research Chair program awarded to the second author and was conducted at the Laboratoire de Cyberpsychologie de l'UQO (http://w3.uqo.ca/ cyberpsy).

## REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Laforest, Bouchard, Bossé and Mesly. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Using Virtual Reality in the Treatment of Gambling Disorder: The Development of a New Tool for Cognitive Behavior Therapy

*Stéphane Bouchard1,2\*, Geneviève Robillard1 , Isabelle Giroux3 , Christian Jacques3 , Claudie Loranger2 , Manon St-Pierre2 , Maxime Chrétien3 and Annie Goulet3*

*1Université du Québec en Outaouais, Gatineau, QC, Canada, 2Centre Intégré de Santé et de Services Sociaux de l'Outaouais, Gatineau, QC, Canada, 3Université Laval, Québec, QC, Canada*

Virtual reality (VR) can be used in the treatment of gambling disorder to provide emotionally

#### *Edited by:*

*Meichun Mohler-Kuo, University of Zurich, Switzerland*

#### *Reviewed by: Andreas Wilke,*

*Clarkson University, USA Colette Joy Browning, RDNS Institute, Australia Tobias Hayer, University of Bremen, Germany*

*\*Correspondence: Stéphane Bouchard stephane.bouchard@uqo.ca*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 20 September 2016 Accepted: 01 February 2017 Published: 24 February 2017*

#### *Citation:*

*Bouchard S, Robillard G, Giroux I, Jacques C, Loranger C, St-Pierre M, Chrétien M and Goulet A (2017) Using Virtual Reality in the Treatment of Gambling Disorder: The Development of a New Tool for Cognitive Behavior Therapy. Front. Psychiatry 8:27. doi: 10.3389/fpsyt.2017.00027*

charged contexts (e.g., induce cravings) where patients can practice cognitive behavior therapy (CBT) techniques in the safety of the therapist's office. This raises practical questions, such as whether the cravings are sufficient to be clinically useful but also manageable enough to remain clinically safe. Pilot data are also needed to test the development of a treatment manual and prepare large randomized control trials. This paper reports on three studies describing (a) cravings induced in VR compared to real gambling and a control game of skill with no money involved (*N* = 28 frequent gamblers and 36 infrequent gamblers); (b) the usefulness of a treatment protocol with only two CBT sessions using VR (*N* = 34 pathological gamblers); and (c) the safety of a four-session treatment program of CBT in VR (*N* = 25 pathological gamblers). Study 1 reveals that immersions in VR can elicit desire and a positive anticipation to gamble in frequent gamblers that are (a) significantly stronger than for infrequent gamblers and for playing a control game of skill and (b) as strong as for gambling on a real video lottery terminal. Study 2 documents the feasibility of integrating VR in CBT, its usefulness in identifying more high-risk situations and dysfunctional thoughts, how inducing cravings during relapse prevention exercises significantly relates to treatment outcome, and the safety of the procedure in terms of cybersickness. Results from Study 3 confirm that, compared to inducing urges to gamble in imagination, using VR does not lead to urges that are stronger, last longer, or feel more out of control. Outcome data and effect sizes are reported for both randomized control pilot trials conducted in inpatient settings. Suggestions for future research are provided, including on increasing the number of VR sessions in the treatment program.

Keywords: gambling disorder, virtual reality therapy, cognitive behavior therapy, cravings, craving behavior intervention, cognitive restructuring, side effect, safety

People who suffer from gambling disorder (GD) are characterized by the inability to resist the urge to gamble, adversely affecting all aspects of their lives including their home, social, professional, and personal life (1). Cognitive behavior therapy (CBT) has repeatedly proven effective for this disorder. It constitutes an empirically validated form of treatment recommended by experts and is considered to be among best practices (2–5). Clinically, all of the founding literature on CBT [e.g., Ref. (6–9)] strongly emphasizes the importance of mastering therapeutic tools in the comfort of the therapist's office and gradually transferring what is learned in the clinical setting to everyday situations of increasing difficulty. The literature review by Ledgerwood and Petry (10) identified this transfer of skills from the therapist's office to a real-life context as important to prevent relapses in individuals suffering from GD. The lack of correlation between the place where CBT takes place—i.e., the therapist's office—and the day-to-day reality of gamblers becomes particularly evident when it comes to the cravings and emotional responses felt by people suffering from addictive disorders when they encounter high-risk situations (3, 11).

There have been various attempts in CBT to help gamblers practice therapeutic strategies in situations of emotional arousal and cravings to gamble (3). For example, research has been done on the effectiveness of imaginal exposure (12, 13), where gamblers picture a high-risk situation in their minds so that they can then imagine themselves using psychotherapeutic strategies to deal with it. The few studies on this topic suggest that imaginal exposure helps reduce cravings in pathological gamblers (12, 14, 15). But, this technique is still limited because (a) not everyone is skilled at bringing the stimuli to life in their mind; (b) therapists have no way of knowing what, exactly, their clients are thinking about during the exercises; (c) the therapeutic exercises performed by clients increase their cognitive load, causing a corresponding decline in their ability to fully imagine themselves in that situation; and (d) therapists sometimes have trouble getting gamblers to put their dysfunctional thoughts into words.

Furthermore, inciting an urge to gamble by thinking of a past situation is still very limited when comparing an imaginary situation to the omnipresence and abundance of indicators capable of triggering a craving in the everyday lives of people who suffer from GD. There are many factors that contribute to high-risk situations for gamblers (16), such as images and logos associated with gambling, being in the presence of video lottery terminals (VLTs) in public places, feeling strong physiological and affective responses, seeing others gamble and win or give up their spot because they lost, being in the relaxed atmosphere of a bar, or the glamorous surroundings of a casino, etc.

Practising CBT in virtual reality (VR) offers a promising alternative (11, 17–20). In the safety of the therapist's office, gamblers can don 3D glasses and be faced with VLTs or visit a casino. The therapist can then conduct various classic CBT interventions [see, for example, Ref. (21)], gradually bringing gamblers into situations that will trigger their urge to gamble. As described in Study 2 and 3 later in this article, therapists can use VR to identify situations, thoughts, and behaviors associated with gambling; conduct cognitive restructuring with dysfunctional beliefs underlying GD; or work on relapse prevention (18). Using VR in combination with traditional CBT has proven effective in a few studies, but these studies are all based on the therapeutic rationale of cue exposure [i.e., habituation leading to extinction of the conditioned response (19, 20)] and not on the goal of inducing emotions and cravings to practice CBT techniques (18, 22). Also, because the latter paradigm is not based on extinction, it raises the very important question of whether the cravings induced by VR are too strong to be used safely, especially in outpatient settings where people can go gamble after the therapy session.

The goal of this paper is to document the potential of VR in the CBT of GD with three consecutive studies that are as follows: (a) an experimental demonstration that VR immersions can induce cravings in GD patients (Study 1); (b) a pilot study documenting the potential of a minimal use of VR in CBT for GD (Study 2); and (c) a second pilot trial to gage the safety, in terms of the intensity of cravings, of a slightly more intensive use of VR in CBT for GD (Study 3). Due to ethical considerations regarding the induction of cravings in patients, the first study was conducted with a subclinical sample, the latter two studies were conducted in inpatient settings, and the number of sessions using VR progressively increased from only two in Study 2 to only four in Study 3. All three studies were approved by UQO's review board of ethical conduct in research, and every participant signed an informed consent form in accordance with Canadian standards of ethical conduct for research involving human participants.

#### STUDY 1

#### Participants

To test whether the virtual environments developed for GD can induce an urge to gamble, adults between the ages of 18 and 65 familiar with VLTs were recruited. The sample included 28 "frequent" players (play VLTs at least once a month) and 36 "occasional" recreational players (play no more than twice a year). Occasional recreational gamblers were recruited as a control group with enough minimal experience with VLTs to know what they are and how to use them. Occasional recreational players were excluded if they scored higher than 1 on the South Oaks Gambling Screen (SOGS). Frequent players were excluded if they scored 9 or more for ethical reasons due to concern about inducing cravings in people suffering from GD [9 is a score clearly within the range of potential GD but below the severity of the majority of those diagnosed with GD (23, 24), p. 11]. Hypersensitivity to cybersickness, defined as a self-reported history of severe motion sickness when calling potential study participants, was also an exclusion criterion. Two additional exclusion criteria were set *a priori* and tested in the lab once the consent form was filled out, having poor stereoscopic vision or being intoxicated during the experiment, but no participant was excluded based on these criteria.

#### Method

For the experiment, participants signed an informed consent form and were invited to play each of the following games for 7 min: (a) Scrabble™ (control condition), (b) a real VLT with participants gambling \$20, (c) a virtual bar with VLTs called *At Fortunes*, and (d) a virtual casino called *The 3Dice* (see **Figures 1**–**3** describing the experimental setup and the virtual environments). The Gambling Craving Scale (GCS) (25) was administered after each game, and each of its subscales will be examined separately: the Anticipation that gambling will be fun (Cronbach's alpha = 0.84), the urgent Desire to gamble (Cronbach's alpha = 0.81), and the

Figure 1 | Experimental setup for Study 1 on the potential to induce cravings with a real video lottery terminal and virtual reality.

expectation that gambling will provide relief from negative affect (Cronbach's alpha = 0.85). For a detailed description of the virtual environments and the VR technology used, see Bouchard et al. (18). The sequence of participation in each condition was randomly distributed. The immersions in VR were conducted using an nVisor SX head-mounted display and a CUBE2 motion tracker.

The data were analyzed with two conditions (frequent gamblers experimental condition vs. occasional gamblers control condition) by four repeated measures (Scrabble™—game of skill control condition, real VLT control condition, immersion in the *At Fortunes* virtual bar with VLTs experimental condition, immersion in the *The 3Dice* virtual casino experimental condition) ANOVAs conducted separately with each subscale of the GCS. A Bonferroni correction was applied to control for inflation of the error rate due to multiple comparisons

Figure 2 | Illustrations (screenshots) of the *At Fortunes* virtual environment used in all three studies. Reproduced from Bouchard et al. (18) under the Creative Commons copyright licence.

Figure 3 | Illustrations (screenshots) of *The 3Dice* virtual environment used in all three studies. Reproduced from Bouchard et al. (18) under the Creative Commons copyright licence.

(i.e., alpha set at 0.016). Statistically significant interactions were followed up by *a priori* planned repeated contrasts with the Scrabble™ game of skill control condition, and effect sizes measured with partial eta-squared are reported. The correlation between severity of GD and urge to gamble after immersions in both virtual environments was explored with Pearson's correlations.

#### Results

Descriptive statistics on the impact of the manipulations on the GCS subscales are reported in **Table 1**. The results of the repeated measures ANOVAs revealed statistically significant main effects on the Anticipation of fun subscale of the GCS [Time *F*(3,186) = 0.5, ns; Condition *F*(1,62) = 10.87, *p* < 0.001; Interaction *F*(3,186) = 11.53, *p* < 0.001] and the Desire to gamble subscale of the GCS [Time *F*(3,186) = 2.35, ns; Condition *F*(1,62) = 5.51, *p* < 0.01; Interaction *F*(3,186) = 4.97, *p* < 0.01]. No significant main effect was found on the Relief from negative affect subscale of the GCS (all *F* < 2.9, ns). The significant repeated measures ANOVAs were followed up by interaction contrasts comparing each exposure to a gambling situation (i.e., real VLT, VR with *At Fortunes*, and VR with *The 3Dice*) with scores following the control condition (i.e., playing Scrabble™). For the Anticipation subscale, all three interaction contrasts were statistically significant [control vs. real VLT *F*(1,62) = 16.31, *p* < 0.001, partial eta-squared = 0.21; control vs. VR with *At Fortunes F*(1,62) = 11.48, *p* < 0.01, partial eta-squared = 0.16; and control vs. VR with *The 3Dice F*(1,62) = 18.3, *p* < 0.001, partial eta-squared = 0.23]. A similar pattern was observed for the Desire subscale [control vs. real VLT *F*(1,62) = 10.97, *p* < 0.001, partial eta-squared = 0.15; control vs. VR with *At Fortunes F*(1,62) = 5.06, *p* < 0.05, partial eta-squared = 0.08; and control vs. VR with *The 3Dice F*(1,62) = 9.96, *p* < 0.01, partial etasquared = 0.14]. In short, the results suggest that, for "frequent" gamblers, gambling on a real VLT or in VR is associated with a significant increase in anticipation and desire to gamble, which were significantly higher than in very occasional gamblers. In addition, urges to gamble measured with the total score of the GCS post immersion in VR were significantly correlated with the severity of gambling addiction as measured with the SOGS (*r* = 0.49, *p* < 0.001 in *At Fortunes* and *r* = 0.49, *p* < 0.001 in *The 3Dice* for the Anticipation subscale; *r* = 0.47, *p* < 0.001 in *At Fortunes* and *r* = 0.49, *p* < 0.001 in *The 3Dice* for the Desire subscale; and *r* = 0.63, *p* < 0.001 in *At Fortunes* and *r* = 0.38, *p* < 0.001 in *The 3Dice* for the Relief subscale).

In summary, the results from Study 1 show that VR can be used to induce cravings in gamblers. Following immersions in VR, the magnitude of the increase on two subscales out of three on the measure assessing the urges to gamble corresponds to large effect sizes that are essentially in the same range as playing on a real VLT. The strong correlations between urges to gamble and the SOGS suggest that results might be generalizable to more severe gamblers, although this remains to be tested empirically. The next step is to test a preliminary treatment protocol with minimal involvement of VR and documents its safety with people suffering from GD.

#### STUDY 2

#### Participants

An initial pilot study was conducted to document the potential clinical usefulness of two VR immersions in the treatment of GD. The sample comprised 34 participants suffering from pathological gambling as defined in the DSM-IV-TR (26) and registered at one of two inpatient treatment centers: the CASA Centre (Saint-Augustin-de-Desmaures, QC, Canada) and La Maison L'Odyssée (Sainte-Marie-de-Beauce, QC, Canada). The sociodemographic characteristics of the study's participants were as follows: 35% women, 87% Canadian, and 14% Natives Americans, median age of 45 (SD = 12.6), average SOGS score of 11.5 (SD = 4.6), average number of days per month spent gambling estimated by the participants at 12.6 (SD = 6.8), number of uncontrollable gambling episodes per month estimated by participants at 9.79 (SD = 8.65), and average amount of money (in Canadian dollars) sunk into gambling per month was estimated by gamblers at \$3,710 (SD = \$4,963).

Following a random assignment, 14 participants received a traditional 28-day cognitive behavioral treatment program with 2 imaginal exposure exercises (imagination condition), and 20 received the same treatment program but with the 2 exposure exercises conducted using VR immersion (VR condition). A few minimal exclusion criteria were applied during the recruitment of participants: suffering from GD but not associated with VLT or casino slot machines, being a minor, suffering from health issues that could be exacerbated by treatment using VR (major cardiac disorder, severe and frequent motion sickness when traveling by car, vestibular or inner ear disorders, recurring migraines, epilepsy, balance or ocular disease) and suffering from a potentially contraindicated mental health issue (schizophrenia, mental retardation, etc.).

Table 1 | Mean (and SDs) Gambling Craving Scale (GCS) after playing a control game (Scrabble**™**) or gambling on a real video lottery terminal (VLT) or in two virtual environments in Study 1.


#### Method

The two treatment centers taking part in the project have been applying the assessment and treatment program for excessive gamblers developed by Ladouceur et al. (27, 28). When this program is administered in a 28-day inpatient setting, the very first session is devoted to identifying situations that increase the patient's risk of gambling and the dysfunctional beliefs that support the maintenance of gambling issues. Another session, in the last week of therapy, focuses on practising relapse prevention skills. During both of these sessions, gamblers are asked to imagine gambling situations that trigger cravings and relive these experiences for about 20 min. The intervention that took place during the two gambling exposure sessions (imagination and VR conditions) was recorded in an audio file, and a random selection of 20 of the 59 recordings available (9 sessions were not recorded due to technical problems) were reviewed to confirm that the interventions were conducted as planned. The immersions in VR were performed using a Vuzix iWear VR920 and a CUBE2 motion tracker. Before and after the first gambling exposure, the participants were asked to list all of their personal high-risk thoughts and situations. Before and after the second gambling exposure session, participants evaluated the intensity of their desire to gamble on a scale of 0 to 10 and filled out the Simulator Sickness Questionnaire (SSQ) (29, 30). Before and after the treatment program, the participants completed the GCS (25). A few months after the end of the data collection with the participants, focus group-type interviews lasting 1 h 45 min were conducted with the four therapists who carried out the interventions at each center during the data collection (all females staff members of the centers, with a bachelor's degree in psychology or social sciences, several years of experience with GD, and 2 h of training on the use VR). The goal of the interview was to get their impressions on the VR immersion and the clinical issues they observed.

The usefulness of VR was documented through (a) a detailed review of the content of the sessions and comparison with a Student's *t*-test of the number of dysfunctional thoughts and highrisk situations identified during the session, (b) a description of the impact of VR immersions on cybersickness using descriptive statistics and a comparison from pre to post immersion using a non-parametric test for comparing means (a Wilcoxon *Z* was used because SSQ scores were not normally distributed), and (c) preliminary data on the impact of treatment using VR were analyzed with 2-condition (VR experimental condition vs. imagination control condition) and 2-time (pre and post-treatment) repeated measures ANOVAs and a multiple regression using residualized change scores.

#### Results

A review of the audio recordings revealed that therapists were more inclined to ask patients to express their thoughts and emotions out loud in VR (93 occurrences in the course of treatment) than in imagination (46 occurrences, Chi-square = 15.89, *p* < 0.001). As illustrated in **Table 2**, VR immersion helps therapists identify more high-risk situations than imaginal exercises. VR also helps to identify twice as many dysfunctional thoughts, but this difference is not statistically significant. Note, however, that the effect size is medium, and that a sample of about 80 participants would provide a 0.80 power to detect a significant difference.

The number of negative unwanted side effects induced by the immersion in VR in the VR condition was measured before and after the second VR therapy session. The results presented in **Table 3** show that the immersion did not lead to an increase in intensity of cybersickness symptoms compared to what was recorded before the immersion (Wilcoxon *Z* test = 0.57, 0.30, and.0, respectively, all ns).

As preliminary data on the effectiveness of the program with a minimal use of VR, a repeated measures ANOVA was conducted for the total GCS score. Results show a large reduction in cravings in participants in the VR condition, from a mean of 28.00 (SD = 16.9) to a mean of 12.69 (SD = 6.66), and a similar reduction in the control imagination condition, from a mean of 23.62 (SD = 14.78) to a mean of 10.88 (SD = 2.75). The reduction was significant for both conditions, with no differences in terms of treatment modality [Time *F*(1,19) = 14.23, *p* < 0.001; Condition *F*(1,19) = 0.61, ns; Interaction *F*(1,19) = 0.12, ns]. The effect size of the interaction, as assessed with the partial eta-squared, was 0.006. A regression using residualized change scores was also conducted to document the relationship between the intensity of the cravings induced during the relapse prevention session and pre-to-posttreatment improvements on the total score of the GCS. The

Table 3 | Unwanted negative side effects induced by the immersion in virtual reality (VR) (i.e., cybersickness) as measured by the Simulator Sickness Questionnaire (SSQ) in Study 2 before and after the session devoted to relapse prevention.




*\*p* < *0.025.*

regression equation was statistically significant [*F*(2,20) = 9.01, *p* < 0.01, Adj *R*<sup>2</sup> = 0.46], with the increase in cravings during the relapse prevention session being significantly related to more improvement at the end of the program (*t* = 4.19, *p* < 0.001, sr2 = 0.48).

The focus group confirmed that therapists were satisfied with the use of VR. No adverse event was reported on the evenings following the therapy sessions where VR was used. **Table 4** lists the advantages of using VR as reported during the focus groups. The therapists also made several suggestions that contributed to improving the treatment program and the development of two additional modules where VR could be used for cognitive restructuring (18).

Overall, the results of Study 2 revealed that therapists can use VR in a clinical setting and that this technology could be useful in eliciting clinically relevant information about patients' thoughts, behaviors, and high-risk situations. The second session where VR was used revealed that it was not associated with significant cybersickness and that the induction of cravings during the relapse prevention exercise was related to the treatment outcome. Finally, dedicating two sessions to VR instead of following the standard treatment program was not associated with a reduction in treatment effectiveness (see the general discussion for more on VR vs. the standard procedure). VR can therefore be used more intensively in the CBT of GD. However, concerns about the safety of inducing cravings in sessions can be addressed much further.

#### STUDY 3

#### Participants

Study 3 primarily aims to document the safety, in terms of intensity of gambling cravings post-session, of applying VR to CBT, and to provide pilot data on increasing the use of VR to four sessions. A sample of 25 adults with a primary diagnosis of GD according to the DSM-5 criteria (1) was recruited following a semi-structured telephone interview conducted by mental health and GD care professionals. The control condition comprised 11 participants; 14 participants were in the experimental condition. Participants were recruited and treated at two GD treatment centers: Centre CASA (St-Augustin-de-Desmaures, QC, Canada) and Maison Jean-Lapointe (Montreal, QC, Canada). The two centers

Table 4 | Overview of the therapists' opinions in Study 2 regarding the advantages of using virtual reality immersions.


taking part in the project applied the Evaluation and Treatment Program for Excessive Gamblers by Ladouceur et al. (27, 28) with group therapy sessions combined with four individual sessions (i.e., the four targeted for applying VR) in their 28-day inpatient treatment program. The sociodemographic characteristics of the participants were as follows: 50% women, 100% Canadians, mean age of 47 (SD = 12.8), average Canadian Problem Gambling Index (CPGI) score of 19.96 (SD = 3.7), average number of diagnostic GD symptoms encountered during the interview of 7.44 (SD = 1.6), average number of hours per week spent gambling estimated by the participants at 19.9 (SD = 16.95), and weekly average amount lost gambling estimated by gamblers at \$1,131 (SD = \$1,190).

#### Method

As in Study 2, the participants received inpatient treatment from the therapists, but this time four CBT sessions were dedicated to VR immersion [for a detailed treatment manual, see Ref. (18)]. The participants were randomly assigned to the VR stimuli (VR-S) condition or to a control imagination stimuli (Imag-S) condition. To balance out the potential effects of being immersed in VR (e.g., cybersickness), the four treatment sessions for the control Imag-S condition were conducted in VR, but the content of the virtual environment was not associated in any way with gambling or with the induction of gambling cravings. Instead of using the *At Fortunes* bar or *The 3Dice* casino, participants in the control Imag-S condition were immersed in an environment representing an empty room, with no cues associated with gambling or money. Once immersed in this environment, participants were invited to imagine themselves in a high-risk situation and apply the CBT techniques.

The VR sessions relied on the same equipment as that the one used in Study 2 and were dedicated to the identification of high-risk situations, cognitive restructuring, and relapse prevention. Audio recordings of the therapy sessions were played back to confirm that the instructions in the treatment manual were followed. All participants were immersed in VR, with 100% of the participants in the VR-S condition being exposed to virtual gambling cues only, and 100% of the participants in the Imag-S condition being exposed to imaginal stimuli only. The integrity scores regarding compliance with the treatment manual (86.7% for the VR-S condition and 82.5% for the Imag-S condition) confirm that the therapists carried out the interventions as planned. The compliance scores for respecting the clinical objectives of each session were even higher (91.9% for the VR-S condition and 96.9% for the Imag-S condition), which is excellent.

To document the safety of immerging people suffering from GD in VR to elicit cravings over four sessions, participants filled out the My Treatment, an in-house questionnaire administered immediately after the session, and then 12, 24, and 36 h postsession. Five questions were asked (see **Table 5**), including two of particular interest to us since they measure the intensity of cravings: (a) "In terms of percentage, how much did you feel an urge to gamble?" and (b) "How often did you feel an urge to gamble?" The second goal of Study 3 was to provide pilot data on the impact of a program comprising four CBT sessions conducted as part of VR immersions on three indices of effectiveness,



*VR-S, experimental condition where the urge is induced by stimuli recreated in virtual reality; Imag-S, control condition where the urge is induced by stimuli evoked in imagination.*

administered before and 2 weeks after the treatment program: (a) the Canadian Problem Gambling Index (CPGI) (31, 32), (b) the number of GD diagnostic criteria according to the Diagnostic Interview for Gambling (DIG) (27, 28), and (c) the Gambling Related Cognitions Scale (GRCS) (33).

A descriptive approach was adopted for the data documenting side effects, accompanied by Student's *t*-tests and two conditions (VR with virtual stimuli associated with gambling experimental condition and VR with imaginal stimuli associated with gambling control condition) by four times repeated measures ANOVAs. Data were analyzed with the goal of documenting the safety of using VR in CBT for GD, and therefore no correction was applied to control the potential inflation of error rate associated with multiple comparisons for assessing side effects. To provide pilot data on the efficacy of replacing standard CBT exercises for GD with the exercise performed in (only) four immersions in VR, 2-condition by 2-time (before treatment and 2 weeks after treatment) repeated measures ANOVAs were conducted using three outcome measures. Effect sizes are reported, Bonferroni corrections applied (i.e., alpha set at 0.016) and intent-to-treat data were used to remain conservative.

#### Results

The issue of post-session intensity of the desire to gamble induced in VR was studied through a review of the questionnaires filled out by the participants after each of the four therapy sessions in which therapists induced the urge to gamble through either VR or imaginal exercises (**Tables 5**–**7**). *T* tests were performed to compare the two conditions based on all of the variables measured immediately following the four therapy sessions. The only statistically significant difference (*t* = 2.29, *p* < 0.05) regarded the impression that the treatment provided helped better control gambling issues, as completed following the third session (i.e., the last cognitive restructuring session) for participants in the VR-S condition. No other comparison1 on the data reported in **Table 5** even came close to the significance threshold. This shows that VR does not induce an urge to gamble that persists post-session longer or more strongly than after imaginal therapy; that the urge remains low; and that the impression of being able to control the urge remains high.

**Tables 6** and **7** show how the urge to gamble becomes more or less intense in the hours following the four sessions. A repeated measures ANOVA reveals a decline in the evaluated percentage in the first session (**Table 6**) over time [*F*(3,75) = 4.01, *p* < 0.025], no difference between conditions [*F*(1,25) = 0.08, ns], and no significant interaction [*F*(3,75) = 0.48, ns]. Once again we see a significant drop over time during the second [*F*(3,69) = 2.86, *p* < 0.05] and third [*F*(3,63) = 6.11, *p* < 0.001] therapy sessions. The conditions and interactions effects are not significant. It is of interest to note that the urge to gamble in the hours and days following the sessions levels off without ever dropping down to zero, most likely pointing to the everyday degree of desire in the gamblers being treated.

As regards frequency of the urge to gamble (**Table 7**) after the sessions, none of the ANOVAs revealed a significant effect except for the time effect following the third therapy session [*F*(3,63) = 2.91, *p* < 0.025]. Overall, the low frequency of episodes of gambling cravings might be indicative of a low urge to gamble following the sessions but likely also reflects a difficulty retrospectively isolating multiple distinct episodes. As such, someone who would constantly feel the urge to gamble throughout the day may report just one episode or at most a few distinct episodes if attention was distracted away from a constant craving. A percentage assessment of the entire day is probably more accurate than the frequency method.

Repeated measures ANOVAs were performed for the three effectiveness measurements (see **Table 8**). The results show large effect sizes (partial eta-squared of.46, 0.9, and 0.85, respectively) and statistically significant improvement in the three measures. Analyses revealed no significant difference with regard to the time × condition interactions, with effect sizes ranging from small (0.001) for the CPGI, medium for the number of diagnostic criteria encountered with the DIG (0.07), and dysfunctional beliefs as measured using the GRCS (0.04). We can thus estimate that with 0.80 power, some 60 participants per condition would be needed for these interactions to be statistically significant. Using a cut-off score of 7 or less for the CPGI post-treatment (i.e., the cut-off score for GD), we get a 50% success rate for VR-S and a 45.5% success rate for the Imag-S control condition. Using a cut-off score of 4 or less for the DIG (i.e., the number of diagnostic criteria required to receive a DG diagnostics), the success rate is 56% for the VR-S condition and 44% for the Imag-S control condition. These differences are not statistically

<sup>1</sup>Results not reported but available upon request.


#### Table 6 | Mean (and SD) of the urge to gamble measured in percentage by My Treatment Questionnaire after therapy sessions and after 12, 24, and 36 h in Study 3.

*VR-S, experimental condition where the urge is induced by stimuli recreated in virtual reality; Imag-S, control condition where the urge is induced by stimuli evoked in imagination.*

Table 7 | Mean (and SD) of the urge to gamble measured in frequency by My Treatment Questionnaire after therapy sessions and after 12, 24, and 36 h in Study 3.


*VR-S, experimental condition where the urge is induced by stimuli recreated in virtual reality; Imag-S, control condition where the urge is induced by stimuli evoked in imagination.*



*Intent-to-treat data, \*\*\*p* < *0.001.*

*VR-S, experimental condition where the urge is induced by stimuli recreated in virtual reality; Imag-S, control condition where the urge is induced by stimuli evoked in imagination; CPGI, Canadian Problem Gambling Index; DIG, Diagnostic Interview of Gambling (number of diagnostic criteria met by the participant); GRCS, Gambling Related Cognitions Scale.*

significant. Such preliminary results accordingly point not only to the success of the VR-S program but also underscore the need to pursue more research to increase its short-term effectiveness on GD symptoms.

Essentially, the results of Study 3 illustrate three phenomena: (a) there is no difference in lasting effects on the urge to gamble between the sessions where VR-S were applied and imaginal stimuli were used; (b) the post-session urge to gamble is comparable across all conditions and hovers around 20% immediately following the therapy sessions for participants in the VR-S and the Imag-S conditions; and (c) the use of only four VR sessions in CBT for GD can lead to success rates between 50 and 56% and medium effect sizes.

#### DISCUSSION

This series of three studies helps determine the potential and safety of VR for the treatment of GD. The studies show that it is possible to induce a significant urge to gamble, as strong as that observed using a real VLT. A gradual progression in the number of sessions while keeping a watchful eye on potential adverse effects sets the stage for a more intensive use of VR with GD. This technology was well accepted and used by our therapists, helping them work with patients' who are more emotionally aroused during therapy sessions. It also helps identify elements that are useful for the therapy, namely high-risk situations. Cravings are induced at levels that are easily manageable by therapists during the sessions and that are not a cause for concern post-session. That does not mean that post-session cravings can be ignored completely, but they are certainly not overly worrisome. Of course, it is up to each therapist to be well informed and know how to handle these cravings with his or her patients. The unwanted negative side effects induced by immersions in VR seem minor. The project provides all of the information necessary to initiate a large-scale clinical trial and increase the number of sessions incorporating VR immersions in CBT strategies to more than four sessions.

The results essentially open the way for a new approach to the treatment of GD. This technology opens the door to powerful new prevention and treatment tools for therapists that will also appeal to gamblers. Yet, therapists and case workers may still question the potential of VR to induce cravings because gamblers make no actual monetary gains when gambling in VR. The demonstration in this paper that an immersion in VR can stimulate the urge to gamble in gamblers builds on the work done by Kushner et al. (34) and Wulfert et al. (35), which showed that gambling urges could be induced by stimuli in a laboratory, and research from Young et al. (36) demonstrating that gambling urges could be influenced, even in VR.

Through a number of pilot cases, research studies from Garcia-Palacios et al. (22), Giroux et al. (19), and Park et al. (20) point to VR's potential as a clinical tool. The data obtained from the current project provide a solid empirical basis justifying a number of new research projects as well as large-scale randomized control clinical trials, such as (a) comparing the effectiveness of immersive and non-immersive versions (i.e., using only the computer screen rather than a head-mounted display); (b) distinguishing cybersickness symptoms from the physical signs of gambling cravings and withdrawal; (c) examining the role played by the sense of presence in VR immersions; (d) using virtual stimuli associated with other addictions (e.g., presence of alcohol in the virtual environment) to see their impact on GD therapy; (e) clarifying the impact of environmental factors (e.g., bank machine) on the urge to gamble; (f) using VR with other forms of psychotherapy, including mindfulness; (g) evaluating how using VR can boost the patient's motivation in therapy; and (h) conducting research on potential VR addiction, an as-yet non-existent phenomenon but clearly one that should be monitored closely.

The fact that VR was not more effective than the control condition raises the question of the relevance of using this technology. There are several elements to consider in this regard. First, the success rates of using VR in CBT were far from inferior to standard CBT, a finding that mimics what was found in the first studies on using VR in the treatment of anxiety disorders and led to a now-flourishing field of useful clinical applications [for a review, see Ref. (37)]. Indeed, it took several trials with people suffering from specific phobia showing that VR was not more or less effective than *in vivo* exposure to develop treatment protocols that fully exploit the potential of VR and make it more effective. Second, CBT was applied using virtual craving stimuli in only very few sessions. Dedicating only two sessions to cognitive restructuring and one to relapse prevention is likely insufficient to exploit the full potential of VR. Now that safety seems sufficiently documented, as many sessions of VR as possible should be integrated in the treatment protocol to really tests if VR can be more effective than the standard procedure. Third, the interventions were integrated in routine 28-day inpatient programs, with much less control in terms of content than what can be found in randomized control

#### REFERENCES


trials. It is difficult to isolate in the two pilot trials presented in this article the contribution of each specific intervention. Also, strong follow-up data are required to fully comment on the efficacy of the interventions. Finally, the advantages of VR should not be examined only in terms of the reduction of symptoms but also in terms of motivation to attract and retain patients in treatment programs, effort by the therapists, and therapists' motivation to actually use exposure to gambling cues (either in imagination or in real-life settings).

Until now, given the purchase cost of the equipment involved, only well-funded research centers had access to VR. But with the advent of a number of hardware companies aiming for the mass market, such as Oculus™ (owned by Facebook), Vive™ (owned by HTC), GearVR™ (owned by Samsung), and Google Cardboard™ (owned by Google), just to name a few, VR will quickly become a mass market product, and applications are now available to gamble real money while immersed in virtual casinos. The advantage with VR applications developed for clinicians over those developed for the gaming industry is that people with GD cannot use them outside the therapist's office to fuel their addiction. The arrival of VR brings with it all of its advantages, namely as regards to availability of new psychotherapeutic tools, and its disadvantages, such as the risk of addiction to various non-therapeutic applications that make it possible to escape the challenges of day-to-day life or gamble from the comfort of the patient's own home.

#### AUTHOR CONTRIBUTIONS

All authors listed have made substantial, direct, and intellectual contribution to the work and approved it for publication.

#### ACKNOWLEDGMENTS

The authors wish to thank collaborators who significantly contributed to the development of this research: Julie Boulanger, Laurent Brisson, Lynda Poirier, Andrée-Chantale Charrette, Mireille Beaulieu, Rachel Fraser, Lucie Laniel, Kathy Tétreault, Isabelle Cyr, Geneviève Morency, Marie-Claude Gagné, Josée Dostie, Claude Boutin, Louis-Philippe Bertrand, Isabelle Provencher, Kathy Ducharme, Jennifer Mercure, Ginette Doucet, and Serge Sévigny. The project received funding from the Fondation Mise sur Toi, the Fondation de l'Université du Québec en Outaouais, the Fonds de Recherche du Québec Société Culture, and the Canada Research Chairs program. Clinical trial registration ISRCTN77484912.


**Conflict of Interest Statement:** SB and GR are consultants to and own equity in Cliniques et Développement In Virtuo, which develops virtual environments. The remaining authors report no financial relationship with commercial interests.

*Copyright © 2017 Bouchard, Robillard, Giroux, Jacques, Loranger, St-Pierre, Chrétien and Goulet. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Transforming Experience: The Potential of Augmented Reality and Virtual Reality for Enhancing Personal and Clinical Change

#### *Giuseppe Riva1,2\*, Rosa M. Baños3,4,5, Cristina Botella4,5,6, Fabrizia Mantovani7 and Andrea Gaggioli1,2*

*1Applied Technology for Neuro-Psychology Laboratory, Istituto Auxologico Italiano, Milan, Italy, 2Centro Studi e Ricerche di Psicologia della Comunicazione, Università Cattolica del Sacro Cuore, Milano, Italy, 3Universidad de Valencia, Valencia, Spain, 4 Instituto Salud Carlos III, Ciber Fisiopatología Obesidad y Nutrición (CB06/03), Madrid, Spain, 5Red de Excelencia PROMOSAM, Mineco, Spain, 6Universitat Jaume I, Castelló de la Plana, Spain, 7Università degli Studi Milano Bicocca, Dipartimento di Scienze Umane per la Formazione "Riccardo Massa", Milan, Italy*

#### *Edited by:*

*François Borgeat, Université de Montréal, Canada*

#### *Reviewed by:*

*Victor Lasebikan, Univeristy of Ibadan, Nigeria Lucia Romo, Paris West University Nanterre La Défense, France*

> *\*Correspondence: Giuseppe Riva giuseppe.riva@unicatt.it*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 27 July 2016 Accepted: 15 September 2016 Published: 30 September 2016*

#### *Citation:*

*Riva G, Baños RM, Botella C, Mantovani F and Gaggioli A (2016) Transforming Experience: The Potential of Augmented Reality and Virtual Reality for Enhancing Personal and Clinical Change. Front. Psychiatry 7:164. doi: 10.3389/fpsyt.2016.00164*

During life, many personal changes occur. These include changing house, school, work, and even friends and partners. However, the daily experience shows clearly that, in some situations, subjects are unable to change even if they want to. The recent advances in psychology and neuroscience are now providing a better view of personal change, the change affecting our assumptive world: (a) the focus of personal change is reducing the distance between self and reality (conflict); (b) this reduction is achieved through (1) an intense focus on the particular experience creating the conflict or (2) an internal or external reorganization of this experience; (c) personal change requires a progression through a series of different stages that however happen in discontinuous and non-linear ways; and (d) clinical psychology is often used to facilitate personal change when subjects are unable to move forward. Starting from these premises, the aim of this paper is to review the potential of virtuality for enhancing the processes of personal and clinical change. First, the paper focuses on the two leading virtual technologies – augmented reality (AR) and virtual reality (VR) – exploring their current uses in behavioral health and the outcomes of the 28 available systematic reviews and meta-analyses. Then the paper discusses the added value provided by VR and AR in transforming our external experience by focusing on the high level of personal efficacy and self-reflectiveness generated by their sense of presence and emotional engagement. Finally, it outlines the potential future use of virtuality for transforming our inner experience by structuring, altering, and/ or replacing our bodily self-consciousness. The final outcome may be a new generation of transformative experiences that provide knowledge that is epistemically inaccessible to the individual until he or she has that experience, while at the same time transforming the individual's worldview.

Keywords: virtual reality, augmented reality, personal change, anxiety disorders, eating disorders, acute pain, post-traumatic stress disorder, body swapping

#### INTRODUCTION: UNDERSTANDING PERSONAL CHANGE

How can we use technologies like virtual reality or augmented reality to support personal and clinical change? A meaningful answer to this question requires an in-depth examination of the process of change. During our life, we undergo many personal changes: we change our house, our school, our work, and even our friends and partners. However, in this paper, we will focus on a peculiar type of change – personal change – whose main effect is a change in the conceptual system of the subject (assumptive world), derived from perceptions of one's own behavior or experience, or other incoming information (1). Personal change plays an adaptive role in managing symptoms of distress produced by life transitions and traumatic events (1, 2). Moreover, high levels of personal change are associated with psychological well-being (2).

Our daily experience shows clearly that, in some situations, subjects are unable to change even if they want to. To help these subjects, clinical psychology is often used to facilitate personal change. However, as noted by Higginson and Mansell (3): "The mechanism of change is not fully understood. This is clear in research demonstrating the efficacy of different therapeutic approaches and the significant rates of natural recovery" (p. 326). On one side different studies suggest the lack of differential effectiveness between therapies – many therapies have equivalently positive outcomes – despite manifestly non-equivalent theories and techniques (4, 5). On the other side, some people experience personal change without the help of any form of psychotherapeutic treatment.

The recent advances in psychology and neuroscience are now providing a better view of personal change that help us in understanding the potential of these technologies. The main tenets are:


We will deepen these points in the following section.

The starting point for our exploration of the process of change is the perceptual control theory (PCT) (9, 10). According to this vision, the process of control is the critical feature of human nature (9): "life is a constant process of comparing how things are with how we want things to be, and if they do not match doing something to get closer to how we want things to be" (p. 250).

Generally, control is a defined as the process of reducing the distance between what we want and what we are (*error*). Interestingly, the source of errors is both *within* and *between* individuals (9, 11). Specifically, PCT suggests that a possible source of error is internal: the coherence between goals and subgoals of the individual (*conflict*).

To eliminate a conflict, the individual must direct his or her awareness to the experience that is creating the conflict. Then, a reorganization is required: a trial and error process, which modifies the characteristics or the conflicting goals (3).

The PCT can be integrated with the vision of a second theory: self-affirmation theory (SAT) (12, 13). According to this view, individuals are motivated to maintain their self-integrity, defined as (7) "a sense of global efficacy, an image of oneself as able to control important adaptive and moral outcomes in one's life" (p. 336). On one side, any threat to it evokes self-defense and psychological stress. On the other side, subjects can import into a critical domain the sense of personal integrity that they feel in another. In the concept of self-integrity, a critical role is played by self-efficacy, the strength of one's belief in one's own ability to complete tasks and reach goals (14, 15). As noted by Bandura (16, 17), self-efficacy determines whether the process of change will be initiated, how much effort will be expended toward it, and how long it will be sustained in the face of obstacles and aversive experiences.

The process of change is also the focus of another vision: the TransTheoretical Model of Behavior Change (18–20). This model describes personal change as a progression through a series of five stages: precontemplation, contemplation, determination, action, and maintenance. These stages represent a temporal dimension that allows both the individuals and the persons supporting them to understand when particular shifts in attitudes, intentions, and behaviors occur.

However, not all personal changes occur in a linear or gradual manner. As noted by Miller and C'de Baca (21), individuals may have "transformative experiences" able to produce a deep and enduring restructuration of one or more personal dimension. According to Mezirow's Transformative Learning Theory (22, 23), these experiences can be triggered by a "disorienting dilemma" usually related to a life crisis or major life transition (e.g., death, illness, separation, or divorce), which forces individuals to critically examine and eventually revise their core assumptions and beliefs. The outcome of a transformative experience is a significant and permanent change in the expectations – mindsets, perspectives and habits of mind – through which we filter and make sense of the world.

As noted by Kottler (6) and Riva (24), by merging these theories, we can identify some important properties of personal change:

	- a. selfhood is affected by a crisis, trauma, or developmental transition;
	- b. a level of pain and discomfort is reached that cannot any longer be ignored or denied;
	- c. there is an awareness or insight that something different must be done (change);

The critical steps in this process are three: the emergence of transformative experiences, the passage between stage *c* and *d* and the one between stage *d* and *e*.

As noted by Gaggioli (25), transformative experiences provide knowledge that is epistemically inaccessible to the individuals until they have that experience. For this reason, transformative experiences cannot be planned in advance but happen suddenly in individuals' lives without a prior control on their contents and their effects.

Instead, the passage between stage *c* and *d* requires *selfreflectiveness*: an intense focus on the particular instance or experience creating the conflict (26). By exploring this experience as thoroughly as possible, the individual can relive and identify all of the significant elements associated with it (e.g., conceptual, behavioral, emotional, and motivational) facilitating their reorganization (24).

Finally, the passage between stage *d* and *e* requires the belief of *personal efficacy* (16, 17): individuals have to believe that they have the power to effect changes through their actions. Without it there, they are not willing to act, or to keep on acting in the face of problems and difficulties.

Starting from these premises, the paper wants to review the potential of virtuality for enhancing the process of personal change. First, the paper will explore the two leading virtual technologies – augmented reality (AR) and virtual reality (VR) – assessing their current uses in behavioral health and the outcomes of the available systematic reviews and meta-analyses. Then the paper will discuss their added value in transforming our external experience by focusing on the elevated level of personal efficacy and self-reflectiveness generated by their sense of presence and emotional engagement. Finally, it will outline the potential future use of virtuality for transforming our inner experience by structuring, altering, and/or replacing our bodily self-consciousness (BSC).

#### THE VIRTUAL TECHNOLOGIES: AUGMENTED REALITY AND VIRTUAL REALITY

#### Experiential Learning through Augmented Reality

Augmented reality can be described as an interactive visualization system (a head-mounted display, a computer, a game console, a smartphone, or a tablet) allowing the merging of digital contents with the real environment surrounding the user (27, 28). In simpler words, AR allows the augmentation of our real experience blending both "real-world elements" and "virtual elements," which may involve not only the view but also hearing, touch, and smell (29). For this reason, within the reality–virtuality continuum (see **Figure 1**) introduced by

Milgram and Kishino (30) to describe all the possible combinations of real and virtual objects, AR is the step just after the real environment.

In fact, the most important feature of AR is that the synthetic objects and data provide the real world with remarkable and valuable information for its user. van Krevelen and Poelman describe this opportunity in the following way (31): "Imagine a technology with which you could see more than others see, hear more than others hear, and perhaps even touch, smell, and taste things that others cannot. What if we had technology to perceive completely computational elements and objects within our real world experience… that help us in our daily activities, while interacting almost unconsciously through mere gestures and speech?" (p. 1).

The additional information offered by AR can be a powerful tool for personal change, because it can support and improve the sense of self-reflectiveness and personal efficacy of its users.

For these features, AR seems to be a promising and useful tool for intervention in the treatment of specific phobias (29, 32), as also supported by a recent systematic review (28) (see **Table 1**) and a narrative review (33).

However, its potential in supporting personal change is wider as demonstrated by different emerging applications: from poststroke (34) and physical rehabilitation (35, 36), social (37) and emotional (38) training for children with autism, to pain reduction (39). Here, we suggest that the added value of AR is related to the support it offers to all the stages of the experiential learning cycle (40) (see **Figure 2**).

According to the influential model introduced by David Kolb, experiential learning is a process consisting of four stages: experience, observation and reflection, abstract reconceptualization, and experimentation (40, 41). The individual starts the learning process from his/her experience, which leads to observations and reflections on its contents. Specifically, abstract conceptualization is used to create a generalization of the experience that is evaluated, integrated with the available knowledge, and converted into recommendations. These recommendations activate new actions and strategies, which can be tested and explored, to adjust the original experience.


A possible example of how AR can be used to support experiential learning in relation to clinical change is the treatment of a specific phobia, for example cockroach phobia (32, 42, 43). In the "concrete experience" stage, the patient observes the cockroaches in the "here-and-now" offered by AR. This experience is the "basis for observation and reflection" (reflective observation): the patient has the opportunity to consider the actions – e.g., avoidance – and emotions – e.g., irritability and fear – experienced and to think about his/her behavior in previous similar experiences (generalization). At this stage, the role of the therapist is important: generalization questions – e.g., individual is asked to compare the performance in earlier exposures and to identify the pros and cons of the different behaviors – can be used to facilitate abstract conceptualization and to identify recommendations supporting the next exposure. During the new exposure, AR can support, in real time, the patient, offering real-time information about his/her status – e.g., the level of emotional arousal – and practical suggestions – e.g., to keep their hands closer to the ones of the therapist. Moreover, AR can enhance the process of change throughout the entire treatment, for instance, Botella and colleagues (44) used an AR-based serious game in a mobile phone in order to facilitate the exposure treatment. The goal of this game was reducing the level of fear and avoidance before the AR exposure session and promoting the over-learning after the AR exposure session as a homework assignment.

In this way, AR facilitates personal change through a cyclical interaction of experience, thought, and reflection (active experimentation).

In other words, AR is the perfect experiential learning tool. On one side, it allows real-time interactivity in an ecological setting improving concentration and motivation (45). On the other side, it provides targeted and non-directive suggestions and guidelines that help users to develop skills and knowledge in a more effective way (46). Finally, as suggested by Baus and Bouchard (32), AR can be used in the actual places the subject encounters his difficulties, facilitating the transfer of the acquired skills to the real world.

#### Virtual Reality as Simulative Technology

Within the reality–virtuality continuum (see **Figure 1**) introduced by Milgram and Kishino (30), VR is the final step, in opposition to reality: if AR adds digital information to the real-world environment, VR completely replaces it with a virtual one. But what is VR?

In computer sciences, VR is usually described as a set of fancy technologies (47, 48): an interactive 3D visualization system (a computer, a game console, or a smartphone) supported by one or more position trackers and head-mounted display. The trackers sense the movements of the individual and report the collected data to the visualization system, which updates the scene in real time.

However, in psychology and neuroscience, VR is instead defined as (49) "an advanced form of human–computer interface that allows the user to interact with and become immersed in a computer-generated environment in a naturalistic fashion" (p. 82). From a psychological perspective, VR is a *subjective experience* cheating the individual out of the illusion that he/she is there, that this experience is real (50). Specifically, VR is different from other media because it induces the sense of "presence": the feeling of "being there" inside the virtual experience produced by the technology (51, 52).

While still is lacking a general consensus about the definition and the etiology of presence [for an introduction to this topic see Ref. (53–66)], most researchers agree about what it is not (65). As underlined by Riva and colleagues (54) "Presence is not the degree of technological immersion, it is not the same thing as emotional engagement, it is not absorption or attention or action; but all of these have a potential role in understanding the experience of presence in interaction – the experience of interacting with presence" (p. 1).

The sense of presence offered by VR can be a powerful tool for personal change because it offers *a world where the individual can stay and live a specific experience* (47, 67, 68). Following the model of personal change discussed before, VR allows a level of *self-reflectiveness* that is higher than the one provided by memory and imagination, and it is more controlled than the one offered by direct "real" experience. In fact, VR can also be defined as an "*advanced imaginal system*" (69–71): an experiential form of imagery that is as effective as reality in inducing emotional responses.

As underlined by Glantz and colleagues (72) "One reason it is so difficult to get people to update their assumptions is that change often requires a prior step – recognizing the distinction between an assumption and a perception. Until revealed to be fallacious, assumptions constitute the world; they seem like perceptions, and as long as they do, they are resistant to change" (p. 96). Using the sense of presence induced by VR, it is easier to develop new, realistic, credible, and informative experiences regarding the surrounding world or the self demonstrating to the individual that what is assumed to be true – e.g., my team disapproves me – in fact is a result of his/her mind. Once this has been understood, it is easier to identify all of the elements supporting the assumption and make them available for reorganization (73).

These features clearly explain the increasing use of VR in behavioral health and, in particular, in the treatment of anxiety disorders. In **Table 2**, we reported all the available systematic reviews and meta-analyses related to the use of VR in behavioral health. To select them, a computer-based search in several databases was performed for relevant publications. Databases used for the search were PsycINFO, PubMed/Medline, and Web of Science (Web of Knowledge). We searched using the string "Virtual Reality" AND ("review" OR "systematic review" OR "meta-analysis" OR "meta-analysis"). English, French, and German were used as language limits. More, we hand-searched the reference lists of all relevant articles to find additional studies (snowball technique).

We have included only articles on VR used for supporting personal and clinical change. Excluded from the analysis were studies related to the use of VR in surgery, physical and cognitive rehabilitation, and review articles lacking basic information about the selection of the discussed papers.

Our initial search yielded 918 non-duplicate citations screened *via* PsycINFO, PubMed/Medline, and Web of Science (Web of Knowledge). After the application of inclusion/exclusion criteria, papers have been reduced to 67 articles. A more in-depth investigation of the full papers resulted in an exclusion of 40 articles. Twenty-seven of them were excluded because their specific focus was not VR, while the remaining 13 were excluded, because they lacked a clear description of the process used to select the discussed papers. In the end, 27 studies met full criteria and were included in **Table 2**.

As expected, the highest number of papers – four meta-analyses and seven systematic reviews – is related to the use of VR in the treatment of emotion-related disorders: anxiety disorders (four meta-analyses and three systematic reviews) (66, 74–79) and stress-related disorders (four systematic reviews) (80–83). The results support the use of VR in the treatment of phobias (66, 74, 75, 79), stress management (81), post-traumatic stress disorders (80, 82, 83), and panic disorders with or without agoraphobia (77). No definitive evidence is available for the treatment of social phobia (66, 75, 76). More, as underlined by most studies (84), and specifically by McCann and colleagues (78), the quality of future research has to be improved using well-specified randomization procedures, assessing treatment adherence, and providing a better standardization of clinical protocols.

As noted by Riva and Mantovani (85), the rationale behind the use VR in anxiety disorders is simple "…in VR, the patient is intentionally confronted with the feared stimuli while allowing the anxiety to attenuate. Avoiding a dreaded situation, reinforces a phobia, and each successive exposure to it reduces the anxiety" (p. 21). In other words, VR is a versatile tool that permits to develop multiple environments that can be presented to the user in many different forms (66, 86). Recent studies show that VR exposure to multiple contexts reduces the recurrence of fear to a greater extent than exposure to only one scenario (87); in the same way, return of fear at posttreatment was significantly reduced by the use of multiple stimuli contexts during exposure (33, 88). Therefore, exposure to different virtual contexts can be an effective way to generalize the results. More, as suggested by Diemer and colleagues (62), VR can be used to induce emotional reactions *via* different routes (perceptual vs. conceptual), with additive effects if combined.

These studies are in agreement with the results obtained by Craske et al. (89) on the inhibitory learning approach presenting exposure optimization strategies such as (a) "deepened

#### Table 2 | Meta-analyses and systematic reviews related to the use of VR in the different areas of behavioral health.


*(Continued)*

#### table 2 | Continued


extinction," where multiple fear stimuli are first extinguished separately before being combined during extinction; (b) "variability," using different stimuli, levels of intensity, and durations; or (c) "exposure to multiple contexts," different in terms of colors and textures. These kinds of effects are not easy to obtain in the "real world" but easier to achieve by using VR.

A similar process can be used in addiction. As underlined by two systematic review by Bordnick and colleagues (90) and Hone-Blanchet and colleagues (91), VR stimuli (e.g., liquor bottles and cigarette packs) and VR environments (e.g., bar and party) are effective in inducing strong craving in cocaine/alcohol/smokingdependent subjects. It is supposed that the reactions induced by VR cues support motivational processes reducing relapse in addicts attempting to remain abstinent (92, 93). The same result is reported in the recent meta-analysis by Pericot-Valverde and colleagues exploring the use of VR in cigarette craving assessment. The paper confirms the potential of VR "for the generation of robust cue-specific craving in cue-reactivity research" (94). As discussed before, the use of VR exposure allows patients to experience arousal and reactivity in a controlled setting, and to develop new coping skills through repeated exposures and practice (95).

The use of VR as a controlled setting in which to develop new skills through trials and errors is also effective with persons with autistic spectrum disorder. The two systematic reviews support the use of VR to learn how to cope with social situations (96, 97), in particular to learn communication and imitation skills.

In summary, as underlined by Pla-Sanjuanelo and colleagues (98) "the capability of developing a large amount of realistic controlled stimuli and, simultaneously, of monitoring the responses generated by the user offers a considerable advantage over real experiences" (p. 145). For example, if an individual experiences a significant fear when exposed to heights, using a virtual elevator simulation, the therapist can assure him/ her that this threat will be experienced only when he/she is prepared to cope with it. The same can be said for all the elements that are present in the situation, which can make it more or less threatening (99, 100). For instance, the height of the spaces, the presence of protecting elements, and the duration of a determined situation.

More, VR allows the construction of "virtual adventures" in which subjects experience themselves as competent and efficacious (47, 101, 102). Specifically, it is possible to design targeted VR experiences with different difficulty levels – from easy performances to very difficult ones – that offer an important source of personal efficacy. Interacting with them individuals discover that the conflicts and/or feared situations can be overcome through confrontation and effort (85).

Finally, as recently suggested by Gaggioli (25), a further opportunity offered by VR is the possibility of simulating impossible worlds – that is, worlds that do not conform to the fundamental laws of logic and nature. For example, Friedman and colleagues used VR to give people the illusion of backwards time travel allowing them to relive a sequence of events in which they can intervene and change history. In this view, time alterations and time paradoxes (e.g., the possibility of changing and restructuring the history) represent a kind of impossible manipulation of physical reality that is feasible in virtual reality and that can be used to elicitate transformative experiences (103).

#### Virtual Reality as Embodied and Transformative Technology

However, VR is more than a tool to provide exposure, training, and desensitization (104), as evidenced by the other areas of behavioral health in which VR systematic reviews have been found: *depression, psychosis, schizophrenia, pain management, obesity, and eating disorders*.

The first three applicative areas are still in their infancy. The systematic review discussing the use of VR in depression (105) and psychosis (106) suggests that VRET may be effective both for reducing depression caused by fear and for investigating psychological processes and mechanisms associated with psychosis. However, actual studies are not conclusive (105, 106). A similar picture is provided by the two systematic reviews assessing the use of VR in schizophrenia (107, 108): there is no clear good quality evidence for or against using VR for treatment compliance among people with schizophrenia (107), even if existing studies suggest a potential for increasing our understanding of psychosis (108). The situation is different for pain management and the treatment of obesity and eating disorders.

The four systematic reviews related to the use of VR in pain management support its use in the treatment of acute pain (109–112) and, in particular, in reducing the pain experienced by burn injury patients during wound dressing changes (109, 110). A strong overall evidence has been found for immediate and short-term pain reduction after VR, while moderate one for short-term effects on physical function. A recent systematic review tried to shed some light on the rationale of this approach (112). The results suggest that if on one side, the sense of presence influences the effectiveness of VR as a distraction tool, on the other side anxiety as well as positive emotions directly affect the experience of pain. The potential of VR in the treatment of pain was recently confirmed by "The Italian Consensus Conference on Pain in Neurorehabilitation" (113). In their analysis of the psychological interventions and psychotherapies that can be used within an integrated approach for patients undergoing neurological rehabilitation for pain, authors included the use of VR (grade of recommendation: D) for acute pain management (113).

A similar outcome is provided by the three systematic reviews related to the use of VR in the treatment of obesity and eating disorders (114–116): both VR cue exposure to food stimuli (115–117) and VR body image treatments (114, 115) are effective. A recent narrative review (117) confirms these data concluding that "VR has, for the past two decades, proven to be a useful adjunctive tool for both assessment and treatment of patients with eating disorders and obesity" (p. 71). In particular, as underlined by Gutiérrez-Maldonado and colleagues (118): "Recent studies indicate that … VR can integrate and extend existing treatments for eating and weight disorders. Future possibilities for VR to improve actual approaches include its use for altering in real time the experience of the body (embodiment) and as a cue exposure tool for reducing food craving" (p. 148).

But what is the link between the use of VR in pain and eating/ weight disorders? Both are effective in modifyng the experience of the body of their users. Let us try to deepen this claim.

Virtual reality can be defined as an "embodied technology" for "its ability of modifying the feeling of presence" (119–121): in VR, subjects can experience the synthetic environment as if it was "their surrounding world" (*incarnation*: the physical body is within a virtual environment) or can experience their synthetic avatars as if they were "their own body" (*embodiment*: the physical body is replaced by the virtual one). In other words, the VR user is present in a virtual world or in a virtual body through the alteration of the cognitive factors regulating our experience of body and space [for an in-depth analysis of this claim, see Ref. (119)]. The side effect of this alteration is the experience of simulation sickness experienced by some VR users: the conflict between the visually perceived movement in the virtual world and the vestibular system's sense of movement (I stand still) may produce negative effect like vomiting, discomfort, disorientation, and fatigue (122).

The increasing interest of cognitive science and social psychology for the experience of the body – Bodily Self-Consciousness – BSC – is providing a better picture of these processes.

First, BSC is apparently experienced by the subject as a single and coherent experience. However, neuroimaging and clinical data suggest that BSC is the outcome of different experiential layers (123–127). Specifically, we become aware of our bodies through exteroceptive signals arising outside the body (e.g., vision, and touch) and through proprioceptive (e.g., skeletal joints/muscles) and interoceptive (e.g., heart rate) signals arising inside the body (128, 129).

Second, the above studies support also the idea that body representations play a central role in structuring cognition and the self (124, 130–132). As underlined by Blanke (124) in his paper for Nature Reviews Neuroscience: "Human adults experience a "real me" that "resides" in "my" body and is the subject (or "I") of experience and thought. This aspect of self-consciousness, namely the feeling that conscious experiences are bound to the self and are experiences of a unitary entity ("I"), is often considered to be one of the most astonishing features of the human mind" (p. 556). For this reason, the experience of the body is strictly connected to processes like cognitive development and autobiographical memory.

Third, we use the "feelings" from the body to sense both our physical condition and emotional state. These feelings range from bodily changes both visible (e.g., posture, touch, and facial expressions) and invisible (e.g., heart rate, endocrine release, and muscle contractions) to an external observer (133).

Fourth, the characteristics of BSC evolve over time following the ontogenetic development of the subject. As suggested by Riva (134), we expand over time our BSC by progressively including new experiences – minimal selfhood, self-location, agency, body ownership, third-person perspective, and body satisfaction – based on different and more sophisticated bodily representations that progressively integrate.

Fifth, bodily representations are usually produced and modulated by sensory inputs, but they can exist and produce qualitatively rich bodily experiences even in the absence of any input signal (e.g., phantom limb syndrome) (135). In this view, the experience of our bodily self is the outcome of a multimodal simulation. As underlined by Wilson (136): "The human perceptual system incorporates an emulator… that is isomorphic to the human body…the emulator draws on body-schematic knowledge derived from the observer's representation of his own body" (p. 221).

Starting from these premises in 2007, two European teams of cognitive neuroscientists independently reported in Science how VR could be used for altering BSC (producing an out-of-body experience) in healthy volunteers (137, 138). Since then, the rapid development of immersive VR environments has allowed a new research line – *virtual embodiment* – (139–142) whose results are discussed in two recent reviews (143, 144). But how the experience of "being" in a synthetic body is achieved in these studies? These experiments are an evolution of the trick used in the low-tech rubber-hand illusion (145): the cross-modal congruence between what a person feels *via* the somatosensory pathways (touch) and what (s)he sees in VR. Using VR, different authors induced the illusion of a fake hand (146) or a fake limb (147) and produced an out-of-body experience (137) by modifying the normal association between touch and vision. Slater and colleagues even used VR to induce a body transfer illusion (147): they transferred a group of male subjects in a life-sized virtual human female body. Interestingly, altering the experience of the body has significant effects also on social cognition: for example, the transfer illusion in a body of different race produced a significant reduction of the implicit bias against that race (148).

But, how can these research data drive a new generation of VR tools aimed at supporting personal and clinical change? Up to now, VR has been used to simulate external reality, which is to make people feel "real" what is actually not really there. The next step is the use of VR for the simulation of our internal reality, including the way we perceive our body, control it, and affectively react to what happens to it. The final outcome is a new generation of transformative experiences that provide knowledge that is epistemically inaccessible to the individual until he or she has that experience, while at the same time transforming the individual's worldview (25).

This opportunity may also open a radically new research field in medicine – Embodied Medicine – allowing new clinical solutions for the treatment of neurological and psychiatric disorders where our BSC seems to be altered (119). Considerable evidence suggests that the etiology of different disorders – including PTSD, eating disorders, depression, chronic pain, phantom limb pain, autism, schizophrenia, Parkinson's and Alzheimer's – may be related to an impaired/altered BSC. More, it may offer a scientific path to improve the level of well-being in non-clinical subjects by inducing positive emotions, improving attitudes, and helping individuals in understanding and controlling the signals of their body.

In general, it is possible to modify our BSC in three different ways (see **Table 1**) (25, 119, 120, 149, 150):

• *By structuring BSC through the focus and reorganization of its contents (mindful embodiment)*: individuals have different levels of body awareness is the extent of sensitivity and attentiveness to bodily signals and sensations (151). VR, if integrated with other technologies like biosensors, can be applied for improving body awareness. For example, in integration with biofeedback, training can be used to assess and control specific body signals like heart rate, galvanic skin response, electromyography, or electroencephalography (152, 153).


#### CONCLUSION

This paper claimed that virtual technologies – AR and VR – have the potential for supporting personal and clinical change: if AR adds virtual information to the real world, VR completely replaces the real environment with a virtual one. Specifically, both AR and VR can transform our external experience through the high level of personal efficacy and self-reflectiveness generated by their sense of presence and emotional engagement. Moreover, VR can also modify our inner experience by structuring, altering, and/or replacing our BSC.

In the first part of the paper, we explored the characteristics of the process of change. By integrating the available literature, we identified three critical steps that may slow down or block the process.

First, the process of change requires *self-reflectiveness*: an intense focus on the particular instance or experience creating the conflict (26). By focusing on this experience as much as possible, the individual can relive and identify any significant element (e.g., conceptual, behavioral, emotional, and motivational) facilitating its reorganization (24).

Second, the process of change requires *personal efficacy* (16, 17): individuals have to believe that they have the power to effect changes through their actions. Without it there, they are not willing to act or to keep on acting in the face of problems and difficulties.

Finally, the process of change can be dramatically boosted by *transformative experiences* (22, 23), forcing individuals to critically examine and eventually revise their core assumptions and beliefs. The key advantage of these experiences is that they are epochal (158): "a sudden, dramatic, and reorienting insight" (p. 86) pushing the individual to an immediate and irreversible change. Unfortunately, most transformative experiences cannot be planned in advance, but happen suddenly in individuals' lives, without a prior control on their contents and their effects.

In the second part of the paper, we analyzed the outcomes of the available systematic reviews and meta-analyses related to the use of virtual technologies in behavioral health to identify and discuss the added values offered by them.

The only available systematic review for AR support its use in the treatment of phobias. Moreover, it also outlines the value of AR as experiential learning tool. On one side, it offers real-time interactivity in an ecological setting improving concentration and motivation (45). On the other side, it provides targeted and non-directive suggestions and guidelines helping individuals to develop skills and knowledge in a more effective way (46). Finally, AR can be used in the actual places where the subject encounter his difficulties, facilitating the transfer of the acquired skills to the real world (32).

The 27 meta-analyses and systematic reviews available for VR support the use of this technology in the treatment of anxiety disorders, stress-related disorders, obesity and eating disorders, and pain management. But still, there is no clear good quality evidence for or against using VR for the treatment of depression and schizophrenia.

In most pathologies, VR is used as simulative tool for controlled exposure to critical/fearful situations. The possibility of presenting realistic controlled stimuli and, simultaneously, of monitoring the responses generated by the user offers a considerable advantage over real experiences. More, the possibility of designing targeted VR experiences with different difficulty levels – from easy performances to very difficult ones – offers an important source of personal efficacy.

However, the use of VR in pain management and in the treatment of obesity and eating disorders suggest a different rationale: VR can also be used as an embodied technology able to alter our experience of the body and space. If most VR applications to date have been used to simulate external reality, it is also possible to use VR for the simulation of our internal reality including the perception and ownership of our body. The final outcome may be a new generation of transformative experiences that provide knowledge that is epistemically inaccessible to the individual until he or she has that experience, while at the same time transforming the individual's worldview and pushing him/her to an immediate and irreversible personal or clinical change (25, 159). More, it may offer a scientific path to improve the level of well-being in non-clinical subjects by inducing positive emotions, improving attitudes, and helping individuals in understanding and controlling the signals of their body.

At the end, the contents of this review offer a sound foundation and rationale for researchers interested in using virtual technologies for improving personal and clinical change.

#### AUTHOR CONTRIBUTIONS

GR performed the literature review and drafted the first version of the manuscript. RB, CB, FM, and AG supervised the rationale

#### REFERENCES


and the scientific contributions. All authors read and approved the final manuscript.

#### FUNDING

This paper was funded by the research project "Tecnologia Positiva e Healthy Ageing" funded by Università Cattolica del Sacro Cuore (D3.2, Ricerche di interesse d'Ateneo 2014).


acceptability. *Neuropsychiatr Dis Treat* (2015) 15:2533–45. doi:10.2147/NDT. S89542


*Subjects*. Farnborough, UK: Army Personnel Research Establishment (1993). Report 93R022.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Riva, Baños, Botella, Mantovani and Gaggioli. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# A Serious Game to Improve Cognitive Functions in Schizophrenia: A Pilot Study

*Isabelle Amado1,2,3\*, Lindsay Brénugat-Herné1,2,3 , Eric Orriols3,4 , Colombe Desombre1 , Maxine Dos Santos1,3,4 , Zelda Prost1,2 , Marie-Odile Krebs1,3 and Pascale Piolino3,4,5*

*1Sainte Anne Hospital, Paris, France, 2Reference Center for Cognitive Remediation and Psychosocial Rehabilitation, INSERM U894, Sainte Anne Hospital, Paris, France, 3Center for Psychiatry and Neuroscience, INSERM U894, Paris Descartes University, Paris, France, 4 Laboratory for Memory and Cognition, Sorbonne Paris Cité, Paris, France, 5 Institut Universitaire de France (IUF), Paris, France*

#### *Edited by:*

*Yasser Khazaal, Geneva University Hospitals, Switzerland*

#### *Reviewed by:*

*Arezoo Shajiei, University of Manchester, UK Nicolas Franck, Claude Bernard University, France*

> *\*Correspondence: Isabelle Amado i.amado@ch-sainte-anne.fr*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 11 February 2016 Accepted: 30 March 2016 Published: 20 April 2016*

#### *Citation:*

*Amado I, Brénugat-Herné L, Orriols E, Desombre C, Dos Santos M, Prost Z, Krebs M-O and Piolino P (2016) A Serious Game to Improve Cognitive Functions in Schizophrenia: A Pilot Study. Front. Psychiatry 7:64. doi: 10.3389/fpsyt.2016.00064*

Cognitive deficits in schizophrenia impair everyday functioning and instrumental daily living activities. These disabilities can be partly responsible for chronicity and institutionalization. We present here a virtual reality (VR) tool in which patients with schizophrenia performed a virtual game in an imaginary town during a 3-month program. In a pilot study, seven patients with schizophrenia (DSM-5), institutionalized for many years, attended weekly 1-h-and-a-half sessions organized by two clinicians. During the first sessions, they listed together the difficulties they experienced in everyday organization and planning. After being familiarized with the joystick and the VR environment, they navigated in the town, and planned actions that were difficult for them to carry out in their usual life (e.g., shopping, memorizing the way to the supermarket or being on time at a meeting point). They had to look for alternative routes and practice a switch from a 2D Map to the 3D Map. They also gathered their efforts to share strategies for each action, or discussed the action plan they could generate to solve concrete problems. The pre/ post-neuropsychological evaluations showed attention, working memory, prospective, and retrospective memory benefits, but no improvement in planning as assessed by the Zoo map test and the action program subtest of Behavioral Assessment of the Dysexecutive Syndrome. Patients also clinically and functionally improved, gaining autonomy. Pragmatically, they reported a strong energy to elaborate concrete plans to search for jobs, or return to activities in the community. Qualitative assessments showed a benefit in sparing time, planning better, enriched relatedness, and better management of their housework. This VR game opens avenue to rehabilitation for patients with schizophrenia experiencing chronicity in their life, less attendance in daycare units, and a better community living. This program might reduce neurocognitive difficulties and might evolve into a true method for cognitive remediation (trial n° 2011-A00988-33).

Keywords: cognition, schizophrenia, virtual reality, serious game

## INTRODUCTION

Cognition impairments in schizophrenia are found in multiple domains, with direct implications in everyday functioning (1). These difficulties limit the patients' access to full time employment, independence in their residential living, and social outcomes (2). Green et al. (3) revealed associations between specific neurocognitive constructs and functional outcome. Josman et al. (4) reported a link between deficits in executive functioning and ability to perform daily activities in subjects with schizophrenia using the instrumental activities of daily living scale (5). Actually, psychosocial therapies and more specifically cognitive remediation are necessary to complete pharmacological treatments to alleviate cognitive deficits and improve patients' everyday functioning.

Although neuropsychological tests provide important information about cognitive disabilities, they generally have low ecological validity and, therefore, have limited ability to predict functioning in daily activities (6). The term "ecological validity" is central in the area of assessment of crucial functions, such as executive functioning, as these functions are fully associated with real-life complex situations (e.g., shopping, preparing a meal, or medication adherence) that require planning, organization, and structuring (7). Very recently, the apprehension of everyday functioning has been enriched with the development of computerized assessment of functional capacities using virtual reality (VR). VR is supposed to mimic the real world in an immersive and potentially remotely deliverable environment. In psychology, VR provides an immersion in a safe, non-stressful, and ecologically valid environment, while maintaining strict experimental control over stimulus delivery and measurement (8). Moreover, VR enables users to be engaged in simulated, interactive environments that are similar to real-world objects and events (9). These environments are multimodal and offer the opportunity to record all modalities of cognitive and behavioral activities (10–12). In the executive functioning domain, studies using VR reported that the immersion in that world and in a meaningful context facilitates performance (13).

In the psychosis domain, Kurtz et al. (14) used a virtual apartment to evaluate the medication monitoring in patients with schizophrenia versus that of controls: patients made more errors in the number of pills taken, were less accurate at taking medication, and were less attentive to the hour the pills had to be taken. These defects reported in VR were in agreement with real-life reports, through a classification of adherence versus non-adherence in medication management skills. Josman et al. (4) compared planning abilities in patients with schizophrenia versus controls with a shopping task, using the Virtual Action Plan supermarket (15), while examining the correlations with the Behavioral Assessment of the Dysexecutive Syndrome [BADS (16)]. Patients had more difficulties than controls, specifically to perform the different actions, such as buying virtually all the items on the shopping list. Zawadzki et al. (12) evaluated visuo-spatial organization and spatial orientation during a virtual navigation in a realistic city. Individuals with schizophrenia had difficulties in route-finding within the virtual city. They were also more likely not to notice the target during passive viewing, not to find novel shortcuts to targets, and more likely to become lost and fail completely in finding the target. Scores were negatively correlated to neurocognition as well as to the Quality of Life Scale and to psychosocial functioning. Lastly, Ruse et al. (17) proposed the virtual reality functional capacity assessment tool (VRFCAT) to evaluate performance in everyday life in an ecological manner, with different versions of short scenarios that include navigating in a kitchen, planning a trip to the grocery store, and therefore taking a bus, organizing all the steps to go shopping. Patients with schizophrenia compared to controls showed difficulties in several items. Also, strong correlations were found with slowing of information processing and working memory evaluated with the Matrix Consensus Cognitive Battery.

However, in the neurocognitive domain, if VR is mainly used to evaluate cognitive abilities in an ecological manner, no program exists to improve executive functioning in a VR environment, supposed to mimic the real world.

Our aim was to test a new method to improve cognitive abilities, especially memory, executive functioning, and planning in patients with schizophrenia, using a virtual city. Therefore, we mainly focused our pre–post-neuropsychological evaluations on retrospective and prospective memory, working memory, learning, and planning abilities. A total of twelve 1-h-and-a-half weekly sessions during a 3-month program were delivered in groups of a maximum of six patients. As VR provides an immersive nonstressful environment, we wanted to propose a VR program to patients with schizophrenia who were institutionalized, with long-time use of daycare programs of therapeutic activities. These patients, because of their very ritualized way of living, were considered as more chronic than other patients and less susceptible to accept spontaneously a direct access to rehabilitation perspectives, such as job employment or professional training. Also, it was a challenge for our unit, because institutionalized patients are difficult to drive to innovative therapies, and to orient toward external activities that take place in the real world. This very preliminary open study was a clinical trial with raters who were not blind to assessments and to treatments, with no randomization, and no control arm.

Our hypothesis stated that a VR environment improves patients' cognitive abilities, mainly in prospective memory and planning, with a clinical benefit and a benefit in social functioning, quality of life and self-esteem, assessed in a pre/post comparison before and after the VR program.

#### MATERIALS AND METHODS

#### Population

Ten outpatients (in two different groups) meeting Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (18) criteria for schizophrenia or schizoaffective disorders were recruited. Patients had been institutionalized for several years, coming several times a week to a daycare activity center (Centre d'activité thérapeutique à temps partiel, Rue Mathurin Régnier, 75015 Paris). Two patients dropped out after the first two sessions. Exclusion criteria included auditory or visual impairment, mental retardation (IQ < 70), traumatic brain injury, presence or history of neurological illness, bad understanding of French or instructions, and/or criteria met for concurrent substance abuse or dependence. All participants provided written informed consent, and all procedures met institutional ethical approval, following the declaration of Helsinki (trial n° 2011-A00988-33). Finally, although eight participants were able to complete the clinical assessments, only seven of them performed the pre/post neuropsychological tests, because one patient refused to be evaluated after the program, and only six participants performed VR game assessment due to a technical problem during the preprogram evaluation.

#### Procedure

This study took place in our Reference Center for Cognitive Remediation and Rehabilitation (C3RP – Paris Descartes University – Sainte Anne Hospital – Paris), in a collaborative partnership with a specialized unit for memory assessment via VR (Laboratory of Memory and Cognition, LMC – Paris Descartes University).

#### The Virtual Town

The virtual town was developed and created by the LMC unit. It consisted of virtual urban environments built with LMC software EditoMem [based on3DVIA Virtools Dev 5.0 software (3dVIA Virtools)] allowing us to create virtual environments inspired by photos of Paris and game scenarios.

These urban environments have been previously used in healthy individuals and patients to assess episodic, spatial, retrospective, and prospective memories as well as working memory [e.g., Ref. (19–23)]. In the present virtual town, the main elements were located so that they could be easily detected (i.e., in front of the subjects, in a corner of a street, or quite salient on the side of the road). Patients had to navigate as pedestrians in a virtual 3D town, using a joystick (simulation software: Simulamem), along several avenues and roads (see **Figures 1A,B**).

Several specific elements are located in the town, such as a bank and several shops offering a large diversity for shopping (bakery, grocery, a supermarket). Hobbies are also represented with a cinema and a "mediatheque," which is a typical French structure where you can freely borrow various types of media. Centers for healthcare also exist with a pharmacy, a dentist practice house and a laboratory for blood analysis. Cafes, restaurants, squares, and a public garden are represented. These different places could be memorized and used as memory cues to orient the navigation, as visual landmarks, or could be a theme for discussion (e.g., what do we have to do in case of a fire beginning in front of you? what are the hypotheses if you come to see a friend and a truck with boxes, which look as a moving out, is located is front of his/her building?). Also, participants, while playing during the VR game, received their own personal 2D map of the virtual town, which they had to complete with street names and mark city's symbolic places. This map could always be referred, while navigating, to help orientation and train switching from 2D to 3D virtual spaces (see **Figure 2**). This VR environment was presented in a quiet room with a computer (15.6″), allowing good visibility for the whole group.

#### Description of a Session

The whole program includes 12 interactive 90-min weekly sessions. Two clinicians, a psychologist and an occupational therapist (ZP/LB-CD) conduct the group sessions. During 60 min, one after the other, the participants have to navigate in the VR town, sharing a joystick. While one of the members in the group navigates, the others are helping him, in an interactive and collective effort. Participants have to find their way (involving, therefore, attention and visuo-spatial organization), to memorize their itinerary (which involves memory for details and topographic memory abilities), to plan different actions (encouraging planning ability, flexibility, executive functions, as well as prospective memory), depending on the instruction they have to follow. After 1 h of navigation and several participants navigating in the town, all the participants in the group discuss during the last 20 min of the session about their actions and the possible transfer to everyday life they can imagine. At the end of each session, a task to perform at home is given, directly linked to the theme of the session (trying to reorganize their administrative papers, organizing a party with friends or family members for the Epiphany, etc.). Two different versions of the virtual town have been used, successively versions 1 and 2 during the 12 sessions for the first program (6 sessions for Version 1 and 4 sessions for Version 2). During the two program sessions, we improved and enriched the city, adding more landmarks. Indeed, we added several bakeries, butcher's shops, and drugstores. We also added a travel agency, two cinemas, a theater, more restaurants and coffee shops, a dairy store, and a florist. Thereby, for the second program, we used a version that could cumulate the different locations existing in Version 1 and the different locations existing in Version 2 for the 12 sessions.

Procedure of the different sessions and thematic contents:


(standing in line and eventual distress they could perceive, planning important things in advance, etc.).


#### Clinical and Neuropsychological Evaluations

Two assessments were done, at W0 before the VR program, and W12 after the end of the last session. For each assessment, there was one clinical interview done by a psychiatrist rater who was not blind for the assessments (Isabelle Amado), and the neuropsychological evaluations done by a neuropsychologist, also not blind to the assessments (Lindsay Brénugat-Herné and Zelda Prost) and also involved in the management of the VR sessions. The interviews and the assessments at W12 also collected the subjective opinions of the participants concerning the program and the subjective benefits or harm they perceived.

#### *Clinical Scales*


#### *Psychosocial Scales and Questionnaires*


#### *Neuropsychological Evaluation*

– Attention, visual scanning abilities, and speed processing: D2 cancellation test (31). The scores considered in the study: D2-KL represents the number of targets correctly canceled; D2-GZ represents the quantitative performance index, that is to say the total number of characters treated; D2-F% represents the quality of the treatment of the information, that is to say, the percentage of errors; D2-GZ-F represents the corrected quantitative performance. Wechsler Adult Intelligence Scale 3rd edition [WAIS-III (32)] – Code subtest: participants have to substitute a number by a symbol as quickly as possible, while the code of the different numbers is noticed in the first line of the sheet. The standard score was considered. This subtest assesses attention and cognitive speed processing – Copy-Code subtest: participants have to copy the symbol appearing in the compartment of the top as quick as possible. The number of compartments completed within the allotted time was considered. This subtest assessed motor speed processing.


of execution, and the number of errors for both versions 1 and 2 of the tests were considered.


Before the second navigation to test the prospective memory performance, and approximately 20 min after the first navigation, subjects were instructed to encode a list of 16 action–intentions from verbal cues and figures presented on a screen: 12 eventbased with incongruent or congruent cue actions (e.g., "at the bus stop, I should sing a song," " take money, at the post office") and 4 time-based actions (e.g., "call a friend after 2 min"). The encoding was rehearsed three times to allow subjects to store up clue actions. If at the third rehearsal, subjects obtained a recall of action–intentions below 8, the prospective task was not administered. Being again immersed in the virtual town, they were requested to stop walking at the appropriate time or place, and tell the experimenter what they had to do. We recorded the total recall of intentions at the third rehearsal before navigation (learning score), the number of correct stop at the appropriate scene or time and the number of correct related action. Then, we asked the participants to recall the list of intentions after navigation [for similar scoring method, see Debarnot et al. (20)].

#### Statistical Analyses

Clinical assessments as well as neuropsychological performance and psychosocial evaluations between W0 and W12 were assessed with a unilateral non-parametric Wilcoxon test (Statistica V.10). For each score, mean and SDs were expressed.

#### RESULTS

#### Socio-Demographical Description of the Participants

Participants were seven men and one woman. Their mean age was 38.6 (12.1), years of education was 12 (1.1). Duration of disease was 13.2 (7.6). All patients were treated, the mean and SD values for chlorpromazine equivalents were162.7 (150.1); for details of treatment, see **Table 1**.

BPRS was also clinically assessed (see **Table 2**).

#### Clinical and Psychosocial Evaluations between W0 and W12

Patients clinically significantly improved at W12 comparatively to W0 for BPRS scores (*p* < 0.001), as well as for GAF scores (*p* < 0.01) (see **Table 2**).

For psychosocial evaluations, there was a significant improvement for the EAS-Total score (*p* < 0.01); when going through the different dimensions, the W12-W0 change for the EAS-personal


#### Table 2 | Clinical and psychosocial evaluations.


*BPRS, Brief Psychiatric Rating Scale; GAF, global Assessment Functioning; EAS, Evaluation of Autonomy Scale; EAS-T, total score; pc, personal care; fc, financial control; al, affective life; S-QOL, scale for quality of life assessment; BIS, Birchwood Insight Scale.*

care dimension was significant (*p* < 0.05) as well as the EAS-affective life (*p* < 0.05) and financial control (*p* < 0.05). For the S-QOL questionnaire, as well as the Insight short form questionnaire, no difference was found for the total scores between W12 and W0. Concerning the SERS, no significant difference was found either between W12 and W0 – and it was not either for positive or negative scores of the SERS.

#### Pre/Post Comparisons in Neuropsychological Assessment

– *Attention, visual scanning and speed processing (See* **Table 3***):* D2 cancelation test: the KL score was significantly different at W12 comparatively to W0 (*p* < 0.05), and so was the GZ-F score (*p* < 0.05). However, there was no significant change for the GZ score (*p* = 0.22), or for the F% score (*p* = 0.19). For the Code subtest of the WAIS-III (32) assessed at W0 and W12, the change for the total score was significant (*p* < 0.05), as well as for the Copy-Code (*p* < 0.05).

– *Verbal and Visual Working memory:* When considering the standard scores of the Digit Span, there was a very significant difference when comparing W12 to W0 (*p* < 0.01), but any difference appeared for the forward span (*p* = 0.25) and the backward span (*p* = 0.14). No significant difference was found for the standard score of the visuo-spatial span subtest (*p*= 0.09) or for the forward span (*p*= 0.11), but a significative difference was found for the backward span (*p* < 0.05).


*WAIS, Wechsler Adult Intelligence Scale; Sd, Standard score; RCFT, Rey–Osterrieth Complex Figure Test; RS, recognition score. Time is given in seconds. Retrospective and prospective memory scores are expressed in terms of ratio.*


Concerning the VR Prospective memory test, event-based component was improved during the navigation unlike timebased component. Participants appeared to be more efficient after remediation, stopping at the correct place (*p* = 0.05), and performing correct actions (*p* < 0.05). There was no difference in learning action–intentions before navigation and cued recall after the navigation.

#### Qualitative Results

At week 12: during the assessment, the clinicians gathered the qualitative opinions of the participants. All of them reported a good tolerance. After the program they noticed: for six participants a better organization, for three patients a gain for planning, two reported an enrichment of relatedness, and more visits done outside their home, for four participants a gain in self-confidence, for five participants more awareness of their own difficulties; for three participants a better rhythm in life, four noticed an effort to search for work or professional training, lastly 3/8 participants requested more therapies, especially a cognitive remediation program or a group for social cognition. Also, they mentioned having done more concrete things in their everyday life, with less stress (e.g., completing administrative files for work employment, opening and sorting out post mails, less mess at home, etc.). One participant went to the employment agency searching for a job. Three participants aimed to return to work, or decided to begin training. One participant wrote a paper in a scientific review to talk about his distressful experience when he was hospitalized and reported the fruitful benefit of the combination of CR and therapeutic activities (37).

#### DISCUSSION

In this study, we proposed to chronic patients with schizophrenia, institutionalized for years in a daycare therapeutic activity center, a VR program focused on cognition, mainly visuo-spatial abilities and planning. Our first hypothesis was that this VR environment improves patients' cognitive abilities, prospective memory, and planning. Furthermore, we hypothesized a clinical benefit and an improvement in social functioning compared to their functional status before the program.

Following our first hypothesis, we observed a significant improvement in the KL and GZ-F sub-scores of the D2 cancelation test. Also, there was significant improvement in the WAIS-Code and Copy-Code. The need to navigate, to develop a good visual orienting in the virtual town, and to establish good landmarks all along the itinerary required an ability to focus on details of the virtual environment (38). Furthermore, patients were instructed to discriminate the streets correctly and rapidly, to take quick decisions to find their way around, and to keep in mind details in the streets, shops, or avenues they had already seen. Lastly, the task they had to perform (shopping in the different shops, action planning, etc.) required a strict attention to the game. On the other hand, the manipulation of the joystick requesting the manual motricity, that could explain the improvement of the motor speed processing. Finally, the standard score of the Digit span was also significantly improved after the program, indicating a greater regularity in verbal working memory performance. Interestingly, we found improvement in backward visual-spatial span, indicating a special enhancement of central executive of visuo-spatial working memory. Once again the richness of details to be kept in mind, as well as the multitasking aspect of this virtual game stimulated visuo-spatial working memory (for example, not forgetting to turn left after the grocery, paying attention to time, buying some bread without forgetting to pick up a friend at the railway station, etc.). In the VR retrospective memory test, significant improvements were found at the end of the program for the "Where" egocentric recognition scores as well as the "When" recognition scores. This result is particularly interesting because schizophrenia is related to a deficit in retrieving contextual information and in binding the different features of an event in memory (39). In the VR prospective memory, the recall of event-based actions during the navigation recall task also improved. The VR program seems to benefit maintenance of action–intentions, detection of prospective event-based cues and intention execution that are generally deficient in schizophrenia (40). Nevertheless, there was no improvement regarding the recall of time-based actions. It could be relevant to augment activities related to time processing (respect of duration or hour) during the VR program. Although encouraging, these positive results have to be cautiously interpreted, as they could reflect performances due to a practice effect. Indeed, the versions used before and after the program were the same. It could be methodologically more rigorous to assess patients with equivalent but different versions of the retrospective and prospective memory tests before/after the program.

Clinically, the BPRS and the GAF scales were very significantly improved. These benefits corroborated the qualitative reports done by the patients. These improvements were noticeable for symptoms as well for qualitative reports of everyday functioning. However, when examining carefully the GAF scores, although there was an average significant improvement, there also was a large variability of the scores after the program. Nevertheless, when examining the BPRS scores, the substantial improvement represented 19.6% of the total scores. This percentage of improvement was similar to what is generally admitted for a pharmacologically active treatment. Hence, this VR program could have a positive, noticeable clinical incidence. However, this result must be tempered by several limitations: the rater was not blind to the treatment, the size of the sample was rather small, and lastly there was no control group to allow a strict comparison versus therapy as usual or versus another treatment-arm. Concerning psychosocial evaluations, there was a significant improvement in the global score of the EAS, with a better personal presentation, a gain in autonomy and improved affective and social relatedness. However, neither the self-esteem, nor the quality of life, nor the insight questionnaires were significantly improved. This has to be considered with a larger group of subjects, or maybe over a longer period of time, as the program only lasted 3 months. This interval is rather short to raise a substantial change for either insight or self-esteem.

These outpatients suffering from schizophrenia or schizoaffective disorder were all chronic users of daycare institutions. They were unemployed, going routinely for years to weekly therapeutic activities, with limited autonomy and complaining of stress and difficulties in their everyday functioning. Two of them were even taking clozapine, indicating a severe form of the disease. The only positive common point when the program began was that they were willing to evolve in their lives. At the end of the program, all reported benefits, with several qualitative points that seemed crucial benefits in everyday functioning: better rhythm in their life, less stress, gain of time, better ability to plan, many things to do at home, concrete things done in their life. The treatment was perfectly well tolerated. The two dropouts happened after the first sessions of the program, probably due to an insufficient motivation to continue. One has to note, however, that a patient did not want to come for the neuropsychological part of the post-program evaluation. This might be due to the challenge this evaluation might represent for some of them. It could be important for the future to put the participants at ease concerning the tests, explaining them that these results will have no incidence for their outcome, treatment, or way of care.

Disconfirming our initial hypothesis, there was no improvement in standard tests of verbal learning and planning. This could be accounted for by several facts: first, the limited power of the tests, due to the small sample. Second, one can notice that the scores obtained by the patients at W0 were high, reaching maybe a ceiling effect. Third, we selected for planning the zoo map test and action planning of the BADS. The zoo map test assesses planning abilities to set up an action plan to solve a problem, define a route for visiting some enclosures in a zoo. This exercise is far from mimicking daily life. Indeed, the participants showed difficulties perceiving the link with the planning abilities required to the everyday life. In addition, the action programs and the zoo tests were not sensitive enough to detect slight variation of performances. Noticeably, ceiling effect was found at W0 for all participants except one at the program action of the BADS, even while participants complained planning difficulties in everyday life activities. However, at W12, patients obtained longer time of planning and execution with the same number of errors, at the Zoo map test, when compared to W0. To explain this discrepancy, we could argue that many participants were penalized, not because of planning disabilities but for taking more time to plan their route. In addition, the action planning and the zoo tests do not provide a sufficiently sensitive quotation that could detect slight variations in performances. It could be better for the next groups to select other tests, such as the modified commission test (41), to obtain a more ecological assessment. In this test, participants have to perform a number of tasks in a neighborhood, taking into account time, load carrying and optimal distance constraints. In this line, the prospective VR test was able to detect improvement in maintenance and execution of action–intentions that are skills related to planning. It will also be interesting to complete the executive functioning assessment by adding inhibition and flexibility tests. Indeed, the VR program involves many steps with quick decisions to take, and when participants are planning different actions, they have to establish priorities to define their schedule. Hence, there could be automatic actions to inhibit in order to be more efficient in action planning, and also more flexible.

When considering all the positive comments the participants reported, this VR program is undoubtedly an asset for our Center for Cognitive Remediation and Rehabilitation. Despite the small number of patients who participated in the program, all the patients who completed it successfully improved, clinically and functionally. Although not demonstrating a substantial neuropsychological improvement in planning, patients showed attention, working memory, prospective, and retrospective memory benefits. Therefore, our initial hypotheses are only partially confirmed. However, the perspective for this program has been highly enhanced by these pioneering results. Patients are truly evolving in a rehabilitation pathway. They are asking to complete this experience with a cognitive remediation program, or to participate in a social cognition group in order to keep progressing in social interactions. One participant even decided to write a paper in a scientific review, on his own, to describe his unique experience (37). He related his drastic evolution in life in these terms (text translated into English):

« In this paper I want to describe my latest professional experience. Even if this page of my life had a bad ending issue, it is important for me to evolve in a constructive way for my future and to better apprehend my illness. It is by confronting myself with the world of work that I can improve my self-knowledge and paradoxically accept my handicap (…) With the help of cognitive remediation and sector teams I realized (…) that I had to think positively about the traumatic events that happened at work and to drive my own evolution in a positive and fruitful manner. (…) Now, I know more about my illness and I accept it. I know my limitations and I acquired a form of stability in life. » (37).

#### Limitations

One of the most important limitations is what has been already mentioned, namely the limited sample included in the group. The second point was the non-parallel versions of some assessments, mainly the prospective and retrospective VR tasks. Also, the participants had very diverse treatments with large variations in the dosage and compounds delivered. However, this study was assessed with patients who are institutionalized and in ecological clinical conditions, containing therefore an exemplarity of users who could visit daily a daycare therapeutic Parisian center.

## CONCLUSION

Virtual reality worlds offer great promises for the future. For the next groups, we plan to develop different versions of the VR program to induce more flexibility and to increase the training for patients by discovering different areas and new routes during their navigation, develop the complexity of scripts and maybe introduce interactive avatars. In the near future, we will introduce a whole network of public transportation (metro and bus) to enable discussions in the group around travel itineraries, as well as some unpredictable events to oblige patients to be more flexible in their routine action plans, while navigating in a non-stressful environment. We think that these points can be highly helpful for patients to manage their daily lives.

These fruitful results encourage our teams to promote serious games as real cognitive remediation methods. Hence, with larger samples, we would have opportunities to demonstrate benefits also in planning, executive functioning, or even social cognition. Also, this program could be helpful for patients with schizophrenia with strong disabilities in daily living, who stay years in residential care unit for example, or living in apartments with a strong clinician support. Thanks to this program, we

#### REFERENCES


could expect that the supportive action of the clinicians who help patients to gain independence in life will be reduced or will focus on the remaining deficits insufficiently improved by the program. More generally, these new VR worlds open avenue for innovative rehabilitation programs for patients with schizophrenia.

#### AUTHOR CONTRIBUTIONS

IA and PP supervised the project. IA was in charge of the clinical assessment. LB-H and ZP were in charge of the neuropsychological assessment and the program sessions. CD was in charge of the program sessions. EO was the engineer who conceived the virtual reality program. MS collected the data and was in charge of the statistical analysis. M-OK and all the participants already mentioned were involved in the redaction of the manuscript.

#### FUNDING

The study was done with the financial help of the CNRS Innovarc Sopsy Project and the financial help of the Janssen Pharmacological company.

environment and single-trial, goal-directed navigation. *Front Behav Neurosci* (2013) **26**(7):180. doi:10.3389/fnbeh.2013.00180


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Amado, Brénugat-Herné, Orriols, Desombre, Dos Santos, Prost, Krebs and Piolino. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Improving Facial Emotion Recognition in Schizophrenia: a Controlled Study Comparing Specific and Attentional Focused Cognitive Remediation

*Baptiste Gaudelus1 \*, Jefferson Virgile1,2, Sabrina Geliot1,3, The GAÏA/RECOS Study Team1,4,5,6† and Nicolas Franck1,7,8*

*1Service Universitaire de Réhabilitation, CL3R, Le Vinatier Hospital, Bron, France, 2Département de Réhabilitation Psycho-sociale, St Cyr au Mont d'Or Hospital, St Cyr au Mont d'Or, France, 3Centre régional de dépistage et de prise en charge des troubles psychiatriques d'origine génétique, Le Vinatier Hospital, Bron, France, 4CRISALID, Clermont de l'Oise Hospital (CHI), Clermont de l'Oise, France, 5Unité de Soins Psycho-Sociaux, HDJ S17-S18, C3RP, Sainte-Anne Hospital, Paris, France, 6C3R Grenoble, Centre référent de réhabilitation psychosociale, Alpes-Isère Hospital (CHAI), St Egrève, France, 7 Lyon 1 Claude Bernard University, Lyon, France, 8UMR 5229, Centre National de la Recherche Scientifique (CNRS), Lyon, France*

Cognitive impairments associated with schizophrenia are very frequent. They concern both neurocognition and social cognition, including facial emotion recognition. These impairments have a negative impact on the daily functioning, in particular the social and vocational rehabilitation of people with schizophrenia. Previous studies in this area clearly demonstrated the interest of cognitive remediation to improve neurocognitive and social cognitive functioning in schizophrenia. They also established clear links between facial emotion recognition skills and attentional processes. The present study compares the GAÏA s-face program (GAÏA arm), which focuses on facial emotion recognition processes, with the RECOS program (RECOS arm), a neurocognitive remediation therapy focusing on selective attention. Forty people with schizophrenia were randomly distributed between each study arm and assessed pre- (T1) and post- (T2) therapy. The single-blind assessment focused on facial emotion recognition (the main criteria), symptoms, social and subjective functioning, and neurocognitive and social cognitive performance. Both programs were conducted by nurses after a 3-day training session. The study showed a significant improvement in facial emotion recognition performance in both groups, with a significantly larger effect in the GAÏA arm. Symptoms and social functioning also improved in the GAÏA arm, and certain neurocognitive and social cognitive processes improved in both study arms. Further studies are recommended, with larger population samples and a follow-up assessing the long-term preservation of these improvements.

Keywords: Schizophrenia, social cognition, facial emotions, cognitive remediation, nursing practice

#### *Edited by:*

*Jérôme Favrod, Univesity of Applied Sciences & Arts of Western Switzerland, Switzerland*

#### *Reviewed by:*

*Frank Larøi, University of Liege, Belgium Ali Oker, Télécom ParisTech, France*

#### *\*Correspondence:*

*Baptiste Gaudelus baptiste.gaudelus@ch-le-vinatier.fr*

*† The GAÏA/RECOS Study Team are: Dupuis M., Hochard C., Josserand AC., Koubichkine A., Lambert T., Perez M., Rouyre B., Scherding P. (therapists); Bralet MC., Demily C., Launay C., Gouache B. (medical investigators); Duboc C., Dubrulle A., Farhat SL., Fourt A., Fluttaz C., Peyroux E., Todd A. (neuropsychologist assessors).*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 24 February 2016 Accepted: 03 June 2016 Published: 21 June 2016*

#### *Citation:*

*Gaudelus B, Virgile J, Geliot S, The GAÏA/RECOS Study Team and Franck N (2016) Improving Facial Emotion Recognition in Schizophrenia: a Controlled Study Comparing Specific and Attentional Focused Cognitive Remediation. Front. Psychiatry 7:105. doi: 10.3389/fpsyt.2016.00105*

## INTRODUCTION

Cognitive impairments associated with schizophrenia affect about four out of five patients. They significantly impact functional recovery, and the social and vocational rehabilitation of people with schizophrenia (1–3). They may also impact all cognitive processes, i.e., neurocognition (memory, attention, executive functions, and processing speed) and social cognition (4–6).

Social cognition can be defined as the ability to construct mental representations about others, oneself and one's relationships to others (5), and covers several processes. Five components of social cognition are mainly impaired in schizophrenia (6–9): theory of mind, attributional style, social perception, social knowledge, and the perception of emotions on faces and in prosody (i.e., the emotional information provided by variations in pitch, loudness, and speech flow).

The ability to understand the emotional state of others is, therefore, a key aspect of social cognition. Its function is to help adapt one's behavior according to the "signs" other people give off. The misinterpretation of other people's emotions or poor communication with one's own emotions negatively impacts relationships, and leads to difficulty in joining social groups. Impaired emotion perception and expression in schizophrenia have been described since Bleuler (1911) (10), and facial emotion recognition deficit is now clearly identified (11). Various studies showed that this deficit was a trait factor (12) and was observed with less intensity in family members not affected by the disease (13). Although facial affect recognition deficit is not specific to schizophrenia, its frequency and intensity are higher in schizophrenia than in other psychiatric disorders. Some authors suggested that facial affect recognition deficits might be a marker of vulnerability to the disease (14).

The origin of facial emotion processing deficits associated with schizophrenia is still not completely understood but is thought to result from the alteration of various processes.

Several authors (15–17) reported correlations between attention deficit and facial emotion recognition deficit. These correlations are supported by many neuroimaging studies that showing hypoactivity in the frontal cortex in patients with schizophrenia (18); the frontal cortex controls both executive and attentional skills, and is also involved in facial emotion recognition (19).

Other studies focused on patterns of visual attention to faces (20, 21): compared to healthy controls, people with schizophrenia showed reduced number and range of visual saccades, increased duration of fixation, and reduced attention to relevant features during emotion recognition tests.

Facial emotion recognition deficit is also frequently correlated with other difficulties in emotional information processing, especially the production of facial emotions (22, 23), and the processing of affective prosody (24). This may be caused by a deficit in a specific cognitive emotional processing module. This hypothesis is also supported by neuroimaging studies showing that the same brain regions are partly involved in the processing of facial and prosodic emotional information, most notably the upper right temporal cortex, the limbic system, and the prefrontal cortex (25, 26).

Recent research has explored the hypothesis of a gap in the processing of all facial information, including non-emotional information. In people with schizophrenia, such deficit may impair the ability to process configural information (i.e., the physical characteristics making up a face and the relations between its components), which can be common to all faces (i.e., first-order configural information) or specific to one face (i.e., second-order configural information). Maurer et al. (27) proposed a distinction between two processing methods operating at the same time: the holistic processing, in which facial information is processed as a whole, and the componential processing that involves a featurebased analysis relying on second-order configural information processing (28). The impact of impaired configural information processing on facial emotion recognition deficit in people with schizophrenia is still poorly understood. However, the differences reported by various studies between patients with schizophrenia and healthy controls (14, 29) call for further investigation in this particular area.

Although the processes underlying social cognition disorders are not completely understood, cognitive remediation programs targeting social cognition or some of its components (especially theory of mind and the perception of facial emotions) have been recently developed (30–32). Cognitive remediation seems to be the most promising intervention to improve social cognition abilities and especially facial emotion recognition (32), whereas antipsychotic treatments have little effect on such processes (33). Most social cognitive remediation programs are group oriented and based on "standardized" exercises, which may be detrimental to patients. It also makes it more difficult to adapt strategies to each participant's cognitive and clinical profile. Furthermore, groups do not facilitate individualized homework tasks, which are designed to help transfer therapeutic strategies into daily life.

In the field of neurocognition, recently, several computerassisted cognitive remediation programs have been developed (34–37). Computer-assisted programs give immediate feedback on the participant's performance and adjust both the difficulty of the exercises and reinforcement methods (38). Furthermore, it seems that prolonged computer stimulation encourages neural plasticity and the learning of new coping strategies, which are central to cognitive remediation therapy (39). Finally, individual cognitive remediation provides exercises adapted to the participant's cognitive profile and functional goals (40), and takes into account the metacognitive difficulties associated with schizophrenia (41–43).

In the field of social cognition, the use of computer-assisted methods seems just as relevant, especially to improve emotion recognition. The advantage of this technology lies in the control of all the processes at play in social interactions, and the development of progressive training. Furthermore, it provides a secure environment where the person can practice without risking negative repercussions in real life or generating much anxiety. Using a virtual environment also limits the bias of attributing emotions to others according to the participant's own emotions. However, the presence of a collaborative therapist remains essential, especially to encourage the transfer and adaptation of strategies into real-life situations (40). Furthermore, the role of the therapist also includes encouraging the participant formulate concrete objectives (increasing, therefore, motivation), and developing individualized exercises to achieve these goals throughout therapy; these exercises will then be put into practice during and outside the sessions.

According to these data, both neurocognitive remediation, focusing on attentional processes, and social cognition remediation, focusing on emotional perception, should increase facial affect recognition in people with schizophrenia. From a clinical perspective, it would be interesting to compare the effects of these strategies on symptoms, and cognitive and social functioning.

The GAÏA s-face (Schizophrenia- Facial Affects recognition Cognitive Enhancement) program (44, 45) is an individual, computer-assisted cognitive remediation therapy focusing on facial emotion recognition. The present controlled efficacy study compares the GAÏA s-face program with the RECOS program, an individual, computer-assisted neurocognitive remediation therapy focusing on attentional processes (46, 47).

#### METHODS

#### Study Design

Clinically stable people with schizophrenia, according to the DSM-IV-TR criteria (48), were recruited in four psychiatric centers in France. They were randomly distributed between two active treatment arms, namely GAÏA and RECOS. The assessments were blind to group allocation and carried out at baseline (T1 = week 0) and post-treatment (T2 = week 11).

The main treatment outcome measure was the TREF (facial emotion recognition test) total score (49).

Secondary measures included clinical ratings, neurocognitive and social cognitive measures, and psychosocial evaluation.

The study was approved by local ethics authority (CPP Lyon Sud–Est VI, project no. AU 940), and declared to the national authority (ANSM: project no. 2011-A00793-38) and on clinicaltrials.gov (project ID: NCT01607424).

After a complete description of the study objectives and procedures, each participant signed a written informed consent.

The two arms of this parallel-group randomized clinical trial consisted of active and comparable interventions in terms of number of sessions, but also in terms of therapist roles and material used. This constitutes an ethical choice, since each participant was provided with an intervention designed to improve his/her cognitive performance and social and professional integration (3, 7, 50). The study design also makes it possible to observe and compare the specific effects of each program on the various cognitive and functional outcomes.

Both interventions were carried out by nurses (except one occupational therapist in the RECOS arm) who all attended a specific 3-day training session. Each nurse was involved in a single arm of the study to ensure equal motivation and investment among therapists for both programs. The possibility to assess the nurses' skills in providing effective individual cognitive remediation interventions might represent a cost-efficient solution to implement cognitive remediation in the routine care of schizophrenia, for the benefit of a larger number of people.

#### Participants

Forty in- and out-patients with stable schizophrenia were included in the trial (see **Figure 1**). Inclusion criteria were as follows: DSM-IV-TR criteria for schizophrenia (48), with clinical stability confirmed after examination by the psychiatrist-investigator from the recruiting department; age 18–45; impairment of facial affect recognition confirmed by a TREF test score below 69% (SD = −1) (49); native French speakers.

Exclusion criteria included history of neurological illness or trauma; use of somatic medication with cerebral or psychological impact; alcohol or drug addiction (except tobacco); and other

cognitive remediation therapy during the period of inclusion in the study.

At T1, there was no significant difference between the GAÏA and RECOS groups in age, gender, symptom intensity measured with the Positive And Negative Symptoms Scale [PANSS (51)], medication doses converted into chlorpromazine equivalent (52), or facial emotion recognition skills measured with the TREF (see **Table 1**).

#### Interventions

The participants were assessed before (T1) and after (T2) intervention. However, both RECOS and GAÏA s-face programs recommend a 6-month follow-up phase, with a 30-min meeting every 2 weeks between the participant and the therapist (see **Figure 2**). This meeting, during which no cognitive remediation



exercises are performed, is aimed at reinforcing the functional outcomes of cognitive remediation (45, 53).

#### RECOS (Control Arm)

RECOS is an individual neurocognitive remediation therapy with proven effectiveness (35). In its usual form (53), it targets one to three out of six neurocognitive functions (verbal memory, working memory, executive functions, memory and visuo–spatial attention, selective attention, and processing speed), according to the functional implications of their impairments and the participant's cognitive and clinical profile. The RECOS program comes after a comprehensive assessment of cognitive functions, clinical symptoms, and social functioning. The therapy consists of three 1-h sessions per week, always alternating paper-and-pencil sessions, computer sessions, and home (without therapist) exercises designed to encourage functional benefits.

In the present study, all the patients working with RECOS were trained on selective attention, since it was demonstrated that attentional deficits were linked to facial emotion processing deficits (15–17). All the participants randomly assigned to the RECOS group attended 30 sessions during a 10-week-long treatment.

#### GAÏA s–Face (Experimental Arm)

GAÏA s-face (44, 45) is an individual, social cognitive remediation program focusing on facial emotion processing. The GAÏA program comes after a comprehensive assessment of neurocognition,

social cognition, clinical symptoms, and social functioning. The therapy is spread over 30 sessions: three 1-h sessions per week for 10 weeks. One of these sessions (called the transfer session) is performed without the therapist in the participant's everyday environment. The GAÏA program is divided into three phases: (1) an exercise phase with photographs, aiming to develop strategies to recognize and discriminate joy, anger, and sadness. (2) A computerized exercise phase using videos, to adapt the strategies developed on photographs to dynamic situations with five levels of increasing difficulty. (3) A generalization phase, consisting of at least five therapist–participant sessions and further computerized exercises determined according to the participant's profile and requests, to recognize and discriminate other basic emotions (fear, disgust, and contempt) and to work on complex emotions.

#### Assessment

#### Main Outcome Measure

The main outcome was measured with the TREF (Facial Emotions Recognition Task) total score (49).

The TREF consists of 54 photographs of six male and female faces expressing disgust, contempt, fear, anger, sadness, and joy. Each emotion is represented with color photos of four different models with nine intensity levels from 20 to 100%. The different emotional expression intensities were obtained using a morphing technique blending the neutral and maximal expression photographs for each emotion and model. The 54 photographs are organized in six lists. The lists are presented in a randomized order.

For each photograph, the task is to select the right emotion from a six-label list: fear, sadness, contempt, anger, joy, and disgust. Each photograph is presented for 10 s; there is no time limit to answer.

The total score is the percentage of correctly labeled emotions. In a previous study (49), we demonstrated that the average score of correct answers in a first population of control subjects was 76.45% (SD = 7.44).

Participants enrolled in this study all presented facial emotion recognition impairment, confirmed by a TREF score below or equal to 69% (SD = 1 below the mean of the reference population).

#### Secondary Outcome Measures

#### *Clinical Assessment*

Symptoms were assessed with the PANSS (51). Total, positive subscale, and negative subscale scores were used.

Delusional beliefs were assessed with the 21-item Peters et al.'s Delusions Inventory (PDI21) (54). Total scores were used.

#### *Functional and Subjective Assessment*

Insight was assessed with the Insight Scale (IS) (55). Total scores were used.

Self-esteem was assessed with the Self-Esteem Rating Scale (SERS) (56, 57). Total scores were used.

Social functioning was assessed with the social autonomy scale ("Echelle d'Autonomie Sociale"-EAS) (58). Total score, "social relatedness (SR)," and "relationship with the environment" (RE) sub-scores were used.

#### *Cognitive Assessment*

A comprehensive neuropsychological assessment of neurocognition and social cognition (see **Table 2**) was performed.

#### Statistical Analysis

Analyses of variance and *t*-tests were used to investigate group differences. When results were significant (*p*-value inferior to 5%), Fisher's least significant difference (LSD) was used. Data were analyzed using statistica (v.10). If significant differences existed between the groups at T1, *t*-tests were used on the change between T1 and T2, in order to compare evolution of measures.

To assess evolution on neuropsychological tests, statistical analyzes were performed for each cognitive process (visuospatial attention and memory, working memory, executive functions, selective attention, and processing speed). The scores were converted into standard notation in order to obtain an overall score per process.

The treatment effect magnitude was calculated with Cohen's *d* effect size between T2 and T1 in each group.

#### Hypothesis

A significant improvement in TREF scores was expected for GAÏA participants compared to the RECOS arm at T2. However, an improvement in facial emotion recognition performance was expected in both study arms at T2 compared to T1, since GAIA specifically targets emotion recognition and RECOS focuses on the role of attentional processes in facial emotion recognition (15, 17).

#### Table 2 | Comprehensive cognitive assessment.


*TAP: Test of Attentional Performance; BADS: Behavioral Assessment of the Disexecutive syndrome.*

Various studies have shown the role of social cognition deficits in the generation and maintenance of the positive and negative symptoms of schizophrenia (15, 33, 73). Thus, decreased PANSS scores were expected in the GAIA arm at T2 compared to T1. According to the literature (34), this effect was not expected for RECOS participants.

No special assumption was made on the effects of each intervention on delusional beliefs measured with the PDI21.

Social cognition has been described as a mediator between neurocognition and social functioning (33). According to this model, an improving effect of the GAIA s-face intervention was expected at T2 on social autonomy in comparison with the RECOS program.

In a previous study (34), the RECOS program showed a positive effect on self-esteem. This effect may be due to metacognitive skills and verbalization techniques raising the participants' awareness of their own resources, and to the problem-solving strategies they implement to manage their daily life. A large part of this methodology has been implemented in the GAÏA s-face program. Thus, an improvement of the participants' self-esteem was expected at T2 in comparison with T1 in both study arms.

The literature describes neurocognition and social cognition as separate processes (74, 75). No effect was, therefore, expected on neurocognitive assessment in the GAÏA arm. By contrast, RECOS participants receiving cognitive remediation targeting selective attention were expected to show improved performances at T2 in tasks targeting this process compared to GAÏA participants.

Social cognition is defined as a set of separate processes (5). A specific effect of the GAÏA s-face intervention on every test measuring emotional processing was expected at T2 vs. T1. No effect of the RECOS program was expected on social cognition tests, apart from facial emotion recognition, as described above.

#### RESULTS

Results are presented in **Tables 3**–**5**.

#### Main Outcome

GAÏA participants showed an average increase of 16.21% on the TREF total score at T2 compared to T1 (*p* < 0.0001), whereas RECOS participants showed an average increase of 8.43% (*p* < 0.009) for the same period. Both groups showed significantly improved T2–T1 TREF means total score (see **Figure 3**; **Table 3**).

Although there was no difference between the two groups at T1 (*p* = 0.71), a significant difference in favor of the GAÏA arm was found when comparing the results at T2 (*p* = 0.005). GAÏA participants showed significantly improved TREF scores in comparison with RECOS participants.

Each group showed a strong effect size, with a greater effect in the GAÏA arm (*d* = 2.41) than in the RECOS arm (*d* = 0.98).

#### Secondary Outcomes

#### Clinical Measures

No effect was found at T2 on PDI21 measures of delusional beliefs in either group.

The GAÏA group showed significantly decreased symptoms on the T2–T1 mean (*p* < 0.001), measured with the PANSS total score. This decrease was also significant on the positive subscale score (*p* < 0.01) and the negative subscale score (*p* < 0.001).

No significant effect was found on the T2–T1 mean PANSS total score (*p* = 0.66), and positive (*p* = 0.08) and negative (*p* = 0.63) subscale scores in the RECOS group (see **Table 4**).

While no significant difference was observed between the two groups in PANSS scores and sub-scores after randomization (**Table 1**), a significant difference in the negative sub-score appeared at T1, when taking into account drop-out participants (see **Table 4** and **Figure 4**). Nevertheless, a significant difference in favor of the GAÏA arm was found when comparing the results at T2 on the PANSS total score (*p* < 0.001). No significant effect was found on the positive subscale between the interventions. The difference in favor of the GAÏA arm on the negative subscale is even more significant at T2 (from *p* < 0.01 to *p* < 0.0001).

#### Psychosocial and Functional Outcomes

No effect was found at T2 on measures of insight with the IS, nor on self-esteem measured with the SERS, in either group.

Measures of social autonomy with the *EAS* significantly improved (i.e., decreased scores were observed) in the GAÏA arm on the T2–T1 mean (see **Table 4**; **Figure 5**). This improvement was found on the total score (*p* < 0.01), "SR" subscore (*p* < 0.04), and ''RE'' subscore (*p* < 0.01). No effect was found on pre–post measure in the RECOS arm.

A significant difference in favor of the GAÏA arm was found when comparing the results at T2 on the EAS total score and SR sub-score (respectively, *p* < 0.0001 and *p* < 0.02 – see **Table 4**). A significant difference in favor of the GAÏA arm was observed on "RE" score at T1 (*p* < 0.003); however, the difference was even more significant at T2 (*p* < 0.0001).

#### Cognitive Functioning Measures

#### *Neurocognitive Assessment*

Social cognitive outcomes are reported in **Table 5.**

A group effect was found in selective attention on the T2–T1 mean. The participants of the RECOS arm showed significantly improved performances at T2 (*p* < 0.0001). No effect was found in the GAÏA arm (*p* = 0.4).

Despite a difference in favor of the GAÏA arm at T1 (**Table 5**), the significant difference found at T2 was in favor of the RECOS arm.

Both groups significantly improved in processing speed, with a larger effect in the RECOS group in comparison with the GAÏA group (respectively, *p* < 0.0001 and *p* < 0.03). No effect was found in group comparison.

Participants in both groups showed significantly improved performance in visuospatial attention and memory at T2 in comparison with T1 (see **Table 5**), but no effect was found in group comparison.

A group effect was found in working memory at T2. GAÏA participants showed significantly improved performance at T2 in comparison with T1 (*p* < 0.01) and with the RECOS arm (*p* < 0.005).

No effect was observed in either group during executive function assessment on T2 vs. T1 (GAÏA *p* < 0.08; RECOS *p* < 0.75).

#### Table 3 | Evolution in social cognition assessment.


*V-SIR, Versailles-Situational Intention Reading test; RMET, Reading the Mind in the Eyes Test; LEAS, Levels of Emotional Awareness Scale; AIHQ, Ambiguous Intentions Hostility Questionnaire; QCAE, Questionnaire of Cognitive and Affective Empathy; (ns), no significant.*

*Bold values indicate the significant result; \*p* < *0.05; \*\*p* < *0.01; \*\*\*p* < *0.001*.

#### Table 4 | Evolution in clinical, psychosocial, and functional assessment.


*RE, relationship with the environment; SR, social relatedness.*

*Bold values indicate the significant result; \*p* < *0.05; \*\*p* < *0.01; \*\*\*p* < *0.001*.

#### Table 5 | Evolution in neurocognitive assessment.


*Bold values indicate the significant result; \*p* < *0.05; \*\*p* < *0.01; \*\*\*p* < *0.001*.

Nevertheless, a significant difference in favor of the GAÏA arm was found at T2 in group comparison (*p* < 0.02).

#### *Social Cognition Assessment*

Social cognitive outcomes are reported in **Table 3**.

Emotion perception was measured with the TREF (main criteria) and the Levels of Emotional Awareness Scale (LEAS).

A group effect measured with the LEAS was found on T2 vs. T1 for GAÏA participants, whereas no difference was found in the RECOS arm. No significant difference was found in group comparison at T2, though a difference in favor of the RECOS arm existed at T1.

No significant difference was observed in either group on the T2–T1 mean in attributional style measured with the

Ambiguous Intentions Hostility Questionnaire, nor in group comparison at T2.

The assessment of theory of mind included several tasks. The Hinting task scores of RECOS participants were significantly higher at T2 than T1, whereas no effect was found in the GAÏA group. A group effect was found at T2 in favor of the RECOS arm (*p* < 0.05).

No effect was found in either group on the Versailles-Situational Intention Reading test, nor on the Reading the Mind in the Eyes test.

Empathy was measured with the Questionnaire of Cognitive and Affective Empathy. A significant increase was found between T2 and T1 in the cognitive empathy sub-score for the RECOS group (*p* < 0.02). No effect was observed in either group on the affective empathy sub-score nor in group comparison.

#### DISCUSSION

The main results include improved facial emotion recognition performance in both groups, reduced symptoms and improved social functioning in the GAÏA group, and the improvement of some neurocognitive and social cognitive processes in both study arms.

The significant improvement in TREF results for the GAÏA arm compared to the RECOS arm is consistent with the data from the literature, since Kurtz and Richardson (32) already showed that facial emotion recognition could be improved using a specific remediation.

The improvement for RECOS participants was an expected effect, based on previously demonstrated links between selective attention and facial emotion recognition skills (15–17).

Various hypotheses may explain the significantly greater improvement of the GAÏA group in facial emotion recognition skills compared to the RECOS group.

The GAÏA program learns participants to focus on relevant facial features to recognize emotions; this specific training probably helped them directly reuse the strategies learned during cognitive remediation to perform the TREF test. As for the RECOS group, the ability to select and focus on relevant facial features may be considered as a transfer and generalization effect of the strategies they practiced during cognitive remediation.

Previous research has shown that modifications in the pattern of visual attention to faces were associated with facial emotion recognition deficit (20, 21). The GAÏA program probably has a positive effect on the pattern of visual attention to faces, supported by an (unexpected) positive effect on visuospatial abilities on pre–post analysis. However, further studies should explore this hypothesis.

The significant improvement of schizophrenia symptoms measured with the PANSS at T2 in the GAÏA arm was expected. Various studies have shown correlations between social cognition, especially facial emotion recognition, and both positive (15, 73) and negative (33) symptoms.

However, links between symptoms and social cognition are not straightforward (33), since reduced symptoms after GAÏA s-face therapy may be a transfer and generalization effect of the benefits of cognitive remediation.

Symptoms did not improve at T2 in the RECOS arm, hypothetically because of a lack of connection between neurocognition and positive symptoms (76), unlike social cognition that may act as a mediator between neurocognition and negative symptoms (33). The improvement of negative symptoms after neurocognitive remediation, thus, constitutes a higher transfer level than facial emotion recognition.

Nevertheless, it will be important to observe whether these benefits are sustainable and whether a generalization effect occurs in the RECOS arm after follow-up, as it was shown in a previous study (34).

A significant improvement in social autonomy was measured with the social autonomy scale (EAS) (58) for GAÏA participants. Such improvement is consistent with the data from the literature, which describe social cognition as a mediator between neurocognitive and social functioning (77, 78). Couture et al. specifically showed correlations between the perception of emotions and community functioning, "social behavior in the milieu" and "social skills," which seems to match the improvement in the ''relationship with the environment'' (RE) and ''social relatedness'' (SR) subdivisions of the EAS. We suggest that this effect was encouraged by individual therapy, which helped introduce a metacognitive dimension to the sessions with the therapist, and adapt exercises to the motivations and clinical profile of participants during the transfer sessions. Furthermore, the structure of the GAÏA s-face program, offering computer exercises adjusted to the participant's everyday environment, may have facilitated the transfer and generalization of benefits to everyday life.

This study did not show any improvement in social functioning for RECOS participants, probably because transfer between neurocognition and social functioning is less straightforward than between social cognition and social functioning. However, the program is open to incorporate a metacognitive dimension and individualization, to facilitate the transfer of strategies to daily tasks.

It will be important for future studies to observe how social functioning evolves in the long term for both programs, whether the encouraging results of the GAÏA arm are long-lasting, and whether functional benefits appear later for the RECOS arm, since the generalization of cognitive benefits requires a certain amount of time.

An improvement in self-esteem was expected in both arms, but this study did not evidence this improvement. RECOS had been proven effective in this area in a previous study (35) related to therapeutic strategies, particularly the research and generation of the participant's own strategies and the positive reinforcement provided by the therapist. These strategies were introduced into the GAÏA s-face program. It will be important for future studies to perform the same measures with more experienced therapists and assess subjective benefits for the participants as well.

This study also highlighted the benefits of cognitive remediation with the RECOS program. The RECOS group underwent a training program focusing on selective attention (in its usual version, the program provides targeted cognitive remediation, adapted to the participant's cognitive profile) and results at T2 showed a significant improvement in selective attention, processing speed, and visuospatial attention tasks.

The GAÏA group also showed positive effects on neuropsychological assessment at T2. Besides visuospatial functions, improved working memory was observed after the therapy. Several hypotheses may explain this unexpected effect. Some neuropsychological tests measuring working memory skills used visual modality and, therefore, also required visuospatial memory skills. Improvement in memory and visuospatial attention for GAÏA participants may underlie improvement in working memory.

Furthermore, the GAÏA s-face therapy requires that participants undertake verbal and visual learning, especially during photograph exercises and generalization phases, where they have to assimilate cues to recognize facial expressions of basic emotions. Therapists probably developed strategies with participants to facilitate these acquisitions and participants may have reused such strategies during T2 working memory tests. However, it will be necessary to repeat these results in future studies designed to test these hypotheses.

The assessment of social cognition evolution at T2 in the GAÏA arm showed a significant improvement in the participants' performance in tests measuring emotional perception. Because of the specific nature of the LEAS (66), the significantly increased scores in this test could be a first step toward improved empathy skills; however, no improvement was found in this area.

Nonetheless, this targeted increase seems in accordance with the literature, which defines social cognition as a complex process with partially independent components (6, 79).

A significant improvement was observed in one of the theory of mind tests [the Hinting task (72)] and in cognitive empathy measured with the QCAE for RECOS participants. The Hinting task assesses the ability to understand implicit components within sentences from daily life situations; the QCAE (cognitive subscore) measures a person's ability to consider a situation from someone else's perspective. Such improvement may be partly explained by improved attention skills, enabling participants to better focus on situations rather than on their own thoughts or feelings, and better detect intentions in verbal messages.

Limitations of the present study mainly include small sample size and lack of evaluation of the long-term preservation of the benefits.

Further studies are needed to confirm the positive outcomes of the present study and verify their sustainability.

The potential impact on the subjective quality of life of the participants should be assessed in future studies, using standardized assessment and real-life social functioning criteria.

Finally, further studies will have to explore the effects of the GAÏA s-face program on every social cognitive process (social perception and social knowledge were not assessed in this study). Comprehensive social cognition test batteries are currently under development (80); they should help better understand the interactions between each process.

#### CONCLUSION

Although the understanding of the cognitive mechanisms improved by the GAÏA program is still incomplete, the results of this study show the applicability and effectiveness of the program to improve facial emotion recognition skills, reduce positive and negative symptoms in schizophrenia, and improve social functioning.

Positive results were obtained for both programs in the main criteria. Facial emotion recognition skills improved in both arms, selective attention was significantly improved in the RECOS group, which received specific training targeting these processes, and the GAÏA program showed positive effects on symptoms and social functioning. These results were obtained with cognitive remediation provided by nurses after a 3-day training session. This is an interesting prospect for the spread of cognitive remediation and its potential inclusion into the routine care of schizophrenia alongside drug treatment, psychoeducation, social skills training, and psychotherapy.

Facial emotion processing is also impaired in other disorders, so it would be interesting for future studies to explore whether the GAÏA s-face program is effective in people with diseases, such as autism spectrum disorders or 22q11.2 deletion syndrome.

#### AUTHOR CONTRIBUTIONS

BG: nurse; GAÏA s-face program conceptor; contibution in study design; and first author. JV: neuropsychologist; cognitive assessment; contibution in study design; DATA analysis; substantial contribution. SG: nurse; symptoms, social functioning and subjective assessment; contribution in study design; substantial contribution. The GAÏA/RECOS study team: Dupuis M., Hochard C., Josserand AC., Koubichkine A., Lambert T., Perez M., Rouyre B.: nurses; therapists; critical review. Scherding P.: occupational therapist; therapist; critical review. Bralet MC., Demily C., Launay C., Gouache B: medical investigators; critical review. Duboc C., Dubrulle A., Farhat SL., Fourt A., Fluttaz C., Peyroux E., Todd A.: neuropsychologist assessors; critical review. Prof. NF (last author): first investigator; substantial contribution in GAÏA s-face program conception; substantial contribution in study design; substantial contribution in writing the article.

#### ACKNOWLEDGMENTS

The authors wish to thank the CSR (scientific research board) of Le Vinatier Hospital for financially supporting this study, D.Gros (Tutoractiv') for his work on the GAÏA s-face software, P.Vianin for his agreement in using the RECOS program and his help in designing the GAÏA s-face program, JY.Baudouin for his help in designing the GAÏA s-face program, G. Collot and F.Apruzzese for editing the manuscript, I. Amado for her final review.

#### REFERENCES


## FUNDING

CSR (Scientific Research Board) of Le vinatier Hospital, Lyon, France.


emotion. *J Pers Assess* (1990) 55:124–34. doi:10.1080/00223891.1990.9 674052


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Gaudelus, Virgile, Geliot, The GAÏA/RECOS Study Team and Franck. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Computerized Exercises to Promote Transfer of Cognitive Skills to Everyday Life

#### *Pascal Vianin\**

*Département de Psychiatrie, Centre Hospitalier Universitaire Vaudois (DP-CHUV), Lausanne, Switzerland*

In recent years, computerized and non-computerized cognitive remediation programs have been designed for both individual and group settings. We believe, however, that a common misconception lies in considering the efficiency of a cognitive remediation therapy as resulting from the sole use of a computer. This omits that metacognitive skills need also to be trained throughout the remediation phase. RECOS is a theory-based therapeutic approach designed to promote the transfer of cognitive skills to functional improvements. It involves working with one person at a time using both paper/ pencil tasks and a set of interactive computer exercises. Paper/pencil exercises are used to promote problem-solving techniques and to help patients to find appropriate suitable strategies. During the following computerized 1-h session, therapists guide participants to the procedural dimension of the action, which refers to knowledge about doing things and relies on retrospective introspection. We assume that each patient has a rich and underestimated procedural knowledge he/she is not aware of. By providing complex and interactive environments, computerized exercises are recommended to bring this knowledge to light. When strategies used by the participant become conscious, conditional knowledge determines when and why to use them in real-life situations.

Keywords: cognitive remediation, computer, metacognition, procedural knowledge, transfer, RECOS therapy

Individuals who suffer from mental illness including schizophrenia, bipolar disorder, or depressive disorder often experience cognitive deficits. Attention, processing speed, executive functions, or memory deficits may lead to severe disability in everyday life. Therefore, improving cognitive abilities is essential for the well being of the patients. During the last decades, cognitive remediation was

Cognitive remediation programs alone have shown moderate improvements on cognitive and functional outcomes. With the objective of improving the beneficial effects of cognitive training, meta-analyses aimed to determine what factors moderate outcome to cognitive remediation therapy. Most of them identified treatment variables. For example, they showed that combining cognitive remediation with a psychiatric rehabilitation program (1, 2) or linking the remediation stage with

#### *Edited by:*

*Yasser Khazaal, Geneva University Hospitals, Switzerland*

#### *Reviewed by:*

*Mª Angeles Gomez Martínez, Universidad Pontificia de Salamanca, Spain Pawel Izdebski, Kazimierz Wielki University, Poland*

#### *\*Correspondence:*

*Pascal Vianin pascal.vianin@chuv.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 12 February 2016 Accepted: 24 March 2016 Published: 15 April 2016*

#### *Citation:*

*Vianin P (2016) Computerized Exercises to Promote Transfer of Cognitive Skills to Everyday Life. Front. Psychiatry 7:56. doi: 10.3389/fpsyt.2016.00056*

real-life situations (3) increased the effectiveness of these programs.

INTRODUCTION

Several studies show, however, that the way cognitive performance is improved by cognitive remediation therapy is different according to the patient's neuropsychological profile (4, 5). For

developed first for traumatic brain injury and later for mental disorders.

that reason, we believe that many programs lack specificity, the patients being generally confronted with similar executive, attention, or memory exercises. Given the heterogeneity of the cognitive profiles of individuals with schizophrenia, remediation should be adjusted to each individual's needs. Therefore, the theoretical foundation of RECOS emphasizes personalization of the treatment by providing specific and targeted cognitive training modules.

Several cognitive remediation programs aim at improving neurocognitive functions (e.g., CRT, NEAR, and RECOS), whereas other interventions target social cognition (e.g., TAR, SCIT, and TomRemed) in severe mental diseases. A further distinction can be made between cognitive remediation techniques relying on paper and pencil support and those that are computer assisted. In a recent review, Grynszpan et al. (6) summarize the advantages of computer-assisted cognitive remediation: unlimited training possibilities, automatic reinforcement, multisensory presentation, objective recording of performance, standardized training tasks, intrinsic motivation, etc.

However, we do not believe that the sole use of a computer may predict the success of the cognitive remediation therapy. We assume that metacognitive skills are more important to consider than the media used for improving cognition in real-world settings. Improving knowledge about his/her own cognition and the way strategies can be used for everyday functioning seems to be essential for recovery. In our view, cognitive remediation therapy should be based on a functional and metacognitive approach. By providing ecological and multisensory strategies, computer activities offer opportunities for improving not only cognitive functioning but also metacognitive skills. The present paper aims to show how computerized exercises used in RECOS therapy should promote the transfer of cognitive skills to functional improvements.

#### RECOS THERAPY

RECOS – *COgnitive REmediation for Schizophrenia or other related disorders* – is an individualized cognitive rehabilitation treatment to take into account the cognitive heterogeneity characterizing this disorder (7). It includes training in six of the most highly altered areas of cognition: verbal memory, visuospatial attention and memory, working memory, selective attention, executive functions, and processing speed. Remediation phase is preceded by a comprehensive neuropsychological assessment. Before beginning cognitive training, functional consequences of cognitive troubles are evaluated with each patient using qualitative criteria. To facilitate generalization of cognitive gains to everyday life, interventions are aimed at concrete goals defined according to the patients' difficulties and discussed regularly throughout the therapy. At the end of the remediation stage, participants are re-evaluated by using a similar neuropsychological battery. In order to explore the effectiveness of the RECOS therapy, a preliminary study showed that effect sizes were more important for each cognitive function when they have been trained rather than untrained (8), suggesting that matching treatment to each individual's level of cognitive functioning is essential for cognitive enhancement. In a randomized study, Franck et al. (9) found that participants to RECOS training improved not only their cognitive performance but also their clinical and social functioning.

RECOS exercises were designed by Scientific Brain Training Company (SBT, Villeurbanne, France – http://www.sbt.fr/) and adapted by our team for specific use in schizophrenia. They were designed to be engaging and similar to everyday situations in order to facilitate generalization to everyday life skills and to enhance motivation. The remediation phase includes 28 1-h twice-weekly sessions with the therapist and 14 sessions completed at home (42 h in total). Each week, participants take part successively in the paper/pencil and computer sessions with the therapist and are asked to do exercises at home.

The first session of the week is dedicated to paper/pencil exercises. Each exercise is chosen according to the cognitive abilities solicited by the individual objective of the participant. Problem-solving strategies are mainly employed during paper/ pencil sessions. The problem-solving cycle begins by identifying and defining the problem. During the next step, participants are asked to suggest a large number of strategies to solve the task. After a discussion with the therapist, the apparently effective solutions are then selected. After this selection, it is important to evaluate the results to determine the best possible solution to the problem. At the end of the paper/pencil session, the following home work is decided, aiming to transfer the learned solution in real-life situations. One week later, the new paper/pencil session begins with a discussion of the exercises made at home. During the whole cognitive remediation program, therapists have to make sure that participants become more and more efficient to transfer the acquired skills to their daily life.

The second weekly session with the therapist is dedicated to computer exercises. Each exercise depicts a daily life situation, such as preparing a cocktail or the storage of supplies. The first two or three levels of each exercise are rather easy, so that the performance is positively reinforced right from the start. To move up a level, it is necessary to achieve a 100% score at this same exercise. Thus, exercises are adapted to the patients' abilities and help maintain patients' motivation. Moreover, results are recorded and automated feedback allows patients to observe their progression permanently from the beginning of the program. It appears that with computer-assisted learning, participants become more attentive to the task. As computers provide multisensory stimulation and personalization of activities, they allow the therapist to observe the participants in a quite similar way they would be in a real environment. Unexpected and non-conscious strategies intervene during the computer exercises, giving access to procedural knowledge.

The role of the therapist is quite different in paper/pencil as compared to computer exercises. The therapist is in front of the participant during the paper/pencil whole session, the attention of both being devoted to the exercise. In order to stimulate problem-solving strategies, the therapist encourages the participant to generate and experiment different kinds of solutions. On the contrary, the participant is fully focused on the exercise during the computer session. The therapist is sited behind him/ her and does not appear in his/her field of vision. The therapist precisely observes the way the participant is solving the problem (by counting with his/her fingers, looking at the bottom left

corner of the screen, moving impulsively his/her mouse, etc.). All this information is useful for the therapist to identify the procedural knowledge of the participant. *Explicitation* interview will complete this information after the end of the exercise (see below).

## TRAINING METACOGNITIVE SKILLS

Although the concept of *metacognition* is frequently used in literature on cognitive remediation, it has been frequently neglected as an object of treatment. During training sessions, metacognitive skills are essential because they enable participants to manage their cognitive skills better and to identify weaknesses that can be corrected by developing new strategies. Three kinds of metacognitive knowledge are targeted during the phase of training.

#### Metacognitive Knowledge

Metacognitive knowledge refers to knowledge not only about one's own cognitive skills but also about cognitive functions in general, and the way cognitive skills may help participants in their daily life. During the first meeting with their therapist, they indicate the main cognitive difficulties they have recently identified. After the following neurocognitive assessment, standardized results are discussed with them. Confronting these results with their initial complaints generally allows them to become aware not only of their difficulties but also of their resources. Throughout the program, participants are then encouraged to identify cognitive processes engaged in their activities and exercises. Moreover, a whole paper/pencil session is dedicated to psychoeducation on cognitive functions at the beginning of the therapy. During this 1-h discussion, the therapist explains the cognitive functions treated by RECOS therapy, the importance to improve functional outcome, the impact of specific impairments in different areas of the daily life, etc.

#### Procedural Knowledge

Procedural knowledge refers to knowledge about doing things. It is typically exercised in the accomplishment of a task. Piaget (10) underlines that the action is a non-conscious and autonomous knowledge. Indeed, participants to RECOS therapy often learn procedural knowledge without even being aware that they are learning. In order to bring this knowledge to light, Vermersch (11) developed the *explicitation interview*, which relies on the participant's retrospective introspection. It draws on Piaget's theory of how experience is processed into reflection. It requires that the participant is guided toward the verbalization of the lived experience (12). To accomplish this, interviewees enter a state of evocation, so that they are "reliving" the activity. The interviewer should establish a state of evocation in the participant by focusing on procedural dimension of the action.

During RECOS therapy, procedural knowledge is best identified during computer exercises (13). Because of computer ecological and multisensory presentation of tasks, participants feel generally present in the virtual environment. While paper/ pencil sessions request a collaborative relation with the therapist, computer sessions allow to evaluate the participant "in action." *Explicitation* interview may be introduced by the therapist after

#### Box 1 | Case report.

Sofia is a young woman with schizophrenia. She works 2 days a week in a garage. Her job is to change the colors for the seats of old cars, according to the clients' choice. Because of her deficits in visuospatial memory, she is not able to remember what she has done as soon as she interrupts her work for some minutes or hours. The objective of remediation is to help her to remember shapes, colors, and patterns of the seats she has to fix. For this purpose, the therapist suggests her to solve the exercise *Heraldry*. In the computerized exercise, Sofia is asked to memorize a coat of arms. She will be asked to recreate it with its components when it will have disappeared from the screen.

After Sofia has finished this exercise successfully, she is asked whether she agreed to be interviewed about this exercise. After her acceptance, she begins the explicitation interview by describing her successive thoughts.

First I look at the shape of the flag. I observe that the top is like the shape of a mouth whereas the bottom of it is like a tongue (laughs) (…) Then I look at the colors of the lines. I see that they are blue and yellow. I repeat several times in my head 'yellow, blue, yellow, blue, yellow, …' (…) and then I observe that the red figure looks like the Agip gas station logo …

The interview lasts several minutes and allows her and the therapist to become aware of the numerous strategies used during the exercise. It is quite remarkable that Sofia generates so many strategies without any suggestion or help from the therapist. Before RECOS therapy, she explained that she suffers from schizophrenia and therefore does not have any strategies to solve her problems of memory!

Note: The actual Case Report introduced in the paper concerns a patient who participated to the RECOS therapy as a patient. I was her therapist, and she did not participate in any study about cognitive remediation. Information has been provided to the patient who agreed to be cited in articles or in a training about cognitive remediation. Her name has been anonymized (Sofia is not her real name). There is no need to submit to an ethical committee for such a case report in our institution.

the participant ends a new exercise. It begins with what is called *a contract of evocation*: "What I suggest to you, if you agree, is to take your time, allowing a moment to explain how you realize this exercise." The therapist encourages participants to evoke a particular episode of the computerized exercise he/she has just realized: "Please tell me what you did as soon as the ingredients of the cocktail appeared on the screen, until the moment you gave your answers by clicking on the validation button." Different cues, such as the tone of the voice, the words used, or the gaze of the participants, reveal whether they are in evocation or not. It is important that the therapist does not sit directly opposite the participants as this interferes with the fact of becoming aware of

the lived experience. In order to be in evocation, the participant should not explain to the therapist the strategies used but has to live the exercise again by verbalizing his/her thoughts when performing it. The interviewer avoids questions such as "why," which brings on rationalizations. When the participant makes judgments about his/her performance, he/she is invited to come back to the way he/she proceeded to solve the task. Results obtained at the end of the exercise should, therefore, not be considered during the *explicitation* interview.

#### Conditional Knowledge

Conditional knowledge refers to knowing when and why to use declarative and procedural knowledge. It allows participants to allocate their resources when using strategies. Conditional knowledge is therefore essential to transfer strategies identified by the *explicitation* interview. Home exercises aim to help participants to transfer these skills to reach the objectives of RECOS therapy. In the case of Sofia (**Box 1**), homework may consist in using strategies used during the *Heraldry* exercise in the professional environment to remember the work done on the seats of the old cars. The therapist invites her to imagine how such abilities would help her to remember colors and shapes of the seats she worked on last time.

These metacognitive skills are key determinants to improve functional abilities of the participants (**Figure 1**). When procedural knowledge becomes conscious, it becomes declarative knowledge and can be directly and explicitly applied to a problemsolving task. Consciousness of his/her procedural knowledge is therefore fundamental because people do understand that they are using strategies. The role of the participant – with the help of

#### REFERENCES


the therapist – is then to identify where and when these strategies become efficient to solve a problem in a specific situation. Conditional knowledge is essential for many patients with schizophrenia because of their lack of cognitive flexibility: they often imagine that a strategy that has worked once is a good strategy. They need to consider the context in which such a strategy may be useful. Improving conditional knowledge is probably the main difficulty RECOS therapists are confronted with.

#### CONCLUSION

We believe that a common misconception is considering the efficiency of a cognitive remediation therapy as resulting from computerized exercises. This omits that metacognitive skills also need to be trained during the whole remediation phase. In our view, efficiency of cognitive remediation therapy depends more on the possibility of the participants to transfer acquired skills during sessions in everyday life than from the type of exercises used in the remediation phase. Our metacognitive approach tries to bridge the gap between basic neurocognition and real-world functioning.

From a neurocognitive point of view, RECOS therapy develops the ability to change personal thought processes since each participant is requested to analyze performance against strategies they have themselves deployed. This work of "metacognitive restructuring" is important because patients tend to think that they have no control over their difficulties. They commonly consider that their failures are simply due to the fact that they suffer from schizophrenia. This sense of "learned helplessness" (14) suggests that the drop in performance is due to repeated failures and the subjective impression that the situation cannot be controlled. On the contrary, the patients must understand that their success is due to the conscious and systematic use of relevant problem-solving strategies and that their failures are due to strategies which are ineffective or inappropriate to the situation.

Vermersch's use of introspection to give access to procedural learning theory has inspired our conception of cognitive remediation in psychiatry. We are convinced that each patient has a rich and underestimated procedural knowledge that he/she is not aware of. By providing complex environments favorable to induce motivation (15) and because of their non-judgmental nature in case of failure (16), computerized exercises are recommended to bring this knowledge to light.

#### AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and approved it for publication.


schizophrenia. *Psychiatry Res* (2014) **221**:204–9. doi:10.1016/j.pscychresns. 2014.01.004


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Vianin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Promoting Personal Recovery in People with Persisting Psychotic Disorders: Development and Pilot Study of a Novel Digital Intervention

*Neil Thomas1,2\*, John Farhall3,4, Fiona Foley1 , Nuwan Dominic Leitan1 , Kristi-Ann Villagonzalo1 , Emma Ladd5 , Cassy Nunan5 , Sue Farnan5 , Rosalie Frankish5 , Tara Smark5 , Susan L. Rossell1,2,6, Leon Sterling7,8, Greg Murray1 , David Jonathon Castle6,9 and Michael Kyrios1,10*

*1Centre for Mental Health, Swinburne University of Technology, Hawthorn, VIC, Australia, 2Monash Alfred Psychiatry Research Centre, Monash University and The Alfred, Melbourne, VIC, Australia, 3Department of Psychology and Counselling, La Trobe University, Melbourne, VIC, Australia, 4NorthWestern Mental Health, Royal Melbourne Hospital, Melbourne, VIC, Australia, 5Wellways Australia, Melbourne, VIC, Australia, 6Department of Psychiatry, St Vincent's Hospital, Fitzroy, VIC, Australia, 7Centre for Design Innovation, Swinburne University of Technology, Hawthorn, VIC, Australia, 8Department of Computing and Information Systems, University of Melbourne, Parkville, VIC, Australia, 9Department of Psychiatry, University of Melbourne, Parkville, VIC, Australia, 10Research School of Psychology, Australian National University, Canberra, ACT, Australia*

#### *Edited by:*

*Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland*

#### *Reviewed by:*

*Karsten Heekeren, University of Zurich, Switzerland Kim T. Mueser, Boston University, USA*

> *\*Correspondence: Neil Thomas neilthomas@swin.edu.au*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 19 September 2016 Accepted: 25 November 2016 Published: 23 December 2016*

#### *Citation:*

*Thomas N, Farhall J, Foley F, Leitan ND, Villagonzalo K-A, Ladd E, Nunan C, Farnan S, Frankish R, Smark T, Rossell SL, Sterling L, Murray G, Castle DJ and Kyrios M (2016) Promoting Personal Recovery in People with Persisting Psychotic Disorders: Development and Pilot Study of a Novel Digital Intervention. Front. Psychiatry 7:196. doi: 10.3389/fpsyt.2016.00196*

Background: For people with persisting psychotic disorders, personal recovery has become an important target of mental health services worldwide. Strongly influenced by mental health service consumer perspectives, personal recovery refers to being able to live a satisfying and contributing life irrespective of ongoing symptoms and disability. Contact with peers with shared lived experience is often cited as facilitative of recovery. We aimed to develop and pilot a novel recovery-based digitally supported intervention for people with a psychotic illness.

Methods: We developed a website to be used on a tablet computer by mental health workers to structure therapeutic discussions about personal recovery. Central to the site was a series of video interviews of people with lived experience of psychosis discussing how they had navigated issues within their own recovery based on the Connectedness– Hope–Identity–Meaning–Empowerment model of recovery. We examined the feasibility and acceptability of an 8-session low intensity intervention using this site in 10 participants with persisting psychotic disorders and conducted a proof-of-concept analysis of outcomes.

Results: All 10 participants completed the full course of sessions, and it was possible to integrate use of the website into nearly all sessions. Participant feedback confirmed that use of the website was a feasible and acceptable way of working. All participants stated that they would recommend the intervention to others. Post-intervention, personal recovery measured by the Questionnaire for the Process of Recovery had improved by an average standardized effect of *d* = 0.46, 95% CI [0.07, 0.84], and 8 of the 10 participants reported that their mental health had improved since taking part in the intervention.

Conclusion: In-session use of digital resources featuring peer accounts of recovery is feasible and acceptable and shows promising outcomes. A randomized controlled trial is the next step in evaluating the efficacy of this low intensity intervention when delivered in conjunction with routine mental health care.

Keywords: schizophrenia, psychosis, personal recovery, mental health services, low intensity interventions, digital health, tablet computers, peer support

#### INTRODUCTION

Psychotic disorders represent one of the leading causes of disability and need for ongoing health care in working age adults. In Australia, for example, approximately 4.5 per 1,000 people receive specialist mental health care for a psychotic disorder each year, not including those treated exclusively by private psychiatrists or in primary care (1). In spite of the routine use of antipsychotic medication, and efforts over the past two decades to ensure psychosis is promptly treated at its first emergence, health outcomes remain unsatisfactory for many. For example, the 2010 Australian National Survey of High Impact Psychosis reported that 92% of people seen in specialist mental health services have either recurring or unremitting episodes, 62% experience continuous symptoms, and 90% have deteriorated social functioning (1).

For individuals who experience persisting symptoms and disability, *personal recovery* has become an important target of mental health services internationally (2–7). Often contrasted with the traditional treatment targets of minimizing symptoms (*clinical recovery*) or improving social and occupational functioning (*functional recovery*), the concept of personal recovery has developed from the perspectives of people who use mental health services to prioritize more personal and subjectively meaningful goals of treatment (8–10). A widely used definition is that recovery is "a deeply personal, unique process of changing one's attitudes, values, feelings, goals, skills and/or roles. It is a way of living a satisfying, hopeful, and contributing life even with limitations caused by the illness" (11). The literature on personal recovery has primarily been based on mental health service consumer narratives about the processes that have been most relevant in their own recovery. Although typically characterized as an individual journey, there has been convergence on identification of processes involved in recovery. An influential synthesis of qualitative studies on recovery has highlighted five themes, summarized by the acronym Connectedness–Hope– Identity–Meaning–Empowerment (CHIME): (C) greater social Connectedness, (H) fostering Hope and optimism, (I) transformation of Identity from one dominated by stigma and a passive patient role, (M) developing new Meaning in life, often deriving meaning from mental health experiences, and (E) Empowerment and responsibility for self-managing mental health (12).

The understanding of processes associated with recovery provides a framework for the development of novel interventions suitable for use in mental health services. This field is at an early stage. Results of the recent REFOCUS trial suggested that delivering training based on the CHIME model to promote recovery-oriented practice in services may have a relatively limited impact on measures of personal recovery (13). On the other hand, positive outcomes for measures of recovery have been found for a number of self-management programs, which include materials on recovery (14–16). Notably, these programs tend to incorporate a strong perspective of learning from shared lived experience, with trialed interventions typically featuring peer co-facilitation and group format delivery, encouraging peer contact and peer-to-peer discussion. Peer-delivered services, peer worker roles, and peer-facilitated interventions have increasingly been a key component of this broader recovery movement (8). In a qualitative metasynthesis of what people find helpful about peer support, peers providing a positive role model, engendering hope, and forming new connections were the key themes (17). Likewise, peer contact is often highlighted as having contributed to recovery in consumer narratives (12), which suggests that hearing directly from others with shared lived experience may be a useful component in recoveryoriented interventions.

In the project reported here [Self-Management and Recovery Technology (SMART)], we developed and piloted a scalable intervention tool suitable for use within mental health services to promote personal recovery. In developing the intervention, we saw potential in creating resources in a digital format that could be incorporated into mental health service consultations using a tablet computer as well as being directly accessible by consumers (18). Initial studies of Internet-based applications with people with persisting psychosis have indicated self-guided use of digital tools to be feasible with this population (19–21). Moreover, the Internet is potentially empowering of people with mental illness in facilitating peer-to-peer connections (22) and is a means of presenting lived experience material in video format that may be useful in portraying positive hopeful views of peers with mental health problems (23). Hence, digital technology offers a number of possibilities for promoting learning from lived experience.

We developed an online intervention tool featuring lived experience accounts of personal recovery as central to a series of modules based on the CHIME framework. This paper presents data on the feasibility and acceptability of using this tool in sessions with a mental health support worker and provides a preliminary examination of outcomes.

#### MATERIALS AND METHODS

#### Website Development

The website was developed through parallel processes of end-user consultation; content conceptualization and writing; development of lived experience video materials; and site design. The consultation process included a reference group of seven mental health service consumers with experiences of psychosis who met every 2 weeks during the development phase; a series of focus groups with mental health practitioners from both clinical services and community support services (24), followed by a monthly practitioner reference group; and a further focus group with family carers.

#### Content Development

The development of content was an iterative process, combining the conceptual framework of CHIME with input from consultations and emerging content from the filming process. The CHIME framework was used to inform the main content themes, presented as modules as follows:


health services; how to get the most out of services; and rights and advocacy.

7. *Life*, covering topics related to developing new meaning in life including consideration of the personal values that make life meaningful and identifying related goals.

The video material that was central to the site was developed by conducting a series of interviews with persons with lived experience of psychosis using a semi-structured interview derived from the content framework. To develop the interview questions, a working group that encompassed academic, practitioner, and lived experience expertise developed ideas for questions based on topics within the CHIME model for each theme, refined further with reference group input. Questions were developed to draw out the following: (a) how the theme had been relevant to the interviewee's experience (e.g., the impact of mental illness and associated stigma on how the person saw themselves) and (b) what the person had done to navigate that issue in their own recovery (e.g., ways in which the person had changed how they viewed themselves during recovery). Questions were worded to generate talking from a first person perspective reflecting on their own experiences (e.g., what they had observed from their experiences) rather than from a second person perspective (e.g., advice they would give to a peer about recovery). Each question was preceded by a briefing to orient the participant to the types of material the interviewer was interested in (see **Figure 1**), with the interviewer actively following up material provided in response, in order to thoroughly explore the topic.

Selected interviewees were first phoned by the interviewer to discuss what participation would involve and to ensure they had thought through the implications of appearing on film. They were then sent information about what their participation would involve and were provided with interview questions a few days prior to filming. To ensure that interviewees felt in control of the experience, it was made clear that the interviewer wanted them to discuss only things that they felt comfortable with. The interviewer revisited whether they were willing for the interview to be used after it had been completed, with the option of having any material deleted if wished. A film crew of two (camera and audio) performed the filming, using a two-camera set-up to facilitate editing, with the interviewer off screen. The interviewer used the semi-structured interview as a guide, but allowed the interview to deviate when useful material was being

# Figure 1 | Example interview question posed to video participants.

generated. Questions could be repeated to be refilmed if needed to improve delivery, or align with the first person experiential style of the interviews. At the end of the filming session, each participant was given the option to review a transcript of their interview prior to giving permission for use. Interviewees were paid for their time.

From a pool of 20 potential interviewees who responded to advertising, 11 were selected to form a group with diversity in terms of age, gender, ethnicity, employment status, and sexuality. Interviews were conducted until the material elicited was judged to have reached saturation in its coverage of the content domains; each took approximately 1–2 h.

An extended editing process aimed to generate a series of 2–3 min videos featuring a selection of four to six interviewees discussing key issues for each topic. Given that recovery is characterized as highly individual (12), we aimed to capture different experiences and points of view within each video. Interviews were transcribed, and each line coded by a member of the research team into the various video topics. The combined filmed material for each video topic was reviewed by the content group, with excerpts selected that represented the most useful or impactful material obtained while reflecting a range of experiences and perspectives. A total of 26 videos were produced using this method. Additionally, 11 videos were made introducing each of the peers.

In addition to these lived experience videos, five videos of mental health professionals' experiences and two videos of family members' experiences (produced in a similar way) were included as part of material on working with services and relationships. Additionally, 12 videos were produced featuring either a consumer leader or academic expert contributing additional material that elaborated on what was raised in the other videos or by addressing points that had not been captured by the interviews. An introduction to each module was also filmed, featuring a consumer leader as guide. In total 64 videos were included in the final package.

Text content and reflective exercises were added to summarize key points and complement the lived experience content with material from relevant therapeutic approaches (e.g., on implementing coping strategies, on changing health behavior, on identifying personal values).

#### Website Design

The design of the site was informed by published guidelines for website development for severe mental illness designed to minimize the impact of difficulties in thinking and memory (25), combined with input from the consultation process, and consideration of how the site could be designed in a way that reinforced recovery. The site was optimized for tablet computer and mobile phone use. Navigation was simplified by organizing content in a minimal number of levels (topic, subtopic), making use of touchscreen scrolling to reduce the required number of page loads and having a single constant menu button. Links between pages were clearly labeled, pages were designed to have minimal distractions, and content was developed to be simple, clear, and logically organized. Key design principles derived from the consultations were (a) simplicity of layout and navigation, (b) flexibility in use (e.g., material can be completed in any order), (c) interactivity, (d) catering to different learning styles and preferences by presenting content in multiple ways, (e) access to any information entered being controlled by the consumer, and (f) promoting a positive emotional experience while engaging with the website (24). Consideration of how recovery processes could be facilitated by the design and features of the site included the following: (a) promoting connectedness by allowing users to comment on material and contribute to a user forum on the site and allowing users to share content with workers, family members, and others; (b) promoting the person taking ownership of their identity by allowing personalization of user profiles and customization of content; and (c) promoting empowerment and responsibility in self-management by the ability to track parameters such as sleep and mood and to set and view goals developed from the material.

The site is accessible only by creation of an account, which enables the user to enter information in reflective exercises, charts, and task lists and to select a username and avatar for posting public comments and using the forum. Forums and comment feeds are monitored by the research team to assess risk of harm to participants, and participants are also able to report any offensive comments for moderator review. Example screenshots are shown in **Figure 2**.

#### Pilot Study

#### Design

To examine feasibility and proof-of-concept of using this tool within service delivery, a pilot study was conducted in the form of a single-arm trial of a mental health worker-facilitated intervention using the SMART website on a tablet computer with participants. All participants received the intervention in addition to treatment as usual during a 3-month delivery window. Assessments were completed at baseline and at 3 months. The project was conducted in accordance with the Declaration of Helsinki, and was approved by the Human Research Ethics Committees at The Alfred (study no. 139-14), St Vincent's Hospital Melbourne (study no. 041.14) and Melbourne Health (study no. 2014.087). All participants gave full informed consent prior to commencement.

#### Participants

Participants were recruited through a combination of mail-outs of consumers, clinician referral at community mental health services, and presentation to consumers at residential services in metropolitan Melbourne. Inclusion criteria were: (a) aged 18–65 years; (b) diagnosis of a psychotic disorder (schizophreniarelated disorder or bipolar disorder or major depressive disorder with the presence of a severe episode with psychotic features within the past 2 years), confirmed using the Structured Clinical Interview for DSM-IV-TR Axis I Disorders [SCID; (26)]; (c) sufficient fluency in English to make use of the resources; (d) overall intellectual functioning within normal limits, having an estimated IQ over 70 based on the Wechsler Test of Adult Reading [WTAR; (27)]; (e) access to Internet at home or *via* smartphone. Exclusion criteria were (f) organic psychosis; and (g) change in antipsychotic medication, in-patient admission,

or the commencement or completion of formal psychological therapy within the preceding 2 months.

#### Procedures

A research assistant met with potential participants to obtain informed consent and complete baseline assessments, including the SCID, WTAR, and first administration of outcome measures. Eligible consenting participants were then provided with a time for their appointment with one of the two facilitators, and went on to complete the eight session intervention. The postintervention assessment was scheduled for 3 months following the baseline assessment, to allow time for missed sessions. The post-intervention assessment also included a treatment evaluation questionnaire. Following this assessment, participants were also contacted by telephone by the project manager to obtain feedback on the site to refine content, during which they were also asked questions about their experience of the intervention more broadly.

#### Intervention

The intervention consisted of eight 50-min face-to-face sessions with an experienced mental health support worker (facilitator), using a tablet computer from which the SMART website was accessed. Participants were assigned to one of two trained facilitators, seconded from the community mental health support sector, and attended sessions at weekly to fortnightly intervals within a window of 3 months, in addition to the participant's routine treatment. An account for the participant to use the website was set up during the first intervention session, and they were shown how to access it to facilitate use outside sessions.

Sessions involved the collaborative selection of content from the seven themes on the site, followed by shared viewing and discussion of website material. Discussions included reflecting on the website content as applicable to the participant's own recovery, considering changes participants may wish to enact based on these reflections, and setting goals for the upcoming week. Facilitators encouraged participants to use the website between sessions, complete reflective exercises, and/or make public posts about the content, if willing.

#### Feasibility and Acceptability

Feasibility and acceptability were indexed by the following:


6. Acceptability in terms of positive versus negative emotional impact of using the site, assessed by the item "Overall, did the website make you feel better, or worse, or no different?" in the post-intervention questionnaire.

#### Outcome Measures

The following measures were completed pre- and post-intervention to provide a preliminary assessment of the outcome.

#### *Personal Recovery*

Personal recovery was the primary outcome. The Questionnaire for the Process of Recovery [QPR; (28)], a 22-item self-report measure developed in conjunction with mental health service users to assess personal recovery. The QPR has good psychometric properties (29) and was used because of its strong alignment with the CHIME framework (30). However, because it was a relatively new measure for which sensitivity to intervention effects was unknown, we also included total score on the more established 41-item Recovery Assessment Scale (31) as a second measure of personal recovery.

#### *Recovery dimensions*

Hope was measured using total score on the Schizophrenia Hope Scale (32), a 9-item questionnaire assessing optimism and hope for the future, rated on 3-point items (*disagree*, *agree*, *strongly agree*). Social connectedness was measured using total score on the Friendship Scale (33), comprising six 5-point (*not at all* to *almost always*) items.

#### *Psychotic Symptoms*

The interviewer-rated Positive and Negative Syndrome Scale [PANSS; (34)] was used to assess the severity of psychotic symptoms and their impact on behavior and functioning. The two research assistants were trained in the standardized administration of the PANSS. In addition, the Subjective Experiences of Psychosis Scale (35) was used to assess the subjective impact of psychotic symptoms in participants reporting ongoing positive symptoms at baseline. This is a 29-item questionnaire on which participants rate the positive and negative impact of their symptoms on aspects of their feelings and behavior, such as "hope for the future" and "ability to socialize." Subjective impact is rated on a 5-point scale from *not at all* to *very much* in the past week. The negative impact subscale score was used in analyses.

#### *Emotional Symptoms*

The 21-item Depression Anxiety Stress Scale [DASS-21; (36)] total score and its associated subscales were used to assess emotional symptoms. Participants reported on their experience of symptoms related to depression, anxiety, and stress in the past week, on a 4-point scale from *did not apply to me at all* to *applied to me very much or most of the time*.

#### *Quality of Life*

Total score on the Assessment of Quality of Life-8 Dimension [AQoL-8D; (37)] was used to assess health-related quality of life. This 35-item questionnaire encompasses eight dimensions of physical and psychosocial health, with lower scores indicating fewer issues related to subjective quality of life in the past week.

#### Process Measures

It was hypothesised that the lived experience content of the intervention would influence outcome by increasing self-efficacy for positive recovery, and by reducing the extent to which mental illness is viewed in negative stigmatized terms. The proposed mechanism was assessed by the following measures.

#### *Self-Stigma*

Self-stigma was used as an index of negative views of illness, measured using the 29-item Internalized Stigma of Mental Illness Scale [ISMI; (38)] which includes subscales of alienation, stereotype endorsement, discrimination experience, social withdrawal, and stigma resistance. Items are rated on a 4-point scale from *strongly disagree* to *strongly agree*.

#### *Self-Efficacy*

Self-efficacy was measured using total score on the Generalized Self-Efficacy Scale (39), a 10-item questionnaire rated on a 4-point scale from *not at all true* to *exactly true*.

Additionally, qualitative feedback on experiences of using the lived experience videos was collated from the post-intervention interview. Use of antipsychotic medication was also recorded at baseline and 3 months as a potential confound.

#### Treatment Evaluation

Subjective perceptions of the helpfulness or otherwise of the intervention were assessed by the item "Do you feel that using the website made the impact of your mental health problems better, or worse, or no different?" based on an item used by our group in previous trials with this population (40, 41). This was rated on a 5-point scale from *much worse* to *much better*. Additionally, 11 5-point Likert items were developed to assess whether participants endorsed changes having occurred in relation to material covered on the site (e.g., "I feel more hopeful about my recovery"). Questions were also included in the qualitative phone interview to gather feedback on use of the site during sessions with a worker, and on specific site elements including the videos.

#### Statistical Analyses

In this pilot study, the emphasis was on examining acceptability and feasibility, as well as allowing a preliminary estimate of treatment effects, rather than using inferential statistics to hypothesis–test specific outcomes. Correspondingly, effect sizes and confidence intervals were calculated for the mean pre- to post-intervention change score. Standardized effect sizes were calculated by dividing the mean change by the average SD of pre and post-intervention scores or, if variances were unequal, the baseline SD (42). A series of paired *t*-tests was also conducted to indicate where two-tailed significances fell within *p* < 0.05. Complete data were available for all participants.

#### RESULTS

Twelve potential participants were recruited for the study. Two of these were excluded at baseline: one due to a recent medication change and the other due to participation in another research project. Ten participants completed the baseline assessment (nine males; mean age 42.6 years, SD 12.47, range 23–62 years; six single, four divorced). Nine had a diagnosis of schizophrenia, and one of schizoaffective disorder. None worked full time; two were in paid part-time work and one in volunteer work. All were receiving antipsychotic medication. At baseline, seven participants reported using the Internet at least daily, one once per week, and two "rarely or never." Half of the participants reported that they "rarely or never" used the Internet to access information about mental health.

#### Feasibility and Acceptability In-Session Use

The website was used in 76 of 80 sessions that were attended. Of the remainder, two were initial sessions spent assisting participants setting up email to use the site, and one was a final session consolidating the program material without use of the site. There was only one session in which the support worker was not able to use the site with the participant. Session notes indicated that the participant's engagement had been threatened in a previous appointment, when the facilitator misunderstood something that the participant was discussing, and subsequently directed them to an unrelated topic on the site. The session was spent having a broader discussion of recovery without use of the site to reengage the participant, and the participant and the worker went on to use the site further in their remaining sessions.

Qualitative feedback on the use of the site in-session is collated in **Table 1**. Responses suggested that the process of integrating the website as a tool in sessions functioned well. Some participants expressed that they would have been less engaged or unable to use the site independently without the facilitator sessions, and a number of participants commented that the site facilitated discussion with the worker.

#### Between-Session Use

Six of the 10 participants independently logged on to the site outside of sessions, with a median of 4.5 log-ons among those who did this (range 1–14). Among these six participants was one of the participants who "rarely or never" used the Internet at baseline, the remainder being daily users. Six participants posted public comments on the site, with a median of two posts among those who posted (range 1–20 posts).

#### Drop Out and Satisfaction

All 10 participants attended the full course of eight intervention sessions, and all 10 participants said during the post-intervention interview that they would recommend the site to others.

#### Emotional Impact of Site Use

No participants indicated a negative emotional impact of use of the site, with all 10 reporting a positive effect of using the site on how they felt (responses *better* or *much better*).

#### Table 1 | Participants' feedback about using the site together with a worker.

P1. I think the technology was, it was a guide, it kept our discussion going in a direction that we wanted it to go in. And it would raise the topic or it would raise the, the next discussion, so it was guiding what we were going through. But we did do a lot of talking with [the facilitator] and I felt the two worked together really, really well … perfect. I did say to [her] though that I felt that the facilitator was needed. I felt that while I was at home I didn't have, there was no accountability. I didn't have anyone looking over my shoulder telling me "You must do that" and "you must go to the website", "you must do a module or whatever", there was no accountability. But with a facilitator where I'm going to see [her], you know, within a few days, and we were going to discuss this, then there was accountability, I had to get some things done.

P2. Yeah it was very easy and [the facilitator] explained everything well to me, yeah. And ah, even, I think even just without her I think I could have gone through it myself it would have been quite easy … yeah, yeah. There was some bits in there, um some bits that were a bit difficult to understand, yeah. Not much, but there were two or three parts that she helped me with … And not only that, ah, having someone there as a support to go through every single one of them, I think that very helpful … I guess the SMART, the website goes into more detail into aspects of my life.

P3. We were able to acknowledge and cover things in more depth than I would have by myself.

P4. It's fine, so the iPad was useful, but I'm not the type of person to sort of sit down and do that sort of stuff. I'm more of an interactive person with whoever I'm talking to. … I enjoyed talking to her more than using the iPad.

P5. I thought it worked well. Yeah. … Like with the iPad, I thought that, what do you want to discuss today? There's always, it was more, this program, it was more about what you had to say and what you thought of situations instead of, instead of feeling intimidated when you go in to other ways to see a worker or feeling like oh what are going to say and then feeling like intimidated. But in this case I didn't feel intimidated, I knew [the facilitator] well, and I thought that she did a great job just explaining everything to me, patience, and all of it.

P6. I thought it was really easy. Really easy. Smooth and, ah yeah, just a pleasant experience … . if it was just sort paper and pencils, it sort of got a bit dull after a while. … but the iPad and the website made it quite colourful and a bit more interesting.

P7. Well I didn't use it, I know it's going to sound funny, but we didn't use it much, just for me to get to where I'd written it at home and then read it out to her, "oh this is what I've written and this is what I've written, and this is what I've done," and then discuss it because I can't type properly on an iPad and I like to type really fast and you know, and be able to check my spelling and everything so I just did it at home., and I was happy. … I'd just log on and do some stuff at home and then in the sessions they were really just to go over what I'd done at home and what had come up and so she was sort of acting therapist, poor [facilitator]. … No with the iPad, because I got to share all, everything I'd written, and we'd talk about what I wrote. Talk about subjects and say, "What subject should I do next?" and "What's that involve?" And, "Maybe this one would work," and then I'd say, "Can you do a print out of the PDF, and yadayadayada." A lot more involved than just seeing a therapist, you know what I mean.

P8. [Without the website] we wouldn't have had nearly as much to talk about. And then I would have been more stuck for words I think. I wouldn't have been able to talk about all the issues that we had discussed about the website so it would have been a bit more difficult I think.

P9. It was just really good. A good experience with [the facilitator] and the iPad.

P10. I would've really hated it if I did it by myself, because I probably wouldn't have got there anyway, I mean anywhere, but that's why I thought, I didn't mind it so much, because um people like [the facilitator] were just such a good guide. But something like, with computers, I couldn't do it myself, even though it seemed pretty simple, once [the facilitator] was showing me what to do, I just said, I really would hate to do it by myself …

#### Outcomes

Estimated effect sizes on the outcome measures are presented in **Table 2**. On the primary outcome of personal recovery, an estimated medium effect size was observed on the QPR, which in spite of the small sample size was statistically significant. A similar magnitude effect was observed on the RAS as a second measure of recovery (also statistically significant). Among other outcomes, medium effects were estimated for the subjective negative impact of psychosis symptoms, emotional symptoms on the DASS, and hope, but there was a negligible effect on social connectedness. A small effect size was estimated on the PANSS and the AQoL-8D.

Among process measures, a small to medium effect was estimated on self-stigma, but negligible effects were evident on self-efficacy. Examination of subscales of the ISMI self-stigma measure suggested that the strongest effects were in reducing perceived alienation with an estimated moderate to large effect size (statistically significant), while the estimated effect on other domains, such as negative stereotype endorsement, were negligible. Two participants had increases and three had decreases in antipsychotic medication dose during the trial, but changes on the personal recovery measures were not correlated with changes in chlorpromazine-equivalent dose.

In response to the question "Do you feel that using the website made the impact of your mental health problems better, or worse, or no different?" eight participants reported that their mental health was better or much better, with the remaining two reporting it was no different. Agreement was also high on all items of the treatment evaluation questionnaire (**Table 3**). Specific feedback on the lived experience videos is collated in **Table 4**.

#### DISCUSSION

This study examined the feasibility of an intervention targeting personal recovery in psychosis, a domain for which intervention development is a priority. It involved the novel combination of lived experience-based content on recovery, presentation *via* a digital medium, and delivery integrated with face-to-face mental health sessions. Overall, it appeared feasible to deliver an intervention in this way, and there were promising findings on the primary outcome of personal recovery.

#### Table 2 | Estimated effects on outcome measures.


*N* = *10, except for SEPS negative impact (N* = *7).*

*QPR, Questionnaire for the Process of Recovery; RAS, Recovery Assessment Scale; SHS, Schizophrenia Hope Scale; PANSS, Positive and Negative Syndrome Scale; SEPS,* 

*Subjective Experience of Psychosis Scale; DASS-21, Depression Anxiety Stress Scale 21; AQoL-8D, Assessment of Quality of Life-8 dimension; GSES, Generalized Self-Efficacy Scale; ISMI, Internalized Stigma of Mental Illness.*

#### Table 3 | Participant responses on the treatment evaluation questionnaire.


Our attempts to develop the main content of the site by using a process of editing together various lived experience interviews showed this to be a feasible approach. We used 11 lived experience speakers to provide diversity in age, gender, ethnicity, and sexuality within the group, which was sufficient to produce material for all topics. Not all content within the topics was covered in this way, but complementary scripted videos from experts and text material was used to complete an effective website in which the dominant content was explicitly authored by peers. While many online interventions include "client perspective" videos to illustrate other material [e.g., Ref. (43, 44)], this is the first online intervention, we are aware of, that has an explicit focus on lived experience material as the main vehicle for change.

Use of the site in-session appeared to be feasible and acceptable to participants. The website was used regularly in-session, no participants dropped out, and participants gave generally positive feedback about how the use of the site integrated with face-to-face work. It appeared from participant feedback that many would have found it harder to use or maintain engagement with the site had it not been integrated with sessions with the facilitator. The feedback is consistent with the broader digital mental health literature, where therapist-assisted interventions tend to be engaged with for longer than self-guided interventions (45). However, lower levels of use of digital technology (46) and higher rates of disability among persons with severe mental illness suggest some people may be more reliant upon support to utilize online materials. A blended approach offers a means of capitalizing

#### Table 4 | Participants' feedback about lived experience videos.

P1. I enjoyed, I guess, what's the name of the word, the reinforcement, or, seeing somebody else going through the same situation with the same feelings, somebody I could relate to, I found, it was something I hadn't been through before and, and that made me feel good, that felt great and a lot more at ease from watching the video. The video also … what I did try and do once or twice was to answer questions without watching the video and then, going back watching the video, realised that the video was actually opening up the scope, it was actually scoping out the area ahead of what the questions were going to be like.

P2. The fact that others are sharing their own experience. … Yeah, and I look at the video and even though I was hospitalised before, when I come out – it's been a while since I came out – I forget that ah, I'm not alone. Yeah.

P3. Really enlightening. Made me feel like I am not alone.

P4. Well … relate to people … what they're saying: this is what happened to me and how I got over it, and what I did. Yeah.

P5. I like the fact that everyone's so different, it's so, like they all have different, and they're all unique, and they all had good things to say. Like what I mean by good is, you know, relevant to people with like, yeah. I didn't feel so alone. So that was a good thing. … I just felt like I could relate to someone. I wasn't so alone.

P6. There were obviously the different individuals who explained their scenario and talked about each topic in the video, and then said what that topics means to them, and how certain questions around the topic are answered, and it was all good, it was all insightful. … Yes, the videos were really good, they were organised, they were structured, quite informative, honest, and um yeah, so like lots of multi-perspectives on topics – yeah, that was good.

P7. Because I could just sit there and watch a whole half hour of them talking and get so inspired, and so moved, you know. … it makes you realise you're not alone, and that you're not some frumpy sort of, the bad image of mental illness: not washed, not clothed well, smells bad, can't coherently keep a sentence together, looks off into the distance, is aggressive or threatening or sullen. You know what I mean?

P8. I could relate to a lot of peoples' stories, and they had a similar experience to mine, so I thought that was good. And then I answered a few questions and sent a few comments to [the facilitator] and that sort of thing. So yeah I just, I gained more insight into my condition I think. I've always had a lot of insight, but just hearing other peoples' experiences; when you think you've got your own mind made up about your illness and you won't listen to anybody about your illness, and you need to think "oh okay," you think you're right, but there are a lot of other people who have varying symptoms, and it was just good to hear other peoples' opinions and impressions of their own diagnosis, and that sort of thing; what they do to tackle their problems. So it was good.

P9. I can relate to some of the things they were talking about in my own life, and it just makes me more aware and more determined to overcome the obstacles that I've been facing.

P10. They had people talking about how to handle stress, and I put my feet in their shoes and sort of could understand, you know, um where those people were coming from, their experiences.

on the scalability of digital interventions to deliver quality structured interventions, while bearing in mind the barriers to independent Internet use experienced by this group. Indeed, although most participants used the site between sessions as well, which was encouraged, not all of the participants in this pilot did so, suggesting that a significant proportion of people in this population would be reliant on in-session use. Indeed, it should also be noted that most of our sample were daily Internet users at baseline, so their existing levels of computer use were higher than average for this population (46). Given our vision for the website as a vehicle to facilitate discussion between the consumer and worker about recovery, the modest between-session use was not problematic. A number of participants' responses confirmed that use of the site did help them discuss issues which otherwise might not have been raised or would have been difficult to raise.

Video-based tools may have benefits beyond the present aim of facilitating discussions about recovery: future research could investigate whether patient–worker interactions around embarrassing or sensitive topics (e.g., discussing ambivalence about medication) are supported by tools of this kind. The technology may have broader applications in practice, such as in promoting supported decision making [e.g., Ref. (47)], or as a tool for formal psychological therapies.

As a preliminary proof-of-concept study with a small sample, analysis of outcome was not designed to hypothesis–test efficacy, but to establish whether estimated effects were in a range suggesting full scale trialing to be worthwhile. While we cannot be certain that other ongoing interventions had no impact on outcomes, results were promising, with a moderate effect size being estimated on both measures of recovery that were used. Feedback from participants was also consistent with the intervention having a beneficial impact upon recovery, and participant feedback additionally identified no negative effects. The estimated effect size on recovery is similar to effects observed for other psychosocial interventions for persisting psychosis (48). Together, these findings suggest value in conducting a larger-scale controlled trial of this intervention (49).

#### AUTHOR CONTRIBUTIONS

All authors participated in the development and pilot of the digital intervention. NT, FF, NL, JF, EL, CN, SF, LS, and GM designed the website and developed its content. NT, JF, FF, RF, and TS developed the therapist intervention protocol. NT, K-AV, and FF conducted analyses and prepared the manuscript. All the authors read and approved the final manuscript.

#### ACKNOWLEDGMENTS

The authors gratefully acknowledge the contributions of the video contributors; Rybazoid (video production); Fidget Friend (software development); Alfred Health, Mind Australia, NorthWestern Mental Health, St Vincent's Hospital Melbourne, Tandem, VICSERV, the Victorian Mental Illness Awareness Council, Wellways Australia (Mental Illness Fellowship); Katrina Lindblom, Friederike Wahl, Robert Shaw, Claire Young, Jo-Anne Abbott, Alex Lopez Lorca, Sonja Pedel, Maheswaree Kissoon Curumsing, and members of the focus and reference groups.

#### REFERENCES


#### FUNDING

This study was funded by the State Government of Victoria Department of Health Mental Illness Research Fund (MIRF33). The funder had no role in the design of the study or reporting of results.

Robertson S, et al., editors. *Proceedings of the 24th TheMHS Mental Health Services Conference 2014*. Balmain, NSW: TheMHS (2015a).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Thomas, Farhall, Foley, Leitan, Villagonzalo, Ladd, Nunan, Farnan, Frankish, Smark, Rossell, Sterling, Murray, Castle and Kyrios. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# What Role Can Avatars Play in e-Mental Health Interventions? Exploring New Models of Client–Therapist Interaction

*Imogen C. Rehm1 , Emily Foenander1 , Klaire Wallace1 , Jo-Anne M. Abbott1 , Michael Kyrios2 and Neil Thomas1,3\**

*1National eTherapy Centre, Centre for Mental Health, Swinburne University of Technology, Melbourne, VIC, Australia, 2Research School of Psychology, College of Medicine, Biology and Environment, Australian National University, Canberra, ACT, Australia, 3Monash Alfred Psychiatry Research Centre, Monash University Central Clinical School and The Alfred, Melbourne, VIC, Australia*

#### *Edited by:*

*Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland*

#### *Reviewed by:*

*Andrew Molodynski, Oxford University, UK Gaudelus Baptiste, Centre Hospitalier Le Vinatier, France*

> *\*Correspondence: Neil Thomas neilthomas@swin.edu.au*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 20 September 2016 Accepted: 03 November 2016 Published: 18 November 2016*

#### *Citation:*

*Rehm IC, Foenander E, Wallace K, Abbott JM, Kyrios M and Thomas N (2016) What Role Can Avatars Play in e-Mental Health Interventions? Exploring New Models of Client–Therapist Interaction. Front. Psychiatry 7:186. doi: 10.3389/fpsyt.2016.00186*

In the burgeoning field of e-mental health interventions, avatars are increasingly being utilized to facilitate online communication between clients and therapists, and among peers. Avatars are digital self-representations, which enable individuals to interact with each other in computer-based virtual environments. In this narrative review, we examine the psychotherapeutic applications of avatars that have been investigated and trialed to date. Five key applications were identified (1) in the formation of online peer support communities; (2) replicating traditional modes of psychotherapy by using avatars as a vehicle to communicate within a wholly virtual environment; (3) using avatar technology to facilitate or augment face-to-face treatment; (4) as part of serious games; and (5) communication with an autonomous virtual therapist. Across these applications, avatars appeared to serve several functions conducive to treatment engagement by (1) facilitating the development of a virtual therapeutic alliance; (2) reducing communication barriers; (3) promoting treatment-seeking through anonymity; (4) promoting expression and exploration of client identity; and (5) enabling therapists to control and manipulate treatment stimuli. Further research into the feasibility and ethical implementation of avatar-based psychotherapies is required.

Keywords: avatars, virtual environments, virtual reality, e-mental health, digital mental health, human–computer interaction, computer-mediated communication

#### INTRODUCTION

In online virtual environments, multiple remotely located users can synchronously communicate and interact with each other *via* an avatar – a digital character that the user can customize to represent his/ her identity. In their seminal 2008 paper, Gorini et al. (1) proposed two key applications of avatars in multiuser, computer-based, three-dimensional virtual worlds, such as *Second Life*, which could potentially transform the delivery of online interventions for mental health problems (i.e., e-mental health interventions). These included (1) an alternative form of technology (i.e., other than head-mounted virtual reality devices) to deliver exposure-based therapy for anxiety disorders and substance-abuse problems and (2) facilitation of online peer support communities. Studies since 2008 have proposed several further applications of avatars in the delivery of e-mental health interventions. For instance, head-mounted virtual reality devices that immerse clients into computer-generated interactions with avatars are increasingly being utilized to treat anxiety disorders (2) and persecutory delusions (3). Indeed, immersive virtual reality technology has previously been reviewed for its significant potential in psychiatric applications (4, 5). In this narrative review, we consider the psychotherapeutic applications of computer-based and online avatar technology; specifically, the ways in which avatars have been used to replace or augment traditional models of client–therapist interaction and communication. We additionally synthesize the functions that avatars can serve in such applications and consider the advantages and challenges of implementing this novel technology within e-mental health interventions.

#### APPLICATIONS OF AVATARS IN e-MENTAL HEALTH: AN OVERVIEW

#### Online Peer Support Communities

The potential to foster cohesive social networks is cited as a strength of virtual worlds in which users can interact with each other by adopting personalized avatars (1). Most attention has focused on the program Second Life, where users can create realistic-looking human avatars that they can use to manipulate stimuli within the virtual environment and remotely interact with other users – also represented as avatars – *via* text and/or audio.

In 2008, there were 68 health-related activities on Second Life, 20% of which were intended primarily for peer support (6). Many peer communities focused on sensitive topics (e.g., sexual health, addictions) or were organized for and by groups of people who are vulnerable to marginalization and discrimination in "real life" (e.g., people with disabilities). The popularity of Second Life peer support communities, and additional health-related activities (e.g., health promotion and education), may result from users' ability to collaborate, interact, and consult with other avatars in real time while maintaining their anonymity (6). By 2013, only 24 health-related Second Life sites were still active, many of which were absent of other users when researchers entered the virtual host spaces (7). Thus, user anonymity may have actually minimized investment in building long-term relationships.

#### Avatar Use to Replicate Traditional Psychotherapy Models in Online Virtual Environments

The Second Life platform has strong potential to replicate models of individual and group-based treatments, but conducted entirely online, with both client and therapist interacting with each other in a virtual environment. To date, this model has been trialed in two uncontrolled studies using individual (8) and group (9) formats.

Yuen et al. (8) conducted a manualized, acceptance-based behavioral treatment for adults with social anxiety disorder, delivered entirely *via* Second Life. Participants (*n* = 14) and therapists (*n* = 3) met in a private, secure virtual therapy room for a 1-h individual treatment session each week for 12 weeks. During role-playing exposure exercises, sessions occurred in other virtual spaces relevant to the exposure scenario (e.g., giving a presentation in a virtual conference room), with confederate therapists facilitating each exercise by adopting pre-made avatars with diverse physical characteristics (e.g., age, gender, ethnicity). Intention-to-treat analyses for a range of mood, psychosocial, and social anxiety measures indicated large posttreatment and 12-week follow-up effect sizes. Future research comparing the intervention to a control condition is required.

Also delivered in Second Life, Hoch et al. (9) developed an 8-week relaxation and mindfulness group, with sessions delivered twice-weekly to groups of up to 10 participants. The first session of each week involved teaching the participants specific relaxation strategies. In the second session, participants met in a virtual teaching space designed to look peaceful (e.g., a virtual forest), where they were asked to review their practice of these strategies. Overall, mental health symptoms measured by the Symptom Checklist 90 decreased from pretreatment to posttreatment, although changes were not statistically significant. Future research may need to use gold-standard measures of specific psychopathology symptoms, which may be more sensitive to changes. However, participants reported that they appreciated the convenience of being able to participate remotely in a virtual group program and also commented that the anonymity of participation made the intervention material more approachable.

#### Avatar-Assisted Face-to-Face Therapies

Several studies have utilized various forms of avatar technology to facilitate or augment treatments that are delivered with the face-to-face support of a therapist. Two models of these avatarassisted therapies have been implemented, which are as follows: (1) applications that require the client to "embody" or represent themselves as an avatar in order to participate in the therapy and (2) applications that do not require the client to embody an avatar, but rather, require the client to *interact with another avatar*, be it the therapist or an "other."

#### Embodied Use of Avatars by the Client

Using Second Life, Kandalaft et al. (10) delivered a manualized social skills training program to eight young adults with high-functioning autism spectrum disorder. Ten sessions were completed over 5 weeks, during which the therapist physically sat alongside and coached the participant through virtual role-playing scenarios. During each session, the therapist – represented as an avatar – directed participants to various virtual spaces (e.g., cafes, parks, shops) where they met with a confederate clinician – also represented as an avatar – to practice social interactions in diverse role-playing situations (e.g., attending a job interview). Clinicianadministered neurocognitive measures of verbal and non-verbal emotional recognition significantly improved from pre- to postprogram, suggesting that the program may improve elements of social communication typically impaired in people with autism.

Using a different commercially available avatar platform (*ProReal*), van Rijn et al. (11) used avatar-mediated communication as a component of face-to-face group therapy in a prison setting. Unlike Second Life avatars, ProReal avatars appear as androgynous, featureless human forms, which users can manipulate in terms of color, size, and expressive gestures. ProReal avatars can also be given virtual props to facilitate symbolic emotional expression, such that participants could use the avatars to explore and communicate their emotions to other group members. Group therapy sessions were 90-min long, ran for 6 weeks, and were facilitated by a counselor. Distress ratings measured by the CORE-10 did reduce from pretreatment to posttreatment, although not statistically significantly. Qualitative feedback suggested that the avatars supported participants to express emotions that were difficult to communicate verbally, and to develop empathy for other group members.

#### Avatar-Mediated and Augmented Therapeutic Interactions

Avatar software can offer clients unique scope to address or confront their symptoms, within a safe and controlled environment, with the support of a therapist. Leff et al. (12) utilized this technology as part of a novel treatment for persecutory auditory hallucinations, with participants (*n* = 26) asked to create an animated avatar face of the entity that they believed was talking to them. In conjunction with voice transformation software to distort the therapist's speech, the therapist used the avatar to role play the person's auditory hallucination during exercises designed to support the person responding to their voice more adaptively. Highly promising results were reported in a pilot trial (12), with reductions in hallucination severity relative to wait-list controls, and some participants reporting a remission of their voices. These findings are currently being examined in a full sale randomized controlled trial (RCT) (13).

The value of avatar technology for learning and practice of new social skills has been recognized by numerous authors (5, 14–16). Both Rus-Calafell et al. (14) and Peyroux and Franck (15, 16) have used avatars to simulate social situations as part of social cognitive remediation programs for people with psychotic disorders. In the twice-weekly, 8-week *Soskitrain* program (14), participants (*n* = 12) practiced social skills with a variety of expressive avatar characters in different situational environments. Therapists could control the avatar's behaviors according to participant responses to promote scaffolded learning and stop the interaction for therapeutic discussion. In an uncontrolled pilot trial of the program, Rus-Calafell et al. (14) reported significant pretreatment to posttreatment improvements to participants' self-reported negative symptoms, social avoidance, and social functioning. System-recorded facial emotion recognition errors and time spent in avatar-based conversations also improved. Gains were maintained at 4-month follow-up. Using a different avatar-based simulation program (*RC2S*), Peyroux and Franck (16) reported two experimental single-case studies that made significant pretreatment to posttreatment improvements in theory of mind abilities and improved facial emotion recognition, social knowledge, self-esteem, and attributional style. Using RC2S (15, 16), participants learned to analyze the mental state, emotions, and intentions of "Tom," an avatar character, to guide Tom's responses to various social situations. As in Soskitrain, the therapist's role was to provide social skills training and feedback to support participant's interactions with the avatar.

#### Avatars to Participate in "Serious Games"

Many video games require that the player embodies an avatar to interact with other players or to interact with automated non-player characters. In "serious games," such game-like elements are incorporated into computerized psychotherapies to achieve a serious health-related goal (e.g., to reduce depression symptoms) (17). SPARX is an example of a serious game in which the participant, as an avatar, progresses through seven modules of a fantasy-based computerized game (18, 19), which incorporates cognitive–behavioral therapy strategies to treat depression in adolescents. At the commencement of each module, participants meet with an automated guide whose role is akin to a therapist. The guide informs participants of their tasks for each module and how they relate to improving depression. At the end of each module the guide provides a summary of what was learned. SPARX has shown efficacy in two RCTs, and participants have reported high levels of treatment satisfaction and acceptability (18, 20).

Unlike aforementioned avatar-based therapies (8–12), SPARX is an entirely self-guided e-mental health intervention. Participant feedback suggested that the self-guided nature of the program was one of its strengths, as were gaming elements that supported treatment engagement such as a story-like narrative throughout the seven modules, and automated characters that were perceived as being warm and caring (21).

#### Avatars as Autonomous Virtual Therapists

Finally, avatars have been utilized as autonomous virtual therapists – also referred to as embodied communicative/relational agents – to facilitate the clinical interview and assessment process, to provide psychoeducation, or to direct individuals to access alternative psychological services (22–25). In these applications, the client is not required to embody an avatar to interact with the therapeutic agent, as in Second Life (8–10) and SPARX (18–20). Furthermore, the therapeutic agent, which can be represented as a realistic-looking human avatar (22–24) or as a two-dimensional animated character (25), is not controlled by a human clinician. Rather, the avatar is an autonomous agent presented on a computer monitor, which responds to the client's text-based, auditory, and/or sensory input on the basis of artificial intelligence or algorithm.

To investigate user experiences, Rizzo et al. (22) conducted clinical interviews with 91 adults who interacted with an autonomous virtual therapist called "Ellie." Their experiences were compared to 120 participants who interacted with a clinicianoperated version of Ellie and 140 participants who participated in face-to-face clinical interviews. Most participants reported that they were willing and felt comfortable to share information with Ellie (in both autonomous and clinician-operated conditions), which is consistent with other studies that have found participants highly rate the therapeutic alliance with relational agents (25). Ratings of rapport and listening skills were significantly greater for the clinician-operated avatar than the autonomous avatar, and rapport ratings for the clinician-operated avatar exceeded those of face-to-face clinical interviewers. This may indicate the importance of realistic transactional elements in virtual interactions for reducing emotional barriers to client engagement (22); however, this suggestion requires investigation.

Pinto et al. (23, 24) investigated the efficacy of an avatar-based self-management intervention for young adults with depression (Electronic Self-Management Resource Training for Mental Health; eSMART-MH). *Via* laptops, participants accessed a virtual primary health clinic to communicate with simulated avatar health-care professionals. The program was designed to provide young people with training and practice in communicating with health-care professionals about depression, and to learn self-management strategies. An RCT (23) demonstrated that self-reported depression symptoms decreased significantly more for the eSMART-MH group (*n* = 12), when compared to an attentional control group; however, there was no significant change in symptoms from baseline to 12-week follow-up in either group. Participant feedback for future versions of the intervention included greater range of user input and avatar response options, the possibility to receive counseling, and access to the program *via* mobile devices (24).

#### THERAPEUTIC FUNCTIONS AND CHALLENGES OF AVATAR TECHNOLOGY

#### Supporting the Therapeutic Relationship through Virtual Presence

As the reviewed studies demonstrate, several applications have involved both the therapist and client utilizing avatars as a form of virtual embodiment (8–10), which allows both parties to feel a sense of social presence within a remotely accessed online environment. This sense of social presence, and the tendency for people to engage in greater self-disclosure during computermediated communication compared to face-to-face interactions (26), has strong potential to facilitate the development of online therapeutic relationships and may be more important for presentations where shame or stigma are central features. Indeed, feelings of copresence, emotional closeness, and interpersonal trust are equivalent for avatar-, audio-, and video-based modes of communication (27). In line with the contention that avatars can generate social presence in virtual environments, participants in a number of the studies listed above noted that their interactions fostered a sense of genuine rapport (8, 10), even when the real/implied therapist was operated on the basis of automated therapeutic scripts (21, 24) or artificial intelligence (22).

#### Reduction of Communication Barriers

The flexibility to use audio- and/or text-based communication in avatar platforms provides options for clients to choose a communication mode with which they feel most comfortable. For instance, Stendal and Balandin (28) illustrated how text-based communication in Second Life reduced communication barriers for a participant with autism spectrum disorder by reducing the ambiguity of social and emotional cues during his interactions with online peers. The participant reported feeling a sense of security and control that he perceived his disability did not afford him in the "real world," empowering him to develop personally valued online friendships. Relatedly, participants with autism spectrum disorder in Kandalaft et al.'s study (10) suggested that being comfortable with computer-mediated communication supported their confidence to participate in the avatar-simulated social situations. For many individuals with physical, mental, and language impairments, the ability to communicate *via* an online medium of choice and from the perceived safety of one's home is likely to be an advantage of avatar-based technology.

Like other forms of online communication, however, technical difficulties can create new communication barriers that impact on the quality of treatment sessions (e.g., flow, timing) (8), and clients must feel confident to use (or learn to use) the technology in the first instance. Another issue to consider is that the lack of visual cues may reduce a sense of accountability to one's conversational partner, as both parties can engage in other tasks without the other being able to see this occurring. Even for therapists, this can foster a sense of disconnectedness, which itself can inhibit the development of communicational and emotional attunement (29).

#### Anonymity to Promote Treatment-Seeking

Unlike videoconferencing, avatars afford the possibility of anonymous engagement with psychological services. Potentially, this may attract individuals to receive treatment they may otherwise not seek (e.g., due to shame or stigma) by engaging them wholly in an avatar-based e-mental health intervention (29) or by encouraging them to seek face-to-face treatment following an anonymous and positive interaction with an autonomous virtual therapist (22, 24). A case study by Quackenbush and Krasner (29) highlighted how a client who was reluctant to engage in online treatment *via* videoconferencing due to fear of racial discrimination was successfully treated in Second Life, where the client was able to use an avatar and pseudonym that disguised his ethnicity.

Reflecting ethical concerns relating to risk management and client safety, anonymous use of avatar technology has been recommended only for instances in which the client is seeking generic psychoeducation and referral information (22, 30). When ongoing avatar-based psychotherapy services are offered, authentication of the client's identity is advised (30). This can still permit pseudonymity and customization of one's avatar. The psychological impact of a client's "anonymized" avatar on both the client and therapist requires investigation; particularly in light of evidence that altering the visual features of an avatar (e.g., height, attractiveness) can both positively and negatively affect the individual's virtual and "real world" behavior (31).

#### Exploration of Client Identity

A particularly novel component of (embodied) avatar use in online psychotherapeutic interventions is the capacity they provide for clients to express, experiment with, explore, and construct a virtual, visual representation of their identity. In the context of using Second Life for social purposes, three functions of avatars as an expression of the individual's "real world" identity have been identified (32). First, avatars can be utilized as a vehicle for engaging in a virtual world, whereby the individual's representation of their true physical characteristics, and their real name, occupation, interests, etc., were conveyed *via* their avatar. Second, avatars could be utilized to enhance one's real world self by embodying features perceived by the individual as positive; for instance, customizing the avatar to appear more youthful or expressing personality traits the individual normally suppresses. Finally, avatars could be utilized to diversify one's identity by adopting an entirely new identity in the virtual world; for instance, adopting a new name, different gender, creating a new "life story," and visually representing one's avatar in a way that does not reflect one's true physical appearance. Participants reported that regardless of whether they utilized their avatar for self-extension, enhancement, or diversification, experiences within Second Life generalized to positive physical, cognitive, social, or emotional outcomes in the real world. Such identitybased functions could have significant implications for the therapeutic use of avatars.

#### Manipulation and Control of Treatment Stimuli

Finally, the ability for the therapist to control the content of virtual stimuli and intensity of virtual situations as part of behavioral interventions is a key advantage of avatar technology over other forms of computer-mediated communication (1). As with virtual reality exposure therapy conducted through head-mounted display devices, computer-based virtual reality applications can be conducted remotely (8, 9) or within the therapist's office (10–16), thereby offering a safe environment for exposure-based treatments; compared to *in vivo* therapies, sessions utilizing the same exposure stimuli or role-playing scenarios can feasibly and conveniently be repeated as often as needed for the client to learn new skills; and clients may feel a greater sense of control over and safety within virtual settings, which could support treatment adherence. Additionally, computer-based applications are unlikely to induce the nausea and sensory distortions that can be experienced with virtual reality technology (33).

Research is required to determine whether computer-based virtual reality applications can generate an equivalent sense of realistic "presence" as do head-mounted simulation technologies; the assumption being that presence will elicit the level of anxiety required for habituation over the course of exposure therapy (34). The role of presence as a mechanism of behavior change requires greater investigation across all modalities of virtual therapies, as does the level of photorealism required of avatars and virtual environments to induce a sense of being immersed in a therapeutic interaction. This may depend on the needs of the clinical population undergoing treatment. For instance, the high drop-out rate (34.6%) from Leff et al.'s (12) treatment for auditory

#### REFERENCES


hallucinations may have indicated that use of avatars to represent experiences more concretely may have been too confronting for many participants to tolerate.

#### CONCLUSION

The use of avatars in e-mental health interventions represents a nascent area of inquiry. As demonstrated in this review, the psychotherapeutic applications of computer-based and online avatar technologies have been diverse. Several of the studies reviewed in this paper were uncontrolled trials with small sample sizes and framed as pilot investigations of feasibility as opposed to efficacy. The diversity in study aims, methodologies, participant groups, intervention types, outcome measures, technologies, and treatment delivery models prevents a conclusion as to the efficacy of avatars in delivering e-mental health interventions – nor was this the aim of this review. However, this diversity does highlight its significant potential and functionality.

As a flexible and creative platform with which to deliver individual and group therapies, peer support, and as a form of e-mental health augmentative intervention, avatar technology offers significant potential to engage a broad range of clients in need of psychological support who may otherwise be unable or unwilling to participate in traditional treatment models. In particular, avatars may foster the development of a strong virtual therapeutic alliance; overcome communication barriers experienced by individuals with various disabilities and mental disorders; offer an anonymized means of seeking treatment; support clients to explore and extend their identity; and provide therapists with greater control over treatment stimuli that involve an element of exposure or skills training. Nevertheless, there appear to be many challenges to the ethical and feasible implementation of avatar-based e-mental health interventions. In light of advancing portable technologies, such as smart phones and tablet devices, it will be interesting to observe how the models of avatar-based e-mental health interventions will further diversify and enable greater access to engaging interventions. Delineating feasible and appropriate models of avatar use for psychotherapeutic purposes, and investigating both consumer and clinician attitudes and preferences toward the technology for this purpose, will be an important endeavor for future research.

#### AUTHOR CONTRIBUTIONS

EF conducted the literature review, and IR drafted the first version of the manuscript. NT, KW, JA, and MK contributed to and edited the final manuscript.


interviewing. In: Luxton DD, editor. *Artificial Intelligence in Behavioral and Mental Health Care*. Oxford, UK: Elsevier (2016). p. 53–79.


**Conflict of Interest Statement:** The authors declare that the manuscript was developed in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Rehm, Foenander, Wallace, Abbott, Kyrios and Thomas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **The Impact of a Videogame-Based Pilot Physical Activity Program in Older Adults with Schizophrenia on Subjectively and Objectively Measured Physical Activity**

*Heather Leutwyler <sup>1</sup> \*, Erin Hubbard<sup>1</sup> , Bruce Cooper <sup>2</sup> and Glenna Dowling<sup>1</sup>*

*<sup>1</sup> Physiological Nursing, University of California San Francisco, San Francisco, CA, USA, <sup>2</sup> School of Nursing, University of California San Francisco, San Francisco, CA, USA*

#### *Edited by:*

*Yasser Khazaal, Geneva University Hospitals, Switzerland*

#### *Reviewed by:*

*Jérôme Favrod, University of Applied Sciences and Arts of Western Switzerland, Switzerland Jude Uzoma Ohaeri, University of Nigeria, Nigeria Khaled M'Hiri, University Hospital of Geneva, Switzerland*

> *\*Correspondence: Heather Leutwyler heather.leutwyler@ucsf.edu*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 23 October 2015 Accepted: 07 December 2015 Published: 21 December 2015*

#### *Citation:*

*Leutwyler H, Hubbard E, Cooper B and Dowling G (2015) The Impact of a Videogame-Based Pilot Physical Activity Program in Older Adults with Schizophrenia on Subjectively and Objectively Measured Physical Activity. Front. Psychiatry 6:180. doi: 10.3389/fpsyt.2015.00180* **Objectives:** The purpose of this report is to describe the impact of a videogame-based pilot physical activity program using the *Kinect for Xbox 360* game system (Microsoft, Redmond, WA, USA) on physical activity in older adults with schizophrenia.

#### **Methods:** In this one group pre-test, post-test pilot study, 20 participants played an active videogame for 30 min, once a week for 6 weeks. Physical activity was measured by self-report with the Yale Physical Activity Survey and objectively with the Sensewear Pro armband at enrollment and at the end of the 6-week program.

**Results:** There was a significant increase in frequency of self-reported vigorous physical activity. We did not detect a statistically significant difference in objectively measured physical activity although increase in number of steps and sedentary activity were in the desired direction.

**Conclusion:** These results suggest participants' perception of physical activity intensity differs from the intensity objectively captured with a valid and reliable physical activity monitor.

**Keywords: schizophrenia, physical activity, videogame**

## **INTRODUCTION**

Older people with schizophrenia tend to spend the majority of their time in sedentary activity (1). Prolonged sedentary activity is associated with various negative health outcomes including poor physical fitness (2). The negative impact of sedentary activity is problematic for any adult, but is particularly burdensome for older adults with schizophrenia. People with schizophrenia have a higher mortality rate than the general population and sedentary behavior contributes to this mortality gap (3, 4). Even small reductions in sedentary behavior, such as short activity intervals of 10 min or less, may positively influence physical and mental health (5).

Despite the obvious need for physical activity programs to decrease sedentary time in older adults with schizophrenia, few interventions exist. Most of the physical activity research in serious mental illness has focused on younger to middle-aged adults (6). In a 2010 Cochrane review, the efficacy of three physical activity RCTs (e.g., walking, weight training) in people with severe mental illness (SMI) was evaluated (6). The authors concluded that the impacts of these programs on health

Leutwyler et al. Physical Activity

outcomes were mixed, but some studies provided evidence of improved fitness. Two physical activity RCTs for people with SMI published after the Cochrane review also produced mixed outcomes. One study provided evidence of a positive impact on weight change but did not report information about impact on physical activity (7). Another study showed a positive impact on self-reported physical activity levels (8).

Not only do we need effective physical activity programs for older adults with schizophrenia but we also need programs that will successfully engage older adults with schizophrenia (6, 9). Videogame-based exercise may be one way to promote physical activity in this vulnerable population (10, 11). Videogames with an interface that requires physical exertion to play, such as the *Kinect for Xbox 360* game system (Microsoft, Redmond, WA, USA), promote physical activity (11, 12). Participants use their body to control the game with the *Kinect's* full-body tracking sensor system that recognizes the participant's body and mirrors those movements in the game.

We designed and tested a videogame-based physical activity program designed to keep older adults with schizophrenia engaged and motivated to participate in physical activity. (10, 11) Consistent with previous findings from videogame research, (13) as our participants became more involved and successful with active videogames, they developed skills that made it easier to engage in activity (10). One of the study objectives was to examine the impact of the program on physical activity in participants' every day life. The purpose of this report is to describe the impact of a videogame-based physical activity program using the *Kinect for Xbox 360* game system (Microsoft, Redmond, WA, USA) on frequency of vigorous physical activity in older adults with schizophrenia.

#### **MATERIALS AND METHODS**

#### **Design**

A one-group pre-test, post-test design was used to examine the impact of our pilot program on physical activity. Institutional review board approval was obtained from the University's Committee on Human Research. Anonymity and confidentiality were maintained according to their guidelines.

#### **Participants and Settings**

Inclusion criteria were that participants be: (1) at least 55 years of age or older; (2) diagnosed with schizophrenia or schizoaffective disorder; and (3) competent to consent based on an evaluation of their comprehension of the consent form. A convenience sample of participants from four different facilities were recruited by facility staff and referred to the researchers. Study fliers were also posted. Patients with a history of a prior myocardial infarction, uncontrolled hypertension, history of angioplasty, history of angina, and/or use of nitroglycerin to treat angina were excluded.

#### **Procedures**

Recruitment and data collection began in May 2012 and concluded in June 2013. The principal investigator and her research staff facilitated the weekly exercise sessions. Once a week for 6 weeks, participants played an active videogame, using the *Kinect for Xbox 360* game system (Microsoft, Redmond, WA, USA), for 30-min. Although current recommendations for adults are to engage in moderate intensity physical activity 150 min each week, (14) the focus of this pilot study was to determine feasibility, acceptability, and adherence to the program. Therefore, we chose a short frequency and duration for our program to first establish participant participation in the program.

At each weekly session, participants choose from a variety of games and were encouraged to use different games each week. Offthe-shelf videogames played included: *Kinect Sports*(e.g., bowling, golf, skiing, darts), *Kinect Carnival Games*, *Kinect Dance Central 2*, *Kinect Adventures*, and *Kinect Your Shape Fitness Evolved*. Games played most often were: bowling, dance, carnival games, skiing, Tai Chi (from the Fitness game), baseball, darts, golf, river rafting, and 20,000 leaks under the sea.

Participants engaged in the program in groups of three to four at a time. The program took place at the facility the person attended: an outpatient community treatment center, a locked mental health facility, a transitional residential facility, or a skilled nursing facility.

Each participant had an appropriate amount of space in order to achieve full range of motion. Approximately 6 feet of free space between the participant and the *X-Box Kinect* sensor was needed. The games offered a variety of levels and each group started off at the beginner level. Participants were taught warning signs to be aware of while exercising (e.g., shortness of breath, dizziness), were encouraged to discontinue the game if they noticed any exercise warning signs and to notify the research staff.

#### **Measures**

A demographic questionnaire obtained information on age, gender, race, smoking status, and living situation at enrollment.

#### Physical Activity

Subjective assessment of physical activity was assessed with the Yale Physical Activity Scale (YPAS). (15) The YPAS is based on self-report and measures five activity dimensions (Vigorous activity, Leisurely walking, Moving on feet, Standing, and Sitting) that occur during a typical week over the past month. Eight summary scores are calculated that include: total time spent per week in all physical activities, total energy expenditure in kcal per week, five individual indices for the activity dimensions, and a total activity dimension index. Indices are calculated from the frequency, duration, and intensity of activities. Preliminary data supports the use of YPAS as a self-report measure of physical activity in older adults with schizophrenia (16). The YPAS was completed at enrollment into the study and upon completion of the final group at week 6.

Objective assessment of physical activity was measured with the SenseWear Pro armband™ (SWA; BodyMedia Inc., Pittsburgh, PA, USA). Participants wore the device for 7 days between week 1 and week 2 and again between week 5 and week 6 over the left triceps muscle at all times except during bathing or water activities. This device samples data from a heat-flux sensor, a galvanic-skin-response sensor, a skin-temperature sensor, a near-body-temperature sensor, and a bi-axial accelerometer. Previous research doucmented that the SWA accurately estimates energy expenditure in older adults. (17) Outcome variables measured with this instrument included daily steps taken during a week and weekly hours of physical activity categorized as sedentary [0–2.9 metabolic equivalents (METS)], moderate (3–5.9 METS), vigorous (6–8.9 METS), and very vigorous (*≥*9 METS). The data can be reduced to a single number in order to determine if activity levels improved during the course of the program. Participants were fitted with the SWA and informed about the activities the device would monitor during data collection. Participants were reminded to take the SWA off only when showering or performing other activities during which the device might get wet. Additionally, they were given an information sheet that described proper placement and contact information for study staff if they had questions at any time. The devices were returned at the next videogame session. If the participant was unable to attend, arrangements were made for a convenient time and location for retrieval.

Adherence was measured with a count of sessions attended and with total minutes attended out of the possible total minutes of attendance. Total possible minutes for the six sessions were 180. The PI or research assistant (RA) logged the participant's attendance at each session. The PI or RA monitored participants throughout the session in order to determine the number of minutes attended as participants were allowed to leave at any point during the group.

#### **Data Analysis**

Statistical analyses were performed using STATA version 13. Descriptive statistics and frequency distributions were generated for sample characteristics. Change in self-reported vigorous physical activity with the YPAS was analyzed with a non-parametric bootstrapped repeated measures *t*-test using bias-corrected and accelerated confidence intervals with 5,000 repetitions to compensate for the non-normal distribution of difference scores.

Change in objectively measured physical activity from week 1 to week 6 was evaluated with a change score coded as a dichotomy, with positive change (i.e., increase) coded 1 and negative (i.e., decrease) or no change coded as 0, subtracting the total amount of activity measured during week 1 from week 6. This approach accommodated the extreme variation in the small number of reports (e.g., SD almost seven times greater than the mean for change in sedentary hours reported) by allowing a simple binomial test for change.

#### **RESULTS**

A total of 20 participants that took part in the study completed the physical activity assessments and are included in the analyses. Sociodemographic and clinical characteristics are presented in **Table 1**. The majority of participants were male and the average age was 60 years. More than half of the patients were current or previous smokers.

The mean number of groups attended was 5.6 out of 6 total (SD = 0.8) and the mean total minutes attended were 169 out of 180 total (SD = 23.7). Seventy percent of participants (*n* = 14) had perfect attendance (i.e., attended six out of six sessions), four

#### **TABLE 1 | Clinical characteristics**.


participants attended five sessions, one participant attended four sessions, and one participant attended three sessions.

Vigorous physical activity was defined as any activity that lasted at least 10 min and caused large increases in breathing, heart rate, or leg fatigue, or caused the person to perspire. At enrollment in the study, 13 participants reported no vigorous activity, 4 participants reported vigorous activity 1–3 times a month, 1 participant reported vigorous activity 1–2 times per week, 1 participant reported vigorous activity 3–4 times per week, and 1 participant reported vigorous activity 5 or more times per week. At completion of the study, nine participants reported no vigorous activity, four participants reported vigorous activity one to three times a month, one participant reported vigorous activity one to two times per week, three participants reported vigorous activity three to four times per week, two participants reported vigorous activity five or more times per week, and one did not know. With the non-parametric bootstrapped repeated measures *t*-test using bias-corrected and accelerated confidence intervals (95% CI) and alpha equal to 0.05, we detected a significant increase in frequency of self-reported vigorous physical activity from enrollment to the end of the program for an average increase of 0.9 points (95% CI: 0.35–2.45). This difference constitutes a standardized effect of 0.46 [Cohen's *d*; (18)], which is in the "medium effect size" range.

Fifteen of the 20 participants wore the SWA. Five participants chose not to wear the SWA. As indicated in**Table 2**, 10 participants increased the number of steps taken during the period between enrollment to program completion and 5 showed no increase. Nine participants had a decrease in the amount of time spent in sedentary activity while six had no increase. Eight participants increased the amount of time spent in moderate activity, and seven showed no increase. We did not detect any vigorous or very vigorous activity. None of these differences were statistically significant in this small sample, although increase in the number of steps taken (67% increased), and the reduction in sedentary activity (61% decreased) were in the desired direction.

#### **TABLE 2 | Objective physical activity results**.


#### **DISCUSSION**

Our videogame-based physical activity program is the first in the literature to show a significant positive impact on the amount and frequency of physical activity in older adults with schizophrenia. While participants self-reported significantly more vigorous physical activity, we did not find a statistically significant difference in objectively measured physical activity. However, the majority of participants increased their number of steps and spent less time in sedentary activity at program completion. It was interesting to find that although we did not detect any vigorous or very vigorous activity with our monitors, participants self-reported an increase in vigorous activity at program completion. Further research need to be done to better understand the differences between perceived and actual levels of physical activity in this population. It is possible that when a sedentary patient begins an exercise program, the patient perceives the activity to be more vigorous than what is objectively recorded by an activity monitor. These data also suggest that our program is an ideal way to begin a physical activity program because it allows participants to begin activity in a safe and feasible manner while providing the opportunity to decrease sedentary activity and increase daily steps taken.

Interestingly, Lindamer et al. (16) also found that older patients with schizophrenia self-reported vigorous activity differed from objectively measured vigorous activity. Soundy et al. (19) conducted a systematic review to identify and synthesize the evidence about levels of physical activity in adults with serious mental illness. Not surprisingly, the authors found that patients spent more time per week in sedentary activity and less time per week in moderate and vigorous activity compared to healthy controls. Physical activity was measured in a variety of ways in these studies, including both actigraphy and self-report with the IPAQ. These authors encourage the use of both rigorous self-report questionnaires as well as accelerometers to capture the range of physical activity. Our results also highlight the importance of evaluating physical activity with both objective and subjective measures in order to capture the full spectrum of activity and inactivity.

Similar to our results, Bartels et al. (8) found in their physical activity program (In SHAPE) for adults of all ages with serious mental illness, that intervention participants self-reported twice as much moderate to vigorous activity compared to controls. The intervention group also had significantly greater total MET minutes of vigorous activity. Their measure of physical activity was self-report using the International Physical Activity Questionnaire (IPAQ). In contrast to the YPAS, the IPAQ only assesses moderate and vigorous physical activity. Other researchers have suggested that the YPAS may be more likely to capture light activity than an objective physical activity monitor (16). The intervention for the In SHAPE program consisted of 1 year of weekly sessions with a fitness trainer plus a fitness club membership. This approach may have also offered a way for participants to begin a physical activity program in an individualized way that was safe and approachable.

Recent research suggests that promoting light intensity physical activity may be a feasible approach to ameliorate the deleterious health consequences associated with sedentary behavior (20). Low levels of physical activity in combination with prolonged sitting time are associated with negative health outcomes (20, 21). Greater sedentary time has been associated with an increased risk for all-cause mortality in the general population (22). Our program may be an avenue to break up prolonged sitting time because it is a feasible gentle physical activity program to begin with that can be adapted to become more strenuous as participants become adept at both playing the games and more comfortable with engaging in physical activity.

Our study has several limitations. Since this was a pilot feasibility study, participants engaged in the program only once a week for 30 min. Perhaps with more frequent activity, the impact on objective measures would have been more pronounced. In addition, the majority of the participants were male. In order to determine if the program is helpful for both men and women, in future work we will attempt to recruit and enroll more females. The lack of a control group makes it difficult to fully understand the impact of the program on participants' physical activity levels. One of the purposes of the study was to determine which games participants preferred however this could have influenced individual participants' activity levels differently. In future work, we plan to standardize the games played during the course of the study so that participants all experience the same games for equal amounts of time. Despite the limitations, our study provides important preliminary data about the initial efficacy of a novel videogame-based physical activity program that has potential to improve physical activity and health outcomes in people with schizophrenia.

Our results indicate that participants have a positive attitude toward videogame-based physical activity. Although participants' perception was that of engaging in vigorous activity, our objective monitor did not detect vigorous activity. However, objectively measured physical activity did increase, showing a trend toward improvement. With a larger sample, more frequent exercise, and increased exercise duration, we hope to see a significant improvement in objectively measured activity. For example, with 40 participants, the medium effect size for change in sedentary time would be statistically significant with alpha = 0.05 (twosided) and power = 0.80. The results from this study represent an important "first step" toward the creation of a novel and effective physical activity program for older adults with schizophrenia that may be easily incorporated into the daily routine of mental health facilities.

#### **AUTHOR CONTRIBUTIONS**

HL, EH, BC, and GD made substantial contributions to the conception of the work, analysis and interpretation of the data for the work. HL and EH made significant contributions to the acquisition of the data. HL drafted the work. HL, EH, BC, and GD revised the work critically for important intellectual content, approved the final version, and agree to be accountable for all aspects of the work.

#### **REFERENCES**


#### **FUNDING**

This work was supported by the National Center for Research Resources (KL2R024130 to HL and UCSF-CTSI Grant Number UL1 RR024131) and the National Institute on Aging (K23AG04438). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources or the National Institutes of Health.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer Jerome Favrod and handling Editor Yasser Khazaal declared a current collaboration and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

*Copyright © 2015 Leutwyler, Hubbard, Cooper and Dowling. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **Reducing delusional conviction through a cognitive-based group training game: a multicentre randomized controlled trial**

*Yasser Khazaal <sup>1</sup> \*, Anne Chatton <sup>1</sup> , Karen Dieben <sup>1</sup> , Philippe Huguelet <sup>1</sup> , Maria Boucherie <sup>1</sup> , Gregoire Monney <sup>1</sup> , Laurent Lecardeur <sup>2</sup> , Virginie Salamin <sup>3</sup> , Fethi Bretel <sup>4</sup> , Silke Azoulay <sup>5</sup> , Elodie Pesenti <sup>6</sup> , Raoul Krychowski <sup>7</sup> , Andreia Costa Prata <sup>7</sup> , Javier Bartolomei <sup>1</sup> , Perrine Brazo <sup>2</sup> , Alexei Traian <sup>8</sup> , Thomas Charpeaud <sup>9</sup> , Elodie Murys <sup>10</sup>, Florent Poupart <sup>11</sup> , Serge Rouvière <sup>12</sup>, Daniele Zullino <sup>1</sup> , Alberto Parabiaghi <sup>13</sup>, Mohamed Saoud 12,14,15 and Jérôme Favrod <sup>16</sup>*

#### *Edited by:*

*Alexandre Andrade Loch, University of São Paulo, Brazil*

#### *Reviewed by:*

*Erica Toledo Piza Peluso, Universidade Bandeirante Anhanguera, Brazil Rafael Teixeira De Sousa, National Institutes of Health, USA*

#### *\*Correspondence:*

*Yasser Khazaal, Geneva University Hospitals, Grand Pré, 70C, Geneva 1206, Switzerland yasser.khazaal@hcuge.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 23 February 2015 Accepted: 14 April 2015 Published: 28 April 2015*

#### *Citation:*

*Khazaal Y, Chatton A, Dieben K, Huguelet P, Boucherie M, Monney G, Lecardeur L, Salamin V, Bretel F, Azoulay S, Pesenti E, Krychowski R, Costa Prata A, Bartolomei J, Brazo P, Traian A, Charpeaud T, Murys E, Poupart F, Rouvière S, Zullino D, Parabiaghi A, Saoud M and Favrod J (2015) Reducing delusional conviction through a cognitive-based group training game: a multicentre randomized controlled trial. Front. Psychiatry 6:66. doi: 10.3389/fpsyt.2015.00066* *<sup>1</sup> Geneva University Hospitals, Geneva University, Geneva, Switzerland, <sup>2</sup> Service de Psychiatrie, CHU de Caen, Université de Caen Basse-Normandie, UMR6301 ISTCT, ISTS Team, Caen, France, <sup>3</sup> Fribourg Network for Mental Health, Fribourg, Switzerland, <sup>4</sup> Service de Psychiatrie Ambulatoire et de Réhabilitation du Pôle Rouen Rive Droite, Centre Hospitalier du Rouvray, Centre de Jour Saint-Gervais, Rouen, France, <sup>5</sup> Centre Psychosocial, Biel, Switzerland, <sup>6</sup> MIDI Foyer SISP EMS, Yverdon Les Bains, Switzerland, <sup>7</sup> La Borde, Foyer SISP EMS, Lausanne, Switzerland, <sup>8</sup> Centre Medico-Psychologique B, Centre Hospitalier Universitaire, Clermont-Ferrand, France, <sup>9</sup> Service de Psychiatrie, Centre Hospitalier de Vichy, Vichy, France, <sup>10</sup> Unité Mobile de Psychiatrie, Centre Hospitalier Princess Grace, Monaco, Monaco, <sup>11</sup> Léon-Jean Grégory Hospital, Thuir, France, <sup>12</sup> CRESOP, Centre Hospitalier le Vinatier, Bron, France, <sup>13</sup> Laboratory of Epidemiology and Social Psychiatry, IRCCS 'Mario Negri' Institute for Pharmacological Research, Milan, Italy, <sup>14</sup> EA 4615, Université de Lyon, Lyon, France, <sup>15</sup> Université Lyon 1, Lyon, France, <sup>16</sup> School of Nursing Sciences La Source, University of Applied Sciences and Arts of Western Switzerland, Lausanne, Switzerland*

Objective:"Michael's game" (MG) is a card game targeting the ability to generate alternative hypotheses to explain a given experience. The main objective was to evaluate the effect of MG on delusional conviction as measured by the primary study outcome: the change in scores on the conviction subscale of the Peters delusions inventory (PDI-21). Other variables of interest were the change in scores on the distress and preoccupation subscales of the PDI-21, the brief psychiatric rating scale, the Beck cognitive insight scale, and belief flexibility assessed with the Maudsley assessment of delusions schedule (MADS).

Methods: We performed a parallel, assessor-blinded, randomized controlled superiority trial comparing treatment as usual plus participation in MG with treatment as usual plus being on a waiting list (TAU) in a sample of adult outpatients with psychotic disorders and persistent positive psychotic symptoms at inclusion.

Results: The 172 participants were randomized, with 86 included in each study arm. Assessments were performed at inclusion (T1: baseline), at 3 months (T2: post-treatment), and at 6 months after the second assessment (T3: follow-up). At T2, a positive treatment effect was observed on the primary outcome, the PDI-21 conviction subscale (*p* = 0.005). At T3, a sustained effect was observed for the conviction subscale (*p* = 0.002). Further effects were also observed at T3 on the PDI-21 distress (*p* = 0.002) and preoccupation

subscales (*p* = 0.001), as well as on one of the MADS measures of belief flexibility ("anything against the belief") (*p* = 0.001).

**Conclusion:** The study demonstrated some significant beneficial effect of MG.

**Keywords: psychotic disorders, schizophrenia, cognitive therapy, game, hypothetical reasoning, randomized controlled study, psychotherapy**

#### **Introduction**

A considerable proportion of patients with psychotic disorders do not respond fully to antipsychotic agents (1). As an adjunct treatment, however, cognitive behavioral therapy (CBT) has, to some extent, a favorable effect on psychotic symptoms (2–7), awareness of illness, distress, preoccupation, conviction, and behavioral consequences of delusional beliefs (8–11). A core component of CBT is based on hypothetical reasoning, which consists of the search for alternative explanations for a given experience (12). Delusions are associated with low belief flexibility, with most patients unable to spontaneously find alternative explanations for their beliefs (13). Promoting an alternative hypothesis may reduce their degree of conviction, preoccupation, and distress associated with delusions (14, 15).

Despite an increasing need for trained professionals who are able to deliver CBT for psychotic symptoms in naturalistic settings, studies have emphasized that training opportunities are lacking and that the numbers of qualified therapists are poor (16– 18). Moreover, most studies in the field have been performed in CBT specialized settings with highly selected patients (19). A number of preliminary studies have shown, however, that new tools that integrate CBT techniques in game format (20, 21) and some forms of computer-assisted therapy (22) represent promising treatment for patients with psychotic disorders.

"Michael's game" (MG) (**Table 1**) is a hypothetical reasoning training module that promotes the dissemination of CBT (specifically, reasoning training) in natural clinical settings. It is based on CBT of psychotic symptoms (5, 23, 24), as described elsewhere (20, 21). The aim of the game is to train people to find alternative hypotheses for a given situation.

Two preliminary studies (20, 21) support the possible impact of the game on psychotic symptoms.

Those earlier studies had, however, a number of limitations such as lack of control group, lack of blind assessment, the noncontrolled character of the pharmacological treatments, as well as the lack of follow-up measures.

Furthermore, the consistency of the intervention was not controlled by audio recording of the sessions. The study at hand aimed to overcome the weakness of the previous works with more rigorous methods such as a randomized controlled design, blind assessments, sessions recording, and a follow-up assessment.

In the present randomized controlled trial, we hypothesized a more important impact of "treatment as usual + Michael's game" (MG) than of "treatment as usual + waiting list" (TAU) on a measure of psychotic symptoms related to conviction (primary outcome) and possibly on measures of distress, preoccupation, symptom intensity, belief flexibility, cognitive insight, awareness of illness, and actions on beliefs (secondary outcomes).

#### **TABLE 1 | "Michael's game."**

#### **Examples of objectives on the cards**

Describe a situation before interpretation Devise the interpretation of a situation as a hypothesis Search for different interpretations of the same situation Identify the cognitive and behavioral consequences of the different hypotheses Search for a link between the interpretation given for a situation and a personal real-life experience Put the hypotheses in hierarchical order in terms of their probability Search for arguments for or against a hypothesis Think of a way of testing a given hypothesis in reality **Examples of cards** *A non-psychotic and non-emotional situation card* Michael sets two bags of different sizes on each side of a scale The big bag has the same weight as the small bag Michael is surprised since the two bags are supposed to be filled with cotton He thinks that the small bag contains a stone

*A psychotic card*

Michael is watching his favorite show on television When the show host appears, Michael is so pleased that he bursts out laughing The show host and another participant in the show start laughing at the same time Michael tells himself: "My joy is contagious"

*The game was conceived by two of the authors, Yasser Khazaal and Jerome Favrod. It has been translated into English, French, Spanish, German, and Italian. The game includes non-psychotic, non-emotional cards (1–11); emotional, non-psychotic cards (12–32); and psychotic cards (33 to the end).*

The aim of the trial is to assess the short-term effect (postintervention) of MG in comparison to TAU on the primary outcome and on further secondary outcomes. The sustained or long-term effect of the intervention in comparison to TAU was also evaluated 6 months after the post-intervention assessment of the primary and secondary outcomes.

#### **Materials and Methods**

Participants were outpatients recruited in psychiatric rehabilitation units and outpatient clinics in Switzerland, France, Monaco, and Italy. Potential participants were identified through systematic screening of medical records. Written informed consent was obtained from all participants. Institutional review boards and the ethical committees in Switzerland, France, and Italy approved the study protocol. The protocol was also made available to all study investigators and was registered (International Standard Randomized Controlled Trial Number Register: ISRCTN37178153)<sup>1</sup> . The study was carried out from October 2008 to September 2011. The study's duration was longer than initially expected because of a delay in the recruitment process. The study had, however, slightly higher recruitment of participants than first planned (172 rather than 166) and a higher retention rate than originally expected (124

<sup>1</sup> http://www.controlled-trials.com/ISRCTN37178153/

patients assessed at the end point rather than the 94 projected). There were no other deviations from the original study protocol.

#### **Randomization**

Eligible patients were randomized to either TAU or to MG after providing informed consent. The randomization scheme was generated by using the website randomisation.com<sup>2</sup> and kept independently by a statistician. The allocation ratio used was one to one. A permuted-block randomization procedure was used with fixed block sizes of four patients, ensuring that the number of subjects in the different groups closely balanced at all times. The study investigators, who were in charge of enrollment, were blinded to the randomization sequence in order to prevent them from predicting patient allocation, thus reducing selection bias. There was no randomization by center.

#### **Assessment Procedures**

Patients were assessed at baseline (T1); at 3 months, after the end of the MG sessions (T2); and 6 months after the second assessment (T3). Psychologists or psychiatrists made the assessments independently from the therapists, and the game leaders and were not informed of the treatment received by the participants (blind evaluation). The patients were given 40 Swiss francs (approximately 35 Euros or 40 US Dollars) as compensation after each evaluation.

#### **Measures**


phenomena related to the principal abnormal belief of a patient. In practice, the MADS improves the assessment of the possible impact of the game on specific and individualized aspects of the main delusional idea of each included patient and on measures of belief flexibility specifically linked with the main delusional belief. The following MADS assessments were included in the study:


#### **Primary Outcome**

The primary outcome was the change after treatment in the conviction score of the PDI-21.

#### **Secondary Outcomes**

The 16 secondary outcomes included the change in scores on the other two subscales of the PDI-21 (distress and preoccupation), the five subscales of the BPRS, the two subscales of the BCIS, the GAF, the SOFAS, and the five items of the MADS.

#### **Sample Size and Power**

An *a priori*sample size was estimated from a pilot study (21), using the conviction subscale of the PDI-21 in the calculation. By using the Diggle formula for the sample size calculation in longitudinal data, we found that a sample size of at least 47 persons per group was needed at each study evaluation (error rate of 0.05; statistical power of 0.8).

#### **Inclusion Criteria**


#### **Exclusion Criteria**


#### **Interventions**

#### Michael's Game

Michael's game is a collaborative group game consisting of 80 cards (**Table 1**). Each card corresponds to a situation and to objectives

<sup>2</sup> http://www.randomization.com

that target the ability to reason with hypotheses (see **Table 1** for an example of a card). Training group leaders (two per session) direct the game during weekly sessions lasting for about 1 h. Because MG is an 80-card game, the mean number of sessions needed to end the game is not *a priori* fixed. The mean number of sessions needed to end the game was 12.1 (SD = 3.41). The mean participation rate of the participants (number of sessions completed by a participant *×* 100/total number of sessions needed to complete the program in the same group) was 91.2% (SD = 14.1). These figures are similar to those described in the preliminary reports related to the game (20). Failure to attend more than three sessions was considered to be MG treatment discontinuation. Participants (in groups of four to eight patients) are led through specific questions to find multiple answers (hypotheses) to the questions and conclusions that Michael draws from the situations that he is confronted with. In other words, participants have to help Michael to find alternatives to the conclusions that he draws from situations described on each card. MG was conceived as a collaborative group card game in order to allow patients to become partners of a fictive character (Michael) and together to interact with cards containing impersonal information that may reflect their own concerns. Participants play together on objectives such as the following: describe the situation reported on a given card,

identify Michael's hypothesis, search for different hypotheses that may explain the same situation, report the possible cognitive and behavioral consequences of the different hypotheses, give arguments for or against a given hypothesis, and imagine how to test a given hypothesis in reality. Game directors were psychiatric care workers (nurses: 14, psychologists: 12, psychiatrists: 6) who were specifically trained to deliver MG according to the training model described below.

#### Treatment as Usual Plus Waiting List (TAU; Control Condition)

Treatment as usual comprises case management, psychosocial interventions, antipsychotic medication, and outpatient and community follow-up. The patients in the TAU condition had to wait until the end of follow-up before participating in a MG.

The collaborative centers had access to similar pharmacological treatment, psychosocial structures, and level of medical training. Medication was monitored during the study. Chlorpromazine equivalences were calculated for antipsychotic medications according to Woods (34). Participants from both groups received treatment as usual throughout the entire study period.

**TABLE 2 | Baseline sociodemographic and clinical characteristics of the participants in the Michael's game (MG) and the treatment as usual (TAU) groups**.


Khazaal et al. A game for hypothetical reasoning


*Summary statistics report means and SDs or percentages.*

*TAU, treatment as usual plus waiting list; MG, treatment as usual plus Michael's game; PDI, Peters delusions inventory; BCIS, Beck cognitive insight scale; MADS, Maudsley assessment of delusions schedule; BPRS, brief psychiatry rating scale; GAF, global assessment of functioning; SOFAS, social and occupational functioning assessment scale.*

#### **Quality Assurance**

Group directors at each of the 16 collaborative centers (nurses, psychologists, psychiatrists) were trained by the same method: a standardized 1 h presentation and a 1 h training session for the game. Training was directed by one of the game's authors or by one person authorized by the authors for his or her educational qualities and experience in directing the game. Supervision was also offered to game directors by the first author upon request. Sessions were audiotaped (not available for all centers) to check their fidelity to the intervention and were assessed with expected therapist strategies extracted from the collaboration, interpersonal effectiveness, understanding, and guided discovery items of the Haddock scale (35). Twenty-five randomly selected sessions were assessed. The percentage of items rated as being compliant with the desirable strategies ranged from 72 to 100%.

#### **Statistical Analyses**

At the patient level, baseline differences were checked between the two study groups, as well as differences between completers and non-completers using *t*-tests, chi-square tests, or Fisher's exact tests where applicable.

The direct effect, or the short-term effect, of treatment on the primary and on the secondary continuous outcomes was evaluated through analyses of covariance (ANCOVAs) by considering outcome values after the intervention at T2 and adjusting for the respective baseline outcomes. The treatment group served as the between-subject factor. These analyses were also controlled for center to correct for a possible cluster effect and for medication. Similarly, the long-term effect at T3 of the intervention was measured by ANCOVAS, using outcome values at T3 and adjusting for the respective baseline outcomes. Finally, categorical MADS variables were analyzed at T2 (short-term effect) and T3 (longterm effect) through logistic regression analyses, with treatment group as the between-subject factor, and were also adjusted for baseline outcomes and controlled for center and medication. The necessary application conditions of all these models were checked and met.

#### **TABLE 3 | Evolution of primary and secondary outcome variables by treatment group**.


*<sup>a</sup>p-value resulting from analysis of short-term treatment effect.*

*<sup>b</sup>Only significant results after Bonferroni's correction are reported.*

*<sup>c</sup>p-value resulting from analysis of long-term treatment effect.*

*<sup>d</sup>n.s., not significant.*

*Interval variables are reported by their mean and SD [M (SD)] and categorical variables by their percentage (%).*

*TAU, treatment as usual plus waiting list; MG, treatment as usual plus Michael's game; T1, baseline; T2, at 3 months post-treatment; T3, at 6 months follow-up; PDI, Peters delusions inventory; BCIS, Beck cognitive insight scale; BPRS, brief psychiatry rating scale; GAF, global assessment of functioning; SOFAS, social and occupational functioning assessment scale.*

*Bold font indicates significant p-value.*

#### Adjusting for Multiple Testing

#### Treatment of Missing Data

The trial was *a priori* powered for multiple time points at a predefined 0.05 significance level, using PDI-21 conviction as the primary outcome. Therefore, this level of significance was kept when the primary outcome results were interpreted. However, to minimize the chance of spurious results pertaining to the secondary outcomes, we adjusted them for multiple testing by Bonferroni's correction (α *≤* 0.05/16 = 0.0031). All the statistical analyses were performed with the Statistical Package for the Social Sciences (SPSS version 18.0, IBM, Chicago, IL, USA).

Analyses were done on an intention-to-treat basis, whereby all randomized subjects were included. Categorical missing data were imputed using logistic regressions. Quantitative missing data were imputed by the expectation–maximization algorithm, a statistical simulation technique that estimates the averages, the matrix of variance and covariance, and the matrix of correlations. After convergence, missing data are replaced by the estimation obtained and the completed data are then analyzed by the usual methods.

#### **Results**

After screening and informed consent, 172 patients were included and randomized for study participation (**Figure 1**).

#### **Baseline Characteristics of Subjects**

The mean age of the sample was 37.1 years (SD = 10.4). A high school diploma or a university degree had been attained by 27.1% of participants. Most participants had a diagnosis of schizophrenia (81.4%), were single (78.5%), lived in a private residence (59.3%), and had a disability pension (73.1%). Concurrent psychiatric disorders at inclusion were common [i.e., substance use (15.6%); anxiety disorders (27.2%)].

At baseline, the PDI-21 distress score was higher for the MG group than for the TAU group at a trend level (*p* = 0.05). There were no differences in the other main clinical measures (**Table 2**).

#### **Dropouts**

At T2, dropout was associated with treatment allocation (*p* = 0.01). Twenty-three of the TAU group patients and 10 of the MG group patients discontinued the study. At T2, dropout was also associated with younger age (*p* = 0.02). Furthermore, dropouts were more common among patients who dismiss the belief in response to hypothetical contradiction (*p* = 0.04) and those more able to plan a behavioral experiment (*p* = 0.04). There was no other statistically significant difference on the other sociodemographic and clinical characteristics. Dropout at T3 was associated with a younger age (*p* = 0.05) and recruiting centers, where Switzerland had a statistically significant high rate of dropouts during this period compared with the other centers (*p* = 0.01). No other statistically significant difference was observed.

#### **Outcomes**

**Table 3** presents patient outcomes at baseline, 3 months, and 9 months for the MG and TAU groups. **Figure 2** depicts the evolution of the PDI conviction raw scores.

#### Short-Term Treatment Effect (at T2) on the Primary Outcome

After baseline values, medication, and the cluster effect of center were controlled for, the results of the ANCOVAS showed a short-term treatment effect for conviction (*p* = 0.005). After the intervention, the estimated mean conviction was 29.0 for the TAU group, whereas it was 24.3 for the MG group, which means a short-term treatment effect of *−*4.7 points (95% CI [*−*7.9 to *−*1.4]).

#### Short-Term Treatment Effect (at T2) on the Secondary Outcomes

No short-term effect was observed for the PDI-21 distress subscale (*p* = 0.1), but an effect was observed for preoccupation (*p* = 0.025), with an estimated mean of 22.3 for the TAU group and 19.0 for the MG group, a treatment effect of *−*3.3 points (95% CI [*−*6.2 to *−*0.4]). However, after Bonferroni's correction, this effect was no longer significant.

No short-term treatment effect was observed for either Beck self-reflectiveness (*p* = 0.08) or Beck self-certainty (*p* = 0.4). For BPRS subscores (activation, affect, positive symptoms, and resistance), no treatment effect was found (*p* = 0.6, *p* = 0.3, *p* = 0.4, and *p* = 0.4, respectively). For BPRS negative symptoms, a treatment effect was observed (*p* = 0.019) that was not sustained after Bonferroni's correction. GAF and SOFAS did not exhibit a treatment effect (*p* = 0.1 and *p* = 0.9, respectively). With regard to the MADS items, there was an effect for "anything against the belief " in which the MG group was more likely to answer "yes" than the TAU group (OR = 3.16, 95% CI [1.30–7.70], *p* = 0.01). This effect, however, was no longer significant after Bonferroni's correction.

#### Long-Term Treatment Effect (at T3) on the Primary Outcome

A sustained effect of the intervention was observed for conviction (*p* = 0.002). At T3, the estimated mean conviction was 24.6 for the TAU group, whereas it was 18.8 for the MG group, a long-term treatment effect of *−*5.7 points (95% CI [*−*9.3 to *−*2.2]).

#### Long-Term Treatment Effect (at T3) on Secondary Outcomes

A treatment effect was observed for distress, preoccupation, GAF, and SOFAS (*p* = 0.002, *p* = 0.001, *p* = 0.03, and *p* = 0.02, respectively). The mean scores for distress and preoccupation were higher in the TAU group, whereas the mean scores for GAF and SOFAS were higher in the MG group than in the TAU group. However, after Bonferroni's correction, the significance remained only for distress and preoccupation, for which treatment effects were estimated at *−*5.5 points (95% CI [*−*9.0 to *−*2.0]) and *−*5.5 points (95% CI [*−*8.6 to *−*2.2]), respectively.

Regarding MADS variables, a sustained treatment effect was observed only for "anything against the belief ": the MG group was more likely to answer "yes" than the TAU group (OR = 4.95, 95% CI [1.85–13.27], *p* = 0.001). This effect remained significant after Bonferroni's correction.

#### **Discussion**

This study investigated the effects of a hypothetical reasoning training module of short duration, called MG, in a sample of 172 outpatients with psychotic symptoms in 16 centers in France, Italy, Monaco, and Switzerland. In accord with the main hypothesis, the conviction score measured with the PDI-21 was improved in the MG condition compared with the TAU condition at T2, the post-intervention assessment. After Bonferroni's correction, no other effects on the secondary outcomes were observed at post-intervention.

Six months later, the effect observed on the conviction score of the PDI-21 was sustained. Furthermore, other effects were observed, after Bonferroni's correction, on the distress and preoccupation subscales of the PDI-21. In addition, these results in the MG group were supplemented with an improvement in belief flexibility, as measured with the MADS item "anything against the belief."

The results do not show specific effects on the BPRS scale. The observed effects occurred on more detailed measures such as the conviction subscale of the PDI-21 (the main outcome), the distress and preoccupation subscales of the PDI-21, and one of the MADS items. This pattern possibly indicates that MG increases belief flexibility and changes the relation of the participants to their symptoms to some extent. As suggested elsewhere (19), the effect of CBT on psychotic disorders is probably of more interest for outcomes such as distress, preoccupation, and conviction related to beliefs than it is for psychotic symptoms as measured by the BPRS. Indeed, through specific training on hypothetical reasoning, MG seems to promote other viewpoints about what happens (as suggested by the observed changes on the PDI-21 and on the MADS), which is considered particularly helpful to patients dealing with stressful positive symptoms (9).

In the present study, as reported in a number of other studies on psychotherapy (36–43), post-intervention effects were sustained at follow-up on the primary outcome. The further improvements observed at follow-up on some outcomes possibly suggests that the effect of the intervention fosters additional changes. One may hypothesize that the first improvements on conviction and the patient's possible appropriation of the model of reasoning with hypotheses may allow additional experiences in daily life, and then improvements for at least some patients, as suggested by a previous case report (44). Future studies are needed, however, in order to assess the process that may be involved in such supposed effects.

One can also hypothesize that the game could be used as a warm-up program before applying it to an individualized CBT intervention for psychosis. The fidelity checks indicate good fidelity to the intervention in the assessed centers. These results show that an easy-to-use program such as MG may facilitate the dissemination of CBT (specifically hypothetical reasoning training) for psychosis in natural settings. The game may also familiarize health care professionals with the use of CBT techniques for psychosis. This of particularly high interest in consideration of the scarcity of psychotherapeutic treatments offered in clinical settings for people with psychotic disorders (16, 17, 45).

Dropout was relatively low. The game format may increase therapeutic alliance, a possible mediating factor of treatment outcomes in patients with psychosis (46). This aspect was not studied in the present paper, however, and may merit further studies. Dropout at T2 was associated with variables such as a younger age and a higher percentage of participants who dismissed the belief in response to a hypothetical contradiction. These patients possibly perceived less utility in the game than other patients did. The dropout rate associated with younger participants is in accordance with other observations (47) and suggests that the module could be improved to better engage them in the intervention or could be completed with other recovery-oriented interventions and psycho-education (20, 48).

In the absence of an active control group in our study, we cannot exclude the possibility that the observed effects were due to non-specific psychotherapeutic effects such as non-specific attention and interactions with the game leaders and other patients (45, 49). In addition, the absence of audiotaped sessions in some centers to control for fidelity to the intervention by group leaders may have missed poor fidelity in some centers. The lack of precise comparison of the TAU components in the 16 centers may have influenced the results. However, the analyses were controlled for center to correct for a possible cluster effect.

The multicentre design of the study may increase the possible generalization of the results. Furthermore, as reported in other studies (50), the limited number of exclusion criteria could contribute to the external validity of the study. We cannot fully rule out the influence of factors such as comorbid disorders (i.e., substance use disorders) or variations in cognitive deficits on the treatment outcomes.

## **Conclusion**

The results of this study indicate that the use of MG, a hypothetical reasoning training module, may improve belief flexibility and selfassessment of delusion in patients with psychotic disorders in comparison with not using it. MG appears to be a short and easyto-use intervention to disseminate CBT for psychosis in routine clinical settings. Further studies with active control group are still needed to improve the understanding of the possible effect of the game.

#### **Acknowledgments**

We thank the participants and the game leaders, as well as Joanne Corpataux, Milena Kriz, Dr. Giacomini, Claude Hayoz (Fondation HorizonSud), Dr. Georges Klein, Phillippe Laffond, Prof. Eric Bonvin, Prof. Sonia Dollfus, Prof. Thierry D'Amato, Prof. Jean-Michel Llorca, Dr. George Berthon, Dr. Céline Plassereaud, Dr. Véronique Roure, Natacha Boureykoff, Marie-Josée Durak, Dr. Maria Teresa Caldera, Dr. Roberta Bossoletti, and Dario Caffaro. *Funding*: Swiss National Science Foundation Grant 32003B-121038.

## **References**


**Conflict of Interest Statement:** Yasser Khazaal and Jerôme Favrod are the authors of the game. The other co-authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Khazaal, Chatton, Dieben, Huguelet, Boucherie, Monney, Lecardeur, Salamin, Bretel, Azoulay, Pesenti, Krychowski, Costa Prata, Bartolomei, Brazo, Traian, Charpeaud, Murys, Poupart, Rouvière, Zullino, Parabiaghi, Saoud and Favrod. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Development of the Positive Emotions Program for Schizophrenia: An Intervention to Improve Pleasure and Motivation in Schizophrenia

*Alexandra Nguyen1 , Laurent Frobert1 , Iannis McCluskey1,2 , Philippe Golay2 , Charles Bonsack2 and Jérôme Favrod1,2\**

*1School of Nursing Science La Source, University of Applied Sciences and Arts of Western Switzerland, Lausanne, Switzerland, 2Social Psychiatry Section, Community Psychiatry Service, Department of Psychiatry, University Hospital Center, Lausanne, Switzerland*

*Edited by: Shervin Assari, University of Michigan, USA*

#### *Reviewed by:*

*Masoumeh Dejman, Johns Hopkins School of Public Heath, USA; University of Social Welfare and Rehabilitation Sciences, Iran Ehsan Moazen Zadeh, Tehran Psychiatric Institute, Iran*

> *\*Correspondence: Jérôme Favrod j.favrod@ecolelasource.ch*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

*Received: 24 November 2015 Accepted: 25 January 2016 Published: 17 February 2016*

#### *Citation:*

*Nguyen A, Frobert L, McCluskey I, Golay P, Bonsack C and Favrod J (2016) Development of the Positive Emotions Program for Schizophrenia: An Intervention to Improve Pleasure and Motivation in Schizophrenia. Front. Psychiatry 7:13. doi: 10.3389/fpsyt.2016.00013*

Objectives: The efficacy of drug-based treatments and psychological interventions on the primary negative symptoms of schizophrenia remains limited. Recent literature has distinguished negative symptoms associated with a diminished capacity to experience, from those associated with a limited capacity for expression. The positive emotions program for schizophrenia (PEPS) is a new method that specifically aims to reduce the syndrome of a diminished capacity to experience.

Methods: The intervention's vital ingredients were identified through a literature review of emotion in schizophrenia and positive psychology. The program has been beta-tested on various groups of health-care professionals.

Results: A detailed description of the final version of PEPS is presented here. The French version of the program is freely downloadable.

Conclusion: PEPS is a specific, short, easy to use, group-based intervention to improve pleasure, and motivation in schizophrenia. It was built considering a recovery-oriented approach to schizophrenia.

Keywords: schizophrenia, anhedonia, apathy, pleasure, motivation, psychosocial interventions

#### INTRODUCTION

Negative symptoms have long been recognized as a central feature of the phenomenology of schizophrenia, dating back to early descriptions by Kraepelin and Bleuler (1). They negatively affect patients' longitudinal social, occupational, and functional outcomes, as well their longterm recovery (2–5). Whereas positive symptoms (hallucinations, delusions) reflect an excess or distortion of normal functions, and negative symptoms (blunted affect, alogia, apathy–avolition, anhedonia, inattentiveness) represent the absence or reduction of normal emotions and behaviors. Negative symptoms are classified as primary or secondary. Primary negative symptoms comprise the core features intrinsic to schizophrenia itself. Secondary negative symptoms are transient;

they are attributable and temporally related to the effects of such factors as unrelieved positive symptoms, depression, the adverse effects of antipsychotic drugs (akinesia), or the social isolation imposed by the stigma of schizophrenia. Primary and secondary negative symptoms may be similar in clinical expression, despite their contrasting etiologies (6). Often, secondary negative symptoms diminish with the resolution of their causative factors.

#### Limitations of Current Techniques

The efficacy of drug-based treatments and psychological interventions on primary negative symptoms remains limited, however (7–9). Fusar-Poli et al. (9), in their meta-analysis of 6,503 patients in the treatment arm and 5,815 patients in the placebo arm, showed that most treatments reduced negative symptoms at follow-up relative to placebo: second-generation antipsychotics, antidepressants, combinations of pharmacological agents, glutamatergic medications, and psychological interventions. However, none of the treatments used reached the threshold for clinically significant improvement as measured by clinicians using the Clinical Global Impression Severity Scale. The mean percentage change in treatment groups was 16.1%, whereas the control group changed by an average 7.9%. Improvements by treatment group, compared with the control group, varied between 4.8% for first-generation antipsychotics and 12.7% for psychological treatments. There is a clear clinical need for developing treatments for negative symptoms. The lack of clinically meaningful efficacy of drug or psychological treatments is in line with clinicians' practical experiences (7, 9). The existing psychological treatments have limitations (10). Family interventions to reduce negative symptoms appear promising when combined with other interventions. However, patients who have family members willing to be involved in treatment may merely represent a subgroup of people with schizophrenia. Furthermore, families who are more willing to participate in psychosocial interventions may provide patient's with greater support anyway and thus may not represent the broader population. Psychosocial interventions, such as cognitive behavior therapy (CBT), social skills training (SST), mindfulness-based intervention (MBI), family therapy (FT), or cognitive remediation (CR), combined with SST require highly trained therapists. The vast majority of studies were not specifically designed to target negative symptoms, and most assessed negative symptoms overall, without looking for the specific effects of the intervention. Several interventions were provided over long durations, requiring a great investment in time by the patients. For example, combining CR with SST may be useful for reducing negative symptoms, particularly social withdrawal, affective flattening, and motor retardation (11, 12), or a global score of negative symptoms (13). These interventions required 32–100 h of training. One 3-year study (14) indicated that improvements in negative symptoms did not occur until the second and third years of therapy. However, a second study (15, 16) indicated improvements in negative symptoms at 12 and 24 months. These comprehensive packages are promising but involve relatively long interventions and call for a broad range of therapeutic techniques. Except SST, very few interventions are provided as group therapy. Group interventions are particularly important since they are more commonly used in community psychiatry settings, where most patients receive treatment services.

#### Experimental Objectives

Recent literature has distinguished the negative symptoms associated with a diminished capacity to experience (apathy, anhedonia) from those who were associated with a limited capacity for expression (emotional blunting, alogia) (17–20). The apathy–anhedonia syndrome tends to be associated with a poorer prognosis than the symptoms related to diminished expression, suggesting that it is the more severe facet of the psychopathology (19). This syndrome is also related to the duration of the untreated psychosis, family history of schizophrenia, and the patient's employment status during first-episode schizophrenia (21). The distinction between diminished experience and limited expression syndromes allows more specific approaches to these problems.

A symptom-specific strategy has been used in the development of specific therapeutic techniques for positive symptoms (22, 23) and led to the development of more effective interventions, such as CBT for delusions or hallucinations (24). More recently, metacognitive training (MCT), which targets associated specific cognitive biases with psychosis, has appeared effective in reducing positive symptoms (25–27). A similar strategy appears to be a good way to develop psychological interventions for negative symptoms. Recent research has also shown that more specific symptom or syndrome approaches enabled a better identification of specific psychological mechanisms. For example, the endorsement of beliefs regarding low expectations of success, and perceptions of limited personal resources, are robustly associated with negative symptoms of diminished experience (anhedonia, avolition, asociality), but are not associated with negative symptoms of diminished expressivity (flattened affect, alogia). Similarly, defeatist performance beliefs are slightly related to diminished experience, but not at all related to diminished expression (28). An impaired ability to envision the future is associated with apathy (29). These results suggest that within the syndrome of diminished capacity to experience, apathy and anhedonia may be the results of the same underlying process: that is, a diminished capacity to anticipate a particular experience or the achievement of a pleasurable goal (18), or a motivational impairment (30). This article presents a new method. It is hoped that it will open new avenues of experimental investigation for interventions to improve the diminished expression syndrome in schizophrenia. The intervention is called the positive emotions program for schizophrenia (PEPS). It comprises a program of 8 1-h sessions applied to groups of 5–10 participants. The results concerning the participants have been published elsewhere (31). This paper presents the development of the program before the pilot study with participants.

#### METHODS VALIDATION

#### Identification of the Component of PEPS

Anhedonia has been defined as a reduction in the ability to experience pleasure. Despite its clinical significance, research into anhedonia has produced a paradoxical set of findings, raising questions about its nature. On the one hand, using self-reported measures of trait social and physical anhedonia, individuals with schizophrenia typically report experiencing lower levels of pleasure in their daily lives than non-patients (32–35). On the other hand, in laboratory studies using emotionally evocative stimuli, individuals with schizophrenia have repeatedly reported experiencing levels of pleasant emotions similar to, or even stronger than control subjects (36–38). Germans and Kring (39) resolved this inconsistency by suggesting that patients do not anticipate that pleasurable activities will indeed be pleasurable, even though they experience pleasant emotions when presented with pleasurable stimuli. This explanation is founded on the distinction between appetitive/anticipatory pleasure (i.e., anticipating the potential pleasure of taking part in a future activity) and consummatory pleasure (i.e., the actual level of pleasure experienced directly from participating in an activity). Anticipatory pleasure is linked to motivational processes that stimulate goal-directed behaviors, whereas consummatory pleasure is associated with satiety. The Temporal Experience of Pleasure Scale (TEPS) is a trait measure of pleasure (40) that distinguishes between "momentary pleasure" and "anticipation of future pleasant activities." A TEPS score study, comparing subjects with schizophrenia to controls, indicated that patients did not differ from controls on the consummatory scale; however, they reported significantly less anticipatory pleasure than controls (41). These results were replicated by the French version of TEPS (42). Bringing out this new way of conceptualizing anhedonia in schizophrenia permits a redefinition and calibration of the symptom complex as a target for treatment. If patients with schizophrenia show a deficit in their ability to anticipate pleasure, rather than experience pleasure, then cognitive training might well help these individuals anticipate pleasure from foreseeable, future activities. Ideally, treatment would lead to a greater ability to anticipate pleasure, and this, in turn, would lead to a meaningful increase in spontaneous daily activities. These considerations led us to explore the potential for an intervention that would train patients who complained of anhedonia, or a lack of desire to engage in activities, in the cognitive skills needed to increase their anticipatory pleasure (43). This first, exploratory pilot study included five participants with schizophrenia, presenting severe anhedonia, and stabilized on atypical antipsychotic medication. They received 10–25 h of training in anticipatory pleasure. Results showed that the patients improved on the anticipatory scale of TEPS. The patients' daily activities were also increased according to a time budget. These preliminary data were, of course, interpreted with caution, given the small study sample, but they seemed to show a promising path toward the development of new interventions to alleviate anhedonia in schizophrenia.

Further emotional deficits may be present in schizophrenia (44) and should be taken into account in the development of new interventions (45, 46). Strauss (46) suggested maximizing positive emotional experiences by using techniques developed in the field of affective science (47, 48) to increase the frequency and duration of positive emotional experiences. Five techniques have been found to specifically and reliably increase the frequency, intensity, and duration of positive emotions, including anticipating the enjoyment. The others were behavioral display (expressing emotions via non-verbal behaviors), being "in the moment" (directing controlled attention toward positive experiences when they occur-savoring), communicating and celebrating positive experiences with others, and recalling previously pleasurable events. Patients reported lower levels of pleasure in savoring past, present, and future events than did normal controls, and stated that they had low expectations of their self-efficacy (49). Individuals with schizophrenia also manifested a lesser ability to maintain positive emotions (50–52). Even though observable, outward signs of emotional expression were lessened in schizophrenia, studies indicated that sufferers continued to display very subtle facial muscle movements (as measured by electromyogram) similar to, and in accordance with, their responses (53). Finally, to the best of our knowledge, it appears that communicating and celebrating positive events with others has not been studied in schizophrenia patients. However, one study showed that impaired perspective-taking – a component of cognitive empathy – was associated with functional capacity and community functioning, even after taking into account the influences of neurocognitive deficits and psychopathology (54).

With this as a background, Jérôme Favrod and Alexandra Nguyen conceived an intervention, which they named the "positive emotions program for schizophrenia," to reduce anhedonia and apathy. The program teaches skills to help overcome defeatist thinking (55, 56) and to increase the anticipation and maintenance of positive emotions (44, 45). PEPS involves eight 1-h group sessions, administered using visual and audio materials as part of a PowerPoint presentation of slides projected onto a screen.

#### Beta-Testing Procedure

During its development, PEPS, as well as the sensitivity of selfreporting instruments, was beta-tested on volunteer health-care professionals in order to improve its efficacy on anticipatory and consummatory pleasure, as well as savoring. Four 1-day training sessions of 7 h of PEPS were beta-tested on four different mixed groups of health-care professionals. At the end of each session, oral feedback and advice were collected, as were pre- and posttest assessments. Sessions 1–3 were conducted between February and March 2014 using version 1.0 of PEPS. Session 4 was conducted in March 2015 using the improved version 1.1 of PEPS, developed on the basis of the results of the previous sessions.

#### Participants

Participants were health-care professionals, including psychiatrists, psychologists, nurses, occupational therapists, and social workers, interested in participating in the program's development. No participant followed more than one session. Participation was anonymous, and the subjects gave only their sex and age. The clinical studies with PEPS have the agreement of the Vaud Cantonal Ethics Commission on Human Research (127/14 and 446/15).

#### Instruments

The following measurement instruments were used:

• *The Savoring Belief Inventory (SBI)* is a self-reported scale for measuring beliefs about one's capacity for savoring things. The scale has 24 items, including a positive scale (12 items) and a negative scale (12 items). The inventory has good validity and a high test–re-test reliability (57). It measures a person's thinking regarding their capacity to savor positive experiences in terms of past experiences, current experiences, and future anticipation. The total SBI score is used.


#### Data Analysis

Two-tailed paired sample *t*-tests of pre- and posttest results were calculated using IBM SPSS Statistics Version 22. Cohen's *d* effect sizes were calculated for within-subjects in correcting for dependence among means. Formula 8 from Morris and DeShon (61) was also used.

#### RESULTS

#### Results of the Beta Tests

**Table 1** shows that, in session 1, the 21 participants improved on their scores significantly and clinically on the anticipatory and consummatory scales of TEPS, but not on ACIPS. Session 2 tested PEPS using the SBI scale, and scores improved clinically and significantly for the 16 participants. Session 3 replicated the previous results of ACIPS and the SBI with 27 participants. Since version 1.0 did not improve interpersonal pleasure scores as measured by ACIPS, version 1.1 was upgraded to put more emphasis on this factor. Version 1.1 includes meditations focusing on caring for others and exercises involving interpersonal pleasure. Version 1.1 was beta-tested in session 4. The 28 participants in session 4 improved their scores for both TEPS and ACIPS. No adverse effects were observed or reported during the four sessions with a total 92 different participants.

#### Detailed Description of the Final Version of PEPS

The pedagogical concept underpinning PEPS was designed according to Kolb and Kolb's model (62) of experiential learning. This model sees the learning process as the transformation of an experience into personal knowledge. The sequential organization of the learning activity starts with the learner experiencing something (the concrete experience phase). This is followed by a stage of distancing oneself from the experience through a period of observation and reflection that seeks to give the experience meaning (the reflective observation phase). Distancing oneself from the experience broadens the learner's understanding, generalizing, and developing concepts through more abstract thought (the abstract conceptualization phase). The learner then initiates an experimental approach to validate the newly acquired knowledge through reality tests (the active experimentation phase). This model's major contribution is its dynamic conception of learning, seen as "a process, not in terms of results" (63). The model claims to provide a supportive environment for all learners since it is based on adults' different learning strategies and styles, all of which can be activated through the four phases. Therefore, Kolb's model is relevant to a therapeutic program insofar as its design corresponds to a sequential logic – alternating phases of experience and reflection. The logo at the top-left of the each slide is a reminder to group leaders as to which phase the session is in.

The program uses a collaborative, egalitarian approach. Group facilitators participate in sessions just as the participants do, by doing the exercises, sharing their experiences, and carrying out


*TEPS, Temporal Experience of Pleasure Scale; SBI, Savoring Belief Inventory; ACIPS, Anticipatory and Consummatory Interpersonal Pleasure Scale.*

the given tasks. Group facilitators receive a day's training before leading a group themselves and are supervised during two 1-h periods during the program.

Each session includes a number of the following steps. Part 1 begins with a welcome, followed by a 5-min relaxation–meditation exercise. In part 2, group leaders go over the homework task given during the previous session. Part 3 involves an exercise in challenging specific defeatist thoughts, which are presented using the program's two fictitious heroes – Jill and Jack. Jill, for example, expresses such defeatist thinking as "I can't relax; I'm useless." The participant's role is to challenge her belief, initially by assigning different reasons to why Jill has difficulty relaxing. They learn to find reasons that might be linked not only to the program's heroine but also to other people or her environment. They subsequently try to develop an alternative, more positive way of thinking.

The following slides show how the exercise appears to the participants. This first slide presents a defeatist belief.


The next slide presents questions to find a more balanced set of reasons to explain Jill's belief that she is unable to relax. Since defeatist beliefs are often linked to the internal attribution of failures, participants are asked to find a set of different reasons to explain why Jill was not able to relax. These questions are asked by the group leaders to the participants. Each of the eight sessions of PEPS will address a defeatist belief using the same methods in order to facilitate patients' understanding of how the method works.


The third slide only appears after the participants have given answers to slide 2. Other suggestions are also given to complement or confirm the work they did for the previous slide.


The fourth slide asks the participants to give an alternative to Jill's conclusion and defeatist belief that she is useless.

The final slide in this sequence about changing defeatist beliefs gives suggestions to complement or confirm the work the participants did for the previous slide.

	-
	-
	-
	-

Subsequently, and according to the session's theme, participants learn and practice a new skill to improve their anticipation or maintenance of pleasure. The session ends with group leaders setting the homework task that the participants must accomplish for the next session.

The skills taught include savoring a pleasant experience, expressing emotions by increasing behavioral expression, making the most of or capitalizing on positive moments, and anticipating pleasant moments in the future. Savoring a pleasant experience involves becoming aware of that pleasure or of the positive emotions the participant feels at a given moment (47). For example, participants are asked to look at a picture of pleasant countryside or listen to soothing music, and hence become aware of the pleasurable experience of doing this and thus appreciate it. Increasing behavioral expression of emotions involves using facial expressions or gestures to accompany that positive emotion. The participants are asked to imitate pictures of actors expressing a positive emotion and to become aware of the sensations this produces. The next slide introduces the exercise.


The next slide is an example of a picture to imitate.

The following slide allows participants to think about the exercise. Group leaders will help each participant to share their feelings during the exercise. The group leaders will also describe what they have felt, in order to help demonstrate ways of sharing experiences and guide participants in self-observation.

Making the most of positive moments entails communicating and celebrating positive events with others. For example, participants are asked to describe positive events to one another through role playing. The next slides show a complete sequence of training with its experiential learning steps: concrete experience, reflective observation, abstract conceptualization, and active experimentation. During the concrete experience phase, participants experience the skill they are to be taught. During reflective observation phase, they consciously express and formulate the ingredients of that experience. During abstract conceptualization phase, the participants receive theoretical information about the skill they are being taught. Finally, during active experimentation phase, they practice the skill in their natural environment as homework. The homework is reviewed at the beginning of the next session.





Anticipating pleasant moments involves imagining the sensations produced by positive future events. This strategy is meant to guide the participants through different positive feelings and emotions. It can engage their senses, for example, by imagining they are eating a smooth, shiny, crunchy, tasty, sweet-smelling fruit, or by anticipating the emotion produced and the physical sensations experienced upon the completion of a pleasurable physical or social activity. A simple homework task is assigned to be done between each session. For example, this could be choosing an image or an object that provokes a positive emotion or feeling in the participant, who must then bring it back and describe it to the group. The original French version of PEPS can be downloaded for free at http://www.seretablir.net/outils-interventions/peps/.

## DISCUSSION

#### Key Findings

Drug and psychological treatments for the negative symptoms of schizophrenia have shown poor clinical efficacy. Furthermore, they require lengthy therapeutic interventions involving highly skilled professionals. This paper presents a specific, short, easy to use, group-based intervention to improve pleasure, and motivation in schizophrenia. A more targeted syndrome approach was grafted to a model of the advancement of psychological therapies for delusions. The program targets apathy and anhedonia as they can be combined into a single reduced-experience syndrome. The program was built with regard to the specific deficits described in the literature on pleasure and motivation. The program was developed by beta-testing on health-care professionals and was improved according to the results of earlier tests. A pilot study was conducted with participants who met the ICD-10 criteria for schizophrenia or schizoaffective disorders (31). Thirty-one participants completed the program; those who dropped out did not differ significantly from completers. Participation in the program was accompanied by statistically significant reductions in the total scores for avolition–apathy and anhedonia–asociality on the Scale for the Assessment of Negative Symptoms, with moderate effect sizes. Furthermore, there was a statistically significant reduction of depression on the CDSS, with a large effect size. Emotional blunting and alogia remained stable during the intervention. The original program in French can be downloaded for free on the Internet. It was designed to be easy to use and applicable in group sessions so as to meet the needs of community care psychiatry.

#### Potential Shortcomings and Limitations

Positive emotions program for schizophrenia was designed using a recovery-oriented approach rather than a deficit-centered approach. The deficits of pleasure, and above all of motivation, may be related to factors that were not selected as targets in the program. This could be mainly because the authors were looking for a rapid, targeted intervention on the diminished experience syndrome. For example, apathy is associated with poor performance on executive tests (64–66). CR has been shown to improve executive functioning, but although CR yields lasting effects on global cognition and functioning, its influence on symptom effects is small and disappears over time (67). However, a recent study indicated that anticipation abnormalities are associated rather to negative beliefs about potentially rewarding social situations than neurocognitive deficits (68). Motivation may also be affected by medication; for example, the D2 antagonistic effect of antipsychotic agents reduces anticipation of a monetary reward (69). PEPS does not include medication in its intervention.

The use of health-care professionals in the beta tests for developing the program may be questionable since they were not representative groups of people with schizophrenia. The high female-to-male ratio of the beta test samples did not fit with the high male-to-female ratio of patients with enduring schizophrenia (70). However, these professionals were easy to reach out to, familiar with the problems surrounding schizophrenia, and able to give knowledgeable feedback on PEPS exercises. However, this familiarity with schizophrenia may also be a bias against innovation. The developers were well aware of these risks. The main reasons for the beta tests were to quickly evaluate the intervention's feasibility, its impact on the self-reporting scales selected, and the program's safety before the clinical phase in the development of PEPS.

## Future Directions

The findings presented here indicate that PEPS is indeed a feasible intervention, and it was associated with an apparently specific reduction of anhedonia and apathy. However, these findings are limited by the absence of a control group and the fact that the rater was not blind to the treatment objectives. A randomized controlled study with blind raters is needed to assess more correctly the efficacy of PEPS.

## AUTHOR CONTRIBUTIONS

AN and JF, in equal measure, conceptualized this research and PEPS, acquired, analyzed, and interpreted the data, and drafted the first version of the manuscript. LF, IM, PG, and CB gave a substantial contribution to the analysis and interpretation of data and critically revised the article for important intellectual content. All the authors approved the final version for publication. All the authors agree to be accountable for all aspects of the work by ensuring that any questions related to its accuracy or integrity can be appropriately investigated and resolved.

#### REFERENCES


#### ACKNOWLEDGMENTS

Sébastien Perroud, known as PET, drew the cartoons; Charlotte Aeschbacher and Yael Horowicz were the models for the photographs depicting positive emotions.

#### FUNDING

This work was supported by a donation from Dr. Alexander Engelhorn and Swiss National Science Foundation grant number 105319\_163355/1.

treatment for patients with a first episode of psychotic illness. *BMJ* (2005) **331**:602. doi:10.1136/bmj.38565.415000.E01


**Conflict of Interest Statement:** The authors declare that this research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Nguyen, Frobert, McCluskey, Golay, Bonsack and Favrod. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Improving Social Cognition in People with Schizophrenia with RC2S: Two Single-Case Studies

*Elodie Peyroux1,2\* and Nicolas Franck1,2,3*

*1Rehabilitation Department (CL3R), Le Vinatier Hospital, Lyon, France, 2UMR 5229, Center of Cognitive Neurosciences, CNRS, Lyon, France, 3University Claude Bernard, University of Lyon, Lyon, France*

Difficulties in social interactions are a central characteristic of people with schizophrenia, and can be partly explained by impairments of social cognitive processes. New strategies of cognitive remediation have been recently developed to target these deficits. The RC2S therapy is an individualized and partly computerized program through which patients practice social interactions and develop social cognitive abilities with simulation techniques in a realistic environment. Here, we present the results of two case-studies involving two patients with schizophrenia presenting with specific profiles of impaired social cognition. Each patient completed three baseline sessions, 14 treatment sessions, and 3 follow-up sessions at the end of the therapy – and for 1 patient, another 3 sessions 9 months later. We used a multiple baseline design to assess specific components of social cognition according to the patients' profiles. Functioning and symptomatology were also assessed at the end of the treatment and 6 months later. Results highlight significant improvements in the targeted social cognitive processes and positive changes in functioning in the long term. The RC2S program seems, thus, to be a new useful program for social cognitive remediation in schizophrenia.

#### *Edited by:*

*Stéphane Bouchard, Université du Québec en Outaouais, Canada*

#### *Reviewed by:*

*Jude Uzoma Ohaeri, University of Nigeria, Nigeria Ali Oker, Télécom ParisTech, France*

*\*Correspondence: Elodie Peyroux elodie.peyroux@gmail.com*

#### *Specialty section:*

*This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry*

> *Received: 31 January 2016 Accepted: 01 April 2016 Published: 25 April 2016*

#### *Citation:*

*Peyroux E and Franck N (2016) Improving Social Cognition in People with Schizophrenia with RC2S: Two Single-Case Studies. Front. Psychiatry 7:66. doi: 10.3389/fpsyt.2016.00066*

Keywords: social cognition, schizophrenia, cognitive remediation, simulation techniques, single-case design

## INTRODUCTION

#### Social Cognition in Schizophrenia

The most consensual definition of social cognition may be the one proposed by Adolphs (1) as "the ability to construct mental representations of the relation between oneself and others and to use those representations flexibly to guide social behavior." In social neurosciences, the study of social cognition is at the interface of basic perception processes and behavior in the social world, and includes research in several domains, from social psychology to cognitive sciences (2). In psychiatry, social cognition has been studied since the 1990s in populations with social dysfunction as a central characteristic and diagnostic criterion, such as people with schizophrenia and related disorders. In this disease, impairments have been highlighted at all levels of social processes, from basic perception to social functioning. Social cognitive deficits are now defined as a central characteristic of schizophrenia by the American Psychiatric Association (3).

In people with schizophrenia, social cognitive impairments are a core feature of the disease (4). According to renown researchers, three to five social cognitive processes are usually altered in schizophrenia (5–8): (a) *emotional processing*, or the ability to identify emotions through facial expressions, gestures, and tone of voice; (b) *Theory of Mind* (ToM), defined as "the ability to attribute mental states – beliefs, intents, desires, pretending, knowledge, etc. – to oneself and others and to understand that others have beliefs, desires, and intentions that differ from one's own" (9); (c) *attributional style*, which refers to how people explain the causes of positive and negative events; in schizophrenia, it is the equivalent of "self-serving" and "personalizing bias," i.e., the tendency to blame others for negative life events rather than share the responsibilities between different sources (10); and (d and e) *social perception and knowledge*, which can be defined as decoding and interpreting social cues from others, taking the social context into account, and being aware of social rules, roles, and goals.

The components of social cognition appear to be related to both symptomatology and functioning in everyday life. The links between cognitive impairments and symptoms in schizophrenia have been studied since the 1990s. The role of certain cognitive processes in the production of symptoms was first highlighted by Frith's (11) hypothesis establishing links between self-monitoring and hallucinations. Research on social cognition is more recent. However, it already appears that the alteration of the social processes that allow for the understanding of others' mental and emotional states may influence both positive and negative symptoms. The positive dimension of schizophrenia has been significantly associated with facial emotion recognition (12) and ToM (13, 14). Attribution bias, and especially hostile attribution bias, has also been correlated with paranoid and persecution delusions (7). Nevertheless, the literature does not always provide consistent results about the links between social cognitive processes and positive symptoms. For the most part, the literature (including recent research) deals with relations between negative symptoms and social cognition and functioning. In this field, specific and significant links between ToM and negative symptoms have been highlighted (15). Other authors have suggested that ToM may be related either to excessive (positive symptoms) or to defective (negative symptoms) attribution of mental states to others (14). Other correlations between negative symptoms and emotional processes have been shown, which suggests that the negative dimension of schizophrenia is closely related to specific processes of social cognition, although both remain two independent constructs (16). To conclude, it seems that social cognitive impairments in schizophrenia, and especially ToM impairments, are directly associated with the severity of negative symptoms (17). According to Couture et al. (18), social cognition and negative symptoms associated with neurocognitive abilities are three pertinent factors to predict the functioning of people with schizophrenia in everyday life.

The social difficulties experienced by people with schizophrenia progressively lead to isolation and relational problems in family, social, and vocational areas. Their quality of life and autonomy are, therefore, strongly impacted, and several studies have looked into the links between social cognition and social functioning. Overall, it seems that social cognition plays a key role in social functioning, professional life, and interpersonal relationships (18, 19). Nevertheless, these links depend on specific social cognitive processes. Couture et al. (19) proposed a conceptual model of relationship between social cognition and functioning. According to them, when people with schizophrenia are facing social interaction, they may first misperceive the emotional expression on the interlocutor's face without observing additional information from the context. Such misperceptions may then result in drawing the wrong conclusion about the interlocutor's emotional state. Subsequently, the next processing phase involves finding an explanation as to why the interlocutor is feeling this way. Biases in attributional style, such as personalizing bias, may lead patients to draw wrong conclusions about the cause of their interlocutor's emotions, since ToM deficits make it difficult for them to walk in someone else's shoes. This inability to grasp the emotional and social context of the interlocutor's behavior may lead patients to act inappropriately. According to Couture et al.'s model, impairments (or biases) in social cognition may impact a variety of indices of functional outcomes.

Other meta-analyses were dedicated to the question. Fett et al. (20) highlighted that social cognitive (especially ToM) impairments had major consequences on community functioning, vocational activities, and interpersonal relationships. By comparing the strength of the relations between cognition and functioning, the authors found evidence that social cognition in schizophrenia is more strongly connected to community functioning than neurocognition. Other studies also showed that the quality of life of people with schizophrenia depended more on social cognition (especially ToM) than neurocognition, whereas the opposite was observed in both healthy controls and relatives (21). Accordingly, social cognition (especially ToM) may be a more significant predictor of functional outcomes in schizophrenia.

Several recent modeling studies have more precisely examined the influence of various cognitive factors on social and functional components. Some authors, thus, suggested that social cognition acted as a mediator between neurocognitive functions and community functioning (18, 22, 23). In this context, neurocognition affected functioning through its impact on social cognitive processes. The mediating role of social cognition was also highlighted for professional life (24). Recently, hybrid models of social cognition, acting both as a mediator and an independent contributor, were developed. In those models, neurocognition and social cognition are two independent components that participate separately but closely in daily functioning; social cognition may act as a mediator between some neurocognitive components and functional dimensions (25, 26).

At the moment, one of the most commonly cited models is Schmidt et al.'s (27). The authors gathered 15 studies assessing the mediating role of social cognition between neurocognition and functional status. By applying a structural equation model, they found that neurocognitive impairments in schizophrenia had a negative effect on social cognitive processes, which increased their negative influence on functioning. Schmidt et al.'s model, therefore, confirmed the role of social cognition as a mediator between neurocognition and functioning. Their distal mediation model explained a large proportion of the variance in difficulties observed in functioning – between 7 and 41% with an average of 20%. Nevertheless, other components, and especially social discomfort, support and skills, motivation (28), and negative symptoms (18, 29) were also highlighted as mediating factors between cognition and functioning in schizophrenia.

## Cognitive Remediation of Social Cognition and Advantages of Virtual Environments

In recent years, several cognitive remediation therapies have been developed to treat social cognitive impairments. These interventions use specific learning strategies that overlap with those used in cognitive remediation therapies focusing on neurocognitive processes (30). Strategies include errorless learning (i.e., avoiding the implicit encoding of errors), scaffolding (i.e., minimizing errors by carefully regulating the complexity of learning material), massed practice (i.e., the repetition of tasks to encourage retention and application of skills), information processing strategies (including verbalization, information reduction, breaking, and simplifying the task into various steps), and positive reinforcement (31). The few meta-analyses that have assessed the impact of social cognitive remediation programs highlighted that such therapies showed encouraging results in both patients' interest and motivation, and social cognitive processes. Kurtz and Richardson's (32) meta-analysis, including data from 19 studies, highlighted significant effect of social cognitive remediation programs on emotion recognition (moderate-to-large effect) and ToM (small-to-moderate effect). However, most programs did not cause any change on measures of attributional bias. The review of Fiszdon and Reddy (33), including nearly 50 studies and focusing on proximal effects of programs targeting social processes, maintenance of those effects, and generalization to functional outcomes, corroborated previous results. According to them, basic processes of social cognition, such as emotion recognition and social perception, could definitely be improved by cognitive remediation. But these techniques were less successful in remediating more complex, higher-order social cognitive functions. The authors suggested that this might be due to difficulties taking into account or compensate for neurocognitive impairments in schizophrenia, or to limited opportunities for patients to practice skills.

In fact, while the use of computer programs has become very popular in the field of neurocognition, as they offer the possibility to adapt the difficulty level to the specific skills of each patient, to give immediate feedback concerning performance, and to readjust the reinforcement methods used (34), very few has been used in social cognition therapy. This probably arises because it is assumed that interacting with a machine does not involve social skills. This is why, in this field, strategies derived from social skills training, such as role play and exposure to social situations, are used in most programs. It seems, however, that social cognition can be learned directly via computer programs because computerized tasks often provide the opportunity to decompose and control the different processes at play in social interactions, and, thus, to offer a progressive training program regarding difficulty. Moreover, other technologies, such as virtual reality or simulation techniques, are even more applicable.

These techniques expose patients to complex, dynamic, and interactive stimuli and can, thus, serve to remedy cognitive, behavioral, and functional impairments of people with schizophrenia in tasks very close to those found in daily life (35). Indeed, virtual reality and simulation techniques can provide safe and harmless environments where patients can learn social skills without negative repercussions on their real life, such as emotional frustration or a feeling of failure. For these reasons, a number of recent clinical studies have explored the advantage of virtual social skills training, with very positive results, in the field of both autism disorders (36–38) and schizophrenic disorders (39–43). Nevertheless, these programs often focus on general social skills and do not take specific processes of social cognition into account, which are in fact essential to social functioning. In this context, simulation techniques seems highly relevant by allowing to transfer skills, developed in paper-and-pencil tasks, to a realistic and complex environment close to difficulties encountered by patients in daily life. Moreover, traditional programs of social cognitive remediation often focus on relatively simple associations and deductive reasoning involving only one component of social interactions. They use a hierarchical step-by-step model where each process is taught independently and thereby lacks the characteristics of real social situations. We postulate that the RC2S program, by providing an environment similar to the real social world for patients to practice skills in specific social interactions, may be helpful in improving higher-order social cognitive functions and to promote one of the greatest challenges for cognitive remediation: the transfer of acquired abilities to everyday life following treatment and the generalization of treatment benefits to other social processes.

#### RC2S Program

RC2S is a comprehensive, individualized, and partly computerized social cognitive remediation program (44). It was developed in France through collaboration between the Rehabilitation Department of Le Vinatier Hospital in Lyon and the SBT Company. The treatment lasts 14 weeks at a pace of one 1-to-1 h 30 min session per week, and includes both paper-and-pencil tasks and computerized exercises. The therapy is divided into three main parts.

First, the preparation sessions usually take place over two sessions; the patient and his/her therapist analyze the social cognitive assessment previously performed to determine the patient's strengths and weaknesses in social cognition. The functional outcomes of social cognitive impairments are examined using a specific questionnaire (Social Cognition-Functional Outcomes Scale or ERF-CS1 ); and information about this cognitive domain are also given thanks to a psychoeducation document (also available in French at the following address: see text footnote 1). Finally, the patient sets two or three concrete objectives designed to improve his/her social functioning by the end of the therapy.

Then, 10 cognitive remediation sessions with the therapist take place once a week for 10 weeks. Each session is divided into four parts: (1) paper-and-pencil tasks for the patient to develop strategies to analyze social situations, focusing first on basic social cognitive processes, such as emotion recognition or social perception, and then on higher order social cognitive functions, such as ToM or attributional style, depending on his/her specific needs and difficulties; (2) a simulation scene in which the patient's goal is to assist a character named Tom in a particular social

<sup>1</sup>http://www.remediation-cognitive.org/documents

situation and guide him by choosing among various propositions of behavioral patterns after each conversation; (3) a review of the situation in which the patient's choices during the simulation scene are decomposed to focus on specific social components, such as contextual information, tone of voice, gestures, and facial expressions; the therapist can also ask the patient to play the scene again to generate specific mental states in the characters Tom is facing (e.g., anger, envy, or discomfort), identify key components that may appear in real-life interactions, and think about the characters' possible reactions or intentions; and finally (4) the determination of a home-based task chosen by the patient in collaboration with the therapist and related to the concrete objectives defined at the beginning of the therapy.

Finally, two transfer sessions take place at the end of the social cognitive remediation program. The therapist and the patient review the work done throughout the therapy and assess the achievement of the objectives. These sessions also help the patient transfer the skills acquired during the remediation program to his/her daily life by working on his/her specific difficulties. Thanks to these transfer sessions, the patient can adapt strategies to other social interactions.

This paper reports the RC2S treatment of two patients suffering from schizophrenia. Since cognitive impairments in patients with schizophrenia or related disorders are very diverse, we carried out single-case studies to assess the impact of this new cognitive remediation program. Single-case experimental design has indeed been considered by several authors to assess individualized interventions involving functional issues (45). The two patients enrolled in the studies underwent a complete cognitive, social, and clinical functioning assessment. According to each patient's profile and objectives, we determined three kinds of baselines before the intervention: targeted component-specific baselines (SBLs); non-SBLs, i.e., measures of neurocognitive processes that should not be affected by the intervention; and IBLs, i.e., measures of social cognitive functions linked to the targeted processes but not directly concerned by the cognitive remediation program. The measures and complete assessments were repeated at the end of the intervention to highlight the impacts of RC2S on social cognitive impairments, and 9 months later (in only one patient) to investigate the possible maintenance of benefits.

#### MATERIALS AND METHODS

#### Participants

Damien (Patient A) is a 28-year-old man with a diagnosis of schizophrenia. He presented with difficulties in social relationships and a tendency to withdrawal. He had difficulties communicating with others and exchange during conversations. At the time of recruitment, Damien's condition was stable after a 5-month hospitalization.

Robin (Patient B) is a 30-year-old man suffering from schizophrenia. He complained about interpersonal relationships, with a difficulty to understand the emotions and mental states of others. He had a tendency to feel hostility from others that affected his vocational integration. Actually, previous relations with his coworkers and superiors were troubled and led to the end of his professional activity as a seasonal worker.

These two single-case studies were carried out in accordance with the Declaration of Helsinki and was approved by the local Ethical Committee (CPP Lyon – Sud Est IV, no. 13/014; AFSAPPS, no. 2013-A00131-44). Written informed consent to take part in the study was received from all participants.

#### Assessments

#### Clinical and Functional Assessments

Before starting the RC2S program, clinical, cognitive, and daily life functioning was assessed (the clinical and functional characteristics of patients A and B are summarized in **Table 1**).

Psychotic symptoms were measured with the positive and negative syndrome scale (PANSS) (46). Before the intervention, symptoms in Damien were not very active, with a moderate score of positive symptoms and a slightly higher score of negative symptoms. Symptoms in Robin were also stable and not very active; both positive and negative symptoms were moderate.

Some aspects of daily functioning were also assessed before the treatment. We measured self-esteem with the Self-Esteem Rating Self (SERS) (47) and mental well-being with the Warwick– Edinburgh Mental Well-Being Scale (WEMWBS) (48). Damien's self-esteem seemed stable and quite positive, whereas Robin's was slightly negative. The patients' mental well-being scores were low and close to the average score of people considering themselves to be in bad health. Finally, the Social Autonomy Scale (*EAS*) (49) was used to assess Damien's daily and social functioning. Overall, total scores and sub-scores were slightly high, indicating limited autonomy.

#### Cognitive Assessments

Neurocognition and social cognition were assessed in both patients by a different neuropsychologist from the RC2S therapist. Damien's neurocognitive functioning was affected by weak executive functioning with impaired planning skills [*test des commissions* – revised version; (50)] and mental flexibility [Trail Making Test (TMT); (51)]. Logical deductive reasoning [Matrix reasoning, WAIS IV; (52)], and capacity of abstraction of verbal concepts [Similarities, WAIS IV; (52)] were, however, completely efficient. We noted a slight sensitivity to interference and weak

Table 1 | Clinical and functional characteristics of patients A and B before the treatment.


inhibition ability [Stroop (53)]. While assessing attentional processes, we observed that information selection was possible but limited depending on the task. The D2 test (54) highlighted decreased speed of processing when the task was costly. Damien was, however, perfectly able to maintain his attention on the task and appropriately divide attentional resources between several sources of information [Test of Attentional Performance (TAP); (55)]. Finally, Damien had very good resources in both shortterm and working memory [Digit span, WAIS IV; (52)].

Neurocognitive functioning of Robin was also globally preserved. His working memory was relatively efficient in both visuospatial [Corsi block-tapping task, MEM III (56)] and verbal modalities [Digit span, Arithmetic and Letter-Number sequencing subtests, WAIS IV; (52)]. Selective and sustained attention was also efficient but speed of processing appeared to be impaired [D2 test (54)]. Robin's performances in executive functioning were poor, especially mental flexibility [Trail Making Test, TMT; (51)]. Difficulties also showed in the Stroop test (53) but seemed associated with decreased speed of processing.

The evaluation of social cognitive functioning focused on five components: (a) facial emotional processing, (b) affective and cognitive, first- and second-order ToM, including the potential influence of verbalization, (c) attributional style, (d and e) social perception and social knowledge, and (f) empathic abilities (the social cognitive functioning assessments of patients A and B are summarized in **Table 2**).

Damien's assessment highlighted poor capacities in identifying and differentiating facial emotions, especially disgust and contempt [TREF (57)]. Damien needed a high intensity of emotion to correctly detect facial expressions. Several ToM measures were performed. MASC-VF [Movie for the Assessment of Social Cognition (58)] showed major and significant deficits. Damien seemed to present with difficulties inferring others' mental states, i.e., lack of mentalization. Other measures emphasized dissociation between cognitive and affective ToM: the ToM–15 (59), which assesses false-belief understanding, clearly highlighted impaired cognitive ToM, while the Reading the Mind in the Eyes Test (RMET) (60) showed relatively well-preserved affective ToM. Damien's answers to the Ambiguous Intentions Hostility Questionnaire (AIHQ) (61) on attributional style were relatively appropriate. We did not observe hostility, aggression, or attribution of responsibility bias. Social perception and social knowledge abilities were assessed with a new test (PerSo, GDR 3557) that is currently under validation. Damien's scores highlighted difficulties using contextual information and selecting relevant cues to understand the social situations depicted in the test. Finally, empathic abilities were assessed using the Empathic Quotient (EQ) (62) and were globally preserved.

Robin's facial emotion recognition assessment with the TREF (57) highlighted poor capacities in identifying and differentiating facial emotions. These capacities were below average for all emotions except for joy and fear, which were correctly detected even with low emotional intensity. Other emotions were confused even when the level of intensity was the highest. The MASC-VF test (58) for ToM assessment showed important difficulties in understanding others' intentions, thoughts, and emotions. Robin's profile seemed to indicate a greater lack of mentalization than tendency to over-interpretation. Moreover, dissociation between cognitive and affective ToM was observed. The ToM-15 (59) clearly revealed impairments, while the RMET (60) showed relatively well-preserved capacities. Robin's responses to the AIHQ attributional style questionnaire (61) significantly highlighted a hostility bias in accidental and ambiguous situations. In accordance with his complaints in daily life, Robin had a tendency to give hostile explanations about the social situations depicted in the test. Moreover, an aggressive bias was also observed since Robin tended to respond aggressively to intentional and ambiguous situations. Social perception and knowledge abilities (PerSo, GDR 3557) showed that Robin checked images in a fragmented manner, which resulted in difficulties interpreting social situations. Finally, both cognitive and affective empathic abilities assessed using the Questionnaire of Cognitive and Affective Empathy (QCAE) (63) were preserved.

#### Method, Measures, and Statistical Approaches of the Single-Case Studies

Single-case experimental designs represent the intensive and prospective study of an individual, using an *a priori* methodology, which includes systematic observation, manipulation of variables, repeated measurement, and data analysis (64). In contrast to an experimental group design, in which one group is compared with another, participants in a single-subject experiment research provide their own control data for the purpose of comparison in a within-subject, rather than a between-subject, design (65). Single-case designs involve a comparison between experimental time periods, known as *phases.* In our studies, an A–B–A methodological protocol, with (A) a phase with three baseline sessions separated by a 1-week interval to assess the stability of measures; (B) the cognitive remediation therapy phase; and (A) another phase with three baseline sessions at a 1 week interval to assess both the stability of measures and the evolution of performances was used. Damien underwent another phase with three baseline sessions 9 months after the end of the RC2S treatment to analyze the maintenance of benefits, but unfortunately we were not able to do the same with Robin. In this kind of design, the goal is to determine whether a causal or functional relationship exists between a researcher-manipulated independent variable (the introduction of the RC2S treatment) and a meaningful change in the dependent variable (social cognitive measures). The dependent variables are, thus, measured repeatedly across and within the conditions of the study (before and after the treatment). Moreover, in order to control intern validity and to check the absence of confounding variables, additional control measures should be proposed through a multiple baseline design.

In these studies, we selected three kinds of baselines for each patient. We first defined SBLs assessing impaired social cognitive processes, which are the targets of social cognitive remediation therapy, and for which a significant improvement was expected at the end of the therapy so we could prove its effectiveness.

In Damien's case, two SBLs focusing on ToM were determined: (1) a verbal baseline (SBL1) for which a score out of 12 was calculated in each session, and consisting of short stories derived from Achim et al. (66); the patient answered questions about the mental states of the characters depicted in the stories

#### Table 2 | Social cognitive functioning assessments of patients A and B before the treatment.


and (2) a visual baseline (SBL2) consisting of photos extracted from rehabilitation material labeled "Colorcards – what are they thinking" and depicting people in various situations, with balloons representing the protagonists' thoughts. For each picture, two independent assessors rated a score out of three to obtain a total score out of 15 for each baseline session.

Since Robin's main social cognitive difficulties lay in attributional style and ToM, two SBLs focusing on these processes were determined. SBL1 aimed to measure causal attributions about positive, negative, or ambiguous events using six sentences for each baseline: two sentences depicting a negative social situation, e.g., "a friend thinks you are stupid"; two sentences depicting a positive social situation, e.g., "a friend tells you that he/she respects you"; and two sentences depicting an ambiguous social situation, e.g., "a friend did not come to the regular appointment." The sentences were derived from both the Internal, Personal, and Situational Attributions Questionnaire [(IPSAQ) (67); French version by (68)] and the MetaCognitive Training program (MCT) (69). For each sentence, Robin was required to write causal explanations for the situation described. The investigator classified Robin's propositions into three categories according to the type of attribution: internal (something to do with Robin), personal (something to do with another person or persons), and situational (something to do with the circumstances of chance) attributions. For each sentence, a score from 1 – when Robin suggested causes belonging to only one type of attributions – to 3 – three types of attributions – was calculated. For each baseline session, Robin's score could, thus, vary from 6 to 18. The second SBL (SBL2) measured ToM by gathering the specific verbal and visual baselines developed for Damien. For each baseline session a total score out of 27 was calculated.

Second, we defined intermediate baselines (IBLs) corresponding to untrained aspects of social functioning that might be impacted by cognitive remediation therapy due to their connections to the targeted processes. The improvement of these measures would evidence the generalization of benefits.

For Damien and Robin, the IBLs targeted two components: (1) facial emotion recognition (IBL1) and (2) social intelligence (IBL2) as defined in Guilford's (70) work, i.e., the ability to understand others' behaviors. The emotion recognition baseline consisted of 12 photos extracted from Ekman's set and eight closeup videos from Baron-Cohen's "Mind Reading." The patient had to determine the basic or complex emotions depicted. A score out of 20 was calculated for each baseline session. The "social intelligence" baseline included parts of the Four Factor Test of Social Intelligence developed by O'Sullivan and Guilford (71): expression grouping, missing cartoons, social transactions, and cartoon predictions. A score out of 18 was measured for each baseline session.

Finally, we defined an unspecific baseline (UsBL) measuring a process untargeted by cognitive remediation therapy, for which no evolution was expected by the end of the treatment, so we could evidence treatment specificity. Both patients had to learn 15 abstract words in five trials. Their score was calculated during each baseline session with the sum of the words they were able to recall.

To analyze results, data from all baseline pre-therapy and posttherapy sessions (and 9-month follow-up sessions for Damien) were collected. In the case of Damien, we obtained a 3 × *k* statistical plan, 3 being the three (pre, post, and follow-up) measurement phases and *k* being the baselines with five modalities: SBL/ ToM–verbal modality (SBL1), SBL/ToM–visual modality (SBL2), IBL/facial emotion recognition (IBL1), IBL/social intelligence (IBL2), and UsBL. For Robin, we obtained a 2 × *k* statistical plan, 2 being the two (pre- and post-) measurement phases and *k* being the baselines with five modalities: SBL/attributional style (SBL1), SBL/ToM (SBL2), IBL/facial emotion recognition (IBL1), IBL/ social intelligence (IBL2), and UsBL.

The analyses were conducted with the *Q*′ (72), which is the equivalent of a non-parametric analysis of variance (ANOVA) based on proportions. The *Q*′ test allows to test the hypothesis of equal proportions and proportion differences in 2 × *k* plan (comparison of two phases in *k* tests or conditions). It derived from Marascuilo's test (1970) by introducing the Wilson's variance. In the 2 × *k* design, the magnitude of difference between two proportions is compared in *k* conditions (where a condition may represent a group, a single observer, or even a single test from individual data). The resulting test statistic has a χ<sup>2</sup> distribution with ν = (*k* − 1) degrees of freedom. The *Q*′ test permits to investigate the main effects and interaction, and a procedure of multiple comparisons can be used to locate statistically significant sources of variance and differences. Since the *Q*′ test allows for the comparison of two phases in *k* tests or conditions (2 × *k* plan), but not a 3 × *k* plan, in the case of Damien we first compared preand post-data to assess the impact of the therapy, and post- and follow-up data to assess the maintenance of benefits.

#### Treatment

#### *Patient A*

The first two sessions allowed Damien to determine the functional outcomes of his social cognitive impairments and objectives for the RC2S therapy. He wanted to enrich his social daily life, be more at ease with others and reconnect with his old friends.

Each cognitive remediation session was divided into three phases: a brief review of the home-based task, paper-and-pencil tasks to develop strategies in specific social situations, and the simulation phase to help transfer strategies into an environment similar to daily life, plus the determination of a new home-based task. At the beginning of the therapy, we worked on strategies deconstructing social situations into microstructures – who, where, when, what – for Damien to take into account their contexts. We then developed a visual scanning of faces strategy to improve his facial emotion recognition abilities. Finally, we worked on ToM reusing the previous strategies. We asked Damien to formulate plausible hypotheses about others' (emotional and unemotional) mental states based on contextual information, facial, postural, and vocal cues. The same strategies were used during the simulation phase to guide Tom's behavior and understand the reactions of his interlocutors. Home-based tasks were linked to Damien's objectives and were intended to favor social interactions, first in family environment and then on the outside to develop friendly relationships.

The last two therapeutic sessions aimed to help transfer strategies into real life. We asked Damien to determine the emotions and mental states of people on the street relying on the previous strategies.

#### *Patient B*

In accordance with Robin's complaints, the functional outcomes of social cognitive impairments assessed with the ERF–CS scale (Social Cognition-Functional Outcomes Scale) mainly concerned attributional style. According to both this scale and the social cognitive evaluation, Robin determined two concrete objectives for the therapy. First, he wanted to be less sensitive to negative information during social interactions. To achieve this goal, we planned to focus on attribution bias by developing strategies that helped Robin take into account different interpretations from those he spontaneously made up in his mind. His second objective was to be able to establish contact with others more easily.

Each cognitive remediation session was divided into different phases: a brief review of the home-based task, paper-and-pencil tasks to develop strategies about ambiguous social situations, the simulation phase designed to help transfer strategies into an environment similar to daily life, and finally the determination of a new home-based task. At the beginning of the therapy, we worked on strategies to keep hostile experience at a distance. To achieve this goal, we worked on the links between facial, prosodic and gestural expressions, and internal feelings. Then we helped Robin acquire functional skills to start and maintain a conversation. At the same time, we worked on attribution bias and developed three strategies using both images or video clips and simulation situations from the RC2S program. The first strategy involved thinking about other interpretations of each social interaction using three stereotypical attribution styles: internal, personal, and situational. The second strategy consisted in separating facts from interpretation, and the third was making suggestions and selecting the most probable assumption concerning others' behaviors. These strategies were also used in home-based tasks and during the last two therapeutic sessions to improve the transfer of strategies to real life.

#### RESULTS

#### Patient A

At the end of the treatment phase, the interactions between factors were significant [*Q*′(4) = 19.10; *p* = 0.0008]. We observed significant main effects between pre- and post-therapy measures [*Q*′(1) = 19.12; *p* < 0.0001], and between the various baselines [*Q*′(4) = 44.54; *p* < 0.0001]. The results obtained after 9 months showed significant interaction between pre- and follow-up measures [*Q*′(4) = 28.29; *p* < 0.0001], and significant main effects of pre- vs. follow-up measurement phases [*Q*′(1) = 12.19; *p*= 0.0005] and between the various baselines [*Q*′(4) = 48.52; *p*< 0.0001]. No interaction was highlighted between post-therapy and follow-up measures [*Q*′(4) = 2.25; *p* = 0.69]. Multiple comparisons were carried out on each baseline to analyze these results.

We verified that the three measures performed for each baseline at a weekly interval prior to the therapy (pre-therapy), after the treatment (post-therapy), and 9 months later (follow-up) were stable. No significant result was observed. This excluded the possibility that the results might be linked to natural behavioral evolution.

The comparison between pre- and post-therapy scores for the SBL1 showed significant improvement (*Q*′= 4.79; *p*< 0.001). This enhancement appeared stable in the long term, since no significant difference was observed between post-therapy and follow-up measures (*Q*′= 0.93; *p*= 0.93) and the difference with pre-therapy measures remained significant (*Q*′ = 5.78; *p* < 0.001). The results are similar for the SBL2 (**Figure 1**). Significant improvements were observed between pre-therapy and both post-therapy (*Q*′ = 4.01; *p* = 0.003) and follow-up scores (*Q*′ = 3.25; *p* = 0.03). No difference was observed between post- and follow-up measures (*Q*′ = 0.71; *p* = 0.97). These results evidence the effectiveness of the RC2S treatment and its maintenance.

The results from the emotion recognition IBL (IBL1) highlighted significant improvement at the end of the treatment when comparing pre- and post-therapy scores (*Q*′ = 3.16; *p* = 0.04). This enhancement appeared to be more fragile in the long term, since even if the comparison between post-therapy and follow-up measures was not significant (*Q*′ = 0.60; *p* = 0.99), the difference between pre-therapy and follow-up scores also failed to achieve significance (*Q*′= 2.57; *p*= 0.16). The results from the social intelligence IBL (IBL2) were similar (**Figure 2**). Significant enhancement was actually observed between pre- and post-therapy measures (*Q*′ = 3.27; *p* = 0.03), and no difference was highlighted between post-therapy and follow-up measures (*Q*′ = 0.24; *p* = 1). However, the difference between pre-treatment and follow-up measures also failed to achieve significance, even if a trend was observed (*Q*′ = 3.02; *p* = 0.06). These data nevertheless confirm the assumption that benefits tend to generalize to processes close to those targeted by cognitive remediation therapy.

Finally, as we expected it, no evolution of the UsBL score was observed between pre- and post-therapy (*Q*′= 1.43; *p*= 0.78), nor between pre-therapy and follow-up (*Q*′ = 0.38; *p* = 1) or between post-therapy and follow-up (*Q*′ = 1.05; *p* = 0.90) (**Figure 3**). These results evidence the specificity of the RC2S program to target social cognitive processes.

#### Patient B

At the end of the treatment phase, the interactions between factors were significant [*Q*′(4) = 24.69; *p* = 0.0001]. We observed significant main effects between pre- and post-therapy measures [*Q*′(1) = 20.73; *p* < 0.0001] and between the various baselines [*Q*′(4) = 14.87; *p* = 0.005]. Multiple comparisons were carried out on each baseline to analyze these results.

We verified that the three measures performed for each baseline at a weekly interval prior to the therapy (pre-therapy) and after the treatment (post-therapy) were stable. No significant result was observed. This excluded the possibility that the results might be linked to natural behavioral evolution.

The comparison between pre- and post-therapy scores for the attributional style SBL (SBL1) showed significant improvement (*Q*′ = 5.26; *p* < 0.001). The results for the SBL2 are similar (**Figure 4**). Significant improvements were observed between pre- and post-therapy scores (*Q*′ = 3.52; *p* = 0.015). These results evidence the effectiveness of RC2S on the processes targeted by the treatment.

The results from the IBL1 highlighted significant improvement at the end of the treatment when compared with pre-therapy scores (*Q*′ = 3.69; *p* = 0.009). The results for the IBL2 are similar (**Figure 5**). Significant enhancement between pre- and posttherapy measures was actually observed (*Q*′ = 3.65; *p* = 0.01). These data confirm the assumption that the benefits generalize to processes close to those targeted by cognitive remediation therapy.

To conclude, no evolution of the UsBL score was observed between pre- and post-therapy (*Q*′ = 0.58; *p* = 0.99), in accordance with our expectations (**Figure 6**). These results confirm the specificity of the RC2S program.

#### Conclusion

The single-case methodology used to assess the influence of the RC2S treatment stressed the positive impact of the therapy on patients' social cognitive skills. To confirm these observations, the social cognitive assessment performed prior to the treatment was repeated at the end of the program – and 9 months later in Damien's case. The results are summarized in **Table 3**.

Results globally corroborated previous conclusions. Damien's assessment highlighted substantially enhanced scores in all ToM tests. The improvement was stable – and even increased – 9 months after the end of the therapy. We also observed improved facial emotion recognition after the therapy and at follow-up, even if the results did not reach statistical significance. Damien was also able to correctly recognize facial expressions with a lower level of intensity. Scores on the attribution bias test emphasized the evolution of scores throughout the treatment, but results were still close to those expected in controls. We also observed improved social perception and knowledge scores. His describing images depicting social situations was richer and his interpreting interactions between characters was more accurate at the end of the treatment.

Robin's results also corroborated the conclusion of the previous the single-case study. We observed substantially enhanced ToM scores. Attributional style was one of the central target components of the therapy, and we noted a very positive and significant evolution of the AIHQ scores (61). Robin's explanations of social situations were more qualified for both accidental and ambiguous situations. He was then able to submit several hypotheses to explain the potential causes of specific situations with different kinds of attributions – internal, external, and situational. We also noted improved facial emotion recognition measured with

#### Table 3 | Pre-, post-, and follow-up comparisons of social cognitive assessments for patients A and B.


*ns, not significant.*

the TREF (57) after the therapy, although the results were not statistically significant.

Damien subjectively reported to have made progress in social functioning. He has recovered his previous circle of friends and meets them regularly. He is more comfortable during family reunions and his parents reported the same improvement. Damien seems to be more flexible during social interactions and said he was more able to adapt his behavior to the social context. At the end of the treatment, Robin reported the subjective feeling of being more comfortable in social situations. He explained he was less overwhelmed by negative interpretations in social interactions and was able to communicate with others more easily.

To assess the objective evolution of social functioning, the scales used before the therapy were repeated at the end of the treatment, and 9 months later in Damien's case. The results are presented in **Table 4**.

A non-significant improvement of Damien's self-esteem was reported at the end of the treatment, but 9 months later the SERS total score (47) came back to its initial level. Damien's mental well-being, measured with the WEMWBS (48), remained stable throughout the treatment. We observed a general trend to improvement in daily social functioning assessed with the *EAS* (49) at the end of the therapy. The most impacted sub–score was "emotional life and social relationships." Nine months after the


Table 4 | Pre-, post-, and follow-up comparisons of functional assessments for patients A and B.

end of the treatment, Damien's total score decreased significantly compared to his initial score. All sub-scores decreased; especially the "external relationships" sub-score changed significantly. Finally, we noted decreasing PANSS total score of psychotic symptoms throughout the treatment, from 76 at the beginning to 72 at the end and 62 nine months after. This improvement seems to show in Damien's behavior. Robin's scores on the scales selected to assess self-esteem [SERS (47)] and mental well-being [WEMWBS (48)] remained stable, as did symptoms measured with the PANSS (46).

#### DISCUSSION

The main objective of these two single-case studies was to assess the impact of the RC2S program on social cognitive impairments in two patients with schizophrenia. We will now discuss the results in the light of the literature on the cognitive remediation of social cognition.

#### RC2S Program Efficiency

RC2S was developed as a comprehensive social cognitive remediation program. It is an individualized treatment designed to adapt the therapy to the specific social cognition difficulties of each patient while taking into account preserved cognitive processes. The two single-case studies highlighted improvement in both patients' abilities in the targeted processes, i.e., ToM in Damien's case, and ToM and attributional style in Robin's case. These improvements were emphasized through SBLs during the studies and social cognitive assessments after the therapy. Damien showed enhanced ToM abilities. The strategies developed during the treatment, such as analyzing context and behavior to make assumptions about another person's mental state, also allowed him to progress in tasks involving social perception processes. As for Robin, we noted normalized scores on the attribution bias test and significantly improved ToM at the end of the therapy. Our two patients seemed to have developed efficient alternative strategies to compensate for their difficulties in specific social cognition domains. Furthermore, the UsBL, involving verbal memory processes that were untargeted by cognitive remediation therapy, remained stable throughout the treatment. This suggests that the results were not due to a general improvement of cognitive functioning.

Moreover, the generalization of the positive impacts of the RC2S therapy was also highlighted by the results for the IBLs and the social cognitive assessment at the end of the therapy. For each patient, IBLs focused on the recognition of basic and complex emotions on the one hand, and the understanding of others' behaviors through social intelligence on the other. The performances on the two IBLs significantly improved for both patients by the end of the therapy. These results suggest that the RC2S program has a positive impact on social cognitive processes close to those targeted by the therapy, even if this enhancement is still weak in the long run. The social cognitive assessments performed at the end of the therapy also showed improved untargeted processes. Actually, even if such improvements were not statistically significant, the patients' scores at the end of the therapy (and 9 months later in Damien's case) were similar to those observed in the general healthy population. This provides additional arguments supporting RC2S as a comprehensive program that is able to target any component of social cognition, and focus on one process or another depending on the patient's profile.

Concerning the transfer of the impacts of the therapy to daily life, the two patients reported subjective benefits on social functioning. Damien has renewed contact with his friends and has been participating more in group activities such as sports. Robin declared it was easier for him to start and maintain conversations with others. However, these subjective impacts were not confirmed by the scales assessing self-esteem and mental well-being. Only the *EAS* (49) highlighted significantly enhanced daily social functioning in Damien's case 9 months after the end of the therapy. We were unfortunately not able to test Robin with the *EAS*; therefore, we could not conclude on the transfer of RC2S benefits to daily life. In addition, at least another criticism might explain these results. The measures assessing that social functioning may have been too different from the aspects of daily life practiced during the therapy. The analysis of the empirical literature on the impact of social cognitive remediation on functional outcomes indeed emphasizes that most studies assess social functioning with measures based on role-plays or real interactions, such as the Social Skills Performance Assessment (SSPA) (73). Such evaluation tools are clearly more adapted to reflect the patients' achievements in social functioning than the questionnaires used in our single-case studies.

#### RC2S Program in the Field of Social Cognitive Remediation

As discussed in the first part of this paper, the few meta-analyses assessing social cognitive remediation in schizophrenia emphasized promising results (32, 33), even if some conclusions seem to be less clear-cut when the data from social cognitive therapy patients are compared to controls (74). Some components of social cognition, especially basic processes, such as emotion recognition and social perception, clearly appear to benefit from social cognitive remediation. ToM seems to be positively impacted, but the long-term maintenance of benefits and the replication of results have only been occasionally studied. Finally, it seems more difficult to have an impact on higher-order social cognitive functions, such as attributional style, for which very few studies report significant improvements.

The results of the two single-case studies presented in this paper partly support the data from the literature. Indeed, we observed a normalization of the capacity to recognize facial emotions, even though this component was not specifically targeted by the therapy. This result confirms that basic processes of social cognition can be definitely improved by cognitive remediation. We also observed enhanced ToM abilities for both patients. This component of social cognition was a therapeutic target for both participants, who have significantly improved their performances on SBLs and pre- and post-therapy tests. These results support the potential for ToM improvement through social cognitive remediation. Lastly, the single-case study carried out on Robin (patient B), who presented with attributional style impairments, emphasized that social cognitive remediation could have an impact on higher-order social cognitive functions. Indeed, Robin improved his performances both on the attributional style SBL and the attribution bias test. The impact of cognitive remediation on attributional style is scarce in the literature but our results are consistent with those obtained with the Social Cognition and Interaction Training program (SCIT) (75). Researchers showed significant attributional style improvement in several studies evaluating the impact of SCIT on social cognition (75–79). We might actually consider that our results are linked to the strategies used in our study. During Robin's therapy, the strategies developed to enhance attributional style were quite similar to those in the SCIT program, namely the detachment from the patient's initial interpretations of specific social situations by focusing on the facts that provided arguments for alternative interpretations. Moreover, in their review, Fiszdon and Reddy (33) suggested that the limited success of social cognitive remediation on higherorder functions might be due to limited opportunities for the patients to practice skills. In RC2S, we chose to bypass this difficulty by using simulation scenes as a primary transfer step to test and adapt the strategies developed during paper-and-pencil tasks to real life-like situations. The gap between training and real life is, therefore, smaller than in other social cognitive remediation programs. This could be one of the strongest assets of the RC2S program.

#### Relevance of an Individual and Computerized Therapy

As described in the introductive part, the use of computerized programs in the field of social cognition is scarce and most therapies are designed for groups, based on the assumption that improved social skills abilities are essentially favored by therapies involving social situations. However, group therapies present a few disadvantages: since the exercises are standardized for the entire group and each participant's specific profile is not taken into account, they neither include adapted strategies to compensate for or restore cognitive deficits, nor personalized exercises aiming to transfer strategies to daily life (80). Furthermore, computerized programs provide advantages, both to adapt the cognitive remediation therapy to patients' specific impairments and abilities, but also in term of impacts on cerebral functioning. Actually, according to a meta-analytic study assessing the efficacy and specificity of computer-assisted cognitive remediation in schizophrenia prolonged multimedia stimulation favor neural plasticity (81), which is a central concept of cognitive remediation.

Even if computerized programs are not widespread in social cognitive remediation, these techniques, especially those involving simulation or virtual reality, are already used in the field of psychiatry and neuropsychology for treating specific phobias, post-traumatic stress disorder, or attention-deficit disorder in children (82). More recently, virtual-reality systems has been used for social skills training of people with schizophrenia (40–43) and authors found these technologies are particularly good at improving conversational skills and assertiveness, but also negative symptoms, psychopathology, and social functioning. Our results confirmed previous data and demonstrated the interest of a computerized and individualized program to improve specific social cognitive functions. Virtual-reality applications seem to be also advantageous in terms of enhancing motivation for therapy. Based on our experience, computerized tasks seem to be actually very motivating for patients, and particularly for young ones. This result is interesting because recent models of cognition suggest that motivation is a core component of the relationship between social functioning and cognition in schizophrenia (83).

#### Improvement of the RC2S Program and Validation Study

These single-case studies also paved the way for enhancing the program. Actually, we have developed various sets of photos and videos to complete the simulation scenes and potentiate the therapeutic impacts on social cognition and functioning. The combination is called RC2S+.

Two sets of photos were developed with the help of professional actors: one set was designed to work on facial and postural emotions, including basic and more complex emotions, such as boredom, pride, shame, or seduction with several levels of intensity; the other set depicts situations involving either one person whose emotions or mental state can be inferred thanks to contextual information, or several people in daily-life activities. This material allows for the training of ToM and social perception abilities and is also a valuable basis for discussions around attributional style. We have also developed videos that specifically target attributional style by presenting both ambiguous interactions and archetypal social situations. All this additional material can be used during paper-and-pencil tasks and allows the therapist to select specific items according to the patient's profile and interests. Moreover, in order to enhance massed practice, we have decided to split up cognitive remediation sessions into two 1-h sessions per week: a paper-and-pencil session to develop strategies and a computerized session to use strategies in simulation scenes. Finally, we are also evaluating the possibility of a mixed individual/ group therapy with individual paper-and-pencil and small-group computerized sessions to favor both the exchange of strategies and discussions between patients, depending on their points of view on the interactions between Tom and other characters.

To conclude, the RC2S program is a new promising tool for reducing social cognitive impairment in schizophrenia. RC2S was also used in other populations presenting with the same kind of deficits, such as a young man with schizoid personality disorder, a woman with 22q11 deletion syndrome, and a patient with

#### REFERENCES


Prader–Willi syndrome without intellectual disability (personal data). All the results were positive and corroborated the conclusions of the two single-case studies reported in this paper. We are currently preparing a randomized controlled trial to establish the validity of the RC2S program in schizophrenia. The study will be conducted in 2016 by several teams in France and aims to assess the impact of RC2S on attributional style compared with cognitive behavioral therapy focusing on positive symptoms.

#### AUTHOR CONTRIBUTIONS

EP and NF both contributed to the conception of the RC2S program. The design of the studies has been proposed by EP, such as the acquisition and analysis. EP and NF both participated in the interpretation of data for the work. EP drafted the paper, and NF and EP both approved the final version.

#### ACKNOWLEDGMENTS

The authors would like to thank Damien and Robin for their implication in the study, Pr. George Michael for his methodological and statistical support, and Florence Apruzzese for editing the manuscript.

#### FUNDING

This research was funded by the research scientific council of Le Vinatier Hospital (CSR C07), which we thank for its support.


spectrum disorders: preliminary findings. *Schizophr Res* (2007) **91**:112–6. doi:10.1016/j.schres.2006.12.010


**Conflict of Interest Statement:** The research was conducted in the absence of any commercial or financial relationship that could be considered as a potential conflict of interest.

*Copyright © 2016 Peyroux and Franck. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

digital media

of impactful research

article's readership