MINI REVIEW article

Front. Psychiatry, 19 July 2019

Sec. Aging Psychiatry

Volume 10 - 2019 | https://doi.org/10.3389/fpsyt.2019.00505

A Literature Overview of Virtual Reality (VR) in Treatment of Psychiatric Disorders: Recent Advances and Limitations

  • 1. Department of Psychiatry, Depression Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea

  • 2. Department of Health Sciences & Technology, Department of Medical Device Management & Research, and Department of Clinical Research Design & Evaluation, Samsung Advanced Institute for Health Sciences & Technology (SAIHST), Sungkyunkwan University, Seoul, South Korea

  • 3. Department of Electronic Engineering, Hallym University, Kangwon, South Korea

Article metrics

View details

381

Citations

90,3k

Views

25,8k

Downloads

Abstract

In this paper, we conduct a literature survey on various virtual reality (VR) treatments in psychiatry. We collected 36 studies that used VR to provide clinical trials or therapies for patients with psychiatric disorders. In order to gain a better understanding of the management of pain and stress, we first investigate VR applications for patients to alleviate pain and stress during immersive activities in a virtual environment. VR exposure therapies are particularly effective for anxiety, provoking realistic reactions to feared stimuli. On top of that, exposure therapies with simulated images are beneficial for patients with psychiatric disorders such as phobia and posttraumatic stress disorder (PTSD). Moreover, VR environments have shown the possibility of changing depression, cognition, even social functions. We review empirical evidence from VR-based treatments on psychiatric illnesses such as dementia, mild cognitive impairment (MCI), schizophrenia and autism. Through cognitive training and social skill training, rehabilitation through VR therapies helps patients to improve their quality of life. Recent advances in VR technology also demonstrate potential abilities to address cognitive and functional impairments in dementia. In terms of the different types of VR systems, we discuss the feasibility of the technology within different stages of dementia as well as the methodological limitations. Although there is room for improvement, its widespread adoption in psychiatry is yet to occur due to technical drawbacks such as motion sickness and dry eyes, as well as user issues such as preoccupation and addiction. However, it is worth mentioning that VR systems relatively easily deliver virtual environments with well-controlled sensory stimuli. In the future, VR systems may become an innovative clinical tool for patients with specific psychiatric symptoms.

Introduction

Virtual reality (VR) is defined as a computer-generated simulation, such as a set of images and sounds that represents a real place or situation, that can be interacted with, in a seemingly real or physical way by a person using special electronic equipment. It can transmit visual, auditory, and various sensations to users through a headset to make them feel as if they are in a virtual or imagined environment (1).

The concept of VR was introduced in the 1950s, and the maturity of VR for entertainment is now evident. Currently, more than 230 companies are producing various products related to VR and performing research and development, including global companies such as Samsung Electronics, Apple, Facebook, Amazon, and Microsoft. VR systems consist of VR headsets, a computer, and video. Recently, chairs, gloves, and sensors have been added. VR headsets refer to head-mounted goggles. They are equipped with a speaker or headphones. VR systems that include the transmission of vibrations and other sensations to the user through a game controller, gloves or chairs are known as haptic feedback systems (2). This tactility is advantageous as the sense of presence can be improved by actually sensing the shock or vibration to the user in the medical field, video games, and military training. A 4D (four-dimensional) system of VR refers to a VR system with a motion chair that enhances reality for users with integrated movement according to the content of the VR video. Depending on the type of system and programing, the user may interact with the environment from a first- or third-person’s point of view. In the case of the latter, the user can move around a virtual representation of themselves, called an ‘avatar’ (3).

In medical fields, multidisciplinary research has tried to apply VR systems to domains of diagnosis, treatment, and so on. Especially, in psychiatry, traditional tools of treatment have mainly been limited to interpersonal psychotherapy and medication. However, VR can provide various types of stimulation (4). Intuitively, it helps in relieving pain, stress, and anxiety in an imagined space, and VR makes it possible to provide efficient educational and psychological training without causing harm to patients (5). It therefore has the possibility of changing PTSD, phobia, anxiety, depression, cognition, and social functions in patients with psychiatric illnesses.

Indeed, over the past few decades, therapeutic virtual reality (VR) has emerged as a successful solution for a wide range of psychiatric disorders. In the 1990s, Rothbaum et al. conducted the first study in the field of psychiatry to investigate the efficacy of VR focusing one treating acrophobia in college students and found that VR is successful in reducing their fear of heights (6). The early studies established the efficacy of VR exposure therapy for a number of anxiety and related disorders. For example, VR exposure therapies have shown benefits for patients with a specific phobia or posttraumatic stress disorder (PTSD) by the extinction of traumatic experiences through their repetitive exposures, and the extinction of pain by pulling the patients focus away from painful conditions. The broad reach of VR has enabled its use in the evaluation and rehabilitation of patients with schizophrenia and autism through improvements of their social activities. The reports included in this review, show that VR is also an efficacious way for Amnestic MCI (mild cognitive impairment) and early to moderate Alzheimer’s disease through cognitive reserve and training.

VR-based treatment currently faces hurdles preventing its wide use as a real tool in psychiatry practice, such as motion sickness and dry eyes as well as user issues such as preoccupation and addiction. However, VR systems can deliver and confront virtual environments with well-controlled sensory stimuli. With a review of the current utilization of VR in the field of psychiatry, we highlight both the benefits and limitations of VR use, as it is just beginning to be applied as a new modality in psychiatry. We have tried to describe the evidence of the utility of VR in psychiatric conditions and the types of procedures followed in those studies

Methods and Results

To identify ‘virtual reality’ in the field of psychiatry, a PubMed literature search was performed which included articles published prior to 31 December 2018, and which included the terms “virtual reality” and “psychiatry” and “treatment” or “therapy.” We then categorized them as articles related to “posttraumatic Stress Disorders (PTSD),” “phobia,” “anxiety disorders,” “schizophrenia,” “autism,” “dementia and mild cognitive impairment (MCI),” and “pain and stress.” Additional relevant articles were found through a manual bibliographic search. Eligibility criteria were: 1) articles in English; 2) human studies; and 3) articles focused on virtual reality and psychiatric disorders. We collect 36 studies that made use of VR to provide various types of stimulations for patients with psychiatric problems (Table 1).

Table 1

Author and date of publicationSubjectsDesignMethodConclusions
Doniger et al., 2018 (7)Middle-aged adults with Alzheimer’s disease family history (n = 125)RCTVR cognitive-motor training, 45 min, twice/week for 12 weeksIncreased cognitive function
Reger et al., 2019 (8)Active duty soldiers with PTSD (n = 108)ObservationalRandomization to exposure via 10 sessions of prolonged exposure or VRE or 5-week minimal attention waitlistNo group differences in average or peak subjective distress during exposure therapy
Flores et al.,
2018 (9)
Two patients with spinal cord injury with psychiatric symptomsCase report4 VR DBT sessions for Patient 1, 2 VR DBT sessions for Patient 2Reductions in negative emotions for Patient 1, mixed results for Patient 2
Peskin et al.,
2018 (10)
Men and women with World trade center-related PTSD (n = 25)RCT100 mg D-cycloserine versus placebo augmented VRE sessions for 12 weeksTemporal relationship between posttraumatic and depressive symptoms. during VRE
Du Sert et al.,
2018 (11)
Schizophrenia patients with refractory AVH
(n = 19)
RCTA 7-week phase-II, randomized, partial cross-over trialSignificant improvements in severity of auditory visual hallucination, depressive symptoms and quality of life
Pot-Kolder et.al., 2018 (12)Patients with a psychotic disorder and paranoid ideation (n = 116)RCTVR-CBT with treatment as usual, 1 h long, 16 individual session versus treatment as usualSignificant reduction in momentary paranoid ideation and anxiety
Gold et al., 2018 (13)Child and adolescent patients (n = 143)RCTTotally 5 min long VR game with standard of care versus standard of care onlySignificant reduction in acute procedural pain and anxiety
Gomez et al., 2017(14)21-year-old Latino male patient with burn injuryObservationalImmersive VR enhanced DBT mindfulness skills training, 4 sessions for 1 monthIncreased positive emotions and decreased negative emotions
Ryu et al., 2017 (15)Children scheduled for elective surgery
(n = 69)
RCTPreoperative VR tour of the operating theatre, 4 min videoLower scores of the Yale Preoperative Anxiety Scale, Induction Compliance Checklist, and Procedural Behavior Rating Scale
Eijlers et al.
2017 (16)
Children undergoing elective day care surgery
(n = 200)
RCTPreoperative VRE intervention, 15 minDiminished preoperative anxiety, postoperative pain, and the use of postoperative analgesics
Beidel et al.,
2017 (17)
Veterans and active duty soldiers with combat-related PTSD (n = 92)RCTVRET plus group treatment versus VRET with psychoeducation controlDecrease on PTSD scale for both group and decrease in social isolation for VRET plus group treatment
Ferrer-Garcia et al., 2017 (18)Patients with bulimia nervosa and binge eating disorder (n = 64)Case controlTwo second-level treatment condition: VR- Cue Exposure Therapy or additional CBTMore proportion of achievement abstinence from binge eating or purging episodes
Blume et al.,
2017 (19)
Children with ADHD (n = 90)RCT15 training sessions of either NIRS based NFT in VR, NIRS based NFT in 2D or biofeedback training in VR, 60–70 min for each sessionNFT in VR is expected to yield greater effects than training in 2D
Shiban et al.,
2017 (20)
29 patients with aviophobiaRCTVRE treatment either with or without diaphragmatic breathingHigher tendency to effectively overcome the fear of flying in VR with diaphragmatic breathing
Bouchard et al.,
2017 (21)
Patients with social anxiety disorder (N = 59)RCT14 weekly sessions for VR exposure or in vivo exposure or waiting listImprovement in both CBT groups, more effective in VRE
Reger et al.,
2016 (22)
Active-duty soldiers
(n = 162)
RCT
Randomization to 10 sessions of Prolonged exposure, VRE, or a minimal attention waitlistSignificant reductions in PTSD symptoms in Prolonged exposure and VRE groups
Norrholm et al.,
2016 (23)
Participants met criteria for PTSD (n = 50)RCT6 weeks of VRE therapy combined with d-cycloserine, alprazolam, or placeboIn the d-cycloserine group, elevated startle before VR therapy predicted better outcome
Son et al.,
2015 (24)
Alcohol dependent subjects (n = 12)Case control study10 sessions of VRET, consisted of 3 steps: relaxation, presentation of a high risk situation, and aversive situationDecreased metabolism in the basal ganglia after VRET (PET shows)
Jahani Shoorab et.al.,
2015 (25)
Primiparous parturient women having labor
(n = 30)
RCTRandomization to VR with standard care group and only standard care groupDecreased pain during the episiotomy repair
When use of VR with local anesthesia
Rothbaum et al., 2014 (26)Iraq and Afghanistan war veterans with PTSD (n = 156)RCTAn introductory session and five sessions of VRE augmented with d-cycloserine or alprazolam or placeboSignificantly improved PTSD symptoms from pre- to posttreatment across all conditions
Marco et al., 2013 (27)
Participants diagnosed with eating disorders
(n = 34)
Case control study15 CBT group sessions and 8 individual psychotherapy sessions with VRImproved body image and this improvement was maintained at the one-year follow-up
Pallavicini et al., 2013 (28)Undergraduate students (n = 39)Case control studySame experience was offered using test, audio, video, and VRVR was less effective than other procedures in eliciting stressor responses
Diemer et al., 2013 (29)Patients with arachnophobia
(n = 58)
RCTA single dose of quetiapine XR or placebo prior to a VREffect of VR challenge on behavioral avoidance, psychophysiological reaction
Malbos et al., 2013 (30)Agoraphobic participants (n = 18)Case control studyVRET only and VRET with cognitive therapyThe isolated effects of VRET did not seem to be less than the effects of VRET with cognitive therapy
McLay et al. 2012 (31)Active duty service members with PTSD
(n = 20)
ObservationalOpen-label, single-group VRETReduction in PTSD symptoms, improvement in PTSD, depression and anxiety
Culbertson et al., 2012 (32)Healthy treatment-seeking cigarette smokers (n = 11)RCTRandomization to CBT plus either smoking-VR Cue Exposure Therapy or placebo-VR Cue Exposure TherapyHigher quit rate, smoking fewer cigarettes per day
Park et al., 2011 (33)Inpatients with schizophrenia (n = 91)RCTComparison social skills training using VR role playing to social skills training using traditional role playing, over 10 semiweekly sessions for 5 weeksImproved more in conversational skills and assertiveness
McLay et al., 2011 (34)Active duty military personnel with combat-related PTSD (n = 10)RCTRandomization to VR-graded exposure therapy or treatment as usualHigher number of improvements reported, more improvement on the CAPS score
St-Jacques et al., 2010 (35)Agoraphobic participants (n = 31)RCTRandomization to in vivo exposure alone or in virtual reality-based exposureVR did not increase motivation toward psychotherapy
Gerardi et al., 2008 (36)A 29-year-old veteranCase report90-min individual session, once weekly over 4 weeksDecreased rating scale scores (CAPS, PTSD Symptom Scale Self-Report)
Difede et al., 2007 (37)Male disaster workers with PTSD (n = 21)Case control studyAssignment to a VR treatment or a waitlist controlSignificant decline in CAPS scores
Walshe et al., 2003 (38)Subjects with simple phobia/accident phobia (n = 14)ObservationalAn open study, computer games and virtual reality therapy, 12 1-h sessionsSignificant post treatment reductions on all measures
Rothbaum et al., 2001 (39)
Male Vietnam combat veterans with PTSD (n = 10)ObservationalOpen clinical trial, 8 to 16 sessions, 2 virtual environmentsSignificant reduction from baseline in symptoms
Rothbaum et al., 2000 (40)Patients with fear of flying (n = 49)Case control studyRandomization to VRE therapy, standard exposure therapy, or a wait-list control, 4 sessions of exposure out of 8 sessionsVRE and standard exposure both superior to wait-list
Rothbaum et al., 1999 (41)A Vietnam combat veteran with PTSDObservationalVRE, 2 virtual environmentsDecrease on CAPS and self-rated PTSD
Rothbaum et al., 1996 (42)A 42-year-old female with a debilitating fear and avoidance of flyingCase reportAnxiety management techniques and the VREAll self-report measures of fear and avoidance of flying decreased

Clinical trials or therapies with virtual reality (VR) in psychiatry.

RCT, randomized controlled trial; VR, virtual reality; VRE, virtual reality exposure; VRET, virtual reality exposure therapy; DBT, dialectical behavior therapy; PTSD, posttraumatic stress disorder; CBT, cognitive-behavioural therapy; NIRS, near-infrared spectroscopy; CAPS, clinician administered PTSD scale; NFT, neurofeedback training.

Discussion

Treatment of Posttraumatic Stress Disorders (PTSD)

Posttraumatic Stress Disorder (PTSD) is a psychological reaction that occurs after experiencing stress that has caused life-threatening extreme mental trauma (42). An individual’s quality of life is greatly reduced by re-experiencing the situation with awakening, anxiety, agitation, and insomnia symptoms. Among PTSD, many VR studies have been focused on veterans who have been exposed to battles in Iraq and Afghanistan, to alleviate their trauma (43), reduce suicidal ideation (44), decrease depression and anger (16), and to improve their PTSD (30). Discharged soldiers can have destructive behaviors to both themselves and others as a result of rage and depression caused by PTSD. However, they can learn to solve these situations in a safe and well-controlled environment called VR. Since the key in the emotion-processing theory (EPT) is to expose and modify their unique fear structure, the virtual environment is ideal in the sense of its flexibility and customization (45). As they are exposed to sources of their disorder, they decrease the feelings of fear and anxiety in the form of VR-based habituation therapy. Dr. Rothbaum at the Emory University Hospital provided a randomized, double-blind, six-session VR exposure treatment in 156 patients diagnosed with PTSD among discharged soldiers returning from the Iraq and Afghan wars (26). The study concluded that VR treatment was associated with the reduction in PTSD diagnoses and symptoms in Iraq and Afghanistan veterans. Another study suggested that VR with skin conductance reactivity is a diagnostic tool for PTSD as well as a treatment (47). Veterans with PTSD displayed larger skin conductance reactivity across VR combat events, but not for non-combat VR events. The VR exposure therapy system, “Bravemind,” is currently distributed to over 50 sites, including VA hospitals, military bases, and university centers to provide relief from PTSD for soldiers (48).

Anxiety Disorders and Specific Phobia

These days, some VR systems create highly immersive experiences using more invasive devices such as Head mounted Display (HMDs). A new generation of realistic simulation can therefore serve as a promising assessment and therapy for erroneous anxiety-provoking thinking. In general, these symptoms are viewed as serious conditions where patients worry about something fearful excessively and persistently. Reproducing the traditional exposure interventions in VR, Maples-Keller et al. reviewed several case studies of social anxiety disorders and generalized anxiety disorders (49). This is merely the beginning of an explosion in potential provided by ever increasing sophisticated technology. It worth noting that the effectiveness of applying VR in this domain is also quantitatively being analyzed (50).

More specifically, phobia is a type of anxiety disorder characterized by marked and persistent fears that are cued by the presence or anticipation of specific objects or situations with a desire to avoid that condition due to high levels of fear and discomfort (43). Phobia includes acrophobia, flight phobia, phobias for insects or animals, and so on. Exposure therapy in VR is helpful because we can deal with such specific phobias in a virtual world, and it can be cost-effectively performed (29). In VR, patients with phobia can reproduce the situation they actually feel fearful of and face it themselves. Repeated use of VR increases the threshold of anxiety and makes it less insensitive, resulting in the reduced incidence of actual situations. Initially, VR graded exposure therapy was found to be successful in reducing fear of spiders (51, 52), social phobia (53), and flight phobia (3, 54) after applying it to a small number of subjects. A self-training program with mobile VR individuals with acrophobia has been safely and successfully applied to reduce fear of heights (55). It can be safely applied at home and at the hospital. It can be easily interrupted or repeated depending on the situation. VR can reduce the degree of anxiety by exposing the patient to a virtual dental care scenario in an incremental manner (56, 57). Similarly, a pilot study has applied a VR headset as a fear reduction tool and pain distraction for fear of needles, where 94.1% of pediatric subjects reported an improvement after using VR during immunization (58). Recently, VR with repetitive transcranial magnetic stimulation (rTMS) over the prefrontal cortex has been applied in participants with spider phobias. It diminished activation in the left inferior frontal gyrus in functional near-infrared spectroscopy (fNIRS) during an emotional Stroop paradigm (59).

Schizophrenia

Patients with schizophrenia show anhedonia, social withdrawal, and a blunted affect, which can lead to rumination and isolation. While exposure therapy in anxiety-related disorders uses VR as a simulation tool, the so-called avatar therapy for negative symptoms of schizophrenia focuses on interactive VR. In a computer-generated virtual world, VR users are no longer simply external observers, but active participants. It is one of the key variables in understanding social environments that need to be controlled, and thus provides exciting applications to research and treatment (60). For example, interactive VR therapy has shown benefits in social skills such as role-playing (33), memory function (61), medication management skills (62), job interviews (63), and vocational training (64). Recent VR-based cognitive rehabilitation programs also manage positive symptoms in schizophrenia such as auditory verbal hallucinations (11).

Autism

Autism is characterized by a state of being trapped in one’s own world. It is a childhood developmental disability. Children with autism do not interact with others. They do not have emotional ties. VR approaches for rehabilitation in autism tend to create virtual environments integrated with other equipment, facilitating cognitive processes of training such as concentration and other functional skills in everyday life. The University of Texas has developed a training program to assist in the social skills training of autistic children (65). It uses brain imaging and Electroencephalography (EEG) monitoring. It also uses avatars to put children in situations such as job interviews and meetings. They practice reading social signals and expressing socially appropriate behaviors. After completion of the program, the activity of the brain area associated with social understanding was found to be increased in participants’ brain image. Smith and colleagues at the Northwestern University Psychiatry Department have reported that young adults diagnosed with autism spectrum have a higher job search rate than the comparative group at six months after receiving job interviews through VR (66). For the purpose of training outdoor activity, individuals with Autism spectrum disorder were placed in a three-dimensional city and given a set of tasks that involved taking buses through a game. A statistically significant increase in measures of knowledge of the process of riding a bus, a reduction in the electro-dermal activity, and a high success rate in their application were found (67).

Dementia and Mild Cognitive Impairment (MCI)

Lessons from the Nun study have revealed that cognitive reserve and training are also important in preventing Alzheimer’s disease (68). Dementia is a broad term describing such disorders of the brain that progress over time. Basically, in evaluating cognitive dysfunction and detecting MCI, VR has been applied and has exhibited very high accuracy. Cushman et al. (69) have investigated navigational impairment of early Alzheimer’s disease, using both real-world and laptop PC based virtual environments in the same subjects; 35 young normal controls, 26 older normal controls, 12 patients with mild cognitive impairment and 14 patients with early Alzheimer's disease (AD). It was found that virtual environment testing provides a valid assessment of navigational skills for aging and Alzheimer’s disease (69). Also, there is a systematic review that presented a status of VR applications for diagnostic assessment and cognitive training in Alzheimer’s disease and MCI. Both semi-immersive and fully-immersive VR technology can be feasible amongst individuals living within the earlier stages of dementia outside of a hospital environment (70). While much of the VR studies appear to focus on the treatment of anxiety or phobias, the population of VR applications is underdeveloped. Even though sample sizes are limited, VR-based cognitive training has shown benefits for episodic memory in Amnestic MCI and early to moderate Alzheimer’s disease (71). Moreover, Moyle el al., explored the feasibility of VR in individuals with a range of cognitive impairments from mild to more severe stages of dementia (72). VR was perceived to have a positive effect on people with dementia, although a greater level of fear and anxiety during VR were experienced compared to those in the normative sample (72). Some individuals in the earlier stages of dementia experienced boredom, and VR technology was also found to increase fear and anxiety in one study. However, it is perhaps not surprising that recent advances in VR rehabilitation applications keep pointing to the feasibility of VR training in healthy elderly persons as well as in pathological populations (73).

Stress and Pain Alleviation

Stress and pain have deleterious effects on the mind and body. In order to decrease one’s attention available for conscious pain processing, VR usage for stress and pain alleviation typically provides simple forms of distraction (e.g., watching videos or playing video games). Although the physical mechanisms are not well understood, the patients focus moves away from the conscious attention on the stressful and painful condition during the occupational activity (74). While patients can learn pain-management techniques as mindfulness, several experimental results suggest that VR techniques has actual benefits for subjective pain reduction (75). For example, Oculus Rift uses DEEP, a meditation application to help users breathe deeply. The application works through a band surrounding the chest to measure breathing. In another pilot study, 44 participants attended a mindfulness conference putting on an Oculus Rift DK2 VR helmet and floated down a calm 3D computer-generated virtual river while listening to digitized DBT mindfulness skills training instructions. Participants reported significantly less sadness, anger, and anxiety but more relaxation (76). Dr. Spiegel’s team at the Cedars-Sinai Hospital has given chronic patients the opportunity to get out of the hospital through VR and to enjoy the natural scenery. This could reduce a patient’s stress and shorten hospital stays (77). Relaxation and meditation in various VR applications have become increasingly widespread for treating patients at home or in hospitals (78).

Studies at the University of Barcelona have shown that applying VR to depressed patients can reduce the severity of their depression and self-degradation and increase satisfaction (79). By limiting distractions from the real world and increasing the sense of presence, VR may facilitate mindfulness practice as well.

Limitations

Clearly, exposure to VR applications may result in significant discomfort for the majority of people, with symptoms of motion sickness including eye fatigue, headaches, nausea, and sweating (80, 81). VR Sickness is different from common motion sickness because motion sickness is caused by visual perception of self-motion while VR sickness does not require actual movement. A conflict between accommodation and vergence depth cues on stereoscopic displays is a significant cause of visual discomfort from VR (82). Dry eyes due to an overheated display in an enclosed space and retinal damage due to blue light are also concerns. As shown in this review, only a few large-sized and well-designed studies have been conducted in psychiatry with VR.

VR is developing to improve real-life adaptation of patients with psychiatric problems. However, patients may become preoccupied or addicted to the VR environment, similar to internet game addiction. If patients with schizophrenia have impairment on reality testing, they may have delusional thinking in the VR environment. Doctor-patient relationships and careful education before using VR are mandatory before applying VR treatments in psychiatric patients. In the near future, a guideline to apply VR treatments to patients with psychiatric illnesses should be established. VR will play a role as an alternative option for psychiatrists to use in supporting psychiatric assessments and treatments in patients.

Conclusion

Many studies and clinical trials have used VR as a simulation, interaction, and distraction tool for patients with psychiatric illnesses such as PTSD, anxiety, specific phobia, schizophrenia, autism, dementia, and heavy stress. VR environments show the possibility of changing their anxiety, depression, cognition, and social functions by effectively exposing them sources of fear, presenting interactive virtual environments of cognitive-behavioral approaches, and contributing to other rehabilitation applications.

In practice, patients with a psychiatric diagnosis such as depression, bipolar disorder, anxiety disorder, schizophrenia, and even alcohol use disorder share common characteristics such as anxiety, avoidance, and poor insight to their illnesses. Modern VR systems can deliver an ideal place where one can confront the problem which needs to be overcome, not only through talking with doctors, but also through virtual environments with well-controlled sensory stimuli. This may produce cognitive and behavioral changes in patients with psychiatric disorders including autism and dementia. They also have benefits in reducing chronic pain and intensive stress. However, VR needs to overcome technical hurdles such as motion sickness and dry eyes, as well as user hurdles such as preoccupation and addiction.

Funding

This work was supported by the Original Technology Research Program for Brain Science through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (No. NRF-2016M3C7A1947307; PI HJJ), and by the Bio & Medical Technology Development Program of the NRF funded by the Korean government, MSIT (No. NRF-2017M3A9F1027323; PI HJJ). They had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Statements

Author contributions

All authors listed have made substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  • 1

    LiLYuFShiDShiJTianZYangJet al. Application of virtual reality technology in clinical medicine. Am J Transl Res (2017) 9:3867–80. doi: 10.4247/AM.2017.ABH187

  • 2

    Bekrater-BodmannRFoellJDiersMKampingSRanceMKirschPet al. The importance of synchrony and temporal order of visual and tactile input for illusory limb ownership experiences - an FMRI study applying virtual reality. PLoS One (2014) 9:e87013. doi: 10.1371/journal.pone.0087013

  • 3

    BausOBouchardS. Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front Hum Neurosci (2014) 8:112. doi: 10.3389/fnhum.2014.00112

  • 4

    MishkindMCNorrAMKatzACRegerGM. Review of virtual reality treatment in psychiatry: evidence versus current diffusion and use. Curr Psychiatry Rep (2017) 19:80. doi: 10.1007/s11920-017-0836-0

  • 5

    MantovaniFCastelnuovoGGaggioliARivaG. Virtual reality training for health-care professionals. Cyberpsychol Behav (2003) 6:389–95. doi: 10.1089/109493103322278772

  • 6

    RothbaumBOHodgesLFKooperROpdykeDWillifordJSNorthM. Effectiveness of computer-generated (virtual reality) graded exposure in the treatment of acrophobia. Am J Psychiatry (1995) 152:626–8. doi: 10.1176/ajp.152.4.626

  • 7

    DonigerGMBeeriMSBahar-FuchsAGottliebATkachovAKenanHet al. Virtual reality-based cognitive-motor training for middle-aged adults at high Alzheimer’s disease risk: a randomized controlled trial. Alzheimers Dement (N Y) (2018) 4:118–29. doi: 10.1016/j.trci.2018.02.005

  • 8

    RegerGMSmolenskiDEdwards-StewartASkoppNARizzoAS, Norr A Does Virtual Reality Increase Simulator Sickness During Exposure Therapy for Post-Traumatic Stress Disorder? Telemed J E Health (2019) in press

  • 9

    FloresALinehanMMToddSRHoffmanHG. The use of virtual reality to facilitate mindfulness skills training in dialectical behavioral therapy for spinal cord injury: a case study. Front Psychol (2018) 9:531. doi: 10.3389/fpsyg.2018.00531

  • 10

    PeskinMWykaKCukorJOldenMAltemusMLeeFSet al. The relationship between posttraumatic and depressive symptoms during virtual reality exposure therapy with a cognitive enhancer. J Anxiety Disord (2019) 61:8288. doi: 10.1016/j.janxdis.2018.03.001

  • 11

    Du SertOPPotvinSLippODellazizzoLLaurelliMBretonRet al. Virtual reality therapy for refractory auditory verbal hallucinations in schizophrenia: a pilot clinical trial. Schizophr Res. (2018) 197:176181. doi: 10.1016/j.schres.2018.02.031

  • 12

    Pot-KolderRGeraetsCNWVelingWVan BeilenMStaringABPGijsmanHJet al. Virtual-reality-based cognitive behavioural therapy versus waiting list control for paranoid ideation and social avoidance in patients with psychotic disorders: a single-blind randomised controlled trial. Lancet Psychiatry (2018) 5:217–26. doi: 10.1016/S2215-0366(18)30053-1

  • 13

    GoldJIMahrerNE. Is virtual reality ready for prime time in the medical space? A randomized control trial of pediatric virtual reality for acute procedural pain management. J Pediatr Psychol (2018) 43:266–75. doi: 10.1093/jpepsy/jsx129

  • 14

    GomezJHoffmanHGBistrickySLGonzalezMRosenbergLSampaioMet al. The use of virtual reality facilitates dialectical behavior therapy(R) “Observing Sounds and Visuals” mindfulness skills training exercises for a latino patient with severe burns: a case study. Front Psychol (2017) 8:1611. doi: 10.3389/fpsyg.2017.01611

  • 15

    RyuJHParkSJParkJWKimJWYooHJKimTWet al. Randomized clinical trial of immersive virtual reality tour of the operating theatre in children before anaesthesia. Br J Surg (2017) 104:1628–33. doi: 10.1002/bjs.10684

  • 16

    EijlersRLegersteeJSDierckxBStaalsLMBerghmansJVan Der SchroeffMPet al. Development of a virtual reality exposure tool as psychological preparation for elective pediatric day care surgery: methodological approach for a randomized controlled trial. JMIR Res Protoc (2017) 6:e174. doi: 10.2196/resprot.7617

  • 17

    BeidelDCFruehBCNeerSMBowersCATrachikBUhdeTWet al. Trauma management therapy with virtual-reality augmented exposure therapy for combat-related PTSD: a randomized controlled trial. J Anxiety Disord (2019) 61:64-74. doi: 10.1016/j.janxdis.2017.08.005

  • 18

    Ferrer-GarciaMGutierrez-MaldonadoJPla-SanjuaneloJVilalta-AbellaFRivaGClericiMet al. A Randomised controlled comparison of second-level treatment approaches for treatment-resistant adults with bulimia nervosa and binge eating disorder: assessing the benefits of virtual reality cue exposure therapy. Eur Eat Disord Rev (2017) 25:479–90. doi: 10.1002/erv.2538

  • 19

    BlumeFHudakJDreslerTEhlisACKuhnhausenJRennerTJet al. NIRS-based neurofeedback training in a virtual reality classroom for children with attention-deficit/hyperactivity disorder: study protocol for a randomized controlled trial. Trials (2017) 18:41. doi: 10.1186/s13063-016-1769-3

  • 20

    ShibanYDiemerJMullerJBrutting-SchickJPauliPMuhlbergerA. Diaphragmatic breathing during virtual reality exposure therapy for aviophobia: functional coping strategy or avoidance behavior? A pilot study. BMC Psychiatry (2017) 17:29. doi: 10.1186/s12888-016-1181-2

  • 21

    BouchardSDumoulinSRobillardGGuitardTKlingerEForgetHet al. Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: a three-arm randomised controlled trial. Br J Psychiatry (2017) 210:276–83. doi: 10.1192/bjp.bp.116.184234

  • 22

    RegerGMKoenen-WoodsPZetochaKSmolenskiDJHollowayKMRothbaumBOet al. Randomized controlled trial of prolonged exposure using imaginal exposure vs. virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). J Consult Clin Psychol (2016) 84:946–59. doi: 10.1037/ccp0000134

  • 23

    NorrholmSDJovanovicTGerardiMBreazealeKGPriceMDavisMet al. Baseline psychophysiological and cortisol reactivity as a predictor of PTSD treatment outcome in virtual reality exposure therapy. Behav Res Ther (2016) 82:2837. doi: 10.1016/j.brat.2016.05.002

  • 24

    SonJHLeeSHSeokJWKeeBSLeeHWKimHJet al. Virtual reality therapy for the treatment of alcohol dependence: a preliminary investigation with positron emission tomography/computerized tomography. J Stud Alcohol Drugs (2015) 76:620–7. doi: 10.15288/jsad.2015.76.620

  • 25

    JahanishoorabNEbrahimzadeh ZagamiSNahviAMazluomSRGolmakaniNTalebiMet al. The effect of virtual reality on pain in primiparity women during episiotomy repair: a randomize clinical trial. Iran J Med Sci (2015) 40:219–24. https://www.ncbi.nlm.nih.gov/pubmed/25999621

  • 26

    RothbaumBOPriceMJovanovicTNorrholmSDGerardiMDunlopBet al. A randomized, double-blind evaluation of D-cycloserine or alprazolam combined with virtual reality exposure therapy for posttraumatic stress disorder in Iraq and Afghanistan War veterans. Am J Psychiatry (2014) 171:640–8. doi: 10.1176/appi.ajp.2014.13121625

  • 27

    MarcoJHPerpinaCBotellaC. Effectiveness of cognitive behavioral therapy supported by virtual reality in the treatment of body image in eating disorders: one year follow-up. Psychiatry Res (2013) 209:619–25. doi: 10.1016/j.psychres.2013.02.023

  • 28

    PallaviciniFCipressoPRaspelliSGrassiASerinoSVignaCet al. Is virtual reality always an effective stressors for exposure treatments? Some insights from a controlled trial. BMC Psychiatry (2013) 13:52. doi: 10.1186/1471-244X-13-52

  • 29

    DiemerJDomschkeKMuhlbergerAWinterBZavorotnyyMNotzonSet al. Acute anxiolytic effects of quetiapine during virtual reality exposure–a double-blind placebo-controlled trial in patients with specific phobia. Eur Neuropsychopharmacol (2013) 23:1551–60. doi: 10.1016/j.euroneuro.2013.01.001

  • 30

    MalbosERapeeRMKavakliM. A controlled study of agoraphobia and the independent effect of virtual reality exposure therapy. Aust N Z J Psychiatry (2013) 47:160–8. doi: 10.1177/0004867412453626

  • 31

    MclayRNGraapKSpiraJPerlmanKJohnstonSRothbaumBOet al. Development and testing of virtual reality exposure therapy for post-traumatic stress disorder in active duty service members who served in Iraq and Afghanistan. Mil Med (2012) 177:635–42. doi: 10.7205/MILMED-D-11-00221

  • 32

    CulbertsonCSShulenbergerSDe La GarzaRNewtonTFBrodyAL. Virtual reality cue exposure therapy for the treatment of tobacco dependence. J Cyber Ther Rehabil (2012) 5:5764. https://www.ncbi.nlm.nih.gov/pubmed/25342999

  • 33

    ParkKMKuJChoiSHJangHJParkJYKimSIet al. A virtual reality application in role-plays of social skills training for schizophrenia: a randomized, controlled trial. Psychiatry Res (2011) 189:166–72. doi: 10.1016/j.psychres.2011.04.003

  • 34

    MclayRNWoodDPWebb-MurphyJASpiraJLWiederholdMDPyneJMet al. A randomized, controlled trial of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychol Behav Soc Netw (2011) 14:223–9. doi: 10.1089/cyber.2011.0003

  • 35

    St-JacquesJBouchardSBelangerC. Is virtual reality effective to motivate and raise interest in phobic children toward therapy? A clinical trial study of in vivo with in virtuo versus in vivo only treatment exposure. J Clin Psychiatry (2010) 71:924–31. doi: 10.4088/JCP.08m04822blu

  • 36

    GerardiMRothbaumBOResslerKHeekinMRizzoA. Virtual reality exposure therapy using a virtual Iraq: case report. J Trauma Stress (2008) 21:209–13. doi: 10.1002/jts.20331

  • 37

    DifedeJCukorJJayasingheNPattIJedelSSpielmanLet al. Virtual reality exposure therapy for the treatment of posttraumatic stress disorder following September 11, 2001. J Clin Psychiatry (2007) 68:1639–47. doi: 10.4088/JCP.v68n1102

  • 38

    WalsheDGLewisEJKimSIO’sullivanKWiederholdBK. Exploring the use of computer games and virtual reality in exposure therapy for fear of driving following a motor vehicle accident. Cyberpsychol Behav (2003) 6:329–34. doi: 10.1089/109493103322011641

  • 39

    RothbaumBOHodgesLFReadyDGraapKAlarconRD. Virtual reality exposure therapy for Vietnam veterans with posttraumatic stress disorder. J Clin Psychiatry (2001) 62:617–22. doi: 10.4088/JCP.v62n0808

  • 40

    RothbaumBOHodgesLSmithSLeeJHPriceL. A controlled study of virtual reality exposure therapy for the fear of flying. J Consult Clin Psychol (2000) 68:1020–6. doi: 10.1037//0022-006X.68.6.1020

  • 41

    RothbaumBOHodgesLAlarconRReadyDShaharFGraapKet al. Virtual reality exposure therapy for PTSD Vietnam Veterans: a case study. J Trauma Stress (1999) 12:263–71. doi: 10.1023/A:1024772308758

  • 42

    RothbaumBOHodgesLWatsonBAKesslerCDOpdykeD. Virtual reality exposure therapy in the treatment of fear of flying: a case report. Behav Res Ther (1996) 34:477–81. doi: 10.1016/0005-7967(96)00007-1

  • 43

    American Psychiatric Association and American Psychiatric AssociationDsm-5 Task Force. Diagnostic and statistical manual of mental disorders DSM-5.Washington, D.C: American Psychiatric Association (2013). doi: 10.1176/appi.books.9780890425596

  • 44

    RothbaumBORizzoASDifedeJ. Virtual reality exposure therapy for combat-related posttraumatic stress disorder. Ann N Y Acad Sci (2010) 1208:126–32. doi: 10.1111/j.1749-6632.2010.05691.x

  • 45

    NorrAMSmolenskiDJRegerGM. Effects of prolonged exposure and virtual reality exposure on suicidal ideation in active duty soldiers: an examination of potential mechanisms. J Psychiatr Res (2018) 103:6974. doi: 10.1016/j.jpsychires.2018.05.009

  • 46

    PowersMBEmmelkampPM. Virtual reality exposure therapy for anxiety disorders: a meta-analysis. J Anxiety Disord (2008) 22:561–9. doi: 10.1016/j.janxdis.2007.04.006

  • 47

    Van ‘T WoutMSpoffordCMUngerWSSevinEBSheaMT. Skin conductance reactivity to standardized virtual reality combat scenes in veterans with PTSD. Appl Psychophysiol Biofeedback (2017) 42:209–21. doi: 10.1007/s10484-017-9366-0

  • 48

    RizzoS. Bravemind: Virtual Reality Exposure Therapy (http://ict.usc.edu/prototypes/pts/). US: USC (2018).

  • 49

    Maples-KellerJLBunnellBEKimSJRothbaumBO. The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harv Rev Psychiatry (2017) 25:103–13. doi: 10.1097/HRP.0000000000000138

  • 50

    OprisDPinteaSGarcia-PalaciosABotellaCSzamoskoziSDavidD. Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depress Anxiety (2012) 29:8593. doi: 10.1002/da.20910

  • 51

    CarlinASHoffmanHGWeghorstS. Virtual reality and tactile augmentation in the treatment of spider phobia: a case report. Behav Res Ther (1997) 35:153–8. doi: 10.1016/S0005-7967(96)00085-X

  • 52

    Garcia-PalaciosAHoffmanHCarlinAFurnessTA3rdBotellaC. Virtual reality in the treatment of spider phobia: a controlled study. Behav Res Ther (2002) 40:983–93. doi: 10.1016/S0005-7967(01)00068-7

  • 53

    KlingerELegeronPRoySCheminILauerFNuguesP. Virtual reality exposure in the treatment of social phobia. Stud Health Technol Inform (2004) 99:91119. doi: 10.1037/e705482011-034

  • 54

    WallachHSBar-ZviM. Virtual-reality-assisted treatment of flight phobia. Isr J Psychiatry Relat Sci (2007) 44:2932. https://www.ncbi.nlm.nih.gov/pubmed/17665808

  • 55

    HongYJKimHEJungYHKyeongSKimJJ. Usefulness of the mobile virtual reality self-training for overcoming a fear of heights. Cyberpsychol Behav Soc Netw (2017) 20:753–61. doi: 10.1089/cyber.2017.0085

  • 56

    RaghavKVan WijkAJAbdullahFIslamMNBernatchezMDe JonghA. Efficacy of virtual reality exposure therapy for treatment of dental phobia: a randomized control trial. BMC Oral Health (2016) 16:25. doi: 10.1186/s12903-016-0186-z

  • 57

    GujjarKRVan WijkASharmaRDe JonghA. Virtual reality exposure therapy for the treatment of dental phobia: a controlled feasibility study. Behav Cogn Psychother (2018) 46:367–73. doi: 10.1017/S1352465817000534

  • 58

    ChadREmaanSJillianO. Effect of virtual reality headset for pediatric fear and pain distraction during immunization. Pain Manag. (2018) 8(3):175179. doi: 10.2217/pmt-2017-0040

  • 59

    DeppermannSNotzonSKroczekARosenbaumDHaeussingerFBDiemerJet al. Functional co-activation within the prefrontal cortex supports the maintenance of behavioural performance in fear-relevant situations before an iTBS modulated virtual reality challenge in participants with spider phobia. Behav Brain Res (2016) 307:208–17. doi: 10.1016/j.bbr.2016.03.028

  • 60

    FreemanD. Studying and treating schizophrenia using virtual reality: a new paradigm. Schizophr Bull (2008) 34:605–10. doi: 10.1093/schbul/sbn020

  • 61

    SohnBKHwangJYParkSMChoiJSLeeJYLeeJYet al. Developing a virtual reality-based vocational rehabilitation training program for patients with schizophrenia. Cyberpsychol Behav Soc Netw (2016) 19:686–91. doi: 10.1089/cyber.2016.0215

  • 62

    KurtzMMBakerEPearlsonGDAsturRS. A virtual reality apartment as a measure of medication management skills in patients with schizophrenia: a pilot study. Schizophr Bull (2007) 33:1162–70. doi: 10.1093/schbul/sbl039

  • 63

    SmithMJFlemingMFWrightMARobertsAGHummLBOlsenDet al. Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophr Res (2015b) 166:8691. doi: 10.1016/j.schres.2015.05.022

  • 64

    TsangMMManDW. A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophr Res (2013) 144:5162. doi: 10.1016/j.schres.2012.12.024

  • 65

    KandalaftMRDidehbaniNKrawczykDCAllenTTChapmanSB. Virtual reality social cognition training for young adults with high-functioning autism. J Autism Dev Disord (2013) 43:3444. doi: 10.1007/s10803-012-1544-6

  • 66

    SmithMJFlemingMFWrightMALoshMHummLBOlsenDet al. Brief report: vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training. J Autism Dev Disord (2015a) 45:3364–9. doi: 10.1007/s10803-015-2470-1

  • 67

    SimoesMBernardesMBarrosFCastelo-BrancoM. Virtual travel training for autism spectrum disorder: proof-of-concept interventional study. JMIR Serious Games (2018) 6:e5. doi: 10.2196/games.8428

  • 68

    SnowdonDA. Aging and Alzheimer’s disease: lessons from the Nun Study. Gerontologist (1997) 37:150–6. doi: 10.1093/geront/37.2.150

  • 69

    CushmanLASteinKDuffyCJ. Detecting navigational deficits in cognitive aging and Alzheimer disease using virtual reality. Neurology (2008) 71:888–95. doi: 10.1212/01.wnl.0000326262.67613.fe

  • 70

    Garcia-BetancesRIArredondo WaldmeyerMTFicoGCabrera-UmpierrezMF. A succinct overview of virtual reality technology use in Alzheimer’s disease. Front Aging Neurosci (2015) 7:80. doi: 10.3389/fnagi.2015.00080

  • 71

    ZygourisSNtovasKGiakoumisDVotisKDoumpoulakisSSegkouliSet al. A preliminary study on the feasibility of using a virtual reality cognitive training application for remote detection of mild cognitive impairment. J Alzheimers Dis (2017) 56:619–27. doi: 10.3233/JAD-160518

  • 72

    MoyleWJonesCDwanTPetrovichT. Effectiveness of a virtual reality forest on people with dementia: a mixed methods pilot study. Gerontologist (2018) 58:478–87. doi: 10.1093/geront/gnw270

  • 73

    La CorteVSperdutiMAbichouKPiolinoP. Episodic memory assessment and remediation in normal and pathological aging using virtual reality: a mini review. Front Psychol (2019) 10:173. doi: 10.3389/fpsyg.2019.00173

  • 74

    JonesTMooreTChooJ. The impact of virtual reality on chronic pain. PLoS One (2016) 11:e0167523. doi: 10.1371/journal.pone.0167523

  • 75

    IndovinaPBaroneDGalloLChiricoADe PietroGGiordanoA. Virtual reality as a distraction intervention to relieve pain and distress during medical procedures: a comprehensive literature review. Clin J Pain (2018) 34:858–77. doi: 10.1097/AJP.0000000000000599

  • 76

    Navarro-HaroMVLopez-Del-HoyoYCamposDLinehanMMHoffmanHGGarcia-PalaciosAet al. Meditation experts try virtual reality mindfulness: a pilot study evaluation of the feasibility and acceptability of virtual reality to facilitate mindfulness practice in people attending a mindfulness conference. PLoS One (2017) 12:e0187777. doi: 10.1371/journal.pone.0187777

  • 77

    TashjianVCMosadeghiSHowardARLopezMDupuyTReidMet al. Virtual reality for management of pain in hospitalized patients: results of a controlled trial. JMIR Ment Health (2017) 4:e9. doi: 10.2196/mental.7387

  • 78

    GarrettBTavernerTMcdadeP. Virtual reality as an adjunct home therapy in chronic pain management: an exploratory study. JMIR Med Inform (2017) 5:e11. doi: 10.2196/medinform.7271

  • 79

    FalconerCJRoviraAKingJAGilbertPAntleyAFearonPet al. Embodying self-compassion within virtual reality and its effects on patients with depression. BJPsych Open (2016) 2:7480. doi: 10.1192/bjpo.bp.115.002147

  • 80

    OhyamaSNishiikeSWatanabeHMatsuokaKAkizukiHTakedaNet al. Autonomic responses during motion sickness induced by virtual reality. Auris Nasus Larynx (2007) 34:303–6. doi: 10.1016/j.anl.2007.01.002

  • 81

    ChenWChaoJGChenXWWangJKTanC. Quantitative orientation preference and susceptibility to space motion sickness simulated in a virtual reality environment. Brain Res Bull (2015) 113:1726. doi: 10.1016/j.brainresbull.2015.01.007

  • 82

    CarnegieKRheeT. Reducing visual discomfort with HMDs using dynamic depth of field. IEEE Comput Graph Appl (2015) 35:3441. doi: 10.1109/MCG.2015.98

Summary

Keywords

virtual reality, psychiatric treatment, psychiatric disorders, dementia, motion sickness

Citation

Park MJ, Kim DJ, Lee U, Na EJ and Jeon HJ (2019) A Literature Overview of Virtual Reality (VR) in Treatment of Psychiatric Disorders: Recent Advances and Limitations. Front. Psychiatry 10:505. doi: 10.3389/fpsyt.2019.00505

Received

02 March 2019

Accepted

27 June 2019

Published

19 July 2019

Volume

10 - 2019

Edited by

Hyun Kook Lim, The Catholic University of Korea, South Korea

Reviewed by

Hernando Santamaría-García, Pontifical Javeriana University, Colombia; Veronica Perez De La Cruz, National Institute of Neurology and Neurosurgery (INNN), Mexico

Updates

Copyright

*Correspondence: Hong Jin Jeon,

This article was submitted to Aging Psychiatry, a section of the journal Frontiers in Psychiatry

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics