Acceptability of an internet-delivered intervention for adjustment disorder and its role as predictor of efficacy

Introduction

Since their appearance in the late 90s, Internet-delivered interventions have been widely investigated and accumulated a strong empirical support. Different meta-analyses pointed out that these interventions can be equally effective as face-to-face psychotherapy (Andrews et al., 2018; Carlbring et al., 2018), not only in controlled trial settings but also in the context of routine psychiatric care (Hedman et al., 2014; Mathiasen et al., 2018). Therefore, internet interventions represent a potentially feasible and effective solution to the current treatment gap, which refers to the difference between the prevalence of mental disorders and the proportion of people affected by mental disorders who receive treatment (Kohn et al., 2004).

According to WHO, up to 50% of people with mental health problems do not receive help in high-income countries (World Health Organization, 2013). This rate is even higher in low- and middle-income countries. Long waiting lists, high cost, and desire to handle the problem on one’s own are some of the reasons for this situation (Saxena et al., 2007; Andrade et al., 2014). The quality of the received treatment is another important problem. An epidemiological survey conducted across 21 countries revealed that only 27.6% of people meeting criteria for an anxiety disorder have received some treatment, though only about 9.8% received a possibly adequate treatment (Alonso et al., 2018). Given these challenges, researching strategies to improve access to evidence-based psychological interventions is paramount since untreated mental health symptoms can lead to prolonged emotional distress, reduced functioning, and even suicide (Brown et al., 2002; Williams et al., 2008). Need for alternative forms of psychotherapy also became evident during the recent COVID-19 pandemic, when the access to conventional (face-to-face) psychotherapy has been hampered by quarantine policies requiring social distancing.

Internet-delivered interventions can help to overcome or sidestep some of the current barriers to psychotherapy. Among the main advantages of internet-delivered interventions are accessibility, flexibility, affordability, confidentiality, and their wide reach (Griffiths et al., 2006; Musiat et al., 2014). However, these advantages are not enough to consider internet-delivered interventions as a feasible alternative to face-to-face psychotherapy. Despite demonstrated effectiveness, the implementation of a certain treatment represents a significant challenge if patients are not willing to use it. Thus, evidence suggests that there is a direct link between treatment acceptability, adherence, and clinical outcomes (Swift and Callahan, 2009). Notwithstanding this evidence, little attention has been paid to the assessment of the variables related to the acceptability of internet-delivered interventions.

Intervention acceptability is usually estimated from take-up and drop-out rates, or from patients’ satisfaction, collecting information about positive and negative aspects of the received treatment (Kaltenthaler et al., 2008). Treatment preferences, expectations, or ease of use have been also reported as indicators of acceptability (Mohr et al., 2010; Botella et al., 2016). Overall, the existing evidence suggests that internet-delivered interventions are a less preferred treatment option than the face-to-face therapy (Mohr et al., 2010; Gun et al., 2011; Kayrouz et al., 2015; Batterham and Calear, 2017). However, a large proportion of people inclined to use internet-delivered interventions (March et al., 2018) and the high satisfaction among those who complete them (Richards et al., 2016; Andrews et al., 2018; Campos et al., 2018; Mira et al., 2019) suggest that internet-delivered interventions represent a potentially attractive treatment alternative for mental disorders.

Certain clinical conditions, such as Adjustment Disorder (AjD), might be more suited for internet interventions (Maercker et al., 2015). AjD is the diagnostic category for the distress triggered by an identifiable stressful event (American Psychiatric Association, 2013). According to the literature, AjD is a highly prevalent condition (Evans et al., 2013; Yaseen, 2017), usually associated with a favorable prognosis (Carta et al., 2009; Yates et al., 2016). In fact, AjD is considered a transient condition, which may be resolved spontaneously (Zelviene and Kazlauskas, 2018). However, AjD involves significant distress, important reduction of quality of life, and severe interference in daily activities (Fernández et al., 2012; Gradus, 2017). Furthermore, sometimes AjD can become chronic, lead to more severe conditions and increase suicide risk (Casey et al., 2015; O’Donnell et al., 2016). Evidence also points out that there are different vulnerability factors for the development of AjD. More specifically, AjD was found to be associated with greater neuroticism, less extraversion, and more maladaptive coping styles (Vallejo-Sánchez and Pérez-García, 2017). Thus, despite its usually transient nature, AjD symptoms can reappear in response to new stressors if an appropriate treatment is not provided. Previous research revealed that AjD is one of the main causes of recurrent sickness absence (Van der Klink et al., 2003; Koopmans et al., 2011). Therefore, a treatment should be considered in order to alleviate the distress and prevent the aggravation or recurrence of the disorder (Bachem and Casey, 2018).

The few controlled studies available suggest that internet interventions constitute an effective treatment option to reduce AjD symptom burden (Eimontas et al., 2018; Moser et al., 2019; Rachyla et al., 2020). However, little is known about their acceptability. The inclusion of therapist, for example, seems to lead to lower dropout rate, especially when it is provided on weekly basis (Rachyla et al., 2020) instead of on demand (Eimontas et al., 2018). The aim of the present work is to address the existing knowledge gap concerning the acceptability of internet interventions for AjD. Data regarding take-up rates, adherence, treatment expectations, satisfaction and perceived usefulness from a randomized controlled trial (RCT) on a guided internet-delivered Cognitive Behavioral Therapy (ICBT) intervention for AjD (Rachyla et al., 2018) were analyzed. The potential impact of acceptability related indicators on intervention efficacy was explored. Finally, qualitative data about ICBT strengths and limitations were collected.

Materials and methods

Research design

The acceptability was assessed among participants of a randomized controlled trial (RCT) aimed at developing and validating an ICBT intervention for AjD (Rachyla et al., 2018). Participants were randomly allocated to either intervention or waiting list control group. In this work only data from the intervention group were used. The assessment was conducted online at five different moments: baseline, post-intervention, 3-, 6-, and 12-month follow-up. A brief post-module assessment was also included. Patients’ expectations about the treatment were assessed after the first module, which included detailed information about the intervention program.

The study protocol was approved by the Ethics Commitee of Universitat Jaume I and registered at https://www.clinicaltrials.gov as NCT02758418. The participation was voluntary. Patients’ informed consent was obtained before their inclusion in the trial. The agreement to join the study was provided by clicking the “Accept” button and answering a brief socio-demographic survey.

Sample characteristics

In the present work, only participants from the intervention group were included. Therefore, the sample consisted of 34 participants between the ages of 18 and 58 (M = 32.59, SD = 10.40), the majority of whom were women (n = 24, 70.6%). Twenty-seven (79.4%) of the participants were from Spain, two (5.9%) from Argentina, two (5.9%) from Colombia, one (2.9%) from Brazil, one (2.9%) from El Salvador, and one (2.9%) from Portugal. Twenty-four (70.6%) participants have completed university education or were university students. Regarding the clinical characteristics of the sample, twenty-five (73.5%) presented mixed anxiety and depressive symptoms, six (17.6%) presented only anxiety symptoms, and three (8.8%) presented low mood. Only six (17.6%) participants were taking medication. Drug name and dosage were registered at baseline and then monitored throughout the intervention. More detailed sample description can be seen in the Table 1.

TABLE 1

TABLE 1. Characteristics of study participants at baseline.

Recruitment and procedure

The sample for the trial was recruited via printed advertisements and announcements on internet, radio and newspaper. Collaborating centers informed about the trial when detected possible cases of AjD. Those interested in participating contacted the research team by email or telephone. A telephone interview was arranged to provide more detailed information about the trial and conduct the diagnostic interview. Participants who fulfilled the inclusion criteria and signed online informed consent were included in the study. The block randomization was conducted by an independent researcher, who was not involved in the RCT, and the assigned group was communicated by telephone. The baseline assessment was completed after the randomization. The intervention group had access to the ICBT 24 h after completing the assessment. The treatment lasted from 7 to 10 weeks. During the intervention period patients received weekly therapist support. The support consisted in a voice call (maximum 10 min duration) aimed at clarifying doubts, reminding the importance of moving forward, and congratulating for their effort. When the intervention was finished, participants completed the assessment online and the diagnostic and opinion interviews were conducted by telephone.

Inclusion and exclusion criteria

Apart from meeting DSM-5 (American Psychiatric Association, 2013) criteria for AjD, participants had to be at least 18 years old, understand and be able to read Spanish, and have an email address, access to internet, and basic knowledge about how to use a computer. Participants were excluded if they were already receiving psychological assistance, met diagnostic criteria for another severe mental disorder (including personality disorder and substance abuse or dependence) or presented high risk of suicide or self-destructive behaviors. The medication was monitored over the course of the trial. A change in the medication or an increase in the dosage was considered an exclusion criterion.

Measures

Table 2 summarizes the instruments completed at each study stage.

TABLE 2

TABLE 2. Timing for each instrument.

Diagnostic interview

Diagnostic Interview for Adjustment Disorders (Quero et al., 2019). It is a semi-structured interview which lasts approximately 60 min and provides information about the main stressor, clinical symptoms developed as a consequence of this stressor and its impact on the person’s normal life and functioning. ADIS clinician’s severity rating scale (Di Nardo et al., 1994) was completed by the interviewers at the end of each interview in order to indicate the level of distress-interference in functioning (from 0/“Absent” to 8/“Very severely disturbing/disabling”).

Patients’ expectations about the treatment

A 6-item scale was adapted from the Credibility/Expectancy questionnaire (Borkovec and Nau, 1972): 1) “How logical does the treatment seem to you?”; 2) “How satisfied are you with the treatment you are going to receive?”; 3) “To what extent would you recommend this intervention to a friend of yours with the same problem?”; 4) “How useful do you feel this treatment would be in treating other psychological problems?”; 5) “To what extent do you feel this treatment is going to be useful for you?”; 6) “How aversive do you feel this treatment will be?”. Each item was rated from 1 (“Not at all”) to 10 (“Very much”). The item 6 was not included in the analysis given that participants reported having difficulties to understand its meaning.

Patients’ satisfaction with the treatment

Patients’ satisfaction after completing the intervention was assessed using the same scale than for expectations, but the items were slightly modified: 1) “How logical did the treatment seem to you?”; 2) “How satisfied are you with the received treatment?”; 3) “To what extent would you recommend this intervention to a friend of yours with the same problem?”; 4) “How useful do you feel this treatment would be in treating other psychological problems?”; 5) “ To what extent do you feel this treatment helped you?”; 6) “How aversive did you find this treatment?”. Again, item 6 was not included in the analysis.

Patient’s opinion about the treatment modules

Before starting a new module participants rated from 0 (“Not at all”) to 10 (“Very much”) the following statements referring to the previous module: 1) “To what extent did you like the module?”, and 2) “To what extent did you find useful the module?”.

Patient’s opinion about the treatment and the received support

A short opinion questionnaire was developed specifically for the study. Participants were asked to rate from 0 (“Not at all”) to 10 (“Very much”) the following statements: 1) “To what extent did you find this internet intervention useful?”; 2) “To what extent do you feel that this internet intervention is an attractive way to receive psychological assistance?”; 3) “To what extent do you feel that this internet intervention is a comfortable way to receive psychological assistance?”. Qualitative data were also collected asking participants about positive and negative features of the intervention. Two questions were developed to assess participants’ opinion on weekly support: 1) “To what extent did you like to receive the weekly call from the therapist?”; 2) “To what extent did you find useful the weekly call from the therapist?”. Participants were asked to rate each of these statements from 0 (“Not at all”) to 10 (“Very much”) and justify their answer.

Treatment efficacy

Treatment efficacy was estimated from a battery of self-report questionnaires which included: Beck Depression Inventory - Second Edition (BDI-II), Beck Anxiety Inventory (BAI), Inventory of Stress and Loss (ISL), Posttraumatic Growth Inventory (PTGI), Positive and Negative Affect Scales (PANAS), and Multidimensional Quality of Life Questionnaire (MQLI). For more detail see the study protocol (Rachyla et al., 2018).

Intervention

The received intervention was TAO, the Spanish acronym for Adjustment Disorders Online. It is a seven-module ICBT, accessible at https://www.psicologiaytecnologia.com/. Participants logged in with a user name randomly generated during registration, thus protecting their privacy. The modules were activated sequentially (one after the other). Participants were advised to progress at their own pace but not more than one module per week, so it took them between 7 and 10 weeks to complete the intervention.

The program presents a linear navigation, so participants could only go forward or backward, following the usual structure of a CBT session (Figure 1): setting agenda; working on agenda (giving periodic summaries and proposing activities intended to put into practice the learned psychological techniques); assigning tasks to do before the next session; checking if patients have understood the contents which have been covered during the session; summarizing the session. Each module followed the same structure and combined texts, videos, pictures, vignettes, interactive exercises, and downloadable pdf files. This format was used in order to, first, make the program more user-friendly and, second, to help participants realize that each module is tailored towards specific treatment goals.

FIGURE 1

FIGURE 1. Intervention screenshots: (A) main menu; (B) agenda and therapeutic goals; (C) therapeutic contents; (D) self-assessment questions.

As for therapeutic contents, TAO includes the following components: 1) psychoeducation on AjD and common reactions to stressful events; 2) behavioral activation and slow breathing technique to manage negative emotions; 3) exposure to avoided situations which contribute to the maintenance of the problem; 4) problem-solving technique to improve the ability to deal with everyday challenges; 5) mindfulness to become aware of the personal experiences (feelings, thoughts, emotions); 6) elaboration and processing of the stressful event through the acceptance, confrontation, and development of a new meaning for the problematic situation; 7) positive psychology strategies to promote personal growth and a new attitude towards problems; and 8) relapse prevention. The first module is introductory and aims at providing information about how the intervention works and the contents of each module, recommendations to get the maximum benefit from the program, and enhancing the motivation for change. Content of each module is shown in the Table 3.

TABLE 3

TABLE 3. Intervention content.

Participants received a brief telephone call every week during the time that the intervention lasted. At the beginning of the intervention participants were assigned a therapist who was responsible for these weekly calls. Only two therapists participated in the study. Both had Master degree in Clinical Psychology and experience providing psychological assistance at the Emotional Disorder Clinic in Universitat Jaume I (Spain). Before the commencement of the RCT, therapists were trained in order to provide the same kind and level of support.

Statistical analyses

Means and standard deviations were computed for treatment expectations, satisfaction, and opinion, as well as for satisfaction with and perceived usefulness of each of the intervention modules and the received telephonic support. For the 26 participants who completed the whole intervention, reliable change indexes (Jacobson and Truax, 1991) were calculated for all treatment efficacy measures to evaluate the statistical significance of the pre-post change. Post-treatment scores can be found in (Rachyla et al., 2020). Stepwise regression analyses were conducted to assess whether the clinical reliable change was predicted by any of the previously analyzed parameters. Finally, qualitative information was grouped according to common themes.

Results

Acceptance of participation

As it is shown in the Figure 2, seventy-eight out of the 144 assessed volunteers fulfilled the inclusion criteria for the trial; 44 contacted because they were interested in an online intervention and 34 were derived from the Emotional Disorders Clinic of Universitat Jaume I or by their family doctor, but were initially interested in a face-to-face therapy. Sixty-eight participants signed the informed consent and were finally included in the RCT (34 were randomized to the intervention group and 34 to the control group). Four of the patients derived from the Emotional Disorders Clinic refused to participate in the study and preferred face-to-face treatment. Finally, six participants did not participate because they were unable to be contacted. Three of these participants were initially interested in an online intervention and three in a face-to-face therapy.

FIGURE 2

FIGURE 2. Flowchart of participants.

Adherence

Out of 34 participants included in the TAO condition, eight participants dropped out from the intervention group before they completed the ICBT program. The reasons for dropping out were: time constraints (n = 2), feeling that the intervention did not match their needs (n = 2), worsening of clinical symptoms and feeling that a professional help was needed to manage them (n = 2). Two participants did not answer the phone or the email, so their reasons for dropping out are unknown.

Participants’ expectations about the treatment

Participants’ scores on Treatment Expectation Scale, differentiating between completers and dropouts, are presented in Table 4. As can be seen, completers scored slightly higher on the scale, as well as on each of the items. Overall, treatment expectations were positive (mean scores above or around 7 on a 0 to 10 scale). Regression analyses revealed a non-significant impact of treatment expectations on the clinically reliable change at post-intervention (ρ > 0.05).

TABLE 4

TABLE 4. Mean and standard deviations on Treatment Expectation Scale^a for completers and dropouts.

Participants’ satisfaction with the treatment

Participants’ scores on Treatment Satisfaction Scale are presented in Table 5. On a 0 to 10 rating scale, users reported moderate levels of satisfaction (mean scores above or around 7). Regression analysis revealed that higher treatment satisfaction reflected in greater pretest-posttest positive change in PANAS+ (R² = 0.32, β = 0.59, t = 3.49, ρ = 0.002) and PTGI (R² = 0.13, β = 0.41, t = 2.13, ρ = 0.044).

TABLE 5

TABLE 5. Mean and standard deviations on treatment satisfaction scale^a.

Intervention modules: Perceived satisfaction and usefulness

Satisfaction with, and perceived usefulness of, intervention modules is reported in Table 6. Module 6 was the module with higher satisfaction (M = 7.76, SD = 1.92) and usefulness (M = 7.76, SD = 1.92) score. Regression analyses showed that satisfaction with module 0 was a significant predictor of posttraumatic growth from baseline to post-intervention (R² = 0.17, β = 0.45, t = 2.44, ρ = 0.023), while satisfaction with module 6 predicted change in positive affect (R² = 0.30, β = 0.57, t = 3.36, ρ = 0.003) and satisfaction with module 5 predicted change in perceived quality of life (R² = 0.17, β = 0.45, t = 2.40, ρ = 0.025). Finally, perceived usefulness of module 6 was predictor of change in positive affect (R² =0 .29, β = 0.56, t = 3.26, ρ = 0.003) and quality of life (R² = 0.13, β = 0.40, t = 2.12, ρ = 0.045).

TABLE 6

TABLE 6. Satisfaction with the intervention modules and their perceived usefulness.

Participants’ opinion about the treatment

The program was perceived as a useful (M = 8.00, SD = 1.48), convenient (M = 8.91, SD = 1.44), and attractive (M = 7.77, SD = 1.77) way of receiving psychological assistance. Results concerning qualitative opinion about the received treatment are presented in Table 7.

TABLE 7

TABLE 7. Qualitative opinion about the internet intervention.

Telephone support

Participants reported that they liked the weekly calls (M = 9.41, SD = 1.10) and also found them highly useful (M = 8.82, SD = 1.37). Reasons for the positive evaluation of the telephone support were: promotes the adherence (motivates to move forward), opportunity to solve doubts (even if you do not have any), knowing that you are not alone (a specialist behind the machine who is monitoring your progress).

Discussion

The aim of the present work was to assess the acceptability of an ICBT intervention for AjD in terms of take-up rates, adherence, as well as treatment expectations, satisfaction, and usefulness. Impact of acceptability on treatment efficacy was also investigated. Another objective was to explore if the ICBT was considered as an attractive and convenient way of receiving psychological assistance among patients with AjD. Finally, positive and negative program features were analyzed.

Results for take-up rates showed that only four out of 78 participants fulfilling eligibility criteria for the RCT rejected to participate, being all of them initially interested in face-to-face therapy. This result is partially supported by the existing literature which suggests that individuals interested in receiving psychological intervention tend to show greater preference for face-to-face interventions (Mohr et al., 2010). However, the other 30 referred patients seeking psychological assistance accepted to participate in the RCT and 44 people contacted because they were interested in an online intervention. These findings suggest that patients with AjD perceived an internet-delivered intervention as a feasible alternative to face-to-face therapy.

With regard to adherence, out of 34 participants, 76.5% completed the whole seven-module ICBT. The dropout rate was 23.5%, slightly below the 30% reported in the literature on the adherence to guided ICBT for other mental disorders (Van Ballegooijen et al., 2014; Andrews et al., 2018) and considerably lower than those reported for completely self-applied interventions for AjD (Bachem and Maercker, 2016; Eimontas et al., 2017; Eimontas et al., 2018), and even for internet interventions with therapist support on demand (Eimontas et al., 2018). These findings suggest that the providence of guidance might elevate patients’ motivation and consequent treatment engagement.

Time constriction was one of the reasons behind treatment dropout. The worsening of symptoms was another reason, in line with previous research which found that patients who dropped out from ICBT reported that treatment demands exceeded their capabilities (Johansson et al., 2015). In our study, the feeling of being completely overwhelmed by the situation prompted two participants to seek greater therapeutic support. In fact, both participants initiated face-to-face intervention after leaving the project. The third reason for dropping out was the feeling that the intervention did not match their needs.

As for expectations with regard to the ICBT for AjD, results were similar to those found in the literature for other disorders, such as depression (Botella et al., 2016) or specific phobia (Campos et al., 2018). Treatment expectations were slightly lower among dropouts than among completers, supporting somehow the assumption that poor treatment expectations can lead to treatment abandonment (Swift and Callahan, 2009). However, due to the small sample size of dropouts in the present study, it could not be explored whether the observed between-group differences were statistically significant, hence further study is needed in order to draw firmer conclusions in this regard. Regression analyses revealed that expectations did not predict treatment efficacy, suggesting that even users with lower initial expectations benefited from the treatment. Since a previous study reported that the providence of information about an internet-delivered intervention influenced positively treatment acceptance among depressed patients (Ebert et al., 2015), this strategy could also be used to encourage patients to continue to use the ICBT intervention, even if it does not seem to meet their demands. It also should be reminded that treatment results are not immediate, in order to prevent patients from getting discouraged and dropping out.

Participants expressed moderate satisfaction with the received intervention. Regression analyses showed that treatment satisfaction did not predict distress reduction. Patients who were less satisfied with TAO reported clinical change on BDI, BAI, ISL, and PANAS negative affect subscale similar to that reported by more satisfied participants. Therefore, it seems that the ICBT was equally effective in all patients. However, the satisfaction predicted posttraumatic growth and increase in positive affect and quality of life. The analyses of the impact of each treatment module revealed that satisfaction with modules 0, 5, and 6 predicted clinically significant change in posttraumatic growth, quality of life, and positive affect respectively. Perceived usefulness of module 6, on the other hand, predicted clinically significant change in positive affect and quality of life. Module 0 was aimed at helping patients to give name to their problem and propose a way to manage it. Feeling more identified with TAO target population and agreeing with the proposed way to approach the problem might lead to higher satisfaction, greater treatment engagement, and consequent posttraumatic growth. On the other hand, the module 6 was aimed at raising awareness about the progress made since the beginning of the intervention, summarizing learned skills, preparing patients for recurrence of some symptoms and identifying warning signs of relapse. Seeing the program as an integrated whole, where every module was focused on treating different parts of the same problem, might help patients to feel more empowered to handle the problem situation. In addition, the relapse prevention component was aimed at promoting the sense of achievement, which also might explain greater improvement in positive affect and quality of life. Finally, the impact of module 5 on the quality of life was an encouraging finding since that module had as objective to change the way of approaching the stressful situation.

Finally, the general opinion of participants was that TAO represented a useful, comfortable and attractive way of receiving psychological assistance. As positive aspects of TAO participants highlighted its contents (inclusion of effective techniques, examples, practical, and attractive exercises), form (structure, intuitive interface, easy understandable language, and brief modules) and format (combination of different multimedia resources, inclusion of videos, and downloadable PDF files). Regarding the advantages of ICBT interventions, participants highlighted confidentiality, accessibility from any place and at any time, comfort of working on your wellbeing from home, without need for transportation or changing your schedule, chance to progress at your own way, reviewing or dedicating more time to certain parts of the intervention, and perceived self-efficacy. However, participants indicated that ICBT required considerable motivation, commitment, and determination. Therapist support was seen as an important resource to overcome this barrier. Despite telephone calls did not include additional clinical content, participants felt that they had someone to help them. This finding is consistent with previous research which found out that support provided in ICBT was effective even when it was provided by a person without clinical experience (Baumeister et al., 2014). Thus, if the main function of therapist call is to increase motivation (Schröder et al., 2016), it might be replaced by other type of automated support. In this regard, automated emails were found to promote both adherence and clinical improvement (Titov et al., 2013). Participants from TAO indicated that a more customizable and interactive interface would also enhance ICBT use.

Limitations

The generalization of present findings could be compromised by several study limitations which should be addressed in future research. First, the sample size was too small to allow definitive conclusions on acceptability of ICBT interventions for the treatment of AjD. Second, 73.5% of the sample was people who contacted because they were interested in ICBT intervention. This initial interest could have affected the results. Third, the majority of the sample were women and most had relatively high education level, factors which were found to be associated with greater willingness to use internet-delivered interventions (Batterham and Calear, 2017).

Conclusion

In sum, the present work showed that ICBT with therapist support was well accepted among patients with AjD. Expectations towards the treatment were positive. The results suggest that treatment expectations were related to adherence. However, treatment outcomes were not significantly affected, indicating that those with more negative expectation also improved with treatment. The drop-out rates were lower than it was expected, probably because of the weekly therapist telephone support. Satisfaction with the treatment was good and it was predictor of increase in positive affect and posttraumatic growth. Three intervention modules were related to treatment satisfaction: introductory module which provided information about the ICBT, relapse prevention module, and also the module aimed at changing the way of perceiving the stressful event. This suggests the convenience of incorporating these components into treatment protocols for AjD. Therapist support promoted treatment adherence. However, adherence might be also achieved through certain system functions as well as including contents aimed at facilitating engagement, and not only clinical change.

To the best of our knowledge, this is the most comprehensive study exploring acceptability of an ICBT intervention for AjD, so it represents a significant step forward in this field. Further research is required since it will allow revealing more about both acceptability and engagement related factors. Development of treatment programs which are effective and also attractive for patients will lead to increased exploitation of all potential offered by internet-delivered interventions.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Ethics Commitee of Universitat Jaume I. The patients/participants provided their written informed consent to participate in this study.

Author contributions

IR analysed the data and drafted the manuscript, with important contributions from SQ and CB. IR, in collaboration with SQ and SM, designed the study and participated in each of its phases. SQ, IR, and DC carried out the Internet-based adaptation of the treatment protocol. All authors participated in the review and revision of the manuscript and approved the final manuscript for publication.

Funding

The study was funded by the following institutions: Ministerio de Economía y Competitividad (Spain; Plan Nacional I + D + I, PSI 2013-41783-R); Plan 2018 de Promoción de la Investigación de la Universitat Jaume I (UJI-2018-57); Generalitat Valenciana pre-PhD grant program (VALi + d: ACIF/2015/181); CIBERObn, an initiative of the Instituto de Salud Carlos III (ISCIII) (ISC III CB06 03/0052); and the grant Ministerio de Ciencia, Innovación y Universidades (Spain; Programa Estatal I+D+i RTI2018-100993-B-100) funded by MCIN/AEI/10.13039/501100011033, by “ERDF A way of making Europe” and by the “European Union”. The aforementioned institutions had no role in the design or execution of the study, analyses and interpretation of the data, or decision to submit results.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor is currently co-organizing a Research Topic with one of the authors (SQ) and confirms the absence of any other collaboration.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alonso, J., Liu, Z., Evans-Lacko, S., Sadikova, E., Sampson, N., Chatterji, S., et al. (2018). Treatment gap for anxiety disorders is global: Results of the World Mental Health Surveys in 21 countries. Depress. Anxiety 35 (3), 195–208. doi:10.1002/da.22711

PubMed Abstract | CrossRef Full Text | Google Scholar

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders: DSM-V. 5a Ed. Washington: American Psychiatric Association.

Google Scholar

Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A., Borges, G., et al. (2014). Barriers to mental health treatment: Results from the WHO world mental health surveys. Psychol. Med. 44 (6), 1303–1317. doi:10.1017/s0033291713001943

PubMed Abstract | CrossRef Full Text | Google Scholar

Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., et al. (2018). Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. J. Anxiety Disord. 55 (2017), 70–78. doi:10.1016/j.janxdis.2018.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Bachem, R., and Casey, P. (2018). Adjustment disorder: A diagnosis whose time has come. J. Affect. Disord. 227, 243–253. doi:10.1016/j.jad.2017.10.034

PubMed Abstract | CrossRef Full Text | Google Scholar

Bachem, R., and Maercker, A. (2016). Self-help interventions for adjustment disorder problems: A randomized waiting-list controlled study in a sample of burglary victims. Cogn. Behav. Ther. 45 (5), 397–413. doi:10.1080/16506073.2016.1191083 Available from: http://www.ncbi.nlm.nih.gov/pubmed/27299909.

PubMed Abstract | CrossRef Full Text | Google Scholar

Batterham, P. J., and Calear, A. L. (2017). Preferences for internet-based mental health interventions in an adult online sample: Findings from an online community survey. JMIR Ment. Health 4 (2), e26. doi:10.2196/mental.7722 Available from: http://mental.jmir.org/2017/2/e26/.

PubMed Abstract | CrossRef Full Text | Google Scholar

Baumeister, H., Reichler, L., Munzinger, M., and Lin, J. (2014). The impact of guidance on Internet-based mental health interventions - a systematic review. Internet Interv. 1 (4), 205–215. doi:10.1016/j.invent.2014.08.003

CrossRef Full Text | Google Scholar

Borkovec, T. D., and Nau, S. D. (1972). Credibility of analogue therapy rationales. J. Behav. Ther. Exp. Psychiatry 3 (4), 257–260. doi:10.1016/0005-7916(72)90045-6 Available from: https://www.sciencedirect.com/science/article/pii/0005791672900456.

CrossRef Full Text | Google Scholar

Botella, C., Mira, A., Moragrega, I., García-Palacios, A., Bretón-López, J., Castilla, D., et al. (2016). An internet-based program for depression using activity and physiological sensors: Efficacy, expectations, satisfaction, and ease of use. Neuropsychiatr. Dis. Treat. 12, 393–406. doi:10.2147/ndt.s93315

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, S. A., Parker, J. D., and Godding, P. R. (2002). Administrative, clinical, and ethical issues surrounding the use of waiting lists in the delivery of mental health services. J. Behav. Health Serv. Res. 29, 217–228. doi:10.1097/00075484-200205000-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

Campos, D., Mira, A., Bretón-López, J., Castilla, D., Botella, C., Baños, R. M., et al. (2018). The acceptability of an internet-based exposure treatment for flying phobia with and without therapist guidance: Patients’ expectations, satisfaction, treatment preferences, and usability. Neuropsychiatr. Dis. Treat. [Internet], 879, 14–92. doi:10.2147/NDT.S153041 Available from: https://www.dovepress.com/the-acceptability-of-an-internet-based-exposure-treatment-for-flying-p-peer-reviewed-article-NDT.

PubMed Abstract | CrossRef Full Text | Google Scholar

Carlbring, P., Andersson, G., Cuijpers, P., Riperd, H., and Hedman-Lagerlöfh, E. (2018). Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: An updated systematic review and meta-analysis. Cogn. Behav. Ther. 47 (1), 1–18. doi:10.1080/16506073.2017.1401115

PubMed Abstract | CrossRef Full Text | Google Scholar

Carta, M., Balestrieri, M., Murru, A., and Hardoy, M. (2009). Adjustment disorder: Epidemiology, diagnosis and treatment. Clin. Pract. Epidemiol. Ment. Health 5 (1), 15. doi:10.1186/1745-0179-5-15

PubMed Abstract | CrossRef Full Text | Google Scholar

Casey, P., Jabbar, F., O’Leary, E., and Doherty, A. M. (2015). Suicidal behaviours in adjustment disorder and depressive episode. J. Affect. Disord. 174, 441–446. doi:10.1016/j.jad.2014.12.003 Available from: http://www.sciencedirect.com/science/article/pii/S0165032714007885.

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Nardo, P. A., Brown, T. A., and Barlow, D. H. (1994). Anxiety disorders interview schedule for DSM-IV: Lifetime version (ADIS-IV-L). Albany, New York: Graywind Publications.

Google Scholar

Ebert, D. D., Berking, M., Cuijpers, P., Lehr, D., Pörtner, M., and Baumeister, H. (2015). Increasing the acceptance of internet-based mental health interventions in primary care patients with depressive symptoms. A randomized controlled trial. J. Affect. Disord. 176, 9–17. doi:10.1016/j.jad.2015.01.056

PubMed Abstract | CrossRef Full Text | Google Scholar

Eimontas, J., Gegieckaite, G., Dovydaitiene, M., Mazulyte, E., Rimsaite, Z., Skruibis, P., et al. (2018). The role of therapist support on effectiveness of an internet-based modular self-help intervention for adjustment disorder: A randomized controlled trial. Anxiety, Stress Coping 31 (2), 146–158. doi:10.1080/10615806.2017.1385065

PubMed Abstract | CrossRef Full Text | Google Scholar

Eimontas, J., Rimsaite, Z., Gegieckaite, G., Zelviene, P., and Kazlauskas, E. (2017). Internet-based self-help intervention for ICD-11 adjustment disorder: Preliminary findings. Psychiatr. Q. 1–10, 451–460. doi:10.1007/s11126-017-9547-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Evans, S. C., Reed, G. M., Roberts, M. C., Esparza, P., Ritchie, P., Maj, M., et al. (2013). Psychologists’ perspectives on the diagnostic classification of mental disorders: Results from the WHO-IUPsyS Global Survey. Int. J. Psychol. 48 (3), 177–193. doi:10.1080/00207594.2013.804189

PubMed Abstract | CrossRef Full Text | Google Scholar

Fernández, A., Mendive, J. M., Salvador-Carulla, L., Rubio-Valera, M., Luciano, J. V., Pinto-Meza, A., et al. (2012). Adjustment disorders in primary care: Prevalence, recognition and use of services. Br. J. Psychiatry 201 (2), 137–142. doi:10.1192/bjp.bp.111.096305

PubMed Abstract | CrossRef Full Text | Google Scholar

Gradus, Gradus J. L. (2017). Prevalence and prognosis of stress disorders - a review of the epidemiologic literature. Clin. Epidemiol. 9, 251–260. doi:10.2147/clep.s106250

PubMed Abstract | CrossRef Full Text | Google Scholar

Griffiths, F., Lindenmeyer, A., Powell, J., Lowe, P., and Thorogood, M. (2006). Why are health care interventions delivered over the internet? A systematic review of the published literature. J. Med. Internet Res. 8 (2), e10–e15. doi:10.2196/jmir.8.2.e10

PubMed Abstract | CrossRef Full Text | Google Scholar

Gun, S. Y., Titov, N., and Andrews, G. (2011). Acceptability of Internet treatment of anxiety and depression. Australas. Psychiatry 19 (3), 259–264. doi:10.3109/10398562.2011.562295

PubMed Abstract | CrossRef Full Text | Google Scholar

Hedman, E., Ljótsson, B., Kaldo, V., Hesser, H., El Alaoui, S., Kraepelien, M., et al. (2014). Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J. Affect. Disord. 155 (1), 49–58. doi:10.1016/j.jad.2013.10.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacobson, N. S., and Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 59 (1), 12–19. doi:10.1037/0022-006x.59.1.12

PubMed Abstract | CrossRef Full Text | Google Scholar

Johansson, O., Michel, T., Andersson, G., and Paxling, B. (2015). Experiences of non-adherence to internet-delivered cognitive behavior therapy: A qualitative study. Internet Interv. 2 (2), 137–142. doi:10.1016/j.invent.2015.02.006

CrossRef Full Text | Google Scholar

Kaltenthaler, E., Sutcliffe, P., Parry, G., Beverley, C., Rees, A., and Ferriter, M. (2008). The acceptability to patients of computerized cognitive behaviour therapy for depression: A systematic review. Psychol. Med. 38 (11), 1521–1530. doi:10.1017/s0033291707002607

PubMed Abstract | CrossRef Full Text | Google Scholar

Kayrouz, R., Dear, B. F., Johnston, L., Keyrouz, L., Nehme, E., Laube, R., et al. (2015). Intergenerational and cross-cultural differences in emotional wellbeing, mental health service utilisation, treatment-seeking preferences and acceptability of psychological treatments for Arab Australians. Int. J. Soc. Psychiatry 61 (5), 484–491. doi:10.1177/0020764014553004

PubMed Abstract | CrossRef Full Text | Google Scholar

Kohn, R., Saxena, S., Levav, I., and Sraceno, B. (2004). The treatment gap in mental health care. Bull. World Health Organ. 82 (03), 858–866. doi:10.1590/S0042-96862004001100011

PubMed Abstract | CrossRef Full Text | Google Scholar

Koopmans, P. C., Bültmann, U., Roelen, C. A. M., Hoedeman, R., van der Klink, J. J. L., and Groothoff, J. W. (2011). Recurrence of sickness absence due to common mental disorders. Int. Arch. Occup. Environ. Health 84 (2), 193–201. doi:10.1007/s00420-010-0540-4 Available from: http://link.springer.com/10.1007/s00420-010-0540-4.

PubMed Abstract | CrossRef Full Text | Google Scholar

Maercker, A., Bachem, R. C., Lorenz, L., Moser, C. T., and Berger, T. (2015). Adjustment disorders are uniquely suited for eHealth interventions: Concept and case study. JMIR Ment. Health 2 (2), e15. doi:10.2196/mental.4157 Available from: http://mental.jmir.org/2015/2/e15/.

PubMed Abstract | CrossRef Full Text | Google Scholar

March, S., Day, J., Ritchie, G., Rowe, A., Gough, J., Hall, T., et al. (2018). Attitudes toward e-mental health services in a community sample of adults: Online survey. J. Med. Internet Res. 20 (2), e59. doi:10.2196/jmir.9109 Available from: http://www.jmir.org/2018/2/e59/.

PubMed Abstract | CrossRef Full Text | Google Scholar

Mathiasen, K., Riper, H., Andersen, T. E., and Roessler, K. K. (2018). Guided internet-based cognitive behavioral therapy for adult depression and anxiety in routine secondary care: Observational study. J. Med. Internet Res. 20 (11), e10927. doi:10.2196/10927 Available from: http://www.jmir.org/2018/11/e10927/.

PubMed Abstract | CrossRef Full Text | Google Scholar

Mira, A., Soler, C., Alda, M., Baños, R., Castilla, D., Castro, A., et al. (2019). Exploring the relationship between the acceptability of an internet-based intervention for depression in primary care and clinical outcomes: Secondary analysis of a randomized controlled trial. Front. Psychiatry 10, 325. doi:10.3389/fpsyt.2019.00325

PubMed Abstract | CrossRef Full Text | Google Scholar

Mohr, D. C., Siddique, J., Ho, J., Duffecy, J., Jin, L., and Fokuo, J. K. (2010). Interest in behavioral and psychological treatments delivered face-to-face, by telephone, and by internet. Ann. Behav. Med. 40 (1), 89–98. doi:10.1007/s12160-010-9203-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Moser, C., Bachem, R., Berger, T., and Maercker, A. (2019). Ziel: Internet-based self-help for adjustment problems : Results of a randomized controlled trial. J. Clin. Med. 8 (10), 1655. doi:10.3390/jcm8101655

PubMed Abstract | CrossRef Full Text | Google Scholar

Musiat, P., Goldstone, P., and Tarrier, N. (2014). Understanding the acceptability of e-mental health: Attitudes and expectations towards computerised self-help treatments for mental health problems. BMC Psychiatry 14 (1), 109. doi:10.1186/1471-244x-14-109 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3999507&tool=pmcentrez&rendertype=abstract.

PubMed Abstract | CrossRef Full Text | Google Scholar

O’Donnell, M. L., Alkemade, N., Creamer, M., McFarlane, A. C., Silove, D., Bryant, R. A., et al. (2016). A longitudinal study of adjustment disorder after trauma exposure. Am. J. Psychiatry 173 (12), 1231–1238. doi:10.1176/appi.ajp.2016.16010071

PubMed Abstract | CrossRef Full Text | Google Scholar

Quero, S., Rachyla, I., Molés, M., Mor, S., Tur, C., Cuijpers, P., et al. (2019). Can between-session homework be delivered digitally? A pilot randomized clinical trial of CBT for adjustment disorders. Int. J. Environ. Res. Public Health 16 (20), 3842. doi:10.3390/ijerph16203842

PubMed Abstract | CrossRef Full Text | Google Scholar

Rachyla, I., Mor, S., Cuijpers, P., Botella, C., Castilla, D., and Quero, S. (2020). A guided internet-delivered intervention for adjustment disorders: A randomized controlled trial. Clin. Psychol. Psychother. 28, 313–324. doi:10.1002/cpp.2518

PubMed Abstract | CrossRef Full Text | Google Scholar

Rachyla, I., Pérez-Ara, M., Molés, M., Campos, D., Mira, A., Botella, C., et al. (2018). An internet-based intervention for adjustment disorder (TAO): Study protocol for a randomized controlled trial. BMC Psychiatry 18 (1), 161. doi:10.1186/s12888-018-1751-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Richards, D., Murphy, T., Viganó, N., Timulak, L., Doherty, G., Sharry, J., et al. (2016). Acceptability, satisfaction and perceived efficacy of “Space from Depression” an internet-delivered treatment for depression. Internet Interv. 5, 12–22. doi:10.1016/j.invent.2016.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Saxena, S., Thornicroft, G., Knapp, M., and Whiteford, H. (2007). Resources for mental health: Scarcity, inequity, and inefficiency. Lancet 370 (9590), 878–889. doi:10.1016/s0140-6736(07)61239-2 Available from: http://www.who.int/mental_.

PubMed Abstract | CrossRef Full Text | Google Scholar

Schröder, J., Berger, T., Westermann, S., Klein, J. P., and Moritz, S. (2016). Internet interventions for depression: New developments. Dialogues Clin. Neurosci. 18 (2), 203–212. doi:10.31887/dcns.2016.18.2/jschroeder

PubMed Abstract | CrossRef Full Text | Google Scholar

Swift, J. K., and Callahan, J. L. (2009). The impact of client treatment preferences on outcome: A meta-analysis. J. Clin. Psychol. 65 (4), 368–381. doi:10.1002/jclp.20553

PubMed Abstract | CrossRef Full Text | Google Scholar

Titov, N., Dear, B. F., Johnston, L., Lorian, C., Zou, J., Wootton, B., et al. (2013). Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: Randomised controlled trial. PLoS One 8 (7), e62873. doi:10.1371/journal.pone.0062873

PubMed Abstract | CrossRef Full Text | Google Scholar

Vallejo-Sánchez, B., and Pérez-García, A. M. (2017). The role of personality and coping in adjustment disorder. Clin. Psychol. Aust. Psychol. Soc. 21 (3), 245–251. doi:10.1111/cp.12064

CrossRef Full Text | Google Scholar

Van Ballegooijen, W., Cuijpers, P., Van Straten, A., Karyotaki, E., Andersson, G., Smit, J. H., et al. (2014). Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: A meta-analysis. PLoS One 9 (7), e100674. doi:10.1371/journal.pone.0100674

PubMed Abstract | CrossRef Full Text | Google Scholar

Van der Klink, J. J. L., Blonk, R. W. B., Schene, A. H., and van Dijk, F. J. H. (2003). Reducing long term sickness absence by an activating intervention in adjustment disorders: A cluster randomised controlled design. Occup. Environ. Med. 60 (6), 429–437. doi:10.1136/oem.60.6.429

PubMed Abstract | CrossRef Full Text | Google Scholar

Williams, M. E., Latta, J., and Conversano, P. (2008). Eliminating the wait for mental health services. J. Behav. Health Serv. Res. 35, 107–114. doi:10.1007/s11414-007-9091-1

PubMed Abstract | CrossRef Full Text | Google Scholar

World Health Organization. Mental health action plan 2013-2020. World Health Organization. Geneva; 2013. 1–44. Available from: http://apps.who.int/iris/bitstream/10665/89966/1/9789241506021_eng.pdf?ua=1%5Cnhttp://apps.who.int/iris/bitstream/10665/89966/1/9789241506021_eng.pdf

Google Scholar

Yaseen, Y. A. (2017). Adjustment disorder: Prevalence, sociodemographic risk factors, and its subtypes in outpatient psychiatric clinic. Asian J. Psychiatr. 28 (2017), 82–85. doi:10.1016/j.ajp.2017.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Yates, W. R. (2016). “Other psychiatric syndromes: Adjustment disorder, fictitious disorder, illicit steroid abuse, cultural syndromes,” in The medical basis of psychiatry. Editors S. H. Fatemi, and P. J. Clayton (Berlin, Germany: Springer).

Google Scholar

Zelviene, P., and Kazlauskas, E. (2018). Adjustment disorder : Current perspectives. Neuropsychiatr. Dis. Treat. 14, 375–381. doi:10.2147/ndt.s121072

PubMed Abstract | CrossRef Full Text | Google Scholar