Satisfaction, engagement, and outcomes in internet-delivered cognitive behaviour therapy adapted for people of diverse ethnocultural groups: an observational trial with benchmarking

Introduction: Depression and anxiety are the most common mental health disorders worldwide. Internet-Delivered Cognitive Behaviour Therapy (ICBT) can reduce barriers to care to broad cross sections of the population. However, People of Diverse Ethnocultural Backgrounds (PDEGs) other than White/Caucasian underutilize mental health services and are under represented in clinical trials of psychological interventions.

Methods: To address this research gap we adapted an evidence-based ICBT program for PDEGs. The current pilot study explores the engagement, satisfaction, and effectiveness in the adapted ICBT program by PDEGs (N=41) when benchmarked against a sample of PDEGs (N=134) who previously completed a non-adapted version of the ICBT program.

Results: An intent-to-treat analyses showed that the adapted ICBT program is effective in reducing anxiety and depression symptoms among PDEGs. Large within-group pre-to post-treatment Cohen’s effect sizes of d = 1.23, 95% CI [0.68, 1.77] and d = 1.24, 95% CI [0.69, 1.79] were found for depression and anxiety, respectively. Further, 81.8% of the PDEGs who received the adapted ICBT reported overall satisfaction, 90.9% reported increased confidence in managing symptoms, and 70.7% completed majority of the psychoeducational lessons in the ICBT program.

Conclusion: No statistically significant differences in the clinical outcomes, engagement, and satisfaction were found between the pilot study and benchmark sample. Future directions for ICBT research with PDEGs are described.

Clinical trial registration: https://beta.clinicaltrials.gov/study/NCT05523492, identifier NCT05523492.

Introduction

Depression and anxiety are among the most common mental health concerns worldwide (1). As is remarked often, with the availability of evidence-based digital mental health interventions such as transdiagnostic Internet-Delivered Cognitive Behaviour Therapy (ICBT) (e.g., 2, 3), help is just a click away. Yet a significant gap exists between those who have access to and use these services. For example, a recent analysis of ICBT utilization trends over six years (i.e., 2013-2019) in an online clinic in Saskatchewan, Canada showed that there has been a consistently lower (~10% compared to the expected population of ~32%) participation from self-identified non-White/Caucasian people in ICBT (4). People of Diverse Ethnocultural Backgrounds (PDEGs) other than White/Caucasian are known to underutilize mental health services (e.g., 5), and are also underrepresented in clinical trials of psychological interventions including ICBT (6, 7). Further, despite the recognition of the salience of ethnocultural backgrounds on mental health (e.g., 8, 9), there has been limited research on adapting ICBT to be appropriate for multiple ethnocultural groups with the aim of improving its utilization by PDEGs in routine care settings (10). Of note, for the lack of better terminology, we have used the phrase People of Diverse Ethnocultural Groups (PDEGs) to refer to all but self-identified Caucasian/White people in this study. In this regard, PDEGs encompass Indigenous Canadians as well as “visible minorities” or “racialized minorities” such as black, Asian, and Latin American. Other terms for this group that are used in the literature include visible monitories, racialized minorities, or Black, Indigenous, People of Colour (BIPOC).

To address this research gap as well as with an aim to increase use of ICBT by PDEGs in an online routine care clinic in Saskatchewan, Canada, we conducted a multi-phase patient-oriented research [see (11)] project and adapted an evidence-based ICBT program based on feedback from self-identified PDEG patients, community representatives providing services to PDEGs, and ICBT clinicians [see (12) for details of the adaptation process]. Suggestions for improvement including acknowledging the role of culture on mental health and mental health stigma in the course materials, diversifying case stories, examples and images, language simplification and audiovisual materials but did not involve modifying the cognitive behavioural approach. Moreover, further trainings were offered to clinicians and efforts were made to improve community outreach (see 12). This approach to adaptation is aligned with the Selective and Directed Treatment Adaptation Framework [SDTAF; (13, 14)], which has strong emerging evidence base to guide adaptations of Western psychological interventions for other populations. The current pilot study was conducted to assess the effectiveness, satisfaction, and engagement of the adapted ICBT by PDEGs benchmarked against data from PDEGs who took a non-adapted version of the same ICBT program in the past. Given the changes made, we expected similar effect sizes, but increased engagement and satisfaction with the adapted ICBT when benchmarked against a sample of PDEGs who took a non-adapted version of the ICBT program (see 15).

Materials and methods

Participants

This registered pilot interventional trial (NCT05523492; https://beta.clinicaltrials.gov/study/NCT05523492) used a single-group pre- to post-treatment design. Prospective participants learned about the Online Therapy Unit (www.onlinetherapyuser.ca) through a variety of sources, such as family physicians, other medical professionals, community mental health clinics, web-based searches, word of mouth, media, and posters or cards. To access the course, participants completed an online questionnaire followed by a telephone interview. During this interview, staff determined if participants met basic eligibility for the Course (i.e., ≥18 years of age, Saskatchewan resident, endorsed depression and/or anxiety symptoms, had access to a computer and the Internet, able to provide a healthcare provider emergency contact), or were experiencing concerns outside of the scope of the Course (i.e., are at a high risk for suicide, have significant alcohol and/or drug usage, or are experiencing unmanaged psychosis or mania). One additional inclusion criterion specific to this study was that patients had to self-identify as belonging to PDEGs. Participants who did not meet the eligibility criteria were directed to more appropriate local services (see 4for further details). All eligible PDEG participants (N=41) enrolled in ICBT at the Online Therapy Unit between June 2022 and November 2022 were included (see Figure 1). Sub-sample (e.g., self-identified non-White/Caucasian) data (N=134) collected between December 2021 and May 2022 at the same site with an equivalent protocol were used as the benchmarking sample [see (15) for details about benchmarking sample].

Figure 1

Figure 1 Participant flow diagram.

At post-treatment ~32% (n=13) consenting participants in the pilot study were interviewed asking about their expectations and experience with the adapted ICBT Wellbeing Course and suggestions for further improvement.

ICBT program: The Wellbeing Course

Participants were enrolled in the Adapted Wellbeing Course. This transdiagnostic course originally developed by Titov, Dear (16) addresses both depression and anxiety symptoms and was adapted as described above for PDEGs. The Course includes five online psychoeducational lessons and described (a) the cognitive-behavioural model and the relationship between thoughts, behaviours and emotions; (b) thought monitoring and challenging; (c) de-arousal strategies and pleasant activity planning; (d) graded exposure and behavioural activation; and (e) relapse prevention. This course is completed over ~ 8 weeks and complemented with once-weekly therapist emails or telephone calls where therapists answer patients’ ‘ questions about the course and provide guidance on use of skills. Each lesson includes a brief video, educational materials, case stories, homework suggestions (i.e., do-it-yourself guides) and frequently asked questions along with additional reading if patients are interested. Weekly automated email reminders are sent to remind patients’ of the Course [see (16) for details].

Measures

Data were self-reported. Demographic variables (e.g., age, sex, ethnicity, education, marital status, employment status, etc.) were recorded at pre-treatment. Considering vast diversity in ethnocultural backgrounds of patients in the Canadian context (17), ethnicity was assessed using broadly used ethno-racial categories, that is, black, white, Asian (east, middle east, south), Indigenous (First Nations, Inuit, Métis), Latin American, and other (with the option for participants to describe their ethnicity).

Consistent with past research (4, 18), ICBT engagement was assessed using number of patient messages, phone calls, and lessons accessed. Following previous research, for treatment satisfaction, at post-treatment, participants answered yes/no items about whether they would recommend the treatment to a friend, and if the course was worth their time. Participants also rated satisfaction with the program, materials, and confidence to manage symptoms, and motivation to seek future treatment if needed (1 to 5 scale). Patients completed measures of depression [Patient Health Questionnaire-9 [PHQ-9]; (19)] and anxiety [Generalized Anxiety Disorder-7 [GAD-7]; (20)] at pre-treatment (baseline), each week for seven weeks, and post-treatment (end of week 8). Scores ≥10 in these measures indicate clinically significant symptoms of depression or generalized anxiety. Cronbach’s α in this study was between.66 and.91, and.81 and.93 for PHQ-9 and GAD-7, respectively.

The post-treatment qualitative interviews included 12 open-ended questions that asked about the patient’s experience with the program, including perceived cultural relevance of different aspects of the ICBT program, and ways to improve accessibility and utilization of the program for PDEGs [see (12) for details].

Data analysis

Statistical analyses were conducted using IBM SPSS Statistics (Version 28.0). Descriptive statistics described patients’ characteristics. Pilot versus benchmark groups were compared on pre-treatment characteristics, engagement and satisfaction measures using Chi square tests for categorical variables and t-tests for continuous variables. A series of mixed model analyses were run using all data available across the nine weekly assessments to examine changes in the PHQ-9 and GAD-7 outcomes over time and to determine if these changes differed between groups. For each outcome, a series of models involving fixed and random effects of intercept and slope (time) were conducted. The fixed-effect models included time, group, and their interaction (time × group). Intraclass correlation coefficients were used to determine if mixed-model analyses were appropriate (21). Various within-individual covariance structures (e.g., scaled identity, diagonal, autoregressive [AR (1)]) were also tested. The models with smallest Akaike’s Information Criterion and Bayesian Information Criterion were selected for the final analysis. Estimates were calculated using the full information maximum likelihood method. These analyses were conducted twice, 1) using an intention-to-treat (ITT) sample, for which data were imputed using the Multiple Imputation method such that all enrolled participants were included, and 2) using a complete case analysis, for which unimputed data were used. For the ITT analyses, twenty multiply imputed data sets were created (22). Since mixed-model analysis with maximum likelihood method of estimation can handle missing data, imputation prior to analysis was not necessary for complete case analysis (23, 24). Pre- to post-treatment effect sizes (Cohen’s d) were computed using estimated means and standard deviations from the mixed model analysis. Further, we compared the groups on reliable change by calculating the percentage of participants who recovered (GAD-7 or PHQ-9 ≤ 4), improved reliably (GAD-7 or PHQ-9 ≤ 9), or did not show reliable improvement (GAD-7 or PHQ-9 ≥ 10). Pre-treatment standard deviations and test-retest reliability estimates α = .84 for PHQ-9 (19) and α = .83 for GAD-7 (20) were used to compute reliable change index (25).

The digitally recorded post-treatment qualitative interviews were transcribed verbatim, coded by combining deductive and inductive approaches, and organised according to themes for thematic analysis (26). The frequency of each code was recorded for descriptive analysis. The semi-structured interview data were coded by a researcher (EV) using NVivo (version 12 Plus) software.

Results

A total of 41 participants from the pilot study (see Figure 1 for the participant flow chart) and 134 participants from the benchmarking study were included (see Table 1 for pre-treatment demographic and clinical characteristics for the groups). No group differences were found (see Table 1). Participants were mostly Indigenous (53.1%), women (74.9%), married (50.3%), with an average age of 32.9 years (SD = 10.6 years) residing in large cities (64%).

Table 1

Table 1 Patients’ pre-treatment characteristics by group.

For both groups, there were no missing data across pre-treatment demographic and clinical variables but there were 55 (41.0%) and 17 (41.5%) missing values in the benchmark and pilot samples, respectively, at post-treatment. In both groups, data were missing mainly due to dropout. There were no statistically significant differences in pre-treatment demographics, and anxiety and depression symptom severity between the dropouts and completers, therefore the data were assumed to be missing at random (27).

Engagement

There were no significant differences in attrition between pilot (41.5%) and benchmark (41.0%) groups on post-treatment outcome variables (p = .55) or mean number of lessons completed (pilot: 3.78 (SD = 1.79) vs benchmark: 3.99 (SD = 1.74), p = .47). Also, the groups did not significantly differ on the mean number of messages and phone calls to therapists (see Table 2).

Table 2

Table 2 Engagement, treatment satisfaction and negative effects by group.

Satisfaction

Although slightly higher percentage of pilot study participants reported satisfaction in most of the variables, there was no statistically significant differences between pilot and benchmark groups on satisfaction variables (see Table 2).

Effectiveness

As expected, ITT analysis yielded a non-significant interaction effect for the PHQ-9 (β₁ = − 0.01, 95% CI = [− 0.28, 0.21], p = .943) and the GAD-7 (β₁ = − 0.13, 95% CI = [− 0.35, 0.09], p = .261). Similar results were obtained for the complete case analysis of the PHQ-9 (β₁ = − 0.03, 95% CI = [− 0.29, 0.34], p = .866) and GAD-7 (β₁ = -0.09, 95% CI = [− 0.41, 0.22], p = .553), suggesting that the change in self-reported depression and generalized anxiety symptoms over 9 weeks was not significantly different between groups (see Figure 2). For both groups, significant reductions on the PHQ-9 (β₁ = − 0.82, 95% CI = [− 1.01, − 0.64], p < .001) and GAD-7 (β₁ = − 0.79, 95% CI = [− 0.94, − 0.65], p < .001) were observed over time in ITT analyses as well as in the complete case analyses, PHQ-9 (β₁ = − 0.91, 95% CI = [− 1.06, − 0.57], p < .001) and GAD-7 (β₁ = − 0.80, 95% CI = [− 0.95, − 0.64], p < .001). Final models used AR (1) within-individual covariance structure. Similar large pre-to-post within-group effect sizes (Cohen’s d) were found for the PHQ-9 in the pilot (d = 1.23, 95% CI [0.68, 1.77]) and benchmark (d = 1.18, 95% CI [0.88, 1.48]) groups and for the GAD-7 pilot (d = 1.24, 95% CI [0.69, 1.79]) and benchmark (d = 1.27, 95% CI [0.97, 1.58]) groups.

Figure 2

Figure 2 Weekly change in PHQ-9 and GAD-7 mean scores.

Clinically significant change

On the PHQ-9 at post-treatment, for the pilot group (n=24), 10 (41.7%), 4 (16.7%), and 10 (41.7%) met criteria for recovered, reliable improvement, and did not improve reliably, respectively. In the benchmark group (n=79), 27 (34.2%), 19 (24.1%), and 33 (41.8%), met criteria for recovered, reliable improvement, and did not improve reliably, respectively, on the PHQ-9. Similarly, on the GAD-7, in the pilot group (n=24), 8 (33.3%), 6 (25.0%), and 10 (41.7%) met criteria for recovered, reliable improvement, and did not improve reliably, respectively. In the benchmark group (n=79), 28 (35.4%), 20 (25.3%), and 31 (39.2%) met criteria for recovered, reliable improvement, and did not improve reliably, respectively.

Negative effects

One (2.4%) pilot study participant and 11 (8.2%) benchmark study participants self-reported negative effects during the treatment period. In both groups, negative effects were related to transient increases in symptoms during ICBT, such as, when confronting bad memories, when challenging thoughts, or thinking about symptoms.

Post-treatment interviews

Thirteen self-identified PDEG participants representing Indigenous (53.8%), Asian (23.1%) and Black (23.1%) communities were interviewed. There were equal number of self-identified men and women (n=6, 46.2% each) and one (7.7%) transgender participants, with an average age of 31 years (SD = 11.4 years).

A majority of the interview participants (n=12, 92.3%) indicated that the ICBT program met or exceeded their expectations and reported having positive experiences with the therapists. Likewise, in terms of cultural fit of the ICBT program, a majority (n=10, 76.9%) said that the program aligns with their cultural beliefs and practices. When asked for suggestions for specific improvements to make the program better for people from their ethnocultural background, seven (53.8%) participants mentioned that there was no need for change in the program. The remaining (n=6, 46.2%) participants provided some suggestions for further improvements: making the public more aware about this freely available program (n=2, 15.4%), adding culture-specific content (n=2, 15.4%), diversifying the cultural background of the main characters represented in the case stories (n=2, 15.4%), language translation (n=1, 7.7%), and increasing therapist support (n=1, 7.7%).

Discussion

We examined the treatment engagement, satisfaction, and outcomes of an ICBT program adapted for PDEGs benchmarked against a standard (non-adapted) ICBT program. The results showed that there were no statistically significant differences in the engagement, satisfaction, and outcomes between the pilot and benchmark samples, which may reflect the sample size and comparison of the adapted ICBT program to an already established and effective ICBT program. It is unclear at this point whether further adaptations to the ICBT program would serve to enhance the engagement, satisfaction, and outcomes above and beyond the standard ICBT program. Interviews revealed patients were largely satisfied with the adapted program and there was little consensus on suggestions for improvement suggesting further adaptations may not result in significant benefit and the current outcomes reflect the maximal benefits with this line of treatment.

While statistically significant differences were not observed, some non-measured benefits have yet to be examined, including whether offering this adapted ICBT course will improve uptake of the course by PDEGs. Furthermore, although this study did not provide strong evidence that patient-oriented adaptation of the ICBT improves clinical outcomes, engaging PDEG patients and other groups who work with PDEGS has provided some assurance that the adapted ICBT program is culturally relevant and clinically significant in reducing symptoms of anxiety and depression across diverse ethnocultural groups. In the future, we intend to explore whether there is an increased uptake of ICBT by PDEGs. However, we acknowledge that underutilization of ICBT could reflect increased resilience, differences in help-seeking behaviour, and that PDEGs may prefer alternative sources of support.

Limitations

This study had several limitations. We could not conduct more detailed ethnocultural subgroup analyses (e.g., Asian group includes many diverse ethnicities and cultures) due to small sample size in our pilot study and the lack of data on place of birth, immigration history, level of acculturation etc., past research (e.g., 28, 29) shows that younger adults tend to engage less and drop-out more from the ICBT program. Further, there is some evidence that younger immigrants likely acculturate faster (see 30, 31) and more acculturated people are less likely to benefit from culturally adapted interventions than less acculturated people (32). In the current study, the mean age of the participants was 31.6 (SD=10.9) (see Table 1), which was lower than in our previous studies (Mean age = 38.14, SD = 11.9 years) (see 4). Further, consistent with the literature, engagement was measured using number of lessons completed and email and/or telephone exchange with therapists. However, although these measures are useful in assessing behavioural aspects of engagement, they do not necessarily reflect the level of engagement (e.g., effort put into reviewing the lessons, practicing skills, continuity in use of the learned skills) with the program. All these factors may have influenced that rate of engagement and effectiveness of the adapted intervention in the current study. Further studies should consider involving more diverse range of age groups and more nuanced measures of engagement (see 33).

Within a setting that includes PDEGs, limited organizational resources do not allow for development of ICBT for each ethnocultural group accessing services. Nevertheless, adaptation to specific larger groups could be helpful such as Indigenous peoples using Indigenous research methods as has been done in Australia (see 34). Meta-analytic evidence suggests that interventions targeted to specific ethnocultural group are more effective than interventions adapted for multicultural settings (32). In the case of internet- and mobile-based interventions for mental health disorders, however, there is a dearth of research that directly compares adapted and non-adapted interventions (35) and more research is needed to understand the benefits of adaptation and which approaches are most effective. Of note, non-adapted ICBT interventions have also been demonstrated to produce equivalent outcomes in migrants and non-migrants in Australia (36) and elsewhere (see 37). We opted for a patient-oriented adaptation as it was not feasible (based on available resources) for us to conduct a comprehensive cultural adaptation of the ICBT program considering ICBT is being implemented in a multicultural routine care setting in Saskatchewan. Therefore, a thought for future studies remains, that is, as the existing research suggests (e.g., 38), perhaps we would obtain better engagement, satisfaction, and treatment outcomes had the ICBT program been culturally adapted and tailored to specific ethnocultural groups. Therefore, the findings of this study are preliminary and should not diminish the importance of need-informed cultural adaption, especially targeting a specific ethnocultural group in the context of digital mental health. Adaptation of therapeutic programs is indeed a complex issue that calls for careful consideration. As noted in a recent review of the literature on the effectiveness of adapted psychological interventions, various types of cultural adaptations (e.g., adaptation of content, method of delivery, training of therapists, etc.) may have different effects on outcomes. Additionally, the review suggests that some ethnocultural groups may benefit more from cultural adaptation than others [e.g., South Asians may benefit more than East Asians; (39)].

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

All procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The studies involving humans were approved by the Institutional Review Board of the University of Regina (benchmark study: REB # 2019-197, December 12, 2019; pilot trial: REB # 2022-012, March 17, 2022). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

RS: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing. EV: Investigation, Project administration, Writing – original draft, Writing – review & editing. BD: Investigation, Validation, Writing – review & editing. NT: Investigation, Validation, Writing – review & editing. HH: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The Saskatchewan Health Research Foundation and Mental Health Research Canada funded the research. The Saskatchewan Ministry of Health funds the Online Therapy Unit to deliver Internet-delivered cognitive behaviour therapy to residents of Saskatchewan.

Acknowledgments

We would like to thank the patients and community stakeholders associated with the project. We would specifically like to acknowledge our working group members, including Vanessa Heron, Lee Bourgeault, Basmah Almosallem, Vibya Natana, Micha Kasongo, Belinda Owusu Nyamike, Arjun Adhikari for their inputs and guidance in research and the Course modification, and all therapists, Kelly Adlam, Aaron Ingrouville, Janet Tzupa, MacKenzie-Martin-Proskie, Katherine-Owens, who provided ICBT to patients in this trial. The authors also gratefully acknowledge Marcie Nugent the Online Therapy Unit Operations Director and University of Regina Research IT Support.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. World Health Organization. Depression and other common mental disorders: global health estimates. Geneva: World Health Organization (2017).

Google Scholar

2. Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, et al. ICBT in routine care: A descriptive analysis of successful clinics in five countries. Internet Interv. (2018) 13:108–15. doi: 10.1016/j.invent.2018.07.006

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Hedman-Lagerlöf E, Carlbring P, Svärdman F, Riper H, Cuijpers P, Andersson G. Therapist-supported Internet-based cognitive behaviour therapy yields similar effects as face-to-face therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. World Psychiatry. (2023) 22:305–14. doi: 10.1002/wps.21088

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Hadjistavropoulos H, Peynenburg V, Thiessen DL, Nugent M, Karin E, Staples L, et al. Utilization, patient characteristics, and longitudinal improvements among patients from a provincially funded transdiagnostic internet-delivered cognitive behavioural therapy program: observational study of trends over 6 years. Can J Psychiatry. (2022) 67:192–206. doi: 10.1177/07067437211006873

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Cook BL, Trinh N-H, Li Z, Hou SS-Y, Progovac AM. Trends in racial-ethnic disparities in access to mental health care, 2004–2012. Psychiatr Serv. (2017) 68:9–16. doi: 10.1176/appi.ps.201500453

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Polo AJ, Makol BA, Castro AS, Colón-Quintana N, Wagstaff AE, Guo S. Diversity in randomized clinical trials of depression: A 36-year review. Clin Psychol Rev. (2019) 67:22–35. doi: 10.1016/j.cpr.2018.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

7. De Jesús-Romero R, Holder-Dixon A, Lorenzo-Luaces L. Reporting and representation of racial and ethnic diversity in randomized controlled trials of internet-based cognitive-behavioral therapy (iCBT) for depression. PsyArXiv Pre-print. (2022). doi: 10.31234/osf.io/kfnhm

CrossRef Full Text | Google Scholar

8. Bhugra D, Gupta S, Bhui K, Craig T, Dogra N, Ingleby JD, et al. WPA guidance on mental health and mental health care in migrants. World Psychiatry. (2011) 10:2–10. doi: 10.1002/j.2051-5545.2011.tb00002.x

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Lewis-Fernández R, Kirmayer LJ. Cultural concepts of distress and psychiatric disorders: Understanding symptom experience and expression in context. Transcult Psychiatry. (2019) 56:786–803. doi: 10.1177/1363461519861795

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Torous J, Benson NM, Myrick K, Eysenbach G. Focusing on digital research priorities for advancing the access and quality of mental health. JMIR Ment Health. (2023) 10:e47898. doi: 10.2196/47898

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Canadian Institutes of Health Research. Strategy for patient-oriented research: patient engagement framework. (2014). Canadian Institutes of Health Research. Available online at: https://cihr-irsc.gc.ca/e/48413.html (Accessed 10 August 2022).

Google Scholar

12. Sapkota RP, Valli E, Wilhelms A, Adlam K, Bourgeault L, Heron V, et al. Patient-oriented research to improve internet-delivered cognitive behavioural therapy for people of diverse ethnocultural groups in routine practice. Healthcare. (2023) 11:2135. doi: 10.3390/healthcare11152135

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Lau AS. Making the case for selective and directed cultural adaptations of evidence-based treatments: Examples from parent training. Clin Psychol: Sci Pract. (2006) 13:295–310. doi: 10.1111/j.1468-2850.2006.00042.x

CrossRef Full Text | Google Scholar

14. McCabe K, Yeh M, Lau A, Argote CB. Parent-child interaction therapy for Mexican Americans: Results of a pilot randomized clinical trial at follow-up. Behav Ther. (2012) 43:606–18. doi: 10.1016/j.beth.2011.11.001

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Peynenburg V, Sapkota RP, Lozinski T, Sundström C, Wilhelms A, Titov N, et al. The impacts of a psychoeducational alcohol resource during internet-delivered cognitive behavioral therapy for depression and anxiety: observational study. JMIR Ment Health. (2023) 10:e44722. doi: 10.2196/44722

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Titov N, Dear BF, Staples LG, Bennett-Levy J, Klein B, Rapee RM, et al. MindSpot clinic: an accessible, efficient, and effective online treatment service for anxiety and depression. Psychiatr Serv. (2015) 66:1043–50. doi: 10.1176/appi.ps.201400477

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Statistics Canada. The Canadian census: A rich portrait of the country's religious and ethnocultural diversity (2022). Statistics Canada. Available online at: https://www150.statcan.gc.ca/n1/en/daily-quotidien/221026/dq221026b-eng.pdf?st=v070_cQJ (Accessed 3 January 2023).

Google Scholar

18. Titov N, Dear BF, Staples LG, Terides MD, Karin E, Sheehan J, et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: A randomized controlled trial. J Anxiety Disord. (2015) 35:88–102. doi: 10.1016/j.janxdis.2015.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. (2006) 166:1092–7. doi: 10.1001/archinte.166.10.1092

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Peugh JL, Enders CK. Using the SPSS mixed procedure to fit cross-sectional and longitudinal multilevel models. Educ Psychol Meas. (2005) 65:717–41. doi: 10.1177/0013164405278558

CrossRef Full Text | Google Scholar

22. Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev Sci. (2007) 8:206–13. doi: 10.1007/s11121-007-0070-9

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Twisk J, de Boer M, de Vente W, Heymans M. Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis. J Clin Epidemiol. (2013) 66:1022–8. doi: 10.1016/j.jclinepi.2013.03.017

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Shin T, Davison ML, Long JD. Maximum likelihood versus multiple imputation for missing data in small longitudinal samples with nonnormality. Psychol Methods. (2017) 22:426–49. doi: 10.1037/met0000094

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. (1991) 59:12–9. doi: 10.1037/10109-042

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. (2008) 62:107–15. doi: 10.1111/j.1365-2648.2007.04569.x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Little RJ, Rubin DB. Statistical analysis with missing data. Hoboken, NJ: John Wiley and Sons (2002). doi: 10.1002/SERIES1345

CrossRef Full Text | Google Scholar

28. Sapkota RP, Peynenburg V, Dear BF, Titov N, Hadjistavropoulos HD. Engagement with homework in an Internet-delivered therapy predicts reduced anxiety and depression symptoms: A latent growth curve analysis. J Consult Clin Psychol. (2023) 91:112–7. doi: 10.1037/ccp0000775

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Edmonds M, Hadjistavropoulos H, Schneider L, Dear B, Titov N. Who benefits most from therapist-assisted internet-delivered cognitive behaviour therapy in clinical practice? Predictors of symptom change and dropout. J Anxiety Disord. (2018) 54:24–32. doi: 10.1016/j.janxdis.2018.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Cheung BY, Chudek M, Heine SJ. Evidence for a sensitive period for acculturation: Younger immigrants report acculturating at a faster rate. Psychol Sci. (2011) 22:147–52. doi: 10.1177/09567976103946

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Dow HD. The acculturation processes: The strategies and factors affecting the degree of acculturation. Home Health Care Manag Pract. (2011) 23:221–7. doi: 10.1177/1084822310390877

CrossRef Full Text | Google Scholar

32. Griner D, Smith TB. Culturally adapted mental health intervention: A meta-analytic review. Psychother: Theory Res Pract Training. (2006) 43:531. doi: 10.1037/0033-3204.43.4.531

CrossRef Full Text | Google Scholar

33. Torous J, Michalak EE, O’Brien HL. Digital health and engagement—looking behind the measures and methods. JAMA network Open. (2020) 3:e2010918–e. doi: 10.1001/jamanetworkopen.2020.1091

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Titov N, Schofield C, Staples L, Dear BF, Nielssen O. A comparison of Indigenous and non-Indigenous users of MindSpot: an Australian digital mental health service. Australas Psychiatry. (2019) 27:352–7. doi: 10.1177/1039856218789784

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Spanhel K, Balci S, Feldhahn F, Bengel J, Baumeister H, Sander LB. Cultural adaptation of internet-and mobile-based interventions for mental disorders: a systematic review. NPJ Digit Med. (2021) 4:128. doi: 10.1038/s41746-021-00498-1

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Kayrouz R, Karin E, Staples LG, Nielssen O, Dear BF, Titov N. A comparison of the characteristics and treatment outcomes of migrant and Australian-born users of a national digital mental health service. BMC Psychiatry. (2020) 20:1–13. doi: 10.1186/s12888-020-02486-3

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Hernandez MEH, Waller G, Hardy G. Cultural adaptations of cognitive behavioural therapy for Latin American patients: unexpected findings from a systematic review. Cognit Behav Ther. (2020) 13:e57. doi: 10.1017/S1754470X20000574

CrossRef Full Text | Google Scholar

38. Ellis DM, Draheim AA, Anderson PL. Culturally adapted digital mental health interventions for ethnic/racial minorities: A systematic review and meta-analysis. J Consult Clin Psychol. (2022) 90:717–33. doi: 10.1037/ccp0000759

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Arundell L-L, Barnett P, Buckman JE, Saunders R, Pilling S. The effectiveness of adapted psychological interventions for people from ethnic minority groups: A systematic review and conceptual typology. Clin Psychol Rev. (2021) 88:102063. doi: 10.1016/j.cpr.2021.102063

PubMed Abstract | CrossRef Full Text | Google Scholar