Age, Motivation, and Emotion Regulation Skills Predict Treatment Outcome in an Internet-Based Self-Help Intervention for COVID-19 Related Psychological Distress

Introduction: First evidence suggests that internet-based self-help interventions effectively reduce COVID-19 related psychological distress. However, it is yet unclear which participant characteristics are associated with better treatment outcomes. Therefore, we conducted secondary analyses on data from a randomized controlled trial investigating the efficacy of a 3-week internet-based self-help intervention for COVID-19 related psychological distress. In this exploratory analysis, we examined several predictors ranging from sociodemographic variables to psychological distress, resource-related, and treatment-related variables. This includes, for example, age, motivation, and emotion regulation skills. Treatment outcomes were defined as post-treatment depressive symptoms and post-treatment resilience.

Methods: In a total of 107 participants with at least mild depressive symptoms, possible predictor variables and treatment outcomes were assessed using self-report measures. For example, emotion regulation skills were assessed by the Self-report measure for the assessment of emotion regulation skills. In a first step, we performed a separate linear regression analysis for each potential predictor. In a second step, predictors meeting a significant threshold of p < 0.05 were entered in linear multiple regression models. Baseline scores of the respective outcome measure were controlled for.

Results: The mean age of the participants was 40.36 years (SD = 14.59, range = 18–81 years) with the majority being female (n = 87, 81.3%). Younger age predicted lower post-treatment depressive symptoms. Additionally, higher motivation to use the intervention and better pre-treatment emotion regulation skills predicted higher post-treatment resilience.

Conclusion: The current study provides preliminary evidence regarding the relationship between participant characteristics and treatment outcome in internet-based self-help interventions for COVID-19 related distress. Our results suggest that under the circumstances surrounding COVID-19 such interventions might be particularly beneficial for young adults regarding depressive symptoms. Moreover, focusing on participants' existing strengths might be a promising approach to promote resilience through internet-based self-help interventions. However, since this was an exploratory analysis in an uncontrolled setting, further studies are needed to draw firm conclusions about the relationship of participant characteristics and treatment outcome in internet-based self-help interventions for COVID-19 related psychological distress.

Introduction

In March 2020, the World Health Organization (WHO) declared the COVID-19 (acute respiratory syndrome coronavirus 2; SARS-CoV-2) outbreak a pandemic (1). At the onset and during the COVID-19 pandemic, studies indicated a deterioration of mental health in the general population (2–6). In particular, evidence for an increase in depression and anxiety symptoms was found (7). For example, in a study in the USA, a tripling of the prevalence of depression symptoms in the general population during the COVID-19 pandemic was reported (8). However, some studies reported that the COVID-19 pandemic and associated restrictions had no impact or even a positive impact on the wellbeing of the general population (9, 10). For example, Aghababa et al. (10) found stable activity patterns among team sport athletes and Albrecht et al. (9) report positive effects of homeschooling on adolescent sleep schedules. Moreover, some studies indicated that the initial increase in psychological distress in the general population decreased over the course of the COVID-19 pandemic (11–13). These findings suggest that on average, the general population might be resilient. Resilience can be defined as the maintenance of stable mental health or recovery from initial psychological distress in the face of major life stressors (14). Nonetheless, a substantial minority of the general population shows heightened psychological distress (15, 16). Accordingly, mental health interventions mitigating this psychological distress are needed.

A promising approach is the use of internet-based self-help interventions since they do not require direct on-site contact and are easily scalable (17–19). Studies indicate that internet-based self-help interventions are an effective treatment option for various psychological problems, including depressive symptoms (20, 21). So far, few studies have investigated the efficacy of internet-based self-help interventions for COVID-19 related psychological distress in the general population. However, first results suggest that internet-based self-help interventions are efficacious in reducing COVID-19 related worry and associated symptoms (22), symptoms of depression, anxiety, and stress (23, 24), as well as in promoting resilience and emotion regulation skills (25). Nonetheless, in one study, there was no significant reduction of depressive symptoms (25). Given that there is still comparatively little research on internet-based self-help interventions for COVID-19 related psychological distress and that it shows mixed results, it is important to find out who might benefit from internet-based self-help for COVID-19 related psychological distress.

In the context of the COVID-19 pandemic, understanding the relationship between participant characteristics and treatment outcome is of particular interest since some studies point toward the need for tailoring interventions for specific risk populations (5, 7). Identifying predictors of treatment outcomes in internet-based self-help for COVID-19 related psychological distress would allow interventions to be tailored to specific needs and thus improve intervention efficacy. Accordingly, knowledge of the predictors of treatment outcomes would inform how interventions could be improved for specific use in target populations or adapted for other target populations. For example, if age predicts treatment outcomes, interventions could be tailored and improved for specific age groups or adapted for those not yet reached. So far, potential risk factors for heightened psychological distress due to the COVID-19 pandemic include for example: pre-existing mental health problems (26–28), pre-existing physical health problems (27), younger age (29–32), identifying as non-binary (27), female gender (27–30), and difficulties in emotion regulation (33, 34).

However, no study to date has investigated predictors of treatment outcome in internet-based self-help interventions for COVID-19 related psychological distress. Therefore, to improve the understanding of the relationship between participant characteristics and treatment outcome in internet-based self-help for COVID-19 related psychological distress, we explored predictors of treatment outcome in an internet-based self-help intervention for COVID-19 related psychological distress called ROCO (25, 35). ROCO is an acronym for Resilience and Optimism during COVID-19. The 3-week ROCO intervention included guidance on demand and the participants had the possibility to contact a psychologist via a chat function. The efficacy of the ROCO intervention was evaluated in a randomized controlled trial, from which the data used in this study are drawn (25). The primary target of the ROCO intervention was a reduction of depressive symptoms. However, a considerable part of the intervention was also aimed at promoting resilience (35). Therefore, in the present study, we defined treatment outcomes as post-treatment depressive symptoms and post-treatment resilience. In the RCT, the 3-week ROCO intervention did not significantly reduce primary depressive symptoms and secondary outcomes such as anxiety and stress symptoms. Moreover, the intervention had no beneficial effects on secondary outcomes such as quality of life, optimism, embitterment, loneliness, and self-efficacy. However, the intervention led to a significant increase in emotion regulation skills and resilience (small-to-medium effect sizes).

The aim of this exploratory analysis is to investigate possible predictors of treatment outcome in an internet-based self-help intervention for COVID-19 related psychological distress. Specifically, we aimed to explore, who improves more during treatment. Based on the above mentioned previous research on possible risk factors for COVID-19 related psychological distress, we decided to explore sociodemographic variables (age, gender, and level of education), psychological distress variables (ever having received a psychiatric diagnosis, previous or current psychotherapy, current medication, anxiety, stress, embitterment, loneliness, and mental and physical health quality), and resource-related variables (emotion regulation skills, optimism, and self-efficacy) as possible predictors. Moreover, we explored if treatment-related variables (motivation to use the self-help intervention, number of completed modules) predict treatment outcome.

Materials and Methods

Study Design

The data used in the current study were obtained in a parallel-group randomized controlled trial (RCT) investigating the efficacy of a short internet-based self-help intervention for COVID-19 related psychological distress called ROCO. The protocol of the study was approved by the Ethics Committee of the Canton of Bern, and the trial was registered on ClinicalTrials.gov (NCT04380909).

In the RCT, an immediate treatment group was compared to a waiting control group, with both groups receiving care as usual [CAU; (25, 35)]. Participants were randomized in a 1:1 ratio to either the immediate treatment group or the waiting control group. Participants in the immediate treatment group received direct access to the 3-week internet-based ROCO intervention, whereas participants in the waiting control group had a waiting period of 3 weeks and then received access to the ROCO intervention (i.e., delayed treatment). Three weeks after randomization, all participants had to fill out a second assessment (post-treatment for the immediate treatment group; pre-treatment for the waiting control group). All participants had to complete a third assessment 6 weeks after the randomization (follow-up for the immediate treatment group; post-treatment for the waiting control group). In the RCT, analyses were conducted according to an intention-to-treat principle (25).

For the present secondary analysis, data from both groups were combined, using the data of the respective treatment phase (immediate or delayed). The investigated predictors of post-treatment outcomes (depressive symptoms and resilience, respectively) were assessed before the respective treatment phase (i.e., for the immediate treatment group at baseline and for the waiting control group after the waiting period). Sociodemographic variables as well as information on previous or current psychological treatments (ever received a psychiatric diagnosis, prior experience with psychotherapy, current psychotherapy, or medication intake) were collected for both groups at baseline.

Participants

We recruited German-speaking participants between April 2020 and February 2021, primarily through newspaper articles, internet forums on mental health, and advertisements on the internet. Interested participants registered on our study homepage and subsequently received the detailed study information. After providing informed consent, participants completed the online baseline assessment, consisting of questions concerning socio-demographic variables, previous or current psychological treatment, and various self-report questionnaires.

The following inclusion and exclusion criteria were evaluated based on this baseline assessment: participants had to be at least 18 years of age, have access to the internet, show sufficient knowledge of the German language, provide an emergency address for the case of an acute crisis, and reach a minimum of 4 points on the Patient Health Questionnaire [PHQ; (36)], which is interpreted as the presence of mild depressive symptoms. Participants were excluded if they reached a cut-off value of 8 points on the Suicide Behavior Questionnaire [SBQ-R; (37)], which would indicate the presence of suicidal tendencies. Furthermore, participants reporting a known psychotic or bipolar disorder diagnosis were also excluded. A total of 107 participants met all the inclusion criteria and none of the exclusion criteria, thus constituting the current study sample.

Measures

All assessments were administered online and consisted of self-report questionnaires. We used the German versions of the self-report questionnaires.

Outcome Measures

Depressive symptoms, the primary treatment target of the internet-based intervention, were measured with the PHQ-9 (36). The PHQ-9 is used to assess the severity of depressive symptoms. For this purpose, nine items are scored on a scale from 0 = not at all to 5 = nearly every day. The items are introduced as follows: “Over the last 2 weeks, how often have you been bothered by any of the following problems?” Examples of items are: “Little interest or pleasure in doing things” and “Feeling down, depressed, or hopeless.” The nine items correspond to the nine DSM-IV criteria for depression. From the nine items, a score is built: a score of 5 corresponds to mild depression, a score of 10 to moderate depression, a score of 15 to moderately severe depression, and a score of 20 to severe depression (38). In the present sample, the PHQ-9 had a satisfactory internal consistency (Cronbach's α = 0.72 at pre-treatment and Cronbach's α = 0.74 at post-treatment).

A secondary treatment target of the internet-based intervention was to promote resilience. Resilience was measured with the Connor-Davidson Resilience Scale [CD-RISC; (39)]. In the present study, the 10-item version of the CD-RISC was used. Examples of items are: “I am able to adapt to change” and “I can handle unpleasant feelings.” The 10 items are answered on a scale from 0 = not true at all to 4 = true nearly all of the time. Higher scores correspond to more resilience. In the present sample, the CD-RISC showed good internal consistency (Cronbach's α = 0.88 at pre-treatment and Cronbach's α = 0.90 at post-treatment).

Predictors

We grouped a total of 18 possible predictor variables into four groups: sociodemographic, psychological distress, resource-related, and treatment-related variables.

Sociodemographic Variables. We assessed age, gender, and level of education.

Psychological Distress Variables

At baseline, we assessed whether participants had ever received a psychiatric diagnosis, had previously been in psychotherapy, were currently in psychotherapy, and were currently taking medication for mental health problems. These variables were chosen as measures of pre-existing mental health problems and current psychological treatment needs, indicative of psychological burden (13, 26).

At pre-treatment, we assessed several variables using self-report questionnaires. Anxiety and stress were measured by the corresponding subscales of the DASS-21 (40). Each subscale consists of seven items, which are answered on a scale from 0 = did not apply to me at all to 3 = applied to me very much or most of the time. Examples of items are: “I found it hard to wind down,” “I felt I was close to panic,” and “I was unable to become enthusiastic about anything.” On the anxiety subscale, a score of 4 represents mild anxiety, a score of 6 moderate anxiety, a score of 8 severe anxiety, and a score of 10 extremely severe anxiety. On the stress subscale, a score of 8 represents mild stress, a score of 10 moderate stress, a score of 13 severe stress, and a score of 17 extremely severe (41). In the present sample, the internal consistency at pre-treatment was close to satisfactory for the anxiety subscale (Cronbach's α = 0.68) and good for the stress subscale (Cronbach's α = 0.81).

Mental health quality and physical health quality were assessed as measures of general health-related quality of life with the respective scales of the 12-Item Short-Form Health Survey (27, 42). An example of an item is: In general, would you say your health is: Excellent, Very Good, Good, Fair, or Poor. Higher scores on the respective subscale indicate better mental health quality, respectively physical health quality. The SF-12 has a good test-retest reliability (43).

Embitterment was measured with the 6-item version of the Bern Embitterment Inventory (44). Embitterment can be defined as the feeling of being disadvantaged by others and fate and might be a mental health reaction to the COVID-19 pandemic (45–47). Examples of items are: “It fills me with bitterness when I think of the goals that have not been achieved” and “I sometimes think that all people are bad and corrupt.” Items are scored on a scale from 0 = I do not agree to 4 = I agree, with higher scores representing more embitterment (48). In the present sample, the internal consistency of the BEI at pre-treatment was good (Cronbach's α = 0.80).

Loneliness was assessed using the 9-item version of the UCLA Loneliness Scale [ULS; (49)] since several studies postulated a link between loneliness and mental health problems and the COVID-19 pandemic has been reported to increase loneliness (50, 51). Examples of items are: “How often do you feel that there are people you can talk to?” and “How often do you feel isolated from others?.” The items are answered on a scale from 1 = never to 4 = often, with higher scores indicating more loneliness. In the present sample, the internal consistency of the ULS at pre-treatment was good (Cronbach's α = 0.85).

Resource-Related Variables

We assessed several resource-related variables using self-report questionnaires at pre-treatment. Self-efficacy Was assessed using the General Self-Efficacy Scale [GSE; (52)]. The 10 items Are scored on a scale From 1 = Not at all true to 4 = exactly true, With higher scores indicating more self-efficacy (52). Examples of items Are: “It Is easy for me to stick to my aims and accomplish my goals” and “I can solve most problems if I invest the necessary effort.” In the present sample, the internal consistency of the GSE at pre-treatment was good (Cronbach's α = 0.89).

Optimism was assessed with the Life Orientation Test Revised [LOT-R; (53)]. The total score of the 10-item LOT-R is built from six items, since four items are filler items. A higher score indicates more optimism. Examples of items are: “In uncertain times, I usually expect the best” and “I rarely count on good things happening to me.” The items are answered on a scale from 0 = strongly disagree to 4 = strongly agree (53). In the present sample, the internal consistency of the LOT-R at pre-treatment was good (Cronbach's α = 0.84).

Emotion regulation skills were assessed with the Self-report measure for the assessment of emotion regulation skills [SEK-27; (54)]. The 27 items of the SEK-27 are answered on a scale from 0 = never to 4 = (almost) always, with higher scores corresponding to better emotion regulation skills (54). The items are introduced as follows: “Dealing with emotions: Last week....” Examples of items then are: “I understood my emotional reactions,” “I was OK with my feelings, even if they were negative,” and “I supported myself in emotional distressing situations.” In the present sample, the internal consistency for the SEK-27 at pre-treatment was excellent (Cronbach's α = 0.93).

Treatment-Related Variables

Motivation to use the internet-based intervention was assessed at baseline with one item (Please indicate your motivation to use the ROCO program in general). Participants could rate their motivation with a regulator From 0 = no motivation at all to 100 = greatest possible motivation.

The number of completed modules was measured after the treatment. It could range from 0 (no module completed) to 6 (all modules completed).

Description of Intervention

The internet-based self-help intervention ROCO was aimed at persons experiencing COVID-19 related psychological distress. The 3-week intervention consisted of six thematic modules, an introduction, and a conclusion. Additionally, the intervention comprised a page with information on what to do in an acute crisis, including a list of emergency contacts, a page with an overview of the weekly exercises, and a page with a symptom-tracking questionnaire, allowing participants to track their self-reported symptoms. The six thematic modules were based on cognitive-behavioral therapy and included texts, videos, graphics, and exercises. Each thematic module had a specific focus: in module 1, psychoeducation about COVID-19 related psychological distress was given. In module 2, participants learned about emotions and emotion regulation. In module 3, the identification and restructuring of thought patterns were addressed. In module 4, participants acquired knowledge about several possibilities to promote resilience. In module 5, relaxation techniques and sleep hygiene were discussed. Finally, in module 6, the topics of self-care and personal growth were approached. For a more detailed description of the intervention, see the study protocol of the ROCO RCT (35). Participants were advised to work through two modules per week. However, the participants could decide for themselves which modules they wanted to work on and in which order. A module took about 40 to 80 mins to complete. Since the internet-based self-help program offered guidance on demand, the participants had the possibility to contact a psychologist via a chat function. The psychologist answered within three working days. Otherwise, there was no scheduled contact.

Statistical Analysis

All analyses were conducted using IBM SPSS Statistics (version 25). Independent samples t-tests and χ²-tests (for nominal data) were performed to examine group differences at baseline and pre-treatment. In a first step, potential predictors were identified using simple linear regression analyses. For each potential predictor a separate linear regression analysis was performed as follows: the potential predictor (e.g., age) was entered as predictor, the post-treatment score of the outcome (depressive symptoms or resilience) was entered as dependent variable, and the pre-treatment score of the respective outcome (e.g., depressive symptoms) was defined as covariate. We predetermined, that predictors had to reach a p-value below 0.05 to be included in the subsequent multiple regression analyses. In a second step, a multiple regression analysis was performed for each outcome with the predictors identified in step 1 entered as predictors and the pre-treatment score of the respective outcome entered as covariate. All tests were two-sided. This approach of stepwise selection of predictors has been criticized (e.g., for leading to bias in predictor effects or variability of selected predictors) and modern prediction methods have been recommended (55, 56). Accordingly, stepwise regression procedures are unfavorable for explanatory purposes. However, since stepwise regression procedures might be of value for exploratory data analysis (57), we refrained from using modern prediction methods.

To account for possible group effects, we additionally tested whether group (immediate vs. delayed treatment) was a significant predictor for the outcome while using the pre-treatment values of the respective outcome as covariate. If the group was a significant predictor (p < 0.050), it was added as covariate in the multiple regression analysis of the respective outcome. We did not replace missing data in the predictor variables. Hence only participants with complete data sets were considered for the respective outcomes (completers analysis).

The assumptions for performing multiple linear regressions were checked in advance (58). Our sample size was N = 107 > 100. However, the number of predictors times ten (18 × 10 = 180) was greater than our sample size, possibly leading to overfitting of regression models. Yet even two observations per parameter have been found to not bias the estimate in linear regression analysis (59). The variances inflation factors (VIF) were between 1 and 2.732 whereas a VIF <1 and VIF > 10 indicates multicollinearity and the predictors explained the dependent variables (Rs = 0.545–0.818, R² s = 0.297–0.669). We did not calculate Durbin-Watson coefficients because our dependent variable (respective treatment outcome) was also in the model as an independent variable (pre-treatment score of the respective outcome) with a time lag and thus the application of the Durbin-Watson statistic is not warranted (60). Moreover, we inspected the distribution of the residuals (homoscedasticity and normal distribution) and checked for outliers. We identified one outlier based on the studentized residuals, however Cook's distances indicated that the case was not influential for our models (61). Therefore, we did not remove this outlier from the data (62).

Results

Sample Characteristics

The total sample consisted of 107 German-speaking participants. On average, they were 40.36 years old (SD = 14.59, range = 18–81 years) and the majority were female (n = 87, 81.3%), had a university degree (n = 64, 59.8%) and previous experience with psychological treatment (n = 68, 63.6 %). Overall, 28 participants (26.2%) were in concurrent psychological treatment and 24 participants (22.4%) were taking medication for psychological problems at baseline. The participants showed, on average, moderate depressive symptoms (M = 10.37, SD = 4.18) and mild anxiety and stress symptoms (M = 4.33, SD = 3.26; M = 8.80, SD = 4.10) at pre-treatment. Approximately one third of the participants (n = 36, 33.6 %) reported having received a psychiatric diagnosis at some point in their lives. Baseline or pre-treatment scores of the predictor variables and outcome measures are displayed in Table 1. There was a significant group difference in terms of education (${\text{χ}}_{(1)}^2$ = 5.055, p = 0.025), indicating that participants in the immediate treatment group had a lower average level of education. Moreover, participants in the delayed treatment group completed significantly fewer modules of the intervention than participants in the immediate intervention group (t_(104.1) = 2.719, p = 0.009). Additionally, the delayed treatment group showed markedly lower pre-treatment depression scores compared to the immediate treatment group [immediate treatment group M (SD) = 11.13 (4.36) vs. delayed treatment group M (SD) = 9.60 (3.89)]. However, the group difference was not significant (t_(102.1) = 1.908, p = 0.055).

TABLE 1

Table 1. Predictors and outcome measures at baseline or pre-treatment, overall and divided by group.

Identifying Predictors of Post-treatment Depressive Symptoms and Resilience

In a first step, variables predicting post-treatment depressive symptoms and resilience were identified using simple linear regressions. We controlled for pre-treatment scores of the corresponding outcome measures (depressive symptoms or resilience). The results of the single predictor analysis are displayed in Table 2. In a second step, the variables that met the previously defined threshold of a p-value below 0.05 were included in a multiple regression model (see Tables 3, 4). All models used centered predictor variables (grand mean-centered) to anticipate possible multicollinearity. Since the variable group (immediate vs. delayed treatment) was a significant covariate for resilience (ΔR² = 0.034, β = −0.184, p = 0.013), it was entered in the respective multiple regression.

TABLE 2

Table 2. Single-predictor linear regression analysis with post-treatment depressive symptoms respectively post-treatment resilience as dependent variable controlling for pre-treatment depressive symptoms, respectively pre-treatment resilience.

TABLE 3

Table 3. Predictors of the post-treatment depressive symptoms (multiple regression).

TABLE 4

Table 4. Predictors of the post-treatment resilience (multiple regression).

Predictors of Post-treatment Depressive Symptoms in Multiple Regression

Within the first multiple linear regression, we examined predictors for post-treatment depressive symptoms (see Table 3). The age of the participants at baseline was a significant predictor of post-treatment depressive symptoms [b (SE) = 0.043 (0.020), p = 0.032]. The older the participants were, the higher their depressive symptoms were post-treatment.

Predictors of Post-treatment Resilience in Multiple Regression

Table 4 displays the results of the second multiple linear regression, in which post-treatment resilience was the outcome. Both motivation at baseline [b (SE) = 0.092 (0.032), p = 0.006] and pre-treatment emotion regulation skills [b (SE) = 0.072 (0.036), p = 0.047] predicted post-treatment resilience. The higher the motivation of the participants to use the intervention was, the higher their resilience was post-treatment. Likewise, the better the emotion regulation skills of the participants were pre-treatment, the higher their resilience was post-treatment.

Discussion

In the present study, we aimed to identify predictors of treatment outcome in users of an internet-based self-help intervention for COVID-19-related psychological distress. With regard to depressive symptoms, being younger predicted lower depressive symptoms after the 3-week intervention. With regard to resilience, higher motivation to use the intervention and better emotion regulation skills pre-treatment predicted higher resilience after the 3-week intervention.

We found that higher age was associated with worse treatment outcomes regarding depressive symptoms. This finding is inconsistent with previous research on predictors of internet-based self-help interventions for depression, in which age was not predictive of treatment outcome (63–67). The present finding is not straightforward to explain but could be related to the specific circumstances of the COVID-19 pandemic. A possible explanation could be a differential influence of various COVID-19-related stressors on psychological distress depending on age and that the intervention under study provided better support in dealing with certain stressors. For example, in a sample of 22-year-olds, secondary consequences of the COVID-19 pandemic, such as disruption of lifestyle or economic disruption were more strongly associated with psychological distress than COVID-19-related health risk exposures (68). Moreover, in one study, avoidant coping moderated the relationship between COVID-19 related psychological distress and depressive symptoms more strongly in younger adults compared to older adults (69). Therefore, in an uncontrolled setting, younger adults might benefit more from an intervention fostering adaptive coping than older adults. Given that research increasingly suggests that young adults are particularly affected mentally by the COVID-19 pandemic, the present finding is promising, despite the difficult explanation (29–32). However, since the ROCO intervention did not significantly reduce depressive symptoms and the present study was uncontrolled, such improvements could also be observed in younger adults not using the intervention.

Regarding resilience, we found that higher motivation to use the intervention and better emotion regulation skills pre-treatment predicted better treatment outcome. To date, there have been no studies examining predictors of treatment outcome in interventions promoting resilience, let alone internet-based interventions. However, in an internet-based self-help intervention for stress, higher motivation seemed to predict better adherence (70). Accordingly, it could be assumed that the effect of higher motivation on treatment outcome regarding resilience is mediated by adherence in our study as well. Yet, this assumption is not supported by our data, as the number of completed modules did not predict the treatment outcome in terms of resilience [b(SE) = 0.162 (0.226), p = 0.475]. However, these results could be attributed to the fact that we measured adherence only by the number of completed modules. Some studies point out that adherence involves much more than mere technological usage (71, 72). Therefore, it could be possible that highly motivated participants are otherwise more engaged with the internet-based intervention, for example, by addressing the content of the intervention in more depth or implementing it more thoroughly in their daily lives, which in turn could improve treatment outcome.

In addition to motivational conditions, better pre-treatment emotion regulation skills also appear to predict better treatment outcomes in an internet-based intervention for COVID-19 related psychological distress in terms of resilience. The better treatment outcome regarding resilience in participants with better pre-treatment emotion regulation skills could be caused by so-called capitalization. Capitalization describes the fact that pre-existing strengths of patients are reinforced and built on in therapy (73). In one study, tailoring treatment by focusing on patients' respective strengths rather than on their respective deficits led to better treatment outcomes in depressed patients (74). Since the intervention under study focuses, among other aspects, on building emotion regulation skills, it could be argued that emotion regulation skills were capitalized in participants with better pre-treatment emotion regulation skills. Previous research found that emotion regulation skills are associated with higher resilience (75) and better emotional adjustment (76). Therefore, capitalizing emotion regulation skills might lead to benefits in resilience. In conclusion, it appears that in the present study, participants with higher pre-treatment resources (motivation or emotion regulation skills) improved more during an internet-based self-help intervention regarding resilience in an uncontrolled setting.

In the current study, multiple possible predictor variables did not predict post-treatment depressive symptoms and resilience. For example, female gender predicted neither treatment outcome. This finding is consistent with studies that found no effect of female gender on treatment outcome (63–65, 77). However, there are also some studies that have shown that female gender predicted better treatment outcome (78–80).

The current study comes with several limitations. First, our sample was relatively small for predictor analysis. Our analyses might have been underpowered since predictor effects in internet-based interventions tend to be small. In addition, the small sample size in combination with the applied prediction procedure leads to an increased chance of incidental findings. Moreover, as only participants with complete data sets were included in the analysis, sample size was further reduced for some outcomes due to drop-out. To prevent further reduction of our sample, other methods (e.g., multiple imputation) could have been used to address missing data instead of listwise deletion. However, we decided to do a completer analysis instead of an intention-to-treat analysis. Second, participants in the delayed treatment group [M (SD) = 2.89 (2.53)] completed significantly less modules than participants in the immediate treatment group [M (SD) = 4.15 (2.27); t_(104.1) = 2.719, p = 0.009], whereas the effect size was d = 0.52 (medium effect size). One possible reason for this result could be that the burden of the participants in the delayed treatment group has already decreased during the waiting period. Accordingly, participants in the immediate treatment group showed higher mean depression scores [M (SD) = 11.13 (4.36)] than participants in the delayed treatment group [M (SD) = 9.60 (3.89)]. However, this difference was not significant (t_(102.1) = 1.908, p = 0.055).The current sample might have been already less burdened at pre-treatment, and therefore might not be representative of people with COVID-19 related psychological distress actively seeking support. Third, we relied only on self-report outcome measures and did not conduct a clinical assessment. Accordingly, responses could be subjectively biased. This could particularly concern information on psychological burden. Fourth, despite several analyses, we did not correct for multiple tests. Accordingly, this could result in type 1 errors. Nonetheless, since analyses were exploratory, avoiding type 2 errors is crucial. However, results must therefore be considered as hypotheses generating and not as definitive findings (81). Fifth, we have not analyzed moderators of treatment outcome. Therefore, we cannot answer whether the ROCO intervention was more effective for certain participants (e.g., younger adults) when compared to a control group. Based on our analyses we can only draw preliminary conclusions about which participants benefitted more in an uncontrolled setting. Sixth, we refrained from using modern prediction methods which might lead to difficulties in replicating our results (56, 82).

Conclusion

Despite the limitations mentioned above, the current study gives preliminary evidence on the relationship between participant characteristics and treatment outcome in internet-based self-help interventions for COVID-19 related distress. One promising finding is that young adults, who can be considered a psychologically vulnerable group in the COVID-19 pandemic, seem to improve more using such an intervention in terms of depressive symptoms. Moreover, participants with higher motivation and better pre-treatment emotion regulation skills seemed to be able to build on their strengths and showed better treatment outcome in terms of resilience. Therefore, it could be beneficial to tailor interventions to respective strengths of the participants in order to promote resilience. However, further studies are needed to make informed decisions about the relationship of participant characteristics and treatment outcome in internet-based self-help interventions for COVID-19 related psychological distress. First, in further studies, the hypotheses generated in this exploratory analysis should be verified. Second, further studies should be conducted in a controlled setting to draw conclusions about the individuals for whom such an intervention is most effective.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethics Committee of the Canton of Bern. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

NB drafted the manuscript and was responsible for its conceptualization, methodology, formal analysis, data curation, and writing. JH was responsible for data curation and writing—reviewing and editing. TB and HZ were responsible for project administration, resources, and writing—reviewing and editing. All authors contributed to the article and approved the submitted version.

Funding

This work was entirely funded by the University of Bern. Open access funding was provided by the University of Bern.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer TL declared a past collaboration with one of the authors TB to the handling editor.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank Céline Bieri for her contribution to the data collection.

References

1. Wiersinga WJ, Rhodes A, Cheng AC, Peacock SJ, Prescott HC. Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease (2019). (COVID-19): a review. Jama. (2020) 324:782–93. doi: 10.1001/jama.2020.12839

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Wang C, Pan R, Wan X, Tan Y, Xu L, Ho CS, et al. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health. (2020) 17:1729. doi: 10.3390/ijerph17051729

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Rajkumar RP. COVID-19 and mental health: a review of the existing literature. Asian J Psychiatr. (2020) 2020:102066. doi: 10.1016/j.ajp.2020.102066

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Qiu J, Shen B, Zhao M, Wang Z, Xie B, Xu Y. A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: implications and policy recommendations. General Psychiatr. (2020) 33:213. doi: 10.1136/gpsych-2020-100213

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Breslau J, Finucane ML, Locker AR, Baird MD, Roth EA, Collins RL, et al. longitudinal study of psychological distress in the United States before and during the COVID-19 pandemic. Prevent Med. (2021) 143:106362. doi: 10.1016/j.ypmed.2020.106362

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Xiong J, Lipsitz O, Nasri F, Lui LM, Gill H, Phan L, et al. Impact of COVID-19 pandemic on mental health in the general population: a systematic review. J Affect Disord. (2020) 20:1. doi: 10.1016/j.jad.2020.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Luo M, Guo L, Yu M, Wang H. The psychological and mental impact of coronavirus disease COVID-19) on medical staff and general public–a systematic review and meta-analysis. Psychiatry Res. (2019) 2020:113190. doi: 10.1016/j.psychres.2020.113190

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic. JAMA Netw Open. (2020) 3:e2019686. doi: 10.1001/jamanetworkopen.2020.19686

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Albrecht JN, Werner H, Rieger N, Widmer N, Janisch D, Huber R, et al. Association between homeschooling and adolescent sleep duration and health during COVID-19 pandemic high school closures. JAMA Netw Open. (2022) 5:e2142100–e. doi: 10.1001/jamanetworkopen.2021.42100

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Aghababa A, Sani SHZ, Rohani H, Nabilpour M, Badicu G, Fathirezaie Z, et al. No evidence of systematic change of physical activity patterns before and during the Covid-19 pandemic and related mood states among iranian adults attending team sports activities. Front Psychol. (2021) 21:12. doi: 10.3389/fpsyg.2021.641895

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Robinson E, Sutin AR, Daly M, Jones A. A systematic review and meta-analysis of longitudinal cohort studies comparing mental health before vs. during the COVID-19 pandemic. medRxiv. (2021). doi: 10.1101/2021.03.04.21252921

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Daly M, Sutin AR, Robinson E. Longitudinal changes in mental health and the COVID-19 pandemic: Evidence from the UK household longitudinal study. Psychol Med. (2020) 2020:1–10. doi: 10.1017/S0033291720004432

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Daly M, Robinson E. Psychological distress and adaptation to the COVID-19 crisis in the United States. J Psychiatr Res. (2021) 136:603–9. doi: 10.1016/j.jpsychires.2020.10.035

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Infurna FJ, Luthar SS. Re-evaluating the notion that resilience is commonplace: a review and distillation of directions for future research, practice, and policy. Clin Psychol Rev. (2018) 65:43–56. doi: 10.1016/j.cpr.2018.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Shevlin M, Butter S, McBride O, Murphy J, Gibson-Miller J, Hartman TK, et al. Refuting the myth of a ‘tsunami’of mental ill-health in populations affected by COVID-19: evidence that response to the pandemic is heterogeneous, not homogeneous. Psychol Med. (2021) 2021:1-9. doi: 10.1017/S0033291721001665

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Ahrens K, Neumann R, Kollmann B, Brokelmann J, Von Werthern N, Malyshau A, et al. Impact of COVID-19 lockdown on mental health in Germany: longitudinal observation of different mental health trajectories and protective factors. Transl Psychiatry. (2021) 11:1–10. doi: 10.1038/s41398-021-01508-2

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Halder S. COVID-19: psychological impact and psychotherapeutic intervention. EC Psychol Psychiatr. (2020) 9:32−5.

Google Scholar

18. Soklaridis S, Lin E, Lalani Y, Rodak T, Sockalingam S. Mental health interventions and supports during COVID-19 and other medical pandemics: A rapid systematic review of the evidence. Gen Hosp Psychiatr. (2020) 66:133–46. doi: 10.1016/j.genhosppsych.2020.08.007

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Wind TR, Rijkeboer M, Andersson G, Riper H. The COVID-19 pandemic: the ‘black swan’for mental health care and a turning point for e-health. Intern Intervent. (2020) 20:317. doi: 10.1016/j.invent.2020.100317

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Andersson G, Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatr. (2014) 13:4–11. doi: 10.1002/wps.20083

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Cuijpers P, Donker T, Johansson R, Mohr DC, van Straten A, Andersson G. Self-guided psychological treatment for depressive symptoms: a meta-analysis. PLoS One. (2011) 6:e21274. doi: 10.1371/journal.pone.0021274

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Wahlund T, Mataix-Cols D, Lauri KO, de Schipper E, Ljótsson B, Aspvall K, et al. Brief online cognitive behavioural intervention for dysfunctional worry related to the COVID-19 pandemic: a randomised controlled trial. Psychother Psychosom. (2021) 90:191–9. doi: 10.1159/000512843

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Al-Alawi M, McCall RK, Sultan A, Al Balushi N, Al-Mahrouqi T, Al Ghailani A, et al. Efficacy of a six-week-long therapist-guided online therapy vs. self-help internet-based therapy for COVID-19–induced anxiety and depression: open-label, pragmatic, randomized controlled trial. JMIR Ment Health. (2021) 8:e26683. doi: 10.2196/26683

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Aminoff V, Sellén M, Sörliden E, Ludvigsson M, Berg M, Andersson G. Internet-Based cognitive behavioral therapy for psychological distress associated with the COVID-19 pandemic: a pilot randomized controlled trial. Front Psychol. (2021) 12:1998. doi: 10.3389/fpsyg.2021.684540

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Brog NA, Hegy JK, Berger T, Znoj H. Effects of an internet-based self-help intervention for psychological distress due to COVID-19: results of a randomized controlled trial. Intern Intervent. (2021) 2021:100492. doi: 10.21203/rs.3.rs-59343/v2

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Asmundson GJ, Paluszek MM, Landry CA, Rachor GS, McKay D, Taylor S. Do pre-existing anxiety-related and mood disorders differentially impact COVID-19 stress responses and coping? J Anxiety Disord. (2020) 74:102271. doi: 10.1016/j.janxdis.2020.102271

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Alonzi S, La Torre A, Silverstein MW. The psychological impact of preexisting mental and physical health conditions during the COVID-19 pandemic. Psychol Trauma: Theory Res Pract Policy. (2020) 12:S236. doi: 10.1037/tra0000840

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Mazza C, Ricci E, Biondi S, Colasanti M, Ferracuti S, Napoli C, et al. A nationwide survey of psychological distress among Italian people during the COVID-19 pandemic: immediate psychological responses and associated factors. Int J Environ Res Public Health. (2020) 17:3165. doi: 10.3390/ijerph17093165

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Li LZ, Wang S. Prevalence and predictors of general psychiatric disorders and loneliness during COVID-19 in the United Kingdom. Psychiatry Res. (2020) 291:113267. doi: 10.1016/j.psychres.2020.113267

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Kimhi S, Marciano H, Eshel Y, Adini B. Resilience and demographic characteristics predicting distress during the COVID-19 crisis. Soc Sci Med. (2020) 265:113389. doi: 10.1016/j.socscimed.2020.113389

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Birditt KS, Turkelson A, Fingerman KL, Polenick CA, Oya A. Age differences in stress, life changes, and social ties during the COVID-19 pandemic: Implications for psychological well-being. Gerontologist. (2021) 61:205–16. doi: 10.1093/geront/gnaa204

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Kowal M, Coll-Martín T, Ikizer G, Rasmussen J, Eichel K, Studzińska A, et al. Who is the most stressed during the COVID-19 pandemic? data from 26 countries and areas. Appl Psychol Health Well-Being. (2020) 12:946–66. doi: 10.1111/aphw.12234

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Tyra AT, Griffin SM, Fergus TA, Ginty AT. Individual differences in emotion regulation prospectively predict early COVID-19 related acute stress. J Anxiety Disord. (2021) 81:102411. doi: 10.1016/j.janxdis.2021.102411

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Panayiotou G, Panteli M, Leonidou C. Coping with the invisible enemy: The role of emotion regulation and awareness in quality of life during the COVID-19 pandemic. J Context Behav Sci. (2021) 19:17–27. doi: 10.1016/j.jcbs.2020.11.002

CrossRef Full Text | Google Scholar

35. Brog NA, Hegy JK, Berger T, Znoj H. An internet-based self-help intervention for people with psychological distress due to COVID-19: study protocol for a randomized controlled trial. Trials. (2021) 22:171. doi: 10.1186/s13063-021-05089-9

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Löwe B, Spitzer RL, Zipfel S, Herzog W. Komplettversion und Kurzform. New York, NY: Pfizer (2002).

Google Scholar

37. Osman A, Bagge CL, Gutierrez PM, Konick LC, Kopper BA, Barrios FX. The suicidal behaviors questionnaire-revised (SBQ-R): validation with clinical and nonclinical samples. Assessment. (2001) 8:443–54. doi: 10.1177/107319110100800409

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann. (2002) 32:509–15. doi: 10.3928/0048-5713-20020901-06

CrossRef Full Text | Google Scholar

39. Connor KM, Davidson JR. Development of a new resilience scale: the connor-davidson resilience scale (CD-RISC). Depress Anxiety. (2003) 18:76–82. doi: 10.1002/da.10113

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav Res Ther. (1995) 33:335–43. doi: 10.1016/0005-7967(94)00075-U

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Gomez F. “A guide to the depression, anxiety and stress scale (DASS 21),” In: Central and Eastern Sydney primary health networks (2016).

Google Scholar

42. Ware JE Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. (1996) 1996:220–33. doi: 10.1097/00005650-199603000-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Gandek B, Ware JE, Aaronson NK, Apolone G, Bjorner JB, Brazier JE, et al. Cross-validation of item selection and scoring for the SF-12 Health Survey in nine countries: results from the IQOLA Project. J Clin Epidemiol. (1998) 51:1171–8. doi: 10.1016/S0895-4356(98)00109-7

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Znoj H, Schnyder U. Verbitterungsfragebogen Kurzversion. Bern: University of Bern (2014).

Google Scholar

45. Znoj H, Abegglen S, Buchkremer U, Linden M. The embittered mind. J Individ Differen. (2016) 16:208. doi: 10.1027/1614-0001/a000208

CrossRef Full Text | Google Scholar

46. Muschalla B, Vollborn C, Sondhof A. Embitterment as a specific mental health reaction during the coronavirus pandemic. Psychopathology. (2021) 54:232–41. doi: 10.1159/000517447

PubMed Abstract | CrossRef Full Text | Google Scholar

47. De Sousa A D'souza R Embitterment: Embitterment: the nature of the construct and critical issues in the light of COVID-19. healthcare. London: Multidisciplinary Digital Publishing Institute (2020).

Google Scholar

48. Znoj H. BVI. Berner Verbitterungs-Inventar.: Hogrefe (2008).

Google Scholar

49. Luhmann M, Bohn J, Holtmann J, Koch T, Eid M. I'm lonely, can't you tell? convergent validity of self-and informant ratings of loneliness. J Res Personal. (2016) 61:50–60. doi: 10.1016/j.jrp.2016.02.002

CrossRef Full Text | Google Scholar

50. Lee CM, Cadigan JM, Rhew IC. Increases in loneliness among young adults during the COVID-19 pandemic and association with increases in mental health problems. J Adolesc Health. (2020) 67:714–7. doi: 10.1016/j.jadohealth.2020.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Okruszek L, Aniszewska-Stańczuk A, Piejka A, Wiśniewska M, Zurek K. “Safe but lonely?,” in: Loneliness, mental health symptoms and COVID-19 (2020).

Google Scholar

52. Jerusalem M, Schwarzer R. SWE-Skala zur Allgemeinen Selbstwirksamkeitserwartung (2003).

Google Scholar

53. Glaesmer H, Hoyer J, Klotsche J, Herzberg PY. Die deutsche version des life-orientation-tests (LOT-R) zum dispositionellen optimismus und pessimismus. Zeitschrift für Gesundheitspsychologie. (2008) 16:26–31. doi: 10.1026/0943-8149.16.1.26

CrossRef Full Text | Google Scholar

54. Berking M, Znoj H. SEK-27. 84Fragebogen zur standardisierten Selbsteinschätzung emotionaler Kompetenzen [Verfahrensdokumentation aus PSYNDEX Tests-Nr. 9005957, Fragebogen und Auswertungsanweisung],“ In: ed. (Hrsg.) ETTZ, editor: In Leibniz-Zentrum für Psychologische Information und Dokumentation (ZPID) (2011).

Google Scholar

55. Smith G. Step away from stepwise. J Big Data. (2018) 5:1–12. doi: 10.1186/s40537-018-0143-6

CrossRef Full Text | Google Scholar

56. Steyerberg EW, Uno H, Ioannidis JP, Van Calster B, Ukaegbu C, Dhingra T, et al. Poor performance of clinical prediction models: the harm of commonly applied methods. J Clin Epidemiol. (2018) 98:133–43. doi: 10.1016/j.jclinepi.2017.11.013

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Thayer JD. Stepwise Regression as an Exploratory Data Analysis Procedure (2002).

Google Scholar

58. Field A. Discovering statistics using IBM SPSS Statistics. London: Sage (2013).

Google Scholar

59. Austin PC, Steyerberg EW. The number of subjects per variable required in linear regression analyses. J Clin Epidemiol. (2015) 68:627–36. doi: 10.1016/j.jclinepi.2014.12.014

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Nerlove M, Wallis KF. Use of the Durbin-Watson statistic in inappropriate situations. Econometrica: J Econometr Soc. (1966) 1966:235-8. doi: 10.2307/1909870

CrossRef Full Text | Google Scholar

61. Blatná D. Outliers in regression. Trutnov. (2006) 30:1–6.

Google Scholar

62. Orr JM, Sackett PR, Dubois CL. Outlier detection and treatment in I/O psychology: A survey of researcher beliefs and an empirical illustration. Pers Psychol. (1991) 44:473–86. doi: 10.1111/j.1744-6570.1991.tb02401.x

CrossRef Full Text | Google Scholar

63. Niles AN, Axelsson E, Andersson E, Hedman-Lagerlöf E, Carlbring P, Andersson G, et al. Internet-based cognitive behavior therapy for depression, social anxiety disorder, and panic disorder: effectiveness and predictors of response in a teaching clinic. Behav Res Ther. (2021) 136:103767. doi: 10.1016/j.brat.2020.103767

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Høifødt RS, Mittner M, Lillevoll K, Katla SK, Kolstrup N, Eisemann M, et al. Predictors of response to Web-based cognitive behavioral therapy with high-intensity face-to-face therapist guidance for depression: a Bayesian analysis. J Med Internet Res. (2015) 17:e4351. doi: 10.2196/jmir.4351

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Warmerdam L, Van Straten A, Twisk J, Cuijpers P. Predicting outcome of Internet-based treatment for depressive symptoms. Psychother Res. (2013) 23:559–67. doi: 10.1080/10503307.2013.807377

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Hobbs MJ, Joubert AE, Mahoney AE, Andrews G. Treating late-life depression: Comparing the effects of internet-delivered cognitive behavior therapy across the adult lifespan. J Affect Disord. (2018) 226:58–65. doi: 10.1016/j.jad.2017.09.026

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Schneider BC, Schröder J, Berger T, Hohagen F, Meyer B, Späth C, et al. Bridging the “digital divide”: a comparison of use and effectiveness of an online intervention for depression between baby boomers and millennials. J Affect Disord. (2018) 236:243–51. doi: 10.1016/j.jad.2018.04.101

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Shanahan L, Steinhoff A, Bechtiger L, Murray AL, Nivette A, Hepp U, et al. Emotional distress in young adults during the COVID-19 pandemic: evidence of risk and resilience from a longitudinal cohort study. Psychol Med. (2020) 2020:1–10. doi: 10.1017/S003329172000241X

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Minahan J, Falzarano F, Yazdani N, Siedlecki KL. The COVID-19 pandemic and psychosocial outcomes across age through the stress and coping framework. Gerontologist. (2021) 61:228–39. doi: 10.1093/geront/gnaa205

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Alfonsson S, Olsson E, Hursti T. Motivation and treatment credibility predicts dropout, treatment adherence, and clinical outcomes in an internet-based cognitive behavioral relaxation program: a randomized controlled trial. J Med Internet Res. (2016) 18:e52. doi: 10.2196/jmir.5352

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Ryan C, Bergin M, Wells JS. Theoretical perspectives of adherence to web-based interventions: a scoping review. Int J Behav Med. (2018) 25:17–29. doi: 10.1007/s12529-017-9678-8

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Sieverink F, Kelders SM, van Gemert-Pijnen JE. Clarifying the concept of adherence to eHealth technology: systematic review on when usage becomes adherence. J Med Internet Res. (2017) 19:e8578. doi: 10.2196/jmir.8578

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Flückiger C, Caspar F, Holtforth MG, Willutzki U. Working with patients' strengths: a microprocess approach. Psychother Res. (2009) 19:213–23. doi: 10.1080/10503300902755300

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Cheavens JS, Strunk DR, Lazarus SA, Goldstein LA. The compensation and capitalization models: A Test of two approaches to individualizing the treatment of depression. Behav Res Ther. (2012) 50:699–706. doi: 10.1016/j.brat.2012.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Min J-A, Yu JJ, Lee C-U, Chae J-H. Cognitive emotion regulation strategies contributing to resilience in patients with depression and/or anxiety disorders. Compr Psychiatr. (2013) 54:1190–7. doi: 10.1016/j.comppsych.2013.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Berking M, Orth U, Wupperman P, Meier LL, Caspar F. Prospective effects of emotion-regulation skills on emotional adjustment. J Couns Psychol. (2008) 55:485. doi: 10.1037/a0013589

PubMed Abstract | CrossRef Full Text | Google Scholar

77. Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, Öberg J, et al. Internet administered guided self-help vs. individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther. (2010) 48:368–76. doi: 10.1016/j.brat.2010.01.005

PubMed Abstract | CrossRef Full Text | Google Scholar

78. Spek V, Nyklíček I, Cuijpers P, Pop V. Predictors of outcome of group and internet-based cognitive behavior therapy. J Affect Disord. (2008) 105:137–45. doi: 10.1016/j.jad.2007.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

79. Clarke G, Kelleher C, Hornbrook M, DeBar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther. (2009) 38:222–34. doi: 10.1080/16506070802675353

PubMed Abstract | CrossRef Full Text | Google Scholar

80. Andersson G, Carlbring P, Rozental A. Response and remission rates in internet-based cognitive behavior therapy: an individual patient data meta-analysis. Front Psychiatry. (2019) 10:749. doi: 10.3389/fpsyt.2019.00749

PubMed Abstract | CrossRef Full Text | Google Scholar

81. Streiner DL, Norman GR. Correction for multiple testing: is there a resolution? Chest. (2011) 140:16–8. doi: 10.1378/chest.11-0523

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Putka DJ, Beatty AS, Reeder MC. Modern prediction methods: new perspectives on a common problem. Organ Res Methods. (2018) 21:689–732. doi: 10.1177/1094428117697041

CrossRef Full Text | Google Scholar