Engagement, efficacy, and experiences of psychotherapy for perinatal populations with depression and anxiety during the COVID-19 pandemic

Background: Engagement in psychotherapy is essential for achieving effective mental health outcomes, yet maintaining participation can be challenging—especially during significant disruptions such as the COVID-19 pandemic. The SUMMIT Trial evaluated a brief behavioral activation treatment for perinatal depression and anxiety, comparing telemedicine with in-person psychotherapy. Due to pandemic-related disruptions, in-person randomization was suspended twice, dividing participants into peak and non-peak COVID-19 timeframes.

Methods: This secondary mixed-methods analysis examined enrollment, retention, satisfaction, and depressive and anxiety symptoms between peak (March 2020-July 2021 & Jan-April 2022) and non-peak COVID-19 periods (Jan-March 2020, Jul 2021-Jan 2022, April 2022-Sep 2023). During peak-COVID-19, participants received telemedicine-only; otherwise, they were randomized to telemedicine or in-person treatment. T-tests compared symptom scores and enrollment rates; chi-square and logistic regression analyzed retention and satisfaction. Qualitative data underwent thematic analysis.

Results: Of 1230 participants, 597 (48.5%) enrolled during peak-COVID and were randomized to telemedicine, while 693 (56.3%) were randomized to telemedicine or in-person treatment. Enrollment (25.4 vs. 21.8 participants/month, p = 0.250), retention (98.12% vs. 98.42%, p = 0.689) and satisfaction (CSQ-8: 3.45 vs. 3.39, p = 0.174) did not differ significantly. No differences were observed in depressive or anxiety symptoms at baseline (EPDS: 15.85 vs. 15.72, p = 0.546; GAD-7: 11.70 vs. 11.96, p = 0.3670) or at 3 months (EPDS: 9.09 vs. 9.09, p = 0.980; GAD-7: 6.42 vs. 6.38, p = 0.907).

Conclusions: Engagement, efficacy, and experience were comparable across pandemic phases, highlighting the feasibility of telemedicine-based adaptations in the midst of public health crises.

Introduction

The COVID-19 pandemic placed an unprecedented strain on healthcare systems, significantly disrupting access to medical care, including mental health services (1). In the perinatal population, the inability to see patients in person further constrained treatment availability (2) and mental health treatments were deprioritized in deference to lifesaving care (3). These challenges—exacerbated by the pandemic—underscore the urgent need to create patient-centered models of care during public health crises and to assess any therapeutic limitations associated with modifying usual care standards.

The Scaling Up Maternal Mental healthcare by Increasing access to Treatment (SUMMIT) trial aimed to aimed to compare telemedicine vs. in-person delivered psychotherapy from specialist (e.g., licensed clinicians) and non-specialist (e.g., nurses, midwives) providers (4, 5). The trial was conducted throughout the COVID-19 pandemic between January 2020 and October 2023 and, required significant modification of its original study design due to pandemic-related factors. The operational changes included strengthening clinical referral pathways, transitioning research staff to remote work with secure server access, and implementing secure telemedicine platforms for treatment delivery and storage of encrypted session recordings (4). The in-person study arms were suspended twice during peak infection prevalence corresponding to the onset of the pandemic through the highly infectious Delta variant, and then again during the height of the Omicron variant when case prevalence increased dramatically. The primary analysis at 3-month post-randomization demonstrated that task-sharing and telemedicine were non-inferior to specialist-delivered, in-person treatment for perinatal depression and anxiety outcome (5). While there had been no planning to accommodate the pandemic, a modified analytical plan was put into place along with in-person restrictions to accommodate the COVID-19 disruptions (ClinicalTrials.gov: NCT04153864).

Similar to all psychotherapy trials, the SUMMIT Trial relied on participant engagement for both scientific rigor and clinical application (6–8). Key variables related to participant engagement included rates of participant enrollment, retention, and satisfaction. Enrollment rates reflected an intervention’s reach, while retention rates were a measure of intervention feasibility and engagement. Finally, participant satisfaction ratings provided insight into the actual experience of treatment and its perceived effectiveness. These factors may be particularly relevant in task-sharing models, where non-specialist providers (individuals with no formal training or mental healthcare background) deliver care, as well as in telemedicine which requires sustained remote engagement to achieve effectiveness (9). The concurrence of the pandemic may impact all of these factors as well as treatment effectiveness.

Thus, the current study aimed to assess the impact of COVID-19-related adaptations within the trial on participant engagement, treatment efficacy and patient experience during peak and non-peak COVID-19 periods. In this study, peak-COVID periods are defined as a phase during which randomization and treatment delivery were restricted to telemedicine as a result of increased COVID cases in the general population often leading to lockdowns (March 2020-July 2021; January 2022-April 2022). The remainder of the trial period was classified as non-peak.

This exploratory study to the SUMMIT Trial had three primary aims:

1. To evaluate patient engagement (rates of recruitment, retention and satisfaction scores) in perinatal depression treatment during the COVID-19 pandemic, by comparing enrollment and retention rates across peak and non-peak pandemic periods.

2. To assess treatment efficacy of brief behavioral activation (BA) delivered via telemedicine during peak versus non-peak COVID-19 periods, as measured by changes in patient symptoms of depression and anxiety from enrollment to 3-month measurement.

3. To explore patient experience during these periods using both quantitative satisfaction data and qualitative analysis of reported barriers and facilitators to care.

Methods

Design and setting

This mixed methods study is a secondary analysis of the Scaling Up Maternal Mental Healthcare by Increasing Access to Treatment (SUMMIT) trial, which was designed to evaluate the non-inferiority of Behavioral Activation (BA) delivered by specialist vs. non-specialist providers and via telemedicine vs. in-person modalities for perinatal depression and anxiety treatment. The design of the SUMMIT trial (ClinicalTrials.gov: NCT04153864) has been described in detail elsewhere (4, 5). The study recruited participants from university-affiliated healthcare settings, clinical sites, and obstetric and family medicine clinics across Chapel Hill, North Carolina, USA; Evanston/Chicago, Illinois, USA; and Toronto, Canada.

Participants

Recruitment began in early 2020, just prior to the COVID-19 pandemic, and continued through late 2023. A total of 1230 pregnant and postpartum women were enrolled across sites. Participants were introduced to the study by clinical providers or trained research assistants. Inclusion and exclusion criteria for the larger trial are described elsewhere (4). All participants provided written or electronic informed consent via REDcap™ before enrollment. Ethical approval was obtained from the following institutional review boards (IRBs): UNC Biomedical IRB (19-1786); Endeavor Health IRB (EH18-129); and Clinical Trials Ontario (1895).

Between March 2020–July 2021 and January 2022–April 2022, a COVID-restricted randomization strategy was employed, where all participants received telemedicine-only treatment but were still able to be randomized to either specialist or non-specialist care. These periods are referred to as “peak-COVID-19.” In addition, any patients who were randomized 1:1 to the two telemedicine arms were grouped into the peak-COVID category (Figure 1). During non-peak COVID-19, participants were randomized to either in-person or telemedicine treatment and as the pandemic resolved, a weighted randomization strategy favoring in-person care was employed to partially restore the balanced allotment that was originally planned (5). Qualitative interview participants during peak COVID-19 completed all BA sessions by telemedicine, while participants interviewed in the non-peak period completed BA either in-person or by telemedicine.

Figure 1

Figure 1. Distribution on Peak and Non-peak COVID-19 cases during the SUMMIT trial (January 2020-September 2023). Due to site-specific differences in response to the Omicron variant, 64 additional participants were categorized as “Non-Peak COVID” from January 2022-April 2022.

Intervention

Participants received up to 8 sessions of a brief, manualized BA intervention delivered by a specialist provider (SP) (e.g., psychologists, psychiatrists, social workers) or a trained non-specialist provider (NSP) (e.g., nurses, midwives, doulas). Treatment was delivered either in-person or by video via a HIPAA/PHIPA-compliant telemedicine video platform, such as Zoom™ or WebEx™, in a setting of the participant’s choice. During peak-COVID periods, all BA sessions were conducted via telemedicine, while during non-peak periods, participants were randomly assigned to receive in-person or telemedicine-based treatment.

BA is a transdiagnostic treatment that is effective for depressive and anxiety symptoms (10–13), including for ethnically-diverse perinatal populations (14, 15). The intervention focuses on increasing engagement in enjoyable and meaningful activities that align with one’s values, aiming to reduce mental health symptoms (16, 17). Further details on the intervention are provided elsewhere (4, 18–20). The BA manual used in the SUMMIT Trial is now freely available (www.thesummittrial.com).

Demographic and clinical characteristics

Demographic and clinical data were collected at baseline and 3 months post-randomization using REDCap™. Demographic variables included participant age, perinatal period (pregnancy or postpartum), race/ethnicity, gender identity, sexual orientation, marital status, employment status, education level, and treatment preferences. Clinical data included psychotropic medication use and pregnancy-related conditions. Participants were categorized into white and racial/ethnic minority groups based on self-report. The number of completed BA sessions was recorded at 3 months post-randomization.

Measures

Enrollment and retention

Enrollment rate was defined as the number of participants randomized per month; retention rate was defined as the number of individuals who (a) remained in the trial before starting BA and (b) remained during the treatment phase. A robust retention strategy was implemented throughout the study to maximize participant retention and patient-centeredness (21). These efforts included pre-notification emails, follow-up messages via email, phone, and text, and routine updates to participant contact information and training all research personnel in common therapeutic skills such as demonstrating empathy and a non-judgmental stance (5). Participant engagement was further supported through quarterly newsletter reminders, in addition to the information about the study progression. As a result, the study achieved a low dropout rate.

Client Satisfaction Questionnaire-8

The Client Satisfaction Questionnaire-8 (CSQ-8) is an 8-item Likert scale designed to gauge satisfaction with psychotherapy services and was completed by the participants at 3 months post-randomization. It assesses the quality of treatment, the likelihood of recommending the service to others, and the probability of seeking future support or returning for more treatment. Responses are averaged on a scale from 1 to 4, with higher scores reflecting greater satisfaction. The CSQ-8 is well-regarded for its reliability and has been linked to symptom improvement (22). It has been validated in studies involving perinatal populations (23) as well as diverse ethnic groups (24).

Symptom scores

Symptom scores were collected at baseline and 3 months post-randomization. Depressive and anxiety scores were measured by the 10-item Edinburgh Postnatal Depression Scale (EPDS) and the 7-item Generalized Anxiety Disorder-7 (GAD-7), respectively. The EPDS is a 10-item, Likert scale (range 0-30) that is highly sensitive to change and is the most common measure of perinatal depressive symptoms in clinical and research settings (25–27). The GAD-7 is a 7-item scale (range 0–21) with good internal consistency, reliability, validity and is sensitive to change (28, 29).

Qualitative interviews

Qualitative data were collected over two time periods: 1) during the peak COVID-19 period of September to December 2020 and 2) during the non-peak period of August to October 2022. Qualitative researchers conducted semi-structured, individual interviews with participants across all three SUMMIT study sites. Qualitative interviews were extended to a subset of consenting participants to further explore perinatal perspectives. Participants who completed a minimum of 5 treatment sessions and consented to a qualitative interview were randomly selected and the sample size was determined through maximum saturation methods. Interviews were conducted using a HIPAA/PHIPA-compliant version of Zoom™. Each interview lasted 40–60 minutes. In the interviews, all participants were asked open-ended questions about their perspectives of facilitators and barriers to positive care experiences, in which themes related to facilitators and barriers were endorsed spontaneously by participants (see Appendix 1 in Supplementary Materials for interview questions).

Statistical methods

All baseline variables of interest were summarized using means and two-sided 95% Confidence Intervals (CIs) for continuous measures, and percentages for categorical measures. Differences in baseline characteristics and symptom scores between peak and non-peak COVID-19 time periods were evaluated using a two-sample t-test for continuous variables and a chi-square test for categorical variables.

To compare participant enrollment rates during peak and non-peak periods, the monthly enrollment rate was analyzed using a two-sample t-test. Retention rates before and during treatment across peak and non-peak COVID-19 periods were assessed with a chi-square test of independence. Logistic regression was employed to examine the relationship between retention rates and COVID-19 periods, while controlling for treatment modality and race/ethnicity.

Client satisfaction during peak and non-peak COVID-19 periods was compared using a two-sample t-test. Imputation methods were applied to incorporate the missing data for individuals who had completed at least one treatment session but did not respond to the CSQ-8 measure. Multiple linear regression was utilized to examine the relationship between CSQ-8 and COVID-19 periods, controlling for patient’s household income, perinatal status (pregnant or postpartum), treatment modality and race/ethnicity.

A subgroup analysis was conducted by excluding in-person participants from the non-peak period to compare the telemedicine-only treatment delivery between peak- and non-peak periods. In this analysis, linear and logistic regression models included only race/ethnicity as a covariate. The selection of covariates was based on descriptive statistics indicating significant differences between groups, and variables with high inter-correlation were excluded from the analysis. For retention rates, a subgroup analysis was not performed as during the non-peak period, both in-person and telemedicine enrollment occurred. A subgroup analysis excluding in-person individuals in the enrollment numbers introduces selection bias and does not account for the context of the entire enrollment process.

All statistical analyses were conducted using SAS 9.4 software.

Qualitative analysis

Using established methodologies, qualitative data were coded using NVivo™ and were analyzed using thematic analysis (21, 30). The interview guide was used to develop an initial coding index. Coding was done through a 3-step process: a line-by-line coding stage, a follow-up coding stage in which new themes emerged and were synthesized, and a final coding stage in which all codes were organized thematically. Inter-rater reliability, assessed using Cohen’s Kappa (κ), demonstrated excellent agreement between coders (κ=0.95). Descriptive statistics (means, ranges, and frequencies) were calculated to summarize qualitative findings.

Results

Baseline characteristics

Among 1230 participants, 597 were peak participants (i.e., randomized to telemedicine arms only) and 633 were non-peak participants (i.e., randomized to telemedicine and in-person arms). The distribution between pregnant and postpartum participants was nearly balanced in both groups (Table 1). Peak participants were significantly younger (32.79 vs. 33.64 years, p = 0.003) and more likely to identify as white (55.77% vs. 45.77, p = 0.0009) than non-peak participants. Peak period participants completed significantly more sessions (7.04 (SD = 1.83) vs. 6.64 (SD = 2.22), p = 0.001) than non-peak participants, likely owing to exclusive telemedicine delivery. Participants’ pre-randomization preference for telemedicine was also higher during the peak period (65.50% vs. 59.57%, p = 0.009). Two individuals endorsed COVID-19 exposure at baseline during the peak time frame, compared to 20 during the non-peak time frame (Peak: n = 2, 0.51% vs. Non-peak: n = 20, 5.03%, p = 0.002), perhaps related to less vigilance among the public when infection prevalence was lower. We also examined baseline characteristics among participants retained at 3 months post-randomization (n = 1098), which demonstrated similar distributions and between-group differences as the full (n = 1230) cohort (Supplementary Table 1). In addition, these characteristics did not differ among participants who dropped out of the study (n = 132; Supplementary Table 2).

Table 1

Table 1. Baseline participant characteristics categorized by peak and non-peak COVID-19 timeframes.

Enrollment and retention rates

There was no statistically significant difference in monthly enrollment rates between groups (Peak: 25.38 vs. Non-Peak: 21.78 participants per month, t-value = -1.16, p = 0.250). Similarly, a chi-square test of independence revealed no significant relationship between participant retention during treatment and the COVID-19 periods (Peak: 98.12% vs. Non-peak: 98.42%, χ² = 0.160, p = 0.689). Logistic regression analysis, accounting for covariates, also showed no significant effect of the peak versus non-peak COVID intervals during treatment (p = 0.383). However, there was a significant difference between “retention until treatment” based on the COVID period (Peak: 96.44% vs. Non-peak: 98.42%, χ² = 5.01, p = 0.025). This means that a lower percentage of enrolled “peak” participants never started therapy. When controlling for covariates, the logistic regression evaluating retention until treatment did not show significance (p = 0.131).

Symptom scores

No statistically significant differences were observed in depressive or anxiety symptoms between the peak and non-peak COVID periods at both baseline (EPDS: 15.85 vs. 15.72, p = 0.546; GAD-7: 11.70 vs. 11.96, p = 0.3670) and 3- months post-randomization (EPDS: 9.09 vs. 9.09, p = 0.980; GAD-7: 6.42 vs. 6.38, p = 0.907) (Table 2).

Table 2

Table 2. Mean depressive and anxiety symptom scores at baseline and 3 months for full cohort and peak and non-peak subgroups (N = 1230).

Participant satisfaction

There was no statistically significant difference in participant satisfaction (CSQ-8) scores between peak and non-peak periods (CSQ-8: 3.45 vs. 3.39, p = 0.174). Nor did linear mixed model analyses reveal a significant effect of the COVID period on patient satisfaction score (β = 0.05, p = 0.304), after controlling for relevant covariates. Only 1033 patients out of the full cohort had at least one treatment session and completed CSQ. Data was imputed for individuals (n=86) who completed sessions but did not complete the CSQ.

Barriers and facilitators to positive care experiences

The qualitative sample (N = 37) consisted of 19 peak and 18 non-peak perinatal participants (for full list of characteristics, see Supplementary Table 3). All reported facilitators and barrier categories are displayed in Figure 2 and the breakdown of these categories are displayed in Table 3. Participants reported similar perceived facilitators to satisfactory care experiences across both periods. These facilitators included: (1) empathetic and non-judgmental providers (Peak n=8, 42.0%; Non-peak n=12, 66.7%), (2) relatable and compatible providers (Peak n=8, 42.0%; Non-peak n=11, 61.1%), (3) providers skilled at delivering BA (Peak n=7, 37.0%; Non-peak n=12, 66.7%), (4) flexible (Peak n=5, 26.0%; Non-peak n=8, 44.4%), (5) skilled at active listening (Peak n=5, 26.0%; Non-peak n=5, 27.8%), and (6) provided helpful notes and feedback (Peak n=2, 11.0%; Non-peak n=6, 33.3%). Participants in the non-peak period endorsed each facilitator more frequently, which may have been shaped by their enrollment in in-person sessions due to COVID or COVID-19 pandemic itself. Nonetheless, as described, there was no statistical difference in treatment satisfaction among participants from both groups.

Figure 2

Figure 2. Barriers & facilitator categories to treatment satisfaction. Figure 2 display the barriers and facilitators treatment reported by peak and non-peak perinatal participants. The same barriers and facilitators were reported by both groups. When asked about barriers, “no barriers” was reported most often by both groups.

Table 3

Table 3. Barriers and facilitators to treatment by peak and non-peak COVID subgroups (n=37).

Representative facilitator comments included those that reflected favorably on the NSP’s relatability as a mother:

“[My provider] also happens to be a mother and has a young child. I felt like that was helpful to me, and was particularly affirming … because they also identify as a woman, there were definite moments when I ranted and then at the end just kind of went ‘well, patriarchy’ [Laughter]. It was nice, they understood and or at least laughed a little bit because some of my observations, she said, resonated with her as well.” (Sinai Participant 61, Non-peak COVID)

and the acknowledgement that the participant was not the only individual with emotional challenges associated with childbirth:

“It was helpful to have [a provider] who is a mom [since] my struggle really stemmed from challenges with breastfeeding … She was really relatable. I’ve been in therapy before and I felt like I haven’t had that same connection with any other therapists.” (Northshore Participant 2, Peak COVID)

When asked about barriers to satisfactory treatment, both peak and non-peak participants reported few barriers that were similar in frequency between peak and non-peak groups. The most common “barrier” was participants’ desire for more sessions to develop a better connection with their provider (Peak n=3, 16.0%; Non-peak n=5, 27.8%):

“I only have good things to say about [provider’s name]. One thing was craving for more time with her [laughter]. I would have liked our sessions to be longer. It’s not feedback on her by any means. I feel a little bit more time could have really enhanced the quality of the conversations.” (Sinai Participant 70, Non-peak COVID)

Overall, it did not appear that COVID-19 peak periods were associated with unique facilitators or barriers and the impact of telemedicine during high COVID-19 prevalence may have accounted for these observations.

Telemedicine-only cases

In order to control for the potential impact of treatment modality (i.e., in-person treatment) that was only assigned during the non-peak-COVID-19 timeframe, we conducted a sub-analysis excluding the non-peak, in-person cases, resulting in 533 and 408 participants, in the peak and non-peak periods, respectively (Table 4).

Table 4

Table 4. Baseline characteristics categorized for telemedicine-only cohort by COVID phases, Peak and Non-peak groups (n=941).

In this analysis, a Chi-square test of independence revealed no significant relationship between participant retention during treatment and the COVID-19 periods (Peak: 98.12% vs. Non-peak: 97.30%, χ² = 0.712, p = 0.399). Additionally, this cohort exhibited no significant differences in the depression and anxiety symptom scores at baseline (EPDS: Peak: 15.85 vs. Non-peak: 15.73, p = 0.619; GAD-7: Peak: 11.70 vs. Non-peak: 11.98, p = 0.384) and 3-months post-randomization (EPDS: Peak: 9.09 vs. Non-peak: 9.22, p = 0.710; GAD-7: Peak: 6.42 vs. Non-peak: 6.45, p = 0.929) (Table 5).

Table 5

Table 5. Depressive and anxiety symptom scores at baseline and 3 months for telemedicine-only cohort and peak and non-peak subgroups (n=941).

Finally, there was no statistically significant difference in participant satisfaction (CSQ-8) scores between peak and non-peak periods for the telemedicine-only group (CSQ-8: 3.45 vs. 3.41, p = 0.349), and Linear mixed model analyses revealed no significant effect of the COVID period on patient satisfaction score (β = 0.05, p = 0.297), after controlling for relevant covariates. Only 816 patients out of the telemedicine cohort had at least one treatment session and completed CSQ. Data was imputed for individuals (n=71) who completed sessions but did not complete the CSQ.

Discussion

The COVID-19 pandemic provided a unique opportunity to understand if the high infection prevalence periods had a negative impact on the measured outcomes for SUMMIT participants. The current study used a mixed methods approach to compare patient engagement and treatment efficacy during peak and non-peak COVID-19 timeframes corresponding to the adaptations put in place to continue SUMMIT safely. Specifically, we examined treatment engagement via rates of participant enrollment, retention during therapy, as well as treatment satisfaction; assessed treatment outcomes via changes in depressive and anxiety symptom scores. We also examined perceived barriers and facilitators of care experiences among a subset of interviewed participants. Our findings contribute to our understanding of how healthcare utilization and satisfaction with treatment may vary during public health crises and underscore the importance of adaptive care delivery models in response to unanticipated public health disruptions.

Telemedicine in times of crisis: maintaining engagement during COVID-19

Enrollment rates in the SUMMIT Trial were similar and stable when comparing peak and non-peak periods. This suggests that the pandemic’s peak did not deter individuals from seeking care once the remote accommodation was put in place. This finding contrasts with other studies reporting challenges in enrollment during the pandemic (31, 32). A global review of clinical trial activity across 37 countries reported a 25-50% decline in in-person enrollment across multiple therapeutic areas between 2019 and 2021 (33). In mental healthcare, telemedicine may help address such challenges by offering a safer and more convenient alternative to in-person care, particularly during times of public health disruption (34). This modality is especially relevant in the context of low enrollment among racial and ethnic minority populations in clinical trials during the pandemic, which coincided with a disproportionate burden of COVID-19-related health and social impacts in these communities (35, 36). Notably, modality adaptations to treatment delivery have been linked to equally high satisfaction rates among racial and ethnic minorities compared to White participants (15). Expanding access to telemedicine-based services may mitigate some barriers by increasing flexibility and reducing logistical and systemic obstacles to participation and care.

Retention rates during treatment in the SUMMIT Trial also remained stable, with no significant differences observed between peak and non-peak periods. This suggests consistent participant engagement once treatment commenced, despite external stressors associated with the pandemic. These findings align with existing literature, which highlights an increased preference for telemedicine among providers and patients during the pandemic (37) and thereafter (38). This growing acceptance of telehealth practices has been associated with higher attendance rates and fewer cancellations across various healthcare domains (39, 40), including among perinatal populations. Similarly, other clinical trials conducted during the COVID-19 pandemic achieved high retention rates, often exceeding 80%, by implementing telemedicine and online health interventions—rates comparable to or higher than those observed in pre-pandemic, in-person trials (41–44). Although there were no observable differences in retention across peak and non-peak periods, there were sociodemographic differences, particularly that peak period participants were more likely to be younger and identify as White than non-peak participants. The difference in age between peak and non-peak participants was minimal (approximately 1 year), and the statistical significance is likely attributable to the large sample size rather than a clinically meaningful difference. Although speculative, differences in the racial makeup of peak vs non-peak enrollment could be related to a few different factors: participants identifying as racial or ethnic minorities may have been more likely to live with extended family (45) and therefore have less private space for virtual visits, be less reliant on outside childcare when school was conducted virtually and therefore have less time to enroll in a research study, and/or have differences in the ability and sources to work virtually (46, 47) that may have impacted enrollment during the telemedicine only periods.

No significant differences in participant satisfaction were observed across COVID-19 periods, with satisfaction remaining consistently high throughout the pandemic. This finding aligns with existing research suggesting that telemedicine-delivered care effectively maintains high levels of treatment satisfaction, upwards of 80% (48). The high satisfaction rates observed in the SUMMIT Trial may also be attributed to the implementation of behavioral activation that was also culturally sensitive, which aimed to strengthen the therapeutic relationship by incorporating the requisite humility to facilitate discussions about race, culture, and ethnicity (15). These findings reflect the SUMMIT model’s patient-centered orientation, which prioritized personalized care. The emphasis on empathy, flexibility, and cultural humility makes participants feel heard and respected.

Exploring treatment outcomes and participant experiences during COVID-19

No statistically significant differences in baseline as well as 3-month measures of depressive or anxiety symptoms were observed in either cohort. These stable symptom trajectories may be attributable to the high and uniform treatment fidelity (5) achieved across modalities (Telemedicine and In-person) and provider (Non-specialist vs. Specialist) groups (as shown in our previous publication), which likely helped maintain intervention effectiveness under pandemic-related stressors. However, these findings diverge from previous literature suggesting that the COVID-19 pandemic exacerbated the prevalence of perinatal depression and anxiety symptoms up to 24.9% and 32.8% respectively (49). This discrepancy may reflect the effectiveness of proactive and supportive care models like SUMMIT (5) in buffering against the adverse mental health effects typically associated with the pandemic. Participants in this study may have benefited from early intervention, consistent access to care via telemedicine, and providers trained in delivering empathetic, evidence-based interventions—all of which could mitigate symptom exacerbation. Furthermore, the structured support offered through SUMMIT may have fostered a sense of continuity and safety during an otherwise unpredictable period, potentially offsetting broader population-level trends reported in other studies. Thus, care models that prioritize provider well-being, patient well-being, and the integration of telemedicine represent promising strategies for expanding access to care in both pandemic and non-pandemic contexts (50–52).

The study’s qualitative data confirmed that SUMMIT provided needed support and a satisfying therapeutic experience to participants, independent of the pandemic. Participants from both peak and non-peak periods identified similar facilitators to treatment delivery, including empathetic and non-judgmental care, compatibility with their provider, effective delivery of BA therapy, flexibility in scheduling, active listening, and thoughtful feedback. Non-peak participants more frequently endorsed these facilitators, potentially reflecting improvements in provider experience and the refinement of telemedicine practices over time. This interpretation aligns with existing literature, which suggests that effective treatment delivery may improve as providers gain technological competence and experience with remote care (53). Barriers to treatment delivery were reported less frequently than facilitators, and types of barriers were consistently reported across both periods. This trend mirrors research that indicates barriers to treatment delivery during the pandemic were less frequently reported than facilitators, while the types of barriers and facilitators remained consistent (54–56).

Strengths and limitations

This study has several key strengths. First, our participant sample included varied racial and ethnic representation, enhancing our findings’ generalizability (57). This is particularly important given the disproportionate impact of the COVID-19 pandemic on racial and ethnic minorities (36, 58) and the historic lack of effective engagement of treatment studies with these populations (59). Second, including in-person participants in our analyses mitigated the risk of selection bias and provided critical context for understanding enrollment patterns across the various phases of the pandemic. Finally, combining quantitative assessments with qualitative data strengthened our understanding of the participants’ experience of receiving mental health care during the pandemic (60).

Our study also has limitations. First, participants who enrolled but did not receive any treatment were unavailable for qualitative interviews. Thus, we cannot determine if the dropout was specific to the pandemic which could modify our conclusions. Second, no analyses were done to account for the potential variation in treatment outcomes between earlier and later peak and non-peak COVID periods. It is possible that improvements in outcomes over time may have reflected increased familiarity with telemedicine, refined clinical workflows, or greater comfort among both providers and participants. Future studies should aim to include potential confounders such as seasonal variations, social support availability, shifting medical and safety concerns and their impact on the study outcomes. These variables were not collected for this study. The studies should also include perspectives from individuals who enrolled but did not initiate treatment to better understand barriers to engagement, particularly in the context of external stressors like a pandemic. Additionally, the temporal trends in treatment outcomes across different phases of the COVID were not assessed due to small sample sizes across various phases, which would not have yielded meaningful results (61). However, future research should explore this to ensure more comprehensive estimates of treatment outcomes. This could help disentangle the effects of evolving contextual factors—such as increasing telehealth proficiency, shifting public health guidelines, and changing participant attitudes—on both engagement and clinical effectiveness.

Conclusions

Adjustment of the SUMMIT trial protocol to the telemedicine only condition in response to the COVID-19 pandemic allowed trial recruitment and treatment to continue during times of peak infection. We observed that clinical engagement (as measured by rates of participant enrollment, retention, and satisfaction), efficacy (as measured by depressive and anxiety symptom scores), and positive participant experiences was preserved throughout the peak and non-peak pandemic timeframes. This study provides key insights into maintaining mental health treatment during public health emergencies that otherwise prevent in-person care.

Data availability statement

The datasets generated and/or analyzed during the current study are not publicly available to allow the study team to collect long-term study outcomes and analyze and potentially publish the results of the secondary aims involving sustained outcomes at 12-months post-randomization as indicated in our detailed Statistical Analysis Plan, but are available from the corresponding author on reasonable request. Requests to access the datasets should be directed to Daisy Singla, ZGFpc3kuc2luZ2xhQHV0b3JvbnRvLmNh.

Ethics statement

The studies involving humans were approved by University of Toronto Research Ethics Board; University of North Carolina Institutional Review Board, Endeavor Health Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written or electronic informed consent via REDcap™ to participate in this study.

Author contributions

HAK: Conceptualization, Methodology, Project administration, Writing – original draft, Writing – review & editing. SS: Conceptualization, Data curation, Formal Analysis, Methodology, Validation, Writing – original draft, Writing – review & editing. JEG: Conceptualization, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing. NA: Conceptualization, Data curation, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing. LL: Writing – original draft, Writing – review & editing. ASL: Conceptualization, Project administration, Writing – original draft, Writing – review & editing. AA: Conceptualization, Data curation, Formal Analysis, Methodology, Validation, Writing – original draft, Writing – review & editing. TSB: Writing – original draft, Writing – review & editing. CD: Writing – original draft, Writing – review & editing. AG: Writing – original draft, Writing – review & editing. ZL: Project administration, Writing – original draft, Writing – review & editing. RGM: Writing – original draft, Writing – review & editing. SM: Writing – original draft, Writing – review & editing. SNV: Writing – original draft, Writing – review & editing. RKS: Writing – original draft, Writing – review & editing. DRS: Funding acquisition, Conceptualization, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declared that financial support was received for this work and/or its publication. SUMMIT is funded as a Pragmatic Clinical Study (PCS) by the Patient-Centered Outcomes Research Institute (PCORI), United States, PCS-2018C1–10621.

Acknowledgments

We thank our SUMMIT participants and treatment providers involved in this study.

Conflict of interest

SM-B receives research funding to UNC for clinical trials sponsored by Sage Therapeutics, Electromedical Products International, and Sirtsei Pharmaceuticals for clinical trials. She serves as a clinical advisor and professional corporation owner for Modern Health, a digital health company. She also serves as a scientific advisor to EmbarkNeuro and Seaport Therapeutics; SV reports royalties from UptoDate Inc for authorship of materials on depression and pregnancy.

The remaining author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2025.1686719/full#supplementary-material

References

1. Centers for Disease Control and Prevention. Emergency preparedness and response for reproductive health (2022). Available online at: https://www.cdc.gov/reproductive-health/emergency-preparation-response/index.html (Accessed March 6, 2025).

Google Scholar

2. Wu Y, Zhang C, Liu H, Duan C, Li C, Fan J, et al. Perinatal depressive and anxiety symptoms of pregnant women during the coronavirus disease 2019 outbreak in China. Am J Obstet Gynecol. (2020) 223:240.e241–240.e249. doi: 10.1016/j.ajog.2020.05.009

PubMed Abstract | Crossref Full Text | Google Scholar

3. WHO. COVID-19 pandemic triggers 25% increase in prevalence of anxiety and depression worldwide (2022). Available online at: https://www.who.int/news/item/02-03-2022-covid-19-pandemic-triggers-25-increase-in-prevalence-of-anxiety-and-depression-worldwide (Accessed March 6, 2025).

Google Scholar

4. Singla DR, Meltzer-Brody SE, Silver RK, Vigod SN, Kim JJ, La Porte LM, et al. Scaling Up Maternal Mental healthcare by Increasing access to Treatment (SUMMIT) through non-specialist providers and telemedicine: a study protocol for a non-inferiority randomized controlled trial. Trials. (2021) 22:186. doi: 10.1186/s13063-021-05075-1

PubMed Abstract | Crossref Full Text | Google Scholar

5. Singla DR, Silver RK, Vigod SN, Schoueri-Mychasiw N, Kim JJ, La Porte LM, et al. Task-sharing and telemedicine delivery of psychotherapy to treat perinatal depression: a pragmatic, noninferiority randomized trial. Nat Med. (2025). doi: 10.1038/s41591-024-03482-w

PubMed Abstract | Crossref Full Text | Google Scholar

6. Ferreira DC, Vieira I, Pedro MI, Caldas P, and Varela M. Patient satisfaction with healthcare services and the techniques used for its assessment: A systematic literature review and a bibliometric analysis. Healthcare (Basel). (2023) 11:639. doi: 10.3390/healthcare11050639

PubMed Abstract | Crossref Full Text | Google Scholar

7. Robinson L, Parsons C, Northwood K, Siskind D, and McArdle P. Patient and clinician experience of using telehealth during the. Schizophr Bull Open. (2023) 4:sgad016. doi: 10.1093/schizbullopen/sgad016

PubMed Abstract | Crossref Full Text | Google Scholar

8. Song L, Qan’ir Y, Guan T, Guo P, Xu S, Jung A, et al. The challenges of enrollment and retention: A systematic review of psychosocial behavioral interventions for patients with cancer and their family caregivers. J Pain Symptom Manage. (2021) 62:e279–304. doi: 10.1016/j.jpainsymman.2021.04.019

PubMed Abstract | Crossref Full Text | Google Scholar

9. Singla DR, Kohrt BA, Murray LK, Anand A, Chorpita BF, and Patel V. Psychological treatments for the world: lessons from low- and middle-income countries. Annu Rev Clin Psychol. (2017) 13:149–81. doi: 10.1146/annurev-clinpsy-032816-045217

PubMed Abstract | Crossref Full Text | Google Scholar

10. Ekers D, Webster L, Van Straten A, Cuijpers P, Richards D, and Gilbody S. Behavioural activation for depression; an update of meta-analysis of effectiveness and subgroup analysis. PLoS One. (2014) 9. doi: 10.1371/journal.pone.0100100

PubMed Abstract | Crossref Full Text | Google Scholar

11. Etherton JL and Farley R. Behavioral activation for PTSD: A meta-analysis. Psychol Trauma. (2022) 14:894–901. doi: 10.1037/tra0000566

PubMed Abstract | Crossref Full Text | Google Scholar

12. Gros DF, Price M, Strachan M, Yuen EK, Milanak ME, and Acierno R. Behavioral activation and therapeutic exposure: an investigation of relative symptom changes in PTSD and depression during the course of integrated behavioral activation, situational exposure, and imaginal exposure techniques. Behav Modif. (2012) 36:580–99. doi: 10.1177/0145445512448097

PubMed Abstract | Crossref Full Text | Google Scholar

13. Stein AT, Carl E, Cuijpers P, Karyotaki E, and Smits JAJ. Looking beyond depression: a meta-analysis of the effect of behavioral activation on depression, anxiety, and activation. Psychol Med. (2021) 51:1491–504. doi: 10.1017/s0033291720000239

PubMed Abstract | Crossref Full Text | Google Scholar

14. Lehmann DC and Bördlein C. A systematic review of culturally adapted behavioral activation treatments for depression. Res Soc Work Pract. (2020) 30:688–702. doi: 10.1177/1049731520915635

Crossref Full Text | Google Scholar

15. Singla DR, Hossain S, Andrejek N, Cohen MJ, Dennis CL, Kim J, et al. Culturally sensitive psychotherapy for perinatal women: A mixed methods study. J Consult Clin Psychol. (2022) 90:770–86. doi: 10.1037/ccp0000754

PubMed Abstract | Crossref Full Text | Google Scholar

16. Dimidjian S, Barrera M, Martell C, Muñoz RF, and Lewinsohn PM. The origins and current status of behavioral activation treatments for depression. Annu Rev Clin Psychol. (2011) 7:1–38. doi: 10.1146/annurev-clinpsy-032210-104535

PubMed Abstract | Crossref Full Text | Google Scholar

17. Jacobson NS, Martell CR, and Dimidjian S. Behavioral activation treatment for depression: returning to contextual roots. Clin Psychology: Sci Pract. (2001) 8:255–70. doi: 10.1093/clipsy.8.3.255

Crossref Full Text | Google Scholar

18. Dimidjian S, Martell CR, Herman-Dunn R, and Hubley S. Behavioral activation for depression. In: Barlow DH, editor. Clinical handbook of psychological disorders: a step-by-step treatment manual, 6th ed. The Guilford Press, New York (NY (2021). p. 339–80.

Google Scholar

19. Santos MM, Nagy GA, Kanter JW, and López SR. Applying a process-oriented model of cultural competence to behavioral activation for depression. Cognit Behav Pract. (2021) 28:127–46. doi: 10.1016/j.cbpra.2020.11.007

PubMed Abstract | Crossref Full Text | Google Scholar

20. Singla DR, Hossain S, Ravitz P, Schiller EC, Andrejek N, Kim J, et al. Adapting behavioral activation for perinatal depression and anxiety in response to the COVID-19 pandemic and racial injustice. J Affect Disord. (2022) 299:180–7. doi: 10.1016/j.jad.2021.12.006

PubMed Abstract | Crossref Full Text | Google Scholar

21. Andrejek N, Hossain S, Schoueri-Mychasiw N, Saeed G, Zibaman M, Puerto Niño AK, et al. Barriers and facilitators to resuming in-person psychotherapy with perinatal patients amid the COVID-19 pandemic: A multistakeholder perspective. Int J Environ Res Public Health. (2021) 18:12234. doi: 10.3390/ijerph182212234

PubMed Abstract | Crossref Full Text | Google Scholar

22. Attkisson CC and Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. (1982) 5:233–7. doi: 10.1016/0149-7189(82)90074-x

PubMed Abstract | Crossref Full Text | Google Scholar

23. Dimidjian S, Goodman SH, Sherwood NE, Simon GE, Ludman E, Gallop R, et al. A pragmatic randomized clinical trial of behavioral activation for depressed pregnant women. J Consult Clin Psychol. (2017) 85:26–36. doi: 10.1037/ccp0000151

PubMed Abstract | Crossref Full Text | Google Scholar

24. Roberts ER, Clifford CA, and Rene MM. Assessing the client satisfaction questionnaire in English and Spanish. Hispanic J Behav Sci. (1984) 6:385–96. doi: 10.1177/07399863840064004

Crossref Full Text | Google Scholar

25. Cox JL, Holden JM, and Sagovsky R. Detection of postnatal depression. Development of the 10-item Edinburgh Postnatal Depression Scale. Br J Psychiatry. (1987) 150:782–6. doi: 10.1192/bjp.150.6.782

PubMed Abstract | Crossref Full Text | Google Scholar

26. Levis B, Negeri Z, Sun Y, Benedetti A, Thombs BD, and Group D. S. D. D. E. Accuracy of the Edinburgh Postnatal Depression Scale (EPDS) for screening to detect major depression among pregnant and postpartum women: systematic review and meta-analysis of individual participant data. BMJ. (2020) 371:m4022. doi: 10.1136/bmj.m4022

PubMed Abstract | Crossref Full Text | Google Scholar

27. Sultan P, Ando K, Elkhateb R, George RB, Lim G, Carvalho B, et al. Assessment of patient-reported outcome measures for maternal postpartum depression using the consensus-based standards for the selection of health measurement instruments guideline: A systematic review. JAMA Netw Open. (2022) 5:e2214885. doi: 10.1001/jamanetworkopen.2022.14885

PubMed Abstract | Crossref Full Text | Google Scholar

28. Kertz S, Bigda-Peyton J, and Bjorgvinsson T. Validity of the Generalized Anxiety Disorder-7 scale in an acute psychiatric sample. Clin Psychol Psychother. (2013) 20:456–64. doi: 10.1002/cpp.1802

PubMed Abstract | Crossref Full Text | Google Scholar

29. Spitzer RL, Kroenke K, Williams JB, and Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. (2006) 166:1092–7. doi: 10.1001/archinte.166.10.1092

PubMed Abstract | Crossref Full Text | Google Scholar

30. Singla DR, Lemberg-Pelly S, Lawson A, Zahedi N, Thomas-Jacques T, and Dennis CL. Implementing psychological interventions through nonspecialist providers and telemedicine in high-income countries: qualitative study from a multistakeholder perspective. JMIR Ment Health. (2020) 7:e19271. doi: 10.2196/19271

PubMed Abstract | Crossref Full Text | Google Scholar

31. Bradt J. Impact of COVID-19 on clinical research. Nordic J Music Ther. (2020) 29:297–-299. doi: 10.1080/08098131.2020.1777785

Crossref Full Text | Google Scholar

32. Fleming TR, Labriola D, and Wittes J. Conducting clinical research during the COVID-19 pandemic: protecting scientific integrity. JAMA. (2020) 324:33–4. doi: 10.1001/jama.2020.9286

PubMed Abstract | Crossref Full Text | Google Scholar

33. McDonald K, Seltzer E, Lu M, Gaisenband SD, Fletcher C, McLeroth P, et al. Quantifying the impact of the COVID-19 pandemic on clinical trial screening rates over time in 37 countries. Trials. (2023) 24:254. doi: 10.1186/s13063-023-07277-1

PubMed Abstract | Crossref Full Text | Google Scholar

34. Arafat MY, Zaman S, and Hawlader MDH. Telemedicine improves mental health in COVID-19 pandemic. J Glob Health. (2021) 11:3004. doi: 10.7189/jogh.11.03004

PubMed Abstract | Crossref Full Text | Google Scholar

35. Tai DBG, Sia IG, Doubeni CA, and Wieland ML. Disproportionate impact of COVID-19 on racial and ethnic minority groups in the United States: a 2021 update. J Racial Ethn Health Disparities. (2022) 9:2334–9. doi: 10.1007/s40615-021-01170-w

PubMed Abstract | Crossref Full Text | Google Scholar

36. Xiao H, Vaidya R, Liu F, Chang X, Xia X, and Unger JM. Sex, racial, and ethnic representation in COVID-19 clinical trials: A systematic review and meta-analysis. JAMA Intern Med. (2023) 183:50–60. doi: 10.1001/jamainternmed.2022.5600

PubMed Abstract | Crossref Full Text | Google Scholar

37. Waite MR, Diab S, and Adefisoye J. Virtual behavioral health treatment satisfaction and outcomes across time. J Patient Cent Res Rev. (2022) 9:158–65. doi: 10.17294/2330-0698.1918

PubMed Abstract | Crossref Full Text | Google Scholar

38. Kissler K, Thumm EB, Smith DC, Anderson JL, Wood RE, Johnson R, et al. Perinatal telehealth: meeting patients where they are. J Midwifery Womens Health. (2024) 69:9–16. doi: 10.1111/jmwh.13560

PubMed Abstract | Crossref Full Text | Google Scholar

39. Cutchin GM, Shelly S, Petty B, van Leer E, Tripp RM, Klein AM, et al. A comparison of voice therapy attendance rates between in-person and telepractice. Am J Speech Lang Pathol. (2023) 32:1154–64. doi: 10.1044/2022_AJSLP-22-00113

PubMed Abstract | Crossref Full Text | Google Scholar

40. Tajan N, Devès M, and Potier R. Tele-psychotherapy during the COVID-19 pandemic: a mini-review. Front Psychiatry. (2023) 14:1060961. doi: 10.3389/fpsyt.2023.1060961

PubMed Abstract | Crossref Full Text | Google Scholar

41. Brown SE, Miller M, Darring J, Inamdar K, Salgaonkar A, Burnsed JC, et al. Impact of the COVID-19 pandemic on a clinical trial: A quantitative report on study engagement and strategies. Pediatr Phys Ther. (2023) 35:439–48. doi: 10.1097/PEP.0000000000001036

PubMed Abstract | Crossref Full Text | Google Scholar

42. Kaur J, Mania I, Tirupathi R, and Polavarapu L. Impact of telemedicine on retention in medications for opioid use disorder (MOUD) treatment with buprenorphine in the times of COVID-19 pandemic: A retrospective chart review. J Rural Ment Health. (2022) 46. doi: 10.1037/rmh0000206

Crossref Full Text | Google Scholar

43. Knowlson C, Tharmanathan P, Arundel C, James S, Flett L, Gascoyne S, et al. Can learnings from the COVID-19 pandemic improve trial conduct post-pandemic? A case study of strategies used by the DISC trial. Res Methods Med Health Sci. (2023) 4:50–60. doi: 10.1177/26320843221128296

PubMed Abstract | Crossref Full Text | Google Scholar

44. Ma C, Adler RH, Neidre DB, Chen RC, Northouse LL, Rini C, et al. Challenges and approaches to recruitment for and retention in a dyad-focused eHealth intervention during COVID-19: randomized controlled trial. J Med Internet Res. (2024) 26:e51877. doi: 10.2196/51877

PubMed Abstract | Crossref Full Text | Google Scholar

45. Lofquist D. (2013). Multigenerational households, in: Paper presented at the Annual Meeting of the American Sociological Association, New York, U.S.A.

Google Scholar

46. Ehmer AC, Scott SM, Smith H, and Ashby BD. Connecting during COVID: The application of teleservices in two integrated perinatal settings. Infant Ment Health J. (2022) 43:127–39. doi: 10.1002/imhj.21958

PubMed Abstract | Crossref Full Text | Google Scholar

47. Shang D, Williams C, and Culiqi H. Telehealth uptake among hispanic people during COVID-19: retrospective observational study. JMIR Med Inf. (2024) 12:e57717. doi: 10.2196/57717

PubMed Abstract | Crossref Full Text | Google Scholar

48. Crawford J, Haffar S, Fernando S, Stephens H, Harvey SB, and Black M. Client perspectives: Telehealth for mental health services. Australas Psychiatry. (2025) 33:96–102. doi: 10.1177/10398562241270986

PubMed Abstract | Crossref Full Text | Google Scholar

49. Delanerolle G, McCauley M, Hirsch M, Zeng Y, Cong X, Cavalini H, et al. The prevalence of mental ill-health in women during pregnancy and after childbirth during the Covid-19 pandemic: a systematic review and Meta-analysis. BMC Pregnancy Childbirth. (2023) 23:76. doi: 10.1186/s12884-022-05243-4

PubMed Abstract | Crossref Full Text | Google Scholar

50. Almuslim H and AlDossary S. Models of incorporating telehealth into obstetric care during the COVID-19 pandemic, its benefits and barriers: A scoping review. Telemed J E Health. (2022) 28:24–38. doi: 10.1089/tmj.2020.0553

PubMed Abstract | Crossref Full Text | Google Scholar

51. Pierce BS, Perrin PB, Tyler CM, McKee GB, and Watson JD. The COVID-19 telepsychology revolution: A national study of pandemic-based changes in U.S. mental health care delivery. Am Psychol. (2021) 76:14–25. doi: 10.1037/amp0000722

PubMed Abstract | Crossref Full Text | Google Scholar

52. Wilson AN, Ravaldi C, Scoullar MJL, Vogel JP, Szabo RA, Fisher JRW, et al. Caring for the carers: Ensuring the provision of quality maternity care during a global pandemic. Women Birth. (2021) 34:206–9. doi: 10.1016/j.wombi.2020.03.011

PubMed Abstract | Crossref Full Text | Google Scholar

53. Galvin E, Desselle S, Gavin B, Quigley E, Flear M, Kilbride K, et al. Stakeholder perspectives and experiences of the implementation of remote mental health consultations during the COVID-19 pandemic: a qualitative study. BMC Health Serv Res. (2023) 23:623. doi: 10.1186/s12913-023-09529-x

PubMed Abstract | Crossref Full Text | Google Scholar

54. Amiri P, Nadri H, and Bahaadinbeigy K. Facilitators and barriers of mHealth interventions during the Covid-19 pandemic: systematic review. BMC Health Serv Res. (2023) 23:1176. doi: 10.1186/s12913-023-10171-w

PubMed Abstract | Crossref Full Text | Google Scholar

55. Gonzalez G, Whiting-Green W, Danos D, Singh S, Witmeier K, and Philippi S. Barriers and facilitators to delivering tele-mental health services to children and families: 2022 follow-up survey results of Louisiana medicaid providers after rapid telehealth implementation during COVID-19. J Technol Behav Sci. (2024) 10:528–39. doi: 10.1007/s41347-024-00458-y

Crossref Full Text | Google Scholar

56. Spagnolo J, Beauséjour M, Fleury MJ, Clément JF, Gamache C, Sauvé C, et al. Perceptions on barriers, facilitators, and recommendations related to mental health service delivery during the COVID-19 pandemic in Quebec, Canada: a qualitative descriptive study. BMC Prim Care. (2022) 23:32. doi: 10.1186/s12875-022-01634-w

PubMed Abstract | Crossref Full Text | Google Scholar

57. Sharghi S, Khalatbari S, Laird A, Lapidus J, Enders FT, Meinzen-Derr J, et al. Race, ethnicity, and considerations for data collection and analysis in research studies. J Clin Transl Sci. (2024) 8:e182. doi: 10.1017/cts.2024.632

PubMed Abstract | Crossref Full Text | Google Scholar

58. Magesh S, John D, Li WT, Li Y, Mattingly-App A, Jain S, et al. Disparities in COVID-19 outcomes by race, ethnicity, and socioeconomic status: A systematic-review and meta-analysis. JAMA Netw Open. (2021) 4:e2134147. doi: 10.1001/jamanetworkopen.2021.34147

PubMed Abstract | Crossref Full Text | Google Scholar

59. Lam PC, Simpson D, John DA, Rodriguez M, Bridgman-Packer D, Cruz AG, et al. Differential engagement by race/ethnicity in experimental trials of mental health treatment interventions: A systematic review. J Clin Psychiatry. (2022) 83:21r14343. doi: 10.4088/JCP.21r14343

PubMed Abstract | Crossref Full Text | Google Scholar

60. Lund T. Combining qualitative and quantitative approaches: some arguments for mixed methods research. Scandinavian J Educ Res. (2012) 56:155–-165. doi: 10.1080/00313831.2011.568674

Crossref Full Text | Google Scholar

61. Rusticus SA and Lovato CY. Impact of sample size and variability on the power and type I error rates of equivalence tests: A simulation study. Pract Assessment Research Eval. (2014) 19.

Google Scholar