Exploring peer education for migrant informal caregivers of mentally ill loved ones: a realist evaluation protocol

Introduction: Informal caregivers with a migration background who care for someone with a mental illness often experience elevated caregiver burdens due to factors such as discrimination, language barriers, and stigma. In The Netherlands, a peer education intervention called ‘They Are not Mental?!’ (TANM) addresses these challenges by reducing stigma, increasing help-seeking behaviors, and improving access to healthcare. This transdisciplinary study evaluates how contextual factors and mechanisms influence its outcomes.

Description: This research protocol outlines a realist evaluation of the intervention using a mixed-methods design, including interviews, post-questionnaires, and observations. The study will develop and refine the program theory to determine for whom, in what contexts, why, and how the intervention works.

Discussion: This protocol shows how we plan to investigate how, why, for whom and under what circumstances TANM produces its intended outcomes, using a realist evaluation approach. This approach is well-suited for evaluating complex interventions because it accommodates for dynamic and iterative interventions. Its focus is on understanding patterns and mechanisms within specific contexts, using program theories.

Conclusion: The evaluation of TANM will inform future intervention adaptations and guide future efforts to support vulnerable populations, particularly migrants navigating caregiving challenges.

1 Introduction

Informal caregivers – often family members or friends who provide unpaid care to loved ones – play an important role in supporting individuals with a diagnosis of a mental health condition (1, 2). For caregivers with a migration background, meaning those who have moved to a new country and are in a situation in which they might be less familiar with local customs, norms, or care system, this responsibility often comes with additional challenges and increased stress, due to stigmatizing beliefs regarding mental illness, language and cultural barriers and limited access to support services (3–5). These caregivers face a “double adaptation burden,” with stress intensified by their caregiving role and experiences as migrants. To alleviate this burden, providing appropriate support is important (5, 6). In response, the culturally sensitive intervention “Ze Zijn Toch Niet Gek?!” (translation: “They Are not Mental”; hereafter TANM) was developed to support informal caregivers with a migration background who care for a loved one with a mental illness. This paper presents the protocol for a planned realist evaluation of TANM, aiming to develop a theoretical framework which explains the mechanisms through which support programs like TANM produce their desired outcomes and gain insights for its improvement.

Members of migrant communities are often at increased psychological health risks, due to challenges such as adapting to a new culture and encountering discrimination. As such, informal caregivers play an important role in supporting those affected (4, 7, 8). Caregiving for someone with a mental illness can be stressful and difficult for all informal caregivers, irrespective of migration status, but caregivers with a migration background face additional challenges that can intensify this caregiver burden (1, 3, 9, 10). These challenges include stigma, which is a general phenomenon across societies and can be influenced by culture-specific explanatory models – such as viewing it as a result of black magic, spirits, punishment of a divine entity or higher power, or the evil eye. Stigma can foster feelings of shame within families and discourage both patients and caregivers from seeking help (3, 4, 11–15). Additionally, cultural expectations around family caregiving, including gendered norms and ideas about family obligations, can further increase caregiver burden (3, 16, 17).

These cultural taboos, combined with systemic barriers like language differences and cultural misunderstandings, can make it difficult for informal caregivers and their loved ones to acknowledge the need for support, limiting the access to formal care (5). These intersecting challenges make caregiving for individuals with mental illness particularly difficult within migrant communities.

One way to address barriers in accessibility is through integrated care, which focuses on breaking down organizational boundaries and aligning services to better meet the needs of individuals, rather than being shaped by what each organization provides separately (18). Integrated care requires collaboration between healthcare, social and community services (18, 19), and ideally, the involvement of the target group. By making care systems more navigable and tailored to the target group’s needs, integrated care is a promising way to empower vulnerable individuals (20). This is particularly relevant for informal caregivers with a migration background, who often face additional challenges in navigating healthcare systems and accessing support (5). For caregivers of individuals with mental illnesses, tailored interventions are important for addressing their physical, psychological, and social needs. Through collaboration with relevant services, integrated care can better meet these needs and enhance the overall quality of psychosocial care by supporting these informal caregivers, who may experience additional barriers due to language, cultural differences, or unfamiliarity with available services (5, 21–23).

To address the challenges faced by informal caregivers with a migration background who care for a loved one with a suspected mental illness, TANM was developed as a peer education program. A key aim of TANM is to enhances caregivers’ resilience – the dynamic process of adapting to adversity, which helps buffer against caregiver burden (24–26) - through improved access to information, reduced stigma surrounding mental health, and stronger connections between migrant families and the healthcare system. Additionally, TANM enhances coordination between community-based support and healthcare organizations, by incorporating elements of integrated care. A detailed description of the TANM, including its structure and implementation, is provided in Section 2.

Since its development in 2014, TANM has undergone slight adjustments, including changes in funding and target groups. Its evolving nature requires an evaluation method that can adapt to these changes. A realist evaluation (RE) design was chosen because it provides a framework not only to assess whether the intervention works, but also to explore why, for whom, and under what circumstances it works. This approach is well-suited for an exploration to understand how underlying contexts and mechanisms – such as stigma and taboos – impact the intended outcomes of TANM (27). Given the complexity of these factors, this research takes a transdisciplinary approach, involving collaboration with the intervention’s coordinating organizations to ensure that different perspectives are integrated into the evaluation process circumstances (27–29). This research aims to evaluate TANM using a RE approach, in order to develop a program theory that explores how contexts and mechanisms, such as stigma and taboos, impact its intended outcomes. The research will answer the following questions:

‘Did intervention participants, ambassadors and coordinators perceive changes in its short-term outcomes related to discussing mental health taboos, understanding mental illnesses and caregiver support, and their perception of the healthcare sector? How did the intervention components and contextual factors contribute to this change?’

Considering the goals of TANM, we expect these changes to have a positive direction; that is, we anticipate that participants will discuss mental health taboos with their friends and families, develop a better understanding of mental illnesses and caregiver support, and hold a more positive perception of the healthcare sector.

In the following sections, we describe key components of our realist evaluation design, including the program theory, data collection methods, and planned analyses.

2 Intervention description

TANM is a peer-facilitated intervention aimed at strengthening the resilience of informal caregivers with a migration background who support a loved one with a suspected mental illness in the second largest city of the Netherlands, Rotterdam. Developed collaboratively by a community empowerment organization, a mental health prevention agency and a support association for families affected by mental health issues, TANM is a 3-session program that aims to increase resilience in the long-term, by (1) addressing mental health taboos (2); improving access to information and care and (3) fostering trust between migrant communities and the healthcare sector. Since the long-term goal of increasing caregivers’ resilience extends beyond the scope of the current research, we will focus on evaluating its short-term objectives, which include: encouraging open discussions about mental health within migrant communities, improving understanding of mental illnesses and available support for caregivers, and fostering trust between migrant families and the healthcare sector to reduce barriers to professional support.

TANM relies on peer educators, referred to as ambassadors, who share similar lived experiences with the participants, particularly through their migration background and, for some ambassadors, their caregiving role. Peer education is an approach to engage ‘hard-to-reach’ populations, such as those with limited resources, language barriers or distrust in the healthcare system. Peers serve as intermediaries who can leverage cultural understandings and trust to identify participants and adapt the intervention’s messages to meet the group’s needs (30–32).

Within TANM, ambassadors are volunteers who are active community members, often demonstrated through a strong social network or prior volunteer experience in their neighborhoods, and they recognize the importance of discussing mental health. Ambassadors are invited through outreach by the community empowerment organization, as well as through referrals from ambassadors of previous years. They are selected based on the following criteria:

• Affinity with and access to target groups;

• Willingness to share personal experiences;

• Ability to lead discussions and educate others;

• Ability to establish contact with healthcare professionals and translate the needs and concerns of the target group to these professionals.

Once selected, ambassadors receive specialized training, covering three main areas: (1) basic knowledge of mental health conditions, such as depression, psychosis and schizophrenia, which includes symptoms, causes, treatment options and the effects of medication and substance use; (2) informal caregiving, including definitions, perceptions, signs of caregiver burden, and available support services; and (3) practical skills in peer education, such as presentation techniques and an introduction of the content of the peer education sessions. The training is delivered in the Dutch language by a prevention specialist with a background in psychology, a Family Experience Expert (FEE; explained in Section 2.1 about Phase 3) and a caregiver support coach affiliated with the local caregiver support service, which has been part of TANMs coordinating organizations since 2023. The training is highly interactive and fosters a culturally sensitive approach by encouraging ambassadors to share personal experiences and explanatory models they are familiar with. A key message throughout the training is that differing perspectives on mental health conditions can coexist, and that creating space for these perspectives during the peer education sessions is essential.

After completing the training, ambassadors recruit participants for their groups and lead the intervention’s peer education sessions in the dominant language spoken within their group. This could be Dutch, but also, for example, Arabic or Turkish. Ongoing support is provided to the ambassadors in two ways, through regular feedback meetings with members of the coordinating organizations and fellow ambassadors, and through direct access to a designated contact person within the coordinating organizations who is available for individual questions, support or guidance through phone, messaging or in-person contact.

Our evaluation will focus on the current round of TANM (September 2024 – March 2025), which is co-organized by four partners, following the addition of a helpline and support service for caregivers in Rotterdam in 2023.

2.1 Intervention structure and content

The intervention is structured into three main phases (see Figure 1 for an overview): the preparation phase, the peer education sessions, and the follow-up training and home visits phase.

Figure 1

Figure 1. Phases of the intervention TANM.

2.1.1 Phase 1 – preparation

The organization of TANM recruits ambassadors – i.e. key figures from various communities in Rotterdam. Ambassadors undergo training to become peer educators. This training equips them with the knowledge and skills needed to identify symptoms of mental illness, facilitate peer education sessions, and recruit participants for Phase 2.

2.1.2 Phase 2 - peer education sessions

Peer educators facilitate three structured sessions, focusing on topics such as understanding mental health issues, the role and challenges of informal caregiving, and available support options. Detailed session topics and objectives are outlined in Appendix I. Ambassadors identify participating informal caregivers who may need additional professional support and refer them Phase 3.

2.1.3 Phase 3 - follow-up training and home visits

This optional phase provides tailored support to meet the needs of the identified informal caregivers. For each group, a member of one of TANM’s coordinating organizations, together with the ambassador, conducts a needs assessment to determine the most appropriate follow-up support for the identified caregivers. This may include five to six training sessions with a psychologist for a group of identified informal caregivers, which are based on the training ‘Psychological vulnerability in your immediate environment’ (33) but adapted to fit the unique needs of migrant caregivers. Alternatively, one-on-one consultations by a Family Experience Expert (FEE) for an individual caregiver may be offered. The FEE is someone who has personal experience supporting a relative with mental health issues and has received specialized training, enabling them to reflect on their own experiences, develop communication skills, and establish professional boundaries. Phase 3 is flexible and not attended by all Phase 2 participants.

3 Methods

Realist evaluation (RE) is a theory-driven approach that provides a framework for evaluating interventions, policies, measures, and other complex social programs or systems. It examines what works, how, for whom, why and under which circumstances, considering the influence of factors like culture and socioeconomic status on outcomes (27, 28, 34, 35). The underlying idea behind REs is that outcomes are the product of multiple causes and depend on mechanisms and contextual elements (36, 37). Together, these form context-mechanism-outcome (CMO) configurations, which form the basis of the so-called program theory (37). There is no standard methodology for REs, but it is encouraged to use a mixed-methods approach, combining both quantitative and qualitative methods (35, 38). This flexibility makes RE particularly suited for evaluating complex interventions, like TANM (27, 28, 35). Our RE of TANM consists of both qualitative and quantitative methods and will follow the RE reporting standards as made clear by the RAMESES-II project (37).

The RE of TANM, combines these methods in four phases. The first phase, which has already been completed (see Figures 2, 3), focused on developing an initial program theory. This theory, presented in section 3.1, serves as the foundation for the remaining phases of our RE. This protocol outlines the steps for phases 2–4, involving: (1) interviews with participants, peer educators, and other stakeholders; (2) post-intervention questionnaires; and (3) observations of the peer education cycle sessions. This study is conducted by a research team with experience in intercultural research, including two researchers who have a migration background themselves. The following sections provide detailed descriptions of each phase and method (see Figure 2).

Figure 2

Figure 2. Four phases and corresponding methodologies of our RE approach.

Figure 3

Figure 3. Initial program theory of TANM.

The study protocol was approved by the Ethics Review Committee of the Department of Psychology, Education and Child Studies, Erasmus University Rotterdam (application number ETH2425-0079).

3.1 Phase 1: development of the initial program theory

The first phase of our RE involved developing an initial program theory, which explains which mechanisms caused what outcomes in what contexts (36). Our theory was based on a logic model, derived from available intervention documentation (e.g., ambassador training manual), six interviews with ambassadors and six interviews with caregivers who were part of the 2023 intervention round, and a focus group with the members of the coordinating organizations. Data were collected and analyzed in April–June 2024, and the initial program theory was drafted by one researcher (MH) and refined through iterative discussions with co-authors (PK, SD). The last draft was discussed with the project lead of TANM and then finalized. The initial program theory forms the basis of the further evaluation, for which the methodology is presented in this protocol. The theory is outlined in the next paragraph and in Figure 3.

In our analysis, we found nine CMO-configurations that fitted in four clustered themes, as can be seen in Figure 3. The first theme we found is related to the taboo on mental illness among migrant communities (pictured in blue). The second theme related to increasing accessibility to information, care, and support (pictured in green). The third theme related to bridging the gap between migrant families and the healthcare sector (pictured in yellow). Lastly, the fourth theme related to community empowerment and social capital (pictured in red). The nine CMO-configurations within these four themes serve as a starting point for the data collection and analysis in the following phases of the RE. For more details about our initial theory, see Supplementary material I where we shortly discuss the cluster themes and details of each CMO-configuration.

3.2 Phase 2: data collection for theory refinement

Phase 1 resulted in a conceptual model which will guide further empirical data collection aimed at testing and refining it – that is, adapting the program theory based on this new evidence (36). During the most recent intervention cycle, which started in September 2024, researchers will collect qualitative data (i.e., interviews, observations), quantitative data (i.e., post-intervention questionnaire) and logbook data (e.g., notes of meetings with TANM’s coordinating organizations and feedback meetings with peer educators). Recurring consultations with the coordinating organizations were held throughout the development of the research protocol. Data will be collected by a team of co-researchers with diverse cultural backgrounds, including Moroccan, Turkish, Indonesian, and Dutch. This equips the team with insights and skills to connect with participants in a culturally sensitive manner. Supplementary material I elaborates on the initial program theory, while Tables 1, 2 provide an overview of how each CMO-configuration is linked to at least one of the above-mentioned data sources. This ensures that all CMO-configurations are addressed through planned data collection and data analysis. With our approach, we will assess short-term and intermediate outcomes of the TANM intervention components and contextual factors.

Table 1

Table 1. Link between CMO-configurations and data sources.

Table 2

Table 2. Overview of data collection methods, CMO-configurations, sample size, frequency, and purpose in realist evaluation.

3.2.1 Interviews with ambassadors

The interviews with ambassadors aim to explore their experiences with TANM and will be guided by topics from our initial program theory. This enables exploration of contextual elements and mechanisms driving the intervention’s intended short-term outcomes. The outcomes are related to discussing taboos with friends and family, a better understanding of mental illnesses and caregiver support, a more positive perception of the healthcare system, and the ripple effect where knowledge is shared outside of the intervention group (39).

Table 1 provides an overview of all CMO-configurations, whereas Table 2 provides an overview of which CMO-configurations will be covered in the interviews with the ambassadors.

3.2.2 Interviews with participants

Like the interviews with the ambassadors, those with TANM’s participants aim to explore their experiences with TANM, focusing on how and which intervention components and contextual factors influence TANM’s observed and experienced short-term outcomes. These interviews will address the same CMO-configurations explored in the interviews with the ambassadors (see Tables 1, 2).

We will interview two types of participants; (1) Those identified as informal caregivers and who attended Phase 2, the peer educational cycle, and Phase 3, the tailored follow-up support; and (2) participants who only took part in Phase 2.

3.2.3 Observations

The observations aim to document how the peer educational sessions are delivered, with a focus on contextual factors and mechanisms. We will look at mechanisms, such as providing informational support and discussing individual experiences, and contexts, such as group dynamics and alternative explanatory models of mental illnesses. See Tables 1, 2 for a full overview of the related CMO-configurations.

Observations were conducted using a semi-structured observation guide, designed to capture context and mechanisms from the initial program theory (e.g., social support based on Cohen & Wills (40), aspects of peer education based on Voorham (32)). Specifically, we will focus on:

1. Social influences: References to cultural norms, values, and alternative explanations for mental illnesses; individual experiences shared by participants or ambassadors.

2. Social support: Types of social support observed and the ways in which they are expressed.

3. Role of the ambassador: Adaptations of content by the ambassador (e.g., translation or cultural tailoring); peer education aspects such as similarity and professionalism.

4. Group dynamics: Interactions, trust, inclusivity, dominance of specific members, and both verbal and nonverbal communication.

The guide includes space for additional notes and researcher reflections after observations (full guide is added in Supplementary material II).

The observations will take place during Phase 2 of TANM, with at least one session per intervention group observed by researchers who are fluent in the language spoken in each group. This ensures an accurate understanding of what is observed. Given the expected sensitivity of the topic and the potential caution of participants toward outsiders like researchers, we made efforts to minimize interference with the natural flow of the educational sessions. Accordingly, we schedule our observations to coincide with planned visits from TANM’s coordinating organizations’ representatives. As such, we observe all final sessions, as we are present to administer the questionnaire (see 3.2.4). Additionally, if an organizational visit is planned during the first or second peer education session, we join these visits, observing these sessions as well. This will total to at least 10 observations from the final peer education session, added by one to 10 observations from the first or second sessions. This will result in at least 10 observations of the final peer education sessions, with an additional one to 10 observations from the first or second sessions.

3.2.4 Questionnaire

The quantitative component of this study is of exploratory nature and involves the use of a post-intervention questionnaire conducted among participants of the peer education sessions. It aims to capture differences between participant groups, and it will collect insights into the context and mechanisms related to social support, perceived stigma, group composition, and group dynamics (see Tables 1, 2).

All Phase 2 participants are asked to complete the questionnaire at the end of the final third session. Based on advice of the coordinating organizations, we address practical challenges associated with the target group, such as expected language barriers, diverse backgrounds, and varying literacy levels, the questionnaire was simplified to a maximum B2-level in Dutch. We minimized the number of questions and made answer options straightforward (e.g., yes/sometimes/no) to avoid confusion.

At the end of the third session, the attending researcher distributes paper questionnaires and explains the study’s goal, questionnaire, and informed consent, and instructs participants to complete them individually. In groups where participants do not understand Dutch, a research assistant fluent in the group’s language translates each question aloud, one at a time, for the entire group. This approach allows participants to listen to the translation and fill in their responses individually.

Rather than relying on pre-validated questionnaires, the items were constructed based on theoretical concepts from our CMO-configurations. The questionnaire starts with demographic questions (i.e., age, gender, country of birth of the participant and their parents), and questions related to caregiving responsibilities (i.e., frequency and care recipient). It is followed by section on group composition (specifying familiarity and relationships with group members and ambassador prior to the intervention), social support mechanisms (inspired by Cohen and Wills (40)), and mental health taboos and group dynamics. Lastly, an open-ended section allows participants to share lessons or insights gained during the intervention. Table 3 provides an overview of the themes and items, with the full questionnaire available in Supplementary material III.

Table 3

Table 3. Overview of questionnaire themes, items and example questions.

3.2.5 Logbook

To document the study process and decisions made throughout, researcher MH makes notes in a logbook. This logbook contains notes of all attended meetings with TANM’s coordinating organizations as well as feedback meetings held with the ambassadors and representatives from the coordinating organizations. The notes capture key decisions during the study and intervention process, feedback received by the coordinating organizations and ambassadors, and personal reflections of the researcher. As such, these notes will provide context for interpreting the findings and aligning them with the program theory, contributing to a nuanced interpretation of the results and supporting the data triangulation process in Phase 3. The template of the logbook has been added to Supplementary material IV.

3.3 Phase 3: data analysis and theory refinement

The analysis process comprises method-specific analyses for each type of data collected, followed by an integrative synthesis to triangulate our findings and refine the initial program theory. Below, we describe each type of analyses, followed by a description of how we will triangulate the data and refine the program theory.

3.3.1 Qualitative data: interviews and observations

Audio recordings of interviews will be transcribed verbatim, and observation schemes, completed during fieldwork, will be digitalized. Additionally, the open-ended question from the post-intervention questionnaire will be transcribed and analyzed as part of the qualitative data.

Thematic analysis will be conducted using both deductive and inductive approaches. Deductive analysis is guided by the initial program theory, with a coding framework structured around the concepts within our CMO-configurations (see Figure 3). At the same time, inductive analysis allows new themes and mechanisms to emerge within the predefined categories of context, mechanisms, and outcomes (41).

The analysis will follow Braun and Clarke’s (42) six-step framework: starting with familiarization with the data through repeated reading, followed by systemic coding, theme development and refinement. Coding will be conducted in Atlas.ti [Version 24; (43)].

3.3.2 Quantitative data: questionnaire

Quantitative data from the post-intervention questionnaire will be analyzed using SPSS [Version 29.0.0.0; (44)] to examine differences across intervention groups and assess the potential influence of contextual factors on perceived mechanisms and outcomes. Based on the initial program theory, we expect differences between groups (i.e., whether participants knew each other or their ambassador beforehand and the cultural homogeneity of the group), which may influence social support and group dynamics (such as sharing personal experiences with caregiving and/or mental illness and perceptions of taboo). To explore these potential mechanisms within our program theory, we will conduct crosstabulations of these independent and dependent variables, supplemented by chi-square tests to assess the statistical significance of observed differences.

Based on previous studies on peer education with migrants [(e.g., 45, 46)], we estimate an effect size of 0.4. Using a power analysis in R (47) with the pwr package (48) we expect to need around 60 completed surveys to achieve sufficient power. Given that we anticipate collecting around 8–12 questionnaires per intervention group, totalling approximately 80–100 questionnaires, we expect to have adequate power for our questionnaire analysis.

Missing data will be included in the analyses as is, unless responses to other questions suggest that the omission can be reasonably filled in based on available information. In such cases, missing responses may be assessed using other relevant data from the same participant. All decisions regarding missing data will be reported transparently in the final analysis.

3.3.3 Data triangulation

The data triangulation process will integrate findings from the interviews, observations, and post-intervention questionnaires. Firstly, we will identify patterns where different findings from different data sources align, confirming the validity of specific CMO-configurations from our initial program theory. Discrepancies will be explored, by analyzing the inconsistencies between data sources to understand the variations in context, mechanisms, or outcomes.

The logbook that was kept by researcher MH will serve as a tool to help identify explanations of discrepancies between sources, by offering rationale behind actions and decisions, as well as personal reflections of the study process.

Inspired by Rees (49), a synthesis matrix will be developed to systematically map findings from all sources to the initial CMO-configurations. This matrix will be organized around the four overarching themes from our initial program theory (See Figure 3). Under each theme, the relevant CMO-configurations will be listed, along with associated concepts from the program theory. Data from the interviews, observations and post-intervention questionnaires will be aligned with these themes, allowing for identification of new or modified CMO-configurations, where applicable. This will ensure a structured approach to integrate qualitative and quantitative findings and provide an overview of how data supports or challenges the initial program theory.

3.3.4 Refinement of the program theory

Insights gained from the integrated data will guide the iterative refinement of the program theory. This process starts with evaluating the existing CMO-configurations of the initial program theory based on the triangulated evidence. If the findings indicate that certain configurations are only partially supported or not confirmed, they will be adjusted accordingly. Additionally, if the data reveals previously unconsidered mechanisms or contextual factors, new CMO-configurations will be developed and integrated into the program theory.

The first author (MH) leads the refinement process, while the co-authors provide feedback and discuss discrepancies. We will mitigate potential bias caused by our positionalities – i.e., our backgrounds and perspectives - through regular discussions within the research team. Additionally, throughout the project, regular meetings are held with the TANM’s coordinating organizations to discuss the progress and research design, review the preliminary results, and adjust accordingly.

3.4 Phase 4: final validation of the program theory and evaluative synthesis

The adjusted program theory (Phase 3) will be subjected to a validation process to ensure its relevance and credibility. The theory will be validated in a focus group with the members of t TANM’s coordinating organizations. Feedback and information received during the focus group could lead to one last round of refinement, before reaching the final program theory of TANM. Once the program theory is finalized, we can answer our research questions concerning the perceptions of a successful change in TANM’s short-term outcomes. The process is concluded with an evaluative synthesis of TANM, that assesses whether it achieves it outcomes and which factors influence its success.

4 Discussion

While previous studies have explored experiences of informal caregivers with a migration background [(e.g., 50, 51)] and the stigma surrounding mental illness within migrant communities [(e.g., 52–54)], this study will build on that foundation by evaluating an intervention that is aimed at supporting informal caregivers with a migration background who care for a loved one with a mental illness. This protocol shows how we plan to investigate how, why, for whom and under what circumstances TANM produces its intended outcomes.

This protocol employs a RE approach, which is suitable for evaluating interventions that are dynamic, culturally sensitive and context-dependent – like TANM. Unlike traditional randomized experiments, which may not capture the complexity or nuances of real-world settings, RE allows for the consideration of variations in context and experiences as integral to understanding outcomes (27, 55). Its flexibility helps to overcome challenges such as language barriers (35). This makes it a fitting approach for evaluating culturally sensitive interventions like TANM. RE extend qualitative insights to broader patterns by linking context, mechanisms and outcomes, making it more useful for informing policy and practice (27, 55) – which is ultimately the goal of our evaluation of TANM.

However, the flexibility of RE also presents challenges, such as the lack of clear methodological guidance (56). To address this, we employed a mixed-methods approach (35) and multiple data sources for triangulation. Additionally, while RE often uses arrows within program theories, they do not always establish universal causality. Depending on the selection of methods applied in a RE, causality is not necessarily implied. Instead of aiming for universally applicable explanations, REs develop so-called “middle-range” theory – explanations that are specific enough to be useful in practice but broad enough to apply beyond a single case. These theories link empirical findings to hypotheses about how an intervention works, helping to guide future research and program implementation, rather than providing definitive conclusions (56). This study’s program theory will serve as a foundation for future research on culturally sensitive caregiver interventions, such as longitudinal studies assessing long-term impacts or comparative research across different cultural contexts to explore their influence on outcomes.

Our RE approach aligns with its growing use in health system research, particularly in transdisciplinary collaborations. Similar to previous studies [(e.g., 57, 58)] we primarily employ qualitative and exploratory methods to evaluate TANM. By sharing the lessons learned from implementing this protocol, we aim to inform the design and evaluation of culturally tailored interventions in integrated healthcare, particularly in reaching underserved and hard-to-reach populations (5, 20).

4.1 Strengths and limitations

A key strength of this study is its use of multiple data sources for triangulation, adapted to participants’ cultural, linguistic and health literacy needs. Additionally, through collaboration with TANM’s coordinating organizations and ambassadors, the relevance and approach of our study is enhanced by including people with lived experiences of integrated care. We expect to have a suitable approach in our research. However, as an exploratory study, it cannot establish causality but rather provide insights for future research and practice (56).

Conducting research alongside intervention implementation also required pragmatic decisions. For instance, the post-intervention questionnaire was only available in Dutch, though spoken translations were provided in Turkish, Arabic, and Tamazight, when possible, either by a researcher or a bilingual participant willing to help. Additionally, recall bias may affect interviews conducted after the intervention. We mitigated this through real-time observations, which provided immediate insights to cross-verify participant accounts.

Cultural and language differences may also introduce biases, such as translation bias, where meaning is lost or altered when translating questions or responses (59); and interpretation bias, where differences in cultural norms or expression lead to the wrong interpretation of participants’ response (59, 60). While efforts were made to minimize these biases through matching researchers with participants (e.g., Turkish-speaking researchers for Turkish-speaking groups), this could either facilitate openness or lead participants to withhold information due to feelings of shame or stigma when the researcher was perceived as an insider of their community (61). To navigate these complexities, we incorporated team reflections to account for cultural nuances and researcher positionality.

5 Conclusion

This protocol outlines our approach for evaluating and refining TANM’s program theory. By integrating multiple data collection methods and applying both inductive and deductive analyses, we aim to understand the intervention’s mechanisms and whether participants, ambassadors, and coordinators perceive improvements in discussing mental health taboos, understanding mental illnesses and caregiver support, and fostering trust in the healthcare system. Beyond TANM, this study contributes to broader research, policy, and practice in integrated care, particularly in reaching vulnerable populations, including those with a migration background.

Ethics statement

The studies involving humans were approved by Ethics Review Committee of the Department of Psychology, Education and Child Studies of the Erasmus University Rotterdam (application number ETH2425-0079). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by each participant.

Author contributions

MH: Investigation, Writing – original draft, Visualization, Project administration, Methodology, Conceptualization. PK: Methodology, Supervision, Conceptualization, Writing – review & editing. ÖU-B: Methodology, Writing – review & editing. MS: Conceptualization, Writing – review & editing. SD: Methodology, Conceptualization, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This study was conducted by the collaborative expertise center Healthy’R, financially supported by Municipality Rotterdam.

Acknowledgments

We thank Indigo Preventie Rijnmond, Dona Daria Rotterdam, MIND Ypsilon, and Mantelfoon for the opportunity to collaborate and evaluate their intervention TANM. We also extend our gratitude to Shaema Feyzi, Nisrine Abbou, Haluk Arslan, Jalila Yachou and Puck Buis for their help and support during the data collection of the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2025.1623903/full#supplementary-material

References

1. Bremmers, LGM, Fabbricotti, IN, Gräler, ES, Uyl-de Groot, CA, and Hakkaart-van, RL. Assessing the impact of caregiving on informal caregivers of adults with a mental disorder in OECD countries: a systematic literature review of concepts and their respective questionnaires. PLoS One. (2022) 17:e0270278. doi: 10.1371/journal.pone.0270278

Crossref Full Text | Google Scholar

2. Zarzycki, M, Seddon, D, Bei, E, Dekel, R, and Morrison, V. How culture shapes informal caregiver motivations: a meta-ethnographic review. Qual Health Res. (2022) 32:1574–89. doi: 10.1177/10497323221110356

PubMed Abstract | Crossref Full Text | Google Scholar

3. Abdullah, T, and Brown, TL. Mental illness stigma and ethnocultural beliefs, values, and norms: an integrative review. Clin Psychol Rev. (2011) 31:934–48. doi: 10.1016/j.cpr.2011.05.003

PubMed Abstract | Crossref Full Text | Google Scholar

4. Bhugra, D, and Jones, P. Migration and mental illness. Adv Psychiatr Treat. (2001) 7:216–22. doi: 10.1192/apt.7.3.216

Crossref Full Text | Google Scholar

5. Wieringen, J, and Grondelle, N. Migrant caregivers: invisible, indispensable overburdened. Bijblijven, (2014), 30: 32–39. doi: 10.1007/s12414-014-0035-4

Crossref Full Text | Google Scholar

6. Knaifel, E. Immigrant caregivers: the double burden experience of immigrants caring for a family member with severe mental illness. Community Ment Health J. (2022) 58:606–17. doi: 10.1007/s10597-021-00863-0

PubMed Abstract | Crossref Full Text | Google Scholar

7. Kunst, AE, Mackenbach, JP, Lamkaddem, M, Rademakers, J, and Devillé, W. Overview and evaluation of results of scientific research into ethnic differences in health, health risks and health care use in the Netherlands. Erasmus MC. (2008). Available online at: https://www.zonmw.nl/sites/zonmw/files/typo3-migrated-files/overzicht_evaluatie_resultaten_onderzoek.pdf.

Google Scholar

8. Özgen, H, Karimi, F, and Braakman, M. Cultural diversity in mental health care. Tijdschr Psychiatr. (2022) 64:513–6.

PubMed Abstract | Google Scholar

9. Hastrup, LH, Van Den Berg, B, and Gyrd-Hansen, D. Do informal caregivers in mental illness feel more burdened? A comparative study of mental versus somatic illnesses. Scand J Public Health. (2011) 39:598–607. doi: 10.1177/1403494811414247

PubMed Abstract | Crossref Full Text | Google Scholar

10. Koopman, E, Heemskerk, M, Beek, AJ, and Coenen, P. Factors associated with caregiver burden among adult (19–64 years) informal caregivers – an analysis from Dutch municipal health service data. Health Soc Care Community. (2020) 28:1578–89. doi: 10.1111/hsc.12982

PubMed Abstract | Crossref Full Text | Google Scholar

11. Carballo, M, and Nerurkar, A. Migration, refugees, and health risks. Emerg Infect Dis. (2001) 7:556–60. doi: 10.3201/eid0707.017733

PubMed Abstract | Crossref Full Text | Google Scholar

12. Hoffer, C. Religious traditions, folk beliefs and religious healing methods in a multicultural Netherlands: implications for mental health care practice. In: Handboek Psychiatrie, Religie En Spiritualiteit, pp. 179–194. (2012).

Google Scholar

13. Liu, CH, Meeuwesen, L, van Wesel, F, and Ingleby, D. Why do ethnic Chinese in the Netherlands underutilize mental health care services? Evidence from a qualitative study. Transcult Psychiatry. (2015) 52:331–52. doi: 10.1177/1363461514557887

PubMed Abstract | Crossref Full Text | Google Scholar

14. Teunissen, E, Sherally, J, van den Muijsenbergh, M, Dowrick, C, van Weel-Baumgarten, E, and van Weel, C. Mental health problems of undocumented migrants (UMs) in the Netherlands: a qualitative exploration of help-seeking behaviour and experiences with primary care. BMJ Open. (2014) 4:e005738. doi: 10.1136/bmjopen-2014-005738

PubMed Abstract | Crossref Full Text | Google Scholar

15. Zolezzi, M, Alamri, M, Shaar, S, and Rainkie, D. Stigma associated with mental illness and its treatment in the Arab culture: a systematic review. Int J Soc Psychiatry. (2018) 64:597–609. doi: 10.1177/0020764018789200

PubMed Abstract | Crossref Full Text | Google Scholar

16. Su, JA, and Chang, CC. Association between family caregiver burden and affiliate stigma in the families of people with dementia. Int J Environ Res Public Health. (2020) 17:2772. doi: 10.3390/ijerph17082772

PubMed Abstract | Crossref Full Text | Google Scholar

17. Werner, P, Mittelman, MS, Goldstein, D, and Heinik, J. Family stigma and caregiver burden in Alzheimer’s disease. The Gerontologist. (2012) 52:89–97. doi: 10.1093/geront/gnr117

PubMed Abstract | Crossref Full Text | Google Scholar

18. Nies, H. Integrated care: concepts and background In: H Nies, editor. Integrated Services for Older People: A resource book for managers. Dublin: EHMA (2004)

Google Scholar

19. Claeys, A, Berdai-Chaouni, S, Tricas-Sauras, S, and De Donder, L. Barriers and facilitators in providing care for patients with a migration background. J Clin Nurs. (2023) 32:912–25. doi: 10.1111/jocn.16491

PubMed Abstract | Crossref Full Text | Google Scholar

20. Janssen, BM, Snoeren, MWC, Van Regenmortel, T, and Abma, TA. Working towards integrated community care for older people: empowering organisational features from a professional perspective. Health Policy. (2015) 119:1–8. doi: 10.1016/j.healthpol.2014.09.016

PubMed Abstract | Crossref Full Text | Google Scholar

21. Bighelli, I, Rodolico, A, García-Mieres, H, Pitschel-Walz, G, Hansen, WP, Schneider-Thoma, J, et al. Psychosocial and psychological interventions for relapse prevention in schizophrenia: a systematic review and network meta-analysis. Lancet Psychiatry. (2021) 8:969–80. doi: 10.1016/S2215-0366(21)00243-1

PubMed Abstract | Crossref Full Text | Google Scholar

22. Lopez Hartmann, M, Wens, J, Verhoeven, V, and Remmen, R. The effect of caregiver support interventions for informal caregivers of community-dwelling frail elderly: a systematic review. Int J Integr Care. (2012) 12:e133. doi: 10.5334/ijic.845

PubMed Abstract | Crossref Full Text | Google Scholar

23. Rexhaj, S, Martinez, D, Golay, P, Coloni-Terrapon, C, Monteiro, S, Buisson, L, et al. A randomized controlled trial of a targeted support program for informal caregivers in adult psychiatry. Front Psych. (2023) 14:1284096. doi: 10.3389/fpsyt.2023.1284096

PubMed Abstract | Crossref Full Text | Google Scholar

24. Cassidy, T, Giles, M, and McLaughlin, M. Benefit finding and resilience in child caregivers. Br J Health Psychol. (2014) 19:606–18. doi: 10.1111/bjhp.12059

PubMed Abstract | Crossref Full Text | Google Scholar

25. Mulud, ZA, and McCarthy, G. Caregiver burden among caregivers of individuals with severe mental illness: testing the moderation and mediation models of resilience. Arch Psychiatr Nurs. (2017) 31:24–30. doi: 10.1016/j.apnu.2016.07.019

PubMed Abstract | Crossref Full Text | Google Scholar

26. Scott, CB. Alzheimer’s disease caregiver burden: does resilience matter? J Hum Behav Soc Environ. (2013) 23:879–92. doi: 10.1080/10911359.2013.803451

Crossref Full Text | Google Scholar

27. Pawson, R, and Tilley, N. Realist Evaluation. London. (2004). Available online at: https://cnxus.org/wp-content/uploads/2022/04/RE_chapter.pdf (Accessed Aug 9, 2023).

Google Scholar

28. Jack, K. What is realist evaluation? Evid Based Nurs. (2022) 25:111–3. doi: 10.1136/ebnurs-2022-103608

PubMed Abstract | Crossref Full Text | Google Scholar

29. Stokols, D, Hall, K, and Vogel, A. Transdisciplinary public health: definitions, core characteristics, and strategies for success In: D Joshu and TD McBride, editors. Transdisciplinary public health: Research, methods, and practice. San Francisco: Jossey-Bass Publishers (2013). 3–30.

Google Scholar

30. Goldfinger, JZ, Arniella, G, Wylie-Rosett, J, and Horowitz, CR. Project HEAL: peer education leads to weight loss in Harlem. J Health Care Poor Underserved. (2008) 19:180–92. doi: 10.1353/hpu.2008.0016

PubMed Abstract | Crossref Full Text | Google Scholar

31. Kocken, P, Voorhan, T, Brandsma, J, and Swart, W. Effects of peer–led AIDS education aimed at Turkish and Moroccan male immigrants in the Netherlands: a randomized controlled evaluation study. Eur J Pub Health. (2001) 11:153–9. doi: 10.1093/eurpub/11.2.153

PubMed Abstract | Crossref Full Text | Google Scholar

32. Voorham, AJJ. Health promotion for and by the target group: Theoretical basis and evaluation among migrants and the elderly. [PhD thesis]. Maastricht University. (2003). Available online at: https://cris.maastrichtuniversity.nl/en/publications/f67ceb0d-8067-46e6-91cb-bd5d6af0a65f (Accessed Sep 17, 2024).

Google Scholar

33. Bransen, E. Psychological vulnerability in your immediate environment: Tools for loved ones. Bilthoven, RIVM: Trimbos-instituut. (2020). Available online at: https://glsb-rivm.api.dms.atabase.nl/files/9d2190b4-67ea-43e2-8284-83c6cd96db65/stream?expires=1736986715&signature=668107242ab29ef09967e884f79ab31f879d5a79f23391244c89b10f1b74b78d.

Google Scholar

34. Marchal, B, Van Belle, S, Olmen, J, Hoerée, T, and Kegels, G. Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation. (2012) 18:192–212. doi: 10.1177/1356389012442444

Crossref Full Text | Google Scholar

35. Mercer, T, and Lacey, P. A brief introduction to realist evaluation. London: Public Health England, 1–10. (2021).

Google Scholar

36. Astbury, B, and Leeuw, FL. Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. (2010) 31:363–81. doi: 10.1177/1098214010371972

Crossref Full Text | Google Scholar

37. Wong, G, Westhorp, G, Manzano, A, Greenhalgh, J, Jagosh, J, and Greenhalgh, T. RAMESES II reporting standards for realist evaluations. BMC Med. (2016) 14:96. doi: 10.1186/s12916-016-0643-1

PubMed Abstract | Crossref Full Text | Google Scholar

38. Anguera, MT, Blanco-Villaseñor, A, Losada, JL, Sánchez-Algarra, P, and Onwuegbuzie, AJ. Revisiting the difference between mixed methods and multimethods: is it all in the name? Qual Quant. (2018) 52:2757–70. doi: 10.1007/s11135-018-0700-2

Crossref Full Text | Google Scholar

39. Manzano, A. The craft of interviewing in realist evaluation. Evaluation. (2016) 22:342–60. doi: 10.1177/1356389016638615

Crossref Full Text | Google Scholar

40. Cohen, S, and Wills, TA. Stress, social support, and the buffering hypothesis. Psychol Bull. (1985) 98:310–57. doi: 10.1037/0033-2909.98.2.310

PubMed Abstract | Crossref Full Text | Google Scholar

41. Williams, M, and Moser, T. The art of coding and thematic exploration in qualitative research. Int Manage Rev. (2019) 15:45–55.

Google Scholar

42. Braun, V, and Clarke, V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3:77–101. doi: 10.1191/1478088706qp063oa

Crossref Full Text | Google Scholar

43. ATLAS.ti Scientific Software Development Gmb H. ATLAS.Ti mac (version 24) qualitative data analysis software. (2024). Available online at: https://atlasti.com.

Google Scholar

44. IBM Corp. IBM SPSS statistics for windows, version 29.0.2.0. Armonk, NY: IBM Corp (2023).

Google Scholar

45. Ho, KHM, Yang, C, Leung, AKY, Bressington, D, Chien, WT, Cheng, Q, et al. Peer support and mental health of migrant domestic workers: a scoping review. Int J Environ Res Public Health. (2022) 19:7617. doi: 10.3390/ijerph19137617

PubMed Abstract | Crossref Full Text | Google Scholar

46. Kocken, PL, van Zwanenburg, EJ, and de Hoop, T. Effects of health education for migrant females with psychosomatic complaints treated by general practitioners. Patient Educ Couns. (2008) 70:25–30. doi: 10.1016/j.pec.2007.09.016

Crossref Full Text | Google Scholar

47. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing (2022).

Google Scholar

48. Champely, S. Pwr: Basic functions for power analysis. (2020). Available online at: https://CRAN.R-project.org/package=pwr

Google Scholar

49. Rees, S. Moving on together: What makes a successful transition into adulthood for young disabled people (14–25 years of age)? PhD thesis, United Kingdom: Bangor University (2016).

Google Scholar

50. Hynek, K, Gotehus, A, Methi, F, Nes, RB, Skirbekk, V, and Hansen, T. Caregiving + migrant background = double jeopardy? Associations between caregiving and physical and psychological health according to migrant backgrounds in Norway. Int J Environ Res Public Health. (2023) 20:5800. doi: 10.3390/ijerph20105800

PubMed Abstract | Crossref Full Text | Google Scholar

51. Tsai, LP, and Barr, JA. The experiences and perceptions of immigrant informal caregivers about engaging with professional services in the host country: a scoping review. Int J Nurs Pract. (2024) 30:e13227. doi: 10.1111/ijn.13227

PubMed Abstract | Crossref Full Text | Google Scholar

52. Makowski, AC, and von dem Knesebeck, O. Depression stigma and migration – results of a survey from Germany. BMC Psychiatry. (2017) 17:381. doi: 10.1186/s12888-017-1549-y

PubMed Abstract | Crossref Full Text | Google Scholar

53. Pouille, A, De Ruysscher, C, Vander Laenen, F, and Vanderplasschen, W. “Watch out for the boogieman”: stigma and substance use recovery among migrants and ethnic minorities. Commun Appl Soc Psychol. (2023) 33:571–86. doi: 10.1002/casp.2657

Crossref Full Text | Google Scholar

54. Seven, ÜS, Stoll, M, Dubbert, D, Kohls, C, Werner, P, and Kalbe, E. Perception, attitudes, and experiences regarding mental health problems and web based mental health information amongst young people with and without migration background in Germany. A qualitative study. Int J Environ Res Public Health. (2020) 18:81. doi: 10.3390/ijerph18010081

PubMed Abstract | Crossref Full Text | Google Scholar

55. Mackenzie, M, Koshy, P, Leslie, W, Lean, M, and Hankey, C. Getting beyond outcomes: a realist approach to help understand the impact of a nutritional intervention during smoking cessation. Eur J Clin Nutr. (2009) 63:1136–42. doi: 10.1038/ejcn.2009.38

PubMed Abstract | Crossref Full Text | Google Scholar

56. Salter, KL, and Kothari, A. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. (2014) 9:115. doi: 10.1186/s13012-014-0115-y

PubMed Abstract | Crossref Full Text | Google Scholar

57. Chavarría Vargas, D d J, Matagira Rondón, G, Agudelo Cifuentes, MC, Morales Mesa, SA, Ouellet, N, and Bergeron, DA. A realist evaluation of a support and training program for family caregivers in the municipality of Envigado: a research protocol. Nurs Open. (2023) 10:4442–51. doi: 10.1002/nop2.1686

PubMed Abstract | Crossref Full Text | Google Scholar

58. Hoens, S, Smetcoren, A, Switsers, L, and De Donder, L. Community health workers and culturally competent home care in Belgium: a realist evaluation. Health Soc Care Community. (2022) 30:630. doi: 10.1111/hsc.13630

Crossref Full Text | Google Scholar

59. Kirkpatrick, P, and van Teijlingen, E. Lost in translation: reflecting on a model to reduce translation and interpretation bias. Open Nurs J. (2009) 3:25–32. doi: 10.2174/1874434600903010025

PubMed Abstract | Crossref Full Text | Google Scholar

60. Poortinga, YH, and Van De Vijver, FJR. Explaining cross-cultural differences: bias analysis and beyond. J Cross-Cult Psychol. (1987) 18:259–82. doi: 10.1177/0022002187018003001

Crossref Full Text | Google Scholar

61. Greene, MJ. On the inside looking in: methodological insights and challenges in conducting qualitative insider research. Qual Rep. (2014) 19:1–13. doi: 10.46743/2160-3715/2014.1106

PubMed Abstract | Crossref Full Text | Google Scholar