Content evaluation of the inclusive eHealth guide: how to develop interventions for people with a lower socioeconomic position?

Objectives: eHealth interventions favor those with higher socio-economic positions (SEPs). This can widen disparities, as people with lower SEPs may lack resources and face digital or financial barriers, making tailored solutions necessary. This study evaluates professionals' perceptions of the Inclusive eHealth Guide (IeG) regarding its content. The aim was to ensure it meets the needs of professionals and the targeted lower SEP demographic, thereby enhancing the effectiveness of eHealth interventions.

Methods: This mixed-method study used qualitative research through semi-structured interviews and the think-aloud method with 13 professionals involved in eight different eHealth lifestyle interventions using the eHealth guide. Quantitative feedback was obtained through a survey with evaluative multiple-choice questions. Participants evaluated the IeG at various stages. They identified positive aspects and points for improvement, and provided recommendations for the guide's content and structure.

Results: Participants valued the IeG's practicality and comprehensiveness, noting its usefulness in developing accessible eHealth solutions for populations with lower SEP. They suggested improving content clarity, expanding informational depth, and refining the guide's structure.

Conclusions: The IeG has potential as a valuable tool for professionals developing eHealth interventions for lower SEP populations. Continuous refinement is crucial to ensure the guide remains relevant and effective, contributing to reducing health disparities.

1 Introduction

The benefits of eHealth interventions are widely acknowledged. They empower individuals by engaging them in healthy lifestyle activities and self-managing chronic illnesses. This enhances health outcomes, and alleviates the burden on healthcare providers (1–3). eHealth encompasses a range of devices and communication tools, from smartphones and wearables to email and text messaging (4). These technologies play a crucial role in health interventions across platforms, websites, and apps, including smoking cessation and managing conditions like diabetes and cardiovascular disease (5). Technology can benefit individuals through improving adherence and effectiveness of interventions (6). However, achieving these benefits depends on the quality—such as the usefulness and usability—of the eHealth technology. Structural barriers to eHealth use are especially significant for people in lower socioeconomic positions (SEPs), who are disproportionately affected by health problems. These individuals often face systemic exclusion from digital health solutions due to factors such as limited financial resources, elevated stress, and lower digital literacy (5, 7, 8). Consequently, their use of eHealth interventions is often lower, which may further exacerbate existing health disparities (8). Lower SEP is a multifaceted concept encompassing various domains such as education, income, occupation, and neighborhood. It affects a significant portion of the population, with 22% of the EU population or 95.3 million individuals in 2022 falling into this category (9). Variations exist across countries, with areas with a higher concentration of lower SEP people also reporting higher rates of cardiovascular disease and mortality (10). The association between lower SEP and poorer health outcomes and health behaviors—such as smoking and unhealthy diets—is shaped largely by systemic, economic, and environmental inequalities. It is not solely the result of individual choices (7, 11, 12).

However, eHealth interventions hold promise in influencing health attitudes and behaviors among people from disadvantaged backgrounds (13). These interventions can achieve this by incorporating appropriate behavior change techniques and by providing accessible information through features like simple language, visuals, animations, and audio (14). Furthermore, eHealth interventions can significantly improve access to healthcare services by overcoming geographical barriers and associated costs, particularly for working people with limited flexibility for appointments (15). By tailoring interventions to individual circumstances, such as economic and cultural backgrounds, these tools have the potential to empower users to make informed health decisions (16, 17), including users with lower SEP.

To realize this potential, the active involvement of professionals—such as researchers, developers, healthcare providers, and policy officers—is essential. These professionals are responsible for the development, implementation, and evaluation of eHealth interventions. However, they frequently encounter substantial challenges in effectively reaching and engaging lower SEP populations, due to limited practical resources, frameworks, and an insufficient understanding of the lived realities faced by these target groups (18, 19). Without sufficient practical guidance and inclusive design tools, professionals may inadvertently create eHealth solutions that perpetuate rather than reduce existing inequalities (11, 19). People with a lower SEP face not only financial barriers, but also daily pressures such as caregiving, unstable work, and bureaucratic hurdles. These stressors can tax cognitive and emotional resources like attention and self-regulation, creating a scarcity mindset that hinders engagement with eHealth (20). Efforts to promote healthier lifestyles through eHealth must therefore move beyond content improvements and embrace a comprehensive approach that recognizes and addresses the broader challenges faced by this population (11).

A promising strategy to ensure usefulness and usability of the technology is to integrate user needs into the complete developmental process. This is a key element of the Human-Centered Design (HCD) approach, which prioritizes the human perspective throughout the design process (21, 22). HCD emphasizes stakeholder (e.g., users of the technology, healthcare professionals who provide the technology) involvement, to assess their needs, wishes, capabilities, and context, thereby developing eHealth solutions that genuinely align with users' requirements for adoption and sustained utilization. This iterative design process involves early and ongoing engagement with stakeholders, feedback incorporation, and continuous field testing (22, 23). This approach successfully integrates both scientific and practical knowledge, thereby enhancing the user-friendliness and customization of eHealth solutions to adequately meet end-users' challenges, ultimately supporting their effectiveness and impact. Despite these advantages, continuous testing and evaluation with stakeholders is often limited. Constraints include time, funding, limited experience with participatory methods, lack of trust between developers and users, and uncertainty about how to engage end-users meaningfully. These factors result in suboptimal design and implementation of eHealth interventions and resulting in their underutilization (19, 24).

1.1 Evaluating the inclusive eHealth guide

To support professionals (e.g., eHealth developers, researchers, health care providers, and policy makers) in designing inclusive eHealth interventions for populations with a lower SEP, we developed the Inclusive eHealth Guide (IeG), a web-based tool grounded in HCD principles. The guide's design was investigated in our prior research (18). The guide was iteratively developed using feedback from professionals across disciplines, ensuring that its content is practically relevant and applicable in real-world settings, providing a comprehensive framework for creating impactful eHealth solutions.

The IeG is structured around five key phases of eHealth intervention development: development, reach, adherence, evaluation, and implementation (see Figure 1). Its content is based on previous scoping reviews and empirical studies involving both professionals and people with lower SEP. Each phase addresses common barriers and links them to practical facilitators and actionable recommendations. These are enriched with user portraits, real-world examples, tips, and links to tools and resources [See also Figure 2, elements (5) and (6)]. A visual navigation structure on the homepage allows users to explore the guide flexibly, depending on their needs and the specific phase of their project. An illustration of the IeG is presented in Figures 1, 2, and a detailed description of the guide is included in Supplementary Material S1.

Figure 1

Figure 1. Different stages of the inclusive eHealth guide. Adapted with permission from “Inclusive eHealth Guide”.

Figure 2

Figure 2. Example page layout from the inclusive eHealth guide, illustrating structure and key features. Screenshot from: “Developing with the target group, Inclusive eHealth Guide.”

To further develop the IeG, we gathered feedback from professionals to explore how they perceive and apply the guide in practice. Understanding these perceptions is crucial to ensure that the guide aligns with practical needs and effectively supports the implementation of eHealth solutions. This feedback helps identify strengths and areas for improvement, guiding the ongoing refinement of the guide. In doing so, we aim to ensure that the IeG continues to meet the evolving needs of professionals and the lower SEP population, thereby enhancing the effectiveness of inclusive eHealth interventions.

2 Methods

2.1 Study design

Ethical approval was obtained from the Psychology Research Ethics Committee at the University of Leiden (Breeman, dr. L.D.-V2-4262). The IeG was assessed using a mixed-methods framework, blending qualitative and quantitative techniques (see Figure 3). Data were collected in one session using semi-structured interviews and the think-aloud method (25, 26). The think-aloud approach enabled participants to verbalize their thoughts and reasoning, offering rich insights into professional perceptions of the IeG, identifying both positive aspects and points for improvement. Quantitative feedback was gathered through a survey featuring evaluative multiple-choice questions, providing a balanced view of professional responses across different phases of eHealth intervention: development, reach, adherence, evaluation, and implementation.

Figure 3

Figure 3. Study design and interview process overview.

2.2 Ehealth interventions and participants recruitment

The relevance of IeG content was evaluated through interviews with researchers and developers from various eHealth-interventions. Data was collected in two steps. First, 8 eHealth lifestyle interventions were selected based on specific criteria (Figure 4) to assess IeG practicality and feasibility. This offered the opportunity to identify areas for improvement to enhance IeG applicability. To find suitable eHealth interventions, we compiled a list of eHealth lifestyle interventions based on internet searches and recommendations from colleagues in the field. Subsequently, we contacted individuals responsible for these interventions, inquiring about case suitability based on the selection criteria. In the second step, upon identifying a suitable case, we inquired via email or face-to-face about individuals' intent to participate in this study. Suitable candidates were those involved in at least one phase of their eHealth intervention, excluding those previously engaged in the IeG development. We did not consider participants' experience with lower SEP groups as a selection criterion, because this study aimed to gather diverse perspectives to identify potential gaps in the IeG and improve its general applicability. Including professionals with varying levels of experience allowed us to explore a range of needs and perceptions. This included both those highly familiar with the target group and those less experienced but still involved in relevant intervention work. This variation provided insight into how the guide is understood and used by professionals across different backgrounds and levels of familiarity with inclusive design. The content of the guide was informed by earlier research involving the target group (27). In this phase, the focus was on professionals, as the IeG is intended to support them in designing inclusive eHealth interventions.

Figure 4

Figure 4. Stages of eHealth intervention and participant selection process.

Semi-structured interviews with researchers, eHealth developers, healthcare professionals, and other experts involved in these interventions were conducted. Feedback from this range of professionals yielded an overall assessment of the guide's applicability based on their experiences and needs. We interviewed 1–3 individuals from each selected intervention, totaling 13 interviews. This purposive sampling approach ensured the inclusion of diverse professional roles and experiences across different phases of eHealth intervention development, allowing for a broad and informative exploration of the guide's applicability The sample size was considered sufficient to reflect a diversity of perspectives aligned with the study's objectives (28).

2.3 Procedure and materials

Interviews were conducted by the first author online or in-person, lasting 50–60 min, and recorded using Microsoft Teams or a voice recorder. The interview process was divided into three segments (see Figure 3). Before the interview, participants received an email with an information letter, a background questionnaire link, and access to the IeG. This pre-interview questionnaire collected data on their roles and experiences in eHealth interventions, particularly concerning lower SEP populations. If participants had not completed the questionnaire prior to the interview, it was completed during the session. The interview began by exploring the resources, documents, and guidelines participants used in their eHealth interventions, their experience with designs for people with a lower SEP, and familiarity with inclusive design principles. This was followed by a discussion on their eHealth intervention involvement, including objectives, target demographics, and SEP considerations (29, 30). Subsequently, the “think-aloud” method was employed. Participants were asked to navigate the IeG online, selecting intervention phases relevant to their experiences. A scenario provided a starting point for exploration, which they could adapt to their needs (see Box 1). They were tasked with examining the homepage, sharing thoughts, noting details, and discussing encountered recommendations and practical advice. In the final part, participants answered multiple-choice questions about the guide's clarity, conciseness, and whether the eHealth intervention phases contained adequate information for developing interventions aimed at people with a lower SEP.

Box 1

Box 1. Scenario.

These items were rated on a 5-point Likert scale, from 1 “very negative” to 5 “very positive”. Participants also provided verbal feedback and answered three open-ended questions regarding guide acceptance, inspired by the Appraisal of Guidelines for Research & Evaluation (AGREE) II Instrument (31). The interview protocol is included in Supplementary Material S2. All information was processed anonymously, with identifying information removed before coding and participant identification codes used instead.

2.4 Data analysis

We conducted a thematic analysis of transcribed interviews using Atlas.ti software (32), following Braun & Clarke's (2006) open and axial coding procedures (25). This approach facilitated the identification and organization of recurring themes. The first and third authors independently coded all the data, ensuring depth in analysis. In collaborative sessions, initial codes and themes were discussed, refined, and unified by consensus, resulting in two main categories: (1) positive aspects, which identify the strengths of the IeG, and (2) points for improvement, which offering future recommendations for the guide's content and structure. Although the primary focus was on collecting content-related evaluations, we also obtained valuable insights evaluations about its structure, which were included in the results section. In addition to the thematic analysis, we also analyzed the quantitative data, including participant background information and the quantitative evaluations of the IeG, using descriptive statistics in SPSS Statistics 29 (IBM Corp). The small sample size did not permit inferential analysis.

3 Results

3.1 Participants

This study involved a total of 13 professionals, including eHealth developers, healthcare providers, researchers, and policy officers, who engaged in eight different eHealth lifestyle interventions, varying from lifestyle management to sleep-related interventions. Of the 54% of participants who reported having worked with people with a lower SEP background, 31% reported having only limited experience (Table 1). The average length of time that participants were involved in these interventions was 2.65 years (SD = 1.32).

Table 1

Table 1. Participant characteristics.

3.1.1 Ehealth experience of participants

The experience of participants in designing eHealth interventions for people with a lower SEP varied. Some directly developed solutions (n = 9), while others provided indirect support through consultations or advisory services (n = 5). Notably, advisory roles often focused on projects for diverse and potentially marginalized groups facing language and literacy challenges (n = 4). One participant designed digital learning interventions outside healthcare to improve access for individuals with limited literacy (n = 1). Three participants lacked direct experience in developing eHealth interventions for lower SEP populations.

Participants reported various methods to maintain and update their knowledge in this area, as derived from the interviews (See Figure 5). Several key resources used for developing eHealth interventions for lower SEP populations included collaborating with the target group (e.g., following the eHealth for All principle), using the Pharos (Dutch Centre of Expertise on Health Disparities) Centre's guidelines, and ensuring B1 level readability, suitable for basic language proficiency.

Figure 5

Figure 5. Overview of the sources used by participants to maintain and update their knowledge.

3.1.2 Ehealth intervention characteristics

The objectives of the eHealth interventions mentioned by the participants included enhancing employee well-being, managing chronic conditions, increasing physical activity, and improving sleep quality. See Supplementary Material S1, Table 1, for more information on the eHealth intervention characteristics Furthermore, inclusivity in eHealth development and evaluation was emphasized as essential for ensuring accessibility across different literacy, digital skill, and cultural levels. This involved simplifying language and using visual or audio elements. A key challenge was balancing simplicity for users with a lower SEP and retaining enough depth for broader audiences. Although this required extra effort, professionals considered it a necessary part of inclusive design.

3.1.3 Suggested users and timing of use

Most participants (84.6%, 11/13) considered the guide valuable for a broad range of professional roles, including developers, researchers, healthcare providers, and project managers. It was perceived as relevant throughout the entire eHealth development process. Participants reported different preferred moments of use depending on their role: developers and project leaders mostly used the guide in the planning or grant-writing phase, while healthcare providers and researchers revisited it during implementation and evaluation. Several participants preferred using it at an early stage, such as during proposal writing or intervention planning, noting it was useful “before you write the grant” (P12). Others highlighted its continued relevance during later phases, such as implementation and evaluation, indicating they would “look at it again” when applying or reviewing content (P2). These findings reflect the guide's flexibility and usability across different professions and project phases. Supplementary Table S2 provides an overview of suggested users and timing of use.

3.2 Guide evaluation

Based on their experience with various phases of their own eHealth interventions, participants evaluated the IeG across different phases: “reach” and “implementation” were assessed by 38% of participants (n = 5 each), followed by “development” and “evaluation” (31%, n = 4 each). The “adherence” phase received less focus (23%, n = 3). Table 2 summarizes participants' ratings of the guide across the eHealth intervention phases.

Table 2

Table 2. Descriptive statistics of questionnaire responses.

3.2.1 Positive aspects of the guide

Qualitative analysis of the interviews identified 67 themes, including 13 main themes with positive aspects and 19 main themes for improvement or future recommendations. Key themes are presented below, and a comprehensive overview of the themes extracted from the interviews is provided Tables 3, 4.

Table 3

Table 3. Positive aspects, descriptions, codes, and quotes per theme and category.

Table 4

Table 4. Recommendations, descriptions, codes, and quotes per theme and category.

Feedback was largely positive across all phases: development (5/5 positive), reach (3/5 positive), adherence (2/3 positive), implementation (3/4 positive), and evaluation (3/4 positive). Furthermore, 69.2% (9/13) reported the recommendations as sufficiently informative for independent progress, while 76.9% (10/13) confirmed the guide's adequacy for tailoring eHealth interventions to a lower SEP audience. Tables 3, 4 present examples of participant quotes highlighting the positive aspects of the IeG, as well as recommendations for its improvement.

3.2.1.1 Content of the guide

Participants found the guide's practical information useful (76.9%, 10/13 positive) and generally recommendations also clear (53.8%, 7/13 positive). The density of information within the recommendations was also positively viewed (61.5%, 8/13 positive). More than half of the participants (7/13) valued the guide concise clarity, appreciating the brevity and straightforward presentation that facilitated quick and easy understanding. One participant (P2, Researcher) noted that “everything is nicely clear, and it is also nice that you can click through,” highlighting the usability of the structure.

Reflecting on the IeG applicability and practicality, almost all participants (12/13) across various phases concurred on several positive aspects. The majority (8/12) valued the website's practical tips, including the concrete guidance, tools, and strategies provided. They appreciated the easy access to information via evaluation instruments, participatory research, and hyperlinks, finding it helpful for implementing interventions and overcoming challenges. As expressed by one participant (P9, Researcher), the guide is “filled with concrete tips that are feasible and understandable”.

Furthermore, several participants (n = 4) emphasized that the guide's principles and practical recommendations are applicable beyond digital health interventions or lower SEP groups. For instance, one participant noted that “most of the tips can also be used for non-digital interventions” (P3), and others highlighted its potential value for broader intervention design in areas such as lifestyle support and rare diseases like Huntington's (P4).

In the development phase all participants (4/4) appreciated the focus on understanding the target audience's technology usage. Practical tips like participatory design techniques and community engagement were seen as instrumental for developers. One interviewee (P4, Researcher) remarked, “This is enjoyable, all these concrete details. Yes, I find that very useful”.

In the adherence phase, participants (2/3) valued the depth and relevance of information. Tips on rewarding participants and setting achievable goals were recognized as crucial for enhancing user engagement and adherence. This appreciation for the guide's practical examples was expressed by one psychologist (P3), who noted: “Good to see there are many cases and examples included. If you need more information, you can read further”.

In the evaluation phase, half of the participants (2/4) valued content practicality, focusing on positivity and convenient processes. Community-based participatory research, expert advice, and stakeholder involvement were believed beneficial. Innovative, inclusive methods for lower SEP populations were also valued. Finally, in the implementation phase, the majority (3/5) found the information directly beneficial for developing intervention strategies, with concrete examples emphasizing the importance of thorough preparation.

3.2.1.2 Design and structure of the guide

Participants expressed positive feedback regarding the website's design and structure. The majority (9/13) found the content presentation to be concise and user-friendly, with the option to delve deeper into the material through additional links without feeling overwhelmed by all information. One researcher highlighted that “you can go through briefly with one sentence. and if you want to know more, you can press on those pluses” (P4), referring to the expandable sections on the website that offer more detailed information.

The recommendations “Works” and “Doesn't Work” were well-received (61.5%, 8/13 positive). Participants (3/13) perceived the “Works” and “Doesn't Work” presentation format as insightful, promoting a deeper understanding of best practices by showcasing successful strategies alongside common pitfalls [see Figure 2, elements (3) and (4)]. The sequential flow of content, transitioning from problem identification to solutions, was considered logical and beneficial for learning and decision-making. The website's aesthetics also received positive remarks, with participants (8/13) praising the clarity of the images, color coding between “Works” and “Doesn't Work” sections, and overall layout.

The necessity for recommendations to be based on scientific evidence was affirmed by 69.2% (9/13 positive). Some participants (4/13) underscored the value of grounding website content in scientific information, such as literature and references, to demonstrate credibility and appeal to academically inclined users (i.e., highlighting the importance of evidence-based and experience-based input). Participants appreciated the inclusion of scientific sources to underpin content validity and reliability. They also preferred practical applications, suggesting a balance between academic rigor and real-world relevance.

3.2.2 Recommendations for the guide

3.2.2.1 Content of the guide

A clear theme emerged around the need for improving the clarity and accessibility of the guide's content. The majority of participants (7/13) underscored the importance of simplifying terminology and reducing jargon to make the content more approachable. This included suggestions to provide clearer explanations and definitions, potentially through hyperlinked terms, to aid in comprehension. One participant (P4, Researcher) noted, “Evaluation—what kind of evaluation is that, because of course, you can evaluate during the development phase,” illustrating the need for more precise terms.

Additionally, some participants (5/7) recommended a nuanced view on repetition; while it can reinforce key points and aid retention, excessive repetition was seen as counterproductive. This was reflected in the question by a policy officer and project leader (P1): “Which professionals are being referred to here? Are they healthcare professionals?”, indicating a need for clarification and consistency.

All participants indicated a preference for more in-depth and actionable content. This included requests for step-by-step instructions on specific processes (e.g., collaboration strategies, team communication), concrete examples, case studies, and links to successful projects for enhanced learning and application. A researcher (P10) remarked, “What are those rules of thumb that I need to take into account?”, emphasizing the desire for practical, directly applicable information. However, some participants (3/13) expressed a desire for concise information delivery to avoid overwhelming readers.

3.2.2.1.1 Expanding information content

The participants offered broad suggestions for enriching the guide's informational content that are applicable to various phases of the guide (see Table 5). The incorporation of both positive and negative examples was suggested to facilitate comprehensive learning, reflecting on past successes and failures. Participants valued insights from other projects' experiences, recommending the sharing of best practices and lessons learned. The balance between acknowledging participation and avoiding undue influence was noted, with suggestions for small incentives like supermarket vouchers to maintain engagement. Additionally, guidance on managing privacy concerns in different project phases was sought, including simplifying complex privacy regulations and effectively communicating them to end-users.

Table 5

Table 5. Recommendations on expanding information content, descriptions, codes, and quotes per theme and category.

Specific recommendations for each phase were made, such as methods for identifying the target audience, analyzing their engagement, and tips for organizing focus groups in the development phase (N = 2). Also the importance of dynamic scheduling and incentivizing focus group participation was indicated. As one researcher (P9) explained: “Also, determine what a good moment and time might be, and then, as a researcher, you will need to be flexible. Because it will not always be, say, the typical 9–5”..

In the reach phase, some participants (2/5) highlighted the importance of social involvement and practical engagement strategies (4/5), suggesting the inclusion of patients' family members in eHealth interventions (e.g., translating, app usage) to enhance audience engagement. However, it was also emphasized that the children's age and the potential burden on them must be considered.

In the adherence phase (N = 3), suggestions focused on personalized engagement and the use of gamification, recommending customization to reflect individual user needs and achievements to enhance motivation and adherence. One researcher (P11) reflected: “…And we also did the gamification…It shouldn't have been too complicated. I think we made it a bit too complicated”..

In the implementation phase, participants (N = 5) emphasized the need for thorough planning covering aspects like financing, project preparation, and privacy. Emphasizing both planning and execution, they advised the inclusion of concrete examples to illustrate the impact of inadequate implementation. One participant (P12, Researcher and Healthcare Provider) stressed: “Implementing the eHealth intervention is crucial; those using it and offering it to patients must be motivated, involved, and believe in the intervention”..

They indicated the necessity of securing structural funding and ownership issues, including intellectual property rights, to ensure the sustainability and success of eHealth interventions. Additionally, the implementation of practical support mechanisms, such as providing direct contact options (e.g., a dedicated phone line and email support), was recommended to improve accessibility and user support.

3.2.2.2 Design and structure of the guide

Design suggestions were provided by participants. They suggested 13 recommendations for improving the website's layout design. Mixed opinions (9/13) were expressed about the “Works Well” and “Doesn't Work” sections, with a preference for direct access to solutions and a need for clearer guidance through a balanced approach to both sections. The importance of logical and intuitive content organization was emphasized by 7/13 participants. Additionally, adding visual aids, such as charts, diagrams, and images, was suggested by 7 participants and found to be valuable.

4 Discussion

This study aimed to explore professional opinions on the IeG (18), a guide designed to support eHealth intervention development for lower SEP populations. The IeG was positively evaluated across different professional fields, particularly for its practicality, broad applicability, and inclusion of real-life examples and actionable suggestions. According to some professionals, the guide's applicability may extend beyond eHealth, offering support in non-digital settings as well. They cited elements such as participatory design and accessible communication as relevant across various intervention contexts. However, further exploration of this potential is warranted. To further enhance the guide's utility, it is necessary to improve accessibility, enrich content with more in-depth information, refine recommendations for specific developmental stages (adherence, implementation, and evaluation), improve the design for a better user experience, and ensure a balance between scientific depth and practical relevance.

4.1 Practicality and relevance of the IeG

The practicality and relevance of the IeG were often highlighted by professionals from different disciplines. They appreciated the guide's practical examples and actionable advice for developing eHealth interventions for people with a lower SEP. Its comprehensive framework cover the full development process, aligning with requirements identified in our previous study (18), and stemming from participatory design approaches (24) and HCD principles (22), focusing on user needs and stakeholder involvement.

Although differences were not systematically explored, professionals with less experience working with lower SEP groups may have underestimated the complexity involved in tailoring eHealth interventions. This potential gap between expectations and the realities of practice could influence perceptions of required time and resources (33).

However, feedback highlighted areas for improvement. Despite positive reception, calls exist for broadening topics and better accessibility through clearer content categorization and providing more detailed implementation guidance with “actionable advice”. Experienced professionals, especially those working with people with a lower SEP, suggest the need for detailed instructions and real-life case studies to navigate unconventional situations. Recommendation reflects a broader agreement on the importance of actionable advice, a critical need identified in our prior study (18). This need has not been fully met because we aimed to keep the content concise, based on the needs of participants from our prior study, and not all information could be supported with practical examples due to the lack of specific information for people with lower SEP. Existing guidelines, such as those from the World Health Organization (WHO) and the National Health Service (NHS), offer some eHealth development direction, focusing on people in vulnerable positions (34–36). However, these often lack the concrete, actionable knowledge—the “how”—professionals need to navigate complex real-world situations. This need for practical, step-by-step guidance was raised by participants in the current study and supported by our previous research that informed the development of the IeG (18). Without such support, professionals may struggle to design interventions that effectively reach underserved populations, thereby contributing to the persistent digital and health divide between those with higher and lower SEP (8). This observation aligns with previous findings that emphasize a disconnect between general eHealth frameworks and the needs of practice. For instance, while existing models are valuable at a conceptual level, they frequently fail to address day-to-day implementation challenges (37). Others have argued that digital health interventions can even exacerbate inequalities when they are not designed with contextual, socio-economic factors in mind (6), highlighting the need for concrete, context-sensitive guidance tailored to underserved populations. Additionally, digital health tools often risk reinforcing stereotypes by framing people with a lower SEP as passive or digitally incapable. This tendency can undermine inclusive design unless explicitly countered through participatory approaches that recognize the diversity and agency within this group (7), something the IeG attempts to address. In line with this, making implicit professional knowledge explicit is critical to improving practical decision-making. And emphasize that surfacing such tacit knowledge supports more responsive, situated design, a core ambition of the IeG (38). The Digital Public Health Framework (39) has highlighted the need for frameworks that consider both system-level ethics and practical utility, yet its emphasis remains at the policy and strategic level. In contrast, tools like the IeG aim to operationalize these principles by offering hands-on, context-specific advice for day-to-day use.

Moreover, gathering feedback from participants underscores the iterative development's essential role in the IeG's evolution. Through a cyclical process of prototyping, testing, analyzing, and refining (24), the guide remains responsive to user feedback and adaptable to emerging challenges. This dynamic approach is crucial for ensuring the IeG meets the field's evolving needs, becoming a more impactful resource for professionals in eHealth development.

5 Implications for practice and research

Our study highlighted the necessity of combining scientific and practical knowledge for eHealth professionals. This integration is essential for developing evidence-based eHealth interventions responsive to users' needs. As illustrated by prior research (40) on workplace interventions, connecting research with practical application ensures research remains relevant, actionable, and closely tailored to the practical contexts and specific needs where it is applied. Implementing a similar approach in eHealth can help guarantee that interventions are rooted in solid scientific evidence and are also user-friendly and tailored to address the challenges faced by end-users, thereby improving the effectiveness and impact of these interventions.

Future research should prioritize assessing the long-term impacts of the IeG on eHealth interventions' development and outcomes. Diversifying communication channels, such as instructional videos and interactive workshops, and engaging a wider range of stakeholders could enhance the IeG's reach and adoption. Furthermore, we propose that establishing professional learning communities among eHealth professionals to share practical experiences and scientific research can enrich the collective knowledge base, aiding in developing more effective, user-centered interventions. Continuous refinement, informed by ongoing user feedback (e.g., co-design) and advancements in eHealth technologies, is essential for its sustained relevance. Regular updates are necessary to keep the IeG aligned with the evolving eHealth landscape and to promote inclusive healthcare practices. Expanding implementation and dissemination, possibly using frameworks like RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) (41), can enhance the guide's utilization.

5.1 Strengths and limitations

The strengths of this study include a mixed-method approach combining quantitative approach with qualitative interviews, using the think-aloud technique to understand the strengths, practical use and areas for improvement of the IeG from the perspective of targeted professional users. The diversity of professionals involved provided a broad evaluation, offering varied viewpoints. However, limitations exist. The selection of participants might have introduced bias, possibly including only professionals who see potential in eHealth for lower SEP. Due to time constraints and professional availability, it was not feasible to involve more than one individual from each eHealth intervention, limiting the breadth of perspectives. Excluding lower SEP individuals from the feedback process, which could have provided critical insights into the guide's accessibility and relevance. Future research should therefore include individuals with lived experience of lower SEP in order to deepen the evaluation and ensure the guide's inclusiveness. Moreover, the potential for socially desirable responses was heightened by the interviewer's dual role as the IeG developer, possibly leading to an overestimation of the guide's. Future evaluations should use log data to objectively assess usage and effectiveness and mitigating potential biases. Finally, evaluating the IeG based on hypothetical scenarios might not fully capture its practical impact. Assessing real-world applications is essential to evaluate its effectiveness and applicability more accurately.

6 Conclusion

The IeG emerges as an valuable resource for professionals aiming to develop accessible eHealth interventions for lower SEP individuals. While the guide is appreciated for its practicality and relevance, the feedback indicates areas for improvement, highlighting the significance of continual refinement. As eHealth evolves, it is crucial that the IeG adapts to meet the dynamic needs of professionals and the communities they serve. By promoting a culture of continuous improvement and integrating scientific and practical insights, the IeG stands as a crucial tool in advancing eHealth solutions, ensuring it remains at the forefront of eHealth innovations and applications.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding authors.

Ethics statement

The studies involving humans were approved by Psychology Research Ethics Committee at the University of Leiden (Breeman, dr. L.D.-V2-4262). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

IA-D: Writing – original draft, Writing – review & editing, Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Validation. LB: Writing – original draft, Writing – review & editing, Conceptualization, Supervision. JF: Validation, Writing – review & editing, Conceptualization. JW: Conceptualization, Methodology, Writing – review & editing. RvdBE: Conceptualization, Writing – review & editing. JK: Conceptualization, Writing – review & editing. VJ: Conceptualization, Writing – review & editing. RK: Writing – review & editing. VV: Conceptualization, Funding acquisition, Writing – review & editing. NC: Conceptualization, Funding acquisition, Writing – review & editing. AE: Conceptualization, Funding acquisition, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. We acknowledge the support from Medical Delta (MD 2.0 eHealth and self-management program), and the NWO Stevin grant (PI A. Evers) and the Netherlands Cardiovascular Research Initiative: An initiative with support of the Dutch Heart Foundation, CVON2016-12 BENEFIT (PI A. Evers). Additional support was provided by the Dutch Heart Foundation and ZonMw and is conducted in collaboration with and supported by the Dutch CardioVascular Allicance, number [01-001-2021-B017] Gezond Leven in een Gezonde Omgeving.

Acknowledgments

The authors express their gratitude to all participants whose insightful contributions during the interviews were invaluable to this research. We also thank Thomas Reijnders, PhD, for his involvement in the initial phase of the IeG development.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2025.1528860/full#supplementary-material

References

1. Bergmo TS. How to measure costs and benefits of ehealth interventions: an overview of methods and frameworks. J Med Internet Res. (2015) 17:e254. doi: 10.2196/jmir.4521

PubMed Abstract | Crossref Full Text | Google Scholar

2. Kohl LFM, Crutzen R, De Vries NK. Online prevention aimed at lifestyle behaviors: a systematic review of reviews. J Med Internet Res. (2013) 15(7):1–13. doi: 10.2196/jmir.2665

Crossref Full Text | Google Scholar

3. Talboom-Kamp EP, Verdijk NA, Harmans LM, Numans ME, Chavannes NH. An eHealth platform to manage chronic disease in primary care: an innovative approach. Interact J Med Res. (2016) 5(1):e5. doi: 10.2196/ijmr.4217

PubMed Abstract | Crossref Full Text | Google Scholar

4. Eysenbach G. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Int Res. (2011) 13:e126. doi: 10.2196/jmir.1923

PubMed Abstract | Crossref Full Text | Google Scholar

5. Kontos E, Blake KD, Chou WYS, Prestin A. Predictors of ehealth usage: insights on the digital divide from the health information national trends survey 2012. J Med Internet Res. (2014) 16(7):1–16. doi: 10.2196/jmir.3117

Crossref Full Text | Google Scholar

6. Veinot TC, Mitchell H, Ancker JS. Good intentions are not enough: how informatics interventions can worsen inequality. J Am Med Inform Assoc. (2018) 25(8):1080–8. doi: 10.1093/jamia/ocy052

PubMed Abstract | Crossref Full Text | Google Scholar

7. Dijkstra I, Penders B, Horstman K. How health promotion prevents itself from tackling health inequalities. A critical analysis of Dutch health promotion’s paradigm through its handbooks (1995–2022). SSM—Qual Res Health. (2025) 7:100533. doi: 10.1016/j.ssmqr.2025.100533

Crossref Full Text | Google Scholar

8. Latulippe K, Hamel C, Giroux D. Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res. (2017) 19(4):e136. doi: 10.2196/jmir.6731

PubMed Abstract | Crossref Full Text | Google Scholar

9. Eurostat. People at Risk of Poverty or Social Exclusion in 2022 (2023). Available online at: https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20230614-1 (Accessed November 14, 2024).

Google Scholar

10. Boylan JM, Robert SA. Neighborhood SES is particularly important to the cardiovascular health of low SES individuals. Soc Sci Med. (2017) 188:60–8. doi: 10.1016/j.socscimed.2017.07.005

PubMed Abstract | Crossref Full Text | Google Scholar

11. Myers-Ingram R. Effectiveness of eHealth weight management interventions in overweight and obese adults from low socioeconomic groups: a systematic review. Syst Rev. (2023) 12(1):59. doi: 10.1186/s13643-023-02207-3

PubMed Abstract | Crossref Full Text | Google Scholar

12. Stringhini S, Carmeli C, Jokela M, Avendaño M, Muennig P, Guida F, et al. Socioeconomic status and the 25 × 25 risk factors as determinants of premature mortality: a multicohort study and meta-analysis of 1·7 million men and women. Lancet. (2017) 389(10075):1229–37. doi: 10.1016/S0140-6736(16)32380-7

PubMed Abstract | Crossref Full Text | Google Scholar

13. Ronteltap A, Bukman AJ, Nagelhout GE, Hermans RCJ, Hosper K, Haveman-Nies A, et al. Digital health interventions to improve eating behaviour of people with a lower socioeconomic position: a scoping review of behaviour change techniques. BMC Nutr. (2022) 8(1):145. doi: 10.1186/s40795-022-00635-3

PubMed Abstract | Crossref Full Text | Google Scholar

14. Crutzen R, van der Vaart R, Evers A, Bode C. Public health, behavioural medicine and eHealth technology. In: Gemert-Pijnen LJ, Kelders SM, Kip HSR, editors. eHealth Research, Theory and Development. Abingdon-on-Thames. UK: Routledge (2018). p. 111–27.

Google Scholar

15. McMaughan DJ, Oloruntoba O, Smith ML. Socioeconomic status and access to healthcare: interrelated drivers for healthy aging. Front Public Health. (2020) 8(June):1–9. doi: 10.3389/fpubh.2020.00231

PubMed Abstract | Crossref Full Text | Google Scholar

16. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med. (2016) 51(5):833–42. doi: 10.1016/j.amepre.2016.06.015

PubMed Abstract | Crossref Full Text | Google Scholar

17. Al-Dhahir I, Reijnders T, Faber JS, van den Berg-Emons RJ, Janssen VR, Kraaijenhagen RA, et al. The barriers and facilitators of eHealth-based lifestyle intervention programs for people with a low socioeconomic status: scoping review. J Med Internet Res. (2022) 24(8):e34229. doi: 10.2196/34229

PubMed Abstract | Crossref Full Text | Google Scholar

18. Faber JS, Al-Dhahir I, Kraal JJ, Breeman LD, van den Berg-Emons RJG, Reijnders T, et al. Guide development for eHealth interventions targeting people with a low socioeconomic position: participatory design approach. J Med Internet Res. (2023) 25(1):e48461. doi: 10.2196/48461

PubMed Abstract | Crossref Full Text | Google Scholar

19. Al-Dhahir I, Breeman LD, Faber JS, Reijnders T, Van Den Berg-Emons H, van der Vaart R, et al. An overview of facilitators and barriers in the development of eHealth interventions for people of low socioeconomic position: a Delphi study. Int J Med Inform. (2023) 177(July):105160. doi: 10.1016/j.ijmedinf.2023.105160

PubMed Abstract | Crossref Full Text | Google Scholar

20. Starcke K, Brand M. Decision making under stress: a selective review. Neurosci Biobehav Rev. (2012) 36(4):1228–48. doi: 10.1016/j.neubiorev.2012.02.003

PubMed Abstract | Crossref Full Text | Google Scholar

21. Gemert-Pijnen L, Kelders S, Jong N, Oinas-Kukkonen H. Persuasive health technology. In: van Gemert-Pijnen L, Kelders SM, Kip H, Sanderman R, editors. eHealth Research, Theory and Development: A Multidisciplinary Approach. London: Routledge (2018). p. 228–46.

Google Scholar

22. Melles M, Albayrak A, Goossens R. Innovating health care: key characteristics of human-centered design. Int J Qual Health Care. (2021) 33(October 2020):37–44. doi: 10.1093/intqhc/mzaa127

PubMed Abstract | Crossref Full Text | Google Scholar

23. Sieverink F, Köhle N, Cheung K, Roefs A, Trompetter H, Keizer J, et al. Evaluating eHealth. In: Gemert-Pijnen Lv, Kelders SM, Kip H, Sanderman R, editors. eHealth Research, Theory and Development: A Multidisciplinary Approach (2018). p. 290–318.

Google Scholar

24. Kip H, Keizer J, da Silva MC, Jong NB, Köhle N, Kelders SM. Methods for human-centered eHealth development: narrative scoping review. J Med Internet Res. (2022) 24(1):e31858. doi: 10.2196/31858

PubMed Abstract | Crossref Full Text | Google Scholar

25. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3(2):77–101. doi: 10.1191/1478088706qp063oa

Crossref Full Text | Google Scholar

26. Jaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. (2009) 78(5):340–53. doi: 10.1016/j.ijmedinf.2008.10.002

PubMed Abstract | Crossref Full Text | Google Scholar

27. Faber JS, Al-Dhahir I, Reijnders T, Chavannes NH, Evers AWM, Kraal JJ, et al. Attitudes towards health, healthcare, and eHealth of people with a low socioeconomic status: a community-based participatory approach. Front Digit Health. (2021) 3(July):68. doi: 10.3389/fdgth.2021.690182

Crossref Full Text | Google Scholar

28. Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud. (1994) 41(3):385–97. doi: 10.1006/ijhc.1994.1065

Crossref Full Text | Google Scholar

29. Wentzel J, Velleman E, van der Geest T. Developing accessibility design guidelines for wearables: accessibility standards for multimodal wearable devices. Lect Notes Comput Sci. (2016) 9737:109–19. doi: 10.1007/978-3-319-40250-5_11

Crossref Full Text | Google Scholar

30. Balbach ED. Using Case Studies Evaluation. California Department of Health Services (1999).

Google Scholar

31. Brouwers MC, Kerkvliet K, Spithof K. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. Br Med J. (2016) 352:i1152. doi: 10.1136/bmj.i1152

PubMed Abstract | Crossref Full Text | Google Scholar

32. Atlas ti 23. (2024). Available online at: https://atlasti.com/ (Accessed March 12, 2024).

Google Scholar

33. Coetzer JA, Loukili I, Goedhart NS, Ket JCF, Schuitmaker-Warnaar TJ, Zuiderent-Jerak T, et al. The potential and paradoxes of eHealth research for digitally marginalised groups: a qualitative meta-review. Soc Sci Med. (2024) 350(October 2023):116895. doi: 10.1016/j.socscimed.2024.116895

PubMed Abstract | Crossref Full Text | Google Scholar

34. NHS. Digital inclusion guide for health and social care. NHS Digital. (2019) (July):51.

Google Scholar

35. NHS. PRN00373. Inclusive Digital Healthcare: A Framework for NHS Action on Digital Inclusion (2023). Available online at: https://www.england.nhs.uk/Long-read/inclusive-digital-healthcare-a-framework-for-nhs-action-on-digital-inclusion/ (Accessed November 14, 2024).

Google Scholar

36. WHO. WHO Guideline: recommendations on digital interventions for health system strengthening. Portal Reg BVS. (2019) 2:150.

Google Scholar

37. van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. (2011) 13(4):e111. doi: 10.2196/jmir.1672

PubMed Abstract | Crossref Full Text | Google Scholar

38. Stappers P, Giaccardi E. Research through Design. Interaction Design Foundation - IxDF (2024). Chapter 41. Available online at: https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/research-through-design

39. Jahnel T, Pan CC, Pedros Barnils N, Muellmann S, Freye M, Dassow HH, et al. Developing and evaluating digital public health interventions: the digital public health framework (DigiPHrame). J Med Internet Res. (2023) 26:e54269. doi: 10.2196/54269

Crossref Full Text | Google Scholar

40. Rasmussen CDN, Højberg H, Bengtsen E, Jørgensen MB. Identifying knowledge gaps between practice and research for implementation components of sustainable interventions to improve the working environment—a rapid review. Appl Ergon. (2018) 67:178–92. doi: 10.1016/j.apergo.2017.09.014

PubMed Abstract | Crossref Full Text | Google Scholar

41. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. (2019) 7(MAR):64. doi: 10.3389/fpubh.2019.00064

PubMed Abstract | Crossref Full Text | Google Scholar