Evaluation of mobile health applications using the RE-AIM model: systematic review and meta-analysis

Background: The Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) model has been used as an instrument to determine the impact of the intervention on health in digital format. This study aims to evaluate, through a systematic review and meta-analysis, the dimensions of RE-AIM in interventions carried out by mobile health apps.

Methods: The systematic review and meta-analysis were conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and involved searching six databases - Medline/PubMed, Embase, CINAHL, Virtual Library in Health, and Cochrane Library. The review included randomized, cross-sectional, and cohort clinical trials assessing the prevalence of each RE-AIM dimension according to the duration of the intervention in days. The quality of the selected studies was evaluated using the Joanna Briggs Institute tool. The random effects meta-analysis method was used to explain the distribution of effects between the studies, by Stata® software (version 11.0) and publication bias was examined by visual inspection of graphs and Egger’s test.

Results: After analyzing the articles found in the databases, and respecting the PRISMA criteria, 21 studies were included, published between 2011 and 2023 in 11 countries. Improvements in health care and self-management were reported for various conditions. The result of the meta-analysis showed a prevalence of 67% (CI: 53–80) for the reach dimension, of 52% (CI: 32–72) for effectiveness, 70% (CI: 58–82) for adoption, 68% (CI: 57–79) for implementation and 64% (CI: 48–80) for maintenance.

Conclusion: The RE-AIM dimensions are useful for assessing how digital health interventions have been implemented and reported in the literature. By highlighting the strengths and areas requiring improvement, the study provides important input for the future development of mobile health applications capable of achieving better clinical and health promotion outcomes.

Systematic Review Registration: PROSPERO. International Prospective Registry of Systematic Reviews. CRD42024556886.

Introduction

The increase in chronic non-communicable diseases (CNCDs) worldwide has led to the rapid expansion of digital mobile health (mHealth). With the increasing use of mobile devices, digital health applications are emerging as promising tools to expand access to the monitoring and managing of these conditions, enabling real-time support and adherence to safe practices that promote behavioral changes essential for the prevention and control of NCDs (1).

Digital health has been advanced by Information and Communication Technologies (ICTs), which enable the transmission of information through digital means, such as computers and wireless networks. These technologies were first introduced in the healthcare sector in the United States in the 1970s (2, 3) and have since been adopted in numerous countries worldwide. With the onset of the COVID-19 pandemic in 2020, there was greater incorporation of these technologies, both in public and private services, for treatment and monitoring of health care adapted to the context of social isolation, leading to a global expansion of the use of online health services for medical consultations and training of health professionals (4).

The literature has shown how the use of ICTs in health services is an excellent tool to ease the flow and, above all, to be able to serve a greater number of people in a universal and equal way. Healthcare applications can be used to optimize results and reduce health risks, as well as to understand the determinants of health promotion and disease (5–7).

Enabled by digital technologies, mobile applications are widely accessible to individuals across various settings, applicable at multiple levels of care -primary, secondary, and tertiary- and have demonstrated a positive impact on health outcomes (8).

In addition, they can be applied in real-life scenarios to provide continuous monitoring and management of health and/or disease parameters (8), on an individual and group basis, encouraging healthy behaviors, preventing or reducing health problems, supporting self-management of chronic diseases, improving users’ knowledge, reducing the number of medical consultations and providing opportunities for different areas of health to develop remote care (9–11).

The implementation of digital health intervention programs has emerged as a key strategy in public health to enhance patient support and improve access to various interventions aimed at health promotion and disease prevention. These programs facilitate treatment adherence and integration while promoting behavioral strategies through messaging, goal tracking, and encouragement of behavior change. By improving individual health outcomes, they contribute to the delivery of high-quality care by healthcare professionals (4, 12).

In this context of digital interventions, the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) model has been used as a tool to help improve the chances of implementing and, above all, planning, monitoring, and evaluating intervention programs in practices and policies focused on promoting healthy habits (13, 14). The RE-AIM framework is made up of five dimensions, translated and adapted from the acronyms: Reach, Effectiveness, Adoption, Implementation, and Maintenance, which together determine the impact on health.

The first study to apply RE-AIM was published in 1999 and since then many researchers and evaluators have used this tool to check what is working and what is not in their programs and use this information to plan quality improvement activities, as well as the long-term sustainability of intervention programs (15–17). Despite its widespread use, there is still no consensus in the literature on the quality of the RE-AIM model for evaluating digital health intervention programs. To this end, this study aims to evaluate mobile health applications using the dimensions of the RE-AIM model, through a systematic review and meta-analysis.

Methodology

Study design and search strategy

This is a systematic review and meta-analysis, conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (18) and registered with the International Prospective Register of Systematic Reviews (PROSPERO), under protocol number CRD4202455688.

The Population, Intervention, Comparator, Outcomes, and Studies (PICOS) method guided the establishment of the inclusion criteria, based on the following guiding question: “What are the scope, effectiveness, adoption, implementation, and maintenance (RE-AIM dimensions) of interventions carried out by mobile health apps?.” The problem considered was mHealth apps, the intervention was the use of the RE-AIM model to evaluate mHealth apps, and there was no comparison. The outcome considered was the RE-AIM dimensions (reach, efficacy, adoption, implementation, and maintenance), and the studies included were cohort, case–control, cross-sectional, and randomized.

The search strategy was developed using keywords from the literature, including previous reviews. Based on these terms, we analyzed which were appropriate among those established by the MeSH (Medical Subject Headings) terms, defining the following terms: Digital Health, Telehealth, Health Program, Health Intervention, and Intervention. To take into account the digital health perspective, the search terms were used in different combinations: (“RE-AIM” AND (“mobile phone” OR “digital health” OR telehealth)); (“RE-AIM” AND (“mobile phone” OR “digital health” OR telehealth) AND “intervention”); (“RE-AIM” AND (“mobile phone” OR “digital health” OR telehealth) AND (“health program” OR “health intervention”)) (Supplementary material—Appendix I).

Data sources

The search was carried out in July 2024, in the electronic databases Medical Literature Analysis and Retrieval System Online (MEDLINE/PubMed), Scopus, Virtual Health Library (VHL), Excerpta Medica dataBASE (EMBASE), and Cumulative Index to Nursing and Allied Health Literature (Cinahls).

The bibliographic software Rayyan (Qatar Computing Research Institute, Qatar) was used to manage the references. No language or publication date filters were used in the search stage.

Eligibility criteria

Only studies that focused on the use of mobile applications in healthcare and the RE-AIM model were included. Review articles, letters, editorials, conference proceedings, commentaries, reports, study protocols, pilot studies, case reports, and experience reports were excluded from this study. There were no date or language restrictions.

Study selection

The selection of studies was conducted independently by three reviewers (ELGMJ, MDP, ALPM) using Rayyan software. Screening of the title, abstract, and text began, and any discrepancies were discussed with another reviewer (JHCS) to resolve them. Each of the three reviewers selected studies for possible inclusion based on the title and content of the abstract. The studies that met the inclusion criteria were analyzed in the full-text review. A reverse search was carried out, analyzing the references of the selected articles to identify any additional relevant studies not captured in the initial search.

Data extraction

A Microsoft Excel spreadsheet (Microsoft Corp., Washington, USA) was used to organize the extracted data. The following data was collected: title, year of publication, authors, country, year of study, study design, app name, sample size, study duration, sample characterization, main intervention, main features of the mobile app, program data, results, and main conclusions.

To extract the data on the dimensions of RE-AIM, we used a table adapted from Burke et al. (19), which includes the five dimensions and the items that make up each one. We considered the percentage of adequacy, concerning adequately reported interventions, and the percentage of inadequacy, in relation to inadequately reported interventions.

Meta-analysis

The primary outcome was the prevalence of Reach, Effectiveness, Adoption, Implementation, and Maintenance according to the duration (in days) of the intervention. The meta-analysis was performed using Stata® software (version 11.0), utilizing the Metaprop and Metareg commands. The random effect model was used to carry out the meta-analysis. Heterogeneity was assessed using the chi-squared test (χ2) with a significance level of 90% (p < 0.10), and its magnitude was determined by the I-squared (I2) (20).

Heterogeneity was classified as low, moderate, or high when I2 values were above 25, 50 and 75%, respectively. Heterogeneity was examined through meta-regressions using the Knapp and Hartung test (21). A univariate analysis was carried out with the following variables: year of publication, duration of the intervention and region or continent of the study (to include). A significance level of 5% was established. The presence of small-study effects was evaluated through visual inspection of the funnel plot and Egger’s test.

Quality and risk of bias assessment

The quality of each study included in the review was assessed using the critical appraisal tool recommended by the Joanna Briggs Institute (22), specific to each type of study, and conducted independently by three researchers. The evaluation is based on a checklist with 13 questions for randomized studies, 11 for cohort studies, and eight for cross-sectional studies. Each question allows answers of yes, no, unclear, or not applicable. The results were measured as a percentage, and each item on the checklist was given the following score: 1 point for “yes” and 0 for “no” “unclear” or “not applicable.” Studies of good quality were those that scored above 75% (23).

Publication bias was examined by visual inspection of funnel plots and Egger’s test through the meta-analysis carried out using Stata® software (version 11.0). The risk of bias was also analyzed using RevMan (version 5.4). The statistical significance of the overall effect size of using the RE-AIM mobile health app assessment was determined by the 95% confidence interval (CI).

Results

Identification of studies

The search across six databases yielded 2,140 articles. After duplicate removal, 744 articles remained and were screened based on their titles and abstracts, resulting in 198 articles. Following the application of eligibility criteria, 107 articles were retained for full-text review. A thorough assessment of the full texts ultimately led to the selection of 21 articles for inclusion in this systematic review and meta-analysis. No additional studies were identified in the reverse search. This process is shown in Figure 1.

Figure 1

Figure 1. Selection flowchart of papers included in the systematic review.

Characteristics of the studies included

The studies were carried out between 2011 and 2022 in different countries: eight in the United States, followed by two in China, two in the Netherlands, two in Uganda, and the rest in Canada, Switzerland, Norway, the United Kingdom, Australia, Zimbabwe and Singapore. A total of 14,823 people were assessed in the 21 studies, with a minimum sample size of 35 and a maximum of 2,961. One study evaluated only adolescents (24), one included adolescent athletes (over 12 years old) in its sample (25), one included patients aged 15 and over with sickle cell anemia, and two included patients over 15 years old who lived in a fishing community (26, 27). The other studies assessed adults and the older adult.

Different interventions were carried out on the following topics: medication use in sickle cell anemia, postectomy programs, reduction of ankle injuries, psychological intervention in chronic pain, management of low back pain, HIV prevention, adherence to antiretroviral treatment, reduction of alcohol consumption, treatment of drug addicts, nutritional monitoring of hospitalized patients, cardiac rehabilitation, health of stroke patients, healthy lifestyle for diabetes patients, promotion of physical activity, reduction of sedentary lifestyles in the workplace, training parents in pediatric care, quality of life and mental health in cancer patients and mental health of university students. The minimum intervention time was 21 days (28) and the maximum was 1,185 days (26, 27). Only one study did not provide an exact intervention time, as it varied according to the length of time patients were hospitalized (29) (Supplementary material—Appendix II).

The results of the studies showed that the use of mobile apps increases adherence to treatments, helps in managing health conditions, preventing diseases, adopting a healthy lifestyle, reducing injuries, controlling pain, and self-monitoring health. These tools were deemed effective and cost-efficient for reaching a broader population. Logistical advantages included the ability to customize app functions to individual needs, enable simultaneous use by healthcare teams, and maintain functionality offline. These features optimize healthcare professionals’ actions and extend coverage, even in areas with limited internet connectivity. Some challenges were reported, such as technical problems and difficulties in using the app, especially for older people and those with low levels of education; the issue of social stigma in cases of diseases such as HIV; the low adherence of some health professionals; and the decline in effectiveness over time, showing the difficulties of sustainability in digital health interventions (Supplementary material—Appendix II).

Assessment of RE-AIM dimensions and items

The 21 studies were evaluated based on the suitability of the five RE-AIM dimensions -reach, effectiveness, adoption, implementation, and maintenance- and demonstrated an overall adequacy of 88.15%. According to the model proposed by Burke et al. (19), each item in these dimensions was assessed (Table 1). It is important to highlight that the RE-AIM dimensions assessed in this review reflect the reporting adequacy and presence of each dimension within the included studies, according to the authors’ own definitions and criteria, and do not directly represent clinical effectiveness or health outcomes achieved by the interventions themselves. For the “reach” dimension, the percentage of adequacy was 76.16%, with 19 studies (90.44%) meeting the “participation rate” item. For the “effectiveness” dimension, 76.16% of the studies showed adequacy, with the majority of studies meeting the “primary outcome measure” (90.44%) and “broader outcome measures” (85.68%) items.

Table 1

Table 1. Inclusion of RE-AIM items across all interventions (N = 21).

In the “adoption” dimension focused on the configuration of the intervention, 67.83% of the studies showed adequacy, with 18 (85.68%) studies meeting the “configuration exclusions” item. In the “adoption” dimension focused on the team, 63.07% of the studies showed adequacy, with 15 (71.40%) of them meeting the “team representativeness” item. In the “implementation” dimension, the studies showed 65.68% adequacy, with 17 (80.92%) of them meeting the “adaptations to the intervention” item. In the individual “maintenance” dimension, the studies were 37.12% adequate, and 11 (52.36%) studies met the item “use of qualitative methods to understand long-term effects.” In the “maintenance” dimension focused on the configuration of the intervention, the studies were 54.74% adequate, and 14 (66.64%) of them met the item “long-term program adaptations.”

A total of 31 RE-AIM items were assessed, and for the 21 studies, the average number of items reported was 19.38, with a minimum of eight and a maximum of 29 items per study.

Although the RE-AIM dimensions, particularly Effectiveness, include health outcome data reported by the original studies, it is important to clarify that the synthesis presented here reflects how these outcomes were categorized and reported according to each study’s own operational definitions and methodologies. The aggregated percentages should therefore be interpreted as a reflection of the reporting and implementation characteristics of the interventions, rather than as direct measures of clinical effectiveness or intervention success at the population level.

Meta-analysis

For the 21 studies analysed, the pooled prevalence estimate for Reach was 67% of the individuals assessed (CI: 53–80), for Effectiveness was 52% (CI: 32–72), for Adoption was 70% (CI: 58–82), for Implementation was 68% (CI: 57–79), and for Maintenance was 64% of the individuals present in the studies analysed (CI: 48–80) (Figure 2).

Figure 2

Figure 2. Meta-analysis of the prevalence of reach, effectiveness, adoption, implementation and maintenance, according to the days of intervention in the studies. *ES: Prevalence Estimates grouped from the studies; 95% CI: 95% Confidence Interval.

The studies showed significant heterogeneity (p < 0.001), considered high according to the I² statistic (<98%), which can be explained by the different interventions and the diversity of the sample, which varied in age (adolescents, adults and the older adult), gender (three studies only considered males), and health conditions (morbidities such as sickle cell anemia, diabetes, cancer, chronic muscle damage and drug addiction, sedentary lifestyle, and hospitalization). Subgroup analyses were carried out for the variables year of publication, duration of intervention and region or continent of the study, but none of them showed statistical significance in any of the models analysed. Other variables of interest, such as age group and gender, were not included in the regression model because there was no complete data in the studies evaluated.

Quality of the studies

Among the 21 articles included in this systematic review and meta-analysis, 11 met the criteria for classification as exceeding 75% based on the evaluation criteria of the Joanna Briggs Institute forms, while the remaining articles scored between 50 and 75%. None of the studies were classified as low quality.

Risk of bias

Figure 3 shows the funnel graphs for visualizing the risk of publication bias. In these analyses, it is possible to observe asymmetry between the investigations. This asymmetry is confirmed by Egger’s test, which showed significant results for “reach” (p = 0.023), “effectiveness” (p = 0.001) and “maintenance” (p = 0.016).

Figure 3

Figure 3. Funnel plot of the prevalence of RE-AIM dimensions, with 95% confidence limits. *ES: prevalence estimates grouped from the studies; se(ES): Standard error of the estimate.

The risk of bias in each study was also analyzed (Figure 4). In summary, one article fully met the quality criteria, and 21 partially met these criteria. The majority of the articles (17/21) provided results and follow-up data, indicating a low risk of attrition bias. Concerning reporting bias, the majority (18/21) had a low risk of bias, as they clearly expressed the characteristics of the participants in their respective studies.

Figure 4

Figure 4. Summary risk of bias for each study included in the meta-analysis.

Discussion

The results of this review reflect how the RE-AIM dimensions were reported in the included studies. More than 60% of the target population was reached, as described by the original authors. Adoption and implementation were the most frequently reported dimensions, with around 70% of individuals engaged according to each study’s criteria. Effectiveness was reported for 52% of participants, based on study-specific outcome definitions. These percentages refer these the reporting adequacy and presence of each RE-AIM dimension within the included studies, and do not directly correspond to clinical health outcomes or intervention success rates.

The “Reach” dimension refers to the level of individual participation, such as patients or health professionals, and involves the proportion and risk characteristics of people who are impacted by or participate in an intervention or program (13). The studies revealed that reach was 67% of the individuals evaluated, suggesting that most studies explicitly reported sample characteristics, as information such as advanced age, limited internet connection and low education, for example, can be a barrier to the reach of digital interventions. These results suggest the need for digital interventions that consider the diverse characteristics of individuals, seeking to serve mainly the most vulnerable, such as those with low income and less education, who have difficulty accessing services.

The “Effectiveness” dimension examines the impact of interventions, checking whether they have generated improvements or losses in the health and quality of life of those involved (13). In this review, effectiveness reached 52% of participants, indicating that not all interventions had a positive impact on those assessed. This can be explained by the lack of detail in some studies, which did not show a comparison of the results after the intervention or were unable to assess the improvements in patients’ health and quality of life after using the app. They also failed to compare those who completed the program with those who dropped out, to identify the real impact of the interventions. It is also important to think that the low effectiveness may be the result of low adherence to the app, as presented by the authors Chang et al. (26), who reported low acceptance of the intervention due to the stigmas associated with HIV.

The lower scores observed in the effectiveness dimension, compared to adoption or reach, may be justified because, while several strategies can be applied to increase the reach and adoption of interventions, health improvements require sustained behavioral changes, and monitoring and measuring these improvements in the digital context is complex. Furthermore, the duration of interventions is limited in most studies, which makes it difficult to detect significant health outcomes.

“Adoption” refers to the rate of acceptance and the representativeness of the places, such as work environments, health institutions, or communities, that implement a program or policy (13). This dimension reached 70% of participants, showing that the majority of those assessed, whether patients or healthcare staff, accepted using the apps for digital interventions. Information such as the number of downloads of the application, the number of login accesses, registrations in intervention programs, responses to questionnaires and interviews about the interventions, time spent using the application, and discontinuation of use, were presented in the evaluation of this dimension. Low adoption rates have been attributed to factors such as cost, insufficient training of professionals in the use of the digital tool, and the format and location of interventions (30, 31). According to Asare et al. (32), not disclosing team details, for example, does not allow us to know which resources need future interventions, which undermines both the effectiveness of the intervention and its adoption on an ongoing basis and in different scenarios.

Some important factors for increased adoption include personalized and timely support for users during implementation, the ability to adapt to patients’ routines, the inclusion of a two-way service model that allows communication between patients and healthcare professionals involved in care, and technological literacy, especially for older patients. Adoption tends to be higher when strategies are less intensive and supported by authorities in the field, in addition to media coverage that highlights the importance of such interventions (33).

The “Implementation” dimension assesses the degree to which an intervention or program is carried out as planned and with the necessary adaptations to achieve the proposed objectives (13). It impacted 68% of the participants in the studies, which highlighted the adaptations made and provided a detailed account of the team’s actions when assessing individuals both before and after the intervention. According to Stirman et al. (34), this information is important so that interventions can be adapted to different contexts while maintaining the essential characteristics. Furthermore, reporting the costs of implementing a given intervention allows future researchers to have a financial reference for developing similar studies (34).

The last dimension, “maintenance,” checks the extent to which interventions continue to be used in the long term, both by individuals and by organizations or communities, becoming part of their routines or practices (13). In the studies included, it reached 64% of participants, and it was reported that they continued to use the app even after the research or funding ended, or when contracts for its use were renewed. Satisfaction surveys and assessments of permanent behavior change were used to understand this dimension. Some studies also reported this dimension at the organizational level, which is important because it shows that the intervention has become part of the routine in health care.

Low retention rates have been attributed to researcher time and financial constraints, and to the lack of reports with generalized information on the impact of interventions (31, 32). According to Blackman et al. (30), it is essential to recognize that advances in new technologies can render current interventions obsolete. Furthermore, technical issues can result in decreased motivation to use the tools or lead to withdrawal from participation over time.

Research indicates that mobile applications play a significant role in healthcare by supporting monitoring, promoting treatment adherence, and enhancing health outcomes. Key features of these applications—such as adaptability to individual needs, the inclusion of health guidance through text and video, and functionality in offline settings—are identified as critical factors driving their utilization in health-related contexts. The studies by Quinn et al. (35) and Kim et al. (36) support these findings, as the authors suggest that incorporating a health education platform within the mobile app significantly enhances outcomes such as blood pressure, blood glucose, and lipid levels. Additionally, it promotes the development of self-care and self-monitoring of risk factors over time, leading to overall improvements in health and quality of life.

Of the 21 studies included (24–29, 39–42, 44–54), 13 reported interventions in primary care, highlighting the significance of digital interventions in preventing the onset of risk factors and the progression of disease. This evidence was also reported in another study (37), indicating that the mobile application inserted at the primary care level allows care, monitoring, and management of health conditions, facilitating patient adherence to treatments and optimizing the work of the health team.

It is important to mention the existence of other models for evaluating digital health interventions, such as the Mobile App Rating Scale (MARS) and the Consolidated Framework for Implementation Research (CFIR) (38). However, RE-AIM was defined for this review because it comprehensively addresses internal and external validity, allows for assessment at the organizational and individual level, and considers short- and long-term results. Furthermore, it allows for consideration of the different contexts and diverse populations that can benefit from mHealth interventions, with emphasis on the field of public health (43).

This study has limitations. First, the high heterogeneity of the included studies. In this regard, we performed subgroup analyses considering the year of publication, duration of the intervention, and location of the study, but we did not find statistically significant differences. It was not possible to analyze factors such as age group and sex, since most studies did not provide this information.

Second, the possible classification bias, since some inconsistencies in the operationalization and interpretation of the RE-AIM dimensions were identified in the studies, such as the variation in the measurement tools and the thresholds for each dimension, which limited the comparability of the results. Despite this, data extraction and categorization were conducted by two reviewers, by consensus, or by the judgment of a third reviewer.

Third, it was not possible to perform a sensitivity analysis of the estimates found, due to the heterogeneity of the studies. Fourth, there was evidence of publication bias for three of the five RE-AIM dimensions (“Reach,” “Effectiveness” and “Maintenance”), as indicated by the visual asymmetry in the funnel plots and the statistically significant results in the Egger test. This suggests that the observed effect sizes may be overestimated due to the selective reporting of positive findings in the published literature. Finally, we understand the possibility of underestimation or incorrect classification of the RE-AIM dimensions, as they were extracted from the studies and each author may have failed to report important aspects of interventions that have been implemented. We also emphasize that we found a scarcity of records on RE-AIM in the context of mental health, which does not necessarily reflect the lack of mHealth interventions in this context.

We recommend that future studies prioritize the detailing and transparency of information on each dimension, so that more robust comparisons are possible, contributing to the advancement of mHealth interventions based on consistent evidence.

Strengths of this study are highlighted. First, the comprehensive search of six large databases, comprising studies from 11 countries over 12 years (2011–2023), which minimized selection bias and increased the representativeness of the evidence, with different population and health contexts.

Second, conducting the review according to the PRISMA guidelines ensured methodological rigor, and the use of the Joanna Briggs Institute (JBI) tool showed low to moderate risk of bias, especially in the reporting and attrition domains, confirming the methodological quality of this review.

Third, the use of a predefined protocol for extraction and categorization of RE-AIM data increased consistency and reduced classification bias, despite variability between studies. Furthermore, the multiple study designs included (cross-sectional studies, cohort studies, and randomized clinical trials) also broadened the scope of evidence.

Fourth, the meta-analysis allowed for the calculation of pooled estimates for each dimension, providing a clearer synthesis of each dimension. Subgroup analyses and assessment of bias using funnel plots and Egger’s test confirm the transparency and methodological rigor of this review.

Finally, by focusing on a wide range of health outcomes and intervention contexts, multiple criteria related to research and important decisions for quality of life in the context of public healththis review contributes valuable insights to researchers, policymakers, and public health practitioners interested in the implementation and impact of digital health interventions guided by the RE-AIM framework.

Conclusion

This study synthesized the evidence on mobile health app-based interventions using the RE-AIM framework to assess impact across five key dimensions: reach, effectiveness, adoption, implementation, and maintenance. The results show that these dimensions are useful for assessing how digital interventions were designed, delivered, and reported. In this review, interventions had broad population reach, were well accepted by individuals and organizations, and demonstrated robust implementation processes. However, effectiveness and maintenance were the most challenging dimensions, reflecting the need to improve long-term impact and ensure sustained use of these tools.

High adoption rates indicate that digital interventions have substantial potential for integration into clinical practice and individuals’ daily routines, especially when delivered through accessible and personalized platforms. However, the observed variation in effectiveness highlights the need to develop more targeted strategies that address not only app use but also patients’ specific health needs and conditions.

This review makes a significant contribution to public health by providing a comprehensive overview of how digital health interventions have been implemented and evaluated in a variety of healthcare settings. By highlighting strengths and areas for improvement, the study provides important insights for the future development of mobile health applications that can achieve better clinical and health promotion outcomes.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

EM: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. EF: Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Visualization, Writing – review & editing. MP: Data curation, Methodology, Visualization, Writing – original draft. AM: Data curation, Methodology, Visualization, Writing – original draft. JS: Data curation, Methodology, Visualization, Writing – original draft. TM: Data curation, Formal analysis, Methodology, Software, Supervision, Visualization, Writing – review & editing. RM: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This study was funded by the Research Support Foundation of the State of Minas Gerais (FAPEMIG), process no. APQ-02708–21.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2025.1611789/full#supplementary-material

Abbreviations

CI, Confidence Interval; CNCDs, Chronic non-communicable diseases; ICTs, Information and Communication Technologies; MeSH, Medical Subject Headings; mHealth, Digital mobile health; PRISMA, Reporting Items for Systematic Reviews and Meta-Analyses; PROSPERO, International Prospective Register of Systematic Reviews; RE-AIM, Reach, Effectiveness, Adoption, Implementation, and Maintenance.

References

1. World Health Organization (WHO). Global diffusion of eHealth: Making universal health coverage achievable. Report of the third global survey on eHealth. Geneva: World Health Organization (2016).

Google Scholar

2. Lopes, JE, and Heimann, C. Uso das tecnologias da informação e comunicação nas ações médicas a distância: um caminho promissor a ser investido na saúde pública. J Health Inform. (2016) 8:26–30. Available at: https://jhi.sbis.org.br/index.php/jhi-sbis/article/view/364/252

Google Scholar

3. Silva, AB. Adaptação transcultural do aplicativo Zero Mothers Die para dispositivos móveis no Brasil: contribuições para a saúde digital com abordagem do cuidado centrado na e-gestante. Rev Bras Saude Matern Infant. (2019) 19:763–75. doi: 10.1590/1806-93042019000400002

Crossref Full Text | Google Scholar

4. Araújo, MFN. Saúde Pública: avanços da tecnologia da informação e comunicação. Saúde Pública em Pauta: Conhecimentos e Inovações. (2023) 1:240–53. doi: 10.37885/230412890

Crossref Full Text | Google Scholar

5. Belisario, JSM. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev. (2015) 7:1–99. doi: 10.1002/14651858.MR000042.pub2

Crossref Full Text | Google Scholar

6. Papazafiropoulou, A. Telemedicine and diabetes during the COVID-19 era. Arch Med Sci Atheroscler Dis. (2022) 7:e131–e135. doi: 10.5114/amsad/150506

Crossref Full Text | Google Scholar

7. Rosta, L, Menyhart, A, Mahmeed, WA, al-Rasadi, K, al-Alawi, K, Banach, M, et al. Telemedicine for diabetes management during COVID-19: what we have learnt, what and how to implement. Front Endocrinol (Lausanne). (2023) 14:1129793. doi: 10.3389/fendo.2023.1129793

PubMed Abstract | Crossref Full Text | Google Scholar

8. Ferreira, ES, Oliveira, AHM, Dias, MA, Costa, GD, Januário, JPT, Botelho, GM, et al. Mobile solution and chronic diseases: Tomasdevelopment and implementation of a mobile application and digital platform for collecting, analysing data, monitoring and managing health care. BMC Health Serv Res. (2024) 24:1009. doi: 10.1186/s12913-024-11505-y

Crossref Full Text | Google Scholar

9. Mehta, P, Stahl, MG, Germone, MM, Nagle, S, Guigli, R, Thomas, J, et al. Telehealth and nutrition support during the COVID-19 pandemic. J Acad Nutr Diet. (2020) 120:1953–7. doi: 10.1016/j.jand.2020.07.013

PubMed Abstract | Crossref Full Text | Google Scholar

10. Peregrin, T. Telehealth is transforming health care: what you need to know to practice telenutrition. J Acad Nutr Diet. (2019) 119:1916–20. doi: 10.1016/j.jand.2019.07.020

PubMed Abstract | Crossref Full Text | Google Scholar

11. Silva, BMC, Rodrigues, JJPC, de la Torre Díez, I, López-Coronado, M, and Saleem, K. Mobile-health: a review of current state in 2015. J Biomed Inform. (2015) 56:265–72. doi: 10.1016/j.jbi.2015.06.003

PubMed Abstract | Crossref Full Text | Google Scholar

12. Brasil. Telessaúde Brasil Redes: uma década de Inovação. Rio de Janeiro: Rede Nacional de Ensino e Pesquisa (2017).

Google Scholar

13. Glasgow, RR, Vogt, TM, and Boles, SM. Evaluating the public health impact of health promotion interventions: the re-AIM framework. Am J Public Health. (1999) 89:1322–7. doi: 10.2105/AJPH.89.9.1322

PubMed Abstract | Crossref Full Text | Google Scholar

14. RE-AIM. (2023). Improve public health relevance and population health impact. Available at: https://re-aim.org/

Google Scholar

15. Cornell, PY, Hua, CL, Halladay, CW, Halaszynski, J, Harmon, A, Koget, J, et al. Benefits and challenges in the use of RE-AIM for evaluation of a national social work staffing program in the veterans health administration. Front Health Serv. (2023) 3:3. doi: 10.3389/frhs.2023.1225829

PubMed Abstract | Crossref Full Text | Google Scholar

16. Loeb, TB, Gholami, M, Ramm, K, Shedd, K, Soetenga, S, Jackson, NJ, et al. Multilevel perceptions of the virtual delivery of th eUniversity of California diabetes prevention program on RE-AIM domains due to COVID-19 mandates. Front Public Health. (2024) 12:1327429. doi: 10.3389/fpubh.2024.1327429

PubMed Abstract | Crossref Full Text | Google Scholar

17. McNamara, S, Bassett-Gunter, R, Townsend, R, and Wilson, W. Promoting adapted physical activity within kinesiology undergraduate programs: a RE-AIM perspective. Sport Educ Soc. (2024) 30:1. doi: 10.1080/13573322.2024.2336585

Crossref Full Text | Google Scholar

18. Page, MJ, McKenzie, JE, Bossuyt, PM, Boutron, I, Hoffmann, TC, Mulrow, CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. (2021) 372:1–9.

Google Scholar

19. Burke, SM, Tomasone, JR, Scime, NV, Ma, JK, Harden, SM, and Wolf, DL. Physical activity self-management interventions for adults with spinal cord injury: part 2 – exploring the generalizability of findings from research to practice. Psychol Sport Exerc. (2018) 37:286–95. doi: 10.1016/j.psychsport.2018.02.013

Crossref Full Text | Google Scholar

20. Knapp, G, and Hartung, J. Improved tests for a random effects meta-regression with a single covariate. Stat Med. (2003) 22:2693–710. doi: 10.1002/sim.1482

PubMed Abstract | Crossref Full Text | Google Scholar

21. Sterne, JA, Sutton, AJ, Ioannidis, JP, Terrin, N, Jones, DR, and Lau, J. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. (2011) 343:4002. doi: 10.1136/bmj.d4002

PubMed Abstract | Crossref Full Text | Google Scholar

22. Moola, S, Munn, Z, Tufanaru, C, Aromataris, E, Sears, K, Sfectu, R, et al. Chapter 7: systematic reviews of etiology and risk. eds. Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z. In: JBI manual for evidence synthesis: JBI. (2020) doi: 10.46658/JBIMES-24-06

Crossref Full Text | Google Scholar

23. de Sousa, FIM, dos Santos, EJF, Cunha, M, Ferreira, RJO, and Marques, AA. Eficácia de consultas realizadas por enfermeiros em pessoas com artrite reumatóide: revisão sistemática. Rev Enferm Ref. (2017) IV Série:147–56. doi: 10.12707/RIV17013

Crossref Full Text | Google Scholar

24. De la Vega, R, Ritterband, L, and Palermo, TM. Assessing digital health implementation for a pediatric chronic pain intervention: comparing the RE-AIM and BIT frameworks against real-world trial data and recommendations for future studies. J Med Internet Res. (2020) 22:e19898. doi: 10.2196/19898

PubMed Abstract | Crossref Full Text | Google Scholar

25. Vriend, I, Coehoorn, I, and Verhagen, E. Implementation of an app-based neuromuscular training programme to prevent ankle sprains: a process evaluation using the RE-AIM framework. Br J Sports Med. (2014) 49:484–8.

Google Scholar

26. Chang, LW, Pollard, R, Mbabali, I, Anok, A, Hutton, H, Amico, KR, et al. Mixed methods, implementation science evaluation of a community health worker strategy for HIV service engagement in Uganda. JAIDS. J Acquir Immune Defic Syndr. (2023) 94:28–36. doi: 10.1097/QAI.0000000000003220

PubMed Abstract | Crossref Full Text | Google Scholar

27. Edwards, AJ, Pollard, R, Kennedy, CE, Mulamba, J, Ismail Mbabali,, Aggrey Anok,, et al. Impact of community health worker intervention on PrEP knowledge and use in Rakai, Uganda: a mixed methods, implementation science evaluation. Int J STD AIDS. (2022) 33:995–1004. doi: 10.1177/09564624221121208

PubMed Abstract | Crossref Full Text | Google Scholar

28. Wang, M, Guo, Y, Zhang, Y, Xie, S, Yu, Z, Luo, J, et al. Promoting healthy lifestyle in Chinese college students: evaluation of a social media-based intervention applying the RE-AIM framework. Eur J Clin Nutr. (2020) 75:335–44. doi: 10.1038/s41430-020-0643-2

PubMed Abstract | Crossref Full Text | Google Scholar

29. Paulsen, MM, Varsi, C, and Andersen, LF. Process evaluation of the implementation of a decision support system to prevent and treat disease-related malnutrition in a hospital setting. BMC Health Serv Res. (2021) 21:281. doi: 10.1186/s12913-021-06236-3

PubMed Abstract | Crossref Full Text | Google Scholar

30. Blackman, KC, Zoellner, J, Berrey, LM, Alexander, R, Fanning, J, Hill, JL, et al. Assessing the internal and external validity of mobile health physical activity promotion interventions: a systematic literature review using the RE-AIM framework. J Med Internet Res. (2013) 15:e224. doi: 10.2196/jmir.2745

PubMed Abstract | Crossref Full Text | Google Scholar

31. Zeiler, M, Kuso, S, Nacke, B, Klesges, LM, and Waldherr, K. Evaluating reach, adoption, implementation and maintenance of internet-based interventions to prevent eating disorders in adolescents: a systematic review. Eur J Pub Health. (2021) 31:ckz130. doi: 10.1093/eurpub/ckz130

PubMed Abstract | Crossref Full Text | Google Scholar

32. Asare, M, Popelsky, B, Akowuah, E, Lanning, BA, and Montealegre, JR. Internal and external validity of social media and mobile technology-driven HPV vaccination interventions: systematic review using the reach, effectiveness, adoption, implementation, maintenance (RE-AIM) framework. Vaccine. (2021) 9:197. doi: 10.3390/vaccines9030197

PubMed Abstract | Crossref Full Text | Google Scholar

33. Cruz-Martínez, RR, Wentzel, J, Asbjørnsen, RA, Noort, PD, van Niekerk, JM, Sanderman, R, et al. Supporting self-management of cardiovascular diseases through remote monitoring technologies: metaethnography review of frameworks, models, and theories used in research and development. J Med Internet Res. (2020) 22:e16157. doi: 10.2196/16157

PubMed Abstract | Crossref Full Text | Google Scholar

34. Stirman, SW, Miller, CJ, Toder, K, and Calloway, A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. (2013) 8:1–12. doi: 10.1186/1748-5908-8-65

PubMed Abstract | Crossref Full Text | Google Scholar

35. Quinn, CC, Khokhar, B, Weed, K, Barr, E, and Gruber-Baldini, AL. Older adult self-efficacy study of mobile phone diabetes management. Diabetes Technol Ther. (2015) 17:455–61. doi: 10.1089/dia.2014.0341

PubMed Abstract | Crossref Full Text | Google Scholar

36. Kim, HS, Yang, SJ, Jeong, YJ, Kim, YE, Hong, SW, and Cho, JH. Satisfaction survey on information technology-based glucose monitoring system targeting diabetes mellitus in private local clinics in Korea. Diabetes Metab J. (2017) 41:213–22. doi: 10.4093/dmj.2017.41.3.213

PubMed Abstract | Crossref Full Text | Google Scholar

37. Ferreira, ES, Franco, FA, Santos, MML, Levcovitz, AA, Dias, MA, Moreira, TR, et al. Effectiveness of a mobile application for monitoring diabetes mellitus and hypertension in the adult and elderly population: systematic review and meta-analysis. BMC Health Serv Res. (2023) 23:855. doi: 10.1186/s12913-023-09879-6

PubMed Abstract | Crossref Full Text | Google Scholar

38. Nilsen, P. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53-79.

Google Scholar

39. Badawy, SM, DiMartino, L, Brambilla, D, Klesges, LM, Baumann, AA, Burns, EN, et al. Impact of the COVID-19 pandemic on the implementation of mobile health to improve the uptake of hydroxyurea in patients with sickle cell disease: mixed methods study. JMIR Form Res. (2022) 6:e41415–5. doi: 10.2196/41415

PubMed Abstract | Crossref Full Text | Google Scholar

40. Berreta, K, Nguyen, C, Stoner, AM, Ridgeway, L, Wilson, A, Fadel, N, et al. A RE-AIM analysis of a mental health app for undergraduate and medical students during the COVID-19 pandemic: a retrospective cross-sectional study. Int J Environ Res Public Health. (2023) 20:6266. doi: 10.3390/ijerph20136266

PubMed Abstract | Crossref Full Text | Google Scholar

41. Breitenstein, SM, Laurent, S, Pabalan, L, Risser, HJ, Roper, P, Saba, MT, et al. Implementation findings from an effectiveness-implementation trial of tablet-based parent training in pediatric primary care. Fam Syst Health. (2019) 37:282–90. doi: 10.1037/fsh0000447

PubMed Abstract | Crossref Full Text | Google Scholar

42. Enggasser, JL, Livingston, NA, Ameral, V, Brief, DJ, Rubin, A, Helmuth, E, et al. Public implementation of a web-based program for veterans with risky alcohol use and PTSD: a RE-AIM evaluation of VetChange. J Subst Abus Treat. (2021) 122:108242. doi: 10.1016/j.jsat.2020.108242

PubMed Abstract | Crossref Full Text | Google Scholar

43. Gaglio, B, Shoup, JA, and Glasgow, RR. The RE-AIM framework: a systematic review of use over time. Am Public Health. (2013) 103:e38–46. doi: 10.2105/AJPH.2013.301299

PubMed Abstract | Crossref Full Text | Google Scholar

44. Gong, E, Sun, L, Long, Q, Xu, H, Gu, W, Bettger, JP, et al. The implementation of a primary care-based integrated Mobile health intervention for stroke Management in Rural China: mixed-methods process evaluation. Front Public Health. (2021) 9:774907. doi: 10.3389/fpubh.2021.774907

PubMed Abstract | Crossref Full Text | Google Scholar

45. Di Sebastiano, KM, Lau, EY, Yun, L, and Faulkner, G. An Evaluation of a Commercialized mHealth Intervention to Promote Physical Activity in the Workplace. Front Public Health. (2022) 10:740350. doi: 10.3389/fpubh.2022.740350

Crossref Full Text | Google Scholar

46. Irvine, AB, Russell, H, Manocchia, M, Mino, DE, Glassen, TC, Morgan, R, et al. Mobile-web app to self-manage low back pain: randomized controlled trial. J Med Internet Res. (2015) 17:e1. doi: 10.2196/jmir.3130

PubMed Abstract | Crossref Full Text | Google Scholar

47. Koot, D, Goh, PSC, Lim, RSM, Tian, Y, Yau, TY, Tan, NC, et al. A mobile lifestyle management program (GlycoLeap) for people with type 2 diabetes: single-arm feasibility study. JMIR Mhealth Uhealth. (2019) 7:e12965. doi: 10.2196/12965

PubMed Abstract | Crossref Full Text | Google Scholar

48. MacDonald, B, Gibson, AM, Janssen, X, and Kirk, A. A mixed methods evaluation of a digital intervention to improve sedentary behaviour across multiple workplace settings. Int J Environ Res Public Health. (2020) 17:4538. doi: 10.3390/ijerph17124538

PubMed Abstract | Crossref Full Text | Google Scholar

49. Quanbeck, A, Gustafson, DH, Marsch, LA, Chih, MY, Kornfield, R, McTavish, F, et al. Implementing a Mobile health system to integrate the treatment of addiction into primary care: a hybrid implementation-effectiveness study. J Med Internet Res. (2018) 20:e37. doi: 10.2196/jmir.8928

PubMed Abstract | Crossref Full Text | Google Scholar

50. Siebenhüner, AR, Mikolasek, M, Witt, CM, and Barth, J. Improvements in health might contradict adherence to mobile health interventions: findings from a self-care cancer app study. J Altern Complement Med. (2021) 27:S-115–23. doi: 10.1089/acm.2020.0111

PubMed Abstract | Crossref Full Text | Google Scholar

51. Tsangaris, E, Hyland, C, Liang, G, O’Gorman, J, Thorpe Huerta, D, Kim, E, et al. Feasibility of implementing patient-reported outcome measures into routine breast cancer care delivery using a novel collection and reporting platform. JAMIA Open. (2023) 6:108. doi: 10.1093/jamiaopen/ooad108

PubMed Abstract | Crossref Full Text | Google Scholar

52. Tran, V, Gwenzi, F, Marongwe, P, Rutsito, O, Chatikobo, P, Murenje, V, et al. REDCap mobile data collection: using implementation science to explore the potential and pitfalls of a digital health tool in routine voluntary medical male circumcision outreach settings in Zimbabwe. Digit Health. (2022) 8:2163. doi: 10.1177/20552076221112163

PubMed Abstract | Crossref Full Text | Google Scholar

53. Van Reijen, M, Asscheman, M, Vriend, I, van Mechelen, W, and Verhagen, E. Users’ perspectives, opportunities, and barriers of the strengthen your ankle app for evidence-based ankle sprain prevention: mixed-methods process evaluation for a randomized controlled trial. JMIR Rehabilit Assist Technol. (2018) 5:e13. doi: 10.2196/rehab.8638

PubMed Abstract | Crossref Full Text | Google Scholar

54. Varnfield, M, Karunanithi, M, Ding, H, Bird, D, and Oldenburg, B. Telehealth for chronic disease management: Do we need to RE-AIM? Stud Health Technol Inform. (2014) 206:93–100. doi: 10.3233/978-1-61499-456-5-93

PubMed Abstract | Crossref Full Text | Google Scholar