Editorial: Using the RE-AIM Framework and Other Implementation Theories, Models, and Frameworks to Guide the Implementation and Evaluation of Rural Health Innovations

Heather Schacht Reisinger^1,2,3*Teresa Schacht Schacht Damush^4,5,6Erin Schacht Schacht Balkenende^3,7Edwin Wong^8,9Kanika Arora¹⁰Rachael R Kenney¹¹Laura Taylor¹²Monica Matthieu^13,14

• ¹Center for Access and Delivery Research and Evaluation, United States Department of Veterans Affairs, Iowa City, United States
• ²Department of Internal Medicine, Carver College of Medicine, University of Iowa, Iowa City, United States
• ³Implementation Science Center, University of Iowa, Iowa City, United States
• ⁴VA Center for Health Information and Communication, Indianapolis, United States
• ⁵Department of General Internal Medicine and Geriatrics, Indiana University School of Medicine, Indianapolis, United States
• ⁶Regenstrief Institute Inc, Indianapolis, United States
• ⁷Center for Access and Delivery Research and Evaluation (CADRE), Iowa City VA Health Care System, Iowa City, United States
• ⁸Center of Innovation for Veteran-Centered and Value Driven Care, VA Puget Sound Health Care System, Seattle, United States
• ⁹University of Washington Department of Health Systems and Population Health, Seattle, United States
• ¹⁰Department of Health Management and Policy, College of Public Health, University of Iowa, Iowa City, United States
• ¹¹Center of Innovation for Veteran Centered and Value Driven Care, VA Eastern Colorado Health Care System, Aurora, United States
• ¹²US Department of Veterans Affairs Office of Patient Care Services, Washington, United States
• ¹³Center for Mental Healthcare & Outcomes Research, Central Arkansas VA Health Care System, Little Rock, United States
• ¹⁴Saint Louis University School of Social Work, St. Louis, United States

While many of the challenges facing rural health are well described [1][2][3][4][5] -from divestment 6 and disparities 7 to long travel distances to care [8][9][10] and workforce shortages 11,12 -the role implementation science can play in addressing those barriers is less well-addressed. 13,14 Theories, models, and frameworks can guide systematic planning and evaluation of implementation that addresses these barriers. 15 They also allow for comparison across innovations and implementation contexts through the examination of shared constructs. 13 The goal of this Research Topic and its compilation of articles is to examine challenges of implementation and evaluation in the rural healthcare context, using theories, models, and frameworks to guide the discussion.In 2006, the United States Congress passed 38 U.S. Code § 7308 thereby establishing the Department of Veterans Affairs (VA) Office of Rural Health (ORH). The mission of VA ORH is to improve the health and well-being of rural veterans through research, innovation and dissemination of best practices. ORH fulfills this mission with three pillars: 1) to promote system-wide and community care solutions for rural veterans, 2) to reduce rural health care workforce disparities, and 3) to enrich rural veteran health research and innovation. One of the ways VA ORH supports its mission is through the funding of enterprise-wide initiatives (EWIs) that seek to spread evidence-based interventions and best practices to rural veterans across the United States (https://www.ruralhealth.va.gov/providers/enterprise_wide_initiatives.asp). Integrated into the EWI program is an evaluation requirement guided by the planning, evaluation, and implementation framework, RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance). 16,17 ORH's EWI program's use of the RE-AIM framework to structure its evaluation requirements presents a unique opportunity to examine real-world implementation of innovations through a standard lens. In most implementation research, the investigative team decides on the intervention, setting, and population; designs the research questions; determines the implementation strategies; and oversees the conduct and analysis of the methods to reach the results. In the context of ORH's EWIs, evaluation teams, often with considerable health services and implementation science expertise, are paired with VA clinical operational and field-based leads to test the implementation of best practices and innovations. In partnership, the operational, field-based, and evaluation teams decide on the evaluation design and outcome measures and the evaluation team then conducts the agreed upon evaluation. The impetus for this Research Topic was to bring together a variety of EWI evaluations and provide examples of how using the RE-AIM framework may lead to broader lessons learned for large-scale implementation of rural health innovations. The Research Topic was also an opportunity to examine other large-scale evaluation programs that focus on rural health, particularly those that have integrated theories, models, and frameworks to guide their efforts. The result is a compilation of 19 papers at the intersection of rural health and implementation science primarily in the VA context, although it includes large-scale programs in the U.S. community health sector (Melhado et al; Petermann et al) and a systematic review focused on rural youth accessing health care across the 54 countries in Africa (Gbaja-Biamila et al). Importantly, the lessons learned are cross-cutting and provide insight into directions implementation science could take to improve rural health globally.We compiled Table 1 to assist researchers in quickly determining which articles in the compilation could best inform their work. Overall, 15 of the 19 articles applied RE-AIM as their framework, which is not surprising given the focus of the special issue and the number of publications on EWIs (n=14). One additional publication utilized RE-AIM in the context of the Practical, Robust Implementation and Sustainability Model (PRISM). Two articles applied Consolidated Framework for Implementation Research (CFIR) and one used the Critical Incident Technique as a framework to more fully examine sustainability.The strength of the majority of publications applying the same framework is that it makes it easier to compare the variation in how a single construct was applied for each individual construct. For example, reach is the first construct of the RE-AIM framework, and this focus allows other implementation scientists, rural health researchers, and policy makers to compare multiple issues related to reach. For example, are there common barriers or facilitators to reaching an a rural-dwelling population? Are there particular implementation strategies that help programs reach this population? Interestingly, 18 of the 19 articles in the special issue address barriers to implementation in some way and what is unique about this compilation is that all are directly related to the implementation process in rural settings.In additional, several articles discuss the strengths and weaknesses of applying specific models to evaluating implementation. Chasco et al and Kenney et al examine the use of RE-AIM across multiple EWIs and describe the challenges of defining and applying the five RE-AIM domains. Melhado et al and Leonard et al elaborate on the use of frameworks (RE-AIM and PRISM+RE-AIM) to iteratively improve implementation of the interventions they focus on in their articles. Overall, most articles reflect Damush and colleagues review that "the pragmatic application of the RE-AIM framework to guide implementation evaluations is appropriate, comprehensive, and recommended for future applications."From a methodological perspective, theories, models, and frameworks are not prescriptive. While a slight majority (n=7) of the articles used qualitative methods to examine the constructs in each program, six relied on a mix of qualitative and quantitative approaches. Quantitative methods were used exclusively in five of the publications and included surveys, validated instruments, and administrative data to answer questions related to the constructs. This compilation of articles provides readers with a rich set of measures across constructs, which will allow researchers to compare these measures and make decisions in their own work about the most effective approaches to use. Our comparison of the articles in this compilation are only an initial review. Other comparisons could be made such as a more in-depth review of barriers to shed light on barriers specific to rural implementation. We look forward to learning of the ways the compilation of these articles together contributes to the scholarship in implementation specific to rural settings. Finally, we anticipate the ability to read these articles as a whole will lead to additional research, evaluation, and policy questions to support improved implementation and outcomes for rural populations. Our goal was to demonstrate the value of applying theories, models, and frameworks in rural health settings and promote the use of implementation science to improve the care and health outcomes of rural patients and community members. We hope this Research Topic and its articles contributes to this conversation.