This is an open-access article distributed under the terms of the
The United States Veterans Health Administration (VHA) Office of Rural Health funds Enterprise-Wide Initiatives (system-wide initiatives) to spread promising practices to rural Veterans. The Office requires that evaluations of Enterprise-Wide Initiatives use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. This presents a unique opportunity to understand the experience of using RE-AIM across a series of evaluations. The authors conducted a study to document the benefits and pitfalls of using RE-AIM, capture the variety of ways that the team captured the elements of RE-AIM, and develop recommendations for the future use of RE-AIM in evaluation.
The authors first conducted a document review to capture pre-existing information about how RE-AIM was used. They subsequently facilitated two focus groups to gather more detailed information from team members who had used RE-AIM. Finally, they used member-checking throughout the writing process to ensure accurate data representation and interpretation and to gather additional feedback.
Four themes emerged from the document review, focus groups, and member checking. RE-AIM: provides parameters and controls the evaluation scope, “buckets” are logical, plays well with other frameworks, and can foster collaboration or silo within a team. Challenges and attributes for each RE-AIM dimension were also described.
Overall, participants reported both strengths and challenges to using RE-AIM as an evaluation framework. The overarching theme around the challenges with RE-AIM dimensions was the importance of context. Many of these benefits and challenges of using RE-AIM may not be unique to RE-AIM and would likely occur when using any prescribed framework. The participants reported on the RE-AIM domains in a variety of ways in their evaluation reports and were not always able capture data as originally planned. Recommendations included: start with an evaluation framework (or frameworks) and revisit it throughout the evaluation, consider applying RE-AIM PRISM (Practical Robust Implementation Framework) to gain a broader perspective, and intentionally integrate quantitative and qualitative team members, regardless of the framework used.
Almost a quarter of United States Veterans (Veterans) reside in rural areas after their military service and Veterans are overwhelmingly more likely to live rurally than the general population. Rural Veterans are also significantly more likely to enroll in the US Veterans Health Administration (VHA) for healthcare than Veterans who live in more urban areas (
The VHA established the Office of Rural Health (ORH) in 2006 with the goal of conducting, coordinating, promoting, and disseminating research on Veterans that live in rural communities (
In 2017, ORH began requiring evaluations of Enterprise-Wide Initiatives to use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework (
RE-AIM is an implementation framework that considers elements of implementation (such as characteristics of sites and those that provide the intervention) and program performance (such as clinical outcomes). Additionally, the RE-AIM dimensions consider the steps necessary to successfully monitor and improve program implementation and success (
Definitions of RE-AIM dimensions.
| Dimension | Office of rural health definition ( |
|---|---|
| Reach | The absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiative, intervention, or program. |
| Effectiveness | The impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes |
| Adoption | The absolute number, proportion and representativeness of settings and staff who actually initiate a program |
| Implementation | How closely did the facilities and staff adhere to the various elements of an intervention's protocol, including consistency of delivery as intended and the time and cost of the intervention |
| Maintenance | The extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies |
While there are many papers and reports of individual evaluations that used RE-AIM as an evaluation framework ( Document the benefits and pitfalls of using RE-AIM Document the variety of ways that the team captured the elements of RE-AIM Develop recommendations for the future use of RE-AIM in evaluation.
The authors are Masters (JPY, RPK, RRK, CEM) and PhD (AM) educated Health Science Specialists with degrees in Public Health, Social Work, Sociology, Clinical Psychology, and Women, Gender, and Sexuality studies. They have 4–15 years of experience in evaluation and work at the Seattle-Denver Center for Innovation in Veteran-Centered and Value-Driven Care (COIN). The Seattle-Denver COIN has completed six Enterprise-Wide Initiative evaluations, with four more currently underway (
Evaluations of enterprise-wide initiatives.
| Years | Initiative |
|---|---|
| FY18-20 | Accessing Telehealth Through Local Area Stations (ATLAS) ( |
| FY18-present | The Medical Foster Home Program ( |
| FY21-present | Mobile Prosthetic Orthotic Care (MoPOC) ( |
| FY21-present | Oral Telemedicine |
| FY18-19 | Specialty Care Access Network-Extension for Community Healthcare Outcomes (SCAN-ECHO) ( |
| FY18-20 | Simulation Learning, Evaluation, Assessment, and Research Network (SimLEARN) ( |
| FY18-19 | State Veterans Telehealth |
| FY21-present | Telediabetes ( |
| FY16-21 | The Transitions Nurse Program (TNP) ( |
| FY17-20 | Virtual Integrated Multisite PACT teams (V-IMPACT) ( |
One of the authors worked on the project.
The authors took an approach reminiscent of collaborative autoethnography to learn from their previous experiences and inform future evaluations of Enterprise-Wide Initiatives. In collaborative ethnography, the personal experiences of the researchers are the primary data (
Project participation.
| Study design | Document review | Invited to focus group | Participated in focus group | Facilitated focus groups | Identification of themes | Participated in member checking |
|
|---|---|---|---|---|---|---|---|
| Author/project manager/qualitative analyst ( |
4 | 4 | 4 | 4 | – | 4 | 4 |
| Author/qualitative analyst ( |
1 | 1 | – | – | 1 | 1 | – |
| Principal investigator |
– | – | 6 | 3 | – | – | 3 |
| Project manager/qualitative analyst ( |
– | – | 2 | 2 | – | – | 3 |
| Qualitative methodologist ( |
– | – | 2 | 2 | – | – | 1 |
| Quantitative methodologist ( |
– | – | 1 | 1 | – | – | – |
| Qualitative analyst ( |
– | – | – | – | – | – | 3 |
| Quantitative analyst or programmer ( |
– | – | – | – | – | – | 4 |
| Total ( |
5 | 5 | 15 | 12 | 1 | 5 | 18 |
Three principal investigators who were invited to a focus group were also qualitative methodologists and a fourth was also a quantitative methodologist.
Everyone was invited to participate in member checking.
The first step was a document review to identify how each RE-AIM dimension was measured, challenges with using RE-AIM as an evaluation framework, data collection methods, project goals, outcomes of interest, and adaptations to RE-AIM.
The document review was followed by focus groups. Completing the document review first grounded the authors in how RE-AIM was used in the evaluations, which allowed them to ground the focus group questions in the existing documentation and use focus groups to explore personal experiences that were not captured in the existing documents. The aim of the focus groups was to gain a deeper understanding of experiences using RE-AIM to conduct an evaluation.
Finally, the authors also applied member checking techniques to ensure that the identified themes truly reflected the experiences of additional team members who worked on the evaluations that were surveyed.
The authors started by compiling relevant information and materials from prior and current evaluations. Documents reviewed included project regulatory documents, evaluation plans, logic models, local department presentations, and quarterly and annual reports (
Four of the authors (RK, RRK, CEM, JPY) completed a review (
All authors collaborated on focus guide development which was informed by document review. The authors chose to conduct focus groups over individual interviews in order to allow for “depth and detail” through group discussion of experiences with using RE-AIM as an evaluation framework (
Key players from each project, including the principal investigator, project manager, qualitative methodologist, and quantitative methodologist, were invited via email to participate in one of two planned virtual focus groups. Each focus group targeted a subset of projects so teams could build on each other's comments. Author AM, who had less direct experience with evaluating Enterprise-Wide Initiatives than the other authors, facilitated two focus groups over Microsoft Teams. This allowed the remaining authors to participate in one focus group as a project team member and observe the other. A total of 15 team members (including four of the authors) were invited and 12 participated. At least one team member from each identified project participated. The focus group guide was sent to all participants in advance of the focus groups. Focus groups lasted approximately 60 min and were recorded and automatically transcribed in Microsoft Teams. Participants were informed that their participation was voluntary and that they could leave at any time.
After the focus groups, the authors reviewed the recordings and transcripts to identify themes. Each author independently listened to the focus group audio and conducted careful line-by-line reading of the transcript of one of the two focus groups to identify key concepts and meaning units (
The authors presented the identified themes at a standing meeting comprised of team members who evaluated Enterprise-Wide Initiatives. The purpose of this presentation was to check the validity, trustworthiness, and credibility of the analysis. All focus group invitees were invited to participate in member checking and member checking was opened up to other roles that were not invited to focus groups, such as qualitative analysts, quantitative analysts, and programmers (
Finally, the authors shared an early version of this paper with all team members who participated in evaluations of Enterprise-Wide Initiatives. Additional feedback was solicited over email, in a team meeting, and in one additional meeting that was set up to capture the perspectives of team members who were unable to participate in other member checking activities. The themes presented below were synthesized across results of the document review, focus group analysis, and member checking.
This work was deemed quality improvement by the Eastern Colorado Research and Development Committee because it was designed and conducted for the purposes of improving internal VA processes in support of activities carried out in accordance with the VA mission and the institutional requirements.
15 evaluators (including four of the authors, CEM, RPK, RRK, JPY) were invited to focus groups and 12 (including four of the authors, CEM, RPK, RRK, JPY) attended. Nine focus group participants (including four of the authors, CEM, RPK, RRK, JPY) and an additional 11 team members who were not invited to focus groups (qualitative analysts, quantitative analysts, and programmers) participated in member checking. At least one person from each evaluation participated in focus groups and member checking (
Themes are grouped into overarching comments about RE-AIM and specific comments about measuring each element of RE-AIM.
Input from member checking is integrated into the themes and noted where substantial changes were made due to member checking. Overall, member checking strengthened the existing themes and did not identify new themes.
Using RE-AIM in evaluation planning led to early identification of data collection methods and clearly defining each RE-AIM dimension as it related to the subject of the evaluation. Participants found it “really helpful” to do up front work to define RE-AIM dimensions before beginning the evaluation. They also commented that without the RE-AIM framework they could “write a 100-page report.”
“
Even though RE-AIM was seen as useful to control the scope, some participants expressed that it was challenging when there was pertinent information that didn't fit clearly into a RE-AIM dimension. The Office of Rural Health provides a standardized report template and the participants reflected that this led to “stuffing,” “pushing,” or “fitting” findings into RE-AIM. Another challenge was when findings could be reported under multiple RE-AIM dimensions, specifically overlap between adoption and implementation. This made it difficult to decide where to include the information in the report.
“
Other participants commented that RE-AIM was broad by nature and didn't limit the evaluation: “I don't think I would want us to turn away from an outcome that was appropriate for, you know, representing the impact of a program” (Principal Investigator/Qualitative Methodologist). Another participant explained: “let [the RE-AIM dimensions] guide data collection, but don't be constrained in terms of expanding the data collection and analysis” (Principal Investigator/Qualitative Methodologist) These participants advised that this supplemental information that didn't “fit” elsewhere in RE-AIM could be reported in the implementation section.
Participants felt that RE-AIM was logical, and that any good evaluation would have data for each of the RE-AIM dimensions. They discussed “filling the RE-AIM buckets,” or making sure that there was data for each element of RE-AIM, when planning the evaluation. One participant commented that they felt that use of RE-AIM was not overly restrictive because the dimensions were so “high level” and another commented that “if you do a full summative evaluation from soup to nuts, you should have stuff in each of those categories” (Qualitative Methodologist)
“
Even though there is no explicit guidance that every RE-AIM dimension should be given the same amount of attention, participants still felt that it was important to explicitly recognize that there should not be an expectation that each RE-AIM dimension hold the same weight. They shared that the focus will naturally shift for different partners, initiatives/programs, and stages of implementation. Some partners were interested in effectiveness (i.e., clinical outcomes), while others were more interested in adoption (i.e., how to spread the initiative). Participants stated that they work closely with partners to ensure that the evaluation focuses on the most important or salient components, rather than trying to give equal weight to each element of RE-AIM: “
While most participants felt that RE-AIM was a logical framework, they commented that it didn't encompass everything of interest: “
Some participants reflected that starting with RE-AIM fostered collaboration between qualitative and quantitative team members early in the evaluation process. Participants shared that RE-AIM can be an opportunity for intentional alignment around goals and methods if it is used early on and throughout the evaluation.
Other participants commented that use of RE-AIM siloed quantitative and qualitative efforts and without intentionality RE-AIM exacerbated existing silos. They commented that quantitative team members primarily focused on the measures of reach and effectiveness, while qualitative team members focused on the remaining dimensions of RE-AIM. Participants reflected that the team might define how each element of RE-AIM would be captured during the planning phase, but then not come back together again as a full team until reporting. These participants felt that when teams were not intentional, the nature of the RE-AIM dimensions led evaluations to be “siloed” and multi-methods, rather than true mixed methods.
“
During member checking, a third perspective emerged. Some team members shared that they believed that the collaboration or lack thereof was due to the Principal Investigator's style and not the evaluation framework that was used. One of these team members commented that they were not aware of RE-AIM prior to participating in member checking.
All evaluations reported RE-AIM dimensions in ways that were specific to the program that they evaluated. Cross cutting themes regarding measuring RE-AIM were: (1) some outcomes were reported in multiple places, sometimes places that did not seem to fit the definition of the RE-AIM element, (2) some proposed outcomes were dropped because of inability to identify appropriate data sources, and (3) the participants utilized resources from the VA Office of Rural Health (
Themes by RE-AIM element.
| Dimension | Theme |
|---|---|
| Reach | Reporting reach can be misleading—reach in early implementation is not representative of what reach will be when a program reaches maintenance |
| Effectiveness | Economic analysis may not accurately represent effectiveness for new initiatives—costs in early implementation are not representative of ongoing program costs |
| Adoption | Adoption requires context—knowing yes/no whether the program was adopted is not enough, it is important to include information about characteristics of those who do, and do not, adopt |
| Implementation | Additional/Supplementary data can often be included in implementation—data that might not seem to “fit” into RE-AIM is often related to implementation factors |
| Maintenance | It takes time for a program to reach maintenance—“maintenance” data in early implementation is speculative and may not represent actual maintenance |
Some participants identified limitations with the application of Reach, especially in early implementation phases. They commented that the number of patients served in early implementation is not representative of the number of patients that will be served when the initiative reaches maintenance. In these cases, participants provided supporting information in their reports to provide context when reporting reach.
“
The document review revealed that nearly all evaluations reported the number and proportion of eligible Veterans served and included information on representativeness. Other items included provider impressions of Veterans who do and do not participate, potential locations for future sites, the number of encounters, program utilization by service line and modality, and fidelity to key program components. One evaluation team proposed but was ultimately unable to capture the number of referrals to the program, number of patients offered the program, and the number of patients who decline the program.
Some participants noted that cost may not be an appropriate metric for programs that are in early implementation. They reflected that new programs are often working to increase reach, which can lead to increased cost, and do not reflect the cost of the program once it reaches maintenance. Participants reported that one project looked at the prevention of death as an alternative to cost.
“
“
“
Other measures of effectiveness that were identified in the document review included descriptions of the experience of participating in the program from both VHA and Veteran perspectives, perceived quality of care, satisfaction, acceptability, and feasibility. Some evaluations included measures of cost. Other information reported in effectiveness was specific to each program or initiative and included access to care, the frequency and values from clinically relevant lab draws, medications prescribed, adverse events, training results, and relational coordination (
One participant described that adoption was intended to be measured as yes/no and it was challenging to find a metric that “feels adequate” to report adoption. Participants described reporting barriers and adaptations in addition to raw numbers of sites and staff who adopted the initiative to provide context and more authentically represent adoption.
“
The document review revealed that while the evaluation reports and other artifacts included the number of settings that initiated the program being evaluated, they did not always include information about representativeness of staff and settings. Other information included in adoption was barriers and facilitators to site and VHA provider/staff participation, adaptations to the program, services and modalities implementing the program, equipment purchased, and staffing. This was another place where one evaluation planned but was unable to collect data on referrals to the program.
Participants reported that RE-AIM is broad by nature, and that any additional or supplemental data collected would fit “under” implementation in the RE-AIM report and “if it didn't fit RE-AIM, I don't think I would want us to turn away from an outcome that was appropriate for, you know, representing the impact of a program” (Principal Investigator/Qualitative Analyst) One team member explained that even though RE-AIM may seem limiting if it is seen as the only way to design the evaluation, “let [the RE-AIM constructs] guide data collection, but don't be constrained in terms of expanding the data collection and analysis.”
“
The document review uncovered that teams included information about site context and fidelity to core program components as well as adaptations when reporting implementation. They also included information about the number and types of visits conducted, cost, acute care, acceptability, feasibility, and barriers and facilitators to implementation among other measures.
Participants reported that many evaluations that began the evaluation process when in pre-implementation or early-implementation did not reach maintenance before the 3-year evaluation ended and that there often was not robust maintenance data to report. Participants reported that they utilized qualitative data about what the partners thought was necessary to maintain the program and referred to this as “perceived maintenance.” The participants noted that “perceived maintenance” may not predict actual maintenance and associated barriers and facilitators. Some participants shared that another strategy they used was to include maintenance data from the first wave of a program to demonstrate what the expected maintenance might look like.
“
As most of the programs were evaluated before they reached maintenance, teams included perceptions of sustainability, suggestions for future program development, and plans for program sustainment in the maintenance section of their reports.
The purpose of this manuscript was to capture the authors’ teams’ experiences using RE-AIM as an evaluation framework across a series of US Veterans Health Administration Enterprise-Wide Initiative evaluations.
Benefits and pitfalls to using RE-AIM as an evaluation framework (Aim 1) were identified in this manuscript. Overall, participants saw both strengths and challenges to using RE-AIM. Participants reported that RE-AIM provided parameters and controlled the evaluation scope, was logical, and paired well with other frameworks. While using RE-AIM as a framework sometimes seemed to improve team communication in some cases, this was not always true and may have more to do with the Principal Investigator than the RE-AIM framework. Other challenges included the importance of providing additional context, limitations of the RE-AIM dimensions, and challenges when RE-AIM seemed to miss important outcomes.
These challenges can be mitigated by thoughtful identification of ways to capture RE-AIM elements (Aim 2) and developing and implementing recommendations for future use of RE-AIM in evaluation (Aim 3). The document review revealed that some measures were reported in multiple RE-AIM elements, some of which may seem to be the “wrong” place, and that some measures were planned and later dropped. While there are tools available on the RE-AIM website that expand on how to measure the RE-AIM elements, the participants primarily consulted guidance from the funder to guide evaluations. This guidance was not as comprehensive as the tools available on the RE-AIM website. Additional recommendations are shared below.
The overarching theme around the challenges with RE-AIM dimensions was the importance of context. Participants reported that without implementation context, often provided by qualitative data, quantitative data about RE-AIM could misconstrue the actual viability of a program, especially in early implementation. While RE-AIM was initially created as a quantitative framework, more recent work points to the importance of including qualitative data when using RE-AIM (
Finally, many of these benefits and challenges of using RE-AIM may not be unique to RE-AIM and would likely occur when using any prescribed framework. In order to understand benefits and challenges unique to RE-AIM, one would need to conduct a comparative analysis of evaluations that were required to use different frameworks. This work sets up the opportunity to look at the transferability of experience between models.
Several findings from this paper were previously addressed in an article written by the founders of RE-AIM (
Ways that projects measured RE-AIM dimensions.
| Definition | Outcomes | ||
|---|---|---|---|
| Reach | The absolute number, proportion and representativeness of individuals who are willing to participate in a given initiative, intervention or program | Number of patients served | Total number of encounters and average number encounters per patient |
| Provider impressions of Veterans who are served, are offered the program and decline, or are unenrolled due to dissatisfaction | Fidelity (e.g., are the screenings required by the program completed, provider training attendance) |
||
| Proportion of eligible patients served | |||
| Representativeness of patients served | |||
| Program utilization by service and by service modality |
|||
| Potential locations for future sites |
|||
| Effectiveness | The impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes | Risk Ratios (ED Visit, PQI hospitalization, AUDIT-C, PHQ-2, Influenza Vaccination) | Adverse events (e.g., PQI Hospitalizations, emergency room visits) |
| Satisfaction and perceived satisfaction | Change in the number of Veterans served | ||
| Perceived quality of Care | Frequency and outcomes from clinically relevant lab draws (e.g., A1c, lipid panel, vitamin B12) | ||
| [for a training program] Intermediate training results | Feasibility |
||
| Cost analysis (e.g., return on investment, cost of the program compared to an alternative program, cost per site, cost per encounter) | Access to care (e.g., were the screenings required by the program completed? Perceived access from interviews) |
||
| Acceptability | Description of experience |
||
| Medications prescribed | |||
| Relational coordination of site staff | |||
| Adoption | The absolute number, proportion and representativeness of settings and staff who actually initiate a program | Number of VA sites engaged | Services and modalities of care initiated |
| Number of partner sites engaged | Equipment Purchased | ||
| Number of providers engaged | Adaptations |
||
| Barriers and facilitators to site participation | |||
| Staffing | |||
| Implementation | How closely did the facilities and staff adhere to the various elements of an intervention's protocol, including consistency of delivery as intended and the time and cost of the intervention | Barriers and facilitators to implementation | Acceptability |
| Adaptations | Feasibility |
||
| Use of community care | Site context |
||
| Fidelity to core program components | Cost of services (e.g., equipment, staffing, space, travel) |
||
| Attrition from VA care | Number and types of visits conducted; average number of encounters per patient |
||
| Medication adherence | Acute care (hospitalizations, ED visits, UC visits) |
||
| VA Travel Reimbursement | No-show rates |
||
| Maintenance | The extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies | Perceptions of sustainability | |
| Suggestions for future program development | |||
| Plans for program sustainment | |||
Planned but unable to capture.
Reported in more than one element or could be reported in a different element.
The findings in this manuscript echo previous work, such as Kwan et al. (
Unlike this manuscript, Kwan et al. and Quinn et al., reported challenges with data acquisition. While this manuscript identified data acquisition issues in the document review, this was not a salient theme in the focus groups. This may be because this paper examines US Veterans Health Administration programs, and, as the largest integrated health system in the United States, there is a plethora of administrative data available. Additionally, this may not be salient in part due to the limited participation of quantitative team members, further discussed in the limitations.
The biggest limitation to this project was that the authors reviewed their own work. While they took precautions, such as member checking and ensuring that all themes and examples were represented in the data, it is still likely that their own experiences played a disproportionate role in the findings. Review by those not intimately involved in the series of evaluations may identified a different set of themes.
A second limitation was that the RE-AIM framework is a requirement for evaluations of Office of Rural Health Enterprise-Wide Initiatives. This colored the experience of the participants. Focus group participants shared extensively about the requirement to use RE-AIM, the standardized reporting format, and the challenges that some faced when switching to RE-AIM after an evaluation was designed with a different framework. This detail was not included in the manuscript because it was specific to evaluations of Enterprise-Wide Initiatives. Evaluators who independently chose RE-AIM as an evaluation framework likely have different experiences using RE-AIM.
Finally, the perspectives of participants with qualitative expertise (
Recommendations drawn from this analysis include:
Start with an evaluation framework and continue to revisit the framework throughout the evaluation. Don't limit oneself to one evaluation framework; even if a specific framework is required, pairing it with additional frameworks may be necessary to ensure that the evaluation is comprehensive and meets the needs of the partners, other interested parties, and the evaluation funder. Think carefully about applying economic analysis; ensure that team members conducting economic analysis have the appropriate expertise, supplement economic analysis with contextual information, and consider conducting further research on applying economic analysis to programs that have not yet met sustainment. Consider RE-AIM PRISM (Practical Robust Implementation Framework) to gain a broader perspective, as the developers of RE-AIM have endorsed this expanded framework. Be intentional about integrating quantitative and qualitative team members; it is easy to lose sight of the possibilities for mixed methods and collaboration over the course of an evaluation.
In closing, while RE-AIM was not without its challenges, participants found RE-AIM to be a logical and practical framework for evaluating Enterprise-Wide Initiatives, especially when providing contextual information to support findings. More work is needed to understand the experience of using dissemination and implementation frameworks in evaluation. This includes further examination of the experience of quantitative team members, exploring the experiences of evaluators who chose the RE-AIM framework on their own accord, and comparing the experience of using RE-AIM with the experience of using other frameworks. This research will both provide guidance for evaluators who utilize the frameworks and shine a light on gaps and misconceptions that framework developers can address with additional guidance and/or resources.
The datasets generated and/or analyzed during the current study are not publicly available due to the Department of Veterans Affairs’ regulatory compliance. Requests to access the datasets should be directed to
This work was deemed quality improvement by the Eastern Colorado Research and Development Committee because it was designed and conducted for the purposes of improving internal VA processes in support of activities carried out in accordance with the VA mission and the institutional requirements.
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
The VA Office of Rural Health funded the projects that were included in this study but did not fund the study itself. The authors undertook this study as an independent unfunded quality improvement effort.
We would like to thank our colleagues for their contributions to this paper: Sherry Ball, Juliana Barnard, Richard Blankenhorn, Evan Carey, Lexus De-Motta, Etana Disasa, Kelty Fehling, Elise Gunzburger, Leah Haverhals, Ed Hess, P. Michael Ho, Lynette Kelley, Marcie Lee, Chelsea Leonard, Lauren McKown, George Sayre, Lauren Stevenson, Krysttel Stryczek, and Chris Wilson.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
The Supplementary Material for this article can be found online at: