Extent, Type and Reasons for Adaptation and Modification When Scaling-Up an Effective Physical Activity Program: Physical Activity 4 Everyone (PA4E1)

Introduction

Physical activity has extensive benefits for health and society (1, 2). One in four adults and four in five adolescents globally are insufficiently active to meet aerobic physical activity guidelines (3, 4). While there are an abundance of evidence-based programs to address physical activity, many of these have been tested under optimal research conditions (5, 6), and few have successfully targeted adolescent physical activity (7, 8). As such, many programs previously tested have utilized technical expertise, skills, resources and infrastructure that are not common in real-world operational environments where they are intended to be implemented (9, 10). Further, research trials often recruit participant groups that differ markedly from those of the target population (11).

As physical activity programs examined in research trials are often unsuitable for replication in more real-world environments, they are frequently changed by end-users as part of efforts to make them more suitable for implementation and scale-up (9, 12). These changes can take two forms – adaptations to a program prior to it being delivered, and modifications that occur during the delivery of the program. Adaptation (8, 13) has been defined as a process of thoughtful and deliberate alteration to the design or delivery of a program, with the goal of improving its fit or effectiveness in a given context (13). Program adaptations can include both adaptations to the evidence-based practices and/or to the implementation support strategies provided to increase adoption of the practices in the setting (such as training for clinicians or teachers who will be delivering the program). Adaptations include those to core components of the program, cultural adaptations, mode of delivery adaptations, target audience adaptations and service setting adaptations (14). Modification has been defined as encompassing any change to a program, whether deliberately and proactively, or in reaction to unanticipated challenges that arise in the context of its delivery (13). Adaptations and modifications can also be made to evaluation methods.

Systematic reviews demonstrate that program adaptations are ubiquitous as part of efforts to scale-up programs in practice. For example, a systematic review of physical activity programs (8) reported that 100% of programs made adaptations to the program tested in an efficacy trial prior to undertaking a trial of its scale-up. The majority of adaptations focused on the “delivery mode” of programs (8, 15, 16), such as giving preference to online or telephone over face-to-face delivery modes, which are often undertaken to enable greater program reach (8). Understanding program adaptations and modifications is important as they can have significant implications to the effectiveness of programs (9, 12, 17). They have been attributed, in part, to a phenomenon labeled “voltage drop” whereby the effects of a program are reduced by 25–50% when they are implemented at scale in real world contexts (8, 16, 17). However, they have also been hypothesized to improve the impact of programs. For example, improvements may be made by allowing tailoring of evidence-based programs and their implementation (i.e., the local culture, historical context, priorities and availability of funding, staffing and resources), strengthening key program components, reducing inequities by improving its cultural relevance, or reducing relative costs via delivery using less expensive modalities (12). Understanding the nature of program adaptations and modifications is also important for the development of explanations about how they may impact program implementation and outcomes as part of trial process evaluations (18).

The Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) (13, 19) was recently developed to support the consistent documentation and reporting of program adaptations and modifications. It provides a taxonomy of classifying adaptations and modifications (13) including what is adapted/modified, the nature of the adaptation/modification, who participated in the adaptation/modification decision, for whom/what is the adaptation/modification made and when it occurred. Despite the existence of FRAME and the need for consistent reporting, both adaptations made prior to program delivery and modifications made during the implementation are often poorly described in research reports (13, 15, 20, 21). That is, individual trials seldom report adaptations for scale-up (prior to program delivery) using consistent taxonomies (13), instead trials rely on descriptions of adaptations that can't be compared between trials (13).

A systematic review of 42 evidence-based public health programs that reported adaptations to the evidence-based program practices (15) found that the most frequent types of adaptation were tailoring (93%) or adding elements (71%). Most commonly these adaptations were to content (100%), context (95%), cultural (74%) and/or delivery (62%). While the review provides useful insights into the frequency of adaptations to the evidence based program practices, it does not explicitly include adaptations to the implementation support strategies used or the evaluation methods (13, 22). Also, the authors relied on published papers to retrospectively code adaptations to evidence-based program practices (15). A limitation of relying on published papers is that sometimes the extent, types, context and reasons for adaptations and modifications may be unclear or absent completely from these documents, remaining instead with those people involved in the scale-up process (15). Additionally, it is also unclear in many programs what modifications occur during delivery, and to the authors' knowledge, no physical activity studies have used a consistent taxonomy to report modifications during program delivery (23). Importantly, prior studies have also not routinely reported who was responsible for program adaptations or modifications, why these were undertaken, and if they were considered to contribute, or detract, from the effects of the program. Such information could be used to help interpret trial findings in implementation-effectiveness studies (13, 22, 24).

In the absence of well-described adaptations prior to delivery and modifications during delivery, we present here a descriptive study of the adaptations and modifications made in the scale-up of an evidence based physical activity program targeting adolescents, Physical Activity 4 Everyone (PA4E1). PA4E1 is a secondary school physical activity program. After an efficacy trial, PA4E1 showed positive results (25–29), PA4E1 was adapted in preparation for scale-up (30). The PA4E1 program includes both an evidence-based program (consisting of seven school physical activity practices) and seven implementation support strategies offered to help schools implement these physical activity practices (implementation support). The aims of the current paper are:

1. To describe the extent, type and reasons for adaptations to PA4E1 that were made for scale-up to the physical activity practices, implementation support strategies and evaluation methods.

2. To describe the extent, type and reasons for modifications during the PA4E1 scale-up trial made to the physical activity practices, implementation support strategies and evaluation methods.

Methods

This research has been conducted and reported in accordance with the requirements of the Standards for Reporting Implementation Studies (StaRI) Statement (Additional File 1) and Template for Intervention Description and Replication (TIDieR) checklist (Additional File 2).

Ethical Approval

The efficacy and scale-up trials have been registered at ACTRN12612000382875 and ACTRN12617000681358, respectively. Ethical approvals were sought from Hunter New England Human Research Ethics Committee (Ref No. 11/03/16/4.05), University of Newcastle (Ref No. H-2011-0210), NSW Department of Education and Communities (SERAP 2011111), Maitland Newcastle Catholic School Diocese, Broken Bay Catholic School Diocese, Lismore Catholic School Diocese, Armidale Catholic School Diocese, and the Aboriginal Health and Medical Research Council (AHMRC).

Stages of Physical Activity 4 Everyone (PA4E1)

An outline of the physical activity practices (evidence based program) for both the efficacy and scale-up trials are shown in Figure 1. The implementation support strategies offered to schools are outlined for both trials in Figure 2 (26, 27, 29–32). As is best practice in implementation science (33), we distinguish between program components, separating the evidence-based program practices (the physical activity practices) from the implementation support strategies, which are designed to assist schools to implement the physical activity practices.

FIGURE 1

Figure 1. Summary of the physical activity practices in the PA4E1 efficacy trial (25–27, 29) and the PA4E1 scale-up trial (30–32).

FIGURE 2

Figure 2. Summary of the implementation support strategies offered to schools in the PA4E1 efficacy trial (25–27, 29) and the PA4E1 scale-up trial (30–32).

Efficacy Trial (2012–2014)

The evaluation methods of the PA4E1 efficacy trial have been reported in a trial protocol (26). Briefly, the PA4E1 efficacy trial was a 2-year (2012–2014) cluster randomized controlled trial involving 10 low-socioeconomic Australian secondary schools (five per group). Hunter New England Local Health District (HNELHD) led PA4E1 (34), supported by two other local health district delivery partners (26, 34) in a research-practice partnership with the University of Newcastle and New South Wales (NSW) Department of Education. PA4E1 had positive effects on students' device-measured moderate-to-vigorous physical activity and unhealthy weight gain (25–27, 29) and was deemed cost-effective (28). The PA4E1 program consisted of seven physical activity practices (Figure 1) to support students to be more physically active (the evidence-based program) and six implementation support strategies designed to embed the physical activity practices within the school environment (Figure 2).

Adaptation Process (2017)

Adaptations were made in 2017 with the goal of scaling up the program (physical activity practices and implementation support strategies) employed in the efficacy trial and testing the effects again as part of a scale-up trial. The process has been reported in more detail elsewhere (30). The scale-up adaptation process sought to retain the effects of the original program by retaining components deemed as core (physical activity practices and implementation support strategies) (25–27, 29) while enabling greater reach (scaling up to more schools). Briefly, to adapt PA4E1 for scale-up, we used a four-stage iterative scale-up process, based on a review of existing models and factors for scaling up public health programs (35) and a scoping review of frameworks for adapting public health programs (36). Firstly, we identified barriers and enablers to the physical activity practices and implementation support strategies. Second, we mapped the identified barriers to the Theoretical Domains Framework (37) and the Behavior Change Wheel (38). Thirdly, we prioritized the components of the program from the perspective of a health service requiring its delivery at scale, considering variables such as affordability, practicability and acceptability (39). Finally, the PA4E1 Expert Advisory Group (comprised of senior health service staff, senior academics, NSW Education sector partners and the PA4E1 project staff) reviewed the prioritized program (physical activity practices and implementation strategies) and made the final judgement regarding the design and components of the resulting PA4E1 program (30).

Scale-Up Trial (2017–2019)

The scale-up trial was a type III hybrid implementation-effectiveness trial (24). Methods have been reported in both a trial protocol (30) and a process evaluation protocol (32). The scale-up trial (2017–2019) tested the adapted PA4E1 program in a larger number of low-socioeconomic secondary schools (n = 49) across a larger geographic area (HNELHD leading three other local health districts, in a research-practice partnership with the University of Newcastle and NSW Department of Education) (34). Program schools were offered seven implementation support strategies (incorporating 23 sub-strategies) to support their adoption of seven physical activity practices (30–32). The scale-up trial recruited 49 schools (24 program, 25 control) (31).

Defining Adaptation and Modification

We operationally defined changes to PA4E1 (inclusive of the physical activity practices, implementation support strategies and the evaluation methods) temporally, as either adaptations which were planned and made prior to the scale-up trial or modifications which were made during the conduct of the trial. The methods are reported by aim. Aim one focuses on adaptations (prior to scale) and aim two focuses on modifications (during implementation of the program in the scale-up trial).

Measures and Procedures

The measures and procedures are reported by aim.

Aim 1: To describe the extent, type and reasons for adaptations to PA4E1 that were made for scale-up to the physical activity practices, implementation support strategies and evaluation methods.

We used the FRAME framework to code the adaptations to the PA4E1 program made for scale-up (13). We applied the FRAME framework coding to the evidence-based practices (physical activity practices), the implementation support strategies (13) and the evaluation methods. The FRAME framework was used to provide a discrete set of codes for each category of adaptation (see Table 2 for a list of codes for each category). Additional coding categories (outlined below) were developed by the author team in line with methods from Rabin et al., (23) to report free-text descriptions of each adaptation.

Table 2 shows the adaptation categories utilized and the response codes. Briefly, these are:

• Description of the adaptation (open text)^*

• What component of the PA4E1 program was the adaptation made to (the physical activity practices, implementation support strategies or evaluation methods)?

• Was the adaptation proposed to have a positive, null or negative impact on the program effectiveness at the time of adaptation?

The following categories were coded for each adaptation from the FRAME framework (6):

• Were adaptations systematic or unsystematic?

• Note: Based on the Model for Adaptation and Impact (MADI) framework (20), we revised the FRAME framework (13) terminology to remove the code “planned” and replace it with the code “systematic.” This emphasizes the importance of how the modification was made (i.e., was it done using a systematic process which involved the use of theory to make the adaptation?). We operationalized the code “systematic” and applied it to adaptations that used both theory and a process to make the adaptation. By contrast, “unsystematic” was the code assigned when either a theory was not used or a process was not used to make the adaptation.

• Were adaptations proactive or reactive?

• Who participated in the decision to adapt? (e.g., program manager, individual practitioners such as in-School Champion)

• What was the goal of adaptation? (e.g., improve feasibility, reduce cost)

• What was adapted? [i.e., context, training and evaluation, implementation and scale-up activity, content (including aspects of the way content was delivered)]

• Context adaptations were made to what? (e.g., format, setting)

• At what level of delivery did adaptations occur? (e.g., unit level - an individual school)

• What is the nature of the content adaptation? (e.g., adding elements, shortening/condensing)

• Were adaptations fidelity consistent (relationship to core elements retained) or fidelity inconsistent?

• Reasons for the adaptation? (including socio-political, organization/setting, provider and recipient reasons).

Certain coding categories were considered “not applicable” to the evaluation methods (as outlined in Table 2).

Firstly, MM identified a list of adaptations between the efficacy and scale-up trial on physical activity practices, implementation support strategies and evaluation by triangulating data from a number of sources. The first source was published PA4E1 research papers: the trial protocol (26), 12 and 24-month outcome papers (27, 29) and cost-effectiveness paper for the efficacy trial (28); and the trial protocol (30), process evaluation protocol (32) and 12-month practice outcome paper (31) for the scale-up trial. Secondly, we drew upon the trial registries for both trials. Finally, to provide important context and resolve discrepancies in recorded adaptations, we drew on the historical knowledge of three authors (LD, EC and RS) who were involved in both trials through meetings with the lead author (MM).

To refine the initial codes, a consensus meeting was held between MM, TM, LD, RS, and EC. Following discussion, MM then finalized the coding for each of the adaptations. Finally, the final codes were agreed upon by email by MM, TM, LD, RS, and EC. Descriptive statistics for each adaptation category were calculated (e.g., the number and type of adaptations that were made).

To synthesize the adaptations, we calculated descriptive statistics for each adaptation category (e.g., the number and type of adaptations that were made) overall, and for adaptations made to physical activity practices, to implementation support strategies and to evaluation.

Aim 2: To describe the extent, type and reasons for modifications during the PA4E1 scale-up trial made to the physical activity practices, implementation support strategies and evaluation methods.

We used the FRAME framework to describe and code modifications made during the delivery of the scaled-up PA4E1 program to the physical activity practices, the implementation support strategies and the evaluation methods.

We used the method as outlined in the process evaluation protocol (32). Throughout the 24-month program (2017–2019), Support Officers involved in the delivery of the program to schools and the PA4E1 Management Team involved in the day-to-day operations of the project continually tracked modifications to PA4E1 during the program in “real-time” by adding them to a Microsoft Excel spreadsheet (23). A monthly meeting (up to 60 min) was held between the PA4E1 Management Team (including at least one Support Officer) to code the identified modifications onto the Stirman et al., (13) framework for modifications in the same spreadsheet (19). MM subsequently updated the coding to reflect the additional categories included within the updated framework by Stirman et al., the FRAME framework (13, 19). This method of ‘real-time ongoing coding’ has previously been found to be feasible to track modifications (23). We used the same coding categories from Aim 1 for each modification (replacing the word adaptation for modification), as well as coding two additional categories. For the category of proposed impact on the program effectiveness at the time of modification (i.e., “positive”, “null” or “negative”), we coded this based on the predicted impact of the modification at the time of coding, rather than the actual or measured impact of the modification.

• Reason for the modification (open text)

• School term the modification was first applied (within the nine school term program).

At the end of the program, the final codes were collated by MM. The final codes were discussed and refined (by RS, EC, LD, TM) to reach consensus. Descriptive statistics for each modification category were calculated (e.g., the number and type of modifications that were made) overall and separately for physical activity practices, implementation support strategies and evaluation methods.

Results

Results are reported by adaptations (aim 1) and modifications (aim 2), respectively.

Adaptations

Aim 1 was to describe the extent, type and reasons for adaptations to PA4E1 that were made for scale-up to the physical activity practices, implementation support strategies and evaluation methods. Table 1 descriptively summarizes the main program adaptations from efficacy trial to scale-up trial for individual physical activity practices and implementation support strategies, including the codes “fidelity consistent or fidelity inconsistent”, “systematic or unsystematic” and “proposed positive, negative or null impact on the project” (as described in the methods). For a more expanded description of adaptations made for the scale-up trial and their coding, see Additional File 3.

TABLE 1

Table 1. Adaptations to each specific component of PA4E1 efficacy trial made for the scale-up trial, including adaptations to individual school physical activity practices and implementation support strategies offered to schools.

Number of Adaptations

A total of 20 adaptations were made to scale-up PA4E1. Eight adaptations were to the school physical activity practices, six to the implementation support strategies and the remaining adaptations were to the evaluation methods (n = 6). Table 2 summarizes the codes according to the FRAME framework for physical activity practices, implementation support strategies and evaluation methods, respectively (13). Additional File 3 outlines each individual adaptation in detail and includes the full set of codes outlined in the methods.

TABLE 2

Table 2. FRAME framework (13) modification codes for adaptations from efficacy to scale-up trial.

The vast majority of adaptations were coded as “systematic” (n = 19), as they were made during the theory-informed iterative scale-up process. By definition, all adaptations (n = 20) were made “proactively” rather than in response to an unknown event or circumstance. Two adaptations were deemed “fidelity inconsistent”, as the core elements or functions had changed as a result of the adaptation (13). Of the 14 adaptations to the practices and strategies, 12 were proposed to have a positive impact on program effectiveness. All adaptations involved both the “Program Manager” and “Treatment/Intervention Team” in the decision-making process, which included the PA4E1 Implementation Team (inclusive of the program manager, project staff and the expert advisory group) as described in the methods.

Types of Adaptations

The goals and types of adaptations varied. The most common goals were to “improve fit with recipients” (n = 8), “improve feasibility” (n = 5), “improve effectiveness/outcomes” (n = 4) and to “reduce cost” (n = 3). Most adaptations were to “content” (n = 13) and “implementation and scale-up activities” (n = 12). Fewer adaptations were to “training and evaluation” (n = 6) or “contextual” (n = 1). All adaptations (n = 20) were made at the “target intervention group” level, meaning that adaptations applied to all schools receiving the program, rather than certain schools or local health districts (e.g., “individual”, “cohort” or “individual practitioner” level). Content adaptations varied, including “substituting” (n = 8) “tailoring/tweaking/refining” (n = 3) and “shortening/condensing” (n = 3).

Reasons for Adaptations

The reasons for adaptations included the broad context of having “funding or resource availability/allocation” (n = 3) and also the PA4E1 Implementation Team having “available resources (funds, staff, technology, space)” (n = 6). School and in-School Champions (i.e., “recipients”) reasons included cultural or religious norms (n = 4), “physical capacity” (n = 2), “motivation and readiness” (n = 2) and “access to resources” (n = 2).

Modifications

Aim 2 was to describe the extent, type and reasons for modifications during the PA4E1 scale-up trial made to the physical activity practices, implementation support strategies and evaluation methods. Table 3 provides a brief description of each modification made to the physical activity practices and implementation support strategies during the scale-up trial delivery.

TABLE 3

Table 3. Modifications to each component of PA4E1 scale-up trial made during the delivery of the program (practices and implementation support strategies).

Number of Modifications

A total of 20 modifications were made during the delivery of the scale-up trial of PA4E1 from 2017–2019. Of these, 16 modifications were made to the implementation support strategies, three to the physical activity practices and one to the evaluation methods. All modifications were deemed “fidelity consistent” and most modifications were proposed to have a positive impact on the effectiveness of the program (n = 15). Table 4 summarizes the modification codes according to the FRAME framework (13). Additional File 4 outlines each individual modification in detail and includes the full set of codes (as outlined in the methods).

TABLE 4

Table 4. FRAME framework modification codes for modifications during the scale-up trial.

Types of Modifications

The vast majority of modifications were “unsystematic” (n = 18), because most modifications did not make deliberate use of theory to make the modification. Two modifications made to support sustainability used theory and a specific process, and therefore were deemed systematic. Most modifications (n = 12) were made “proactively”, however eight modifications were made in response to an unknown event or circumstance and were therefore coded as “reactive”. Most modifications involved the “program manager” (n = 17). “Individual practitioners” participated in half of all decisions (n = 10). Other groups and individuals participated in the decision making process less frequently, including the whole PA4E1 Implementation Team “treatment/intervention team” (n = 3), local health districts “administrator” (n = 2) and in-School Champion “recipients” (n = 1).

The goals and types of modifications varied. The most common goals were to “improve fit with recipients” (n = 7), “increase satisfaction” (n = 6), “improve effectiveness/outcomes” (n = 5), “increase reach or engagement” (n = 4) and to “increase retention” (n = 4). All modifications were to either “content” (n = 14) or “training and evaluation” (n = 6). The nature of content modifications varied, including “adding elements” (n = 5), “tailoring/tweaking/refining” (n = 3) and “removing/skipping elements” (n = 2). Most modifications (n = 15) were made across all schools, in all local health districts, and were therefore coded the “target intervention group” level. The remaining modifications occurred at particular local health districts “cohort level” (n = 3) or individual schools “clinic/unit level” (n = 4).

Reasons for Modifications

Reasons for modifications were primarily related to the “available resources” (funds, staff, tech, space) (n = 14) of the PA4E1 Implementation Team, i.e., the “provider”. The second most common reason was schools and in-School Champions (i.e., recipient) “motivation and readiness” (n = 8). Further reasons for modification were few, but included “historical context” (n = 1) and the “social context” (n = 1) surrounding the “PA4E1 Implementation Team”.

Discussion

Principal Findings

To the authors' knowledge, this is the first study to use a taxonomy to comprehensively report the number, types and reasons for both adaptations made to scale-up a program and also the modifications that occurred during the delivery of the scaled-up program. Our findings show that 20 adaptations were made to the PA4E1 program for scale-up (26, 27, 29), including eight adaptations to the physical activity practices, six to the implementation support strategies and six to the evaluation methods. Most adaptations were proposed to have a positive impact on the effectiveness of the program (n = 12). Additionally, 20 modifications were made during the delivery of the scaled-up program, of which 16 were proposed to have a positive impact on the effectiveness of the program. Most modifications were to the implementation support strategies (n = 16). Given that the use of adaptation and modifications data is being encouraged to explain the findings of scale-up trials (20), the findings of this study provide valuable data, together with detailed process evaluation data (32) to help to explain the findings of the scale-up trial (18, 32).

“Funding and resource availability” was a common reason adaptations and modifications were made. The occurrence of adaptations as part of a research trial, involving the original trial developers, and with a good understanding of the programs and its mechanism of effect may also explain the frequency with which they were fidelity consistent, and thought to have a beneficial impact. The findings underscore the importance of selecting programs that are congruent with the available resources to deliver them at scale, and in doing so, reduce the need for significant adaptations. The use of scalability assessment tools may assist policy makers and practitioners can assist with this process (43).

Both adaptations and modifications to the scale-up of PA4E1 were primarily made to “improve fit with recipients” (i.e., schools). Most adaptations were coded as “fidelity consistent” and “systematic” (i.e., informed by theory and used a process). Such findings are perhaps unsurprising, given they occurred in the context of a funded trial, involving the original trial developers with a good understanding of the program and its mechanism and who employed a considered process to informing adaptations (30). It is also consistent with previous reviews of public health programs (15). Adaptability is a key component of scalable programs, where optimal adaptations are those which are made to fit different contexts and environments while retaining fidelity consistency (12, 13, 21, 43).

Modifications, in contrast to adaptations, were found to be “unsystematic”, likely reflecting the rapid and reactive contexts in which these changes were made by school staff (in-School Champions) and practitioners delivering the program (Support Officers, Program Managers). Previous research into public health prevention programs has similarly found that modifications were unsystematic, suggesting they may be more likely to detract from intervention core functions, resulting in negative impacts on implementation-effectiveness outcomes (44, 45). However, modifications to the program were all coded as “fidelity consistent”, and so retain the intended core functions of the program (outlined in Additional File 3). School staff may have, as a result of the training undertaken as part of the scale-up strategy, or via their existing tacit knowledge, have a good understanding of how the program may impact on the intended outcome and been mindful of this when undertaking modifications. Further research is warranted to explore and better explain such findings.

Reviews of scaled-up physical activity (8) and obesity programs (16) that characterized the nature of adaptations made for scale-up, concluded that adaptations to the “mode of delivery” of programs were particularly prevalent (8, 14, 16). Similarly, we found 13 content adaptations to scale-up PA4E1 and 14 content modifications during delivery of the scaled-up PA4E1.While changes to the delivery modes or content modifications may be perceived as fidelity consistent or improving the overall impacts of a program, for example, by increasing reach and the number of people who may benefit, they may also reduce the absolute effect size of a program (8, 46, 47). That is, the “voltage drop” phenomena whereby the effect sizes of physical activity programs are reduced at scale, may be acceptable from a population perspective if the scaled-up program is capable of reaching and so benefiting (due to delivery mode adaptations) more people, at lower relative cost. Taking a population-level perspective is therefore important when weighing and assessing the potential impact of adaptations or modifications.

Strengths and Limitations

It has been recommended that researchers consider potential causal pathways of modifications, considering both the intended and unintended impacts of modifications on outcomes (20). In line with our process evaluation protocol (32), we have comprehensively described the extent, type and reasons for adaptations and modifications to PA4E1. A strength of this study is the use of real-time tracking of modifications during delivery to record deviations from the planned protocols, which is expected during trial delivery but often not documented well. Indeed, we found the method to be feasible and informative within our study. We found coding using the FRAME to initially be quite difficult, despite the existence of a coding manual (48). We would suggest to future researchers to consider annotating the FRAME framework coding for their own context (13, 48). We also emphasize the importance of going beyond published papers to generate a list of adaptations. By also using the knowledge of those involved in both the efficacy and scale-up trials, we were able to code more accurately the reasons for adaptation. However, a limitation of our research is that we were not blinded to the outcomes of the implementation-effectiveness trial outcomes, which may have influenced our interpretation (18). Additionally, although this study drew upon historical knowledge of those involved in delivering both the efficacy and scale-up trials, it is possible that some specific details were forgotten, given that the efficacy trial was completed in 2014 (7 years ago). Future studies reporting adaptations should therefore aim to do so prospectively (13, 18). Finally, we used the FRAME to report adaptations and modifications to the physical activity practices, implementation support strategies and evaluation methods. Subsequent to our data analysis, the FRAME-IS (Implementation Strategy) was released which is designed for implementation support strategies and organized into modules. The use of both the FRAME (13) and FRAME-IS (22) may have improved our coding of adaptations and reduced the frequency of consensus meetings required. Additional support to use FRAME to code adaptations and modifications to evaluation methods may also be useful.

Conclusions

Adaptations and modifications to public health programs are common. Modifications have the potential to impact the implementation and effectiveness of programs. Our findings underscore the importance of comprehensive reporting of the extent, type and reasons for modifications as part of process evaluations, as this data may be important to the interpretation of trial findings. Making modifications that retain core components but better suit a particular context (program adaptability) is considered to be an important component of successfully scaled-up programs. However, it will be important for future programs to identify project management strategies to mitigate the occurrence of reactive operational modifications that are fidelity inconsistent.

Describing the extent, type and reasons for adaptations and modifications made to public health programs provides valuable process evaluation data to help explain the findings of the program. For example, the data may be used to explain the expected reduction in effect size when they are scaled-up. The comprehensive and transparent description of adaptations and modifications will assist us to generate hypotheses relating to the trial process evaluation and implementation outcome data, which will be explored further.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by Hunter New England Area Human Research Ethics Committee. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author Contributions

MM: conceived the design of the study, led the data analysis, and led the development of the manuscript. MM, JW, EC, RS, and LW: developed the research questions, these were further refined by MM, EC, and LW. RS, EC, LW, and JW: obtained funding for the research. MM: led the ongoing data collection, with significant contributions from EC, RS, TM, and LD. MM, EC, RS, TM, and LD: were involved in the coding of modifications. All authors contributed to the interpretation of findings, provided critical comment on multiple versions of the manuscript, and approved the final manuscript.

Funding

This project is funded by the NSW Ministry of Health, Translational Research Grant Scheme. The NSW Ministry of Health has not had any role in the design of the study as outlined in this protocol and will not have a role in data collection, analysis of data, interpretation of data and dissemination of findings. This work was also supported by Cancer Council New South Wales. The project received infrastructure support from the Hunter Medical Research Institute (HMRI). RS is supported by a NHMRC TRIP Fellowship (APP1150661). LW is supported by a NHMRC Career Development Fellowship (APP1128348), Heart Foundation Future Leader Fellowship (101175) and a Hunter New England Clinical Research Fellowship.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to acknowledge the staff from the Central Coast, Mid-North Coast, South-West Sydney and Hunter New England Local Health Districts involved in delivering this trial.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2021.719194/full#supplementary-material

Abbreviations

AHMRC, Aboriginal Health and Medical Research Council; FRAME, Framework for Reporting Adaptations and Modifications-Enhanced; FRAME-IS, Framework for Reporting Adaptations and Modifications-Enhanced - Implementation HMRI: Hunter Medical Research Institute; HNELHD, Hunter New England Local Health District; MADI, Model for Adaptation Design and Impact; NESA, New South Wales Education Standards Authority; NSW, New South Wales; PA, Physical Activity; PA4E1, Physical Activity 4 Everyone; PE, Physical Education; RT4T, Resistance Training 4 Teens; SAAFE, Supportive, Active, Autonomous, Fair and Enjoyable; SERAP, State Education Research Applications Process.

References

1. Physical Activity Guidelines Advisory Committee. 2018 Physical Activity Guidelines Advisory Committee Scientific Report. U.S. Department of Health and Human Services (2018).

2. The International Society for Physical Activity and Health (ISPAH). The Bangkok Declaration on Physical Activity for Global Health and Sustainable Development. Br J Sports Med. (2017) 51:1389–91. doi: 10.1136/bjsports-2017-098063

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Guthold R, Stevens GA, Riley LM, Bull FC. Global trends in insufficient physical activity among adolescents: a pooled analysis of 298 population-based surveys with 1.6 million participants. Lancet Child Adolesc Health. (2019) 4:23–35. doi: 10.1016/S2352-4642(19)30323-2

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Guthold R, Stevens GA, Riley LM, Bull FC. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1.9 million participants. Lancet Glob Health. (2018) 6:e1077–e86. doi: 10.1016/S2214-109X(18)30357-7

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Finch M, Jones J, Yoong S, Wiggers J, Wolfenden L. Effectiveness of centre-based childcare interventions in increasing child physical activity: a systematic review and meta-analysis for policymakers and practitioners. Obes Rev. (2016) 17:412–28. doi: 10.1111/obr.12392

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Yoong SL, Wolfenden L, Clinton-McHarg T, Waters E, Pettman TL, Steele E, et al. Exploring the pragmatic and explanatory study design on outcomes of systematic reviews of public health interventions: a case study on obesity prevention trials. J Public Health. (2014) 36:170–6. doi: 10.1093/pubmed/fdu006

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Love R, Adams J, van Sluijs EMF. Are school-based physical activity interventions effective and equitable? A meta-analysis of cluster randomized controlled trials with accelerometer-assessed activity. Obes Rev. (2019). 20:859–70. doi: 10.1111/obr.12823

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lane C, McCrabb S, Nathan N, Naylor P-J, Bauman A, Milat A, et al. How effective are physical activity interventions when they are scaled-up: a systematic review. Int J Behav Nutr Phys Act. (2021) 18:16. doi: 10.1186/s12966-021-01080-4

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Milat AJ, King L, Bauman AE, Redman S. The concept of scalability: increasing the scale and potential adoption of health promotion interventions into policy and practice. Health Promot Int. (2013) 28:285–98. doi: 10.1093/heapro/dar097

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Milat AJ, Bauman AE, Redman S, Curac N. Public health research outputs from efficacy to dissemination: a bibliometric analysis. BMC Public Health. (2011) 11:934. doi: 10.1186/1471-2458-11-934

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Averitt AJ, Weng C, Ryan P, Perotte A. Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations. NPJ Digit Med. (2020) 3:67. doi: 10.1038/s41746-020-0277-8

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. (2013) 8:117. doi: 10.1186/1748-5908-8-117

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. (2019) 14:58. doi: 10.1186/s13012-019-0898-y

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. (2016) 51(4 Suppl 2):S124–31. doi: 10.1016/j.amepre.2016.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R, et al. A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci. (2018) 13:125. doi: 10.1186/s13012-018-0815-9

PubMed Abstract | CrossRef Full Text | Google Scholar

16. McCrabb S, Lane C, Hall A, Milat A, Bauman A, Sutherland R, et al. Scaling-up evidence-based obesity interventions: a systematic review assessing intervention adaptations and effectiveness and quantifying the scale-up penalty. Obes Rev. (2019) 20:964–82. doi: 10.1111/obr.12845

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Movsisyan A, Arnold L, Evans R, Hallingberg B, Moore G, O'Cathain A, et al. Adapting evidence-informed complex population health interventions for new contexts: a systematic review of guidance. Implement Sci. (2019) 14:105. doi: 10.1186/s13012-019-0956-5

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: medical research council guidance. BMJ. (2015) 350:h1258. doi: 10.1136/bmj.h1258

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Stirman SW, Miller CJ, Toder K, Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci. (2013) 8:65. doi: 10.1186/1748-5908-8-65

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Kirk MA, Moore JE, Wiltsey Stirman S, Birken SA. Towards a comprehensive model for understanding adaptations' impact: the model for adaptation design and impact (MADI). Implement Sci. (2020) 15:56. doi: 10.1186/s13012-020-01021-y

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Yoong SL, Bolsewicz K, Grady A, Wyse R, Sutherland R, Hodder RK, et al. Adaptation of public health initiatives: expert views on current guidance and opportunities to advance their application and benefit. Health Educ Res. (2020) 35:243–57. doi: 10.1093/her/cyaa014

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. (2021) 16:36. doi: 10.1186/s13012-021-01105-3

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Rabin BA, McCreight M, Battaglia C, Ayele R, Burke RE, Hess PL, et al. Systematic, multimethod assessment of adaptations across four diverse health systems interventions. Front Public Health. (2018) 6:102. doi: 10.3389/fpubh.2018.00102

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. (2012) 50:217–26. doi: 10.1097/MLR.0b013e3182408812

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hollis JL, Sutherland R, Campbell L, Morgan PJ, Lubans DR, Nathan N, et al. Effects of a “school-based” physical activity intervention on adiposity in adolescents from economically disadvantaged communities: secondary outcomes of the “Physical Activity 4 Everyone” RCT. Int J Obes. (2016) 40:1486–93. doi: 10.1038/ijo.2016.107

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Sutherland R, Campbell E, Lubans DR, Morgan PJ, Okely AD, Nathan N, et al. A cluster randomised trial of a school-based intervention to prevent decline in adolescent physical activity levels: study protocol for the “Physical Activity 4 Everyone” trial. BMC Public Health. (2013) 13:57. doi: 10.1186/1471-2458-13-57

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sutherland R, Campbell L, Lubans D, Morgan P, Okely AD, Nathan N, et al. “Physical Activity 4 Everyone” school-based intervention to prevent decline in adolescent physical activity levels: 12 month (mid-intervention) report on a cluster randomised trial. Br J Sports Med. (2015). 50:488–95. doi: 10.1136/bjsports-2014-094523

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Sutherland R, Reeves P, Campbell E, Lubans DR, Morgan PJ, Nathan N, et al. Cost effectiveness of a multi-component school-based physical activity intervention targeting adolescents: the “Physical Activity 4 Everyone” cluster randomized trial. Int J Behav Nutr Phys Act. (2016) 13:94. doi: 10.1186/s12966-016-0418-2

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Sutherland RL, Campbell EM, Lubans DR, Morgan PJ, Nathan NK, Wolfenden L, et al. The Physical Activity 4 Everyone cluster randomized trial: 2-year outcomes of a school physical activity intervention among adolescents. Am J Prev Med. (2016) 51:195–205. doi: 10.1016/j.amepre.2016.02.020

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Sutherland R, Campbell E, Nathan N, Wolfenden L, Lubans DR, Morgan PJ, et al. A cluster randomised trial of an intervention to increase the implementation of physical activity practices in secondary schools: study protocol for scaling up the Physical Activity 4 Everyone (PA4E1) program. BMC Public Health. (2019) 19:883. doi: 10.1186/s12889-019-6965-0

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Sutherland R, Campbell E, McLaughlin M, Nathan N, Wolfenden L, Lubans DR, et al. Scale-up of the Physical Activity 4 Everyone (PA4E1) intervention in secondary schools: 12-month implementation outcomes from a cluster randomized controlled trial. Int J Behav Nutr Phys Act. (2020) 17:100. doi: 10.1186/s12966-020-01000-y

PubMed Abstract | CrossRef Full Text | Google Scholar

32. McLaughlin M, Duff J, Sutherland R, Campbell E, Wolfenden L, Wiggers J. Protocol for a mixed methods process evaluation of a hybrid implementation-effectiveness trial of a scaled-up whole-school physical activity program for adolescents: Physical Activity 4 Everyone (PA4E1). Trials. (2020) 21:268. doi: 10.1186/s13063-020-4187-5

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI): explanation and elaboration document. BMJ Open. (2017) 7:e013318. doi: 10.1136/bmjopen-2016-013318

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Wolfenden L, Yoong SL, Williams CM, Grimshaw J, Durrheim DN, Gillham K, et al. Embedding researchers in health service organizations improves research translation and health service performance: the Australian hunter New England population health example. J Clin Epidemiol. (2017) 85:3–11. doi: 10.1016/j.jclinepi.2017.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Milat AJ, Bauman A, Redman S. Narrative review of models and success factors for scaling up public health interventions. Implement Sci. (2015) 10:113. doi: 10.1186/s13012-015-0301-6

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Escoffery C, Lebow-Skelley E, Udelson H, Boing EA, Wood R, Fernandez ME, et al. A scoping study of frameworks for adapting public health evidence-based interventions. Transl Behav Med. (2018) 19:1–10. doi: 10.1093/tbm/ibx067

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. (2012) 7:37. doi: 10.1186/1748-5908-7-37

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. (2011) 6:42. doi: 10.1186/1748-5908-6-42

PubMed Abstract | CrossRef Full Text

39. Michie S, Atkins L, West R. The Behaviour Change Wheel: A Guide to Designing Interventions. Great Britain: Silverback Publishing (2014).

PubMed Abstract | Google Scholar

40. Lubans DR, Lonsdale C, Cohen K, Eather N, Beauchamp MR, Morgan PJ, et al. Framework for the design and delivery of organized physical activity sessions for children and adolescents: rationale and description of the “SAAFE” teaching principles. Int J Behav Nutr Phys Act. (2017) 14:24. doi: 10.1186/s12966-017-0479-x

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Lubans DR, Morgan PJ, Callister R, Collins CE, Plotnikoff RC. Exploring the mechanisms of physical activity and dietary behavior change in the program x intervention for adolescents. J Adolesc Health. (2010) 47:83–91. doi: 10.1016/j.jadohealth.2009.12.015

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Kennedy SG, Smith JJ, Morgan PJ, Peralta LR, Hilland TA, Eather N, et al. Implementing resistance training in secondary schools: a cluster randomized controlled trial. Med Sci Sports Exerc. (2018) 50:62–72. doi: 10.1249/MSS.0000000000001410

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Lee K, Milat A, Grunseit A, Conte K, Wolfenden L, Bauman A. The intervention scalability assessment tool: a pilot study assessing five interventions for scalability. Public Health Res Pract. (2020) 30:3022011. doi: 10.17061/phrp3022011

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Carvalho ML, Honeycutt S, Escoffery C, Glanz K, Sabbs D, Kegler MC. Balancing fidelity and adaptation: implementing evidence-based chronic disease prevention programs. J Public HealthManag Pract. (2013) 19:348–56. doi: 10.1097/PHH.0b013e31826d80eb

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Mackie TI, Ramella L, Schaefer AJ, Sridhar M, Carter AS, Eisenhower A, et al. Multi-method process maps: an interdisciplinary approach to investigate ad hoc modifications in protocol-driven interventions. J Clin Transl Sci. (2020) 4:260–9. doi: 10.1017/cts.2020.14

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Stewart AL, Gillis D, Grossman M, Castrillo M, Pruitt L, McLellan B, et al. Diffusing a research-based physical activity promotion program for seniors into diverse communities: CHAMPS III. Prev Chronic Dis. (2006) 3:A51.

PubMed Abstract | Google Scholar

47. Hardy LL, Mihrshahi S, Gale J, Nguyen B, Baur LA, O'Hara BJ. Translational research: are community-based child obesity treatment programs scalable? BMC Public Health. (2015) 15:652. doi: 10.1186/s12889-015-2031-8

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Stirman. FRAME Coding Manual. (2020). Available online at: http://med.stanford.edu/fastlab/research/adaptation.html. (accessed April 22, 2021).