The Building Blocks of Implementation Frameworks and Models in Primary Care: A Narrative Review

Abstract

Background: Our aim is to identify the core building blocks of existing implementation frameworks and models, which can be used as a basis to further develop a framework for the implementation of complex interventions within primary care practices. Within the field of implementation science, various frameworks, and models exist to support the uptake of research findings and evidence-based practices. However, these frameworks and models often are not sufficiently actionable or targeted for use by intervention designers. The objective of this research is to map the similarities and differences of various frameworks and models, in order to find key constructs that form the foundation of an implementation framework or model that is to be developed.

Methods: A narrative review was conducted, searching for papers that describe a framework or model for implementation by means of various search terms, and a snowball approach. The core phases, components, or other elements of each framework or model are extracted and listed. We analyze the similarities and differences between the frameworks and models and elaborate on their core building blocks. These core building blocks form the basis of an overarching model that we will develop based upon this review and put into practice.

Results: A total of 28 implementation frameworks and models are included in our analysis. Throughout 15 process models, a total of 67 phases, steps or requirements are extracted and throughout 17 determinant frameworks a total of 90 components, constructs, or elements are extracted and listed into an Excel file. They are bundled and categorized using NVivo 12© and synthesized into three core phases and three core components of an implementation process as common elements of most implementation frameworks or models. The core phases are a development phase, a translation phase, and a sustainment phase. The core components are the intended change, the context, and implementation strategies.

Discussion: We have identified the core building blocks of an implementation framework or model, which can be synthesized in three core phases and three core components. These will be the foundation for further research that aims to develop a new model that will guide and support intervention designers to develop and implement complex interventions, while taking account contextual factors.

Introduction

Initiating and sustaining change within primary care is challenging (1). Most change that is introduced in primary care takes the form of a complex intervention, meaning that it involves concepts that are rather difficult to measure and its components are often interconnected (2, 3). Nowadays, there are increased efforts to shift toward a more patient-centered approach (1), as this proves to improve disease outcomes and quality of life (4). However, such a shift highly challenges current primary care practices and there is therefore no consensus on how to best implement it (5). This indicates a gap between scientific evidence and actual practice: an evidence-to-practice gap (3). This can also be referred to as “the black box of knowledge translation” (6), meaning that much uncertainty exists about understanding why evidence-based practices do not find their way into real world settings and investigating how such complexities can best be approached.

Concrete initiatives and strategies for implementation often do not match with targeted problems (7). In the end, too much is expected from practitioners' ability and goodwill to consult, interpret, and adapt their practices in line with best evidence of research findings (8). The World Health Report 2008¹ stated that “providing a sense of direction to health systems requires a set of specific and context-sensitive reforms that respond to the health challenges of today and prepare for those of tomorrow.” It is thus key to carefully define specific interventions that aim to transform current practices, while at the same time tailoring them to local circumstances (9, 10). To do this, it is essential to gain insight in the process of implementation as well as in potential barriers and facilitators that might hinder or support the implementation process. This is studied in the field of implementation science, which is “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice(11).” The goal of implementation science is to close the gap between evidence-based practices and the extent to which research findings are integrated into real world settings and practices (3, 12).

Within the field of implementation science, many theories, models and frameworks have been created by various disciplines. Moreover, there is a variety of guidelines and tools aimed at facilitating the integration of knowledge of implementation science into either the development or the initiation of interventions and how to document this process. Examples are the ImpRes tool (13), NCEC Implementation Guide & Toolkit for National Clinical Guidelines (14), RNAO Toolkit: Implementation of Best Practice Guidelines (2^nd ed). (15), STaRi Standards for Reporting Implementation Studies (16) and Implementation Research Logic Model (17). However, the landscape of implementation science is rather difficult to navigate, as there is a lack of guidance for selecting theories, frameworks, models, or tools that best fit specific implementation objectives (18). A first step toward a better comprehension of such guidance on implementation efforts and to focus on concepts that are more meaningful to the actors in the field, is to gain better understanding in the common thread throughout the wide variety of models and frameworks that form the basis of such tools.

Current approaches to guide the implementation process are mainly characterized by a single-discipline, medical perspective in which a limited number and types of barriers are taken into account (19). This is insufficient to provide a deeper understanding of implementation success or failure or to increase the chance of success of the implementation (20). Existing frameworks and models tend to incorporate a selection of barriers, but do not allow to give more guidance about their validity or relative importance in specific contexts (20). Moreover, many frameworks and models remain very abstract and fall short in giving concrete guidance for intervention designers on how to navigate the implementation process (21). As many of such frameworks or models remain untested, this again questions their operability (7).

Therefore, an overarching framework is needed that provides both an explanatory approach (3), but also allows to prioritize those variables that are essential to achieve implementation success (22). This means that such a framework should provide a pathway that clarifies the core phases and steps throughout an implementation process and that highlight the core constructs that, within each phase, need to be defined, acted upon, and reflected upon. These phases and constructs should be accessible and meaningful to actors that will conduct implementation efforts. It is key for such framework to transcend disciplines and to bundle insights from different approaches (7).

This research is a first step in the development of a generic framework that incorporates such an approach. We therefore looked into existing theories, models, and frameworks from implementation science and combined insights across various disciplines. The similarities and differences between various frameworks informed us about the main building blocks of such frameworks and about how and why they differ. In doing this, we were guided by a rather broad research question: “What are the main components of implementation frameworks and models in order to structure and guide implementation processes?” This resulted into the identification of core building blocks that form a common thread throughout implementation models and frameworks. Such synthesis will in future research help to develop an overarching model that puts forward clear and meaningful constructs for intervention designers, and that provides both a pathway as well as an explanatory structure to define, act, and reflect upon each component of a complex intervention.

Methods

To determine the building blocks of an overarching implementation framework, we conducted a literature review. Various disciplines were represented in the included literature, for which the initial search had been conducted by a multi-disciplinary team of medical researchers, sociologists, social work, and agogic sciences. We opted for a narrative review, which can be defined as “comprehensive narrative syntheses of previously published information (23)” and which helps to “pull many pieces of information together into a readable format (23).” This reviewing technique is particularly helpful for grasping a broad perspective on a topic; it enables us to transcend a purely medical view on primary care and incorporate other perspectives such as social welfare. Moreover, since the field of implementation science is rather fragmented and consists of a wide range of sources, it requires a wider scoping (24). Instead of focusing on a more rigor methodology to answer a very specific, narrowly-focused research question (24), a narrative review allows for interpretation and critique, aiming to deepen the overall understanding of the subject specifically targeted at our problem (24). This corresponds to our goal to identify and possibly simplify the complexities of implementing an intervention by extracting the core phases and components that are common in most models. According to Green, Johnson and Adams (23), a successful narrative review synthesizes available evidence in relation to a topic and present it in a structured way, conveying a clear message. Our aim is thus to provide an overview of existing implementation frameworks and models and to analyze how they are structured and build.

Our initial search started with articles that were key in identifying other models and frameworks: Nilsen (25) which categorized many frameworks and models and Damschroder et al. (26) which provided a list of references on which the consolidated framework for advancing implementation science was based. Our search continued with consulting the three databases PubMed, Web of Science, and Google Scholar, which are most commonly used in this type of literature. The key words that were used are listed in Table 1. Article titles and abstracts were screened for references about a specific framework, model, or theory for implementation, followed by an additional search for theoretical papers on these frameworks, models, or theories. Subsequently, the search terms were adapted and redefined based upon our findings, thus creating an iterative process that ensures covering literature in a comprehensive way (27). Also, a snowball approach was used and additional literature was found in the references of the papers.

Articles were searched for and consulted between October 2019 and May 2020. They were mostly published between the years 2000 and 2020, but we did include some older source material if a model or framework was considered to be relevant (e.g., the paper was often referred to by other relevant articles). All articles were available as full text in English. We looked for articles which primarily consisted of a theoretical elaboration (and/or application) of a specific framework or model. Frameworks and models that were highly targeted toward a single case or strategy were excluded, as they were difficult to generalize for overall primary care settings.

To compare and analyze the frameworks and models, they were listed and classified according to Nilsen's (25) categorization (see: Table 2). We built our analysis upon process models and determinant frameworks, as they allowed to extract clear steps, actions, barriers, and facilitators that can be transformed into guidance for intervention designers, which was the main aim of our research. For additional understanding of the component evaluation that came up in several models and frameworks, we also looked into three evaluation frameworks. Several classic theories [e.g., Theory of Diffusion (28)] and implementation theories [e.g., Normalization Process Theory (29)] were initially identified, but were not included in our analysis as their approach and structure did not match with our goal to extract clear building blocks of an implementation process that could be used to reconstruct a generic framework.

To analyze, all relevant frameworks and models were listed in an Excel file, with an overview of how they were constructed. For process models, their main phases (steps, stages) were listed, together with relevant details or components within the process they described. For determinant frameworks, the main components (constructs, elements) were listed, together with any details or further clarification about each of the components described. The first step to analyze was to bundle each of the phases or components that had a similar approach or meaning. This was done by the main researcher and validated by the three senior researchers. An overarching concept was appointed to each group of concepts. Then, NVivo 12© was used to structure the main themes and concepts and to analyze their similarities and differences. The overarching concepts were entered as the main nodes in NVivo 12©, whereby details or explanation about each concept from the different models and frameworks were again coded when we noticed overlap with approaches from different frameworks or models. By structuring the phases and components this way and by analyzing the details that were given for each component, we could synthesize it into core building blocks.

Results

Fifteen process models and 17 determinant frameworks were identified. Four models had characteristics of both a process model as well as a determinant framework: the Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors (22), the Consolidated Framework for Implementation Research (26), The Ottawa Model of Health Care Research (30) and the Generic Implementation Framework (19). The frameworks or models focus on various domains. They were either developed specifically to apply within a certain research domain or development was based upon a single discipline. Table 3 gives an overview of the process models and determinant frameworks that were incorporated in our analysis per research domain. As we have only included English literature, this is largely represented in the geographical distribution of the included literature: 18 articles derive from authors affiliated with institutions located in the United States of America, 5 in the United Kingdom, 2 in Canada, 1 in Australia (in collaboration with a Spanish and Portuguese institution), 1 in Ireland, and 1 in Sweden.

Through analysis of both process models and determinant frameworks, we were able to grasp (1) a logical pathway in which different actions need to be taken in order to successfully implement a complex intervention, and (2) the main building blocks of which the intervention consists.

Table 4 gives an overview of the 15 process models with the main phases, steps, or requirements we could detract in each model (67 in total) and Table 5 gives an overview of the 17 determinant frameworks and the main components, constructs, or elements that were put forward in these frameworks (90 in total). This served as a basis on which we detracted the common thread in each of these models and frameworks. We identified three main phases which most models have in common: a development phase, a translation phase, and a sustainment phase. Throughout all process models, 54 phases, steps, or requirements could directly be linked to these three phases. We also identified three main components: the intended change, the context, and the implementation strategies. A total of 67 components, constructs, or elements from all determinant frameworks could be directly linked to these three main components (see: Table 5). Thirteen components from 10 different process models could also be linked to these three main components (see: Table 4). Additionally, 17 components from 10 different determinant frameworks could indirectly be linked to the three main components as either outcomes, actors or processes (see: Table 5), leaving only 6 components that were not linked to the core phases and components we identified.

The three core phases we identified simplify the implementation process and are relevant to distinguish between different actions that need to be taken at different points in the process. The three components we identified are the core building blocks of the intervention: the way these components are approached and interact with each other will determine implementation success. Therefore, intervention designers need to reflect on how to approach each of the components within each of the phases.

Phases of an Implementation Process

To examine different phases of an implementation process, we look at process models, as defined by Nilsen (25). Such models are built to make sense of the different phases or steps of the implementation process of an intervention (25). The goal is to construct and clarify a “logical pathway” that can give concrete guidance for intervention designers. Many models were designed with the objective of translating research evidence into real world practice (33, 39, 44, 46) or the so called shift from knowledge to action [cfr. Wilson et al. (46)]. They tend to depart from an evidence base that needs to be translated into real world settings (33, 39, 44, 46). Other models incorporate a research development phase (38, 46, 47) in which best practices are still to be defined.

We find variation among models as to what is viewed as the main process of implementation. In some models such process takes the form of a stepwise approach to ensure successful implementation of an intervention (30–32, 42, 49, 51). Nilsen (25) calls these action models. They are built upon critical steps or phases that need to be followed or focused upon in order to reach successful implementation. These main phases or steps can either be aimed at the implementation process itself (31, 32) or at the process of using research to initiate change (42, 51). In such models, key drivers or components tend to be highlighted that are necessary for change (33, 49) and/or they have a thorough focus on those strategies that will lead to sustainable change, which is referred to as general implementation strategies (31), transfer strategies (30), capacity-building strategies (32) et cetera.

Another approach for describing a process is to have models differentiate between the main phases of how implementation efforts takes form, in order to make sense of the implementation process itself (22, 38, 43, 45–47). These models describe similar phases. They distinguish between either a development (38, 47), preparation (43) or exploration phase (22, 45), a pre-adoption phase [such as piloting (38), installation (45), or the intent/decision to adopt (22, 43, 47)], an actual implementation- (22, 38, 45, 47) or translation phase (43, 46) and a sustainment (22) or institutionalization (46, 47) phase. We reduce these models to three core phases: a development phase, a translation phase and a sustainment phase—as depicted in Table 6. This is a simplification that is relevant for intervention designers and practitioners, as these phases make most sense to them as distinct phases that require other types of action from them.

Development Phase

The development phase is the initial phase in which preparatory activities are conducted in order to successfully introduce the intervention. In the different models, various elements are considered to be relevant in this initial phase, which leads to a variety of actions that can be taken to prepare for and develop an intervention. Overall, the development phase comprises:

• Synthesizing or collecting research evidence on which an intervention can be based;

• Exploring the host setting;

• Considering the overall fit of an intervention within a particular setting;

• Ensuring readiness and intend to adopt the intervention.

Most models require that intervention designers synthesize existing evidence (38, 39, 42, 44), or that they conduct their own (discovery) research (44, 46). This will lead to either a theory (38), approach or practice (46), or research findings that can be translated into an evidence based practice (EBP) standard (42) or guidelines (44). Other models have a different focus and depart from the idea of planning (26) for an intervention or a more general exploration phase (22, 45). This is less focused on research translation and more intended to gain awareness of an issue (22), and to explore practices and implementation strategies that might respond to this issue (22). Exploration could also refer to assessing the feasibility of implementation intentions or examining the readiness of the setting in which an intervention should take place (45). This is in line with Meyers, Durlak and Wandersman (32) who mention the importance of “initial considerations regarding the host setting,” which refers to exploring whether there is a fit between an intervention and the host setting.

The fit between an intervention and the host setting (32, 45) can be linked to the need to asses contextual factors in this initial development phase (10, 53). The Ottawa Model of Health Care Research refers to “assess” as a first step, which means that the implementation environment, potential adopters, and the evidence-based innovation itself have to be examined (30). This is relevant when trying to assess the feasibility and compatibility of the intervention within a specific context. Pronovost et al. (39) mention a barrier assessment, which is a similar approach as the Conceptual Model of Evidence-Based Practice Implementation in which much emphasis is placed on mapping various hindering and promoting context variables in order to increase implementation success (22). These models recognize the importance of scanning contextual variables to identify barriers and facilitators that will affect implementation efforts.

Lastly, some models incorporate the decision or intend to adopt as a key element of the initial phase (22, 30, 43, 47). Lukas et al. (49) refer to this as the “impetus to transform,” which indicates that the decision to adopt a certain intervention is affected by various elements (22). This relates back to overall practitioner readiness (45), and the fit between the intervention and the setting in which it will be implemented (32, 43). According to the Quality Implementation Framework, a key step in the initial phase is also to create a structure for implementation (32). This can mean having a plan for implementation (32), but also to form a team that is dedicated to ensure implementation of an intervention (32, 42). The CFIR also recognizes the importance of engaging different actors that are involved in the intervention and views it as one of the core activities in the first phases of developing an intervention (26).

Translation Phase

Many frameworks refer to an implementation phase (22, 38, 42, 45, 47). It can also be called executing (26), adoption (30), improvement initiatives (49), or practice integration (44). Following the definition of Blanchard et al. (45), the core of this phase is to integrate the intervention into everyday practice, relying on the preparatory work started in the initial phase. We decided to follow the approach of the NCCDPHP Knowledge to Action Framework for Public Health (46) and the Stetler Model of Research Utilization (43) in which this phase is called the translation phase. They view the implementation process as translating research into practice. The core of these phases is however similar: it refers to the entire process of putting research into practice (46), thus implementing change into real world settings. In short, the actions that are key within the translation phase are:

• Introducing the intervention by applying the strategies as defined in the development phase;

• Monitoring how different components interact with each other to ensure continuous improvement.

All models with a translation phase will agree that key activities within this phase are applying those strategies (30, 45) or types of support (42, 46) that have been defined in the development phase, in order to introduce the intervention. For example, training or coaching is organized (45, 46), leadership- or communication structures are put in place (42, 49), technical assistance is provided or financial resources are made available (46). The Ottawa Model of Health Care Research (30) sees this as a monitoring phase, which means that strategies for introducing and implementing the intervention are to be observed and adjusted if necessary. Within the Research Utilization Model (47), the term “adaptation” is introduced, which means that “over time, an innovation, the social system into which it is introduced, or both, may change or be modified to facilitate use of the innovation.” This suggests that interaction is expected between the intervention, the strategies used and the context or setting in which the intervention takes place.

Sustainment Phase

Seven process models that we included in our analysis mention some form of sustainment phase. Aarons et al. (22) directly incorporate a sustainment phase and define it as “the continued use of an innovation in practice.” This corresponds with what is named the “institutionalization phase” in the NCCDPHP Knowledge to Action Framework for Public Health (46) and the Research Utilization Model (47). Institutionalization of an intervention means that the intended change within an intervention becomes an established activity or norm within the setting it is implemented (46). It becomes integrated into the routines and practices of this setting (47), and it should be ensured that the intervention is applied to all of whom it is aimed (39). Central in the sustainment phase is:

• Applying the strategies as defined in the development phase to help sustain the intervention;

• Reflecting upon the actions taken and ensuring continuous improvement.

Indeed, the aim of the sustainment phase of an intervention is that the intended change is maintained and becomes part of the daily routines and practices. This goes beyond a mere adoption of an intervention. The Organizational model for transformational change in health care systems (49) incorporates a similar idea, which is referred to as integration. Blanchard et al. (45) also ^*speak of integration of new learnings into practice, which they call full implementation. All of these imply that an intended change is adopted and in time harmonizes with, or replaces previously existing practices and activities.

A sustainment phase is also the phase in which continuous improvements ensure a fit between the intervention and the setting in which it is implemented. The Quality Implementation Framework (32) sees the improvement of future applications as the core of this final phase. This is learning from experience. Through reflection and feedback from the setting in which the intervention is introduced, strengths, and weaknesses of the intervention can be detected and acted upon (32). For Blanchard et al. (45) this implies achieving fidelity and improving outcomes. This phase can directly be linked to evaluation, which four process models include as a separate phase.

This notion of continuous improvement can be linked to reflection and evaluation as a part of the process. Several process models include evaluation or measuring performance and outcomes as a phase of the implementation process, for example in the Medical Research Council guidance (38), the CFIR (26), the Ottawa Model of Health Care Research (30), the IOWA model (42), the ACE Star Model of Knowledge Transformation (44), the Stetler Model of Research Utilization (43) and Advancing Understanding of Mechanism of Change in Implementation Science (31), and the model for large scale knowledge translation (39). These frameworks or models generally include minor guidance about how to assess success or failure. There are however also frameworks that are designed specifically to guide the evaluation process, examples of which are the RE-AIM framework (54), the PRECEDE-PROCEED model (55) and the Implementation Outcomes Framework (IOF) (56).

Framework Components

Throughout the three phases of the implementation process, we distinguish components that have to be taken into account within each phase. Therefore, we looked into what Nilsen (25) calls determinant frameworks. These are designed with the intent to understand and explain what influences implementation outcomes, and thus provide information on which components to focus for implementation success. Some frameworks tend to mainly focus on enlisting relevant context variables [e.g., Theoretical Domains Framework 2.0 (51)], while others also specify the relationships and interactions between types of determinants (25). These frameworks provide valuable input when describing different types of context variables that might hinder or facilitate intervention efforts.

Table 7 gives an overview of how various determinant frameworks refer to the three components that we have extracted: intended change, context and/or strategies. They will provide further guidance on how to understand and work with these elements and how they can affect implementation outcomes.

Intended Change

The intended change deals with any conscious change into current practices of primary care providers or any actions that actors undertake (57), which are expected to solve a care or quality gap (58). This can take the form of a task-oriented change in practice (33), require behavioral change (59) either at individual or group/team level (40) and/or have a broader organizational impact whereby a more complex transformational change is initiated (33). The intended change derives from the objectives of the intervention, with the assumption that the initiated change will contribute to realizing these objectives (58).

Twelve of the determinant frameworks mention a component similar to the intended change as part of the implementation process. This is referred to as (characteristics of) an intervention (3, 26, 52), innovation (19, 30, 48, 60), change (40), program (36) or issue (34), or involves an evidence based practice (20, 30, 33, 41) or guidelines (20, 35). Determinant frameworks that do not mention the intervention as a separate component either focus on context variables (40), domains (51) or barriers (20), or incorporate intervention aspects in general implementation characteristics (35, 37).

The CFIR (26), the Interactive Systems Framework for Dissemination and Implementation (48) and i-PARiHS specifically zoom in on the characteristics of such an intended change (in these models referred to as intervention, innovation, or evidence-based practice). This indicates that an intervention or intended change is complex, multi-faceted, and different components will be interacting with each other (26). Characteristics that are mentioned are among others compatibility (33, 35, 48), adaptability (36, 48), complexity (26, 33, 35, 36), and/or relative advantage (33, 35). Such inherent characteristics of the intervention will have an impact on its overall implementation success.

As the intended change is expected to contribute to realizing the objectives of the intervention, it is important to define what outcomes are expected from the intended change. Four determinant frameworks incorporate results (52), output (52), outcomes (30), (implementation or innovation) effectiveness (37), or successful implementation (33) as separate components. This helps focusing on the objectives that are set when defining an intervention and the benefits that arise when implementation is successful (37). The time frame in which results can be observed, can differ majorly. Certain results are obtained early on, while others only exist in the long-term even after the intervention is finished (52). When defining the intended change, it is thus key to not only define the behavioral or organizational change that is expected, but also the expected results and how this can be evaluated.

Context

Context variables can be defined as “the set of circumstances or unique factors that surround a particular implementation effort (26).” They are dynamic factors that interact, influence, modify, and facilitate or constraint intervention and implementation efforts (53). Context variables are most prominent in what Nilsen (25) defines as determinant frameworks, in which the main objective is to gain insight in those barriers and facilitators that impact implementation outcomes (25). Some are built with the interaction of context variables (40), context domains (51), or barriers (20) as a main focus. Most frameworks indeed incorporate some form of context variables as an essential part of the implementation process. i-PARiHS (33), the Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors (22), the CFIR (26), the CAIMeR theory (52), and the GIF (19) directly incorporate context, contextual readiness, inner and outer context, context domains, setting, or factors as a component of the framework. A distinction is sometimes made between inner- and outer context or setting (22, 26), which mentions inner context variables as being specific to a person, team our organization (on micro and meso level), while outer context variables are broader in nature such as socio-economic or policy variables (on macro level).

When referring to context, some frameworks only incorporate context variables on the macro level. They zoom in on the so called outer context (50), external context (3), or external environment (36). Elements on an organizational or individual-adopter level are then incorporated under a different name. For example, organizational aspects can also be referred to as system characteristics (35), system antecedents or system readiness for innovation (50), practice environment (30), system and process barriers (20), implementation and sustainability infrastructure (36), organizational culture (41) or climate for implementation (37), intervention infrastructure (41), or factors relevant to the prevention delivery system (48).

When it comes to the micro context, individual adopter characteristics are mentioned by fewer frameworks. They are referred to as professional (3), or provider characteristics (35, 48), or more specifically as cognitive-behavioral barriers, attitudinal, or rational-emotional barriers or professional barriers (20), which indicates that individual adopter characteristics can cover a wide range of micro level aspects. This is also noticeable in the Theoretical Domains Framework (51), in which a wide variety of “domains” is mentioned, many of which are individual adopter characteristics such as professional role, beliefs about capabilities, etc.

On the micro level, context variables highly relate to the actors to which the intended change concerns. Greenhalgh et al. (50) state that “people are not passive recipients of innovations.” The dynamic interplay of how individuals relate to the organization in which they work (26) and their general assumptions about people, society and their profession (52) influences their perception and the way in which they make sense of an intended change. Six determinant frameworks include actors (52), individuals involved (26), potential adopters (30, 50), recipients (36), or the user group (34) as a core component. Incorporate actors as one of the components strengthens the view that actors have an impact on the way an intervention is realized. In five determinant frameworks, the influence actors have on implementation success is recognized by including individual attitudes, cognitions, or professional characteristics as a context variable (3, 20, 35, 48, 51). The component actors can thus be incorporated as a separate component of an implementation model, but it can also be included as a micro level context variable.

Overall, there is a wide belief that the context in which a primary care intervention takes place highly determines implementation success (10). This makes scanning and taking into account the context key for each phase of the implementation process. When determining implementation strategies, context variables must be taken into account in order for strategies to be tailored and fit local circumstances (10). This is in line with realist evaluation, whereby the general aim is to find out “what works, for whom, and under what conditions?” (6). In this approach, context variables are the conditions in which an intervention takes place.

Strategies

Implementation strategies can be defined as the approach(es) and means that are used to ensure or enhance the adoption of the target behaviors and other requirements of the primary care intervention by the targeted actors (10, 61). Whereas, the intended change refers to what is to be implemented, the strategies refer to how they are to be implemented and is linked to the process or mechanism that intervention designers want to trigger in order to accomplish implementation.

Implementation strategies are directly referred to in few process models, such as Advancing Understanding of Mechanism of Change in Implementation Science (31), whereby a first step to implementation is to specify the implementation strategies; the Ottawa Model of Health Care Research (30) in which transferring strategies is a part of monitoring the uptake of the intervention and in the GIF (19), in which the strategies are viewed as the approaches to respond to barriers and facilitators. Throughout other determinant frameworks, a component similar to implementation strategies is included in eight of the models we included in our analysis, either in the form of facilitation (33), support (e.g., training, assistance) (20, 48), implementation characteristics (35) and dissemination and/or diffusion of strategies (34, 41, 50). Frameworks also tend to incorporate those elements that are considered to be most decisive as strategies, such as communicational aspects (50), coordination of different stakeholders (41), or the use of incentives and disincentives (37).

Implementation strategies are discussed more in-depth in the Expert Recommendations for Implementing Change (ERIC) study, in which a compilation of 73 implementation strategies was made (62, 63). This can serve as a guide for when the most fitting implementation strategies have to be selected for the implementation of a certain intervention. To make more sense of the wide diversity of implementation strategies, they often are categorized. For example, Powell et al. (64) distinguishes between strategies that are related to either planning, educating, financing, restructuring, managing quality, and/or attending to policy context. Another categorization can be found in Charif et al. (65), who differentiate strategies that are related to either the health infrastructure, policy and regulation, financing, human resource, or patients (65).

Implementation strategies can be very different depending on the type of change that is initiated, and should ideally be tailored to fit the inner and outer context (10, 66), making use of the facilitators or barriers that are observed in order to ensure a fit between the intervention and its context (3). When defining implementation strategies to implement one's intervention, Proctor et al. (67) have set up guiding principles to name, define, and operationalize implementation strategies by firstly specifying the following elements: (1) actor, (2) action, (3) action target, (4) temporality, (5) dose, (6) implementation outcome affected, and (7) justification. These can support intervention designers in defining implementation strategies.

In short, implementation strategies are expected to lead to an intended change in a given context. This means that there is an underlying process that will bring about this change. Three determinant frameworks include this (implementation) process (26, 60) or mechanism (52) as one of the core components. These frameworks have a more explanatory approach and put more emphasis on understanding the process of change. For complex interventions, this consists of many interdependent sub-processes that may or may not follow a clear path to success (26). The process involves decision making activities, the use of resources, communication, and collaboration (50). Blom and Morén (52) view this as an either social, socio-psychological, or psychological mechanism that is at the base of change. Greenhalgh et al. (50) and Lewis et al. (31) also refer to linkages or effect modifiers and Aarons et al. (22) speak about interconnections, referring to the fit between an innovation and a system or organization that comes into play when introducing a change. These frameworks thus incorporate the process or mechanism of change as a core element that needs to be understood in order to fully know how to target certain interventions in specific settings. When choosing implementation strategies, it is thus recommended to make explicit the assumptions of how a certain strategy will lead to the intended change in a given context.

Discussion

We have identified the core building blocks of an overarching implementation framework for complex interventions in primary care services. Throughout our narrative review, three core phases are detracted that describe the process of implementation in relation to three core components. This process can roughly be divided in a development phase, a translation phase, and a sustainment phase. For each phase, three main components are essential to define, tailor, and manage to successfully implement an intervention in a specific setting. These are the intended change, the context, and the implementation strategies. Other related components that are closely linked to these three components may still be relevant, such as actors, the process or mechanism, and the outcomes and evaluation of the intervention.

An overarching implementation framework is needed to transcend the solely theoretical models and to aim for a model that is both explanatory as well as actionable. Context variables should be given a prominent place in this, as tailoring interventions to local circumstances is considered key for reaching implementation success (9, 10). By focusing on the core components intended change, context, and strategies we propose meaningful concepts to intervention designers and practitioners for reflecting upon the interactions of these components. The next step is synthesizing these core building blocks into a framework that consists of a clear and actionable pathway for intervention designers, and which enables them to prioritize and reflect upon those actions that need to be taken for the implementation of complex interventions.

Our research is part of a larger project that intends to make progress in three main research areas: to improve goal oriented care, self-management, and inter-professional collaboration. In each of the three areas, one or more interventions will be used for developing and evaluating the implementation of interventions in these three areas. The model that we will further develop will allow to develop and implement interventions with broad consideration of the setting or context in which they will be introduced, and how this interacts with the intended change and the implementation strategies that are used.

A limitation of our review is that we did not gather and include our sources in a systematic way. We used a more intuitive approach whereby sources were gathered mainly through expertise from our research team, by database searches with a set of different key words and by further use of a snowball approach that lead to the most prominent frameworks and models that exist. Furthermore, as we have only included English literature, there seems to be a slight overrepresentation of literature deriving from native English authors and/or institutions. Moreover, we have no view on gray literature or literature written in foreign languages, which might further limit our scope.

Although there is no assurance that we have covered all relevant literature, the methodology of a narrative review allowed us to explore the broad range of implementation literature and interpret various approaches in the light of interventions that aim toward pro-active, person-centered primary care. This way, we could harmonize literature into insightful constructs and phases which are to be made concrete when further applying them in the defining and execution of interventions.

Conclusion

An overarching implementation model is needed to bridge the gap between scientific evidence and actual practice in primary care. Through a narrative review, we have identified the core building blocks that form the common thread of existing implementation frameworks or models and we synthesized it in three core phases (a development phase, a translation phase and a sustainment phase) and three core components (the intended change, the context and the implementation strategies). These core building blocks can be used to develop an overarching implementation model that is both explanatory, as well as actionable. The main phases and components are the basis on which further guidance for intervention designers will be elaborated. A strength of the model that we will develop based upon this research is that it will be further developed and refined in collaboration with three research teams that will actively use the model to develop and introduce one or more interventions in primary care. This allows for direct feedback on its applicability and therefore ensures its actionability.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1.

  CrabtreeBFNuttingPAMillerWLMcDanielRRStangeKCJaenCRet al. Primary care practice transformation is hard work: insights from a 15-year developmental program of research. Med Care. (2011) 49:S28. 10.1097/MLR.0b013e3181cad65c

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 2.

  ShepperdSLewinSStrausSClarkeMEcclesMPFitzpatrickRet al. Can we systematically review studies that evaluate complex interventions?PLoS Med. (2009) 6:e1000086. 10.1371/journal.pmed.1000086

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 3.

  LauRStevensonFOngBNDziedzicKTreweekSEldridgeSet al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. (2015) 11:40. 10.1186/s13012-016-0396-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 4.

  EpsteinRMFiscellaKLesserCSStangeKC. Why the nation needs a policy push on patient-centered health care. Health Affairs. (2010) 29:1489–95. 10.1377/hlthaff.2009.0888

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 5.

  KitsonAMarshallABassettKZeitzK. What are the core elements of patient-centred care? A narrative review and synthesis of the literature from health policy, medicine and nursing. J Adv Nurs. (2013) 69:4–15. 10.1111/j.1365-2648.2012.06064.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 6.

  SalterKLKothariA. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. (2014) 9:115. 10.1186/s13012-014-0115-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 7.

  GrolRWensingMEcclesMDavisD. Improving Patient Care: The Implementation of Change in Health Care. New Jersey: John Wiley & Sons (2013). 10.1002/9781118525975

  • CrossRef
  • Google Scholar
  • View reference in article

• 8.

  OlswangLBPrelockPA. Bridging the gap between research and practice: implementation science. J Speech Lang Hear Res. (2015) 58:S1818–26. 10.1044/2015_JSLHR-L-14-0305

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 9.

  MayCRJohnsonMFinchT. Implementation, context and complexity. Implement Sci. (2016) 11:141. 10.1186/s13012-016-0506-3

  • CrossRef
  • Google Scholar
  • View reference in article

• 10.

  PfadenhauerLMGerhardusAMozygembaKLysdahlKBBoothAHofmannBet al. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci. (2017) 12:21. 10.1186/s13012-017-0552-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 11.

  EcclesMPMittmanBS. Welcome to Implementation Science. New York, NY: Springer (2006). 10.1186/1748-5908-1-1

  • CrossRef
  • Google Scholar
  • View reference in article

• 12.

  WensingMGrolR. Knowledge translation in health: how implementation science could contribute more. BMC Med. (2019) 17:88. 10.1186/s12916-019-1322-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 13.

  HullLGouldingLKhadjesariZDavisRHealeyABakolisIet al. Designing high-quality implementation research: development, application, feasibility and preliminary evaluation of the implementation science research development (ImpRes) tool and guide. Implement Sci. (2019) 14:80. 10.1186/s13012-019-0897-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 14.

  National Clinical Effectiveness Committee. Implementation Guide and Toolkit for National Clinical Guidelines: Department of Health (DoH), (2018). Available online at: http://hdl.handle.net/10147/623839

  • Google Scholar
  • View reference in article

• 15.

  Registered Nurses' Association of Ontario (RNAO). Toolkit: Implementation of Best Practice Guidelines (2nd ed), Toronto, ON (2012).

  • Google Scholar
  • View reference in article

• 16.

  PinnockHBarwickMCarpenterCREldridgeSGrandesGGriffithsCJet al. Standards for Reporting Implementation Studies (StaRI) statement. BMJ. (2017) 356:i6795. 10.1136/bmj.i6795

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 17.

  SmithJDLiDHRaffertyMR. The implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. (2020) 15:1–12. 10.1186/s13012-020-01041-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 18.

  BirkenSAPowellBJSheaCMHainesERKirkMALeemanJet al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci. (2017) 12:124. 10.1186/s13012-017-0656-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 19.

  MoullinJCSabater-HernándezDFernandez-LlimosFBenrimojSI. A systematic review of implementation frameworks of innovations in healthcare and resulting generic implementation framework. Health Res Policy Syst. (2015) 13:16. 10.1186/s12961-015-0005-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 20.

  CochraneLJOlsonCAMurraySDupuisMToomanTHayesS. Gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Continuing Educ Health Professions. (2007) 27:94–102. 10.1002/chp.106

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 21.

  DaviesPWalkerAEGrimshawJM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. (2010) 5:14. 10.1186/1748-5908-5-14

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 22.

  AaronsGAHurlburtMHorwitzSM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. (2011) 38:4–23. 10.1007/s10488-010-0327-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 23.

  GreenBNJohnsonCDAdamsA. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropractic Med. (2006) 5:101–17. 10.1016/S0899-3467(07)60142-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 24.

  GreenhalghTThorneSMalterudK. Time to challenge the spurious hierarchy of systematic over narrative reviews?Eur J Clin Invest. (2018) 48:e12931. 10.1111/eci.12931

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 25.

  NilsenP. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53. 10.1186/s13012-015-0242-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 26.

  DamschroderLJAronDCKeithREKirshSRAlexanderJALoweryJC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. 10.1186/1748-5908-4-50

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 27.

  ArkseyHO'MalleyL. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. (2005) 8:19–32. 10.1080/1364557032000119616

  • CrossRef
  • Google Scholar
  • View reference in article

• 28.

  RogersEM. Diffusion of Innovations: Simon and Schuster, New York, NY (2010).

  • Google Scholar
  • View reference in article

• 29.

  MayCFinchT. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. (2009) 43:535–54. 10.1177/0038038509103208

  • CrossRef
  • Google Scholar
  • View reference in article

• 30.

  LoganJGrahamID. Toward a comprehensive interdisciplinary model of health care research use. Sci Commun. (1998) 20:227–46. 10.1177/1075547098020002004

  • CrossRef
  • Google Scholar
  • View reference in article

• 31.

  LewisCCKlasnjaPPowellBJLyonARTuzzioLJonesSet al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. (2018) 6:136. 10.3389/fpubh.2018.00136

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 32.

  MeyersDCDurlakJAWandersmanA. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. (2012) 50:462–80. 10.1007/s10464-012-9522-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 33.

  StetlerCBDamschroderLJHelfrichCDHagedornHJ. A guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. (2011) 6:99. 10.1186/1748-5908-6-99

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 34.

  JacobsonNButterillDGoeringP. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy. (2003) 8:94–9. 10.1258/135581903321466067

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 35.

  GursesAPMarstellerJAOzokAAXiaoYOwensSPronovostPJ. Using an interdisciplinary approach to identify factors that affect clinicians' compliance with evidence-based guidelines. Critic Care Med. (2010) 38:S282–91. 10.1097/CCM.0b013e3181e69e02

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 36.

  FeldsteinACGlasgowRE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission J Qual Patient Safety. (2008) 34:228–43. 10.1016/S1553-7250(08)34030-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 37.

  KleinKJSorraJS. The challenge of innovation implementation. Acad Manage Rev. (1996) 21:1055–80. 10.5465/amr.1996.9704071863

  • CrossRef
  • Google Scholar
  • View reference in article

• 38.

  CraigPDieppePMacintyreSMichieSNazarethIPetticrewM. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. (2008) 337:a1655. 10.1136/bmj.a1655

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 39.

  PronovostPJBerenholtzSMNeedhamDM. Translating evidence into practice: a model for large scale knowledge translation. BMJ. (2008) 337:a1714. 10.1136/bmj.a1714

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 40.

  FerlieEBShortellSM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Quart. (2001) 79:281–315. 10.1111/1468-0009.00206

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 41.

  BradleyEHWebsterTRBakerDSchlesingerMInouyeSKBarthMCet al. Translating research into practice: speeding the adoption of innovative health care programs. Issue Brief. (2004) 724:1–12. 10.1111/j.1532-5415.2004.52510.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 42.

  DoodyCMDoodyO. Introducing evidence into nursing practice: using the IOWA model. Br J Nurs. (2011) 20:661–4. 10.12968/bjon.2011.20.11.661

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 43.

  StetlerCB. Updating the Stetler model of research utilization to facilitate evidence-based practice. Nurs Outlook. (2001) 49:272–9. 10.1067/mno.2001.120517

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 44.

  StevensK. The impact of evidence-based practice in nursing and the next big ideas. Online J Issues Nurs. (2013) 18:4. 10.3912/OJIN.Vol18No02Man04

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 45.

  BlanchardCLivetMWardCSorgeLSorensenTDMcClurgMR. The active implementation frameworks: a roadmap for advancing implementation of comprehensive medication management in primary care. Res Soc Admin Pharmacy. (2017) 13:922–9. 10.1016/j.sapharm.2017.05.006

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 46.

  WilsonKMBradyTJLesesneCTranslationNWGo. Peer reviewed: an organizing framework for translation in public health: the knowledge to action framework. Prev Chronic Dis. (2011) 8:A46. Available online at: http://www.cdc.gov/pcd/issues/2011/mar/10_0012.htm

  • Google Scholar
  • View reference in article

• 47.

  DavisSMPetersonJCHelfrichCDCunningham-SaboL. Introduction and conceptual model for utilization of prevention research. Am J Prevent Med. (2007) 33:S1–5. 10.1016/j.amepre.2007.04.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 48.

  DurlakJADuPreEP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. (2008) 41:327–50. 10.1007/s10464-008-9165-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 49.

  LukasCVHolmesSKCohenABRestucciaJCramerIEShwartzMet al. Transformational change in health care systems: an organizational model. Health Care Manage Rev. (2007) 32:309–20. 10.1097/01.HMR.0000296785.29718.5d

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 50.

  GreenhalghTRobertGMacfarlaneFBatePKyriakidouO. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Quart. (2004) 82:581–629. 10.1111/j.0887-378X.2004.00325.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 51.

  AtkinsLFrancisJIslamRO'ConnorDPateyAIversNet al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci. (2017) 12:77. 10.1186/s13012-017-0605-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52.

  BlomBMorénS. Explaining social work practice — the CAIMeR theory. J Soc Work. (2009) 10:98–119. 10.1177/1468017309350661

  • CrossRef
  • Google Scholar

• 53.

  PfadenhauerLMMozygembaKGerhardusAHofmannBBoothALysdahlKBet al. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes. (2015) 109:103–14. 10.1016/j.zefq.2015.01.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 54.

  GlasgowREVogtTMBolesSM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. 10.2105/AJPH.89.9.1322

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55.

  CrosbyRNoarSM. What is a planning model? An introduction to PRECEDE-PROCEED. J Public Health Dentistry. (2011). 71:S7–15. 10.1111/j.1752-7325.2011.00235.x

  • CrossRef
  • Google Scholar

• 56.

  ProctorESilmereHRaghavanRHovmandPAaronsGBungerAet al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38:65–76. 10.1007/s10488-010-0319-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57.

  ChenHT. Theory-driven evaluation: conceptual framework, application and advancement. In: StroblRLobermeierOHeitmeyerW, editors. Evaluation von Programmen und Projekten für eine demokratische Kultur. Wiesbaden: Springer (2012). p. 17–40. 10.1007/978-3-531-19009-9_2

  • CrossRef
  • Google Scholar

• 58.

  CrableELBiancarelliDWalkeyAJAllenCGProctorEKDrainoniML. Standardizing an approach to the evaluation of implementation science proposals. Implement Sci. (2018) 13:71. 10.1186/s13012-018-0770-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 59.

  MichieSRichardsonMJohnstonMAbrahamCFrancisJHardemanWet al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. (2013) 46:81–95. 10.1007/s12160-013-9486-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60.

  GreenhalghTRobertGBatePMacfarlaneFKyriakidouO. Diffusion of Innovations in Health Service Organisations: A Systematic Literature Review. New York, NY: John Wiley & Sons (2008).

  • Google Scholar

• 61.

  NetaGBrownsonRCChambersDA. Opportunities for epidemiologists in implementation science: a primer. Am J Epidemiol. (2017) 187:899–910. 10.1093/aje/kwx323

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 62.

  WaltzTJPowellBJMatthieuMMDamschroderLJChinmanMJSmithJLet al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. (2015) 10:109. 10.1186/s13012-015-0295-0

  • CrossRef
  • Google Scholar

• 63.

  PowellBJWaltzTJChinmanMJDamschroderLJSmithJLMatthieuMMet al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. (2015) 10:21. 10.1186/s13012-015-0209-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 64.

  PowellBJMcMillenJCProctorEKCarpenterCRGriffeyRTBungerACet al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. (2012) 69:123–57. 10.1177/1077558711430690

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65.

  CharifABZomahounHTVLeBlancALangloisLWolfendenLYoongSLet al. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implement Sci. (2017) 12:139. 10.1186/s13012-017-0672-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66.

  BungerACPowellBJRobertsonHAMacDowellHBirkenSASheaC. Tracking implementation strategies: a description of a practical approach and early findings. Health Res Policy Syst. (2017) 15:15. 10.1186/s12961-017-0175-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 67.

  ProctorEKPowellBJMcMillenJC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. (2013) 8:139. 10.1186/1748-5908-8-139

  • Pubmed Abstract
  • CrossRef
  • Google Scholar