Historically in public health research, policy was more typically conceptualized as a distal “outer setting” determinant and not as the central “intervention” [i.e., as described by Curran using plain language as, “the thing being implemented” (4, 6, 7, 24)]. Policy can be conceptualized as the “thing” of interest, or an “implementation strategy,” (i.e., as described by Curran, “the stuff we do to try to help people and/or places to do the thing”) (24), or a determinant that influences the implementation of strategies (6). For example, school nutrition standards intend to decrease consumption of sugary, low-nutrient foods and beverages. In this case, the “thing being implemented” or the intervention is the policy that intends to make healthier foods and beverages more accessible to students in the school built environment. In comparison, earmarked taxes, defined as “taxes for which revenue can be spent only on specific activities” are conceptualized as an implementation strategy that facilitates access to evidence-based practices, such as mental health services (25). For the policy of interest, articulating clearly its place as “the thing” or an “implementation strategy” early in study conception is critical to the selection of frameworks, study designs, and associated methods (6). We contend that it matters less how it is conceptualized; rather, the important point is that it is clearly described.

Two frameworks: (a) Equity in Implementation Research (EquIR); and (b) targeted universalism inform this work. Briefly, EquIR aims to address inequalities during implementation and was selected because it calls for an explicit and intentional focus on equity—particularly on social determinants of health—from the planning and design phases (23). The framework encourages researchers to consider a continuum of participatory approaches that center historically disadvantaged groups' priorities. Such efforts will avoid constructing historically disadvantaged communities as “homogenous groups with static traits and shared beliefs” (26). Targeted universalism is defined as “pursuing targeted strategies that respond to the urgent needs of some people, and wrapping those strategies in a universal goal that holds wide appeal” (22). This framework was selected because it offers an equity-driven focus to policy strategies and aligned with approaches to health; for example, goals may include providing food, housing, and affordable health care, and the “targeting” component involves measuring the impediments to filling gaps, not with reference to each other but to the universal goal (22). The examples described next are designed with the principles of both EquIR (intentionally building equity into study approaches) and targeted universalism (developing equity-driven policy strategies that promote structural change). Finally, an essential component of equity-centered work is the need for researchers' to deeply engage in reflexive practices (27). Reflexivity requires researchers to continuously check their own social positions and deeply examine the ways in which they exercise and are influenced by power, as well as the ways in which these positions influence the particular research subject (27). While this practice should be conducted in all research, policy is fraught with ideology and values; thus, researchers should be mindful and transparent about their own biases and privileges.

Figure 1 illustrates two key considerations. First, at the top of the figure, researchers are reminded to consider equity early in the design phase and across effectiveness and implementation considerations. Second, to address “effectiveness,” researchers may consider what the policy intended to do and identify the appropriate “evidence” (described next) to support success, failure, or other (6). Unlike clinical interventions, policy as an “intervention” can be ambiguous (i.e., unclear policy language or multiple goals); in response, researchers may conduct (pre-study or phased) activities to better understand policy goals, such as policy analysis (28) or qualitative interviews with key policy actors and community groups (6). Policy analysis may be part of a policy surveillance, the latter is defined as “the ongoing, systematic collection, analysis, interpretation and dissemination of information about a given body of public health law and policy” (28).

We offer a few dimensions of “evidence” to inform policy success or failure. Scholars have outlined the problematic nature of applying post-positivist, clinical hierarchies of “evidence” (for example, based on large, randomized samples, and controlled for confounding variables) to the study of public health policy (3, 29). This discussion sits within a broader conversation and advocacy toward transforming the ways that “evidence” on policy effectiveness is generated and disseminated for decision making (3, 30). Policy “evidence” requires broader data that is derived from multiple sources/actor groups, tailored to context, and responds to the interests of those impacted by the policy (3, 30). Parkhurst and Abeysinghe (2016) suggest considering key questions: “(1) what are the policy concerns at hand (and is the evidence selected the most useful to address the multiple policy concerns at hand)?; (2) are the data constructed in ways that best serve policy goals?; and (3) do we have reason to believe that the evidence is applicable to our local policy context?” (29). Finally, and importantly, we highlight the equity dimension: “evidence” generation has historically been the privilege of white, Western, male researchers, with the intentional exclusion of historically disadvantaged groups, including women and people of color (27, 28). Here again, inclusion of a broader range of outcomes as “evidence,” such as nonbiomedical outcomes more salient to communities of interest is paramount (27). Researchers may determine implementation considerations with an equity lens by identifying who the policy will impact (i.e., is it targeted to a specific historically disadvantaged group or group with documented disparities?) and whether the current “evidence” points to any existing disparities or differential outcomes across groups (23). We restate here that policy “evidence” without this equity lens can unintentionally lead to exacerbation of inequitable conditions (3, 21).

In addition to investigating outcomes to better understand policy effectiveness (e.g., health or behavioral outcomes), hybrid studies examine implementation outcomes, such as acceptability, feasibility, sustainability and costs (31). Previous scholars have adapted these definitions and identified quantitative measures (outcomes and determinants) for policy IS research (23); for example, acceptability may measure perceptions of historically disadvantaged groups to understand if the policy is “agreeable” and why/why not. Implementation cost may measure total costs of implementation for historically disadvantaged and non-disadvantaged groups and calculate a final adjusted cost-effectiveness (23). More work is needed to further test and develop psychometric properties of such tools, as well as ensure rigorous qualitative and mixed methods approaches to measuring outcomes (23).

Table 1 provides key characteristics of each hybrid type to align with policy IS research. Again, they are presented as distinct types here, but can be conceptualized more as a continuum. As illustrated in Figure 1, Hybrid Type 1 is considered when there is the lowest availability of “evidence” on whether the policy is effective (9), while considering whether outcomes differ in historically disadvantaged groups: if the policy “works,” did it “work” across all groups? For example, in the case of sugary beverages taxes, a policy goal may be a decrease in purchasing of sugary beverages, based on prior research that has shown differential rates of exposure to targeted marketing across ethnic groups (33, 34). In addition to behavioral outcomes at the individual level, studies may consider policy-level outcomes, such as revenue generated by the tax (35), and community-level outcomes, such as the number of community organizations offering tax-funded programs. A secondary aim is to understand why (or why not) the policy “worked”, including equity-focused implementation outcomes, may examine fidelity to the policy (e.g., were all groups of interest within the jurisdiction able to implement the policy as intended?), as well as perceptions of acceptability of the policy among historically disadvantaged groups. A tertiary aim for the Hybrid Type 1 approach is to understand whether there were unique barriers or facilitators within historically disadvantaged groups that impacted implementation. Addition of contextual and determinant factors allows for a comprehensive understanding: what unique assets facilitated implementation; what were barriers that could inform future implementation strategies? (36).

Type 2a approaches aim to understand policy effectiveness and pilot test a potential implementation strategy. There may be some evidence to support the policy but effectiveness data is still of interest. A co-primary aim is to understand whether the policy showed differential effectiveness across groups, which requires a baseline understanding of the existing historically disadvantaged groups and potential disparate health status. In addition, a co-primary aim for Type 2a is to test the feasibility of an implementation strategy and “preliminary effectiveness” of the strategy on implementation outcomes (e.g., adoption or fidelity); the latter could be part of a readiness assessment to understand whether historically disadvantaged groups and partners are ready to adopt the strategy of interest. Conducting a rigorous readiness assessment—with an explicit equity emphasis—is considered an important strategy for policy IS research.

Type 2b approaches may be considered when there is interest in comparing two (or more, including packages of) implementation strategies in their ability to facilitate implementation, along with implementation outcomes, such as adoption and fidelity. Like Type 2a, a co-primary aim is to measure effectiveness of the policy amongst the historically disadvantaged groups and/or across all groups of interest. Another co-primary aim is to compare effectiveness outcomes between the two (or more) implementation strategies. For example, the same sugary beverage tax study may compare two implementation strategies: (1) a retailer education program to improve knowledge of sugary beverage among retailers; versus (2) a random check monitoring strategy that checks compliance to sugary beverage tax, to determine which of these strategies was more successfully adopted with fidelity. This type of examination may also use a commonly applied framework—such as RE-AIM—to evaluate equity-centered implementation outcomes associated with each implementation strategy, such as how many and what types of retailers received the trainings, to understand if there was differential uptake across retailers and why (37).

Type 3 studies are recommended when there is substantial “evidence” available supporting the effectiveness of the policy (e.g., a systematic review). Pilot testing of implementation strategies—including readiness assessment—would already be completed via a partner-informed approach. Like Hybrid Type 2b, the primary aim is to understand which of the implementation strategies (or packages) “worked” better in facilitating implementation of the policy (comparison of strategies). A secondary aim is to also gather policy effectiveness outcomes (as described in Type 1) to determine if there was success, including in historically disadvantaged groups.

Figure 1 includes the design or refinement of implementation strategies across the continuum of types as an equity-centered activity. This process aligns with a “targeted universalism” process, where targeted strategies—not a “one size fits all”—are designed based on partner-informed data (22). There are several compilations of general (e.g., Expert Recommendations for Implementing Change (ERIC), school [e.g., School Implementation Strategies, Translating ERIC Resources (38)] and policy-specific implementation strategies [e.g., Bullock et al. (17)] available. In addition, once implementation strategies are identified, a partner-engaged readiness assessment is recommended to understand whether the target partners have the capacity and motivations to implement (39). Many of the processes used in designing for dissemination (e.g., stakeholder engagement, participatory codesign, context analysis) will facilitate partner-focused implementation (40).