The discovery and subsequent administration of penicillin in 1943 was a major milestone in clinical medicine, saving countless lives (1). However, even before the drug was approved for clinical use, the first reports of antimicrobial resistance were described. Less than 20 years after initial approval, more than 80% of Staphylococcus aureus strains were penicillin-resistant (2). In the decades since, a similar story has been described for every antibiotic brought to market. Host-pathogen relationships are inherently dynamic: as hosts develop ways to combat an infectious diseases threat— whether through immunity or treatment—pathogens evolve to evade our advancements.

In addition to an inherently evolving host-pathogen interaction, many factors impacting this interaction are also constantly changing and need to be considered, measured, and integrated into public health policy making. Factors that change over time and therefore determine public policy impact include resource availability, case fatality rate, understanding about modes of transmission, human behaviors, societal expectations, the evidence basis for treatment and prevention and therefore our understanding about the disease, among others.

Maximizing public health policy impact for combating infectious diseases threats necessitates that all of these dynamic factors be measured and evaluated in real-time to continually adapt response plans and achieve maximum public health benefit. Updating policies and recommendations to elevate some interventions and de-escalate others as contextual factors continually evolve is the best evidence-based policy strategy (3, 4). Achieving this ideal requires re-imaging infectious diseases public health policy making as a dynamic and constantly evolving process with the anticipation of change inherently built into health communications and public expectations (5–7). Infrastructure that can support integration of point-of-care data with emerging and evolving evidence and public input to assess ongoing feasibility, acceptability, appropriateness, and costs are needed to achieve safe, efficient and higher quality care and policies (8).

Viewing pandemic responses and the expected impacts of public health policy through the lens of implementation science would enhance emergency preparedness for future pandemics and ultimately improve public health policy impact. The objectives of this review are to: (1) discuss the pipeline from clinical effectiveness to implementation outcomes to ultimate public health impact and introduce the concept of dynamic policy effectiveness (9), (2) to present the COVID-19 pandemic and public health policy as a case study for considering the dynamic host, pathogen, contextual, and evidence changes that evolved over the course of the world-wide emergency responses, and (3) to propose future innovations to support a real-time, learning public health infrastructure that is more adaptable based on changing conditions, context, and evidence. A practical tool for operationalizing the Dynamic Infectious Diseases Public Health Response Framework is presented. The tool is designed to facilitate integration of key implementation outcomes and considerations into infectious diseases response planning (Supplementary material).

Traditional clinical trials evaluate efficacy and effectiveness of an intervention in a controlled setting, whereas implementation trials evaluate how to promote uptake and how to translate potential benefits into tangible improvements (9). Public health policy impact is a downstream consequence of the potential efficacy of the intervention as well as its real-world implementation (Figure 1). The Dynamic Sustainability Framework (DSF) (10) highlights the importance of adapting interventions on individual and systems levels to maintain and maximize longitudinal impacts in the setting of constantly changing evidence and contexts.

DSF principles also apply to public health systems and public health policy impact. Public health policies are typically composed of a bundle of multiple interventions. To achieve sustained public health impact, public health policies and their component interventions (whether targeted at an individual, organization or society) must adjust and adapt to changing contexts and evidence. In other words, public health policies must be viewed as having dynamic public health impacts and dynamic effectiveness; a constant level of public health impact while other changes in the system are occurring cannot be assumed.

Efficacy and effectiveness are the traditional measures for evaluating clinical and public health policies and interventions in controlled research studies. These measures focus on the absolute and relative differences in outcomes among exposed and unexposed groups and are generally conceived of as static estimates; in other words, the relative risk reduction associated with receipt of a particular medical intervention is assumed to be constant over time.

Implementation science focuses on different outcomes (Table 1) (11). Early implementation outcomes include acceptability, appropriateness, feasibility/availability, and adoption, which is defined as the decision to recommend a specific public health intervention (in contrast with uptake, which is the actual use of the intervention). Later implementation outcomes include fidelity, penetration, and costs. The longest-term implementation outcome is sustainability. In implementation science, both sustainability and sustainment are key terms, with sustainability referring to a property/characteristic of an intervention related to its likely long-term usability whereas sustainment refers to the outcome of whether an intervention was used over a long period of time.

Importantly, while often not considered when evaluating clinical efficacy and effectiveness, implementation outcomes directly impact the expected benefit and impact of public health policies and their components (the individual interventions) (Figure 1). Interventions that are promising in laboratory settings or in idealized clinical trial settings have limited or no impact on public and population health if they are infeasible/unavailable, unacceptable, and/or perceived to be inappropriate by end-users. Further complicating longitudinal evaluations of public health impact and ongoing recommendations, these implementation outcomes themselves are not static – feasibility, which is related to availability (12), acceptability, appropriateness, and costs all vary according to contexts, evolving evidence, resource availability, progress, available alternatives and perceived benefits (Figures 2, 3). Sustainability is a perennial challenge in implementation, particularly if day-to-day behavior change is required and if the intervention is perceived to have substantial downsides. Thus, implementation outcomes are key determinants of public health policy and intervention effectiveness and impact. Implementation outcomes are also constantly evolving.

Consideration of dynamic public health policy effectiveness and impact is particularly important for developing and adapting responses to infectious disease threats. As humans make advancements, such as the development of therapeutics or vaccines, the pathogen evolves in response to human progress (Figure 4). For example, delta and omicron variants both emerged in part due to pressure from vaccine and infection-induced immunity. Mutations arose that rendered once highly effective monoclonal antibody therapies for early treatment and prophylaxis obsolete. Antimicrobial resistance, another critical public health threat in infectious diseases, is a direct downstream consequence of pathogen evolution in response to human innovation. Antimicrobial resistance highlights the generalizability of the dynamic nature of the management and containment of infectious diseases beyond the COVID-19 pandemic.

COVID-19 is presented as a case study for considering public health policy impacts and adaption through an implementation science lens. Principles about dynamic change are inherent to the host-pathogen interaction and generalizable beyond the specifics of the public health response to the COVID-19 pandemic. Key lessons learned include the changing public health policy effectiveness of interventions as a function of conditions, contexts, and political environments and the need to consider the aims of sustainability. Rather than focusing on maintaining compliance with any specific intervention, to improve health, public health policy goals should aim to reduce severe health outcomes.

Achieving the ideal of adapting policy to sustain benefit will require re-focusing public health surveillance and evaluation methods to include consideration of implementation outcomes and changing contextual factors. Ideally, systems will be developed so that key inflection points, or phase transitions (e.g., from the early phases characterized by fear and limited resources to the later phases characterized by reduced mortality) can be measured and acted upon in real time through ongoing policy updates. Empirical evidence for the importance of novel data sources and integrating implementation outcomes into infectious diseases pandemic planning is illustrated in the modeling data from Chang et al., which demonstrated that integration of cell phone movement data improved outbreak prediction model accuracy (46). Authors also found that integration of data about compliance with social distancing policy (fidelity to the public health intervention), which varied with time, lead to persistent improvements in model prediction accuracy. This study therefore highlights the importance of integrating data about key implementation outcomes, such as fidelity, to improve infectious diseases management and public health policy.

The Dynamic Infectious Diseases Public Health Response Framework is a model for evaluating ongoing public health policy impact in the context of a constantly changing and evolving system (Figures 3, 6). Concepts are grounded in implementation science theory and are a direct extension of the Dynamic Sustainability Framework (10). Key considerations are generalizable to many infectious diseases threats and associated mitigation measures and prevention responses.

Focusing on maximizing public health outcomes has additional implications. For populations that are high-risk of severe disease despite vaccination (or for other reasons, depending upon the specific infectious disease in question) (47–49), different policy recommendations may be needed to achieve public health goals. For example, mitigation measures designed to prevent any COVID-19 case applied in skilled nursing facilities, dialysis centers, and chemotherapy units are expected to have a more substantial direct public health impact than when those same mitigation measures are applied to lower-risk populations, such as interventions implemented in elementary and secondary school settings. Acknowledgment of differential population risk was one of the reasons masking requirements were maintained in hospital settings longer than in the community (50). Thus, implementation of a Dynamic Infectious Diseases Public Health response requires measurement of population risk and adaptation of policy to match level of risk. In this setting, risk should be defined broadly, and harms and costs of the intervention should be included when crafting public health policy.

Achieving future improvements in real-time infectious disease policy responses will require major infrastructure investments to collect the data necessary to inform ongoing public policy decisions. Traditional surveillance systems focus on measuring cases via reporting from state and local health departments. These traditional systems do not have the infrastructure or linkage to data elements that would facilitate evaluation and integration of implementation outcomes. Novel mechanisms for gathering and interpreting data in near-real time are needed; practically, this will likely include a national data repository, technologic advancements in data cleaning and real-time analysis, and integration of non-traditional sources of information, such as social media, for ongoing assessment of public perceptions about different public health policies. For example, measuring trends about discussion of masking policies and school closures on Twitter and other social media sites likely would have provided valuable insight into changing acceptability of these public health recommendations, and allowed for public health policy makers to integrate perceptions of end-users into ongoing policy updates. Similarly, early in the pandemic, ongoing discussions of lack of access to testing may have served as a signal to invest more heavily in this strategy.

Inherent in the Dynamic Infectious Diseases Public Health Framework is the concept that as implementation outcomes evolve longitudinally, so does the effectiveness of public health policy responses (Figure 1). Thus, as a direct result of constantly evolving conditions, risk of the setting and to the individual, public perceptions, and political contexts, public health policy needs constant revision and evolution to maintain the same societal benefits (7).

Challenges with sustainment of behavior change to control disease spread are not specific to COVID-19. Recommendations for use of barrier methods to prevent transmission of sexually transmitted infections, particularly HIV infection, and to reduce the number of sexual contacts, were eventually abandoned in favor of a more harm-reduction focused approach (51), as early abstinence-based public health recommendations were found to be unacceptable to many and therefore less effective than other approaches, such as treatment and PrEP (52–55). In the summer of 2022, when the Mpox outbreak occurred, public health recommendations that focused on reducing sexual contacts were heavily criticized, due in part to lack of acceptability and perceived appropriateness of the intervention among recipients of the public health campaign (56–59).

The ideal of the Learning Health System is that data generated in real time can be leveraged to advance our scientific understanding of a problem and that advancement in our understanding can be leveraged to improve care delivery (6). Inherent in this model is the idea that “the evidence” is constantly changing and evolving, and that a variety of data elements are needed to realize potential benefits of this approach (60). Particularly when applied to the context of public health programs, data elements include not only traditional data elements but also other factors that impact real-world effectiveness – cultures, believes, attitudes and contexts. Integration of these other elements into policy evaluation is critical for achieving and sustaining public health benefit.

Adaptation to changing conditions and contexts is the best way to incorporate evidence into policy changes but achieving the ideal synergy of specific policy recommendation to context and public perception requires substantial advancements and investment in data collection, analysis, and implementation. Substantial changes in contexts, evidence and phases need to be identified in near-real time, so that public health policy can be aligned with the current conditions. The more heavily reliant an intervention is on day-to-day, person-level appropriate use for effectiveness, the more strongly the duration of the intervention should be considered in weighing recommendations. Ideally, evaluations of the effectiveness of individual-level interventions should be ongoing, with constant reassessments of whether the intervention is continuing to achieve the desired outcome and immediate policy change once the intervention is no longer effective or found to be harmful. Costs and harms of an intervention should be evaluated in parallel with benefits and on an ongoing basis. Improvements in real-time data analytics are likely to result in earlier policy de-implementation as human behaviors and expectations are inherently integrated into public health policy making. Once the balance tips away from benefit and toward harm, the policy should be de-implemented, with clear and open communication about the emerging evidence and rationale for the change.

Recognizing the inherent limitations of observational data, advancements in data management and rapid data analysis are needed for the next pandemic to enhance our understanding about real-time policy effectiveness. Given the role of perceived appropriateness of policy in driving compliance—and therefore effectiveness—improvements should also include mechanisms to assess and integrate public feedback into public health policy and messaging. Data about the feasibility, acceptability, and costs of different interventions, such as social distancing policies and business and school closures should be collected by public health officials and state and local governments from publicly available sources, such as social media platforms and google movements as well as directly from interested parties and individuals impacted by specific policy interventions. These data elements should then be integrated into planning and policy responses. Data about ongoing use could be used to inform whether policy effectiveness is likely to have changed due to changing conditions and public perceptions. Potential harms of interventions should be prospectively measured and incorporated into assessments of the ongoing acceptability and appropriateness of the public health policy. These data elements could then be mapped to quantitative data and used to tailor public health policy recommendations based on input from those most impacted and ongoing expected benefits and harms. An example application of integrating implementation outcomes into public policy design is presented in Table 3, which ranks NPIs according to feasibility and acceptability. In the future, information about implementation outcomes and public perceptions could be integrated with evidence about relative policy effectiveness to adapt public health policy recommendations to align with current conditions to sustain benefits that are informed by factors that are important to the public.

Achieving this ideal – rapid data collection and analysis to inform on-the-ground policy recommendations will require substantial investments in national informatics infrastructure to achieve a public health ‘Learning Health System' (6, 7, 61). Leveraging advancements in artificial intelligence to link and analyze novel data sources, such as social media commentary, to assess ongoing public perceptions, feasibility, and acceptability may help in the future to realize these ideals. Setting up systems where data are assembled, analyzed and interpreted, leading to knowledge of the needed interventions and how to manage them, and then tracking data to understand practices changes in real-time are learning health system steps required for the real-world application of the Dynamic Infectious Diseases Public Health Framework (60).

A practical tool to assist policy makers in applying the framework is presented in Supplementary material 1. The practical tool with pre-populated responses to key determinants of public health policy impacts directs iterative assessments of key implementation outcomes and contextual factors that can be evaluated in real-time to inform and adapt public health policy. It is designed to facilitate measurements and incorporation of implementation outcomes, such as feasibility, appropriateness and costs to inform and adapt decision making, which may include policy adaptation, policy de-implementation, or determination that additional information is needed. If additional information is needed, the tool can help to identify knowledge gaps and direct scientific investigation to close these gaps. Integration of novel information sources, such as public input from social media platforms, is encouraged as part of ongoing assessments of public perceptions about policy appropriateness and harms. Key population-specific factors that may impact decision-making, such as risk of disease, are also included to inform policy development and adaptation. Optimizing the effectiveness of public health policy necessitates pre-planned, ongoing evaluation of each of these different variables, with updates as changes are identified.

A flow chart to direct application and use of the tool is presented in Figures 7A, B. Figure 7A depicts the first step of the dynamic response, which includes consideration of pathogen characteristics, key contextual factors including the current evidence base and potential alternatives to the current strategy, assessment of implementation outcomes (defined in Table 1), and public perceptions. These steps are delineated to assist with policy making and to identify key challenges and evidence gaps to inform subsequent decision-making. At the end of dynamic step 1, users should define the timeframe for re-evaluation. Dynamic step 2 is depicted in Panel B. Step 2 defines key factors that may merit a policy change and directs ongoing planning, including collection of new evidence. Step 2 is designed to be repeated as many times as is necessary for the duration of the response.

A refrain throughout the pandemic has been “the science has changed!” Public health policy making is complex – and the message that impacts of public health policies are dynamic is a difficult one to convey. While knowledge and evidence have evolved and expanded, fundamental scientific principles have not. Diagnostics, therapeutics, preventative interventions, and viral variants did change, as did public perceptions, tolerances, and behaviors. In future policy responses and policy messaging, uncertainty must be acknowledged and embraced. Public health officials should also be upfront that change in policy is an expected outcome of the most evidence-based practice, as we learn more, the context, conditions, and evidence change, and even the goals of the public health response evolve. Both implementation and de-implementation plans should be incorporated into planning.

The Dynamic Infectious Diseases Public Health Response Framework is presented through the lens of the COVID-19 pandemic response but is broadly applicable to public health interventions that include complex and ever-evolving host-pathogen interactions. Consideration of implementation outcomes, in addition to more traditional measures of clinical effectiveness, may help to improve evaluations of public health programs and impact and to facilitate matching policy recommendations with evolving contexts. Public health policy goals, feasibility, costs, and perceived acceptability and appropriateness change as the context, evidence, and resources change, highlighting the importance of viewing the impact and effectiveness of public health policies and impacts as dynamic elements of a larger constantly evolving and changing system. These implementation outcomes are determinants of ultimate public health policy impact.

Public health policy and pandemic responses are not just about the evidence– or just about the evidence at one moment in time. This is particularly true for the management and control of infectious diseases, which always involve a dynamic interplay between the host and the pathogen. Real-time decision making requires sensitivity to conditions on the ground and adaptation of intervention at all levels as implementation outcomes, such as acceptability, appropriateness, and fidelity change and as contexts evolve. When asking about the public health effectiveness and impact of non-pharmaceutical interventions, the focus should be on when, how, and for how long they can achieve public health impact – definitive statements such as “masks work” or “masks don't work” fail to capture how interventions work in real world settings and contexts.

Static effectiveness estimates cannot be assumed in a constantly changing system. Policy impacts are dynamic and need to be recognized and evaluated as such. Just as NPI policy should change, our public health infrastructure needs to adapt to maintain effectiveness in the background of constant change and to maintain relevance and benefit to the end-user of these policy recommendations: the public.

All authors contributed to concept review and analysis, drafting, and editing of the manuscript. In addition, all authors met with the graphic designer and assisted in the design and revision of the figures presented in the manuscript.