Emanuel Krebs
Deirdre Weymann
Tania Bubela
Dean A. Regier- 1Regulatory Science Lab, BC Cancer Research Institute, Vancouver, BC, Canada
- 2Faculty of Health Sciences, Simon Fraser University, Burnaby, BC, Canada
- 3Aga Khan University, Karachi, Pakistan
- 4School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada
Precision oncology uses omics-based diagnostic technologies to inform histology-agnostic cancer treatment. To date, health system implementation remains limited owing to high uncertainty in regulatory and reimbursement evidence submissions. In this perspective, we describe a life-cycle approach to the evaluation of precision oncology technologies that addresses evidentiary uncertainty and is grounded in real-world evidence (RWE) derived using data routinely collected by healthcare systems. We consider the role for RWE in international regulatory and reimbursement decision-making, review common biases for observational precision oncology evaluations, make specific recommendations for RWE study design and analysis, and specify healthcare system requirements for data collection. We then explore how decision-grade real-world data can support the generation of decision-grade RWE, ultimately enabling real-world life-cycle assessment for precision oncology.
Introduction
Regulatory decisions aim to ensure that healthcare technologies are safe and effective, with reimbursement agencies in many jurisdictions adding an analysis of cost-effectiveness. Collectively, these decision processes manage uncertainty in the evidence base. Advances in precision oncology challenge regulatory and reimbursement decisions. Precision oncology refers to a suite of ‘omics-based diagnostics (e.g., genomics, transcriptomics, and metabolomics) that drive histology-agnostic treatment decisions (1). As a result, these diagnostics and drugs typically benefit small patient populations, and their clinical evaluations rarely include a randomized control group, leading to uncertainty around health and economic outcomes, both at the clinical and healthcare system level (2, 3). This uncertainty has understandably limited the reach and implementation of precision oncology technologies, with stakeholders demanding additional evidence generation mechanisms to better manage uncertainty (4–7).
In this perspective, we describe how life-cycle real-world evidence (RWE) can bridge evidentiary gaps when evaluating precision oncology technologies, including diagnostics and drugs. We define RWE as evidence that relies on purposefully generated real-world data (RWD) from healthcare systems, which includes any data collected outside the confines of trials or experiments (8, 9). The increasing reliance on expedited regulatory pathways has led to more drugs being approved on the basis of fewer or less robust clinical studies, including studies using surrogate markers as primary endpoints (10). To generate evidence not available for market approval, regulatory agencies need to rely more on life-cycle RWE for the evaluation of safety, efficacy, and comparative effectiveness (11, 12). We expand on our previously published framework for life-cycle health technology assessment (HTA), which proposes the ongoing assessment of health, economic, and societal impacts of precision oncology technologies throughout their life-cycles (3). Herein, we specify how life-cycle RWE can facilitate such assessment, reducing uncertainty about precision oncology and supporting informed healthcare decision-making. We suggest that the acceptability and validity of using RWE for supporting regulatory and reimbursement decisions require advances in methods of causal inference to inform the safety, comparative effectiveness, and cost-effectiveness of health technologies.
In the following sections, we consider how RWE can be used for regulatory and reimbursement decision-making, overviewing current RWE initiatives and identifying gaps for RWE generation and uptake. We then provide a detailed overview of common observational study biases, propose specific study designs and analytic methods for producing decision-grade RWE for precision oncology, identify real-world data system changes needed to enable these analyses, and detail how RWE can support life-cycle assessment. We conclude with a discussion of life-cycle equity analyses and RWE considerations for global healthcare systems. Box 1 defines RWE terminology used throughout this perspective (13–15).
Box 1 Glossary of real-world evidence terminology.
.
A role for real-world evidence in decision-making?
To manage evidentiary uncertainty, including for precision oncology, international regulatory, and reimbursement agencies have stated that RWE will increasingly be used in regulatory pre- and post-market decision-making and for reimbursement (16, 17). In practice, the use of RWE for regulatory market approval has been rare despite bodies such as the United States Food and Drug Administration (FDA) having a long history of using RWE to monitor and evaluate drug safety post-authorization (e.g., Sentinel System) and the European Medicines Agency (EMA) increasingly considering RWE on a case-by-case basis for regulatory purposes (18–21). Instances of regulatory approvals based on RWE have primarily been in precision oncology and rare disease settings (22). This is because single-arm trial designs are common in these settings, where small benefiting populations prohibit rapid trial accrual. In the absence of randomized controlled trial data, decision-makers require additional non-trial evidence for understanding comparators (3, 23). While the reliability of RWE is a topic of continued debate (24), several regulatory and HTA bodies support RWE backed by strong study designs and RWD for establishing safety and comparative effectiveness when randomized trials are unavailable or unable to reliably inform decision-making (25, 26). This acknowledged support signals the acceptability of RWE for causal inference, in a similar way that the satisfactory design and conduct of randomized trials can allow causal inference from a sample of people to generalize to a broader population (27, 28).
Principles for decision-grade RWE generation from non-randomized observational studies of precision oncology thus require consideration of key study design elements and methodological improvements (27, 29). Objective assessments of RWE are challenging without points of reference typical of study protocols for randomized trials (30, 31). Aiming to promote a better understanding and acceptance of RWE, regulatory and HTA bodies have been issuing study design guidance and frameworks for enabling decision-grade evidence generation. This guidance is agnostic of health technology and clinical setting, and is informative in situations in which randomized trial evidence is unavailable, when trial generalizability is of concern, or when post-market uncertainties remain. Table 1 presents RWE study design components found in selected regulatory and HTA frameworks.
Table 1. Study design components in frameworks for the use of real-world evidence in regulatory approval and health technology assessment.
While sharing similar aims for improving the validity and acceptability of RWE for regulatory and reimbursement deliberations, specific RWE framework components, definitions, and implementation differ across jurisdictions. For example, guidance from the National Institute for Health and Care Excellence (NICE) in the United Kingdom is prescriptive in terms of specific study design considerations for estimating causal effects of interest in non-randomized studies, requiring specification of a target trial for emulation (25). To promote understanding of study requirements to generate causal evidence of effectiveness and/or evidence of safety, the FDA recently updated draft guidance recommendations for non-interventional study designs, which mandate similarly transparent reporting of study design considerations, but do not explicitly require targeted trial emulation (16, 26). Canada's Drug Agency (CDA-AMC), Health Canada, and the Institut national d'excellence en santé et en services sociaux (INESSS) collaborated on a reporting framework to assist appraisal of RWE study components (32). CDA-AMC also released methods guidance for HTA that highlights RWE quality appraisal considerations and, similar to the FDA, suggests modern causal inference frameworks such as target trial emulation, but makes no specific methodological recommendations (32, 33). The EMA's recent draft reflection paper on key methodological considerations for non-interventional studies to generate RWE for regulatory purposes draws from both existing and in-development frameworks of the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) (34–36). In an effort to align RWE policies across Asia, a collaboration of academics and HTA agencies in the region, the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) working group, developed non-binding recommendations that include specific guidance on statistical methods but do not recommend any specific study design (37, 38). Calls remain to establish methodological standards for RWE studies (35, 39–41), and the FDA continues to oversee and support demonstration research projects to determine study designs that can generate substantial evidence for regulatory decisions (26, 42).
We contend that harmonization across regulatory and HTA bodies on study design components will promote a better understanding and acceptance of RWE and ultimately form the backbone of agile regulation environments adapted to rapid healthcare innovation, which is inherent to precision oncology. Harmonization efforts must be centered on accepted methodological approaches and study designs that mitigate potential sources of bias to reliably inform regulatory and reimbursement decision-making.
What is the real-world evidence study design and method for analysis?
Skepticism about using RWE in regulatory and reimbursement decisions primarily arises from concerns about biases common in observational studies, but absent in well-conducted randomized controlled trials (24, 43). In particular, confounding bias occurs when a patient characteristic or external factor is associated with both the mechanism of treatment assignment and the outcome of interest. For example, in the case of precision oncology, a key confounder may be the timing of an ‘omics-based diagnostic or treatment relative to cancer diagnosis (44). Failure to adjust for this confounding obfuscates the estimated effect of treatment, resulting in the inability to conclude causality of the effect. The act of randomization eliminates confounding bias by producing a treatment and control group that is well balanced in both observable and unobservable characteristics, including genetic factors. This balance ensures that any measured change in outcomes for the treatment group is attributable to treatment itself rather than systematic differences across the two groups studied.
To avoid confounding bias and ensure that RWE can be reliably used for precision oncology decision-making, we contend that RWE generation must be grounded within a causal theoretical framework. The framework should make explicit the complex, intersecting relationships between observed and unobserved variables and the estimand of interest. Based on the theory underlying Structural Causal Modeling, directed acyclic graphs (DAGs) are a graphical tool for transparently mapping these inter-variable relationships (45). DAGs are a non-parametric, diagrammatic means of representing the complete set of variables involved in a data generating process, that use nodes and unidirectional arcs to depict possible causal and non-causal relationships (46). Included variables and their theorized relationships should mirror real-world patient experiences and can be based on systematic evidence review, behavioral models, large-language models, and stakeholder consultation (47, 48). DAGs enable the manual or algorithmic determination of minimally sufficient covariate adjustment sets for comparative evaluations (49–51). Adjusting for only those covariates included in a sufficient adjustment set guarantees we are able to answer our causal question, without blocking causal paths or unnecessarily reducing degrees of freedom, better powering precision oncology evaluations in rare indications.
Analytical approaches for precision oncology applications
In non-randomized, single-arm precision oncology settings, quasi-experimental approaches can be used alongside causal theoretical frameworks to reliably support regulatory and reimbursement deliberations (50). Quasi-experimental approaches, such as matching or inverse probability of treatment weighting, appear in HTA evidence packages, particularly for new drug classes such as tumor-agnostic treatments and orphan drugs (52, 53). These approaches can enable counterfactual identification of single-arm precision oncology trial participants while addressing both observed and unobserved sources of confounding occurring before and/or after study inclusion. The appropriate quasi-experimental approach depends on the research question, clinical context, primary endpoints, sources of confounding, and available data. Careful consideration of strengths, limitations, and underlying assumptions for competing methods is necessary for unbiased and statistically efficient estimation. For example, matching emulates the effects of randomization on cohort characteristics and enables estimation of an average treatment effect on the treated (ATT) population, but also reduces available data and can be problematic to apply in finite samples common to precision oncology, where genomic heterogeneity results in small benefiting populations (54). Inverse probability of treatment weighting instead estimates an average treatment effect (ATE) using all available data and performs relatively well in rare outcome settings (55), but is sensitive to model misspecification and extreme weights. Both approaches assume positivity and ignorability, requiring thorough overlap and balance assessments to confirm validity.
Once a quasi-experimental approach is selected, the study must be carefully designed to avoid self-inflicted biases (30, 56). Study criteria, exposure, and follow-up definitions each have implications for time and selection-related biases that must be considered alongside analytical correction methods. For example, defining study eligibility and initiating follow-up after receipt of a precision oncology intervention may lead to both selection bias and prevalent user bias (56, 57). Selection bias occurs when study eligibility is non-random, resulting in a study cohort that no longer represents the target population of interest (58). Prevalent-user bias occurs when patients most likely to experience the outcome of interest, such as disease progression or death, experience attrition, and follow-up is limited to only low-risk patients (59). Even study calendar time can introduce bias if differential access to care occurring across treated and control patients remains unaccounted for (60).
For progressive diseases such as cancer, patient prognosis, and treatment landscapes change continuously over time, and study indexing and follow-up definitions are especially important. If the study design allows for delayed entry, where follow-up is initiated after the natural time of origin for some but not all patients, time-varying characteristics, and prognosis may be systematically different from at the time of origin, introducing confounding that must be adjusted (61, 62). Immortal time bias is related to delayed entry and occurs when, by design, there is a period of follow-up time during which either treated or control patients must remain outcome-free (63). For example, if patients experience a delay in accessing ‘omics-based testing after diagnosis, or in receiving targeted treatment after biomarker results (64). While immortal time bias remains common in oncology evaluations (65, 66), this bias can be mitigated through immortal time adjustment methods, such as time-dependent analysis, multiple imputation, participant cloning, or the use of a prevalent new-user cohort design (60, 64, 67). Best practice guidelines on RWE study design and analysis are critically needed to avoid biases common to precision oncology evaluations and practically support decision-grade evidence generation throughout health product life-cycles.
What are real-world data and data systems?
DAGs outline the minimum set of RWD elements needed for causal inference on how a precision oncology intervention affects clinical, economic, and health system outcomes. DAG-informed data elements may include a mix of historical and prospectively collected data, necessitating longitudinal integration of genomic profiling information with other health data such as patient socio-demographic characteristics, disease characteristics, patient-reported outcomes, and clinical outcomes such as overall and progression-free survival (48, 68). Additionally, retrospective and future data on resource use and cost of genomic profiling, prescribed treatment, in-patient hospital stays, physician visits, non-genomic laboratory tests, clinical outcomes, adverse events, and non-cancer prescription drugs are important for valid causal inference on health outcomes and for estimating downstream cost-effectiveness, respectively.
While past efforts have defined core RWD elements needed for precision oncology evaluations (44, 68), the ability of healthcare systems to curate the required data is variable. Missing elements threaten the generalizability and validity of RWE outcomes analyses, with missing data points, measurement errors, and data misclassifications contributing to information bias (58, 69). Data curation is the organization and integration of data collected from siloed sources (70, 71). RWD curation involves the annotation and maintenance of data over time, and may require abstracting important text, image, and other health services information from a variety of structured and unstructured sources, such as electronic medical records, administrative data, registries, patient-reported surveys, or social media (9). As such, data curation incorporates a range of activities and processes, which include cleaning and normalizing data, adding metadata and quality verification, all necessary to create, manage, maintain, and validate required data elements, grounded in a common data model (70, 71). These standardization efforts are critical for achieving complete capture of precision oncology core data elements, facilitating data sharing across institutions and jurisdictions to not only increase generalizability but also augment available sample sizes in rare indications (72).
RWD curation, sharing, and use in precision oncology decision-making remain limited in practice. Healthcare systems face legal, structural, and operational barriers. For example, the generation and use of RWD may be impeded by data stewards' interpretation and implementation of legislation that governs privacy and health information, resulting in persistent data siloing across institutions and jurisdictions (72–74). Systematic use of RWD has been largely confined to individual clinical decision-making, hospital performance reporting, and quality improvement (8, 75, 76). Addressing these barriers while generating regularly refreshed, curated, and transformed actionable data based on harmonized and documented processes is necessary for generalizable decision-grade RWD. Broad privacy-preserving access to decision-grade RWD can then support the generation of decision-grade RWE, ultimately enabling real-world life-cycle assessment.
Real-world life-cycle assessment
Life-cycle activities have always been part of the market approval and reimbursement process (12, 77); however, there remains a need for greater transparency and guidance on the implementation of life-cycle assessment (78, 79). Figure 1 illustrates a hierarchy for the acceptability of real-world life-cycle assessment that builds on previous sections detailing the needs for the harmonization of RWD and data systems and of causal RWE approaches.
Figure 1. Hierarchy for the acceptability of real-world life-cycle assessment.
Life-cycle assessment of outcomes can be used both in regulatory and reimbursement decisions as well as for re-appraisal and disinvestment (3). For example, prior to market authorization and health system reimbursement, a precision oncology intervention may be eligible for managed access, which could support conditional access to technologies whose comparative outcomes are uncertain. To create a managed access agreement, the healthcare system and the sponsor of the health product would agree on research that uses historical data alongside prospectively collected clinical trial and healthcare system data. The RWD would align to the DAG-identified minimum adjustment set and to a core data set for the cost-effectiveness of precision oncology. Broad initiatives for operationalizing RWD for causal analysis are showing promise (48). RWE generated can inform the continuation of managed access, as well as address any post-market conditions of regulatory authorization. Using an evidence-based definition to determine when substantial evidence exists based on life-cycle RWE would also promote transparency and accountability for both stakeholders and regulators (80). In the post-reimbursement phase, continual outcomes surveillance and health technology management provide the opportunity for comparative evaluation of precision oncology interventions versus alternative historical or contemporaneous standards of care, using quasi-experimental cohort comparisons. The outcomes of these phases can inform disinvestment in low-value technologies, freeing resources for other areas of innovation or healthcare.
A scoping review of regulatory and HTA policies recently underscored the need for better alignment on the use of RWE methods (40, 81, 82). Greater clarity and standardization of guidelines can help leverage RWD for improving healthcare decisions (39, 48). Similar needs have been highlighted in the context of international collaboration on RWD and RWE for regulatory decision-making (41). A real-world life-cycle assessment approach with transparent decision-making that limits duplicative efforts for evidence generation in precision oncology can provide meaningful value in addressing the decision problems that expedited regulatory pathways aim to resolve.
Discussion
Real-world life-cycle assessment is a promising solution for addressing precision oncology uncertainty. While regulators and reimbursement agencies are receptive to RWE in settings where randomized controlled trials are limited or unavailable, RWE continues to be infrequently used in decision-making. Routine, health system-integrated life-cycle assessment of precision oncology diagnostics and treatments remains elusive. Stakeholder-driven data and methods guidance are needed to achieve consensus on acceptable forms of RWE, promote evidence generation, and ensure uptake. This guidance should be anchored in the study design and analytical principles discussed throughout this perspective. Any RWE used for regulatory and reimbursement purposes should adhere to an underlying causal theoretical framework, with carefully justified and validated study designs and model estimation that mitigate biases common to observational precision oncology evaluations. While there is emerging consensus found in RWE guidance frameworks supporting causal designs based on target trial emulation (25, 26, 32, 33), its use in submissions is still limited (83). Specific recommendations on important drivers of uncertainty in precision oncology evaluations, such as accurately defining index dates and approaches to emulate randomization for external control arms (84–86), remain necessary to promote the use of real-world trial emulation (82, 83, 87). Supporting RWD curation must meet core precision oncology data requirements for enabling causal analyses, while harmonizing to allow for cross-jurisdictional RWD sharing. Combined, these efforts can facilitate real-world life-cycle assessment that drives sustainable resource allocation for precision oncology.
To simultaneously achieve equitable precision oncology implementation, equity analyses should be considered and embedded in every phase of the life-cycle assessment. During early product prioritization, RWE characterizing differential condition prevalence, access to care, and opportunity to benefit can ensure that research and development explicitly considers the priority population and avoids creating or exacerbating inequities (88, 89). This evidence is particularly salient for precision oncology, in which both drug development and patient selection for treatment depend on the use of genomic variant background libraries that historically underrepresent populations of non-European ancestry (90, 91). Following product development and evaluation, distributional analyses of comparative effectiveness and cost-effectiveness can provide regulators and reimbursement agencies with evidence of precision oncology outcomes variation, including for equity-seeking groups (2, 92, 93). This evidence can inform implementation strategies that mitigate disparities, supported by post-market RWE monitoring of access and uptake across diverse populations.
To fully realize the potential of life-cycle RWE for achieving equitable healthcare, careful considerations of global health system factors will be crucial to future research and operations initiatives. Health systems will vary in their ability to effectuate the life-cycle framework described herein, with reliable real-world assessments of precision oncology interventions requiring substantial investment in RWD infrastructure alongside legal and regulatory modernization. Both financial and non-financial resources will be necessary to support continuous RWE generation and integration into decision-making. Low- and middle-income countries face competing challenges for cancer control, such as public awareness and health literacy, healthcare funding and infrastructure, oncology workforce capacity, cancer screening availability and uptake, diagnostic turnaround times, treatment abandonment, and a lack of palliative care (94). High costs of precision oncology diagnostics and drugs are prohibitive in constrained resource settings, where feasible, high-impact, cost-effective interventions are prioritized (95–97). Federated analytics and validation of RWE transportability across jurisdictions present near-term opportunities for translating life-cycle assessment into global health systems (9, 97). These solutions require thoughtful, partnered design, validation, adjustment, and regulation to ensure suitability for all contexts (98, 99).
Conclusion
Uncertain comparative evidence for ‘omics-based diagnostics and treatments challenges regulatory and reimbursement decision-making globally. As a result, precision oncology implementation remains limited to select patient populations in a few jurisdictions. Real-world life-cycle assessment can reduce evidentiary uncertainty, supporting the sustainable and equitable implementation of precision oncology across healthcare systems. Stakeholder-driven guidance on acceptable RWE methods for unbiased causal inference and significant health data system augmentation is, however, required to achieve a life-cycle assessment. In the long run, global partnerships that leverage innovative, diagonal financing mechanisms alongside health system strengthening may be necessary to deploy real-world life-cycle assessment and bridge the evidentiary gaps in precision oncology worldwide.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
EK: Conceptualization, Data curation, Visualization, Writing – original draft, Writing – review & editing. DW: Conceptualization, Writing – original draft, Writing – review & editing. TB: Conceptualization, Funding acquisition, Writing – original draft, Writing – review & editing. DR: Conceptualization, Funding acquisition, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. This research was supported by Genome British Columbia/Genome Canada (G05CHS) and BC Cancer's Real-World Evidence and Evaluation program, which was funded by the BC Cancer Foundation, aligning with a key priority in BC's 10-year Cancer Action Plan to innovate and advance data and digital means to inform and improve care.
Conflict of interest
The following authors declare competing interests outside of the submitted work: DR reports that his institution has received research funding for a project from Roche Canada. DW co-directs IMPRINT Research Consulting and has consulted for AstraZeneca Canada, Birota Economics Group, and LHS labs.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declare that no Gen AI was used in the creation of this manuscript.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Tran A, Klossner Q, Crain T, Prasad V. Shifting, overlapping and expanding use of “precision oncology” terminology: a retrospective literature analysis. BMJ Open. (2020) 10:e036357. doi: 10.1136/bmjopen-2019-036357
2. Ehman M, Punian J, Weymann D, Regier DA. Next-generation sequencing in oncology: challenges in economic evaluations. Expert Rev Pharmacoecon Outcomes Res. (2024) 24:1115–32. doi: 10.1080/14737167.2024.2388814
3. Regier DA, Pollard S, McPhail M, Bubela T, Hanna TP, Ho C, et al. A perspective on life-cycle health technology assessment and real-world evidence for precision oncology in Canada. NPJ Precis Oncol. (2022) 6:1–7. doi: 10.1038/s41698-022-00316-1
4. Roberts MC, Kennedy AE, Chambers DA, Khoury MJ. The current state of implementation science in genomic medicine: opportunities for improvement. Genet Med. (2017) 19:858–63. doi: 10.1038/gim.2016.210
5. Bayle A, Bonastre J, Chaltiel D, Latino N, Rouleau E, Peters S, et al. ESMO study on the availability and accessibility of biomolecular technologies in oncology in Europe. Ann Oncol. (2023) 34:934–45. doi: 10.1016/j.annonc.2023.06.011
6. Weymann D, Pollard S, Lam H, Krebs E, Regier DA. Toward best practices for economic evaluations of tumor-agnostic therapies: a review of current barriers and solutions. Value Health. (2023) 26:1608–17. doi: 10.1016/j.jval.2023.07.004
7. Mateo J, Steuten L, Aftimos P, André F, Davies M, Garralda E, et al. Delivering precision oncology to patients with cancer. Nat Med. (2022) 28:658–65. doi: 10.1038/s41591-022-01717-2
8. Makady A, van Veelen A, Jonsson P, Moseley O, D'Andon A, de Boer A, et al. Using real-world data in health technology assessment (HTA) practice: a comparative study of five HTA agencies. Pharmacoeconomics. (2018) 36:359–68. doi: 10.1007/s40273-017-0596-z
9. Liu F, Panagiotakos D. Real-world data: a brief review of the methods, applications, challenges and opportunities. BMC Med Res Methodol. (2022) 22:287. doi: 10.1186/s12874-022-01768-6
10. Skydel JJ, Zhang AD, Dhruva SS, Ross JS, Wallach JD, US. Food and drug administration utilization of postmarketing requirements and postmarketing commitments, 2009–2018. Clin Trials. (2021) 18:488–99. doi: 10.1177/17407745211005044
11. Wallach JD, Ross JS, Naci H. The US food and drug administration's expedited approval programs: addressing premarket flexibility with enhanced postmarket evidence generation. Clin Trials. (2018) 15:243–6. doi: 10.1177/1740774518770657
12. Cipriani A, Ioannidis JPA, Rothwell PM, Glasziou P, Li T, Hernandez AF, et al. Generating comparative evidence on new drugs and devices after approval. Lancet. (2020) 395:998–1010. doi: 10.1016/S0140-6736(19)33177-0
13. Angrist JD, Pischke JS. Mostly Harmless Econometrics: An Empiricist's Companion. New Jersey, NJ: Princeton University Press (2009). doi: 10.2307/j.ctvcm4j72
14. Pearl J. Causal inference in statistics: an overview. Stat Surv. (2009) 3:96–146. doi: 10.1214/09-SS057
16. Rahman M, Dal Pan G, Stein P, Levenson M, Kraus S, Chakravarty A, et al. When can real-world data generate real-world evidence? Pharmacoepidemiol Drug Saf. (2024) 33:e5715. doi: 10.1002/pds.5715
17. Burns L, Roux NL, Kalesnik-Orszulak R, Christian J, Hukkelhoven M, Rockhold F, et al. Real-world evidence for regulatory decision-making: guidance from around the world. Clin Ther. (2022) 44:420–37. doi: 10.1016/j.clinthera.2022.01.012
18. Platt R, Brown JS, Robb M, McClellan M, Ball R, Nguyen MD, et al. The FDA sentinel initiative — an evolving national resource. N Engl J Med. (2018) 379:2091–3. doi: 10.1056/NEJMp1809643
19. Ball R, Robb M, Anderson S, Dal Pan G. The FDA's sentinel initiative—a comprehensive approach to medical product surveillance. Clin Pharmacol Ther. (2016) 99:265–8. doi: 10.1002/cpt.320
20. Flynn R, Plueschke K, Quinten C, Strassmann V, Duijnhoven RG, Gordillo-Marañon Gordillo-Marañon M, et al. Marketing authorization applications made to the European medicines agency in 2018–2019: what was the contribution of real-world evidence? Clin Pharmacol Ther. (2022) 111:90–7. doi: 10.1002/cpt.2461
21. Bakker E, Plueschke K, Jonker CJ, Kurz X, Starokozhko V, Mol PGM. Contribution of real-world evidence in European medicines agency's regulatory decision making. Clin Pharmacol Ther. (2023) 113:135–51. doi: 10.1002/cpt.2766
22. Feinberg BA, Gajra A, Zettler ME, Phillips TD, Phillips EG, Kish JK. Use of real-world evidence to support FDA approval of oncology drugs. Value Health. (2020) 23:1358–65. doi: 10.1016/j.jval.2020.06.006
23. Agrawal S, Arora S, Amiri-Kordestani L, de Claro RA, Fashoyin-Aje L, Gormley N, et al. Use of single-arm trials for US food and drug administration drug approval in oncology, 2002-2021. JAMA Oncol. (2023) 9:266–72. doi: 10.1001/jamaoncol.2022.5985
24. Collins R, Bowman L, Landray M, Peto R. The magic of randomization versus the myth of real-world evidence. N Engl J Med. (2020) 382:674–8. doi: 10.1056/NEJMsb1901642
25. National Institute for Health and Care Excellence (NICE). NICE real-world evidence framework. National Institute for Health and Care Excellence (NICE) (2022). Available online at: http://www.nice.org.uk/corporate/ecd9 (Accessed January 10, 2025).
26. United States Food & Drug Administration (FDA). Real-World Evidence: Considerations Regarding Non-Interventional Studies for Drug and Biological Products. FDA (2024). Available online at: https://www.fda.gov/regulatory-information/search- fda-guidance-documents/real-world-evidence-considerations-regarding-non-interven tional-studies-drug-and-biological-products (Accessed January 10, 2025).
27. Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. (2016) 183:758–64. doi: 10.1093/aje/kwv254
28. Al-Faruque F. FDA's Califf: expect to see more RWE-based regulatory decisions. (RAPS Regulatory Focus) (2023). Available online at: https://www.raps.org/News-and-Articles/News-Articles/2023/9/FDA%E2%80%99s-Califf-Expect-to-see-more-RWE-based-regulato (Accessed November 10, 2024).
29. Wang SV, Schneeweiss S, RCT-DUPLICATE initiative, Franklin JM, Desai RJ, Feldman W, et al. Emulation of randomized clinical trials with nonrandomized database analyses: results of 32 clinical trials. JAMA. (2023) 329:1376–85. doi: 10.1001/jama.2023.4221
30. Hernán MA, Wang W, Leaf DE. Target trial emulation: a framework for causal inference from observational data. JAMA. (2022) 328:2446–7. doi: 10.1001/jama.2022.21383
31. Gomes M, Latimer N, Soares M, Dias S, Baio G, Freemantle N, et al. Target trial emulation for transparent and robust estimation of treatment effects for health technology assessment using real-world data: opportunities and challenges. Pharmacoeconomics. (2022) 40:577–86. doi: 10.1007/s40273-022-01141-x
32. Canada's Drug Agency - L'Agence des médicaments du Canada (CDA-AMC). Guidance for Reporting Real-World Evidence. (2023). Available online at: https://www.cda-amc.ca/guidance-reporting-real-world-evidence (Accessed November 10, 2024).
33. Canada's Drug Agency - L'Agence des médicaments du Canada (CDA-AMC). Methods Guide for Health Technology Assessment. (2024). Available online at: https://www.cda-amc.ca/sites/default/files/MG%20Methods/MG0030-Quantitative-Methods-Manual-Line-Numbered.pdf (Accessed December 5, 2024).
34. European Medicines Agency (EMA). Reflection paper on use of real-world data in non-interventional studies to generate real-world evidence - Scientific guideline. (2024). Report No.: EMA/CHMP/150527/2024. Available online at: https://www.ema.europa.eu/en/reflection-paper-use-real-world-data-non-interventional-studies-generate-real-world-evidence-scientific-guideline (Accessed December 5, 2024).
35. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). General Principles on Plan, Design and Analysis of Pharmacoepidemiological Studies That Utilize Real-World Data for Safety Assessment of Medicines [Draft Guidance]. (2024). Available online at: https://database.ich.org/sites/default/files/ICH_M14_Step3_DraftGuideline_2024_0521.pdf (Accessed January 10, 2025).
36. International Council for Harmonisation Oo Technical Requirements for Pharmaceuticals for Human Use (ICH). E9(R1) Statistical Principles for Clinical Trials: Addendum: Estimands and Sensitivity Analysis in Clinical Trials. FDA (2021) Available online at: https://www.fda.gov/regulatory-information/search- fda-guidance-documents/e9r1-statistical-principles-clinical-trials-addendum-estimands-and-sensitivity-analysis-clinical (Accessed January 10, 2025).
37. Kc S, Lin LW, Bayani DBS, Zemlyanska Y, Adler A, Ahn J, et al. What, where, and how to collect real-world data and generate real-world evidence to support drug reimbursement decision-making in Asia: a reflection into the past and a way forward. Int J Health Policy Manag. (2023) 12:1–9. doi: 10.34172/ijhpm.2023.6858
38. Lou J, Kc S, Toh KY, Dabak S, Adler A, Ahn J, et al. Real-world data for health technology assessment for reimbursement decisions in Asia: current landscape and a way forward. Int J Technol Assess Health Care. (2020) 36:474–80. doi: 10.1017/S0266462320000628
39. Ramsey SD, Onar-Thomas A, Wheeler SB. Real-world database studies in oncology: a call for standards. J Clin Oncol. (2024) 42:977–80. doi: 10.1200/JCO.23.02399
40. Sarri G, Hernandez LG. The maze of real-world evidence frameworks: from a desert to a jungle! An environmental scan and comparison across regulatory and health technology assessment agencies. J Comp Eff Res. (2024) 13:e240061. doi: 10.57264/cer-2024-0061
41. European Medicines Agency (EMA). Global regulators call for international collaboration to integrate real-world evidence into regulatory decision-making. European Medicines Agency (2022). Available from: https://www.ema.europa.eu/en/news/global-regulators-call-international-collaboration-integrate-real-world-evidence-regulatory-decision (Accessed November 5, 2024).
42. United States Food & Drug Administration (FDA). Advancing Real-World Evidence Program. (2024). Available online at: https://www.fda.gov/drugs/development-resources/advancing-real-world-evidence-program (Accessed November 10, 2024).
43. Nabhan C, Klink A, Prasad V. Real-world evidence—what does it really mean? e͡xtitJAMA Oncol. (2019) 5:781–3. doi: 10.1001/jamaoncol.2019.0450
44. Dienstmann R, Hackshaw A, Blay JY, Le Tourneau C. Core variables for real-world clinicogenomic data collection in precision oncology. ESMO Real World Data Digit Oncol. (2025) 7:100117. doi: 10.1016/j.esmorw.2025.100117
45. Lipsky AM, Greenland S. Causal directed acyclic graphs. JAMA. (2022) 327:1083–4. doi: 10.1001/jama.2022.1816
46. Tennant PWG, Murray EJ, Arnold KF, Berrie L, Fox MP, Gadd SC, et al. Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations. Int J Epidemiol. (2021) 50:620–32. doi: 10.1093/ije/dyaa213
47. Jiralerspong T, Chen X, More Y, Shah V, Bengio Y. Efficient causal graph discovery using large language models. arXiv [Preprint]. arXiv:2402.01207v1 (2024). Available online at: http://arxiv.org/abs/2402.01207 (Accessed December 6, 2024).
48. Desai RJ, Matheny ME, Johnson K, Marsolo K, Curtis LH, Nelson JC, et al. Broadening the reach of the FDA Sentinel system: a roadmap for integrating electronic health record data in a causal analysis framework. NPJ Digit Med. (2021) 4:1–6. doi: 10.1038/s41746-021-00542-0
49. Evans D, Chaix B, Lobbedez T, Verger C, Flahault A. Combining directed acyclic graphs and the change-in-estimate procedure as a novel approach to adjustment-variable selection in epidemiology. BMC Med Res Methodol. (2012) 12:156. doi: 10.1186/1471-2288-12-156
50. Arif S, MacNeil MA. Utilizing causal diagrams across quasi-experimental approaches. Ecosphere. (2022) 13:e4009. doi: 10.1002/ecs2.4009
51. Pearl J, Glymour M, Jewell NP. Causal Inference in Statistics: A Primer. John Wiley & Sons (2016).
52. CADTH. Entrectinib (Rozlytrek). Can J Health Technol. (2023) 3:1–296. doi: 10.51731/cjht.2023.565
53. Canada's Drug Agency - L'Agence des médicaments du Canada (CDA-AMC). Metreleptin (Myalepta). (2024). Available online at: https://www.cda-amc.ca/metreleptin (Accessed January 10, 2025).
54. Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. (2015) 34:3661–79. doi: 10.1002/sim.6607
55. Franklin JM, Eddings W, Austin PC, Stuart EA, Schneeweiss S. Comparing the performance of propensity score methods in healthcare database studies with rare outcomes. Stat Med. (2017) 36:1946–63. doi: 10.1002/sim.7250
56. Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol. (2016) 79:70–5. doi: 10.1016/j.jclinepi.2016.04.014
57. Nguyen VT, Engleton M, Davison M, Ravaud P, Porcher R, Boutron I. Risk of bias in observational studies using routinely collected data of comparative effectiveness research: a meta-research study. BMC Med. (2021) 19:279. doi: 10.1186/s12916-021-02151-w
58. Sterrantino AF. Observational studies: practical tips for avoiding common statistical pitfalls. Lancet Reg Health Southeast Asia. (2024) 25:100415. doi: 10.1016/j.lansea.2024.100415
59. Acton EK, Willis AW, Hennessy S. Core concepts in pharmacoepidemiology: key biases arising in pharmacoepidemiologic studies. Pharmacoepidemiol Drug Saf. (2023) 32:9–18. doi: 10.1002/pds.5547
60. Suissa S. Single-arm trials with historical controls: study designs to avoid time-related biases. Epidemiology. (2021) 32:94. doi: 10.1097/EDE.0000000000001267
61. Brown S, Lavery JA, Shen R, Martin AS, Kehl KL, Sweeney SM, et al. Implications of selection bias due to delayed study entry in clinical genomic studies. JAMA Oncol. (2022) 8:287–91. doi: 10.1001/jamaoncol.2021.5153
62. Betensky RA, Mandel M. Recognizing the problem of delayed entry in time-to-event studies: better late than never for clinical neuroscientists. Ann Neurol. (2015) 78:839–44. doi: 10.1002/ana.24538
63. Hanley JA. Foster BJ. Avoiding blunders involving ‘immortal time'. Int J Epidemiol. (2014) 43:949–61. doi: 10.1093/ije/dyu105
64. Weymann D, Krebs E, Regier DA. Addressing immortal time bias in precision medicine: practical guidance and methods development. Health Serv Res. (2025) 60:e14376. doi: 10.1111/1475-6773.14376
65. Dall'Olio FG, Rizzo A, Mollica V, Massucci M, Maggio I, Massari F. Immortal time bias in the association between toxicity and response for immune checkpoint inhibitors: a meta-analysis. Immunotherapy. (2021) 13:257–70. doi: 10.2217/imt-2020-0179
66. Newman NB, Brett CL, Kluwe CA, Patel CG, Attia A, Osmundson EC, et al. Immortal time bias in national cancer database studies. Int J Radiat Oncol. (2020) 106:5–12. doi: 10.1016/j.ijrobp.2019.07.056
67. Maringe C, Benitez Majano S, Exarchakou A, Smith M, Rachet B, Belot A, et al. Reflection on modern methods: trial emulation in the presence of immortal-time bias. Assessing the benefit of major surgery for elderly lung cancer patients using observational data. Int J Epidemiol. (2020) 49:1719–29. doi: 10.1093/ije/dyaa057
68. Pollard S, Weymann D, Chan B, Ehman M, Wordsworth S, Buchanan J, et al. Defining a core data set for the economic evaluation of precision oncology. Value Health. (2022) 25:1371–80. doi: 10.1016/j.jval.2022.01.005
69. Hudson M, Suissa S. Avoiding common pitfalls in the analysis of observational studies of new treatments for rheumatoid arthritis. Arthritis Care Res. (2010) 62:805–10. doi: 10.1002/acr.20124
70. Asiimwe R, Lam S, Leung S, Wang S, Wan R, Tinker A, et al. From biobank and data silos into a data commons: convergence to support translational medicine. J Transl Med. (2021) 19:493. doi: 10.1186/s12967-021-03147-z
71. Diaz O, Kushibar K, Osuala R, Linardos A, Garrucho L, Igual L, et al. Data preparation for artificial intelligence in medical imaging: a comprehensive guide to open-access platforms and tools. Phys Med. (2021) 83:25–37. doi: 10.1016/j.ejmp.2021.02.007
72. Pollard S, Ehman M, Hermansen A, Weymann D, Krebs E, Ho C, et al. “I Just assumed this was already being done”: Canadian patient preferences for enhanced data sharing for precision oncology. JCO Precis Oncol. (2024) 8:e2400184. doi: 10.1200/PO.24.00184
73. McPhail M, McCabe C, Regier DA, Bubela T. The importance of and challenges with adopting life-cycle regulation and reimbursement in Canada. Healthc Policy. (2022) 17:81–90. doi: 10.12927/hcpol.2022.26726
74. Bubela T, Garcia R, Beschastnikh I, Talhouk A. Towards a Proportionate and Risk-Based Approach to Federated Data Access in Canada. Toronto, ON: CIFAR (2023).
75. Husereau D, Nason E, Ahuja T, Nikaï E, Tsakonas E, Jacobs P. Use of real-world data sources for Canadian drug pricing and reimbursement decisions: stakeholder views and lessons for other countries. Int J Technol Assess Health Care. (2019) 35:181–8. doi: 10.1017/S0266462319000291
76. Barbazza E, Allin S, Byrnes M, Foebel AD, Khan T, Sidhom P, et al. The current and potential uses of Electronic Medical Record (EMR) data for primary health care performance measurement in the Canadian context: a qualitative analysis. BMC Health Serv Res. (2021) 21:820. doi: 10.1186/s12913-021-06851-0
77. Kristensen FB, Husereau D, Huić M, Drummond M, Berger ML, Bond K, et al. Identifying the need for good practices in health technology assessment: summary of the ISPOR HTA council working group report on good practices in HTA. Value Health. (2019) 22:13–20. doi: 10.1016/j.jval.2018.08.010
78. Pichler FB, Boysen M, Mittmann N, Gilardino R, Bruce A, Bond K, et al. Lifecycle HTA: promising applications and a framework for implementation. An HTAi global policy forum task force report. Int J Technol Assess Health Care. (2024) 40:e50. doi: 10.1017/S0266462324000187
79. Pichler FB, Boysen M, Mittmann N, Gilardino R, Bruce A, Bond K, et al. An operationalization framework for lifecycle health technology assessment: a health technology assessment international global policy forum task force report. Int J Technol Assess Health Care. (2024) 40:e45. doi: 10.1017/S0266462324000199
80. Krebs E, Bubela T, McPhail M, McCabe C, Regier DA, McCabe C. Putting the substance in substantial evidence: an evidence-based approach to flexible drug regulation. Front Med. (2025) 12:fmed.2025.1337890. doi: 10.3389/fmed.2025.1337890
81. Claire R, Elvidge J, Hanif S, Goovaerts H, Rijnbeek PR, Jónsson P, et al. Advancing the use of real world evidence in health technology assessment: insights from a multi-stakeholder workshop. Front Pharmacol. (2024) 14:1289365. doi: 10.3389/fphar.2023.1289365
82. Duffield S, Jónsson P. The real-world impact of national institute for health and care excellence's real-world evidence framework. J Comp Eff Res. (2023) 12:e230135. doi: 10.57264/cer-2023-0135
83. Castanon A, Duffield S, Ramagopalan S, Reynolds R. Why is target trial emulation not being used in health technology assessment real-world data submissions? J Comp Eff Res. (2024) 13:e240091. doi: 10.57264/cer-2024-0091
84. Curtis LH, Sola-Morales O, Heidt J, Saunders-Hastings P, Walsh L, Casso D, et al. Regulatory and HTA considerations for development of real-world data derived external controls. Clin Pharmacol Ther. (2023) 114:303–15. doi: 10.1002/cpt.2913
85. Gray CM, Grimson F, Layton D, Pocock S, Kim J. A framework for methodological choice and evidence assessment for studies using external comparators from real-world data. Drug Saf. (2020) 43:623–33. doi: 10.1007/s40264-020-00944-1
86. Graziadio S, Gregg E, Allen AJ, Neveux P, Monz B, Davenport C, et al. Is the comparator in your diagnostic cost-effectiveness model “Standard of Care”? Recommendations from literature reviews and expert interviews on how to identify and operationalise it. Value Health. (2024) 27:585–97. doi: 10.1016/j.jval.2024.02.003
87. Krebs E, Weymann D, Ho C, Weppler A, Bosdet I, Karsan A, et al. Clinical effectiveness and cost-effectiveness of multigene panel sequencing in advanced melanoma: a population-level real-world target trial emulation. JCO Precis Oncol. (2025) 9:e2400631. doi: 10.1200/PO-24-00631
88. Sullivan JA, Gold ER. Exploring regulatory flexibility to create novel incentives to optimize drug discovery. Front Med. (2024) 11:1379966. doi: 10.3389/fmed.2024.1379966
89. Frutos Pérez-Surio A, Gimeno-Gracia M, Alcácera López MA, Sagredo Samanes MA, Pardo Jario MDP, Salvador Gómez MDT. Systematic review for the development of a pharmaceutical and medical products prioritization framework. J Pharm Policy Pract. (2019) 12:21. doi: 10.1186/s40545-019-0181-2
90. Landry LG, Ali N, Williams DR, Rehm HL, Bonham VL. Lack Of diversity in genomic databases is a barrier to translating precision medicine research into practice. Health Aff. (2018) 37:780–5. doi: 10.1377/hlthaff.2017.1595
91. Dutta R, Vallurupalli M, McVeigh Q, Huang FW, Rebbeck TR. Understanding inequities in precision oncology diagnostics. Nat Cancer. (2023) 4:787–94. doi: 10.1038/s43018-023-00568-1
92. Diaby V, Ali A, Babcock A, Fuhr J, Braithwaite D. Incorporating health equity into value assessment: frameworks, promising alternatives, and future directions. J Manag Care Spec Pharm. (2021) 27:S22–9. doi: 10.18553/jmcp.2021.27.9-a.s22
93. Ward T, Mujica-Mota RE, Spencer AE, Medina-Lara A. Incorporating equity concerns in cost-effectiveness analyses: a systematic literature review. Pharmacoeconomics. (2022) 40:45–64. doi: 10.1007/s40273-021-01094-7
94. Bamodu OA, Chung CC. Cancer care disparities: overcoming barriers to cancer control in low- and middle-income countries. JCO Glob Oncol. (2024) 10:e2300439. doi: 10.1200/GO.23.00439
95. Bharadwaj M, Vallurupalli M, Huang FW. Global precision oncology: a call to action on expanding access to targeted cancer therapies. Oncologist. (2021) 26:353–5. doi: 10.1002/onco.13708
96. Radich JP, Briercheck E, Chiu DT, Menon MP, Torra OS, Yeung CCS, et al. Precision medicine in low- and middle-income countries. Annu Rev Pathol Mech Dis. (2022) 17:387–402. doi: 10.1146/annurev-pathol-042320-034052
97. Levy NS, Arena PJ, Jemielita T, Mt-Isa S, McElwee S, Lenis D, et al. Use of transportability methods for real-world evidence generation: a review of current applications. J Comp Eff Res. (2024) 13:e240064. doi: 10.57264/cer-2024-0064
98. Dychiao RG, Nazer L, Mlombwa D, Celi LA. Artificial intelligence and global health equity. BMJ. (2024) 387:q2194. doi: 10.1136/bmj.q2194
99. Kaushik A, Barcellona C, Mandyam NK, Tan SY, Tromp J. Challenges and opportunities for data sharing related to artificial intelligence tools in health care in low- and middle-income countries: systematic review and case study from Thailand. J Med Internet Res. (2025) 27:e58338. doi: 10.2196/58338
Keywords: real-world evidence (RWE), real-world data (RWD), life-cycle assessment, causal inference, regulatory science, decision making, regulatory acceptance and use, precision oncology
Citation: Krebs E, Weymann D, Bubela T and Regier DA (2025) How life-cycle real-world evidence can bridge evidentiary gaps in precision oncology. Front. Med. 12:1563950. doi: 10.3389/fmed.2025.1563950
Received: 20 January 2025; Accepted: 31 July 2025;
Published: 02 September 2025.
Edited by:
Hubert G. Leufkens, Utrecht University, NetherlandsReviewed by:
Jason Robert Guertin, Laval University, CanadaCopyright © 2025 Krebs, Weymann, Bubela and Regier. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Dean A. Regier, ZGVhbi5yZWdpZXJAdWJjLmNh
†These authors have contributed equally to this work and share first authorship