Skip to main content

Novel methods and technologies for the evaluation of drug outcomes and policies

Cover image for Research Topic "Novel methods and technologies for the evaluation of drug outcomes and policies"
101kviews
Editorial18 April 2024
Editorial: Novel methods and technologies for the evaluation of drug outcomes and policies

Globally, providing quality, equitable healthcare by accelerating patient access to new, promising health technologies while balancing the impact of their increased expenditures remains a global challenge. In parallel, the landscape of techniques and tools available to evaluate the safety and effectiveness of drugs is rapidly evolving with the advent of novel technologies and methodologies, thereby re-inventing the way we evaluate health outcomes and policies. This special issue of Frontiers in Pharmacology presents a compelling collection of scientific papers that delve into these advancements, offering insights into the latest developments in this dynamic field.

Artificial intelligence (AI) and machine learning (ML) methodologies are key themes explored in three papers within this special issue. The study by Md Muntasir Zitu and colleagues on the generalizability of ML methods in detecting adverse drug events from clinical narratives in electronic medical records is a testament to the potential of AI in enhancing drug safety monitoring. Adamson and colleagues' application of AI and ML in extracting real-world data from electronic health records (EHRs) is a stride forward in oncology research. This approach exemplifies how technology can enhance the curation of health records into valuable data sources. Vithlani et al. systematically review the conduct and reporting of health economic evaluations for AI-based healthcare interventions. Their work reveals the rapid growth in this area and the necessity for specific reporting standards to enhance transparency and decision-making in AI intervention evaluations. Some believe the increasing use of AI and ML raises ethical concerns regarding data privacy, bias, and transparency.

There was vital discussion around dominant topics in Health Technology Assessment (HTA); integrating real-world evidence (RWE) in HTA, expanding analytical approaches (cost-effectiveness, equity-informed analyses) to include considerations beyond clinical and economic value drivers, and exploring new, reactive HTA approaches. Claire et al. bring forward insights, drawn from a multi-stakeholder workshop, to address the slow adoption of RWE in HTA compared to regulatory processes and the underlying reasons for staying behind. They emphasize the need for developing resources to promote best practices for conducting RWE studies, comprehensive training, stakeholder collaboration, and impactful research projects to bridge this gap, thereby enhancing HTA's evidence base for informed healthcare decisions. Muir et al.'s review on integrating additional value elements in HTA modeling methods is a call to broaden the scope of health technology assessments. By incorporating societal values and health equity, their work advocates for a more holistic approach to evaluating new therapies. In the same direction, Zebrowska and team’s groundbreaking work on quantifying the impact of novel metastatic cancer therapies on health inequalities is a sobering reminder of the disparities in healthcare by offering an example of equity-informed analysis. Their study highlights how advancements in treatments may inadvertently widen the survival gap among different patient groups, emphasizing the need for more equitable healthcare solutions.

Cheyne et al. draw parallels between 'living' clinical practice guidelines and HTA. Their reflections on incorporating continuous evidence synthesis in HTA processes offer a new paradigm in healthcare evaluation, ensuring that HTA remains responsive and current in a rapidly evolving evidence landscape.

Moving to topics on advanced RWD analysis techniques in health economics and outcomes research (HEOR), the selected articles presented solutions for evaluating effectiveness and safety for new drugs in rare and very rare diseases and presented case study applications in causal inference and pharmacoepidemiology. Mackay and Springford's advocacy for Bayesian methods in evaluating treatments for rare indications addresses a critical gap in HEOR. They argue for the use of Bayesian approaches to overcome challenges in small sample sizes and disconnected evidence networks, paving the way for more nuanced and robust analysis in rare disease settings. Franchini and colleagues introduce an innovative approach in discrete event simulation, focusing on event-specific probabilities and distributions, especially in the context of censored data. Their methodological advancements in modeling competing events hold significant promise for more accurate and nuanced analysis in pharmacoeconomic studies.

Causal inference principles applied by Polito and team to external control analysis in observational data is a noteworthy contribution. By defining the estimand attributes and selecting appropriate estimators, their study offers a refined approach to evaluating long-term survival outcomes in metastatic non-small cell lung cancer. Jeong and team's use of network analysis to elucidate the dynamic landscape of drug-drug interactions offers a novel perspective. Their work underscores the potential of computational methods in identifying key research areas and informing clinical practice. The analysis of the FDA Adverse Event Reporting System by Si Su and colleagues offers a deep dive into the adverse event profiles of Denosumab and Zoledronic acid. Their findings provide invaluable insights for clinicians and policymakers, highlighting the importance of ongoing safety monitoring in pharmacovigilance.

Finally, the living systematic review by Elvidge and colleagues provides a crucial update on the economic evaluations of COVID-19 diagnostics and treatments emphasizing the need for a real time, regularly updated decision-making. Two years into the pandemic, their work synthesizes cost-effectiveness evidence for various interventions, highlighting the importance of making informed healthcare decisions in the rapidly changing landscape of COVID-19 management as new data emerges. This study underscores the ongoing need for living robust economic evaluations in guiding healthcare strategies, especially in a rapidly-changing pandemic setting.

This special issue not only reflects the recent trends in HEOR and rapid advancements in drug evaluation and policy research but also underscores the need for continuous adaptation and integration of novel methods in healthcare decision-making. It remains a potential challenge in accessing and implementing novel technologies, particularly in resource-limited settings. From the economic evaluations of emerging therapies to the cutting-edge use of AI and ML in data analysis, these studies collectively push the boundaries of current knowledge, paving the way for more informed, efficient, and equitable healthcare systems.

2,218 views

ORIGINAL RESEARCH
26 January 2024
Applying the estimand and target trial frameworks to external control analyses using observational data: a case study in the solid tumor setting
Letizia Polito, 6 more and Dominik Heinzmann

7,383 views

10 citations

METHODS
24 November 2023
Quantifying the impact of novel metastatic cancer therapies on health inequalities in survival outcomes
Karolina Zebrowska, 4 more and Koen Degeling

2,871 views

5 citations

CORRECTION
17 November 2023
Corrigendum: Therapy by physician–pharmacist combination and economic returns for cancer pain management in China: a cost-effectiveness analysis
Xikui Lu, 8 more and Jian Xiao

1,313 views

ORIGINAL RESEARCH
1 November 2023
Indication and adverse event profiles of denosumab and zoledronic acid: based on U.S. FDA adverse event reporting system (FAERS)
Si Su, 5 more and Kuan Li

8,000 views

17 citations

ORIGINAL RESEARCH
3 October 2023
Revealing the dynamic landscape of drug-drug interactions through network analysis
Eugene Jeong, 4 more and You Chen

4,867 views

6 citations

ORIGINAL RESEARCH
30 October 2023
Implementing competing risks in discrete event simulation: the event-specific probabilities and distributions approach
Fanny Franchini, 2 more and Koen Degeling

2,882 views

3 citations

Background: As artificial intelligence (AI) continues to advance with breakthroughs in natural language processing (NLP) and machine learning (ML), such as the development of models like OpenAI's ChatGPT, new opportunities are emerging for efficient curation of electronic health records (EHR) into real-world data (RWD) for evidence generation in oncology. Our objective is to describe the research and development of industry methods to promote transparency and explainability.We applied NLP with ML techniques to train, validate, and test the extraction of information from unstructured documents (e.g., clinician notes, radiology reports, lab reports, etc.) to output a set of structured variables required for RWD analysis. This research used a nationwide electronic health record (EHR)-derived database. Models were selected based on performance. Variables curated with an approach using ML extraction are those where the value is determined solely based on an ML model (i.e., not confirmed by abstraction), which identifies key information from visit notes and documents. These models do not predict future events or infer missing information.We developed an approach using NLP and ML for extraction of clinically meaningful information from unstructured EHR documents and found high performance of output variables compared with variables curated by manually abstracted data. These extraction methods resulted in research-ready variables including initial cancer diagnosis with date, advanced/metastatic diagnosis with date, disease stage, histology, smoking status, surgery status with date, biomarker test results with dates, and oral treatments with dates. Conclusions: NLP and ML enable the extraction of retrospective clinical data in EHR with speed and scalability to help researchers learn from the experience of every person with cancer.

14,505 views

107 citations

Evaluating efficacy and real-world effectiveness for novel therapies targeting rare mutations or patient subpopulations with unmet needs is a growing challenge in health economics and outcomes research (HEOR). In these settings it may be difficult to recruit enough patients to run adequately powered randomized clinical trials, resulting in greater reliance on single-arm trials or basket trial designs. Additionally, evidence networks for performing network meta-analysis may be sparse or disconnected when comparing available treatments in narrower patient populations. These challenges create an increased need for use of appropriate methods for handling small sample sizes, structural modelling assumptions and more nuanced decision rules to arrive at “best-available evidence” on comparative and non-comparative efficacy/effectiveness. We advocate for greater use of Bayesian methods to address these challenges as they can facilitate efficient and transparent borrowing of information across varied data sources under flexible modelling assumptions, probabilistic sensitivity analysis to assess model assumptions, and more nuanced decision-making where limited power reduces the utility of classical frequentist hypothesis testing. We illustrate how Bayesian methods have been recently used to overcome several challenges of rare indications in HEOR, including approaches to borrowing information from external data sources, evaluation of efficacy in basket trials, and incorporating non-randomized studies into network meta-analysis. Lastly, we provide several recommendations for HEOR practitioners on appropriate use of Bayesian methods to address challenges in the rare disease setting.

3,264 views

5 citations

Economic evaluations of artificial intelligence-based healthcare interventions: a systematic literature review of best practices in their conduct and reporting
Objectives: Health economic evaluations (HEEs) help healthcare decision makers understand the value of new technologies. Artificial intelligence (AI) is increasingly being used in healthcare interventions. We sought to review the conduct and reporting of published HEEs for AI-based health interventions.We conducted a systematic literature review with a 15-month search window (April 2021 to June 2022) on 17 th June 2022 to identify HEEs of AI health interventions and update a previous review. Records were identified from 3 databases (Medline, Embase, and Cochrane Central). Two reviewers screened papers against predefined study selection criteria. Data were extracted from included studies using prespecified data extraction tables. Included studies were quality assessed using the National Institute for Health and Care Excellence (NICE) checklist. Results were synthesized narratively.Results: A total of 21 studies was included. The most common type of AI intervention was automated image analysis (9/21, 43%) mainly used for screening or diagnosis in general medicine and oncology. Nearly all were cost-utility (10/21, 48%) or cost-effectiveness analyses (8/21, 38%) that took a healthcare system or payer perspective. Decision-analytic models were used in 16/21 (76%) studies, mostly Markov models and decision trees. Three (3/16, 19%) used a short-term decision tree followed by a longer-term Markov component. Thirteen studies (13/21, 62%) reported the AI intervention to be cost effective or dominant. Limitations tended to result from the input data, authorship conflicts of interest, and a lack of transparent reporting, especially regarding the AI nature of the intervention.Published HEEs of AI-based health interventions are rapidly increasing in number. Despite the potentially innovative nature of AI, most have used traditional methods like Markov models or decision trees. Most attempted to assess the impact on quality of life to present the cost per QALY gained. However, studies have not been comprehensively reported. Specific reporting standards for the economic evaluation of AI interventions would help improve transparency and promote their usefulness for decision making. This is fundamental for reimbursement decisions, which in turn will generate the necessary data to develop flexible models better suited to capturing the potentially dynamic nature of AI interventions.

13,143 views

67 citations

PERSPECTIVE
24 August 2023
What could health technology assessment learn from living clinical practice guidelines?
Saskia Cheyne, 4 more and Sarah Norris

3,565 views

5 citations

We assessed the generalizability of machine learning methods using natural language processing (NLP) techniques to detect adverse drug events (ADEs) from clinical narratives in electronic medical records (EMRs).We constructed a new corpus correlating drugs with adverse drug events using total 1394 clinical notes of 47 randomly selected patients who received immune checkpoint inhibitors (ICIs) from 2011 to 2018 at The Ohio State University James Cancer Hospital, annotating189 drug-ADE relations in single sentences within the medical records. We also used data of Harvard's publicly available 2018 National Clinical Challenge (n2c2), which includes 505 discharge summaries with annotations of 1355 single-sentence drug-ADE relations. We applied classical machine learning (support vector machine [SVM]), deep learning (convolutional neural network [CNN] and bidirectional long short-term memory [BiLSTM]), and state-of-the-art transformer-based (bidirectional encoder representations from transformers [BERT] and ClinicalBERT) methods trained and tested in the two different corpora and compared performance among them to detect drug-ADE relationships.ClinicalBERT detected drug-ADE relationships better than the other methods when trained using our data set and tested in n2c2 (ClinicalBERT F-score, 0.78; other methods, F-scores, 0.61-0.73) and when trained using the n2c2 data set and tested in ours (ClinicalBERT F-score, 0.74; other methods, F-scores, 0.55-0.72).Comparison among several machine learning methods demonstrated the superior performance and therefore greatest generalizability of findings of ClinicalBERT for the detection of ADEs for the detection of drug-ADE relations from clinical narratives in electronic medical records.

6,109 views

37 citations