Data sources and applied methods for paclitaxel safety signal discernment

Background Following the identification of a late mortality signal, the Food and Drug Administration (FDA) convened an advisory panel that concluded that additional clinical study data are needed to comprehensively evaluate the late mortality signal observed with the use of drug-coated balloons (DCB) and drug-eluting stent (DES). The objective of this review is to (1) identify and summarize the existing clinical and cohort studies assessing paclitaxel-coated DCBs and DESs, (2) describe and determine the quality of the available data sources for the evaluation of these devices, and (3) present methodologies that can be leveraged for proper signal discernment within available data sources. Methods Studies and data sources were identified through comprehensive searches. original research studies, clinical trials, comparative studies, multicenter studies, and observational cohort studies written in the English language and published from January 2007 to November 2021, with a follow-up longer than 36 months, were included in the review. Data quality of available data sources identified was assessed in three groupings. Moreover, accepted data-driven methodologies that may help circumvent the limitations of the extracted studies and data sources were extracted and described. Results There were 39 studies and data sources identified. This included 19 randomized clinical trials, nine single-arm studies, eight registries, three administrative claims, and electronic health records. Methodologies focusing on the use of existing premarket clinical data, the incorporation of all contributed patient time, the use of aggregated data, approaches for individual-level data, machine learning and artificial intelligence approaches, Bayesian approaches, and the combination of various datasets were summarized. Conclusion Despite the multitude of available studies over the course of eleven years following the first clinical trial, the FDA-convened advisory panel found them insufficient for comprehensively assessing the late-mortality signal. High-quality data sources with the capabilities of employing advanced statistical methodologies are needed to detect potential safety signals in a timely manner and allow regulatory bodies to act quickly when a safety signal is detected.

Background: Following the identification of a late mortality signal, the Food and Drug Administration (FDA) convened an advisory panel that concluded that additional clinical study data are needed to comprehensively evaluate the late mortality signal observed with the use of drug-coated balloons (DCB) and drug-eluting stent (DES).The objective of this review is to (1) identify and summarize the existing clinical and cohort studies assessing paclitaxel-coated DCBs and DESs, (2) describe and determine the quality of the available data sources for the evaluation of these devices, and (3) present methodologies that can be leveraged for proper signal discernment within available data sources.Methods: Studies and data sources were identified through comprehensive searches.original research studies, clinical trials, comparative studies, multicenter studies, and observational cohort studies written in the English language and published from January 2007 to November 2021, with a followup longer than 36 months, were included in the review.Data quality of available data sources identified was assessed in three groupings.Moreover, accepted data-driven methodologies that may help circumvent the limitations of the extracted studies and data sources were extracted and described.Results: There were 39 studies and data sources identified.This included 19 randomized clinical trials, nine single-arm studies, eight registries, three administrative claims, and electronic health records.Methodologies focusing on the use of existing premarket clinical data, the incorporation of all contributed patient time, the use of aggregated data, approaches for individual-level data, machine learning and artificial intelligence approaches, Bayesian approaches, and the combination of various datasets were summarized.

Introduction
Drug-coated balloons (DCBs) and drug-eluting stents (DESs) are frequently used in revascularization procedures among patients diagnosed with atherosclerosis.More specifically, devices coated or eluting paclitaxel have been associated with a decreased risk of restenosis and reintervention (1,2).Paclitaxel hinders scar tissue from forming in the treated vessel, thus preventing restenosis.On December 18, 2018, Katsanos et al., published a meta-analysis of long-term mortality rates in 28 randomized controlled trials (RCTs) in subjects treated with paclitaxel-coated devices, compared to uncoated control devices, in the femoral or popliteal arteries (3).The meta-analysis included publicly available data from clinical trials that evaluated DCB and DES.The clinical trials included devices available within and outside the United States (US) and captured 1-, 2-, and 5-year studylevel mortality data.The authors concluded that the risk of death was significantly greater in patients treated with DCB and DES devices than the control devices at each assessed timepoint.
In June 2019, the Food and Drug Administration (FDA) convened a public advisory committee meeting to discuss late mortality signal and provide recommendations on the necessary regulatory actions (4).The committee reviewed the existing evidence on the use of DCB and DES and noted that the studies thus far, including the meta-analysis, suffer from critical limitations.These limitations include the lack of patient-level data, cause of death information, detailed paclitaxel dose information, and information regarding missing data and followup data.Given these limitations, the panel and FDA agreed that additional clinical study data are needed to comprehensively evaluate the late mortality signal.
High-quality data with applied appropriate statistical methods are needed to accurately ascertain a signal from a device that may not be performing as anticipated in premarket clinical trials.While RCTs and other clinical studies provide foundational evidence on the safety and effectiveness of a device, real-world data sources that capture the clinical use of these devices among the broader population can provide further insight into the devices' performance.Even when available, high-quality data sources are not sufficient for the assessment of devices.Appropriate statistical methods relevant to the leveraged data sources need to be employed to minimize bias and produce the needed evidence to inform regulatory and clinical decision making.
The objective of this review is thus to (1) identify and summarize the existing clinical and cohort studies assessing paclitaxel-coated DCBs and DESs, (2) describe and determine the quality of the available data sources for the evaluation of these devices, and (3) present methodologies that can be leveraged for proper signal discernment within available data sources.

Identification of studies
Comprehensive searches conducted in MEDLINE, EMBASE, and clinicaltrials.govidentified relevant completed or ongoing studies and data sources.The search strategy used the following terms: "paclitaxel-coated balloon," "paclitaxel-eluting stent," "paclitaxel drug-coated balloon," "paclitaxel drug-eluting stent," "DCB," and "DES."Studies initiated, and data sources available from January 2007 to November 2021 were identified.Including studies that have been initiated but not yet completed allows for the comprehensive assessment of existing collected data and upcoming soon-to-be available data.Bibliographies were crossreferenced for additional citations that did not arise in the original search.Original research, clinical trials, comparative studies, multicenter studies, and observational cohort studies written in the English language and evaluating the paclitaxelcoated balloons or paclitaxel-eluting stents were included in the review.Identified studies or data sources with a follow-up duration of fewer than 36 months were excluded.

Data sources quality assessment
Data quality considerations from regulatory, international societies, and initiative guidance were reviewed.These documents indicate the need for data relevance, reliability, and robustness to have sufficient quality to be "fit for purpose" and address research questions.
Based on the recommended quality assessment criteria for realworld evidence (RWE) and considerations for signal discernment regarding a long-term safety outcome, we determined that data quality could be assessed in three groupings: (1) availability of critical data elements, (2) study design for the original data collection or data source analysis, and (3) four questions specific to data quality assessment (5)(6)(7)(8)(9).Critical data elements included data related to device exposure, mortality, lifestyle, comorbidities, medications, procedures, and physical status/frailty.Additionally, the number of patients in the study at the time of the procedure and the 3 and 5 years following the procedure were recorded.Questions related to the study design, data quality, to the objectives of the original study, generalizability of findings, and the underlying population from the original study were included in the assessment.
The four additional questions specific to data quality were: • Were there any changes in variable capture or by study site over the study period?This question clarifies whether there were substantial changes in an RCT protocol or transitions in coding elements for RWE (e.g., transition from ICD-9-CM to ICD-10-CM) during the study period.
• What was the timing between points of data capture?This question clarifies whether there were extended time periods between data collection points.• Will the data source owners (or researchers conducting signal refinement) be able to utilize patient-level data for additional analysis?This question clarifies whether investigators could perform additional analyses on the data collected.• Do the data source owners (or researchers conducting signal refinement) have the ability to obtain and utilize clinical records for patients included in the data source?This question clarifies whether the data is accessible for validation purposes and further hypothesis testing with covariates not collected in the original study.
The authors assessed these aspects of data quality for each RCT, single-arm, and RWE data source.

Data-driven methodologies for the assessment of identified data sources
Following the extraction of relevant studies and data sources, data-driven methodologies commonly used within the various data types and established within the statistical, regulatory, and clinical communities that may help circumvent the limitations of the respective studies and data sources were identified.

Results
There were 39 studies and data sources identified.This included 19 RCTs, nine single-arm studies, eight registries, three administrative claims, and electronic health records.All the included studies and data sources are summarized in Table 1.

RCTs leading to device approval
Eight brands of paclitaxel-coated devices were evaluated or are currently being evaluated in 14 premarket RCTs (Table 1), of which 7 RCTs were conducted outside of the US (OUS).The total sample size of these trials ranged between 100 and 532 subjects.The number of patients treated with paclitaxel-coated devices in these RCTs ranged from 48 to 524 subjects.Four trials compared DCBs vs. DCBs, DCBs vs. DES, and DES vs. DES.The remaining randomized subjects received paclitaxel-coated devices or plain old balloon angioplasty (POBA).The majority of these RCTs (n = 11) had primary endpoints for the safety and efficacy of these devices, and the same number have a follow-up duration of five years.

RCTs conducted postmarket
Four European postmarket RCTs, including two registry-based RCTs, were identified (Table 1).The studies evaluated two FDAapproved DES and paclitaxel-coated devices in Sweden.These postmarket RCTs included more patients (220-2,400 subjects) than the premarket studies with up to 1,200 subjects exposed to a paclitaxel-coated device.The majority of the trials evaluate effectiveness as the primary endpoint.All trials have a follow-up duration of 5 years and are expected to be completed in one to five years or by 2025 at the latest.

Single-arm and cohort studies
Paclitaxel-coated devices were evaluated in two single-arm US studies and four OUS studies (Table 1).Half of these studies include safety and effectiveness as primary endpoints or have a follow-up duration of 5 years.The sample sizes range between 13 and nearly 1,500 subjects.One study evaluated a DCB not approved for commercial use in the US (as of August 2020).A retrospective cohort study examined all-cause mortality comparing DES (n = 285) with non-paclitaxel-coated devices (POBA or bare-metal stents, n = 1,250) in Japan was also identified.This cohort study had a median follow-up of 3.4 years (interquartile range: 2.1, 5.7).

Identified data sources
A total of 7 distinct registry-based RCTs were identified.The data sources included information on seven brands of FDAapproved DCBs (three approved for commercial use in the US) and two brands of FDA-approved DESs.In addition to the registry-based RCTs identified, one coordinated registry network linking five data sources capturing vascular procedures internationally was identified.Three additional data sources captured private or commercial state and national level administrative claims as well as electronic health records.

Quality assessment
Almost all data sources capture either all-cause or cardiovascular (CV)-related mortality in regular intervals of 1, 6, 12, 24, and 36-month intervals.Most data sources only present aggregate data and rarely make patient-level data available.Given that most data sources are clinical trials, patient records beyond what is collected are often not accessible.Quality assessments of the included studies and data sources are summarized in Table 2.

Identified data-driven methodologies
Given the extracted studies, data sources, and their respective limitations, methodologies focusing on the use of existing premarket clinical data, the use of real-world data (RWD) to overcome RCT limitations (i.e., lost to follow-up), approaches for individual-level data, machine learning and artificial intelligence approaches, Bayesian approaches, and the combination of various datasets were summarized.

Discussion
The review identified 39 studies and data sources that can aid in the signal detection of paclitaxel DCBs and DESs.While RCTs provide critical information to regulatory bodies prior to approval and, have the potential to produce high-quality, detailed data and have minimal risk of introducing confounding due to randomization, they suffer from critical limitations.Trials, however, are limited to specific study populations, may greatly differ in eligibility criteria for included patients between studies, have low external validity, have short follow-up periods and are plagued by possible high rates of loss to follow-up.If a device is authorized, its application and performance in clinical practice must be continuously assessed, under both the existing and broader conditions of use, to detect any potential safety and effectiveness signals promptly.High-quality RWD, as characterized by the FDA (5), employing appropriate statistical methods, summarized below, can build upon data collected from premarket studies and provide a more comprehensive and continuous assessment of devices.

Existing real-world data sources for the assessment of paclitaxel-coated devices 4.1.1 Strategically coordinated registry networks (CRNs)
Coordinated Registry Networks (CRNs) create a robust and comprehensive source for medical device evaluation by growing existing data sources' capacity through the organization and linkage data systems to circumvent the limitations of individual data sources and create a robust and comprehensive source for medical device evaluation (10,11).As with all databases, the quality and capability of a registry play a vital role in its ability for accurate and timely evaluations.Robust registries continuously and consistently collect data relevant to multiple stakeholders, including patients, physicians, manufacturers, and regulatory bodies.It is paramount that registries be generalizable to the population utilizing the medical device and afford evaluation of meaningful outcomes that improve the quality of patient care (12).Registries that incorporate standardized data elements and standardized libraries for device identification, such as the Fast Healthcare Interoperability Resources (FHIR) and unique device identifiers (UDI), should be employed.The standardization of data elements and device identifiers improves interoperability with other data sources and device identification capabilities.High-quality registries have numerous advantages.They can capture a large number and variety of procedures and devices, reflect current medical practice, have high external validity, and have the potential for long followup times to assess devices over their total product life cycle.However, some limitations include that individuals in registries are not randomized.In addition, limited demographic and clinical data may be collected on individuals in the registry.The risk of confounding in analyses may, therefore, be increased.High-quality registries can be linked to several claims databases, such as the Center for Medicare and Medicaid Services (CMS) claims and the Statewide Planning and Research Cooperative System (SPARCS).Claims complement registries by collecting comprehensive patientlevel characteristics, diagnoses, treatments, hospitalizations, and charges for inpatient as well as outpatient services.Thus allowing researchers to evaluate all reported events or diagnoses that are related and unrelated to the medical device.
The Vascular Implant Surveillance & Interventional Outcomes Network (VISION) CRN captures detailed demographic and clinical data of patients who undergo vascular procedures with the ultimate goal of improving the quality, safety, effectiveness, and cost of vascular healthcare.VISION covers 605,322 patients in the VQI registry from over 600 academic and community hospitals across the US and Canada (13).To augment the VISION-CRN, the Vascular Quality Initiative (VQI) registry captures mortality through follow-up data submitted by providers and linkage to the social security index data.International efforts of the VISION-CRN include the International Consortia of Vascular Registries (ICVR), which has direct data sharing from national registries in 13 countries and distributed systems for research and surveillance.ICVR continues to engage in international collaborations to perform studies within health insurance claims and registry data, such as the German administrative claims database.These analyses include thousands of health insurance claims, survival data, and event outcomes occurring between 2007 and 2017.

Administrative claims databases
Health insurance claims can be leveraged to identify and study paclitaxel-coated devices among commercially insured patients.Claims produce procedure codes in the form of current procedural terminology (CPT) and International Classification of Diseases (ICD) codes that only identify whether a medical device-related procedure was performed.It is important to recognize that these codes are input for billing purposes and not research purposes.Unlike national drug codes (NDC) that can The National Patient-Centered Clinical Research Network (PCORNet) captures and combines Electronic Health Record (EHR) data from multiple institutions within a given area.The captured data permit the identification of procedures and provide information on follow-up visits within a network of institutions.The New York City Insight Clinical Research Network gathers EHR data from five major hospitals in the city.Implanted paclitaxel devices can be identified using the Healthcare Common Procedure Coding System (HCPCS).Follow-up data regarding the patients receiving these interventions can be examined using PCORNet data.

Data-driven methodologies for the assessment of paclitaxel-coated devices
High-quality data are available and accessible, though still useful in many other aspects of medical-device-related research, and may not be sufficient to properly detect the signals needed to raise regulators a device's performance.Appropriate statistical analyses tailored to the type, amount, and elements available in data sources need to be employed to take advantage of a data source's capabilities and accurately identify any potential signals (16).

Leveraging existing premarket clinical data
Data from RCTs may be utilized to assess devices not only in the premarket phase but also postmarket.RCTs may be utilized to identify potential signals when real-world data (RWD) capturing the devices of interest are not yet available.One may initially fill the gaps with existing data that has already been analyzed.While randomization generally provides balance at the baseline between two alternatives (e.g., devices with and without paclitaxel), missing visits or loss-to-follow-up within an RCT over time can lead to unbalanced groups in the assessment of long-term outcomes (e.g., mortality 3-5 years after initial procedure).Thus, analyzing the RCTs as RWE cohorts may elucidate important, data-driven factors affecting the outcome.For such analyses, accounting for both time-varying factors and competing risks (e.g., loss to follow-up due to death) is essential.Additionally, accounting for differences in patients who were and were not included in RCTs is crucial.The separate evaluation of patients in RWE studies who would have met RCT inclusion/ exclusion criteria vs. those who were ineligible may enlighten researchers regarding critical differences between those included and excluded from RCTs, important confounders, timing of outcomes, and other interactions with healthcare.

Incorporating all contributed patient time
Time-to-event statistical models are beneficial for assessing long-term outcomes of medical devices in RWD because they consider the entirety of patient-contributed time.These methods mitigate the effects of loss to follow-up and allow individuals with varying follow-up times who may or may not have experienced an outcome of interest to contribute to the analysis.

General approaches for aggregated data
• When aggregated data are the only data available, then traditional meta-analytic approaches that combine data across studies comparing the same treatments are commonly used to generate estimates.Thus, estimates from studies comparing the control, plain old balloon angioplasty (POBA), with paclitaxel-coated devices can be combined to provide a new (combined) device effect.A network meta-analysis combines treatment estimates that have been compared within a study (called a direct estimate) to provide a more precise device effect estimate but also undertakes indirect comparisons of two different devices that have used the same comparator but have not been compared head-to-head.Cross-design synthesis involves combining effect estimates from randomized trials with strong internal validity with observational studies with strong external validity.All of these approaches are appropriate for determining estimates using distributed computing systems.

General approaches for individual-level data
When individual data are available, more flexibility in estimating device effects is possible, as well as a greater ability to assess the required statistical assumptions.For instance, when interest focuses on determining if patient characteristics modify device performance, individual-level data provide more power to identify the interaction than aggregated data.Assumptions about transitivity, consistency, validity, and selection bias are still required.Due to heterogeneity in the data sources, such as different clinical trials, different database registries, different countries, random effects for each data source are virtually always required.

Machine learning and artificial intelligence approaches
With the expansion and growing amount of RWD, researchers can utilize artificial intelligence approaches such as machine learning to better understand and, in turn, predict how patient, clinical, and device-related factors may influence decisions relating to procedures and relevant outcomes (17).Machinelearning-based models have several advantages over traditional regression-based models (18) and may thus generate robust predictive models that can predict when a risk of a particular outcome, such as late-stage mortality, is higher among patients treated with specific devices, including paclitaxel-coated devices.

Bayesian approaches
Bayesian techniques provide a natural way to integrate RWD and RCTs, allowing the incorporation of prior information and providing flexible yet interpretable models.Several strategies have been developed to incorporate such studies to provide robust predictive models (19).These include using the so-called power priors (20), commensurate priors (21), and the considered "gold standard" of hierarchical modeling (22,23).More recent techniques integrate patient-level information from observational studies and previous trials as synthetic or external controls, using confounding adjustment methods (24)(25)(26).Bayesian methods have the potential to provide a more comprehensive understanding of long-term mortality for paclitaxel-coated devices.

Additional methods
Data integration from different multinational studies implies the need for parsing the risks posed by various genetic and environmental factors that can affect underlying conditions, treatment choices, and ultimately treatment-related health outcomes.Many populations from the pivotal studies cited in Table 1 are expected to differ in demographics, including socioeconomic and other race/ethnicityrelated characteristics.Black race/ethnicity, for instance, has been associated with increased risk of peripheral arterial disease, an underlying condition for the use of DCB and DES (27).PAD has been associated with two SNPs-Single Nucleotide Polymorphisms (28) both of which demonstrate race/ethnicity-related differences in Frontiers in Cardiovascular Medicine their risk allele populational frequencies (e.g., Africans vs. Europeans/ Caucasians) (27,28).In general, genetic risk assessment of devicerelated adverse outcomes in patient subpopulations requires laborious efforts on biomarker discovery and validation, which are beyond the scope of the currently proposed research methodology.However, when partitioning the race/ethnicity-related risk factors in this endeavor, it is important to consider a complex, and in some cases opposing, interplay of genetic and environmental components, instead of anticipating a negative summation or potentiation of socioeconomic and genetic effects in the ethnic minority patients.

Combining evidence types
Due to a multitude of potential device-, drug-, and patientrelated factors contributing to the sum-effect, the presumed increase of all-cause mortality should be investigated using coherent linkage of multidisciplinary data that can transcend the disciplinary boundaries.The resultant interdisciplinary evidence is expected to move from bioengineering (device), pharmacological (drug), and epidemiological (patient) silos to promote the more comprehensive examination of potential synergistic effects that may remain undetected otherwise (Figure 1).
The original report on increased late mortality from paclitaxelcontaining devices in femoropopliteal applications suggested a combined role of drug-and patient-related factors (i.e., paclitaxel dose and peripheral arterial disease in the lower limbs as an underlying condition, respectively) (3).Although not all subsequent studies (11,29) confirmed the initial findings, the elusive risk increase was also attributed to patient-related factors such as the length of lesion as well as different comorbidities (30,31).With the actual causes still unknown, non-target paclitaxel embolization was indicated as a plausible mechanism (32).This suggests the need for more inclusive preclinical and clinical data analyses aimed at exploring the drug-, device-, and patient-attributable modifications of thromboembolism.While the drug-related risk component in thromboembolism may include paclitaxel effects on vascular homeostasis (32), the device-related risk component may involve thrombogenicity as a possible manifestation of inflammatory vascular tissue remodeling due to device/material bioreactivity (33).
Thus, while the siloed approaches may obscure the intersectional risk of increased mortality, which is likely limited to certain patient/device subgroups, the root-cause analysis employing interdisciplinary evidence can apportion the mortality risk more adequately and, most importantly, can minimize a potential failure to recognize the complex interplay of various risk modifiers.

Conclusion
The meta-analysis that sparked the regulatory action occurred 17 years after the first clinical trial assessing a paclitaxel-coated device was initiated.Even with the multitudes of available studies reviewed within the committee assembled by the FDA, it was agreed upon that additional data were needed to comprehensively assess the late-mortality signal.While several RWD sources exist and may help further assess the safety signal produced among paclitaxel-coated devices and their relevant outcomes among greater patient populations, each data source has limitations and varies in quality (34).Combining the myriad of clinical studies, available RWD, and additional evidence types may allow for a more comprehensive assessment of the safety signal produced by paclitaxel-coated devices across the product's lifecycle and the role of patient-, device-, and drug-related factors.The amalgamation of the identified high-quality data sources with sophisticated statistical methods will allow for the generation of real-world evidence needed to identify and confirm the safety signal promptly and accurately.Thus providing the FDA with the needed high-quality evidence to make relevant and correct regulatory decisions regarding the safety of paclitaxelcoated devices.
, IDDM, HTN, CAD, CHF, RD/D -Disease characteristics: Rutherford -Lesion characteristics: LL, CTO, RVD, HxIL, TASC -Medications: ATT, aspirin, P2Y12 receptor blockers, statins SAFE-, IDDM, HTN, CAD, CHF, RD/D, COPD -Disease characteristics: Rutherford, prior amputation -Lesion characteristics: LL, CTO, RVD, HxIL -Medications: ATT, aspirin, P2Y12 receptor blockers, It is continuously linked to Medicare data provided by the Centers for Medicare & Medicaid Services (CMS) claims, a nationally representative dataset of Medicareinsured individuals above the age of 65 covered by FFS Medicare (14).The registry is also continuously linked to state and city representative datasets, including the California and New York Statewide Planning and Research Cooperative System (SPARCS) dataset and the New York City Clinical Data Research Network (NYC-CDRN) dataset (15).

FIGURE 1
FIGURE 1Graphic illustrating the role of interdisciplinary evidence synthesis in the evaluation of treatments.

TABLE 1
Data sources for paclitaxel signal discernment.

TABLE 2 Quality
Assessment of data sources for paclitaxel signal Discernment.
medications by type, formulation, and dose, CPT codes are not granular and cannot identify which specific medical device was used.Despite the lack of granularity in the identification of the device, claims have the potential to follow individuals over a long period of time, allowing for the evaluation of long-term outcomes throughout the product's life cycle.Optum and FAIR Health Data are two administrative claims data repositories of over 20,000 patients within the US that follow patients for up to 48 months.The available data either captures all-cause mortality directly or relies on linkages with vital statistics to capture mortality.It is important to note that all-cause death and date of death may be missing for patients who are no longer captured by the dataset because they changed insurance plans or are no longer eligible for a specific type of insurance. identify