Factors Affecting Combination Trial Success (FACTS): Investigator Survey Results on Early-Phase Combination Trials

Experimental therapeutic oncology agents are often combined to circumvent tumor resistance to individual agents. However, most combination trials fail to demonstrate sufficient safety and efficacy to advance to a later phase. This study collected survey data on phase 1 combination therapy trials identified from ClinicalTrials.gov between January 1, 2003 and November 30, 2017 to assess trial design and the progress of combinations toward regulatory approval. Online surveys (N = 289, 23 questions total) were emailed to Principal Investigators (PIs) of early-phase National Cancer Institute and/or industry trials; 263 emails (91%) were received and 113 surveys completed (43%). Among phase 1 combination trials, 24.9% (95%CI: 15.3%, 34.4%) progressed to phase 2 or further; 18.7% (95%CI: 5.90%, 31.4%) progressed to phase 3 or regulatory approval; and 12.4% (95%CI: 0.00%, 25.5%) achieved regulatory approval. Observations of “clinical promise” in phase 1 combination studies were associated with higher rates of advancement past each milestone toward regulatory approval (cumulative OR = 11.9; p = 0.0002). Phase 1 combination study designs were concordant with Clinical Trial Design Task Force (CTD-TF) Recommendations 79.6% of the time (95%CI: 72.2%, 87.1%). Most discordances occurred where no plausible pharmacokinetic or pharmacodynamic interactions were expected. Investigator-defined “clinical promise” of a combination is associated with progress toward regulatory approval. Although concordance between study designs of phase 1 combination trials and CTD-TF Recommendations was relatively high, it may be beneficial to raise awareness about the best study design to use when no plausible pharmacokinetic or pharmacodynamic interactions are expected.

Experimental therapeutic oncology agents are often combined to circumvent tumor resistance to individual agents. However, most combination trials fail to demonstrate sufficient safety and efficacy to advance to a later phase. This study collected survey data on phase 1 combination therapy trials identified from ClinicalTrials.gov between January 1, 2003 and November 30, 2017 to assess trial design and the progress of combinations toward regulatory approval. Online surveys (N = 289, 23 questions total) were emailed to Principal Investigators (PIs) of early-phase National Cancer Institute and/or industry trials; 263 emails (91%) were received and 113 surveys completed (43%). Among phase 1 combination trials, 24.9% (95%CI: 15.3%, 34.4%) progressed to phase 2 or further; 18.7% (95%CI: 5.90%, 31.4%) progressed to phase 3 or regulatory approval; and 12.4% (95%CI: 0.00%, 25.5%) achieved regulatory approval. Observations of "clinical promise" in phase 1 combination studies were associated with higher rates of advancement past each milestone toward regulatory approval (cumulative OR = 11.9; p = 0.0002). Phase 1 combination study designs were concordant with Clinical Trial Design Task Force (CTD-TF) Recommendations 79.6% of the time (95%CI: 72.2%, 87.1%). Most discordances occurred where no plausible pharmacokinetic or pharmacodynamic interactions were expected. Investigator-defined "clinical promise" of a combination is associated with progress toward regulatory approval. Although concordance between study designs of phase 1 combination trials and CTD-TF Recommendations was relatively high, it may be beneficial to raise awareness about the best study design to use when no plausible pharmacokinetic or pharmacodynamic interactions are expected.
Keywords: clinical trials, combination therapy, regulatory approval, early phase, trial design, drug combinations, phase 1 trials INTRODUCTION Recent advances in genomic sequencing (1), molecular characterization of cancers and companion biomarkers (2), immune system knowledge (3), and other areas of research are uncovering new cancer therapies. These novel therapies are reshaping the field of cancer medicine and increasingly being evaluated in combination with other novel drugs as well as with approved treatments (4) in an effort to circumvent tumor resistance to individual agents, enhance synergy, and employ dual pathway inhibition. Combination trials now account for more than 25% of clinical trials in oncology, and trials supported by the National Institutes of Health (NIH) are significantly more likely to use drug combinations than those supported by industry (5). Trials involving combination of agents pose distinct challenges, including design of clinical trials that provide informative results, selection of agents with acceptable toxicity and improved efficacy and logistical and regulatory challenges. Phase I trials are the initial step in combination regimen clinical evaluation. This article presents an assessment of phase 1 combination trials in ClinicalTrials.gov between January 1, 2003 and November 30, 2017 to determine the proportion that achieved regulatory approval, and factors associated with success.
To date, most drug combination trials fail to demonstrate sufficient safety and efficacy to advance to later phases of development (6). The design and conduct of early-phase combination trials present specific challenges, such as determining which agents to combine, choosing an appropriate dose and schedule (including which agent to escalate), and addressing drug-drug interactions and overlapping toxicities (7,8). Furthermore, supportive measures that may effectively treat chemotherapy-related toxicities are insufficient in dealing with toxicities brought on by molecularly targeted agents, including rashes and elevated liver transaminases (9). Molecularly targeted agents usually require continuous dosing until disease progression, and thus are associated with toxicities that may not have been observed during the dose-limiting toxicity (DLT) assessment period, are more difficult to manage, and are exacerbated in combination therapy trials (9).
Given the increasing importance of combination regimens and the challenges associated with their development, the National Cancer Institute (NCI) Investigational Drug Steering Committee appointed a Clinical Trial Design Task Force (CTD-TF) to develop pragmatic clinical guidelines for the design of phase 1 combination clinical trials that were published in 2014 (10). The guidelines, shown in Figure 1 recommend investigators use a biologic or pharmacologic rationale supported by clinical, preclinical and/or other evidence to justify the combination, describe next steps in development of the combination and potential clinical results, and then take into account overlapping DLTs and potential pharmacodynamic (PD) and pharmacokinetic (PK) interactions in order to select the most effective trial design.
The Task Force members agreed on a set of five factors that need to be considered in early-phase combination clinical trial design: (i) therapeutic effect (11,12); (ii) mechanism of action and related PD markers (13)(14)(15); (iii) toxicity (e.g., non-overlapping dose limiting toxicities, chronic administration toxicity) (16); (iv) PK such as drug-drug interactions in which one drug may alter the metabolism of another and reduce or enhance its anticancer effect (17); and (v) dose schedule (e.g., low-dose continuous administration vs. high-dose intermittent administration) (18)(19)(20).
Publication bias results in limited data from negative studies available in journals, making literature reviews to characterize differences between positive and negative trials problematic (21). This study tests the hypothesis that a survey (Factors Affecting Combination Trial Success-FACTS) can be used to improve understanding of phase 1 trial design decisions. Specifically, the FACTS survey aimed to (i) assess proportions of combinations achieving each milestone toward regulatory approval, (ii) identify factors associated with these proportions, and (iii) assess the extent to which phase 1 trials were concordant with the CTD-TF guidelines. Because the CTD-TF guidelines were designed to help translational researchers improve the probability that a combination will advance toward regulatory approval, concordance with those guidelines may be a marker for predicting regulatory approval of the combination.
We relied on a survey for this work, despite the limitations of this approach, because theinformation we sought was only rarely included and nearly always incomplete in the manuscripts that were published and, of course, was not available at all when clinical trials were undertaken but no manuscript had been published. Thus, critically important patterns of clinical trial design decision-making was available only in the memories of the PIs. The Investigational Drug Steering Committee of NCI published new design guidelines in 2014 with the goal of improving the rate of success of early phase clinical trials in successfully move new treatments forward toward regulatory approval. We used the PI survey, then, to try to measure progress toward implementation of the new guidelines as a step toward accelerating adoption of those guidelines.

Participants
Survey participants were principal investigators (PIs) of earlyphase cancer treatment trials that evaluated combinations of experimental therapeutic agents. To identify PIs eligible for this study, we conducted a search of the ClinicalTrials.gov database in September 2015 to identify cancer intervention clinical trials listed as phase 1, 1b, or 1/2 that evaluated combinations of two or more therapeutic agents (N = 389) in both solid tumors and hematologic malignancies. The therapeutic agents included molecularly targeted agents, immune-oncology drugs, and antibody drug conjugates as well as chemotherapies. The list of participants was updated with additional queries to ClinicalTrials.gov through November 2017. Contact information was available for 289 trials led by 243 PIs (36 PIs were responsible for multiple trials, range 2-6.), a majority were Cancer Therapy Evaluation Program (CTEP) investigators from the Experimental Therapeutics Clinical Trials Network (ETCTN; n = 138) under Survey A 23-question online survey was developed to collect information on trial design decisions made by the PI and the progress the combination made toward regulatory approval. Three key content areas were assessed within the survey: (i) biomarker decisions (types of biomarkers in the study, whether clinical data was used for rationale, and the presence of primary/secondary biomarker objectives); (ii) phase 1 combination decisions (trial design type, preclinical factors supporting the combination, pre-defined criteria used to determine success/failure, expected interactions, and results of the phase 1 trial, including further investigation warranted, secondary endpoints met, and results published); and (iii) status of combination progression (current status of the phase 2/phase 3 of combination, results of the phase 2/phase 3 trial, whether the phase 2/phase 3 met secondary endpoints, whether the phase 2/phase 3 results were published, and whether regulatory approval of the combination was granted). Additional questions asked about whether the trial was investigator-initiated, the trial's funding source, and PI familiarity with the 2014 CTD-TF recommendations. In-depth phone interviews were conducted with five PIs prior to survey dissemination to review and revise the survey draft questions to ensure clarity and comprehension of the questions.

Milestone Achievements in Clinical Trial Development
The endpoint for this analysis was the number of clinical trial milestones each combination successfully achieved (i.e., further investigation beyond phase 1, further investigation beyond phase 2, positive phase 3 results, and regulatory approval; see Figure 2). Note that the investigation of some combinations was still in progress at the time of data acquisition (e.g., the phase 2 trial was positive, but the phase 3 trial was not yet initiated). For these combinations, the outcome is right-censored, as the highest milestone ultimately achieved was unknown, but was greater than or equal to the one achieved at data acquisition. This scenario was indicated by a "+" (e.g., if a phase 3 trial was ongoing, the endpoint was 2+).

Concordance Between CTD-TF Recommendations and Phase 1 Study Design
Concordance meant any of the following: • Overlapping DLTs or plausible PD leading to DLTs were expected and a formal phase 1 evaluation with pre-defined success criteria was used. • No overlapping DLTs and no plausible PD interactions were expected, but plausible PK interactions were, and a drug-drug interaction design with a PK primary endpoint was used. • No plausible PD or PK interactions were expected, and no formal phase 1 study was performed. Frontiers in Medicine | www.frontiersin.org

Procedure
An online survey platform was developed by Insilica Corporation at NCITrialPub.org to automatically generate email invitations to PIs and collect and manage the data. Survey links were created for each eligible trial (N = 289) and emailed to PIs from July-December 2017 in batches of 30 every week by the Emmes Corporation, an NCI contractor. Emails were re-sent to nonresponders after 10 business days for a maximum of 5 reminders.

STATISTICAL METHODOLOGY
Maximum likelihood estimation was used to estimate the probabilities of achieving each milestone; the likelihood function and how right-censoring was handled are detailed in the Supplementary Methods. Likelihood ratio tests were used to assess the associations between individual study characteristics and the probabilities of achieving each milestone, with the Benjamini-Hochberg procedure (22) used to adjust for multiple testing. Multivariate models of these probabilities given the study characteristics were constructed using logistic regression subject to Elastic Net constraints (see Supplementary Methods).
The proportion of combinations in which the phase 1 trial study design and CTD-TF Recommendations were concordant was estimated with 95% confidence intervals. A chi-square test was used to assess whether the expected interactions and DLTs were independent of the study design used. A Mann-Whitney U Test was used to assess the association between familiarity of the PI with CTD-TF Recommendations and concordance of the phase 1 trial study design with CTD-TF Recommendations.

RESULTS
The survey was dispatched to 289 PIs between July and December 2017. Delivery was successful for 263 surveys, and valid responses were received for 113 (39%) trials (Figure 3). A data verification in which two coders reviewed the literature on 10% of the combinations and answered the survey questions independently showed 99% agreement between the publications and the survey responses.

Probabilities of Advancement Past Each Milestone Toward Regulatory Approval
Of the combinations, 39.8% (45/113; 95% CI: 30.8%, 48.8%) advanced beyond phase 1. The estimate for the proportion advancing beyond phase 2 was 24.9% (95% CI: 15.3%, 34.4%), and 15 of the 113 combinations in the data had achieved this milestone by the time of data acquisition. Note that the estimate of the proportion advancing beyond a milestone may not be equal to the proportion of combinations in the data achieving the milestone at the time of data acquisition due to the rightcensoring. The former takes into account that combinations may achieve additional milestones in the future.
The estimate for the proportion for which the phase 3 trial was positive was 18.7% (95% CI: 5.90%, 31.4%); three of the 113 combinations were associated with a positive phase 3 trial by the    Figure 1).
Frontiers in Medicine | www.frontiersin.org  time of data acquisition. The estimate of the proportion achieving regulatory approval was 12.4% (95% CI: 0.00%, 25.5%), with two of the 113 combinations achieving regulatory approval by the time of data acquisition. These results are shown in Table 1. Table 2 compares expected DLTs and PK and PD interactions with the type of phase 1 study design used, and shows concordance between CTD-TF Recommendations and phase 1 study design was observed in 79.6% of the interactions (90 out of 113; 95% CI: 72.2%, 87.1%). However, formal phase 1 designs with pre-determined success criteria were used in 110 of the 113 surveyed trials, including in all 20 trials in which the CTD-TF would not have recommended using this design. The p-value of the test of independence between expected DLTs and PK and PD interactions vs. the type of phase 1 study used was 0.956. Investigators whose designs were in concordance with the CTD-TF Recommendations reported greater familiarity with the guidelines than PIs in non-concordant studies (19% vs. 3% reporting they were "very familiar") ( Table 3). The Mann-Whitney U Test of the degree of familiarity with CTD-TF Recommendations vs. concordance of phase 1 study designs with CTD-TF Recommendations indicated little evidence of association between these two variables (p = 0.304). Because 108 out of 113 of the surveyed trials were submitted to ClinicalTrials.gov before the Recommendations were published in August 2014 (10), low concordance was to be expected. However, the data in this study provide a baseline against which to measure improvement in concordance over time.

Phase 1 Study Characteristics Associated With Advancement Toward Regulatory Approval
Summary statistics for the phase 1 combination survey results are provided in Table 4. At the α = 0.05 level, the data provided evidence of a significant association between clinical promise in the phase 1 trial (i.e., evidence of sufficient activity, e.g., decrease in tumor size or FDG uptake or prolonged progression-free or overall survival, at tolerable levels of toxicity to move forward with a registration-directed investigation) and advancement toward regulatory approval (p = 0.0002). Clinical promise was associated with higher probabilities of achieving each of the milestones toward regulatory approval ( Table 5) trials that did not explicitly require a demonstration of evidence still showed clinical promise 25% of the time, and 68% of these trials still advanced past phase 1 when clinical promise was observed. Table 6 lists regression coefficient estimates for a multivariate model of progression toward regulatory approval given study characteristics. The regression coefficient estimates associated with observed clinical promise in the phase 1 study and inclusion of phase 1 biomarker-driven objectives were linked to higher probabilities of progressing past each clinical trial milestone. The regression coefficient estimates associated with the following characteristics were negative, indicating that they were linked to lower probabilities of progressing past each milestone: the rationale for the combination study was based on in vitro evidence of activity; results other than establishing safe, tolerable, or optimal doses, determining the sequence of drug administration, or observing pharmacokinetic or pharmacodynamic effects or clinical promise in the phase 1 trial; PK interactions were expected; PK was a predefined criterion for the success of the phase 1 study; and overlapping dose-limiting toxicities were expected of the combination.

DISCUSSION
Observing clinical promise of a combination (e.g., sufficient activity at tolerable levels of toxicity to warrant moving forward with registration-based investigation) in a phase 1 trial is associated with progress toward regulatory approval. However, nearly one-quarter of phase I trials that did not report clinical promise from phase 1 still moved into a phase 2 study. We estimate that 12% of all combinations will ultimately achieve regulatory approval: While only 5% of combinations that do not report clinically promising results in phase 1 achieve regulatory approval, 40% of combinations that do report clinically promising phase 1 results achieve regulatory approval. Only 47% of surveyed trials referenced clinical promise as a requirement of success, but clinical promise was nonetheless observed in 25% of the trials that did not require the observation.
In trials lacking the clinical promise requirement, clinical promise was still strongly associated with phase 1 success. The data may indicate that clinical promise should be closely examined in phase 1 trials especially given that clinical promise was observed in ∼50% of the trials that require it, but only observed 25% of the time in trials that did not require it and strongly associated with success in both cases (64-68% of trials where clinical promise was observed past phase 1). Further, these data may suggest investigators consider foregoing phase 2 studies for combinations that show little phase 1 clinical promise. Although concordance of phase 1 designs with the CTD-TF Recommendations occurred in 79.6% of the trials, formal phase 1 designs were used in 97% of trials, including in all 20 cases (18%) in which the CTD-TF would not have recommended this design. Thus, a large proportion of investigators employ formal phase 1 designs even when expected interactions indicate that formal phase 1 designs are not ideal (p-value of test of independence of expected interactions and design: 0.956). This high level of concordance between phase 1 designs and CTD-TF Recommendations occurred despite more than 95% of the trials being submitted to ClinicalTrials.gov before the Recommendations were published. Follow-up with trials designed after the Recommendations were published will be needed to determine the impact of the Recommendations on improving factors toward success. Because greater familiarity was associated with concordance with the CTD-TF guidelines, additional benefit may be gained by raising awareness of the best study design to use when no plausible pharmacokinetic or pharmacodynamic interactions are expected.
Even with a sample size of 113, evidence of signal was found in some trial design characteristics with regard to advancement toward regulatory approval. Observation of phase 1 clinical promise to move forward with registration-directed investigation was significantly associated with advancement past each milestone toward regulatory approval. In addition, evidence of association with advancement toward regulatory approval were observed for (i) biomarker-driven objectives included in phase 1 design, (ii) assessment of therapeutic pharmacokinetic levels in phase 1, and (iii) findings from phase 1 trials other than establishment of safe, tolerable, or optimal doses, determining the sequence of drug administration, or observation of pharmacokinetic or pharmacodynamic effects or clinical promise. These associations are consistent with reported causes of failure of oncology drugs in late-stage clinical development that demonstrated lack of a biomarker-driven strategy and failure to attain proof of concept (23). In addition, observation of any pharmacodynamic or pharmacokinetic interactions was associated with lower probabilities of achieving all subsequent milestones toward regulatory approval, a finding consistent with reports of overlapping toxicities as significant contributors to the failure of drug combinations to reach regulatory approval (4).
Extending this survey to more combinations, including those not evaluated in CTEP-sponsored trials, will not only improve power to detect associations between these design characteristics and advancement toward regulatory approval, but also allow development and evaluation of a predictor of whether a combination will achieve each of these milestones based on one or more of these trial characteristics. Such a predictor may help inform investigators and funding sources in determining which combinations to include in phase 1 trial design.
One major limitation of this study is that we excluded chemo-radiation combinations. Our rationale was that we were seeking consistency of endpoints. We anticipate followon research will include combinations with radiation. The potential for bias based on only 39% response rate is another limitation as is the potential for recall bias. Although we are asking investigators to report information going back many years, the investigators played pivotal roles in the design, execution, and manuscript preparation, so their recall may be stronger than respondents without that close association. In addition, our survey instrument linked the responding investigator to that investigator's publication on the trial to help them with accuracy of recall. Our initial focus on CTEP-funded trials provided consistent, complete information that provides a strong launch of the FACTS program. A more inclusive database of combination trials, with regular progress updates toward regulatory approval and additional curation of structured data on clinical trials, may help to automatically identify promising clinical trials and/or alert practitioners of potential problems in their trial design.