Usefulness of Bnet, a Simple Linear Metric in Discerning Torsades De Pointes Risks in 28 CiPA Drugs

The Comprehensive in vitro Proarrhythmia Assay (CiPA) project suggested the torsade metric score (TMS) which requires substantial computing resources as a useful biomarker to predict proarrhythmic risk from human ether-à-go-go–related gene (hERG) and a few other ion channel block data. The TMS was useful to predict low TdP risks of drugs blocking Na+ (ranolazine) and Ca2+ (verapamil) channels as well as the hERG channel. However, Mistry asserted that the simple linear metric, Bnet reflecting net blockade of a few influential ion channels has similar predictive power. Here we compared the predictability of Bnet and TMS for the 12 training and 16 validation CiPA drugs which were pre-classified into three categories according to the known TdP risks (low, intermediate, and high risk) by CiPA. Bnet at 5×Cmax (Bnet5×Cmax) was calculated using the ion-channel IC50 and Hill coefficients of CiPA drugs collected from previous reports by the CiPA team and others. The receiver operating characteristic curve area under curve (ROC AUC) values for TMS and Bnet5×Cmax as performance metrics in discerning low versus intermediate/high risk categories for the 28 CiPA drugs were similar. However, Bnet5×Cmax was much inferior to TMS at discerning between intermediate- and high-risk drugs. Dynamic Bnet, which used in silico hERG dynamic parameters unlike conventional Bnet, improved the misspecification. Thus, we propose that Bnet5×Cmax is used for quick screening of TdP risks of drug candidates and if the “intermediate/high” risk is predicted by Bnet5×Cmax, in silico approaches, such as dynamic Bnet or TMS, may be further considered.


INTRODUCTION
The International Council on Harmonization (ICH) established the guidelines, S7B for non-clinical evaluation and E14 for clinical evaluation of the proarrhythmic risk of drugs. As recommended by the guidelines, the conventional practice to evaluate the Torsades de Pointes (TdP) risks has been focused on the QTc interval from blockade of human ether-à-go-go-related gene (hERG) channel (Shah, 2005) that is associated with rapidly activating delayed rectifier potassium current I Kr . (Sager et al., 2014) Although ICH S7B and E14 regulatory guidelines have been successful in screening TdP risks of new drugs, there are several low TdP risk drugs with the prolonged action potential duration (APD) and QTc interval. Thus, the current practice according to ICH guidelines is sensitive but not specific enough to evaluate proarrhythmic (TdP) risks.
One of the major objectives of the Comprehensive in vitro Proarrhythmia Assay (CiPA) initiative was to improve the current ICH guidelines to avoid the misclassification of TdP risks by evaluating mechanistically based in vitro assays and in silico reconstruction of the cardiac action potential. The CiPA ion channel working group and in silico modeling group suggested qNet and the torsade metric score (TMS) as conclusive markers via the CiPAORdv1.0, the mechanistic in silico model (Li et al., 2019b) based on a series of modification of O'Hara-Rudy (ORd) human ventricular myocyte model (O'hara et al., 2011).
However, a few groups have raised questions on the superiority in accuracies of model-driven in silico approaches. (Mistry et al., 2015;Mistry, 2017;Parikh et al., 2017;Mistry, 2018;Parikh et al., 2019;Mistry, 2019a) Especially, Mistry asserted that Bnet, a simple linear metric using the net difference between inward and outward ion channel blocking, has predictive power similar to that of TMS. (Mistry, 2019a) Mistry questioned the usefulness of the complicated in silico approaches proposed by CiPA if the performance to assess the proarrhythmic risk is similar, although the CiPA researchers asserted the superiority of in silico approaches that consider the trapping of the hERG and other channels through rigorous validation of the model (Li et al., 2019b).
In this report, we compared the performance of Bnet and TMS in discerning TdP risks of the whole 28 CiPA drugs (12 training and 16 validation) to gain insight into potentials and limitations of in silico approaches by CiPA.

Channel Block Data to Calculate Bnet
The CiPA have chosen 12 training and 16 validation drugs which have been classified by a team of clinical cardiologists and electrophysiologists into three categories according to the known TdP risks (high, intermediate, and low risk) (Colatsky et al., 2016).
To compare the relationship between TMS and Bnet of the 28 drugs, we first collected the ion channel block data (IC50 and Hill coefficients by the drugs) that were used to estimate the TMS (Li et al., 2019b). They were used to calculate Bnet values. Because the CiPA aimed to automate the assays by using high-throughput patch-clamp systems (HTS) (Sager et al., 2014), hybrid patchclamp data collected using both manual (for hERG channel) and automated (other channels) methods were compared with data from the manual method for all channels. The performance of the hybrid and manual methods seemed equivalent (Li et al., 2019b). However, we picked the TMS values obtained from the manual method that has long been used as a standard in patch clamp studies.
In the many ion channels, only the four channels that were finally chosen by CiPA as significantly influencing the qNet and TMS: rapidly activating delayed rectifier potassium current (I Kr ), late sodium current (INaL), L-type calcium current (ICaL), and peak sodium current (INa) (Li et al., 2019b). Thus, for the calculation of Bnet, we used the IC50 and Hill coefficients for the four channels identical to those used to estimate qNet and TMS by CiPA. Those for the 12 training drugs were retrieved from the report by Crumb et al. (2016) that was utilized by Li et al. (2019b) In the case of the 16 validation drugs, the CiPA researchers did not use published data but have performed patch-clamp studies on their own. (Li et al., 2019b) Thus, we retrieved the IC50 and Hill coefficients for INaL, ICaL, and INa channels from the report by Li et al. However, the CiPA researchers did not use simple channel block, but employed a channel-trapping model in the case of the hERG channel and the IC50 or Hill coefficients for hERG channel for the 16 validation drugs were not mentioned in their report at all. (Li et al., 2019b). Thus, we had to search other published data ( Table 1) to replace those for I Kr (hERG) of the 16 validation drugs. The percentage block (%block) against a repolarization or depolarization ion-channel inputted into the Bnet 5×Cmax model was calculated using the mean maximal concentration observed (C max ) corrected for plasma protein binding (thus, unbound concentration) (Mistry, 2018).
Hill Hi max max l ll Bnet 5×Cmax was defined as the net difference in %block of the four most influential channels on the AP shape (%block of hERG channel − sum of %blocks of the other channels) at 5× C max (Supplementary Table 1).
where Ri and Dj represent the %block against repolarization (I Kr ) and depolarization (INaL, ICaL, and INa) ion-channels, respectively for a specific drug.
There are three major differences between the original Bnet (Mistry, 2018) and Bnet 5×Cmax . First, compared to the original Bnet proposed by Mistry, %block in our study (Bnet 5×Cmax ) took Hill coefficient into consideration. Second, the original Bnet proposed by Mistry did not include INa, we included it because the four channels have been selected to calculate TMS by CiPA researchers. Third, the original Bnet used values at 1× C max but Bnet 5×Cmax used values at 5× C max .

Calculation of Dynamic Bnet
We compared TMS and Bnet 5×Cmax with "dynamic Bnet" (Mistry, 2019a), which reflects hERG dynamics as TMS used. We utilized the publicly available data set that Mistry provided (Mistry, 2019b) and in the data set, hERG dynamics was included into Bnet by replacing the static hERG block with the dynamic hERG blocking using IC50 and maximal inhibition at the 1× C max .

Torsade Metric Score
The TMS, mean of qNet values at 1×, 2×, 3×, or 4× C max derived from the CiPAORdv1.0 model was digitized from a report by Li et al. (2019b). As mentioned in the previous section, only the TMS values from manually measured data were collected for comparison.

Ranking Performance Measures
Statistical analysis was performed using R Statistical Software version 3.6.0 (R Core Team, 2019). The ROC AUC (receiver operating characteristic curve area under the curve) (Zou et al., 2007) for TMS and Bnet 5×Cmax was calculated based on the known risk classifier. A logistic regression analysis using maximum likelihood estimation of the metric and the torsadogenic risk categories was performed by the rms R package (Harrell, 2019).

Risk Misspecification by TMS and Bnet 5×Cmax in the Validation Drug Data Set
The TMS and Bnet 5×Cmax of all 12 training drugs tested with Crumb's data (Crumb et al., 2016) were accordant to the risk categories (low vs. intermediate/high) (Figure 1). In the case of the 16 validation drugs, there were a few mismatches in categories both in the TMS ( Figure 1A) and Bnet 5×Cmax methods ( Figure 1B): tamoxifen and metoprolol (low-risk drugs) were located in the intermediate-risk cluster in the TMS and clarithromycin, domperidone, and risperidone (intermediate-risk drugs) were located in the low-risk cluster in the Bnet 5×Cmax . Dynamic Bnet using hERG dynamics decreased the misspecification but a drug of the validation data set, risperidone, still was misclassified ( Figure 1C).

Strong Correlation Between TMS and Bnet 5×Cmax in the Training Drug Data Set
Bnet 5×Cmax has shown performance similar to that of TMS as they are correlated with each other (r 2 = 0.663) (Figure 2). The correlation between Bnet 5×Cmax and TMS was stronger in the training drugs (r 2 = 0.867) than in the validation drugs (r 2 = 0.597), suggesting that the training drugs may possess better in vitro (patch-clamp study) data quality.

Performance Comparison: TMS and Bnet 5×Cmax
Both TMS and Bnet 5×Cmax seemed to discriminate low proarrhythmic risk drugs from intermediate/high-risk drugs quite well because the TMS and Bnet 5×Cmax values of low-risk drugs were significantly different (t-test, p < 0.005, and p < 0.001, respectively) from those of intermediate-and high-risk drugs (Supplementary Figure 1). The ranking performance measure for TMS and Bnet 5×Cmax was evaluated using ROC AUC (low vs. intermediate/high risk), and the values were 0.956 and 0.959, respectively (Table 2, Supplementary Figure 2

DISCUSSION
This is the first study to examine the performance of Bnet 5×Cmax , a simple metric calculated as the gap in blocking four representative channels by 28 CiPA drugs. We showed that Bnet 5×Cmax provided predictability comparable to the large-scale mechanistic model.
The therapeutic C max value directly affects the TMS and the Bnet metric. The TMS is calculated by averaging qNet values at 1×, 2×, 3×, or 4× C max and Bnet 5×Cmax is calculated based on %block at 5× C max . We have first screened Bnet for 1×, 5×, and 10× C max and the Bnet 5×Cmax showing the best performance was further used in our study (Supplementary Table 2).
Although the TMS and Bnet 5×Cmax of all 12 training drugs tested with Crumb's data (Crumb et al., 2016) were in the exact order of risk categories pre-defined by previous reports (Shah, 2005), the data based on the 16 validation drugs showed a few incorrect predictions both in TMS and Bnet 5×Cmax . This misspecification seems to have been caused by the patch-clamp experiments on the validation drugs not as qualified as in the 12 training drugs.
The reliability of patch-clamp experiment data is known to be highly variable by laboratories and skillfulness of the experimenter. Thus, measurement of IC50 and Hill coefficients using the patchclamp method performed by well-trained personnel appears critical for the appropriate assessment of both TMS and Bnet, regardless of using the silico method. The CiPA's attempt to estimate TMS with combined in vitro and in silico approaches is worthwhile in that the variability in multiple channel blocking is rigorously validated. Nonetheless, the performance of TMS is also dependent on the quality of patch-clamp experiment data for the ion channels that are input into the in silico simulation step.
The major limitation of Bnet is the inability to discriminate drugs with atypical binding kinetics (Li et al., 2019a). Because the hERG trapping observed in some drugs is not measured by the conventional ion channel blocking assay, CiPA has used the fIGURe 2 | Correlation between Bnet 5×Cmax and median TMS (torsade metric score, the average of qNet at 1-4× C max ) of (A) 12 training drugs, (B) 16 validation drugs, and (C) all 28 CiPA drugs. C max , peak plasma concentration. dynamic-hERG binding model for the data obtained using the Milnes protocol (Li et al., 2017). When the conventional IC50 is used to calculate Bnet for drugs that are significantly trapped in the hERG channel (e.g., dofetilide, bepridil, and terfenadine exemplified by Li et al. (2017)), their Bnet would be underestimated. However, the three drugs have shown Bnet values high enough to fall in the "intermediate/high" cluster in our study (Figure 1). Recently, Mistry calculated "dynamic Bnet" (Mistry, 2019a) using hERG dynamic IC50 which may partly reflect binding kinetics and showed the higher correlation of dynamic Bnet with TMS of the 28 CiPA drugs (r 2 = 0.86) than the conventional Bnet 5×Cmax (r 2 = 0.66) presented in Figure 2 in this report. The ROC AUC values of low-versus intermediate/high-risk for TMS, Bnet 5×Cmax , and dynamic Bnet were 0.956, 0.959, and 0.994, respectively. Although dynamic Bnet showed the best performance, it also requires the additional in silico approach and the time and resources spent to acquire the metric in the discovery or preclinical stage may still be substantial. The Bnet 5×Cmax can be a straightforward, accessible, and simple screening tool to discern the low-risk drugs.
The highest prediction performance of low/intermediate-risk versus high-risk drugs was observed in the TMS (ROC AUC = 0.99, Table 2). However, in the actual early development process, drug candidates with an intermediate risk often cannot survive to the next development step, and we believe that this limitation of poor discerning between intermediate and high risks may not affect go/no-go decision at the early stage in almost of therapeutic areas except for antiarrhythmics.
The Bnet 5×Cmax metric may be used as a simple screening biomarker in drug discovery and early development. We demonstrated that the Bnet 5×Cmax (or Bnet at concentrations regarded high enough when no Cmax data are available) provides initial information whether a candidate is at low proarrhythmic risk or not. For a candidate worthy of further development even with intermediate/high risk according to the Bnet 5×Cmax metric, in silico approaches proposed by CiPA or dynamic Bnet may be helpful.

AUTHOR'S NOTe
The abstract was submitted to Safety Pharmacology Society Anual Meeting 2019, held in Barcelona, Spain from September 23rd to 26th, 2019.

AUTHOR CONTRIBUTIONS
SeH, K-SK, H-AL, and D-SY contributed to the acquisition of data and provided experimental data guidance. SuH, SeH, and D-SY contributed to designing the work and carried out simulations. SuH, SeH, and D-SY contributed to the analysis and interpretation of the data and writing of the manuscript.