<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Archiving and Interchange DTD v2.3 20070202//EN" "archivearticle.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="methods-article" dtd-version="2.3">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Pharmacol.</journal-id>
<journal-title>Frontiers in Pharmacology</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Pharmacol.</abbrev-journal-title>
<issn pub-type="epub">1663-9812</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fphar.2019.01586</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Pharmacology</subject>
<subj-group>
<subject>Methods</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Deep Learning Based Drug Metabolites Prediction</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname>Wang</surname>
<given-names>Disha</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/883618"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Liu</surname>
<given-names>Wenjun</given-names>
</name>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Shen</surname>
<given-names>Zihao</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/840014"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Jiang</surname>
<given-names>Lei</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/883616/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Wang</surname>
<given-names>Jie</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/818104"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Li</surname>
<given-names>Shiliang</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/473776"/>
<xref ref-type="author-notes" rid="fn001">
<sup>*</sup>
</xref>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Li</surname>
<given-names>Honglin</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/421355"/>
<xref ref-type="author-notes" rid="fn001">
<sup>*</sup>
</xref>
</contrib>
</contrib-group>
<aff id="aff1">
<sup>1</sup>
<institution>Shanghai Key Laboratory of New Drug Design, State Key Laboratory of Bioreactor Engineering, School of Pharmacy, East China University of Science and Technology</institution>, <addr-line>Shanghai</addr-line>, <country>China</country>
</aff>
<aff id="aff2">
<sup>2</sup>
<institution>Research and Development Department, Jiangzhong Pharmaceutical Co., Ltd.</institution>, <addr-line>Nanchang</addr-line>, <country>China</country>
</aff>
<author-notes>
<fn fn-type="edited-by">
<p>Edited by: Alex Zhavoronkov, Biogerontology Research Foundation, United Kingdom</p>
</fn>
<fn fn-type="edited-by">
<p>Reviewed by: Cao Dongsheng, Central South University, China; Mingyue Zheng, Chinese Academy of Sciences, China</p>
</fn>
<fn fn-type="corresp" id="fn001">
<p>*Correspondence: Shiliang Li, <email xlink:href="mailto:slli403@163.com">slli403@163.com</email>; Honglin Li, <email xlink:href="mailto:hlli@ecust.edu.cn">hlli@ecust.edu.cn</email>
</p>
</fn>
<fn fn-type="other" id="fn002">
<p>This article was submitted to Experimental Pharmacology and Drug Discovery, a section of the journal Frontiers in Pharmacology</p>
</fn>
</author-notes>
<pub-date pub-type="epub">
<day>30</day>
<month>01</month>
<year>2020</year>
</pub-date>
<pub-date pub-type="collection">
<year>2019</year>
</pub-date>
<volume>10</volume>
<elocation-id>1586</elocation-id>
<history>
<date date-type="received">
<day>03</day>
<month>10</month>
<year>2019</year>
</date>
<date date-type="accepted">
<day>09</day>
<month>12</month>
<year>2019</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#xa9; 2020 Wang, Liu, Shen, Jiang, Wang, Li and Li</copyright-statement>
<copyright-year>2020</copyright-year>
<copyright-holder>Wang, Liu, Shen, Jiang, Wang, Li and Li</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>Drug metabolism research plays a key role in the discovery and development of drugs. Based on the discovery of drug metabolites, new chemical entities can be identified and potential safety hazards caused by reactive or toxic metabolites can be minimized. Nowadays, computational methods are usually complementary tools for experiments. However, current metabolites prediction methods tend to have high false positive rates with low accuracy and are usually only used for specific enzyme systems. In order to overcome this difficulty, a method was developed in this paper by first establishing a database with broad coverage of SMARTS-coded metabolic reaction rule, and then extracting the molecular fingerprints of compounds to construct a classification model based on deep learning algorithms. The metabolic reaction rule database we built can supplement chemically reasonable negative reaction examples. Based on deep learning algorithms, the model could determine which reaction types are more likely to occur than the others. In the test set, our method can achieve the accuracy of 70% (Top-10), which is significantly higher than that of random guess and the rule-based method SyGMa. The results demonstrated that our method has a certain predictive ability and application value.</p>
</abstract>
<kwd-group>
<kwd>deep learning</kwd>
<kwd>drug metabolism</kwd>
<kwd>metabolites prediction</kwd>
<kwd>reaction rules</kwd>
<kwd>SMARTS</kwd>
</kwd-group>
<counts>
<fig-count count="8"/>
<table-count count="4"/>
<equation-count count="0"/>
<ref-count count="42"/>
<page-count count="10"/>
<word-count count="3710"/>
</counts>
</article-meta>
</front>
<body>
<sec id="s1" sec-type="intro">
<title>Introduction</title>
<p>The discovery of small molecule drugs is time-consuming, expensive and labor-intensive. (<xref ref-type="bibr" rid="B8">Dickson and Gagnon, 2004</xref>; <xref ref-type="bibr" rid="B32">Paul et al., 2010</xref>; <xref ref-type="bibr" rid="B9">Dimasi et al., 2015</xref>) It is resource intensive, and involves typical timelines of 10&#x2013;20 years and costs that range from US$0.5 billion to US$2.6 billion (<xref ref-type="bibr" rid="B32">Paul et al., 2010</xref>; <xref ref-type="bibr" rid="B1">Avorn, 2015</xref>). In addition to economic and technical reasons, the main reason is that almost half of the candidate drugs failed in clinical trials. Up to 25% of compounds were withdrawn due to metabolic, pharmacokinetic, or toxic problems (<xref ref-type="bibr" rid="B19">Hwang et al., 2016</xref>). Drug metabolism can produce metabolites with physicochemical and pharmacological properties, which are significantly different from the physical and pharmacological properties of parent drugs (<xref ref-type="bibr" rid="B22">Kirchmair et al., 2013</xref>). As <xref ref-type="fig" rid="f1">
<bold>Figure 1</bold>
</xref> shows, when drugs or other exogenous substances enter the human body, they are largely controlled by three stages of drug metabolism. In the first stage, reactive groups are introduced by oxidation, reduction, or hydrolysis. In the second stage, conjugation reactions with macromolecules occur <italic>in vivo</italic>. In the third stage, allogeneic and metabolites are removed from liver and intestinal cells. After these three stages, exogenous substances such as drugs may be transformed into non-physiological active substances or toxic metabolites. 70% clinical drugs are removed by the body&#x2019;s metabolic system, so as part of drug development, it is also necessary to conduct in-depth research on drug metabolism (<xref ref-type="bibr" rid="B17">Grant et al., 2001</xref>; <xref ref-type="bibr" rid="B13">Embrechts and Ekins, 2007</xref>; <xref ref-type="bibr" rid="B28">Lazar and Birnbaum, 2012</xref>; <xref ref-type="bibr" rid="B4">Damsky and Bosenberg, 2012</xref>; <xref ref-type="bibr" rid="B6">Di, 2014</xref>; <xref ref-type="bibr" rid="B30">Mackenzie et al., 2017</xref>; <xref ref-type="bibr" rid="B21">Kang et al., 2018</xref>). Understanding the metabolic process of drugs is essential for successful drug discovery and development, and helps to optimize the stability of drugs, so as to optimize the half-life <italic>in vivo</italic>.</p>
<fig id="f1" position="float">
<label>Figure 1</label>
<caption>
<p>General pathway of drug metabolism.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g001.tif"/>
</fig>
<p>In order to reduce the risk caused by metabolic characteristics of candidate drugs, effective andreliable methods are needed to predict drug metabolism <italic>in vitro</italic>. Many experimental methods can be used to explore the metabolic process of drugs (<xref ref-type="bibr" rid="B7">Diao et al., 2016</xref>; <xref ref-type="bibr" rid="B30">Mackenzie et al., 2017</xref>). For example, fast LC-MS scans can be carried out to specifically detect predicted metabolites. However, experimental methods are still highly demanding in terms of equipment, expertise, cost, and time (<xref ref-type="bibr" rid="B22">Kirchmair et al., 2013</xref>). Therefore, it has great prospects to develop computational tools for predicting drug metabolism with lower cost and higher throughput than existing tools. Many different methodologies to predict metabolites or sites of metabolism have been reported recently. Various methods in predicting drug metabolism using in silico approaches have been reviewed (<xref ref-type="bibr" rid="B14">Fox and Kriegl, 2006</xref>; <xref ref-type="bibr" rid="B16">Gleeson et al., 2011</xref>; <xref ref-type="bibr" rid="B42">Zhang et al., 2011</xref>; <xref ref-type="bibr" rid="B40">Tan et al., 2017</xref>). However, most of these methods are limited to P450 catalytic reactions and represent only unstable sites, rather than predicting the actual metabolites formed.</p>
<p>Metabolic sites (SOMs) and metabolite structure are two main research directions of computer-aided metabolic prediction methods, which can provide decisive support and guidance for experimenters. SOM prediction methods usually have high prediction accuracy. The program MetaSite estimate the possibility of metabolic reactions at an atomic site using protein structure information, GRID-derived MIFs of protein, and ligand and molecular orbital calculations (<xref ref-type="bibr" rid="B15">Gabriele et al., 2005</xref>). The program SMARTCyp contains a pre-calculated energy reaction analysis table for density functional theory activation, where a large number of ligand fragments pass through CYP3A4 or CYP2D6 mediated transformation (<xref ref-type="bibr" rid="B34">Rydberg et al., 2010a</xref>; <xref ref-type="bibr" rid="B35">Rydberg et al., 2010b</xref>). A method called cypscore, in which 2400 CYP-mediated transformations and 850 literature compounds are used as data bases (<xref ref-type="bibr" rid="B18">Hennemann et al., 2010</xref>). However, most of these methods are limited to CYP450 catalytic reactions and can only predict unstable sites rather than metabolite structures. Furthermore, predicted SOMs are not identical to identifying the correct bioinformations that will occur at an atomic location, and they do not provide information about the type of reaction that will occur. Therefore, these limitations make it difficult to draw any quantitative conclusions about the metabolic possibilities of a molecule.</p>
<p>So far, only a few computational methods have been developed for predicting the structure ofmetabolites. Existing methods can be divided into two categories: expert rule-based anddescriptor-based. Rule-based approaches use data mining techniques. Large databases with data onmetabolism are used to extract generalized rules to determine the part of a molecule that undergoes metabolic alteration (<xref ref-type="bibr" rid="B3">Cariello et al., 2002</xref>). The ligand-based approach relies on the assumption that the metabolic fate of compounds is entirely determined by their chemical structure and properties. These methods include quantum mechanics methods. Descriptor-based methods to obtain an idea of the route of a drug through the metabolic system, the identification of the involved enzymes, and the reaction pathways is necessary (<xref ref-type="bibr" rid="B29">Livingstone, 2010</xref>). The program of Bioprint contains a database of most marketed drugs together with reference compounds and data from a wide variety of biological and <italic>in-vitro</italic> ADME assays, called the Biological fingerprint (<xref ref-type="bibr" rid="B26">Krejsa et al., 2003</xref>). Thus, the possible results of new compounds can be calculated by neighborhood relation and QSAR model. In the MetaDrug database (<xref ref-type="bibr" rid="B12">Ekins et al., 2006</xref>; <xref ref-type="bibr" rid="B11">Ekins et al., 2005b</xref>), metabolic reactions with substrates (including primary and secondary metabolites), xenobiotic reactions, and kinetic data on enzyme inhibition are stored. 317 molecules (parent drug and primary and secondary metabolite) were randomly selected from this database to build kernel-partial least squares models for metabolism rules (<xref ref-type="bibr" rid="B13">Embrechts and Ekins, 2007</xref>). Metabolite prediction is usually accomplished through a large set of transformation rules. Given the reactant, all rules are then matched to determine the site of metabolic instability. Expert systems such as METEOR (<xref ref-type="bibr" rid="B41">Testa et al., 2010</xref>; <xref ref-type="bibr" rid="B2">Button et al., 2015</xref>), META (<xref ref-type="bibr" rid="B23">Klopman et al., 1994</xref>; <xref ref-type="bibr" rid="B39">Talafous et al., 1994</xref>; <xref ref-type="bibr" rid="B24">Klopman et al., 1997</xref>), MetabolExpert (<xref ref-type="bibr" rid="B5">Darvas, 1988</xref>), RD-Metabolizer (<xref ref-type="bibr" rid="B31">Meng et al., 2017</xref>), MetaDrug (<xref ref-type="bibr" rid="B25">Korolev et al., 2003</xref>; <xref ref-type="bibr" rid="B10">Ekins et al., 2005a</xref>), and KnowItAll (<xref ref-type="bibr" rid="B38">Stouch et al., 2003</xref>) are based on these databases and provide a ranked list of most likely metabolites. In a study described by AstraZeneca (<xref ref-type="bibr" rid="B36">Scott et al., 2007</xref>), the substrates and reaction centers of the metabolite database were stored as fingerprints in two databases. Then the query molecule powders are compared with the two databases, and the proposed SOM is ranked by using the number of clicks as a weighted scheme. An approach called SyGMa based on the MDL metabolite database was developed (<xref ref-type="bibr" rid="B33">Ridder and Wagener, 2010</xref>). According to the corresponding rules of MDL metabolite data coding, the structure of possible metabolites is predicted, and probability scores are assigned to each metabolite, covering 70% of all known human metabolic reactions.</p>
<p>So far, one of the difficulties in predicting possible metabolites is that this task means identifying the reaction site (SOM) and the type of metabolic reaction correctly. Current methods for predicting metabolite structure tend to have high false-positive rates and can only be used for specific enzymes without covering all the metabolic enzymes involved in human reactions. In view of the above problems, we mainly designed a deep learning algorithm combined with drug metabolism characteristics.</p>
<p>In this work, by combining metabolic reaction template and Deep Learning, we have established a model to predict the main metabolites of drugs (<xref ref-type="fig" rid="f2">
<bold>Figure 2</bold>
</xref>). Our method has the following innovations: (1) Data enhancement strategy, which provides chemically reliable examples of negative reactions through the metabolic reaction template library; (2) the implementation and validation of a neural network-based model, which can obtain that some reaction modes are more or less likely to occur than other potential modes.</p>
<fig id="f2" position="float">
<label>Figure 2</label>
<caption>
<p>Metabolic reaction product prediction flow chart.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g002.tif"/>
</fig>
</sec>
<sec id="s2" sec-type="materials|methods">
<title>Materials and Methods</title>
<sec id="s2_1">
<title>Data Collection and Processing</title>
<p>We collected metabolic reaction data from MDL Database (2011 edition). Here we used only human metabolic reactions with effective substrates and metabolites. The data were filtered to remove unreasonable structures, such as reactants and products containing R groups, free radicals, metal chelating, and structural errors, which could make it impossible to distinguish the reaction records of reaction sites. The pretreatment resulted in 7,380 reaction records, of which 74 reactions had only chiral changes. We randomly selected 300 response records from them as standby for external test sets.</p>
</sec>
<sec id="s2_2">
<title>Generation of Metabolic Reactions Template Library</title>
<p>The process of constructing the metabolic reactions template library is shown in <xref ref-type="fig" rid="f3">
<bold>Figure 3</bold>
</xref>. At present, the methods based on expert system mainly use the general metabolic rules deduced by experts to predict the structure of metabolites. However, this method has some drawbacks. The model needs to understand the influence of coding reaction functional groups. Such rules can not completely produce the desired response because the complete background of molecules is ignored. The remaining Non-coding functional groups of the molecule may affect or react competitively. So maybe even if the rules are matched, the ideal reaction product cannot be produced. Therefore, reaction rules need to be annotated with relevant information, such as functional groups, priority of reactions. However artificial code rules are time-consuming, laborious, and lack of internal ranking mechanism. Based on this, expert rules cannot be implemented on a large scale. Marwin H.S. (<xref ref-type="bibr" rid="B27">Law et al., 2009</xref>; <xref ref-type="bibr" rid="B37">Segler et al., 2018</xref>) has proved that in predicting the products of inverse synthesis reaction, the artificial extraction of reaction rules based on expert rules is far less effective than the algorithm which automatically judges the type of reaction according to the reactants and products in the deep learning model training. In order to construct a database of metabolic templates, we also adopted the heuristic driving algorithm of Law (<xref ref-type="bibr" rid="B27">Law et al., 2009</xref>).</p>
<fig id="f3" position="float">
<label>Figure 3</label>
<caption>
<p>The generation of metabolic reactions template.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g003.tif"/>
</fig>
<p>
<xref ref-type="table" rid="T1">
<bold>Table 1</bold>
</xref> shows the most common reaction types in the database. It can be seen that the most common metabolic reactions are amide hydrolysis, carboxylic acid hydrolysis, and hydroxylation of N, O, S atoms.</p>
<table-wrap id="T1" position="float">
<label>Table 1</label>
<caption>
<p>The most common type of reactions and SMART fragments in the dataset.</p>
</caption>
<table frame="hsides">
<thead>
<tr>
<th valign="top">Template SMART</th>
<th valign="top" align="center">Example</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top">O = C-[NH;+0:1]-[C:2]&gt; &gt; [C:2][NH2;+0:1]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g009.tif"/>
</td>
</tr>
<tr>
<td valign="top">O = C-[NH;+0:1]-[c:2]&gt; &gt; [NH2;+0:1]-[c:2]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g010.tif"/>
</td>
</tr>
<tr>
<td valign="top">O = C-[O;H0;+0:1]-[C:2]&gt; &gt; [C:2]-[OH;+0:1]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g011.tif"/>
</td>
</tr>
<tr>
<td valign="top">[C]-[O;H0;+0:1]-[C:2] = [O:3]&gt; &gt; [O:3] = [C:2]-[OH;+0:1]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g012.tif"/>
</td>
</tr>
<tr>
<td valign="top">[C:1]-[N;H0;+0:2](-[C:3])-[C:4]&gt; &gt; [C:1]-[N+;H0:2](-[C:3])(-[C:4])</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g013.tif"/>
</td>
</tr>
<tr>
<td valign="top">[c:1]-[S;H0;+0:2] [c:3]&gt; &gt; O = [S;H0;+0:2](-[c:1])-[c:3]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g014.tif"/>
</td>
</tr>
<tr>
<td valign="top">[C:1]-[CH2;+0:2]-[C:3]&gt; &gt; O-[CH;+0:2](-[C:1])-[C:3]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g015.tif"/>
</td>
</tr>
<tr>
<td valign="top">[c:1]:[n;H0;+0:2]:[c:3]&gt; &gt; [O-]-[n+;H0:2](:[c:1]):[c:3]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g016.tif"/>
</td>
</tr>
<tr>
<td valign="top">[c:1]:[cH;+0:2]:[c:3]&gt; &gt; O-[c;H0;+0:2](:[c:1]):[c:3]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g017.tif"/>
</td>
</tr>
<tr>
<td valign="top">[C:1]-[S;H0;+0:2]-[C:3]&gt; &gt; O = [S;H0;+0:2](-[C:1])-[C:3]</td>
<td valign="top" align="right">
<inline-graphic xlink:href="fphar-10-01586-g018.tif"/>
</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s2_3">
<title>Producing Candidate Metabolites</title>
<p>The above-mentioned metabolic reaction templates are stored in the database for subsequent production of positive and negative potential metabolites. For each atom mapping reaction in the dataset, the reaction center is defined by determining which product atoms are different from the corresponding reactant atoms. The reaction center is expanded to include the surrounding environment, and then other factors that play a role in the reaction is found out. Adjacent atoms are defined as non-hydrogen substituents, where high coverage is achieved at the expense of low specificity. Metabolic Templates are defined with SMART format strings encoding reaction centers. The reaction template generated in this way does not depend on manual extraction, marking, or sorting.</p>
<p>As shown in <xref ref-type="fig" rid="f4">
<bold>Figure 4</bold>
</xref>, we can match the reaction template of the metabolic database one by one and produce a large number of potential metabolic reaction products by using RDKit. Positive compounds are the products recorded in the database, and the rest are all negative products. This strategy can continuously produce negative products for later use.</p>
<fig id="f4" position="float">
<label>Figure 4</label>
<caption>
<p>Flow chart for potential metabolites production.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g004.tif"/>
</fig>
</sec>
<sec id="s2_4">
<title>Model Training</title>
<p>For deep learning and supervised learning, we need to input eigenvalues. What we need to consider is using what molecular descriptors to characterize the whole metabolic process. Here we choose molecular fingerprints to describe the atomic and functional characteristics of metabolic reactions. The abstract representation of molecular fingerprints, which encodes molecular transformation into a series of vectors, makes it easier for molecules to compare with each other. If two molecules are similar, there must be many common fragments between them. Then molecules with similar fingerprints will have a high probability of being similar in 2D structures. Here we use RDKit to generate 1,024-dimensional Morgan molecular fingerprints. Molecular fingerprint ECFP is suitable for machine learning because it contains more molecular structure details. The metabolites generated above through the metabolic reaction template will be scored separately by Deep Learning model. Here we use Python library of Keras. The input layer consists of molecular fingerprints of products and reactants, with a total of 2,048 dimensions. One reaction corresponds to multiple potential metabolites. Thus, the input layer generates a matrix of 2,048 dimensions with <italic>n</italic> vectors. We use keras wrapper to realize each vector, that is, each individual potential metabolite is fully connected independently, which increases the ability of the model to achieve one-to-many and many-to-many. The probability of all potential reaction products is finally mapped out by the output layer activation function softmax, so that the total probability of all potential metabolites is 1. According to the score of the output layer, it is most likely to describe which metabolites actually exist. Deep neural network models are trained here to solve problems similar to classification problems: given hundreds of possible classes (potential metabolites), predicting real classes (recording reaction products), each metabolic reaction may correspond to multiple classifications. We use cross-entropy as the loss function during training. This objective function can be understood as the negative logarithm of probability allocated to the true class (true metabolites). During the training period, we use five-fold cross-validation to divide all the data sets into five parts, one of them is taken as the validation set without repetition, and the other four are used as training model of training set. Cross-validation can avoid over-fitting and under-fitting, and the final results are more convincing.</p>
</sec>
</sec>
<sec id="s3" sec-type="results">
<title>Results and Discussion</title>
<sec id="s3_1">
<title>Accuracy of Prediction Results</title>
<p>Following the above steps, we cross-validated the model with five folds by using 200 epochs. The training set, validation set, test set segmentation was 7:1:2. The objective of the training period is to minimize the cross-entropy loss of classification, which is the natural logarithm of probability allocated to real metabolites. Considering that there may be more than one metabolic reaction product for a drug, we believe that the top ten predicted products may have more reference value. As shown in <xref ref-type="table" rid="T2">
<bold>Table 2</bold>
</xref>, the model achieves an average test set accuracy of 70% for Top-10 in the five-fold cross-validation. In addition, we also calculated the accuracy of Top-1, Top-3, and Top-6 rankings. Since our metabolites are generated automatically by the metabolic template obtained by the algorithm, as long as the template is matched, the reaction products can be formed. There is a problem with the explosion of potential metabolite combinations. It is a great challenge for the model to hit the product of the real reaction in the reaction product, but at the same time, the model can learn a lot of false product information because of the production of a large number of false metabolites, thus enhancing the learning ability.</p>
<table-wrap id="T2" position="float">
<label>Table 2</label>
<caption>
<p>Prediction accuracies of the test set.</p>
</caption>
<table frame="hsides">
<thead>
<tr>
<th valign="top" align="center"/>
<th valign="top" align="center">Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top">Top-1</td>
<td valign="top" align="center">34%</td>
</tr>
<tr>
<td valign="top">Top-3</td>
<td valign="top" align="center">51%</td>
</tr>
<tr>
<td valign="top">Top-6</td>
<td valign="top" align="center">68%</td>
</tr>
<tr>
<td valign="top">Top-10</td>
<td valign="top" align="center">70%</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Here, we conducted external tests on 300 reaction records that were not used for model training. It is also compared with the rule-based prediction method SyGMa. The accuracy rates of Top-1, Top-3, Top-6, and Top-10 are 35, 55, 67, and 78% respectively for our method (<xref ref-type="fig" rid="f5">
<bold>Figure 5</bold>
</xref>). The accuracies of SyGMa for Top-1, Top-3, Top-6, and Top-10 are 20, 39, 50, and 70 respectively. The accuracy rates of our method are higher than SyGMa&#x2019;s. The main reason is that SyGMa does not produce the correct metabolites in some reactions.</p>
<fig id="f5" position="float">
<label>Figure 5</label>
<caption>
<p>Comparison results on external test set.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g005.tif"/>
</fig>
<p>As can be seen from <xref ref-type="fig" rid="f6">
<bold>Figure 6</bold>
</xref>, correctly predicted metabolic reaction products by our method are common metabolic reactions, because these types of reaction samples account for the vast majority of the training set. Some of the metabolic reactions that cannot be correctly predicted are due to reaction types being uncommon with fewer occurrences in data sets, or because the reactants are too complicated and have multiple reaction sites. Furthermore, because usually only one metabolite of a compound is recorded in the reaction record, the Top-1 metabolite predicted by our method may not exactly be the recorded one, but it may still be one of the metabolites. Besides, in the reaction record involving multi-site and multi-step reactions, we can only predict a single-step metabolic reaction at one site. For our model, it is difficult for us to learn the changes in ring-opening and ring-closing reactions, because too much information is lost in those processes. It is difficult to characterize those metabolic processes only by the molecular fingerprints of reactants and products.</p>
<fig id="f6" position="float">
<label>Figure 6</label>
<caption>
<p>Reaction cases for correct and incorrect predictions.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g006.tif"/>
</fig>
<p>The amount of our data is small for Deep Learning to learn more information. The more reaction records that focused on a specific reaction, the more accurate the prediction of the reaction is. Thus we need to expand the data set for training. Next, we will collect more and more metabolic reactions from KEGG and other databases to train models, so as to improve the prediction accuracy of the models.</p>
</sec>
<sec id="s3_2">
<title>Influences of Molecular Fingerprint Radius on the Results</title>
<p>We retrieved Morgan molecular fingerprints with radius 3 from potential metabolic reaction products in training set and retrained them with AutoEncoder algorithm (<xref ref-type="fig" rid="f7">
<bold>Figure 7</bold>
</xref>). Morgan molecular fingerprint with a radius of 3 is equivalent to ECFP6, which will contain more information about molecular fragments.</p>
<fig id="f7" position="float">
<label>Figure 7</label>
<caption>
<p>Flow chart of AutoEncoder combined with molecular fingerprint.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g007.tif"/>
</fig>
<p>As shown in <xref ref-type="table" rid="T3">
<bold>Table 3</bold>
</xref>, increasing fingerprint radius did not improve the prediction accuracy of Top-1 and Top-3, but did improve the prediction accuracy of Top-10. The results suggested that increasing fingerprint radius can improve the accuracy of the model to a certain extent, and AutoEncoder algorithm can help improve the prediction ability of the model as well.</p>
<table-wrap id="T3" position="float">
<label>Table 3</label>
<caption>
<p>Prediction accuracies at molecular fingerprint radius of 3.</p>
</caption>
<table frame="hsides">
<tbody>
<tr>
<td valign="top" align="center"/>
<th valign="top" align="center">
<bold>Accuracy</bold>
</th>
</tr>
<tr>
<td valign="top">Top-1</td>
<td valign="top" align="center">32%</td>
</tr>
<tr>
<td valign="top">Top-3</td>
<td valign="top" align="center">51%</td>
</tr>
<tr>
<td valign="top">Top-6</td>
<td valign="top" align="center">68%</td>
</tr>
<tr>
<td valign="top">Top-10</td>
<td valign="top" align="center">81%</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Here we take Zileuton as an example to analyze its prediction results of metabolites. It is an inhibitor of 5-lipoxygenase for the maintenance treatment of asthma. The main metabolic pathways of Zileuton are hydroxylation of benzene ring, oxidation of sulfur atoms on sulfur-containing heterocycles, and hydrolysis of nitrogen atoms on amide groups (<xref ref-type="bibr" rid="B20">Joshi et al., 2004</xref>) (<xref ref-type="fig" rid="f8">
<bold>Figure 8</bold>
</xref> and <xref ref-type="table" rid="T4">
<bold>Table 4</bold>
</xref>).</p>
<fig id="f8" position="float">
<label>Figure 8</label>
<caption>
<p>Metabolic pathways of Zileuton.</p>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g008.tif"/>
</fig>
<table-wrap id="T4" position="float">
<label>Table 4</label>
<caption>
<p>Top-10 predicted metabolites for Zileuton.</p>
</caption>
<table frame="hsides">
<tbody>
<tr>
<td valign="top" align="center">
<bold>Rank</bold>
</td>
<td valign="top" align="center">
<bold>Compounds</bold>
</td>
</tr>
<tr>
<td valign="top" align="center">1</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g019.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">2</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g020.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">3</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g021.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">4</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g022.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">5</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g023.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">6</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g024.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">7</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g025.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">8</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g026.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">9</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g027.tif"/>
</td>
</tr>
<tr>
<td valign="top" align="center">10</td>
<td valign="top" align="center">
<inline-graphic mimetype="image" mime-subtype="tif" xlink:href="fphar-10-01586-g028.tif"/>
</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Here, three metabolites of zileuton were predicted correctly by our method, namely hydroxylation of the benzene ring and oxidation of sulfur atoms on sulfur-containing heterocycles. But our model has not predicted the hydrolysis of N atom of the side chain amide. The possible reason is that our training set has too little reactions to this type and the model has not adequately learned.</p>
</sec>
</sec>
<sec id="s4">
<title>Conclusion</title>
<p>In summary, we developed a deep learning based drug metabolites prediction algorithm to complement the experimental methods. By generating a broad coverage of metabolic reaction templates, we can generate a large number of potential metabolic reactants, and rank all metabolites by deep neural network algorithm to get the right metabolites ranked high. The accuracy of Top-1, Top-3, Top-6, and Top-10 in 300 external test sets with metabolic reactions is 35, 55, 67, and 78% respectively, which is significantly higher than that of random guess and the rule-based method SyGMa. Nevertheless, our method still has some limitations. It can rank the metabolites, but cannot give the probability of occurrence of metabolic sites. Besides, despite the relatively high prediction accuracy, it still has a high false-positive problem. To sum up, A approach of drug metabolites prediction based Deep learning was developed in this paper, which has certain predictive ability and can be used to provide some guidance information for researchers to improve the metabolic properties of lead compounds.</p>
</sec>
<sec id="s5">
<title>Data Availability Statement</title>
<p>The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation, to any qualified researcher.</p>
</sec>
<sec id="s6">
<title>Author Contributions</title>
<p>DW conducted the research and wrote the paper. WL validated the model using known drugs and natural products. ZS helped collect the metabolic reaction data. LJ participated in building the metabolic reactions template library. JW tested the trained model. HL and SL designed and performed research, interpreted data, and approved the final manuscript.</p>
</sec>
<sec id="s8">
<title>Funding</title>
<p>This work was supported by the National Key Research and Development Program (2016YFA0502304 to HL); the National Natural Science Foundation of China (grant 81825020 to HL, 81803437 to SL); the National Science &amp; Technology Major Project &#x201c;Key New Drug Creation and Manufacturing Program,&#x201d; China (Number: 2018ZX09711002); the Fundamental Research Funds for the Central Universities, Special Program for Applied Research on Super Computation of the NSFC-Guangdong Joint Fund (the second phase) under Grant No.U1501501. SL is also sponsored by Shanghai Sailing Program (No. 18YF1405100). HL is also sponsored by National Program for Special Supports of Eminent Professionals and National Program for Support of Top-notch Young Professionals.</p>
</sec>
<sec id="s7">
<title>Conflict of Interest</title>
<p>Author WL was employed by company Jiangzhong Pharmaceutical Co., Ltd.</p>
<p>The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
<p>The reviewer MZ declared a past co-authorship with one of the authors HL to the handling editor.</p>
</sec>
</body>
<back>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Avorn</surname> <given-names>J.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>The $2.6 Billion Pill &#x2014; Methodologic and Policy Considerations</article-title>. <source>New Engl. J. Med.</source> <volume>372</volume> (<issue>20</issue>), <fpage>1877</fpage>&#x2013;<lpage>1879</lpage>. doi: <pub-id pub-id-type="doi">10.1056/NEJMp1500848</pub-id>
</citation>
</ref>
<ref id="B2">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Button</surname> <given-names>W. G.</given-names>
</name>
<name>
<surname>Judson</surname> <given-names>P. N.</given-names>
</name>
<name>
<surname>Long</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Vessey</surname> <given-names>J. D.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>Using absolute and relative reasoning in the prediction of the potential metabolism of xenobiotics</article-title>. <source>Cheminform</source> <volume>34</volume> (<issue>50</issue>), <fpage>1371</fpage>&#x2013;<lpage>1377</lpage>. doi: <pub-id pub-id-type="doi">10.1021/ci0202739</pub-id>
</citation>
</ref>
<ref id="B3">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Cariello</surname> <given-names>N. F.</given-names>
</name>
<name>
<surname>Wilson</surname> <given-names>J. D.</given-names>
</name>
<name>
<surname>Britt</surname> <given-names>B. H.</given-names>
</name>
<name>
<surname>Wedd</surname> <given-names>D. J.</given-names>
</name>
<name>
<surname>Burlinson</surname> <given-names>B.</given-names>
</name>
<name>
<surname>Gombar</surname> <given-names>V.</given-names>
</name>
<etal/>
</person-group>. (<year>2002</year>). <article-title>Comparison of the computer programs DEREK and TOPKAT to predictbacterial mutagenicity</article-title>. <source>Mutagenesis</source> <volume>17</volume> (<issue>4</issue>), <fpage>321</fpage>&#x2013;<lpage>329</lpage>. doi: <pub-id pub-id-type="doi">10.1093/mutage/17.4.3</pub-id>
</citation>
</ref>
<ref id="B4">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Damsky</surname> <given-names>W. E.</given-names>
</name>
<name>
<surname>Bosenberg</surname> <given-names>M.</given-names>
</name>
</person-group> (<year>2012</year>). <article-title>From bedding to bedside: genetically engineered mouse models of cancer inform concurrent clinical trials</article-title>. <source>Pigment Cell Melanoma Res.</source> <volume>25</volume> (<issue>4</issue>), <fpage>404</fpage>&#x2013;<lpage>405</lpage>. doi: <pub-id pub-id-type="doi">10.1111/j.1755-148X.2012.01013.x</pub-id>
</citation>
</ref>
<ref id="B5">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Darvas</surname> <given-names>F.</given-names>
</name>
</person-group>(<year>1988</year>). <article-title>Predicting metabolic pathways by logicprogramming</article-title>. <source>J. Mol. Graphics</source> <volume>6</volume> (<issue>2</issue>), <fpage>80</fpage>&#x2013;<lpage>86</lpage>. doi: <pub-id pub-id-type="doi">10.1016/0263-7855(88)85004-5</pub-id>
</citation>
</ref>
<ref id="B6">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Di</surname> <given-names>L.</given-names>
</name>
</person-group> (<year>2014</year>). <article-title>The role of drug metabolizing enzymes in clearance</article-title>. <source>Expert Opin. Drug Metab. Toxicol.</source> <volume>10</volume> (<issue>3</issue>), <fpage>379</fpage>&#x2013;<lpage>393</lpage>. doi: <pub-id pub-id-type="doi">10.1517/17425255.2014.876006</pub-id>
</citation>
</ref>
<ref id="B7">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Diao</surname> <given-names>X.</given-names>
</name>
<name>
<surname>Scheidweiler</surname> <given-names>K. B.</given-names>
</name>
<name>
<surname>Wohlfarth</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Pang</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Kronstrand</surname> <given-names>R.</given-names>
</name>
<name>
<surname>Huestis</surname> <given-names>M. A.</given-names>
</name>
</person-group> (<year>2016</year>). <article-title>
<italic>In vitro</italic> and <italic>in vivo</italic> Human Metabolism of Synthetic Cannabinoids FDU-PB-22 and FUB-PB-22</article-title>. <source>AAPS J.</source> <volume>18</volume> (<issue>2</issue>), <fpage>455</fpage>&#x2013;<lpage>464</lpage>. doi: <pub-id pub-id-type="doi">10.1208/s12248-016-9867-4</pub-id>
</citation>
</ref>
<ref id="B8">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Dickson</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Gagnon</surname> <given-names>J. P.</given-names>
</name>
</person-group> (<year>2004</year>). <article-title>The cost of new drug discovery and development</article-title>. <source>Discovery Med.</source> <volume>4</volume> (<issue>22</issue>), <fpage>172</fpage>&#x2013;<lpage>179</lpage>.</citation>
</ref>
<ref id="B9">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Dimasi</surname> <given-names>J. A.</given-names>
</name>
<name>
<surname>Grabowski</surname> <given-names>H. G.</given-names>
</name>
<name>
<surname>Hansen</surname> <given-names>R. W.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>The Cost of Drug Development</article-title>. <source>New Engl. J. Med.</source> <volume>372</volume> (<issue>20</issue>), <fpage>1972</fpage>. doi: <pub-id pub-id-type="doi">10.1056/NEJMc1503146</pub-id>
</citation>
</ref>
<ref id="B10">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Ekins</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Andreyev</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Ryabov</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Kirillov</surname> <given-names>E.</given-names>
</name>
<name>
<surname>Rakhmatulin</surname> <given-names>E. A.</given-names>
</name>
<name>
<surname>Bugrim</surname> <given-names>A.</given-names>
</name>
<etal/>
</person-group>. (<year>2005</year>a). <article-title>Computational prediction of human drugmetabolism</article-title>. <source>Expert Opin. Drug Metab. Toxicol.</source> <volume>1</volume> (<issue>2</issue>), <fpage>303</fpage>&#x2013;<lpage>324</lpage>. doi: <pub-id pub-id-type="doi">10.1517/17425255.1.2.303</pub-id>
</citation>
</ref>
<ref id="B11">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Ekins</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Nikolsky</surname> <given-names>Y.</given-names>
</name>
<name>
<surname>Nikolskaya</surname> <given-names>T.</given-names>
</name>
</person-group> (<year>2005</year>b). <article-title>Techniques: application of systems biology to absorption, distribution, metabolism, excretion and toxicity</article-title>. <source>Trends Pharmacol. Sci.</source> <volume>26</volume> (<issue>4</issue>), <fpage>202</fpage>&#x2013;<lpage>209</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.tips.2005.02.006</pub-id>
</citation>
</ref>
<ref id="B12">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Ekins</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Andreyev</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Ryabov</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Kirillov</surname> <given-names>E.</given-names>
</name>
<name>
<surname>Rakhmatulin</surname> <given-names>E. A.</given-names>
</name>
<name>
<surname>Sorokina</surname> <given-names>S.</given-names>
</name>
<etal/>
</person-group>. (<year>2006</year>). <article-title>A combined approach to drug metabolism and toxicity assessment</article-title>. <source>Drug Metab. Disposition</source> <volume>34</volume> (<issue>3</issue>), <fpage>495</fpage>&#x2013;<lpage>503</lpage>. doi: <pub-id pub-id-type="doi">10.1124/dmd.105.008458</pub-id>
</citation>
</ref>
<ref id="B13">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Embrechts</surname> <given-names>M. J.</given-names>
</name>
<name>
<surname>Ekins</surname> <given-names>S.</given-names>
</name>
</person-group> (<year>2007</year>). <article-title>Classification of metabolites with kernel-partial least squares (K-PLS)</article-title>. <source>Drug Metab. Disposition</source> <volume>35</volume> (<issue>3</issue>), <fpage>325</fpage>&#x2013;<lpage>327</lpage>. doi: <pub-id pub-id-type="doi">10.1124/dmd.106.013185</pub-id>
</citation>
</ref>
<ref id="B14">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Fox</surname> <given-names>T.</given-names>
</name>
<name>
<surname>Kriegl</surname> <given-names>J. M.</given-names>
</name>
</person-group> (<year>2006</year>). <article-title>Machine learning techniques for <italic>in silico</italic> modeling of drug metabolism</article-title>. <source>Curr. Topics Medicinal Chem.</source> <volume>6</volume> (<issue>15</issue>), <fpage>1579</fpage>&#x2013;<lpage>1591</lpage>. doi: <pub-id pub-id-type="doi">10.2174/156802606778108915</pub-id>
</citation>
</ref>
<ref id="B15">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Gabriele</surname> <given-names>C.</given-names>
</name>
<name>
<surname>Carosati</surname> <given-names>E.</given-names>
</name>
<name>
<surname>De.Boeck.</surname> <given-names>B.</given-names>
</name>
<name>
<surname>Ethirajulu.</surname> <given-names>K.</given-names>
</name>
<name>
<surname>Mackie</surname> <given-names>C.</given-names>
</name>
<name>
<surname>Trevor Howe</surname> <given-names>A.</given-names>
</name>
<etal/>
</person-group>. (<year>2005</year>). <article-title>MetaSite:&#x2009; understanding metabolism in human cytochromes from the Perspective of the Chemist</article-title>. <source>J. Medicinal Chem.</source> <volume>48</volume> (<issue>22</issue>), <fpage>6970</fpage>&#x2013;<lpage>6979</lpage>. doi: <pub-id pub-id-type="doi">10.1021/jm050529c</pub-id>
</citation>
</ref>
<ref id="B16">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Gleeson</surname> <given-names>M. P.</given-names>
</name>
<name>
<surname>Hersey</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Hannongbua</surname> <given-names>S.</given-names>
</name>
</person-group> (<year>2011</year>). <article-title>In-Silico ADME Models: A General Assessment of their Utility in Drug Discovery Applications</article-title>. <source>Curr. Topics Medicinal Chem.</source> <volume>11</volume> (<issue>4</issue>), <fpage>358</fpage>&#x2013;<lpage>381</lpage>. doi: <pub-id pub-id-type="doi">10.2174/156802611794480927</pub-id>
</citation>
</ref>
<ref id="B17">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Grant</surname> <given-names>J. A.</given-names>
</name>
<name>
<surname>Pickup</surname> <given-names>B. T.</given-names>
</name>
<name>
<surname>Nicholls</surname> <given-names>A.</given-names>
</name>
</person-group> (<year>2001</year>). <article-title>A smooth permittivity function for Poisson&#x2013;Boltzmann solvation methods</article-title>. <source>J. Comput. Chem.</source> <volume>22</volume> (<issue>6</issue>), <fpage>608</fpage>&#x2013;<lpage>640</lpage>. doi: <pub-id pub-id-type="doi">10.1002/jcc.1032</pub-id>
</citation>
</ref>
<ref id="B18">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Hennemann</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Friedl</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Lobell</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Keldenich</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Hillisch</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Clark</surname> <given-names>T.</given-names>
</name>
<etal/>
</person-group>. (<year>2010</year>). <article-title>CypScore: Quantitative prediction of reactivity toward cytochromes P450 based on semiempirical molecular orbital theory</article-title>. <source>Chemmedchem</source> <volume>4</volume> (<issue>4</issue>), <fpage>657</fpage>&#x2013;<lpage>669</lpage>. doi: <pub-id pub-id-type="doi">10.1002/cmdc.200800384</pub-id>
</citation>
</ref>
<ref id="B19">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Hwang</surname> <given-names>T. J.</given-names>
</name>
<name>
<surname>Carpenter</surname> <given-names>D.</given-names>
</name>
<name>
<surname>Lauffenburger</surname> <given-names>J. C.</given-names>
</name>
</person-group> (<year>2016</year>). <article-title>Failure of Investigational drugs in late-stage clinical development and publication of trial results</article-title>. <source>JAMA Intern Med.</source> <volume>176</volume> (<issue>12</issue>), <fpage>1826</fpage>&#x2013;<lpage>1833</lpage>. doi: <pub-id pub-id-type="doi">10.1001/jamainternmed.2016.6008</pub-id>
</citation>
</ref>
<ref id="B20">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Joshi</surname> <given-names>E.  M.</given-names>
</name>
<name>
<surname>Heasley</surname> <given-names>B.  H.</given-names>
</name>
<name>
<surname>Chordia</surname> <given-names>M.  D.</given-names>
</name>
<name>
<surname>Macdonald</surname> <given-names>T. L.</given-names>
</name>
</person-group> (<year>2004</year>). <article-title>
<italic>In vitro</italic> metabolism of 2-acetylbenzothiophene: relevance to zileuton hepatotoxicity</article-title>. <source>Chem. Res. Toxicol.</source> <volume>17</volume> (<issue>2</issue>), <fpage>137</fpage>&#x2013;<lpage>143</lpage>. doi: <pub-id pub-id-type="doi">10.1021/tx0341409</pub-id>
</citation>
</ref>
<ref id="B21">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kang</surname> <given-names>Y. P.</given-names>
</name>
<name>
<surname>Ward</surname> <given-names>N. P.</given-names>
</name>
<name>
<surname>Denicola</surname> <given-names>G. M.</given-names>
</name>
</person-group> (<year>2018</year>). <article-title>Recent advances in cancer metabolism: a technological perspective</article-title>. <source>Exp. Mol. Med.</source> <volume>50</volume> (<issue>4</issue>), <fpage>31</fpage>. doi: <pub-id pub-id-type="doi">10.1038/s12276-018-0027-z</pub-id>
</citation>
</ref>
<ref id="B22">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kirchmair</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Howlett</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Peironcely</surname> <given-names>J. E.</given-names>
</name>
<name>
<surname>Murrell</surname> <given-names>D. S.</given-names>
</name>
<name>
<surname>Williamson</surname> <given-names>M. J.</given-names>
</name>
<name>
<surname>Adams</surname> <given-names>S. E.</given-names>
</name>
<etal/>
</person-group>. (<year>2013</year>). <article-title>How do metabolites differ from their parent molecules and how are they excreted</article-title>? <source>J. Chem. Inf. Modeling</source> <volume>53</volume> (<issue>2</issue>), <fpage>354</fpage>&#x2013;<lpage>367</lpage>. doi: <pub-id pub-id-type="doi">10.1021/ci300487z</pub-id>
</citation>
</ref>
<ref id="B23">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Klopman</surname> <given-names>G.</given-names>
</name>
<name>
<surname>Dimayuga</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Talafous</surname> <given-names>J.</given-names>
</name>
</person-group> (<year>1994</year>). <article-title>META. 1. A program for the evaluation of metabolic transformation of chemicals</article-title>. <source>J. Chem. Inf. Comput. Sci.</source> <volume>34</volume> (<issue>6</issue>), <fpage>1320</fpage>. doi: <pub-id pub-id-type="doi">10.1021/ci00022a014</pub-id>
</citation>
</ref>
<ref id="B24">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Klopman</surname> <given-names>G.</given-names>
</name>
<name>
<surname>Tu</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Talafous</surname> <given-names>J.</given-names>
</name>
</person-group> (<year>1997</year>). <article-title>META. 3. A genetic algorithm for metabolic transform priorities optimization</article-title>. <source>J. Chem. Inf Comput. Sci.</source> <volume>37</volume> (<issue>2</issue>), <fpage>329</fpage>&#x2013;<lpage>334</lpage>. doi: <pub-id pub-id-type="doi">10.1021/ci9601123</pub-id>
</citation>
</ref>
<ref id="B25">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Korolev</surname> <given-names>D.</given-names>
</name>
<name>
<surname>Balakin</surname> <given-names>K. V.</given-names>
</name>
<name>
<surname>Nikolsky</surname> <given-names>Y.</given-names>
</name>
<name>
<surname>Kirillov</surname> <given-names>E.</given-names>
</name>
<name>
<surname>Ivanenkov</surname> <given-names>Y. A.</given-names>
</name>
<name>
<surname>Savchuk</surname> <given-names>N.&#xa0;P.</given-names>
</name>
<etal/>
</person-group>. (<year>2003</year>). <article-title>Modeling of human cytochrome p450-mediated drug metabolism using unsupervised machine learning approach</article-title>. <source>J. Medicinal Chem.</source> <volume>46</volume> (<issue>17</issue>), <fpage>3631</fpage>&#x2013;<lpage>3643</lpage>. doi: <pub-id pub-id-type="doi">10.1021/jm030102a</pub-id>
</citation>
</ref>
<ref id="B26">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Krejsa</surname> <given-names>C. M.</given-names>
</name>
<name>
<surname>Horvath</surname> <given-names>D.</given-names>
</name>
<name>
<surname>Rogalski</surname> <given-names>S. L.</given-names>
</name>
<name>
<surname>Penzotti</surname> <given-names>J. E.</given-names>
</name>
<name>
<surname>Mao</surname> <given-names>B.</given-names>
</name>
<name>
<surname>Barbosa</surname> <given-names>F.</given-names>
</name>
<etal/>
</person-group>. (<year>2003</year>). <article-title>Predicting ADME properties and side effects: the BioPrint approach</article-title>. <source>Curr. Opin. Drug Discovery Devel</source> <volume>6</volume> (<issue>4</issue>), <fpage>470</fpage>&#x2013;<lpage>480</lpage>.</citation>
</ref>
<ref id="B27">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Law</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Zsoldos</surname> <given-names>Z.</given-names>
</name>
<name>
<surname>Simon</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Reid</surname> <given-names>D.</given-names>
</name>
<name>
<surname>Liu</surname> <given-names>Y.</given-names>
</name>
<name>
<surname>Khew</surname> <given-names>S. Y.</given-names>
</name>
<etal/>
</person-group>. (<year>2009</year>). <article-title>Route designer: a retrosynthetic analysis tool utilizing automated retrosynthetic rule generation</article-title>. <source>J. Chem. Inf. Modeling</source> <volume>49</volume> (<issue>3</issue>), <fpage>593</fpage>. doi: <pub-id pub-id-type="doi">10.1021/ci800228y</pub-id>
</citation>
</ref>
<ref id="B28">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Lazar</surname> <given-names>M. A.</given-names>
</name>
<name>
<surname>Birnbaum</surname> <given-names>M. J.</given-names>
</name>
</person-group> (<year>2012</year>). <article-title>Physiology</article-title>. <source>De-meaning Metab. Sci.</source> <volume>336</volume> (<issue>6089</issue>), <fpage>1651</fpage>&#x2013;<lpage>1652</lpage>. doi: <pub-id pub-id-type="doi">10.1126/science.1221834</pub-id>
</citation>
</ref>
<ref id="B29">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Livingstone</surname> <given-names>D. J.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>The characterization of chemical structures using molecular properties. A survey</article-title>. <source>J. Chem. Inf Comput. Sci.</source> <volume>31</volume> (<issue>23</issue>), <fpage>195</fpage>&#x2013;<lpage>209</lpage>. doi: <pub-id pub-id-type="doi">10.1021/ci990162i</pub-id>
</citation>
</ref>
<ref id="B30">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Mackenzie</surname> <given-names>P. I.</given-names>
</name>
<name>
<surname>Somogyi</surname> <given-names>A. A.</given-names>
</name>
<name>
<surname>Miners</surname> <given-names>J. O.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>Advances in Drug Metabolism and Pharmacogenetics Research in Australia</article-title>. <source>Pharmacol. Res.</source> <volume>116</volume>, <fpage>7</fpage>&#x2013;<lpage>19</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.phrs.2016.12.008</pub-id>
</citation>
</ref>
<ref id="B31">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Meng</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Li</surname> <given-names>S.</given-names>
</name>
<name>
<surname>Liu</surname> <given-names>X.</given-names>
</name>
<name>
<surname>Zheng</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Li</surname> <given-names>H.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>RD-Metabolizer: an integrated and reaction types extensive approach to predict metabolic sites and metabolites of drug-like molecules</article-title>. <source>Chem. Cent. J.</source> <volume>11</volume> (<issue>1</issue>), <fpage>65</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s13065-017-0290-4</pub-id>
</citation>
</ref>
<ref id="B32">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Paul</surname> <given-names>S. M.</given-names>
</name>
<name>
<surname>Mytelka</surname> <given-names>D. S.</given-names>
</name>
<name>
<surname>Dunwiddie</surname> <given-names>C. T.</given-names>
</name>
<name>
<surname>Persinger</surname> <given-names>C. C.</given-names>
</name>
<name>
<surname>Munos</surname> <given-names>B. H.</given-names>
</name>
<name>
<surname>Lindborg</surname> <given-names>S.&#xa0;R.</given-names>
</name>
<etal/>
</person-group>. (<year>2010</year>). <article-title>How to improve R&amp;D productivity: the pharmaceutical industry&#x2019;s grand challenge</article-title>. <source>Nat. Rev. Drug Discovery</source> <volume>9</volume> (<issue>3</issue>), <fpage>203</fpage>&#x2013;<lpage>214</lpage>. doi: <pub-id pub-id-type="doi">10.1038/nrd3078</pub-id>
</citation>
</ref>
<ref id="B33">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Ridder</surname> <given-names>L.</given-names>
</name>
<name>
<surname>Wagener</surname> <given-names>M.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>SyGMa: combining expert knowledge and empirical scoring in the prediction of metabolites</article-title>. <source>Chemmedchem</source> <volume>3</volume> (<issue>5</issue>), <fpage>821</fpage>&#x2013;<lpage>832</lpage>. doi: <pub-id pub-id-type="doi">10.1002/cmdc.200700312</pub-id>
</citation>
</ref>
<ref id="B34">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rydberg</surname> <given-names>P.</given-names>
</name>
<name>
<surname>Gloriam</surname> <given-names>D. E.</given-names>
</name>
<name>
<surname>Olsen</surname> <given-names>L.</given-names>
</name>
</person-group> (<year>2010</year>a). <article-title>The SMARTCyp cytochrome P450 metabolism prediction server</article-title>. <source>Bioinformatics</source> <volume>26</volume> (<issue>23</issue>), <fpage>2988</fpage>&#x2013;<lpage>2989</lpage>. doi: <pub-id pub-id-type="doi">10.1093/bioinformatics/btq584</pub-id>
</citation>
</ref>
<ref id="B35">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rydberg</surname> <given-names>P.</given-names>
</name>
<name>
<surname>Gloriam</surname> <given-names>D. E.</given-names>
</name>
<name>
<surname>Zaretzki</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Breneman</surname> <given-names>C.</given-names>
</name>
<name>
<surname>Olsen</surname> <given-names>L.</given-names>
</name>
</person-group> (<year>2010</year>b). <article-title>SMARTCyp: a 2D method for prediction of cytochrome p450-mediated drug metabolism</article-title>. <source>ACS Medicinal Chem. Lett.</source> <volume>1</volume> (<issue>3</issue>), <fpage>96</fpage>&#x2013;<lpage>100</lpage>. doi: <pub-id pub-id-type="doi">10.1021/ml100016x</pub-id>
</citation>
</ref>
<ref id="B36">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Scott</surname> <given-names>B.</given-names>
</name>
<name>
<surname>Amby.</surname> <given-names>C. H.</given-names>
</name>
<name>
<surname>Carlsson</surname> <given-names>L.</given-names>
</name>
<name>
<surname>Smith</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Stein</surname> <given-names>V.</given-names>
</name>
<name>
<surname>Glen</surname> <given-names>R. C.</given-names>
</name>
</person-group> (<year>2007</year>). <article-title>Reaction Site Mapping of Xenobiotic Biotransformations</article-title>. <source>J. Chem. Inf. Modeling</source> <volume>47</volume> (<issue>2</issue>), <fpage>583</fpage>. doi: <pub-id pub-id-type="doi">10.1021/ci600376q</pub-id>
</citation>
</ref>
<ref id="B37">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Segler</surname> <given-names>M. H. S.</given-names>
</name>
<name>
<surname>Preuss</surname> <given-names>M.</given-names>
</name>
<name>
<surname>Waller</surname> <given-names>M. P.</given-names>
</name>
</person-group> (<year>2018</year>). <article-title>Planning chemical syntheses with deep neural networks and symbolic AI</article-title>. <source>Nature</source> <volume>555</volume> (<issue>7698</issue>), <fpage>604</fpage>. doi: <pub-id pub-id-type="doi">10.1038/nature25978</pub-id>
</citation>
</ref>
<ref id="B38">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Stouch</surname> <given-names>T. R.</given-names>
</name>
<name>
<surname>Kenyon</surname> <given-names>J. R.</given-names>
</name>
<name>
<surname>Johnson</surname> <given-names>S. R.</given-names>
</name>
<name>
<surname>Chen</surname> <given-names>X. Q.</given-names>
</name>
<name>
<surname>Doweyko</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Yi</surname> <given-names>L.</given-names>
</name>
</person-group> (<year>2003</year>). <article-title>
<italic>In silico</italic> ADME/Tox: why models fail</article-title>. <source>J. Comput. Aided Mol. Des.</source> <volume>17</volume> (<issue>2-4</issue>), <fpage>83</fpage>&#x2013;<lpage>92</lpage>. doi: <pub-id pub-id-type="doi">10.1023/A:1025358319677</pub-id>
</citation>
</ref>
<ref id="B39">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Talafous</surname> <given-names>J.</given-names>
</name>
<name>
<surname>Sayre</surname> <given-names>L. M.</given-names>
</name>
<name>
<surname>Mieyal</surname> <given-names>J. J.</given-names>
</name>
<name>
<surname>Klopman</surname> <given-names>G.</given-names>
</name>
</person-group> (<year>1994</year>). <article-title>META. 2. A dictionary model of mammalian xenobiotic metabolism</article-title>. <source>J. Chem. Inf. Comput. Sci.</source> <volume>34</volume> (<issue>6</issue>), <fpage>1326</fpage>. doi: <pub-id pub-id-type="doi">10.1021/ci00022a015</pub-id>
</citation>
</ref>
<ref id="B40">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Tan</surname> <given-names>B. H.</given-names>
</name>
<name>
<surname>Pan</surname> <given-names>Y.</given-names>
</name>
<name>
<surname>Dong</surname> <given-names>A. N.</given-names>
</name>
<name>
<surname>Ong</surname> <given-names>C. E.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>
<italic>In vitro</italic> and <italic>in silico</italic> Approaches to Study Cytochrome P450-Mediated Interactions</article-title>. <source>J. Pharm. Pharm Sci.</source> <volume>20</volume> (<issue>1</issue>), <fpage>319</fpage>. doi: <pub-id pub-id-type="doi">10.18433/J3434R</pub-id>
</citation>
</ref>
<ref id="B41">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Testa</surname> <given-names>B.</given-names>
</name>
<name>
<surname>Balmat</surname> <given-names>A. L.</given-names>
</name>
<name>
<surname>Long</surname> <given-names>A.</given-names>
</name>
<name>
<surname>Judson</surname> <given-names>P.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>Predicting drug metabolism&#x2013;an evaluation of the expert system METEOR</article-title>. <source>Chem. Biodiversity</source> <volume>2</volume> (<issue>7</issue>), <fpage>872</fpage>&#x2013;<lpage>885</lpage>. doi: <pub-id pub-id-type="doi">10.1002/cbdv.200590064</pub-id>
</citation>
</ref>
<ref id="B42">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Zhang</surname> <given-names>T.</given-names>
</name>
<name>
<surname>Chen</surname> <given-names>Q.</given-names>
</name>
<name>
<surname>Li</surname> <given-names>L.</given-names>
</name>
<name>
<surname>Angela Liu</surname> <given-names>L.</given-names>
</name>
<name>
<surname>Wei</surname> <given-names>D. Q.</given-names>
</name>
</person-group> (<year>2011</year>). <article-title>In silico prediction of cytochrome P450-mediated drug metabolism</article-title>. <source>Comb. Chem. High Throughput Screening</source> <volume>14</volume> (<issue>5</issue>), <fpage>388</fpage>&#x2013;<lpage>395</lpage>. doi: <pub-id pub-id-type="doi">10.2174/138620711795508412</pub-id>
</citation>
</ref>
</ref-list>
</back>
</article>