All training set data used to construct BBB permeability model were comprised of non-proprietary data harvested from published literature (e.g., PubMed, Web of Science v.5.34, Scopus, Elsevier, and Google Scholar), US FDA approval packages (e.g., Drugs@FDA and PharmaPendium^®), EMA approval packages (e.g., PharmaPendium^®), and patents. All references for BBB databases are provided in Supplementary Table S1.

The BBB permeability database contains blood/plasma (B/P) or blood/brain (B/B) ratios obtained from rodents that were treated via intravenous, intraperitoneal, or oral routes. For the majority of data entries, the amount of the chemical present in the brain and blood or plasma was measured in the animals 30 min to a several hours after administration. However, in some cases, the animals were sacrificed at certain intervals after treatment and different B/P ratios were reported. In such cases, the ratio of the area under the curve (AUC) for the brain and plasma concentrations were used. In experiments where different amounts of a chemical were reported in different parts of the brain, the average value was considered. All findings were transformed into a binary scoring system for modeling purposes, where “0” denotes a negative finding (no brain penetration) and “1” denotes a positive finding (brain penetration). Chemicals with a log BB ≥ -1 were considered positive while chemicals with a log BB < -1 were considered negative (Vilar et al., 2010). The final BBB database is comprised of 921 compounds with 52% positives. The dataset and references are provided in Supplementary Table S1.

The chemical structures were obtained from SciFinder^® and published literature. Electronic representations of chemical structures were created using structure data file (SDF) format. Inorganic chemicals, noble gases, mixtures, single atoms, metals, and high molecular weight compounds (MW ≥ 1800; polysaccharides, proteins, polymers, etc.) were excluded from the training set due to processing limitations within the QSAR software. Furthermore, the neutralized free form of any simple salt was included. A final manual inspection was performed to ensure the chemicals, their associated data and references were accurately recorded.

Two commercial QSAR software platforms, Leadscope Enterprise (LS) version 3.9 (Instem Inc., United States), and CASE Ultra (CU) version 1.8.0.1 (MultiCASE Inc., United States) were used to construct two distinct binary QSAR models. All software programs were acquired and used under Research Collaboration Agreements between FDA/CDER and the software providers mentioned above.

The predictive performance of the BBB models was assessed using an external validation set comprised of 83 chemicals (42 positives and 41 negatives) obtained from published literature. All references and activity scores are provided in Supplementary Table S2 for the external validation set.

To examine the combined predictive performance of LS and CU, a positive prediction from any one software platform was used to justify an overall positive prediction. Similarly, an equivocal prediction from any one software platform was used to justify an overall equivocal prediction, in the absence of a positive prediction. In the case that one of the models was OOD and the other model generated a prediction, the OOD was disregarded and the prediction was used to generate an overall call. An overall negative prediction was reported when a statistical model generated a negative prediction in the absence of positive or equivocal predictions from the other model.

In order to evaluate the performance of individual model outputs, Cooper statistics was employed. Briefly, predictive performance was evaluated using a classic 2x2 contingency table containing counts of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Chemicals classified as OOD and equivocal were excluded from Cooper statistic calculations. Statistics such as sensitivity [TP/(TP + FN)], specificity [TN/(TN + FP)], positive predictivity [TP/(TP + FP)], negative predictivity [TN/(TN-FN)], and accuracy [(TP + TN)/(TP + TN + FP + FN)] were calculated as described by Cooper et al., 1979 (Cooper et al., 1979). Coverage was calculated as the percentage of all chemicals screened for which a prediction could be made (OOD results do not constitute a prediction).

In the present study, the rodent BBB permeability dataset was compiled from publicly available data sources and the original study data were used. The results from rats and mice were treated as equivalent since previous studies show no significant difference in brain permeability between rats and mice (Murakami et al., 2000; Abraham et al., 2006). The final BBB permeability database contains 921 unique chemicals of which 621 compounds are from in vivo studies in rats and 300 in mice. The database is well-balanced with a total of 478 compounds scored as positive and 443 as negative (activity scores provided in Supplementary Table S1). Furthermore, the database is comprised of 263 drug substances approved between 1939 and 2022, 21 drug derivatives, 13 drug metabolites, 61 investigational drugs undergoing clinical trials, 14 prodrugs, and 549 other non-drug molecules. This database covers a broad range of chemical space, functional groups, and Anatomical Therapeutic Chemical (ATC) classes as presented in Figure 1. Most functional groups and ATC classes have an equal distribution between positives and negatives in the database. Chemicals that contain carboxylic acid, sulfone, sulfonyl and sulfonamide functional groups were mostly negative. As expected, the majority of central nervous system drugs in the database cross the BBB. However, two triptan analgesics (rizatriptan and almotriptan) were identified among negative drugs (Figure 2B). A possible reason is that triptans are usually substrates of human, but not rat, BBB uptake transporters (Zhang et al., 2016). Another interesting finding is that majority of the cardiovascular drugs in the database cross the BBB. A review of the literature suggested that the lipophilicity of many cardiovascular drugs, specifically beta blocking agents, may be the reason for their BBB permeability (McAinsh and Cruickshank, 1990; Goldner, 2012; Shah et al., 2020). When compared to several of the previously described models, the current training set showed improved coverage of almost all chemical functional groups (Supplementary Table S5).

Previous modeling efforts employed calculated physicochemical descriptors such as polar surface area (PSA), number of hydrogen donors/acceptors, and molecular weight to predict BBB (Young et al., 1988; van de Waterbeemd and Kansy, 1992b; Abraham et al., 1994; Lombardo et al., 1996; Norinder et al., 1998; Clark, 1999; Luco, 1999; Feher et al., 2000; Keserü and Molnár, 2001; Platts et al., 2001; Abraham, 2004; Abraham et al., 2006). While these properties influence BBB permeability of molecules and can be applied to simple cases, they are limited in their ability to comprehensively predict BBB permeability of drugs that pass through more complex mechanisms. In the present study, machine-learning algorithms were used to examine all structural features present in the training set and global molecular properties that are useful to predict and interpret BBB permeability. The two modeling platforms that were used to construct BBB models are LS and CU.

The LS QSAR model was optimized by manual refinement of chemical structural features and physicochemical descriptors. Highly predictive features were identified for retention while 14 redundant and less discriminating chemical features were removed. The total number of chemical features present in the final BBB model is 386. Examples of chemical features with highest and lowest Z-scores, corresponding to highest and lowest BBB permeability are presented in Figure 2A. Chemical features with highest Z-scores are comprised of aliphatic and aromatic rings while carboxylic acids and carbonyls have the lowest Z-scores. Moreover, polycyclic secondary and tertiary amines are also positive features.

Additionally, six physicochemical descriptors including molecular weight, number of rotatable bonds, number of hydrogen bond donors, Lipinski score, AlogP, and PSA were assessed for their predictive ability. The overall results showed a very poor correlation between the individual physicochemical descriptors and log BB alone. Specifically, the squared Pearson correlation coefficient (R ²) values for log BB ratio and molecular weight, PSA, and number of hydrogen bond donors are 0.02, 0.08 and 0.02, respectively (see Figure 2B and Supplementary Figure S1). However, it should be noted that a 3% increase in statistical performance was observed upon inclusion of the six molecular descriptors. The predictivity of the model and frequency of the compounds as a function of probability is presented in Figure 2C. The U-shaped plots indicate the optimum regression, with the maximum probability located at the two ends of the axis. The lowest predictivity and frequency was identified to be at approximately 0.5 and selected as the equivocal range.

The CU models were optimized using ROC plots that were generated by varying the classification threshold which defines a positive prediction (Figure 3A). The optimal classification threshold was identified to be 0.55 (Figure 3A; orange dot). The number of chemical fragments present in the CU model is 171. Selected examples of chemical fragments with the highest number of positive and negative compounds are presented in Figure 3B. Chemical fragments with the highest number of chemicals that permeate the BBB contain aromatic moieties and amines while chemical fragments with the highest number of negative chemicals contain carboxylic acids and cyclic ethers.

The predictive performance statistics for the BBB models based on 10% LMO cross-validation experiments as well as the external validation experiments are presented in Table 2. However, it should be noted that the coverage in the cross-validation statistics from LS and CU cannot be compared directly as they are calculated differently. The CU coverage is calculated using domain analysis while LS provides a prediction for all the chemicals in the cross-validation experiment. The LS model achieved a sensitivity of 82% and a negative predictivity of 80% in cross-validation, while the CU model achieved a sensitivity of 85% and a negative predictivity of 83%. Furthermore, when using an external validation set of 83 chemicals (51% positive; 28 drugs and 55 non-drug molecules), the LS model achieved a sensitivity of 70% and a negative predictivity of 72%, while the CU model achieved a sensitivity of 75% and a negative predictivity of 70%. The partitioned predictive performance for drugs and nondrugs is provided in Supplementary Table S3. Additionally, a prediction comparison analysis for LS and CU by functional groups and drug classes is provided in Supplementary Figures S2–4.

In a subsequent evaluation, the combined predictive performance of the LS and CU models was assessed (Table 2). Here, the models achieved a sensitivity of 80% and negative predictivity of 70%. However, a decrease in specificity (51%) and positive predictivity (64%) was observed when predictions across the two software programs were combined. These results were anticipated given that combining predictions across different software platforms results in an increase of false positive predictions. A total of 11 chemicals were outside the applicability domain for LS while 18 were outside applicability domain for CU. However, when predictions from the LS and CU were combined, 93% of all chemicals were within the applicability domain.

Obtaining meaningful alerts and a robust QSAR model depend heavily on the quality of the training set data. In the present study, efforts were made to identify and extract high quality data for the BBB permeability model. One of the most commonly reported and trusted measures for BBB permeability is log BB; this parameter is generated by most pharmaceutical companies for drug candidates. Among the challenges of combining Log BB data from multiple sources is the potential for introducing conflicting data into models thereby affecting the quality of the data. To enhance the overall quality of the underlying data, chemicals with contradictory and/or equivocal study results were reviewed and resolved or removed from the databases.

Recently, several studies have suggested that the steady state unbound brain-to-plasma ratio, K_p,uu,brain is a relevant parameter to measure drug concentration as the key driving force for drug distribution is the free concentration in the brain. However, publicly available data for K_p,uu,brain are very limited and therefore a viable model to predict K_p,uu,brain could not be developed at this time.

Previously published models employed calculated physicochemical descriptors such as lipophilicity, PSA and/or hydrogen bonding (Young et al., 1988; Van de Waterbeemd and Kansy, 1992a; Abraham et al., 1994; Clark, 1999). There is a general agreement that these specific descriptors can influence log BB (Clark, 2003). For instance, lipophilicity is positively correlated with log BB (Young et al., 1988; Calder and Ganellin, 1994; Kaliszan and Markuszewski, 1996; Salminen et al., 1997; Goodwin and Clark, 2005) while hydrogen bonding is negatively correlated to brain penetration (Van de Waterbeemd and Kansy, 1992a; Calder and Ganellin, 1994; Clark, 1999). In addition, several reports indicate log BB is negatively correlated to molecular weight (Calder and Ganellin, 1994; Kaliszan and Markuszewski, 1996; Salminen et al., 1997; Kaznessis et al., 2001; Platts et al., 2001). In this investigation, the use physicochemical descriptors was found to improve the overall performance of the LS models (by 3%) when combined with chemical features, although physicochemical descriptors were poorly correlated with experimental log BB parameter alone. This can be attributed to the larger log BB data set that covers a more diverse chemical space (Brito-Sanchez et al., 2015). It is anticipated that as more data become available, finding a single equation that describes log BB as a function of physicochemical descriptors will become more difficult. Therefore, models that use a combination of chemical and physicochemical features may be advantageous.

A review of alerts and deactivating chemical features in LS and CU models revealed that the top features with highest activity scores belong to polycyclic aromatic compounds. The training set structures representing these alerts are relatively nonpolar (lipophilic), which is favorable for crossing the BBB. In addition, unlike primary amines, polycyclic secondary and tertiary amines are among the top positive alerts. Beside the reduced polarity of those amines, the ability of making hydrogen bonds is also reduced compared to primary amines, which may explain their higher BBB permeability (Silverman et al., 2009). In contrast, features that contained carboxylic acids and alcohols had the lowest activity scores presumably due to their ability to form hydrogen bonds (Abraham et al., 1994). At physiological pH of 7.4, carboxylic acids are dissociated to carboxylate ions, which improves their water solubility and the ability to form hydrogen bonds (Bredael et al., 2022). The low lipophilicity of carboxylic acids, at physiological pH, also limits their BBB penetration (Soloway et al., 1960). Additionally, ethers were also found among negative features due to their ability to accept hydrogen bonds. However, one should be aware of exceptions to the hydrogen bond rule. As discussed earlier, there is a low correlation between log BB and number of hydrogen bond acceptors/donors. A detailed assessment of the functional groups that are present in the BBB database showed that the current training set contains 135 compounds that have carboxylic acid groups with 30 being BBB permeable (Figure 1A). Compounds with carboxylic acids that pass BBB are typically substrates of uptake transporters. An example of this is l-DOPA, a precursor for dopamine, which has a carboxylic acid and a diol group and is capable of crossing BBB (Di et al., 2013).

In this investigation, an external validation set was used to examine BBB models individually and by combining predictions across LS and CU. In a regulatory setting, high sensitivity and negative predictivity are preferred to reduce the risk of false negatives and minimize the risk to public health. Towards this end, the current BBB models were tuned to achieve high sensitivity and negative predictivity while maintaining good overall predictive performance in other statistical parameters. Specifically, the new models showed a sensitivity ranging from 70 to 75% and negative predictivity ranging from 70 to 72% in external validation. Furthermore, when predictions from the two methodologies were combined, a sensitivity of 80% and coverage of 93% was achieved. While the increase in the false positive rate is not ideal when predictions are combined, it can be mitigated by evaluating the alerts behind the positive prediction and examining structurally similar analogs. Perhaps the most striking finding was that OOD chemicals in CU were successfully predicted by LS, suggesting that the two software platforms interpret chemicals differently resulting in different OOD domain predictions. Moreover, the overall increase in coverage is desirable for predicting a large diversity of chemicals.

BBB penetration of drugs is a complicated process involving passive diffusion and active transport (efflux or uptake). The current data set includes known substrates of active transporters. The model is agnostic to such complicated processes. Furthermore, the log BB data entries are collected from different experiments where drugs are administered through different routes and brain samples are collected at various time points post administration. Despite all these complications, the model can estimate BBB permeation with relatively high precision. In future, utilization of a combination of models for different transport mechanisms may further improve the log BBB predictivity.