Abstract
The blood-brain barrier (BBB) is a selective and semipermeable boundary that maintains homeostasis inside the central nervous system (CNS). The BBB permeability of compounds is an important consideration during CNS-acting drug development and is difficult to formulate in a succinct manner. Clinical experiments are the most accurate method of measuring BBB permeability. However, they are time taking and labor-intensive. Therefore, numerous efforts have been made to predict the BBB permeability of compounds using computational methods. However, the accuracy of BBB permeability prediction models has always been an issue. To improve the accuracy of the BBB permeability prediction, we applied deep learning and machine learning algorithms to a dataset of 3,605 diverse compounds. Each compound was encoded with 1,917 features containing 1,444 physicochemical (1D and 2D) properties, 166 molecular access system fingerprints (MACCS), and 307 substructure fingerprints. The prediction performance metrics of the developed models were compared and analyzed. The prediction accuracy of the deep neural network (DNN), one-dimensional convolutional neural network, and convolutional neural network by transfer learning was found to be 98.07, 97.44, and 97.61%, respectively. The best performing DNN-based model was selected for the development of the “DeePred-BBB” model, which can predict the BBB permeability of compounds using their simplified molecular input line entry system (SMILES) notations. It could be useful in the screening of compounds based on their BBB permeability at the preliminary stages of drug development. The DeePred-BBB is made available at https://github.com/12rajnish/DeePred-BBB.
Introduction
Neurological diseases are among the most predominant health issues, with an approximately 28% prevalence in all age groups of patients (). Despite a decrease in communicable neurological diseases, the number of deaths due to neurological diseases has increased to 39% in the last three decades (). This substantial increase in the absolute number of patients indicates that available therapeutics are scarce to prevent and manage neurological diseases in the current changing global demography. Therefore, it is imperative to find novel and effective therapeutics to target the central nervous system (CNS) to meet the challenges of the ever-increasing absolute number of patients with neurological diseases. An alternative method targeting the molecular and signaling mechanisms at BBB rather than the traditional approaches has become the recent trend in drug target validation (). Drugs must cross the blood-brain barrier (BBB) to act on the CNS. There is a higher attrition rate of drug candidates failing in clinical research due to non-permeability to the BBB compared to potency issues (; ; ; ). The BBB is a semipermeable and selective boundary that maintains the steady state of the CNS by protecting it from external compounds (98%) (Figure 1; ). As drugs need to enter the CNS to impart therapeutic activity, it becomes crucial to determine BBB permeability during the initial stages of CNS-acting drug design and development (; ; ).
FIGURE 1
The BBB separates the CNS from the bloodstream, preventing contagions from invading the brain. Brain endothelial cells, astrocytes, neurons, and pericytes are four major components of the BBB. The largest constituent of the BBB is a layer containing brain endothelial cells, which serve as the first line of defense from the CNS surroundings. Endothelial cells are connected with tight junctions and adherence junctions, which create a strong barrier, restricting pinocytosis and decreasing vesicle-facilitated transcellular transport (; ). The BBB not only acts as a physical barrier but also serves as a metabolic barrier, transport interface, and secretory layer (; ). Neurons reside very close to brain capillaries and play a vital role in maintaining ion balance in the local environment ().
Clinical experiments to determine the BBB permeability of compounds are accurate; however, they are time-consuming and labor-intensive (; ; ). Additionally, it is difficult to perform clinical experiments with diverse types of drug candidates (; ). Therefore, it is crucial to predict and forecast BBB permeability using computational algorithms or in vitro BBB mimics to elucidate the permeability of compounds across the BBB (). There have been numerous attempts to predict the BBB permeability of compounds since the advent of artificial intelligence (AI), primarily using machine learning (ML) algorithms such as support vector machines (SVMs), artificial neural networks (ANNs), k-nearest neighbors (kNNs), naïve Bayes (NB), and random forests (RFs) (; , ; ; ). In addition to above, some future directions of BBB permeability also seem promising such as application of humanized self-organized models, organoids, 3D cultures and human microvessel-on-a-chip platforms especially those which are amenable for advanced imaging such as transmission electron microscope and expansion microscopy since they enable real-time monitoring of BBB permeability (; ). BBB permeability prediction models developed using AI algorithms can further be assisted with the high throughput screening (), computer aided drug designing (), and knowledge based rules, e.g., Lipinski rule of five (hydrogen bond donor ≤ 5, hydrogen bond acceptor ≤ 10, molecular weight ≤ 500, CLogP ≤ 5), Veber rule (rotatable bonds count ≤ 10, polar surface area ≤ 140), BBB rule (hydrogen bond = 8–10, molecular weight = 400–500, no acids), etc., to screen potential drug candidates with desirable end-point for prevention, mitigation and cure of neurological disorders (; ; ).
In an attempt to develop the BBB permeability prediction model, applied SVM with a radial basis function (RBF) kernel (). They used a dataset of 1,562 compounds containing 694 BBB permeable (BBB++) and 868 BBB non-permeable (BBB-) compounds. The overall accuracy, sensitivity, and specificity were reported to be more than 85%. The next year, used the decision tree algorithm on 581 compounds and found that the BBB permeability prediction accuracy increased by 2.93% (). However, this study was performed on a much smaller dataset than Jiang et al.’s study. In another study, developed SVM-based BBB prediction model using a larger dataset of 1,990 compounds with a prediction accuracy of 93.96% (). The sensitivity and specificity of the model were reported to be 94.3 and 91.0%, respectively. In the same year, applied SVM and kNN algorithms using 2,358 compounds (). The prediction accuracy of the best-performing model was found to be 2.64% higher than that of the prediction model. However, the model lagged in terms of sensitivity (0.925) and specificity (0.899). The next year, applied a deep learning (DL) algorithm to 462 compounds. The accuracy of the model was reported to be 97%, with decent AUC (0.98) and F1 scores (0.92). However, the dataset used for the DL study was very small compared to the earlier ML-based models for BBB prediction. Recently, proposed a recurrent neural network (RNN) algorithm-based model using 2,342 compounds for the prediction of BBB permeability (). The developed model had better performance metrics with an accuracy, sensitivity, and specificity of 96.53, 94.91, and 98.09%, respectively. The Matthews correlation coefficient (MCC) (93.14) and area under the curve (AUC) (98.6) of the prediction were also found to be satisfactory. In another study, applied a light gradient boosting machine algorithm to a dataset of 7,162 compounds for the prediction of BBB permeability (). Although the study involved a very large dataset compared to previously reported studies, the model’s accuracy was reported to be 90%, which was approximately 6.5% less than the BBB permeability prediction model proposed by . In the same year, used random forest, multilayer perceptron, and sequential minimal optimization using 605 compounds to develop the BBB permeability prediction model (). Upon validation of the developed model using 1,566 compounds, the prediction accuracy was found to be 86.5% only. Very recently, proposed an ML-based BBB permeability prediction model using 1,978 compounds (). The study group found that SVM with the RBF kernel yielded an accuracy of 96.77% with AUC and F1 score values of 0.964 and 0.975, respectively, which outperformed the kNN, random forest, and naïve Bayes algorithms in the prediction of BBB permeability on the same dataset.
The major challenge while applying ML algorithms is selecting optimal features to develop predictive models based on labeled BBB permeability datasets (; ). To overcome this challenge, we applied DL algorithms and compared their performance with traditional ML algorithms.
Materials and Methods
Data Collection
A total of 3,971 compounds with BBB permeability classes were collected from ; , and . The PubChem database1 was used to retrieve available PubChem IDs of the collected compounds. The collected datasets were checked to remove redundant compounds. After careful curation, we obtained a dataset of 3,605 non-redundant clean compounds containing 2,607 BBB permeable and 998 BBB non-permeable compounds (Table 1 and Supplementary File). The class labels for BBB non-permeable and permeable compounds were kept as “0” and “1,” respectively.
TABLE 1
| Dataset | BBB permeable compounds | BBB non-permeable compounds | Total |
| 819 | 366 | 1,185 | |
| 1,398 | 393 | 1,791 | |
| 390 | 239 | 629 | |
| Total | 2,607 | 998 | 3,605 |
The final dataset and its distribution.
Feature Calculation
Three types of feature sets viz. physicochemical properties, molecular access system (MACCS) fingerprints and substructure fingerprints were used in this study. Physicochemical properties contain different types of physical and chemical information encoded in a compound, e.g., molecular weight, molecular volume, solubility, partition coefficient, etc. The molecular fingerprints are fixed-length vectors that indicate the presence/absence of an atom type or functional group in a compound. All features were calculated by open-source PaDel (). Each compound was encoded with 1,917 features containing 1,444 physicochemical (1D and 2D) properties, 166 MACCS, and 307 substructure fingerprints. This feature set was used for the ML, DNN, and CNN-1D algorithms using Keras framework. For CNN-VGG16, the Python package RDKit was used to generate the structure images of the compounds using their Simplified molecular input line entry system (SMILES) notations (; ). The Python package RDKit is a collection of ML and cheminformatic software and contains functions to modify chemical compounds. The RDKit package was used to generate 2D images of size 300 * 300 pixels (RGB) from SMILES notations of compounds. RDKit-generated images contain different colors to express the chemical information viz. carbon = black, oxygen = red, nitrogen = blue, sulfur = yellow, chlorine = green, and phosphorous = orange. Images generated by RDKit always fit the entire molecule, so there was no issue with different molecular sizes. The dataset was split into training and test sets at a ratio of 3:1. The test set was separated from the training set to avoid any bias (Table 2). To handle the data imbalance, we have already applied cost-sensitive augmentation via the class_weight argument on the fit() function when training models.
TABLE 2
| Dataset | BBB permeable compounds | BBB non-permeable compounds | Total |
| Training set | 1,955 | 749 | 2,704 |
| Test set | 652 | 249 | 901 |
| Total | 2,607 | 998 | 3,605 |
Distribution of the dataset in the training and test sets.
Development of Prediction Models
In this study, ML-based algorithms (SVM, kNN, RF, and NB) and DL-based algorithms DNN, CNN-1D were developed using keras framework with libraries; python, numpy, pandas, keras, and tensorflow on Anaconda 3–5.2. CNN (VGG16) was implemented using transfer learning through cloud-based computational resource of Google Colaboratory to develop prediction models for the BBB permeability of the compounds. Based on the performance of the generated prediction models, the DNN-based “DeePred-BBB” is proposed for BBB permeability prediction. DeePred-BBB performance was compared with ML algorithms viz. SVM, NB, kNN, RF, and DL algorithms CNN-1D and CNN (VGG16).
Machine Learning-Based Models
Support vector machine with four different kernels (RBF, polynomial, sigmoid, and linear), NB, kNN, and RF were applied to the training set of 2,704 compounds and tested with an independent set of 901 compounds. Principal component analysis (PCA) () was used for feature reduction. The component range (10, 20, 30, 40, 50, and 100) was used to find the best prediction accuracy for each applied ML algorithm. Tenfold cross-validation was applied to evaluate the efficacy of the model during training.
Support Vector Machine
Support vector machine is among the robust ML algorithms used for classification and regression (; ). It searches for the optimal hyperplane with maximized margins using support vectors for classification (). This algorithm plots the data to the N-dimensional feature space and finds a hyperplane (Θ.x + b = 0) to classify the data sets with minimized loss using the hinge loss function. The loss function is given in Eq. 1.
Support vector machine was applied using kernels to map the data to higher dimensions to linearly classify the data (). A penalty parameter “C” (Cost NAÏVE) adjusts the balance between training errors and forcing rigid margins. Another parameter, “γ,” regulates the kernel function amplitude (). Various values of C (1, 5, 10, 50, 90) and γ (0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05) were tested to find the best combination. An optimized combination of C and γ was used for each SVM kernel (RBF, C = 10, γ = 0.005; polynomial, C = 1, γ = 0.005; sigmoid C = 90, γ = 0.0005; linear, C = 1, γ = 0.05). For the polynomial kernel, 2–6 values of degree (d) were applied and evaluated. The best performance of the polynomial kernel was found at d = 3.
Naïve Bayes
The naïve Bayes algorithm is based on the Bayes theorem. It is a probabilistic method that works on the assumption of class conditional independence (Eq. 2) (). Each feature present in a class is independent and individually contributes to the probability with nil dependency on other features (). It is fast, readily manages a large dataset, and generally produces better results than other classification techniques when features existing in a class are independent.
where P(X|Y) is the posterior probability of X (class) for a given Y (feature), P(Y|X) is the likelihood, P(X) is the prior probability of class X, and P(Y) is the marginal probability of feature Y.
k-Nearest Neighbor
k-nearest neighbor is a simple and non-parametric classifier that assumes that nearby data points are similar and tend to have similar classes. Feature similarity is used to find the class label of a new data instance. It commonly uses Euclidean distance to find the closeness of the data points, and depending upon the class matching with considered k-points, the class labels are decided (). Here, k is the number of neighbors. The Euclidean distance between data points x (x1, x2, x3) and y (y1, y2, y3) is calculated using Eq. 3.
To determine the optimal k, a range of k-values (1–10) was evaluated. The best-performing prediction model at k = 3 was selected for further analysis.
Random Forest
Random forest uses ensemble learning to create a collection of decision trees (forest) that run concurrently and classify data instances (). Tree construction is performed using arbitrary input vectors and node division on arbitrary feature subsets. Each tree of the RF predicts a certain class, and depending upon the highest votes, the final class label is predicted (). In the current study, the developed prediction models were tested with variable trees in a forest (4, 8, 12, 32, 64). Each decision tree’s various depths (2–5) and estimators (5, 10, 20, 30, 40) were tested to find the best performing prediction model.
Deep Learning-Based Models
Deep learning algorithms use multiple neurons and hidden layers to extract high-level functions from input data. The major advantage of DL algorithms is their inherent property of selecting the most relevant features from the training dataset. Therefore, unlike ML algorithms, separate feature selection algorithms are not required (). In this study, three DL algorithms, DNN, CNN-1D, and CNN-VGG16, were applied. The tenfold cross-validation method was used to assess the model’s efficiency while training. The training dataset was further divided into ten subsets, iteratively training models using all subsets except one held out to test the performance.
Deep Neural Network
For DNN, 2,704 compounds, each encoded with 1,917 features (1,444 physicochemical properties, 166 MACCS, and 307 substructure fingerprints), were used to develop BBB permeability prediction models. Initial layers receive compounds encoded with feature vectors and subject them to the hidden layers. These hidden layers obtain the relevant information from the input vectors and project the freshly extracted features to the batch normalization layer. This layer increases the training process by reducing the intradata covariance. Dropout layers were applied to reduce the problem of coadaptation of neurons and overfitting (). These layers randomly drop the nodes as per the dropout rate. Rectified linear unit (ReLU) activation function was used, which adaptively transforms rectifier parameters. Furthermore, ReLU transforms the neuronal output by mapping it to the highest possible value or zero (if the value is negative) (). ReLU function is given in Eq. 4.
where xi is input for activation function f on channel “i.”
The “softmax” activation function was applied on the output layer to map the hidden layer output between 0 to 1 intervals. The Adam optimizer was used to minimize the loss value from the cross-entropy cost function.
The network performance of a DNN depends upon its depth and breadth. Therefore, it is vital to determine the optimal depth and breadth and optimize other parameters, e.g., the learning rate and dropout ratio. To achieve this, we kept other parameters fixed and evaluated the prediction accuracy by varying the hidden layers (K = 1–5) and neurons (100, 200, 300, 500, 800 neurons per layer). The DNNs were also simultaneously evaluated for five dropout ratios (0.1, 0.2, 0.3, 0.4, 0.5), and prediction accuracy was evaluated. Furthermore, various network configurations were evaluated for epochs (100, 200, 400, 500, 800) and learning rates (0.0001, 0.0002, 0.0003, 0.001, 0.002, 0.003) optimization. Table 3 summarizes the explored values of hyperparameters for the development of the DNN-based BBB permeability prediction model.
TABLE 3
| Parameter | Values |
| Number of hidden layers | 1–5 |
| Number of neurons | 100, 200, 300, 500, 800 |
| Dropout ratio | 0.1, 0.2, 0.3, 0.4, 0.5 |
| Learning rate | 0.0001, 0.0002, 0.0003, 0.001, 0.002, 0.003 |
| Epochs | 100, 200, 400, 500, 800 |
Hyperparameter values explored for the DNN model.
Convolutional Neural Network-1 Dimension (CNN-1D)
CNN is a particular type of DL that is widely used for image data classification (). There are three major layers in the CNN: convolutional, pooling, and fully connected layers. Cube-shaped weights and multiple filters (kernels) are applied in the convolutional layers to extract features and develop feature maps from the images (; ; ). The filter size may downsample the outputs; therefore, the size and number of kernels are vital (). To overcome the issue of downsampling, an optimized padding value is applied, which allows the filter kernels to create feature maps of the input image size.
Furthermore, other parameters of the convolutional layer also needed to be optimized, e.g., regularization type and value, activation function, and stride. Pooling layers specifically perform average or max-pooling in the filter region to lower the number of parameters and calculations by downsampling the representations. The fully connected layers flatten the output prior to the classification and are usually kept at the end. CNNs are created to process and learn from images. However, CNN-1D can be applied similarly to one-dimensional data containing physicochemical properties and fingerprints. We used three filters (15, 32, 64) to determine the local pattern in the 1,917 features, which were calculated from PaDel. After the CNN layers, dense layers (1 and 2) were tested for three dropout ratios (0.2, 0.3, 0.5). Table 4 summarizes the explored hyperparameter values for the development of the CNN-1D model.
TABLE 4
| Parameter | Values |
| Number of filters | 15, 32, 64 |
| Number of dense layers | 1, 2 |
| Dropout ratio | 0.2, 0.3, 0.4 |
| Learning rate | 0.0001, 0.0002, 0.0003, 0.001, 0.002, 0.003 |
| Epochs | 100, 200, 400, 500, 600 |
Explored hyperparameter values for the CNN-1D model.
Convolutional Neural Network by VGG16 Transfer Learning (CNN-VGG16)
The CNN processes the input 2D images to distinguish the image objects by allocating weights and biases. CNN captures temporal and spatial relationships using the tiny squares of input images by processing them through a series of convolution layers. Filters in each convolutional layer skid on the image to find relevant and specific features, e.g., edge detection, sharpen or blur the image and produce the feature map. The feature map’s size depends on filter numbers, filter slide-over pixels, and zero-padding (image borders are padded with zero). The 2D-array values of the feature map were subjected to the individual layer activation function (ReLU). Dimensionality reduction of each feature map is processed using pooling without any loss of information. The pooling layer’s output is sent into fully connected layers, which classify the images. The CNN with transfer learning (VGG16) was used in this study using RDKit-generated images. The images were scaled to a pixel size of 128 * 128 to develop and validate the BBB permeability prediction model.
Furthermore, image data argumentation was performed by randomly zooming (up to 10%) and flipping the images. The CNN (VGG16) hyperparameters are given in Table 5. The developed model was tested with an independent test set consisting of 901 images. Figure 2 depicts the adopted methodology to develop the DL-based prediction models.
TABLE 5
| Parameters | VGG16 |
| Convolutional blocks | Convolutional layers, Kernel size, Filters, Max-Pooling, Zero Padding: Predefined |
| Dense layers | 02 |
| Dense layers neurons | 150, 104 |
| Dropout ratio | 0.5 |
| Learning rate | 0.02 |
| Batch size | 132 |
| Epochs | 800 |
The hyperparameters for CNN (VGG16).
FIGURE 2
Results and Discussion
The performance metrics of the ten developed models (ML = 7, DL = 3) for BBB permeability prediction were compared to determine the best-performing model. The performance indicators used in this study were area under the curve (AUC), area under the precision-recall curve (AUPRC), average precision (AP), F1 score and accuracy, and Hamming distance (HD) of the prediction models. Among the developed ML prediction models, the SVM (RBF kernel)-based prediction model outperformed the NB, kNN, and RF algorithms for BBB permeability prediction with test set data. The accuracy of SVM (RBF) was found to be approximately 6% higher than that of NB and RF and approximately 1% higher than that of kNN. Moreover, SVM (RBF) yielded better prediction values of other performance indicators in BBB permeability prediction on the given dataset. However, the performance metrics of SVM (polynomial) at degree 3 were found to be very comparable to the SVM (RBF).
The performance metrics of the DL algorithms were found to be very close to each other. The prediction accuracies of DNN, CNN-1D, and CNN (VGG16) were 98.07, 97.44, and 97.66, respectively. However, the DNN model was superior in AUC, AUPRC, AP, F1, and HD when compared to that of CNN-1D and CNN (VGG16) (Table 6). The comparison of receiver operating characteristic (ROC) curves between SVM (RBF), DNN, CNN-1D, and CNN (VGG16) also indicates the superiority of DNN in BBB permeability prediction with the given dataset (Figure 3). Furthermore, the accuracy and loss plots of the DNN model are given in Figure 4. The accuracy plot shows good coherence between the training (red) and test (blue) accuracy, suggesting that the model is not overfitted. Additionally, coherence in the training (red) and validation/test (blue) loss in the loss plot (binary cross-entropy loss) is indicative of an unbiased model (Figure 4).
TABLE 6
| Algorithm | AUC | AUPRC | AP | F1 | A (%) | HD | FPR (%) | FNR (%) |
| SVM (RBF) | 0.964 | 0.988 | 0.975 | 0.985 | 96.29 | 0.022 | 6.451 | 0.724 |
| SVM (Polynomial d = 3) | 0.948 | 0.965 | 0.965 | 0.98 | 96.01 | 0.029 | 9.756 | 0.579 |
| SVM (Sigmoid) | 0.921 | 0.971 | 0.944 | 0.962 | 94.45 | 0.055 | 13.359 | 2.519 |
| SVM (Linear) | 0.916 | 0.969 | 0.938 | 0.963 | 94.56 | 0.054 | 15.242 | 1.497 |
| NB | 0.844 | 0.948 | 0.899 | 0.935 | 90.18 | 0.098 | 14.543 | 3.202 |
| kNN (3) | 0.927 | 0.974 | 0.949 | 0.968 | 95.3 | 0.047 | 12.891 | 1.615 |
| RF (3, 20) | 0.815 | 0.943 | 0.887 | 0.938 | 90.29 | 0.0971 | 26.666 | 1.471 |
| DNN | 0.992 | 0.997 | 0.996 | 0.987 | 98.07 | 0.019 | 4.048 | 1.159 |
| CNN-1D | 0.969 | 0.956 | 0.975 | 0.983 | 97.44 | 0.026 | 4.118 | 2.017 |
| CNN (VGG16) | 0.972 | 0.983 | 0.983 | 0.946 | 97.61 | 0.0804 | 4.581 | 2.326 |
Performance metrics of ML and DL algorithms.
AUC, area under curve; AUPRC, area under precision-recall curve; AP, average precision; F1, F1 score; A, accuracy; HD, hamming distance; FPR, false positive rate; FNR, false negative rate; SVM, support vector machine; RBF, radial basis function; d, degree; NB, naïve Bayes; kNN, k-nearest neighbor; RF, random forest; DNN, deep neural network; CNN-1D, convolution neural network-one dimension; CNN (VGG16), convolution neural network- visual geometry group16. Best performing model (highlighted in bold).
FIGURE 3
FIGURE 4
The better performance of DL algorithms compared to ML could be due to their ability to handle the large dataset and extract the most relevant features of their own. The performance metrics of DNN, CNN-1D, and CNN (VGG16) were very comparable. To our surprise, DNN was found to be slightly better in overall performance based on accuracy and other performance indicators compared to the CNN models. DNN appears to be a better option to handle the compounds encoded with physicochemical and fingerprint features for classifications and predictions. Based on the overall performance, we selected the DNN model for the development of “DeePred-BBB.” DeePred-BBB can predict BBB permeability based on chemical SMILES notation. It uses PaDel to calculate the features from the SMILES notation and sends them as input to the DNN model. The output is either permeable or non-permeable.
Compounds can penetrate the BBB using various different mechanisms, such as transmembrane diffusion, adsorptive endocytosis, saturable transporters, and extracellular pathways. Most drugs in clinical use till date are small, lipid soluble molecules that cross the BBB by transmembrane diffusion. The prediction models to determine the mechanism of BBB permeability require a mechanism-based set of compounds with their permeability class labels for each mechanism. However, the current study deals with the prediction of the BBB permeability of compounds (irrespective of how they penetrate the BBB) using their SMILES notations. The study holds limitations in identifying the mechanism by which compounds are BBB permeable. DeePred-BBB does not take multiple SMILES notations for prediction. The user needs to input the SMILES notations of compounds one at a time for accurate prediction of BBB permeability. A comparison between DeePred-BBB and previously reported BBB permeability prediction models is given in Table 7.
TABLE 7
| Algorithm | Data set | Prediction performance | Study group |
| SVM (RBF) | 1,562 compounds (BBB+ = 694, BBB- = 868) | >85% accuracy, sensitivity, and specificity | |
| Decision trees | 581 compounds | Accuracy = 87.93%, Sensitivity = 86.67% Specificity = 89.29% | |
| SVM (RBF) | 1,990 compounds (BBB permeable = 1,550, BBB non-permeable = 440) | Accuracy = 93.96%, Sensitivity = 94.3%, Specificity = 91.0%, MCC = 0.84 | |
| SVM, kNN | 2,358 compounds | Accuracy = 96.6%, Sensitivity = 92.5%, Specificity = 89.9% | ; |
| DL | 462 compounds (BBB permeable = 250, BBB non-permeable = 212) | Accuracy = 97%, AUC = 0.98, F1 = 0.92 | |
| RNN | 2,342 compounds | Accuracy = 96.53%, Sensitivity = 94.91%, Specificity = 98.09%, MCC = 0.931, AUC = 0.986 | |
| Light Gradient Boosting Machine Algorithm | 7,162 compounds (BBB permeable = 5,453 BBB non-permeable = 1,709) | Accuracy = 90%, sensitivity = 85%, specificity = 94% | |
| RF, Multilayer perceptron, Sequential minimal optimization | 605 compounds (training) +1,566 compounds (validation) | Accuracy = 86.5% | |
| SVM (RBF) | 1,978 compounds (BBB permeable = 1,550, BBB non-permeable = 440) | Accuracy = 96.77%, AUC = 0.964, F1 = 0.975 | |
| DNN | 3,605 compounds (BBB permeable = 2,704 BBB non-permeable = 901) | Accuracy = 98.07%, AUC = 0.992, AP = 0.997, F1 = 0.987 | Current study |
Comparative analysis of DeePred-BBB with recently published BBB permeability prediction models.
AUC, area under curve; AUPRC, area under precision-recall curve; AP, average precision; BBB, blood brain barrier; DL, deep learning; F1, F1 score; kNN, k-nearest neighbor; MCC, Matthews correlation coefficient; RBF, radial basis function; RF, random forest; RNN, recurrent neural network; SVM, support vector machine.
Conclusion
Deep learning and machine learning algorithms were applied to a dataset of 3,605 compounds to develop a prediction model that could accurately predict the BBB permeability of compounds using their SMILES notations as input. The comparative analysis of the performance metrics of the developed models suggested that the overall performance of DNN-based BBB permeability prediction is better than that of the ML and CNN models. It was discovered that the notion of “deeper the network, better the accuracy” does not often hold true. An optimal depth of the network is required beyond which the performance of the network does not improve. A DNN model with three layers (depth) having 200, 100, and 2 nodes each was the most accurate. It was also observed that in the case of compounds, the physicochemical properties and fingerprint-based DL models yield slightly better performance than 2D-structure image-based models in BBB permeability prediction.
Based on this study, we propose the DeePred-BBB model for BBB permeability prediction of compounds using their SMILES notations as input. In DeePred-BBB, the best performing DNN model is integrated with the open-source PaDel tool to calculate features. The calculated features are automatically fed to the DNN model as input, which predicts whether the compound will be BBB permeable or non-permeable. DeePred-BBB could assist in making quality decisions regarding which compound to carry forward in subsequent drug development stages and could potentially help in reducing the attrition rate of CNS-acting drug candidates failing due to BBB non-permeability. Inevitably, such drug candidates need further in vivo validation to arrive at efficacious and safe drugs at a faster rate and lower cost. DeePred-BBB could be accessed at https://github.com/12rajnish/DeePred-BBB.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Statements
Data availability statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.
Author contributions
RK and AS access to all of the data analyzed in this study, drafted the manuscript, and performed the statistical analysis. RK takes responsibility for the integrity and accuracy of the study data analysis and results. RK, AS, AA and GA involved in the study design, concept, analysis, and interpretation of data. AB, MK, AA, and GA involved in critical revision of the manuscript. All authors contributed to the article and approved the submitted version.
Funding
The Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, Saudi Arabia, has funded this project under Grant No. KEP-36-130-42. The authors therefore acknowledge with thanks DSR’s technical and financial support.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2022.858126/full#supplementary-material
Footnotes
References
1
AbbottN. J.RonnbackL.HanssonE. (2006). Astrocyte-endothelial interactions at the blood-brain barrier.Nat. Rev. Neurosci.741–53. 10.1038/nrn1824
2
AldewachiH.Al-ZidanR. N.ConnerM. T.SalmanM. M. (2021). High-throughput Screening Platforms in the discovery of novel drugs for neurodegenerative diseases.Bioengineering8:30. 10.3390/bioengineering8020030
3
AlsenanS.Al-TuraikiI.HafezA. (2020). A Recurrent neural network model to predict blood-brain barrier permeability.Comput. Biol. Chem.89:107377. 10.1016/j.compbiolchem.2020.107377
4
BaldiP.SadowskiP. (2014). The dropout learning algorithm.Artif. Intell.21078–122. 10.1016/j.artint.2014.02.004
5
BanksW. A. (2009). Characteristics of compounds that cross the blood-brain barrier.BMC Neurol.9:S3. 10.1186/1471-2377-9-S1-S3
6
BenetL. Z.HoseyC. M.UrsuO.OpreaT. I. (2016). BDDCS, the rule of 5 and drugability.Adv. Drug Deliv. Rev.10189–98. 10.1016/j.addr.2016.05.007
7
Ben-HurA.OngC. S.SonnenburgS.SchölkopfB.RätschG. (2008). Support vector machines and kernels for computational biology.PLoS Comput. Biol.4:e1000173. 10.1371/journal.pcbi.1000173
8
BentoA. P.HerseyA.FélixE.LandrumG.GaultonA.AtkinsonF.et al (2020). An open source chemical structure curation pipeline using rdkit. j.Cheminform.12:51. 10.1186/s13321-020-00456-1
9
BickelU. (2005). How to measure drug transport across the blood-brain barrier.NeuroRx215–26. 10.1602/neurorx.2.1.15
10
Castillo-GaritJ. A.Casanola-MartinG. M.Le-Thi-ThuH.Pham-TheH.BarigyeS. J. (2017). A Simple method to predict blood-brain barrier permeability of drug- like compounds using classification trees.Med. Chem.13664–669. 10.2174/1573406413666170209124302
11
DanemanR. (2015). Prat A. the blood-brain barrier.Cold Spring Harb. Perspect. Biol.7:a020412. 10.1101/cshperspect.a020412
12
DieterichH. J.ReutershanJ.FelbingerT. W.EltzschigH. K. (2003). Penetration of intravenous hydroxyethyl starch into the cerebrospinal fluid in patients with impaired blood-brain barrier function.Anesth. Analg.961150–1154. 10.1213/01.ane.0000050771.72895.66
13
DonigerS.HofmannT.YehJ. (2002). Predicting CNS permeability of drug molecules: comparison of neural network and support vector machine algorithms.J. Comput. Biol.9849–864. 10.1089/10665270260518317
14
D’souzaR. N.HuangP. Y.YehF. C. (2020). Structural analysis and optimization of convolutional neural networks with a small sample size.Sci. Rep.10:834. 10.1038/s41598-020-57866-2
15
EstevaA.KuprelB.NovoaR. A.KoJ.SwetterS. M.BlauH. M.et al (2017). Dermatologist-level classification of skin cancer with deep neural networks.Nature542115–118. 10.1038/nature21056
16
FeiginV. L.VosT.NicholsE.OwolabiM. O.CarrollW. M.DichgansM.et al (2020). The global burden of neurological disorders: translating evidence into policy.Lancet Neurol.19255–265. 10.1016/S1474-4422(19)30411-9
17
GaudilloJ.RodriguezJ. J. R.NazarenoA.BaltazarL. R.VilelaJ.BulalacaoR.et al (2019). Machine learning approach to single nucleotide polymorphism-based asthma prediction.PLoS One14:e0225574. 10.1371/journal.pone.0225574
18
GhandiM.LeeD.Mohammad-NooriM.BeerM. A. (2014). Enhanced regulatory sequence prediction using gapped k-mer features.PLoS Comput. Biol.10:e1003711. 10.1371/journal.pcbi.1003711
19
GiulianiA. (2017). The application of principal component analysis to drug discovery and biomedical data.Drug Discov. Today221069–1076. 10.1016/j.drudis.2017.01.005
20
GuptaM.LeeH. J.BardenC. J.WeaverD. F. (2019). The blood-brain barrier (BBB) Score.J. Med. Chem.629824–9836. 10.1021/acs.jmedchem.9b01220
21
HendricksB. K.Cohen-GadolA. A.MillerJ. C. (2015). Novel delivery methods bypassing the blood-brain and blood-tumor barriers.Neurosurg. Focus38:E10. 10.3171/2015.1.FOCUS14767
22
HuY.ZhaoT.ZhangN.ZhangY.ChengL. (2019). A Review of recent advances and research on drug target identification methods.Curr. Drug Metab.20209–216. 10.2174/1389200219666180925091851
23
IsenseeF.JaegerP. F.KohlS. A. A.PetersenJ.Maier-HeinK. H. (2021). NNU-NET: a self-configuring method for deep learning-based biomedical image segmentation.Nat. Methods18203–211. 10.1038/s41592-020-01008-z
24
JiangL.ChenJ.HeY.ZhangY.LiG. (2016). A method to predict different mechanisms for blood-brain barrier permeability of CNS activity compounds in Chinese herbs using support vector machine.J. Bioinform. Comput. Biol.14:1650005. 10.1142/S0219720016500050
25
KhanA. I.LuQ.DuD.LinY.DuttaP. (2018). Quantification of kinetic rate constants for transcytosis of polymeric nanoparticle through blood-brain barrier.Biochim. Biophys Acta Gen. Subj.18622779–2787. 10.1016/j.bbagen.2018.08.02016
26
KumarR.SharmaA.SiddiquiM. H.TiwariR. K. (2018). Promises of machine learning approaches in prediction of absorption of compounds.Mini. Rev. Med. Chem.18196–207. 10.2174/1389557517666170315150116
27
KumarR.SharmaA.VaradwajP.AhmadA.AshrafG. M. (2011). Classification of oral bioavailability of drugs by machine learning approaches: a comparative study.J. Comp. Interdisc. Sci.2179–196. 10.6062/jcis.2011.02.03.0045
28
LeCunY.BengioY.HintonG. (2015). Deep learning.Nature521436–444. 10.1038/nature14539
29
LovrićM.MoleroJ.M.KernR.SparkP. Y.KitR. D. (2019). Moving towards big data in cheminformatics.Mol. Inform.38:e1800082. 10.1002/minf.201800082
30
MainB. S.VillapolS.SloleyS. S.BartonD. J.ParsadanianM.AgbaegbuC.et al (2018). Apolipoprotein E4 impairs spontaneous blood brain barrier repair following traumatic brain injury.Mol. Neurodegener13:17. 10.1186/s13024-018-0249-5
31
MalikJ.KiranyazS.GabboujM. (2021). Self-organized operational neural networks for severe image restoration problems.Neural. Netw.135201–211. 10.1016/j.neunet.2020.12.014
32
MasseyS. C.UrcuyoJ. C.MarinB. M.SarkariaJ. N.SwansonK. R. (2020). Quantifying glioblastoma drug response dynamics incorporating treatment sensitivity and blood brain barrier penetrance from experimental data.Front. Physiol.11:830. 10.3389/fphys.2020.00830
33
MenkenM.MunsatT. L.TooleJ. F. (2000). The global burden of disease study: implications for neurology.Arch. Neurol.57418–420. 10.1001/archneur.57.3.418
34
MiY.MaoY.ChengH.KeG.LiuM.FangC.et al (2020). Studies of blood-brain barrier permeability of gastrodigenin in vitro and in vivo.Fitoterapia140104447. 10.1016/j.fitote.2019.104447
35
MiaoR.XiaL. Y.ChenH. H.HuangH. H.LiangY. (2019). Improved classification of blood-brain-barrier drugs using deep learning.Sci. Rep.9:8802. 10.1038/s41598-019-44773-4
36
PardridgeW. M. (2002). Why is the global CNS pharmaceutical market so under-penetrated? Drug discov.Today75–7. 10.1016/s1359-6446(01)02082-7
37
PardridgeW. M. (2005). The blood-brain barrier: bottleneck in brain drug development.NeuroRx23–14. 10.1602/neurorx.2.1.3
38
ReeseT. S.KarnovskyM. J. (1967). Fine structural localization of a blood-brain barrier to exogenous peroxidase.J. Cell. Biol.34207–217. 10.1083/jcb.34.1.207
39
RheaE. M.BanksW. A. (2019). Role of the Blood-Brain Barrier in Central Nervous System Insulin Resistance.Front. Neurosci.13:521. 10.3389/fnins.2019.00521
40
RoyD.HingeV. K.KovalenkoA. (2019). To Pass or Not To Pass: Predicting the Blood-Brain Barrier Permeability with the 3D-RISM-KH Molecular Solvation Theory.ACS Omega.416774–16780. 10.1021/acsomega.9b01512
41
SalmanM. M.Al-ObaidiZ.KitchenP.LoretoA.BillR. M.Wade-MartinsR. (2021). Advances in Applying Computer-Aided Drug Design for Neurodegenerative Diseases.Int. J. Mol. Sci.22:4688. 10.3390/ijms22094688
42
SalmanM. M.KitchenP.YoolA. J.BillR. M. (2022). Recent breakthroughs and future directions in drugging aquaporins.Trends Pharmacol. Sci.4330–42. 10.1016/j.tips.2021.10.009
43
SalmanM. M.MarshG.KustersI.DelincéM.Di CaprioG.UpadhyayulaS.et al (2020). Design and validation of a human brain endothelial microvessel-on-a-chip open microfluidic model enabling advanced optical imaging.Front. bioeng. biotechnol.8:573775. 10.3389/fbioe.2020.573775
44
SalmanpourM. R.ShamsaeiM.SaberiA.KlyuzhinI. S.TangJ.SossiV.et al (2020). Machine learning methods for optimal prediction of motor outcome in parkinson’s disease.. Phys. Med.69233–240. 10.1016/j.ejmp.2019.12.022
45
SaundersN. R.DreifussJ. J.DziegielewskaK. M.JohanssonP. A.HabgoodM. D.MøllgårdK.et al (2014). The rights and wrongs of blood-brain barrier permeability studies: a walk through 100 years of history.Front. Neurosci.8:404. 10.3389/fnins.2014.00404
46
SaxenaD.SharmaA.SiddiquiM. H.KumarR. (2019). Blood brain barrier permeability prediction using machine learning techniques: an update.Curr. Pharm. Biotechnol.201163–1171. 10.2174/1389201020666190821145346
47
SaxenaD.SharmaA.SiddiquiM. H.KumarR. (2021). Development of machine Learning based blood-brain barrier permeability prediction models using physicochemical properties, maccs and substructure fingerprints.Curr. Bioinform.16855–864. 10.2174/1574893616666210203104013
48
SchidlowskiM.BolandM.RüberT.StöckerT. (2020). Blood-brain barrier permeability measurement by biexponentially modeling whole-brain arterial spin labeling data with multiple T2 -weightings.NMR Biomed.33:e4374. 10.1002/nbm.4374
49
SchlageterK. E.MolnarP.LapinG. D.GroothuisD. R. (1999). Microvessel organization and structure in experimental brain tumors: microvessel populations with distinctive structural and functional properties.Microvasc. Res.58312–328. 10.1006/mvre.1999.2188
50
ShakerB.YuM. S.SongJ. S.AhnS.RyuJ. Y.OhK. S.et al (2020). LightBBB: Computational prediction model of blood-brain-barrier penetration based on LightGBM.Bioinformatics371135–1139. 10.1093/bioinformatics/btaa918
51
ShanW.LiX.YaoH.LinK. (2021). Convolutional neural network-based virtual screening.Curr. Med. Chem.282033–2047. 10.2174/0929867327666200526142958
52
SharmaA.KumarR.RanjtaS.VaradwajP. K. (2021). SMILES to Smell: Decoding the structure-odor relationship of chemical compounds using the deep neural network approach.J. Chem. Inf. Model61676–688. 10.1021/acs.jcim.0c01288
53
ShenJ.ChengF.XuY.LiW.TangY. (2010). Estimation of ADME properties with substructure pattern recognition.J. Chem. Inf. Model501034–1041. 10.1021/ci100104j
54
ShenY.LiY.ZhengH. T.TangB.YangM. (2019). Enhancing ontology-driven diagnostic reasoning with a symptom-dependency-aware naïve bayes classifier.BMC Bioinform20:330. 10.1186/s12859-019-2924-0
55
SinghM.DivakaranR.KondaL. S. K.KristamR. (2020). A classification model for blood brain barrier penetration.J. Mol. Graph. Model.96:107516. 10.1016/j.jmgm.2019.107516
56
SuenderhaufC.HammannF.HuwylerJ. (2012). Computational prediction of blood-brain barrier permeability using decision tree induction.Molecules1710429–10445. 10.3390/molecules170910429
57
TietzS.EngelhardtS. B. (2015). Brain barriers: crosstalk between complex tight junctions and adherens junctions.J. Cell. Biol.209493–506. 10.1083/jcb.201412147
58
van TellingenO.Yetkin-ArikB.de GooijerM. C.WesselingP.WurdingerT.de VriesH. E. (2015). Overcoming the blood-brain tumor barrier for effective glioblastoma treatment.Drug Resist. Updat.191–12. 10.1016/j.drup.2015.02.002
59
VeberD. F.JohnsonS. R.ChengH. Y.SmithB. R.WardK. W.KoppleK. D. (2002). Molecular properties that influence the oral bioavailability of drug candidates.J.Med. Chem.452615–2623. 10.1021/jm020017n
60
WangD.ZengJ.LinS. B. (2021b). Random Sketching for neural networks with relu.IEEE Trans Neural. Netw. Learn. Syst.32748–762. 10.1109/TNNLS.2020.2979228
61
WangM. W. H.GoodmanJ. M.AllenT. E. H. (2021a). Machine learning in predictive toxicology: recent applications and future directions for classification models.Chem. Res. Toxicol.34217–239. 10.1021/acs.chemrestox.0c00316
62
WangZ.YangH.WuZ.WangT.LiW.TangY.et al (2018). In silico prediction of blood-brain barrier permeability of compounds by machine learning and resampling methods.Chem. Med. Chem.132189–2201. 10.1002/cmdc.201800533
63
WeversN. R.KasiD. G.GrayT.WilschutK. J.SmithB.van VughtR.et al (2018). A perfused human blood-brain barrier on-a-chip for high-throughput assessment of barrier function and antibody transport.Fluids barriers CNS15:23. 10.1186/s12987-018-0108-3
64
YangL.WuH.JinX.ZhengP.HuS.XuX.et al (2020). Study of cardiovascular disease prediction model based on random forest in eastern China.Sci. Rep.10:5245. 10.1038/s41598-020-62133-5
65
YaoD.ZhanX.ZhanX.KwohC. K.LiP.WangJ. (2020). A random forest based computational model for predicting novel lncrna-disease associations.BMC Bioinform21:126. 10.1186/s12859-020-3458-1
66
YapC. W. (2011). PADEL-descriptor: an open source software to calculate molecular descriptors and fingerprints.J. Comput. Chem.321466–1474. 10.1002/jcc.21707
67
YuanY.ZhengF.ZhanC. G. (2018). Improved prediction of blood-brain barrier permeability through machine learning with combined use of molecular property-based descriptors and fingerprints.AAPS J.20:54. 10.1208/s12248-018-0215-8
68
ZhangD.XiaoJ.ZhouN.ZhengM.LuoX.JiangH.et al (2015). A genetic algorithm based support vector machine model for blood-brain barrier penetration prediction.Biomed. Res. Int.2015:292683. 10.1155/2015/292683
69
ZhangL.ZhuH.OpreaT. I.GolbraikhA.TropshaA. (2008). QSAR 21odelling of the blood-brain barrier permeability for diverse organic compounds.Pharm. Res.251902–1914. 10.1007/s11095-008-9609-0
70
ZhaoY. H.AbrahamM. H.IbrahimA.FishP. V.ColeS.LewisM. L.et al (2007). Predicting penetration across the blood-brain barrier from simple descriptors and fragmentation schemes.J. Chem. Inf. Model.47170–175. 10.1021/ci600312d
Summary
Keywords
blood-brain barrier, convolutional neural network, deep learning, machine learning, prediction, CNS-permeability
Citation
Kumar R, Sharma A, Alexiou A, Bilgrami AL, Kamal MA and Ashraf GM (2022) DeePred-BBB: A Blood Brain Barrier Permeability Prediction Model With Improved Accuracy. Front. Neurosci. 16:858126. doi: 10.3389/fnins.2022.858126
Received
19 January 2022
Accepted
14 March 2022
Published
03 May 2022
Volume
16 - 2022
Edited by
Corinne Lasmezas, The Scripps Research Institute, United States
Reviewed by
Sezen Vatansever, Icahn School of Medicine at Mount Sinai, United States; Mootaz M. Salman, University of Oxford, United Kingdom
Updates
Copyright
© 2022 Kumar, Sharma, Alexiou, Bilgrami, Kamal and Ashraf.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ghulam Md Ashraf, ashraf.gm@gmail.com, gashraf@kau.edu.sa
This article was submitted to Neurodegeneration, a section of the journal Frontiers in Neuroscience
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.