As the biological common sense, RNA contains two categories of mRNA and ncRNA. The ncRNA includes long non-coding RNA, which is longer than 200 nt, and small ncRNA, like miRNA and snoRNA and there are different biological functions among them (Pan et al., 2016). To demonstrate the robustness and stability of SAWRPI, different RNA-protein interactions benchmark datasets are used to validate, which including mRNA-protein and lncRNA-protein datasets. In practice, dataset RPI488 (Pan et al., 2016)and RPI369 (Muppiral et al., 2011), RPI1807 (Suresh et al., 2015) were chosen to evaluate. The first one is lncRNA-protein dataset, while the last two datasets stand for mRNA-protein. RPI488 is a non-redundant dataset of lncRNA-protein interactions, containing 245 negative samples and 243 positive samples among 25 lncRNAs and 247 proteins (Huang et al., 2010; Puton et al., 2012). Dataset RPI369 also is non-redundant with 332 RNA chains and 338 protein chains, generated from RPIDB (Lewis et al., 2010), a comprehensive database calculated from PDB (Berman et al., 2000), and has no ribosomal protein or ribosomal RNAs. It contains a total of 369 positive interactive pairs. RPI1807, a non-redundant dataset, generated by NDB (Lu et al., 2013), includes 1,078 RNAs and 1807 proteins, and then consist 1807 pairwise positive samples and 1,436 pairwise negative samples. Table 1 illustrates details of these three benchmark datasets.

In this study, to predict ncRNA-protein interactions, we developed a computational method SAWRPI. Due to the difference of structure between ncRNA and protein, we extracted sequence information of two entities through different ways. For proteins, extracting conjoint triad (3-mers) from 7 groups of amino acids and generating 3-mers sparse matrix. Immediately, SVD is utilized to reduce the sparse matrix into a vector, which is seen as raw features. For ncRNA, word embedding method is used to extract raw representation of ncRNA symbol with the local fusion strategy (LFS). Before predicting through the classification strategy, Hilbert Transformation (HT) is used to further extract information of raw features. Finally, making prediction through the classifier with our strategy of stacking ensemble with adaptive weight initialization. Figure 1 deploys the detail of this process.

To preliminarily obtain raw features, for each protein sequence, 20 amino acids are partitioned into 7 groups (Pan et al., 2010), “AGV”, “TMTS”, “ILFP”, “HNQW”, “DE”, “RK” and “C”, based on the dipole moments and side chain volume. Protein sequence with length of n, can be expressed using only seven symbols, and under sequence dividing into n-(k-1) subsequences, there are 7 ^k different possible k-mer. Then the k is set to 3 which is commonly accepted as empirical parameter (Shen et al., 2007; Yi et al., 2018). As Table 2 shown, the features of conjoint triad p _j p _j+1 p _j+2 based on the seven groups for each protein can be extracted as a sparse matrix L _p with the dimension of 7 ^k ×(n-(k-1)) (You et al., 2016), which can be defined as follows: L p = ( a i j ) , i ∈ [ 0 , 7 k − 1 ] , j ∈ [ 0 , ( n − ( k − 1 ) ) ] (1) a i j = { 1 ,      if   p j p j + 1 p j + 2   =   k - m e r ( i ) 0 ,     else (2)

Furthermore, the SVD is used to extract the vector with dimension of 7 ^k ×1 from sparse matrix L _p . While, for each ncRNA sequence with length of m, k-mer composition is also used to divide them into m-(k-1) subsequences and the semantic information is utilized, which is different from the treatment processes of protein sequences. Each ncRNA can be considered as “sentence” and the subsequences (e.g., AAA, AAC, … , UUU) can be seen as “word”. Word embedding techniques have demonstrated the promise in natural language processing applications. Therefore, we used this technique to encode each subsequence. Specifically, features of global word co-occurrence probability are extracted through model of GloVe (Pennington et al., 2014), the details following the next section. Each “word” can be expressed as a feature vector, and each sentence with length of m-(k-1) are expressed as a feature matrix with dimension of d×(m-(k-1)), where d stands for dimension of embedding and is set to 32 in this experiment.

For long non-code RNA, there are more than 200-(k-1) words to be embedded. The count of feature factors is a tremendous overwhelming number. To solve it, many methods select the way of directly truncate, which is helpful but may loss many information of sequence (You et al., 2018; Chen et al., 2019; Yi H.-C. et al, 2020). Inspired by. Zeng et al. (2021) and motivated by spatial pyramid pooling-net (He et al., 2015), we proposed a novel local fusion strategy named LFS to fully explore the evolutionary features that after subsequence embedding, as Figure 2 shown, an average pooling layer is used to produce the patterns of the subsequence, and then combining all the pattern to a vector with certain dimension. Notably, if the length of RNA is too short to satisfy the setting dimension, zero will be filled. Finally, the raw feature vectors of each ncRNA and protein sequence can be extracted. And we set the number of groups as 11.

One reason of deep learning technology developing rapidly is remarkably disposing of corpora in various fields. There are now many natural language processing methods and word embedding methods having been adopted, like iDeepSubMito (Hou et al., 2021), iCircRBP-DHN(Yang et al., 2021), Latent Semantic Analysis (LSA) (Dumais 2004), word2vec (Mikolov, et al., 2013; Mikolov, et al., 2013) and Global Vectors for Word Representation (GloVe) (Pennington et al., 2014). While in this paper, we exploit the model of GloVe to learning the embedding vectors of ncRNA “words”.

The model of GloVe can overcome the drawback of first two embedding methods mentioned previously that the high computational burden and utilization of partial corpus. It produces a word vector space, which has meaningful substructure, based on making full use of the information of global word-word co-occurrence. In detail, implementation of the GloVe is in a three-steps procedure. Firstly, constructing a co-occurrence matrix X based on ncRNA “word” corpus. Each co-occurrence matrix element p _ij stands for probability of co-occurrence rather than count of co-occurrence, following the formula: p i j = P ( j | i ) = x i j x i (3) where x _ij represents for the appearing number of word j in the context environment of word i, and x _i stands for the total appearing number of all word in the context environment of the word i. Then, generating the word vector to construct approximation relationship with the co-ocurrence matrix through the function as follows. ω i ⊤ ω ˜ j + b i + b ˜ j = log ( x i j ) (4) where ω i and ω ˜ j respectively mean the embedding vectors of word i and word j, while b i and b ˜ j respectively mean bias terms. In the end, obtaining and minimizing the loss function: J = ∑ i , j = 1 V f ( x i j ) ( ω i ⊤ ω ˜ j + b i + b ˜ j − log ( x i j ) ) 2 (5) f ( x ) = { ( x / x max ) α   if  x < x max 1                 otherwise (6) where the f ( ⋅ ) is a weight function used to make the value of appearing number between the words rarely appearing much lower. In the experiment, we set embedding dimension as 32. After splitting nucleic acids sequences into 3-mers, each “words” can be indicated as a vector.

To fully exploit sequence information, we further extract information from raw features. Hilbert transform (Johansson 1999) is used to generate features easily analyzing based on the raw features of ncRNA and protein. Hilbert transformation is usually used to analyze signal in the time and frequency, which acts as a 90° phase shifter without changing energy and amplitude, phase-shifting −90° to part of positive frequency, while phase-shifting 90° to part of negative frequency, and it can also be used as a tool of features extracting in the field of biology (Pan et al., 2021). The transformation function can be defined as: x ^ ( t ) = x ( t ) 1 π t = 1 π ∫ − ∞ ∞ x ( τ ) t − τ d τ = − 1 τ ∫ − ∞ ∞ x ( t + τ ) τ d τ (7) where x(t) is seen as each feature vectors. And the back-transformation is defined as: x ( t ) = − x ^ ( t ) 1 π t = − 1 π ∫ − ∞ ∞ x ^ ( τ ) t − τ d τ = 1 τ ∫ − ∞ ∞ x ^ ( t + τ ) τ d τ (8)

Specifically, in this work, we respectively used model of SVD and GloVe to obtain the raw feature of protein and ncRNA. Then each protein and ncRNA is encoded as vectors with dimension of 7 × 7 × 7 and dimension of 11 × 32. Finally, after the processing of Hilbert transforming, hidden high-level features can be extracted.

In this work, four kinds of machine learning base classifiers are utilized to integrate, including XGBoost (Chen and Guestrin 2016), SVM (Cortes and Vapnik 1995; Chang and Lin 2011), ExtraTree (Geurts et al., 2006) and Random Forest (Breiman 2001). SVM is used for classification, regression or other work, through constructing one or multiple hyperplanes in a high-dimension space. Intuitively, a decent segmentation using the hyperplane can maximize the distance of function margins (points of training data) in any class. It is usually used in high dimension space with high-performance, although the sample size is lower than data dimension. However, if the number of samples is much lower than the number of the data features, SVM may overfitting and need to select efficient kernel to avoid.

Supposing the training dataset with label [(x _i , y _i ), i = 0, 1, … , n, y _i = (1, -1), x _i ∈ R] and regarding (w(x)+b) = 0 as a separating hyperplane. In the linear separable problems, to maximize the margin, SVM minimizes subject of ||w||²/2 to find the separation hyperplane through the constraint: y i ( w x i + b ) ≥ 1 , ∀ x i (9) And in the linear non-separable problems, slack variables are introduced to look for the optimal separating hyperplane, then minimizing the function: | | w | | 2 / 2 + C ∑ i = 1 n ξ i , ξ i ≥ 0 , ∀ x i (10) y i ( w x i + b ) ≥ 1 − ξ i , ξ i ≥ 0 , ∀ x i (11) where C is user-adjustable parameter. Kernel of Radial Basis Function (RBF) is adopted, which is defined as: f ( x ) = e − γ | | x − x ' | | 2 (12)

XGBoost, a model of end-to-end tree boosting, can perceive sparsity data well called sparsity-aware. To control complexity of the model, XGBoost adds a regularization term to cost function, which can reduce the variance of the model as well as prevent situation of overfitting, and then performs second-order Taylor expansion. For a larger learning space, XGBoost diminishes the impact of each tree through multiplying the weight of leaf nodes. Its objective function is defined as follows. O b j = ∑ i = 1 n l ( y i , y ^ i ) + ∑ k = 1 K Ω ( f k ) (13) Ω ( f t ) = γ T + λ 2 ∑ j = 1 T w j 2 (14) where l is used to compute difference between target y i and prediction y ^ i . Then, Ω ( ⋅ ) stands for regular term containing T, count of leaf nodes, and the sum of l ₂ modulus square of score on each leaf. XGBoost supports column sampling and draws on the method of Random Forest, which can avoid over-fitting and save computation resources.

Random Forest is a representative ensemble classification algorithm, which is based on the decision tree evaluator to introduce randomness features selection into the process of decision tree training. Specifically, it uses multiple decision tree to reduce variance of output. For each node of decision tree, randomly selecting a subset containing K features from the node features set, and then optimal features can be selected from subset to split. The K is used to control degree of randomness. Supposing the label sets is { c 1 , c 2 , ... , c N } and the prediction of ith base classifier on the sample is ( h i 1 ( x ) , h i 2 ( x ) , ... , h i N ( x ) ) ⊤ . For integrating results of each base classifier, majority voting and averaging methods are often used, which are respectively defined as: H ( x ) = { c j ,           ∑ i = 1 T h i j ( x ) > 0.5 + ∑ k = 1 N ∑ i = 1 T h i k ( x ) r e j e c t ,     o t h e r w i s e (15) H ( x ) = 1 T ∑ i = 1 T w i h i ( x ) (16) where w _i is weight of ith base classifier. Extremely randomized tree (ExtraTree) is on the basis of random forest to further random on splitting threshold. And extremely randomized tree essentially builds totally randomized trees, which selects attribute and cut-point with strongly randomizing when it splits a tree node. Tree structure is independent of the output value. It can further enhance randomness of segmentation points that choosing suitable parameter according specific task. Under the segmentation rule, selecting the best threshold for each candidate feature from these randomly generated thresholds.

And all the parameters were set as follows. The sklearn tool was used in this paper to training four models. For the parameters of XGBoost, we set max_depth = 6 and booster = 'gblinear’. The kernel of ‘rbf’ is set for SVM model. There are four parameters to Random Forest model, criterion = 'gini’, n_estimators = 25, random_state = 1 and n_jobs = 2. Model of ExtraTree uses default parameters.

Ensemble learning method accomplished learning task through constructing and combining multiple evaluators rather than one learning machine, which considers multiple results of each evaluator and integrates into a comprehensive result. In most situations, multiple evaluators are better than single evaluators in performance of classification and regression task.

Generally, different performances are present in different classifiers (evaluators). And how to efficiently integrate different classifiers to generate the target function is so crucial. Previously, there are many studies of integrating multiple classifiers, containing majority voting (Breiman 2001), averaging results of each base model (Pan et al., 2011) and stacked ensemble method (Töscher et al., 2009). Majority voting and averaging has been detailed previously. While, stacked ensembling follows the intuition of the deep neural network, uniting with encoder layer and successive decoder layer. Specifically, the level 0 classifiers, regarded as encoder layer, firstly generate prediction probability score, and then, the level 1 classifier integrate results from single classifier through logistic regression. Figure 3 shows the detail as follows.

In the encoding layer with cth base classifier, the training set Tr will be split divided into four equal fractions Tr _i and encoded in four runs. In ith run, training sub-set of Tr _i is encoded by the sub-encoder learning from the rest of the training sub-sets through cth base classifier, and the testing set Te also is encoded as a vector of te _i ^c . After four iterations, with cth classifier, the training set Tr can be expressed in tr ^c , and the testing set Te can be expressed in te ^c through the function as follows: t e c = 1 N ∑ i = 1 N t e i c (17) where N means the number of base classifiers. Through all of the base classifiers, encoding matrix of Tr and Te can be generated, whose rows stand for encoding vectors of all the samples. Then, level 1 layer of logistic regression satisfies the following equations: P w ( y = ± 1 | x ) = 1 1 + e − y w ⊤ x (18) where x is encoding vector, and w is learning weight vector for each classifier. When w is same constant for each classifier, it is equivalent to strategy of averaging, however, if only one element of is non-zero, it is like strategy of majority voting.

In this work, we provided a strategy of adaptive weight initialization through initialization parameter λ ^c for cth classifier which is defined as follows. λ c = − 1 ( 1 − 1 ( w c ) 2 ) N (19) w c = 1 N ∑ i = 1 N w i c (20) where w _i ^c stands for the AUC score of Tr _i prediction with cth classifier in each run mentioned above. The aim of arising parameter λ ^c is making the importance of weaker classifier to reduce before feeding the vectors to decoder layer to improve performance by fine-tuning. Thus, Tr and Te can be expressed in λ ^c ×tr ^c and λ ^c ×te ^c respectively with cth classifier.

In this article, the performance of SAWRPI is evaluated by five-fold cross validation. And each validation makes full use of the frequently utilized metrics to assess robustness and effectiveness of the proposed method. Including Accuracy (Acc.), Sensitivity (Sen.), Precision (Prec.), F1 (Macro F1) and MCC (Matthews’s Correlation Coefficient). These evaluation indicators can be represented as follows: A c c . = T P + T N T N + T P + F N + F P (21) P r e c . = T P T P + F P (22) S e n . = T P T P + F N (23) F 1 = 2 × P r e c . × S e n . P r e c . + S e n . (24) M C C = T P × T N − F P × F N ( T P + F P ) × ( T N + F N ) × ( T N + F P ) × ( T P + F N ) (25) where TP and FN are treated as the number of positive samples which are correctly predicted as positive and incorrectly predicted as negative, respectively, then TN and FP respectively stand for the number of negative samples which are correctly detected as negative and incorrectly detected as positive. Apart from the above indicators, AUC, the area under the ROC curves, is constructed to evaluate our model. The mean value of the results of five validation is used to ensure low-variance and unbiased evaluations.

In this work, to demonstrate performance and robustness of SAWRPI, three datasets, indicating two kinds of ncRNA-protein interactions, have been used to validate, including mRNA-protein and lncRNA-protein datasets. Furthermore, the five-fold cross-validation can enhance the persuasion of the predicting results. Specifically, dataset RPI369, RPI488 and RPI1807 is used to evaluate SAWRPI. Table 3 reveals the result of prediction. Certainly, the same experiments with the other classifiers are reported in Supplementary Material.

As the table shown, the average scores of Acc reach 0.710, 0.895, and 0.967 in all three datasets. When applying SAWRPI to RPI1807, we obtained the highest average score of Acc, Prec, Sen, F1 and MCC of 0.967, 0.961, 0.981, 0.971, and 0.934, with the standard deviation of 0.005, 0.006, 0.005, 0.004, and 0.011, respectively. On the dataset of RPI369, whose type of interaction is same to RPI1807, obtained average Acc, Prec, Sen, F1 and MCC of 0.710, 0.693, 0.756, 0.723 and 0.422, with the standard deviation of 0.023, 0.022, 0.034, 0.024 and 0.047, respectively. Comparing these results, it is easy to see that SAWRPI is more applicable to the dataset of RPI1807. Thus, the size of dataset can cause effect on prediction result. The other type dataset RPI488 reached average Acc, Prec, Sen, F1 and MCC of 0.895, 0.938, 0.844, 0.888 and 0.791, with the standard deviation of 0.024, 0.042, 0.036, 0.027 and 0.052, respectively. At the view of interaction type, our model may be more effective on the interaction type of lncRNA-protein. One reason may be that our method of representing ncRNA can capture more distal sequence information, which may bring some noise at the same time. Even then, it is undeniable that SAWRPI still achieved a fabulous capability of ncRNA-protein interactions prediction.

AUC, the area under ROC curve, is regarded as an important criterion for evaluating the performance of the classification model. To verify the superiority of our strategy of stacking ensemble with adaptive weight initialization, we compared it with two different integrating methods in the same features of ncRNA and protein. As Table 4 shown, our integrating strategy is more advantageous on dataset of RPI369 and RPI488, and competitive on dataset of RPI1807. The results of other evaluation parameters are reported in Supplementary Material.

Moreover, to reveal the improvement of stacking ensemble strategy, we also contrasted our strategy with the four classifiers, which are used as base predictors of our method. Integrating four base predictors through a Logistic Regression function automatically. As Table 5 illustrates, on the RPI369 dataset, SAWRPI obtained five the highest values of Acc, Prec, Sen, F1 and MCC of 0.710, 0.692, 0.756, 0.723 and 0.422, respectively. On the RPI488 dataset, SAWRPI got four the highest values of Acc, Prec, F1 and MCC of 0.895, 0.938, 0.888 and 0.791, respectively. On the RPI1807 dataset, SAWRPI obtained four the highest values of Acc, Sen, F1 and MCC of 0.967, 0.981, 0.971 and 0.934, respectively. Although the results of our method are not the best on each criterion, it still obtained comparable results which are only 0.004 and 0.005 lower than the best value, respectively. For further description of the model reliability, three ROC curves displayed following, shown by Figures 4–6. To verify that the results are truly significant, statistical learning method is used to plot boxplots, shown by Figure 7. Additionally, ROC curves figures of comparing all classifying strategies in three datasets and five-fold cross-validation results on three datasets by different classifying strategies are shown in Supplementary Material.

To illustrate the effectiveness of feature extraction method, HT was compared with some correlatively common methods, including Auto-covariance (AC) (Zeng et al., 2009) and Discrete Wavelet transform (DWT) (Nanni et al., 2012). As shown in Table 6, on the RPI369 and RPI1807 dataset, our method got the highest prediction values on all evaluation criteria of 0.710, 0.692, 0.756, 0.723, 0.422 and 0.746, and 0.967, 0.961, 0.981, 0.971, 0.934 and 0.992, respectively. And on the RPI488 dataset, our method obtained only 0.008 lower accuracy in term of Sen, comparing the highest value. Obviously, the performance of our feature extracting strategies is better than the others. To verify that the results are truly significant, statistical learning method is used to plot boxplots shown by Figure 8. Notably, the five-fold cross-validation results table and the ROC curve figures of each classification method mentioned above based on different feature extracting strategies are reported in the Supplementary Material.

Furthermore, in order to verify effectiveness and stability of SAWRPI, we compared SAWRPI with other state-of-the-art computational approaches in the same three datasets that RPI488, RPI369 and RPI 1807. The contrast methods include RPISeq-RF (Muppirala et al., 2011), lncPro(Lu et al., 2013), SDA-RF (Pan et al., 2016) and SDA-FT-RF (Pan et al., 2016), which are based on sequence information and similar to SAWRPI. The authors, proposing method of RPISeq-RF, also developed another method RPISeq-SVM to predict. We only used RPISeq-RF which has better performance as comparation. Comparison methods of SDA-RF and SDA-FT-RF respectively used stacked denoising autoencoder through RF classification and stacked denoising autoencoder with fine tuning through RF classification. Table 7 shows all of the results of comparison. Through comparing with any other methods, it can be indicated that a little better performance of our method with Acc of 0.710, Sen of 0.756, F1 of 0.723 and MCC of 0.422. For the RPI1807 dataset, SAWRPI also gives a good performance in Prec, Sen and F1 with 0.961, 0.987 and 0.971. On RPI369 and RPI1807 datasets, SAWRPI obtained acceptable performance and got the highest value in term of F1 with 0.723 and 0.971 respectively. For the lncRNA-protein interactions dataset RPI488, our method achieved significant dominance in the important parameter AUC with 0.922 and displayed the performance with the outstanding improvements of 0.025–0.015, 0.028–0.006, 0.051–0.029 and 0.021–0.013 against others in terms of Acc, Prec, MCC and AUC respectively. Proposed method got the highest result in multiple criteria on three datasets, and notably, the best results in terms of highest AUC were obtained on RPI488. This illustrates that our method has more obvious advantages in task of predicting lncRNA-protein interactions. Without a doubt, SAWRPI is a powerful method of predicting ncRNA-protein interactions.