Predicting HLA CD4 Immunogenicity in Human Populations

Abstract

Background:

Prediction of T cell immunogenicity is a topic of considerable interest, both in terms of basic understanding of the mechanisms of T cells responses and in terms of practical applications. HLA binding affinity is often used to predict T cell epitopes, since HLA binding affinity is a key requisite for human T cell immunogenicity. However, immunogenicity at the population it is complicated by the high level of variability of HLA molecules, potential other factors beyond HLA as well as the frequent lack of HLA typing data. To overcome those issues, we explored an alternative approach to identify the common characteristics able to distinguish immunogenic peptides from non-recognized peptides.

Methods:

Sets of dominant epitopes derived from peer-reviewed published papers were used in conjunction with negative peptides from the same experiments/donors to train neural networks and generate an “immunogenicity score.” We also compared the performance of the immunogenicity score with previously described method for immunogenicity prediction based on HLA class II binding at the population level.

Results:

The immunogenicity score was validated on a series of independent datasets derived from the published literature, representing 57 independent studies where immunogenicity in human populations was assessed by testing overlapping peptides spanning different antigens. Overall, these testing datasets corresponded to over 2,000 peptides and tested in over 1,600 different human donors. The 7-allele method prediction and the immunogenicity score were associated with similar performance [average area under the ROC curve (AUC) values of 0.703 and 0.702, respectively] while the combined methods reached an average AUC of 0.725. This increase in average AUC value is significant compared with the immunogenicity score (p = 0.0135) and a strong trend toward significance is observed when compared to the 7-allele method (p = 0.0938). The new immunogenicity score method is now freely available using CD4 T cell immunogenicity prediction tool on the Immune Epitope Database website (http://tools.iedb.org/CD4episcore).

Conclusion:

The new immunogenicity score predicts CD4 T cell immunogenicity at the population level starting from protein sequences and with no need for HLA typing. Its efficacy has been validated in the context of different antigen sources, ethnicities, and disparate techniques for epitope identification.

Introduction

The identification of T cell epitopes has an important implication in several immunological contexts spanning from vaccine design to diagnostics in cancer, allergies, and infectious diseases fields. Most of the epitope identification is currently performed using bioinformatics prediction systems aimed to identify T cell immunogenicity and also to dissect the mechanisms underlying development of T cell responses. Currently, the majority of the T cell prediction methods are based on prediction of HLA binding affinity, which is a key requisite for human T cell immunogenicity. However, there is a lack of effective strategies able to predict immunogenitcity at the population level, which is of particular importance when HLA typing data are not available. To overcome this issue, it is important to identify the common HLA binding affinity characteristics able to distinguish immunogenic peptides from non-recognized peptides. Two main classes of HLA molecules are important in the immunological context. Class I molecules presents epitopes to CD8 T cells, while class II molecules present epitopes to CD4 T cells. Prediction of HLA class I binding has reached high accuracy with area under the ROC curve (AUC) values greater than 0.9 (1–7), similarly HLA class II predictions have significantly improved in the most recent years reaching significant levels of accuracy (with AUC values in the range of 0.760–0.870) (8–10). However, HLA molecules are highly polymorphic and epitope prediction at the population level has to take into account this high level of heterogeneity.

We previously shown that in the case of HLA class I, focusing on 25–30 main HLA A and B allelic variants provides coverage of a large fraction of the general population (11). Similarly, in the case of HLA class II, about 40–50 allelic variants provide coverage of most frequent allelic variants (12). Prediction of HLA binding is usually performed with allele-specific algorithms, since binding motifs of different HLAs are rather diverse. However, in the case of HLA class II, it is also noted that a high degree of overlap exists between the epitope binding of different variants (13). Indeed, it was shown that the epitopes dominantly recognized are often capable of binding to many different HLA class II alleles. These epitopes (named promiscuous epitopes) account for 50% or more of the total responses at the population level (14).

The “7-allele method” was specifically optimized for prediction of HLA class II responses at the population level (15) based on the prediction of promiscuous epitopes. While this method is associated with significant predictive value, it is also expected that many of the peptides that are predicted or experimentally shown to bind HLA class II molecules may not induce T cell responses. This is because although HLA binding is necessary it is not sufficient by itself for T cell immunogenicity. Other factors such as antigen processing and the size of the TCR repertoire capable of recognizing any given MHC/epitope complex are key factors in ultimately determining immunogenicity (16–18).

In particular, it has been shown that the TCR repertoire is a key factor in shaping epitope immunodominance (19–23). In the case of HLA class I, different algorithms have been devised that evaluate a peptide sequence for the presence of certain amino acids, presumably interacting with TCRs, as a contributing factor to epitope’s intrinsic immunogenic potential (24–26).

In the present study, we evaluate an approach to predict HLA class II immunogenicity at the population level, regardless of specific HLA haplotype, by training neural networks (NNs) with well-characterized sets of immunogenic epitopes dominant in general human populations. This approach could thus probe not only the influence of HLA binding but also potentially detect factors beyond HLA class II binding that would be encoded in the primary sequence of potential epitopes.

Materials and Methods

Datasets

The datasets used for training were derived entirely from experimental data generated in our laboratory using congruent techniques as a mean to rely on tightly controlled datasets. In addition, we also utilized epitopes that were associated with positive tetramer data as part of the training, because tetramer data are regarded as “gold standard” of quality and specificity in analyzing T cell response. Conversely, the datasets used for validation were derived from scientific literature using a broad variety of techniques and antigens, and generated from different laboratories worldwide. This choice was made to ensure the robustness of the validation provided.

Training Dataset Assembly

We used 15-mer peptides derived from several datasets described in peer-reviewed articles or obtained by in-house studies following same experimental approach (Table 1). In some cases, the epitope sets were selected based on interim analysis and do not exactly match the final epitope lists in the published articles. The peptides were tested for immune recognition in cohorts of 5–150 donors by ELISPOT assays for one of the following cytokines: IFNγ, IL-5, IL-17, or IL-10. A full list of these epitopes is described in Table S1 in Supplementary Material. In total, 1,032 epitopes were selected as positives in this study. Negative peptides were selected from the same datasets listed in Table 1 following specific criteria: peptides should be negative in all tests, only peptides from proteins with at least one positive peptide recognized were included. In addition, any peptide tested more than once (due to several studies testing antigens/allergens from the same organism) giving opposite responses for the same donor was removed from the dataset. Overall, 5,739 negative peptides (Table S2 in Supplementary Material) were obtained. In some cases, set-specific adjustments in the criteria were necessary for technical reasons, as detailed below.

Mycobacterium Tuberculosis (TB) Antigens

We selected 65 previously known 15-mer epitopes identified from the vaccine candidate antigens and that captured 80% of the response (27–29).

Timothy Grass (TG) Known Allergens

Previous studies identified 20 epitopes that accounted for 79.5% of the total response to a set of TG-derived pollen antigens (Phl p allergens) in TG allergic individuals (14, 31, 32). Most of the datasets are composed by 15-mers as they were based on HLA class II binding prediction (15, 99). However, since some of those epitopes were not 15-mers, to compare those with the rest of the dataset longer epitopes were dissected into the composing 15-mers and each 15-mers belonging to the longer peptides has been classified as a positive, with the same process being used for negative peptides. In addition, 19 peptides were described to cover an NTGAp19 peptide pool, which were selected to encompass at least 40% of the total IL-5 response directed against all NTGA peptides screened (30).

House Dust Mite (HDM) Allergens

The peptide set included the 34 most dominant epitopes cumulatively accounting for 90% of the total allergen-specific response detected in our screen (32). Analogous to the TG set, longer regions were deconstructed into 15-mers, which yielded 52 peptides in total.

Cockroach (CR) Allergens

71 most dominant epitopes were selected based on total spot forming cells (SFC) values greater than 1,000 (33).

Dengue (DENV) Antigens

Peptides predicted to bind various frequent DRB1 alleles were tested in about 10 HLA-matched donors. The sets comprised 325 epitopes, positive in at least two donors with PBMC derived from normal blood donors from the Colombo (Sri Lanka) region that were seropositive for DENV antibodies and thus representative of natural infection (34). Negative peptides were those tested in at least 10 donors and found to be uniformly negative.

Erythropoietin

Tangri et al. screened overlapping peptides and reported nine epitopes recognized by at least 40% donors (35).

CRJ1 and CRJ2 Japanese Cedar Allergens

This set contained overlapping 15-mers spanning the CRJ1 and CRJ2 allergens (36). We selected 30 dominant epitopes based on average response magnitude of >100 SFC (sum of IL-5 and IFNγ) in either of two group of allergic donors: those who lived in Japan for extended periods of time and USA sensitized donors who had not lived in Japan. A total of 18 control negative peptides were derived from allergens CRJ1 and CRJ2 and selected based on a response frequency of one donor or less and an individual SFC response <100 SFC.

Mouse Allergens

Peptides derived from mouse allergens, largely selected by the 7-allele algorithm were tested in 22 donors (37). A total of 89 dominant epitopes were defined on the basis of total SFC >150 and recognized in at least two donors.

Novel House Dust Mite Antigens

The peptides screened were predicted with the 7-allele method from 96 HDM (novel and known) proteins in 20 HDM allergic donors (38). We selected the 106 more dominant epitopes, recognized in multiple donors and with an overall magnitude of >300 SFC total (accounting for about 50% of the total response).

Pertussis Vaccine Antigens

The peptide set was comprised of 16-mers overlapping by eight residues, spanning the entire sequence of the antigens. We selected the top 100 epitopes recognized in at least 4 of the 53 total donors analyzed, and accounting for approximately 75% of the total response (39).

Ragweed Allergens

This set included 16-mers overlapping by eight amino acids, spanning the entire sequence of the antigens (40). A total of 15 epitopes accounting for 75% of the total response was selected. If variants were present, the most common variant was selected.

Tetanus Toxoid (TT) Antigen

We selected a set of 28 epitopes, recognized in at least 2 out of 20 donors tested (41), and predicted by the 7-alleles method (15). As a control, we selected a set of 57 peptides, which were studied but not recognized, neither in the Immune Epitope Database (IEDB, www.iedb.org) nor in the study by Antunes et al. (41), and an additional set of 41 peptides that were not predicted by the method and also neither recognized in the study by Antunes et al. nor identified in the IEDB as positive human responses. In the case of the third set of 41 peptides, there were 261 15-mers in the Tetanus set. Among them 124 were predicted to be binders with predicted 7-allele median percentile rank ≤20.0. Out of the 137 non-predicted peptides, those with predicted 7-allele median percentile rank >40.0 were selected (67 peptides) for screening to be included in non-predicted AND non-epitope set. From this list, we eliminated peptides that were overlapping by more than five AA residues with any of the epitope (recognized in our study or annotated as positive in IEDB). The remaining 41 peptides were included in the set of “control peptides” that were not predicted and neither recognized in the Antunes et al. study nor identified as positive response in IEDB.

ZIKA Virus (ZIKV) Antigens

A set of 15-mer peptides spanning the entire sequence of the ZIKV proteome was tested with a 14 days re-stimulation protocol in 18 donors. A total of 48 epitopes were defined as being positives in at least two donors (Grifoni et al., unpublished).

Yellow Fever (YF) Antigens

The set of epitopes tested includes 94 previously described YF CD4 T cell epitopes with known HLA class II restriction (IEDB) and sets of peptides predicted to bind different HLA DRB1 molecules. CD4+ T cells from 42 donors vaccinated with YF17 vaccine were co-cultured with autologous antigen-presenting cells and HLA-matched YF DRB1 predicted peptides. After 14 days, IFNγ response against individual peptides was determined as previously described (100). Epitopes were defined as peptides eliciting an SFC of 664 SFC/10⁶ or more. This resulted in the identification of 42 unique peptides (Weiskopf et al., unpublished).

IEDB Validation Datasets

To generate additional datasets to evaluate the performance of the various predictive schema, we sought to identify literature records reporting overlapping peptide studies. Accordingly, we queried the IEDB for papers which contained both positive and negatives records curated in the paper, related to HLA class II restricted T cells. This query identified 870 papers; which were further refined by filtering by “overlapping” mentioned in the abstract, resulting in 183 records.

The abstracts of those records were manually inspected, to select papers truly related to study of immunogenicity of overlapping peptide sets. At this stage, we excluded records relating to Phl p, TT, TB (already represented in the previous sets) and studies based on transgenic mice to obtain 102 relevant papers.

We next removed papers where the peptide size was less than 15, or where less than 10 donors were studied (resulting in 82 papers). Each of these 82 papers was manually inspected and additional papers were discarded upon manual inspection for a variety of reasons, including the paper not reporting testing for full sets of overlapping peptides, ambiguous reporting of negative results or peptide size tested, no clear discrimination between positive and negative responses, testing pools of peptides with no deconvolution, and similar problems.

This resulted in a final selection of 57 papers (Table 1). For each paper, based on the data disclosed and on the author’s interpretations, we captured the most dominant epitopes accounting for the majority of responses and/or consistently positive in multiple donors. We selected peptides that were consistently negatives as corresponding negative controls. In studies where large numbers of donors were tested and essentially all peptides were positives, we selected the peptides positives in one or more donors. A list of PUBMED Ids, and the criteria used to select the “top” epitopes and the “bottom” negative controls is provided in Table S3A in Supplementary Material. A list of positive and negative control peptides is provided in Table S3B in Supplementary Material.

Tetramer Training Dataset

A dataset corresponding to epitopes described as positive in tetramer staining experiments was downloaded from IEDB (accessed June 2015) (101) using the following selection criteria: “Positive Assays Only, Epitope Structure: Linear Sequence, T Cell Assays: qualitative binding/multimer/tetramer (tetramer), No B cell assays, No MHC ligand assays, MHC Restriction Type: Class II, Host Organism: Homo sapiens (human) (ID:9606, human).” The exported dataset was filtered keeping only 15-mer epitopes for which a source antigen protein ID was available. For each unique positive peptide, we took its source protein sequence using the antigen genome ID and scanned that protein for all possible 15-mers overlapping by 10 amino acids. The original positive peptide was then considered as an immunogenic one and the rest of the obtained peptides were used as negatives. The tetramer dataset had 124 unique positives and 5,319 negatives that are presented in Table S4 in Supplementary Material.

Artificial Neural Network (ANN)-Based Predictions Using NNAlign Method

The NN training for peptide sequences was performed using the NNAlign method (102). The method uses classified peptide data for training and identifies nested shorter sequence patterns that constitute an informative motif to separate positive from negative examples. As an input to NNAlign, we used sequences of our 15-mer peptides and their assigned observed immunogenicity score (1.0 for immunogenic and 0.0 for non-immunogenic). The method was trained using extensive cross-validation where part of the data is left out of the training process and is used for evaluation purpose only. For each peptide, the method returns a predicted score between 0.0 and 1.0, with high values identifying more immunogenic peptides and low values non-immunogenic peptides. The NNAlign-1.4 software package was downloaded from http://www.cbs.dtu.dk/services/NNAlign/. The method was trained for each possible motif length varying from 1 to 15. The data for cross-validation was split based on common motifs within peptides with a maximum overlap to nine and varying the motif length. Input peptides were encoded using Sparse and BLOSUM schemes. No rescaling was done to the input data. We also chose to preserve repeated flanks in the original data and do not realign networks with offset. The method was trained with 5 hidden neurons using 10 seeds for each network architecture. It is possible that other encoding approaches, choice of NN design, or choice of other learning algorithms could have let to better results, but such a comparison was outside the scope of our current manuscript.

Receiver Operating Characteristic (ROC) Curves and AUC Values

To measure how different approaches are capable of classifying peptides into epitopes and non-epitopes, ROC curves were used (103). Varying cutoff for predicted scores, peptides were classified into immunogenic and non-immunogenic and the numbers of true positives (TPs) and false positives (FPs) were obtained. The ROC curve was made by plotting TP rate as a function of FP rate at each cutoff. AUC is a useful measure for assessing predictive performance of a prediction method. AUC values range from 0.5 to 1, where 0.5 corresponds to random and 1 to perfect predictions. The AUC value can be interpreted as the probability that the predicted score for a randomly chosen immunogenic peptide is higher than the score of a randomly chosen non-immunogenic peptide.

HLA Binding Predictions

We utilized the previously described 7-allele method (15) to derive HLA binding propensities. The 7-allele method predicts immunogenicity based on the median percentile predicted binding of seven alleles representative of the binding motifs most commonly recognized in the general human population, and is available on the IEDB website (104).

Generation of Two-Sample Logo

The two-sample logo was created with 15-mer peptides (15 residues from N-terminal were extracted in case of longer peptides) from all the datasets combined, for epitopes and non-epitopes. For two-sample logo, both epitopes and non-epitopes datasets (in FASTA formatted files) were submitted to the online tool (http://www.twosamplelogo.org/cgi-bin/tsl/tsl.cgi) with default settings except for p-value, which was set to 0.01 and resolution of 600 dpi (105).

Statistical Analysis

The statistical analysis was performed using Prism 7 (Graph-Pad Software, San Diego, CA, USA). The non-parametric Wilcoxson matched-pair signed rank test with method of Pratt was utilized to assess the significance differences between sets of different AUC values.

Results

Derivation and Validation of an ANNs-Derived Immunogenicity Score

We assembled T cell epitope datasets from different previously published peptide screening studies performed in our laboratory (Table 1). In all cases, peptides were screened using ELISPOT assays to detect which peptides stimulated secretion of cytokines. Table 1 summarizes the number of donors that were screened for each peptide set and if the peptides were selected to overlap specific antigens, or if they were selected based on predicted binding affinity. Dominant epitopes accounting for a majority of the T cell responses as described in more detail in the Section “Materials and Methods” were considered positives (Table S1 in Supplementary Material, N = 1,032 peptides). Peptides that did not give any response in any donor but that came from proteins for which at least one peptide was positive, were considered negatives (Table S2 in Supplementary Material, N = 5,739 peptides). This additional criterion for negative classification was used to ensure that the lack of recognition was not simply due to lack of availability of the source protein necessary for antigen presentation.

This initial dataset was used to train an ANN-based method called NNAlign (102). The NNAlign method takes an unaligned peptide set and aims to find a linear sequence core within the peptides, which differentiates the positive (immunogenic) from the negative (non-immunogenic) peptides. The length of the sequence core was varied systematically from a single residue to 15 residues immunogenicity score (ranging 0–1) for each variation is retrieved, and prediction quality was assessed using fivefold cross-validation. Several sequence core lengths showed AUC values greater than 0.7 which is generally considered as a good prediction quality value, suggesting that the ANNs shows differences between positive and negative peptides based on the peptide motif. In terms of sequence motif length, the cross-validation did not indicate a clear optimal length, as the prediction performance was similar for motif lengths between 3 and 12 (Figure 1). A motif length of nine residues is consistent with the known size of peptide core region engaging HLA and TCR. For this reason, a motif length of nine was selected for the following analyses.

Figure 1

Combining Immunogenicity and HLA Binding Predictions

To consider both HLA binding and the immunogenicity prediction (which presumably incorporate the capacity of being recognized by TCR), we combined our ANN-based immunogenicity predictions with HLA class II binding predictions. Only one method has been described to predict epitopes based on HLA binding at the population level, namely the 7-allelle method, which was previously empirically optimized based on immunogenicity datasets (15).

To combine immunogenicity and HLA binding scores, we used the median percentile rank score (HLA_score) of the 7-allele method (ranging from 0 to 100) and combined it with our NN-based immunogenicity score after converting it to a percentile score, so that it would also range from 0 to 100 and could be comparable with the HLA_score, using the formula (Imm_score) = (1 − neural network immunogenicity) × 100. The two scores were combined as follows:

Next, we systematically varied the value of α in the interval of 0 ≤ α ≤ 1. From the equation above, when α = 1 the results depend only on the immunogenicity predictions by the NN, while with α = 0 only HLA binding predictions are used to define immunogenicity.

To assess the performance of the immunogenicity score, the 7-allele method and their combination, we used independent literature-based datasets. Specifically, we searched the IEDB for papers which described results of testing overlapping peptide sets related to human HLA class II restricted T cells. These epitope sets thus represent a broad range of studies, representing a “real-life” portrait of epitope identification studies performed in the worldwide scientific community. These epitope sets are listed in Table S3A in Supplementary Material and described in more detail in the Section “Materials and Methods” and the sequences are provided in Table S3B in Supplementary Material. Overall, a total of 57 different sets derived from independent literature studies were curated, entailing a total of 530 positive and 1,758 negative peptides. Figure 2 depicts the predictive performance of the combined score, displaying the average of the different AUC values obtained for each of the different datasets. The 7-allele method was associated with AUC values of 0.695, and the immunogenicity score was associated with an average AUC value of 0.670. In terms of combination of the two algorithms, the performance increased and reached a peak at 0.71 for an α value of 0.50.

Figure 2

Performance of the Immunogenicity Score, Eliminating Redundancy Between Training and Testing Datasets

It is expected that inclusion of additional data points would increase the performance of an NN model. Accordingly, we incorporated an additional dataset of CD4 T cell epitopes identified by tetramer mapping studies. We reasoned that this would provide high quality epitopes since the tetramer-staining assay is commonly considered a “gold standard” assay for epitope characterization. The dataset was obtained by querying the IEDB for 15-mer peptides that were tested positive by tetramer staining assays. For each positive peptide, its source protein was scanned for 15-mer peptides overlapping by 10 amino acids, with the positive peptide sequences being removed and the remaining peptides used to construct a negative dataset. The final tetramer dataset is composed of 124 unique positives and 5,319 unique negative peptides (Table S4 in Supplementary Material).

The datasets utilized to train and evaluate the NN models contained some redundancies, which could affect the evaluation and inflate performance. To avoid this issue, we eliminated any redundancy between the training set (Table 1 and tetramer set combined) and the validation set of the 57 independent studies (Table 1) by filtering out any peptide sharing a common 9-mer sequence.

In the analysis performed, a clear optimal alpha was not observed. The data in Figure 2 seemed to indicate an optimal alpha around 0.2–0.3, while the analysis from Figure 3 indicates two optimal peaks at about 0.4 and 0.6. Since the data in Figure 3 are inherently more reliable because of training with a higher number of data points, we empirically selected 0.4 as the alpha to include in the next set of analyses. When this analysis was performed, the 7-allele method prediction and the immunogenicity score were associated with similar performance (average AUC values of 0.703 and 0.702, respectively) while the combined methods again afforded gain in performance, reaching an average AUC of 0.725 (Figure 3). This increase in average AUC values of the combined methods is significant when compared with the average AUC values of the immunogenicity method with a p value of 0.0135 using Wilcoxon matched-pairs signed rank test, and a strong trend toward significance when compared to the 7-allele method with a p value of 0.0938. These results, together with the ones obtained with the tetramer dataset confirm that both the 7-allele and the immunogenicity score method had significant predictive value on their own which are in both cases enhanced by their combination.

Figure 3

Two-Sample Logo of a General Immunogenicity Motif

Next, we analyzed the epitopes and non-epitopes from all the datasets combined for their positional residue conservation and plotted two-sample logo using 15 residues from the N-terminus of the peptides (105) (Figure 4). The two-sample logo represents amino acids which were significantly different in epitopes and non-epitopes based on p-value (<0.01) calculated using t-test. Amino acid residues enriched in the epitope dataset are mostly positively charged, while amino acid residues depleted in the epitope dataset (and enriched in the non-epitope dataset) are mostly negatively charged. In other words, epitopes have higher numbers of positive charged residues like arginine (R) or lysine (K) at positions 9th and 11th–14th, whereas non-epitopes contained aspartate (D) and glutamate (E) at positions 7th–9th and 11th–13th. A preference for hydrophobic residues is also observed [such as proline (P) and alanine (A)] in non-epitopes, whereas isoleucine (I), phenylalanine (F), and asparagine (N) are enriched in the epitope set. To further address the significance of the logo, we split the dataset into five sets, where each set contains 80% of the total dataset, the results in Figure S1 in Supplementary Material confirm that the most prevalent feature revealed by the logo are in consistent with the two-sample logo created using whole dataset (Figure 4). These results suggest that some of these preferences may be contributing to T cell recognition or MHC binding or represent a result of processing enzymes. These possibilities will be addressed in future studies.

Figure 4

Epitope Prediction Threshold and Implementation of an Online Tool

We next determined the performance of the combined score using different cutoff values ranging from 0 to 100 (Table 2) for each study. To this end, we calculated the performance of overlapping datasets derived from literature at different threshold settings using the percentile combined score at α = 0.4. As a first step, for each study we calculated the numbers of: true negative (TN) defined as non-immunogenic peptides predicted as non-immunogenic, FP defined as non-immunogenic peptides predicted as immunogenic, false negative (FN) defined as immunogenic peptides predicted as non-immunogenic, and TP defined as the immunogenic peptides predicted as immunogenic. Based on these values we calculated sensitivity [= (TP/TP + FN) × 100] and specificity [= (TN/TN + FP) × 100]. Finally, we determined that cutoff values of 8, 36, and 66 allowed capturing, respectively, 20, 51, and 75% of the epitopes with a corresponding specificity of 91, 65, and 37%. We also estimated the fraction of peptides needed to test in order to observe a defined fraction of epitopes using the following formula: [(TP + FP)/(TP + TN + FP + FN)] × 100. A value of 43 was associated with equal sensitivity and specificity (59). To make this approach user friendly, we also implemented an online version of this algorithm (Figure 5). The tool is freely available in the IEDB website at http://tools.iedb.org/CD4episcore/.

Figure 5

Discussion

Bioinformatics predictions to identify T cell epitopes are frequently used in the context of designing and testing vaccines and diagnostics for infectious diseases, allergies, and cancer. While several HLA allele-specific predictive algorithms (10) and T cell epitopes predictive strategies based on MHC class II binding have been described (106–108), development of effective strategies to predict immunogenicity at the population level are lacking and remain therefore of significant interest. This is important, since in the real-life applications most often encountered HLA typing data is often unavailable.

Here, we report an approach to identify sequence motifs distinguishing immunogenic peptides recognized by CD4+ T cells from non-recognized peptides, independent of the restricting HLA class II allele. We confirm that the previously described 7-allele method (15) is effective in predicting epitopes and could narrow the range of peptides to be used for biological testing. Importantly, we find significant improvements of a combined HLA binding + immunogenicity approach over immunogenicity predictions alone and a strong trend toward significance of a combined HLA binding + immunogenicity approach over HLA binding predictions alone.

The machine learning algorithm we applied (NNalign) was developed to identify sequence motifs of a specific length that distinguish peptide sets—in our case immunogenic from non-immunogenic peptides. We found that motif lengths between 8 and 11 residues gave the best performance in the classification of the different datasets. This motif length is in line with what has been described with epitope residues in contact with the T cell receptor, and the length of the epitope core binding characteristic of the HLA class II molecule, which is also about nine amino acids long (20, 109).

The fact that the increase over predictions performed on HLA binding alone is rather small suggests, in line with previous studies, that HLA binding is a dominant force in shaping the repertoire of T cell epitopes. It is also possible, however, that this relatively small increase might be related to coordinate evolution between HLA binding and antigen processing and TCR recognition as suggested before by other studies (110).

Since the method was derived on immunogenicity outcomes only, it is possible that the motif defined herein is not only related to HLA binding but also incorporates overall preferences for TCR residue contacts. However, given the unbiased nature in which it was derived, it cannot be ruled out that the method may also reflect completely different processes, such as modulation by HLA-DM or increase in HLA binding stability over affinity is the actual source of the motif (111).

The predictive ability of very short motifs (3, 4 residues) is striking. Potential structural or mechanistic bases for this could be reflective of dominant influence of short stretches of residues incorporating dominant residues for HLA binding in close proximity to residues also dominant in TCR recognition (15). Examining the residues in the motif suggests that peptides with small amino acid side chains are avoided in the middle of the motif, while residues with longer side chains are overrepresented. This is qualitatively similar to what we had previously found for HLA class I restricted epitopes, and which has been reported in experimental studies using single residue substitutions (112, 113). This further supports that the motif identified coincides with properties of peptides more likely to engage a TCR. The F, M, L enrichment in the positions close to the N-terminus maybe at least in part corresponding to the P1 anchor of the MHC-II, which has similar specificity in several loci and allelic variants.

Our results have been trained over an extended set of data, derived from different methodologies and from populations of diverse ethnicities, and related to infectious diseases, allergy, and autoimmunity. The tetramer-trained algorithm seems to perform better, despite a bias toward certain HLA alleles and possible inclusion of many epitopes in negative set (i.e., other epitopes from the same protein other than the tetramer considered). We speculate that this may be due to the fact that tetramer epitopes represent usually dominant epitopes which in turn have been shown to correspond to promiscuous HLA binders. Overall, the combined training sets corresponded to over 14 thousand peptides, from over 300 different antigens and tested in over 2,500 different human donors. We believe this is an important aspect of our study, as it ensures that our building model (as related to both the 7-allele method, the immunogenicity score and the combined approach) are valid irrespective of antigen source, different ethnicities and disparate techniques for epitope identification. Our prediction method may be useful for generating off the shelf vaccine peptide libraries for pathogens or common tumor markers. Conversely, this method may be useful for an optimum selection of peptides covering individualized tumor derived neo-epitopes after NGS sequencing in HLA-typed individuals.

The algorithm is available on the IEDB website (101), and we estimate that the use of the combined immunogenicity score and 7-allele method will allow capturing 50% of the total epitopes by synthesis of 24% of the total possible overlapping 15-mers. This would translate in coverage of a 300 residues protein with 72 15-mer peptides. Future improvements of T cell epitope predictions may benefit from the increased availability of large scale datasets of peptides eluted from HLA class II molecules, datasets of specific TCRs recognizing epitopes, and datasets unraveling the role of mediators in the MHC class II processing pathway such as HLA-DM.

Even with this approach the AUC values are lower than for MHC-I analysis (1). However, it should be kept in mind that these AUC values refers to prediction at the population level encompassing T cell with diverse restriction, while the higher AUC values for MHC-I usually refers to allele-specific predictions. However, the application of the current approach from MHC-II to MHC-I, faces specific challenges. In MHC-I it is thought there is much more HLA-specific selection of epitopes, arguing against a straightforward application of the current approach, but it is possible that the alpha analysis could identify any HLA-independent components. Finally, it will be of interest to develop a similar approach to develop HLA agnostic predictors of HLA class I epitopes. Recent data suggest that it is possible to empirically develop HLA class I epitope “megapools” that afford coverage of general populations, irrespective of ethnicity (114, 115). Future studies will be focused on similar methods for HLA-agnostic prediction of class I restricted epitopes.

References

• 1

  PetersBBuiHHFrankildSNielsonMLundegaardCKostemEet alA community resource benchmarking predictions of peptide binding to MHC-I molecules. PLoS Comput Biol (2006) 2(6):e65.10.1371/journal.pcbi.0020065

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 2

  KumarNMohantyD. Structure-based identification of MHC binding peptides: benchmarking of prediction accuracy. Mol Biosyst (2010) 6(12):2508–20.10.1039/c0mb00013b

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3

  HuXMamitsukaHZhuS. Ensemble approaches for improving HLA class I-peptide binding prediction. J Immunol Methods (2011) 374(1–2):47–52.10.1016/j.jim.2010.09.007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4

  TrolleTMetushiIGGreenbaumJAKimYSidneyJLundOet alAutomated benchmarking of peptide-MHC class I binding predictions. Bioinformatics (2015) 31(13):2174–81.10.1093/bioinformatics/btv123

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5

  AndreattaMNielsenM. Gapped sequence alignment using artificial neural networks: application to the MHC class I system. Bioinformatics (2016) 32(4):511–7.10.1093/bioinformatics/btv639

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6

  NielsenMAndreattaM. NetMHCpan-3.0; improved prediction of binding to MHC class I molecules integrating information from multiple receptor and peptide length datasets. Genome Med (2016) 8(1):33.10.1186/s13073-016-0288-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7

  JurtzVPaulSAndreattaMMarcatiliPPetersBNielsenM. NetMHCpan-4.0: improved peptide-MHC class I interaction predictions integrating eluted ligand and peptide binding affinity data. J Immunol (2017) 199(9):3360–8.10.4049/jimmunol.1700893

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8

  KarosieneERasmussenMBlicherTLundOBuusSNielsenM. NetMHCIIpan-3.0, a common pan-specific MHC class II prediction method including all three human MHC class II isotypes, HLA-DR, HLA-DP and HLA-DQ. Immunogenetics (2013) 65(10):711–24.10.1007/s00251-013-0720-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9

  DhandaSKUsmaniSSAgrawalPNagpalGGautamARaghavaGPS. Novel in silico tools for designing peptide-based subunit vaccines and immunotherapeutics. Brief Bioinform (2017) 18(3):467–78.10.1093/bib/bbw025

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10

  FleriWPaulSDhandaSKMahajanSXuXPetersBet alThe immune epitope database and analysis resource in epitope discovery and synthetic vaccine design. Front Immunol (2017) 8:278.10.3389/fimmu.2017.00278

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11

  WeiskopfDAngeloMAde AzeredoELSidneyJGreenbaumJAFernandoANet alComprehensive analysis of dengue virus-specific responses supports an HLA-linked protective role for CD8+ T cells. Proc Natl Acad Sci U S A (2013) 110(22):E2046–53.10.1073/pnas.1305227110

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12

  McKinneyDMSouthwoodSHinzDOseroffCArlehamnCSSchultenVet alA strategy to determine HLA class II restriction broadly covering the DR, DP, and DQ allelic variants most commonly expressed in the general population. Immunogenetics (2013) 65(5):357–70.10.1007/s00251-013-0684-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  GreenbaumJSidneyJChungJBranderCPetersBSetteA. Functional classification of class II human leukocyte antigen (HLA) molecules reveals seven different supertypes and a surprising degree of repertoire sharing across supertypes. Immunogenetics (2011) 63(6):325–35.10.1007/s00251-011-0513-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14

  OseroffCSidneyJKotturiMFKollaRAlamRBroideDHet alMolecular determinants of T cell epitope recognition to the common Timothy grass allergen. J Immunol (2010) 185(2):943–55.10.4049/jimmunol.1000405

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  PaulSLindestam ArlehamnCSScribaTJDillonMBOseroffCHinzDet alDevelopment and validation of a broad scheme for prediction of HLA class II restricted T cell epitopes. J Immunol Methods (2015) 422:28–34.10.1016/j.jim.2015.03.022

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16

  YewdellJWBenninkJR. Immunodominance in major histocompatibility complex class I-restricted T lymphocyte responses. Annu Rev Immunol (1999) 17:51–88.10.1146/annurev.immunol.17.1.51

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17

  AssarssonESidneyJOseroffCPasquettoVBuiHHFrahmNet alA quantitative analysis of the variables affecting the repertoire of T cell specificities recognized after vaccinia virus infection. J Immunol (2007) 178(12):7890–901.10.4049/jimmunol.178.12.7890

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18

  KotturiMFPetersBBuendia-LaysaFJrSidneyJOseroffCBottenJet alThe CD8+ T-cell response to lymphocytic choriomeningitis virus involves the L antigen: uncovering new tricks for an old virus. J Virol (2007) 81(10):4928–40.10.1128/JVI.02632-06

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19

  Stewart-JonesGBMcMichaelAJBellJIStuartDIJonesEY. A structural basis for immunodominant human T cell receptor recognition. Nat Immunol (2003) 4(7):657–63.10.1038/ni942

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20

  TurnerSJDohertyPCMcCluskeyJRossjohnJ. Structural determinants of T-cell receptor bias in immunity. Nat Rev Immunol (2006) 6(12):883–94.10.1038/nri1977

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21

  KotturiMFScottIWolfeTPetersBSidneyJCheroutreHet alNaive precursor frequencies and MHC binding rather than the degree of epitope diversity shape CD8+ T cell immunodominance. J Immunol (2008) 181(3):2124–33.10.4049/jimmunol.181.3.2124

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22

  JenkinsMKChuHHMcLachlanJBMoonJJ. On the composition of the preimmune repertoire of T cells specific for peptide-major histocompatibility complex ligands. Annu Rev Immunol (2010) 28:275–94.10.1146/annurev-immunol-030409-101253

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23

  QiQLiuYChengYGlanvilleJZhangDLeeJYet alDiversity and clonal selection in the human T-cell repertoire. Proc Natl Acad Sci U S A (2014) 111(36):13139–44.10.1073/pnas.1409155111

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  FrankildSde BoerRJLundONielsenMKesmirC. Amino acid similarity accounts for T cell cross-reactivity and for "holes" in the T cell repertoire. PLoS One (2008) 3(3):e1831.10.1371/journal.pone.0001831

  • CrossRef
  • Google Scholar

• 25

  CalisJJMaybenoMGreenbaumJAWeiskopfDDe SilvaADSetteAet alProperties of MHC class I presented peptides that enhance immunogenicity. PLoS Comput Biol (2013) 9(10):e1003266.10.1371/journal.pcbi.1003266

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26

  GlanvilleJHuangHNauAHattonOWagarLERubeltFet alIdentifying specificity groups in the T cell receptor repertoire. Nature (2017) 547(7661):94–8.10.1038/nature22976

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27

  ArlehamnCSSidneyJHendersonRGreenbaumJAJamesEAMoutaftsiMet alDissecting mechanisms of immunodominance to the common tuberculosis antigens ESAT-6, CFP10, Rv2031c (hspX), Rv2654c (TB7.7), and Rv1038c (EsxJ). J Immunol (2012) 188(10):5020–31.10.4049/jimmunol.1103556

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28

  Lindestam ArlehamnCSGerasimovaAMeleFHendersonRSwannJGreenbaumJAet alMemory T cells in latent Mycobacterium tuberculosis infection are directed against three antigenic islands and largely contained in a CXCR3+CCR6+ Th1 subset. PLoS Pathog (2013) 9(1):e1003130.10.1371/journal.ppat.1003130

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29

  Lindestam ArlehamnCSMcKinneyDMCarpenterCPaulSRozotVMakgotlhoEet alA quantitative analysis of complexity of human pathogen-specific CD4 T cell responses in healthy M. tuberculosis infected South Africans. PLoS Pathog (2016) 12(7):e1005760.10.1371/journal.ppat.1005760

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30

  SchultenVGreenbaumJAHauserMMcKinneyDMSidneyJKollaRet alPreviously undescribed grass pollen antigens are the major inducers of T helper 2 cytokine-producing T cells in allergic individuals. Proc Natl Acad Sci U S A (2013) 110(9):3459–64.10.1073/pnas.1300512110

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31

  WesternbergLSchultenVGreenbaumJANataliSTrippleVMcKinneyDMet alT-cell epitope conservation across allergen species is a major determinant of immunogenicity. J Allergy Clin Immunol (2016) 138(2):571–8.e7.10.1016/j.jaci.2015.11.034

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32

  HinzDOseroffCPhamJSidneyJPetersBSetteA. Definition of a pool of epitopes that recapitulates the T cell reactivity against major house dust mite allergens. Clin Exp Allergy (2015) 45(10):1601–12.10.1111/cea.12507

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33

  DillonMBSchultenVOseroffCPaulSDullantyLMFrazierAet alDifferent Bla-g T cell antigens dominate responses in asthma versus rhinitis subjects. Clin Exp Allergy (2015) 45(12):1856–67.10.1111/cea.12643

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  WeiskopfDBangsDJSidneyJKollaRVDe SilvaADde SilvaAMet alDengue virus infection elicits highly polarized CX3CR1(+) cytotoxic CD4(+) T cells associated with protective immunity. Proc Natl Acad Sci U S A (2015) 112(31):E4256–63.10.1073/pnas.1505956112

  • CrossRef
  • Google Scholar

• 35

  TangriSMotheBREisenbraunJSidneyJSouthwoodSBriggsKet alRationally engineered therapeutic proteins with reduced immunogenicity. J Immunol (2005) 174(6):3187–96.10.4049/jimmunol.174.6.3187

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36

  OseroffCPhamJFrazierAHinzDSidneyJPaulSet alImmunodominance in allergic T-cell reactivity to Japanese cedar in different geographic cohorts. Ann Allergy Asthma Immunol (2016) 117(6):680–689.e1.10.1016/j.anai.2016.10.014

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37

  SchultenVWesternbergLBirruetaGSidneyJPaulSBussePet alAllergen and epitope targets of mouse-specific T cell responses in allergy and asthma. Front Immunol (2018) 9:235.10.3389/fimmu.2018.00235

  • CrossRef
  • Google Scholar

• 38

  OseroffCChristensenLHWesternbergLPhamJLaneJPaulSet alImmunoproteomic analysis of house dust mite antigens reveals distinct classes of dominant T cell antigens according to function and serological reactivity. Clin Exp Allergy (2017) 47(4):577–92.10.1111/cea.12829

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  BancroftTDillonMBda Silva AntunesRPaulSPetersBCrottySet alTh1 versus Th2 T cell polarization by whole-cell and acellular childhood pertussis vaccines persists upon re-immunization in adolescence and adulthood. Cell Immunol (2016) 304-305:35–43.10.1016/j.cellimm.2016.05.002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40

  PhamJOseroffCHinzDSidneyJPaulSGreenbaumJet alSequence conservation predicts T cell reactivity against ragweed allergens. Clin Exp Allergy (2016) 46(9):1194–205.10.1111/cea.12772

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41

  AntunesRDSPaulSSidneyJWeiskopfDDanJMPhillipsEet alDefinition of human epitopes recognized in tetanus toxoid and development of an assay strategy to detect ex vivo tetanus CD4(+) T cell responses. PLoS One (2017) 12(1):e0169086.10.1371/journal.pone.0169086

  • CrossRef
  • Google Scholar

• 42

  ManfrediAAProttiMPWuXDHowardJFJrConti-TronconiBM. CD4+ T-epitope repertoire on the human acetylcholine receptor alpha subunit in severe myasthenia gravis: a study with synthetic peptides. Neurology (1992) 42(5):1092–100.10.1212/WNL.42.5.1092

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43

  HerreraSEscobarPde PlataCAvilaGICorradinGHerreraMA. Human recognition of T cell epitopes on the Plasmodium vivax circumsporozoite protein. J Immunol (1992) 148(12):3986–90.

  • Pubmed Abstract
  • Google Scholar

• 44

  SjostedtASandstromGTarnvikAJaurinB. Nucleotide sequence and T cell epitopes of a membrane protein of Francisella tularensis. J Immunol (1990) 145(1):311–7.

  • Pubmed Abstract
  • Google Scholar

• 45

  RzepczykCMCsurhesPABaxterEPDoranTJIrvingDOKereN. Amino acid sequences recognized by T cells: studies on a merozoite surface antigen from the FCQ-27/PNG isolate of Plasmodium falciparum. Immunol Lett (1990) 25(1–3):155–63.10.1016/0165-2478(90)90108-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  ZeveringYHoughtenRAFrazerIHGoodMF. Major population differences in T cell response to a malaria sporozoite vaccine candidate. Int Immunol (1990) 2(10):945–55.10.1093/intimm/2.10.945

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47

  GoodMFPomboDQuakyiIARileyEMHoughtenRAMenonAet alHuman T-cell recognition of the circumsporozoite protein of Plasmodium falciparum: immunodominant T-cell domains map to the polymorphic regions of the molecule. Proc Natl Acad Sci U S A (1988) 85(4):1199–203.10.1073/pnas.85.4.1199

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48

  CarballidoJMCarballido-PerrigNKagiMKMeloenRHWuthrichBHeusserCHet alT cell epitope specificity in human allergic and nonallergic subjects to bee venom phospholipase A2. J Immunol (1993) 150(8 Pt 1):3582–91.

  • Pubmed Abstract
  • Google Scholar

• 49

  SalvettiMRistoriGD’AmatoMButtinelliCFalconeMFieschiCet alPredominant and stable T cell responses to regions of myelin basic protein can be detected in individual patients with multiple sclerosis. Eur J Immunol (1993) 23(6):1232–9.10.1002/eji.1830230606

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 50

  BilsboroughJCarlisleMGoodMF. Identification of Caucasian CD4 T cell epitopes on the circumsporozoite protein of Plasmodium vivax. T cell memory. J Immunol (1993) 151(2):890–9.

  • Pubmed Abstract
  • Google Scholar

• 51

  ManfrediAAProttiMPDaltonMWHowardJFJrConti-TronconiBM. T helper cell recognition of muscle acetylcholine receptor in myasthenia gravis. Epitopes on the gamma and delta subunits. J Clin Invest (1993) 92(2):1055–67.10.1172/JCI116610

  • CrossRef
  • Google Scholar

• 52

  MoiolaLProttiMPManfrediAAYuenMHHowardJFJrConti-TronconiBM. T-helper epitopes on human nicotinic acetylcholine receptor in myasthenia gravis. Ann N Y Acad Sci (1993) 681:198–218.10.1111/j.1749-6632.1993.tb22887.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 53

  AtkinsonMABowmanMACampbellLDarrowBLKaufmanDLMaclarenNK. Cellular immunity to a determinant common to glutamate decarboxylase and coxsackie virus in insulin-dependent diabetes. J Clin Invest (1994) 94(5):2125–9.10.1172/JCI117567

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 54

  ChayeHOuDChongPGillamS. Human T- and B-cell epitopes of E1 glycoprotein of rubella virus. J Clin Immunol (1993) 13(2):93–100.10.1007/BF00919265

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55

  DamhofRADrijfhoutJWSchefferAJWilterdinkJBWellingGWWelling-WesterS. T cell responses to synthetic peptides of herpes simplex virus type 1 glycoprotein D in naturally infected individuals. Arch Virol (1993) 130(1–2):187–93.10.1007/BF01319007

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  KellermannSAMcCormickDJFreemanSLMorrisJCConti-FineBM. TSH receptor sequences recognized by CD4+ T cells in Graves’ disease patients and healthy controls. J Autoimmun (1995) 8(5):685–98.10.1006/jaut.1995.0051

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57

  MullerCPAmmerlaanWFleckensteinBKraussSKalbacherHSchneiderFet alActivation of T cells by the ragged tail of MHC class II-presented peptides of the measles virus fusion protein. Int Immunol (1996) 8(4):445–56.10.1093/intimm/8.4.445

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58

  ZhangLYangMChongPMohapatraSS. Multiple B- and T-cell epitopes on a major allergen of Kentucky Bluegrass pollen. Immunology (1996) 87(2):283–90.10.1046/j.1365-2567.1996.467533.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 59

  PenderMPCsurhesPAHoughtenRAMcCombePAGoodMF. A study of human T-cell lines generated from multiple sclerosis patients and controls by stimulation with peptides of myelin basic protein. J Neuroimmunol (1996) 70(1):65–74.10.1016/S0165-5728(96)00105-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60

  MarttilaJIlonenJLehtinenMParkkonenPSalmiA. Definition of three minimal T helper cell epitopes of rubella virus E1 glycoprotein. Clin Exp Immunol (1996) 104(3):394–7.10.1046/j.1365-2249.1996.54762.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 61

  WangZYOkitaDKHowardJJrConti-FineBM. Th1 epitope repertoire on the alpha subunit of human muscle acetylcholine receptor in myasthenia gravis. Neurology (1997) 48(6):1643–53.10.1212/WNL.48.6.1643

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 62

  Raulf-HeimsothMChenZRihsHPKalbacherHLiebersVBaurX. Analysis of T-cell reactive regions and HLA-DR4 binding motifs on the latex allergen Hev b 1 (rubber elongation factor). Clin Exp Allergy (1998) 28(3):339–48.10.1046/j.1365-2222.1998.00230.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 63

  KammererRKettnerAChvatchkoYDufourNTiercyJMCorradinGet alDelineation of PLA2 epitopes using short or long overlapping synthetic peptides: interest for specific immunotherapy. Clin Exp Allergy (1997) 27(9):1016–26.10.1111/j.1365-2222.1997.tb01253.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 64

  FlanaganKLPlebanskiMAkinwunmiPLeeEAReeceWHRobsonKJet alBroadly distributed T cell reactivity, with no immunodominant loci, to the pre-erythrocytic antigen thrombospondin-related adhesive protein of Plasmodium falciparum in West Africans. Eur J Immunol (1999) 29(6):1943–54.10.1002/(SICI)1521-4141(199906)29:06<1943::AID-IMMU1943>3.0.CO;2-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65

  MarttilaJIlonenJNorrbyESalmiA. Characterization of T cell epitopes in measles virus nucleoprotein. J Gen Virol (1999) 80(Pt 7):1609–15.10.1099/0022-1317-80-7-1609

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66

  LamonacaVMissaleGUrbaniSPilliMBoniCMoriCet alConserved hepatitis C virus sequences are highly immunogenic for CD4(+) T cells: implications for vaccine development. Hepatology (1999) 30(4):1088–98.10.1002/hep.510300435

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 67

  WoodfolkJASungSSBenjaminDCLeeJKPlatts-MillsTA. Distinct human T cell repertoires mediate immediate and delayed-type hypersensitivity to the Trichophyton antigen, Tri r 2. J Immunol (2000) 165(8):4379–87.10.4049/jimmunol.165.8.4379

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 68

  StottLMBarkerRNUrbaniakSJ. Identification of alloreactive T-cell epitopes on the Rhesus D protein. Blood (2000) 96(13):4011–9.

  • Pubmed Abstract
  • Google Scholar

• 69

  Tejada-SimonMVHongJRiveraVMZhangJZ. Reactivity pattern and cytokine profile of T cells primed by myelin peptides in multiple sclerosis and healthy individuals. Eur J Immunol (2001) 31(3):907–17.10.1002/1521-4141(200103)31:3<907::AID-IMMU907>3.0.CO;2-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 70

  MarttilaJJuhelaSVaaralaOHyotyHRoivainenMHinkkanenAet alResponses of coxsackievirus B4-specific T-cell lines to 2C protein-characterization of epitopes with special reference to the GAD65 homology region. Virology (2001) 284(1):131–41.10.1006/viro.2001.0917

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 71

  HolenEBolannBElsayedS. Novel B and T cell epitopes of chicken ovomucoid (Gal d 1) induce T cell secretion of IL-6, IL-13, and IFN-gamma. Clin Exp Allergy (2001) 31(6):952–64.10.1046/j.1365-2222.2001.01102.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72

  WertheimerAMMinerCLewinsohnDMSasakiAWKaufmanERosenHR. Novel CD4+ and CD8+ T-cell determinants within the NS3 protein in subjects with spontaneously resolved HCV infection. Hepatology (2003) 37(3):577–89.10.1053/jhep.2003.50115

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73

  de SilvaHDGardnerLMDrewACBeezholdDHRollandJMO’HehirRE. The hevein domain of the major latex-glove allergen Hev b 6.01 contains dominant T cell reactive sites. Clin Exp Allergy (2004) 34(4):611–8.10.1111/j.1365-2222.2004.1919.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 74

  ElsayedSEriksenJOysaedLKIdsoeRHillDJ. T cell recognition pattern of bovine milk alphaS1-casein and its peptides. Mol Immunol (2004) 41(12):1225–34.10.1016/j.molimm.2004.05.010

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 75

  SoneTDairikiKMorikuboKShimizuKTsunooHMoriTet alIdentification of human T cell epitopes in Japanese cypress pollen allergen, Cha o 1, elucidates the intrinsic mechanism of cross-allergenicity between Cha o 1 and Cry j 1, the major allergen of Japanese cedar pollen, at the T cell level. Clin Exp Allergy (2005) 35(5):664–71.10.1111/j.1365-2222.2005.02221.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 76

  Schulze zur WieschJLauerGMDayCLKimAYOuchiKDuncanJEet alBroad repertoire of the CD4+ Th cell response in spontaneously controlled hepatitis C virus infection includes dominant and highly promiscuous epitopes. J Immunol (2005) 175(6):3603–13.10.4049/jimmunol.175.6.3603

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 77

  SarobePLasarteJJGarciaNCiveiraMPBorras-CuestaFPrietoJ. Characterization of T-cell responses against immunodominant epitopes from hepatitis C virus E2 and NS4a proteins. J Viral Hepat (2006) 13(1):47–55.10.1111/j.1365-2893.2005.00653.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 78

  RuiterBTregoatVM’RabetLGarssenJBruijnzeel-KoomenCAKnolEFet alCharacterization of T cell epitopes in alphas1-casein in cow’s milk allergic, atopic and non-atopic children. Clin Exp Allergy (2006) 36(3):303–10.10.1111/j.1365-2222.2006.02436.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 79

  MaYBogdanosDPHussainMJUnderhillJBansalSLonghiMSet alPolyclonal T-cell responses to cytochrome P450IID6 are associated with disease activity in autoimmune hepatitis type 2. Gastroenterology (2006) 130(3):868–82.10.1053/j.gastro.2005.12.020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 80

  KasprowiczVIsaATolfvenstamTJefferyKBownessPKlenermanP. Tracking of peptide-specific CD4+ T-cell responses after an acute resolving viral infection: a study of parvovirus B19. J Virol (2006) 80(22):11209–17.10.1128/JVI.01173-06

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 81

  SukatiHWatsonHGUrbaniakSJBarkerRN. Mapping helper T-cell epitopes on platelet membrane glycoprotein IIIa in chronic autoimmune thrombocytopenic purpura. Blood (2007) 109(10):4528–38.10.1182/blood-2006-09-044388

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 82

  Schulze Zur WieschJLauerGMTimmJKuntzenTNeukammMBericalAet alImmunologic evidence for lack of heterologous protection following resolution of HCV in patients with non-genotype 1 infection. Blood (2007) 110(5):1559–69.10.1182/blood-2007-01-069583

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 83

  ImmonenAKinnunenTSirvenPTaivainenAHouitteDPerasaariJet alThe major horse allergen Equ c 1 contains one immunodominant region of T cell epitopes. Clin Exp Allergy (2007) 37(6):939–47.10.1111/j.1365-2222.2007.02722.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 84

  MalhotraIWamachiANMungaiPLMzunguEKoechDMuchiriEet alFine specificity of neonatal lymphocytes to an abundant malaria blood-stage antigen: epitope mapping of Plasmodium falciparum MSP1(33). J Immunol (2008) 180(5):3383–90.10.4049/jimmunol.180.5.3383

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 85

  MasuyamaKChikamatsuKIkagawaSMatsuokaTTakahashiGYamamotoTet alAnalysis of helper T cell responses to Cry j 1-derived peptides in patients with nasal allergy: candidate for peptide-based immunotherapy of Japanese cedar pollinosis. Allergol Int (2009) 58(1):63–70.10.2332/allergolint.08-OA-0008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 86

  SoneTDairikiKMorikuboKShimizuKTsunooHMoriTet alRecognition of T cell epitopes unique to Cha o 2, the major allergen in Japanese cypress pollen, in allergic patients cross-reactive to Japanese cedar and Japanese cypress pollen. Allergol Int (2009) 58(2):237–45.10.2332/allergolint.08-OA-0027

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 87

  MadsenDCantwellERO’BrienTJohnsonPAMahonBP. Adeno-associated virus serotype 2 induces cell-mediated immune responses directed against multiple epitopes of the capsid protein VP1. J Gen Virol (2009) 90(Pt 11):2622–33.10.1099/vir.0.014175-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 88

  PastorelloEAMonzaMPravettoniVLonghiRBonaraPScibiliaJet alCharacterization of the T-cell epitopes of the major peach allergen Pru p 3. Int Arch Allergy Immunol (2010) 153(1):1–12.10.1159/000301573

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 89

  MatsuyaNKomoriMNomuraKNakaneSFukudomeTGotoHet alIncreased T-cell immunity against aquaporin-4 and proteolipid protein in neuromyelitis optica. Int Immunol (2011) 23(9):565–73.10.1093/intimm/dxr056

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 90

  ChaduvulaMMurtazaAMisraNNarayanNPRameshVPrasadHKet alLsr2 peptides of Mycobacterium leprae show hierarchical responses in lymphoproliferative assays, with selective recognition by patients with anergic lepromatous leprosy. Infect Immun (2012) 80(2):742–52.10.1128/IAI.05384-11

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 91

  EttoTde BoerCPrickettSGardnerLMVoskampADaviesJMet alUnique and cross-reactive T cell epitope peptides of the major Bahia grass pollen allergen, Pas n 1. Int Arch Allergy Immunol (2012) 159(4):355–66.10.1159/000338290

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 92

  RavkovEVPavlovIYMartinsTBGleichGJWagnerLAHillHRet alIdentification and validation of shrimp-tropomyosin specific CD4 T cell epitopes. Hum Immunol (2013) 74(12):1542–9.10.1016/j.humimm.2013.08.276

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 93

  SchwaigerJAberleJHStiasnyKKnappBSchreinerWFaeIet alSpecificities of human CD4+ T cell responses to an inactivated flavivirus vaccine and infection: correlation with structure and epitope prediction. J Virol (2014) 88(14):7828–42.10.1128/JVI.00196-14

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 94

  RonkaALKinnunenTTGoudetARytkonen-NissinenMASairanenJKailaanmakiAHet alCharacterization of human memory CD4(+) T-cell responses to the dog allergen Can f 4. J Allergy Clin Immunol (2015) 136(4):1047–54.e10.10.1016/j.jaci.2015.02.025

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 95

  KailaanmakiAKinnunenTRonkaARytkonen-NissinenMLidholmJMattssonLet alHuman memory CD4+ T cell response to the major dog allergen Can f 5, prostatic kallikrein. Clin Exp Allergy (2016) 46(5):720–9.10.1111/cea.12694

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 96

  OshimaMDeitikerPJankovicJAokiKRAtassiMZ. Submolecular recognition of the C-terminal domain of the heavy chain of botulinum neurotoxin type A by T cells from toxin-treated cervical dystonia patients. Immunobiology (2016) 221(4):568–76.10.1016/j.imbio.2015.12.002

  • CrossRef
  • Google Scholar

• 97

  GaidoCMStoneSChopraAThomasWRLe SouefPNHalesBJ. Immunodominant T-cell epitopes in the VP1 capsid protein of rhinovirus species A and C. J Virol (2016) 90(23):10459–71.10.1128/JVI.01701-16

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 98

  OshimaMDeitikerPJankovicJAtassiMZ. Submolecular recognition regions of the HN domain of the heavy chain of botulinum neurotoxin type A by T cells from toxin-treated cervical dystonia patients. J Neuroimmunol (2016) 300:36–46.10.1016/j.jneuroim.2016.09.013

  • CrossRef
  • Google Scholar

• 99

  PaulSSidneyJSetteAPetersB. TepiTool: a pipeline for computational prediction of T cell epitope candidates. Curr Protoc Immunol (2016) 114:18.19.1–18.19.24.10.1002/cpim.12

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 100

  WeiskopfDAngeloMAGrifoniAO’RourkePHSidneyJPaulSet alHLA-DRB1 alleles are associated with different magnitudes of dengue virus-specific CD4+ T-cell responses. J Infect Dis (2016) 214(7):1117–24.10.1093/infdis/jiw309

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 101

  VitaROvertonJAGreenbaumJAPonomarenkoJClarkJDCantrellJRet alThe immune epitope database (IEDB) 3.0. Nucleic Acids Res (2015) 43(Database issue):D405–12.10.1093/nar/gku938

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 102

  AndreattaMSchafer-NielsenCLundOBuusSNielsenM. NNAlign: a web-based prediction method allowing non-expert end-user discovery of sequence motifs in quantitative peptide data. PLoS One (2011) 6(11):e26781.10.1371/journal.pone.0026781

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 103

  SwetsJA. Measuring the accuracy of diagnostic systems. Science (1988) 240(4857):1285–93.10.1126/science.3287615

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 104

  KimYPonomarenkoJZhuZTamangDWangPGreenbaumJet alImmune epitope database analysis resource. Nucleic Acids Res (2012) 40(Web Server issue):W525–30.10.1093/nar/gks438

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 105

  VacicVIakouchevaLMRadivojacP. Two sample logo: a graphical representation of the differences between two sets of sequence alignments. Bioinformatics (2006) 22(12):1536–7.10.1093/bioinformatics/btl151

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 106

  DhandaSKGuptaSVirPRaghavaGP. Prediction of IL4 inducing peptides. Clin Dev Immunol (2013) 2013:263952.10.1155/2013/263952

  • CrossRef
  • Google Scholar

• 107

  DhandaSKVirPRaghavaGP. Designing of interferon-gamma inducing MHC class-II binders. Biol Direct (2013) 8:30.10.1186/1745-6150-8-30

  • CrossRef
  • Google Scholar

• 108

  NagpalGUsmaniSSDhandaSKKaurHSinghSSharmaMet alComputer-aided designing of immunosuppressive peptides based on IL-10 inducing potential. Sci Rep (2017) 7:42851.10.1038/srep42851

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 109

  Sant’AngeloDBRobinsonEJanewayCAJrDenzinLK. Recognition of core and flanking amino acids of MHC class II-bound peptides by the T cell receptor. Eur J Immunol (2002) 32(9):2510–20.10.1002/1521-4141(200209)32:9<2510::AID-IMMU2510>3.0.CO;2-Q

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 110

  NielsenMLundegaardCLundOKesmirC. The role of the proteasome in generating cytotoxic T-cell epitopes: insights obtained from improved predictions of proteasomal cleavage. Immunogenetics (2005) 57(1–2):33–41.10.1007/s00251-005-0781-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 111

  YinLCalvo-CalleJMDominguez-AmorochoOSternLJ. HLA-DM constrains epitope selection in the human CD4 T cell response to vaccinia virus by favoring the presentation of peptides with longer HLA-DM-mediated half-lives. J Immunol (2012) 189(8):3983–94.10.4049/jimmunol.1200626

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 112

  AlexanderJSidneyJSouthwoodSRuppertJOseroffCMaewalAet alDevelopment of hi0067h potency universal DR-restricted helper epitopes by modification of high affinity DR-blocking peptides. Immunity (1994) 1(9):751–61.10.1016/S1074-7613(94)80017-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 113

  HungCFTsaiYCHeLWuTC. DNA vaccines encoding Ii-PADRE generates potent PADRE-specific CD4+ T-cell immune responses and enhances vaccine potency. Mol Ther (2007) 15(6):1211–9.10.1038/sj.mt.6300121

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 114

  Carrasco ProSSidneyJPaulSLindestam ArlehamnCWeiskopfDPetersBet alAutomatic generation of validated specific epitope sets. J Immunol Res (2015) 2015:763461.10.1155/2015/763461

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 115

  WeiskopfDCerpasCAngeloMABangsDJSidneyJPaulSet alHuman CD8+ T-cell responses against the 4 dengue virus serotypes are associated with distinct patterns of protein targets. J Infect Dis (2015) 212(11):1743–51.10.1093/infdis/jiv289

  • CrossRef
  • Google Scholar