Abstract
Antibodies represent the largest class of biotherapeutics thanks to their high target specificity, binding affinity and versatility. Recent breakthroughs in Artificial Intelligence (AI) have enabled information-rich in silico representations of antibodies, accurate prediction of antibody structure from sequence, and the generation of novel antibodies tailored to specific characteristics to optimize for developability properties. Here we summarize state-of-the-art methods for antibody analysis. This valuable resource will serve as a reference for the application of AI methods to the analysis of antibody sequencing datasets.
1 Introduction
Antibodies are the largest class of biotherapeutics, with a projected market size of US$ 300 Billion by 2025 (). They are used for treating cancer, autoimmune and infectious diseases (; Weiner et al., 2010; ), as they can be designed to recognize any antigen at high specificity and binding affinity. Antibody discovery is traditionally performed with directed evolution using experimental assays such as hybridoma or phage display (). Although well-established, these methods remain costly, time-consuming and prone to fail due to experimental challenges.
The introduction of Next-Generation Sequencing (NGS) for antibody screening in place of random colony picking has enabled to cover a much larger sequence diversity, a wider binding affinity range, and isolate sequences that target distinct epitopes (). Short read sequencing is limited to a single chain, either heavy (VH) and light chain (VL), while long reads can obtain paired information of both chains, increasing our understanding of inter-chain residue dependencies ().
Recently, Artificial Intelligence (AI) has experienced accelerated progress, particularly in the fields of Deep Learning (DL) and Natural Language Processing (NLP), and biology has been greatly benefited from it (; ; ; ; ; ). A notable example is the model AlphaFold2 for structural biology (), which brought sequence-based protein structure prediction close to experimental accuracy.
The success of the Transformer architecture (Vaswani et al., 2023) in NLP has led to the creation of Large Language Models (LLM), statistical models trained on large collections of texts to capture semantic similarity among words in the form of vector representations, called embeddings, without relying on expensive and hard to obtain labels. Embeddings are very versatile, with applications that include text classification and generation. In biology, LLMs trained on curated databases of millions of protein sequences [UniProt (UniProt Consortium et al., 2023), UniRef (Suzek et al., 2007) and BFD (; Steinegger and Söding, 2018)] were shown to be able to learn secondary and tertiary structural information from sequence (; ) and can be used to predict protein function. More recently LLMs have been trained on databases of antibody sequences, such as the Observed Antibody Space (OAS), leading to the creation of antibody-specific language models (ALMs) (; ; ; ).
Despite the availability of these models, bringing an antibody from discovery to the patient remains challenging. Once a candidate antibody has been found, it must be optimized to match the properties of therapeutic antibodies, grouped under the term of developability. A consensus is lacking in the literature for which properties are part of developability (; ; ; ; Zhang et al., 2023; ). Some of these properties are humanization, prediction of solubility and aggregation, for which several ML methods have been proposed (; ; ).
The type of license associated with a ML model (code, weights and training data) plays a key role in the choice of integration into industrial applications. A commercially permissive license favors rapid prototyping in research and development within an industrial setting and rapid, cost-free integration into a product. In this review, we indicate the license type associated to the methods presented, in the hope that this resource will serve as a reference to accelerate the adoption of these models in industry.
Several reviews have been published that discuss ML applications to antibody discovery and development (; ; ). These reviews are focused on giving an academic perspective on the field. Our review stands out not only for its breadth, by providing a comprehensive, up-to-date overview of the state-of-the-art AI methods and resources for antibody sequence, structure and developability, but also for the particular focus on practical considerations in regard to product integration, such as licenses.
Providing a comprehensive benchmark for these methods is outside the scope of this review, and would require testing against specific benchmark datasets like ProteinGym () for general protein language models and FlAb () for antibody language models.
This review is divided into three parts (Figure 1). The first part covers recent applications of LLMs to protein (PLM) and antibody (ALM) sequences. The second part focuses on folding models, models that can predict protein structure from sequence, and inverse folding models, models that can identify the sequences a specific structure can fold into. The third part covers ML methods that can be used to optimize developability properties. Finally, we conclude with some remarks and perspectives.
FIGURE 1
2 Antibody language models
The field of NLP was revolutionized by the introduction of the Transformer (Vaswani et al., 2017), a DL architecture that was able to achieve unprecedented accuracy in understanding and generation of written and spoken languages, programming languages, images and videos (). At the core of the Transformer is the attention layer, a neural network layer inspired by cognitive attention, the human ability to focus on important signals and exclude irrelevant information. Through attention the model learns the relative importance all parts of the input sequence (tokens) have with respect to each other. This is used to generate a vector representation of each token in the sequence (embedding) that can be leveraged for specific tasks. Training is performed with either a Masked Language Modelling (MLM) objective, the prediction of a randomly chosen subset of masked tokens, a Causal Language Modelling (CLM) objective, the prediction of the next token based on the preceding tokens, or both (Figure 1).
When trained on large collections of protein sequences [UniProt (UniProt Consortium et al., 2023), UniRef (Suzek et al., 2007) and BFD (Steinegger and Söding, 2018)] as protein Language Models (PLMs) (Table 1), these models capture information on evolutionary constraints, secondary and tertiary structures (; ; ). For a comprehensive overview of NLP applied to the protein sequence domain we refer to the available reviews (; Valentini et al., 2023; ).
TABLE 1
| Model | Base | Params | Code | Training data | License | Refs | Year |
|---|---|---|---|---|---|---|---|
| ProtTrans | Multiple | 224M-11B | GH, HF | UniRef50, BFD | AFL-3.0 | 2020 | |
| ESM-2 | BERT | 8M-15B | GH, HF | UniRef50 | MIT | 2022 | |
| ProteinBERT | BERT | 16M | GH | UniRef50 | NA | 2022 | |
| ProtGPT2 | GPT2 | 738M | HF | UniRef50 | Apache-2.0 | 2022 | |
| EvoDiff | Diffusion | 38M-640M | GH | UniRef50 | MIT | 2023 | |
| Ankh | T5 | 450M-1.15B | GH, HF | UniRef50 | CC NC SA 4.0 | 2023 | |
| AntiBERTy | BERT | 26M | GH | OAS | MIT | 2021 | |
| AntiBERTa | RoBERTa | 86M | GH | OAS | Apache-2.0 | 2021 | |
| Sapiens | RoBERTa | 569K | GH | OAS | MIT | 2021 | |
| IgLM | GPT-2 | 1.4M-13M | GH | OAS | JHU | 2021 | |
| AntiBERTa2 | RoFormer | 203M | HF | OAS | NC | 2022 | |
| AbLang | RoBERTa | 86M | GH | OAS | BSD | 2022 | |
| ProGen2-OAS | GPT | 764M | GH | OAS | BSD-3-Clause | 2022 | |
| blBALM | RoBERTa | 650M | GH | MIT | 2023 | ||
| bioBALM | ESM-2 | 150M | GH | OAS | MIT | 2023 | |
| SC-AIR-BERT | BERT | 23M | GH | BCRs and TCRs | PNL | Zhao et al. (2023) | 2023 |
| AbLang2 | ESM-2 | 45M | GH | OAS | BSD | 2024 | |
| IgBert, IgT5 | BERT, T5 | 420M-3B | HF | OAS | MIT | 2024 |
Specification for general protein language models (top) and antibody-specific language models (bottom). Base, base model architecture; Params, number of trainable parameters; Code, model and code availability; Training data, dataset used for training; License, release license; Refs, references; Year, year of first release. ProtTrans is a collection of models with base architecture Transformer-XL (), XLNet (Yang et al., 2020), BERT (), Albert (), Electra (), T5 (). For disambiguation we refer to Baseline Antibody Language Model (BALM) as blBALM and to Bio-inspired Antibody Language Model (BALM) as bioBALM. GH: GitHub. HF: HuggingFace.
The concept of PLMs was later applied to antibody sequences, resulting in Antibody Language Models (ALM) (Table 1). Most of these models have been trained on unpaired data, either a single model including both chain types (AntiBERTa (), AntiBERTy (), IgLM (), BALM-unpaired (), Bio-inspired Antibody Language Model ()) or with chain-specific models [Sapiens (), AbLang ()]. Other models, such as BALM-paired (), ESM2-paired (), SC-AIR-BERT (Zhao et al., 2023), and AbLang2 (), make use of paired sequence information to capture inter-chain residue dependencies. Applications of these models include paratope prediction [AntiBERTa ()), humanization (Sapiens ()], sequence completion [AbLang ()] and generation conditioned on species and chain type [AntiBERTy (), IgLM ()]. pAbT5 () stands out as it is tasked to predict one chain type starting from the other.
3 Antibody folding and inverse folding
AlphaFold2 () led to impressive improvements in the accuracy of protein sequence-to-structure prediction. One of major bottlenecks for the runtime of AlphaFold2 is the need to construct a Multiple Sequence Alignment (MSA) from the input sequence. Recently, structural information learned with PLMs has been leveraged to substitute the MSA dependency leading to the release of sequence-only models (ESMFold (), BALMFold (), OmegaFold (Wu Ruidong et al., 2022), HelixFold-Single () and EMBER3D (Weissenow et al., 2022)). Barret and coauthors () compared an AlphaFold architecture using either only MSA or sequence (MonoFold) or both inputs together (PolyFold) and showed that the two input modes are complementary to each other, although using MSA has still higher performance. The reliance of AlphaFold2 on the MSA is also reflected in the lower accuracy when predicting the structure of orphan and de novo proteins. Both RGN2 () and trRosettaX-Single (Wang et al.) have been proposed to address this limitation. More recently several models have been published addressing the structure prediction at the atomic level [Protpardelle (), EquiFold (), RoseTTAFold All-Atom ()] instead of the amino acid level, opening up new possibilities for modelling protein complexes with DNA, RNA, and small molecules. This is also the focus of AlphaFold3 ().
The prediction of antibody structure carries additional challenges with respect to other proteins. The Complementary Determining Regions (CDRs) responsible for the binding with the antigen are the most variable and therefore difficult to predict, especially the CDR3 region of the heavy chain (HCDR3). Several models have been published to address these challenges [ABlooper (Brennan et al., 2022), IgFold (), EquiFold (), DeepAB (), ABodyBuilder2 ()]. ABodyBuilder2 is part of the ImmuneBuilder () suite and has better performance with respect to ABlooper, IgFold, EquiFold and AlphaFold-Multimer (), specifically for the prediction of the HCDR3 loops, achieving a RMSD of 2.81Â Ã…. This improvement was achieved by using an ensemble of four models built on the structure module of AlphaFold-Multimer followed by refinement with OpenMM and pdbfixer. tFold-Ab (Wu Jiaxiang et al., 2022) first computes single chain structure predictions using the PLM ProtXLNet () and then predicts the multimer conformation of the heavy and light chains using a simplified version of the Evoformer module of AlphaFold that takes single sequence in input. However, the availability of this method only as a web-server hinders the possibility to assess its performance with respect to available benchmarks.
DL has been recently applied to the inverse folding problem, that is the problem of determining which sequences can fold into a predefined structure. This is especially useful in the context of protein and antibody design. For instance, the structure for a particular antibody sequence can be first derived with folding models, further optimized in structure space for developability properties and then converted back into sequence format for experimental validation. Inverse folding models for general proteins include ESM-IF1 (), KW-Design (), ProRefiner (Zhou et al., 2023), GraDe_IF (Yi et al., 2023), ProteinMPNN () and SeqPredNN (). Inverse folding methods specifically designed for antibodies are AntiFold (), AbMPNN (), IgDesign (Shanehsazzadeh et al., 2023) and DiscoTope-3.0 (). AntiFold is a version of ESM-IF1 fine-tuned on experimental and predicted antibody structures.
Table 2 summarizes the information of the models mentioned in this section with the respective licenses.
TABLE 2
| Model | Params | Code | Training data | License | Refs | Year |
|---|---|---|---|---|---|---|
| AlphaFold2 | 93M | GH | PDB | Apache 2.0 | 2021 | |
| RGN2 | NA | GH | ProteinNet12, ASTRAL SCOPe | NA | 2021 | |
| OpenFold | 93M | GH | PDB | Apache 2.0 | 2022 | |
| ESMFold | 692M | GH | UniRef50, PDB, AlphaFold2 predictions | MIT | 2022 | |
| trRosettaX-Single | NA | Web-server | PDB | NA | Wang et al. (2022) | 2022 |
| OmegaFold | 795M | GH | UniRef50, PDB | Apache 2.0 | Wu et al. (2022a) | 2022 |
| EquiFold | 2M | GH | Rocklin2017 (), SAbDab | Apache 2.0 | 2022 | |
| HelixFold-Single | NA | GH | UniRef30, Uniclust30, PDB, AFDB | Apache 2.0 | 2022 | |
| EMBER3D | NA | GH | SidechainNet | MIT | Weissenow et al. (2022) | 2022 |
| MonoFold, PolyFold | NA | GH | PDB | NC-SA 4.0 | 2022 | |
| BALMFold | NA | GH | SAbDab | MIT | 2023 | |
| Protpardelle | 22M | GH | CATH S40 | MIT | 2023 | |
| RoseTTAFold All-Atom | NA | GH | PBD, Cambridge Structural Database | BSD | 2023 | |
| AlphaFold3 | NA | NA | PDB, MGnify, Rfam, JASPAR | NA | 2024 | |
| ABlooper | 662K | GH | SAbDab | BSD 3-Clause | Brennan et al. (2022) | 2021 |
| DeepAb | 6.4M x5 | GH | SAbDab | Rosetta-DL | 2021 | |
| ABodyBuilder2 | 7.6M + 26.8M x3 | GH | SAbDab | BSD 3-Clause | 2022 | |
| tFold-Ab | NA | NA | SAbDab | NA | Wu et al. (2022b) | 2022 |
| IgFold | 1.5M x4 | GH | SAbDab, AlphaFold predictions | JHU | 2023 | |
| ESM-IF1 | 124M | GH,HF | CATH | MIT | 2022 | |
| ProRefiner | 2.5M | GH | CATH | NA | Zhou et al. (2023) | 2023 |
| GraDe_IF | 3.8M | GH | CATH | NA | Yi et al. (2023) | 2023 |
| ProteinMPNN | 1.7M | GH | PDB | Apache 2.0 | 2023 | |
| SeqPredNN | NA | GH | PDB | GPL-3.0 | 2023 | |
| AntiFold | 141.6M | OPIG | SAbDab, ABodyBuilder2 predictions | BSD 3-Clause | 2023 | |
| AbMPNN | NA | Zenodo | SAbDab, ABodyBuilder2 predictions | CC-4.0 | 2023 | |
| IgDesign | NA | NA | PDB, SAbDab | NA | Shanehsazzadeh et al. (2023) | 2023 |
| DiscoTope-3.0 | NA | BioLib | PDB | On request | 2024 | |
| KW-Design | NA | GH | CATH | Apache 2.0 | 2024 |
Specification for general protein folding models (top), antibody-specific folding models (middle) and inverse folding models (bottom). DeepAb and ABodyBuilder2 are ensemble of models. The table follows the same structure as Table 1, apart from the Base column that does not apply here. The models for which we could not determine the number of parameters are indicated as NA in the Params column. GH: GitHub. HF: HuggingFace.
4 Developability
Screening for a high affinity antibody is only the first step in the antibody development process. To match the characteristics of therapeutic antibodies, the selected antibody must be further optimized to adhere to the properties of therapeutic antibodies (developability) (; ). Raybould and coauthors () developed the Therapeutic Antibody Profile, a webserver used to evaluate antibody developability as including immunogenicity, solubility, specificity, stability, manufacturability, and storability. They focused on five metrics calculated from the CDRs based on total length, surface hydrophobicity, positive and negative charge of surface patches, and net charge of VH and VL chains.
Habib and coauthors () have compiled a list of 40 sequence- and 46 structure-based developability parameters (DP). They showed that sequence DPs are better predictors than structure DPs in single DP ablation experiments using Multiple Linear Regression (MLR) layer, especially when using sequence-based embeddings generated with the PLM ESM-1v as features. This reflects the fact that ESM-1v has direct access to the sequence information it is trained on and only learns structure information indirectly from sequence.
For the scope of this review, we will focus on ML methods for humanization and prediction of solubility, Methods for aggregation predictions are not mentioned, and viscosity. Humanization is the process of lowering the risk of immunogenicity by increasing the human-like content of the antibody sequence while maintaining binding affinity (). Solubility, aggregation, and viscosity are properties that determine if an antibody will perform well in a solution. These are properties that are important for sub-cutaneous delivery, which represent an attractive alternative with respect to intra-venous delivery because of ease and speed of administration (Viola et al., 2018) but require maximizing the dose (). If an antibody has been selected in a screening procedure that uses the naive immune repertoire of a non-human species, when administered to a patient it can elicit an immunogenic response, whereby Anti-Drug Antibodies (ADAs) are raised against the engineered antibody by the immune system of the host. Humanization is the process by which non-human residues lying outside of the epitope-binding regions are iteratively swapped with human-like residues to increase the humanness of the antibody while retaining its binding affinity. Several ML methods have been presented to address humanization. Hu-mAb () uses a set of V gene type-specific Random Forest (RF) models to iteratively select the top scoring single-site mutation in the framework region based on a humanness score until it reaches a target score.
BioPhi () is a platform for humanness evaluation and humanization. Humanness is evaluated using OASis, a database of 9-mers (k-mers of nine residues) constructed from over 188 million sequences from 231 human subjects comprising 26 studies. Humanization is performed with Sapiens, which comprises two chain-specific ALMs, one for the heavy and one for the light chain, trained using a MLM objective on 20 million heavy chain human sequences from 38 OAS studies from 2011 to 2017, and 19 million light chain human sequences from 14 OAS studies from 2011 to 2017. The Sapiens network returns per-position posterior probabilities for all 20 amino acids conditioned on the input sequence that are used to introduce humanizing mutations. BioPhi includes an interface (Designer) that allows to select which of the suggested mutations the user would like to introduce. The software is available as a web-server or with a command line interface for processing set of sequences.
An alternative method not based on DL is AntPack (). First, the authors developed a new antibody numbering method that is much faster than existing methods. Then they fitted a gaussian mixture model on the numbered antibody sequences using 60 million heavy and 70 million light sequences from the cAbRep database (). The authors originally trained the model on human sequences of the OAS dataset and by inspecting the sequences of the training set that were responsible of giving unusually high probability to mouse sequences were able to identify more than 7,000 sequences that had been incorrectly labelled as both mouse and human.
Therapeutic antibodies are often produced and utilized at high concentration, so they require high solubility and low aggregation. Several methods have been proposed to predict solubility from sequence-based and structure-based features. SOLart () is a Random Forest model trained on a combination of 52 sequence-based and structure-based features. In a comparison with 9 SOTA methods using a dataset of experimentally determined and modelled structures of S. cerevisiae it was able to achieve a Pearson correlation of 0.65. Language models have been recently employed for binary prediction of soluble versus non soluble proteins, either by using fine-tuning or by training only the last classification layer of the neural network. NetSolP (Thumuluri et al., 2022), an ensemble of fine-tuned ESM1b models, achieves a performance comparable to the version of ESM with MSA input on datasets of proteins expressed in E. coli that were assessed for solubility.
Another favorable property related to developability for antibodies to perform well in a solution is low viscosity. proposed PfAbNet-viscosity, a 3D convolutional neural network to predict viscosity from antibody structures trained under a low training data regime. The authors used data augmentation to try to mitigate the limitations of working with few antibodies for training. PfAbNet-viscosity outperformed two SOTA models, Sharma () and Surface Charge Model (SCM) ().
5 Discussion
The field of antibody discovery and development is experiencing an acceleration thanks to the successes of DL in protein structure prediction and representation learning from protein sequences. Here we focused on the applicability of these methods and resources in an industrial setting, especially with respect to the possibility of integrating these methods into commercial products. In Table 3 we highlighted a selection of protein and antibody-specific models for sequence, folding, inverse folding and developability (humanization). Our choice is based on our assessment of usability and license considerations.
TABLE 3
| Type | Models | Strengths | Limitations |
|---|---|---|---|
| PLM | ESM-2, ProtTrans | available on HuggingFace multiple model sizes | no antibody-specific pretraining |
| ProtGPT2 | available on HuggingFace | ||
| Folding | AlphaFold2, OpenFold | accurate prediction of single protein chains and protein complexes (AlphaFold-Multimer) | only residue-level modelling |
| RosettaFold All-Atom | atomic-level modelling can predict multiple types of protein complexes | not antibody-specific | |
| Inverse Folding | ESM-IF1 | large augmented training set | not antibody-specific |
| ProteinMPNN | experimentally validated | ||
| ALM | AbLang2 | focal loss mitigates germline bias | |
| IgBert, IgT5 | large antibody training set size | not available on HuggingFace limited training set for paired antibodies | |
| Ab Folding | ABodyBuilder2 | computational cost improved accuracy at HCDR3 regions integrated in SPACE2 () | not available on HuggingFace |
| ABlooper | CDR specific | ||
| Ab Inverse Folding | AntiFold | antibody-specific | not available on HuggingFace |
| Developability | BioPhi (Sapiens) | humanness report web-based and command line interfaces | trained on unpaired chain data |
| AntPack | computational cost |
A selection of models with highlighted strengths and limitations. General protein models (top) and antibody-specific models (bottom).
To evaluate the performance of these models, a benchmark has been recently proposed, Fitness Landscape for Antibodies (FLAb), that covers six properties of therapeutic antibodies: expression, thermostability, immunogenicity, aggregation, polyreactivity and binding affinity (). The models considered in the study include decoder-only generative models trained with next token prediction [ProGen2 (), IgLM () and ProtGPT2 ()], encoder-only models for representation learning [AntiBERTy ()], inverse folding models [ProteinMPNN (), ESM-IF ()] and a physics-based model [Rosetta ()]. The authors showed that none of the models outperformed all the other models for all the tasks, underscoring the challenges in the development and application of these models for specific tasks.
Additional insights on how to improve these models will come from extending these benchmarks to different models, including PLMs like ESM-2 fine-tuned on antibody sequence data. With the increased availability of predicted antibody structures the current direction in the field is to integrate structural information in ALMs, with AntiBERTa2 () being an example. Antibody datasets, such as OAS, suffer from a germline content bias that can prevent the model from suggesting mutations that are further away from the germline sequence space. AbLang2 () is a recent model that addresses this bias, where the authors trained the model to focus more on non-germline residues using focal loss instead of cross-entropy loss to handle the class imbalance of germline versus non-germline residues in the training data.
The availability of models specifically developed for antibody structure prediction and inverse folding models allows to address the developability optimization problem both in sequence and structure space. This is particularly important as optimization in sequence space appears to be more constrained than in structure space (as mentioned in the developability cartography study ()).
Humanization is the process that is best addressed by current developability methods, while other methods aimed at predicting solubility and viscosity suffers from the limited availability of experimental data for training and have been exploring less the application of language models.
These are exciting times for antibody discovery and development with AI that is being leveraged as a catalyst to accelerate and de-risk drug development on many fronts. We are starting to see how the process of bringing a drug from discovery to pre-clinical and clinical trials can be shortened and how the costs of this process can be reduced. The next few years will continue to see a fast-paced development and integration of these methods and resources in industrial applications, with the goal of ensuring that a newly found treatment can arrive faster to the patients.
Statements
Author contributions
LS: Writing–original draft, Writing–review and editing, Investigation. MB: Investigation, Writing–original draft. IX: Conceptualization, Writing–review and editing. BA: Conceptualization, Supervision, Writing–review and editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
Authors LS, MB, IX, and BA were employed by JSR Life Sciences.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AbramsonJ.AdlerJ.DungerJ.EvansR.GreenT.PritzelA.et al (2024). Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature630, 493–500. 10.1038/s41586-024-07487-w
2
Adriaan LateganF.SchreiberC.PattertonH. G. (2023). SeqPredNN: a neural network that generates protein sequences that fold into specified tertiary structures. BMC Bioinforma.24 (1), 373. 10.1186/s12859-023-05498-4
3
AgrawalN. J.HelkB.KumarS.ModyN.SathishH. A.SamraH. S.et al (2015). Computational tool for the early screening of monoclonal antibodies for their viscosities. mAbs8 (1), 43–48. 10.1080/19420862.2015.1099773
4
AhdritzG.BouattaN.FloristeanC.KadyanS.XiaQ.GereckeW.et al (2022). OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization. BioRxiv, Preprint. (Accessed November 22, 2022). 10.1038/s41592-024-02272-z
5
AhmedE.EssamH.Salah-EldinW.MoustafaW.MohamedE.RochereauC.et al (2023). Ankh: optimized Protein Language model unlocks general-purpose modelling. preprint, Bioinformatics. (Accessed January 16 2023). 10.1101/2023.01.16.524265
6
AhmedE.HeinzingerM.DallagoC.RehawiG.WangYuJonesL.et al (2022). ProtTrans: toward understanding the language of Life through self-supervised learning. IEEE Trans. Pattern Analysis Mach. Intell.44 (10), 7112–7127. 10.1109/TPAMI.2021.3095381
7
AlamdariS.ThakkarN.Van Den BergR.LuA. X.FusiN.AminiA. P.et al (2023). Protein generation with evolutionary diffusion: sequence is all you need. BioRxiv. (Accessed September 12, 2023). 10.1101/2023.09.11.556673
8
BarrettT. D.Villegas-MorcilloA.RobinsonL.GaujacB.AdméteD.SaquandE.et al (2022). ManyFold: an efficient and flexible library for training and validating protein folding models. Bioinformatics39 (1). 10.1093/bioinformatics/btac773
9
BartonJ.JacobD. G.LeemJ. (2024). Enhancing Antibody Language Models with structural information. BioRxiv. (Accessed January 04, 2024). 10.1101/2023.12.12.569610
10
BenderA.Cortes-CirianoI. (2021). Artificial intelligence in drug discovery: what is realistic, what are illusions? Part 2: a discussion of chemical and biological data. Drug Discov. Today26 (4), 1040–1052. 10.1016/j.drudis.2020.11.037
11
BenderA.Cortés-CirianoI. (2021). Artificial intelligence in drug discovery: what is realistic, what are illusions? Part 1: ways to make an impact, and why we are not there yet. Drug Discov. Today26 (2), 511–524. 10.1016/j.drudis.2020.12.009
12
BrandesN.OferD.PelegY.RappoportN.LinialM. (2022). ProteinBERT: a universal deep-learning model of protein sequence and function. Bioinformatics38 (8), 2102–2110. 10.1093/bioinformatics/btac020
13
BrennanA.GeorgesG.BujotzekA.DeaneC. M. (2022). ABlooper: fast accurate antibody CDR loop structure prediction with accuracy estimation. Bioinformatics38 (7), 1877–1880. 10.1093/bioinformatics/btac016
14
BrennanA.WongW.KiBoylesF.GeorgesG.BujotzekA.ImmuneBuilderC. M. D. (2023). ImmuneBuilder: deep-Learning models for predicting the structures of immune proteins. Commun. Biol.6 (1), 575. 10.1038/s42003-023-04927-7
15
BurbachS. M.BrineyB. (2024). Improving antibody language models with native pairing. Patterns (N Y). 5 (5), 100967. 10.1016/j.patter.2024.100967
16
CarterP. J.RajpalA. (2022). Designing antibodies as therapeutics. Cell185 (15), 2789–2805. 10.1016/j.cell.2022.05.029
17
ChanA. C.CarterP. J. (2010). Therapeutic antibodies for autoimmunity and inflammation. Nat. Rev. Immunol.10 (5), 301–316. 10.1038/nri2761
18
ChowdhuryR.BouattaN.BiswasS.RochereauC.ChurchG. M.SorgerP. K.et al (2021). Single-sequence protein structure prediction using language models from deep learning. preprint. BioRxiv. (Accessed August 04, 2021). 10.1038/s41587-022-01432-w
19
ChuA. E.ChengL.El NesrG.XuM.HuangP.-S. (2023). An all-atom protein generative model. BioRxiv. (Accessed May 25, 2023). 10.1101/2023.05.24.542194
20
ChungyounM.RuffoloJ.GrayJ. (2024). FLAb: benchmarking deep learning methods for antibody fitness prediction. preprint. BioRxiv. (Accessed January 15, 2024). 10.1101/2024.01.13.575504
21
ClarkK.LuongM.-T.QuocV.LeChristopherD. (2020). Manning. ELECTRA: pre-training text encoders as discriminators rather than generators. arXiv:2003.10555. 10.48550/arXiv.2003.10555
22
DaiZ.YangZ.YangY.CarbonellJ.LeQ. V.SalakhutdinovR. (2019). Transformer-XL: attentive Language Models beyond a fixed-length context. arXiv:1901.02860, 1. 10.48550/arXiv.1901.02860
23
DauparasJ.AnishchenkoI.BennettN.BaiH.RagotteR. J.MillesL. F.et al (2023). Robust deep learning based protein sequence design using ProteinMPNN. Sci.378 (6615), 49–56. 10.1126/science.add2187
24
DounasA.CotetT.-S.YermanosA. (2024). Learning immune receptor representations with protein language models. arXiv. 10.48550/arXiv.2402.03823
25
DreyerF. A.CuttingD.SchneiderC.KenlayH.DeaneC. M. (2023). Inverse folding for antibody sequence design using deep learning. arXiv. 10.3389/fmolb.2023.1237621
26
EvansR.O’NeillM.PritzelA.AntropovaN.SeniorA.GreenT.et al (2021). Protein complex prediction with AlphaFold-Multimer. BioRxiv. (Accessed March 10, 2022). 10.1101/2021.10.04.463034
27
EversA.MalhotraS.SoodV. D. (2023). In silico approaches to deliver better antibodies by design – the past in The present and the future. arXiv. 10.48550/arXiv.2305.07488
28
FangX.WangF.LiuL.HeJ.LinD.XiangY.et al (2023). A method for multiple-sequence-alignment-free protein structure prediction using a protein language model. Nat. Mach. Intell.5 (10), 1087–1096. 10.1038/s42256-023-00721-6
29
Fernández-QuinteroM. L.LjungarsA.WaiblF.GreiffV.AndersenJ. T.GjølbergT. T.et al (2023). Assessing developability early in the discovery process for novel biologics. mAbs15 (1), 2171248. 10.1080/19420862.2023.2171248
30
FerruzN.SchmidtS.HöckerB. (2022). ProtGPT2 is a deep unsupervised language model for protein design. Nat. Commun.13 (1), 4348. 10.1038/s41467-022-32007-7
31
GaoZ.TanC.ChenX.ZhangY.XiaJ.LiS.et al (2024). KW-DESIGN: pushing the limit of protein design via knowledge refinement. arXiv. 10.48550/arXiv.2305.15151
32
GravesJ.JacobB.PriegoE.MakkapatiN.ParishS.MedellinB.et al (2020). A review of deep learning methods for antibodies. Antibodies9 (2), 12. 10.3390/antib9020012
33
GuoY.ChenK.KwongP. D.ShapiroL.ShengZ. (2019). cAb-rep: a database of curated antibody repertoires for exploring antibody diversity and predicting antibody prevalence. Front. Immunol.10, 2365. 10.3389/fimmu.2019.02365
34
HabibB.SmorodinaE.ParisetM.ZhongJ.AkbarR.ChernigovskayaM.et al (2023). Biophysical cartography of the native and human-engineered antibody landscapes quantifies the plasticity of antibody developability. BioRxiv. (Accessed December 13, 2023). 10.1101/2023.10.26.563958
35
Haraldson HøieM.GadeF. S.JohansenJ. M.WürtzenC.WintherO.NielsenM.et al (2024). DiscoTope-3.0: improved B-cell epitope prediction using inverse folding latent representations. Front. Immunol.15, 1322712. 10.3389/fimmu.2024.1322712
36
Haraldson HøieM.HummerA. M.OlsenT. H.NielsenM.DeaneC. M. (2024a). AntiFold: improved antibody structure design using inverse folding. arXiv. 10.48550/arXiv.2405.03370
37
HouQ.Marc KwasigrochJ.RoomanM.PucciF. (2020). SOLart: a structure-based method to predict protein solubility and aggregation. Bioinformatics36 (5), 1445–1452. 10.1093/bioinformatics/btz773
38
HsuC.VerkuilR.LiuJ.LinZ.HieB.SercuT.et al (2022). Learning inverse folding from millions of predicted structures. BioRxiv. (Accessed September 06, 2022). 10.1101/2022.04.10.487779
39
IslamS.ElmekkiH.AhmedE.BentaharJ.DrawelN.RjoubG.et al (2023). A comprehensive survey on applications of transformers for deep learning tasks. arXiv. 10.48550/arXiv.2306.07303
40
JacobD.ChangM.-W.LeeK.ToutanovaK. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. arXiv. 10.48550/arXiv.1810.04805
41
JaffeD. B.ShahiP.AdamsB. A.ChrismanA. M.FinneganP. M.RamanN.et al (2022). Functional antibodies exhibit light chain coherence. Nature611 (7935), 352–357. 10.1038/s41586-022-05371-z
42
JingH.GaoZ.XuS.ShenT.PengZ.HeS.et al (2023). Accurate prediction of antibody function and structure using bio-inspired Antibody Language model. Preprint. Bioinformatics.25 (4), bbae245. 10.1093/bib/bbae245
43
JiskootW.HaweA.MenzenT.VolkinD. B.CrommelinD. J. A. (2022). Ongoing challenges to develop high concentration monoclonal antibody-based formulations for subcutaneous administration: quo vadis?J. Pharm. Sci.111 (4), 861–867. 10.1016/j.xphs.2021.11.008
44
JumperJ.EvansR.PritzelA.GreenT.FigurnovM.RonnebergerO.et al (2021). Highly accurate protein structure prediction with AlphaFold. Nature596 (7873), 583–589. 10.1038/s41586-021-03819-2
45
KenlayH.DreyerF. A.KovaltsukA.MiketaD.DouglasP.DeaneC. M. (2024). Large scale paired antibody language models. arXiv. 10.48550/arXiv.2403.17889
46
KhakzadH.IgashovI.SchneuingA.GoverdeC.BronsteinM.CorreiaB. (2023). A new age in protein design empowered by deep learning. Cell Syst.14 (11), 925–939. 10.1016/j.cels.2023.10.006
47
KhetanR.CurtisR.DeaneC. M.HadsundJ. T.KarU.KrawczykK.et al (2022). Current advances in biopharmaceutical informatics: guidelines, impact and challenges in the computational developability assessment of antibody therapeutics. mAbs14 (1), 2020082. 10.1080/19420862.2021.2020082
48
KimJ.McFeeM.FangQ.AbdinO.KimP. M. (2023). Computational and artificial intelligence-based methods for antibody development. Trends Pharmacol. Sci.44 (3), 175–189. 10.1016/j.tips.2022.12.005
49
Koehler LemanJ.WeitznerB. D.LewisS. M.Adolf-BryfogleJ.AlamN.AlfordR. F.et al (2020). Macromolecular modeling and design in Rosetta: recent methods and frameworks. Nat. Methods17 (7), 665–680. 10.1038/s41592-020-0848-2
50
KrishnaR.WangJ.AhernW.SturmfelsP.KalvetI.LeeG. R.et al (2024). Generalized biomolecular modeling and design with RoseTTAFold all-atom. Sci.384 (6693), eadl2528. 10.1126/science.adl2528
51
LanZ.ChenM.GoodmanS.GimpelK.SharmaP.SoricutR. (2019). ALBERT: a lite BERT for self-supervised learning of language representations. arXiv. 10.48550/arXiv.1909.11942
52
LeeJ. H.YadollahpourP.WatkinsA.FreyN. C.Leaver-FayA.RaS.et al (2022). Protein structure prediction with a novel coarse-grained structure representation. Preprint. BioRxiv. (Accessed October 08, 2022). 10.1101/2022.10.07.511322
53
LeemJ.MitchellL. S.FarmeryJ. H. R.BartonJ.GalsonJ. D. (2022). Deciphering the language of antibodies using self-supervised learning. Patterns3 (7), 100513. 10.1016/j.patter.2022.100513
54
LinZ.AkinH.RaoR.HieB.ZhuZ.LuW.et al (2023). Evolutionary-scale prediction of atomic level protein structure with a language model. Sci.379 (6637), 1123–1130.
55
LuR.-M.HwangY.-C.LiuI.-JuLeeC.-C.TsaiH.-Z.LiH.-J.et al (2020). Development of therapeutic antibodies for the treatment of diseases. J. Biomed. Sci.27 (1), 1. 10.1186/s12929-019-0592-z
56
MarksC.HummerA. M.ChinM.DeaneC. M. (2021). Humanization of antibodies using a machine learning approach on large-scale repertoire data. Bioinform.37 (22), 4041–4047.
57
Nam KimD.McNaughtonA. D.KumarN. (2024). Leveraging artificial intelligence to expedite antibody design and enhance antibody–antigen interactions. Bioengineering11 (2), 185. 10.3390/bioengineering11020185
58
NijkampE.JeffreyR.WeinsteinE. N.NaikN.AliM. (2022). ProGen2: exploring the boundaries of Protein Language Models. arXiv. 10.48550/arXiv.2206.13517
59
NotinP.KollaschA. W.RitterD.van NiekerkL.PaulS.SpinnerH.et al (2023). ProteinGym: large-scale benchmarks for protein fitness prediction and design. BioRxiv. 10.1101/2023.12.07.570727
60
OferD.BrandesN.LinialM. (2021). The language of proteins: NLP, machine learning and protein sequences. Comput. Struct. Biotechnol. J.19, 1750–1758. 10.1016/j.csbj.2021.03.022
61
OlsenT. H.MoalI. H.DeaneC. M. (2024a). Addressing the antibody germline bias and its effect on language models for improved antibody design. BioRxiv. (Accessed February 07, 2024). 10.1101/2024.02.02.578678
62
OlsenT. H.MoalI. H.DeaneC. M. (2024b). Addressing the antibody germline bias and its effect on language models for improved antibody design. bioRxiv. Available at: https://www.biorxiv.org/content/early/2024/02/07/2024.02.02.578678.full.pdf. 10.1101/2024.02.02.578678
63
OlsenT. H.MoalI. H.DeaneC. M. (2022). AbLang: an antibody language model for completing antibody sequences. Bioinforma. Adv.2 (1), vbac046. 10.1093/bioadv/vbac046
64
ParkinsonJ.WangW. (2024). For antibody sequence generative modeling, mixture models may be all you need. Bioinformatics40, btae278. 10.1093/bioinformatics/btae278
65
PrihodaD.MaamaryJ.WaightA.JuanV.Fayadat-DilmanL.SvozilD.et al (2022). BioPhi: a platform for antibody design, humanization, and humanness evaluation based on natural antibody repertoires and deep learning. mAbs14 (1), 2020203. 10.1080/19420862.2021.2020203
66
PujolsJ.IglesiasV.SantosJ.KuriataA.VenturaS. (2022). A3D 2.0 update for the prediction and optimization of protein solubility. Methods Mol Biol.2406, 65–84. 10.1007/978-1-0716-1859-2_3
67
RaffelC.ShazeerN.RobertsA.LeeK.NarangS.MatenaM.et al (2023). Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv.
68
RaiB. K.ApgarJ. R.BennettE. M. (2023). Low-data interpretable deep learning prediction of antibody viscosity using a biophysically meaningful representation. Sci. Rep.13 (1), 2917. 10.1038/s41598-023-28841-4
69
RaybouldM. I. J.TurnbullO. M.SuterA.GulogluB.DeaneC. M. (2024). Contextualising the developability risk of antibodies with lambda light chains using enhanced therapeutic antibody profiling. Commun. Biol.7 (1), 62. 10.1038/s42003-023-05744-8
70
RivesA.MeierJ.SercuT.GoyalS.LinZ.LiuJ.et al (2021). Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proc. Natl. Acad. Sci.118 (15), e2016239118. 10.1073/pnas.2016239118
71
RocklinG. J.ChidyausikuT. M.GoreshnikI.FordA.HoulistonS.LemakA.et al (2017). Global analysis of protein folding using massively parallel design, synthesis, and testing. Science357 (6347), 168–175. 10.1126/science.aan0693
72
RuffoloJ. A.ChuL.-S.MahajanS. P.GrayJ. J. (2023). Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies. Nat. Commun.14 (1), 2389. 10.1038/s41467-023-38063-x
73
RuffoloJ. A.GrayJ. J.SulamJ. (2021). Deciphering antibody affinity maturation with language models and weakly supervised learning. arXiv. 10.48550/arXiv.2112.07782
74
RuffoloJ. A.SulamJ.GrayJ. J. (2022). Antibody structure prediction using interpretable deep learning. Patterns3 (2), 100406. 10.1016/j.patter.2021.100406
75
ShanehsazzadehA.AlverioJ.KasunG.LevineS.KhanJ. A.ChungC.et al (2023). In vitro validated antibody design against multiple therapeutic antigens using generative inverse folding. BioRxiv. Absci Corporation, 570889. 10.1101/2023.12.08.570889
76
SharmaV. K.PatapoffT. W.KabakoffB.PaiS.HilarioE.ZhangB.et al (2014). In silico selection of therapeutic antibodies for development: viscosity, clearance, and chemical stability. Proc. Natl. Acad. Sci. U. S. A.111 (52), 18601–18606. 10.1073/pnas.1421779112
77
ShuaiR. W.RuffoloJ. A.GrayJ. J. (2021). Generative language modeling for antibody design. BioRxiv. (Accessed December 20, 2022). 10.1101/2021.12.13.472419
78
SimonK. S. C.WeiK. Y. (2023). Generative antibody design for complementary chain pairing sequences through encoder-decoder Language Model. arXiv. 10.48550/arXiv.2301.02748
79
SpoendlinF. C.AbanadesB.RaybouldM. I. J.WongW.KiGeorgesG.DeaneC. M. (2023). Improved computational epitope profiling using structural models identifies a broader diversity of antibodies that bind to the same epitope. Front. Mol. Biosci.10, 1237621. 10.3389/fmolb.2023.1237621
80
SteineggerM.SödingJ. (2018). Clustering huge protein sequence sets in linear time. Nat. Commun.9 (1), 2542. 10.1038/s41467-018-04964-5
81
SuzekB. E.HuangH.McGarveyP.MazumderR.WuC. H. (2007). UniRef: comprehensive and non-redundant UniProt reference clusters. Bioinformatics23 (10), 1282–1288. 10.1093/bioinformatics/btm098
82
ThumuluriV.MartinyH.-M.ArmenterosJ. J. A.SalomonJ.NielsenH.JohansenA. R. (2022). NetSolP: predicting protein solubility in Escherichia coli using language models. Bioinformatics38 (4), 941–946. 10.1093/bioinformatics/btab801
83
UniProt ConsortiumT.BatemanA.MartinM.-J.OrchardS.MagraneM.AhmadS.et al (2023). UniProt: the universal protein knowledgebase in 2023. Nucleic Acids Res.51 (D1), D523–D531. 10.1093/nar/gkac1052
84
ValentiniG.MalchiodiD.GliozzoJ.MesitiM.Soto-GomezM.CabriA.et al (2023). The promises of large language models for protein design and modeling. Front. Bioinforma.3, 1304099. 10.3389/fbinf.2023.1304099
85
VaswaniA.ShazeerN.ParmarN.UszkoreitJ.JonesL.GomezA. N.et al (2017). Attention is all you need. arXiv:1706.03762. 10.48550/arXiv.1706.03762
86
ViolaM.SequeiraJ.SeiçaR.VeigaF.SerraJ.SantosA. C.et al (2018). Subcutaneous delivery of monoclonal antibodies: how do we get there?J. Control. Release286, 301–314. 10.1016/j.jconrel.2018.08.001
87
WangW.PengZ.YangJ. (2022). Single-sequence protein structure prediction using supervised transformer protein language models. Nat. Comput. Sci.2, 804–814. 10.1038/s43588-022-00373-3
88
WeinerL. M.SuranaR.WangS. (2010). Monoclonal antibodies: versatile platforms for cancer immunotherapy. Nat. Rev. Immunol.10 (5), 317–327. 10.1038/nri2744
89
WeissenowK.HeinzingerM.SteineggerM.RostB. (2022). Ultra-fast protein structure prediction to capture effects of sequence variation in mutation movies. BioRxiv. (Accessed November 16, 2022). 10.1101/2022.11.14.516473
90
WuJ.WuF.JiangB.LiuW.ZhaoP. (2022b). tFold-ab: fast and accurate antibody structure prediction without sequence homologs. BioRxiv. (Accessed November 13, 2022). 10.1101/2022.11.10.515918
91
WuR.DingF.WangR.ShenR.ZhangX.LuoS.et al (2022a). High-resolution de novo structure prediction from primary sequence. BioRxiv. (Accessed July 22, 2022). 10.1101/2022.07.21.500999
92
YangZ.DaiZ.YangY.CarbonellJ.SalakhutdinovR.QuocV.et al (2020). Generalized autoregressive pretraining for language understanding. arXiv. 10.48550/arXiv.1906.08237
93
YiK.ZhouB.ShenY.LiòP.WangYu G. (2023). Graph denoising diffusion for inverse protein folding. arXiv. 10.48550/arXiv.2306.16819
94
ZhangW.WangH.FengN.LiY.GuJ.WangZ. (2023). Developability assessment at early-stage discovery to enable development of antibody-derived therapeutics. Antib. Ther.6 (1), 13–29. 10.1093/abt/tbac029
95
ZhaoYuSuX.ZhangW.MaiS.XuZ.QinC.et al (2023). SC-AIR-BERT: a pre-trained single-cell model for predicting the antigen-binding specificity of the adaptive immune receptor. Briefings Bioinforma.24 (4), bbad191. 10.1093/bib/bbad191
96
ZhouX.ChenG.YeJ.WangE.ZhangJ.MaoC.et al (2023). ProRefiner: an entropy-based refining strategy for inverse protein folding with global graph attention. Nat. Commun.14 (1), 7434. 10.1038/s41467-023-43166-6
Summary
Keywords
LLM, ALM (antibody language model), developability, inverse folding, deep learning, artificial intelligence
Citation
Santuari L, Bachmann Salvy M, Xenarios I and Arpat B (2024) AI-accelerated therapeutic antibody development: practical insights. Front. Drug Discov. 4:1447867. doi: 10.3389/fddsv.2024.1447867
Received
12 June 2024
Accepted
12 August 2024
Published
03 September 2024
Volume
4 - 2024
Edited by
Ferdinand Molnár, Nazarbayev University, Kazakhstan
Reviewed by
Markus Heinonen, Aalto University, Finland
Updates
Copyright
© 2024 Santuari, Bachmann Salvy, Xenarios and Arpat.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Bulak Arpat, barpat@jsrlifesciences.com
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.