Gene-LLMs: A Comprehensive Survey of Transformer-Based Genomic Language Models for Regulatory and Clinical Genomics

Balakrishnan PAnny Leema^*Dhivyashree A Anny LeemaMohammad SaadMohan Babu

• VIT University, Vellore, India

The convergence of natural language processing (NLP) and genomics has given rise to a new class of transformer-based models-Genome Large Language Models (Gene-LLMs)-capable of interpreting the language of life at unprecedented scale and resolution. It represent a revolution in the field of bioinformatics since they use only raw nucleotide sequences, gene expression data, and multi-omic annotations and their capacity of self-supervised pretraining to decipher complex regulatory grammars that are hidden in the genome. This survey proposal is an exemplar inclusive of the entire lifecycle of Gene-LLMs, including such stages as raw data ingestion and k-mer or gene-level tokenization, and tasks like masked nucleotide prediction and sequence alignment that are conducted for pretext learning. We specify their wide-range usage that goes through crucial subsequent activities such as finding the enhancer or promoter, modeling the chromatin state, predicting the RNA-protein, and creating synthetic sequences. We further explore how the Gene-LLMs approach has created an impact in functional genomics, clinical diagnostics and evolutionary inference by analysing a number of recent benchmarks: CAGI5, GenBench, NT-Bench, and BEACON.We also point to some of the new encoder-decoder modifications as well as theinclusion of position embeddings which are a feature specific to living organisms hinting at their interpretability and translational potential. The paper finally outlines a route to federated genomic learning, multimodal sequence modeling, and low-resource adaptation for rare variant discovery, branding Gene-LLMs as a corner stone technology for the responsible and proactive future of biomedicine.