Advancing Pre-training and Fine-tuning in Foundation Models, Volume II

This Research Topic is the second volume of Research Topic "Advancing Pre-training and Fine-tuning in Foundation Models". Please see the first Volume here.

Foundation models have fundamentally reshaped artificial intelligence by delivering powerful general-purpose capabilities across diverse tasks, domains, and modalities. Built upon the conventional two-stage paradigm of large-scale pre-training and task-specific fine-tuning, foundation models have achieved tremendous success in broad AI scenarios. However, recent technical advances have further evolved this workflow into a more comprehensive lifecycle, incorporating sophisticated post-training refinement and task-level adaptation pipelines, including instruction tuning, preference optimization, reinforcement learning alignment, multimodal alignment, and domain-specific specialization. These emerging techniques extend foundation model development from basic representation learning toward systematic capability acquisition, value alignment, domain specialization, and reliable real-world deployment.

Despite the substantial progress brought by classic pre-training and fine-tuning paradigms, newly emerging challenges continue to restrict the practical deployment and generalization of modern foundation models. As models grow larger, more multimodal, and more domain-integrated, pre-training faces critical bottlenecks regarding data quality and diversity, training scalability, computational and environmental costs, and effective fusion of heterogeneous modalities. Beyond traditional fine-tuning, post-training and advanced adaptation procedures struggle to balance model alignment, computational efficiency, knowledge retention, and robustness. Such limitations become especially prominent under real-world conditions, including distribution shifts, limited-data scenarios, and cross-domain or cross-modal transfer tasks.

Building upon the progress of Volume I, this Research Topic aims to address these next-generation challenges by advancing innovative pre-training, post-training, and adaptation methodologies that enable foundation models to be more efficient, robust, adaptable, and trustworthy. We invite contributions that cover the extended lifecycle of modern foundation models, ranging from scalable and sustainable pre-training strategies to advanced parameter-efficient adaptation, multimodal alignment, domain-specific specialization, and robust cross-domain generalization. Research bridging theoretical innovation and practical deployment, improving model reliability and alignment, or demonstrating tangible real-world application value is particularly encouraged.
This article collection invites original research, reviews, and perspectives that advance both the theoretical foundations and practical methodologies of next-generation foundation model training and adaptation. We particularly encourage work emphasizing data-centric pre-training, sustainable large-scale training, post-training alignment, parameter-efficient fine-tuning, robust cross-domain adaptation, and personalized continual learning. Relevant topics include novel model architectures, advanced optimization strategies, multimodal and domain-specific foundation models, efficient inference, knowledge transfer and retention, interpretability, reliability, and responsible AI governance.

Authors are encouraged to highlight how their work integrates pre-training, post-training, and downstream adaptation, addresses emerging real-world limitations, and demonstrates practical impacts in challenging application scenarios, such as precision healthcare, autonomous robotics, personalized education, and scientific discovery. Submissions should clearly articulate methodological innovations, practical significance, and ethical considerations for modern foundation model development. All manuscripts will undergo rigorous peer review following standard journal policies to ensure high quality, originality, and relevance to this Research Topic.

To further enhance the visibility and impact of this collection, one Best Paper Award will be presented to the most outstanding paper selected from all accepted submissions. The author(s) of the Best Paper will be granted a full waiver.