An Efficient Strategy for Fine-Tuning Large Language Models

Benjamin Marsh^1*Adam Michaleas²Darrell Ricke²Shaun Monera¹Shriya Zembruski¹

• ¹Marine Corps Tactical Systems Support Activity, US Marine Corps, Arlington, United States
• ²Massachusetts Institute of Technology Lincoln Laboratory, Lexington, United States

ABSTRACT Large Language Models (LLMs) are powerful tools for Natural Language Processing tasks; fine-tuning them for domain-specific applications is resource-intensive, requiring significant effort to develop datasets and considerable compute resources for fine-tuning. This work establishes a combination of techniques that enable users to rapidly fine-tune LLMs for domain-specific tasks using limited datasets and performs a comparative analysis across established fine-tuning methods using this strategy to determine the most efficient fine-tuning method to use in conjunction with these techniques. This strategy utilizes the Distillation Step-by-Step method for dataset development and model training applied to a language translation task as a case study using a teacher model to generate labels and rationales using Chain-of-Thought prompting. To determine the most efficient fine-tuning method, three approaches were compared with hyperparameter sweeps: full-precision, Low Rank Adaptation, and Quantized Low Rank Adaptation. Optimal results were obtained by combining Distilling Step-by-Step with full-precision fine-tuning. For resource-constrained environments, Distilling Step-by-Step coupled with Low Rank Adaptation for fine-tuning with Alpha to Rank ratio of 4:1 to balance performance and computation consumption. For environments where GPU compute is at a premium, utilize Distilling Step-by-Step with Quantized Low Rank Adaptation fine-tuning with Alpha to Rank ratio of 4:1. These methods were applied to three sizes of exemplar Text-to-Text Transfer Transformer (T5) fine-tuned Language Net LLM models; T5 models have broad utility in text classification, question answering, and language translation. This work provides a guide for efficiently fine-tuning LLMs for domain-specific tasks with limited data availability.