Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations

TL;DR

This paper addresses enterprise needs for domain-specific LLMs trained on private documents and code, proposing a practical workflow that combines data-preparation recipes, compute estimation, and PEFT-based fine-tuning (notably LoRA/QLoRA) to fit within limited hardware. Through experiments on proprietary document and code corpora using LLaMA 2 across multiple sizes, it analyzes the effects of quantization, PEFT hyperparameters, and RAG-vs-fine-tuned pipelines, yielding actionable guidelines for practitioners. The key contributions include four text-data formats, three code-data formats, and a structured evaluation of memory-time trade-offs, guiding efficient on-premise deployment. Overall, the work demonstrates that domain adaptation via PEFT can achieve quality improvements with modest resources, while highlighting the importance of dataset quality and prompt design to mitigate hallucinations.

Abstract

There is a compelling necessity from enterprises for fine tuning LLMs (Large Language Models) o get them trained on proprietary domain knowledge. The challenge is to imbibe the LLMs with domain specific knowledge using the most optimial resource and cost and in the best possible time. Many enterprises rely on RAG (Retrieval Augmented Generation) which does not need LLMs to be ine-tuned but they are limited by the quality of vector databases and their retrieval capabilities rather than the intrinsic capabilities of the LLMs themselves. In our current work we focus on fine tuning LLaMA, an open source LLM using proprietary documents and code from an enterprise repository and use the fine tuned models to evaluate the quality of responses. As part of this work, we aim to guide beginners on how to start with fine tuning an LLM for documentation and code by making educated guesses on size of GPU required and options that are available for formatting the data. We also propose pre processing recipes for both documentation and code to prepare dataset in different formats. The proposed methods of data preparation for document datasets are forming paragraph chunks, forming question and answer pairs and forming keyword and paragraph chunk pairs. For code dataset we propose forming summary and function pairs. Further, we qualitatively evaluate the results of the models for domain specific queries. Finally, we also propose practical guidelines and recommendations for fine tuning LLMs.

Figures (3)

• Figure 1: Fine tuning Workflow (with LLaMA model as an example)
• Figure 2: Inference time of fine tuned Llama 2 7B Chat model
• Figure 3: Fine tuning time of Llama 2 Chat models on different fine tuning configurations