Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Model Based on Large Language Models

Shenglin Zhang, Pengtian Zhu, Minghua Ma, Jiagang Wang, Yongqian Sun, Dongwen Li, Jingyu Wang, Qianying Guo, Xiaolei Hua, Lin Zhu, Dan Pei

TL;DR

The paper tackles domain-specific QA under privacy and compute constraints by proposing Self-Evolution, a three-phase, iterative framework that uses a lightweight open-source LLM with LoRA fine-tuning to generate and refine domain data from unlabeled documents. It introduces an IFD-based data selection mechanism to filter high-value samples across iterations, guiding subsequent fine-tuning and data augmentation. The approach, demonstrated on 4,000 China Mobile documents with a 7B base model, achieves substantial gains over strong baselines (including 72B-parameter models) and is deployed in production for 117 days, with an overall improvement in domain QA performance and operational efficiency. The authors also release the Self-Evolution code and discuss future work on extending capabilities to multi-turn dialogue using unsupervised data, preserving privacy and reducing computational demands. Key mathematical components include the IFD score $IFD_\theta(Q,A)=\frac{s_\theta(A|Q)}{s_\theta(A)}$ and the iteration update $\theta_{i+1}=IFT(\theta_i,q,a)$, with performance quantified by the relative BLEU-based score $Score = \frac{BLEU(\theta)}{BLEU(\theta_{HQ})}$.

Abstract

Large language models (LLMs) excel at general question-answering (Q&A) but often fall short in specialized domains due to a lack of domain-specific knowledge. Commercial companies face the dual challenges of privacy protection and resource constraints when involving LLMs for fine-tuning. This paper propose a novel framework, Self-Evolution, designed to address these issues by leveraging lightweight open-source LLMs through multiple iterative fine-tuning rounds. To enhance the efficiency of iterative fine-tuning, Self-Evolution employ a strategy that filters and reinforces the knowledge with higher value during the iterative process. We employed Self-Evolution on Qwen1.5-7B-Chat using 4,000 documents containing rich domain knowledge from China Mobile, achieving a performance score 174% higher on domain-specific question-answering evaluations than Qwen1.5-7B-Chat and even 22% higher than Qwen1.5-72B-Chat. Self-Evolution has been deployed in China Mobile's daily operation and maintenance for 117 days, and it improves the efficiency of locating alarms, fixing problems, and finding related reports, with an average efficiency improvement of over 18.6%. In addition, we release Self-Evolution framework code in https://github.com/Zero-Pointer/Self-Evolution.

Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Model Based on Large Language Models

TL;DR

The paper tackles domain-specific QA under privacy and compute constraints by proposing Self-Evolution, a three-phase, iterative framework that uses a lightweight open-source LLM with LoRA fine-tuning to generate and refine domain data from unlabeled documents. It introduces an IFD-based data selection mechanism to filter high-value samples across iterations, guiding subsequent fine-tuning and data augmentation. The approach, demonstrated on 4,000 China Mobile documents with a 7B base model, achieves substantial gains over strong baselines (including 72B-parameter models) and is deployed in production for 117 days, with an overall improvement in domain QA performance and operational efficiency. The authors also release the Self-Evolution code and discuss future work on extending capabilities to multi-turn dialogue using unsupervised data, preserving privacy and reducing computational demands. Key mathematical components include the IFD score and the iteration update , with performance quantified by the relative BLEU-based score .

Abstract

Large language models (LLMs) excel at general question-answering (Q&A) but often fall short in specialized domains due to a lack of domain-specific knowledge. Commercial companies face the dual challenges of privacy protection and resource constraints when involving LLMs for fine-tuning. This paper propose a novel framework, Self-Evolution, designed to address these issues by leveraging lightweight open-source LLMs through multiple iterative fine-tuning rounds. To enhance the efficiency of iterative fine-tuning, Self-Evolution employ a strategy that filters and reinforces the knowledge with higher value during the iterative process. We employed Self-Evolution on Qwen1.5-7B-Chat using 4,000 documents containing rich domain knowledge from China Mobile, achieving a performance score 174% higher on domain-specific question-answering evaluations than Qwen1.5-7B-Chat and even 22% higher than Qwen1.5-72B-Chat. Self-Evolution has been deployed in China Mobile's daily operation and maintenance for 117 days, and it improves the efficiency of locating alarms, fixing problems, and finding related reports, with an average efficiency improvement of over 18.6%. In addition, we release Self-Evolution framework code in https://github.com/Zero-Pointer/Self-Evolution.
Paper Structure (20 sections, 4 equations, 3 figures, 5 tables)

This paper contains 20 sections, 4 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Instruction Fine-tuning
  4. Instruction Data Generation
  5. Instruction Data Selection
  6. Method
  7. QA generation
  8. Data Selection And Training
  9. Next Iteration
  10. Experimental Setup
  11. Model and Dataset
  12. Baseline
  13. Qwen1.5-7B-Chat-Fine-Tuned by High Quality QA
  14. Original LLM
  15. Evaluation Metrics
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Self-Evolution
  • Figure 2: The x-axis represents the number of iterations of the Self-Evolution, while the y-axis shows the performance scores of different models. The horizontal lines in the graph represent the performance of four distinct models, and the line graph depicts the performance of the Self-Evolution at each iteration.
  • Figure 3: Ablation Experiment Results.