Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning

Jing-Cheng Pang, Liu Sun, Chang Zhou, Xian Tang, Haichuan Ma, Kun Jiang, Jianlong Wang, Kai Zhang, Sijie Wu, Haoran Cai, Chenwei Wu, Xubin Li, Xin Chen

TL;DR

This paper tackles the challenge of efficiently fine-tuning domain-specific LLMs when high-quality data is scarce, by replacing static curricula with a dynamic, entropy-driven approach. EDCO continuously estimates inference entropy to select high-uncertainty samples, forming a growing curriculum that challenges the model at its learning frontier; this is implemented via Quick-Answer Prompting and Prefix Entropy Approximation to keep computations tractable. The method is compatible with both supervised and reinforcement learning fine-tuning and demonstrates consistent performance gains across communication, medical, and legal domains, with substantial speedups in entropy estimation (up to 83.5% reduction). The work provides a practical pathway to more data-efficient domain adaptation and highlights the importance of maintaining exploration during fine-tuning for improved generalization.

Abstract

Domain-specific large language models (LLMs), typically developed by fine-tuning a pre-trained general-purpose LLM on specialized datasets, represent a significant advancement in applied AI. A common strategy in LLM fine-tuning is curriculum learning, which pre-orders training samples based on metrics like difficulty to improve learning efficiency compared to a random sampling strategy. However, most existing methods for LLM fine-tuning rely on a static curriculum, designed prior to training, which lacks adaptability to the model's evolving needs during fine-tuning. To address this, we propose EDCO, a novel framework based on two key concepts: inference entropy and dynamic curriculum orchestration. Inspired by recent findings that maintaining high answer entropy benefits long-term reasoning gains, EDCO prioritizes samples with high inference entropy in a continuously adapted curriculum. EDCO integrates three core components: an efficient entropy estimator that uses prefix tokens to approximate full-sequence entropy, an entropy-based curriculum generator that selects data points with the highest inference entropy, and an LLM trainer that optimizes the model on the selected curriculum. Comprehensive experiments in communication, medicine and law domains, EDCO outperforms traditional curriculum strategies for fine-tuning Qwen3-4B and Llama3.2-3B models under supervised and reinforcement learning settings. Furthermore, the proposed efficient entropy estimation reduces computational time by 83.5% while maintaining high accuracy.

EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning

TL;DR

This paper tackles the challenge of efficiently fine-tuning domain-specific LLMs when high-quality data is scarce, by replacing static curricula with a dynamic, entropy-driven approach. EDCO continuously estimates inference entropy to select high-uncertainty samples, forming a growing curriculum that challenges the model at its learning frontier; this is implemented via Quick-Answer Prompting and Prefix Entropy Approximation to keep computations tractable. The method is compatible with both supervised and reinforcement learning fine-tuning and demonstrates consistent performance gains across communication, medical, and legal domains, with substantial speedups in entropy estimation (up to 83.5% reduction). The work provides a practical pathway to more data-efficient domain adaptation and highlights the importance of maintaining exploration during fine-tuning for improved generalization.

Abstract

Domain-specific large language models (LLMs), typically developed by fine-tuning a pre-trained general-purpose LLM on specialized datasets, represent a significant advancement in applied AI. A common strategy in LLM fine-tuning is curriculum learning, which pre-orders training samples based on metrics like difficulty to improve learning efficiency compared to a random sampling strategy. However, most existing methods for LLM fine-tuning rely on a static curriculum, designed prior to training, which lacks adaptability to the model's evolving needs during fine-tuning. To address this, we propose EDCO, a novel framework based on two key concepts: inference entropy and dynamic curriculum orchestration. Inspired by recent findings that maintaining high answer entropy benefits long-term reasoning gains, EDCO prioritizes samples with high inference entropy in a continuously adapted curriculum. EDCO integrates three core components: an efficient entropy estimator that uses prefix tokens to approximate full-sequence entropy, an entropy-based curriculum generator that selects data points with the highest inference entropy, and an LLM trainer that optimizes the model on the selected curriculum. Comprehensive experiments in communication, medicine and law domains, EDCO outperforms traditional curriculum strategies for fine-tuning Qwen3-4B and Llama3.2-3B models under supervised and reinforcement learning settings. Furthermore, the proposed efficient entropy estimation reduces computational time by 83.5% while maintaining high accuracy.
Paper Structure (34 sections, 6 equations, 5 figures, 7 tables, 1 algorithm)

This paper contains 34 sections, 6 equations, 5 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Problem Formulation and LLM fine-tuning
  4. The Role of Entropy in Maintaining Exploration
  5. Method
  6. Dynamic Curriculum Orchestration via Inference Entropy
  7. Efficient Entropy Estimation with Prefix Tokens
  8. Quick-Answer Prompting
  9. Prefix Entropy Approximation
  10. Compatibility with Fine-Tuning Paradigms
  11. Algorithm Summary
  12. Algorithm Summary
  13. Related Work
  14. Domain-specific Large Language Models
  15. Uncertainty-Driven and Entropy-Based Data Selection
  16. ...and 19 more sections

Figures (5)

  • Figure 1: An overview of EDCO method, with three panels: (A) Overall training procedure; (B) Entropy-based curriculum orchestration module that periodically updates the training curriculum; (C) Efficient entropy estimation module that calculates the sample entropy.
  • Figure 2: Performance of various fine-tuning strategies on communication domains. The reported results represent the answer accuracy, averaged over three evaluations.
  • Figure 3: Analysis of the training process of EDCO method. (A) The model's inference entropy during the training. (B) The number of first-time samples added in each training interval.
  • Figure 4: Analysis of the efficient entropy estimation module in EDCO. (A) Comparison of entropy estimation using a 128-token prefix versus the full sequence. (B) Ablation study on the prefix length. To better visualize data trends, the sample indices are arranged in ascending order of the entropy estimated with the 128-token prefix. Ablation study on QAP is in Appendix \ref{['app:ablation_qap']}.
  • Figure 5: Ablation study on the quick-answer prompting. Prefix-token entropy estimation has a strong correlation with the full-length estimation (see blue and red lines in the figure).