Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation

Hailin Chen, Amrita Saha, Steven Hoi, Shafiq Joty

TL;DR

The paper tackles the inefficiency of standard distillation for open-source LLMs in code generation by introducing personalised distillation (PERsD), which tailors teacher guidance to the student’s current capabilities through input and output personalization and execution-feedback driven refinements. It defines three PERsD variants (PERsD-combined, PERsD-refine, PERsD) and an iterative inference regime that leverages seen unit tests to progressively improve code solutions. Empirical results across CodeGen-mono-6B/16B and StarCoder show that PERsD substantially outperforms StanD with far less data, and multi-step inference further boosts performance, with strong results on HumanEval and MBPP. The work demonstrates superior data efficiency, scalable refinement via feedback, and potential for online, personalized distillation, offering a practical pathway to enhance open-source LLMs by emulating human-like personalised teaching.

Abstract

With the rise of powerful closed-sourced LLMs (ChatGPT, GPT-4), there are increasing interests in distilling the capabilies of close-sourced LLMs to smaller open-sourced LLMs. Previous distillation methods usually prompt ChatGPT to generate a set of instructions and answers, for the student model to learn. However, such standard distillation approach neglects the merits and conditions of the student model. Inspired by modern teaching principles, we design a personalised distillation process, in which the student attempts to solve a task first, then the teacher provides an adaptive refinement for the student to improve. Instead of feeding the student with teacher's prior, personalised distillation enables personalised learning for the student model, as it only learns on examples it makes mistakes upon and learns to improve its own solution. On code generation, personalised distillation consistently outperforms standard distillation with only one third of the data. With only 2.5-3K personalised examples that incur a data-collection cost of 4-6$, we boost CodeGen-mono-16B by 7% to achieve 36.4% pass@1 and StarCoder by 12.2% to achieve 45.8% pass@1 on HumanEval.

Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation

TL;DR

The paper tackles the inefficiency of standard distillation for open-source LLMs in code generation by introducing personalised distillation (PERsD), which tailors teacher guidance to the student’s current capabilities through input and output personalization and execution-feedback driven refinements. It defines three PERsD variants (PERsD-combined, PERsD-refine, PERsD) and an iterative inference regime that leverages seen unit tests to progressively improve code solutions. Empirical results across CodeGen-mono-6B/16B and StarCoder show that PERsD substantially outperforms StanD with far less data, and multi-step inference further boosts performance, with strong results on HumanEval and MBPP. The work demonstrates superior data efficiency, scalable refinement via feedback, and potential for online, personalized distillation, offering a practical pathway to enhance open-source LLMs by emulating human-like personalised teaching.

Abstract

With the rise of powerful closed-sourced LLMs (ChatGPT, GPT-4), there are increasing interests in distilling the capabilies of close-sourced LLMs to smaller open-sourced LLMs. Previous distillation methods usually prompt ChatGPT to generate a set of instructions and answers, for the student model to learn. However, such standard distillation approach neglects the merits and conditions of the student model. Inspired by modern teaching principles, we design a personalised distillation process, in which the student attempts to solve a task first, then the teacher provides an adaptive refinement for the student to improve. Instead of feeding the student with teacher's prior, personalised distillation enables personalised learning for the student model, as it only learns on examples it makes mistakes upon and learns to improve its own solution. On code generation, personalised distillation consistently outperforms standard distillation with only one third of the data. With only 2.5-3K personalised examples that incur a data-collection cost of 4-6$, we boost CodeGen-mono-16B by 7% to achieve 36.4% pass@1 and StarCoder by 12.2% to achieve 45.8% pass@1 on HumanEval.
Paper Structure (34 sections, 3 figures, 13 tables, 1 algorithm)

This paper contains 34 sections, 3 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Distillation from ChatGPT
  4. Code Generation with Feedback
  5. Reinforcement Learning from (Human) Feedback
  6. Method
  7. Standard Distillation
  8. Personalised Distillation
  9. Iterative Inference
  10. Experimental Setup
  11. Baselines
  12. Pretraining Data Construction
  13. Model Evaluation
  14. Pretraining Setup
  15. Experimental Results
  16. ...and 19 more sections

Figures (3)

  • Figure 1: Example: (Top) Personalised refinement from student's attempt and execution feedback; (Bottom) Direct solution generated by teacher conditioned on task.
  • Figure 2: Prompt templates to query personalised refinement. Top(a): prompt template for first turn conversation, Botton(b): prompt template for second turn conversation.
  • Figure 3: Prompt template for code refinement finetuning.