L-TUNING: Synchronized Label Tuning for Prompt and Prefix in LLMs

TL;DR

Fine-tuning large language models for classification is hampered by prompt/prefix tuning with non-semantic tokens. L-Tuning processes label tokens through a frozen pre-trained LM to exploit its semantic knowledge, producing distinct label embeddings for each class and enabling two training regimes: LT-prefix and LT-prompt. The paper formalizes the L-Tuning procedure and objective, and shows that training only label-related parameters yields faster convergence and higher accuracy than conventional methods, with larger gains on large LLMs. The results on multiple datasets indicate the approach is scalable and practically impactful for efficient task-specific fine-tuning of LLMs in NLP.

Abstract

Efficiently fine-tuning Large Language Models (LLMs) for specific tasks presents a considerable challenge in natural language processing. Traditional methods, like prompt or prefix tuning, typically rely on arbitrary tokens for training, leading to prolonged training times and generalized token use across various class labels. To address these issues, this paper introduces L-Tuning, an efficient fine-tuning approach designed for classification tasks within the Natural Language Inference (NLI) framework. Diverging from conventional methods, L-Tuning focuses on the fine-tuning of label tokens processed through a pre-trained LLM, thereby harnessing its pre-existing semantic knowledge. This technique not only improves the fine-tuning accuracy and efficiency but also facilitates the generation of distinct label embeddings for each class, enhancing the model's training nuance. Our experimental results indicate a significant improvement in training efficiency and classification accuracy with L-Tuning compared to traditional approaches, marking a promising advancement in fine-tuning LLMs for complex language tasks.

Figures (2)

• Figure 1: Prefix tuning and L-Tuning for Prefix, highlighting label embedding integration and classification pathways.
• Figure 2: Validation Loss Comparison on the CoLA dataset across different steps for traditional Prompt Tuning (solid lines) and L-Tuning for Prompt (dashed lines) across models Llama (blue), Falcon (green), and Bloom (red).