Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning

Moses Ananta, Muhammad Farid Adilazuarda, Zayd Muhammad Kawakibi Zuhri, Ayu Purwarianti, Alham Fikri Aji

TL;DR

QLESS tackles the memory bottleneck in gradient-based data valuation for fine-tuning large language models by integrating a quantization stage into LESS. It first derives low-dimensional gradient representations via a LoRA-based random projection, then quantizes these gradients with an absmax-based scheme to dramatically reduce storage, while normalizing the quantized vectors during influence computation. The approach preserves the ability to identify influential training samples through cosine similarities across checkpoints, achieving comparable data-selection performance to LESS under much smaller memory footprints, with surprising robustness even at 1-bit precision. Across multiple models and benchmarks, QLESS demonstrates substantial memory savings (often >2x to >8x) and practical viability for resource-constrained settings, highlighting its potential for scalable instruction tuning and data curation.

Abstract

Fine-tuning large language models (LLMs) is often constrained by the computational costs of processing massive datasets. We propose \textbf{QLESS} (Quantized Low-rank Gradient Similarity Search), which integrates gradient quantization with the LESS framework to enable memory-efficient data valuation and selection. QLESS employs a two-step compression process: first, it obtains low-dimensional gradient representations through LoRA-based random projection; then, it quantizes these gradients to low-bitwidth representations. Experiments on multiple LLM architectures (LLaMA, Mistral, Qwen) and benchmarks (MMLU, BBH, TyDiQA) show that QLESS achieves comparable data selection performance to LESS while reducing memory usage by up to 16x. Even 1-bit gradient quantization preserves data valuation quality. These findings underscore QLESS as a practical, scalable approach to identifying informative examples within strict memory constraints.

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning

TL;DR

QLESS tackles the memory bottleneck in gradient-based data valuation for fine-tuning large language models by integrating a quantization stage into LESS. It first derives low-dimensional gradient representations via a LoRA-based random projection, then quantizes these gradients with an absmax-based scheme to dramatically reduce storage, while normalizing the quantized vectors during influence computation. The approach preserves the ability to identify influential training samples through cosine similarities across checkpoints, achieving comparable data-selection performance to LESS under much smaller memory footprints, with surprising robustness even at 1-bit precision. Across multiple models and benchmarks, QLESS demonstrates substantial memory savings (often >2x to >8x) and practical viability for resource-constrained settings, highlighting its potential for scalable instruction tuning and data curation.

Abstract

Fine-tuning large language models (LLMs) is often constrained by the computational costs of processing massive datasets. We propose \textbf{QLESS} (Quantized Low-rank Gradient Similarity Search), which integrates gradient quantization with the LESS framework to enable memory-efficient data valuation and selection. QLESS employs a two-step compression process: first, it obtains low-dimensional gradient representations through LoRA-based random projection; then, it quantizes these gradients to low-bitwidth representations. Experiments on multiple LLM architectures (LLaMA, Mistral, Qwen) and benchmarks (MMLU, BBH, TyDiQA) show that QLESS achieves comparable data selection performance to LESS while reducing memory usage by up to 16x. Even 1-bit gradient quantization preserves data valuation quality. These findings underscore QLESS as a practical, scalable approach to identifying informative examples within strict memory constraints.

Paper Structure

This paper contains 39 sections, 12 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Gradient-Based Data Valuation
  4. LESS: Low-Rank Gradient Similarity Search
  5. Quantization in Distributed and Federated Learning
  6. Quantized Random Projection (QRP)
  7. QLESS: Integrating LESS with Quantization
  8. Quantizing Randomly Projected Gradients
  9. Influence Computation with Quantized-Normalized Gradients
  10. Experiments
  11. Experimental Setup
  12. Datasets.
  13. Models.
  14. Baselines.
  15. Our Method.
  16. ...and 24 more sections

Figures (5)

  • Figure 1: Data selection method comparison on average model performance across various models and benchmarks. The x-axis shows the average performance (%), while the y-axis represents different data selection methods. QLESS with various quantization levels achieves comparable performance to LESS.
  • Figure 2: Overview of the data valuation and selection process. LESS and QLESS differ from each other in step 2 and 3.
  • Figure 3: Absmax- vs absmean-based quantization comparison based on the distribution of values in the gradient tensors.
  • Figure 4: The average benchmark results of Llama 2 7B and Qwen 2.5 7B when fine-tuned on different percentages of QLESS selected data with 1-bit gradient store.
  • Figure 5: Subset distribution of the top $5\%$ selected examples for different quantization levels.