One-for-All Pruning: A Universal Model for Customized Compression of Large Language Models

TL;DR

UniCuCo delivers a universal framework for customized pruning of large language models by learning a request-to-pruning-strategy mapping through StratNet. To overcome the expensive evaluation and non-differentiability of pruning, it leverages a Gaussian process estimator to predict pruning performance and enable gradient-based updates, with an alternating GP-StratNet training loop. Empirical results show up to 28x faster processing for 64 requests and, in non-uniform pruning at 70% sparsity, about 3% higher average accuracy than score-based baselines. The approach thus offers a practical, uncertainty-aware path to efficiently service diverse compression requests in real-world LLM deployment, albeit with scalability considerations for very large models and multi-LLM scenarios.

Abstract

Existing pruning methods for large language models (LLMs) focus on achieving high compression rates while maintaining model performance. Although these methods have demonstrated satisfactory performance in handling a single user's compression request, their processing time increases linearly with the number of requests, making them inefficient for real-world scenarios with multiple simultaneous requests. To address this limitation, we propose a Univeral Model for Customized Compression (UniCuCo) for LLMs, which introduces a StratNet that learns to map arbitrary requests to their optimal pruning strategy. The challenge in training StratNet lies in the high computational cost of evaluating pruning strategies and the non-differentiable nature of the pruning process, which hinders gradient backpropagation for StratNet updates. To overcome these challenges, we leverage a Gaussian process to approximate the evaluation process. Since the gradient of the Gaussian process is computable, we can use it to approximate the gradient of the non-differentiable pruning process, thereby enabling StratNet updates. Experimental results show that UniCuCo is 28 times faster than baselines in processing 64 requests, while maintaining comparable accuracy to baselines.

Figures (9)

• Figure 1: A comparison of various approaches in terms of effectiveness and efficiency when providing pruning strategies for compression requests.
• Figure 2: Flowchart of UniCuCo.
• Figure 3: The optimal pruning strategy $\boldsymbol{x}$ obtained using (a) the weighted sum function and (b) the weighted Tchebycheff function under a concave Pareto front.
• Figure 4: The comparison of total time for generating pruning strategies between UniCuCo and EvoPress as the number of requests increases.
• Figure 5: The impact of Gaussian process updates on depth pruning, evaluated using adjusted perplexity (higher values indicate better effectiveness).