ROCKET: Rapid Optimization via Calibration-guided Knapsack Enhanced Truncation for Efficient Model Compression

TL;DR

ROCKET addresses the challenge of efficient, training-free compression for large transformers by coupling calibration-guided sparse dictionary factorization in the whitened activation space with a global budget allocator formulated as a constrained multi-choice knapsack problem. A closed-form, single-shot factorization replaces costly iterative dictionary learning, while dynamic programming yields an optimal per-layer compression configuration under a global parameter budget $C_{ ext{total}}$. The method achieves state-of-the-art performance at 20–50% compression across text, vision-language, and speech modalities, retaining over 90% of original accuracy at 30% compression without fine-tuning and enabling notable recovery with modest fine-tuning. ROCKET also demonstrates generalization to multimodal tasks and offers tangible environmental and efficiency benefits, albeit with some scalability limitations for extremely large MoE architectures and potential gains from learned sparsity patterns during healing. Overall, the work presents a practical, scalable approach to deploying compact, high-performing LLMs without expensive retraining or fine-tuning.

Abstract

We present ROCKET, a training-free model compression method that achieves state-of-the-art performance in comparison with factorization, structured-sparsification and dynamic compression baselines. Operating under a global compression budget, ROCKET comprises two key innovations: First, it formulates layer-wise compression allocation as a multi-choice knapsack problem, selecting the optimal compression level for each layer to minimize total reconstruction error while adhering to a target model size. Second, it introduces a single-step sparse matrix factorization inspired by dictionary learning: using only a small calibration set, it sparsifies weight coefficients based on activation-weights sensitivity and then updates the dictionary in closed form via least squares bypassing iterative optimization, sparse coding, or backpropagation entirely. ROCKET consistently outperforms existing compression approaches across different model architectures at 20-50\% compression rates. Notably, it retains over 90\% of the original model's performance at 30\% compression without any fine-tuning. Moreover, when applying a light fine-tuning phase, recovery is substantially enhanced: for instance, compressing Qwen3-14B to an 8B-parameter model and healing it with just 30 million tokens yields performance nearly on par with the original Qwen3-8B. The code for ROCKET is at github.com/mts-ai/ROCKET/tree/main.

Figures (6)

• Figure 1: Overview of the proposed method. Left: Budget allocation formulated as a shortest-path problem on a directed graph, where nodes represent compression options and edges encode cost (reconstruction error), solved via DP algorithm to find the optimal sequence of operations. Right: The selected optimal path determines per-layer compression parameters (rank $K_i$ and sparsity $S_i$), which are then applied to each layer via Eigen value decomposition (EVD) followed by structured hard thresholding sparsification ($\mathcal{T}(.)$) of coefficients.
• Figure 2: Comparison of Accuracy and Inverse Log Perplexity for Llama3-8B and Llama3.2-1B.
• Figure 3: Comparison of ROCKET with alternative budget allocation methods (Uniform, ARSars, Dobi-SVDdobisvd, and ARAara) on three model configurations: Qwen3-8B at 20% and 40% pruning, and LLaMA2-7B at 40% pruning. Subplots show normalized performance on eight benchmarks (C4 perplexity inverted so higher is better), scaled to each model’s dense baseline (value=1.0). ROCKET consistently retains the most performance under the same parameter constraints.
• Figure 4: Exact state-space graph matching. Each state $(i, p)$ represents having processed the first $i$ layers with $p$ scaled parameters retained. From $(i, p)$, the algorithm branches to all options for layer $i$, producing states $(i+1, p + \Delta p)$. The sink is reachable only from states with $p \geq p_{\min}$, enforcing the global compression ratio by construction.
• Figure 5: Comparison between PyTorch’s built-in sparse-matrix vector multiplication and Macko across MLP layers in Qwen3-8B. Macko shows consistently better running time for large coefficients (gate and up projections), while being on par with PyTorch for smaller sparse matrices (down projection).