PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models

TL;DR

PTQTP introduces a novel structured post-training quantization scheme that decomposes full-precision LLM weights into two trit-planes with scaling, achieving a 1.58-bit representation and multiplication-free inference via $\hat{W} = \mathrm{diag}(\alpha^{(1)}) T^{(1)} + \mathrm{diag}(\alpha^{(2)}) T^{(2)}$. By performing row-wise direct trit-plane approximation with closed-form ridge regression and applying progressive, adaptive regularization alongside a fine-grained, group-wise search over trit-plane entries, PTQTP achieves robust, model-agnostic quantization without retraining. Empirical results across LLaMA3.x and Qwen3 models (0.6B–70B) show PTQTP consistently outperforms existing 1–3-bit PTQ methods, approaches or surpasses 4-bit FP16 baselines on many tasks, and preserves mathematical reasoning capabilities with high fidelity (e.g., Math-500 at ~82.4% accuracy). The method yields substantial practical benefits, including up to ~28.8× quantization-speedups over certain baselines and a modest memory footprint, enabling efficient deployment on edge and resource-constrained platforms without architecture-specific adjustments. These findings establish structured ternary PTQ as a viable, efficient pathway for scalable LLM deployment with minimal computational overhead.

Abstract

Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency and model expressiveness. While existing ultra-low-bit PTQ methods rely on binary approximations or complex compensation mechanisms, they suffer from either limited representational capacity or computational overhead that undermines their efficiency gains. We introduce PTQ to Trit-Planes (PTQTP), the first ternary-weight PTQ framework that decomposes weight matrices into structured ternary {-1, 0, 1} trit-planes using 2x1.58-bit representation. PTQTP achieves multiplication-free inference, identical to 1-bit quantization, while maintaining superior expressiveness through its novel structured decomposition. Our approach provides: (1) a theoretically grounded progressive approximation algorithm ensuring global weight consistency; (2) model-agnostic deployment across diverse modern LLMs without architectural modifications; and (3) uniform ternary operations that eliminate the need for mixed-precision or compensation schemes. Comprehensive experiments across LLaMA3.x and Qwen3 model families (0.6B-70B parameters) demonstrate that PTQTP significantly outperforms existing low-bit PTQ methods, achieving 82.4% mathematical reasoning retention versus 0% for competing approaches. PTQTP approaches and sometimes surpasses 1.58-bit quantization-aware training performance while requiring only single-hour quantization compared to 10-14 GPU days for training-based methods. These results establish PTQTP as a practical solution for efficient LLM deployment in resource-constrained environments. The code will be available at https://github.com/HeXiao-55/PTQTP.

Figures (5)

• Figure 1: Qwen3 performance evaluation. (a) PTQTP outperforms existing low-bit quantization methods (1/2/3-bit) while maintaining equivalent model size. (b) Quantization runtime comparison showing PTQTP achieves 17.73$\times$-28.79$\times$ speedup over ARB-LLMRC and 1.53$\times$-1.57$\times$ over AWQ. (c) PTQTP performance approaches both 4-bit quantization and FP16 baselines across model scales. (d) Minimal precision degradation observed across all benchmarks when applying PTQTP to Qwen32B.
• Figure 2: PTQTP workflow overview: (top) Linear layer transformation pathway for ternary quantization in LLaMA architecture; (bottom) Group-wise progressive trit-plane approximation process, where $G$ represents group size and $T_{max}$ indicates maximum iteration count.
• Figure 3: Effect of progressive search iterations on quantization time (left sub-figure) and perplexity (PPL) (middle and right sub-figures) for LLaMA3.1-8B and LLaMA3.2-3B models.
• Figure 4: Trade-off between tolerance bounds ($\epsilon$), quantization time (left sub-figure), and model perplexity (PPL) (middle and right sub-figures) for LLaMA3.1-8B and LLaMA3.2-3B architectures.
• Figure 5: The single trit-plane update process during the optimization iterations on LLaMA3.1-8B.