Tail-Aware Post-Training Quantization for 3D Geometry Models

TL;DR

Tail-Aware Post-Training Quantization (TAPTQ) targets the unique challenges of 3D geometry foundation models by incorporating progressive calibration data selection, a ternary-search based interval optimization, and Tail Relative Error (TRE)-guided module-wise compensation. The approach reduces calibration overhead from linear to logarithmic complexity, selectively mitigates cross-layer quantization error accumulation, and demonstrates improved reconstruction accuracy and camera-parameter prediction under low-bit quantization on VGGT and Pi3 across 7Scenes, ETH3D, and Co3Dv2. Ablation studies validate the complementary effect of its components, and the method achieves robust performance gains with significantly faster calibration times. These advances enable practical edge deployment of large 3D geometry transformers for real-time multi-view tasks.

Abstract

The burgeoning complexity and scale of 3D geometry models pose significant challenges for deployment on resource-constrained platforms. While Post-Training Quantization (PTQ) enables efficient inference without retraining, conventional methods, primarily optimized for 2D Vision Transformers, fail to transfer effectively to 3D models due to intricate feature distributions and prohibitive calibration overhead. To address these challenges, we propose TAPTQ, a Tail-Aware Post-Training Quantization pipeline specifically engineered for 3D geometric learning. Our contribution is threefold: (1) To overcome the data-scale bottleneck in 3D datasets, we develop a progressive coarse-to-fine calibration construction strategy that constructs a highly compact subset to achieve both statistical purity and geometric representativeness. (2) We reformulate the quantization interval search as an optimization problem and introduce a ternary-search-based solver, reducing the computational complexity from $\mathcal{O}(N)$ to $\mathcal{O}(\log N)$ for accelerated deployment. (3) To mitigate quantization error accumulation, we propose TRE-Guided Module-wise Compensation, which utilizes a Tail Relative Error (TRE) metric to adaptively identify and rectify distortions in modules sensitive to long-tailed activation outliers. Extensive experiments on the VGGT and Pi3 benchmarks demonstrate that TAPTQ consistently outperforms state-of-the-art PTQ methods in accuracy while significantly reducing calibration time. The code will be released soon.

Figures (9)

• Figure 1: Overview of TAPTQ.(I) Our proposed Representative Calibration Data via Two-Stage Cluster. (II) Our proposed Quantization Interval Optimization via Ternary Search. (III) Our proposed Tail-Relative-Error (TRE)-Guided Compensation Module.
• Figure 2: Acc. and Comp. performance under different calibration data set. Each bubble corresponds to one method configuration on 7Scenes (W4A8), where bubble size indicates the number of calibration samples (4 / 8 / 20 scans). Stab. score denotes a stability-based calibration sample selection strategy (see Appendix \ref{['appendix:stability']}).
• Figure 3: Unimodal behavior of similarity with respect to quantization interval. The similarity between quantized and full-precision outputs is evaluated over a discrete set of interval candidates for both activations and weights. In both cases, the similarity exhibits an approximately unimodal trend, enabling efficient interval search via ternary search.
• Figure 4: Accumulated MSE and Tail Relative Error across W4A8 quantized Transformer modules. The accumulated MSE is measured at the output of each quantized module along the network depth. Results are reported for a quantized VGGT model with layer-wise calibration applied.
• Figure 5: Visualization of activation values and corresponding quantization errors. Each horizontal pair shows the full-precision activations (left) and the corresponding quantization errors (right) for representative attention modules. Both distributions exhibit heavy-tailed characteristics, with large-magnitude activations contributing disproportionately to the observed quantization error.