QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge

TL;DR

QuartDepth tackles the obstacle of real-time monocular depth estimation on edge hardware by applying post-training quantization to 4-bit weights and activations, enabling ASIC deployment. The framework introduces LogNP activation polishing to normalize deviant per-channel activation distributions, activation loss compensation to offset quantization error, and a weight reconstruction strategy based on Fisher/KFAC with AdaRound to reduce second-order loss from quantization. A flexible, programmable ASIC accelerator is designed to support kernel fusion, varied precision (W4A4/W4A8), and on-chip vector computation to maximize throughput and energy efficiency. Experimental results across Metric3D and Depth Anything demonstrate competitive accuracy and substantial speedups/energy improvements on ASICs, validating practical edge deployment of large foundation MDE models.

Abstract

Monocular Depth Estimation (MDE) has emerged as a pivotal task in computer vision, supporting numerous real-world applications. However, deploying accurate depth estimation models on resource-limited edge devices, especially Application-Specific Integrated Circuits (ASICs), is challenging due to the high computational and memory demands. Recent advancements in foundational depth estimation deliver impressive results but further amplify the difficulty of deployment on ASICs. To address this, we propose QuartDepth which adopts post-training quantization to quantize MDE models with hardware accelerations for ASICs. Our approach involves quantizing both weights and activations to 4-bit precision, reducing the model size and computation cost. To mitigate the performance degradation, we introduce activation polishing and compensation algorithm applied before and after activation quantization, as well as a weight reconstruction method for minimizing errors in weight quantization. Furthermore, we design a flexible and programmable hardware accelerator by supporting kernel fusion and customized instruction programmability, enhancing throughput and efficiency. Experimental results demonstrate that our framework achieves competitive accuracy while enabling fast inference and higher energy efficiency on ASICs, bridging the gap between high-performance depth estimation and practical edge-device applicability. Code: https://github.com/shawnricecake/quart-depth

Figures (10)

• Figure 1: Visualization of accuracy vs. latency on ASIC for monocular depth estimation models with different backbones.
• Figure 2: Latency profliling for 3 different resolutions using Metric3D with ViT-Large backbone. Vector includes activation functions (e.g. ReLU) and element-wise addition and multiplication.
• Figure 3: Visualization of outliers in various channels at the decoder depth_decoder of Metric3D model on NYUv2 dataset.
• Figure 4: Visualization for the activation data distribution of different channels in decoder with $\log$ frequency.
• Figure 5: Visualization for the polished activation data distribution of different channels in decoder with $\log$ frequency.