LMDepth: Lightweight Mamba-based Monocular Depth Estimation for Real-World Deployment

TL;DR

LMDepth addresses the need for accurate monocular depth estimation on resource-constrained devices by introducing a lightweight Mamba-based architecture that captures global context with a Modified Pyramid Spatial Pooling head and decodes depth via Depth Mamba Blocks. The approach combines an efficient image encoder, semantic scene guidance, and linear-complexity feature fusion to produce depth distributions over learned bins, enabling end-to-end depth maps with low FLOPs. It delivers state-of-the-art performance among lightweight methods on NYUDv2 and KITTI, while supporting real-world embedded deployment through ONNX export and INT8 quantization, achieving substantial model size reductions and speedups with minimal accuracy loss. Overall, LMDepth demonstrates that Mamba-based State Space Models can rival Transformer-based methods in accuracy while markedly improving efficiency, offering practical deployment benefits for autonomous systems and edge devices.

Abstract

Monocular depth estimation provides an additional depth dimension to RGB images, making it widely applicable in various fields such as virtual reality, autonomous driving and robotic navigation. However, existing depth estimation algorithms often struggle to effectively balance performance and computational efficiency, which poses challenges for deployment on resource-constrained devices. To address this, we propose LMDepth, a lightweight Mamba-based monocular depth estimation network, designed to reconstruct high-precision depth information while maintaining low computational overhead. Specifically, we propose a modified pyramid spatial pooling module that serves as a multi-scale feature aggregator and context extractor, ensuring global spatial information for accurate depth estimation. Moreover, we integrate multiple depth Mamba blocks into the decoder. Designed with linear computations, the Mamba Blocks enable LMDepth to efficiently decode depth information from global features, providing a lightweight alternative to Transformer-based architectures that depend on complex attention mechanisms. Extensive experiments on the NYUDv2 and KITTI datasets demonstrate the effectiveness of our proposed LMDepth. Compared to previous lightweight depth estimation methods, LMDepth achieves higher performance with fewer parameters and lower computational complexity (measured by GFLOPs). We further deploy LMDepth on an embedded platform with INT8 quantization, validating its practicality for real-world edge applications.

Figures (8)

• Figure 1: Complextity and performance comparisons of representative lightweight depth estimation methods. The size of $\circ$ and $\Delta$ represents GFlops, with larger sizes indicating higher GFlops. Compared with other methods, our method has higher accuracy while keeping fewer parameters and lower GFlops across different image resolutions.
• Figure 2: Comparison of computational processes of tokens among CNN, Transformer, and MAMBA. It highlights why our Mamba-based depth estimation method is both efficient and powerful. Compared to CNN, MAMBA achieves an improved receptive field. In contrast to Transformer, MAMBA reduces computational complexity from quadratic to linear.
• Figure 3: The overall network architecture of our LMdepth. It consists of an image encoder, an MPSP head, and a decoder. The MPSP head outputs predicted scene classification and depth bins, while the decoder, composed of four custom-designed DMBs, predicts the depth distribution probabilities.
• Figure 4: The overview of VMamba Module. A one-dimensional convolution operation is added to the original Mamba to enhance the extraction of local attention.
• Figure 5: Varing projection features on 2 and 4 MPSP layers.Red lines show the trend of $\delta_1$ accuracy, and Blue lines show trend of FLOPs. LMDepth is selected for the highest accuracy, and LMDepth-S is selected for the equilibrium point between efficiency and performance.