Lightweight RGB-D Salient Object Detection from a Speed-Accuracy Tradeoff Perspective

TL;DR

The paper tackles RGB-D salient object detection under limited-resource constraints by introducing SATNet, a lightweight network that achieves high accuracy with real-time speed. Core innovations include the Depth Anything Model for robust depth priors, a Decoupled Attention Module for efficient cross-modal fusion, a Dual Information Representation Module to enrich feature space, and a Dual Feature Aggregation Module to fuse texture and saliency cues. Across five public RGB-D SOD datasets, SATNet sets new benchmarks for lightweight models (5.2M parameters, 415 FPS) and even surpasses several heavyweight CNN-based approaches in efficiency while maintaining competitive accuracy. The approach enables practical edge deployment and shows promise for related tasks such as RGB-T SOD and medical image segmentation, with potential future integration of large vision foundation models.

Abstract

Current RGB-D methods usually leverage large-scale backbones to improve accuracy but sacrifice efficiency. Meanwhile, several existing lightweight methods are difficult to achieve high-precision performance. To balance the efficiency and performance, we propose a Speed-Accuracy Tradeoff Network (SATNet) for Lightweight RGB-D SOD from three fundamental perspectives: depth quality, modality fusion, and feature representation. Concerning depth quality, we introduce the Depth Anything Model to generate high-quality depth maps,which effectively alleviates the multi-modal gaps in the current datasets. For modality fusion, we propose a Decoupled Attention Module (DAM) to explore the consistency within and between modalities. Here, the multi-modal features are decoupled into dual-view feature vectors to project discriminable information of feature maps. For feature representation, we develop a Dual Information Representation Module (DIRM) with a bi-directional inverted framework to enlarge the limited feature space generated by the lightweight backbones. DIRM models texture features and saliency features to enrich feature space, and employ two-way prediction heads to optimal its parameters through a bi-directional backpropagation. Finally, we design a Dual Feature Aggregation Module (DFAM) in the decoder to aggregate texture and saliency features. Extensive experiments on five public RGB-D SOD datasets indicate that the proposed SATNet excels state-of-the-art (SOTA) CNN-based heavyweight models and achieves a lightweight framework with 5.2 M parameters and 415 FPS.

Figures (11)

• Figure 1: Illustration of the motivation for our SATNet. (a) is the comparison of original depth maps and pseudo depth maps generated by the Depth Anything Model ref-81; (b) is the different attention comparison; (c) is the comparison of performance and parameter of different RGB-D SOD methods.
• Figure 2: Framework of proposed SATNet. It consists of four parts, including RGB and Depth Encoders, Decoupled Attention Modules (DAM), a Dual Information Representation Module (DIRM), and a Decoder with the Dual Feature Aggregation Modules (DFAM). Note that we employ the depth anything model to generate pseudo depth map with more high quality than original one.
• Figure 3: The illustration of the proposed DAM. The CMP is max pooling along with channels.
• Figure 4: The visualization of the proposed DAM and other variants with channel Attention (CA), Spatial Attention (SA), Hybrid Attention (CBAM), and Self-Attention (SelfA).
• Figure 5: The feature visualization of the proposed DIRM, including the features before and after the DIRM.