CFPNet: Improving Lightweight ToF Depth Completion via Cross-zone Feature Propagation

TL;DR

CFPNet tackles depth completion with lightweight ToF sensors by addressing the cross-zone propagation gap between zone and outside-zone areas. It introduces two propagation modules: a Direct-Attention-based Propagation Module (DAPM) and a Large-Kernel-based Propagation Module (LKPM), which together enable robust depth feature transfer across FOV gaps. The method achieves state-of-the-art performance on ZJU-L5, outperforming prior lightweight-ToF baselines in both quantitative metrics and qualitative depth consistency, while ablations confirm the complementary benefits of the two modules. This work advances practical depth completion for low-cost sensing and has potential implications for real-time robotics and AR applications where ToF-based sensing is preferred for power and cost reasons.

Abstract

Depth completion using lightweight time-of-flight (ToF) depth sensors is attractive due to their low cost. However, lightweight ToF sensors usually have a limited field of view (FOV) compared with cameras. Thus, only pixels in the zone area of the image can be associated with depth signals. Previous methods fail to propagate depth features from the zone area to the outside-zone area effectively, thus suffering from degraded depth completion performance outside the zone. To this end, this paper proposes the CFPNet to achieve cross-zone feature propagation from the zone area to the outside-zone area with two novel modules. The first is a direct-attention-based propagation module (DAPM), which enforces direct cross-zone feature acquisition. The second is a large-kernel-based propagation module (LKPM), which realizes cross-zone feature propagation by utilizing convolution layers with kernel sizes up to 31. CFPNet achieves state-of-the-art (SOTA) depth completion performance by combining these two modules properly, as verified by extensive experimental results on the ZJU-L5 dataset. The code is available at https://github.com/denyingmxd/CFPNet.

Figures (11)

• Figure 1: L5 sensing principle and performance comparison. (a) L5 would return zones of resolution $8 \times 8$, and each zone provides depth distribution information. (b) We overlay aligned zone areas on the paired RGB image and display the error maps of DELTAR and our CFPNet. The largest rectangle is the zone area, and some zones are missing due to too few received photons or inconsistency in measurement. Notice that CFPNet obtains smaller errors in outside-zone areas (the yellow rectangle).
• Figure 2: The architecture of our CFPNet. Our CFPNet takes RGB image and depth distribution from ToF sensors as inputs and outputs the depth completion prediction. Our newly proposed DAPM and LKPM are located in the fusion module and allow effective cross-zone feature propagation from zone areas to outside-zone areas.
• Figure 3: The pipeline of proposed DAPM. We conduct cross attention between pixels from zone areas and outside-zone areas. Additional convolution and skip connection are added to capture local contexts and promote propagation of gradients, respectively.
• Figure 4: Designs of ConvNeXt Block and our LKPM. Different from ConvNext, we use a $s \times s$ convolution layer where $s$ could be as large as 31 instead of fixing $s$ as 7. Moreover, we adaptively set $s$ based on the resolution of feature maps.
• Figure 5: Qualitative results comparing different kernel designs in our LKPM on ZJU-L5 dataset. Compared with using only small or large kernel sizes, our adaptive kernel design achieves the best performance.