DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution

TL;DR

This work tackles blind depth super-resolution under unknown real-world degradation by learning degradation representations from LR depth and guiding RGB-D fusion with degradation priors. The proposed DORNet combines a self-supervised degradation learning module with a degradation-oriented fusion scheme (DOFT) and a degradation-regularization pathway, enabling adaptive, multi-scale degradation handling without degradation labels. Key contributions include router-based degradation selection, multi-scale degradation kernel generators, and a loss design that couples reconstruction, degradation consistency, and contrastive degradation supervision. Empirical results on real-world RGB-D-D, TOFDSR, and synthetic NYU-v2 demonstrate state-of-the-art performance, strong robustness to noise, and favorable parameter efficiency, highlighting the method’s practicality for real-world depth sensing and fusion tasks.

Abstract

Recent RGB-guided depth super-resolution methods have achieved impressive performance under the assumption of fixed and known degradation (e.g., bicubic downsampling). However, in real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments (e.g., low reflective surfaces, varying illumination). Consequently, the performance of these methods significantly declines when real-world degradation deviate from their assumptions. In this paper, we propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes through implicit degradation representations. Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data using routing selection-based degradation regularization. To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors. Extensive experimental results on both real and synthetic datasets demonstrate the superiority of our DORNet in handling unknown degradation, outperforming existing methods. The code is available at https://github.com/yanzq95/DORNet.

Figures (12)

• Figure 1: Previous methods (a) directly fuse the RGB information aligned with the LR depth, while our method (b) focuses more on modeling the degradation representation of the LR depth to provide targeted guidance for HR depth recovery.
• Figure 2: Visual results of LR depth, HR depth, and degradation representation. (b) and (c) are the synthetic and the real-world LR depth, respectively. (d) is the learned degradation representation $\boldsymbol {\tilde{D}}$. (e)-(g) are the HR depth predicted by FDSR he2021towards, DCTNet zhao2022discrete, and SGNet wang2024sgnet, while (h) is produced by our DORNet. (i) is the histogram of real-world LR, synthetic LR, and ground-truth (GT) depth.
• Figure 3: Overview of DORNet. Given $\boldsymbol D_{up}$ as input, the degradation learning first encodes it to produce degradation representations $\boldsymbol {\tilde{D}}$ and $\boldsymbol D$. Then, $\boldsymbol {\tilde{D}}$, $\boldsymbol D$, $\boldsymbol D_{lr}$, and $\boldsymbol I$ are fed into multiple degradation-oriented feature transformation (DOFT) modules, generating the HR depth $\boldsymbol D_{hr}$. Finally, $\boldsymbol D$ and $\boldsymbol D_{hr}$ are sent to the degradation regularization to obtain $\boldsymbol D_{d}$, which is used as input for the degradation loss $\mathcal{L} _{deg}$ and the contrastive loss $\mathcal{L}_{cont}$. The degradation regularization only applies during training and adds no extra overhead in testing.
• Figure 4: Visualization of error maps and degradation representation $\boldsymbol {\tilde{D}}$ (a), and their gradient histograms (b).
• Figure 5: Details of DOFT. $\otimes$ is element-wise multiplication while ⓒ is concatenation. Orange rectangular box: residual group zhang2018image.