Decoupling Fine Detail and Global Geometry for Compressed Depth Map Super-Resolution

TL;DR

This work tackles compressed depth map super-resolution by decoupling global geometry from fine geometric details. It introduces FGDE to extract and preserve fine details from RGB guidance and GGE to obtain a robust low-rank global representation that suppresses noise. The GDNet framework, combining these encoders with a depth decoder and trained with SILog loss, achieves state-of-the-art performance on AIM 2024 compressed-depth benchmarks and a synthesized Compressed-NYU dataset, both qualitatively and quantitatively. The approach offers a practical path for reliable depth reconstruction in bandwidth-limited scenarios and demonstrates strong generalization across datasets.

Abstract

Recovering high-quality depth maps from compressed sources has gained significant attention due to the limitations of consumer-grade depth cameras and the bandwidth restrictions during data transmission. However, current methods still suffer from two challenges. First, bit-depth compression produces a uniform depth representation in regions with subtle variations, hindering the recovery of detailed information. Second, densely distributed random noise reduces the accuracy of estimating the global geometric structure of the scene. To address these challenges, we propose a novel framework, termed geometry-decoupled network (GDNet), for compressed depth map super-resolution that decouples the high-quality depth map reconstruction process by handling global and detailed geometric features separately. To be specific, we propose the fine geometry detail encoder (FGDE), which is designed to aggregate fine geometry details in high-resolution low-level image features while simultaneously enriching them with complementary information from low-resolution context-level image features. In addition, we develop the global geometry encoder (GGE) that aims at suppressing noise and extracting global geometric information effectively via constructing compact feature representation in a low-rank space. We conduct experiments on multiple benchmark datasets, demonstrating that our GDNet significantly outperforms current methods in terms of geometric consistency and detail recovery. In the ECCV 2024 AIM Compressed Depth Upsampling Challenge, our solution won the 1st place award. Our codes are available at: https://github.com/Ian0926/GDNet.

Figures (5)

• Figure 1: (a) Illustration of the Degradations during Data Acquisition and Transmission. To be specific, the quality of the depth map is compromised due to downsampling, bit-depth compression, and noise introduced during data acquisition and transmission. (b) The Motivation behind the Proposed GDNet. The core idea is to leverage the compressed depth map for capturing global geometric information, while utilizing the RGB image to extract detailed geometric features. As a result, our GDNet can effectively reconstruct a high-quality depth map.
• Figure 2: The Overall Framework of the Proposed GDNet. Our GDNet leverages RGB images to capture fine geometric details while utilizing compressed depth maps to provide global depth information. By employing the above decoupling strategy, the proposed GDNet is able to reconstruct high-quality depth maps with improved accuracy. Specifically, GDNet comprises three main components: a fine geometry detail encoder, responsible for detailed geometric feature extraction; a global geometry encoder, aiming at capturing global geometric features; a depth decoder to produce high-quality depth map.
• Figure 3: (a) The Detailed Structure of the Proposed Fine Geometry Detail Encoder (FGDE). The purpose of FGDE is to preserve fine geometric details in high-resolution low-level image features while augmenting them with supplementary information derived from low-resolution context-level image features. (b) The Process of Low-rank Feature Reconstruction in Global Geometry Encoder (GGE). By integrating low-rank feature reconstruction, GGE aims at completing feature reconstruction in a low-rank space, thereby achieving the objectives of noise suppression and effective extraction of global geometric cues.
• Figure 4: Visual Comparisons on AIM 2024 Compressed Depth Upsampling Challenge Dataset. The first two rows show the RGB image, ground truth depth map, compressed depth map, and predicted depth maps produced by various methods. The third row displays the error maps for each method. In the error map, darker areas indicate smaller errors. Our method shows superior performance in recovering details within regions of subtle depth variation, offering more accurate predictions of the geometric structure. Furthermore, error maps reveal that the depth errors in our predictions are notably lower than those produced by other methods.
• Figure 5: Visual Comparisons on Compressed-NYU Dataset between Our GDNet and Other Well-known Methods. The first row displays the ground truth depth alongside the predicted results of various approaches, while the second row shows the corresponding RGB image and error maps. For error maps, darker areas indicate smaller errors. As demonstrated, our method generates depth maps that are highly consistent with the ground truth and exhibit significantly reduced errors compared to other techniques. In regions with fine geometry details, especially along the edges, other methods often produce blurred results, while our approach delivers much sharper outputs.