Depth-Aware Super-Resolution via Distance-Adaptive Variational Formulation

TL;DR

Depth-Aware Super-Resolution via Distance-Adaptive Variational Formulation proposes a depth-varying SR framework that explicitly accounts for distance-dependent degradation through a distance-dependent pseudodifferential operator $\mathcal{K}_{\mathcal{D}}$ and a depth-adaptive energy $E[u] = \tfrac{1}{2} \|\mathcal{K}_{\mathcal{D}} u - u_0\|^2 + \lambda \mathcal{R}_{\mathcal{D}}[u]$. The degradation symbol and spectral constraints are analyzed to derive a depth-specific cutoff $\xi_c(d)$, informing a distance-adaptive reconstruction kernel; the solver uses discrete gradient-flow dynamics implemented by depth-conditioned residual blocks, with a consistency loss to ensure convergence to the energy minimizer. Distance-adaptive priors $g(d,|\nabla u|)$ and $h(d)$ are learned via neural networks, enabling near-field preservation of details and far-field smoothing guided by atmospheric scattering constraints. Empirical results on outdoor KITTI data show state-of-the-art PSNR/SSIM at $\times$2 and $\times$4 scales, with robust improvements on depth-variant scenes and competitive performance on traditional SR benchmarks, validating the practical impact of incorporating geometric depth into SR.

Abstract

Single image super-resolution traditionally assumes spatially-invariant degradation models, yet real-world imaging systems exhibit complex distance-dependent effects including atmospheric scattering, depth-of-field variations, and perspective distortions. This fundamental limitation necessitates spatially-adaptive reconstruction strategies that explicitly incorporate geometric scene understanding for optimal performance. We propose a rigorous variational framework that characterizes super-resolution as a spatially-varying inverse problem, formulating the degradation operator as a pseudodifferential operator with distance-dependent spectral characteristics that enable theoretical analysis of reconstruction limits across depth ranges. Our neural architecture implements discrete gradient flow dynamics through cascaded residual blocks with depth-conditional convolution kernels, ensuring convergence to stationary points of the theoretical energy functional while incorporating learned distance-adaptive regularization terms that dynamically adjust smoothness constraints based on local geometric structure. Spectral constraints derived from atmospheric scattering theory prevent bandwidth violations and noise amplification in far-field regions, while adaptive kernel generation networks learn continuous mappings from depth to reconstruction filters. Comprehensive evaluation across five benchmark datasets demonstrates state-of-the-art performance, achieving 36.89/0.9516 and 30.54/0.8721 PSNR/SSIM at 2 and 4 scales on KITTI outdoor scenes, outperforming existing methods by 0.44dB and 0.36dB respectively. This work establishes the first theoretically-grounded distance-adaptive super-resolution framework and demonstrates significant improvements on depth-variant scenarios while maintaining competitive performance across traditional benchmarks.

Figures (8)

• Figure 1: t-SNE visualization of feature distributions across benchmark datasets. KITTI outdoor scenes exhibit distinct clustering patterns separated from indoor/general datasets (DIV2K, Set5, Set14, BSD100, Urban100), indicating fundamentally different degradation characteristics requiring specialized reconstruction approaches.
• Figure 2: Complete pipeline of the proposed depth-aware super-resolution method. The framework processes low-resolution inputs through degradation modeling, kernel generation, and distance-adaptive reconstruction to produce high-resolution outputs.
• Figure 3: Structure of the gradient flow blocks showing residual connections and feature modulation components. The cascaded arrangement enables progressive refinement from coarse structure recovery to fine texture enhancement.
• Figure 4: Visual comparison on Set14 ($\times$2), BSD100 ($\times$4), and Urban100($\times$8) benchmarks. PSNR/SSIM values shown for quantitative assessment.
• Figure 5: Qualitative results on KITTI outdoor scenes at $\times$2, $\times$3, and $\times$4 scales demonstrating depth-aware reconstruction under atmospheric degradation.