Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Incorporating Degradation Estimation in Light Field Spatial Super-Resolution

Zeyu Xiao, Zhiwei Xiong

TL;DR

This work tackles blind light-field super-resolution under unknown and diverse degradations by proposing LF-DEST, a two-step framework that first estimates degradation factors (blur kernels and noise maps) and then restores a high-quality HR light field. A key innovation is the degradation estimator N_DE, coupled with a robust restoration pipeline that uses modulated and selective fusion (MSF) and spatial-angular versatile (SAV) blocks to fuse degradation representations with LR features. End-to-end training and the use of a self-constraint loss for degradation estimation enable LF-DEST to handle a broad range of degradations, including real-world data, and to outperform state-of-the-art methods on both synthetic and real LF SR benchmarks. The approach advances practical light-field imaging by improving robustness to complex degradations and offering a scalable framework for integrating degradation awareness into SR models.

Abstract

Recent advancements in light field super-resolution (SR) have yielded impressive results. In practice, however, many existing methods are limited by assuming fixed degradation models, such as bicubic downsampling, which hinders their robustness in real-world scenarios with complex degradations. To address this limitation, we present LF-DEST, an effective blind Light Field SR method that incorporates explicit Degradation Estimation to handle various degradation types. LF-DEST consists of two primary components: degradation estimation and light field restoration. The former concurrently estimates blur kernels and noise maps from low-resolution degraded light fields, while the latter generates super-resolved light fields based on the estimated degradations. Notably, we introduce a modulated and selective fusion module that intelligently combines degradation representations with image information, allowing for effective handling of diverse degradation types. We conduct extensive experiments on benchmark datasets, demonstrating that LF-DEST achieves superior performance across a variety of degradation scenarios in light field SR.

Incorporating Degradation Estimation in Light Field Spatial Super-Resolution

TL;DR

This work tackles blind light-field super-resolution under unknown and diverse degradations by proposing LF-DEST, a two-step framework that first estimates degradation factors (blur kernels and noise maps) and then restores a high-quality HR light field. A key innovation is the degradation estimator N_DE, coupled with a robust restoration pipeline that uses modulated and selective fusion (MSF) and spatial-angular versatile (SAV) blocks to fuse degradation representations with LR features. End-to-end training and the use of a self-constraint loss for degradation estimation enable LF-DEST to handle a broad range of degradations, including real-world data, and to outperform state-of-the-art methods on both synthetic and real LF SR benchmarks. The approach advances practical light-field imaging by improving robustness to complex degradations and offering a scalable framework for integrating degradation awareness into SR models.

Abstract

Recent advancements in light field super-resolution (SR) have yielded impressive results. In practice, however, many existing methods are limited by assuming fixed degradation models, such as bicubic downsampling, which hinders their robustness in real-world scenarios with complex degradations. To address this limitation, we present LF-DEST, an effective blind Light Field SR method that incorporates explicit Degradation Estimation to handle various degradation types. LF-DEST consists of two primary components: degradation estimation and light field restoration. The former concurrently estimates blur kernels and noise maps from low-resolution degraded light fields, while the latter generates super-resolved light fields based on the estimated degradations. Notably, we introduce a modulated and selective fusion module that intelligently combines degradation representations with image information, allowing for effective handling of diverse degradation types. We conduct extensive experiments on benchmark datasets, demonstrating that LF-DEST achieves superior performance across a variety of degradation scenarios in light field SR.
Paper Structure (15 sections, 12 equations, 9 figures, 5 tables)

This paper contains 15 sections, 12 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Degradation Model
  5. Problem Formulation
  6. Method
  7. Overview
  8. Degradation Estimation
  9. High-Resolution Light Field Restoration
  10. Experiments
  11. Experimantal Settings
  12. Quantitative and Qualitative Comparisons
  13. Model Analysis
  14. Limitations and Discussions
  15. Conclusion

Figures (9)

  • Figure 1: Examples of light field SR. Visual comparisons (central views) of $\times 4$ SR on a severe degraded light field scenes (kernel width=$4.5$ and noise level=$15$). Existing degradation-aware light field SR method, LF-DMNet, tends to generate blurry results with obvious artifacts. Our proposed LF-DEST can recover faithful details from the blurry and noisy input light field.
  • Figure 2: An overview of the proposed LF-DEST. LF-DEST consists of two parts: (a) degradation estimation and (b) light field restoration. For simplicity, we use a light field with a $3 \times 3$ angular resolution as an example to illustrate LF-DEST.
  • Figure 3: Details of the degradation estimator $\mathcal{N}_{DE}$.
  • Figure 4: Details of the MSF module. For simplicity, we utilize $\bm{F}_1$ as an example.
  • Figure 5: Visual results achieved by different methods on synthetically degraded light field (top: kernel width=3, noise level=15 and bottom: kernel width=1.5, noise level=50) for $\times 4$ SR. The super-resolved center view images and EPIs are shown.
  • ...and 4 more figures