Exploiting Deblurring Networks for Radiance Fields

TL;DR

DeepDeblurRF tackles the problem of synthesizing sharp novel views from blurred multi-view training data by integrating DNN-based image deblurring with radiance-field construction in an RF-guided, alternating framework. It supports multiple 3D representations, including Plenoxels and Gaussian Splatting, and introduces BlurRF-Synth and BlurRF-Real to enable large-scale training and realistic evaluation. Experimental results demonstrate state-of-the-art novel-view synthesis quality with substantially reduced training time across camera motion and defocus blur, and robustness to non-linear artifacts like saturation and noise. The work broadens the practical applicability of radiance-field methods to blurred real-world data and provides valuable benchmarks for future research in radiance-field deblurring.

Abstract

In this paper, we propose DeepDeblurRF, a novel radiance field deblurring approach that can synthesize high-quality novel views from blurred training views with significantly reduced training time. DeepDeblurRF leverages deep neural network (DNN)-based deblurring modules to enjoy their deblurring performance and computational efficiency. To effectively combine DNN-based deblurring and radiance field construction, we propose a novel radiance field (RF)-guided deblurring and an iterative framework that performs RF-guided deblurring and radiance field construction in an alternating manner. Moreover, DeepDeblurRF is compatible with various scene representations, such as voxel grids and 3D Gaussians, expanding its applicability. We also present BlurRF-Synth, the first large-scale synthetic dataset for training radiance field deblurring frameworks. We conduct extensive experiments on both camera motion blur and defocus blur, demonstrating that DeepDeblurRF achieves state-of-the-art novel-view synthesis quality with significantly reduced training time.

Figures (8)

• Figure 1: Given a set of multi-view blurry images even with non-linear outliers such as saturated pixels and noise, DeepDeblurRF performs high-quality novel-view synthesis with highly efficient training. DeepDeblurRF-P and DeepDeblurRF-G are the results of our framework, where radiance fields are constructed using Plenoxels fridovich2022plenoxels and 3D Gaussian Splatting kerbl2023gaussiansplatting, respectively. Note that DP-NeRF lee2023dpnerf was trained with two GPUs due to its memory demands, whereas other models were trained on a single NVIDIA TITAN RTX GPU.
• Figure 2: Overall framework and intermediate result of each step of DeepDeblurRF.
• Figure 3: As iterations progress, the rendered images contain increasingly high-quality scene information, which subsequently improves the RF-guided deblurring network's performance in the next iteration.
• Figure 4: Examples of the BlurRF-Synth and BlurRF-Real datasets. The examples show blurred views in BlurRF-Real, while the top and bottom rows in BlurRF-Synth include blurred views and their corresponding sharp views.
• Figure 5: Qualitative results of novel-view synthesis on BlurRF-Synth test scenes.