Blind Image Deconvolution by Generative-based Kernel Prior and Initializer via Latent Encoding

TL;DR

This paper tackles blind image deconvolution by recovering the latent image $\bm{x}$ from a blurred observation $\bm{y}$, modeled as $\bm{y}=\bm{k}\otimes\bm{x}+\bm{n}$ with unknown kernel $\bm{k}$. It introduces a two-stage framework that first learns a GAN-based kernel generator to model kernel priors and an encoder to map blurry images to kernel latent codes, enabling a compact latent kernel manifold. During BID, the kernel is initialized in latent space and jointly optimized with a DIP-based image network, often by updating only the kernel feature to constrain the search space. Experiments on synthetic data and Lai’s benchmark show state-of-the-art performance, especially for large kernels, and demonstrate faster convergence and improved kernel estimation; limitations include data-hungry pre-training and current assumption of uniform blur, with future work on non-uniform blur and meta-learning approaches.

Abstract

Blind image deconvolution (BID) is a classic yet challenging problem in the field of image processing. Recent advances in deep image prior (DIP) have motivated a series of DIP-based approaches, demonstrating remarkable success in BID. However, due to the high non-convexity of the inherent optimization process, these methods are notorious for their sensitivity to the initialized kernel. To alleviate this issue and further improve their performance, we propose a new framework for BID that better considers the prior modeling and the initialization for blur kernels, leveraging a deep generative model. The proposed approach pre-trains a generative adversarial network-based kernel generator that aptly characterizes the kernel priors and a kernel initializer that facilitates a well-informed initialization for the blur kernel through latent space encoding. With the pre-trained kernel generator and initializer, one can obtain a high-quality initialization of the blur kernel, and enable optimization within a compact latent kernel manifold. Such a framework results in an evident performance improvement over existing DIP-based BID methods. Extensive experiments on different datasets demonstrate the effectiveness of the proposed method.

Figures (16)

• Figure 1: Illustration of the latent encoding-based kernel initialization and fine-tuning mechanism in our BID framework. The blur kernel is first initialized from the blurry image by a pre-trained kernel initializer via the latent code $\bm{z}_0$, which is expected to be close to the code of the ground truth kernel, $\bm{z}_{\mathrm{gt}}$. The corresponding kernels $\bm{k}_0$ and $\bm{k}_{\mathrm{gt}}$ in the kernel manifold is also expected to be close. Then the optimization is performed within the latent space, such that $\bm{z}_0$ is fine-tuned to $\bm{z}_{\mathrm{opt}}$, and the final estimated kernel $\bm{k}_{\mathrm{opt}}$ is closer to the ground truth $\bm{k}_{\mathrm{gt}}$. See Sec. \ref{['sec:method']} for more details.
• Figure 2: Illustration of the initialization effect of the DIP-based BID. The two rows correspond to two independent runs of SelfDeblur ren2020neural. From left to right: the randomly initialized kernel, the finally estimated kernel, and the deblurred image.
• Figure 3: Overview of the proposed BID process, with the pre-trained kernel generator and initializer, of our proposed method.
• Figure 4: Comparison of blur kernels synthesized according to physical model and generated by our pre-trained generator. Top row: blur kernels synthesized according to lai2016comparative. Bottom row: blur kernels generated by the pre-trained kernel generator.
• Figure 5: Illustration of the estimating ability of the pre-trained kernel initializer. Left: the blurry image. Top-right: the estimated blur kernel by the kernel initializer. Bottom-right: the ground-truth blur kernel.