Deep Image Prior with L0 Gradient Regularizer for Image Smoothing

TL;DR

This work addresses image smoothing without training data by integrating an L0 gradient regularizer into the Deep Image Prior (DIP) framework, aiming for piecewise-constant outputs that preserve sharp edges. The method formulates the objective as $min_theta ||f - g_theta(x)||_2^2 + lambda ||nabla g_theta(x)||_0$ and solves it with an ADMM scheme that introduces an auxiliary variable and a Region Fusion-based update, plus exponential output averaging. The approach, DIP-ell0, operates as an unsupervised deep filter resembling a global filter and demonstrates strong performance on edge-preserving smoothing and JPEG artifact removal, often outperforming a broad set of baselines including several supervised deep models. This indicates that carefully regularized deep priors can rival supervised methods while avoiding training data, albeit with higher computational cost, and suggests extensions to other image processing tasks.

Abstract

Image smoothing is a fundamental image processing operation that preserves the underlying structure, such as strong edges and contours, and removes minor details and textures in an image. Many image smoothing algorithms rely on computing local window statistics or solving an optimization problem. Recent state-of-the-art methods leverage deep learning, but they require a carefully curated training dataset. Because constructing a proper training dataset for image smoothing is challenging, we propose DIP-$\ell_0$, a deep image prior framework that incorporates the $\ell_0$ gradient regularizer. This framework can perform high-quality image smoothing without any training data. To properly minimize the associated loss function that has the nonconvex, nonsmooth $\ell_0$ ``norm", we develop an alternating direction method of multipliers algorithm that utilizes an off-the-shelf $\ell_0$ gradient minimization solver. Numerical experiments demonstrate that the proposed DIP-$\ell_0$ outperforms many image smoothing algorithms in edge-preserving image smoothing and JPEG artifact removal.

Figures (4)

• Figure 1: Example of image smoothing results on NKS dataset of the top four methods with highest average PSNRs. PSNR/SSIM are shown. Bold denotes best result.
• Figure 2: 10 clip art images for the JPEG compression artifact removal experiments.
• Figure 3: Top four results on JPEG compression artifact removal on Fig. \ref{['fig:miku']} with compression quality factor of 10%.
• Figure 4: Sensitivity analysis on the model parameters $\lambda, \beta,$ and $T$ on the average PSNR of 20 images of the NKS dataset.