VibrantLeaves: A principled parametric image generator for training deep restoration models

TL;DR

A synthetic image generator relying on a few simple principles, integrated in a classical Dead Leaves model, that yields better robustness to various geometric and radiometric perturbations of the test sets.

Abstract

Even though Deep Neural Networks are extremely powerful for image restoration tasks, they have several limitations. They are poorly understood and suffer from strong biases inherited from the training sets. One way to address these shortcomings is to have a better control over the training sets, in particular by using synthetic sets. In this paper, we propose a synthetic image generator relying on a few simple principles. In particular, we focus on geometric modeling, textures, and a simple modeling of image acquisition. These properties, integrated in a classical Dead Leaves model, enable the creation of efficient training sets. Standard image denoising and super-resolution networks can be trained on such datasets, reaching performance almost on par with training on natural image datasets. As a first step towards explainability, we provide a careful analysis of the considered principles, identifying which image properties are necessary to obtain good performances. Besides, such training also yields better robustness to various geometric and radiometric perturbations of the test sets.

Figures (17)

• Figure 1: Graphical summary of the VibrantLeaves model. We integrate three key properties in an occlusion-based model: (1) Geometry, through a random shape generator based on concave hulls, (2) Textures with an exemplar-free parametric texture generator of both pseudo-periodic and micro textures, (3) Depth with a depth-of-field generator and perspective effects. The generated images match natural images statistics much better than other statistical image models, while remaining abstract and free of real image dataset biases.
• Figure 2: Random shape generation. We start by sampling $n$ points (here $n=100$) uniformly in a disk (see first picture), thereafter ensuring rotation invariance. Then, we compute the concave hull of $(x_i)_{i\leq n}$ with a ratio $\alpha$. As shown here, the larger the $\alpha$, the closer the concave hull is to the convex hull.
• Figure 3: Gaussian Smoothing of a random concave hull. From left to right: original shape, shape after Gaussian blurring, and final shape, obtained after thresholding.
• Figure 4: Samples from our shape generator. First row: sharp polygons. Second row: shapes after Gaussian smoothing.
• Figure 5: Pseudo-Periodic textures samples from our texture model in either 1 or 2 dimensions. We start from simple sinusoids and enrich them by: (1) applying a thresholding to the interpolation fields, (2) stacking sinusoids of random periods , and (2) randomly distorting texture maps with gaussian vector fields.