One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns

TL;DR

This work tackles the rigidity of procedural noise by learning a single unified generative model that can produce and seamlessly blend a wide range of noise patterns, including spatially varying blends, without training data that contains spatial variation. It achieves this with a denoising diffusion probabilistic model conditioned via SPADE on noise class and parameters, augmented by a novel CutMix strategy to encourage localized control. Key contributions include a continuous, controllable noise space, a training scheme that enables spatial variation from uniform data, and an application to inverse procedural material design that improves reconstruction fidelity and offers editing capabilities. The approach yields tileable, high-resolution noise textures and demonstrates practical utility for texture authoring and material graph optimization, while acknowledging limitations in interpolating certain noise pairs and low-density modes inherent to diffusion models.

Abstract

Procedural noise is a fundamental component of computer graphics pipelines, offering a flexible way to generate textures that exhibit "natural" random variation. Many different types of noise exist, each produced by a separate algorithm. In this paper, we present a single generative model which can learn to generate multiple types of noise as well as blend between them. In addition, it is capable of producing spatially-varying noise blends despite not having access to such data for training. These features are enabled by training a denoising diffusion model using a novel combination of data augmentation and network conditioning techniques. Like procedural noise generators, the model's behavior is controllable via interpretable parameters and a source of randomness. We use our model to produce a variety of visually compelling noise textures. We also present an application of our model to improving inverse procedural material design; using our model in place of fixed-type noise nodes in a procedural material graph results in higher-fidelity material reconstructions without needing to know the type of noise in advance.

Figures (23)

• Figure 1: Our model produces noise patterns whose characteristics (i.e. scale, ripples, distortion, etc.) interpolate naturally, creating seamless and coherent transitions. By contrast, traditional alpha-blending results in images with overlapping features, inconsistent feature opacity, and a lack of sensible transitions between the noise characteristics.
• Figure 2: Our DDPM is trained using CutMix data augmentation. We first transform the current data sample (highlighted in blue) by cutting and patching together a set of other random samples from the dataset, resulting in a training image $\mathbf{x}_0$. The noise parameters for each image patch are passed to an MLP, which projects the parameter sets into an embedding space that encodes both the noise type (class) and the noise parameters. The resulting feature vectors are tiled to form a feature grid, which is used as a conditioning signal in the U-Net's SPADE blocks, as outlined in Section \ref{['sec:method_conditioning']}.
• Figure 3: At inference time, we query our network using artificially constructed feature grids, enabling a flexible way to synthesize spatially-varying noise patterns. Here we embed four sets of noise parameters, pictorially shown as one of four colors. We blend the feature vectors using bilinear interpolation, creating a smoothly-varying feature grid, which our U-Net is able to transform into a Voronoi noise pattern with non-uniform scale and distortion characteristics.
• Figure 4: Samples from our noise dataset, procured from Adobe Substance 3D Designer. We sample 18 noises along with a variety of their parameters. Note that our dataset does not contain samples with spatially-varying properties.
• Figure 5: We compare our method to a neural texture synthesizer, PSGAN PeriodicSpatialGAN, as well as a non-parametric texture blending method, Image Melding ImageMelding. In the case of Image Melding, the first and last quarter of the image are given, only the remaining interior region is filled in. Both prior methods suffer from artifacts and repeated visual details, whereas our method is able to blend smoothly while synthesizing novel details throughout the canvas. We note that PSGAN produces anisotropic features that are not characteristic of the data distribution (e.g. horizontal streaks in bottom left example).