Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Long Range Constraints for Neural Texture Synthesis Using Sliced Wasserstein Loss

Liping Yin, Albert Chua

TL;DR

This work addresses the challenge of enforcing long-range texture constraints in single-image synthesis without user-provided spatial tags. It introduces a Sliced Wasserstein-based statistics framework, augmented with a height-projection term to form $\mathcal{L}_{\text{Slicing}} = \mathcal{L}_{\text{SW}} + \mathcal{L}_{\text{SW},H}$, and applies a coarse-to-fine multi-scale synthesis to improve nonstationary texture details. Through experiments on 34 textures, the proposed loss yields long-range structure comparable to spectrum-based approaches while avoiding hyperparameter tuning, demonstrating competitive LPIPS, FID, and KID metrics. The work provides unsupervised, scalable statistics for neural texture synthesis, with practical impact on generating textures that exhibit global coherence without manual tagging or extensive tuning.

Abstract

In the past decade, exemplar-based texture synthesis algorithms have seen strong gains in performance by matching statistics of deep convolutional neural networks. However, these algorithms require regularization terms or user-added spatial tags to capture long range constraints in images. Having access to a user-added spatial tag for all situations is not always feasible, and regularization terms can be difficult to tune. Thus, we propose a new set of statistics for texture synthesis based on Sliced Wasserstein Loss, create a multi-scale method to synthesize textures without a user-added spatial tag, study the ability of our proposed method to capture long range constraints, and compare our results to other optimization-based, single texture synthesis algorithms.

Long Range Constraints for Neural Texture Synthesis Using Sliced Wasserstein Loss

TL;DR

This work addresses the challenge of enforcing long-range texture constraints in single-image synthesis without user-provided spatial tags. It introduces a Sliced Wasserstein-based statistics framework, augmented with a height-projection term to form , and applies a coarse-to-fine multi-scale synthesis to improve nonstationary texture details. Through experiments on 34 textures, the proposed loss yields long-range structure comparable to spectrum-based approaches while avoiding hyperparameter tuning, demonstrating competitive LPIPS, FID, and KID metrics. The work provides unsupervised, scalable statistics for neural texture synthesis, with practical impact on generating textures that exhibit global coherence without manual tagging or extensive tuning.

Abstract

In the past decade, exemplar-based texture synthesis algorithms have seen strong gains in performance by matching statistics of deep convolutional neural networks. However, these algorithms require regularization terms or user-added spatial tags to capture long range constraints in images. Having access to a user-added spatial tag for all situations is not always feasible, and regularization terms can be difficult to tune. Thus, we propose a new set of statistics for texture synthesis based on Sliced Wasserstein Loss, create a multi-scale method to synthesize textures without a user-added spatial tag, study the ability of our proposed method to capture long range constraints, and compare our results to other optimization-based, single texture synthesis algorithms.
Paper Structure (11 sections, 9 equations, 7 figures, 3 tables, 2 algorithms)

This paper contains 11 sections, 9 equations, 7 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background on exemplar-based texture synthesis
  3. Background on Sliced Wasserstein Loss
  4. Theoretical justifications for SW loss
  5. Our contributions
  6. Texture Synthesis Algorithm
  7. Comparison with Other Methods
  8. Ablation Study: Varying the Number of Slices
  9. Improvements via a Multi-scale Approach
  10. Additional comparisons
  11. Conclusions

Figures (7)

  • Figure 1: Comparison of results for pseudoperiodic textures. Left: Reference. Mid Left: SW Loss. Mid Right: Spectrum. Right: Using \ref{['eqn: new slicing loss']} [Ours].
  • Figure 2: Comparison of results for more complex textures. Left: Reference. Mid Left: SW Loss. Mid Right: Spectrum. Right: Using \ref{['eqn: new slicing loss']} [Ours].
  • Figure 3: Textures Generated using a different number of directions. Left: Reference. Mid Left:$16$ directions. Mid Right:$64$ directions. Right:$256$ directions.
  • Figure 4: Multi-scale procedure at different scales. Left: Reference. Mid Left:$K = 0$. Mid Right:$K = 1$. Right:$K = 2$.
  • Figure 5: Progression of synthesis that lead to repetitions. Left: Reference Texture. Middle Left:$K = 0$. Middle Right:$K = 1$. Right:$K = 2$.
  • ...and 2 more figures