Lightweight texture transfer based on texture feature preset

TL;DR

The paper tackles the inefficiency of texture transfer arising from highly repetitive texture features that require deep, parameter-heavy encodings. It introduces Texture Feature Preset (TFP), a lightweight framework that precomputes universal texture feature maps from noise and fuses them with shallow content features to generate texture transfers, supplemented by a semantic texture fusion loss and a semantic conditional texture generation branch to preserve content semantics. The approach yields dramatic model-size reductions (3.2× to 3538×) and speedups (1.8× to 5.6×) while delivering competitive or superior texture transfer quality, and enables multiple outputs via random noise sampling. Together, these contributions enable real-time, flexible texture transfer with diverse outputs and strong style fidelity, advancing practical applications in image editing and synthesis.

Abstract

In the task of texture transfer, reference texture images typically exhibit highly repetitive texture features, and the texture transfer results from different content images under the same style also share remarkably similar texture patterns. Encoding such highly similar texture features often requires deep layers and a large number of channels, making it is also the main source of the entire model's parameter count and computational load, and inference time. We propose a lightweight texture transfer based on texture feature preset (TFP). TFP takes full advantage of the high repetitiveness of texture features by providing preset universal texture feature maps for a given style. These preset feature maps can be fused and decoded directly with shallow color transfer feature maps of any content to generate texture transfer results, thereby avoiding redundant texture information from being encoded repeatedly. The texture feature map we preset is encoded through noise input images with consistent distribution (standard normal distribution). This consistent input distribution can completely avoid the problem of texture transfer differentiation, and by randomly sampling different noise inputs, we can obtain different texture features and texture transfer results under the same reference style. Compared to state-of-the-art techniques, our TFP not only produces visually superior results but also reduces the model size by 3.2-3538 times and speeds up the process by 1.8-5.6 times.

Figures (9)

• Figure 1: Ablation study of feature decoding consistency loss.$\mathcal{L}_{mtv}$ ensures that shallow features in the shallow and fusion decoders' outputs yield similar results.
• Figure 2: Architecture illustration of the proposed CTDP. See Section 3 for details.
• Figure 3: Architecture illustration of the proposed CTDP. See Section 3 for details.
• Figure 4: Architecture illustration of the proposed CTDP. See Section 3 for details.
• Figure 5: Architecture illustration of the proposed CTDP. See Section 3 for details.