SemiNFT: Learning to Transfer Presets from Imitation to Appreciation via Hybrid-Sample Reinforcement Learning

TL;DR

Extensive experiments show that SemiNFT not only outperforms state-of-the-art methods on standard preset transfer benchmarks but also demonstrates remarkable intelligence in zero-shot tasks, such as black-and-white photo colorization and cross-domain (anime-to-photo) preset transfer.

Abstract

Photorealistic color retouching plays a vital role in visual content creation, yet manual retouching remains inaccessible to non-experts due to its reliance on specialized expertise. Reference-based methods offer a promising alternative by transferring the preset color of a reference image to a source image. However, these approaches often operate as novice learners, performing global color mappings derived from pixel-level statistics, without a true understanding of semantic context or human aesthetics. To address this issue, we propose SemiNFT, a Diffusion Transformer (DiT)-based retouching framework that mirrors the trajectory of human artistic training: beginning with rigid imitation and evolving into intuitive creation. Specifically, SemiNFT is first taught with paired triplets to acquire basic structural preservation and color mapping skills, and then advanced to reinforcement learning (RL) on unpaired data to cultivate nuanced aesthetic perception. Crucially, during the RL stage, to prevent catastrophic forgetting of old skills, we design a hybrid online-offline reward mechanism that anchors aesthetic exploration with structural review. % experiments Extensive experiments show that SemiNFT not only outperforms state-of-the-art methods on standard preset transfer benchmarks but also demonstrates remarkable intelligence in zero-shot tasks, such as black-and-white photo colorization and cross-domain (anime-to-photo) preset transfer. These results confirm that SemiNFT transcends simple statistical matching and achieves a sophisticated level of aesthetic comprehension. Our project can be found at https://melanyyang.github.io/SemiNFT/.

Figures (12)

• Figure 1: Illustration of the SemiNFT. (a) The architecture integrates reference and source images as inputs through causal attention. The cold-start and post-training LoRA modules are trained in the cold-start and post-training stage, respectively. (b) The scoring process of SemiNFT includes online reward and offline reward. (c) We fine-tune the velocity predictor using DiffusionNFT, enhanced by incorporating a prepared SFT dataset.
• Figure 2: Visual comparison with baselines. The above three lines show the results on the realistic dataset, while the last two lines show results on the synthetic dataset.
• Figure 3: Applications to extended scenes, including black-and-white coloration, cross-domain preset transfer, and text-based restoration for old photos. (the down-left or down-right image is the reference image)
• Figure 4: User Study: Selection ratios of preset similarity and human aesthetics on realistic and synthetic dataset.
• Figure 5: Visualization of ablation studies. Here we include SemiNFT, 'w/o offline samples', 'w/o realistic dataset', and 'w/o RL'. For clarity, we omit 'w/o cold-start stage' as it's the worst.