StyMam: A Mamba-Based Generator for Artistic Style Transfer

TL;DR

This work revisits GAN and proposes a mamba-based generator, termed as StyMam, to produce high-quality stylized images without introducing artifacts and disharmonious patterns and demonstrates that the proposed method outperforms state-of-the-art algorithms in both quality and speed.

Abstract

Image style transfer aims to integrate the visual patterns of a specific artistic style into a content image while preserving its content structure. Existing methods mainly rely on the generative adversarial network (GAN) or stable diffusion (SD). GAN-based approaches using CNNs or Transformers struggle to jointly capture local and global dependencies, leading to artifacts and disharmonious patterns. SD-based methods reduce such issues but often fail to preserve content structures and suffer from slow inference. To address these issues, we revisit GAN and propose a mamba-based generator, termed as StyMam, to produce high-quality stylized images without introducing artifacts and disharmonious patterns. Specifically, we introduce a mamba-based generator with a residual dual-path strip scanning mechanism and a channel-reweighted spatial attention module. The former efficiently captures local texture features, while the latter models global dependencies. Finally, extensive qualitative and quantitative experiments demonstrate that the proposed method outperforms state-of-the-art algorithms in both quality and speed.

Figures (3)

• Figure 1: The overview network of StyMam.
• Figure 2: Qualitative comparisons with SOTA diffusion-based and GAN-based style transfer methods.
• Figure 3: Ablation studies of our StyMam.