Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Dawn of KAN in Image-to-Image (I2I) Translation: Integrating Kolmogorov-Arnold Networks with GANs for Unpaired I2I Translation

Arpan Mahara, Naphtali D. Rishe, Liangdong Deng

TL;DR

This paper tackles unpaired image-to-image translation by integrating Kolmogorov-Arnold Networks (KAN) into the CUT framework, replacing the traditional two-layer MLP with a two-layer efficient KAN. The authors design a KAN-CUT model that leverages B-splines, concatenation with GLUs, and PatchNCE-based contrastive learning within a GAN setup to produce higher-quality target-domain images. They demonstrate through experiments on Horse→Zebra and Cat→Dog that KAN-CUT achieves superior FID scores and improved visual fidelity compared to established baselines, validating the potential of KAN in generative AI tasks. The work highlights the broader impact of KAN on representation learning for generative models and suggests further exploration across more datasets and tasks.

Abstract

Image-to-Image translation in Generative Artificial Intelligence (Generative AI) has been a central focus of research, with applications spanning healthcare, remote sensing, physics, chemistry, photography, and more. Among the numerous methodologies, Generative Adversarial Networks (GANs) with contrastive learning have been particularly successful. This study aims to demonstrate that the Kolmogorov-Arnold Network (KAN) can effectively replace the Multi-layer Perceptron (MLP) method in generative AI, particularly in the subdomain of image-to-image translation, to achieve better generative quality. Our novel approach replaces the two-layer MLP with a two-layer KAN in the existing Contrastive Unpaired Image-to-Image Translation (CUT) model, developing the KAN-CUT model. This substitution favors the generation of more informative features in low-dimensional vector representations, which contrastive learning can utilize more effectively to produce high-quality images in the target domain. Extensive experiments, detailed in the results section, demonstrate the applicability of KAN in conjunction with contrastive learning and GANs in Generative AI, particularly for image-to-image translation. This work suggests that KAN could be a valuable component in the broader generative AI domain.

The Dawn of KAN in Image-to-Image (I2I) Translation: Integrating Kolmogorov-Arnold Networks with GANs for Unpaired I2I Translation

TL;DR

This paper tackles unpaired image-to-image translation by integrating Kolmogorov-Arnold Networks (KAN) into the CUT framework, replacing the traditional two-layer MLP with a two-layer efficient KAN. The authors design a KAN-CUT model that leverages B-splines, concatenation with GLUs, and PatchNCE-based contrastive learning within a GAN setup to produce higher-quality target-domain images. They demonstrate through experiments on Horse→Zebra and Cat→Dog that KAN-CUT achieves superior FID scores and improved visual fidelity compared to established baselines, validating the potential of KAN in generative AI tasks. The work highlights the broader impact of KAN on representation learning for generative models and suggests further exploration across more datasets and tasks.

Abstract

Image-to-Image translation in Generative Artificial Intelligence (Generative AI) has been a central focus of research, with applications spanning healthcare, remote sensing, physics, chemistry, photography, and more. Among the numerous methodologies, Generative Adversarial Networks (GANs) with contrastive learning have been particularly successful. This study aims to demonstrate that the Kolmogorov-Arnold Network (KAN) can effectively replace the Multi-layer Perceptron (MLP) method in generative AI, particularly in the subdomain of image-to-image translation, to achieve better generative quality. Our novel approach replaces the two-layer MLP with a two-layer KAN in the existing Contrastive Unpaired Image-to-Image Translation (CUT) model, developing the KAN-CUT model. This substitution favors the generation of more informative features in low-dimensional vector representations, which contrastive learning can utilize more effectively to produce high-quality images in the target domain. Extensive experiments, detailed in the results section, demonstrate the applicability of KAN in conjunction with contrastive learning and GANs in Generative AI, particularly for image-to-image translation. This work suggests that KAN could be a valuable component in the broader generative AI domain.
Paper Structure (20 sections, 11 equations, 6 figures, 1 table)

This paper contains 20 sections, 11 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Approach
  4. Kolmogorov-Arnold Networks (KAN)
  5. B-Splines
  6. Efficient Two-Layer KAN
  7. Integration of Basis and Spline Functions via Concatenation and Gated Linear Units (GLUs)
  8. Formulation of KAN-CUT Model
  9. Adversarial Loss for Generation of Realistic-Looking Images
  10. Final Loss Functions
  11. Experiments
  12. Datasets
  13. $Horse$→$Zebra$
  14. $Cat$→$Dog$
  15. Evaluation Metrics
  16. ...and 5 more sections

Figures (6)

  • Figure 1: Comparison of Embedding Visualizations of Simulated Data with Transformation by Simple MLP and KAN Models
  • Figure 2: Illustration of Two-Layer KAN.$\Phi_1$ and $\Phi_2$ represent the collection of all 1D functions parametrized by B-Spline in layer 1 and layer 2, respectively.
  • Figure 3: Illustration showing that images should share information based on patches. Images are sourced from the AFHQ dataset choi2020stargan and are reproduced under the Creative Commons Attribution-NonCommercial 4.0 International License.
  • Figure 4: Comparison between the Two-Layer MLP and Two-Layer KAN architectures.
  • Figure 5: Architecture of Generator used in KAN-CUT. The cat and dog images are sourced from the AFHQ dataset choi2020stargan and are reproduced under the Creative Commons Attribution-NonCommercial 4.0 International License.
  • ...and 1 more figures