Color Matching Using Hypernetwork-Based Kolmogorov-Arnold Networks

TL;DR

This work introduces cmKAN, a lightweight, hypernetwork-guided color matching framework that leverages Kolmogorov-Arnold Networks to perform spline-based, spatially adaptive color transformations between source and target color distributions. By generating per-pixel KAN parameters through a dedicated Generator Network (with Illumination Estimator, Color Transformer, and Color Feature Modulator), cmKAN handles nonuniform illumination and ISP-induced variations across raw and sRGB spaces. The approach supports supervised, unsupervised, and paired optimization, and is validated on a large paired-camera dataset along with standard benchmarks, consistently achieving state-of-the-art color accuracy with reduced computational cost. A dedicated cmKAN-Light variant enables efficient online paired optimization, and extensive ablations quantify the contributions of each architectural component. The work provides significant practical impact for camera ISP design, cross-camera color consistency, and post-capture color editing, complemented by publicly released datasets, models, and code.

Abstract

We present cmKAN, a versatile framework for color matching. Given an input image with colors from a source color distribution, our method effectively and accurately maps these colors to match a target color distribution in both supervised and unsupervised settings. Our framework leverages the spline capabilities of Kolmogorov-Arnold Networks (KANs) to model the color matching between source and target distributions. Specifically, we developed a hypernetwork that generates spatially varying weight maps to control the nonlinear splines of a KAN, enabling accurate color matching. As part of this work, we introduce a first large-scale dataset of paired images captured by two distinct cameras and evaluate the efficacy of our and existing methods in matching colors. We evaluated our approach across various color-matching tasks, including: (1) raw-to-raw mapping, where the source color distribution is in one camera's raw color space and the target in another camera's raw space; (2) raw-to-sRGB mapping, where the source color distribution is in a camera's raw space and the target is in the display sRGB space, emulating the color rendering of a camera ISP; and (3) sRGB-to-sRGB mapping, where the goal is to transfer colors from a source sRGB space (e.g., produced by a source camera ISP) to a target sRGB space (e.g., from a different camera ISP). The results show that our method outperforms existing approaches by 37.3% on average for supervised and unsupervised cases while remaining lightweight compared to other methods. The codes, dataset, and pre-trained models are available at: https://github.com/gosha20777/cmKAN

Figures (16)

• Figure 1: We present cmKAN, a learning framework for color matching. Our framework is versatile, supporting three color-matching scenarios: supervised and unsupervised offline training, as well as paired-based optimization. It is suitable for different color-matching tasks, such as (1) raw-to-raw, (2) raw-to-sRGB and (3) sRGB-to-sRGB mapping. Compared to other methods (e.g., SIRLUT li2024sirlut), our method demonstrates promising results across all tasks with a small number of parameters.
• Figure 2: a) Overview of the cmKAN architecture, b) Generator Network ($\mathcal{G}$) and its key components: c) Illumination Estimator (IE), d) Color Transformer (CT), and e) Color Feature Modulator (CFM).
• Figure 3: The proposed Multi-Scale Color Attention (MCA) to efficiently exploit the channel-wise dependencies.
• Figure 4: Qualitative comparison of unsupervised raw-to-raw mapping on the dataset in afifi2021semi. Our cmKAN method achieves the most accurate color mapping with lower $\Delta E$ errors, while being more lightweight than RawFormer perevozchikov2024rawformer. Gamma operator is applied to aid visualization. Best viewed in the electronic version.
• Figure 5: Qualitative comparison of raw-to-sRGB rendering on the Zurich raw-to-sRGB dataset ignatov2020replacing. Our cmKAN method achieves the most accurate color mapping with lower $\Delta E$ errors. Best viewed in the electronic version.