Discovering an Image-Adaptive Coordinate System for Photography Processing

TL;DR

The paper addresses the inefficiency of end-to-end RGB mappings in photography processing by introducing an image-adaptive coordinate system (IAC) that precedes curve-based adjustments. IAC learns three projection vectors forming an invertible 3×3 matrix to map RGB into a task-specific coordinate space, applies per-axis 1D curves, and then maps back to RGB, enabling lightweight yet powerful edits. The approach is end-to-end trainable and demonstrates SOTA performance on photo retouching, exposure correction, and white balance editing, while maintaining low parameter counts (~39.7K) and fast inference. This provides a practical, mobile-friendly solution for real-time ISP-style processing with flexible adaptation to each image.

Abstract

Curve & Lookup Table (LUT) based methods directly map a pixel to the target output, making them highly efficient tools for real-time photography processing. However, due to extreme memory complexity to learn full RGB space mapping, existing methods either sample a discretized 3D lattice to build a 3D LUT or decompose into three separate curves (1D LUTs) on the RGB channels. Here, we propose a novel algorithm, IAC, to learn an image-adaptive Cartesian coordinate system in the RGB color space before performing curve operations. This end-to-end trainable approach enables us to efficiently adjust images with a jointly learned image-adaptive coordinate system and curves. Experimental results demonstrate that this simple strategy achieves state-of-the-art (SOTA) performance in various photography processing tasks, including photo retouching, exposure correction, and white-balance editing, while also maintaining a lightweight design and fast inference speed.

Figures (7)

• Figure 1: Compare of our IAC and previous image-adaptive curve $\&$ LUT methods.
• Figure 2: We adopt image-adaptive coordinate system (IAC) for various photography processing tasks, including photo retouching, exposure correction and white balance editing.
• Figure 3: An overview of IAC's network architecture, image $x$ would pass by network $\mathbb{N}$ to generate coordinate system vectors {$\Vec{n_1}, \Vec{n_2}, \Vec{n_3}$} and curves {$\textit{curve}_1, \textit{curve}_2, \textit{curve}_3$}.
• Figure 4: Visulization of photo retouching results on MIT-Adobe FiveK dataset fivek_dataset.
• Figure 5: Visulization of white balance editing results on dataset afifi2019colour_knn's Set2.