FlowIID: Single-Step Intrinsic Image Decomposition via Latent Flow Matching

TL;DR

This work tackles intrinsic image decomposition (IID) with a need for fast, resource-efficient models. It introduces FlowIID, a latent flow matching framework that operates in a compact latent space guided by a VAE, enabling single-step decomposition of an image into albedo and shading from a single forward pass. With about 52M parameters, FlowIID delivers competitive or superior results on MIT Intrinsic and ARAP benchmarks compared to heavier diffusion-based approaches, making it appealing for real-time and embedded applications. The method combines a VAE-guided latent space, a UNet encoder–decoder, and a two-stage training regime (VAEGAN pretraining and flow-matching optimization) to transport information in latent space and recover $S$, from which $A$ is obtained as $A = I / S$.

Abstract

Intrinsic Image Decomposition (IID) separates an image into albedo and shading components. It is a core step in many real-world applications, such as relighting and material editing. Existing IID models achieve good results, but often use a large number of parameters. This makes them costly to combine with other models in real-world settings. To address this problem, we propose a flow matching-based solution. For this, we design a novel architecture, FlowIID, based on latent flow matching. FlowIID combines a VAE-guided latent space with a flow matching module, enabling a stable decomposition of albedo and shading. FlowIID is not only parameter-efficient, but also produces results in a single inference step. Despite its compact design, FlowIID delivers competitive and superior results compared to existing models across various benchmarks. This makes it well-suited for deployment in resource-constrained and real-time vision applications.

Figures (2)

• Figure 1: Architecture of FlowIID. The upper part illustrates that VAE and a discriminator are trained in a VAEGAN setup, and the lower part illustrates model inference. The model follows an encoder–decoder design, where the input image and noise are mapped to a latent representation via the encoder and a UNet backbone with modified residual blocks (shown on the right). The VAE decoder generates the shading component from the latent vector, and the albedo is obtained by dividing the input image by its shading. The VAE encoder and discriminator are used only during training and are not required at inference time.
• Figure 2: Qualitative comparison of proposed model with existing work, from left column - (i) input image, (ii) Lettry et al. lettry2018unsupervised, (iii) Niid-net luo2020niid, (iv) Careaga and Askoy careagaIntrinsic, (v) Ours, (vi) Ground Truth. The figure shows the albedo and shading components predicted by our model alongside the ground truth and prior methods. Our model produces consistent shading while preserving the color fidelity of the albedo image.