CoatFusion: Controllable Material Coating in Images

TL;DR

This work defines Material Coating as a thin-layer edit that preserves an object's geometry while adding a controllable coating. It introduces DataCoat110K, a large synthetic dataset of 110k before/after coating pairs, and CoatFusion, a diffusion-based editor conditioned on a 2D albedo texture, a coating mask, and global PBR-style trait embeddings (roughness, metalness, transmission, thickness) implemented via LoRA. The method achieves realistic, controllable coatings and outperforms material transfer baselines and Photoshop heuristics, with a user study confirming perceptual realism. Limitations include reliance on projected albedo, handling of transmissive materials, and missing environment interactions, pointing to avenues for future enhancement.

Abstract

We introduce Material Coating, a novel image editing task that simulates applying a thin material layer onto an object while preserving its underlying coarse and fine geometry. Material coating is fundamentally different from existing "material transfer" methods, which are designed to replace an object's intrinsic material, often overwriting fine details. To address this new task, we construct a large-scale synthetic dataset (110K images) of 3D objects with varied, physically-based coatings, named DataCoat110K. We then propose CoatFusion, a novel architecture that enables this task by conditioning a diffusion model on both a 2D albedo texture and granular, PBR-style parametric controls, including roughness, metalness, transmission, and a key thickness parameter. Experiments and user studies show CoatFusion produces realistic, controllable coatings and significantly outperforms existing material editing and transfer methods on this new task.

Figures (11)

• Figure 2: Thickness parameter: Examples of applying a coating with minimal and maximal thickness levels. Note the increase in smoothness for the opaque coating (top row), and the decrease in transmissivity for the translucent coating (bottom row).
• Figure 3: Examples from DataCoat110K, generated in Blender. Each row shows (from left to right) the coating parameters, the original image, masked albedo, and the resulting image.
• Figure 4: Data generation (left) and training architecture (right). We generate a synthetic dataset by taking each of the objects, randomly generate a scene, render the original image (no coating) and then generate 64 random coating materials and render their result as well as the projected albedo and coat mask on the object in the scene (see Section \ref{['sec:dataset']}). During training we provide the image conditionings: original render, projected albedo and projected coat mask as well as the global traits and task type. We use their respective coat render as the target image. During inference we provide user input for all the aforementioned conditionings images and parameters (see Section \ref{['sec:method']}).
• Figure 5:
• Figure 6: Comparison of CoatFusion with baselines.