InstantSticker: Realistic Decal Blending via Disentangled Object Reconstruction

TL;DR

InstantSticker presents a disentangled reconstruction pipeline grounded in Image-Based Lighting to enable realistic decal blending on reconstructed objects with real-time rendering and instant editing. The approach combines ARAP-based pre-parameterization, a neural albedo texture map, a local reflection network, and a shadow factor to decouple geometry, appearance, and lighting while preserving high-frequency decal details. It introduces a Ratio Variance Warping (RVW) metric to quantify decal warping and demonstrates state-of-the-art performance in editing quality, speed, and rendering efficiency across multiple datasets. The practical impact lies in enabling robust AR/VR and digital content workflows where decals must convincingly conform to complex geometry and lighting without expensive optimization. Limitations include dependence on high-quality input meshes, with future work focusing on relaxing this constraint and expanding editable regions.

Abstract

We present InstantSticker, a disentangled reconstruction pipeline based on Image-Based Lighting (IBL), which focuses on highly realistic decal blending, simulates stickers attached to the reconstructed surface, and allows for instant editing and real-time rendering. To achieve stereoscopic impression of the decal, we introduce shadow factor into IBL, which can be adaptively optimized during training. This allows the shadow brightness of surfaces to be accurately decomposed rather than baked into the diffuse color, ensuring that the edited texture exhibits authentic shading. To address the issues of warping and blurriness in previous methods, we apply As-Rigid-As-Possible (ARAP) parameterization to pre-unfold a specified area of the mesh and use the local UV mapping combined with a neural texture map to enhance the ability to express high-frequency details in that area. For instant editing, we utilize the Disney BRDF model, explicitly defining material colors with 3-channel diffuse albedo. This enables instant replacement of albedo RGB values during the editing process, avoiding the prolonged optimization required in previous approaches. In our experiment, we introduce the Ratio Variance Warping (RVW) metric to evaluate the local geometric warping of the decal area. Extensive experimental results demonstrate that our method surpasses previous decal blending methods in terms of editing quality, editing speed and rendering speed, achieving the state-of-the-art.

Figures (10)

• Figure 1: We select a target area in the user interface to position the decal by adjusting the viewpoint and selecting four anchors to define a quadrilateral. Then, we upload an image, which will blend realistically within this quadrilateral.
• Figure 2: During the geometry stage, we reconstruct the mesh, select and parameterize a UV area, and rasterize the mesh. During appearance disentangling, we assign learnable features to each vertex and use barycentric interpolation to obtain fragment features. The environment lighting is represented as a mipmap, filtered from a learnable environment map. After reconstruction, users can upload an image and blend it into any position in UV area by overwritting its pixel colors to the albedo texture map. Please note the distinction between different forms of the same symbol. For example, the symbol $\mathbf{a}$ represents the general form used in equations, $\hat{\mathbf{a}}$ denotes the learnable vertex feature, and $\dot{\mathbf{a}}$ indicates the fragment feature interpolated from $\hat{\mathbf{a}}$.
• Figure 3: For post-parameterization, the rendered color within a triangle is limited to interpolating the features of its three vertices, hindering high-frequency texture representation. We address this by pre-parameterizing to obtain UVs and training a high-resolution texture map.
• Figure 4: Manually editing the roughness of the blending area can create a diffuse effect on a specular surface.
• Figure 5: Disentangling and decal blending results of our method.