See-through: Single-image Layer Decomposition for Anime Characters

TL;DR

The work tackles the problem of converting a single anime illustration into a fully layered 2.5D representation for animation. It introduces a two-stage latent-diffusion framework—comprising semantic RGBA body-part decomposition and a depth-informed reconstruction pipeline—augmented by a Body Part Consistency Module to enforce global coherence across parts. A scalable data engine bootstraps 2.5D annotations from Live2D assets, producing 19 semantic parts with occluded-area labels and per-fragment drawing order, enabling robust training. The approach yields high-fidelity, manipulatable layers suitable for production workflows in puppet animation and real-time VTubing, while offering a strong 2D parsing tool for anime body parts and occlusion-aware reconstruction.

Abstract

We introduce a framework that automates the transformation of static anime illustrations into manipulatable 2.5D models. Current professional workflows require tedious manual segmentation and the artistic ``hallucination'' of occluded regions to enable motion. Our approach overcomes this by decomposing a single image into fully inpainted, semantically distinct layers with inferred drawing orders. To address the scarcity of training data, we introduce a scalable engine that bootstraps high-quality supervision from commercial Live2D models, capturing pixel-perfect semantics and hidden geometry. Our methodology couples a diffusion-based Body Part Consistency Module, which enforces global geometric coherence, with a pixel-level pseudo-depth inference mechanism. This combination resolves the intricate stratification of anime characters, e.g., interleaving hair strands, allowing for dynamic layer reconstruction. We demonstrate that our approach yields high-fidelity, manipulatable models suitable for professional, real-time animation applications.

Figures (19)

• Figure 1: Data engine for 2D anime body part segmentation. We derive coarse "seed" masks from Grad-CAM responses of individual classes, and snap them to Live2D ArtMesh visibility masks for pixel-accurate boundaries. We then refine the masks with the SAM prior, producing our final labels. © USTC LEO ACG Club.
• Figure 2: Training process for our body part decomposition framework.
• Figure 3: Visualization of our Body Part Consistency Module. (a) Input; (b) Reconstruction with the Module; (c) Reconstruction without the Module. Without the module, the body part decomposition tend to be incomplete.
• Figure 4: Depth-guided stratification within a semantic layer. We cluster pseudo-depth values into front/back strata and then inpaint the newly exposed regions.
• Figure 5: Showcase results. For each example, we show the input illustration ($I$), our decomposed semantic RGBA layers, the predicted pseudo-depth, and the reconstructed composite ($R$). The top-left example contains a minor artefact where the wine glass is duplicated, which can be easily corrected in a layer editor.