Physics-Grounded Attached Shadow Detection Using Approximate 3D Geometry and Light Direction

TL;DR

This work addresses the lack of attached-shadow detection by introducing a geometry-informed, dual-module framework that jointly detects cast and attached shadows and estimates light direction from a single image with normals. It leverages a closed-loop refinement where a geometry-derived partial attached-shadow map informs shadow detection and, conversely, shadow predictions refine light estimation. A new dataset with per-pixel cast and attached masks, normals, and heuristic light directions enables rigorous evaluation and training. Experimental results demonstrate significant improvements in attached-shadow detection (BER reduction of at least 33% relative to prior methods) while preserving strong cast-shadow performance. The approach highlights the value of integrating surface geometry and illumination cues for more accurate shadow understanding and scene reasoning.

Abstract

Attached shadows occur on the surface of the occluder where light cannot reach because of self-occlusion. They are crucial for defining the three-dimensional structure of objects and enhancing scene understanding. Yet existing shadow detection methods mainly target cast shadows, and there are no dedicated datasets or models for detecting attached shadows. To address this gap, we introduce a framework that jointly detects cast and attached shadows by reasoning about their mutual relationship with scene illumination and geometry. Our system consists of a shadow detection module that predicts both shadow types separately, and a light estimation module that infers the light direction from the detected shadows. The estimated light direction, combined with surface normals, allows us to derive a geometry-consistent partial map that identifies regions likely to be self-occluded. This partial map is then fed back to refine shadow predictions, forming a closed-loop reasoning process that iteratively improves both shadow segmentation and light estimation. In order to train our method, we have constructed a dataset of 1,458 images with separate annotations for cast and attached shadows, enabling training and quantitative evaluation of both. Experimental results demonstrate that this iterative geometry-illumination reasoning substantially improves the detection of attached shadows, with at least 33% BER reduction, while maintaining strong full and cast shadow performance.

Figures (8)

• Figure 1: Existing shadow detection methods primarily focus on cast shadow detection and do not differentiate between cast and attached shadow types (b, c, d). In contrast, our method accurately segments both cast and attached shadows, separately (e, f).
• Figure 2: Overview. Our network consists of two modules: a shadow detection module and a light estimation module. The shadow detection module simultaneously predicts both cast and attached shadows, while the light estimation module predicts a three-dimensional light direction and generates a partial attached shadow map from the estimated light direction. This partial map is then fed back into the shadow detection module as an additional input during iterative training.
• Figure 3: Problem Setup. The goal is to separately predict cast and attached shadows. Cast shadows appear on external surfaces, while attached shadows form directly on the object itself.
• Figure 4: Examples of our proposed dataset. We curate the dataset from the WSRD vasluianu2024ntire, SOBA wang2020instance, and CUHK Hu2019RevisitingSD datasets. Each image is paired with its corresponding normal map, light direction, cast shadow, attached shadow, and "undefined" shadow masks. The light direction is specified in a camera-centric frame, with $+x$ right, $+y$ down, and $+z$ inward.
• Figure 5: Qualitative comparison of the proposed framework for detecting cast and attached shadows against retrained BDRARzhu18b, FSDNetHu2019RevisitingSD, FDRNetzhu2021mitigating, and SILTyang2023silt models on our dataset. Colored masks are overlaid on the original image (Green: Attached, Red: Cast). Our proposed physics-grounded iterative approach better captures attached shadows.