Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era

Xiaowei Hu, Zhenghao Xing, Tianyu Wang, Chi-Wing Fu, Pheng-Ann Heng

TL;DR

Shadows critically affect scene understanding and visual realism; this work surveys deep-learning approaches for shadow detection, removal, and generation in images and videos. It provides a standardized benchmark framework, cross-dataset generalization studies, and analyses of model size/speed versus performance, while highlighting the growing role of large vision models and AIGC implications. Key contributions include a taxonomic survey, fair experimental comparisons, and publicly available resources to accelerate future research. The findings underscore strong progress yet emphasize gaps in generalization, multi-object/video consistency, and dataset diversity, informing practical applications in image/video editing and synthesis.

Abstract

Shadows are created when light encounters obstacles, resulting in regions of reduced illumination. In computer vision, detecting, removing, and generating shadows are critical tasks for improving scene understanding, enhancing image quality, ensuring visual consistency in video editing, and optimizing virtual environments. This paper offers a comprehensive survey and evaluation benchmark on shadow detection, removal, and generation in both images and videos, focusing on the deep learning approaches of the past decade. It covers key aspects such as tasks, deep models, datasets, evaluation metrics, and comparative results under consistent experimental settings. Our main contributions include a thorough survey of shadow analysis, the standardization of experimental comparisons, an exploration of the relationships between model size, speed, and performance, a cross-dataset generalization study, the identification of open challenges and future research directions, and the provision of publicly available resources to support further research in this field.

Unveiling Deep Shadows: A Survey and Benchmark on Image and Video Shadow Detection, Removal, and Generation in the Deep Learning Era

TL;DR

Shadows critically affect scene understanding and visual realism; this work surveys deep-learning approaches for shadow detection, removal, and generation in images and videos. It provides a standardized benchmark framework, cross-dataset generalization studies, and analyses of model size/speed versus performance, while highlighting the growing role of large vision models and AIGC implications. Key contributions include a taxonomic survey, fair experimental comparisons, and publicly available resources to accelerate future research. The findings underscore strong progress yet emphasize gaps in generalization, multi-object/video consistency, and dataset diversity, informing practical applications in image/video editing and synthesis.

Abstract

Shadows are created when light encounters obstacles, resulting in regions of reduced illumination. In computer vision, detecting, removing, and generating shadows are critical tasks for improving scene understanding, enhancing image quality, ensuring visual consistency in video editing, and optimizing virtual environments. This paper offers a comprehensive survey and evaluation benchmark on shadow detection, removal, and generation in both images and videos, focusing on the deep learning approaches of the past decade. It covers key aspects such as tasks, deep models, datasets, evaluation metrics, and comparative results under consistent experimental settings. Our main contributions include a thorough survey of shadow analysis, the standardization of experimental comparisons, an exploration of the relationships between model size, speed, and performance, a cross-dataset generalization study, the identification of open challenges and future research directions, and the provision of publicly available resources to support further research in this field.
Paper Structure (54 sections, 2 figures, 12 tables)

This paper contains 54 sections, 2 figures, 12 tables.

Table of Contents

  1. Introduction
  2. History and Scope
  3. Related Previous Surveys
  4. Contributions of Our Work
  5. Shadow Detection
  6. Deep Models for Image Shadow Detection
  7. Component Learning
  8. Single-Task Learning
  9. Multi-Task Learning
  10. Semi-Supervised Learning
  11. Self-Supervised Learning
  12. Large Vision Models
  13. Deep Models for Video Shadow Detection
  14. Shadow Detection Datasets
  15. Image Datasets for Shadow Detection
  16. ...and 39 more sections

Figures (2)

  • Figure 1: Shadow detection methods on the SBU-Refine (top) and CUHK-Shadow (bottom) datasets: accuracy, parameters (indicated by the area of the bubbles), and speed.
  • Figure 2: Shadow removal methods on the SRD (top) and ISTD+ (bottom) datasets: accuracy, parameters (indicated by the area of the bubbles), and speed.