When Dance Video Archives Challenge Computer Vision
Philippe Colantoni, Rafique Ahmed, Prashant Ghimire, Damien Muselet, Alain Trémeau
TL;DR
The paper tackles robust 3D pose estimation in archival dance videos by introducing a modular nine-step pipeline that delivers 2D/3D poses, tracks, and scene reconstructions in world coordinates. It integrates state-of-the-art components (SMPL-X, Multi-HMR, SAM2, DUSt3R/MASt3R, and RBF interpolation) with scene-cut detection and camera-motion estimation to semantically enrich digitized performances. It also provides data and XR visualization tools to aid analysts in selecting and refining results under varied content and quality, demonstrating the workflow on PREMIERE archive videos. The work underscores the need for domain-specific pipelines in dance data and outlines concrete avenues for quantitative evaluation and lighting-estimation extensions to further improve realism and utility.
Abstract
The accuracy and efficiency of human body pose estimation depend on the quality of the data to be processed and of the particularities of these data. To demonstrate how dance videos can challenge pose estimation techniques, we proposed a new 3D human body pose estimation pipeline which combined up-to-date techniques and methods that had not been yet used in dance analysis. Second, we performed tests and extensive experimentations from dance video archives, and used visual analytic tools to evaluate the impact of several data parameters on human body pose. Our results are publicly available for research at https://www.couleur.org/articles/arXiv-1-2025/