Hand Shadow Art: A Differentiable Rendering Perspective

TL;DR

The work addresses generating hand shadow art by deforming 3D hand meshes so their rendered silhouette matches a given target image under a fixed viewing configuration, using differentiable rendering with MANO hand models. It optimizes pose, rotation, and translation for both hands to minimize the L2 difference $||I - \mathcal{R}(\mathcal{C}, \mathcal{M}_L, \mathcal{M}_R)||_{2}$, while enforcing a self-intersection penalty and restricting joint motion to 15 joints per hand over 5000 iterations in a single-view setup. The approach enables shadow-based interpolation between two target shadows with two hands, but performance is sensitive to initialization and can struggle with large pose transitions, highlighting avenues for initialization-free and multi-view extensions. This differentiable-rendering-based framework offers a tool for the graphics community to synthesize and interpolate hand shadow art for entertainment, education, and artistic installations.

Abstract

Shadow art is an exciting form of sculptural art that produces captivating artistic effects through the 2D shadows cast by 3D shapes. Hand shadows, also known as shadow puppetry or shadowgraphy, involve creating various shapes and figures using your hands and fingers to cast meaningful shadows on a wall. In this work, we propose a differentiable rendering-based approach to deform hand models such that they cast a shadow consistent with a desired target image and the associated lighting configuration. We showcase the results of shadows cast by a pair of two hands and the interpolation of hand poses between two desired shadow images. We believe that this work will be a useful tool for the graphics community.

Figures (4)

• Figure 1: Different hand configurations to render the target shadow image during the optimization process from one hand (first row) and two hands (second and third row). The images are rendered in $256 \times 256$ resolution.
• Figure 2: The outline of the proposed differentiable rendering framework to simulate hand shadow art. The framework is optimized over $5000$ iterations to reach convergence.
• Figure 3: Different hand configurations (top: hand meshes, bottom: rendered image) while interpolating from a Rabbit to a Bird silhouette. Here is a https://gifyu.com/image/SgUsb.
• Figure 4: Failure cases over large and abrupt transitions and sensitivity to the initial hand configuration.