Light-scattering reconstruction of transparent shapes using neural networks

TL;DR

The paper tackles the challenge of visualizing and reconstructing time-evolving 3D shapes of transparent thin sheets in fluid flows using a low-cost, single-camera setup that employs stacked light sheets and Rayleigh scattering to generate a spatio-temporal hypercloud. A neural autoencoder is trained to map discrete 3D points plus time to a continuous disk surface via a pair of networks, with isometricity penalties enabling robust reconstruction when folds cause self-overlaps. Validation on synthetic data and experiments with sedimenting elastic disks demonstrates accurate 3D shape recovery, preserving approximate in-plane area and capturing complex deformations while managing noise and non-Lagrangian surface parametrizations. The method provides a practical, scalable approach for non-intrusive 3D shape imaging in slow-to-moderate dynamics, with open-access code and clear guidelines for parameter choices and boundary extraction using an $\alpha$-shape framework.

Abstract

The accurate characterisation of the 3D deformations of slender fibres and thin sheets in flow, is a key experimental challenge in the study of particle-laden flows. We propose a high-resolution, single-camera method to visualise non-intrusively the shape of a transparent crumpled sheet, as it translates, rotates and deforms. We perform periodic scans of the crumpled shape by illuminating it with a sequence of stacked light sheets at a rate much faster than its deformation and image the scattered light signal in a plane near-orthogonal to the plane of lighting. Processing of the data using a pinhole camera model yields a noisy spatio-temporal dataset of the strongly deformed time-evolving surface of the sheet, which we reconstruct in 3D using a neural autoencoder. We validate the robustness of the shape reconstruction algorithm to noise using synthetic data sets, and demonstrate the accurate reconstruction of laboratory sedimentation experiments with elastic disks. We find that the inclusion of isometricity-enforcing penalties into the cost function of the autoencoder enables us to robustly reconstruct highly folded shapes, where different regions of the sheet overlap.

Figures (12)

• Figure 1: Schematic diagram of the experimental set-up (to scale), with a single plane of light shining through an elastic sheet. The transparent planes show the light sheets created by the first and last rows of pixels of the projector. Inset: a typical image captured by the top-view camera during a scan of a crumpled sheet.
• Figure 2: (a) Mapping of the distorted camera image due to lens aberration with pixel coordinates $(\xi, \eta)$ to the corrected coordinate $(X,Y)$. (b) Geometric optics of the top view camera capturing a luminous point in air. (c) Geometric optics of the top view camera capturing a luminous point inside a liquid.
• Figure 3: (a) Schematic diagram of the projector system in side view. (b,c) Spatio-temporal diagram of the illumination cast by the HD projector, where $k(t)$ is the coordinate of the illuminated rows of pixels; (b) operation in static mode where multiple rows of pixels spaced $\delta_2$ apart are displayed simultaneously; (c) operation in dynamic mode where rows of pixels separated by $\delta_1$ are sequentially illuminated to performs a scan in the region of interest of size $N_\mathrm{R}$. Upon completion of each scan, the projector flashes and the region of interest is shifted up by $\delta_2$ pixels
• Figure 4: The time series of the total brightness of a binarized image as the object passes through stationary planes of light (blue line). The orange line shows the low-pass filtered time-series, whose minima (red points) are used to split the video into separate scans in the static mode.
• Figure 5: (a) A typical stacked image of an elastic disk, scanned in dynamic mode, following Gaussian blur convolution and prior to removal of specks of dust. (b) The filtered dataset after removal of specks of dust.