Event-based Visual Deformation Measurement

TL;DR

This work tackles dense deformation measurement in challenging, dynamic scenes where traditional frame-based VDM struggles due to large inter-frame motion and prohibitive data storage. It introduces an event-frame fusion strategy and an Affine Invariant Simplicial (AIS) framework that linearizes the deformation field into low-parameter, locally affine sub-regions, combined with a neighborhood-greedy optimization to suppress long-term error accumulation. A new benchmark with temporally aligned event and frame data across 120+ sequences demonstrates robust performance, achieving a survival rate of 65.7% for large displacements and substantially lower data storage than high-speed video methods. Overall, the approach enables accurate, storage-efficient dense deformation tracking with potential impact on structural health monitoring, robotics, and biomechanics.

Abstract

Visual Deformation Measurement (VDM) aims to recover dense deformation fields by tracking surface motion from camera observations. Traditional image-based methods rely on minimal inter-frame motion to constrain the correspondence search space, which limits their applicability to highly dynamic scenes or necessitates high-speed cameras at the cost of prohibitive storage and computational overhead. We propose an event-frame fusion framework that exploits events for temporally dense motion cues and frames for spatially dense precise estimation. Revisiting the solid elastic modeling prior, we propose an Affine Invariant Simplicial (AIS) framework. It partitions the deformation field into linearized sub-regions with low-parametric representation, effectively mitigating motion ambiguities arising from sparse and noisy events. To speed up parameter searching and reduce error accumulation, a neighborhood-greedy optimization strategy is introduced, enabling well-converged sub-regions to guide their poorly-converged neighbors, effectively suppress local error accumulation in long-term dense tracking. To evaluate the proposed method, a benchmark dataset with temporally aligned event streams and frames is established, encompassing over 120 sequences spanning diverse deformation scenarios. Experimental results show that our method outperforms the state-of-the-art baseline by 1.6% in survival rate. Remarkably, it achieves this using only 18.9% of the data storage and processing resources of high-speed video methods.

Figures (8)

• Figure 1: Overview of Event-based Deformation Measurement. (a) Our approach regresses deformation fields by leveraging affine-invariant spatiotemporal trajectory flows extracted from both events and images. (b) Measuring tire deformation while rolling over a step obstacle. Demonstrating robust and accurate measurement capabilities of our event-based VDM approach for objects experiencing rapid self-motion.
• Figure 2: Illustration of the key steps of our method. (a) Data input and event warping strategy. (b) Overall pipeline of the proposed event-based VDM framework. (c) Illustration of event-to-subregion association through vector cross product with anchor trajectories and subsequent displacement interpolation. (d) Image intensity sampling scheme within each subregion.
• Figure 3: The optimization steps. (a) Hierarchical subregion optimization: we first estimate rigid motion parameters of the object, then progressively split and optimize subregions from coarse to fine. (b) Neighborhood-greedy optimization: well-optimized subregions serve as anchors to guide neighboring regions toward better convergence, achieving faster convergence to the global optimum compared to direct high dimensional optimization.
• Figure 4: Hardware configuration of the hybrid event-frame system.(a) Left: Measurement setup where a pressure/stretching machine applies controlled force to the rubber sample. Right: The hybrid system comprises an event camera (Prophesee EVK4) and a 210fps grayscale camera with identical pixel size and flange distance. A signal generator produces square wave synchronization signals. The cameras are collocated by mounting a 50:50 split ratio beam splitter in front of them. Spatial calibration is performed before each data acquisition. (b) Data transmission topology of the system. A 210Hz square wave synchronization signal is used for triggering the frame camera and synchronizing the event stream. Simultaneously acquired event streams and frame data are transmitted to the computer for processing.
• Figure 5: Qualitative comparisons. Results on samples under clamping and twisting (I, II), and subjected to tip pressure (III, IV).