EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision

TL;DR

This paper tackles limited dataset diversity in event-based vision from Dynamic Vision Sensors (DVS) and shows that preserving spatial integrity and temporal continuity (SI and TC) during augmentation is essential. It introduces EventZoom, a progressive embedding augmentation that sequentially inserts transformed event sequences into originals via progressive scaling and shifting, with $\lambda_t$ drawn from $Beta(\lambda_{min},\lambda_{max})$ and linearly interpolated in time to preserve coherence. Evaluations across supervised, semi-supervised, and unsupervised learning on DVS-CIFAR10, N-Caltech101, and UCF101-DVS show that EventZoom achieves state-of-the-art performance against existing augmentation methods. Ablation confirms the central role of SI and TC, and results indicate strong generalization to real-world high-dynamics scenes.

Abstract

Dynamic Vision Sensors (DVS) capture event data with high temporal resolution and low power consumption, presenting a more efficient solution for visual processing in dynamic and real-time scenarios compared to conventional video capture methods. Event data augmentation serve as an essential method for overcoming the limitation of scale and diversity in event datasets. Our comparative experiments demonstrate that the two factors, spatial integrity and temporal continuity, can significantly affect the capacity of event data augmentation, which are guarantee for maintaining the sparsity and high dynamic range characteristics unique to event data. However, existing augmentation methods often neglect the preservation of spatial integrity and temporal continuity. To address this, we developed a novel event data augmentation strategy EventZoom, which employs a temporal progressive strategy, embedding transformed samples into the original samples through progressive scaling and shifting. The scaling process avoids the spatial information loss associated with cropping, while the progressive strategy prevents interruptions or abrupt changes in temporal information. We validated EventZoom across various supervised learning frameworks. The experimental results show that EventZoom consistently outperforms existing event data augmentation methods with SOTA performance. For the first time, we have concurrently employed Semi-supervised and Unsupervised learning to verify feasibility on event augmentation algorithms, demonstrating the applicability and effectiveness of EventZoom as a powerful event-based data augmentation tool in handling real-world scenes with high dynamics and variability environments.

Figures (12)

• Figure 1: Visualization of Boxplots of multiple experiments using event augmentation methods in settings lacking temporal continuity and spatial integrity.
• Figure 2: The data augmentation process for EventZoom is illustrated. Event Sample 1-3 undergoes a progressive scaling and shifting along the temporal dimension. The scaled sample are then incorporated into Original Sample . Depending on the mixnum settings, the number of samples varies. Each time step is assigned a unique label, which is synthesized based on the proportion of events inserted.
• Figure 3: Visualization of the samples generated by EventZoom. The zoomed sample is embeded into the original sample.
• Figure 4: Comparison of different data augmentation methods at different time steps.
• Figure 5: The figure shows the original (first row) and EventZoom enhanced (second row) version of an event sample. We set $mixnum$ to 2 for illustrative purposes.