Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ShapeAug: Occlusion Augmentation for Event Camera Data

Katharina Bendig, René Schuster, Didier Stricker

TL;DR

This work tackles occlusion in event-camera data by introducing ShapeAug, an occlusion-aware augmentation that simulates moving foreground shapes to generate both occlusion and the associated events across temporal slices $T$. By sampling $N\in[1,5]$ shapes with random start positions, sizes up to $s_{max}$, speed $v$, and movement angle, ShapeAug preserves temporal coherence while injecting realistic occlusion dynamics. Evaluations on multiple DVS datasets and the Gen1 automotive dataset show consistent improvements in top-1 accuracy (up to $6.5\%$) and pedestrian AP (over $5\%$), demonstrating robustness to occlusion and compatibility with other augmentations. The results highlight ShapeAug’s practical value for robust event-based classification and detection in dynamic driving scenarios, while pointing to future work in more complex shapes and motion patterns to further bridge the gap to real-world scenes.

Abstract

Recently, Dynamic Vision Sensors (DVSs) sparked a lot of interest due to their inherent advantages over conventional RGB cameras. These advantages include a low latency, a high dynamic range and a low energy consumption. Nevertheless, the processing of DVS data using Deep Learning (DL) methods remains a challenge, particularly since the availability of event training data is still limited. This leads to a need for event data augmentation techniques in order to improve accuracy as well as to avoid over-fitting on the training data. Another challenge especially in real world automotive applications is occlusion, meaning one object is hindering the view onto the object behind it. In this paper, we present a novel event data augmentation approach, which addresses this problem by introducing synthetic events for randomly moving objects in a scene. We test our method on multiple DVS classification datasets, resulting in an relative improvement of up to 6.5 % in top1-accuracy. Moreover, we apply our augmentation technique on the real world Gen1 Automotive Event Dataset for object detection, where we especially improve the detection of pedestrians by up to 5 %.

ShapeAug: Occlusion Augmentation for Event Camera Data

TL;DR

This work tackles occlusion in event-camera data by introducing ShapeAug, an occlusion-aware augmentation that simulates moving foreground shapes to generate both occlusion and the associated events across temporal slices . By sampling shapes with random start positions, sizes up to , speed , and movement angle, ShapeAug preserves temporal coherence while injecting realistic occlusion dynamics. Evaluations on multiple DVS datasets and the Gen1 automotive dataset show consistent improvements in top-1 accuracy (up to ) and pedestrian AP (over ), demonstrating robustness to occlusion and compatibility with other augmentations. The results highlight ShapeAug’s practical value for robust event-based classification and detection in dynamic driving scenarios, while pointing to future work in more complex shapes and motion patterns to further bridge the gap to real-world scenes.

Abstract

Recently, Dynamic Vision Sensors (DVSs) sparked a lot of interest due to their inherent advantages over conventional RGB cameras. These advantages include a low latency, a high dynamic range and a low energy consumption. Nevertheless, the processing of DVS data using Deep Learning (DL) methods remains a challenge, particularly since the availability of event training data is still limited. This leads to a need for event data augmentation techniques in order to improve accuracy as well as to avoid over-fitting on the training data. Another challenge especially in real world automotive applications is occlusion, meaning one object is hindering the view onto the object behind it. In this paper, we present a novel event data augmentation approach, which addresses this problem by introducing synthetic events for randomly moving objects in a scene. We test our method on multiple DVS classification datasets, resulting in an relative improvement of up to 6.5 % in top1-accuracy. Moreover, we apply our augmentation technique on the real world Gen1 Automotive Event Dataset for object detection, where we especially improve the detection of pedestrians by up to 5 %.
Paper Structure (18 sections, 3 equations, 2 figures, 4 tables)

This paper contains 18 sections, 3 equations, 2 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Occlusion-aware RGB Image Augmentation
  4. Event Data Augmentation
  5. METHOD
  6. Event Data Handling
  7. Shape Augmentation
  8. EXPERIMENTS AND RESULTS
  9. Datasets
  10. Implementation
  11. Event Data Classification
  12. Comparison with Existing Literature on Robustness
  13. Comparison to State-of-the-Art.
  14. Robustness.
  15. Combination of Methods.
  16. ...and 3 more sections

Figures (2)

  • Figure 1: Visualization of the shape parameters (position, size and direction) that are randomly chosen for each simulated object (\ref{['fig:shape1']}). The objects move between timesteps and are used to simulate the events that their movement would cause (\ref{['fig:shape2']}).
  • Figure 2: ShapeAug pipeline example for DVS-Gesturedvsgesture.