Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FEDORA: Flying Event Dataset fOr Reactive behAvior

Amogh Joshi, Adarsh Kosta, Wachirawit Ponghiran, Manish Nagaraj, Kaushik Roy

TL;DR

FEDORA tackles the lack of high-fidelity, high-rate training data for flying tasks by presenting a fully synthetic dataset that combines frame, event, and IMU data with depth, ego-pose, and optical flow ground truths at rates exceeding existing datasets. Generated in a Gazebo-based simulation with PX4/ROS integration, FEDORA provides multi-rate optical flow (10/25/50 Hz) and millimeter-accurate depth across varied environments, enabling training of end-to-end perception pipelines for latency-critical flight. The work includes disparity, optical flow, and ego-pose experiments that show higher-rate ground truth improves learning and highlights domain-transfer characteristics between FEDORA and driving-era datasets. By offering a single, high-fidelity package, FEDORA aims to accelerate development of real-time, robust perception systems for autonomous aerial navigation and safer drone operations.

Abstract

The ability of resource-constrained biological systems such as fruitflies to perform complex and high-speed maneuvers in cluttered environments has been one of the prime sources of inspiration for developing vision-based autonomous systems. To emulate this capability, the perception pipeline of such systems must integrate information cues from tasks including optical flow and depth estimation, object detection and tracking, and segmentation, among others. However, the conventional approach of employing slow, synchronous inputs from standard frame-based cameras constrains these perception capabilities, particularly during high-speed maneuvers. Recently, event-based sensors have emerged as low latency and low energy alternatives to standard frame-based cameras for capturing high-speed motion, effectively speeding up perception and hence navigation. For coherence, all the perception tasks must be trained on the same input data. However, present-day datasets are curated mainly for a single or a handful of tasks and are limited in the rate of the provided ground truths. To address these limitations, we present Flying Event Dataset fOr Reactive behAviour (FEDORA) - a fully synthetic dataset for perception tasks, with raw data from frame-based cameras, event-based cameras, and Inertial Measurement Units (IMU), along with ground truths for depth, pose, and optical flow at a rate much higher than existing datasets.

FEDORA: Flying Event Dataset fOr Reactive behAvior

TL;DR

FEDORA tackles the lack of high-fidelity, high-rate training data for flying tasks by presenting a fully synthetic dataset that combines frame, event, and IMU data with depth, ego-pose, and optical flow ground truths at rates exceeding existing datasets. Generated in a Gazebo-based simulation with PX4/ROS integration, FEDORA provides multi-rate optical flow (10/25/50 Hz) and millimeter-accurate depth across varied environments, enabling training of end-to-end perception pipelines for latency-critical flight. The work includes disparity, optical flow, and ego-pose experiments that show higher-rate ground truth improves learning and highlights domain-transfer characteristics between FEDORA and driving-era datasets. By offering a single, high-fidelity package, FEDORA aims to accelerate development of real-time, robust perception systems for autonomous aerial navigation and safer drone operations.

Abstract

The ability of resource-constrained biological systems such as fruitflies to perform complex and high-speed maneuvers in cluttered environments has been one of the prime sources of inspiration for developing vision-based autonomous systems. To emulate this capability, the perception pipeline of such systems must integrate information cues from tasks including optical flow and depth estimation, object detection and tracking, and segmentation, among others. However, the conventional approach of employing slow, synchronous inputs from standard frame-based cameras constrains these perception capabilities, particularly during high-speed maneuvers. Recently, event-based sensors have emerged as low latency and low energy alternatives to standard frame-based cameras for capturing high-speed motion, effectively speeding up perception and hence navigation. For coherence, all the perception tasks must be trained on the same input data. However, present-day datasets are curated mainly for a single or a handful of tasks and are limited in the rate of the provided ground truths. To address these limitations, we present Flying Event Dataset fOr Reactive behAviour (FEDORA) - a fully synthetic dataset for perception tasks, with raw data from frame-based cameras, event-based cameras, and Inertial Measurement Units (IMU), along with ground truths for depth, pose, and optical flow at a rate much higher than existing datasets.
Paper Structure (12 sections, 10 equations, 8 figures, 7 tables)

This paper contains 12 sections, 10 equations, 8 figures, 7 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. DATASET
  4. Sequences
  5. Data Format
  6. OPTICAL FLOW GENERATION
  7. DISCUSSION
  8. EXPERIMENTS
  9. Disparity Experiments
  10. Optical Flow Experiments
  11. Ego-Pose Experiments
  12. CONCLUSION

Figures (8)

  • Figure 1: Worflow of the Dataset Generation Simulator
  • Figure 2: Simulated Quadcopter instrumented with Imaging and Inertial Sensors. The Body frame of reference of the quadcopter is shown in the bottom-left corner
  • Figure 3: Sample RGB frame from sequence Baylands-Day2
  • Figure 4: Sample RGB frame from sequence Racetrack-Night
  • Figure 5: Example RGB and Depth data from the Baylands-Day2 sequence along with the corresponding event and optical flow renders
  • ...and 3 more figures