M-SEVIQ: A Multi-band Stereo Event Visual-Inertial Quadruped-based Dataset for Perception under Rapid Motion and Challenging Illumination

TL;DR

M-SEVIQ addresses perception under rapid motion and challenging illumination for legged robots by linking stereo event cameras with RGB-D, IMU, RTK, and joint encoders on a quadruped. The dataset delivers more than $30$ sequences across indoor and outdoor settings with thorough intrinsic, extrinsic, and temporal calibration to enable precise sensor fusion and benchmarking for perception tasks including semantic segmentation. The authors compare SAM and LISA on event-stream segmentation, illustrating the potential of combining high-temporal-resolution events with language-guided segmentation in extreme conditions. By filling gaps in stereo-event data for quadrupeds and multi-modal sensing, M-SEVIQ offers a practical platform for advancing fast robotics perception and sensor fusion research, with future expansion plans for broader coverage and autonomous calibration.

Abstract

Agile locomotion in legged robots poses significant challenges for visual perception. Traditional frame-based cameras often fail in these scenarios for producing blurred images, particularly under low-light conditions. In contrast, event cameras capture changes in brightness asynchronously, offering low latency, high temporal resolution, and high dynamic range. These advantages make them suitable for robust perception during rapid motion and under challenging illumination. However, existing event camera datasets exhibit limitations in stereo configurations and multi-band sensing domains under various illumination conditions. To address this gap, we present M-SEVIQ, a multi-band stereo event visual and inertial quadruped dataset collected using a Unitree Go2 equipped with stereo event cameras, a frame-based camera, an inertial measurement unit (IMU), and joint encoders. This dataset contains more than 30 real-world sequences captured across different velocity levels, illumination wavelengths, and lighting conditions. In addition, comprehensive calibration data, including intrinsic, extrinsic, and temporal alignments, are provided to facilitate accurate sensor fusion and benchmarking. Our M-SEVIQ can be used to support research in agile robot perception, sensor fusion, semantic segmentation and multi-modal vision in challenging environments.

Figures (6)

• Figure 1: Our Quadruped robot for data collection. (a)Side view of the robot (b)Top view of the robot
• Figure 2: (a) Acceleration and velocity from decoder (b) Acceleration from imu
• Figure 3: The dataset is systematically organized in a hierarchical manner to facilitate extensive benchmarking. It encompasses a wide range of scenarios, captured using a suite of event-centric sensors. This multi-sensor data provides the foundation for evaluating performance on several perception and navigation tasks. The primary utility of the dataset is demonstrated by benchmarking a set of representative algorithms, establishing a baseline for future research.
• Figure 4: (a) warningpost_slow_night; (b) lawn_slow_night;(c) bikeparkinglot_slow_night; (d) building_fast_day; (e) street_slow_day
• Figure 5: Segment the picture automatically