CEAR: Comprehensive Event Camera Dataset for Rapid Perception of Agile Quadruped Robots

TL;DR

CEAR tackles the perception challenge for agile quadrupeds where motion blur and harsh lighting degrade RGB-only perception. It introduces a multimodal dataset collected on the Mini Cheetah that fuses event cameras with RGB-D, LiDAR, IMU, and joint encoders, covering 106 sequences in indoor/outdoor settings, including backflips, with $6$ DoF ground-truth poses. The work details sensor calibration and a post-hoc temporal synchronization approach, and benchmarks several SLAM pipelines (event- and frame-based) using metrics $ATE$ and $RPE$ to reveal the robustness of event-based methods under dynamic gaits and lighting variations. By enabling rapid perception research for legged robots, CEAR is positioned as a practical benchmark to drive multimodal fusion and robust state estimation in hazardous, time-sensitive tasks.

Abstract

When legged robots perform agile movements, traditional RGB cameras often produce blurred images, posing a challenge for rapid perception. Event cameras have emerged as a promising solution for capturing rapid perception and coping with challenging lighting conditions thanks to their low latency, high temporal resolution, and high dynamic range. However, integrating event cameras into agile-legged robots is still largely unexplored. Notably, no dataset including event cameras has yet been developed for the context of agile quadruped robots. To bridge this gap, we introduce CEAR, a dataset comprising data from an event camera, an RGB-D camera, an IMU, a LiDAR, and joint encoders, all mounted on a dynamic quadruped, Mini Cheetah robot. This comprehensive dataset features more than 100 sequences from real-world environments, encompassing various indoor and outdoor environments, different lighting conditions, a range of robot gaits (e.g., trotting, bounding, pronking), as well as acrobatic movements like backflip. To our knowledge, this is the first event camera dataset capturing the dynamic and diverse quadruped robot motions under various setups, developed to advance research in rapid perception for quadruped robots.

Figures (7)

• Figure 1: An overview of the dataset under various gaits. The top row illustrates three different gaits and a backflip motion. The subsequent rows show different sensor data corresponding to each gait. Ratios of images are adjusted to fit into the figure.
• Figure 2: Configuration of the sensors. All sensors are rigidly mounted on top of the Mini Cheetah robot. The coordinate systems for the sensors, the robot's body $\mathbb{B}$, and the global frame $\mathbb{G}$ are indicated by the red, green, and blue axes, representing the $x$, $y$, and $z$ directions, respectively.
• Figure 3: Time synchronization across different sensors. The red vertical line indicates alignment of the RealSense camera and VectorNav IMU, and the event camera's angular velocity, while the orange line shows the alignment of LiDAR, MoCap, and Mini Cheetah's joint angle. For visualization, we magnified the average offset by 10 to distinguish the difference between the original and synchronized data.
• Figure 4: Histograms of forward/vertical accelerations, and pitch angular velocity during different gaits. The histogram depicts the unique features of each gait. Compared to the stable trot gait, the pronking gait exhibits high-dynamic vertical movement, as indicated by high $\ddot{z}$ values. Similarly, the backflip motion shows distinctive high body pitch velocity ($\dot{\theta}_y$) and a wide range of acceleration ($\ddot{x}$, $\ddot{z}$).
• Figure 5: Overview of 10 outdoor environments. (a) is the area around a campus building with a direct sunlight source. (b) and (c) are downtown areas containing dynamic elements like pedestrians, vehicles, and blinking neon lights. (d) is a forest environment that includes slippery ground. (e) is a grassy environment with featureless ground, expansive space, and distant sky. (f) includes two environments, one is a short sidewalk path and the other one is a route between two buildings, highlighting a standard pedestrian space environment. (g) is a parking lot during a cloudy day and a foggy night. (h) is a residential area with complete darkness at night. (i) presents a long sidewalk path, illustrating several pavement types. Event and RGB images are presented with satellite views at each site, showing day and night scenes at the same location, indicated by green circles. The yellow star marks the start/end points, and the red path denotes the trajectory.