BikeActions: An Open Platform and Benchmark for Cyclist-Centric VRU Action Recognition

TL;DR

This work introduces FUSE-Bike, the first fully open perception platform of its kind and presents BikeActions, a novel multi-modal dataset comprising 852 annotated samples across 5 distinct action classes, specifically tailored to improve VRU behavior modeling.

Abstract

Anticipating the intentions of Vulnerable Road Users (VRUs) is a critical challenge for safe autonomous driving (AD) and mobile robotics. While current research predominantly focuses on pedestrian crossing behaviors from a vehicle's perspective, interactions within dense shared spaces remain underexplored. To bridge this gap, we introduce FUSE-Bike, the first fully open perception platform of its kind. Equipped with two LiDARs, a camera, and GNSS, it facilitates high-fidelity, close-range data capture directly from a cyclist's viewpoint. Leveraging this platform, we present BikeActions, a novel multi-modal dataset comprising 852 annotated samples across 5 distinct action classes, specifically tailored to improve VRU behavior modeling. We establish a rigorous benchmark by evaluating state-of-the-art graph convolution and transformer-based models on our publicly released data splits, establishing the first performance baselines for this challenging task. We release the full dataset together with data curation tools, the open hardware design, and the benchmark code to foster future research in VRU action understanding under https://iv.ee.hm.edu/bikeactions/.

Figures (6)

• Figure 1: Qualitative examples from the BikeActions dataset recorded with the FUSE-bike platform. The top row shows RGB camera views with projected 2D skeleton overlays for three distinct urban scenarios (hand sign in adverse lightning conditions, pedestrian crossing, narrow bicycle lane). The bottom row displays the corresponding sparse depth images from the long-range LiDAR, colorized to represent depth on a 0-50m scale.
• Figure 2: The FUSE-Bike hardware prototype and its corresponding CAD model. The design features a rigid front sensor mount with stacked LiDARs and a camera, with the main electronics housed in a rear-mounted case.
• Figure 3: System architecture of the FUSE-Bike, showing the data flow between the Jetson compute unit, the PTP-synchronized sensors, and other key electronics.
• Figure 4: Statistics of the BikeActions dataset. (\ref{['fig:dataset_stats_frames']}) total number of raw frames per sequence; and (\ref{['fig:dataset_stats_length']}) Histogram showing the distribution of action sample duration.
• Figure 5: Qualitative action recognition samples. Each row displays the sequence (left) alongside the corresponding class label (right). The skeleton is color-coded: right side in blue, left side in green, and central joints in orange.