NanoBench: A Multi-Task Benchmark Dataset for Nano-Quadrotor System Identification, Control, and State Estimation

TL;DR

NanoBench is introduced, an open-source multi-task benchmark collected on the commercially available Crazyflie 2.1 nano-quadrotor in a Vicon motion capture arena, and is the first public dataset to jointly provide actuator commands, controller internals, and estimator outputs with millimeter-accurate ground truth on a commercially available nano-scale aerial platform.

Abstract

Existing aerial-robotics benchmarks target vehicles from hundreds of grams to several kilograms and typically expose only high-level state data. They omit the actuator-level signals required to study nano-scale quadrotors, where low-Reynolds number aerodynamics, coreless DC motor nonlinearities, and severe computational constraints invalidate models and controllers developed for larger vehicles. We introduce NanoBench, an open-source multi-task benchmark collected on the commercially available Crazyflie 2.1 nano-quadrotor (takeoff weight 27 g) in a Vicon motion capture arena. The dataset contains over 170 flight trajectories spanning hover, multi-frequency excitation, standard tracking, and aggressive maneuvers across multiple speed regimes. Each trajectory provides synchronized Vicon ground truth, raw IMU data, onboard extended Kalman filter estimates, PID controller internals, and motor PWM commands at 100 Hz, alongside battery telemetry at 10 Hz, aligned with sub-0.5 ms consistency. NanoBench defines standardized evaluation protocols, train/test splits, and open-source baselines for three tasks: nonlinear system identification, closed-loop controller benchmarking, and onboard state estimation assessment. To our knowledge, it is the first public dataset to jointly provide actuator commands, controller internals, and estimator outputs with millimeter-accurate ground truth on a commercially available nano-scale aerial platform.

Figures (6)

• Figure 1: All 12 NanoBench trajectory types visualized from recorded Vicon ground-truth data. Categories A (excitation), B (geometric tracking), and C (battery-drain hover).
• Figure 2: Task 1: Position MAE grows monotonically with prediction horizon for all models. The physics model dominates at $h=1$ but diverges beyond $h=15$ steps. The hybrid model achieves the lowest cumulative error.
• Figure 3: Task 1: Predicted (colored) vs. Vicon ground-truth (black) trajectories for each baseline on a held-out Category B (Helix trajectory) sequence, along each axis.
• Figure 4: Task 2: Closed-loop position (xyz, m) for each baseline vs. Ground Truth. Real-flight (PID, Mellinger) and offline (BC-MLP, BC-LSTM, MPPI) trajectories.
• Figure 5: Task 2: Motor PWM outputs over time for each controller baseline (normalized).