NanoCockpit: Performance-optimized Application Framework for AI-based Autonomous Nanorobotics

TL;DR

NanoCockpit addresses the bottleneck of inefficient, serialized task execution on resource-constrained nano-drones by introducing coroutine-based multitasking, high-throughput camera drivers, and a zero-copy Wi-Fi stack that enable tight overlap between image acquisition, TinyML inference, and data exchange across multiple MCUs. Implemented on the Bitcraze Crazyflie with the GAP8 and ESP32, the framework achieves ideal end-to-end latency and demonstrably improves closed-loop control across three real-world NanoCockpit applications. Key contributions include the coroutine-based cooperative scheduling layer, multi-buffer camera pipelines up to 150 fps, zero-copy inter-device streaming, and integration with DORY/PULP-NN for onboard inference. The open-source release provides practical tools for deploying ultra-lightweight perception and control on nano-drones, accelerating embedded robotics research and development.

Abstract

Autonomous nano-drones, powered by vision-based tiny machine learning (TinyML) models, are a novel technology gaining momentum thanks to their broad applicability and pushing scientific advancement on resource-limited embedded systems. Their small form factor, i.e., a few 10s grams, severely limits their onboard computational resources to sub-\SI{100}{\milli\watt} microcontroller units (MCUs). The Bitcraze Crazyflie nano-drone is the \textit{de facto} standard, offering a rich set of programmable MCUs for low-level control, multi-core processing, and radio transmission. However, roboticists very often underutilize these onboard precious resources due to the absence of a simple yet efficient software layer capable of time-optimal pipelining of multi-buffer image acquisition, multi-core computation, intra-MCUs data exchange, and Wi-Fi streaming, leading to sub-optimal control performances. Our \textit{NanoCockpit} framework aims to fill this gap, increasing the throughput and minimizing the system's latency, while simplifying the developer experience through coroutine-based multi-tasking. In-field experiments on three real-world TinyML nanorobotics applications show our framework achieves ideal end-to-end latency, i.e. zero overhead due to serialized tasks, delivering quantifiable improvements in closed-loop control performance ($-$30\% mean position error, mission success rate increased from 40\% to 100\%).

Figures (8)

• Figure 1: Scientific publications on nano-quadrotors over the last five years, according to the Elsevier Scopus citation database.
• Figure 2: Hardware (A) and software (B) overviews of the Crazyflie nano-drone with the AI-deck companion board and a remote computer.
• Figure 3: (A) Serialized execution without our NanoCockpit framework vs. (B) pipelined (double-buffered) execution with our framework.
• Figure 4: Closed-loop control performance in the A) drone-to-drone localization and B) nano-drone racing obstacle avoidance experiments.
• Figure 5: Co-routine-based cooperative multi-tasking. A) C API example of our co-routine implementation. B) Execution flow with interactions among tasks and event loop. C) Co-routine state machine with start, running, and end states and the three suspension points. D) Internal implementation of co-routine structures.