Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Neural Network Mode for PX4 on Embedded Flight Controllers

Sindre M. Hegre, Welf Rehberg, Mihir Kulkarni, Kostas Alexis

TL;DR

The paper tackles the challenge of deploying neural network controllers on resource-constrained embedded flight controllers by implementing an onboard neural-control module within the PX4 stack, enabled by TensorFlow Lite Micro. Policies trained end-to-end in the Aerial Gym Simulator are converted to a TFLM-compatible format, embedded as a C-array, and used to replace the traditional control cascade, yielding a direct position-to-motor RPM control path. The authors demonstrate a real-world, sim-to-real pipeline on a 1.2 kg quadcopter with onboard inference achieving latency in the tens of microseconds range and a full control loop under $\sim$ $138$ μs, while maintaining performance comparable to simulation. The work provides an open-source, flight-ready framework that lowers the barrier to testing learning-based control on standard embedded flight controllers, accelerating research and potential practical applications. Key contributions include the neural control module within PX4, the end-to-end onboard policy deployment, and demonstrated sim-to-real transfer under realistic hardware constraints.

Abstract

This paper contributes an open-sourced implementation of a neural-network based controller framework within the PX4 stack. We develop a custom module for inference on the microcontroller while retaining all of the functionality of the PX4 autopilot. Policies trained in the Aerial Gym Simulator are converted to the TensorFlow Lite format and then built together with PX4 and flashed to the flight controller. The policies substitute the control-cascade within PX4 to offer an end-to-end position-setpoint tracking controller directly providing normalized motor RPM setpoints. Experiments conducted in simulation and the real-world show similar tracking performance. We thus provide a flight-ready pipeline for testing neural control policies in the real world. The pipeline simplifies the deployment of neural networks on embedded flight controller hardware thereby accelerating research on learning-based control. Both the Aerial Gym Simulator and the PX4 module are open-sourced at https://github.com/ntnu-arl/aerial_gym_simulator and https://github.com/SindreMHegre/PX4-Autopilot-public/tree/for_paper. Video: https://youtu.be/lY1OKz_UOqM?si=VtzL243BAY3lblTJ.

A Neural Network Mode for PX4 on Embedded Flight Controllers

TL;DR

The paper tackles the challenge of deploying neural network controllers on resource-constrained embedded flight controllers by implementing an onboard neural-control module within the PX4 stack, enabled by TensorFlow Lite Micro. Policies trained end-to-end in the Aerial Gym Simulator are converted to a TFLM-compatible format, embedded as a C-array, and used to replace the traditional control cascade, yielding a direct position-to-motor RPM control path. The authors demonstrate a real-world, sim-to-real pipeline on a 1.2 kg quadcopter with onboard inference achieving latency in the tens of microseconds range and a full control loop under μs, while maintaining performance comparable to simulation. The work provides an open-source, flight-ready framework that lowers the barrier to testing learning-based control on standard embedded flight controllers, accelerating research and potential practical applications. Key contributions include the neural control module within PX4, the end-to-end onboard policy deployment, and demonstrated sim-to-real transfer under realistic hardware constraints.

Abstract

This paper contributes an open-sourced implementation of a neural-network based controller framework within the PX4 stack. We develop a custom module for inference on the microcontroller while retaining all of the functionality of the PX4 autopilot. Policies trained in the Aerial Gym Simulator are converted to the TensorFlow Lite format and then built together with PX4 and flashed to the flight controller. The policies substitute the control-cascade within PX4 to offer an end-to-end position-setpoint tracking controller directly providing normalized motor RPM setpoints. Experiments conducted in simulation and the real-world show similar tracking performance. We thus provide a flight-ready pipeline for testing neural control policies in the real world. The pipeline simplifies the deployment of neural networks on embedded flight controller hardware thereby accelerating research on learning-based control. Both the Aerial Gym Simulator and the PX4 module are open-sourced at https://github.com/ntnu-arl/aerial_gym_simulator and https://github.com/SindreMHegre/PX4-Autopilot-public/tree/for_paper. Video: https://youtu.be/lY1OKz_UOqM?si=VtzL243BAY3lblTJ.
Paper Structure (18 sections, 3 figures)

This paper contains 18 sections, 3 figures.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. METHODOLOGY
  4. Policy Training
  5. Aerial Gym Simulator
  6. System Identification
  7. RL Policy Training
  8. Deployment
  9. PX4
  10. TensorFlow Lite Micro
  11. From the Aerial Gym Simulator to PX4
  12. Neural Control Module in PX4
  13. EXPERIMENTS
  14. Experimental setup
  15. Results
  16. ...and 3 more sections

Figures (3)

  • Figure 1: The Learning-based Micro Flyer (LMF) in flight with the neural controller.
  • Figure 2: Left: The Learning-based Micro Flyer platform. Right: The PX4 control cascade. The NN in the Neural Control mode replaces the classical controllers as well as control allocation, which is called the mixer in the PX4 documentation. Diagram was taken from PX4 documentation px4_webpage.
  • Figure 3: Left: Performance of the neural network in simulation. Right: Performance of the neural network on the real platform.