FLARE: Agile Flights for Quadrotor Cable-Suspended Payload System via Reinforcement Learning

TL;DR

FLARE tackles the challenging problem of agile flight for a quadrotor with a cable-suspended payload, which exhibits strong nonlinear and hybrid dynamics due to slack and taut cable modes. It proposes a model-free reinforcement learning approach trained in a high-fidelity Genesis simulator to learn a direct state-to-action policy that handles the coupled dynamics without explicit mode switching. The method achieves zero-shot sim-to-real transfer and real-time onboard execution, and reports a 3x speedup over state-of-the-art optimization during gate traversal along with robust agility across three challenging tasks. The work demonstrates that model-free RL can substantially enhance agility and onboard computational efficiency for suspended payload systems, with potential extension to dynamic environments.

Abstract

Agile flight for the quadrotor cable-suspended payload system is a formidable challenge due to its underactuated, highly nonlinear, and hybrid dynamics. Traditional optimization-based methods often struggle with high computational costs and the complexities of cable mode transitions, limiting their real-time applicability and maneuverability exploitation. In this letter, we present FLARE, a reinforcement learning (RL) framework that directly learns agile navigation policy from high-fidelity simulation. Our method is validated across three designed challenging scenarios, notably outperforming a state-of-the-art optimization-based approach by a 3x speedup during gate traversal maneuvers. Furthermore, the learned policies achieve successful zero-shot sim-to-real transfer, demonstrating remarkable agility and safety in real-world experiments, running in real time on an onboard computer.

Figures (10)

• Figure 1: Autonomous Agile Flights of a Quadrotor Cable-suspended Payload System via Reinforcement Learning Policy. The system demonstrates its capabilities in three challenging real-world scenarios: (a) agile waypoint passing, (b) payload targeting, and (c1, c2) one-time and sequential gate traversal.
• Figure 2: Overview of our proposed approach. We design three representative application scenarios focused on motion planning of cable-suspended payload system, each with a unique goal. In each scenario, a control policy is learned with customized observations and reward setting in Genesis simulator Genesis.
• Figure 3: Definition and formulation of the safety reward.
• Figure 4: Simulation results of scenario 1.
• Figure 5: Robustness analysis regarding parameter variations in scenario 1. Success rates and average completion times are compared under varying payload masses (top) and cable lengths (bottom).