Reinforcement Learning Driven Cooperative Ball Balance in Rigidly Coupled Drones

TL;DR

The paper tackles time-varying center of gravity in multi-drone cooperative transport by proposing a leader-follower scheme where a PID-controlled leader steers the object and a Soft Actor-Critic follower governs the follower drone using local state and minimal leader feedback. The approach leverages a realistic rigid-body dynamic model to train and validate in simulation, and then demonstrates preliminary ball-balancing experiments with two drones on a rod. Compared against an adaptive CG controller, the RL-based follower achieves smoother trajectories, better settling, and resilience to CG speed and payload changes. This work introduces the first known deployment of deep RL (SAC) for decentralized follower control under moving CG in a rigidly coupled two-drone CT system and provides initial experimental validation. The results suggest practical potential for robust, communication-light cooperative transport in uncertain payload scenarios, with future work extending to full flight tests and broader CG dynamics.

Abstract

Multi-drone cooperative transport (CT) problem has been widely studied in the literature. However, limited work exists on control of such systems in the presence of time-varying uncertainties, such as the time-varying center of gravity (CG). This paper presents a leader-follower approach for the control of a multi-drone CT system with time-varying CG. The leader uses a traditional Proportional-Integral-Derivative (PID) controller, and in contrast, the follower uses a deep reinforcement learning (RL) controller using only local information and minimal leader information. Extensive simulation results are presented, showing the effectiveness of the proposed method over a previously developed adaptive controller and for variations in the mass of the objects being transported and CG speeds. Preliminary experimental work also demonstrates ball balance (depicting moving CG) on a stick/rod lifted by two Crazyflie drones cooperatively.

Figures (7)

• Figure 1: Free body diagram of the entire system used for package delivery (showing moving CG).
• Figure 2: Performance of SAC implemented on the follower.
• Figure 3: Position of the geometric center of payload or object being transported by two drones for adaptive and RL controllers implemented on follower drone.
• Figure 4: Position of the geometric center of payload $p_o$ using deep RL follower and PID leader tested for different speeds of CG.
• Figure 5: Position of the geometric center of payload $p_o$ using deep RL follower and PID leader tested for the different mass of the object being transported.