Non-Prehensile Aerial Manipulation using Model-Based Deep Reinforcement Learning

TL;DR

This work tackles non-prehensile aerial manipulation under unknown environment dynamics by casting the problem as model-based deep reinforcement learning. It adapts the DreamerV3 world-model framework to a flying platform in the Aerial Gym simulation, jointly learning perception, state prediction, and a control policy to push objects toward goal regions while adapting to varying friction. The authors introduce a specialized state, action, and reward design, and employ domain randomization to promote sim-to-real transfer and robustness to contact dynamics. The results demonstrate repeatable push behaviors across a spectrum of friction values and task configurations, indicating potential for reliable hardware deployment and broader operational scenarios where occlusions and clutter are present.

Abstract

With the continual adoption of Uncrewed Aerial Vehicles (UAVs) across a wide-variety of application spaces, robust aerial manipulation remains a key research challenge. Aerial manipulation tasks require interacting with objects in the environment, often without knowing their dynamical properties like mass and friction a priori. Additionally, interacting with these objects can have a significant impact on the control and stability of the vehicle. We investigated an approach for robust control and non-prehensile aerial manipulation in unknown environments. In particular, we use model-based Deep Reinforcement Learning (DRL) to learn a world model of the environment while simultaneously learning a policy for interaction with the environment. We evaluated our approach on a series of push tasks by moving an object between goal locations and demonstrated repeatable behaviors across a range of friction values.

Figures (6)

• Figure 1: Aerial vehicle pushing an object between goal points on alternating ends of the table in simulated environment. Object's trajectory is shown in purple and vehicle's trajectory is shown in blue.
• Figure 2: Aerial manipulation platform with a fixed gripper for non-prehensile and grasping tasks.
• Figure 3: Example aerial manipulation experimental scenario. The vehicle started approaching perpendicular to the scene and a traditional controller required the vehicle to move to the side of the scene to grasp the tomato sauce can. In this work, we explore capabilities to more robustly accomplish this task, for example, in cases where the side may be inaccessible the alphabet soup would need to be pushed in order to reach the target object.
• Figure 4: Algorithmic architecture for model-based deep reinforcement learning in simulated Aerial Gym environments.
• Figure 5: Simulation environment for baseline experiments.