Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning

Shlok Deshmukh, Javier Alonso-Mora, Sihao Sun

TL;DR

The paper tackles robust end-effector pose control for a minimal underactuated aerial manipulator (DSAM) by marrying a PPO-based outer-loop policy with reliable inner-loop INDI/PID controllers. Training in a highly randomized simulation environment with domain randomization enables sim-to-real transfer, and onboard inference demonstrates real-world feasibility. Real experiments show centimeter-scale position accuracy and degree-level orientation under disturbances, including payloads and pushing tasks, with ablations highlighting critical design choices for observation space, inner-loop control, and randomness. This work establishes the viability of learning-based whole-body control on lightweight aerial platforms and points toward more capable, contact-rich aerial manipulation in the future.

Abstract

Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity. In this work, we study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control. While the minimal design enables simplicity and reduced payload, it also introduces challenges such as underactuation and sensitivity to external disturbances, including manipulation of heavy loads and pushing tasks. To address these, we employ reinforcement learning, training a Proximal Policy Optimization (PPO) agent in simulation to generate feedforward commands for quadrotor acceleration and body rates, along with joint angle targets. These commands are tracked by an incremental nonlinear dynamic inversion (INDI) attitude controller and a PID joint controller, respectively. Flight experiments demonstrate centimeter-level position accuracy and degree-level orientation precision, with robust performance under external force disturbances. The results highlight the potential of learning-based control strategies for enabling contact-rich aerial manipulation using simple, lightweight platforms.

Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning

TL;DR

The paper tackles robust end-effector pose control for a minimal underactuated aerial manipulator (DSAM) by marrying a PPO-based outer-loop policy with reliable inner-loop INDI/PID controllers. Training in a highly randomized simulation environment with domain randomization enables sim-to-real transfer, and onboard inference demonstrates real-world feasibility. Real experiments show centimeter-scale position accuracy and degree-level orientation under disturbances, including payloads and pushing tasks, with ablations highlighting critical design choices for observation space, inner-loop control, and randomness. This work establishes the viability of learning-based whole-body control on lightweight aerial platforms and points toward more capable, contact-rich aerial manipulation in the future.

Abstract

Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity. In this work, we study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control. While the minimal design enables simplicity and reduced payload, it also introduces challenges such as underactuation and sensitivity to external disturbances, including manipulation of heavy loads and pushing tasks. To address these, we employ reinforcement learning, training a Proximal Policy Optimization (PPO) agent in simulation to generate feedforward commands for quadrotor acceleration and body rates, along with joint angle targets. These commands are tracked by an incremental nonlinear dynamic inversion (INDI) attitude controller and a PID joint controller, respectively. Flight experiments demonstrate centimeter-level position accuracy and degree-level orientation precision, with robust performance under external force disturbances. The results highlight the potential of learning-based control strategies for enabling contact-rich aerial manipulation using simple, lightweight platforms.
Paper Structure (30 sections, 11 equations, 6 figures, 3 tables)

This paper contains 30 sections, 11 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Design
  4. Planning and Control
  5. Preliminaries
  6. Methodology
  7. Control architecture
  8. Reinforcement Learning Policy
  9. Inner-loop Quadrotor Attitude Controller
  10. Training of the RL Policy
  11. Training Environment
  12. Rewards
  13. References and Resets
  14. Domain Randomization
  15. Real-World Experiments
  16. ...and 15 more sections

Figures (6)

  • Figure 1: Snapshot of a DSAM aerial manipulator, consisting of a quadrotor base with an arm mounted through a differential mechanism that provides 2-DoF motion.
  • Figure 2: Overview of proposed controller architecture and training methodology.
  • Figure 3: a) End-effector pose control between two randomly sampled goal poses, b) Pose control with a 140 g payload held by the gripper, c) DSAM pushing a box weighing 590 g from right to left.
  • Figure 4: Comparison of pose control performance with and without payload (we show only the first 5 setpoints for readability).
  • Figure 5: 3D trajectory of path followed by end-effector (starting point indicated by green dot)
  • ...and 1 more figures