Aerial Manipulation with Contact-Aware Onboard Perception and Hybrid Control

TL;DR

This work tackles aerial manipulation tasks requiring physical contact without relying on external motion capture. It introduces a fully onboard perception–control pipeline consisting of a contact-aware VIO, image-based visual servoing, and a hybrid force–motion controller to regulate contact wrenches and lateral motion. The contact factor enhances state estimation during interaction, IBVS guides approach using onboard velocity, and the hybrid controller enables stable normal-force regulation with minimal disturbance to lateral movement. Real-world and simulation experiments demonstrate a 66.01% reduction in velocity drift at contact and robust force holding, illustrating the approach's potential for deployable, MoCap-denied aerial manipulation in wild environments.

Abstract

Aerial manipulation (AM) promises to move Unmanned Aerial Vehicles (UAVs) beyond passive inspection to contact-rich tasks such as grasping, assembly, and in-situ maintenance. Most prior AM demonstrations rely on external motion capture (MoCap) and emphasize position control for coarse interactions, limiting deployability. We present a fully onboard perception-control pipeline for contact-rich AM that achieves accurate motion tracking and regulated contact wrenches without MoCap. The main components are (1) an augmented visual-inertial odometry (VIO) estimator with contact-consistency factors that activate only during interaction, tightening uncertainty around the contact frame and reducing drift, and (2) image-based visual servoing (IBVS) to mitigate perception-control coupling, together with a hybrid force-motion controller that regulates contact wrenches and lateral motion for stable contact. Experiments show that our approach closes the perception-to-wrench loop using only onboard sensing, yielding an velocity estimation improvement of 66.01% at contact, reliable target approach, and stable force holding-pointing toward deployable, in-the-wild aerial manipulation.

Figures (7)

• Figure 1: Our fully-actuated aerial manipulator performs contact inspection using solely onboard sensing for motion tracking and force regulation. The inset shows the camera-based feature detection of our contact-aware visual inertial odometry.
• Figure 2: Overview of the proposed pipeline. A contact-aware VIO module fuses visual, inertial, and force/torque measurements to provide state estimates and activates contact factors during interaction. An image-based visual servo (IBVS) drives the vehicle during the approach phase using VIO-estimated body velocity. Upon contact, a hybrid motion–wrench control strategy regulates contact forces and tracks lateral motion. Finally, a control allocation module maps the commanded wrenches into rotor speeds $\boldsymbol{\Omega}$ for execution on the fully-actuated hexarotor platform.
• Figure 3: Pose graph of the proposed contact-aware VIO
• Figure 4: Snapshots of real-world experiments. (a) Visual servoing initialized. (b) Approaching the target. (c) Force holding.
• Figure 5: Peg-in-hole experiment in simulation. (a) a snapshot of the contact moment. The first person view (FPV) is in the lower right corner. (b) Alignment error converted from the pixel plane. (c) Scaling error in pixel plane. (d) Filtered force measurement.