FlyAware: Inertia-Aware Aerial Manipulation via Vision-Based Estimation and Post-Grasp Adaptation

TL;DR

FlyAware tackles time-varying inertial parameters in aerial manipulators by pairing a vision-language–driven pre-sensing stage with a disturbance-observer–based post-grasp adaptation stage. The method yields real-time updates to object mass $\hat{m}_o$ and inertia $\hat{\mathbf{J}}_o$, which feed into a system inertia $\hat{\mathbf{J}}_t$ used by an inertia-aware gain-scheduling controller. Key contributions include a two-stage inertia estimation framework, an inertia-compensated adaptive control strategy with $\mathbf{K}_k^{rate} = \mathbf{J}_a^{-1} \hat{\mathbf{J}}_t$, and a frequency-domain robustness assessment demonstrating stability under payloads up to 400 g. Real-world experiments show sub-3% inertia estimation error within ~2 s and notable improvements in position and attitude tracking during manipulation and transport, indicating strong practical impact for robust aerial manipulation in dynamic environments.

Abstract

Aerial manipulators (AMs) are gaining increasing attention in automated transportation and emergency services due to their superior dexterity compared to conventional multirotor drones. However, their practical deployment is challenged by the complexity of time-varying inertial parameters, which are highly sensitive to payload variations and manipulator configurations. Inspired by human strategies for interacting with unknown objects, this letter presents a novel onboard framework for robust aerial manipulation. The proposed system integrates a vision-based pre-grasp inertia estimation module with a post-grasp adaptation mechanism, enabling real-time estimation and adaptation of inertial dynamics. For control, we develop an inertia-aware adaptive control strategy based on gain scheduling, and assess its robustness via frequency-domain system identification. Our study provides new insights into post-grasp control for AMs, and real-world experiments validate the effectiveness and feasibility of the proposed framework.

Figures (10)

• Figure 1: Real-world experiment: The aerial manipulator pre-senses and grasps objects arbitrarily placed on a table by a human operator.
• Figure 2: System overview. Our system processes two inputs: an RGB-D image stream of the scene and text specifying the target object. Stage 1: Pre-Sensing: Before grasping, the unknown target object's inertial parameters (mass ${\tilde{m}}_o$ and MoI $\tilde{\mathbf{J}}_o$) are pre-estimated by large vision and language models, and the object frame origin $\mathcal{O}_O$ is also obtained. Stage 2: Adaptation: Upon grasping, the Perception Adaptation module updates the object's mass and MoI to ${\hat{m}}_o$ and $\hat{\mathbf{J}}_o$ using onboard force sensor feedback, then computes the system's total MoI $\hat{\mathbf{J}}_t$. The Control Adaptation module's inertia-aware gain scheduling (IAGS) controller then calculates the inertia-aware gain, generating desired thrust magnitude $T^{\mathrm{des}}$ and torque $\bm{\tau}^{\mathrm{des}}$.
• Figure 3: Pose estimation results in the camera coordinate frame.
• Figure 4: System prompt for object scale factor and density estimation.
• Figure 5: Inertia-compensated adaptive strategy. (A) Quadrotor Control Structure: Comprises a position controller, an attitude controller, and a mixer. (B) Angular Rate Control Loop: The PID control gains $\mathbf{K}_k^{\text{rate}}$ are dynamically adjusted through Inertia-Aware Gain Scheduling (IAGS) based on the manipulator joint angles $\bm{\theta}$.