Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Robust Control Policies for Inverted Pose on Miniature Blimp Robots

Yuanlin Yang, Lin Hong, Fumin Zhang

TL;DR

A novel framework that enables robust control policy learning for inverted pose on miniature blimp robots and confirms that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully upside-down pose in real-world settings.

Abstract

The ability to achieve and maintain inverted poses is essential for unlocking the full agility of miniature blimp robots (MBRs). However, developing reliable control methods for MBRs remains challenging due to their complex and underactuated dynamics. To address this challenge, we propose a novel framework that enables robust control policy learning for inverted pose on MBRs. The proposed framework operates through three core stages: First, a high-fidelity three-dimensional (3D) simulation environment was constructed, which was calibrated against real-world MBR motion data to ensure accurate replication of inverted-state dynamics. Second, a robust policy for MBR inverted control was trained within the simulation environment via a domain randomization strategy and a modified Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. Third, a mapping layer was designed to bridge the sim-to-real gap for the learned policy deployment. Comprehensive evaluations in the simulation environment demonstrate that the learned policy achieves a higher success rate compared to the energy-shaping controller. Furthermore, experimental results confirm that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully upside-down pose in real-world settings.

Learning Robust Control Policies for Inverted Pose on Miniature Blimp Robots

TL;DR

A novel framework that enables robust control policy learning for inverted pose on miniature blimp robots and confirms that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully upside-down pose in real-world settings.

Abstract

The ability to achieve and maintain inverted poses is essential for unlocking the full agility of miniature blimp robots (MBRs). However, developing reliable control methods for MBRs remains challenging due to their complex and underactuated dynamics. To address this challenge, we propose a novel framework that enables robust control policy learning for inverted pose on MBRs. The proposed framework operates through three core stages: First, a high-fidelity three-dimensional (3D) simulation environment was constructed, which was calibrated against real-world MBR motion data to ensure accurate replication of inverted-state dynamics. Second, a robust policy for MBR inverted control was trained within the simulation environment via a domain randomization strategy and a modified Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. Third, a mapping layer was designed to bridge the sim-to-real gap for the learned policy deployment. Comprehensive evaluations in the simulation environment demonstrate that the learned policy achieves a higher success rate compared to the energy-shaping controller. Furthermore, experimental results confirm that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully upside-down pose in real-world settings.
Paper Structure (17 sections, 11 equations, 13 figures, 7 tables, 2 algorithms)

This paper contains 17 sections, 11 equations, 13 figures, 7 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Problem Formulation
  3. Dynamic Model of MBRs
  4. Inverted Control Problem Statement
  5. Methodology
  6. Simulation Environment
  7. Physics-informed Domain Randomization
  8. TD3 with Multi-buffer and Clipping
  9. Implementation Details
  10. Evaluation and Experiment
  11. Performance of the Trained Policy to $m_w$ Variations
  12. Performance of the Trained Policy to $\lambda$ Variations
  13. Performance of the Trained Policy to $g_m$ Variations
  14. Performance of the Trained Policy to $m_w$, $\lambda$ and $g_m$ Variations
  15. Ablation Study
  16. ...and 2 more sections

Figures (13)

  • Figure 1: Overview of the proposed method for inverted control of MBRs. The MBR with an upright pose can reach and maintain the inverted pose with the learned robust policy.
  • Figure 2: Dynamic model analysis of the MBR: (a) Upright pose; (b) Inverted pose.
  • Figure 3: Pipeline of developing and deploying robust policy for inverted control of MBRs.
  • Figure 4: 3D simulation environment designed for robust policy learning for inverted control of MBRs.
  • Figure 5: The MBR's structure in the simulation environment.
  • ...and 8 more figures