Reinforcement Learning-based Robust Wall Climbing Locomotion Controller in Ferromagnetic Environment

TL;DR

This work tackles robust wall climbing on ferromagnetic steel under adhesion uncertainty using a reinforcement learning framework. It combines a physics-based electropermanent-magnetic (EPM) foot adhesion model with a three-phase gravity-rotation curriculum to progressively train from ground crawling to vertical climbing, including stochastic adhesion failures to promote recovery. The approach employs PPO in RaiSim with domain randomization and a compliant 3-DOF ankle, achieving high adhesion retention and slip recovery in simulation and on untethered hardware, and it outperforms an adhesion-ideal MPC baseline under adhesion loss. The results demonstrate the effectiveness of curriculum-driven sim-to-real transfer for magnetic climbing robots, enabling robust operation in complex ferromagnetic environments and providing a foundation for more versatile climbing capabilities in industry settings.

Abstract

We present a reinforcement learning framework for quadrupedal wall-climbing locomotion that explicitly addresses uncertainty in magnetic foot adhesion. A physics-based adhesion model of a quadrupedal magnetic climbing robot is incorporated into simulation to capture partial contact, air-gap sensitivity, and probabilistic attachment failures. To stabilize learning and enable reliable transfer, we design a three-phase curriculum: (1) acquire a crawl gait on flat ground without adhesion, (2) gradually rotate the gravity vector to vertical while activating the adhesion model, and (3) inject stochastic adhesion failures to encourage slip recovery. The learned policy achieves a high success rate, strong adhesion retention, and rapid recovery from detachment in simulation under degraded adhesion. Compared with a model predictive control (MPC) baseline that assumes perfect adhesion, our controller maintains locomotion when attachment is intermittently lost. Hardware experiments with the untethered robot further confirm robust vertical crawling on steel surfaces, maintaining stability despite transient misalignment and incomplete attachment. These results show that combining curriculum learning with realistic adhesion modeling provides a resilient sim-to-real framework for magnetic climbing robots in complex environments.

Figures (6)

• Figure 1: Snapshots of performing robust vertical climbing. The learned controller maintains adhesion and recovers from slips even with non-ferromagnetic patches, enabling the robot to continue crawling without detaching from the wall.
• Figure 2: Overview of the proposed learning framework for vertical locomotion under adhesion uncertainty. The process consists of four main components: (1) ground locomotion pre-training, (2) curriculum-based gravity rotation to adapt to climbing orientation, (3) probabilistic foot adhesion modeling to simulate imperfect magnetic contact, and (4) integration of simulation-to-real transfer with a compliant magnetic foot model. The policy receives proprioceptive observations with history and a crawl clock input, while an estimator predicts foot contacts. The actor--critic architecture is trained via reinforcement learning and imitation signals to produce joint torques and magnetic adhesion actions.
• Figure 3: Magnetic force of the magnetic feet with respect to air gap. Even with a 1 mm air gap, the adhesion force drops to about 7% of the maximum value.
• Figure 4: Training curves of climbing policy optimization. The vertical dashed lines indicate phase transitions of the multi-phase learning curriculum. Phase 1 trains locomotion on flat ground. In Phase 2, the wall inclination $\theta$ increases from $0^\circ$ to $90^\circ$ with perfect adhesion ($\text{Prob}_{\text{attach}}=1.0$). Phase 3 fixes $\theta=90^\circ$ while gradually reducing the adhesion probability from $1.0$ to $0.85$, exposing the policy to adhesion failures.
• Figure 5: Snapshots from a hardware trial. (Top) MPC fails once adhesion is lost. (Bottom) Our RL policy recovers from a failed adhesion and resumes stable crawling.