Morphology-Aware Graph Reinforcement Learning for Tensegrity Robot Locomotion

TL;DR

This work tackles the challenge of controlling tensegrity robots whose dynamics are highly coupled and underactuated. It proposes a morphology-aware reinforcement learning framework by incorporating a graph neural network (GNN) actor into Soft Actor-Critic (SAC), encoding the robot topology as a graph with end-caps as nodes and tendons/rods as edges. The approach yields faster learning, greater robustness to noise, stiffness variations, and terrain, and enables zero-shot sim-to-real transfer on a 3-bar tensegrity robot across straight tracking and turning primitives. These results demonstrate the value of embedding structural priors into RL for tensegrity locomotion and point to scalable, composable policies for real-world deployment.

Abstract

Tensegrity robots combine rigid rods and elastic cables, offering high resilience and deployability but posing major challenges for locomotion control due to their underactuated and highly coupled dynamics. This paper introduces a morphology-aware reinforcement learning framework that integrates a graph neural network (GNN) into the Soft Actor-Critic (SAC) algorithm. By representing the robot's physical topology as a graph, the proposed GNN-based policy captures coupling among components, enabling faster and more stable learning than conventional multilayer perceptron (MLP) policies. The method is validated on a physical 3-bar tensegrity robot across three locomotion primitives, including straight-line tracking and bidirectional turning. It shows superior sample efficiency, robustness to noise and stiffness variations, and improved trajectory accuracy. Notably, the learned policies transfer directly from simulation to hardware without fine-tuning, achieving stable real-world locomotion. These results demonstrate the advantages of incorporating structural priors into reinforcement learning for tensegrity robot control.

Figures (9)

• Figure 1: Morphology-aware graph reinforcement learning for tensegrity locomotion. The robot’s states (end-cap positions and velocities) are encoded as node features in a graph-based policy, which propagates information along the robot’s structural connections. The network outputs tendon length commands to actuate the tensegrity robot to roll forward in physical experiments.
• Figure 2: Physical 3-bar tensegrity robot platform and reference coordinate definitions. $\varphi$ indicates the waypoint angle between forward direction and tracking direction.
• Figure 3: Overview of the proposed morphology-aware GNN-SAC framework for tensegrity robot locomotion. The Soft Actor-Critic (SAC) algorithm integrates a graph neural network (GNN)-based policy that encodes the robot’s topology via message passing among end-cap nodes. The actor generates tendon length commands based on structured observations, enabling morphology-aware learning in both simulation and real-world environments.
• Figure 4: Benchmark of learning performance across algorithms and network depths. The proposed GNN-SAC consistently outperforms MLP-based SAC (M-SAC), PPO, and TD3 in terms of training reward and sample efficiency for all three locomotion primitives. Subplots (a,c,e) compare algorithms, while (b,d,f) analyze the effect of GNN encoder depth, showing improved performance with multi-layer message passing.
• Figure 5: Simulation evaluation of learned motion primitives between Graph-based SAC (G-SAC) and MLP-based SAC (M-SAC): (a) Straight-line tracking error for different waypoint yaw angles; (b) Yaw rate and stability in bidirectional turning tasks.