CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrains

TL;DR

The paper tackles robust humanoid locomotion under observation noise and sim-to-real gaps by introducing Contractive Mapping for Robustness (CMR), a framework that learns contractive latent dynamics through a combination of contrastive representation learning and Lipschitz regularization. Theoretical contributions provide bounds on the noise-induced return gap, including $J(\pi^*) - J(\pi) \le \mathcal{O}(H L_r L_f^H) M \delta_{max}$ for non-contractive dynamics and $J(\pi^*) - J(\pi) \le \mathcal{O}(\frac{\eta}{1-\kappa})$ under a contractive embedding with $0<\kappa<1$, which are horizon-sensitive and horizon-independent respectively. The approach integrates seamlessly with PPO via a composite loss $\mathcal{L}_{\text{CMR}} = \mathcal{L}_{\text{InfoNCE}} + \lambda \mathcal{L}_{\text{Lipschitz}} + \mathcal{L}_{\text{PPO}}$, enabling robust, perceptually aware control. Empirically, CMR outperforms baselines across diverse terrains and noise levels, exhibits strong sim-to-sim zero-shot transfer to MuJoCo, and offers interpretable latent representations via visualization, indicating practical impact for robust humanoid robotics in unstructured environments.

Abstract

Robust disturbance rejection remains a longstanding challenge in humanoid locomotion, particularly on unstructured terrains where sensing is unreliable and model mismatch is pronounced. While perception information, such as height map, enhances terrain awareness, sensor noise and sim-to-real gaps can destabilize policies in practice. In this work, we provide theoretical analysis that bounds the return gap under observation noise, when the induced latent dynamics are contractive. Furthermore, we present Contractive Mapping for Robustness (CMR) framework that maps high-dimensional, disturbance-prone observations into a latent space, where local perturbations are attenuated over time. Specifically, this approach couples contrastive representation learning with Lipschitz regularization to preserve task-relevant geometry while explicitly controlling sensitivity. Notably, the formulation can be incorporated into modern deep reinforcement learning pipelines as an auxiliary loss term with minimal additional technical effort required. Further, our extensive humanoid experiments show that CMR potently outperforms other locomotion algorithms under increased noise.

Figures (6)

• Figure 1: The left panel illustrates diverse types of challenging terrains I–VI for humanoid locomotion tasks under noisy observations. While the radar plot on the right shows that CMR (Ours) consistently outperforms the baselines under Noise Case 3. More results are further reported in Fig. \ref{['fig:noise_level_comparison']}.
• Figure 2: Overview of the CMR framework. Noisy observations, including current and historical proprioception ($o_t^{\text{proprio}}+\delta_t^1$, $o_{t-h}^{\text{proprio}}+\delta_t^2$) and perception ($o_t^{\text{percept}}+\delta_t^3$), are encoded into the contractive embedding $I_t$. Training $I_t$ uses a Lipschitz regularization term and a contrastive objective. As a result, the trained contractive embedding $I_t$ is compatible with deep-RL pipelines as an additional input.
• Figure 3: Latent space visualizations of five trajectories from deployments of CMR (left) and HIM (right) under Noise Case I.
• Figure 4: Latent space visualizations of trajectories across terrains from deployments of CMR (left) and HIM (right).
• Figure 5: Locomotion Distances for deployment of CMR (Ours) and baselines across varing terrains under Noise Case 4/5.

Theorems & Definitions (4)

• Theorem 1: Return Gap under Observation Noise
• Theorem 2: Return Gap in Contractive Embedding Space
• proof
• proof