Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ROPES: Robotic Pose Estimation via Score-Based Causal Representation Learning

Pranamya Kulkarni, Puranjay Datta, Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, Ali Tajer

TL;DR

ROPES reframes robot pose estimation as a causal representation learning problem, leveraging score-based interventional CRL to identify and recover pose-related latent variables from images without pose labels. It introduces a three-stage autoencoder–based pipeline that uses interventional regularizers to disentangle joint-angle factors in latent space, guided by score differences across interventions. Empirical results on a Panda-Gym setup show strong latent identifiability for several joints (high MCC) and low pose error (MSE), along with robustness to occlusion, and competitive performance against a label-dependent RoboPEPP baseline. The work demonstrates the viability of CRL as a near-practical framework for robotics, with implications for scalable state estimation and future video-world model integration in robotic systems.

Abstract

Causal representation learning (CRL) has emerged as a powerful unsupervised framework that (i) disentangles the latent generative factors underlying high-dimensional data, and (ii) learns the cause-and-effect interactions among the disentangled variables. Despite extensive recent advances in identifiability and some practical progress, a substantial gap remains between theory and real-world practice. This paper takes a step toward closing that gap by bringing CRL to robotics, a domain that has motivated CRL. Specifically, this paper addresses the well-defined robot pose estimation -- the recovery of position and orientation from raw images -- by introducing Robotic Pose Estimation via Score-Based CRL (ROPES). Being an unsupervised framework, ROPES embodies the essence of interventional CRL by identifying those generative factors that are actuated: images are generated by intrinsic and extrinsic latent factors (e.g., joint angles, arm/limb geometry, lighting, background, and camera configuration) and the objective is to disentangle and recover the controllable latent variables, i.e., those that can be directly manipulated (intervened upon) through actuation. Interventional CRL theory shows that variables that undergo variations via interventions can be identified. In robotics, such interventions arise naturally by commanding actuators of various joints and recording images under varied controls. Empirical evaluations in semi-synthetic manipulator experiments demonstrate that ROPES successfully disentangles latent generative factors with high fidelity with respect to the ground truth. Crucially, this is achieved by leveraging only distributional changes, without using any labeled data. The paper also includes a comparison with a baseline based on a recently proposed semi-supervised framework. This paper concludes by positioning robot pose estimation as a near-practical testbed for CRL.

ROPES: Robotic Pose Estimation via Score-Based Causal Representation Learning

TL;DR

ROPES reframes robot pose estimation as a causal representation learning problem, leveraging score-based interventional CRL to identify and recover pose-related latent variables from images without pose labels. It introduces a three-stage autoencoder–based pipeline that uses interventional regularizers to disentangle joint-angle factors in latent space, guided by score differences across interventions. Empirical results on a Panda-Gym setup show strong latent identifiability for several joints (high MCC) and low pose error (MSE), along with robustness to occlusion, and competitive performance against a label-dependent RoboPEPP baseline. The work demonstrates the viability of CRL as a near-practical framework for robotics, with implications for scalable state estimation and future video-world model integration in robotic systems.

Abstract

Causal representation learning (CRL) has emerged as a powerful unsupervised framework that (i) disentangles the latent generative factors underlying high-dimensional data, and (ii) learns the cause-and-effect interactions among the disentangled variables. Despite extensive recent advances in identifiability and some practical progress, a substantial gap remains between theory and real-world practice. This paper takes a step toward closing that gap by bringing CRL to robotics, a domain that has motivated CRL. Specifically, this paper addresses the well-defined robot pose estimation -- the recovery of position and orientation from raw images -- by introducing Robotic Pose Estimation via Score-Based CRL (ROPES). Being an unsupervised framework, ROPES embodies the essence of interventional CRL by identifying those generative factors that are actuated: images are generated by intrinsic and extrinsic latent factors (e.g., joint angles, arm/limb geometry, lighting, background, and camera configuration) and the objective is to disentangle and recover the controllable latent variables, i.e., those that can be directly manipulated (intervened upon) through actuation. Interventional CRL theory shows that variables that undergo variations via interventions can be identified. In robotics, such interventions arise naturally by commanding actuators of various joints and recording images under varied controls. Empirical evaluations in semi-synthetic manipulator experiments demonstrate that ROPES successfully disentangles latent generative factors with high fidelity with respect to the ground truth. Crucially, this is achieved by leveraging only distributional changes, without using any labeled data. The paper also includes a comparison with a baseline based on a recently proposed semi-supervised framework. This paper concludes by positioning robot pose estimation as a near-practical testbed for CRL.
Paper Structure (44 sections, 1 theorem, 9 equations, 11 figures, 14 tables)

This paper contains 44 sections, 1 theorem, 9 equations, 11 figures, 14 tables.

Table of Contents

  1. Introduction
  2. A generative viewpoint on pose.
  3. Interventions via controllable variables.
  4. Methodology.
  5. ML-based Pose Estimation (Supervised Deep Learning).
  6. Model-based Pose Estimation.
  7. Contributions.
  8. Problem Setting: CRL for Robotic Pose Estimation
  9. Pose estimation as CRL.
  10. Latent causal generative model.
  11. Interventions.
  12. Objective.
  13. Score-based Methodology: ROPES
  14. Score-Based Interventional CRL: Key Properties
  15. Data Generation: Interventions via Manipulation
  16. ...and 29 more sections

Key Result

Theorem 1

Assume that the latent distribution $p$ has non-zero density over ${\mathbb {R}}^d$, $f$ is a diffeomorphism onto its image, and that pair $(q^i,\bar{q}^i)$ satisfies interventional discrepancy. Then, the global optimizer $(h^*,g^*)$ of $\mathcal{L}(h,g)$ recovers latent $z_i$ up to an elementwise t

Figures (11)

  • Figure 1: Conceptual overview of ROPES, highlighting its three-stage pipeline. The output visualization marks the specific joints targeted for intervention, along with their respective axes of rotation.
  • Figure 2: Overview of ROPES pipeline. A shared Autoencoder 1 (AE1) compresses each $128 \times 128 \times 1$ image into an $8 \times 8 \times 1$ feature map. The subsequent data processing depends on the experimental setup. Single-camera case (analyzing 3 joints): feature map is directly fed into both the LDR network and Autoencoder 2 (AE2). AE2 then produces the final $3 \times 1$ disentangled pose vector. Two-camera case (analyzing 6 joints): feature maps from both views are concatenated along the channel axis, forming an $8 \times 8 \times 2$ input to the LDR and AE2. AE2 then outputs the final $6 \times 1$ pose vector.
  • Figure 3: Single Camera: Scatter-plots of ground-truth vs. estimated angles for joints 2, 4, and 6
  • Figure 4: A step-by-step visualization of the reconstruction process for an occluded input. The final reconstruction from autoencoder-2, generated by passing its output through the decoder of autoencoder-1, successfully inpaints the occluded region.
  • Figure 5: Two different hard interventions on Joint 4.
  • ...and 6 more figures

Theorems & Definitions (1)

  • Theorem 1: Theorem 22 varici2025score, reworded