Redundancy-aware Action Spaces for Robot Learning

TL;DR

The paper tackles the trade-off between fast learning with task-space actions and precise full-body control with joint-space actions for overactuated robotic arms. It introduces End-effector Redundancy (ER), with ERAngle (ERA) and ERJoint (ERJ) as concrete implementations that enable end-effector control while resolving kinematic redundancies through IK-based translation. Across RLBench simulations and real-world imitation tasks, ERJ delivers superior performance in precision-critical scenarios, while ERA shines on simpler tasks and traditional spaces lag in either efficiency or control. This work provides a practical path toward data-efficient, robust manipulation for robots with more than six DoF and supplies code to facilitate replication and future research.

Abstract

Joint space and task space control are the two dominant action modes for controlling robot arms within the robot learning literature. Actions in joint space provide precise control over the robot's pose, but tend to suffer from inefficient training; actions in task space boast data-efficient training but sacrifice the ability to perform tasks in confined spaces due to limited control over the full joint configuration. This work analyses the criteria for designing action spaces for robot manipulation and introduces ER (End-effector Redundancy), a novel action space formulation that, by addressing the redundancies present in the manipulator, aims to combine the advantages of both joint and task spaces, offering fine-grained comprehensive control with overactuated robot arms whilst achieving highly efficient robot learning. We present two implementations of ER, ERAngle (ERA) and ERJoint (ERJ), and we show that ERJ in particular demonstrates superior performance across multiple settings, especially when precise control over the robot configuration is required. We validate our results both in simulated and real robotic environments.

Figures (8)

• Figure 1: (a) The configuration of the entire arm is crucial for manipulation in confined spaces, e.g., removing an object from the cabinet. A correct elbow position enables the robot to enter the cabinet (left), but an incorrect elbow position causes unsuccessful execution of the action (right). (b) Our ER action spaces (ERJ and ERA) outperform other action spaces in tasks where control over the entire robot configuration is required (6 tasks shown on the right).
• Figure 2: ER family of action spaces. (a) ERA is constrained directly by the angle of the elbow around the axis from the shoulder to the wrist ($\phi$). (b) ERJ is constrained through removing joints from the inverse kinematics and controlling these separately (the base joint $j_\theta$ in the diagram).
• Figure 3: Results on 6 tasks where control over the entire robot arm configuration is required to succeed or perform reliably (4 runs).
• Figure 4: Comparing action modes on a standard suite of 8 manipulation tasks. (4 runs).
• Figure 5: Pictures representative of the 5 real-world robotic tasks solved with imitation learning.

Theorems & Definitions (5)

• Definition III.1: Task Space
• Definition III.2: Joint Space
• Definition IV.1: ER Space
• Definition IV.2: ER Angle space
• Definition IV.3: ER Joint space