Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equivariant Transporter Network

Haojie Huang, Dian Wang, Robin Walters, Robert Platt

TL;DR

The paper tackles sample-efficient robotic pick-and-place by making both pick and place components equivariant to object and placement orientations. By enforcing $C_n\times C_n$ symmetry through equivariant convolutions and a quotient representation for gripper orientation, the Equivariant Transporter Network generalizes across rotations more effectively than the original Transporter Net. Empirical results on Ravens-10, gripper-augmented variants, and real-robot experiments demonstrate improved performance with fewer demonstrations and faster convergence, while ablations show place-equivariance as the major contributor to gains. The approach offers a principled way to incorporate symmetry into learning-based manipulation, with practical impact for faster, more reliable policy learning on real robots.

Abstract

Transporter Net is a recently proposed framework for pick and place that is able to learn good manipulation policies from a very few expert demonstrations. A key reason why Transporter Net is so sample efficient is that the model incorporates rotational equivariance into the pick module, i.e. the model immediately generalizes learned pick knowledge to objects presented in different orientations. This paper proposes a novel version of Transporter Net that is equivariant to both pick and place orientation. As a result, our model immediately generalizes place knowledge to different place orientations in addition to generalizing pick knowledge as before. Ultimately, our new model is more sample efficient and achieves better pick and place success rates than the baseline Transporter Net model.

Equivariant Transporter Network

TL;DR

The paper tackles sample-efficient robotic pick-and-place by making both pick and place components equivariant to object and placement orientations. By enforcing symmetry through equivariant convolutions and a quotient representation for gripper orientation, the Equivariant Transporter Network generalizes across rotations more effectively than the original Transporter Net. Empirical results on Ravens-10, gripper-augmented variants, and real-robot experiments demonstrate improved performance with fewer demonstrations and faster convergence, while ablations show place-equivariance as the major contributor to gains. The approach offers a principled way to incorporate symmetry into learning-based manipulation, with practical impact for faster, more reliable policy learning on real robots.

Abstract

Transporter Net is a recently proposed framework for pick and place that is able to learn good manipulation policies from a very few expert demonstrations. A key reason why Transporter Net is so sample efficient is that the model incorporates rotational equivariance into the pick module, i.e. the model immediately generalizes learned pick knowledge to objects presented in different orientations. This paper proposes a novel version of Transporter Net that is equivariant to both pick and place orientation. As a result, our model immediately generalizes place knowledge to different place orientations in addition to generalizing pick knowledge as before. Ultimately, our new model is more sample efficient and achieves better pick and place success rates than the baseline Transporter Net model.
Paper Structure (60 sections, 5 theorems, 33 equations, 12 figures, 3 tables)

This paper contains 60 sections, 5 theorems, 33 equations, 12 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background on Symmetry Groups
  4. The Groups $\mathrm{SO}(2)$ and $C_n$
  5. Representation of a Group
  6. Feature Map Transformations
  7. Equivariant Mappings
  8. Transporter Network
  9. Problem Statement
  10. Description of Transporter Net
  11. Equivariance of Transporter Net
  12. Equivariant Transporter
  13. Equivariant Pick
  14. Model
  15. Equivariance Relationships
  16. ...and 45 more sections

Key Result

Proposition 1

The Transporter Net place network $f_{\mathrm{place}}$ is $C_n$-equivariant. That is, given $g \in C_n$, object image crop $c$ and scene image $o_t$,

Figures (12)

  • Figure 1: If Transporter Network zeng2020transporter learns to pick and place an object when it is presented in one orientation, the model is immediately able to generalize to new object orientations. We view this as $C_n$-equivariace of the model.
  • Figure 2: Our proposed Equivariant Transporter Network is able to generalize over both pick and place orientation. We view this as $C_n \times C_n$-equivariace of the model.
  • Figure 3: Illustration of the action of $T_g^{\mathrm{reg}}$ on a $2 \times 2$ image.
  • Figure 4: The Architecture of Transporter Net.
  • Figure 5: Illustration of the main part of the proof of Proposition \ref{['prop:equivtransporter']}. Rotating the crop $c$ induces a cyclic shift in the channels of the output $\psi(\mathcal{R}_n(T_g^0 c)) = \rho_\mathrm{reg}(-g)\psi(\mathcal{R}_n(c)).$
  • ...and 7 more figures

Theorems & Definitions (10)

  • Proposition 1
  • Proposition 2
  • Lemma 8.1
  • proof
  • Lemma 8.2
  • proof
  • Lemma 8.3
  • proof
  • proof
  • proof