Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning

Yichen Xie, Yixiao Wang, Shuqi Zhao, Cheng-En Wu, Masayoshi Tomizuka, Jianwen Xie, Hao-Shu Fang

Abstract

The generalization ability of imitation learning policies for robotic manipulation is fundamentally constrained by the diversity of expert demonstrations, while collecting demonstrations across varied environments is costly and difficult in practice. In this paper, we propose a practical framework that exploits inherent scene diversity without additional human effort by scaling camera views during demonstration collection. Instead of acquiring more trajectories, multiple synchronized camera perspectives are used to generate pseudo-demonstrations from each expert trajectory, which enriches the training distribution and improves viewpoint invariance in visual representations. We analyze how different action spaces interact with view scaling and show that camera-space representations further enhance diversity. In addition, we introduce a multiview action aggregation method that allows single-view policies to benefit from multiple cameras during deployment. Extensive experiments in simulation and real-world manipulation tasks demonstrate significant gains in data efficiency and generalization compared to single-view baselines. Our results suggest that scaling camera views provides a practical and scalable solution for imitation learning, which requires minimal additional hardware setup and integrates seamlessly with existing imitation learning algorithms. The website of our project is https://yichen928.github.io/robot_multiview.

Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning

Abstract

The generalization ability of imitation learning policies for robotic manipulation is fundamentally constrained by the diversity of expert demonstrations, while collecting demonstrations across varied environments is costly and difficult in practice. In this paper, we propose a practical framework that exploits inherent scene diversity without additional human effort by scaling camera views during demonstration collection. Instead of acquiring more trajectories, multiple synchronized camera perspectives are used to generate pseudo-demonstrations from each expert trajectory, which enriches the training distribution and improves viewpoint invariance in visual representations. We analyze how different action spaces interact with view scaling and show that camera-space representations further enhance diversity. In addition, we introduce a multiview action aggregation method that allows single-view policies to benefit from multiple cameras during deployment. Extensive experiments in simulation and real-world manipulation tasks demonstrate significant gains in data efficiency and generalization compared to single-view baselines. Our results suggest that scaling camera views provides a practical and scalable solution for imitation learning, which requires minimal additional hardware setup and integrates seamlessly with existing imitation learning algorithms. The website of our project is https://yichen928.github.io/robot_multiview.

Paper Structure

This paper contains 22 sections, 9 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Imitation Learning for Robot Manipulation.
  4. Multiview Visual Representation Learning.
  5. Methodology
  6. Preliminary: Imitation Learning for Visuomotor Policy
  7. Camera View Scaling-up for Pseudo-Demonstrations
  8. Robot Actions in View-Scaled Imitation Learning
  9. Robot Base Space
  10. End-Effector Space
  11. Camera Space
  12. Multi-View Action Aggregation in Deployment
  13. Experiments
  14. Experiment Setup
  15. Tasks
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Camera View Scaling-Up. Each expert demonstration can be converted to multiple pseudo-demonstrations by exploiting multiple camera perspective views.
  • Figure 2: Action Space for Pseudo-Demonstrations. We consider three potential action spaces: base, camera, and EEF.
  • Figure 3: Visualization of Attention Scores. When trained with randomly picked camera views, the model learns to focus on the most important regions to finish the task. Otherwise, the attention is distracted to unrelated regions.
  • Figure 4: Real-World Robot Setup. We use a FANUC CRX-10iA robotic arm equipped with a parallel-jaw gripper with silicone soft fingertips and two camera views.