Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation

TL;DR

This work tackles latent redundancy in Behavior Cloning for robot manipulation by applying the Information Bottleneck (IB) to compress multimodal inputs into a task-relevant latent $Z$, minimizing $I(X; Z)$ while preserving $I(Z; A)$. It extends IB to BC by defining the BC-IB objective and analyzes two fusion schemes for multimodal inputs, supported by theoretical generalization bounds. Through large-scale experiments on CortexBench and LIBERO, BC with IB consistently improves generalization across backbones and tasks, and the authors provide MI-based visualizations and ablations to illuminate the underlying mechanics. The findings demonstrate the practical value of redundancy reduction in robotic imitation learning and establish a principled, information-theoretic framework for future representation learning in robotics.

Abstract

Behavior Cloning (BC) is a widely adopted visual imitation learning method in robot manipulation. Current BC approaches often enhance generalization by leveraging large datasets and incorporating additional visual and textual modalities to capture more diverse information. However, these methods overlook whether the learned representations contain redundant information and lack a solid theoretical foundation to guide the learning process. To address these limitations, we adopt an information-theoretic perspective and introduce mutual information to quantify and mitigate redundancy in latent representations. Building on this, we incorporate the Information Bottleneck (IB) principle into BC, which extends the idea of reducing redundancy by providing a structured framework for compressing irrelevant information while preserving task-relevant features. This work presents the first comprehensive study on redundancy in latent representations across various methods, backbones, and experimental settings, while extending the generalizability of the IB to BC. Extensive experiments and analyses on the CortexBench and LIBERO benchmarks demonstrate significant performance improvements with IB, underscoring the importance of reducing input data redundancy and highlighting its practical value for more practical applications. Project Page: https://baishuanghao.github.io/BC-IB.github.io.

Figures (12)

• Figure 1: Policy architecture of BC. Current BC methods (black arrows) do not impose restrictions on the latent representations $Z$, potentially allowing redundant information from the input representations $X$.
• Figure 2: Model architectures used in this study. Based on feature fusion methods, we categorize the BC methods in robot manipulation into two types: spatial fusion and temporal fusion. After extracting features from each modality a), spatial fusion b) extracts spatial features at a given time step or concatenates features across multiple time steps using encoders like MLPs or CNNs. Temporal Fusion c) fuses input features by modeling dynamic relationships and dependencies between time steps using RNNs or Temporal Transformers. The latent representations are then decoded into actions via the policy head.
• Figure 3: (a) BC loss variation for ResNet in spatial and temporal fusion methods on the bin-picking task of the Meta-World. (b) Averaged success rates of ResNet and VC1 in spatial and temporal fusion methods across the Meta-World and DMControl.
• Figure 4: Real-world robot experiments conducted on a tabletop setup with two settings. (a) Left: the experimental setup. (a) Right: an example of predicted trajectories alongside policy execution. (b) and (c): quantitative evaluation results across two settings, where blue denotes the vanilla BC method and red denotes the method with IB. $^*$ denotes the unseen tasks.
• Figure 5: Effect of the Lagrange multiplier $\beta$ in BC-VILT+IB across three suites of LIBERO. When $\beta$=0, the method reduces to vanilla BC-VILT.

Theorems & Definitions (4)

• Theorem 4.1
• Theorem 4.2
• Theorem 4.3
• proof