Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames

TL;DR

This work investigates multi-task imitation learning with behavioral cloning and introduces the concept of a blindfolded expert to induce exploratory behavior that improves generalization to unseen tasks. The authors formulate a Task- conditioned BC setting, derive a generalization bound that includes an information-theoretic term $\sqrt{ I_{T;Z} / m }$, and show that reducing task information via blindfolding can enhance cross-task performance without sacrificing training efficiency. Empirically, they validate the approach on Procgen maze/heist games and a real robotic peg-in-hole task, demonstrating that BF-BC (blindfolded BC) generalizes better to unseen tasks than standard BC, even under comparable data budgets. The results highlight the potential of designing demonstrations with information bottlenecks to improve foundation-model-scale imitation, offering a principled route to robust, transferable visuo-motor policies.

Abstract

Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving generalization requires countless demonstrations of a multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) optimal behavior. In this paper, we propose to hide some of the task's information from the demonstrator. This ``blindfolded'' expert is compelled to employ non-trivial exploration to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We conduct experiments of real-world robot peg insertion tasks with (limited) human demonstrations, alongside videogames from the Procgen benchmark. Additionally, we support our findings with theoretical analysis, which confirms that the generalization error scales with $\sqrt{I/m}$, where $I$ measures the amount of task information available to the demonstrator, and $m$ is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts generalizes better with fewer demonstrated tasks. Project page with videos and code: https://sites.google.com/view/blindfoldedexperts/home

Figures (10)

• Figure 1: Illustration of the learning process. Note that the mask on observations only applies to the blindfolded expert, while the observations in the logged trajectories are unmasked in both cases.
• Figure 2: Demonstration of actual experts' observations. For the Procgen maze (left-pair), Procgen heist (middle-pair), and robotic peg insertion (right-pair). We show the full observation of the Expert and the masked observation of the Blindfolded-Expert.
• Figure 3: Game score as a function of training epochs in the Procgen maze and heist. Left-pair: performance on $200$ training levels. Right-pair: performance on unseen test levels. The mean and standard deviation are computed over 10 seeds.
• Figure 4: Procgen maze score as a function of training epochs. Left: performance on $100$ training levels. Right: performance on unseen test levels. Policies $\pi_{BC}$ and $\pi_{BF-BC}$ are trained on $2K$ trajectories, the former clearly overfits. We also train $\pi_{BC-ext}$ on $4.6K$ non-blindfolded experts' trajectories, matching the total number of steps of $\pi_{BF-BC}$. The resulting $\pi_{BC-ext}$ overfits despite access to twice the amount of data. The mean and standard deviation are computed over $10$ seeds.
• Figure 5: The robotic arm configured for peg insertion (left). Close-up view of various peg shapes and their boards (right).

Theorems & Definitions (9)

• theorem 3.1
• theorem A.1: Theorem \ref{['thr:generalization']}
• proof
• lemma A.2
• proof
• lemma A.3: Sample complexity of behavioral cloning foster2024behavior
• proof
• lemma A.4: Information bottleneck generalization gap kawaguchi2023does
• proof