MaxMI: A Maximal Mutual Information Criterion for Manipulation Concept Discovery

TL;DR

This work tackles the challenge of grounding manipulation concepts from unannotated demonstrations by introducing a Maximal Mutual Information (MaxMI) criterion that identifies physically significant key states without human labeling. It develops a Key State Localization Network (KSL-Net) that localizes key states by maximizing the MI between a key state and its preceding state, with MI estimated by a differentiable neural estimator, and integrates discovered concepts into a concept-guided manipulation policy via the CoTPC framework. The approach is evaluated on complex tasks from ManiSkill2 and Franka Kitchen, showing that discovered concepts align with human semantics, enrich concept granularity, and yield higher success rates and better generalization than strong baselines, including in unseen configurations and zero-shot scenarios. Overall, the MaxMI-based framework reduces labeling needs, enhances grounding between low-level states and high-level manipulation concepts, and improves policy performance and robustness in diverse robotic tasks.

Abstract

We aim to discover manipulation concepts embedded in the unannotated demonstrations, which are recognized as key physical states. The discovered concepts can facilitate training manipulation policies and promote generalization. Current methods relying on multimodal foundation models for deriving key states usually lack accuracy and semantic consistency due to limited multimodal robot data. In contrast, we introduce an information-theoretic criterion to characterize the regularities that signify a set of physical states. We also develop a framework that trains a concept discovery network using this criterion, thus bypassing the dependence on human semantics and alleviating costly human labeling. The proposed criterion is based on the observation that key states, which deserve to be conceptualized, often admit more physical constraints than non-key states. This phenomenon can be formalized as maximizing the mutual information between the putative key state and its preceding state, i.e., Maximal Mutual Information (MaxMI). By employing MaxMI, the trained key state localization network can accurately identify states of sufficient physical significance, exhibiting reasonable semantic compatibility with human perception. Furthermore, the proposed framework produces key states that lead to concept-guided manipulation policies with higher success rates and better generalization in various robotic tasks compared to the baselines, verifying the effectiveness of the proposed criterion.

Figures (5)

• Figure 1: Grounding manipulation concepts via multimodal foundation models. (a): A manipulation concept is grounded using the multimodal encoders from CLIP radford2learning by checking the cosine similarity between the image features and the text embedding of the concept; (b): A multimodal LLM (GPT-4V) can also be used to ground a manipulation concept by directly asking it if the physical state presented renders the manipulation concept achieved. These examples demonstrate that concept grounding using large multimodal foundation models still lags due to the lack of robotic training data.
• Figure 2: Mutual information between a random state variable and its preceding one achieves a maximum when the state variable coincides with a key state (manipulation concept), as verified across four manipulation tasks, namely, Turn Faucet, Peg Insertion, Pick Cube, and Stack Cube. The subfigures highlight the moments when the mutual information arrives at a peak, together with images that illustrate the corresponding key states. This phenomenon is commonly observed and suggests that one can discover manipulation concepts by maximizing such mutual information quantity.
• Figure 3: The proposed Key State Localization Network (KSL-Net) for manipulation concept discovery. Every key concept (to be discovered and localized) is represented by a learnable embedding ($e_k$); the concept embedding is then appended to all state vectors along a trajectory. These augmented state vectors are further processed by a fully convolutional encoder and a multi-layer perceptron (MLP) to derive the probability ($\mathrm{p}^i_k$) of each state being the identified key state.
• Figure 4: Examples of manually annotated key states and those discovered by the proposed pipeline, across four distinct tasks: Pick & Place Cube, Stack Cube, Turn Faucet, and Peg Insertion Side. As observed, our method not only discovers the key states that align with human semantics, but also promotes more fine-grained manipulation concepts, which we show can effectively benefit the concept-guided policy learning.
• Figure 5: Key states discovered with different terms as discussed in Tab. \ref{['tab:criterion_selection']}.