Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination

Rakshit Trivedi, Kartik Sharma, David C Parkes

TL;DR

MIMIC (Modeling Inner Motivations for Imitation and Control), a framework that uses language as an internal representation of behavioral intent that enables fine-grained steering of behavior at inference time by conditioning the agent on behavior-specific speech.

Abstract

Effective human-AI coordination requires artificial agents capable of exhibiting and responding to human-like behaviors while adapting to changing contexts. Imitation learning has emerged as one of the prominent approaches to build such agents by training them to mimic human-demonstrated behaviors. However, current methods struggle to capture the inherent diversity and non-Markovian nature of human behavior and lack the ability to steer behavior at inference time. Drawing inspiration from the theory of human cognitive processes, where inner speech guides action selection before execution, we propose MIMIC (Modeling Inner Motivations for Imitation and Control), a framework that uses language as an internal representation of behavioral intent. MIMIC employs the novel use of vision-language models as linguistic scaffolding to train a conditional variational autoencoder capable of generating inner speech from observations. A diffusion-based behavior cloning policy then selects actions conditioned on current observations and the generated inner speech. MIMIC enables fine-grained steering of behavior at inference time by conditioning the agent on behavior-specific speech. Experiments across robotic manipulation tasks and human-AI collaboration games demonstrate that MIMIC significantly enhances both behavior diversity and fidelity to human demonstrations while enabling nuanced behavioral steering without training on additional demonstrations. We open source our code and provide pre-trained MIMIC agents and qualitative demos at: https://mimic-research.github.io.

Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination

TL;DR

MIMIC (Modeling Inner Motivations for Imitation and Control), a framework that uses language as an internal representation of behavioral intent that enables fine-grained steering of behavior at inference time by conditioning the agent on behavior-specific speech.

Abstract

Effective human-AI coordination requires artificial agents capable of exhibiting and responding to human-like behaviors while adapting to changing contexts. Imitation learning has emerged as one of the prominent approaches to build such agents by training them to mimic human-demonstrated behaviors. However, current methods struggle to capture the inherent diversity and non-Markovian nature of human behavior and lack the ability to steer behavior at inference time. Drawing inspiration from the theory of human cognitive processes, where inner speech guides action selection before execution, we propose MIMIC (Modeling Inner Motivations for Imitation and Control), a framework that uses language as an internal representation of behavioral intent. MIMIC employs the novel use of vision-language models as linguistic scaffolding to train a conditional variational autoencoder capable of generating inner speech from observations. A diffusion-based behavior cloning policy then selects actions conditioned on current observations and the generated inner speech. MIMIC enables fine-grained steering of behavior at inference time by conditioning the agent on behavior-specific speech. Experiments across robotic manipulation tasks and human-AI collaboration games demonstrate that MIMIC significantly enhances both behavior diversity and fidelity to human demonstrations while enabling nuanced behavioral steering without training on additional demonstrations. We open source our code and provide pre-trained MIMIC agents and qualitative demos at: https://mimic-research.github.io.
Paper Structure (47 sections, 1 theorem, 14 equations, 12 figures, 8 tables, 2 algorithms)

This paper contains 47 sections, 1 theorem, 14 equations, 12 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. MIMIC: Inner Speech as Behavior Guides
  4. Theoretical Formulation of Inner Speech
  5. Inner Speech as a Generative Mediating Process
  6. Mathematical Formulation
  7. Architectural Correspondence
  8. Learning Inner Speech and Behavior Generation from Demonstrations
  9. Inner Speech-conditioned Behavior Cloner
  10. Behavior-Conditioned Inner Speech Generator
  11. Experiments
  12. Setup
  13. Results
  14. Does MIMIC achieve high fidelity imitation of diverse behaviors on D3IL benchmark?
  15. Do MIMIC agents serve as effective in silico human surrogates, achieving successful coordination when evaluated with human proxy models in collaborative tasks?
  16. ...and 32 more sections

Key Result

Proposition 1

Instead of directly learning the human action distribution conditioned on the environment's state, $\pi_{\theta}(a \mid s) \approx p_{\mathcal{H}\xspace}(a \mid s)$, we propose to model human behavior through inner speech mediation: $P_{\mathcal{H}\xspace}(a \mid s) = \int p_{\mathcal{H}\xspace}(a \

Figures (12)

  • Figure 1: Paradigm comparison.(a) direct state-to-action mapping. (b) inner speech $m_t$ mediates between perception and action. Extended discussion available in Appendix \ref{['app:paradigm']}.
  • Figure 2: Overview of MIMIC: Agent inner speech is scaffolded by using a pre-trained VLM to discriminate different behaviors in human demonstrations. Next, we train a DDPM-T (diffusion policy with transformer architecture) behavior cloner conditioned on this inner speech and a VAE-based inner speech generator conditioned on the history of states. During simulation, the inner speech is periodically generated to influence the behavior cloner given the actions it generated in the past.
  • Figure 3: GPT-4o evaluation. '-' denotes no update.
  • Figure 4: Effect on Success Rate and Entropy performance of MIMIC on changing various components on the Aligning dataset: (a) removing inner speech during simulation, (b) changing the embedding model, (c-d) using different VLMs for training and scaffolding.
  • Figure 6: Contrasting theoretical frameworks for IL. (a) The behaviorist approach models human behavior as a direct mapping from environmental states to actions ($s_t \mapsto_{\mathcal{H}} a_t$), treating cognitive processes as opaque transformations. (b) The cognitive approach instantiated by MIMIC introduces inner speech as a mediational layer ($s_t \rightarrow m_t \rightarrow a_t$), where $m_t$ represents linguistically-structured internal deliberation that enables behavioral diversity and contextual adaptation.
  • ...and 7 more figures

Theorems & Definitions (1)

  • Proposition 1: Imitating with Inner Speech $m$