Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latent Perspective-Taking via a Schrödinger Bridge in Influence-Augmented Local Models

Kevin Alcedo, Pedro U. Lima, Rachid Alami

TL;DR

This work tackles decision-making under uncertainty in socially interactive environments by learning factored mental-models and estimating others' mental-states. It combines an Influence-Augmented Local Model (IALM) with an amortized Schrödinger Bridge to perform belief transport from egocentric to other-centric perspectives, enabling decision-time mental-state planning in latent belief space. A neuro-symbolic, discrete-latent world model is paired with a perspective-shift operator that tilts reference dynamics, trained via epistemic counterfactuals and optimized with a composite loss. Preliminary MiniGrid experiments show faster learning and higher returns compared to context-free baselines, suggesting a practical path toward more socially aware and robust robot behavior in open, uncertain worlds.

Abstract

Operating in environments alongside humans requires robots to make decisions under uncertainty. In addition to exogenous dynamics, they must reason over others' hidden mental-models and mental-states. While Interactive POMDPs and Bayesian Theory of Mind formulations are principled, exact nested-belief inference is intractable, and hand-specified models are brittle in open-world settings. We address both by learning structured mental-models and an estimator of others' mental-states. Building on the Influence-Based Abstraction, we instantiate an Influence-Augmented Local Model to decompose socially-aware robot tasks into local dynamics, social influences, and exogenous factors. We propose (a) a neuro-symbolic world model instantiating a factored, discrete Dynamic Bayesian Network, and (b) a perspective-shift operator modeled as an amortized Schrödinger Bridge over the learned local dynamics that transports factored egocentric beliefs into other-centric beliefs. We show that this architecture enables agents to synthesize socially-aware policies in model-based reinforcement learning, via decision-time mental-state planning (a Schrödinger Bridge in belief space), with preliminary results in a MiniGrid social navigation task.

Latent Perspective-Taking via a Schrödinger Bridge in Influence-Augmented Local Models

TL;DR

This work tackles decision-making under uncertainty in socially interactive environments by learning factored mental-models and estimating others' mental-states. It combines an Influence-Augmented Local Model (IALM) with an amortized Schrödinger Bridge to perform belief transport from egocentric to other-centric perspectives, enabling decision-time mental-state planning in latent belief space. A neuro-symbolic, discrete-latent world model is paired with a perspective-shift operator that tilts reference dynamics, trained via epistemic counterfactuals and optimized with a composite loss. Preliminary MiniGrid experiments show faster learning and higher returns compared to context-free baselines, suggesting a practical path toward more socially aware and robust robot behavior in open, uncertain worlds.

Abstract

Operating in environments alongside humans requires robots to make decisions under uncertainty. In addition to exogenous dynamics, they must reason over others' hidden mental-models and mental-states. While Interactive POMDPs and Bayesian Theory of Mind formulations are principled, exact nested-belief inference is intractable, and hand-specified models are brittle in open-world settings. We address both by learning structured mental-models and an estimator of others' mental-states. Building on the Influence-Based Abstraction, we instantiate an Influence-Augmented Local Model to decompose socially-aware robot tasks into local dynamics, social influences, and exogenous factors. We propose (a) a neuro-symbolic world model instantiating a factored, discrete Dynamic Bayesian Network, and (b) a perspective-shift operator modeled as an amortized Schrödinger Bridge over the learned local dynamics that transports factored egocentric beliefs into other-centric beliefs. We show that this architecture enables agents to synthesize socially-aware policies in model-based reinforcement learning, via decision-time mental-state planning (a Schrödinger Bridge in belief space), with preliminary results in a MiniGrid social navigation task.
Paper Structure (9 sections, 4 equations, 2 figures)

This paper contains 9 sections, 4 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Influence-Augmented Local Model
  4. Schrödinger Bridge
  5. Method
  6. Neuro-Symbolic World Model
  7. Perspective-Shift Operator
  8. Preliminary Results, Limitations & Next Steps
  9. Related Work

Figures (2)

  • Figure 1: (a) Influence-Augmented Local Model; (b) Neuro-Symbolic World Model Architecture.
  • Figure 2: (a) Perspective-shift representations. Left: global state. Right: ground-truth observations (top); factored, discrete latent states (second); actions (third); next-step latent states (fourth); reconstructed observation (last). (b) Median episodic returns over 5 seeds, shaded band denotes 95% bootstrap CI of the median; dashed line: Perfect-Information baseline final median.