Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Information Gathering Under Dynamics Uncertainty

Fernando Palafox, Jingqi Li, Jesse Milzman, David Fridovich-Keil

TL;DR

This paper addresses how to perform active information gathering when the true dynamics are unknown, by decoupling information costs from modeling choices through a modular framework. It introduces a directed-information based cost $J_{info}$, $J_{info}(oldsymbol{o}, oldsymbol{u}, oldsymbol{ o}) = -I(oldsymbol{ o} ightarrow oldsymbol{o} parallel oldsymbol{u})$, which remains valid across diverse dynamics models, belief updaters, and observation models, and shows that the traditional mutual-information cost is a special case. It also establishes a direct link between information gathering and parameter learning in linearized Bayesian estimation via Extended Kalman Filter updates, providing theoretical justification for MI-based active learning. Through experiments on single- and multi-agent, linear and nonlinear dynamics, the framework demonstrates that higher information-gathering weights improve parameter convergence and generalization, validating both the theory and its practical applicability.

Abstract

An agent operating in an unknown dynamical system must learn its dynamics from observations. Active information gathering accelerates this learning, but existing methods derive bespoke costs for specific modeling choices: dynamics models, belief update procedures, observation models, and planners. We present a unifying framework that decouples these choices from the information-gathering cost by explicitly exposing the causal dependencies between parameters, beliefs, and controls. Using this framework, we derive a general information-gathering cost based on Massey's directed information that assumes only Markov dynamics with additive noise and is otherwise agnostic to modeling choices. We prove that the mutual information cost used in existing literature is a special case of our cost. Then, we leverage our framework to establish an explicit connection between the mutual information cost and information gain in linearized Bayesian estimation, thereby providing theoretical justification for mutual information-based active learning approaches. Finally, we illustrate the practical utility of our framework through experiments spanning linear, nonlinear, and multi-agent systems.

Generalized Information Gathering Under Dynamics Uncertainty

TL;DR

This paper addresses how to perform active information gathering when the true dynamics are unknown, by decoupling information costs from modeling choices through a modular framework. It introduces a directed-information based cost , , which remains valid across diverse dynamics models, belief updaters, and observation models, and shows that the traditional mutual-information cost is a special case. It also establishes a direct link between information gathering and parameter learning in linearized Bayesian estimation via Extended Kalman Filter updates, providing theoretical justification for MI-based active learning. Through experiments on single- and multi-agent, linear and nonlinear dynamics, the framework demonstrates that higher information-gathering weights improve parameter convergence and generalization, validating both the theory and its practical applicability.

Abstract

An agent operating in an unknown dynamical system must learn its dynamics from observations. Active information gathering accelerates this learning, but existing methods derive bespoke costs for specific modeling choices: dynamics models, belief update procedures, observation models, and planners. We present a unifying framework that decouples these choices from the information-gathering cost by explicitly exposing the causal dependencies between parameters, beliefs, and controls. Using this framework, we derive a general information-gathering cost based on Massey's directed information that assumes only Markov dynamics with additive noise and is otherwise agnostic to modeling choices. We prove that the mutual information cost used in existing literature is a special case of our cost. Then, we leverage our framework to establish an explicit connection between the mutual information cost and information gain in linearized Bayesian estimation, thereby providing theoretical justification for mutual information-based active learning approaches. Finally, we illustrate the practical utility of our framework through experiments spanning linear, nonlinear, and multi-agent systems.
Paper Structure (19 sections, 1 theorem, 30 equations, 6 figures, 2 algorithms)

This paper contains 19 sections, 1 theorem, 30 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. A Modular Framework for Sequential Decision-Making Under Dynamics Uncertainty
  4. Execution
  5. Online Dynamics Parameter Learning
  6. Planning
  7. A General Information-Gathering Cost for Dynamics Uncertainty
  8. Mutual Information
  9. Directed Information
  10. The Directed Information Cost
  11. Connection to Mutual Information Cost
  12. An Explicit Connection Between Information Gathering and Dynamics Parameter Learning
  13. Experiments
  14. Hypothesis 1: Information gathering accelerates parameter learning and reduces uncertainty compared with random actions or passive learning, where the agent does not actively seek information-rich observations.
  15. Hypothesis 2: Information gathering produces learned models that generalize better to unseen data.
  16. ...and 4 more sections

Key Result

theorem thmcountertheorem

Under the following assumptions: the directed information cost eq:final_directed_cost reduces to eq:mutual_info_cost, which is the sum of conditional mutual informations between the starting parameter belief $\boldsymbol{{\vartheta}}_t$ and the states $\hat{\mathbf{x}}_{i}, \forall t \leq i \leq t + T$, i.e., $\sum_{i = t}^{t + T - where $\hat{\mathbf{S}}_i = \hat{\mathbf{F}}_i \boldsymbol{\bolds

Figures (6)

  • Figure 1: Causal graph for states $\mathbf{x}_i$, observations $\mathbf{o}_i$, controls $\mathbf{u}_i$, parameters $\boldsymbol{\theta}_i$, beliefs $\boldsymbol{{\vartheta}}_i$, and noise $\boldsymbol{\epsilon}_i$. Planning is done at time $t$ for a planning horizon $T$. Zigzag lines denote axis breaks. A variable with a hat denotes a prediction made during planning. Red lines emphasize the causal dependencies between states and beliefs during prediction: since the true parameters are not available, the states must be predicted using the latest dynamics parameters belief.
  • Figure 2: Double integrator
  • Figure 3: Damped pendulum
  • Figure 4: PE with LQR policy
  • Figure 5: PE with differentiable MPC
  • ...and 1 more figures

Theorems & Definitions (2)

  • theorem thmcountertheorem: Mutual Information Cost as a Special Case of the Directed Information Cost
  • proof