From Embedding to Control: Representations for Stochastic Multi-Object Systems

TL;DR

The paper addresses the challenge of modeling and controlling stochastic nonlinear multi-object systems with non-uniform interactions and random topologies. It introduces Graph Controllable Embeddings (GCE), which embed conditional distributions $P(O_t|A_t,H_t)$ into a RKHS to render dynamics linear in the embedding, enabling linear controllers. A mean-field based adaptive interaction model is developed to capture non-uniform neighbor influence with provable low sample complexity, and a graph neural network builds topology-aware kernel features for generalization to unseen graphs. Theoretical guarantees of existence and convergence accompany empirical results across robotics, physical systems, and power grids, showing improvements over baselines in both in-distribution and few-shot settings. This work enables scalable, data-efficient control of complex multi-object environments and highlights avenues for extending to richer relational structures.

Abstract

This paper studies how to achieve accurate modeling and effective control in stochastic nonlinear dynamics with multiple interacting objects. However, non-uniform interactions and random topologies make this task challenging. We address these challenges by proposing \textit{Graph Controllable Embeddings} (GCE), a general framework to learn stochastic multi-object dynamics for linear control. Specifically, GCE is built on Hilbert space embeddings, allowing direct embedding of probability distributions of controlled stochastic dynamics into a reproducing kernel Hilbert space (RKHS), which enables linear operations in its RKHS while retaining nonlinear expressiveness. We provide theoretical guarantees on the existence, convergence, and applicability of GCE. Notably, a mean field approximation technique is adopted to efficiently capture inter-object dependencies and achieve provably low sample complexity. By integrating graph neural networks, we construct data-dependent kernel features that are capable of adapting to dynamic interaction patterns and generalizing to even unseen topologies with only limited training instances. GCE scales seamlessly to multi-object systems of varying sizes and topologies. Leveraging the linearity of Hilbert spaces, GCE also supports simple yet effective control algorithms for synthesizing optimal sequences. Experiments on physical systems, robotics, and power grids validate GCE and demonstrate consistent performance improvement over various competitive embedding methods in both in-distribution and few-shot tests

Figures (15)

• Figure 1: Hilbert space embedding of conditional distributions. Left: Stochastic nonlinear dynamics as evolving conditional distributions $P(O_t|a_t, h_t)$ (red curves: probability distributions, black dots: realizations). Right: After embedding into RKHS, dynamics become linear under $\mathcal{C}_{O|AH}$ (red dots: expectations).
• Figure 2: An example illustration of the GCE framework. The robot with multiple interconnected objects is initialized on top of a thin pole. At time step $t$, the observation is embedded into an RKHS via characteristic feature maps. A mean field approximation is then applied: the history features are computed via an element-wise product to reduce computation, while the action features preserve their structure for optimization (see math in Equation \ref{['Meta-representation of controllable embedding']}). The predicted features are mapped back to predict the $(t+1)$-step observation, enabling the robot to stabilize on the pole.
• Figure 3:
• Figure 4: Control error and control cost for varying bandwidth $\sigma$ values in Rope and Soft environments. Results are reported as mean and standard deviation over multiple runs.
• Figure 5: Examples of the random parametrized shape of the rope system. The red dots in each sub-figure represent the control targets.

Theorems & Definitions (14)

• Definition 1: Hilbert Space Embedding of Conditional Distributions sriperumbudur2010hilbert
• Remark 1
• Proposition 1
• Proposition 2
• Remark 2
• Theorem a: Sample Complexity of $\mathbf{Hom+Mean}$
• Theorem b: Sample Complexity of $\mathbf{Hom}$
• Theorem c: Sample Complexity of $\mathbf{Dense}$
• Lemma 1: Matrix Bernstein Inequality vershynin2018hightropp2015introduction
• Lemma 2: Error Bound of Cross-Covariance Operator