Path Integral Bottleneck: An Algorithm-Agnostic Framework of Computation and Control

TL;DR

The paper introduces the Path Integral Bottleneck (PI-IB), a framework that unifies information-theoretic (IB) and stochastic optimal control (PI) perspectives to quantify the compute effort required to realize a given closed-loop trajectory across diverse compute platforms. By decomposing compute cost into per-timestep information bottlenecks and ground-truth PI-based cost weights, PI-IB enables algorithm-agnostic comparisons of control performance versus computational expense. The key contributions include a continuous-variable (Gaussian-prior) derivation linking IB to linear encoders and LQR, a discrete-variable formulation, and a cart-pole simulation demonstrating distinct compute-control tradeoffs across balancing and swing-up tasks. The framework provides a principled way to analyze and compare complex controllers, including biological implementations, without relying on specific control laws or hardware details. This work advances cross-platform, performance-computation tradeoff analysis in nonlinear, potentially nonlinear, control systems.

Abstract

Executing a control sequence requires computation. While this is a simple observation, developing a framework that relates a controller's required computation to its ability to successfully control a system (e.g. lower control cost) is challenging, especially when the controller appears on alternative compute platforms (e.g. biological neural networks). More specifically, we want a framework where, given an observed closed-loop trajectory, we can quantify the computation effort needed to produce that trajectory. To enable effective comparisons of closed-loop systems across alternative compute platforms, we present the Path Integral Bottleneck (PI-IB), a method to produce an analytical, algorithm-agnostic description of the compute-control relationship. With the PI-IB framework, we can plot tradeoffs between performance and computation effort for any given plant description and control cost function. Simulations of the cart-pole reveal fundamental control-compute tradeoffs, exposing regions where the task performance-per-compute is higher than others.

Figures (8)

• Figure 1: Two probability distributions over control sequences: $P_{IB}(U)$ through the Information Bottleneck and $P_{PI}(U)$ through the Path Integral. Neither impose an interpretation on the probabilities. a) $P_{IB}(U)$ is constrained by the bottleneck size, which is the mutual information $I(X;Y)$. $X$ is not necessarily a state space vector (e.g. images). b) $P_{PI}(U)$ is constrained by a cost function $\mathcal{L}(x(t), u(t))$. Here, $x$ is a state space vector.
• Figure 2: A distribution $P(X)$'s shape changes its entropy and the random variable's predictability. The parameter $\lambda > 0$ from the PI formulation changes how flat or peaked the distribution is.
• Figure 3: The IB and PI affect compute cost in opposite directions. Optimizing the IB problem ($\min I(X;Y)$) lowers the compute cost, while optimizing the PI problem ($\min \mathcal{S}$) raises the compute cost. The parameter $\beta$ from the IB problem (\ref{['eqn:F_IB']}) determines which side is prioritized.
• Figure 4: Results of cartpole control with MPPI, with starting angles 5-180$^{\circ}$. $0^{\circ}$ is the balance point. All controls successfully drive the angle to the balance point. Transitioning from the balancing task to the swing-up task is visually distinct. We name the three distinct regions "Balancing" (5-60$^{\circ}$), "Swing-up A" (65-120$^{\circ}$), "Swing-up B" (125-180$^{\circ}$). The discrete timestep for the control is .03s.
• Figure 5: Random samples (7188 shown) of viable cart-pole control across initial angles $5-180^{\circ}$. Points are sparser towards the envelope and denser towards towards the center. Some timesteps have a narrower range than others. This emergent shape is evidence of a sophisticated controller.