Toward Neuronal Implementations of Delayed Optimal Control

TL;DR

The paper addresses how neural circuits implement delayed optimal controllers in sensorimotor systems. It analyzes a scalar neuromuscular model with a delayed linear-quadratic regulator, derives the delayed controller transfer function $G(z)$, and maps three minimal realizations to stylized neurons under a delay-graph framework. Key contributions include showing that three minimal neural circuit configurations can implement any second-order delayed controller, developing a formal compatibility framework between controller structures and delay graphs, and demonstrating through simulations that identical control behavior can arise from distinct neural firing patterns. This work provides a constructive method to infer plausible neural circuits from observed behavior and delay constraints, enabling data-driven selection among multiple neural implementations.

Abstract

Animal sensorimotor behavior is frequently modeled using optimal controllers. However, it is unclear how the neural circuits within the animal's nervous system implement optimal controller-like behavior. In this work, we study the question of implementing a delayed linear quadratic regulator with linear dynamical "neurons" on a muscle model. We show that for any second-order controller, there are three minimal neural circuit configurations that implement the same controller. Furthermore, the firing rate characteristics of each circuit can vary drastically, even as the overall controller behavior is preserved. Along the way, we introduce concepts that bridge controller realizations to neural implementations that are compatible with known neuronal delay structures.

Figures (11)

• Figure 1: Behavior vs. implementation in neuroscience and control theory. While control theory offers explanations for behaviors seen in neuroscience experiments, it currently offers few explanations for neural circuitry (i.e., implementation) underlying these behaviors.
• Figure 2: (Left) The neuromuscular control system; the muscle is the 'plant', and the nervous system is the 'controller'. The Golgi tendon organ senses the muscle force and communicates this to the nervous system, which fires a motor neuron to induce muscle activation. (Right) Delay graph associated with this system. There is a delay of 1 timestep from $v_1$ (Golgi tendon organ) to $v_2$ (nervous system), and a delay of 1 timestep from $v_2$ to $v_3$ (muscle). No other communication paths exist between the three vertices.
• Figure 3: Open-loop system response of linear discrete-time neuromuscular model to a unit pulse input. Increasing motor neuron firing rate results in increased muscle force (with some transient dynamics); decreasing firing rate results in decreased force. Simulation parameters are given in Section \ref{['sec:simulations']}.
• Figure 4: Controller structure for the optimal delayed controller described by \ref{['eq:augmented_states']} and \ref{['eq:ifp_controller']}. Broken lines indicate one timestep of delay. This structure is not compatible with the specified communication delays.
• Figure 5: Closed-loop system response to a unit pulse disturbance. The control output (motor neuron firing rate) responds to the disturbance with a slight delay, and restores the force to equilibrium after the disturbance ends. Simulation parameters are given in Section \ref{['sec:simulations']}.

Theorems & Definitions (6)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Definition 6