Exploring the role of structure in a time constrained decision task

TL;DR

This work assesses how network topology affects performance in time-constrained decision tasks by using Echo State Networks with both randomized and structured variants. It introduces dual slow and fast pathways, plus a continuous topological reservoir (M*), and trains readouts via online reinforcement learning; the architectures are evaluated on a temporal two-arm bandit task with motor indirection and overlapping inputs. The main finding is that dual-pathway ESNs ($M^{2}$, $M^{3}$, $M^{*}$) outperform the single-reservoir baseline ($M^{0}$), with the deepest topology achieving $89.5\%$ accuracy compared to $74.0\%$ for the baseline, particularly when the best cue appears early. These results highlight the functional advantage of integrating slow and fast processing for handling late information and rapid adaptation, offering a basal ganglia-inspired blueprint for reservoir computing in time-sensitive decision tasks.

Abstract

The structure of the basal ganglia is remarkably similar across a number of species (often described in terms of direct, indirect and hyperdirect pathways) and is deeply involved in decision making and action selection. In this article, we are interested in exploring the role of structure when solving a decision task while avoiding to make any strong assumption regarding the actual structure. To do so, we exploit the echo state network paradigm that allows to solve complex task based on a random architecture. Considering a temporal decision task, the question is whether a specific structure allows for better performance and if so, whether this structure shares some similarity with the basal ganglia. Our results highlight the advantage of having a slow (direct) and a fast (hyperdirect) pathway that allows to deal with late information during a decision making task.

Figures (7)

• Figure 1: Top Model architecture with a motor output (direction of movement). The black arrows are fixed and the red is plastic. Bottom The red (worst) and blue (best) stimuli can have different onset/offset times and the reward is receives at a fixed time.
• Figure 2: Model are composed of one to several chained ESN, all connected to the readout, all receiving feedback from the output. M$^\mathbf{0}$: Regular ESN. M$^\mathbf{1}$: Dual pathway each made of a single ESN. M$^\mathbf{2}$: Dual pathway each made of two chained ESNs. M$^\mathbf{3}$: Dual pathway each made of three chained ESNs. M$^\mathbf{*}$: Dual pathway each made of one continuous ESN.
• Figure 3: Left Internal structure of the M$^\mathbf{*}$ model. The two pathways are shaded in blue (top) and green (bottom) and are completely segregated (no reciprocal connections between them). The input to the top pathway has ben shaded in darker blue while the output is not represented for clarity because output receives connection from virtually all units. The input to the bottom pathway is similar but not represented for clarity. Instead, a typical unit (dot) connection pattern is represented with red for incoming connections and blue for outgoing projections. Right The connection pattern of a unit is governed by an angle $\theta$ (ranging from 0 to 90), a fixed radius $r$ and a connection probability $P_c$. Top) $\theta=90^{\circ}$, $P_c=1.0$ Middle) $\theta=90^{\circ}$, $P_c=0.4$ Bottom) $\theta=70^{\circ}$, $P_c=1.0$
• Figure 4: The two stimuli $V_i$ are characterized by their respective onset ($t_i^+$) and offset ($t_i^-$) time. The time of decision $t_{reward}$ is fixed and constant across trials.
• Figure 5: Top Training process. The curves correspond to the percentage of successful choice using a moving average that takes the 50 last trials. Bottom The training process is categorized in two scenarios based on when the best cue appears. This enables to observe that the major difference in performances occurs when the best cue appears first.