Two-Layer Attention Optimization for Bimanual Coordination

TL;DR

The paper tackles bimanual coordination under attention constraints by proposing a two-layer optimization: an upper-layer attention trajectory $\mathbf{q}$ that is constrained by a hyperbolic feasible region, and two lower-layer per-hand controllers that track the resulting trajectories. The method clarifies how coordination and attention tradeoffs emerge, and demonstrates that incorporating an explicit attention layer enables the agent to reduce overall attention and control effort in a Pong rally. Key contributions include the formulation of tracking and coordination attention problems, a gradient-enabled interior-point approach for the upper layer, and a detailed simulation study of how asymmetry and centering influence performance. The work has practical implications for bio-inspired robotics and human-machine interaction, offering a scalable framework to reason about sensorimotor resource allocation in coordinated tasks.

Abstract

Bimanual tasks performed by human agents present unique optimal control considerations compared to cyberphysical agents. These considerations include minimizing attention, distributing attention across two isolated hands, and coordinating the two hands to reach a broader goal. In this work, we propose a two-layer controller that captures these considerations. The upper layer solves an attention distribution problem, while the two lower layer controllers (one per hand) tracks a trajectory using the solution given by the upper layer. We introduce a formulation of the attention controller where attention is a vector that is bound within a hyperbolic feasible region, which is determined by specifications of the task the lower layer controllers. This two-layer controller is used to optimize a single-player game of pong, where the agent must rally the ball between two paddles for as long as possible. We find that adding an attention layer on top of the lower controllers allows the agent to coordinate the left and right hands, which minimizes attention and control effort over the course of the rallying task.

Figures (8)

• Figure 1: Block diagram of two-layer optimization. Variables from Table \ref{['tab:variables']}.
• Figure 2: Diagram of the pong problem with some of the states labelled. Superscripts $p$ and $v$ indicate position and velocity respectively. Subscripts $L$ and $R$ indicate left and right respectively.
• Figure 3: Two strategies for reducing attention. Attention can be limited when the ball is far away from the paddle. Attention can also be saved by manipulating the ball's reflection angle towards the opposite paddle.
• Figure 4: Solving (\ref{['eqn:position_problem']}) with gradient descent, where q$_p$ is restricted to two dimensions ($N$ and $N-1$). This shows that the constraint is a convex hyperbola in the positive orthant. The value at $N-1$ is allowed to be much lower than the value at $N$.
• Figure 5: Attention per timestep ($q_p[t]$) after solving (\ref{['eqn:position_problem']}). Hard tasks have a larger standard deviation $\sigma$ across time, indicating an attention window that extends further back. In the Easy task, fixing the attention values to 0 for $t=2...8$ does not change the solution. The total attention costs $\mathcal{J}_{pA}$ for the Hard and Easy task are 39748 and 192 respectively.