Learning-based cognitive architecture for enhancing coordination in human groups

TL;DR

Problem: enhance coordination in human-avatar groups performing periodic motor tasks. Approach: a reinforcement-learning-based cognitive architecture trained in simulation on Kuramoto oscillator networks and deployed in real groups to adjust avatar frequency for improved synchronization, captured by the net order parameter $\langle r_{\mathrm{net}} \rangle$. Key contributions: analytical insights for small groups, a DQN-based CA trained on synthetic data, numerical validation showing robust synchronization gains, and preliminary experiments indicating seamless avatar integration with humans though with mixed improvements. Significance: offers a scalable, adaptive mechanism for group rehabilitation and sports training where avatars can dynamically support coordination in real-time.

Abstract

As interactions with autonomous agents-ranging from robots in physical settings to avatars in virtual and augmented realities-become more prevalent, developing advanced cognitive architectures is critical for enhancing the dynamics of human-avatar groups. This paper presents a reinforcement-learning-based cognitive architecture, trained via a sim-to-real approach, designed to improve synchronization in periodic motor tasks, crucial for applications in group rehabilitation and sports training. Extensive numerical validation consistently demonstrates improvements in synchronization. Theoretical derivations and numerical investigations are complemented by preliminary experiments with real participants, showing that our avatars can integrate seamlessly into human groups, often being indistinguishable from humans.

Figures (6)

• Figure 1: (Top) Example of phase reconstruction (purple) from a position signal (blue) through Algorithm \ref{['alg:phase_estimation']}; orange and gold lines are $A^{p > 0}(t)$ and $A^{p < 0}(t)$. (Bottom) Difference between phases estimated with the Hilbert transform (offline) and Algorithm \ref{['alg:phase_estimation']} (online).
• Figure 2: Enhancement of synchronization achieved by the CA. $n_\mathrm{p}=5$, connected in a ring. (a) without CA ($n_\mathrm{a} = 0$, $n = 5$); (b) with CA connected to all ($n_\mathrm{a} = 1$, $n = 6$). Each pixel is the average of $15$ simulations: in each, $\{\omega_i\}_{i \in \mathcal{I}_\mathrm{p}}$ drawn randomly in $[4 - \delta_{\omega}, 4 + \delta_{\omega}]$. $\theta_i(k=0) = \pi/2, \forall i$. $T = 100 \,\text{s}$.
• Figure 3: Optimality of the behavior of the CA. $n_\mathrm{p} = 7$, connected in a ring. $c$ and $\{\omega_i(k)\}_{i \in \mathcal{I}_\mathrm{p}}$ (time-varying, drawn from Gaussian distributions) taken from alderisio2017interaction. $\theta_i(k=0) = \pi/2, \forall i$. $T = 300\,\text{s}$. (Left) The blue line and shading are mean and standard deviation (simulations repeated $5$ times) obtained by an agent having frequency $\omega$ ($x$-axis) added to the group and connected to all; red dashed line and shading are obtained by a CA added and connected to all. (Top right) $\omega_\mathrm{a}(k)$ selected by the CA. (Bottom right) $\{\theta_i(k)\}_{i \in \mathcal{I}_\mathrm{p}}$ (gray) and $\theta_\mathrm{a}(k)$ (red) in the first (of $5$) simulation.
• Figure 4: Effect of node degree on synchronization performance and comparison between the CA and a naive avatar. $n_\mathrm{p} = 7$, connected in a ring. $c$ and $\{\omega_i(k)\}_{i \in \mathcal{I}_\mathrm{p}}$ (time-varying, drawn from Gaussian distributions) taken from alderisio2017interaction. $\theta_i(k=0) = \pi/2, \forall i$. $T = 100\,\text{s}$. Box plots are distributions over all possible arrangements of the CA/NA's edges, and $5$ simulations per case. Numbers in black are $p$-values of the null hypothesis that the mean performance of CA is not superior to that of NA (one-tailed $t$-tests). $\Delta \lambda_2 \coloneqq \lambda_2^{\mathrm{a}} - \lambda_2^{\neg\mathrm{a}}$, where $\lambda_2^{\mathrm{a}}$ and $\lambda_2^{\neg\mathrm{a}}$ are the algebraic connectivity when the avatar (CA/NA) is absent and present, respectively.
• Figure 5: (a) Experimental setup. (b) Chronos interface.