A Dynamic Systems Approach to Modelling Human-Machine Rhythm Interaction

TL;DR

The paper addresses modeling human rhythmic perception and synchronization using a cerebellum-inspired reservoir computing framework. It introduces a topology-preserving, wave-based reservoir built from a 2D-FDTD discretization with dual neuron types and tunable weights $c$ and $k$, trained with $W$, $W_{in}$, and $W_{out}$ to predict beats with horizon $Δt = n δt$. It adds a dynamical selection mechanism and fast adaptation for continual learning, plus closed-loop feedback and Wasserstein-distance–based customization to capture individual variability. The results show human-like meter perception and robust motor-auditory interaction across single- and multi-channel tasks, suggesting broad applicability to temporal cognitive tasks and brain-like dynamical modeling.

Abstract

In exploring the simulation of human rhythmic perception and synchronization capabilities, this study introduces a computational model inspired by the physical and biological processes underlying rhythm processing. Utilizing a reservoir computing framework that simulates the function of cerebellum, the model features a dual-neuron classification and incorporates parameters to modulate information transfer, reflecting biological neural network characteristics. Our findings demonstrate the model's ability to accurately perceive and adapt to rhythmic patterns within the human perceptible range, exhibiting behavior closely aligned with human rhythm interaction. By incorporating fine-tuning mechanisms and delay-feedback, the model enables continuous learning and precise rhythm prediction. The introduction of customized settings further enhances its capacity to stimulate diverse human rhythmic behaviors, underscoring the potential of this architecture in temporal cognitive task modeling and the study of rhythm synchronization and prediction in artificial and biological systems. Therefore, our model is capable of transparently modelling cognitive theories that elucidate the dynamic processes by which the brain generates rhythm-related behavior.

Figures (6)

• Figure 1: Illustration of a typical task for the model and the reservoir structure. Panel (a) shows a typical task: the model is primed with one rhythm (noted as visual here, referring to the human priming further on in this paper) while exposed to another rhythm based on the same meter; its task is to predict the beats of the primed rhythm and continue doing so after the primer fades. Panel (b) shows the proposed reservoir structure. It illustrates the components $p$, and $\mathbf{o}$ of the reservoir.
• Figure 2: The proposed reservoir demonstrates internal meter perception. (a) Comparison of the difference in inter-beat interval mean and standard deviation before and after adaptation of global parameters. (b) Comparison of the time offset ratio before and after adaptation of global parameters.
• Figure 3: Display of interaction procedure. (a) Human-computer interaction experiment. The upper part depicts the procedure of the experiment with people. During a learning phase, the participant is guided by a visual reference that gradually fades, while a different audio rhythm gradually fades in from a computer, along with a shared metronome. The participant is expected to tap the rhythm in sync with the visual reference, even after the latter has stopped. The lower part outlines the procedure for applying our model to a similar experimental setting. Given the model's proven metronome perception, the metronome is not included as an input. During learning, the input mirrors the human learning procedure, while the output weight undergoes fine-tuning to the dual input. After the visual reference stops, the model feeds back its own tapping. (b) Human interaction procedure. The upper part illustrates the model output weight updating procedure, which remains consistent with the human-computer interaction. The lower part demonstrates the customization added to the model's output weight. The model is affected by itself and the other channel to varying degrees.
• Figure 4: The proposed reservoir can interact with a computer similarly to a human. (a) Time offset ratios for both channels in all combinations are compared. (b) The effect of closed-loop on the continuation task for channel 1 in all samples. (c) Comparison of the distribution between the skip-one target inter-beat interval and the prediction's distribution for all samples. (d) Illustration of the skip-a-while prediction's inter-beat interval error ratio for all samples. (e) After channel 2 increases the inter-beat interval by 2%, a comparison of the prediction's inter-beat interval error ratio before and after the increase for all samples.
• Figure 5: Variation in Human Interaction with Identical Rhythms. Each subplot in a row represents a different participant interacting with the same rhythm combination. The subplots in the same row correspond to the same group of participants, illustrating their varied interactions with the given rhythm. Within each subplot, four groups of participants are shown alongside their respective models. Hollow circles indicate the beats dropped by both participants and models. The angle of each hollow circle denotes the phase shift relative to the cycle (6 * metronome inter-beat interval), while the distance from the center indicates the interval between consecutive beats.