Parallel Delayed Memory Units for Enhanced Temporal Modeling in Biomedical and Bioacoustic Signal Analysis
Pengfei Sun, Wenyu Jiang, Paul Devos, Dick Botteldooren
TL;DR
The paper addresses the challenge of efficient temporal modeling in bio-signal and audio domains where data may be limited. It introduces the Parallel Delayed Memory Unit (PDMU), a delay-gated extension of the Legendre Memory Unit that enhances short-term memory interactions while enabling parallel training and real-time inference; it further proposes Bi-PDMU, EPDMU, and Spiking DMU variants to optimize for bidirectionality and energy efficiency. Through experiments on cough, EEG, SHD, speech, and permuted MNIST benchmarks, PDMU demonstrates improved memory capacity and competitive accuracy with substantial training-time gains and reduced parameter overhead. The results indicate strong potential for edge-enabled, real-time bio-signal analysis and broader temporal sequence tasks, with broad applicability across neuromorphic and conventional architectures. Overall, PDMU offers a modular, plug-and-play approach to incorporate short-term temporal credit assignment into linear RNNs, enhancing performance without sacrificing parallelism or real-time causal inference.
Abstract
Advanced deep learning architectures, particularly recurrent neural networks (RNNs), have been widely applied in audio, bioacoustic, and biomedical signal analysis, especially in data-scarce environments. While gated RNNs remain effective, they can be relatively over-parameterised and less training-efficient in some regimes, while linear RNNs tend to fall short in capturing the complexity inherent in bio-signals. To address these challenges, we propose the Parallel Delayed Memory Unit (PDMU), a {delay-gated state-space module for short-term temporal credit assignment} targeting audio and bioacoustic signals, which enhances short-term temporal state interactions and memory efficiency via a gated delay-line mechanism. Unlike previous Delayed Memory Units (DMU) that embed temporal dynamics into the delay-line architecture, the PDMU further compresses temporal information into vector representations using Legendre Memory Units (LMU). This design serves as a form of causal attention, allowing the model to dynamically adjust its reliance on past states and improve real-time learning performance. Notably, in low-information scenarios, the gating mechanism behaves similarly to skip connections by bypassing state decay and preserving early representations, thereby facilitating long-term memory retention. The PDMU is modular, supporting parallel training and sequential inference, and can be easily integrated into existing linear RNN frameworks. Furthermore, we introduce bidirectional, efficient, and spiking variants of the architecture, each offering additional gains in performance or energy efficiency. Experimental results on diverse audio and biomedical benchmarks demonstrate that the PDMU significantly enhances both memory capacity and overall model performance.