Memory-DD: A Low-Complexity Dendrite-Inspired Neuron for Temporal Prediction Tasks

TL;DR

The paper tackles temporal prediction under resource constraints by introducing Memory-DD, a low-complexity dendrite-inspired recurrent cell. Memory-DD uses two dendrite-inspired neuron groups with Hadamard-product gating and a memory update mechanism to capture temporal dependencies with minimal parameters. Across 18 UCR classification datasets it achieves 89.41% accuracy, exceeding LSTM by 4.25% while using about half the parameters and 27.7% fewer FLOPs; on 9 regression datasets it matches LSTM with substantially fewer parameters. Ablation studies confirm the essential roles of the Hadamard product, residual connections, and weight sharing, supporting Memory-DD as a practical, biologically inspired solution for edge-temporal tasks.

Abstract

Dendrite-inspired neurons have been widely used in tasks such as image classification due to low computational complexity and fast inference speed. Temporal data prediction, as a key machine learning task, plays a key role in real-time scenarios such as sensor data analysis, financial forecasting, and urban traffic management. However, existing dendrite-inspired neurons are mainly designed for static data. Studies on capturing dynamic features and modeling long-term dependencies in temporal sequences remain limited. Efficient architectures specifically designed for temporal sequence prediction are still lacking. In this paper, we propose Memory-DD, a low-complexity dendrite-inspired neuron model. Memory-DD consists of two dendrite-inspired neuron groups that contain no nonlinear activation functions but can still realize nonlinear mappings. Compared with traditional neurons without dendritic functions, Memory-DD requires only two neuron groups to extract logical relationships between features in input sequences. This design effectively captures temporal dependencies and is suitable for both classification and regression tasks on sequence data. Experimental results show that Memory-DD achieves an average accuracy of 89.41% on 18 temporal classification benchmark datasets, outperforming LSTM by 4.25%. On 9 temporal regression datasets, it reaches comparable performance to LSTM, while using only 50% of the parameters and reducing computational complexity (FLOPs) by 27.7%. These results demonstrate that Memory-DD successfully extends the low-complexity advantages of dendrite-inspired neurons to temporal prediction, providing a low-complexity and efficient solution for time-series data processing.

Figures (4)

• Figure 1: The overall architecture of the proposed Memory-DD model. (a) The biological neuron, which provides the biological inspiration for our model by integrating signals through its dendrites. (b) The architecture of a basic Dendritic Neuron model, which utilizes a dendritic layer to perform non-linear computation via the Hadamard product and incorporates a residual connection. (c) The detailed structure of the core Memory-DD Cell, which is composed of a linear layer and two primary functional modules. Dendritic-inspired neuron group 1 serves as the memory update module, updating the cell state to $C_t$ by performing a dendritic logical operation (Hadamard product) between the previous cell state $C_{t-1}$ and an intermediate state $D_t$ (generated from the current input $X_t$ and the previous hidden state $H_{t-1}$). Dendritic-inspired neuron group 2 acts as the decision-making module, generating the new hidden state $H_t$ based on the updated cell state $C_t$ and the intermediate state $D_t$. (d) The unrolled architecture of the complete Memory-DD model for processing sequential data. The model processes an input sequence step-by-step, and the final hidden state is passed through a Layer Normalization and a fully connected layer to produce the final prediction.
• Figure 2: Comparison of model complexity for the baseline models and Memory-DD (Ours). (a) The number of parameters (in millions). (b) The computational cost in Giga Floating Point Operations (G-FLOPs).
• Figure 3: The trade-off between average accuracy and model complexity. (a) Left figure: Average accuracy plotted against the number of parameters (M). (b) Right figure: Average accuracy plotted against the computational cost in Giga Floating Point Operations (G-FLOPs). The top-left corner represents the ideal balance of high accuracy and low complexity.
• Figure 4: Model Performance, Parameters, and FLOPS Comparison for LSTM, GRU, BiLSTM, and Memory-DD models.