SigGate: Enhancing Recurrent Neural Networks with Signature-Based Gating Mechanisms

TL;DR

This work addresses the bottleneck of local-context gating in recurrent networks by introducing path signatures as learnable gates. By replacing the forget gate in LSTM and the reset gate in GRU with signature-based computations, SigLSTM and SigGRU leverage the full history of input trajectories to regulate memory. Empirical results on cryptocurrency time-series forecasting tasks show consistent improvements over traditional architectures, with task-dependent gains and a notable increase in computational cost. The approach opens practical avenues for enhanced time-series modeling and provides open-source tools for reproducibility.

Abstract

In this paper, we propose a novel approach that enhances recurrent neural networks (RNNs) by incorporating path signatures into their gating mechanisms. Our method modifies both Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures by replacing their forget and reset gates, respectively, with learnable path signatures. These signatures, which capture the geometric features of the entire path history, provide a richer context for controlling information flow through the network's memory. This modification allows the networks to make memory decisions based on the full historical context rather than just the current input and state. Through experimental studies, we demonstrate that our Signature-LSTM (SigLSTM) and Signature-GRU (SigGRU) models outperform their traditional counterparts across various sequential learning tasks. By leveraging path signatures in recurrent architectures, this method offers new opportunities to enhance performance in time series analysis and forecasting applications.