Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unified Implementations of Recurrent Neural Networks in Multiple Deep Learning Frameworks

Francesco Martinuzzi

TL;DR

The paper tackles the lack of centralized, cross-framework RNN implementations by introducing three open-source libraries—torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl—that unify recurrent cells, layers, and wrappers across PyTorch, Flux, and Lux with consistent APIs. They collectively offer around 30 diverse cells, including gated, physics-inspired, and alternative integration designs, enabling easy experimentation and cross-framework comparison. The libraries are MIT-licensed, well-documented, and CI-tested, supporting reliable integration into existing workflows. This work lowers barriers to exploring novel RNN architectures and sets the stage for benchmarks and broader adoption in academic research.

Abstract

Recurrent neural networks (RNNs) are a cornerstone of sequence modeling across various scientific and industrial applications. Owing to their versatility, numerous RNN variants have been proposed over the past decade, aiming to improve the modeling of long-term dependencies and to address challenges such as vanishing and exploding gradients. However, no central library is available to test these variations, and reimplementing diverse architectures can be time-consuming and error-prone, limiting reproducibility and exploration. Here, we introduce three open-source libraries in Julia and Python that centralize numerous recurrent cell implementations and higher-level recurrent architectures. torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl offer a consistent framework for constructing and extending RNN models, providing built-in mechanisms for customization and experimentation. All packages are available under the MIT license and actively maintained on GitHub.

Unified Implementations of Recurrent Neural Networks in Multiple Deep Learning Frameworks

TL;DR

The paper tackles the lack of centralized, cross-framework RNN implementations by introducing three open-source libraries—torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl—that unify recurrent cells, layers, and wrappers across PyTorch, Flux, and Lux with consistent APIs. They collectively offer around 30 diverse cells, including gated, physics-inspired, and alternative integration designs, enabling easy experimentation and cross-framework comparison. The libraries are MIT-licensed, well-documented, and CI-tested, supporting reliable integration into existing workflows. This work lowers barriers to exploring novel RNN architectures and sets the stage for benchmarks and broader adoption in academic research.

Abstract

Recurrent neural networks (RNNs) are a cornerstone of sequence modeling across various scientific and industrial applications. Owing to their versatility, numerous RNN variants have been proposed over the past decade, aiming to improve the modeling of long-term dependencies and to address challenges such as vanishing and exploding gradients. However, no central library is available to test these variations, and reimplementing diverse architectures can be time-consuming and error-prone, limiting reproducibility and exploration. Here, we introduce three open-source libraries in Julia and Python that centralize numerous recurrent cell implementations and higher-level recurrent architectures. torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl offer a consistent framework for constructing and extending RNN models, providing built-in mechanisms for customization and experimentation. All packages are available under the MIT license and actively maintained on GitHub.
Paper Structure (7 sections, 1 figure)

This paper contains 7 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Software Overview
  3. Installation
  4. Models
  5. Modularity and Extensibility
  6. Code Quality
  7. Conclusion and Outlook

Figures (1)

  • Figure 1: Structure of recurrent components in the recurrent libraries. Each library defines recurrent cells and corresponding layers, which can be extended using wrappers. Single, double, and custom recurrent architectures share a unified interface, while framework-specific differences (e.g., in torchrecurrent) are indicated with dashed arrows.