LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction

TL;DR

This work tackles the problem of long-horizon sequence prediction with frozen LLMs that lack mutable memory. It introduces LLM-as-RNN, a framework that maintains a bounded natural-language memory state $h_t$ within a fixed token budget $\lambda$ and updates it at each step via a feedback signal $e_t$, enabling online learning without parameter updates. The approach formalizes recurrent inference for LLMs, proposes a three-step memory mechanism (Contextualization, Reflection, Memory Update), and demonstrates strong gains across healthcare (MIMIC-IV), meteorology (Weather), and finance (S&P 500) using multiple backbones (Llama, Gemma, GPT). It also emphasizes interpretable, human-readable learning traces and discusses practical considerations, limitations, and potential risks in deploying memory-updating LLMs for real-world sequential tasks.

Abstract

Large language models are strong sequence predictors, yet standard inference relies on immutable context histories. After making an error at generation step t, the model lacks an updatable memory mechanism that improves predictions for step t+1. We propose LLM-as-RNN, an inference-only framework that turns a frozen LLM into a recurrent predictor by representing its hidden state as natural-language memory. This state, implemented as a structured system-prompt summary, is updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates. Under a fixed token budget, LLM-as-RNN corrects errors and retains task-relevant patterns, effectively performing online learning through language. We evaluate the method on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT model families. LLM-as-RNN significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average, while producing interpretable, human-readable learning traces absent in standard context accumulation.

Figures (5)

• Figure 1: Illustrative comparison. (a) Simple LLM lacks memory. (b) LLM with Context suffers from growing input size. (c) LLM-as-RNN uses an iterative memory state to summarize historical information from evaluating outputs.
• Figure 2: Overview of LLM-as-RNN framework. At each time step, the system fuses the previous memory state with new input to generate a response, evaluates that response to create a feedback signal, and then updates the natural language memory state to guide future interactions.
• Figure 3: Temporal dynamics across iterative timesteps ($t{=}1\ldots5$) for three datasets. As feedback-driven memory updates accumulate, the performance increase.
• Figure 4: Scaling across backbone families. Performance vs. model size (B params, log-scale) for Zero-shot, FHC, MemPrompt, and LLM-as-RNN. Rows: Llama/Gemma/GPT backbones; columns: MIMIC Acc@1, Weather Align, S&P 500 MSE. LLM-as-RNN yields consistent gains across sizes and families.
• Figure 5: Transition heatmap of primary-diagnosis correctness between visits. Rows indicate correctness at time $t$ and columns indicate correctness at time $t{+}1$.