Titans Revisited: A Lightweight Reimplementation and Critical Analysis of a Test-Time Memory Model
Gavriel Di Nepi, Federico Siciliano, Fabrizio Silvestri
TL;DR
Titans Revisited resolves reproducibility gaps around Titans by delivering a lightweight, explicit reimplementation and evaluating memory-augmented inference across NLP, time-series, and recommendation tasks. It clarifies design ambiguities, implements multiple strategies for unresolved choices, and demonstrates that neural memory consistently enriches information beyond attention-only baselines, though gains depend on chunking and task domain. The study shows memory helps bridge discontinuous segments (e.g., improved MRR on MovieLens and competitive MLM results) but also reveals limitations of memory updates when the backbone is frozen, underscoring the need for coordinated memory-backsone adaptation. Overall, the work highlights the potential of test-time memory for long-context reasoning while outlining practical challenges that must be overcome to realize robust, generalizable TTL improvements.
Abstract
By the end of 2024, Google researchers introduced Titans: Learning at Test Time, a neural memory model achieving strong empirical results across multiple tasks. However, the lack of publicly available code and ambiguities in the original description hinder reproducibility. In this work, we present a lightweight reimplementation of Titans and conduct a comprehensive evaluation on Masked Language Modeling, Time Series Forecasting, and Recommendation tasks. Our results reveal that Titans does not always outperform established baselines due to chunking. However, its Neural Memory component consistently improves performance compared to attention-only models. These findings confirm the model's innovative potential while highlighting its practical limitations and raising questions for future research.