Efficient Temporal-aware Matryoshka Adaptation for Temporal Information Retrieval

TL;DR

Temporal information retrieval in RAG systems is bottlenecked by retrievers that struggle to encode time-sensitive cues. The authors introduce Temporal-aware Matryoshka Representation Learning (TMRL), a plug-and-play method that adds a temporal subspace to Matryoshka embeddings via a LoRA-based temporal adapter and a temporal projector, trained with a temporal subspace contrastive loss and self-distillation. They also propose a temporal contrastive data augmentation pipeline using the Temporal Nobel Prize dataset and TimeQA to provide clean supervision. Across multiple TEMs and two benchmarks, TMRL improves temporal retrieval and downstream temporal RAG while enabling efficient accuracy–efficiency trade-offs with no additional inference overhead. This work advances scalable temporal retrieval by integrating temporal encoding directly into a single model and augmenting training data for robust supervision.

Abstract

Retrievers are a key bottleneck in Temporal Retrieval-Augmented Generation (RAG) systems: failing to retrieve temporally relevant context can degrade downstream generation, regardless of LLM reasoning. We propose Temporal-aware Matryoshka Representation Learning (TMRL), an efficient method that equips retrievers with temporal-aware Matryoshka embeddings. TMRL leverages the nested structure of Matryoshka embeddings to introduce a temporal subspace, enhancing temporal encoding while preserving general semantic representations. Experiments show that TMRL efficiently adapts diverse text embedding models, achieving competitive temporal retrieval and temporal RAG performance compared to prior Matryoshka-based non-temporal methods and prior temporal methods, while enabling flexible accuracy-efficiency trade-offs.

Figures (7)

• Figure 1: Effectiveness of the proposed augmented data pipeline and Temporal-aware Matryoshka Representation Learning (TMRL) on the Temporal RAG benchmark of Temporal Nobel Prize (TNP). The results show that poor retrieval limits RAG performance regardless of Qwen3-8B’s reasoning ability. Adapting Contriever with TMRL enables the RAG system to achieve competitive performance while reducing storage footprint and inference latency.
• Figure 2: Our Temporal-aware Matryoshka embeddings are obtained by jointly optimizing a LoRA-adapted text embedding model and a temporal projector $\mathcal{P}$. The projector $\mathcal{P}$ operates only on temporal tokens (Eq. \ref{['eq:temporal_tokens']}) to learn a compact and compatible $t$-dimensional temporal representation (Eq. \ref{['eq:projected_temporal_tokens_mean_pooling']}). These representations are used in the temporal subspace contrastive loss $\mathcal{L}_{\mathrm{Temp}}$ (Eq. \ref{['eq:temporal_query_infonce']}), a core component of TMRL (Eq. \ref{['eq:final_loss']}), to endow the Matryoshka embeddings with a dedicated temporal subspace, yielding the final embeddings defined in Eq. \ref{['eq:temporal_subspace']} and Eq. \ref{['eq:temporal_matryoshka']}.
• Figure 3: Comparison between the TNP dataset (left) and our augmented version (right). We follow a simple principle: split long paragraphs into shorter paragraphs, discard sentences with multiple temporal expressions, ensuring that each passage contains one temporal expression before augmenting positive and negative queries.
• Figure 4: Matryoshka embedding results on TNP. TMRL preserves general semantic performance on NQ while improving temporal retrieval, particularly nDCG@10, compared to the strong LoRA-based MRL baseline, which already outperforms LoRA fine-tuning and Ts-Retriever on full-dimensional embeddings.
• Figure 5: Ablation of the temporal subspace dimension $t$ and the temporal subspace coefficient $\alpha$. The results show the increase/decrease relative to the LoRA-based MRL baseline on TNP using contriever and bge.