Utilizing Metadata for Better Retrieval-Augmented Generation

TL;DR

The paper tackles the challenge of retrieval-augmented generation (RAG) in highly structured, repetitive corpora by treating metadata as a first-class signal, not just a filter. It systematically compares metadata-as-text, dual-encoder unified embeddings, and late-fusion strategies using the RAGMATE-10K SEC filing dataset, showing that metadata integration consistently improves retrieval, with unified embeddings often matching or surpassing prefix-based methods while simplifying maintenance. Embedding-space analysis reveals that metadata tightens intra-document cohesion, reduces inter-document confusion, and increases score separability, clarifying why these methods work. The work contributes a modular framework for metadata integration, field-level ablations, and a publicly released dataset, offering practical guidance for deploying robust RAG systems in structured domains like finance and law.

Abstract

Retrieval-Augmented Generation systems depend on retrieving semantically relevant document chunks to support accurate, grounded outputs from large language models. In structured and repetitive corpora such as regulatory filings, chunk similarity alone often fails to distinguish between documents with overlapping language. Practitioners often flatten metadata into input text as a heuristic, but the impact and trade-offs of this practice remain poorly understood. We present a systematic study of metadata-aware retrieval strategies, comparing plain-text baselines with approaches that embed metadata directly. Our evaluation spans metadata-as-text (prefix and suffix), a dual-encoder unified embedding that fuses metadata and content in a single index, dual-encoder late-fusion retrieval, and metadata-aware query reformulation. Across multiple retrieval metrics and question types, we find that prefixing and unified embeddings consistently outperform plain-text baselines, with the unified at times exceeding prefixing while being easier to maintain. Beyond empirical comparisons, we analyze embedding space, showing that metadata integration improves effectiveness by increasing intra-document cohesion, reducing inter-document confusion, and widening the separation between relevant and irrelevant chunks. Field-level ablations show that structural cues provide strong disambiguating signals. Our code, evaluation framework, and the RAGMATE-10K dataset are publicly hosted.

Figures (7)

• Figure 1: Context@K and Title@K metric improvement in retrieval when using and not using metadata across query types (Dual Encoder Unified Embeddings)
• Figure 2: Conceptual overview of document and metadata embedding strategies for retrieval. Dotted arrows indicate query propagation.
• Figure 3: Comparative retrieval performance vs. plain baseline across query types using the Dual Encoder Unified Embedding approach.
• Figure 4: Comparative retrieval performance vs. plain baseline across query types using Metadata as Text (Prefix)
• Figure 5: Retrieval performance across metadata weight $\alpha$ for Dual encoder with late-fusion scoring. Metadata improves results when moderately weighted; full reliance on either content or metadata reduces performance.

Theorems & Definitions (3)

• proposition thmcounterproposition: Intra-document cohesion increases
• proposition thmcounterproposition: Inter-document confusion decreases
• proposition thmcounterproposition: Score variance increases