Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts

Anwesan Pal, Karen Hovsepian, Tinghao Guo, Mengnan Zhao, Somendra Tripathi, Nikos Kanakaris, George Mihaila, Sumit Nigam

TL;DR

Long-context LLMs underperform on intricate, context-spanning QA tasks despite large context windows. Tagging-Augmented Generation (TAG) injects semantic XML-style tags into input contexts and prompts, guiding attention without model changes; TAG supports multiple tagging methods (LLM IE, classification prompts, spaCy NER) and a two-stage chunking-and-tagging pipeline. Contributions include NoLiMa+ and NovelQA+ benchmarks with empirical validation showing up to 17% accuracy gains at 32K tokens and 2.9% for multi-hop reasoning, plus analysis of tagging strategies (TD vs TD+TC) and infrastructure-free prompting. This approach offers a scalable, low-latency alternative to retrieval-based long-context QA with broad applicability and minimal engineering overhead.

Abstract

Recent investigations into effective context lengths of modern flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. While approaches like retrieval-augmented generation (RAG) and chunk-based re-ranking attempt to mitigate this issue, they are sensitive to chunking, embedding and retrieval strategies and models, and furthermore, rely on extensive pre-processing, knowledge acquisition and indexing steps. In this paper, we propose Tagging-Augmented Generation (TAG), a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios, without degrading and altering the integrity and composition of retrieved documents. We validate our hypothesis by augmenting two challenging and directly relevant question-answering benchmarks -- NoLima and NovelQA -- and show that tagging the context or even just adding tag definitions into QA prompts leads to consistent performance gains over the baseline -- up to 17% for 32K token contexts, and 2.9% in complex reasoning question-answering for multi-hop queries requiring knowledge across a wide span of text. Additional details are available at https://sites.google.com/view/tag-emnlp.

Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts

TL;DR

Long-context LLMs underperform on intricate, context-spanning QA tasks despite large context windows. Tagging-Augmented Generation (TAG) injects semantic XML-style tags into input contexts and prompts, guiding attention without model changes; TAG supports multiple tagging methods (LLM IE, classification prompts, spaCy NER) and a two-stage chunking-and-tagging pipeline. Contributions include NoLiMa+ and NovelQA+ benchmarks with empirical validation showing up to 17% accuracy gains at 32K tokens and 2.9% for multi-hop reasoning, plus analysis of tagging strategies (TD vs TD+TC) and infrastructure-free prompting. This approach offers a scalable, low-latency alternative to retrieval-based long-context QA with broad applicability and minimal engineering overhead.

Abstract

Recent investigations into effective context lengths of modern flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. While approaches like retrieval-augmented generation (RAG) and chunk-based re-ranking attempt to mitigate this issue, they are sensitive to chunking, embedding and retrieval strategies and models, and furthermore, rely on extensive pre-processing, knowledge acquisition and indexing steps. In this paper, we propose Tagging-Augmented Generation (TAG), a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios, without degrading and altering the integrity and composition of retrieved documents. We validate our hypothesis by augmenting two challenging and directly relevant question-answering benchmarks -- NoLima and NovelQA -- and show that tagging the context or even just adding tag definitions into QA prompts leads to consistent performance gains over the baseline -- up to 17% for 32K token contexts, and 2.9% in complex reasoning question-answering for multi-hop queries requiring knowledge across a wide span of text. Additional details are available at https://sites.google.com/view/tag-emnlp.
Paper Structure (19 sections, 12 figures, 2 tables)

This paper contains 19 sections, 12 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Proposed Approach
  4. Experiments
  5. Conclusion
  6. Appendix
  7. Background and Related Work
  8. Long-Context Challenges in LLMs
  9. Input Tagging and Modular Prompt Conditioning
  10. Datasets
  11. NoLiMa
  12. NovelQA
  13. NoLiMa+ Dataset Creation Pipeline
  14. Prompts Used
  15. User prompts
  16. ...and 4 more sections

Figures (12)

  • Figure 1: High-level diagram comparing key steps in (a) traditional long-document QA and (b) the proposed tagging-augmented QA. The diagram highlights the plug-and-play use of imported, domain-specific taggers in an agentic flow. Our framework involves semantically enriching context by means of tagging. Furthermore, we also augment the prompt with an imported list of semantic and entity tag names and definitions to guide the LLM towards recognizing such semantic entities.
  • Figure 2: Overview of the proposed TAG framework. Input text is first segmented into de-duplicated chunks using configurable strategies (e.g., sentence-level, paragraph-level, or semantic chunking). These chunks are processed through tagging modules, producing XML-style semantic annotations that preserve document structure while providing explicit attention guidance for downstream tasks. Finally, we have the enriched documents with tags embedded within the text.
  • Figure 3: User prompt for NoLima+ dataset.
  • Figure 4: User prompt for NovelQA+ dataset.
  • Figure 5: System prompt for untagged context with tag definition in prompt.
  • ...and 7 more figures