Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Resolving the Robustness-Precision Trade-off in Financial RAG through Hybrid Document-Routed Retrieval

Zhiyuan Cheng, Longying Lai, Yue Liu

Abstract

Retrieval-Augmented Generation (RAG) systems for financial document question answering typically follow a chunk-based paradigm: documents are split into fragments, embedded into vector space, and retrieved via similarity search. While effective in general settings, this approach suffers from cross-document chunk confusion in structurally homogeneous corpora such as regulatory filings. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices the precision of targeted chunk retrieval. We identify this robustness-precision trade-off through controlled evaluation on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve this trade-off, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk-based retrieval scoped to the identified document(s). HDRR eliminates cross-document confusion while preserving targeted chunk precision. Experimental results demonstrate that HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a failure rate of only 6.4%, a correctness rate of 67.7% (+18.7 pp over CBR), and a perfect-answer rate of 20.1% (+6.3 pp over CBR, +11.6 pp over SFR). HDRR resolves the trade-off by simultaneously achieving the lowest failure rate and the highest precision across all five experimental groups.

Resolving the Robustness-Precision Trade-off in Financial RAG through Hybrid Document-Routed Retrieval

Abstract

Retrieval-Augmented Generation (RAG) systems for financial document question answering typically follow a chunk-based paradigm: documents are split into fragments, embedded into vector space, and retrieved via similarity search. While effective in general settings, this approach suffers from cross-document chunk confusion in structurally homogeneous corpora such as regulatory filings. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices the precision of targeted chunk retrieval. We identify this robustness-precision trade-off through controlled evaluation on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve this trade-off, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk-based retrieval scoped to the identified document(s). HDRR eliminates cross-document confusion while preserving targeted chunk precision. Experimental results demonstrate that HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a failure rate of only 6.4%, a correctness rate of 67.7% (+18.7 pp over CBR), and a perfect-answer rate of 20.1% (+6.3 pp over CBR, +11.6 pp over SFR). HDRR resolves the trade-off by simultaneously achieving the lowest failure rate and the highest precision across all five experimental groups.

Paper Structure

This paper contains 65 sections, 13 equations, 2 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Proposed Approach
  4. Contributions
  5. Paper Organization
  6. Related Work
  7. Retrieval-Augmented Generation
  8. Hybrid Retrieval and Reranking
  9. Financial Document Analysis
  10. LLM Structured Output
  11. Long-Context LLMs and Document-Level Reasoning
  12. Document Routing and Classification
  13. Multi-Stage and Cascaded Retrieval
  14. Gap Analysis
  15. Problem Formulation
  16. ...and 50 more sections

Figures (2)

  • Figure 1: Chunk-Based RAG (CBR) pipeline. The offline phase indexes documents into dual stores (FTS5 for keyword search, FAISS for semantic search). The online phase retrieves, fuses, reranks, and generates.
  • Figure 2: Hybrid Document-Routed Retrieval (HDRR) pipeline. Stage 1 routes the query to the correct document via LLM structured output. Stage 2 performs scoped hybrid search (FTS + semantic + RRF + reranking) restricted to the routed document's chunks. If routing fails, the system falls back to full-corpus retrieval.

Theorems & Definitions (2)

  • definition 1: Naming Regularity
  • definition 2: Structural Homogeneity