Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applications

Rishi Kalra, Zekun Wu, Ayesha Gulley, Airlie Hilliard, Xin Guan, Adriano Koshiyama, Philip Treleaven

TL;DR

HyPA-RAG tackles the limitations of LLMs in AI legal and policy tasks by introducing a query-complexity classifier for adaptive parameter tuning, and a hybrid retrieval stack that fuses dense, sparse, and knowledge-graph sources. Tested on NYC Local Law 144, the approach demonstrates improvements in retrieval fidelity, contextual precision, and overall correctness through a rigorous evaluation framework and targeted metrics. The work contributes a pragmatic, domain-adaptive RAG architecture, an MG-backed knowledge integration strategy, and comprehensive ablations highlighting the value of adaptivity in high-stakes legal contexts. These findings support scalable, accountable AI tooling for policy and legal decision-support while outlining ethical considerations and future directions for domain-aware RAG systems.

Abstract

Large Language Models (LLMs) face limitations in AI legal and policy applications due to outdated knowledge, hallucinations, and poor reasoning in complex contexts. Retrieval-Augmented Generation (RAG) systems address these issues by incorporating external knowledge, but suffer from retrieval errors, ineffective context integration, and high operational costs. This paper presents the Hybrid Parameter-Adaptive RAG (HyPA-RAG) system, designed for the AI legal domain, with NYC Local Law 144 (LL144) as the test case. HyPA-RAG integrates a query complexity classifier for adaptive parameter tuning, a hybrid retrieval approach combining dense, sparse, and knowledge graph methods, and a comprehensive evaluation framework with tailored question types and metrics. Testing on LL144 demonstrates that HyPA-RAG enhances retrieval accuracy, response fidelity, and contextual precision, offering a robust and adaptable solution for high-stakes legal and policy applications.

HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applications

TL;DR

HyPA-RAG tackles the limitations of LLMs in AI legal and policy tasks by introducing a query-complexity classifier for adaptive parameter tuning, and a hybrid retrieval stack that fuses dense, sparse, and knowledge-graph sources. Tested on NYC Local Law 144, the approach demonstrates improvements in retrieval fidelity, contextual precision, and overall correctness through a rigorous evaluation framework and targeted metrics. The work contributes a pragmatic, domain-adaptive RAG architecture, an MG-backed knowledge integration strategy, and comprehensive ablations highlighting the value of adaptivity in high-stakes legal contexts. These findings support scalable, accountable AI tooling for policy and legal decision-support while outlining ethical considerations and future directions for domain-aware RAG systems.

Abstract

Large Language Models (LLMs) face limitations in AI legal and policy applications due to outdated knowledge, hallucinations, and poor reasoning in complex contexts. Retrieval-Augmented Generation (RAG) systems address these issues by incorporating external knowledge, but suffer from retrieval errors, ineffective context integration, and high operational costs. This paper presents the Hybrid Parameter-Adaptive RAG (HyPA-RAG) system, designed for the AI legal domain, with NYC Local Law 144 (LL144) as the test case. HyPA-RAG integrates a query complexity classifier for adaptive parameter tuning, a hybrid retrieval approach combining dense, sparse, and knowledge graph methods, and a comprehensive evaluation framework with tailored question types and metrics. Testing on LL144 demonstrates that HyPA-RAG enhances retrieval accuracy, response fidelity, and contextual precision, offering a robust and adaptable solution for high-stakes legal and policy applications.
Paper Structure (42 sections, 8 figures, 9 tables)

This paper contains 42 sections, 8 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. System Design
  4. AI Legal and Policy Corpus
  5. Performance Evaluation
  6. Dataset Generation
  7. Evaluation Metrics
  8. Correctness Evaluation
  9. Chunking Method
  10. Query Complexity Classifier
  11. Training Data
  12. Model Training
  13. Classifier Results
  14. RAG System Architecture
  15. Parameter-Adaptive RAG (PA-RAG)
  16. ...and 27 more sections

Figures (8)

  • Figure 1: Hybrid Parameter Adaptive RAG (HyPA-RAG) System Diagram
  • Figure 2: Spearman Coefficient Comparison, showing the correlation between model performance and human evaluation.
  • Figure 3: RAG Evaluation Metrics for Sentence-Level, Semantic, and Pattern-Based Chunking Methods
  • Figure 4: Demo screenshots showing each key stage of the user experience.
  • Figure 5: Overall RAG Development Workflow Diagram
  • ...and 3 more figures