Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SQuAI: Scientific Question-Answering with Multi-Agent Retrieval-Augmented Generation

Ines Besrour, Jingbo He, Tobias Schreieder, Michael Färber

TL;DR

SQuAI addresses hallucination and verification challenges in open-domain scientific QA by introducing a four-agent, multi-agent RAG framework that decomposes complex questions, employs hybrid sparse-dense retrieval, and enforces fine-grained inline citations with citation context. The system operates over the unarXive 2024 corpus to enable scalable, real-time QA across multiple scientific domains, and includes a dedicated evaluation benchmark of 1,000 QA-evidence triplets. Empirical results show up to a 12% improvement in Faithfulness and Relevance over a strong RAG baseline, driven by query decomposition, adaptive filtering, and rich citation grounding. The work provides a transparent, scalable platform for trustworthy scientific QA and offers a publicly available benchmark and interface to facilitate reproducibility and further research.

Abstract

We present SQuAI (https://squai.scads.ai/), a scalable and trustworthy multi-agent retrieval-augmented generation (RAG) framework for scientific question answering (QA) with large language models (LLMs). SQuAI addresses key limitations of existing RAG systems in the scholarly domain, where complex, open-domain questions demand accurate answers, explicit claims with citations, and retrieval across millions of scientific documents. Built on over 2.3 million full-text papers from arXiv.org, SQuAI employs four collaborative agents to decompose complex questions into sub-questions, retrieve targeted evidence via hybrid sparse-dense retrieval, and adaptively filter documents to improve contextual relevance. To ensure faithfulness and traceability, SQuAI integrates in-line citations for each generated claim and provides supporting sentences from the source documents. Our system improves faithfulness, answer relevance, and contextual relevance by up to +0.088 (12%) over a strong RAG baseline. We further release a benchmark of 1,000 scientific question-answer-evidence triplets to support reproducibility. With transparent reasoning, verifiable citations, and domain-wide scalability, SQuAI demonstrates how multi-agent RAG enables more trustworthy scientific QA with LLMs.

SQuAI: Scientific Question-Answering with Multi-Agent Retrieval-Augmented Generation

TL;DR

SQuAI addresses hallucination and verification challenges in open-domain scientific QA by introducing a four-agent, multi-agent RAG framework that decomposes complex questions, employs hybrid sparse-dense retrieval, and enforces fine-grained inline citations with citation context. The system operates over the unarXive 2024 corpus to enable scalable, real-time QA across multiple scientific domains, and includes a dedicated evaluation benchmark of 1,000 QA-evidence triplets. Empirical results show up to a 12% improvement in Faithfulness and Relevance over a strong RAG baseline, driven by query decomposition, adaptive filtering, and rich citation grounding. The work provides a transparent, scalable platform for trustworthy scientific QA and offers a publicly available benchmark and interface to facilitate reproducibility and further research.

Abstract

We present SQuAI (https://squai.scads.ai/), a scalable and trustworthy multi-agent retrieval-augmented generation (RAG) framework for scientific question answering (QA) with large language models (LLMs). SQuAI addresses key limitations of existing RAG systems in the scholarly domain, where complex, open-domain questions demand accurate answers, explicit claims with citations, and retrieval across millions of scientific documents. Built on over 2.3 million full-text papers from arXiv.org, SQuAI employs four collaborative agents to decompose complex questions into sub-questions, retrieve targeted evidence via hybrid sparse-dense retrieval, and adaptively filter documents to improve contextual relevance. To ensure faithfulness and traceability, SQuAI integrates in-line citations for each generated claim and provides supporting sentences from the source documents. Our system improves faithfulness, answer relevance, and contextual relevance by up to +0.088 (12%) over a strong RAG baseline. We further release a benchmark of 1,000 scientific question-answer-evidence triplets to support reproducibility. With transparent reasoning, verifiable citations, and domain-wide scalability, SQuAI demonstrates how multi-agent RAG enables more trustworthy scientific QA with LLMs.
Paper Structure (14 sections, 3 equations, 1 figure, 1 table)

This paper contains 14 sections, 3 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Overview
  4. Multi-Agent System Design
  5. Agent-1: Decomposer
  6. Hybrid Retrieval
  7. Agent-2: Generator
  8. Agent-3: Judge
  9. Agent-4: Answer Generator
  10. User Interface
  11. Evaluation
  12. Benchmarks & Metrics
  13. Results
  14. Conclusion

Figures (1)

  • Figure 1: An overview of the SQuAI framework including four agents and hybrid retrieval for scientific QA.