SymbioticRAG: Enhancing Document Intelligence Through Human-LLM Symbiotic Collaboration

TL;DR

SymbioticRAG addresses the mismatch between user intent and retrieval relevance in RAG by enabling bidirectional learning between humans and LLMs. It implements Level 1: document-processing, retriever, and UI for direct content curation, and proposes Level 2: personalization from interaction logs. Evaluations across literature, geology, and education show reduced human–retriever distance ($D$) and higher user satisfaction ($S$) for SymbioticRAG compared with baselines, indicating improved relevance and engagement, with $D$ reductions to roughly $0.5$–$0.6$ and $S$ improvements to $3.7$–$4.1$ in several scenarios. A human-on-the-loop validation interface supports data quality and domain adaptation. The authors plan open access to foster broader research and accelerate Level 2 development.

Abstract

We present \textbf{SymbioticRAG}, a novel framework that fundamentally reimagines Retrieval-Augmented Generation~(RAG) systems by establishing a bidirectional learning relationship between humans and machines. Our approach addresses two critical challenges in current RAG systems: the inherently human-centered nature of relevance determination and users' progression from "unconscious incompetence" in query formulation. SymbioticRAG introduces a two-tier solution where Level 1 enables direct human curation of retrieved content through interactive source document exploration, while Level 2 aims to build personalized retrieval models based on captured user interactions. We implement Level 1 through three key components: (1)~a comprehensive document processing pipeline with specialized models for layout detection, OCR, and extraction of tables, formulas, and figures; (2)~an extensible retriever module supporting multiple retrieval strategies; and (3)~an interactive interface that facilitates both user engagement and interaction data logging. We experiment Level 2 implementation via a retriever strategy incorporated LLM summarized user intention from user interaction logs. To maintain high-quality data preparation, we develop a human-on-the-loop validation interface that improves pipeline output while advancing research in specialized extraction tasks. Evaluation across three scenarios (literature review, geological exploration, and education) demonstrates significant improvements in retrieval relevance and user satisfaction compared to traditional RAG approaches. To facilitate broader research and further advancement of SymbioticRAG Level 2 implementation, we will make our system openly accessible to the research community.

Figures (7)

• Figure 1: The overview of SymbioticRAG system design
• Figure 2: Four stages of competence proposed by Noel Burch (source: Wikipedia). Current RAG systems effectively address Conscious Incompetence---stage where users can recognize knowledge gaps. However, they struggle with Unconscious Incompetence---users remain unaware of key knowledge deficits, necessitating more exploratory means to uncover these "unknown unknowns."
• Figure 3: SymbioticRAG concept illustration
• Figure 4: Human-on-the-loop validation interfaces for the document processing pipeline, supporting comprehensive validation of layout analysis, OCR, table extraction, figure processing and mathematical formula recognition and understanding.
• Figure 5: SymbioticRAG UI demonstration example