Neurosymbolic Retrievers for Retrieval-augmented Generation

TL;DR

This work tackles the opacity of retrieval-augmented generation in high-stakes settings by introducing Neurosymbolic RAG, which tightly couples symbolic knowledge graphs with neural retrieval. It presents three architectures—Knowledge Modulation-aligned Retrieval (MAR), KG-Path RAG, and Process Knowledge-infused RAG (Proknow-RAG)—to produce interpretable, workflow-consistent retrieval and reasoning. Through mental-health detection tasks, the authors demonstrate that incorporating symbolic features, graph-based provenance, and procedural instruments improves transparency and, in many cases, retrieval performance. The study highlights the potential of grounded, auditable retrieval systems to support safer, more trustworthy decision-making in clinical and similar domains.

Abstract

Retrieval Augmented Generation (RAG) has made significant strides in overcoming key limitations of large language models, such as hallucination, lack of contextual grounding, and issues with transparency. However, traditional RAG systems consist of three interconnected neural components - the retriever, re-ranker, and generator - whose internal reasoning processes remain opaque. This lack of transparency complicates interpretability, hinders debugging efforts, and erodes trust, especially in high-stakes domains where clear decision-making is essential. To address these challenges, we introduce the concept of Neurosymbolic RAG, which integrates symbolic reasoning using a knowledge graph with neural retrieval techniques. This new framework aims to answer two primary questions: (a) Can retrievers provide a clear and interpretable basis for document selection? (b) Can symbolic knowledge enhance the clarity of the retrieval process? We propose three methods to improve this integration. First is MAR (Knowledge Modulation Aligned Retrieval) that employs modulation networks to refine query embeddings using interpretable symbolic features, thereby making document matching more explicit. Second, KG-Path RAG enhances queries by traversing knowledge graphs to improve overall retrieval quality and interpretability. Lastly, Process Knowledge-infused RAG utilizes domain-specific tools to reorder retrieved content based on validated workflows. Preliminary results from mental health risk assessment tasks indicate that this neurosymbolic approach enhances both transparency and overall performance

Figures (2)

• Figure 1: Overview of KG-Path RAG training and inference pipeline. Top: Graph-based ranking loss training process across three steps. Initially, the knowledge graph contains uncertain relationships (Step 1). During training, the system learns to identify satisfied edges (green) and filter noisy connections (red X), progressively building reliable reasoning paths through entities ($e_1, e_2, e_3, e_4$) to satisfy query constraints ($qe$) (Steps 2-3). Bottom: Inference pipeline showing query enrichment from a single user question to $K$ queries (including the original), followed by multi-hop graph traversal with BFS (Breadth First Search)-guided exploration that distinguishes strong and weak connections across 1st and 2nd hop neighbors. FAISS (Facebook AI Semantic Search) retrieves candidate documents while the ranking function $q(x)$ separates positives from negatives. The optimized retrieval results combine relevant queries and top-$k$ documents, which are structured through a template (Query/Question by User, Retrieved Paragraphs, Generated Answer by LLM) and fed to the T5 model for final answer generation. This joint optimization of KG traversal and retrieval enables explicit path-based provenance while maintaining semantic grounding. The bi-directional arrow between user and T5 represents a RAG-disabled case, similar to MentalLLAMA, a fine-tuned model answering mental health queries.
• Figure 2: Human evaluation comparing KG-Path RAG, Proknow-RAG, and MentalLLAMA-33B on mental health detection tasks sourced from the IMHI datasetyang2024mentallama. Three attributes were measured: Interpretability (Interp.), Clinical Relevance (Clinic. Rel.), and Computational Efficiency (Comp. Eff.). Interpretability assesses whether retrieved chunks provide transparent reasoning for answer generation. Clinical Relevance evaluates the presence and accuracy of clinical entities in retrieved chunks that align with the input query. Computational Efficiency measures the time required to generate complete responses.