SOPRAG: Multi-view Graph Experts Retrieval for Industrial Standard Operating Procedures

TL;DR

This work tackles industrial SOP retrieval by introducing SOPRAG, a structure-aware, MoE-inspired retrieval framework that uses a sparse Procedure Card layer and three specialized graph experts—Entity, Causal, and Flow—to address proprietary structure, condition-dependent relevance, and actionable execution. An automated, agent-based benchmark construction pipeline (Collector, Archivist, Auditor, Examiner) enables scalable evaluation in four industrial domains. Empirical results show SOPRAG outperforms lexical, dense, and graph-based RAG baselines in both retrieval and generation, achieving high-quality, executable step-by-step outputs and, in at least one domain, a perfect SOP Quality Score. The proposed architecture advances practical, grounding-aware SOP retrieval and provides a scalable approach to benchmark generation for future research.

Abstract

Standard Operating Procedures (SOPs) are essential for ensuring operational safety and consistency in industrial environments. However, retrieving and following these procedures presents unique challenges, such as rigid proprietary structures, condition-dependent relevance, and actionable execution requirement, which standard semantic-driven Retrieval-Augmented Generation (RAG) paradigms fail to address. Inspired by the Mixture-of-Experts (MoE) paradigm, we propose SOPRAG, a novel framework specifically designed to address the above pain points in SOP retrieval. SOPRAG replaces flat chunking with specialized Entity, Causal, and Flow graph experts to resolve industrial structural and logical complexities. To optimize and coordinate these experts, we propose a Procedure Card layer that prunes the search space to eliminate computational noise, and an LLM-Guided gating mechanism that dynamically weights these experts to align retrieval with operator intent. To address the scarcity of domain-specific data, we also introduce an automated, multi-agent workflow for benchmark construction. Extensive experiments across four industrial domains demonstrate that SOPRAG significantly outperforms strong lexical, dense, and graph-based RAG baselines in both retrieval accuracy and response utility, achieving perfect execution scores in real-world critical tasks.

Figures (7)

• Figure 1: Overview of SOP Retrieval in practice. (Top) An operator encounters a system failure and uses our SOPRAG to generate executable step-by-step guidance. (Bottom) A representative sample of a SOP source document used for retrieval.
• Figure 2: Architecture overview of SOPRAG. (A) Expert Construction: delineating the Procedure Card (PC) layer and the three multi-view graph experts (Entity, Causal, and Flow); (B) MoE-Style Retrieval: illustrating the sparse activation of the PC layer, the LLM-guided gating mechanism (Router), and the scoring and aggregation process of the graph experts; (C) Structure-Aware Generation: showing the generation of a step-by-step actionable response using the flow graph of retrieved SOP as a structured prompt context. (Further extensions are shown in Appendix \ref{['app:agent-exe']}.)
• Figure 3: The overview of the Agent-based Simulation Environment for automated dataset construction. Given a specific domain, four specialized agents (Collector, Archivist, Auditor, and Examiner) mimic human expert workflows to automatically generate a high-quality benchmark dataset.
• Figure 4: Retrieval vs. Generation Trade-off. Points represent mean scores of all evaluation metrics across four domain-specific datasets. The dashed line indicates the Efficiency Frontier of existing methods. SOPRAG noticeably transcends this boundary, demonstrating an optimal balance between retrieval and generation.
• Figure 5: Kernel Density Estimation (KDE) of gating weights ($w_E, w_C, w_F$) across four industrial domains, based on 250 evaluation queries per domain.