Agentic AI-Driven Technical Troubleshooting for Enterprise Systems: A Novel Weighted Retrieval-Augmented Generation Paradigm

TL;DR

This work tackles enterprise technical troubleshooting across heterogeneous data silos by introducing a dynamically weighted Retrieval-Augmented Generation framework. It combines all-MiniLM-L6-v2 embeddings with FAISS indexing, a per-source dynamic weighting mechanism, and top-K aggregation to retrieve diverse, highly relevant information, followed by a LLaMA-based generation and self-evaluation loop to ensure accuracy. Key contributions include the dynamic weighting strategy (with $ ilde{D}_{k,i} = w_k \cdot D_{k,i}$), per-source threshold filtering, and a self-evaluation module that improves accuracy by about $5.6$ percentage points over standard RAG, yielding higher accuracy ($90.8\%$) and relevance ($0.89$) on enterprise data tasks. The approach reduces hallucinations, accelerates resolution in complex troubleshooting, and offers scalable, extensible integration via a facade architecture, with potential for real-time learning and conversational troubleshooting in future work.

Abstract

Technical troubleshooting in enterprise environments often involves navigating diverse, heterogeneous data sources to resolve complex issues effectively. This paper presents a novel agentic AI solution built on a Weighted Retrieval-Augmented Generation (RAG) Framework tailored for enterprise technical troubleshooting. By dynamically weighting retrieval sources such as product manuals, internal knowledge bases, FAQs, and troubleshooting guides based on query context, the framework prioritizes the most relevant data. For instance, it gives precedence to product manuals for SKU-specific queries while incorporating general FAQs for broader issues. The system employs FAISS for efficient dense vector search, coupled with a dynamic aggregation mechanism to seamlessly integrate results from multiple sources. A Llama-based self-evaluator ensures the contextual accuracy and confidence of the generated responses before delivering them. This iterative cycle of retrieval and validation enhances precision, diversity, and reliability in response generation. Preliminary evaluations on large enterprise datasets demonstrate the framework's efficacy in improving troubleshooting accuracy, reducing resolution times, and adapting to varied technical challenges. Future research aims to enhance the framework by integrating advanced conversational AI capabilities, enabling more interactive and intuitive troubleshooting experiences. Efforts will also focus on refining the dynamic weighting mechanism through reinforcement learning to further optimize the relevance and precision of retrieved information. By incorporating these advancements, the proposed framework is poised to evolve into a comprehensive, autonomous AI solution, redefining technical service workflows across enterprise settings.