Towards Responsible and Explainable AI Agents with Consensus-Driven Reasoning
Eranga Bandara, Tharaka Hewa, Ross Gore, Sachin Shetty, Ravi Mukkamala, Peter Foytik, Abdul Rahman, Safdar H. Bouk, Xueping Liang, Amin Hass, Sachini Rajapakse, Ng Wee Keong, Kasun De Zoysa, Aruna Withanage, Nilaan Loganathan
TL;DR
The paper addresses the challenge of achieving explainability and responsibility in autonomous agentic AI systems. It proposes a consensus-driven architecture that pairs a heterogeneous LLM/VLM consortium with a centralized reasoning-layer governance agent to produce explainable, auditable decisions. Through five real-world use cases—news podcast generation, neuromuscular reflex analysis, dental gingivitis detection, psychiatric diagnosis, and RF signal classification—it demonstrates improved robustness, reduced hallucinations, and stronger governance compared to single-model approaches. The architecture provides a practical blueprint for building production-grade, trustworthy agentic AI by separating decision generation from governance and preserving cross-model evidence traces. This work offers concrete guidance for deploying scalable yet responsible AI systems across high-stakes domains.
Abstract
Agentic AI represents a major shift in how autonomous systems reason, plan, and execute multi-step tasks through the coordination of Large Language Models (LLMs), Vision Language Models (VLMs), tools, and external services. While these systems enable powerful new capabilities, increasing autonomy introduces critical challenges related to explainability, accountability, robustness, and governance, especially when agent outputs influence downstream actions or decisions. Existing agentic AI implementations often emphasize functionality and scalability, yet provide limited mechanisms for understanding decision rationale or enforcing responsibility across agent interactions. This paper presents a Responsible(RAI) and Explainable(XAI) AI Agent Architecture for production-grade agentic workflows based on multi-model consensus and reasoning-layer governance. In the proposed design, a consortium of heterogeneous LLM and VLM agents independently generates candidate outputs from a shared input context, explicitly exposing uncertainty, disagreement, and alternative interpretations. A dedicated reasoning agent then performs structured consolidation across these outputs, enforcing safety and policy constraints, mitigating hallucinations and bias, and producing auditable, evidence-backed decisions. Explainability is achieved through explicit cross-model comparison and preserved intermediate outputs, while responsibility is enforced through centralized reasoning-layer control and agent-level constraints. We evaluate the architecture across multiple real-world agentic AI workflows, demonstrating that consensus-driven reasoning improves robustness, transparency, and operational trust across diverse application domains. This work provides practical guidance for designing agentic AI systems that are autonomous and scalable, yet responsible and explainable by construction.