X-SYS: A Reference Architecture for Interactive Explanation Systems

TL;DR

X-SYS reframes explainable AI as an information-systems problem and provides a reference architecture to operationalize interactive explanations. It defines STAR quality attributes (scalability, traceability, adaptability, responsiveness) and a five-component decomposition (XUI Services, Explanation Services, Model Services, Data Services, Orchestration and Governance) with contract-based interfaces to decouple UI evolution from backend computation. The paper contributes both the architectural blueprint and a concrete instantiation, SemanticLens, which demonstrates semantic search and activation steering in vision-language models and showcases offline/online separation and DTO-driven interactions. This work advances practical, auditable, and scalable interactive explanation systems that accommodate evolving XAI methods and deployment constraints, facilitating end-to-end design under governance requirements.

Abstract

The explainable AI (XAI) research community has proposed numerous technical methods, yet deploying explainability as systems remains challenging: Interactive explanation systems require both suitable algorithms and system capabilities that maintain explanation usability across repeated queries, evolving models and data, and governance constraints. We argue that operationalizing XAI requires treating explainability as an information systems problem where user interaction demands induce specific system requirements. We introduce X-SYS, a reference architecture for interactive explanation systems, that guides (X)AI researchers, developers and practitioners in connecting interactive explanation user interfaces (XUI) with system capabilities. X-SYS organizes around four quality attributes named STAR (scalability, traceability, responsiveness, and adaptability), and specifies a five-component decomposition (XUI Services, Explanation Services, Model Services, Data Services, Orchestration and Governance). It maps interaction patterns to system capabilities to decouple user interface evolution from backend computation. We implement X-SYS through SemanticLens, a system for semantic search and activation steering in vision-language models. SemanticLens demonstrates how contract-based service boundaries enable independent evolution, offline/online separation ensures responsiveness, and persistent state management supports traceability. Together, this work provides a reusable blueprint and concrete instantiation for interactive explanation systems supporting end-to-end design under operational constraints.

Figures (8)

• Figure 1: X-SYS organizes around four quality attributes and maps each deployment challenge to its primary architectural target: interaction latency to responsiveness (Panel 1), governance to traceability (Panel 2), workflow integration to adaptability (Panel 3), and multi-stakeholder scaling to scalability (Panel 4).
• Figure 2: X-SYS Overview: Motivated by XAI deployment challenges, we present quality attributes for XAI systems and informed by it, a reference architecture. SemanticLens, an interactive explanation system, is presented as implementation of the reference architecture.
• Figure 3: X-SYS reference architecture showing the five core components and their primary interactions. XUI Services request interaction demand that is supplied by Explanation and Model Services. Data Services form the foundational persistence layer, while Orchestration and Governance provide cross-cutting coordination.
• Figure 4: With SemanticLens, users can explore neural representations via text-based search in the Concept Map perspective (left), or inspect and steer representations at the prediction level in the Model Interaction perspective (right).
• Figure 5: The Concept Map presents encoded knowledge as clusters of components and their semantic relations. These clusters support users in obtaining an overview of component structure (1). Users can search for encodings (2): the query "pasta" in ResNet50 returns components with high similarity such as "carbonara (#858)". Users can select the aligned components (3) to review their details (4a-d).