MATCH: Engineering Transparent and Controllable Conversational XAI Systems through Composable Building Blocks

TL;DR

Interactive AI systems remain opaque despite model-level XAI techniques. MATCH introduces a 5-layer auditable framework of structural and explanatory building blocks that can be composed into pipelines and exposed via APIs, enabling both human and LLM agents to audit, reason about, and control system behavior. The paper contributes an auditable architecture, a practical MATCH implementation exposing code as composable building blocks, and a catalog of both structural and explanatory blocks, demonstrated on a loan-approval use case and designed to merge GUI and conversational XAI. This approach aims to deliver safer, more transparent, and controllable multi-agent AI systems with potential for extension beyond tabular data.

Abstract

While the increased integration of AI technologies into interactive systems enables them to solve an increasing number of tasks, the black-box problem of AI models continues to spread throughout the interactive system as a whole. Explainable AI (XAI) techniques can make AI models more accessible by employing post-hoc methods or transitioning to inherently interpretable models. While this makes individual AI models clearer, the overarching system architecture remains opaque. This challenge not only pertains to standard XAI techniques but also to human examination and conversational XAI approaches that need access to model internals to interpret them correctly and completely. To this end, we propose conceptually representing such interactive systems as sequences of structural building blocks. These include the AI models themselves, as well as control mechanisms grounded in literature. The structural building blocks can then be explained through complementary explanatory building blocks, such as established XAI techniques like LIME and SHAP. The flow and APIs of the structural building blocks form an unambiguous overview of the underlying system, serving as a communication basis for both human and automated agents, thus aligning human and machine interpretability of the embedded AI models. In this paper, we present our flow-based approach and a selection of building blocks as MATCH: a framework for engineering Multi-Agent Transparent and Controllable Human-centered systems. This research contributes to the field of (conversational) XAI by facilitating the integration of interpretability into existing interactive systems.

Figures (4)

• Figure 1: An example loan approval prediction ensemble to illustrate our proposed architecture. Layer 1 shows the structural building blocks, consisting of our building blocks (in blue) integrated into the ML pipeline (in green). Layer 2 converts the structural building blocks into a callable API, usable by the explanatory building blocks in layer 3, assembled into the UI of layer 4. The LLM of layer 5 can interact with the underlying system via the API of layer 2, so that all agents operate on a common knowledge base. Agents can communicate among themselves via a chat interface.
• Figure 2: Prototype of our approach, using the loan approval prediction ensemble. The UI (layer 4) shows each structural building block (layer 1) influencing predictions, exposing system behavior to both human and automated agents (layer 5) through explanatory building blocks (layer 3) and an API (layer 2), respectively. The image shows a LIME explanation for both the NeuralNetwork1 network and the total chain. Both the UI and LLM have access to the same API and are able to use the same visual components.
• Figure 3: Grammar in EBNF for defining workflows consisting of building blocks in MATCH. The AI codebase is linked to individual BuildingBlocks as lists of Callables (i.e., methods). Each type of building block is a Runnable that can be chained sequentially via pipeline operators, or in parallel via ParallelBlocks.
• Figure 4: Diagram of the dynamically generated API. The API class recursively traverses a Runnable, which can be a BuildingBlock, Chain, or ParallelBlock, to expose their methods as REST endpoints via the EndpointGenerator utility class. Methods annotated with CRUD or prediction decorators are automatically mapped to HTTP methods (POST, GET, PUT, DELETE). The structure of the entire Runnable is serialized via the ChainSerializer class. This workflow enables users to interact with the underlying AI workflow via a UI using the API, while the ConversationBot (in our case, ECHO vanbrabant2025echo) enables conversational access via generated tools.