Logic Explanation of AI Classifiers by Categorical Explaining Functors
Stefano Fioravanti, Francesco Giannini, Paolo Frazzetto, Fabio Zanasi, Pietro Barbiero
TL;DR
This work introduces an explaining functor grounded in category theory to map concept-based fuzzy representations to coherent Boolean explanations, ensuring fidelity and compositional integrity with the underlying model reasoning. It defines the δ-COH category of δ-coherent functions, constructs a functor to Boolean explanations, and extends the framework to non-δ-coherent functions via domain-extension, output-modification, and quotient-function strategies. The authors provide theoretical guarantees of coherence and compositionality, and demonstrate practical benefits on synthetic benchmarks (e.g., XOR and fuzzy OR) using Logic Explained Networks, with an extended post-hoc approach improving explanations in non-coherent regions. Overall, the paper lays a principled foundation for self-explainable AI by linking explainability to structure-preserving morphisms and outlines clear pathways for applying the framework to broader XAI methods and data modalities.
Abstract
The most common methods in explainable artificial intelligence are post-hoc techniques which identify the most relevant features used by pretrained opaque models. Some of the most advanced post hoc methods can generate explanations that account for the mutual interactions of input features in the form of logic rules. However, these methods frequently fail to guarantee the consistency of the extracted explanations with the model's underlying reasoning. To bridge this gap, we propose a theoretically grounded approach to ensure coherence and fidelity of the extracted explanations, moving beyond the limitations of current heuristic-based approaches. To this end, drawing from category theory, we introduce an explaining functor which structurally preserves logical entailment between the explanation and the opaque model's reasoning. As a proof of concept, we validate the proposed theoretical constructions on a synthetic benchmark verifying how the proposed approach significantly mitigates the generation of contradictory or unfaithful explanations.