Actionable Interpretability Must Be Defined in Terms of Symmetries
Pietro Barbiero, Mateo Espinosa Zarlenga, Francesco Giannini, Alberto Termine, Filippo Bonchi, Mateja Jamnik, Giuseppe Marra
TL;DR
The paper reframes interpretability as a set of actionable invariances inspired by the Erlangen program, arguing that four symmetries—Inference Equivariance, Information Invariance, Concept Closure Invariance, and Structural Invariance—define the class of interpretable models. It builds a probabilistic, category-theoretic framework (rooted in Markov categories) that formalizes the interactions between a human user and models, and shows how alignment, interventions, and counterfactuals can be expressed as Bayesian inversion within a unified diagrammatic language. By introducing concepts, concept-based translations, and a category of interpretable models, the work provides principled criteria and constructive recipes for building and evaluating interpretable systems, rather than relying on descriptive explanations alone. The framework enables principled model design, tractable verification via information compression, and user-centric reasoning grounded in invariances, with practical implications for robust, explainable AI deployments.
Abstract
This paper argues that interpretability research in Artificial Intelligence is fundamentally ill-posed as existing definitions of interpretability are not *actionable*: they fail to provide formal principles from which concrete modelling and inferential rules can be derived. We posit that for a definition of interpretability to be actionable, it must be given in terms of *symmetries*. We hypothesise that four symmetries suffice to (i) motivate core interpretability properties, (ii) characterize the class of interpretable models, and (iii) derive a unified formulation of interpretable inference (e.g., alignment, interventions, and counterfactuals) as a form of Bayesian inversion.