Explanations Go Linear: Interpretable and Individual Latent Encoding for Post-hoc Explainability

TL;DR

Illume tackles the challenge of post-hoc explainability for black-box classifiers by unifying local and global explanations through a latent-space approach. It learns instance-specific locally linear encodings via a meta-encoder to produce a latent representation $oldsymbol{z}_i = W_i^b oldsymbol{x}_i$, on which a global surrogate operates, while an explanation generator maps back to human-interpretable explanations. The method is regularized to ensure decision conditioning, local linearity, and stable mappings, and uses KL-based objectives to align input, latent, and mapping distributions. Experiments on synthetic and real-world tabular data show Illume improves fidelity, robustness, and efficiency of explanations compared to existing surrogates and local explainers, with explanations that can be feature-attribution or rule-based and readily mapped to input space. This approach enables scalable, faithful post-hoc explanations compatible with a variety of surrogate models and offers a practical path toward end-to-end or cross-dataset explanation inference for tabular data.

Abstract

Post-hoc explainability is essential for understanding black-box machine learning models. Surrogate-based techniques are widely used for local and global model-agnostic explanations but have significant limitations. Local surrogates capture non-linearities but are computationally expensive and sensitive to parameters, while global surrogates are more efficient but struggle with complex local behaviors. In this paper, we present ILLUME, a flexible and interpretable framework grounded in representation learning, that can be integrated with various surrogate models to provide explanations for any black-box classifier. Specifically, our approach combines a globally trained surrogate with instance-specific linear transformations learned with a meta-encoder to generate both local and global explanations. Through extensive empirical evaluations, we demonstrate the effectiveness of ILLUME in producing feature attributions and decision rules that are not only accurate but also robust and faithful to the black-box, thus providing a unified explanation framework that effectively addresses the limitations of traditional surrogate methods.

Figures (10)

• Figure 1: Accuracy-interpretability trade-off for neural encoding.
• Figure 2: illume steps: (1) the meta-encoder $f^b$ is trained using input instances $X$ and black-box decisions $Y$; (2)$f^b$ generates specific encoding functions $\eta^b_i$ to individually map instances $\mathbf{x}_i$ into latent representations $\mathbf{z}_i$; (3) the set of latent vectors $Z$ is used to train a surrogate model $g$ for imitating the black-box $Y=b(X)$; (4) given a test instance $\mathbf{x}_{test}$, it is mapped into $\mathbf{z}_{test}$ with $\eta^b_{test}$ obtained by $f^b$, then the explanation is obtained with $e_g(\mathbf{z}_{test}, \eta^b_{test})$ by combining the surrogate logic $g$ with local mapping $\eta^b_{test}$. Training algorithms are marked with white squares. Learned functions are marked with different colored boxes.
• Figure A1: Exemplification of illume's inference phase on compas dataset when generating feature importance local explanations. Given an input data-point $\mathbf{x}_{test}$ (A) and its corresponding black-box prediction $b(\mathbf{x}_{test})$ (B), the method outputs instance-specific explanation $e_g(\mathbf{z}_{test}, \eta^b_{test})$ (C). This explanation is derived: (i) by encoding the instance into a latent representation $\mathbf{z}_{test}$ (D), (ii) extracting the logic of the global surrogate $g$ (E), and (iii) combining it with local interpretable mapping $\eta^b_{test}$ (F), represented by a sparse and linear transformation returned by the meta-encoder.
• Figure A2: Feature importance explanations for two similar records of the wine dataset. Feature values are reported on the left. In the center, explanations are derived with illume, lime and shap. On the right, the prediction probability returned by an XGB classifier.
• Figure A3: Decision rule explanations for two similar records of the wine dataset. Feature values are reported on the left. In the center, explanations derived with illume, lore and anchor. On the right, the prediction probability returned by an LGB classifier.