Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Rule-Based Explanations of Black Box Decision Systems

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, Fosca Giannotti

TL;DR

LORE tackles the challenge of explaining opaque black-box decisions by constructing a local, interpretable predictor trained on a genetically generated neighborhood around a target instance. The method outputs a single decision rule plus counterfactual rules, enabling both justification and actionable recourse. Through extensive experiments on tabular datasets, LORE outperforms baselines like LIME and Anchors in fidelity, stability, and explanation quality, while producing compact, comprehensible explanations. The work advances model-agnostic explainability by coupling local rule extraction with counterfactual reasoning and demonstrates practical impact for trustworthy decision systems in sensitive domains.

Abstract

The recent years have witnessed the rise of accurate but obscure decision systems which hide the logic of their internal decision processes to the users. The lack of explanations for the decisions of black box systems is a key ethical issue, and a limitation to the adoption of machine learning components in socially sensitive and safety-critical contexts. %Therefore, we need explanations that reveals the reasons why a predictor takes a certain decision. In this paper we focus on the problem of black box outcome explanation, i.e., explaining the reasons of the decision taken on a specific instance. We propose LORE, an agnostic method able to provide interpretable and faithful explanations. LORE first leans a local interpretable predictor on a synthetic neighborhood generated by a genetic algorithm. Then it derives from the logic of the local interpretable predictor a meaningful explanation consisting of: a decision rule, which explains the reasons of the decision; and a set of counterfactual rules, suggesting the changes in the instance's features that lead to a different outcome. Wide experiments show that LORE outperforms existing methods and baselines both in the quality of explanations and in the accuracy in mimicking the black box.

Local Rule-Based Explanations of Black Box Decision Systems

TL;DR

LORE tackles the challenge of explaining opaque black-box decisions by constructing a local, interpretable predictor trained on a genetically generated neighborhood around a target instance. The method outputs a single decision rule plus counterfactual rules, enabling both justification and actionable recourse. Through extensive experiments on tabular datasets, LORE outperforms baselines like LIME and Anchors in fidelity, stability, and explanation quality, while producing compact, comprehensible explanations. The work advances model-agnostic explainability by coupling local rule extraction with counterfactual reasoning and demonstrates practical impact for trustworthy decision systems in sensitive domains.

Abstract

The recent years have witnessed the rise of accurate but obscure decision systems which hide the logic of their internal decision processes to the users. The lack of explanations for the decisions of black box systems is a key ethical issue, and a limitation to the adoption of machine learning components in socially sensitive and safety-critical contexts. %Therefore, we need explanations that reveals the reasons why a predictor takes a certain decision. In this paper we focus on the problem of black box outcome explanation, i.e., explaining the reasons of the decision taken on a specific instance. We propose LORE, an agnostic method able to provide interpretable and faithful explanations. LORE first leans a local interpretable predictor on a synthetic neighborhood generated by a genetic algorithm. Then it derives from the logic of the local interpretable predictor a meaningful explanation consisting of: a decision rule, which explains the reasons of the decision; and a set of counterfactual rules, suggesting the changes in the instance's features that lead to a different outcome. Wide experiments show that LORE outperforms existing methods and baselines both in the quality of explanations and in the accuracy in mimicking the black box.

Paper Structure

This paper contains 20 sections, 4 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem and Explanations
  4. Classification, black boxes, and interpretable predictors.
  5. Black Box Outcome Explanation.
  6. Proposed Method
  7. Neighborhood Generation
  8. Distance Function
  9. Local Rule-Based Classifier and Explanation Extraction
  10. Experiments
  11. Experimental Setup
  12. Analysis of Neighborhood Generation
  13. Comparing Distance Functions
  14. Validation of Local Explanations
  15. Comparison with the State-of-Art
  16. ...and 5 more sections

Figures (10)

  • Figure 1: Crossover.
  • Figure 2: Mutation
  • Figure 3: Random forest black box: purple vs green decision. Starred instance $x$. (Top) Uniformly random (left) and genetic generation (right). (Bottom) Density of random (left) and genetic (right) generation. (Best view in color).
  • Figure 4: Example decision tree.
  • Figure 5: Impact of the number of generations $G$ and of population size $N$ parameters of the genetic neighborhood generation. Bottom plots also report elapsed running times.
  • ...and 5 more figures

Theorems & Definitions (3)

  • definition 1: Black Box Outcome Explanation
  • definition 2: Explanation Through Interpretable Models
  • definition 3: Local Explanation