Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Global and Local Explanations for Tree-Ensemble Learning Methods by Answer Set Programming

Akihiro Takemura, Katsumi Inoue

TL;DR

This work presents a novel method for generating global and local explanations of tree-ensemble methods by extracting rule bodies from base trees and solving a declarative rule-set selection problem with Answer Set Programming. It demonstrates a two-step framework: (1) faithful rule extraction from ensembles, and (2) ASP-encoded constraint-driven optimization to produce concise, user-specified explanations. Across 14 public datasets, the approach yields smaller rule sets than traditional rule-learner baselines while maintaining reasonable fidelity to the original models, with global explanations generally trading some fidelity for simplicity and local explanations achieving competitive precision and strong, consistent runtimes. The framework’s declarative nature enables flexible inclusion of constraints and preferences, supporting rapid prototyping and potential extension to regression and model-agnostic settings, albeit with scalability challenges tied to the combinatorial search space. Overall, the method provides a practical, interpretable pathway to understand complex tree-ensemble decisions through explainable, rule-based descriptions grounded in ASP.

Abstract

We propose a method for generating rule sets as global and local explanations for tree-ensemble learning methods using Answer Set Programming (ASP). To this end, we adopt a decompositional approach where the split structures of the base decision trees are exploited in the construction of rules, which in turn are assessed using pattern mining methods encoded in ASP to extract explanatory rules. For global explanations, candidate rules are chosen from the entire trained tree-ensemble models, whereas for local explanations, candidate rules are selected by only considering rules that are relevant to the particular predicted instance. We show how user-defined constraints and preferences can be represented declaratively in ASP to allow for transparent and flexible rule set generation, and how rules can be used as explanations to help the user better understand the models. Experimental evaluation with real-world datasets and popular tree-ensemble algorithms demonstrates that our approach is applicable to a wide range of classification tasks. Under consideration in Theory and Practice of Logic Programming (TPLP).

Generating Global and Local Explanations for Tree-Ensemble Learning Methods by Answer Set Programming

TL;DR

This work presents a novel method for generating global and local explanations of tree-ensemble methods by extracting rule bodies from base trees and solving a declarative rule-set selection problem with Answer Set Programming. It demonstrates a two-step framework: (1) faithful rule extraction from ensembles, and (2) ASP-encoded constraint-driven optimization to produce concise, user-specified explanations. Across 14 public datasets, the approach yields smaller rule sets than traditional rule-learner baselines while maintaining reasonable fidelity to the original models, with global explanations generally trading some fidelity for simplicity and local explanations achieving competitive precision and strong, consistent runtimes. The framework’s declarative nature enables flexible inclusion of constraints and preferences, supporting rapid prototyping and potential extension to regression and model-agnostic settings, albeit with scalability challenges tied to the combinatorial search space. Overall, the method provides a practical, interpretable pathway to understand complex tree-ensemble decisions through explainable, rule-based descriptions grounded in ASP.

Abstract

We propose a method for generating rule sets as global and local explanations for tree-ensemble learning methods using Answer Set Programming (ASP). To this end, we adopt a decompositional approach where the split structures of the base decision trees are exploited in the construction of rules, which in turn are assessed using pattern mining methods encoded in ASP to extract explanatory rules. For global explanations, candidate rules are chosen from the entire trained tree-ensemble models, whereas for local explanations, candidate rules are selected by only considering rules that are relevant to the particular predicted instance. We show how user-defined constraints and preferences can be represented declaratively in ASP to allow for transparent and flexible rule set generation, and how rules can be used as explanations to help the user better understand the models. Experimental evaluation with real-world datasets and popular tree-ensemble algorithms demonstrates that our approach is applicable to a wide range of classification tasks. Under consideration in Theory and Practice of Logic Programming (TPLP).

Paper Structure

This paper contains 31 sections, 2 theorems, 5 equations, 2 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Tree-Ensemble Learning Algorithms
  4. Answer Set Programming
  5. Pattern Mining
  6. Rule Set Generation
  7. Problem Statement
  8. Rule Extraction from Decision Trees
  9. Computing Metrics and Meta-data for Selection
  10. Encoding Inclusion Criteria and Constraints
  11. Optimizing Rule Sets
  12. Rule Set Generation for Global and Local Explanations
  13. Experiments
  14. Experimental Setup
  15. Datasets
  16. ...and 16 more sections

Key Result

Proposition 1

The maximum size of $R$, constructed by only considering the rules at the leaf nodes, is $K \times 2^h$.

Figures (2)

  • Figure 1: Overview of our framework
  • Figure 2: A simple decision tree-ensemble consisting of two decision trees. The rule associated with each node is given by the conjunction of all conditions associated with nodes on the paths from the root node to that node.

Theorems & Definitions (10)

  • Proposition 1
  • Proposition 2
  • proof
  • Example 1
  • Example 2
  • Example 3
  • Definition 1
  • Example 4
  • Definition 2
  • Example 5