Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A multi-criteria approach for selecting an explanation from the set of counterfactuals produced by an ensemble of explainers

Ignacy Stępka, Mateusz Lango, Jerzy Stefanowski

TL;DR

The paper tackles the challenge of selecting a single, high-quality counterfactual explanation when multiple methods produce conflicting solutions. It introduces a four-stage framework that builds an ensemble of diverse explainers, enforces validity and actionability, filters dominated solutions via Pareto dominance, and selects a final counterfactual with the simple, computation-efficient Ideal Point method. Empirical results across four datasets show the approach consistently yields actionable counterfactuals and achieves strong trade-offs across multiple quality criteria, outperforming individual base methods and random ensembles in overall ranking. The proposed MCDA-based strategy reduces choice overload, provides robust, dataset-agnostic performance, and offers a foundation for user-preference-driven or interactive extension in future work.

Abstract

Counterfactuals are widely used to explain ML model predictions by providing alternative scenarios for obtaining the more desired predictions. They can be generated by a variety of methods that optimize different, sometimes conflicting, quality measures and produce quite different solutions. However, choosing the most appropriate explanation method and one of the generated counterfactuals is not an easy task. Instead of forcing the user to test many different explanation methods and analysing conflicting solutions, in this paper, we propose to use a multi-stage ensemble approach that will select single counterfactual based on the multiple-criteria analysis. It offers a compromise solution that scores well on several popular quality measures. This approach exploits the dominance relation and the ideal point decision aid method, which selects one counterfactual from the Pareto front. The conducted experiments demonstrated that the proposed approach generates fully actionable counterfactuals with attractive compromise values of the considered quality measures.

A multi-criteria approach for selecting an explanation from the set of counterfactuals produced by an ensemble of explainers

TL;DR

The paper tackles the challenge of selecting a single, high-quality counterfactual explanation when multiple methods produce conflicting solutions. It introduces a four-stage framework that builds an ensemble of diverse explainers, enforces validity and actionability, filters dominated solutions via Pareto dominance, and selects a final counterfactual with the simple, computation-efficient Ideal Point method. Empirical results across four datasets show the approach consistently yields actionable counterfactuals and achieves strong trade-offs across multiple quality criteria, outperforming individual base methods and random ensembles in overall ranking. The proposed MCDA-based strategy reduces choice overload, provides robust, dataset-agnostic performance, and offers a foundation for user-preference-driven or interactive extension in future work.

Abstract

Counterfactuals are widely used to explain ML model predictions by providing alternative scenarios for obtaining the more desired predictions. They can be generated by a variety of methods that optimize different, sometimes conflicting, quality measures and produce quite different solutions. However, choosing the most appropriate explanation method and one of the generated counterfactuals is not an easy task. Instead of forcing the user to test many different explanation methods and analysing conflicting solutions, in this paper, we propose to use a multi-stage ensemble approach that will select single counterfactual based on the multiple-criteria analysis. It offers a compromise solution that scores well on several popular quality measures. This approach exploits the dominance relation and the ideal point decision aid method, which selects one counterfactual from the Pareto front. The conducted experiments demonstrated that the proposed approach generates fully actionable counterfactuals with attractive compromise values of the considered quality measures.
Paper Structure (18 sections, 1 equation, 3 figures, 7 tables, 1 algorithm)

This paper contains 18 sections, 1 equation, 3 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Counterfactual Explanation Methods
  4. Most related methods for generating several counterfactual explanations
  5. Quality Measures for Counterfactuals
  6. The proposed multi-criteria approach
  7. Constructing an ensemble of explainers
  8. Enforcement of validity and actionability constraints
  9. Filtering of dominated solutions
  10. Selection of a counterfactual with Ideal Point method
  11. Approach walkthrough with a toy example
  12. Experimental evaluation
  13. Experimental setup
  14. Analysis of the Pareto front of counterfactuals generated by the base explainers in the ensemble
  15. Evaluating counterfactuals with different quality measures
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Visualization of the algorithm steps in our proposed approach.
  • Figure 2: An example demonstrating the application of dominance relations and the Ideal Point method. Note the optimization directions for Proximity and Feasibility are min, and for DiscriminativePower - max.
  • Figure 3: The barycentric plots depicting the best method for a given dataset according to the utility functions with different weights assigned to quality criteria. The datasets are: German (upper-left), Adult (upper-right), Compas (lower-left), Fico (lower-right).