Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are Good Explainers Secretly Human-in-the-Loop Active Learners?

Emma Thuong Nguyen, Abhishek Ghose

TL;DR

The paper investigates whether using explanations to guide data selection amounts to a human-in-the-loop Active Learning workflow. It offers a formal mathematical framework that maps explanations to retrieval-based data selection, using matrices $A$, $B$, and $C$ and retrieval weights $W$ to minimize the generalization loss $\mathcal{L}_v$ over $\theta$ and $\Psi$. Empirically, it demonstrates initial results on SST-5 with Partition SHAP explanations and a linear SVM, showing the explanation-driven strategy can outperform standard AL methods in early iterations and remain competitive thereafter, while enabling simulation-based evaluation. The work highlights practical implications for evaluating XAI-based data curation and outlines directions for broader datasets, alternative explainers, and user studies to validate the approach.

Abstract

Explainable AI (XAI) techniques have become popular for multiple use-cases in the past few years. Here we consider its use in studying model predictions to gather additional training data. We argue that this is equivalent to Active Learning, where the query strategy involves a human-in-the-loop. We provide a mathematical approximation for the role of the human, and present a general formalization of the end-to-end workflow. This enables us to rigorously compare this use with standard Active Learning algorithms, while allowing for extensions to the workflow. An added benefit is that their utility can be assessed via simulation instead of conducting expensive user-studies. We also present some initial promising results.

Are Good Explainers Secretly Human-in-the-Loop Active Learners?

TL;DR

The paper investigates whether using explanations to guide data selection amounts to a human-in-the-loop Active Learning workflow. It offers a formal mathematical framework that maps explanations to retrieval-based data selection, using matrices , , and and retrieval weights to minimize the generalization loss over and . Empirically, it demonstrates initial results on SST-5 with Partition SHAP explanations and a linear SVM, showing the explanation-driven strategy can outperform standard AL methods in early iterations and remain competitive thereafter, while enabling simulation-based evaluation. The work highlights practical implications for evaluating XAI-based data curation and outlines directions for broader datasets, alternative explainers, and user studies to validate the approach.

Abstract

Explainable AI (XAI) techniques have become popular for multiple use-cases in the past few years. Here we consider its use in studying model predictions to gather additional training data. We argue that this is equivalent to Active Learning, where the query strategy involves a human-in-the-loop. We provide a mathematical approximation for the role of the human, and present a general formalization of the end-to-end workflow. This enables us to rigorously compare this use with standard Active Learning algorithms, while allowing for extensions to the workflow. An added benefit is that their utility can be assessed via simulation instead of conducting expensive user-studies. We also present some initial promising results.
Paper Structure (10 sections, 2 equations, 2 figures)

This paper contains 10 sections, 2 equations, 2 figures.

Table of Contents

  1. Introduction
  2. XAI-based Data Selection is Active Learning
  3. Mathematical Formulation
  4. Task Formulation
  5. Objective
  6. Metrics
  7. Extensions
  8. Review of Assumptions
  9. Experiments
  10. Conclusions and Future Work

Figures (2)

  • Figure 1: Workflow representing use of explanations to identify data to retrain a model. This shows one iteration of such a workflow, where we start with Model v1 and create a more accurate model Model v2, based on sampling new training instances from a data pool, $X_{inc}$. Please see Section \ref{['sec:intro']} for details. As we shown in Equation \ref{['eqn:one_eqn']} in Section \ref{['sec:formulation']}, many of these steps may be distilled into a single mathematical expression.
  • Figure 2: F1-macro evaluated on $(X_{test}, y_{test})$ for the dataset SST5. The x-axis describes the number of instances used in each model training iteration. The solid line represents the accuracy for our explanation based AL (Section \ref{['sec:task']}). The dotted lines represent results for different popular AL methods: entropy sampling, random sampling, maximum margin sampling. The band around each line indicates 95% confidence intervals across 4 runs.