Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error Detection and Constraint Recovery in Hierarchical Multi-Label Classification without Prior Knowledge

Joshua Shay Kricheli, Khoa Vo, Aniruddha Datta, Spencer Ozgur, Paulo Shakarian

TL;DR

This work tackles hierarchical multi-label classification without requiring pre-specified hierarchy constraints. It introduces Focused-EDR, a method that learns explainable error rules and uses a ratio-based submodular optimization to recover constraints across two granularity levels. The approach yields improved error detection, robust constraint recovery under noise, and enables neurosymbolic learning with LTNs to boost performance. An open-source military vehicle dataset is released to support future HMC and neurosymbolic studies.

Abstract

Recent advances in Hierarchical Multi-label Classification (HMC), particularly neurosymbolic-based approaches, have demonstrated improved consistency and accuracy by enforcing constraints on a neural model during training. However, such work assumes the existence of such constraints a-priori. In this paper, we relax this strong assumption and present an approach based on Error Detection Rules (EDR) that allow for learning explainable rules about the failure modes of machine learning models. We show that these rules are not only effective in detecting when a machine learning classifier has made an error but also can be leveraged as constraints for HMC, thereby allowing the recovery of explainable constraints even if they are not provided. We show that our approach is effective in detecting machine learning errors and recovering constraints, is noise tolerant, and can function as a source of knowledge for neurosymbolic models on multiple datasets, including a newly introduced military vehicle recognition dataset.

Error Detection and Constraint Recovery in Hierarchical Multi-Label Classification without Prior Knowledge

TL;DR

This work tackles hierarchical multi-label classification without requiring pre-specified hierarchy constraints. It introduces Focused-EDR, a method that learns explainable error rules and uses a ratio-based submodular optimization to recover constraints across two granularity levels. The approach yields improved error detection, robust constraint recovery under noise, and enables neurosymbolic learning with LTNs to boost performance. An open-source military vehicle dataset is released to support future HMC and neurosymbolic studies.

Abstract

Recent advances in Hierarchical Multi-label Classification (HMC), particularly neurosymbolic-based approaches, have demonstrated improved consistency and accuracy by enforcing constraints on a neural model during training. However, such work assumes the existence of such constraints a-priori. In this paper, we relax this strong assumption and present an approach based on Error Detection Rules (EDR) that allow for learning explainable rules about the failure modes of machine learning models. We show that these rules are not only effective in detecting when a machine learning classifier has made an error but also can be leveraged as constraints for HMC, thereby allowing the recovery of explainable constraints even if they are not provided. We show that our approach is effective in detecting machine learning errors and recovering constraints, is noise tolerant, and can function as a source of knowledge for neurosymbolic models on multiple datasets, including a newly introduced military vehicle recognition dataset.
Paper Structure (4 sections, 1 theorem, 2 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 4 sections, 1 theorem, 2 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. Experimental Results

Key Result

theorem 1

Given a training set $\mathcal{T}$, a solution $DC^*_y$ for the maximization problem in Equation eqn:ratio_maximization of a class $y \in \mathcal{Y}_\mathcal{G}$ also maximizes the error F1-score for class $y$ with respect to the prediction vector $\hat{Y}_{\hat{\theta}}^{\mathcal{T}}$.

Figures (4)

  • Figure : (a)
  • Figure : (a)
  • Figure : (b)
  • Figure : (c)

Theorems & Definitions (1)

  • theorem 1