Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging

Muxi Chen, Chenchen Zhao, Qiang Xu

TL;DR

This work tackles systematic failures of vision models by automatically discovering coherent error slices and repairing models. It proposes HiBug2, a closed-loop framework that uses structured attribute generation, a BFS-based slice enumeration, and unseen-slice prediction to identify and address failure modes. Across image classification, pose estimation, and object detection, HiBug2 yields higher-quality, more coherent error slices, accelerates analysis with large speedups over naive approaches, and delivers tangible improvements in model repair compared with prior methods. By emphasizing contextual (background and global) attributes and scalable, interpretable tooling, HiBug2 offers a practical pathway for robust, real-world model debugging.

Abstract

Despite the significant success of deep learning models in computer vision, they often exhibit systematic failures on specific data subsets, known as error slices. Identifying and mitigating these error slices is crucial to enhancing model robustness and reliability in real-world scenarios. In this paper, we introduce HiBug2, an automated framework for error slice discovery and model repair. HiBug2 first generates task-specific visual attributes to highlight instances prone to errors through an interpretable and structured process. It then employs an efficient slice enumeration algorithm to systematically identify error slices, overcoming the combinatorial challenges that arise during slice exploration. Additionally, HiBug2 extends its capabilities by predicting error slices beyond the validation set, addressing a key limitation of prior approaches. Extensive experiments across multiple domains, including image classification, pose estimation, and object detection - show that HiBug2 not only improves the coherence and precision of identified error slices but also significantly enhances the model repair capabilities.

HiBug2: Efficient and Interpretable Error Slice Discovery for Comprehensive Model Debugging

TL;DR

This work tackles systematic failures of vision models by automatically discovering coherent error slices and repairing models. It proposes HiBug2, a closed-loop framework that uses structured attribute generation, a BFS-based slice enumeration, and unseen-slice prediction to identify and address failure modes. Across image classification, pose estimation, and object detection, HiBug2 yields higher-quality, more coherent error slices, accelerates analysis with large speedups over naive approaches, and delivers tangible improvements in model repair compared with prior methods. By emphasizing contextual (background and global) attributes and scalable, interpretable tooling, HiBug2 offers a practical pathway for robust, real-world model debugging.

Abstract

Despite the significant success of deep learning models in computer vision, they often exhibit systematic failures on specific data subsets, known as error slices. Identifying and mitigating these error slices is crucial to enhancing model robustness and reliability in real-world scenarios. In this paper, we introduce HiBug2, an automated framework for error slice discovery and model repair. HiBug2 first generates task-specific visual attributes to highlight instances prone to errors through an interpretable and structured process. It then employs an efficient slice enumeration algorithm to systematically identify error slices, overcoming the combinatorial challenges that arise during slice exploration. Additionally, HiBug2 extends its capabilities by predicting error slices beyond the validation set, addressing a key limitation of prior approaches. Extensive experiments across multiple domains, including image classification, pose estimation, and object detection - show that HiBug2 not only improves the coherence and precision of identified error slices but also significantly enhances the model repair capabilities.

Paper Structure

This paper contains 41 sections, 1 equation, 12 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Error slice discovery
  4. Visual attribute and tag generation
  5. Method: automatic attribute and tag generation
  6. Key challenges in attribute and tag generation
  7. Structured and comprehensive generation
  8. Attribute generation
  9. Tag determination and assignment
  10. Method: exploring data slices
  11. Efficient slice discovery and repair
  12. Basic notations
  13. Key challenges in slice enumeration
  14. Properties of data slices
  15. Algorithm design
  16. ...and 26 more sections

Figures (12)

  • Figure 1: The main workflow of HiBug2 for closed-loop model debugging and repair.
  • Figure 2: Common errors of deep learning models can be grouped into data distribution issues and inherent task difficulties.
  • Figure 3: Attributes and tags generated by the error slice discovery methods.
  • Figure 4: Comparison of the slice enumeration methods.
  • Figure 5: Ablation study across varying numbers of images and tag sets.
  • ...and 7 more figures