Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy

TL;DR

Deep neural networks excel in prediction but remain opaque; the paper addresses reproducibility and scalability in XAI by introducing three open-source tools: Zennit for local, rule-based attribution in PyTorch, CoRelAy for constructing dataset-wide analysis pipelines, and ViRelAy for interactive visualization of results. Zennit implements LRP with a flexible rule-based framework, CoRelAy orchestrates pipeline stages (embedding, clustering, etc.) with caching, and ViRelAy presents a web interface to relate attributions, embeddings, and clusters. The authors demonstrate a dataset-wide analysis workflow on large-scale data (e.g., ImageNet) to uncover systematic biases such as Clever Hans-like artifacts, highlighting the practical impact of standardized XAI tooling. Overall, the work advances reproducibility and accessibility in XAI by providing integrated, extensible software for local explanations and global insights.

Abstract

Deep Neural Networks (DNNs) are known to be strong predictors, but their prediction strategies can rarely be understood. With recent advances in Explainable Artificial Intelligence (XAI), approaches are available to explore the reasoning behind those complex models' predictions. Among post-hoc attribution methods, Layer-wise Relevance Propagation (LRP) shows high performance. For deeper quantitative analysis, manual approaches exist, but without the right tools they are unnecessarily labor intensive. In this software paper, we introduce three software packages targeted at scientists to explore model reasoning using attribution approaches and beyond: (1) Zennit - a highly customizable and intuitive attribution framework implementing LRP and related approaches in PyTorch, (2) CoRelAy - a framework to easily and quickly construct quantitative analysis pipelines for dataset-wide analyses of explanations, and (3) ViRelAy - a web-application to interactively explore data, attributions, and analysis results. With this, we provide a standardized implementation solution for XAI, to contribute towards more reproducibility in our field.

Figures (1)

• Figure 1: Heatmaps of attributions of lighthouses, using the pre-trained VGG-16 network provided by Torchvision. The CompositeEpsilonGammaBox was used and the attributions were visualized with the color map coldnhot (negative relevance is light-/blue, irrelevant pixels are black, positive relevance is red to yellow).