Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Readable Twins of Unreadable Models

Krzysztof Pancerz, Piotr Kulicki, Michał Kalisz, Andrzej Burda, Maciej Stanisławski, Jaromir Sarzyński

TL;DR

The paper addresses the challenge of explaining unreadable deep learning models by constructing readable twins, imprecise information flow models, via a Sequential Information System and rough set flow graphs. It details a stepwise pipeline (training, clustering activations, SIS construction, RSFG creation, path mining with an evolutionary algorithm, and visualization) and demonstrates the approach on MNIST digit classification. The key contribution is a concrete methodology that translates internal model behavior into human-readable, path-based explanations, with visual summaries. This work enables principled interpretability of deep models and opens avenues for ontology-based extensions and alternative graphical representations such as Petri nets.

Abstract

Creating responsible artificial intelligence (AI) systems is an important issue in contemporary research and development of works on AI. One of the characteristics of responsible AI systems is their explainability. In the paper, we are interested in explainable deep learning (XDL) systems. On the basis of the creation of digital twins of physical objects, we introduce the idea of creating readable twins (in the form of imprecise information flow models) for unreadable deep learning models. The complete procedure for switching from the deep learning model (DLM) to the imprecise information flow model (IIFM) is presented. The proposed approach is illustrated with an example of a deep learning classification model for image recognition of handwritten digits from the MNIST data set.

Readable Twins of Unreadable Models

TL;DR

The paper addresses the challenge of explaining unreadable deep learning models by constructing readable twins, imprecise information flow models, via a Sequential Information System and rough set flow graphs. It details a stepwise pipeline (training, clustering activations, SIS construction, RSFG creation, path mining with an evolutionary algorithm, and visualization) and demonstrates the approach on MNIST digit classification. The key contribution is a concrete methodology that translates internal model behavior into human-readable, path-based explanations, with visual summaries. This work enables principled interpretability of deep models and opens avenues for ontology-based extensions and alternative graphical representations such as Petri nets.

Abstract

Creating responsible artificial intelligence (AI) systems is an important issue in contemporary research and development of works on AI. One of the characteristics of responsible AI systems is their explainability. In the paper, we are interested in explainable deep learning (XDL) systems. On the basis of the creation of digital twins of physical objects, we introduce the idea of creating readable twins (in the form of imprecise information flow models) for unreadable deep learning models. The complete procedure for switching from the deep learning model (DLM) to the imprecise information flow model (IIFM) is presented. The proposed approach is illustrated with an example of a deep learning classification model for image recognition of handwritten digits from the MNIST data set.

Paper Structure

This paper contains 9 sections, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. From Unreadable Models to Their Readable Twins
  3. Step 1:
  4. Step 2:
  5. Step 3:
  6. Step 4:
  7. Step 5:
  8. Step 6:
  9. Conclusions

Figures (10)

  • Figure 1: An idea of creation of a readable twin (an imprecise information flow model) for an unreadable original model (a deep learning model).
  • Figure 2: A flowchart of the transformation procedure of DLM into IIFM.
  • Figure 3: A fragment of the MNIST data set (by Suvanjanprasai - Own work, CC BY-SA 4.0 https://commons.wikimedia.org/w/index.php?curid=156115980).
  • Figure 4: Creation of a readable twin for an unreadable original model: Step 1.
  • Figure 5: Creation of a readable twin for an unreadable original model: Step 2.
  • ...and 5 more figures