Topological Interpretability for Deep-Learning

TL;DR

This paper tackles the challenge of interpreting deep learning decisions in high-stakes settings by introducing a topological interpretability framework based on the Mapper algorithm to construct a feature-space graph that preserves data topology and ties explanations directly to the training data. It combines Topological Data Analysis with a geometric, distance-to-measure approach to identify features and words that inform classifications, yielding both global and local explanations that are stable against perturbation-based methods like LIME and SHAP. The authors implement the framework on a multi-task CNN for cancer pathology reports and on a CNN for the 20 Newsgroups dataset, demonstrating the ability to recover clinically relevant keywords and class-specific terms, along with Lipschitz-style stability measurements. The work provides a practical path toward trustworthy DL decisions by revealing the features and regions the model relies on, potentially enabling vocabulary reduction and improved model calibration in deployed systems.

Abstract

With the growing adoption of AI-based systems across everyday life, the need to understand their decision-making mechanisms is correspondingly increasing. The level at which we can trust the statistical inferences made from AI-based decision systems is an increasing concern, especially in high-risk systems such as criminal justice or medical diagnosis, where incorrect inferences may have tragic consequences. Despite their successes in providing solutions to problems involving real-world data, deep learning (DL) models cannot quantify the certainty of their predictions. These models are frequently quite confident, even when their solutions are incorrect. This work presents a method to infer prominent features in two DL classification models trained on clinical and non-clinical text by employing techniques from topological and geometric data analysis. We create a graph of a model's feature space and cluster the inputs into the graph's vertices by the similarity of features and prediction statistics. We then extract subgraphs demonstrating high-predictive accuracy for a given label. These subgraphs contain a wealth of information about features that the DL model has recognized as relevant to its decisions. We infer these features for a given label using a distance metric between probability measures, and demonstrate the stability of our method compared to the LIME and SHAP interpretability methods. This work establishes that we may gain insights into the decision mechanism of a DL model. This method allows us to ascertain if the model is making its decisions based on information germane to the problem or identifies extraneous patterns within the data.

Figures (6)

• Figure 1: The Mapper algorithm proceeds as follows: In (a), observe a point cloud, filter function, chosen as the height function, and an open cover, denoted by the dashed lines. Fig. (b) we have covered the range of values in (a) by consecutive intervals. Clustering on the intervals in (b), and creating the nerve. In fig. (c), we have the resulting Mapper construction, demonstrating the compressed representation of the original data.
• Figure 2: Our multi-task convolutional neural network architecture used for cancer pathology report information extraction. The network consists of three parallel convolutional filters, followed by a maxpooling and concatenation layer. To each maxpooling layer, we employ a dropout rate of 50%. After applying the dropout, we concatenate the remaining vectors and pass them to a task-specific softmax classification layer.
• Figure 3: A visual representation of our mapper graph for the site task. Colors designate the ground truth label, and the nodes are proportional to the number of documents in the associated node. The purple subgraph at the center is associated with C50 (breast cancer), and the C61 (prostate cancer) cluster is the tan nodes directly above the breast cancer subgraph.
• Figure 4: Two dimensional representation of the learned word embeddings from regions of high-predictive accuracy in our mapper graph. The clusters of words from multiple classes are common words indicating a cancer diagnosis, but are not specific as to a particular site; high-probability words are singletons typically found in regions with other words associated with the same class.
• Figure 5: A visual representation of our mapper graph over the 20 Newsgroups subset. Colors designate the ground truth label, and the nodes are proportional to the number of documents in the associated node. Note that not all subgraphs associated with a specific class label are connected. These smaller, disconnected regions contain documents that share similar text characteristics and are likely to be misclassified by our model.

Theorems & Definitions (2)

• Definition
• Definition