Explaining Deep Neural Networks by Leveraging Intrinsic Methods
Biagio La Rosa
TL;DR
This work tackles the opacity of deep neural networks by advancing intrinsic explainability through self-explainable architectures, memory-augmented modules, and latent-space analyses. It introduces Prototype-based Interpretable Graph Neural Networks (PIGNN), memory-tracking mechanisms in Simplified Differentiable Neural Computers (SDNC) and Memory Wrap, Graph Concept Whitening (GCW), and Clustered Compositional Explanations (CoEx), each designed to preserve or enhance predictive performance while delivering faithful explanations. Additionally, it surveys the integration of explanations into visual analytics systems to bring explanations to end-users and researchers alike. Collectively, the contributions yield interpretable, faithful, and actionable insights for developers, researchers, and practitioners, enabling debugging, bias detection, and trustworthy AI deployment across graph, sequential, and vision tasks. The work emphasizes a practical balance between interpretability and performance, providing quantitative and qualitative evidence that intrinsic explanations can rival post-hoc methods while offering richer, more usable explanations for real-world use cases, including drug discovery and CV applications. $f_{FE}$ and $f_{CL}$ are used to denote feature extractors and classifiers, prototypes $P$, memory $M$, and concept axes $c_j$, with equations and design choices grounded in $IoU$, fidelity metrics, and memory-tracking mappings that connect internal states to human-interpretable explanations.
Abstract
Despite their impact on the society, deep neural networks are often regarded as black-box models due to their intricate structures and the absence of explanations for their decisions. This opacity poses a significant challenge to AI systems wider adoption and trustworthiness. This thesis addresses this issue by contributing to the field of eXplainable AI, focusing on enhancing the interpretability of deep neural networks. The core contributions lie in introducing novel techniques aimed at making these networks more interpretable by leveraging an analysis of their inner workings. Specifically, the contributions are threefold. Firstly, the thesis introduces designs for self-explanatory deep neural networks, such as the integration of external memory for interpretability purposes and the usage of prototype and constraint-based layers across several domains. Secondly, this research delves into novel investigations on neurons within trained deep neural networks, shedding light on overlooked phenomena related to their activation values. Lastly, the thesis conducts an analysis of the application of explanatory techniques in the field of visual analytics, exploring the maturity of their adoption and the potential of these systems to convey explanations to users effectively.