Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization

TL;DR

This work enhances Layer-wise Relevance Propagation (LRP) for neural network interpretability by introducing neural-network graphs, heatmap visualizations, and an optimized path-selection algorithm to pinpoint the most influential neuron paths. By leveraging forward/backward differences through the GetOptimizer method and evaluating via MSE and SMAPE, the approach yields clearer, path-driven explanations on the VGG16 backbone, complemented by deconvolution-based feature-map reconstructions. The combination of path-focused heatmaps and backward visualization improves transparency in CNN decisions, supporting more trustworthy AI for computer vision tasks. The results suggest that targeted neuron-path analysis can make complex models more interpretable without sacrificing performance, facilitating regulatory compliance and user trust in real-world applications.

Abstract

Interpreting complex neural networks is crucial for understanding their decision-making processes, particularly in applications where transparency and accountability are essential. This proposed method addresses this need by focusing on layer-wise Relevance Propagation (LRP), a technique used in explainable artificial intelligence (XAI) to attribute neural network outputs to input features through backpropagated relevance scores. Existing LRP methods often struggle with precision in evaluating individual neuron contributions. To overcome this limitation, we present a novel approach that improves the parsing of selected neurons during LRP backward propagation, using the Visual Geometry Group 16 (VGG16) architecture as a case study. Our method creates neural network graphs to highlight critical paths and visualizes these paths with heatmaps, optimizing neuron selection through accuracy metrics like Mean Squared Error (MSE) and Symmetric Mean Absolute Percentage Error (SMAPE). Additionally, we utilize a deconvolutional visualization technique to reconstruct feature maps, offering a comprehensive view of the network's inner workings. Extensive experiments demonstrate that our approach enhances interpretability and supports the development of more transparent artificial intelligence (AI) systems for computer vision applications. This advancement has the potential to improve the trustworthiness of AI models in real-world machine vision applications, thereby increasing their reliability and effectiveness.

Figures (7)

• Figure 1: LRP computational process with two phases: forward propagation of activation and backward propagation of relevance.
• Figure 2: Architecture of VGG16.
• Figure 3: Highlight longest paths in red, emphasizing their relevance in backward propagation (LRP relevance scores).
• Figure 4: Visualization heatmap based on k-path selection.
• Figure 5: (A) Visualization of Original Images, (B) Model Predictions, (C) Graphs, and (D) Back-predictions for Comparative Analysis in Three Categories: Castle, Barn, and Zebra.