EVO-LRP: Evolutionary Optimization of LRP for Interpretable Model Explanations

TL;DR

EVO-LRP tackles the challenge of producing interpretable explanations for deep networks by optimizing Layer-wise Relevance Propagation (LRP) rules with Covariance Matrix Adaptation Evolution Strategy (CMA-ES). By directly tuning LRP hyperparameters against objective interpretability metrics ($FC$, $SP$, $AS$), it yields explanations that are more faithful, concise, and robust, and that adapt to specific target classes. The framework demonstrates that principled, task-aligned optimization can systematically improve attribution quality and enable class-discriminative heatmaps, including complementary composite maps. Empirically, EVO-LRP outperforms standard baselines on ImageNet with clear qualitative gains in the coherence and discriminative power of class-specific relevance maps, suggesting practical impact for high-stakes domains requiring transparent decision-making.

Abstract

Explainable AI (XAI) methods help identify which image regions influence a model's prediction, but often face a trade-off between detail and interpretability. Layer-wise Relevance Propagation (LRP) offers a model-aware alternative. However, LRP implementations commonly rely on heuristic rule sets that are not optimized for clarity or alignment with model behavior. We introduce EVO-LRP, a method that applies Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to tune LRP hyperparameters based on quantitative interpretability metrics, such as faithfulness or sparseness. EVO-LRP outperforms traditional XAI approaches in both interpretability metric performance and visual coherence, with strong sensitivity to class-specific features. These findings demonstrate that attribution quality can be systematically improved through principled, task-specific optimization.

Figures (6)

• Figure 1: Quantitative benchmark comparison of EVO-LRP (URO LRP-$\alpha\beta$ and LRP-$\epsilon$) against baselines across key XAI metrics. Error bars: standard deviation. While EVO-LRP systematically optimizes LRP-$\epsilon$ to high sparsity, LRP-$\alpha\beta$ achieves a superior balance of high faithfulness, high sparseness, and low sensitivity. This comprehensive improvement is crucial for generating trustworthy, understandable, and reliable explanations, addressing shortcomings of existing methods.
• Figure 2: LRP-$\alpha\beta$ all-class relevance maps from EVO-LRP, optimized for individual metrics (faithfulness, avg. sensitivity, sparseness). Faithfulness/sensitivity maps capture broad semantic regions critical to model decisions. The sparseness-optimized map (right) distinctively highlights object boundaries (positive: red, negative: blue). This emergent edge-detection is interesting as it mirrors a fundamental aspect of human visual perception. Edge-detection suggests the model may recognize objects via salient contours, offering insight into its learned features beyond just diffuse areas.
• Figure 3: Comparison of relevance maps: Our composite EVO-LRP map (far right) versus standard baselines (GradCAM, LIME, IG, LRP-0). Positive relevance is red, negative blue. We clamped the top and bottom 1% of relevance scores to minimize the effect of outliers. The composite map, synthesizing strengths from different EVO-LRP optimizations, is visibly sharper, less noisy, and more semantically aligned.
• Figure 4: EVO-LRP class-specific maps demonstrate clear distinction between competing classes and identification of relevant features. Heatmaps (positive: red, negative: blue) highlight unique semantic features for each target class. This ability to provide targeted, discriminative explanations is vital for understanding why a model chose a specific class over alternatives or what features it associates with a class, crucial for debugging and trust.
• Figure 5: Relevance maps from LRP-$\alpha\beta$ all-class models after bi-objective EVO-LRP optimization for combinations of faithfulness, average sensitivity, and sparseness. For visual clarity and to emphasize the primary positive contributions supporting class predictions, these maps focus on positive relevance scores (red). The rightmost column shows maps from single-objective faithfulness optimization for comparison. The visual similarity across strategies, particularly for specific metric combinations like Sparseness/Faithfulness, supports the quantitative finding that multi-objective optimization often converges to solutions similar to strong single-objective results.