Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attribution Upsampling should Redistribute, Not Interpolate

Vincenzo Buono, Peyman Sheikholharam Mashhadi, Mahmoud Rahat, Prayag Tiwari, Stefan Byttner

Abstract

Attribution methods in explainable AI rely on upsampling techniques that were designed for natural images, not saliency maps. Standard bilinear and bicubic interpolation systematically corrupts attribution signals through aliasing, ringing, and boundary bleeding, producing spurious high-importance regions that misrepresent model reasoning. We identify that the core issue is treating attribution upsampling as an interpolation problem that operates in isolation from the model's reasoning, rather than a mass redistribution problem where model-derived semantic boundaries must govern how importance flows. We present Universal Semantic-Aware Upsampling (USU), a principled method that reformulates upsampling through ratio-form mass redistribution operators, provably preserving attribution mass and relative importance ordering. Extending the axiomatic tradition of feature attribution to upsampling, we formalize four desiderata for faithful upsampling and prove that interpolation structurally violates three of them. These same three force any redistribution operator into a ratio form; the fourth selects the unique potential within this family, yielding USU. Controlled experiments on models with known attribution priors verify USU's formal guarantees; evaluation across ImageNet, CIFAR-10, and CUB-200 confirms consistent faithfulness improvements and qualitatively superior, semantically coherent explanations.

Attribution Upsampling should Redistribute, Not Interpolate

Abstract

Attribution methods in explainable AI rely on upsampling techniques that were designed for natural images, not saliency maps. Standard bilinear and bicubic interpolation systematically corrupts attribution signals through aliasing, ringing, and boundary bleeding, producing spurious high-importance regions that misrepresent model reasoning. We identify that the core issue is treating attribution upsampling as an interpolation problem that operates in isolation from the model's reasoning, rather than a mass redistribution problem where model-derived semantic boundaries must govern how importance flows. We present Universal Semantic-Aware Upsampling (USU), a principled method that reformulates upsampling through ratio-form mass redistribution operators, provably preserving attribution mass and relative importance ordering. Extending the axiomatic tradition of feature attribution to upsampling, we formalize four desiderata for faithful upsampling and prove that interpolation structurally violates three of them. These same three force any redistribution operator into a ratio form; the fourth selects the unique potential within this family, yielding USU. Controlled experiments on models with known attribution priors verify USU's formal guarantees; evaluation across ImageNet, CIFAR-10, and CUB-200 confirms consistent faithfulness improvements and qualitatively superior, semantically coherent explanations.
Paper Structure (67 sections, 24 theorems, 33 equations, 9 figures, 10 tables)

This paper contains 67 sections, 24 theorems, 33 equations, 9 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Spatial Resolution in Feature Attribution
  4. Saliency Map Refinement
  5. Axiomatic Foundations in Explainability
  6. Faithfulness Evaluation
  7. From Interpolation to Redistribution
  8. Problem Setup
  9. Lattice and partitions.
  10. Attribution and mass.
  11. Error model.
  12. Why Interpolation Fails
  13. Desiderata for Faithful Upsampling
  14. Uniqueness and the USU Operator
  15. The USU Operator.
  16. ...and 52 more sections

Key Result

theorem 1

Any interpolation kernel whose support extends across neighbourhood boundaries with non-zero weight violates mass conservation. Specifically, there exists a coarse attribution $A$ such that $\sum_{x \in N_k} \tilde{A}(x) \neq M_k$.

Figures (9)

  • Figure 1: USU replaces interpolation with semantically guided mass redistribution.Left: The pipeline decomposes upsampling into score-potential computation, emergent boundary detection, and ratio-form mass redistribution. Right: Standard interpolation (bilinear, bicubic) bleeds attribution across object boundaries; USU-IWMR produces sharp, semantically coherent saliency maps that faithfully reflect model reasoning.
  • Figure 2: Upsampling corrupts saliency maps through reconstruction errors. Classical saliency upsampling kernels can introduce reconstruction errors, leading to incorrect visual explanations. RRL-constrained 7db2afdc5eb5db46cc64185d0a51ed079b0976e8 model on synthetic shapes; see the supplemental material for setup and extended results.
  • Figure 3: Identical saliency for distinct inputs. Each row shows (left) the input, (middle) the raw heatmap, (right) the upsampled saliency map, and (rightmost) the ground-truth attribution priors. Despite distinct ground truths, the upsampled explanations converge to a single pattern: interpolation cannot distinguish what the model attends to from what it ignores. RRL-constrained 7db2afdc5eb5db46cc64185d0a51ed079b0976e8 model on synthetic shapes; see the supplemental material for setup and extended results.
  • Figure 4: Qualitative comparison. Top row: synthetic patterns with known ground-truth attribution priors. Bottom row: ImageNet examples with GradCAM attributions. Interpolation methods (bilinear, bicubic, Lanczos) produce ringing artifacts, boundary bleeding, and aliasing; USU preserves semantic boundaries and concentrates attribution within model-relevant regions.
  • Figure 5: Complete USU pipeline. Given input $x$, the pipeline proceeds in three stages. Scoring (bottom right): masked perturbation evaluates segment importance, producing the score matrix $\Phi$. Hierarchical refinement (bottom left): the depth fusion loop convolves $\Phi$ with the diagonal difference kernel to compute the H-map, selects heterogeneous segments for re-segmentation, and merges coarse and fine scores so that segment boundaries emerge naturally from the model's own reasoning; convergence is governed by the comparator threshold $\mathcal{L}$. Redistribution (top): the coarse attribution $A(x)$ is projected onto the refined segmentation and redistributed via pixel-level weighting with the tensor potential, yielding $A^{\uparrow}$.
  • ...and 4 more figures

Theorems & Definitions (59)

  • theorem 1: Interpolation Violates Mass Conservation
  • theorem 2: Interpolation Ignores Semantics
  • theorem 3: Interpolation Violates Locality
  • theorem 4: Uniqueness of the Ratio Form
  • proof : Proof sketch
  • theorem 5: (D1)--(D4) Satisfaction
  • corollary 1: Global Conservation
  • proof : Proof of \ref{['thm:interp-d2']} (Interpolation Ignores Semantics)
  • proof : Proof of \ref{['thm:interp-d1']} (Mass Conservation Violation)
  • proof : Proof of \ref{['thm:interp-d4']} (Locality Violation)
  • ...and 49 more