Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition

Lukas Miklautz, Chengzhi Shi, Andrii Shkabrii, Theodoros Thirimachos Davarakis, Prudence Lam, Claudia Plant, Jennifer Dy, Stratis Ioannidis

TL;DR

H-SPLID introduces a saliency-preserving latent decomposition that splits a neural network's latent space into a low-dimensional salient subspace and a non-salient subspace, guided by HSIC-based regularization. The method uses masked latent representations and a combination of clustering and HSIC penalties to encourage task-relevant information while compressing redundancy, yielding improved robustness to perturbations without adversarial training. Theoretical guarantees link perturbation sensitivity to the salient subspace dimension $s$ and the HSIC between inputs and salient representations, and bound the vulnerable region volume. Empirically, H-SPLID achieves superior robustness and saliency transfer across diverse datasets (synthetic and real-world) and maintains high clean accuracy, validating the saliency-preserving decomposition as a robust representation learning strategy.

Abstract

We introduce H-SPLID, a novel algorithm for learning salient feature representations through the explicit decomposition of salient and non-salient features into separate spaces. We show that H-SPLID promotes learning low-dimensional, task-relevant features. We prove that the expected prediction deviation under input perturbations is upper-bounded by the dimension of the salient subspace and the Hilbert-Schmidt Independence Criterion (HSIC) between inputs and representations. This establishes a link between robustness and latent representation compression in terms of the dimensionality and information preserved. Empirical evaluations on image classification tasks show that models trained with H-SPLID primarily rely on salient input components, as indicated by reduced sensitivity to perturbations affecting non-salient features, such as image backgrounds. Our code is available at https://github.com/neu-spiral/H-SPLID.

H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition

TL;DR

H-SPLID introduces a saliency-preserving latent decomposition that splits a neural network's latent space into a low-dimensional salient subspace and a non-salient subspace, guided by HSIC-based regularization. The method uses masked latent representations and a combination of clustering and HSIC penalties to encourage task-relevant information while compressing redundancy, yielding improved robustness to perturbations without adversarial training. Theoretical guarantees link perturbation sensitivity to the salient subspace dimension and the HSIC between inputs and salient representations, and bound the vulnerable region volume. Empirically, H-SPLID achieves superior robustness and saliency transfer across diverse datasets (synthetic and real-world) and maintains high clean accuracy, validating the saliency-preserving decomposition as a robust representation learning strategy.

Abstract

We introduce H-SPLID, a novel algorithm for learning salient feature representations through the explicit decomposition of salient and non-salient features into separate spaces. We show that H-SPLID promotes learning low-dimensional, task-relevant features. We prove that the expected prediction deviation under input perturbations is upper-bounded by the dimension of the salient subspace and the Hilbert-Schmidt Independence Criterion (HSIC) between inputs and representations. This establishes a link between robustness and latent representation compression in terms of the dimensionality and information preserved. Empirical evaluations on image classification tasks show that models trained with H-SPLID primarily rely on salient input components, as indicated by reduced sensitivity to perturbations affecting non-salient features, such as image backgrounds. Our code is available at https://github.com/neu-spiral/H-SPLID.
Paper Structure (49 sections, 8 theorems, 75 equations, 4 figures, 18 tables, 1 algorithm)

This paper contains 49 sections, 8 theorems, 75 equations, 4 figures, 18 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Salient Feature Learning.
  4. Feature Decomposition and Selection.
  5. Adversarial Robustness and Saliency.
  6. Methodology
  7. Problem Setup
  8. Saliency-Aware Latent Decomposition
  9. Regularizing and Preserving the Separated Features
  10. The H-SPLID Algorithm
  11. Theoretical Guarantees
  12. Experiments
  13. Datasets and Encoder Models
  14. Comparison Methods.
  15. Testset Attacks.
  16. ...and 34 more sections

Key Result

Theorem 3.2

Let $\mathbf{x}$ be sampled from a tMVN distribution $\mathbf{x} \sim \mathcal{N}_R(0, \sigma^2 I_d)$ with radius $R$, and let the neural network $h_{\boldsymbol\theta} : \mathbb{R}^d \to \mathbb{R}^k$ be differentiable almost everywhere with an $L$-Lipschitz encoder $f_\psi$ and a bounded linear ou where $\mathbf{z}_s := \mathbf{M}_s f_{\boldsymbol\psi}(\mathbf{x})$ is the salient representation

Figures (4)

  • Figure 1: H-SPLID learns to ignore irrelevant input by decomposing the latent space into salient and non-salient components.Left: A simple diagnostic test for saliency, where the model is trained to classify the left digit (only labels for the left are provided) and it should ignore the right. Middle: A model trained with cross-entropy loss achieves high test accuracy (96.86%) but produces entangled representations, making it sensitive to perturbations on the right digit (accuracy drops to 31.76% under high-magnitude PGD attack). Right: H-SPLID separates the latent space into a salient subspace, which captures class-discriminative structure (ACC 97.14%), and a non-salient subspace, which contains no class-relevant information (ACC 8.57%). This separation enables robustness to perturbations on irrelevant input (ACC 87.46%), showing proper learning of salient features. Embeddings are visualized using t-SNE tsne.
  • Figure 2: Overview of H-SPLID. The salient information for classifying the black bear is encoded in the salient space $\mathbf{z}_s$, whereas the background information is encoded in the non-salient space $\mathbf{z}_n$, allowing H-SPLID to be more robust to perturbations $\delta$ of the background.
  • Figure 3: (a) Samples from the CounterAnimal dataset. Common set (left), counter set (right). (b) Adversarial attacks on non-salient features. We attack the non-salient background of COCO images (left) given their corresponding block (upper right) or background mask (lower right) to test whether models successfully learned salient features.
  • Figure 4: Samples from the three ImageNet9 variations: IN-9 (original), Only-FG, and Mixed-Rand.

Theorems & Definitions (15)

  • Theorem 3.2: HSIC-Based Robustness Bound
  • Corollary 3.3: Salient Region Volume Bound via HSIC
  • Proposition A.1: Covariance Bounded by HSIC gretton2005measuringrobustlearning-v119-greenfeld20a
  • Lemma C.1: Bounded NN with Saliency Space
  • Corollary C.2: Bounded Function Spaces
  • Lemma C.3: Rescaled RKHS Equals $\mathcal{C}_b(\mathcal{X}, \mathbb{R}^d)$
  • Lemma C.4: Scaling of Supremum Covariance — Sum Version
  • Lemma C.5: Stein’s Lemma for Scalar-Valued Functions on a Bounded Domain
  • proof
  • proof
  • ...and 5 more