Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Unified Matrix-Spectral Framework for Stability and Interpretability in Deep Learning

Ronald Katende

TL;DR

This work introduces a unified matrix-spectral framework for stability and interpretability in deep learning by representing networks through data-dependent matrix operators and aggregating their spectra into a Global Matrix Stability Index $\mathfrak{S}(f_\theta;\mu,\nu)$. It couples forward sensitivity, attribution robustness, NTK conditioning, and curvature into a single spectral scale and adds spectral entropy $H_S$ to capture typical, not just worst-case, sensitivity, yielding refined analytic guarantees and diagnostics. The Canonical Unified Matrix Stability Theorem connects finite $\mathfrak{S}$ to stability properties, while the Spectral Entropy framework (including SERR) links initialization and depth to stability behavior under NTK dynamics. Empirical results on synthetic data, MNIST, CIFAR-10, and CIFAR-100 show that modest spectral regularization improves attribution stability and reduces distributional changes in attributions, even when global spectra change little, providing a practical toolkit for robustness-aware design and training.

Abstract

We develop a unified matrix-spectral framework for analyzing stability and interpretability in deep neural networks. Representing networks as data-dependent products of linear operators reveals spectral quantities governing sensitivity to input perturbations, label noise, and training dynamics. We introduce a Global Matrix Stability Index that aggregates spectral information from Jacobians, parameter gradients, Neural Tangent Kernel operators, and loss Hessians into a single stability scale controlling forward sensitivity, attribution robustness, and optimization conditioning. We further show that spectral entropy refines classical operator-norm bounds by capturing typical, rather than purely worst-case, sensitivity. These quantities yield computable diagnostics and stability-oriented regularization principles. Synthetic experiments and controlled studies on MNIST, CIFAR-10, and CIFAR-100 confirm that modest spectral regularization substantially improves attribution stability even when global spectral summaries change little. The results establish a precise connection between spectral concentration and analytic stability, providing practical guidance for robustness-aware model design and training.

A Unified Matrix-Spectral Framework for Stability and Interpretability in Deep Learning

TL;DR

This work introduces a unified matrix-spectral framework for stability and interpretability in deep learning by representing networks through data-dependent matrix operators and aggregating their spectra into a Global Matrix Stability Index . It couples forward sensitivity, attribution robustness, NTK conditioning, and curvature into a single spectral scale and adds spectral entropy to capture typical, not just worst-case, sensitivity, yielding refined analytic guarantees and diagnostics. The Canonical Unified Matrix Stability Theorem connects finite to stability properties, while the Spectral Entropy framework (including SERR) links initialization and depth to stability behavior under NTK dynamics. Empirical results on synthetic data, MNIST, CIFAR-10, and CIFAR-100 show that modest spectral regularization improves attribution stability and reduces distributional changes in attributions, even when global spectra change little, providing a practical toolkit for robustness-aware design and training.

Abstract

We develop a unified matrix-spectral framework for analyzing stability and interpretability in deep neural networks. Representing networks as data-dependent products of linear operators reveals spectral quantities governing sensitivity to input perturbations, label noise, and training dynamics. We introduce a Global Matrix Stability Index that aggregates spectral information from Jacobians, parameter gradients, Neural Tangent Kernel operators, and loss Hessians into a single stability scale controlling forward sensitivity, attribution robustness, and optimization conditioning. We further show that spectral entropy refines classical operator-norm bounds by capturing typical, rather than purely worst-case, sensitivity. These quantities yield computable diagnostics and stability-oriented regularization principles. Synthetic experiments and controlled studies on MNIST, CIFAR-10, and CIFAR-100 confirm that modest spectral regularization substantially improves attribution stability even when global spectral summaries change little. The results establish a precise connection between spectral concentration and analytic stability, providing practical guidance for robustness-aware model design and training.
Paper Structure (34 sections, 6 theorems, 35 equations, 3 figures, 1 table)

This paper contains 34 sections, 6 theorems, 35 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Notation
  4. Unified Matrix Framework
  5. Unified matrix observables and stability profile
  6. A Unified Matrix Stability Principle
  7. Unified operator family and global stability index
  8. Standing regularity conditions.
  9. Unified operator family.
  10. A single spectral scale.
  11. Interpretation and immediate consequences.
  12. Aggregation choice.
  13. Canonical unified matrix stability theorem
  14. Standing assumptions.
  15. Analytic Stability via Spectral Structure
  16. ...and 19 more sections

Key Result

Proposition 3.3

The nonzero eigenvalues of $K_\theta$ equal the squared singular values of $G_\theta$.

Figures (3)

  • Figure 1: Synthetic validation of analytic sensitivity bounds and NTK perturbation behavior.
  • Figure 2: Jacobian spectral entropy constrains attribution instability regimes without inducing strict sample-wise ordering.
  • Figure 3: Visual attribution behavior for CIFAR--10 and CIFAR--100. Columns show the input image, unstable gradients, stable gradients, and absolute attribution differences.

Theorems & Definitions (14)

  • Definition 3.1: Spectral concentration
  • Definition 3.2: Matrix stability profile
  • Proposition 3.3: NTK spectrum
  • Definition 4.1: Global Matrix Stability Index
  • Theorem 4.2: Canonical Unified Matrix Stability
  • Remark 4.3: Unification
  • Theorem 5.1: Spectral entropy and expected sensitivity
  • Remark 5.2
  • Theorem 5.3: NTK spectral sensitivity
  • Corollary 5.4: Entropy and worst-case amplification
  • ...and 4 more