Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN

Junpeng Zhang, Lei Cheng, Qing Li, Liang Lin, Quanshi Zhang

TL;DR

The paper examines how DNN generalization relates to the complexity of interactions encoded in the network. It introduces a theoretically grounded framework of AND-OR interactions to explain inferences and defines confusing samples as those driven by non-generalizable patterns, with a universal matching property ensuring a sparse surrogate model can replicate outputs across masked inputs. Through extensive experiments, it shows that high-order, complex interactions emerge mainly during overfitting on a small subset of samples, and that different networks—even with similar performance—have largely different confusing samples. Crucially, the composition of confusing samples is governed by the randomness of low-layer parameters, supporting an extended view of the lottery ticket hypothesis where low-layer initialization largely determines representation power, while high-layer parameters and architecture play a comparatively smaller role. These findings offer a new lens on generalization, suggesting that addressing low-layer randomness could more effectively curb overfitting and improve interpretability of DNN decisions, with implications for targeted regularization and model design.

Abstract

In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover that the confusing samples of a DNN, which are represented by non-generalizable interactions, are determined by its low-layer parameters. In comparison, other factors, such as high-layer parameters and network architecture, have much less impact on the composition of confusing samples. Two DNNs with different low-layer parameters usually have fully different sets of confusing samples, even though they have similar performance. This finding extends the understanding of the lottery ticket hypothesis, and well explains distinctive representation power of different DNNs.

Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN

TL;DR

The paper examines how DNN generalization relates to the complexity of interactions encoded in the network. It introduces a theoretically grounded framework of AND-OR interactions to explain inferences and defines confusing samples as those driven by non-generalizable patterns, with a universal matching property ensuring a sparse surrogate model can replicate outputs across masked inputs. Through extensive experiments, it shows that high-order, complex interactions emerge mainly during overfitting on a small subset of samples, and that different networks—even with similar performance—have largely different confusing samples. Crucially, the composition of confusing samples is governed by the randomness of low-layer parameters, supporting an extended view of the lottery ticket hypothesis where low-layer initialization largely determines representation power, while high-layer parameters and architecture play a comparatively smaller role. These findings offer a new lens on generalization, suggesting that addressing low-layer randomness could more effectively curb overfitting and improve interpretability of DNN decisions, with implications for targeted regularization and model design.

Abstract

In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover that the confusing samples of a DNN, which are represented by non-generalizable interactions, are determined by its low-layer parameters. In comparison, other factors, such as high-layer parameters and network architecture, have much less impact on the composition of confusing samples. Two DNNs with different low-layer parameters usually have fully different sets of confusing samples, even though they have similar performance. This finding extends the understanding of the lottery ticket hypothesis, and well explains distinctive representation power of different DNNs.

Paper Structure

This paper contains 25 sections, 1 theorem, 7 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Defining confusing samples with non-generalizable inference patterns
  3. Preliminaries: extracting interactions as inference patterns used by a DNN for inference
  4. Connection between the interaction complexity and the generalization power
  5. Using interactions to define confusing samples
  6. Exploring the key factor that determines the confusing samples encoded by a DNN
  7. Randomness of confusing samples
  8. Impact of parameters in low layers
  9. Impact of high-layer parameters and architecture
  10. How to understand the randomness of low-layer parameters
  11. Conclusions
  12. Related work
  13. Properties of the AND interaction
  14. Common conditions for sparse interactions
  15. Details to extract the sparsest AND-OR interactions
  16. ...and 10 more sections

Key Result

Theorem 2.1

Given a DNN $v$ and an input sample $\mathbf{x}$, if the scalar weights $I^{\text{and}}_T$ and $I^{\text{or}}_T$ in the logical model are set as $\forall T\subseteq N, I^{\text{and}}_T = \sum\nolimits_{L \subseteq T} (-1)^{\vert T \vert - \vert L \vert} u^{\text{and}}_L$ and $I^{\text{or}}_T = - \ \ where $\mathbf{x}_S$ represents a masked input sample only containing input variables in $S$. All o

Figures (11)

  • Figure 1: ren2024we have proven that we can construct a surrogate logical model $h(\mathbf{x})$ consisting of sparse AND-OR interactions, which can universally predict the DNN's inference scores $v(\mathbf{x})$ on an exponential number of masked states of the sample $\mathbf{x}$.
  • Figure 2: Complex and mutually offsetting interactions emerge only on a few samples (confusing samples) in the overfitting phase.
  • Figure 3: Jaccard similarity between interactions\ref{['ref::salient-interactions']} extracted from training samples and those extracted from testing samples. Low Jaccard similarity of high-order interactions indicate the weak generalization power of high-order interactions.
  • Figure 4: (a) Curves of the training loss and testing loss during the training process. (b) The loss gap between the training loss and the testing loss during the training process. (c) Distribution of interactions\ref{['ref::salient-interactions']} over different orders at the end of the learning phase and during the overfitting phase. We averaged the distributions extracted from different samples. Complex and mutually offsetting interactions emerge in the over-fitting phase
  • Figure 5: Distribution of interactions over different orders. We averaged the distributions extracted from different hard samples and averaged the distributions extracted from different easy samples.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Theorem 2.1: Universal matching property, proven in chen2024defining
  • Definition 2.2
  • proof