Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Features

Junpeng Zhang, Qing Li, Liang Lin, Quanshi Zhang

TL;DR

The paper reframes DNN generalization in terms of primitive inference patterns called interactions and reveals a two-phase dynamic in how these interactions of varying order are learned during training. Phase one inhibits medium- and high-order interactions to promote simpler, more generalizable patterns, while phase two gradually increases interaction order, enabling the emergence of overfitted features. The authors provide a formal AND-OR interaction framework, demonstrate alignment between the two-phase dynamics and the training-testing loss gap, and show that high-order interactions generalize less effectively across diverse architectures and tasks. These findings offer a granular mechanism for understanding how DNN representations evolve from underfitting to overfitting and inform when models begin to rely on overfitted features.

Abstract

This paper investigates the dynamics of a deep neural network (DNN) learning interactions. Previous studies have discovered and mathematically proven that given each input sample, a well-trained DNN usually only encodes a small number of interactions (non-linear relationships) between input variables in the sample. A series of theorems have been derived to prove that we can consider the DNN's inference equivalent to using these interactions as primitive patterns for inference. In this paper, we discover the DNN learns interactions in two phases. The first phase mainly penalizes interactions of medium and high orders, and the second phase mainly learns interactions of gradually increasing orders. We can consider the two-phase phenomenon as the starting point of a DNN learning over-fitted features. Such a phenomenon has been widely shared by DNNs with various architectures trained for different tasks. Therefore, the discovery of the two-phase dynamics provides a detailed mechanism for how a DNN gradually learns different inference patterns (interactions). In particular, we have also verified the claim that high-order interactions have weaker generalization power than low-order interactions. Thus, the discovered two-phase dynamics also explains how the generalization power of a DNN changes during the training process.

Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Features

TL;DR

The paper reframes DNN generalization in terms of primitive inference patterns called interactions and reveals a two-phase dynamic in how these interactions of varying order are learned during training. Phase one inhibits medium- and high-order interactions to promote simpler, more generalizable patterns, while phase two gradually increases interaction order, enabling the emergence of overfitted features. The authors provide a formal AND-OR interaction framework, demonstrate alignment between the two-phase dynamics and the training-testing loss gap, and show that high-order interactions generalize less effectively across diverse architectures and tasks. These findings offer a granular mechanism for understanding how DNN representations evolve from underfitting to overfitting and inform when models begin to rely on overfitted features.

Abstract

This paper investigates the dynamics of a deep neural network (DNN) learning interactions. Previous studies have discovered and mathematically proven that given each input sample, a well-trained DNN usually only encodes a small number of interactions (non-linear relationships) between input variables in the sample. A series of theorems have been derived to prove that we can consider the DNN's inference equivalent to using these interactions as primitive patterns for inference. In this paper, we discover the DNN learns interactions in two phases. The first phase mainly penalizes interactions of medium and high orders, and the second phase mainly learns interactions of gradually increasing orders. We can consider the two-phase phenomenon as the starting point of a DNN learning over-fitted features. Such a phenomenon has been widely shared by DNNs with various architectures trained for different tasks. Therefore, the discovery of the two-phase dynamics provides a detailed mechanism for how a DNN gradually learns different inference patterns (interactions). In particular, we have also verified the claim that high-order interactions have weaker generalization power than low-order interactions. Thus, the discovered two-phase dynamics also explains how the generalization power of a DNN changes during the training process.
Paper Structure (16 sections, 3 theorems, 8 equations, 4 figures)

This paper contains 16 sections, 3 theorems, 8 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Analyzing the starting point of learning over-fitted features
  4. Preliminaries: interactions
  5. The two-phase phenomenon
  6. Experiments
  7. Proof and discussion
  8. Conclusion
  9. Experimental detail
  10. Training settings.
  11. Details about how to calculate interactions for different DNNs.
  12. Detail of three conditions for the relatively stable output of a DNN.
  13. Strategies of adding noise labels
  14. Proof details
  15. Proof of Theorem 3 in the main paper
  16. ...and 1 more sections

Key Result

Theorem 1

Let us be given an input sample $\mathbf{x}$ with $n$ input variables. Let us use a threshold $\tau$ to select a set of salient AND interactions $\Gamma$, subject to $\vert I_{\text{and}}(S\vert \mathbf{x})\vert > \tau$. If the DNN's outputs score $v(\mathbf{x}_{T})$ on all $2^n$ samples $\{\mathbf{

Figures (4)

  • Figure 1: (a) Illustration of interactions encoded by a DNN. Each interaction is a metric to measure a non-linear relationship among a specific set $S$ of input variables. (b) The two-phase dynamics is temporally aligned with the change of the gap between the testing and training losses during the learning process. (c) Illustration of the two-phase phenomenon.
  • Figure 2: The positive and negative strength of salient interactions over different orders. Experiments show that various DNNs trained on different datasets for different tasks all exhibited the two-phase phenomenon, and the two-phase phenomenon is temporally aligned with the change of the gap between the testing and training losses.
  • Figure 3: The mean of the Jaccard similarity over the first $10$ categories between interactions extracted from training samples and that extracted from testing samples.
  • Figure 4: The distribution of interactions over different orders extracted from $100$ original samples and extracted from $100$ incorrectly labeled samples.

Theorems & Definitions (5)

  • Theorem 1: Sparsity property, proved by ren2023we
  • Theorem 2: Universal matching, proved by zhou2023explaining
  • Theorem 3: proved in \ref{['prove:throrem:3']}
  • proof
  • proof