Bulk-boundary decomposition of neural networks

TL;DR

The bulk-boundary decomposition is presented as a new framework for understanding the training dynamics of deep neural networks and a field-theoretic formulation of neural dynamics based on this decomposition is developed.

Abstract

We present the bulk-boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a data-independent bulk term and a data-dependent boundary term. The bulk captures the intrinsic dynamics set by network architecture and activation functions, while the boundary reflects stochastic interactions from training samples at the input and output layers. This decomposition exposes the local and homogeneous structure underlying deep networks. As a natural extension, we develop a field-theoretic formulation of neural dynamics based on this decomposition.

Figures (2)

• Figure 1: (a) and (b) illustrate two equivalent representations of a deep neural network. The input and output layers act as data-dependent boundaries, while the interior bulk represents architecture-dependent dynamics. Local interactions along the depth direction lead to translational symmetry and enable a field-theoretic continuum limit.
• Figure 2: Illustration of the local neural network architecture considered in the example. Each neuron interacts only with nearby neurons through weights $W$, ensuring locality along the width direction. Periodic boundary conditions are imposed along the width direction. The depth direction corresponds to layer index $m$, along which locality and translation symmetry emerge through the bulk--boundary decomposition.