Pointwise convergence of Fourier series and deep neural network for the indicator function of d-dimensional ball
Ryota Kawasumi, Tsuyoshi Yoneda
TL;DR
The paper addresses the problem of pointwise convergence when approximating the indicator of a $d$-dimensional ball using Fourier series versus a deliberately designed deep ReLU network. It shows that spherical partial sums of Fourier series exhibit a third phenomenon (Kuratsubo) that prevents pointwise convergence for $d\ge 5$, while constructing a specific deep neural network whose gradient-descent path yields pointwise convergence to the target function with rate $\|f_N(W^t)-f^ irc_N\|^r_{L^r}\lesssim t^{-1/3}$ and whose limit $f^ irc_N$ approaches $f^ irc$ as $N\to\infty$. The contributions include a rigorous, explicit backpropagation and region-decomposition analysis that demonstrates how carefully crafted network architecture and initialization can overcome non-separable function-space barriers. The work illuminates a fundamental difference between Fourier-based approximations and DNN dynamics for discontinuous targets, with implications for understanding DNNs’ capacity to deal with sharp interfaces and Gibbs-type phenomena.
Abstract
In this paper, we clarify the crucial difference between a deep neural network and the Fourier series. For the multiple Fourier series of periodization of some radial functions on $\mathbb{R}^d$, Kuratsubo (2010) investigated the behavior of the spherical partial sum and discovered the third phenomenon other than the well-known Gibbs-Wilbraham and Pinsky phenomena. In particular, the third one exhibits prevention of pointwise convergence. In contrast to it, we give a specific deep neural network and prove pointwise convergence.