Deep Regression for Repeated Measurements

TL;DR

This work investigates nonparametric mean function regression with repeated measurements using fully connected deep neural network estimators with flexible shapes and illustrates a key phenomenon, the phase transition in the convergence rate, inherent to repeated measurements and its connection to the curse of dimensionality.

Abstract

Nonparametric mean function regression with repeated measurements serves as a cornerstone for many statistical branches, such as longitudinal/panel/functional data analysis. In this work, we investigate this problem using fully connected deep neural network (DNN) estimators with flexible shapes. A novel theoretical framework allowing arbitrary sampling frequency is established by adopting empirical process techniques to tackle clustered dependence. We then consider the DNN estimators for Hölder target function and illustrate a key phenomenon, the phase transition in the convergence rate, inherent to repeated measurements and its connection to the curse of dimensionality. Furthermore, we study several examples with low intrinsic dimensions, including the hierarchical composition model, low-dimensional support set and anisotropic Hölder smoothness. We also obtain new approximation results and matching lower bounds to demonstrate the adaptivity of the DNN estimators for circumventing the curse of dimensionality. Simulations and real data examples are provided to support our theoretical findings and practical implications.

Figures (6)

• Figure 1: Finite sample mean squared error of the DNN estimators for $n=100,200,300,400$ and sampling frequencies $m=1,2,3,5,8,10,12,15,20,25,30,40,50,60,80$. Each subplot portrays sampling frequencies ($m$) on the horizontal axis and logarithmic empirical mean squared error on the vertical axis.
• Figure 2: (a) Graphs depicting estimated function profiles for distinct sample sizes and sampling frequencies. (b) Multiple-dimensional scaling visualization. The numerical values denote $m$, the colors represent $n$, and the black triangles symbolize the estimation with full data.
• Figure 3: Plots of the estimated function with respect to each covariate. In each subplot, other covariates are held at their 0.25/0.50/0.75 quantile.
• Figure S.4: Finite sample mean squared error of DNN estimators for $n=100,200,300,400$ and sampling frequencies $m=1,2,3,5,8,10,12,15,20,25,30,40,50,60,80$. Each subplot portrays sampling frequencies ($m$) on the horizontal axis and logarithmic empirical mean squared error on the vertical axis.
• Figure S.5: Finite sample mean squared error of the DNN estimators for sample size $n=25,100,400$ and overall observation size $nm=400,800,1200,2000,3200,4000,4800,6000,8000,12000,16000,20000,24000,32000$. Each subplot portrays the overall observation size ($nm$) on the horizontal axis and the logarithmic empirical mean squared error on the vertical axis.

Theorems & Definitions (54)

• Definition 2.1: Rademacher fixed point
• Theorem 2.1
• Corollary 2.1
• Corollary 2.2
• Definition 3.1: Hölder space $\mathcal{C}^{s}\left(\Sigma\right)$
• Theorem 3.1
• Theorem 3.2
• Definition 4.1: Hierarchical composition model $\mathcal{H}^{l, \mathcal{G}}(\Sigma)$
• Theorem 4.1
• Theorem 4.2