Nonparametric Exponential Family Regression Under Star-Shaped Constraints
Guanghong Yi, Matey Neykov
TL;DR
This work studies the minimax estimation problem for the natural parameter $\boldsymbol{\theta}$ in nonparametric exponential family regression under star-shaped parameter constraints $K \subset [-M,M]^n$. It develops a universal tree-based estimator, deriving matching lower and upper bounds via KL-divergence/Fano arguments and local entropy, culminating in a minimax rate of $\varepsilon^{*2} \wedge d^2$ with $\varepsilon^{*}=\sup\{\varepsilon: \varepsilon^2 \kappa(M) \le \log N^{\operatorname{loc}}(\varepsilon, c)\}$. An illustrative Bernoulli monotone example demonstrates regime-specific rates: $\varepsilon^2 \asymp n^{1/3}$ for $q=1$, $\varepsilon^2 \asymp \sqrt{n}\log n$ for $q=2$, and $\varepsilon^2 \asymp n^{1-1/q}$ for $q\ge 3$. The results extend minimax theory from Gaussian sequence models to general smooth exponential families under star-shaped constraints, with implications for nonparametric regression under complex shape constraints and local-geometric complexity.
Abstract
We study the minimax rate of estimation in nonparametric exponential family regression under star-shaped constraints. Specifically, the parameter space $K$ is a star-shaped set contained within a bounded box $[-M, M]^n$, where $M$ is a known positive constant. Moreover, we assume that the exponential family is nonsingular and that its cumulant function is twice continuously differentiable. Our main result shows that the minimax rate for this problem is $\varepsilon^{*2} \wedge \operatorname{diam}(K)^2$, up to absolute constants, where $\varepsilon^*$ is defined as \[ \varepsilon^* = \sup \{\varepsilon: \varepsilon^2 κ(M) \leq \log N^{\operatorname{loc}}(\varepsilon)\}, \] with $N^{\operatorname{loc}}(\varepsilon)$ denoting the local entropy and $κ(M)$ is an absolute constant allowed to depend on $M$. We also provide an example and derive its corresponding minimax optimal rate.