Minimax rates of convergence for nonparametric regression under adversarial attacks
Jingfu Peng, Yuhong Yang
TL;DR
This work investigates the minimax rates of nonparametric regression under adversarial input perturbations in the sup-norm. It derives a closed-form for the ideal adversarial loss and shows that the adversarial minimax rate decomposes into the standard non-adversarial rate plus the maximal variation of the true function within the adversarial neighborhood, enabling a modular analysis. An adversarial plug-in estimator, built from a standard minimax-optimal estimator, is shown to achieve the optimal rate, and the results are specialized to isotropic and anisotropic Hölder classes, with simulations validating the theory. The findings provide a general framework for robust nonparametric estimation under adversarial perturbations and offer practical guidance for designing resistant estimators in high-stakes settings.
Abstract
Recent research shows the susceptibility of machine learning models to adversarial attacks, wherein minor but maliciously chosen perturbations of the input can significantly degrade model performance. In this paper, we theoretically analyse the limits of robustness against such adversarial attacks in a nonparametric regression setting, by examining the minimax rates of convergence in an adversarial sup-norm. Our work reveals that the minimax rate under adversarial attacks in the input is the same as sum of two terms: one represents the minimax rate in the standard setting without adversarial attacks, and the other reflects the maximum deviation of the true regression function value within the target function class when subjected to the input perturbations. The optimal rates under the adversarial setup can be achieved by an adversarial plug-in procedure constructed from a minimax optimal estimator in the corresponding standard setting. Two specific examples are given to illustrate the established minimax results.