Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Outlier Robust Multivariate Polynomial Regression

Vipul Arora, Arnab Bhattacharyya, Mathews Boban, Venkatesan Guruswami, Esty Kelman

TL;DR

This work generalizes Kane, Karmalkar, and Price's results to the $n$-variate setting, showing an algorithm that achieves a sample complexity of $O_n(d^n\log d)$, where the hidden constant depends on $n$, if $\chi$ is the $n$-dimensional Chebyshev distribution.

Abstract

We study the problem of robust multivariate polynomial regression: let $p\colon\mathbb{R}^n\to\mathbb{R}$ be an unknown $n$-variate polynomial of degree at most $d$ in each variable. We are given as input a set of random samples $(\mathbf{x}_i,y_i) \in [-1,1]^n \times \mathbb{R}$ that are noisy versions of $(\mathbf{x}_i,p(\mathbf{x}_i))$. More precisely, each $\mathbf{x}_i$ is sampled independently from some distribution $χ$ on $[-1,1]^n$, and for each $i$ independently, $y_i$ is arbitrary (i.e., an outlier) with probability at most $ρ< 1/2$, and otherwise satisfies $|y_i-p(\mathbf{x}_i)|\leqσ$. The goal is to output a polynomial $\hat{p}$, of degree at most $d$ in each variable, within an $\ell_\infty$-distance of at most $O(σ)$ from $p$. Kane, Karmalkar, and Price [FOCS'17] solved this problem for $n=1$. We generalize their results to the $n$-variate setting, showing an algorithm that achieves a sample complexity of $O_n(d^n\log d)$, where the hidden constant depends on $n$, if $χ$ is the $n$-dimensional Chebyshev distribution. The sample complexity is $O_n(d^{2n}\log d)$, if the samples are drawn from the uniform distribution instead. The approximation error is guaranteed to be at most $O(σ)$, and the run-time depends on $\log(1/σ)$. In the setting where each $\mathbf{x}_i$ and $y_i$ are known up to $N$ bits of precision, the run-time's dependence on $N$ is linear. We also show that our sample complexities are optimal in terms of $d^n$. Furthermore, we show that it is possible to have the run-time be independent of $1/σ$, at the cost of a higher sample complexity.

Outlier Robust Multivariate Polynomial Regression

TL;DR

This work generalizes Kane, Karmalkar, and Price's results to the -variate setting, showing an algorithm that achieves a sample complexity of , where the hidden constant depends on , if is the -dimensional Chebyshev distribution.

Abstract

We study the problem of robust multivariate polynomial regression: let be an unknown -variate polynomial of degree at most in each variable. We are given as input a set of random samples that are noisy versions of . More precisely, each is sampled independently from some distribution on , and for each independently, is arbitrary (i.e., an outlier) with probability at most , and otherwise satisfies . The goal is to output a polynomial , of degree at most in each variable, within an -distance of at most from . Kane, Karmalkar, and Price [FOCS'17] solved this problem for . We generalize their results to the -variate setting, showing an algorithm that achieves a sample complexity of , where the hidden constant depends on , if is the -dimensional Chebyshev distribution. The sample complexity is , if the samples are drawn from the uniform distribution instead. The approximation error is guaranteed to be at most , and the run-time depends on . In the setting where each and are known up to bits of precision, the run-time's dependence on is linear. We also show that our sample complexities are optimal in terms of . Furthermore, we show that it is possible to have the run-time be independent of , at the cost of a higher sample complexity.
Paper Structure (30 sections, 42 theorems, 146 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 30 sections, 42 theorems, 146 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Main results
  3. Main technical contributions
  4. Related Work
  5. Technical Overview
  6. KKP's Algorithm and Analysis
  7. Our Results
  8. Generalizing to the multivariate case ($n>1$):
  9. Overview of the algorithms and analyses:
  10. Relating $\ell_\infty$ and $\ell_1$ norms of $p$.
  11. Organization.
  12. Preliminaries
  13. Notations.
  14. Chebyshev Partition.
  15. Main algorithmic result
  16. ...and 15 more sections

Key Result

Theorem 1

Let $\sigma\geq 0,\eta>0$, and $\rho$ be any constant $<1/2$. There is an algorithm that almost solves the defn:problem_defn with a constant approximation factor, up to an additive error of $\eta$. The output of the algorithm is $\widehat{p}\in \mathcal{P}_{d}$ that satisfies with probability at least $2/3$. It uses $M=O_n(d^n\log d)$ samples drawn from the multidimensional Chebyshev distribution

Figures (3)

  • Figure 1: An illustration of a 2-dimensional $(7,2)$-Chebyshev partition (in red) super-imposed on the 2-dimensional solid cube $\mathop{\mathrm{\cal C}}\nolimits_2=[-1,1]^2$, with boundary in blue. The cells are indexed by their bottom-left Chebyshev extremas (the red points). \ref{['thm:optimal-l-infty-bound']} essentially proves that on any cell, for example, $\mathop{\mathrm{\cal C}}\nolimits_{(5,5)}$ (in black), $p$ can be well approximated by its evaluation on one arbitrary point $\mathbf{x}^{((5,5))}\in\mathop{\mathrm{\cal C}}\nolimits_{(5,5)}$.
  • Figure 2: An illustration of cell-refinement in 2-dimensional Chebyshev grids: a $(3m,2)$-grid (in green) super-imposed on a $(m,2)$-Chebyshev cell $\mathop{\mathrm{\cal C}}\nolimits_{(j_1,j_2)}$ (in red). The samples from middle-most cell $\mathfrak{C}_{(3j_1-1,3j_2-1)}$ (in blue) only are retained, and median-recovery is applied on them.
  • Figure 3: Translation for constructing $J_2$: For every $\mathbf{x}_i\in J_1$ (blue), the line segments $J_1^{(\mathbf{x}_i)}$ (green) are translated to have their left ends at $0$, so that they all cover the rectangle (red) of height $|J_1|\geq r$, and width $r$. Thus, $|J_2|\geq r^2$.

Theorems & Definitions (89)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Remark 6
  • Remark 7
  • Theorem 8
  • Theorem 9
  • Proposition 9
  • ...and 79 more