Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On robust recovery of signals from indirect observations

Yannis Bekri, Anatoli Juditsky, Arkadi Nemirovski

TL;DR

This work addresses robust recovery of a linear image $w_*=Bx_*$ from indirect observations with both random and adversarial perturbations. It introduces an uncertainty-immunization framework based on polyhedral estimates and leverages ellitope structure to obtain computable, near-minimax risk bounds via convex optimization, including special treatments for uncertain-but-bounded nuisances and outliers. The main contributions are (i) a general risk-bound machinery for polyhedral estimates under elliptical signal sets, (ii) efficient design methods for contrast matrices under ellitope and co-ellitope nuisance models, (iii) extensions to sparse outliers with aggregated contrasts and (iv) numerical demonstrations validating improved recovery and risk-bounding performance. The results offer a scalable, theory-backed approach for robust linear inverse problems applicable to signals constrained by quadratic-surface sets, with practical impact in robust regression, compressed sensing, and indirect measurement settings.

Abstract

We consider an uncertain linear inverse problem as follows. Given observation $ω=Ax_*+ζ$ where $A\in {\bf R}^{m\times p}$ and $ζ\in {\bf R}^{m}$ is observation noise, we want to recover unknown signal $x_*$, known to belong to a convex set ${\cal X}\subset{\bf R}^{n}$. As opposed to the "standard" setting of such problem, we suppose that the model noise $ζ$ is "corrupted" -- contains an uncertain (deterministic dense or singular) component. Specifically, we assume that $ζ$ decomposes into $ζ=Nν_*+ξ$ where $ξ$ is the random noise and $Nν_*$ is the "adversarial contamination" with known $\cal N\subset {\bf R}^n$ such that $ν_*\in \cal N$ and $N\in {\bf R}^{m\times n}$. We consider two "uncertainty setups" in which $\cal N$ is either a convex bounded set or is the set of sparse vectors (with at most $s$ nonvanishing entries). We analyse the performance of "uncertainty-immunized" polyhedral estimates -- a particular class of nonlinear estimates as introduced in [15, 16] -- and show how "presumably good" estimates of the sort may be constructed in the situation where the signal set is an ellitope (essentially, a symmetric convex set delimited by quadratic surfaces) by means of efficient convex optimization routines.

On robust recovery of signals from indirect observations

TL;DR

This work addresses robust recovery of a linear image from indirect observations with both random and adversarial perturbations. It introduces an uncertainty-immunization framework based on polyhedral estimates and leverages ellitope structure to obtain computable, near-minimax risk bounds via convex optimization, including special treatments for uncertain-but-bounded nuisances and outliers. The main contributions are (i) a general risk-bound machinery for polyhedral estimates under elliptical signal sets, (ii) efficient design methods for contrast matrices under ellitope and co-ellitope nuisance models, (iii) extensions to sparse outliers with aggregated contrasts and (iv) numerical demonstrations validating improved recovery and risk-bounding performance. The results offer a scalable, theory-backed approach for robust linear inverse problems applicable to signals constrained by quadratic-surface sets, with practical impact in robust regression, compressed sensing, and indirect measurement settings.

Abstract

We consider an uncertain linear inverse problem as follows. Given observation where and is observation noise, we want to recover unknown signal , known to belong to a convex set . As opposed to the "standard" setting of such problem, we suppose that the model noise is "corrupted" -- contains an uncertain (deterministic dense or singular) component. Specifically, we assume that decomposes into where is the random noise and is the "adversarial contamination" with known such that and . We consider two "uncertainty setups" in which is either a convex bounded set or is the set of sparse vectors (with at most nonvanishing entries). We analyse the performance of "uncertainty-immunized" polyhedral estimates -- a particular class of nonlinear estimates as introduced in [15, 16] -- and show how "presumably good" estimates of the sort may be constructed in the situation where the signal set is an ellitope (essentially, a symmetric convex set delimited by quadratic surfaces) by means of efficient convex optimization routines.
Paper Structure (34 sections, 8 theorems, 113 equations, 2 figures)

This paper contains 34 sections, 8 theorems, 113 equations, 2 figures.

Table of Contents

  1. Situation and goals
  2. Introduction
  3. Organization of the paper.
  4. The problem
  5. Observation noise assumption.
  6. Ellitopes
  7. Examples.
  8. Main assumption.
  9. Notation.
  10. Uncertain-but-bounded nuisance
  11. Bounding the $\epsilon$-risk of polyhedral estimate
  12. Synthesis of "presumably good" contrast matrices
  13. Ellitopic case
  14. Co-ellitopic case
  15. Observations with outliers
  16. ...and 19 more sections

Key Result

Proposition 2.1

Given a $m\times I$ contrast matrix $G=[g_1,...,g_I]$, consider the optimization problem where and, from now on, for a nonempty compact set ${\cal Z}\subset {\mathbf{R}}^N$ is the support function of ${\cal Z}$. Let $(\lambda,\mu,\gamma)$ be a feasible solution to the problem in (f_G2). Then i.e., the $\epsilon$-risk of the estimate ${\widehat{w}}_G$ is upper bounded with $f_G(\lambda,\mu,\gam

Figures (2)

  • Figure 1: Left plot: distributions of $\|\cdot\|_2$-errors of recovery of $x_*$ and theoretical upper bounds on ${\hbox{\rm Risk}}_{0.05}$ (red horizontal bars); right plot: distributions of $\|\cdot\|_2$-errors and theoretical upper bounds on ${\hbox{\rm Risk}}_{0.05}$ of recovery of $\nu_*$.
  • Figure 2: A typical signal/estimates realization and recovery errors.

Theorems & Definitions (8)

  • Proposition 2.1
  • Proposition 2.2
  • Theorem 2.1
  • Proposition 3.1
  • Theorem 3.1
  • Proposition 3.2
  • Theorem 3.2
  • Proposition A.1