Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tractable downfall of basis pursuit in structured sparse optimization

Maya V. Marmary, Christian Grussler

TL;DR

The paper investigates when Basis Pursuit ($\ell_1$-minimization) fails to recover the sparsest solution to $V u = y$ for structured matrices $V$ that arise in control and signal processing. It provides a deterministic failure criterion based on $p_k=\|V_{(:,(1:r))}^{\dagger}V_{(:,\{r+k\})}\|_1$ and shows how total positivity yields tractable guarantees; for system matrices $V=\mathcal{C}^N(A,b)$, $p(k)$ is unimodal, enabling efficient identification of non-recoverable tail entries and a binary search over a critical index. The work connects to dual-norm theory, variation bounding, and the theory of sign-consistent/totally positive matrices, and demonstrates applications to fuel-optimal discrete-time control and to compressed sensing with function dictionaries. These results offer design insights for sparse optimization problems, suggesting how horizon length, pole placement, and matrix structure can influence BP performance while providing a principled route to certify uniqueness and recovery properties. The findings also contribute to the algebraic theory of log-concavity for symmetric polynomials and extend beyond control to broader structured sensing scenarios.

Abstract

The problem of finding the sparsest solution to a linear underdetermined system of equations, often appearing, e.g., in data analysis, optimal control, system identification, or sensor selection problems, is considered. This non-convex problem is commonly solved by convexification via $\ell_1$-norm minimization, known as basis pursuit (BP). In this work, a class of structured matrices, representing the system of equations, is introduced for which (BP) tractably fails to recover the sparsest solution. In particular, this enables efficient identification of matrix columns corresponding to unrecoverable non-zero entries of the sparsest solution and determination of the uniqueness of such a solution. These deterministic guarantees complement popular probabilistic ones and provide insights into the a priori design of sparse optimization problems. As our matrix structures appear naturally in optimal control problems, we exemplify our findings based on a fuel-optimal control problem for a class of discrete-time linear time-invariant systems. Finally, we draw connections of our results to compressed sensing and common basis functions in geometric modeling.

Tractable downfall of basis pursuit in structured sparse optimization

TL;DR

The paper investigates when Basis Pursuit (-minimization) fails to recover the sparsest solution to for structured matrices that arise in control and signal processing. It provides a deterministic failure criterion based on and shows how total positivity yields tractable guarantees; for system matrices , is unimodal, enabling efficient identification of non-recoverable tail entries and a binary search over a critical index. The work connects to dual-norm theory, variation bounding, and the theory of sign-consistent/totally positive matrices, and demonstrates applications to fuel-optimal discrete-time control and to compressed sensing with function dictionaries. These results offer design insights for sparse optimization problems, suggesting how horizon length, pole placement, and matrix structure can influence BP performance while providing a principled route to certify uniqueness and recovery properties. The findings also contribute to the algebraic theory of log-concavity for symmetric polynomials and extend beyond control to broader structured sensing scenarios.

Abstract

The problem of finding the sparsest solution to a linear underdetermined system of equations, often appearing, e.g., in data analysis, optimal control, system identification, or sensor selection problems, is considered. This non-convex problem is commonly solved by convexification via -norm minimization, known as basis pursuit (BP). In this work, a class of structured matrices, representing the system of equations, is introduced for which (BP) tractably fails to recover the sparsest solution. In particular, this enables efficient identification of matrix columns corresponding to unrecoverable non-zero entries of the sparsest solution and determination of the uniqueness of such a solution. These deterministic guarantees complement popular probabilistic ones and provide insights into the a priori design of sparse optimization problems. As our matrix structures appear naturally in optimal control problems, we exemplify our findings based on a fuel-optimal control problem for a class of discrete-time linear time-invariant systems. Finally, we draw connections of our results to compressed sensing and common basis functions in geometric modeling.

Paper Structure

This paper contains 24 sections, 15 theorems, 75 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations
  4. Sets
  5. Matrices
  6. Functions
  7. Dual Minimum Norm Problem
  8. Linear Time-Invariant Discrete-Time Systems
  9. Variation bounding and total positivity
  10. Main results
  11. General Failure Condition
  12. Tractability for Systems Matrices
  13. General Tractability & Compressed Sensing
  14. Example -- Fuel Optimal Control
  15. Conclusion
  16. ...and 9 more sections

Key Result

Proposition 1

[proposition]lem:opt_cond Let $V \in\mathds{R}^{m\times n}$ and $y\in\mathds{R}^m$ be such that Then, $\min_{u\in D}\| {} u{} \|_{ } = \max_{\|V^\mathsf{T}\beta\|_\ast\leq 1} y^\mathsf{T} \beta$ and if the minimum is achieved by $u^\ast$ and the maximum by $\beta^\ast$, then $u^\ast$ is aligned with $V^\mathsf{T} \beta^\ast$.

Figures (5)

  • Figure 1: blueThe solution of \ref{['eq:ell_1']} applied to \ref{['eq:ctrl_intro']} with $A = \textnormal{diag}(0.80.70.60.50.4)$, $b = 1\cdots1^\mathsf{T}$, $\xi = 8.063233.9728163.715110229534.2432^\mathsf{T}$ and $N=40$. While the unique solution to \ref{['eq:ctrl_intro']} is given by $u^\ast$ with $u^\ast(0) = 1$, $u^\ast(9) = -1$ and $u^\ast(t) = 0$ for $t \not \in \{0,9\}$, \ref{['eq:ell_1']} fails to recover $u^\ast$ and instead returns \ref{['line:recov_v2']} with twice as many non-zero entries.
  • Figure 2: blueIllustration of $p_k := \| {} V_{(:,(1:m))}^{-1}V_{(:,\{k\})}{} \|_{ \ell_1}$ for $V = bA b\dotsA^{49} b$ with $A$ and $b$ as for \ref{['fig:ctrl_results_intro']}. Our results guarantee that $p_k$ is log-concave and, thus, unimodal (i.e., single-peaked). Consequently, since $p_k$ crosses $1$ at $k = 36$, $p_k$ has to remain below $1$ for all $k \geq 36$. It is, further, shown that this implies that a solution $u^\ast$ to \ref{['eq:ctrl_intro']} can only be recovered by \ref{['eq:ell_1']} if $u^\ast(t) = 0$ for all $t \geq 36$. Thus, providing a tractable explanation for the failure of \ref{['eq:ell_1']} in \ref{['fig:ctrl_results_intro']}.
  • Figure 3: Illustration of the vector $p := (\| {} V_{(:,1:m)}^{\dagger }V_{(:,\{k\})}{} \|_{ \ell_1})_{k \in (1:n)}\in \mathds{R}^N$ corresponding to \ref{['ex:hankel_multi_mod']} with $N=20$. $p$ contains many local extrema and drops below $1$ at $k=11$ and $k \in (17:20)$, which makes the application of a principled search method challenging.
  • Figure 4: Illustration of the vector $p := (\| {} V_{(:,1:m)}^{\dagger }V_{(:,\{k\})}{} \|_{ \ell_1})_{k \in (1:n)}\in \mathds{R}^{11}$ in \ref{['example:bernstein']}, where $V$ results from the Bernstein basis polynomials of order $10$. Despite $\Delta (V^\mathsf{T}) \not \in {SC}_{4}$, $p$ is still unimodal, indicating that unimodality can be achieved under less restrictive assumptions than those in \ref{['thm:unimodal']}. Moreover, $p$ drops below $1$ at $k = 10$, showing that our discussed failure phenomena can also be found within contexts other than system matrices.
  • Figure 5: Illustration of the predicted unimodality of the vector $p_k := \| {} V_{((:,(1:m))}^{-1}V_{(:,\{k\})}{} \|_{ \ell_1}$ when $A$ is as in \ref{['exm:mat_v1']} and $k \in (1:500)$. $p_k$ remains above $1$ due to the narrowly clustered eigenvalues of $A$.

Theorems & Definitions (37)

  • Proposition 1
  • Definition 1
  • Definition 2
  • Lemma 1
  • Definition 3
  • Lemma 2
  • Proposition 2
  • Proposition 3
  • Lemma 3
  • Theorem 1
  • ...and 27 more