The Generalized Matrix Norm Problem

TL;DR

It is shown that the dual formulation of the problem of computing the operator norm of a matrix with respect to norms induced by linear operators can be expressed as a max-min problem which can be solved in polynomial time.

Abstract

We study the computability of the operator norm of a matrix with respect to norms induced by linear operators. Our findings reveal that this problem can be solved exactly in polynomial time in certain situations, and we discuss how it can be approximated in other cases. Along the way, we investigate the concept of push-forward and pull-back of seminorms, which leads us to uncover novel duality principles that come into play when optimizing over the unit ball of norms.

Figures (5)

• Figure 1: An overview of the hardness results presented in Kulmburg2024, adapted to reflect the complexity of computing generalized $p\mapsto q;\bm{\underline{{B}}}$-norms. The complexity class $\mathsf{FPTAS}$ is the class of problems that can be approximated within arbitrary accuracy in polynomial time with respect to the representation size of $\bm{\underline{{A}}}$ and $\bm{\underline{{B}}}$ and the accuracy parameter. A solid boundary line means that the boundary is included in the region. For more information, we refer to Kulmburg2024. This graph was inspired by Bhattiprolu2023.
• Figure 1: Numerical results for $q=1$ and $p=1.5$. The solid line corresponds to the theoretical worst case approximation ratio from Theorem \ref{['thm:main']}, while the dashed line corresponds to that of Corollary \ref{['cor:main_nesterov']}. Both theoretical bounds over-approximate the worst-case approximation ratio we measured experimentally.
• Figure 2: An overview of our results from Sections \ref{['sec:tractable']} and \ref{['sec:approximable']} for the computability/approximability of generalized matrix norms. A solid boundary line means that the boundary is included in the region. Here, $C(p,l,m) = \min\{\gamma_p\sqrt{m},l^{1/p-1/2}\}/\gamma_1$, where $m$ and $l$ are the number of rows of $\bm{\underline{{A}}}$ and $\bm{\underline{{B}}}$, respectively, and $\gamma_s$ for $s\in [1,\infty)$ is the $s$-th root of the $s$-th absolute moment of a Gaussian random variable. This graph was inspired by Bhattiprolu2023.
• Figure 2: Numerical results for $q=1$ and $p=2$. The semidefinite relaxation yields, as expected, a worst-case approximation ratio better than $\gamma_2/\gamma_1 = \sqrt{\pi/2}$. It also performs significantly better than the linear relaxation.
• Figure 3: Numerical results for $q=1$ and $p=1$. Since, in this case, the exact result can be formulated as a zonotope containment problem, a larger number of cases could be evaluated.

Theorems & Definitions (48)

• Definition 2.1: Extended Seminorms and Norms
• Definition 2.2: $L_{p,q}$- and $L_{p,q}^\top$-norms
• Lemma 2.3: Dual of $L_{p,q}$-norms
• Proof 1
• Definition 2.4: Banach Space Adjoint
• Lemma 2.5
• Proof 2
• Proposition 3.1: Constrained Matrix Norm Optimization
• Corollary 3.2
• Proof 3