Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Which variables of a numerical problem cause ill-conditioning?

Nick Dewaele

TL;DR

This work develops a general framework for the condition number of constant-rank elimination problems (CREPs) with latent variables, enabling analysis of ill-conditioning when solving for a subset of variables $y$ given input $x$ and latent $z$. It recasts conditioning in geometric terms via a canonical solution map $H$ whose derivative $DH(x_0)$ defines the condition number $\kappa_{x\mapsto y}[F](x_0,y_0,z_0)=\|DH(x_0)\|$, and shows $\kappa_{x\mapsto y}[F]\leq\kappa_{x\mapsto(y,z)}[F]$, implying solving for $y$ is no harder than solving for both $y$ and $z$ together. The paper provides practical computation routes, including a matrix-form expression $\kappa_{x\mapsto y}=\|A^+-Q^T J_x0\|$ and a LS-based formulation, all hinging on Jacobians and projections. It then applies the framework to orthogonal Tucker decompositions, deriving explicit condition numbers: for each mode $d$, $\kappa_{\mathpzc{X}\mapsto U_d}=0$ if $U_d$ is square, and $1/\sigma_{\min}(\mathpzc{C}_{(d)})$ otherwise, with the full decomposition conditioning given by the maximum over these values; and $\kappa_{\mathpzc{X}\mapsto \mathpzc{C}}=1$. This contributes a principled lens to identify and quantify ill-conditioned components in nonlinear, potentially underdetermined systems, including tensor decompositions, and offers practical tools for sensitivity analysis and model reduction.

Abstract

We study a broad class of numerical problems that can be defined as the solution of a system of (nonlinear) equations for a subset of the dependent variables. Given a system of the form $F(x,y,z) = c$ with multivariate input $x$ and dependent variables $y$ and $z$, we define and give concrete expressions for the condition number of solving for a value of $y$ such that $F(x,y,z) = c$ for some unspecified $z$. This condition number can be used to determine which of the dependent variables of a numerical problem are the most ill-conditioned. We show how this can be used to explain the condition number of the problem of solving for all dependent variables, even if the solution is not unique. The concepts are illustrated with Tucker decomposition of tensors as an example problem.

Which variables of a numerical problem cause ill-conditioning?

TL;DR

This work develops a general framework for the condition number of constant-rank elimination problems (CREPs) with latent variables, enabling analysis of ill-conditioning when solving for a subset of variables given input and latent . It recasts conditioning in geometric terms via a canonical solution map whose derivative defines the condition number , and shows , implying solving for is no harder than solving for both and together. The paper provides practical computation routes, including a matrix-form expression and a LS-based formulation, all hinging on Jacobians and projections. It then applies the framework to orthogonal Tucker decompositions, deriving explicit condition numbers: for each mode , if is square, and otherwise, with the full decomposition conditioning given by the maximum over these values; and . This contributes a principled lens to identify and quantify ill-conditioned components in nonlinear, potentially underdetermined systems, including tensor decompositions, and offers practical tools for sensitivity analysis and model reduction.

Abstract

We study a broad class of numerical problems that can be defined as the solution of a system of (nonlinear) equations for a subset of the dependent variables. Given a system of the form with multivariate input and dependent variables and , we define and give concrete expressions for the condition number of solving for a value of such that for some unspecified . This condition number can be used to determine which of the dependent variables of a numerical problem are the most ill-conditioned. We show how this can be used to explain the condition number of the problem of solving for all dependent variables, even if the solution is not unique. The concepts are illustrated with Tucker decomposition of tensors as an example problem.

Paper Structure

This paper contains 13 sections, 9 theorems, 42 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Contributions
  3. Outline
  4. Notation
  5. Geometric characterisation of the condition number
  6. Projections of CREPs
  7. Proofs of the main results
  8. Numerical computation of the condition number
  9. Application to Tucker decompositions
  10. Tucker decomposition as a CREP
  11. Condition numbers
  12. Conclusion
  13. Acknowledgements

Key Result

Theorem 2

Let $F(x,y,z) = c$ be a CREP with $F\colon \mathcal{X} \times \mathcal{Y} \times \mathcal{Z} \to \mathcal{W}$ and let $(x_0, y_0, z_0)$ be a solution to the CREP. Assume that $\mathcal{Y}$ has a Riemannian metric with an induced distance function $d_{\mathcal{Y}}$. Then there exists a neighbourhood is well-defined and smooth around $x_0$.

Figures (1)

  • Figure 1: Solution sets of a CREP and the canonical solution map $H$. Given an exact and perturbed input $x_0$ and $x$, the solution sets for both inputs are subsets of $\mathcal{Y} \times \mathcal{Z}$ (in this case, curves). For sufficiently small neighbourhoods $\widehat{\mathcal{Z}} \subseteq \mathcal{Z}$ of $z_0$, the solution map can be visualised as follows. For a time parameter $t \ge 0$, define the cylinder $C(t) := B(t) \times \widehat{\mathcal{Z}}$ where $B(t)$ is the closed disc around $y_0$ of radius $t$. Let $t$ increase from $0$ until $C(t)$ touches the solution set of $x$ at some point $(y,z)$. The unique $y$-coordinate of this point is $H(x)$ by definition.

Theorems & Definitions (24)

  • Definition 1
  • Theorem 2
  • Definition 3
  • Proposition 4
  • Proposition 5
  • proof
  • Remark 6
  • Lemma 7
  • proof
  • Theorem 8
  • ...and 14 more