Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iterative Optimization of Multidimensional Functions on Turing Machines under Performance Guarantees

Holger Boche, Volker Pohl, H. Vincent Poor

TL;DR

The paper investigates whether iterative, block-coordinate optimization on Turing machines can guarantee a predefined approximation bound for the global minimizer of a continuous function over a computable rectangle. It shows this goal is not generally achievable by constructing two simple computable functions $f_{1}$ and $f_{2}$ that are convex in each coordinate but whose assignment mappings or coordinate-wise reachability of global minimizers are noncomputable or non-effective. Consequently, even when a computable global minimizer exists, the coordinate-wise iterative scheme may fail to compute it effectively or to converge in a computable manner. The results extend to higher smoothness and to general multi-block coordinate methods, underscoring fundamental computability limits for algorithmic termination in decentralized optimization. These findings have practical implications for implementing distributed optimization on digital hardware and motivate consideration of global or alternative approaches when strict termination guarantees are required.

Abstract

This paper studies the effective convergence of iterative methods for solving convex minimization problems using block Gauss--Seidel algorithms. It investigates whether it is always possible to algorithmically terminate the iteration in such a way that the outcome of the iterative algorithm satisfies any predefined error bound. It is shown that the answer is generally negative. Specifically, it is shown that even if a computable continuous function which is convex in each variable possesses computable minimizers, a block Gauss--Seidel iterative method might not be able to effectively compute any of these minimizers. This means that it is impossible to algorithmically terminate the iteration such that a given performance guarantee is satisfied. The paper discusses two reasons for this behavior. First, it might happen that certain steps in the Gauss--Seidel iteration cannot be effectively implemented on a digital computer. Second, all computable minimizers of the problem may not be reachable by the Gauss--Seidel method. Simple and concrete examples for both behaviors are provided.

Iterative Optimization of Multidimensional Functions on Turing Machines under Performance Guarantees

TL;DR

The paper investigates whether iterative, block-coordinate optimization on Turing machines can guarantee a predefined approximation bound for the global minimizer of a continuous function over a computable rectangle. It shows this goal is not generally achievable by constructing two simple computable functions and that are convex in each coordinate but whose assignment mappings or coordinate-wise reachability of global minimizers are noncomputable or non-effective. Consequently, even when a computable global minimizer exists, the coordinate-wise iterative scheme may fail to compute it effectively or to converge in a computable manner. The results extend to higher smoothness and to general multi-block coordinate methods, underscoring fundamental computability limits for algorithmic termination in decentralized optimization. These findings have practical implications for implementing distributed optimization on digital hardware and motivate consideration of global or alternative approaches when strict termination guarantees are required.

Abstract

This paper studies the effective convergence of iterative methods for solving convex minimization problems using block Gauss--Seidel algorithms. It investigates whether it is always possible to algorithmically terminate the iteration in such a way that the outcome of the iterative algorithm satisfies any predefined error bound. It is shown that the answer is generally negative. Specifically, it is shown that even if a computable continuous function which is convex in each variable possesses computable minimizers, a block Gauss--Seidel iterative method might not be able to effectively compute any of these minimizers. This means that it is impossible to algorithmically terminate the iteration such that a given performance guarantee is satisfied. The paper discusses two reasons for this behavior. First, it might happen that certain steps in the Gauss--Seidel iteration cannot be effectively implemented on a digital computer. Second, all computable minimizers of the problem may not be reachable by the Gauss--Seidel method. Simple and concrete examples for both behaviors are provided.
Paper Structure (15 sections, 8 theorems, 72 equations, 3 figures, 1 algorithm)

This paper contains 15 sections, 8 theorems, 72 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation and preliminaries
  3. Computability analysis
  4. Optimization of smooth functions
  5. Minimum value and minimizer
  6. Computability of the minimum value and the minimizer
  7. Iterative optimization methods
  8. Algorithmic computability of assignment functions
  9. A function with no computable assignment function
  10. Convergence of the iterative procedure for $f_{1}$
  11. Approximation of the assignment function
  12. Reachability of global minimizers
  13. Extensions
  14. Summary and discussion
  15. An auxiliary function

Key Result

Lemma 2.1

Let $a\in \mathbb{R}_{\mathrm{c}}$, $a>0$, let $\alpha \in [-1,1]$ be arbitrary, and let $G_{\alpha} : [-a,a] \to \mathbb{R}$ be the function defined by Then $G_{\alpha}$ is not Banach--Mazur computable and therefore not Borel--Turing computable.

Figures (3)

  • Figure 1: Illustration of the function $f_{1}$ constructed in the proof of \ref{['thm:MainThm2D']}. The plot on the right shows the function $f_{1}(x_{1},x_{2})$ (solid line) and $f_{1}(x_{1},-x_{2})$ (dotted line) for fixed $x_{2} = 0$ (blue), $x_{2} = \pm 1.0$ (red), and $x_{2} = \pm 3.0$ (green).
  • Figure 2: Illustration of the function $f_{2}$ defined in \ref{['equ:f2']} with $\alpha = 0.1$ and using a $g_{*}$ based on $\xi_{*} = 1/2$ and with the sequence $\xi_{n} = \xi_{*} + 2^{-n}$ (cf. \ref{['sec:AppAuxFunc']}). The plot on the right shows the function $f_{2}(x_{1},x_{2})$ (solid line) and $f_{2}(x_{1},-x_{2})$ (dotted line) for fixed $x_{2} = 0$ (blue), $x_{2} = \pm 1.0$ (red), and $x_{2} = \pm 2.0$ (green).
  • Figure 3: Illustration of the function $g_{*}$ defined in \ref{['equ:gStar']} for $\xi_{*} = 1/20$ (blue), $\xi_{*} = 1/2$ (red), and $\xi_{*} = 0.95$ (green). In all three cases, the sequence $\xi_{n} = \xi_{*} + 2^{-n}$, $n\in \mathbb{N}$ was used to produce these plots.

Theorems & Definitions (33)

  • Definition 1: Effective convergence
  • Definition 2: Computable number and vector
  • Definition 3: Computable sequence
  • Definition 4: Computable function
  • Lemma 2.1
  • proof : Sketch of proof
  • Definition 5: Computable continuous function
  • Remark 1
  • Proposition 3.1
  • Remark 2
  • ...and 23 more