Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Univariate Polynomial and Conformal Approximation

Kingsley Yeon

TL;DR

The paper introduces deep univariate polynomial approximations formed by composing low-degree polynomials and analyzes their training, normalization, and optimization. It proves that deep composites can achieve exponential convergence for targets like $|x|$ on $[-1,1]$ and demonstrates practical gains over shallow polynomials across several classical functions, including Runge and Bessel functions. It also extends the framework to inverse $p$th roots and explores conformal maps as preconditioners to alleviate Runge-type issues, supported by extensive numerical experiments and a deflation-based strategy to navigate nonconvex landscapes. The results suggest that deep polynomial architectures, together with conformal preconditioning, offer a powerful toolkit for univariate function approximation with favorable convergence properties and potential applications to matrix functions.

Abstract

A deep approximation is an approximating function defined by composing more than one layer of simple functions. We study deep approximations of functions of one variable using layers consisting of low-degree polynomials or simple conformal transformations. We show that deep approximations to $|x|$ on $[-1,1]$ achieve exponential convergence with respect to the degrees of freedom. Computational experiments suggest that a composite of two and three polynomial layers can give more accurate approximations than a single polynomial with the same number of coefficients. We also study the related problem of reducing the Runge phenomenon by composing polynomials with conformal transformations.

Deep Univariate Polynomial and Conformal Approximation

TL;DR

The paper introduces deep univariate polynomial approximations formed by composing low-degree polynomials and analyzes their training, normalization, and optimization. It proves that deep composites can achieve exponential convergence for targets like on and demonstrates practical gains over shallow polynomials across several classical functions, including Runge and Bessel functions. It also extends the framework to inverse th roots and explores conformal maps as preconditioners to alleviate Runge-type issues, supported by extensive numerical experiments and a deflation-based strategy to navigate nonconvex landscapes. The results suggest that deep polynomial architectures, together with conformal preconditioning, offer a powerful toolkit for univariate function approximation with favorable convergence properties and potential applications to matrix functions.

Abstract

A deep approximation is an approximating function defined by composing more than one layer of simple functions. We study deep approximations of functions of one variable using layers consisting of low-degree polynomials or simple conformal transformations. We show that deep approximations to on achieve exponential convergence with respect to the degrees of freedom. Computational experiments suggest that a composite of two and three polynomial layers can give more accurate approximations than a single polynomial with the same number of coefficients. We also study the related problem of reducing the Runge phenomenon by composing polynomials with conformal transformations.

Paper Structure

This paper contains 23 sections, 2 theorems, 45 equations, 20 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Deep polynomials, definition, training
  3. A Note on Chebyshev Basis
  4. The gradient
  5. Normalization
  6. Lemma 1
  7. Proof
  8. Remark
  9. Theorem 2
  10. Proof
  11. Optimization
  12. Composite approximation to the inverse $p$th root
  13. Examples
  14. Example 1: Logistic $\tanh(\alpha x) = \frac{e^{\alpha x}-e^{-\alpha x}}{e^{\alpha x}+e^{-\alpha x}}$ interval of $(-1,1)$
  15. Example 2: Runge $\frac{1}{1+25x^2}$
  16. ...and 8 more sections

Key Result

Proposition 1

If $p(x) = x^e + ... + a_1x$, i.e monic and zero constant-term, and $q(x) = b_dx^d + ... + b_0$, then $q(p(x))=g(x)$ and $q,p$ uniquely determined by $g$.

Figures (20)

  • Figure 1: Relative error of the deep polynomial approximation to $\lvert x\rvert$ on $[-1,1]$. The exponential decay of the error aligns with the theoretical linear convergence rate.
  • Figure 2: Deep polynomial approximation to $\lvert x\rvert$ on $[-1,1]$. The graph illustrates accurate convergence to $\lvert x\rvert$ across the interval, including endpoints and zero.
  • Figure 3: Logistic 10 random run, $L_2$ error: 2.411e-02, $p = 1x^3 -(2.5\cdot 10^{-5})x^2 - 1.94x + 0$$q = (4.43\cdot 10^{-1})x^3 -(2.92\cdot 10^{-7})x^2 -1.42x -(2.28\cdot 10^{-6})$
  • Figure 4: Sigmoid, $\alpha$ = 50, type (15, 15)
  • Figure 5: Runge 2 layer vs 1 layer
  • ...and 15 more figures

Theorems & Definitions (5)

  • Proposition 1
  • Theorem 3.1: Exponential convergence of composite polynomial to $\lvert x\rvert$
  • proof
  • Remark
  • Remark