Recursive construction of biorthogonal polynomials for handling polynomial regression

TL;DR

The paper tackles ill-conditioning in polynomial regression arising from Gram matrix inversions by introducing an adaptive biorthogonalisation that builds classic-like biorthogonal polynomials beta_n^k biorthogonal to monomials on the finite-dimensional space V(I,w). The orthogonal projector onto V_k can be represented biorthogonally as P_Vk f = sum_{i=0}^k x^i ⟨f, beta_i^k⟩, enabling regression without matrix inversion and allowing recursive upgrading or downgrading of the basis. The authors derive concrete constructions from Type A (Legendre, Laguerre) and Type B (Legendre, Chebyshev) OPS, providing explicit beta_n^k formulas and projection coefficients, including analytic examples such as exponential decay. This framework supports flexible, sparse polynomial approximations and has practical impact for high-degree regression and related numerical tasks, with potential extensions to additional OPS families.

Abstract

An adaptive procedure for constructing polynomials which are biorthogonal to the basis of monomials in the same finite-dimensional inner product space is proposed. By taking advantage of available orthogonal polynomials, the proposed methodology reduces the well-known instability problem arising from the matrix inversion involved in classical polynomial regression. The recurrent generation of the biorthogonal basis facilitates the upgrading of all its members to include an additional one. Moreover, it allows for a natural downgrading of the basis. This convenient feature leads to a straightforward approach for reducing the number of terms in the polynomial regression approximation. The merit of this approach is illustrated through a series of examples where the resulting biorthogonal basis is derived from Legendre, Laguerre, and Chebyshev orthogonal polynomials.

Figures (6)

• Figure 1: Polynomial approximations to the simulated noisy chirp data represented as green points displayed in (a). The black dashed line in all fourth plots represent the original (noiseless) chirp function $f(x)= \cos(7 \pi x^2)$ on the interval $[0,1]$. (b) The green line corresponds to the polynomial approximation using the first $18$ Legendre polynomials corresponding to $k=17$. (c) The green line is the downgraded approximation $f_K$ to $15$ terms of the approximation displayed in figure (b) by removing three terms $x, x^4$, and $x^{17}$ following our methodology. (d) The green line is the polynomial approximation of the simulated data using the first $15$ Legendre polynomials $f_k$ with $k=14$ (the same number of terms as in (c)). Figure (e) presents the downgraded approximation $f_K$ to $13$ terms of the approximation in figure (b) by removing five terms $x, x^2, x^3, x^4$, and $x^{17}$. (f) The green line is the polynomial approximation of the simulated data using the first $13$ Legendre polynomials $f_k$ with $k=12$ (the same number of terms as in (e)).
• Figure 2: (a) The exponential decay function $f(x)=\exp(-\alpha x)$ with $\alpha=1$ is given by the black dashed curve defined on the interval $[0,10]$. Approximations using our methodology outlined in the main text using the Laguerre polynomials with $k=14$ (blue solid curve) and Legendre polynomials with $k=9$ (red solid curve) are presented (both approximations lie directly under each other). (b) Plot of the absolute pointwise errors of both approximations in logarithmic scale from figure (a). (c) The gamma distribution $f(x)=x\exp(-\alpha x)$ (black dashed curve) is approximated by Laguerre polynomials with $k=17$ (blue solid curve) and Legendre polynomials with $k=11$ (red solid curve), again with both approximations lying directly on top of each other. (d) Corresponding absolute pointwise errors of the approximations in logarithmic scale from (c).
• Figure 3: The $L^2$-norm error norm, in logarithmic scale, for the downgraded polynomial approximation for each of the 50 random realisations of the polynomial $f(x)$. The olive green dots correspond to the $6$-term approximation constructed using the methodology of (i). The purple dots correspond to the $6$-term approximation for case (ii).
• Figure 4: Example of the approximations from realisation 31. (a) The approximation of a randomly generated polynomial $f(x)$ of degree at most 19 on the interval $[-1,1]$ is shown by the black dashed curve. The $K=6$-term approximation $g_K$ by Legendre orthogonal polynomials (case (i)) is represented by the olive green solid curve. The $K=6$-term approximation $h_K$ achieved by the biorthogonal polynomials (case (ii)) is represented by the purple solid curve indistinguishable from the black dashed curve for $f(x)$. (b) Absolute pointwise error of the two approximations to the randomly generated polynomial $f(x)$. The olive green curve is the error associated to $g_K$ (case (i)) and the purple curve is the error associated to $h_K$ (case (ii)).
• Figure 5: The mean $L^2$-norm error norm (a) and mean maximum absolute pointwise error (b), in semi-logarithmic scale, averaged across all $50$ random realisations for the downgraded polynomial approximations computed via case (i) and case (ii) respectively. The mean errors are plotted versus the level of downgrading, i.e., the number of terms $K=6,\ldots,19$ in the approximations. We observe in all scenarios, case (ii) yields more accurate approximations.

Theorems & Definitions (19)

• Definition 1: Type A Orthonormal Polynomial Sequence
• Definition 2: Type B Orthonormal Polynomial Sequence
• proof
• Definition 3: Direct sum of subspaces
• Definition 4: Orthogonal complement subspace
• Definition 5: Orthogonal projector
• Theorem 1
• proof
• Theorem 2
• proof