On the Stability of a non-hyperbolic nonlinear map with non-bounded set of non-isolated fixed points with applications to Machine Learning

TL;DR

The paper addresses the convergence of the SUCPA calibration map, a non-hyperbolic nonlinear map with a non-bounded set of non-isolated fixed points lying on a straight line ${\mathcal{S}}_b$ and slope 1. It develops a non-standard, geometry-based analysis for a two-class case, proving that every initial condition converges to a fixed point on ${\mathcal{S}}_b$ and characterizing the Jacobian with a single center mode alongside a stable direction. Central to the approach are auxiliary functions $\alpha_1(x)$ and $\alpha_2(x)$ and an intercept-update function $\phi(x)$, which together establish existence, uniqueness, and global attractivity of ${\mathcal{S}}_b$; numerical experiments on sentiment polarity, MNLI entailment, and dog-vs-cat image classification corroborate the theory. The results provide convergence guarantees for SUCPA under prior mismatch in the two-class setting and offer insight into the algorithm’s behavior in higher-class scenarios, with practical implications for calibration in NLP and vision tasks.

Abstract

This paper deals with the convergence analysis of the SUCPA (Semi Unsupervised Calibration through Prior Adaptation) algorithm, defined from a first-order non-linear difference equations, first developed to correct the scores output by a supervised machine learning classifier. The convergence analysis is addressed as a dynamical system problem, by studying the local and global stability of the nonlinear map derived from the algorithm. This map, which is defined by a composition of exponential and rational functions, turns out to be non-hyperbolic with a non-bounded set of non-isolated fixed points. Hence, a non-standard method for solving the convergence analysis is used consisting of an ad-hoc geometrical approach. For a binary classification problem (two-dimensional map), we rigorously prove that the map is globally asymptotically stable. Numerical experiments on real-world application are performed to support the theoretical results by means of two different classification problems: Sentiment Polarity performed with a Large Language Model and Cat-Dog Image classification. For a greater number of classes, the numerical evidence shows the same behavior of the algorithm, and this is illustrated with a Natural Language Inference example. The experiment codes are publicly accessible online at the following repository: https://github.com/LautaroEst/sucpa-convergence

Figures (10)

• Figure 1: Schematic representation of local stability at a fixed point $\boldsymbol{\beta^\ast}\!\!\in\!{\mathcal{S}}_b$ within a ball of radius $\varepsilon$ centered at $\boldsymbol{\beta^\ast}$, $B(\boldsymbol{\beta^\ast}\!,\varepsilon)$. The stability direction $E^s$ (blue line). The direction of $E^c$ matches that of the ${\mathcal{S}}_b$. The local dynamics is a replica for each $\boldsymbol{\beta^\ast}\!\!\in{\mathcal{S}}_b$ within the $2\varepsilon$-width strip ${\mathcal{S}}_b(\varepsilon)$
• Figure 2: Any unitary slope straight line ${\mathcal{S}}_x$ above the line of fixed points, ${\mathcal{S}}_b$, is mapped onto another one, $\mathbf{f}({\mathcal{S}}_x)={\mathcal{S}}_{\phi(x)}$ (in blue), placed between them, but strictly above ${\mathcal{S}}_x$
• Figure 3: Example with $K\!=\!2$. Orbits of five different i.c.: $\boldsymbol{\beta}^{[0]}_1\!=\![0,2]$ (red), $\boldsymbol{\beta}^{[0]}_2\!=\![1.5,3.5]$ (blue), $\boldsymbol{\beta}^{[0]}_3\!=\![3,0]$ (cyan), $\boldsymbol{\beta}^{[0]}_4\!=\![4,-1]$ (green) and $\boldsymbol{\beta}^{[0]}_5\!=\![5,1]$ (magenta). Only five points of each orbit were plotted due to rapid convergence. Also the line of fixed points, ${\mathcal{S}}_b$, with $b\!=\!-1.39726$. The shift vector is $\boldsymbol{\lambda}\!=\![1.5,1.5]$
• Figure 4: Example with $K\!=\!3$. Orbits of five different i.c.: $\boldsymbol{\beta}^{[0]}\!=\![1,-1,1]$ (red), $\boldsymbol{\beta}^{[0]}\!=\![2,-2,2]$ (blue), $\boldsymbol{\beta}^{[0]}\!=\![2.5,-2,1]$ (magenta), $\boldsymbol{\beta}^{[0]}\!=\![2.5,-0.5,1]$ (green) and $\boldsymbol{\beta}^{[0]}\!=\![1,-1.5,1.5]$ (cyan). Five points of each orbit are plotted. Also the line of fixed points, ${\mathcal{S}}(\boldsymbol{\beta^*})$
• Figure 5: Example with $K\!=\!2$ in an image classification task. Orbits of eight different i.c. are plotted. At least $200$ points of each one were needed to reach the corresponding limit point $\boldsymbol{\beta^*}$

Theorems & Definitions (30)

• Definition 2.1
• Definition 2.2
• Conjecture 2.3
• Lemma 3.1
• proof
• Corollary 3.2
• proof
• Conjecture 3.3
• Lemma 3.4
• proof