Fibonacci-Driven Recursive Ensembles: Algorithms, Convergence, and Learning Dynamics
Ernest Fokoué
TL;DR
The paper tackles how to endow ensemble learning with memory by introducing second-order Fibonacci-type update flows, yielding learning dynamics with inertia and momentum. It develops a formal RKHS framework, a state-space representation, and a continuous-time limit to connect discrete updates with dynamical systems. The key contributions include three algorithms (Fibonacci Boosting, Rao–Blackwellized Fibonacci Flow, Orthogonalized Recursive Ensemble), convergence and golden-ratio stability results under spectral-radius constraints, non-asymptotic generalization bounds via Rademacher complexity and stability, and empirical validation on kernel ridge, spline smoothing, and random Fourier features. The results demonstrate improved approximation and generalization, and the framework positions learning dynamics with memory as a unifying lens for recursive ensembles with potential links to momentum methods and neural ODEs.
Abstract
This paper develops the algorithmic and dynamical foundations of recursive ensemble learning driven by Fibonacci-type update flows. In contrast with classical boosting Freund and Schapire (1997); Friedman (2001), where the ensemble evolves through first-order additive updates, we study second-order recursive architectures in which each predictor depends on its two immediate predecessors. These Fibonacci flows induce a learning dynamic with memory, allowing ensembles to integrate past structure while adapting to new residual information. We introduce a general family of recursive weight-update algorithms encompassing Fibonacci, tribonacci, and higher-order recursions, together with continuous-time limits that yield systems of differential equations governing ensemble evolution. We establish global convergence conditions, spectral stability criteria, and non-asymptotic generalization bounds under Rademacher Bartlett and Mendelson (2002) and algorithmic stability analyses. The resulting theory unifies recursive ensembles, structured weighting, and dynamical systems viewpoints in statistical learning. Experiments with kernel ridge regression Rasmussen and Williams (2006), spline smoothers Wahba (1990), and random Fourier feature models Rahimi and Recht (2007) demonstrate that recursive flows consistently improve approximation and generalization beyond static weighting. These results complete the trilogy begun in Papers I and II: from Fibonacci weighting, through geometric weighting theory, to fully dynamical recursive ensemble learning systems.