FlyPrompt: Brain-Inspired Random-Expanded Routing with Temporal-Ensemble Experts for General Continual Learning

TL;DR

The paper tackles General Continual Learning under online, single-pass data streams with blurred task boundaries by decomposing the problem into expert routing and expert competence. It proposes FlyPrompt, a brain-inspired framework combining a Random Expanded Analytic Router (REAR) for non-iterative, closed-form expert selection with a Temporal Ensemble of Task-wise Experts (TE2) that consolidates knowledge across multiple time scales via exponential moving averages. Theoretical guarantees accompany an extensive empirical evaluation on CIFAR-100, ImageNet-R, and CUB-200, showing substantial improvements over state-of-the-art baselines with modest parameter overhead and online efficiency. The work highlights the value of neuro-inspired design for GCL, offering a scalable approach with potential applicability to real-world continual learning systems and NeuroAI directions.

Abstract

General continual learning (GCL) challenges intelligent systems to learn from single-pass, non-stationary data streams without clear task boundaries. While recent advances in continual parameter-efficient tuning (PET) of pretrained models show promise, they typically rely on multiple training epochs and explicit task cues, limiting their effectiveness in GCL scenarios. Moreover, existing methods often lack targeted design and fail to address two fundamental challenges in continual PET: how to allocate expert parameters to evolving data distributions, and how to improve their representational capacity under limited supervision. Inspired by the fruit fly's hierarchical memory system characterized by sparse expansion and modular ensembles, we propose FlyPrompt, a brain-inspired framework that decomposes GCL into two subproblems: expert routing and expert competence improvement. FlyPrompt introduces a randomly expanded analytic router for instance-level expert activation and a temporal ensemble of output heads to dynamically adapt decision boundaries over time. Extensive theoretical and empirical evaluations demonstrate FlyPrompt's superior performance, achieving up to 11.23%, 12.43%, and 7.62% gains over state-of-the-art baselines on CIFAR-100, ImageNet-R, and CUB-200, respectively. Our source code is available at https://github.com/AnAppleCore/FlyGCL.

Figures (12)

• Figure 1: Any-time average accuracy of GCL methods over three datasets using Sup-21K. Dashed lines indicate task transition. $\Delta\rm{auc}$, the improvement of area-under-curve score by FlyPrompt.
• Figure 2: Empirical analysis of GCL. (a) A schematic illustration of GCL viewed as multi-expert collaboration. (b) Prompt selection accuracy for methods with explicit expert routing designs. (c) Final average accuracy ($A_{\rm{last}},\uparrow$) when using a test-time oracle to provide the correct prompt identity. Results evaluated across three benchmarks with Sup-21K. FP, FlyPrompt. RP, Random Projection.
• Figure 3: CKA similarity of feature representations between experts of MVP on three datasets.
• Figure 4: Method overview. Inspired by the fruit fly's olfactory memory system (Left), FlyPrompt incorporates a random-expanded analytic router (Middle) and temporal ensemble-based experts (Right). ORNs, olfactory receptor neurons. PNs, projection neurons. KCs, Kenyon cells.
• Figure 5: Analysis of hyperparameters in REAR. (a-c) Different projection dimension $M$ with fixed $\lambda=10^4$: we report $A_{\rm{auc}}$ and extra storage cost (bar) given $M$. (d-f) Different regularization parameter $\lambda$ with fixed $M=10^4$. Dashed lines indicate the optimal choice of each hyperparameter.

Theorems & Definitions (13)

• Theorem 1: REAR, informal
• Theorem 2: TE2, informal
• Theorem : REAR, full
• Lemma 1
• proof
• Lemma 2
• proof
• Lemma 3
• proof
• proof