Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning

Hengyi Wu, Zhenyi Wang, Heng Huang

TL;DR

The paper tackles catastrophic forgetting in continual learning by introducing GRACE, a layer-aware framework that dynamically regulates learning per layer based on entropy and past task performance. It combines entropy scaling via a Bayesian-inspired per-layer regulator with adaptive training to preserve strong layers while promoting plasticity in weaker ones, formalized through a per-layer objective and evidence-based updates. The authors provide PAC-Bayes-style generalization guarantees and demonstrate empirical superiority on standard benchmarks, with ablations showing the critical roles of both entropy scaling and adaptive training. The approach is modular, compatible with replay and regularization-based CL, and shows consistent gains across datasets and backbones, highlighting its practical impact for more robust continual learning systems.

Abstract

Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability for plasticity or vice versa. However, different layers naturally exhibit varying levels of uncertainty (entropy) when classifying tasks. High-entropy layers tend to underfit by failing to capture task-specific patterns, while low-entropy layers risk overfitting by becoming overly confident and specialized. To address this imbalance, we propose an entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its entropy. Specifically, our approach reduces entropy in high-entropy layers to mitigate underfitting and increases entropy in overly confident layers to alleviate overfitting. This adaptive regulation encourages the model to converge to wider local minima, which have been shown to improve generalization. Our method is general and can be seamlessly integrated with both replay- and regularization-based approaches. Experiments on various datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.

Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning

TL;DR

The paper tackles catastrophic forgetting in continual learning by introducing GRACE, a layer-aware framework that dynamically regulates learning per layer based on entropy and past task performance. It combines entropy scaling via a Bayesian-inspired per-layer regulator with adaptive training to preserve strong layers while promoting plasticity in weaker ones, formalized through a per-layer objective and evidence-based updates. The authors provide PAC-Bayes-style generalization guarantees and demonstrate empirical superiority on standard benchmarks, with ablations showing the critical roles of both entropy scaling and adaptive training. The approach is modular, compatible with replay and regularization-based CL, and shows consistent gains across datasets and backbones, highlighting its practical impact for more robust continual learning systems.

Abstract

Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability for plasticity or vice versa. However, different layers naturally exhibit varying levels of uncertainty (entropy) when classifying tasks. High-entropy layers tend to underfit by failing to capture task-specific patterns, while low-entropy layers risk overfitting by becoming overly confident and specialized. To address this imbalance, we propose an entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its entropy. Specifically, our approach reduces entropy in high-entropy layers to mitigate underfitting and increases entropy in overly confident layers to alleviate overfitting. This adaptive regulation encourages the model to converge to wider local minima, which have been shown to improve generalization. Our method is general and can be seamlessly integrated with both replay- and regularization-based approaches. Experiments on various datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.
Paper Structure (24 sections, 4 theorems, 60 equations, 3 figures, 7 tables, 1 algorithm)

This paper contains 24 sections, 4 theorems, 60 equations, 3 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Preliminaries
  5. Guided EntRopy-Adaptive Feedback for Continual LEarning (GRACE)
  6. Theoretical Analysis
  7. Cumulative entropy deviation.
  8. Experiment
  9. Experiment Setup
  10. Baselines
  11. Implementation details
  12. Results
  13. Ablation study
  14. Conclusion
  15. Appendix
  16. ...and 9 more sections

Key Result

Theorem 4.4

Fix $\delta\in(0,1)$. For task $t$ with sample size $n_t\ge 2$, let where $\ell_t\in[0,1]$ and $\Delta_t({\bm{\theta}})=\sum_{\ell=1}^L (H_l({\bm{\theta}})) - H_{\ell,t}^{*})^2$. Let $\hat{\mathcal{L}}_t({\bm{\theta}})=\hat{R}_t({\bm{\theta}})+\lambda\,\hat{\Delta}_t({\bm{\theta}})$ be the empirical analogue on $n_t$ samples. For any posterior $q_t$ absolutely conti Moreover, if $p_t=\mathcal{N}(

Figures (3)

  • Figure 1: As the input data propagates through each successive block of the network, its feature vector at each layer is fed into a dedicated classification head. We calculate the entropy of the resulting output distribution from each head. This entropy measurement then dynamically adjusts the strength of the regularization term applied to that specific layer, allowing for adaptive regularization throughout the model. Without our method, layer output entropies (solid red, green, and blue) exhibit high variance, indicating that some layers become over-confident while others remain uncertain. Our method guides the entropy of each layer towards a stable, medium-entropy target (dashed red line), promoting more consistent representations throughout the network.
  • Figure 2: In a baseline model without our intervention, a significant entropy divergence emerges during training. Earlier layers consistently maintain high entropy, while deeper layers collapse to a low-entropy state, suggesting over-confidence.
  • Figure 3: With entropy scaling, the entropies across all layers are successfully regularized. They converge towards a stable, medium-entropy state, showing that our method prevents individual layers from becoming either over-confident or under-confident.

Theorems & Definitions (6)

  • Theorem 4.4: PAC-Bayes generalization with entropy control
  • Theorem 4.5: Forgetting bound via parameter drift
  • Theorem A.1: PAC-Bayes generalization with entropy control
  • proof : Proof of Theorem \ref{['thm:pacbayes-entropy']}
  • Theorem A.2: Forgetting bound via parameter drift
  • proof : Proof of Theorem \ref{['thm:forgetting-drift']}