Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Path-Coordinated Continual Learning with Neural Tangent Kernel-Justified Plasticity: A Theoretical Framework with Near State-of-the-Art Performance

Rathin Chandra Shit

TL;DR

This work addresses catastrophic forgetting in sequential learning by integrating NTK-justified plasticity with a path-coordination strategy, statistically validated path selection, and a multi-metric path quality framework. The approach leverages Neural Tangent Kernel eigenspectrum analysis to bound plasticity and uses a Wilson confidence interval to rigorously validate computation paths, enabling selective path freezing via an EWC-Hybrid scheme. On Split-CIFAR10, the method achieves an average accuracy of $66.7\%$ with $23.4\%$ forgetting, approaching state-of-the-art performance and exhibiting a self-stabilizing decrease in forgetting across task sequences; NTK condition numbers serve as early warnings for capacity exhaustion at $\kappa>10^{11}$. The work provides principled theoretical insights and practical guidelines for adaptive regularization and capacity management in fixed-capacity continual learning systems, with implications for scalable lifelong learning in real-world settings.

Abstract

Catastrophic forgetting is one of the fundamental issues of continual learning because neural networks forget the tasks learned previously when trained on new tasks. The proposed framework is a new path-coordinated framework of continual learning that unites the Neural Tangent Kernel (NTK) theory of principled plasticity bounds, statistical validation by Wilson confidence intervals, and evaluation of path quality by the use of multiple metrics. Experimental evaluation shows an average accuracy of 66.7% at the cost of 23.4% catastrophic forgetting on Split-CIFAR10, a huge improvement over the baseline and competitive performance achieved, which is very close to state-of-the-art results. Further, it is found out that NTK condition numbers are predictive indicators of learning capacity limits, showing the existence of a critical threshold at condition number $>10^{11}$. It is interesting to note that the proposed strategy shows a tendency of lowering forgetting as the sequence of tasks progresses (27% to 18%), which is a system stabilization. The framework validates 80% of discovered paths with a rigorous statistical guarantee and maintains 90-97% retention on intermediate tasks. The core capacity limits of the continual learning environment are determined in the analysis, and actionable insights to enhance the adaptive regularization are offered.

Path-Coordinated Continual Learning with Neural Tangent Kernel-Justified Plasticity: A Theoretical Framework with Near State-of-the-Art Performance

TL;DR

This work addresses catastrophic forgetting in sequential learning by integrating NTK-justified plasticity with a path-coordination strategy, statistically validated path selection, and a multi-metric path quality framework. The approach leverages Neural Tangent Kernel eigenspectrum analysis to bound plasticity and uses a Wilson confidence interval to rigorously validate computation paths, enabling selective path freezing via an EWC-Hybrid scheme. On Split-CIFAR10, the method achieves an average accuracy of with forgetting, approaching state-of-the-art performance and exhibiting a self-stabilizing decrease in forgetting across task sequences; NTK condition numbers serve as early warnings for capacity exhaustion at . The work provides principled theoretical insights and practical guidelines for adaptive regularization and capacity management in fixed-capacity continual learning systems, with implications for scalable lifelong learning in real-world settings.

Abstract

Catastrophic forgetting is one of the fundamental issues of continual learning because neural networks forget the tasks learned previously when trained on new tasks. The proposed framework is a new path-coordinated framework of continual learning that unites the Neural Tangent Kernel (NTK) theory of principled plasticity bounds, statistical validation by Wilson confidence intervals, and evaluation of path quality by the use of multiple metrics. Experimental evaluation shows an average accuracy of 66.7% at the cost of 23.4% catastrophic forgetting on Split-CIFAR10, a huge improvement over the baseline and competitive performance achieved, which is very close to state-of-the-art results. Further, it is found out that NTK condition numbers are predictive indicators of learning capacity limits, showing the existence of a critical threshold at condition number . It is interesting to note that the proposed strategy shows a tendency of lowering forgetting as the sequence of tasks progresses (27% to 18%), which is a system stabilization. The framework validates 80% of discovered paths with a rigorous statistical guarantee and maintains 90-97% retention on intermediate tasks. The core capacity limits of the continual learning environment are determined in the analysis, and actionable insights to enhance the adaptive regularization are offered.

Paper Structure

This paper contains 16 sections, 10 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Formulation
  5. NTK-Justified Plasticity Adaptation
  6. Temperature-Scaled Path Discovery
  7. Wilson Confidence Interval Validation
  8. Multi-Metric Path Quality
  9. EWC-Hybrid Training with Path Freezing
  10. Experimental Evaluation
  11. Experimental Setup and Main Results
  12. Analysis of Validation, Dynamics, and Capacity Limits
  13. Discussion
  14. Theoretical Insights and Methodological Contributions
  15. Capacity Limits and Future Research Directions
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Comprehensive results on Split-CIFAR10. (a) Accuracy matrix showing task retention patterns. (b) Learning curve declining from 98% to 67%. (c) Catastrophic forgetting decreasing from 27% to 17%. (d) NTK plasticity stabilizing at 0.10. (e) Wilson CI validation with all CIs above threshold. (f) Path quality scores 0.833-0.890. (g) Per-Task Accuracy (At Training). (h) Comparison table showing improvement over baseline.