Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Continual Learning: A Systematic Literature Review of Approaches, Challenges, and Benchmarks

Seyed Amir Bidaki, Amir Mohammadkhah, Kiyan Rezaee, Faeze Hassani, Sadegh Eskandari, Maziar Salahi, Mohammad M. Ghassemi

TL;DR

This paper addresses the challenge of learning from real-time data streams without catastrophic forgetting in Online Continual Learning (OCL). It presents the first Systematic Literature Review (SLR) of OCL, analyzing 81 approaches, 500+ components, 1,000+ features, and 83 datasets to map the field, categorize methods, and identify gaps. The study finds a strong emphasis on replay-based strategies, with growing interest in hybrid approaches that combine replay with architectural or regularization techniques, and it outlines practical directions such as self-supervised multimodal learning, adaptive memory with sparse retrieval and generative replay, and scalable frameworks for noisy task boundaries. The results provide a comprehensive resource for researchers and practitioners, including a public data pipeline and methodological blueprint to advance robust, efficient OCL in real-world settings.

Abstract

Online Continual Learning (OCL) is a critical area in machine learning, focusing on enabling models to adapt to evolving data streams in real-time while addressing challenges such as catastrophic forgetting and the stability-plasticity trade-off. This study conducts the first comprehensive Systematic Literature Review (SLR) on OCL, analyzing 81 approaches, extracting over 1,000 features (specific tasks addressed by these approaches), and identifying more than 500 components (sub-models within approaches, including algorithms and tools). We also review 83 datasets spanning applications like image classification, object detection, and multimodal vision-language tasks. Our findings highlight key challenges, including reducing computational overhead, developing domain-agnostic solutions, and improving scalability in resource-constrained environments. Furthermore, we identify promising directions for future research, such as leveraging self-supervised learning for multimodal and sequential data, designing adaptive memory mechanisms that integrate sparse retrieval and generative replay, and creating efficient frameworks for real-world applications with noisy or evolving task boundaries. By providing a rigorous and structured synthesis of the current state of OCL, this review offers a valuable resource for advancing this field and addressing its critical challenges and opportunities. The complete SLR methodology steps and extracted data are publicly available through the provided link: https://github.com/kiyan-rezaee/ Systematic-Literature-Review-on-Online-Continual-Learning

Online Continual Learning: A Systematic Literature Review of Approaches, Challenges, and Benchmarks

TL;DR

This paper addresses the challenge of learning from real-time data streams without catastrophic forgetting in Online Continual Learning (OCL). It presents the first Systematic Literature Review (SLR) of OCL, analyzing 81 approaches, 500+ components, 1,000+ features, and 83 datasets to map the field, categorize methods, and identify gaps. The study finds a strong emphasis on replay-based strategies, with growing interest in hybrid approaches that combine replay with architectural or regularization techniques, and it outlines practical directions such as self-supervised multimodal learning, adaptive memory with sparse retrieval and generative replay, and scalable frameworks for noisy task boundaries. The results provide a comprehensive resource for researchers and practitioners, including a public data pipeline and methodological blueprint to advance robust, efficient OCL in real-world settings.

Abstract

Online Continual Learning (OCL) is a critical area in machine learning, focusing on enabling models to adapt to evolving data streams in real-time while addressing challenges such as catastrophic forgetting and the stability-plasticity trade-off. This study conducts the first comprehensive Systematic Literature Review (SLR) on OCL, analyzing 81 approaches, extracting over 1,000 features (specific tasks addressed by these approaches), and identifying more than 500 components (sub-models within approaches, including algorithms and tools). We also review 83 datasets spanning applications like image classification, object detection, and multimodal vision-language tasks. Our findings highlight key challenges, including reducing computational overhead, developing domain-agnostic solutions, and improving scalability in resource-constrained environments. Furthermore, we identify promising directions for future research, such as leveraging self-supervised learning for multimodal and sequential data, designing adaptive memory mechanisms that integrate sparse retrieval and generative replay, and creating efficient frameworks for real-world applications with noisy or evolving task boundaries. By providing a rigorous and structured synthesis of the current state of OCL, this review offers a valuable resource for advancing this field and addressing its critical challenges and opportunities. The complete SLR methodology steps and extracted data are publicly available through the provided link: https://github.com/kiyan-rezaee/ Systematic-Literature-Review-on-Online-Continual-Learning
Paper Structure (63 sections, 7 equations, 8 figures, 7 tables)

This paper contains 63 sections, 7 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Develop Research Questions
  4. Design Conceptual Framework
  5. Construct Selection Criteria
  6. Develop Search Strategy
  7. Select Studies Using Selection Criteria
  8. Coding Studies
  9. Assess the Quality of Studies
  10. Synthesis Results of Individual Studies to Answer the Research Question
  11. Report Findings
  12. Results and Analysis
  13. OCL Definition
  14. CL Formulation
  15. CL Variants
  16. ...and 48 more sections

Figures (8)

  • Figure 1: Our systematic methodology in this work. Our methodology starts with (1) defining research objectives, followed by developing research questions. (2) A conceptual framework is designed, and (3) selection criteria such as relevancy are constructed. (4) A search strategy is implemented using initial keywords and refined to extract more publications. (5) Studies are selected by applying the criteria to identify highly relevant papers. (6) The selected studies are coded to classify entities such as approaches, features, and quality attributes. (7) Quality assessment is performed to ensure high relevance and high-quality papers are included using defined questions. Finally, (8) extracted data are synthesized, and (9) findings are reported to address the research questions.
  • Figure 2: Distribution of articles extracted from multiple digital libraries. This chart depicts the number of OCL-related studies retrieved from various sources, highlighting ScienceDirect as the largest contributor, with 776 articles from a total of 2061.
  • Figure 3: Classification of OCL approaches by strategy. The figure categorizes 81 identified approaches into replay-based, regularization-based, and architecture-based strategies, along with novel methods grouped as "Other". Superscripts denote the reference numbers corresponding to each approach: 1. chaudhry2018efficient, 2. aljundi2019gradient, 3. jiang2021multi, 4. caccia2019online, 5. caccia2020online, 6. fini2020online, 7. chrysakis2020online 8. caccia2020online, 9. NEURIPS2020_ca4b5656, 10. gupta2020look, 11. kj2020meta, 12. wang2021acae, 13. cai2021online, 14. shim2021online, 15. pham2020bilevel, 16. koh2021online, 17. pham2021contextual, 18. wiewel2021entropy, 19. caccia2021new, 20. jin2021gradient, 21. jiang2021multi, 22. yin2021mitigating, 23. mai2021supervised, 24. kim2021continual, 25. cai2022improving, 26. guo2022adaptive, 27. pham2022continualnormalizationrethinkingbatch, 28. 10.1007/978-3-031-20044-1_36, 29. wojcik2022neural, 30. pourcel2022online, 31. zou2022efficient, 32. gu2022not, 33. HAN2022104966, 34. kwon2022toward, 35. yang2022online, 36. liang2022optimizing, 37. wong2022online, 38. pmlr-v162-guo22g, 39. sangermano2022sample, 40. bang2022online, 41. zhang2022simple, 42. wang2022schedule, 43. michel2022contrastivelearningonlinesemisupervised, 44. han2022selecting, 45. prabhu2023online, 46. michel2023learning, 47. xiao2023online, 48. bonicelli2023effectiveness, 49. vodisch2023codeps, 50. Vodisch_2023_CVPR, 51. huo2024non, 52. Guo_2023_CVPR, 53. jung2023new, 54. schiemer2023online, 55. Han_2023, 56. CHEN2023105549, 57. soutif2023improving, 58. lin2023pcr, 59. dam2022scalable, 60. yu2023mitigatingforgettingonlinecontinual, 61. harun2023siesta, 62. gu2023summarizing, 63. HONG2023126527, 64. kong2023trust, 65. seo2024just, 66. kim2024online, 67. seo2024learning, 68. wang2024improving, 69. raghavan2024delta, 70. yan2024orchestrate, 71. wei2024online, 72. li2024progressive, 73. kim2024adaptive, 74. lyu2024overcoming, 75. eeckt2024unsupervised, 76. wang2024forgetting, 77. zhao2024srtfd, 78. lin2024er, 79. wang2024dual, 80. cheng2024nlocl, 81. pan2024adaptive, respectively.
  • Figure 4: Overview of key strategies in OCL. This figure illustrates three primary strategies in OCL: (1) Replay-based methods that store past data in memory or use generative models to sample and replay data during training, depicted by green and blue batches representing task data; (2) Regularization-based methods that preserve critical network weights, highlighted in green for important weights and red for less important weights, ensuring retention of prior knowledge; and (3) Architecture-based methods that dynamically allocate specific neurons or subnetworks to tasks, with color-coded neurons (green, red, and blue) representing task-specific knowledge while protecting previously learned information.
  • Figure 5: Prevalent components in OCL literature. The figure highlights frequently used components across 81 OCL studies, categorized by (1) Model architectures, (2) Optimization techniques, (3) Loss and activation functions, (4) Regularization strategies, (5) Replay and buffer techniques, (6) Data augmentation techniques, (7) Classification and clustering methods, and (8) Libraries and frameworks, with corresponding frequencies.
  • ...and 3 more figures