Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online-BLS: An Accurate and Efficient Online Broad Learning System for Data Stream Classification

Chunyu Lei, Guang-Ze Chen, C. L. Philip Chen, Tong Zhang

TL;DR

This work introduces Online-BLS, a framework for online data stream classification that delivers closed-form weight updates to overcome suboptimal gradient-based learning. By replacing matrix inverses with a Cholesky-based weight estimation (EWEA) and implementing an efficient rank-one update via Givens rotations (EOUS), it achieves both high accuracy and low online update overhead. Theoretical analyses establish error bounds and favorable time complexity, while extensive experiments on stationary and concept-drift datasets demonstrate superiority over state-of-the-art incremental BLS and online deep learning baselines. The approach also extends naturally to non-stationary environments through a forgetting factor, highlighting practical applicability for real-time data streams.

Abstract

The state-of-the-art online learning models generally conduct a single online gradient descent when a new sample arrives and thus suffer from suboptimal model weights. To this end, we introduce an online broad learning system framework with closed-form solutions for each online update. Different from employing existing incremental broad learning algorithms for online learning tasks, which tend to incur degraded accuracy and expensive online update overhead, we design an effective weight estimation algorithm and an efficient online updating strategy to remedy the above two deficiencies, respectively. Specifically, an effective weight estimation algorithm is first developed by replacing notorious matrix inverse operations with Cholesky decomposition and forward-backward substitution to improve model accuracy. Second, we devise an efficient online updating strategy that dramatically reduces online update time. Theoretical analysis exhibits the splendid error bound and low time complexity of our model. The most popular test-then-training evaluation experiments on various real-world datasets prove its superiority and efficiency. Furthermore, our framework is naturally extended to data stream scenarios with concept drift and exceeds state-of-the-art baselines.

Online-BLS: An Accurate and Efficient Online Broad Learning System for Data Stream Classification

TL;DR

This work introduces Online-BLS, a framework for online data stream classification that delivers closed-form weight updates to overcome suboptimal gradient-based learning. By replacing matrix inverses with a Cholesky-based weight estimation (EWEA) and implementing an efficient rank-one update via Givens rotations (EOUS), it achieves both high accuracy and low online update overhead. Theoretical analyses establish error bounds and favorable time complexity, while extensive experiments on stationary and concept-drift datasets demonstrate superiority over state-of-the-art incremental BLS and online deep learning baselines. The approach also extends naturally to non-stationary environments through a forgetting factor, highlighting practical applicability for real-time data streams.

Abstract

The state-of-the-art online learning models generally conduct a single online gradient descent when a new sample arrives and thus suffer from suboptimal model weights. To this end, we introduce an online broad learning system framework with closed-form solutions for each online update. Different from employing existing incremental broad learning algorithms for online learning tasks, which tend to incur degraded accuracy and expensive online update overhead, we design an effective weight estimation algorithm and an efficient online updating strategy to remedy the above two deficiencies, respectively. Specifically, an effective weight estimation algorithm is first developed by replacing notorious matrix inverse operations with Cholesky decomposition and forward-backward substitution to improve model accuracy. Second, we devise an efficient online updating strategy that dramatically reduces online update time. Theoretical analysis exhibits the splendid error bound and low time complexity of our model. The most popular test-then-training evaluation experiments on various real-world datasets prove its superiority and efficiency. Furthermore, our framework is naturally extended to data stream scenarios with concept drift and exceeds state-of-the-art baselines.

Paper Structure

This paper contains 33 sections, 3 theorems, 39 equations, 4 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Online Learning
  4. Incremental Broad Learning System
  5. Method
  6. Problem Setting
  7. Online-BLS Framework
  8. Effective Weight Estimation Algorithm
  9. Efficient Online Update Strategy
  10. Handing Concept Drift
  11. Theoretical Analysis
  12. Error Bound
  13. Time Complexity
  14. Experiments
  15. Experiments on Stationary Datasets
  16. ...and 18 more sections

Key Result

Lemma 1

The solution $\mathbf{\hat{W}}$ derived via forward substitution satisfies where $\mathbf{L}\in\mathbb{R}^{m \times m}$ is a lower triangular matrix, $C_m$ denotes a constant positively correlated with $m$, and $\mathcal{O}(u^{2})$ represents the error tail term.

Figures (4)

  • Figure 1: Average and Standard Deviation (SD) of online update time. M1: I-BLS; M2: BLS-CIL; M3: RI-BLS; M4: Ours w/o EOUS; M5: Ours.
  • Figure 2: Convergence curves of average and SD. Ours w/o EOUS is omitted because its results are consistent with those of Ours.
  • Figure 3: Average OCE under different $n_3$ values. Ours w/o EOUS is omitted because its results are consistent with those of Ours.
  • Figure 4: Average and SD of convergence curves of Online-BLS with and without forgetting factor on non-stationary datasets.

Theorems & Definitions (7)

  • Remark 1
  • Remark 2
  • Lemma 1
  • Remark 3: Definition of $u$
  • Lemma 2
  • Theorem 1
  • proof