Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds

Yan V. G. Ferreira, Igor B. Lima, Pedro H. G. Mapa S., Felipe V. Campos, Antonio P. Braga

Abstract

Most machine learning methods assume fixed probability distributions, limiting their applicability in nonstationary real-world scenarios. While continual learning methods address this issue, current approaches often rely on black-box models or require extensive user intervention for interpretability. We propose SyMPLER (Systems Modeling through Piecewise Linear Evolving Regression), an explainable model for time series forecasting in nonstationary environments based on dynamic piecewise-linear approximations. Unlike other locally linear models, SyMPLER uses generalization bounds from Statistical Learning Theory to automatically determine when to add new local models based on prediction errors, eliminating the need for explicit clustering of the data. Experiments show that SyMPLER can achieve comparable performance to both black-box and existing explainable models while maintaining a human-interpretable structure that reveals insights about the system's behavior. In this sense, our approach conciliates accuracy and interpretability, offering a transparent and adaptive solution for forecasting nonstationary time series.

Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds

Abstract

Most machine learning methods assume fixed probability distributions, limiting their applicability in nonstationary real-world scenarios. While continual learning methods address this issue, current approaches often rely on black-box models or require extensive user intervention for interpretability. We propose SyMPLER (Systems Modeling through Piecewise Linear Evolving Regression), an explainable model for time series forecasting in nonstationary environments based on dynamic piecewise-linear approximations. Unlike other locally linear models, SyMPLER uses generalization bounds from Statistical Learning Theory to automatically determine when to add new local models based on prediction errors, eliminating the need for explicit clustering of the data. Experiments show that SyMPLER can achieve comparable performance to both black-box and existing explainable models while maintaining a human-interpretable structure that reveals insights about the system's behavior. In this sense, our approach conciliates accuracy and interpretability, offering a transparent and adaptive solution for forecasting nonstationary time series.
Paper Structure (15 sections, 10 equations, 9 figures, 4 tables)

This paper contains 15 sections, 10 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. SyMPLER
  4. Piecewise-Linear Approximation
  5. Adding New Models
  6. VC-Theoretical Buffer Size
  7. Switching between Local Models
  8. Experiments and Discussion
  9. Simulation Data
  10. Real Data
  11. Additional Experiments
  12. Experiments with Concept Drifts
  13. Experiments with High-Dimensional Data
  14. Experiments with Noisy Data
  15. Conclusion

Figures (9)

  • Figure 1: SyMPLER switches between existing local models depending on the value of the input $\mathbf{x}(t)$. Each local model approximates the desired function linearly around its approximation point $\mathbf{x}(t) = \mathbf{p}_i$. As the domain of the problem grows, more local models are added. In this example, SyMPLER switches to the model colored in red as $\mathbf{x}(t)$ approaches $\mathbf{p}_4$ and adds a new model for the new region of the input.
  • Figure 2: Smallest training size $l$ for a model with VC-dimension $h$, following cherkassky.
  • Figure 3: Continual learning results: (a) prediction squared error, (b) input data and (c) number of members in the network. Data in (a) and (c) start appearing later than in (b) because the network has no member until the first $2(n+1)+10$ samples arrive.
  • Figure 4: SyMPLER approximation in feature space. Although each individual model is linear, as shown with the coloured lines in (a), switching between them allows the network to approximate non-linear functions piecewise-linearly, as shown in (b). The zoomed area shows the piecewise-linear behavior of the model.
  • Figure 5: Long-term forecasting of the angular position of the pendulum using SyMPLER and the linearized model for the first cycle of the system (left figure) and its last cycle (right figure).
  • ...and 4 more figures