IMLP: An Energy-Efficient Continual Learning Method for Tabular Data Streams
Yuandou Wang, Filip Gunnarsson, Rihan Hai
TL;DR
This work addresses energy-efficient continual learning for tabular data streams by introducing IMLP, an Incremental MLP that leverages a windowed self-attention mechanism over a fixed-size latent feature buffer to avoid storing raw past data. The model concatenates attended context with current features and processes them through shared feed-forward layers, achieving constant memory usage and lightweight computation. The authors define NetScore-T to jointly evaluate accuracy and energy consumption and provide hardware-grounded measurements showing IMLP achieves up to 27.6x higher energy efficiency than TabNet and 85.5x higher than TabPFN with competitive accuracy. Experiments on 36 TabZilla datasets demonstrate favorable energy–accuracy trade-offs and that IMLP sits on the neural Pareto frontier under no-replay, enabling practical on-device continual learning for streaming tabular data. The work suggests strong practical impact for real-time decision-making on resource-constrained devices and offers a foundation for further energy-aware CL research in tabular domains.
Abstract
Tabular data streams are rapidly emerging as a dominant modality for real-time decision-making in healthcare, finance, and the Internet of Things (IoT). These applications commonly run on edge and mobile devices, where energy budgets, memory, and compute are strictly limited. Continual learning (CL) addresses such dynamics by training models sequentially on task streams while preserving prior knowledge and consolidating new knowledge. While recent CL work has advanced in mitigating catastrophic forgetting and improving knowledge transfer, the practical requirements of energy and memory efficiency for tabular data streams remain underexplored. In particular, existing CL solutions mostly depend on replay mechanisms whose buffers grow over time and exacerbate resource costs. We propose a context-aware incremental Multi-Layer Perceptron (IMLP), a compact continual learner for tabular data streams. IMLP incorporates a windowed scaled dot-product attention over a sliding latent feature buffer, enabling constant-size memory and avoiding storing raw data. The attended context is concatenated with current features and processed by shared feed-forward layers, yielding lightweight per-segment updates. To assess practical deployability, we introduce NetScore-T, a tunable metric coupling balanced accuracy with energy for Pareto-aware comparison across models and datasets. IMLP achieves up to $27.6\times$ higher energy efficiency than TabNet and $85.5\times$ higher than TabPFN, while maintaining competitive average accuracy. Overall, IMLP provides an easy-to-deploy, energy-efficient alternative to full retraining for tabular data streams.