EdgeOL: Efficient in-situ Online Learning on Edge Devices

TL;DR

EdgeOL addresses the challenge of maintaining high real-time inference accuracy on edge devices while minimizing the energy and time overhead of continual model fine-tuning. It combines Dynamic Adaptive Fine-tuning Frequency (DAF) to adapt the timing of fine-tuning rounds with similarity-guided freezing (SimFreeze) to selectively freeze converged layers based on Centered Kernel Alignment (CKA). The framework achieves substantial performance gains, including a 64% reduction in fine-tuning time, 52% energy savings, and a 1.75% increase in average inference accuracy on CV tasks (and 67%/54% with NLP tasks), outperforming state-of-the-art methods like Egeria, SlimFit, RigL, and Ekya. Additionally, EdgeOL supports semi-supervised learning and remains compatible with quantization, highlighting its practicality for on-device continual learning in dynamic environments.

Abstract

Emerging applications, such as robot-assisted eldercare and object recognition, generally employ deep learning neural networks (DNNs) and naturally require: i) handling streaming-in inference requests and ii) adapting to possible deployment scenario changes. Online model fine-tuning is widely adopted to satisfy these needs. However, an inappropriate fine-tuning scheme could involve significant energy consumption, making it challenging to deploy on edge devices. In this paper, we propose EdgeOL, an edge online learning framework that optimizes inference accuracy, fine-tuning execution time, and energy efficiency through both inter-tuning and intra-tuning optimizations. Experimental results show that, on average, EdgeOL reduces overall fine-tuning execution time by 64%, energy consumption by 52%, and improves average inference accuracy by 1.75% over the immediate online learning strategy

Figures (2)

• Figure 1: Examples of immediate online learning and fixed fine-tuning frequency online learning.
• Figure 2: Computation of DNN training.