Memory Efficient Continual Learning for Edge-Based Visual Anomaly Detection

TL;DR

The paper tackles visual anomaly detection (VAD) on resource-constrained edge devices under continual learning (CL). It evaluates STFPM and introduces PaSTe within a CLAD framework, employing Replay and novel Compressed Replay variants to balance memory and accuracy. Results show PaSTe achieves a 10% improvement in pixel-level F1 with Replay over STFPM and, with compressed replay, memory can be reduced by up to 91.5% with manageable performance trade-offs. The work demonstrates the feasibility of edge-adaptive, continually learning VAD and provides practical guidance for memory-efficient CL on limited hardware.

Abstract

Visual Anomaly Detection (VAD) is a critical task in computer vision with numerous real-world applications. However, deploying these models on edge devices presents significant challenges, such as constrained computational and memory resources. Additionally, dynamic data distributions in real-world settings necessitate continuous model adaptation, further complicating deployment under limited resources. To address these challenges, we present a novel investigation into the problem of Continual Learning for Visual Anomaly Detection (CLAD) on edge devices. We evaluate the STFPM approach, given its low memory footprint on edge devices, which demonstrates good performance when combined with the Replay approach. Furthermore, we propose to study the behavior of a recently proposed approach, PaSTe, specifically designed for the edge but not yet explored in the Continual Learning context. Our results show that PaSTe is not only a lighter version of STPFM, but it also achieves superior anomaly detection performance, improving the f1 pixel performance by 10% with the Replay technique. In particular, the structure of PaSTe allows us to test it using a series of Compressed Replay techniques, reducing memory overhead by a maximum of 91.5% compared to the traditional Replay for STFPM. Our study proves the feasibility of deploying VAD models that adapt and learn incrementally on CLAD scenarios on resource-constrained edge devices.

Figures (6)

• Figure 1: Considered CL setting for the AD problem clad. Each task corresponds to a new item. The AD models must be able to detect the anomalous products (image-level) and the defects inside the image (pixel-level) of a new item while remembering to perform well on previously seen items.
• Figure 2: Comparison between the STFPM and PaSTe architectures paste. PaSTe reduces the memory required and computation resources to the minimum.
• Figure 3: On the left, we present the scheme of the Replay approach, where current data is combined with images from previous tasks, stored in the replay memory. On the right, we show the general framework for all Compressed Replay approaches, including Feature Replay, FQ Replay, and PQ Replay. In this approach, the replay memory stores either feature representations or compressed versions, which are then decompressed (when necessary) and then mixed with the representations of the images of the current task.
• Figure 4: Behavior of the f1 pixel-level metric over time for STFPM and PaSTe. Each point represents the average performance on all the tasks seen so far. The considered strategies are the same as defined in \ref{['clstrategies']}
• Figure 5: Behavior of the f1 pixel-level metric over time for PaSTe. Each point represents the average performance on all the tasks seen so far. The considered strategies are the same as defined in 4.2