Continual Learning Approaches for Anomaly Detection

TL;DR

This study investigates the problem of Visual Anomaly Detection at Pixel-Level in the Continual Learning setting, where the model adapts to the new data while maintaining the knowledge of old data and proposes a novel approach called SCALE, which performs high compression levels while preserving image quality through Super-Resolution techniques.

Abstract

Anomaly Detection is a relevant problem that arises in numerous real-world applications, especially when dealing with images. However, there has been little research for this task in the Continual Learning setting. In this work, we introduce a novel approach called SCALE (SCALing is Enough) to perform Compressed Replay in a framework for Anomaly Detection in Continual Learning setting. The proposed technique scales and compresses the original images using a Super Resolution model which, to the best of our knowledge, is studied for the first time in the Continual Learning setting. SCALE can achieve a high level of compression while maintaining a high level of image reconstruction quality. In conjunction with other Anomaly Detection approaches, it can achieve optimal results. To validate the proposed approach, we use a real-world dataset of images with pixel-based anomalies, with the scope to provide a reliable benchmark for Anomaly Detection in the context of Continual Learning, serving as a foundation for further advancements in the field.

Figures (7)

• Figure 1: Scheme of the Continual learning for Anomaly Detection (CLAD). Over time, new items (tasks) arrive. The model must learn the new items and differentiate between anomalous and normal samples. In the testing phase, each pixel is classified as normal (black) or anomalous (white). During this phase, the model is evaluated on both new products and past ones to ensure that previous knowledge is maintained.
• Figure 2: (a) is the proposed CLAD framework for the Replay. Here, the Memory Module is represented by the replay memory that contains past samples. (b) is the proposed CLAD framework for the Compressed Replay. In this case, the Memory Module (a model) is used to reconstruct images from the compressed samples of old tasks. Note that it is accepted (but not mandatory) that a single model can act simultaneously as a Memory Module and an AD Module (e.g., the VAE). Then, (c) is the CLAD framework for the Generative Replay. Here, the Memory Module is a generative model that creates images belonging to old tasks as needed.
• Figure 3: Image examples from the MVTec Dataset AD. For each object is shown a normal sample (in green) and an anomalous sample (in red). For each sample, the entire object is displayed, as well as a zoomed-in view of the region containing the defect. The samples illustrate the range of variations and abnormalities found in the dataset.
• Figure 4: Scheme of the proposed approach for compressed Replay called SCALE. $SR_i$ represents the Super Resolution model at task $i$. First, the current task images are scaled (compressed) and saved in memory (yellow area). Then, they are rescaled ($input$), and the model is trained to reconstruct the original image ($target$). Instead, for old tasks, the images are taken from memory, rescaled, and reconstructed using the SR model from the previous task to retain old knowledge (light blue area). After that, the rescaled blurry image ($input_2$) is fed into the current SR model, with the target corresponding to the image reconstructed by SR in Task $i-1$ (i.e., $target_2$).
• Figure 5: Each plot shows the performance for a CL method comparing different architectures in terms of AD performance, i.e., Average f1 score. The Average f1 metric is shown on axis y, and the index of the current training task is shown on axis x.