An Incremental Unified Framework for Small Defect Inspection

TL;DR

The paper addresses defect inspection in dynamic manufacturing environments by introducing the Incremental Unified Framework (IUF), which enables object-incremental learning within a reconstruction-based defect-detection setting. It combines Object-Aware Self-Attention to create object-specific semantic boundaries, Semantic Compression Loss to reserve capacity for unseen objects, and a gradient-based updating strategy to preserve prior semantic memory during updates. Empirical results on MVTec-AD and VisA show state-of-the-art performance at both image- and pixel-level defect localization while mitigating catastrophic forgetting, without relying on explicit memory banks. This approach provides a scalable, adaptable solution for industrial inspection pipelines subject to frequent production changes.

Abstract

Artificial Intelligence (AI)-driven defect inspection is pivotal in industrial manufacturing. Yet, many methods, tailored to specific pipelines, grapple with diverse product portfolios and evolving processes. Addressing this, we present the Incremental Unified Framework (IUF), which can reduce the feature conflict problem when continuously integrating new objects in the pipeline, making it advantageous in object-incremental learning scenarios. Employing a state-of-the-art transformer, we introduce Object-Aware Self-Attention (OASA) to delineate distinct semantic boundaries. Semantic Compression Loss (SCL) is integrated to optimize non-primary semantic space, enhancing network adaptability for novel objects. Additionally, we prioritize retaining the features of established objects during weight updates. Demonstrating prowess in both image and pixel-level defect inspection, our approach achieves state-of-the-art performance, proving indispensable for dynamic and scalable industrial inspections. Our code will be released at https://github.com/jqtangust/IUF.

Figures (7)

• Figure 1: Different framework in small defect inspection. (A) shows the most common One-Model-One-Object pattern defard2021padimli2022towardsliu2020towardscohen2020subreiss2021panda, which trains a separate model for each of the different objects. (B), based on (A), the types of defects are incrementing yildiz2022automatedchang2022tiresun2023continualsun2022newchen2021defect, which improves the generalization performance in detecting different defects. (C) shows a unified model you2022unifiedzhao2023omnial for multi-objects. (D) use a memory bank to incrementally record features for all objects for distinguishing li2022towards. (E) is our Incremental Unified framework, and it combines the advantages of both (C) and (D).
• Figure 2: Semantic space in our methodology (B-D). In (A), all objects in the original semantic space are tightly coupled, leading to "catastrophic forgetting", i.e., the learning of new objects will result in forgetting previously learned objects. Our method firstly builds a semantic boundary of each object (B), then compacts the non-primary semantic space (C), and finally suppresses semantic updating in the previous objects' feature space (D).
• Figure 3: Problems in task stream. $\mathbf{10-1\ with\ 5\ Steps}$ is an example of a task stream protocol, where we first train on 10 basic objects and then add one object at a time, with the process being completed in 5 steps (Please see Sec. \ref{['experimentsetup']}(Task Protocol) for more details.). (A-1) and (A-2) demonstrate the performance of image-level and pixel-level models under the previous unified framework, UniAD you2022unified, where catastrophic forgetting significantly occurs. The upper boundary represents the best performance when we can use all previous objects for joint training. (B) demonstrates the reason for catastrophic forgetting. When training the current step, the training model overwrites the previous semantic patterns, causing severe feature conflicts in the reconstructed network.
• Figure 4: Identify object semantics by Object-Aware Self-Attention (Sec. \ref{['m1']}). (A) is the reconstructed network from the previous method you2022unified. (B) is our current setup, which inserts the category attributes of an image into the reconstructed network via Object-Aware Self-Attention, thus constraining the semantic space to the corresponding image features and constructing the semantic boundaries of the network.
• Figure 5: Semantic space operation. We perform an SVD decomposition of the semantic space, based on the feature values that represent semantic importance, and compact the non-primary space in Sec. \ref{['m2']}. In addition, when learning a new object, we project the update weights to the semantic space of previous features, and then block the weight updates that are semantically significant for previous information in Sec. \ref{['m3']}.