LogiCode: an LLM-Driven Framework for Logical Anomaly Detection

TL;DR

LogiCode addresses the need to detect high-level logical anomalies in industrial settings, beyond traditional structural defect detection. It uses LLMs to translate normal-rule knowledge into executable Python code that analyzes images and provides explanations for detected anomalies. The approach introduces LOCO-Annotations and LogiBench datasets for evaluating detection accuracy, code-generation reliability, and reasoning quality, reporting strong binary accuracy and reasoning performance, with code-generation still improving. The work suggests that LLM-driven reasoning and programmable APIs can enhance interpretability and efficiency in industrial quality control, with potential broad impact across manufacturing contexts.

Abstract

This paper presents LogiCode, a novel framework that leverages Large Language Models (LLMs) for identifying logical anomalies in industrial settings, moving beyond traditional focus on structural inconsistencies. By harnessing LLMs for logical reasoning, LogiCode autonomously generates Python codes to pinpoint anomalies such as incorrect component quantities or missing elements, marking a significant leap forward in anomaly detection technologies. A custom dataset "LOCO-Annotations" and a benchmark "LogiBench" are introduced to evaluate the LogiCode's performance across various metrics including binary classification accuracy, code generation success rate, and precision in reasoning. Findings demonstrate LogiCode's enhanced interpretability, significantly improving the accuracy of logical anomaly detection and offering detailed explanations for identified anomalies. This represents a notable shift towards more intelligent, LLM-driven approaches in industrial anomaly detection, promising substantial impacts on industry-specific applications.

Figures (7)

• Figure 1: Comparison between the structural anomaly and the logical anomaly. (Top) Toy example. (Bottom) Examples from the MVTec LOCO AD dataset Bergmann2022Beyond. In toy example, the yellow squares represent normal components, while the red squares signify structural deviations from the norm. Notably, even though the individual components in logical anomaly are normal, they collectively defy logical constraints. For the LOCO examples, structural anomaly is exemplified by damaged connector, while logical anomaly is indicated by misplaced cable connections.
• Figure 2: Comparison between the existing method (left) and our LogiCode framework (right). The existing method do not accurately capture the underlying logical relationships and lack explanation for logical anomalies. In contrast, the proposed method starts from the perspective of logical constraints and gives the result with reason.
• Figure 3: Overview of LogiCode framework LogiCode is an LLM-empowered logical anomaly detection framework. It comprises three main modules: Code Prompt, Code Generation, and Code Execution. a) Code Prompt module formulates user-defined tasks and logical rules by analyzing examples of normal and abnormal images. b) Code Generation module utilizes an LLM to parse these rules into executable Python codes, selecting appropriate APIs for image analyses. c) Code Execution module applies logical and visual parsing to detect and report anomalies. This comprehensive process not only detects anomalies but also provides rule-based explanations for them, mimicking human problem-solving and decision-making abilities in industrial settings.
• Figure 4: The diagram contrasts the MVTec LOCO dataset (Top) with the LOCO-Annotations (Bottom). Unlike MVTec’s broad focus, the LOCO-Annotations dataset zeroes in on logical anomalies, offering an enriched dataset with detailed explanations for each anomaly. This provides LLMs with the context and specificity required for a nuanced understanding of anomalies, thereby facilitating a more sophisticated and targeted development in industrial quality control systems.
• Figure 5: Quantity Anomaly: The expected count for the cable is one, yet two is present. Position Anomaly: Cable should attach to the connectors with the same order defined by position; however, misplaced attachment is observed here. Size Anomaly: The cable length is anticipated to fall within a specific range; however, it is observed to deviate from the normal range. Matching Anomaly: The cable color is expected to match connector number; however, a mismatch is observed here.