MAU-GPT: Enhancing Multi-type Industrial Anomaly Understanding via Anomaly-aware and Generalist Experts Adaptation

TL;DR

The paper addresses the challenge of fine-grained industrial anomaly understanding amid limited dataset coverage and domain generalization. It introduces MAU-Set, a large, multi-domain dataset with a hierarchical five-task QA structure spanning 6 industrial domains, and MAU-GPT, a domain-adapted multimodal model equipped with AMoE-LoRA to unify anomaly-aware and generalist adaptation. AMoE-LoRA blends a generalist Mixture-of-Experts with a sample-specific anomaly adapter via a hypernetwork, producing an output $o = o_0 + o_1 + o_2$ where $o_1 = \frac{1}{r} \sum_{i=1}^N \omega_i B_i A_i x$ and $o_2 = \frac{\alpha}{r} B_0 A_0 x$, enabling robust cross-domain anomaly reasoning. Experiments show MAU-GPT achieves state-of-the-art or near state-of-the-art performance across Discriminative QA, Open-Ended QA, and MMAD benchmarks, with strong parameter efficiency, indicating practical potential for scalable industrial inspection and anomaly analysis.

Abstract

As industrial manufacturing scales, automating fine-grained product image analysis has become critical for quality control. However, existing approaches are hindered by limited dataset coverage and poor model generalization across diverse and complex anomaly patterns. To address these challenges, we introduce MAU-Set, a comprehensive dataset for Multi-type industrial Anomaly Understanding. It spans multiple industrial domains and features a hierarchical task structure, ranging from binary classification to complex reasoning. Alongside this dataset, we establish a rigorous evaluation protocol to facilitate fair and comprehensive model assessment. Building upon this foundation, we further present MAU-GPT, a domain-adapted multimodal large model specifically designed for industrial anomaly understanding. It incorporates a novel AMoE-LoRA mechanism that unifies anomaly-aware and generalist experts adaptation, enhancing both understanding and reasoning across diverse defect classes. Extensive experiments show that MAU-GPT consistently outperforms prior state-of-the-art methods across all domains, demonstrating strong potential for scalable and automated industrial inspection.

Figures (5)

• Figure 1: Overview of the MAU-GPT model and MAU-Set dataset. The dataset spans 35 product types and over 100 defect categories across 6 major industrial domains, supporting 5 tasks ranging from discriminative question answering (QA) to open-ended visual reasoning. Objs refer to object types, while Defs denote defect categories.
• Figure 2: The data collection and annotation of MAU-Set. Ind. Comp. and Constr. Mat. denote Industrial Components and Construction Materials; Elec. Comp.,Cons. Prod.,Mech. Parts and Opt. Insp. are defined analogously.
• Figure 3: The data collection and annotation of MAU-Set.
• Figure 4: The AMoE-LoRA architecture combines generalist experts with an anomaly-aware expert, enabling the MLLM to incorporate both general knowledge and domain-specific insights for industrial anomaly understanding.
• Figure 5: (a) describes the influence of the number and rank of the generalist experts. (b) shows that our model receives the highest endorsement from human experts. (c) is the 2D representation of hypernetwork-generated parameters across samples.