VQ-Flow: Taming Normalizing Flows for Multi-Class Anomaly Detection via Hierarchical Vector Quantization

TL;DR

VQ-Flow extends normalizing flows to multi-class anomaly detection by introducing hierarchical vector quantization to learn a Conceptual Prototype Codebook (CPC) for concept distinction and a Concept-Specific Pattern Codebook (CSPC) for concept-specific normal patterns. The flow is conditioned on CSPC and uses a concept-aware mixed Gaussian in latent space $p_oldsymbol{Z}(oldsymbol{z} \,|\, \\hat{oldsymbol{y}}) = \sum_{k=1}^{K_{cp}} \mathcal{N}(oldsymbol{z}; \mu(\boldsymbol{c}_k), \sigma(\boldsymbol{c}_k)^2)$, enabling faithful modeling of intricate multi-class distributions. Empirical results on MVTec AD, VisA, and CIFAR-10 show state-of-the-art or competitive AUROCs for both detection and localization under MCAD, with real-time inference (>30 FPS) and a publicly available codebase. The work demonstrates that combining flow-based density estimation with hierarchical VQ yields robust, interpretable concept differentiation and pattern modeling, advancing practical MCAD capabilities. The approach provides a general blueprint for incorporating discrete concept representations into powerful generative density models for complex, unlabeled multiclass data.

Abstract

Normalizing flows, a category of probabilistic models famed for their capabilities in modeling complex data distributions, have exhibited remarkable efficacy in unsupervised anomaly detection. This paper explores the potential of normalizing flows in multi-class anomaly detection, wherein the normal data is compounded with multiple classes without providing class labels. Through the integration of vector quantization (VQ), we empower the flow models to distinguish different concepts of multi-class normal data in an unsupervised manner, resulting in a novel flow-based unified method, named VQ-Flow. Specifically, our VQ-Flow leverages hierarchical vector quantization to estimate two relative codebooks: a Conceptual Prototype Codebook (CPC) for concept distinction and its concomitant Concept-Specific Pattern Codebook (CSPC) to capture concept-specific normal patterns. The flow models in VQ-Flow are conditioned on the concept-specific patterns captured in CSPC, capable of modeling specific normal patterns associated with different concepts. Moreover, CPC further enables our VQ-Flow for concept-aware distribution modeling, faithfully mimicking the intricate multi-class normal distribution through a mixed Gaussian distribution reparametrized on the conceptual prototypes. Through the introduction of vector quantization, the proposed VQ-Flow advances the state-of-the-art in multi-class anomaly detection within a unified training scheme, yielding the Det./Loc. AUROC of 99.5%/98.3% on MVTec AD. The codebase is publicly available at https://github.com/cool-xuan/vqflow.

Figures (7)

• Figure 1: Previous unified methods are prone to model the intricate multi-class data distribution to a simplistic single Gaussian distribution, while our VQ-Flow transforms to the mixed Gaussian distribution through concept-aware distribution modeling.
• Figure 2: The illustration of our main idea in VQ-Flow for multi-class anomaly detection. Multi-class normal data is quantized through hierarchical vector quantization into a Conceptual Prototype Codebook for concept distinction and a Concept-Specific Pattern Codebook to capture concept-specific normal patterns. Best viewed in color.
• Figure 3: The overview of our proposed VQ-Flow for multi-class anomaly detection. For clarity, only the hierarchical vector quantization for the first branch of flow models $F_1$ is depicted, and the other flow models $F_2$ and $F_3$ are overlooked.
• Figure 4: The illustration of hierarchical (residual) vector quantization for Conceptual Prototype Codebook $\boldsymbol{C}_\textrm{cp}$ and Concept-Specific Pattern Codebook $\boldsymbol{C}_\textrm{csp}$ learning in our VQ-Flow. The naive quantization for Concept-Agnostic Pattern Codebook $\boldsymbol{C}_\textrm{cap}$ is also depicted for comparison.
• Figure 5: The AUROC variation with different $K_\textrm{cp}$ and $K_\textrm{csp}$ in our VQ-Flow on MVTec AD dataset:mvtec and CIFAR-10 dataset:cifar10. The best results are framed by red boxes.