Unsupervised Multi-View Visual Anomaly Detection via Progressive Homography-Guided Alignment

TL;DR

This work tackles unsupervised multi-view visual anomaly detection by learning viewpoint-invariant representations through explicit homography-guided cross-view alignment. It introduces ViewSense-AD (VSAD), combining a Multi-View Alignment Module (MVAM) with a View-Align Latent Diffusion Model (VALDM) and a Fusion Refiner Module (FRM) to progressively align and refine cross-view features during denoising, followed by a multi-level memory-bank detector. Key contributions include the homography-based MVAM, progressive multi-stage alignment in VALDM, and cross-view refinement with L$_d$ and L$_r$ losses, achieving state-of-the-art results on RealIAD and MANTA across pixel-, view-, and sample-level metrics. The approach significantly improves robustness to large viewpoint changes and complex textures, enabling finer anomaly localization in industrial inspection settings.

Abstract

Unsupervised visual anomaly detection from multi-view images presents a significant challenge: distinguishing genuine defects from benign appearance variations caused by viewpoint changes. Existing methods, often designed for single-view inputs, treat multiple views as a disconnected set of images, leading to inconsistent feature representations and a high false-positive rate. To address this, we introduce ViewSense-AD (VSAD), a novel framework that learns viewpoint-invariant representations by explicitly modeling geometric consistency across views. At its core is our Multi-View Alignment Module (MVAM), which leverages homography to project and align corresponding feature regions between neighboring views. We integrate MVAM into a View-Align Latent Diffusion Model (VALDM), enabling progressive and multi-stage alignment during the denoising process. This allows the model to build a coherent and holistic understanding of the object's surface from coarse to fine scales. Furthermore, a lightweight Fusion Refiner Module (FRM) enhances the global consistency of the aligned features, suppressing noise and improving discriminative power. Anomaly detection is performed by comparing multi-level features from the diffusion model against a learned memory bank of normal prototypes. Extensive experiments on the challenging RealIAD and MANTA datasets demonstrate that VSAD sets a new state-of-the-art, significantly outperforming existing methods in pixel, view, and sample-level visual anomaly proving its robustness to large viewpoint shifts and complex textures.

Figures (5)

• Figure 1: (a) Conventional methods process views independently, yielding discrete and inconsistent features that struggle to differentiate viewpoint changes from true defects. (b) Our method (VSAD) employs homography-based alignment to establish correspondences between views, learning a continuous and consistent representation that enables robust anomaly detection.
• Figure 2: Overall architecture of ViewSense-AD (VSAD). (i.) During training, multi-view images are encoded into latent space. The View-Align Latent Diffusion Model (VALDM) performs progressive denoising, where at each Unet layer, the Multi-View Alignment Module (MVAM) aligns features from neighboring views using homography. A Fusion Refiner Module (FRM) then enhances global consistency. The model is trained with a denoising loss $\mathcal{L}_{\text{d}}$ and a refinement loss $\mathcal{L}_{\text{r}}$. (ii.) At inference, multi-level refined features are extracted via DDIM inversion and compared against a normal memory bank for anomaly scoring. (iii.) The architecture of the UNet encoder/decoder block used in Stable Diffusion. The proposed MVAM module is integrated after the ResBlock. (iv.) Detailed architecture of the MVAM. (v.) Detailed architecture of the FRM.
• Figure 3: Qualitative anomaly localization results on RealIAD (left) and MANTA (right). Compared to baselines like PatchCore and CKAAD, our method (VSAD) produces significantly more accurate and fine-grained localization maps with fewer false positives, demonstrating its superior ability to handle viewpoint variations and subtle defects.
• Figure 4: Visualization of the effect of component ablations on localization performance, shown on RealIAD (left) and MANTA (right).
• Figure 5: t-SNE visualization of multi-view features for the ‘USB’ class from RealIAD. Left: Before alignment, features from different views (colors) are scattered. Middle: After MVAM, features begin to form view-specific clusters. Right: After FRM, the clusters become tighter and more distinct, indicating improved feature discriminability.