AnomalyMatch: Discovering Rare Objects of Interest with Semi-supervised and Active Learning

TL;DR

AnomalyMatch tackles the challenge of discovering rare anomalies in large image datasets under severe label scarcity by integrating a FixMatch-based semi-supervised binary classifier with an active-learning loop and a user-friendly GUI. The approach uses EfficientNetLite0 as a backbone, robust two-tier augmentations, and scalable data handling to achieve high AUROC and AUPRC across astronomy- and vision-based benchmarks, while drastically reducing labeling effort. Key findings show strong performance starting from only five to ten labeled anomalies, rapid gains in the initial cycles, and manageable overfitting when training iterations are kept modest. The work demonstrates practical utility for upcoming surveys and datasets within ESA Datalabs, with implications for explainability, multimodal extensions, and domain-specific anomaly discovery in astronomy and beyond.

Abstract

Anomaly detection in large datasets is essential in astronomy and computer vision. However, due to a scarcity of labelled data, it is often infeasible to apply supervised methods to anomaly detection. We present AnomalyMatch, an anomaly detection framework combining the semi-supervised FixMatch algorithm using EfficientNet classifiers with active learning. AnomalyMatch is tailored for large-scale applications and integrated into the ESA Datalabs science platform. In this method, we treat anomaly detection as a binary classification problem and efficiently utilise limited labelled and abundant unlabelled images for training. We enable active learning via a user interface for verification of high-confidence anomalies and correction of false positives. Evaluations on the GalaxyMNIST astronomical dataset and the miniImageNet natural-image benchmark under severe class imbalance display strong performance. Starting from five to ten labelled anomalies, we achieve an average AUROC of 0.96 (miniImageNet) and 0.89 (GalaxyMNIST), with respective AUPRC of 0.82 and 0.77. After three active learning cycles, anomalies are ranked with 76% (miniImageNet) to 94% (GalaxyMNIST) precision in the top 1% of the highest-ranking images by score. We compare to the established Astronomaly software on selected 'odd' galaxies from the 'Galaxy Zoo - The Galaxy Challenge' dataset, achieving comparable performance with an average AUROC of 0.83. Our results underscore the exceptional utility and scalability of this approach for anomaly discovery, highlighting the value of specialised approaches for domains characterised by severe label scarcity.

Figures (12)

• Figure 1: Overview of the AnomalyMatch workflow. A FixMatch-based semi-supervised learning loop trains an EfficientNet backbone using weak and strong augmentations. An active learning interface supports user verification and labelling of additional samples. The final model can be applied in batch mode to large datasets, with detected anomalies exported for further analysis.
• Figure 2: AnomalyMatch active learning interface built with ipywidgets. Users can inspect and label model-flagged candidates, adjust visual settings (e.g., brightness, contrast, RGB channels), and monitor performance via AUROC and AUPRC curves. Model and label states can be saved and reloaded across sessions.
• Figure 3: Sample images from GalaxyMNIST (top), Galaxy Zoo 2 (centre), and MiniImageNet (bottom). Red borders indicate anomaly classes used during evaluation, while normal samples are shown with white borders. For GalaxyMNIST all classes were tested as anomaly class in a dedicated run.
• Figure 4: ROC (left) and PR (right) curves after three active learning cycles for the miniImageNet Hourglass anomaly class starting from five labelled anomalies, adding ten after each cycle. The model achieves an AUROC of 0.96 and an AUPRC of 0.80, highlighting robust anomaly detection capability under severe class imbalance.
• Figure 5: Anomaly Detection Efficiency curves for the Hourglass class in miniImageNet. Each point on the x-axis represents a percentage of the ranked predictions inspected, sorted by descending anomaly score (i.e., the most anomalous images are checked first). The y-axis shows the percentage of true anomalies recovered within the inspected data subset. Active learning started with five labelled anomalies and adds ten samples after each cycle.