AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing

TL;DR

AutoSciDACT tackles the problem of detecting subtle scientific novelties in large, high-dimensional datasets while providing statistically robust discovery statements. It introduces a two-phase pipeline that combines contrastive embeddings learned with domain-informed supervision and simulations (pre-training) with a Nyström-approximate New Physics Learning Machine (NPLM) for distributional hypothesis testing (discovery). The authors validate the approach across five datasets from astronomy, physics, biology, and computer vision, showing sensitivity to small signal injections and rigorous p-value based quantification. Compared with traditional GoF tests and density-based baselines, AutoSciDACT demonstrates cross-domain effectiveness and robust statistical performance. The framework offers a transferable, principled path toward automated scientific discovery with potential to accelerate research across disciplines.

Abstract

Novelty detection in large scientific datasets faces two key challenges: the noisy and high-dimensional nature of experimental data, and the necessity of making statistically robust statements about any observed outliers. While there is a wealth of literature on anomaly detection via dimensionality reduction, most methods do not produce outputs compatible with quantifiable claims of scientific discovery. In this work we directly address these challenges, presenting the first step towards a unified pipeline for novelty detection adapted for the rigorous statistical demands of science. We introduce AutoSciDACT (Automated Scientific Discovery with Anomalous Contrastive Testing), a general-purpose pipeline for detecting novelty in scientific data. AutoSciDACT begins by creating expressive low-dimensional data representations using a contrastive pre-training, leveraging the abundance of high-quality simulated data in many scientific domains alongside expertise that can guide principled data augmentation strategies. These compact embeddings then enable an extremely sensitive machine learning-based two-sample test using the New Physics Learning Machine (NPLM) framework, which identifies and statistically quantifies deviations in observed data relative to a reference distribution (null hypothesis). We perform experiments across a range of astronomical, physical, biological, image, and synthetic datasets, demonstrating strong sensitivity to small injections of anomalous data across all domains.

Figures (16)

• Figure 1: Illustration of the scientific method (top row) and the AutoSciDACT pipeline (bottom row), emphasizing the corresponding methodological steps implemented within AutoSciDACT.
• Figure 2: Contrastive embeddings for Particle physics (left) and Astronomy (right) datasets. The high-dimensional input is projected down to four dimensions ($\alpha, \beta, \gamma, \delta$). Background classes are shown in hues of blue and green, while the anomaly is overlaid in hot pink.
• Figure 3: Statistical significances ($Z$ scores) of NPLM and other baseline methods for detecting various fractions of anomalous signals injected into background-dominated samples in (a) scanning additional random variables to the same Synthetic toy at a fixed fraction of 0.6% with 10k Background, and then for the fraction for (b) Synthetic (2k), (c) particle physics, (d) astronomy, (e) histology, and (f) image datasets. In all cases, NPLM is able to discover very small signals with high confidence. The upper limit of the empirical $Z$ scores is indicated by a gray line at roughly 2.88$\sigma$ and is set by the fixed number of pseudo-experiments (500), so empirical numbers are not quoted beyond this point. The Asymptotic NPLM approximates the large pseudo-experiment limit at large $Z$ scores.
• Figure 4: Z-score detection statistic for an anomaly using Mahalanobis distance, and supervised learning on the raw and learned embedded space. (a) We observe the impact of additional signals on sensitivity to find an anomaly. (b) We observe the change in sensitivity vs the number of additional random variables obtained from the average of 100 toys, adding an additional signal model and discriminating variables with each increment on the x-axis. The green corresponds to the Mahalanobis distance on raw inputs, the orange corresponds to a fully supervised algorithm on raw inputs, and the red corresponds to a supervised algorithm trained on the ideal contrastive space, where the signal is known. The blue shows the result of a supervised training (knowing signal) on a contrastive space where the signal is unknown; the brown shows the result of the Mahalanobis distance on the same space.
• Figure 5: Example LIGO signal waveforms: Signals from binary black holes (left) and white noise burst (right).