Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

It's not a FAD: first results in using Flows for unsupervised Anomaly Detection at 40 MHz at the Large Hadron Collider

Francesco Vaselli, Chang Sun, Thea Aarrestad, Dimitrios Danopoulos, Roope Oskari Niemi, Maciej Mikolaj Glowacki, Katya Govorkova, Vladimir Loncar, Felice Pantaleo, Maurizio Pierini

TL;DR

This work tackles real-time unsupervised anomaly detection at the LHC Level-1 trigger by deploying a Continuous Normalizing Flow (CNF) trained with Flow Matching. It introduces a hardware-friendly anomaly score, AS(x) = ||v_t(x, t | φ)||^2, enabling a single-pass FPGA inference suitable for 40 MHz operation and FPGA resource constraints. The approach demonstrates competitive anomaly detection performance against a VAE baseline while enabling aggressive quantization (PTQ and HGQ) that reduces latency to as low as tens of nanoseconds and uses a small FPGA footprint. These results establish CNFs as a viable model-agnostic tool for real-time discovery in high-rate collider environments and outline clear avenues for further optimization and score diversification.

Abstract

We present the first implementation of a Continuous Normalizing Flow (CNF) model for unsupervised anomaly detection within the realistic, high-rate environment of the Large Hadron Collider's L1 trigger systems. While CNFs typically define an anomaly score via a probabilistic likelihood, calculating this score requires solving an Ordinary Differential Equation, a procedure too complex for FPGA deployment. To overcome this, we propose a novel, hardware-friendly anomaly score defined as the squared norm of the model's vector field output. This score is based on the intuition that anomalous events require a larger transformation by the flow. Our model, trained via Flow Matching on Standard Model data, is synthesized for an FPGA using the hls4ml and da4ml libraries. We demonstrate that our approach effectively identifies a variety of beyond-the-Standard-Model signatures with performance comparable to existing machine learning-based triggers. The algorithm achieves a latency of a few hundred nanoseconds, or even less when using advanced quantization techniques, and requires minimal FPGA resources, establishing CNFs as a viable new tool for real-time, data-driven discovery at 40 MHz.

It's not a FAD: first results in using Flows for unsupervised Anomaly Detection at 40 MHz at the Large Hadron Collider

TL;DR

This work tackles real-time unsupervised anomaly detection at the LHC Level-1 trigger by deploying a Continuous Normalizing Flow (CNF) trained with Flow Matching. It introduces a hardware-friendly anomaly score, AS(x) = ||v_t(x, t | φ)||^2, enabling a single-pass FPGA inference suitable for 40 MHz operation and FPGA resource constraints. The approach demonstrates competitive anomaly detection performance against a VAE baseline while enabling aggressive quantization (PTQ and HGQ) that reduces latency to as low as tens of nanoseconds and uses a small FPGA footprint. These results establish CNFs as a viable model-agnostic tool for real-time discovery in high-rate collider environments and outline clear avenues for further optimization and score diversification.

Abstract

We present the first implementation of a Continuous Normalizing Flow (CNF) model for unsupervised anomaly detection within the realistic, high-rate environment of the Large Hadron Collider's L1 trigger systems. While CNFs typically define an anomaly score via a probabilistic likelihood, calculating this score requires solving an Ordinary Differential Equation, a procedure too complex for FPGA deployment. To overcome this, we propose a novel, hardware-friendly anomaly score defined as the squared norm of the model's vector field output. This score is based on the intuition that anomalous events require a larger transformation by the flow. Our model, trained via Flow Matching on Standard Model data, is synthesized for an FPGA using the hls4ml and da4ml libraries. We demonstrate that our approach effectively identifies a variety of beyond-the-Standard-Model signatures with performance comparable to existing machine learning-based triggers. The algorithm achieves a latency of a few hundred nanoseconds, or even less when using advanced quantization techniques, and requires minimal FPGA resources, establishing CNFs as a viable new tool for real-time, data-driven discovery at 40 MHz.

Paper Structure

This paper contains 18 sections, 5 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Data Samples
  3. Preprocessing
  4. Normalizing Flow Models
  5. Continuous Normalizing Flows
  6. Flow Matching
  7. Model architecture and training
  8. VAE model for comparison
  9. Anomaly Detection Scores
  10. Flow ODE
  11. Flow $\mathbf{v}_t$
  12. VAE Score
  13. Further considerations and outlook
  14. Results at Floating Point precision
  15. Model Compression and HLS C Simulation results
  16. ...and 3 more sections

Figures (2)

  • Figure 1: ROC curves for the 4 anomalous samples. For each sample we show the results for the flow, for both the Flow$\mathbf{v}_t$ and Flow ODE anomaly scores, and for the VAE. We show as a dashed red line the threshold of $10^{-5}$ FPR at which we evaluate the fraction of TPR for each model; and as a dotted grey line the baseline performance of a random classifier. The flow is capable of detecting the anomalies, with both ROC AUC and TPR better than a VAE trained under the same condition.
  • Figure 2: Distribution of bit-widths for the weights of the HGQ-trained model. 84.2% of the weights are quantized to 0 bits (pruned), while the remaining weights are distributed between 1 and 5 bits.