Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Particle Transformer for Jet Tagging

Huilin Qu, Congqiao Li, Sitian Qian

TL;DR

JetClass provides the largest public jet tagging dataset to date (100M jets across 10 types) and introduces Particle Transformer (ParT), a Transformer variant with pairwise interaction bias (P-MHA) that leverages physics-inspired features. ParT consistently outperforms prior baselines, including ParticleNet, on JetClass and yields substantial gains when pre-trained on JetClass and fine-tuned on other tagging tasks. The work demonstrates strong data-efficiency, clear ablations validating the pairwise bias, and transferability to top-quark and quark-gluon tagging, signaling a new robust baseline for jet tagging in high-energy physics.

Abstract

Jet tagging is a critical yet challenging classification task in particle physics. While deep learning has transformed jet tagging and significantly improved performance, the lack of a large-scale public dataset impedes further enhancement. In this work, we present JetClass, a new comprehensive dataset for jet tagging. The JetClass dataset consists of 100 M jets, about two orders of magnitude larger than existing public datasets. A total of 10 types of jets are simulated, including several types unexplored for tagging so far. Based on the large dataset, we propose a new Transformer-based architecture for jet tagging, called Particle Transformer (ParT). By incorporating pairwise particle interactions in the attention mechanism, ParT achieves higher tagging performance than a plain Transformer and surpasses the previous state-of-the-art, ParticleNet, by a large margin. The pre-trained ParT models, once fine-tuned, also substantially enhance the performance on two widely adopted jet tagging benchmarks. The dataset, code and models are publicly available at https://github.com/jet-universe/particle_transformer.

Particle Transformer for Jet Tagging

TL;DR

JetClass provides the largest public jet tagging dataset to date (100M jets across 10 types) and introduces Particle Transformer (ParT), a Transformer variant with pairwise interaction bias (P-MHA) that leverages physics-inspired features. ParT consistently outperforms prior baselines, including ParticleNet, on JetClass and yields substantial gains when pre-trained on JetClass and fine-tuned on other tagging tasks. The work demonstrates strong data-efficiency, clear ablations validating the pairwise bias, and transferability to top-quark and quark-gluon tagging, signaling a new robust baseline for jet tagging in high-energy physics.

Abstract

Jet tagging is a critical yet challenging classification task in particle physics. While deep learning has transformed jet tagging and significantly improved performance, the lack of a large-scale public dataset impedes further enhancement. In this work, we present JetClass, a new comprehensive dataset for jet tagging. The JetClass dataset consists of 100 M jets, about two orders of magnitude larger than existing public datasets. A total of 10 types of jets are simulated, including several types unexplored for tagging so far. Based on the large dataset, we propose a new Transformer-based architecture for jet tagging, called Particle Transformer (ParT). By incorporating pairwise particle interactions in the attention mechanism, ParT achieves higher tagging performance than a plain Transformer and surpasses the previous state-of-the-art, ParticleNet, by a large margin. The pre-trained ParT models, once fine-tuned, also substantially enhance the performance on two widely adopted jet tagging benchmarks. The dataset, code and models are publicly available at https://github.com/jet-universe/particle_transformer.
Paper Structure (8 sections, 5 equations, 3 figures, 6 tables)

This paper contains 8 sections, 5 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. The JetClass Dataset
  3. Related Work
  4. Model Architecture
  5. Experiments
  6. Experiments on JetClass Dataset
  7. Fine-Tuning for Other Datasets
  8. Discussion and Conclusion

Figures (3)

  • Figure 1: Illustration of jet tagging at the CERN LHC. High-energy proton-proton collisions at the LHC can produce new unstable particles that decay and yield a collimated spray of outgoing particles. These outgoing particles are measured by complex particle detector systems, and jets can be built ("reconstructed") from these measured particles. The goal of jet tagging is to classify the jets and identify those arising from particles of high interest, e.g., the Higgs boson, the $W$ or $Z$ boson, or the top quark.
  • Figure 2: Examples of the 10 types of jets in the JetClass dataset, viewed as particle clouds. Each particle is displayed as a marker, with its coordinates corresponding to the flying direction of the particle, and its size proportional to the energy. The circles, triangles (upward- or downward-directed), and pentagons represent the particle types, which are hadrons, leptons (electrons or muons), and photons, respectively. The solid (hollow) markers stand for electrically charged (neutral) particles. The marker color reflects the displacement of the particle trajectory from the interaction point of the proton-proton collision, where a larger displacement results in more blue.
  • Figure 3: The architecture of (a) Particle Transformer (b) Particle Attention Block (c) Class Attention Block.