Entity -- Hardware-agnostic Particle-in-Cell Code for Plasma Astrophysics. I: Curvilinear Special Relativistic Module

TL;DR

Entity introduces a GPU-accelerated, open-source PIC code capable of solving the Vlasov-Maxwell system in general curvilinear coordinates using a coordinate-agnostic framework. It achieves hardware portability through Kokkos and emphasizes usability via a modular problem-generator approach, enabling custom grid geometries without core code changes. The paper details the curvilinear SR module, data structures, kernels, and communication patterns, and validates the code with Cartesian and axisymmetric tests, including Weibel-driven turbulence, reconnection, and magnetosphere configurations. The combination of conformal current deposition, on-the-fly metric evaluation, and extensive post-processing tooling positions Entity as a versatile platform for astrophysical plasma simulations across diverse geometries and architectures.

Abstract

Entity is a new-generation, fully open-source particle-in-cell (PIC) code developed to overcome key limitations in astrophysical plasma modeling, particularly the extreme separation of scales and the performance challenges associated with evolving, GPU-centric computing infrastructures. It achieves hardware-agnostic performance portability across various GPU and CPU architectures using the Kokkos library. Crucially, Entity maintains a high standard for usability, clarity, and customizability, offering a robust and easy-to-use framework for developing new algorithms and grid geometries, which allows extensive control without requiring edits to the core source code. This paper details the core general-coordinate special-relativistic module. Entity is the first PIC code designed to solve the Vlasov-Maxwell system in general coordinates, enabling a coordinate-agnostic framework that provides the foundational structure for straightforward extension to arbitrary coordinate geometries. The core methodology achieves numerical stability by solving particle equations of motion in the global orthonormal Cartesian basis, despite using generalized coordinates like Cartesian, axisymmetric spherical, and quasi-spherical grids. Charge conservation is ensured via a specialized current deposition technique using conformal currents. The code exhibits robust scalability and performance portability on major GPU platforms (AMD MI250X, NVIDIA A100, and Intel Max Series), with the 3D particle pusher and the current deposition operating efficiently at about 2 nanoseconds per particle per timestep. Functionality is validated through a comprehensive suite of standard Cartesian plasma tests and the accurate modeling of relativistic magnetospheres in curvilinear axisymmetric geometries.

Figures (12)

• Figure 1: Physical, $x$-$y$, and code, $\mathtt{x}^1$-$\mathtt{x}^2$, coordinates of the 2D Cartesian grid.
• Figure 2: Physical, $\mathcal{R}$-$\mathcal{T}$ ($r$-$\theta$), and code, $\mathtt{x}^1$-$\mathtt{x}^2$, coordinates of the 2D axisymmetric quasi-spherical grid. Here $r_0=(2/3)r_{\rm min}$, and $\vartheta = 0.4$.
• Figure 3: Stacked plot showing the weak scaling performed by simulating the 3D Weibel instability in a periodic box. The size of each meshblock, as well as the total number of particles, is kept constant per each GPU (or each device unit), i.e., the total size of the box and the number of particles are increased proportionally to the number of GPUs. The name of the supercomputer, the GPU node architecture, as well as the total number of particles per GPU are indicated at the top right of each panel.
• Figure 4: Duration of the current deposition and the particle pusher substeps as a function of the total number of particles per GPU on the Perlmutter NVIDIA A100 GPU with 40 GB of VRAM (strong scaling).
• Figure 5: Same experiment as in Figure \ref{['fig:strong_scaling']}, except the number of particles and cells is kept constant, while we change the dimensionality of the problem.