Towards a Scalable In Situ Fast Fourier Transform

TL;DR

This paper addresses the need for scalable, in situ spectral analysis on HPC platforms by delivering a dedicated FFT endpoint within the SENSEI in situ framework, built on FFTW with MPI support. It details data marshaling between the SENSEI bridge (VTK-based) and FFTW, enabling forward and inverse $FFT$ on multi-dimensional data as part of a heterogeneous multi-stage workflow, including Python-based bandpass filtering and visualization. The key contributions are the XML-configurable FFT endpoint, MPI-enabled 2D FFT execution, and an end-to-end prototype workflow demonstrating in situ transformation, spectral-domain processing, and real-domain reconstitution. The work paves the way for scalable in situ FFT analyses, reducing I/O overhead and enabling richer spectral analytics directly within simulation pipelines, with future plans for larger problem sizes and rank-mapping optimizations.

Abstract

The Fast Fourier Transform (FFT) is a numerical operation that transforms a function into a form comprised of its constituent frequencies and is an integral part of scientific computation and data analysis. The objective of our work is to enable use of the FFT as part of a scientific in situ processing chain to facilitate the analysis of data in the spectral regime. We describe the implementation of an FFT endpoint for the transformation of multi-dimensional data within the SENSEI infrastructure. Our results show its use on a sample problem in the context of a multi-stage in situ processing workflow.

Figures (2)

• Figure 1: High-level component diagram of the FFT Endpoint.
• Figure 2: These images show visual results at each of the processing stages of the in situ workflow from Fig. \ref{['fig:components_diagram']}.