Scalable CFD Simulations in Multi-Billion Voxel Micro-CT Images of Porous Materials Using OpenFOAM on ARCHER2

Abstract

This study investigates the use of High-Performance Computing (HPC) to simulate flow and transport in ultra-large micro-CT images of porous materials using Computational Fluid Dynamics (CFD). Two distinct rock samples, representative of two different rock formations - Bentheimer sandstone and Estaillades carbonate - are investigated. The Bentheimer sandstone image, with dimensions 1,950x1,950x10,800 voxels at 6 micron resolution, comprising 41 billion voxels, represents a largely homogeneous structure, while the Estaillades carbonate image, at 1,144x1,144x6,000 voxels and 3.9676 micron resolution, amounting to 8 billion voxels, features greater heterogeneity, including micro-porous regions. These images are used for direct CFD simulations with GeoChemFoam, our OpenFOAM-based numerical solver, leveraging the computational resources of the UK supercomputer ARCHER2. One of the key aspects of the study is the use of the Darcy-Brinkman-Stokes approach, for which the solid surface is represented using a volumetric indicator function, rather than a complex mesh. This enables the use of simple Cartesian meshes that can be generated in parallel in an efficient and scalable manner. The study explores both weak and strong scaling through subvolume decomposition, demonstrating that, due to the strong scalability and the computational power of ARCHER2, full-resolution CFD simulations can be carried out without the need for image size reduction. This work illustrates the potential of HPC to perform detailed, full-scale simulations on large, high-resolution micro-CT data. The approach relies on a meshing strategy that leverages simple, parallelisable Cartesian grids derived from volumetric indicator functions, eliminating the need for complex surface-conforming meshes and allowing scalable simulation of flow and transport in geological and engineering applications.

Figures (12)

• Figure 1: Micro-CT image of Bentheimer (from Bentheimer). Black represents vitton. Greys are assumed to be solid, and white is pore. The solid surfaces are smoothed to remove the artificial roughness generated by the segmentation
• Figure 2: Middle slice of the Estaillades micro-CT image segmented into 14 labels
• Figure 3: Illustration of the regular Cartesian mesh decomposition. The full domain (left) is split into equally sized subdomains (right), each assigned to a processor. Processor indices increase first along the x-direction, then y, and finally z, enabling a predictable mapping of subdomains.
• Figure 4: Computational domains: (a) Standard approach (40M grid blocks); (b) New approach (225M grid blocks); (c) Fluid and interface cells in new approach (53M grid blocks).
• Figure 5: Maximum image size (in billions of voxels) that can be processed as a function of node count for the standard and new workflows. The standard method is limited to approximately 225 million voxels. The new workflow scales linearly with node count.