Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A GPU-Accelerated Sharp Interface Immersed Boundary Solver for Large Scale Flow Simulations

Sushrut Kumar, Joshua Romero, Jung-Hee Seo, Massimiliano Fatica, Rajat Mittal

Abstract

Immersed boundary methods (IBMs) facilitate the simulation of flows around stationary, moving, and deforming bodies on Cartesian grids. However, extending these simulations to the large grid sizes required for realistic flow problems remains a significant computational challenge. In this work, we present the implementation and acceleration of ViCar3D, a sharp-interface immersed boundary solver, on graphical processing units (GPUs). We utilize OpenACC, CUDA Fortran and MPI to reprogram \emph{ViCar3D}, a sharp-interface immersed boundary solver, on multi-GPU architectures. Verification and scalability studies are performed for two benchmark cases: two-dimensional flow past a circular cylinder and direct numerical simulation (DNS) of flow past a finite rectangular wing. For the latter, we observe an approximately 20X speedup (node-to-node comparison) relative to the CPU-based implementation. The GPU-accelerated solver is capable of simulating complex 3D flows with up to 200 million mesh points on a single node equipped with four GPUs. Strong and weak scaling tests demonstrate maximum scaling efficiencies of 92\% and 93\%, respectively, on multi-GPU systems. We further test the code to simulate fluid flow past complex-shaped single-body and multi-body cases.

A GPU-Accelerated Sharp Interface Immersed Boundary Solver for Large Scale Flow Simulations

Abstract

Immersed boundary methods (IBMs) facilitate the simulation of flows around stationary, moving, and deforming bodies on Cartesian grids. However, extending these simulations to the large grid sizes required for realistic flow problems remains a significant computational challenge. In this work, we present the implementation and acceleration of ViCar3D, a sharp-interface immersed boundary solver, on graphical processing units (GPUs). We utilize OpenACC, CUDA Fortran and MPI to reprogram \emph{ViCar3D}, a sharp-interface immersed boundary solver, on multi-GPU architectures. Verification and scalability studies are performed for two benchmark cases: two-dimensional flow past a circular cylinder and direct numerical simulation (DNS) of flow past a finite rectangular wing. For the latter, we observe an approximately 20X speedup (node-to-node comparison) relative to the CPU-based implementation. The GPU-accelerated solver is capable of simulating complex 3D flows with up to 200 million mesh points on a single node equipped with four GPUs. Strong and weak scaling tests demonstrate maximum scaling efficiencies of 92\% and 93\%, respectively, on multi-GPU systems. We further test the code to simulate fluid flow past complex-shaped single-body and multi-body cases.

Paper Structure

This paper contains 14 sections, 3 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Computational Methods
  3. Governing Equations and Discretization Methods
  4. Treatment of Immersed Body
  5. Multi-GPU Implementation
  6. Results and Discussion
  7. Performance Comparison
  8. Verification Cases
  9. 2D Flow past a Circular Cylinder at $Re=1000$
  10. 3D Flow over a Rectangular Wing at $Re=1000$
  11. Complex Test Cases
  12. Flow over a Conceptual Flying Vehicle at $Re = 25,000$
  13. Flow over an array of ellipsoidal particulate flow at $Re = 10,000$
  14. Summary

Figures (8)

  • Figure 1: Data management and GPU offloading
  • Figure 2: Treatment of Sharp-Interface Immersed Boundary within Solver. (a) Cartesian grid for flow fields with immersed body represented using triangulated mesh. (b) Ray tracing procedure to mark different cells. (c) Calculation of ghost cell values. (d) Calculation of fresh cell values.
  • Figure 3: Schematic for domain decomposition. (a) pencil like domain decomposition of cartesian indices. (b) Communication of data between neighboring partitions
  • Figure 4: Scaling tests. (a) Weak scaling test. (b) Strong scaling test
  • Figure 5: Aerodynamic forces for flow over 2D Cylinder at $Re=1000$
  • ...and 3 more figures