Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solutions for Distributed Memory Access Mechanism on HPC Clusters

Jan Meizner, Maciej Malawski

TL;DR

The paper investigates remote memory access mechanisms for multi-node HPC clusters, motivated by genomics workflows such as STAR. It compares a LD_PRELOAD–VFS approach over Lustre with an MPI-based RDMA memory access method. Experiments show MPI-based remote memory access can approach local memory performance, while the VFS approach degrades under heavier workloads. The work highlights MPI-based memory sharing as a promising path for multi-tenant HPC environments and proposes future FUSE-based MPI memory sharing to support closed-source software.

Abstract

Paper presents and evaluates various mechanisms for remote access to memory in distributed systems based on two distinct HPC clusters. We are comparing solutions based on the shared storage and MPI (over Infiniband and Slingshot) to the local memory access. This paper also mentions medical use-cases that would mostly benefit from the described solution. We have found out that results for remote access esp. backed by MPI are similar to local memory access.

Solutions for Distributed Memory Access Mechanism on HPC Clusters

TL;DR

The paper investigates remote memory access mechanisms for multi-node HPC clusters, motivated by genomics workflows such as STAR. It compares a LD_PRELOAD–VFS approach over Lustre with an MPI-based RDMA memory access method. Experiments show MPI-based remote memory access can approach local memory performance, while the VFS approach degrades under heavier workloads. The work highlights MPI-based memory sharing as a promising path for multi-tenant HPC environments and proposes future FUSE-based MPI memory sharing to support closed-source software.

Abstract

Paper presents and evaluates various mechanisms for remote access to memory in distributed systems based on two distinct HPC clusters. We are comparing solutions based on the shared storage and MPI (over Infiniband and Slingshot) to the local memory access. This paper also mentions medical use-cases that would mostly benefit from the described solution. We have found out that results for remote access esp. backed by MPI are similar to local memory access.

Paper Structure

This paper contains 7 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Description of the Problem
  3. Related Work
  4. Solution of the Problem
  5. Experiments and results
  6. Discussion of the results
  7. Conclusions and future work

Figures (3)

  • Figure 1: Sharing duplicated data (green) between nodes to conserve resources for other jobs (A, B) or to enable running bigger jobs memory-wise (C,D).
  • Figure 2: Proposed architectures for memory access - a baseline and two distributed.
  • Figure 3: Local memory access vs VFS vs MPI RDMA on: (A) Ares (Infiniband), (B) Helios (Slingshot).