An Asynchronous Distributed-Memory Parallel Algorithm for k-mer Counting

TL;DR

This work addresses the computational bottleneck of k-mer counting (KC) in genomics by introducing DAKC, an asynchronous distributed algorithm that minimizes global synchronization through fine-grained one-sided communication and multi-layer message aggregation. Implemented on the HClib Actor runtime with Conveyors, DAKC adopts a FABSP-style workflow to reduce synchronization rounds while leveraging aggressive data aggregation to lower communication volume. The authors develop an analytical model to dissect hardware utilization and validate it against measurements, showing KC is predominantly limited by data movement and that asynchronous communication yields substantial speedups. Empirical results on a 256-node cluster demonstrate 2–9x speedups over state-of-the-art distributed baselines and up to 2x over the best shared-memory KC, highlighting practical improvements for scalable KC in large-genome workloads.

Abstract

This paper describes a new asynchronous algorithm and implementation for the problem of k-mer counting (KC), which concerns quantifying the frequency of length k substrings in a DNA sequence. This operation is common to many computational biology workloads and can take up to 77% of the total runtime of de novo genome assembly. The performance and scalability of the current state-of-the-art distributed-memory KC algorithm are hampered by multiple rounds of Many-To-Many collectives. Therefore, we develop an asynchronous algorithm (DAKC) that uses fine-grained, asynchronous messages to obviate most of this global communication while utilizing network bandwidth efficiently via custom message aggregation protocols. DAKC can perform strong scaling up to 256 nodes (512 sockets / 6K cores) and can count k-mers up to 9x faster than the state-of-the-art distributed-memory algorithm, and up to 100x faster than the shared-memory alternative. We also provide an analytical model to understand the hardware resource utilization of our asynchronous KC algorithm and provide insights on the performance.

Figures (13)

• Figure 1: Speedup of DAKC over baseline on synthetic and real genomes. Scatter dot sizes are proportional input size.
• Figure 2: Per core memory overhead of 1D/2D/3D Conveyors for strong scaling experiment of Synthetic 32.
• Figure 3: Last-level cache misses predicted by our analytical model and the observed values from the hardware counters. We performed the experiments using $8$ nodes ($192$ cores) of Phoenix. Each experimental data point is the average of three consecutive runs and the error bars represent the standard deviation across runs.
• Figure 4: Time is taken by two phases of k-mer counting as measured in experiments and predicted by our model. We performed the experiments using $8$ nodes ($192$ cores) of Phoenix. Each experimental data point is the best observed time from three consecutive runs.
• Figure 5: Percentage of total execution time spent in computation, intranode and internode communication in distributed k-mer counting of Synthetic 30 dataset, using $32$ nodes ($768$ cores) as per our analytical model. We assume no computational communication overlap for this figure.