Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast Iteration of Spaced k-mers

Lucas Czech

Abstract

We present efficient approaches for extracting spaced k-mers from nucleotide sequences. They are based on bit manipulation instructions at CPU level, making them both simpler to implement and up to an order of magnitude faster than existing methods. We further evaluate common pitfalls in k-mer processing, which can cause major inefficiencies. Combined, our approaches allow the utilization of spaced k-mers in high-performance bioinformatics applications without major performance degradation, offering a throughput of up to 750MB of sequence data per second per core. Availability: The implementation in C++20 is published under the MIT license, and freely available at https://github.com/lczech/fisk

Fast Iteration of Spaced k-mers

Abstract

We present efficient approaches for extracting spaced k-mers from nucleotide sequences. They are based on bit manipulation instructions at CPU level, making them both simpler to implement and up to an order of magnitude faster than existing methods. We further evaluate common pitfalls in k-mer processing, which can cause major inefficiencies. Combined, our approaches allow the utilization of spaced k-mers in high-performance bioinformatics applications without major performance degradation, offering a throughput of up to 750MB of sequence data per second per core. Availability: The implementation in C++20 is published under the MIT license, and freely available at https://github.com/lczech/fisk

Paper Structure

This paper contains 8 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Results
  4. Encoding and k-mer Extraction
  5. Spaced k-mer Extraction
  6. Benchmarks
  7. Comparison to Existing Approaches
  8. Conclusion

Figures (2)

  • Figure 1: Spaced k-mer extraction.(a) An input sequence of nucleotide characters. (b) Three consecutive $k$-mers of length $8$ are extracted from the sequence, each overlapping by $k-1 = 7$ characters, forming a sliding window of length $k$ over the sequence. (c) A fixed mask $m$ of span $k=8$ and weight $w=5$ is applied to each $k$-mer, to select characters where the mask is $1$, while leaving out characters where the mask is $0$. (d) The final spaced $k$-mers, which we here call $w$-mers for clarity.
  • Figure 2: Extracting spaced k-mers from a sequence.(a) Single mask. The time per $w$-mer is shown, i. e., total time divided by number of $w$-mers in the sequence, averaged across runs with distinct masks, for different hardware and compilers. The naive algorithm loops over the mask for each $w$-mer; the PEXT intrinsic is the fastest where available as a dedicated hardware instruction; the Butterfly algorithm and its SIMD-accelerated variants offer a performant alternative otherwise. (b) Multiple masks (here, 9 distinct masks). The resulting time per distinct $w$-mer is shown. Compared to a single mask, this amortizes the cost for iterating over the underlying $k$-mers in the sequence, which only needs to be performed once, followed by separate bit extractions for each mask.