Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IBB: Fast Burrows-Wheeler Transform Construction for Length-Diverse DNA Data

Enno Adler, Stefan Böttcher, Rita Hartel, Cederic Alexander Steininger

TL;DR

This work presents a novel external algorithm called Improved-Bucket Burrows-Wheeler transform (IBB) for constructing the BWT of DNA datasets with highly diverse sequence lengths that is 10% to 40% faster than the best existing state-of-the-art BWT construction algorithms on most datasets while maintaining competitive memory consumption.

Abstract

The Burrows-Wheeler transform (BWT) is integral to the FM-index, which is used extensively in text compression, indexing, pattern search, and bioinformatic problems as de novo assembly and read alignment. Thus, efficient construction of the BWT in terms of time and memory usage is key to these applications. We present a novel external algorithm called Improved-Bucket Burrows-Wheeler transform (IBB) for constructing the BWT of DNA datasets with highly diverse sequence lengths. IBB uses a right-aligned approach to efficiently handle sequences of varying lengths, a tree-based data structure to manage relative insert positions and ranks, and fine buckets to reduce the necessary amount of input and output to external memory. Our experiments demonstrate that IBB is 10% to 40% faster than the best existing state-of-the-art BWT construction algorithms on most datasets while maintaining competitive memory consumption.

IBB: Fast Burrows-Wheeler Transform Construction for Length-Diverse DNA Data

TL;DR

This work presents a novel external algorithm called Improved-Bucket Burrows-Wheeler transform (IBB) for constructing the BWT of DNA datasets with highly diverse sequence lengths that is 10% to 40% faster than the best existing state-of-the-art BWT construction algorithms on most datasets while maintaining competitive memory consumption.

Abstract

The Burrows-Wheeler transform (BWT) is integral to the FM-index, which is used extensively in text compression, indexing, pattern search, and bioinformatic problems as de novo assembly and read alignment. Thus, efficient construction of the BWT in terms of time and memory usage is key to these applications. We present a novel external algorithm called Improved-Bucket Burrows-Wheeler transform (IBB) for constructing the BWT of DNA datasets with highly diverse sequence lengths. IBB uses a right-aligned approach to efficiently handle sequences of varying lengths, a tree-based data structure to manage relative insert positions and ranks, and fine buckets to reduce the necessary amount of input and output to external memory. Our experiments demonstrate that IBB is 10% to 40% faster than the best existing state-of-the-art BWT construction algorithms on most datasets while maintaining competitive memory consumption.

Paper Structure

This paper contains 15 sections, 5 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. IBB
  5. Right Alignment
  6. Insert Position
  7. Buckets
  8. Trees
  9. Navigation
  10. Sort Words
  11. External Storage of Level-k Buckets
  12. Experimental Results
  13. Conclusion
  14. List of Symbols
  15. Evaluation Results

Figures (4)

  • Figure 1: IBB structure for $k = 2$. All subfigures show the iteration $t = 6$ of the construction of the words in (a).
  • Figure 2: Insertion of $W_2[6]$ and $W_3[6]$ into IBB structure. Blue backgrounds mean that the numbers are changed. The green and blue $C$ highlight which $C$ is used in which location to determine the used tree or value.
  • Figure 3:
  • Figure 4: BWT construction times and maximum resident set sizes (max-rss). A missing point means that the construction algorithm aborts or does not create an output file.