A survey of BWT variants for string collections
Davide Cenzato, Zsuzsanna Lipták
TL;DR
This paper surveys the Burrows-Wheeler Transform (BWT) variants used for string collections, showing that 18 tools implement six distinct transforms, with outputs differing in the run count $r$ and often depending on input order. It formalizes two families of BWTs for collections: the order-independent $ ext{eBWT}$ and separator-based variants, including $ ext{mdolBWT}$, $ ext{dolEBWT}$, $ ext{concBWT}$, $ ext{colexBWT}$, and $ ext{optBWT}$, and develops a framework around the notion of interesting intervals where differences occur. The authors prove that separator-based BWT differences are confined to these intervals and analyze how the input ordering $ ho$ maps to output permutations $oldsymbol{\\pi}$ (with examples showing $oldsymbol{\pi}_{ ext{mdol}}= ho$, $oldsymbol{\pi}_{ ext{dolE}}= ext{id}$, $oldsymbol{\pi}_{ ext{colex}}=oldsymbol{\gamma}$, and $oldsymbol{\pi}_{ ext{conc}}$ as the BWT of a rank-encoded meta-string). They quantify how these variants affect $r$, presenting both theoretical bounds (e.g., $r$ proximity via $r_{ ext{OPT}}$ and $c_{oldsymbol{\mathcal M}}$) and empirical results across eight genomic datasets, where short, repetitive collections show the largest discrepancies (up to a $4.2 imes$ difference in $r$ and up to 12% pairwise Hamming distance). The study recommends standardizing the $r$-definition (favoring $ ext{colexBWT}$ or $ ext{optBWT}$) to improve comparability and reproducibility, and provides code and data to reproduce the results. This work highlights that BWT-based space estimates and repetitiveness measures depend critically on the chosen BWT variant, with significant implications for indexing and downstream analyses in pangenomics and metagenomics.
Abstract
In recent years, the focus of bioinformatics research has moved from individual sequences to collections of sequences. Given the fundamental role of the Burrows-Wheeler Transform (BWT) in string processing, a number of dedicated tools have been developed for computing the BWT of string collections. While the focus has been on improving efficiency, both in space and time, the exact definition of the BWT employed has not been at the center of attention. As we show in this paper, the different tools in use often compute non-equivalent BWT variants: the resulting transforms can differ from each other significantly, including the number $r$ of runs, a central parameter of the BWT. Moreover, with many tools, the transform depends on the input order of the collection. In other words, on the same dataset, the same tool may output different transforms if the dataset is given in a different order. We studied $18$ dedicated tools for computing the BWT of string collections and have been able to identify $6$ different BWT variants computed by these tools. We review the differences between these BWT variants, both from a theoretical and from a practical point of view, comparing them on $8$ real-life biological datasets with different characteristics. We find that the differences can be extensive, depending on the datasets, and are largest on collections of many similar short sequences. The parameter $r$, the number of runs of the BWT, also shows notable variation between the different BWT variants; on our datasets, it varied by a multiplicative factor of up to $4.2$. Source code and scripts to replicate the results and download the data used in the article are available at \url{https://github.com/davidecenzato/BWT-variants-for-string-collections}