Enabling Fast, Accurate, and Efficient Real-Time Genome Analysis via New Algorithms and Techniques
Can Firtina
TL;DR
This work tackles the central challenge of noise in genome analysis by developing a quartet of techniques that enable fast, accurate, real-time processing across sequencing modalities. BLEND provides noise-tolerant fuzzy seeding for seeds, RawHash enables hash-based real-time matching of raw nanopore signals, RawHash2 enhances noise handling and mapping decisions, and Rawsamble enables all-vs-all overlapping and de novo assembly directly from raw signals. Together, these methods push real-time genome analysis toward scalable, high-accuracy operations on large genomes while enabling new applications like assembly from raw signals without basecalling. The results show substantial throughput and accuracy gains over state-of-the-art baselines, with practical implications for adaptive sampling, contamination detection, and metagenomics, and they open avenues for hardware-software co-design and in-storage processing to further accelerate genome pipelines.
Abstract
The advent of high-throughput sequencing technologies has revolutionized genome analysis by enabling the rapid and cost-effective sequencing of large genomes. Despite these advancements, the increasing complexity and volume of genomic data present significant challenges related to accuracy, scalability, and computational efficiency. These challenges are mainly due to various forms of unwanted and unhandled variations in sequencing data, collectively referred to as noise. In this dissertation, we address these challenges by providing a deep understanding of different types of noise in genomic data and developing techniques to mitigate the impact of noise on genome analysis. First, we introduce BLEND, a noise-tolerant hashing mechanism that quickly identifies both exactly matching and highly similar sequences with arbitrary differences using a single lookup of their hash values. Second, to enable scalable and accurate analysis of noisy raw nanopore signals, we propose RawHash, a novel mechanism that effectively reduces noise in raw nanopore signals and enables accurate, real-time analysis by proposing the first hash-based similarity search technique for raw nanopore signals. Third, we extend the capabilities of RawHash with RawHash2, an improved mechanism that 1) provides a better understanding of noise in raw nanopore signals to reduce it more effectively and 2) improves the robustness of mapping decisions. Fourth, we explore the broader implications and new applications of raw nanopore signal analysis by introducing Rawsamble, the first mechanism for all-vs-all overlapping of raw signals using hash-based search. Rawsamble enables the construction of de novo assemblies directly from raw signals without basecalling, which opens up new directions and uses for raw nanopore signal analysis.