Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FASTR: Reimagining FASTQ via Compact Image-inspired Representation

Adrian Tkachenko, Sepehr Salem, Ayotomiwa Ezekiel Adeniyi, Zulal Bingol, Mohammed Nayeem Uddin, Akshat Prasanna, Alexander Zelikovsky, Serghei Mangul, Can Alkan, Mohammed Alser

TL;DR

FASTR rethinks FASTQ by encoding each base and its quality into a single 8-bit value and by eliminating per-read header redundancy with a global header, achieving at least a 2× reduction in storage while remaining fully reversible. The approach includes a novel implicit partitioned range encoding, flexible quality-score processing, and a sentinel-delimited binary stream that enables parallel, lossless decoding. Across Illumina, HiFi, and ONT data, FASTR accelerates encoding/decoding, improves general-purpose compression performance, and remains directly usable by downstream tools like minimap2 with minimal code changes. This combination of efficiency, compatibility, and extensibility positions FASTR as a practical next-generation FASTQ format for scalable genomics and real-time sequencing workflows.

Abstract

Motivation: High-throughput sequencing (HTS) enables population-scale genomics but generates massive datasets, creating bottlenecks in storage, transfer, and analysis. FASTQ, the standard format for over two decades, stores one byte per base and one byte per quality score, leading to inefficient I/O, high storage costs, and redundancy. Existing compression tools can mitigate some issues, but often introduce costly decompression or complex dependency issues. Results: We introduce FASTR, a lossless, computation-native successor to FASTQ that encodes each nucleotide together with its base quality score into a single 8-bit value. FASTR reduces file size by at least 2x while remaining fully reversible and directly usable for downstream analyses. Applying general-purpose compression tools on FASTR consistently yields higher compression ratios, 2.47, 3.64, and 4.8x faster compression, and 2.34, 1.96, 1.75x faster decompression than on FASTQ across Illumina, HiFi, and ONT reads. FASTR is machine-learning-ready, allowing reads to be consumed directly as numerical vectors or image-like representations. We provide a highly parallel software ecosystem for FASTQ-FASTR conversion and show that FASTR integrates with existing tools, such as minimap2, with minimal interface changes and no performance overhead. By eliminating decompression costs and reducing data movement, FASTR lays the foundation for scalable genomics analyses and real-time sequencing workflows. Availability and Implementation: https://github.com/ALSER-Lab/FASTR

FASTR: Reimagining FASTQ via Compact Image-inspired Representation

TL;DR

FASTR rethinks FASTQ by encoding each base and its quality into a single 8-bit value and by eliminating per-read header redundancy with a global header, achieving at least a 2× reduction in storage while remaining fully reversible. The approach includes a novel implicit partitioned range encoding, flexible quality-score processing, and a sentinel-delimited binary stream that enables parallel, lossless decoding. Across Illumina, HiFi, and ONT data, FASTR accelerates encoding/decoding, improves general-purpose compression performance, and remains directly usable by downstream tools like minimap2 with minimal code changes. This combination of efficiency, compatibility, and extensibility positions FASTR as a practical next-generation FASTQ format for scalable genomics and real-time sequencing workflows.

Abstract

Motivation: High-throughput sequencing (HTS) enables population-scale genomics but generates massive datasets, creating bottlenecks in storage, transfer, and analysis. FASTQ, the standard format for over two decades, stores one byte per base and one byte per quality score, leading to inefficient I/O, high storage costs, and redundancy. Existing compression tools can mitigate some issues, but often introduce costly decompression or complex dependency issues. Results: We introduce FASTR, a lossless, computation-native successor to FASTQ that encodes each nucleotide together with its base quality score into a single 8-bit value. FASTR reduces file size by at least 2x while remaining fully reversible and directly usable for downstream analyses. Applying general-purpose compression tools on FASTR consistently yields higher compression ratios, 2.47, 3.64, and 4.8x faster compression, and 2.34, 1.96, 1.75x faster decompression than on FASTQ across Illumina, HiFi, and ONT reads. FASTR is machine-learning-ready, allowing reads to be consumed directly as numerical vectors or image-like representations. We provide a highly parallel software ecosystem for FASTQ-FASTR conversion and show that FASTR integrates with existing tools, such as minimap2, with minimal interface changes and no performance overhead. By eliminating decompression costs and reducing data movement, FASTR lays the foundation for scalable genomics analyses and real-time sequencing workflows. Availability and Implementation: https://github.com/ALSER-Lab/FASTR
Paper Structure (17 sections, 2 figures, 5 tables)

This paper contains 17 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Overview
  4. File Header and Metadata Encoding
  5. Individual Read Headers
  6. Implicit Partitioned Range Encoding
  7. Quality Score Processing and Scaling
  8. Record Delimiting and Lossless Stream Decoding
  9. Results
  10. Evaluated Datasets
  11. FASTR is the Best Successor to FASTQ
  12. FASTR Boosts Compression Performance
  13. FASTR is Directly Usable Without Decompression
  14. Conclusion
  15. Competing interests
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of FASTR processing modes compared to a FASTQ record
  • Figure 2: FASTR’s implicit partitioned range encoding scheme within the 8-bit integer space. In a single byte, we have 255 (0–254) possible scalar values, each of which represents a unique pair of base and its quality score. FASTR reserves the value 255 as an end-of-header character for its safe operation.