Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AgentGC: Evolutionary Learning-based Lossless Compression for Genomics Data with LLM-driven Multiple Agent

Sun Hui, Ding Yanfeng, Huidong Ma, Chang Xu, Keyan Jin, Lizheng Zu, Cheng Zhong, xiaoguang Liu, Gang Wang, Wentong Cai

TL;DR

AgentGC tackles the scalability challenge of genomic data compression by introducing an evolutionary, LLM‑driven multi‑agent system that jointly optimizes algorithm, dataset, and hardware. The Leader coordinates memory availability, data representation, and few‑shot parameter tuning, while the Worker implements the AMKLCF compression framework using static and dynamic models plus arithmetic encoding. With three modes (CP, TP, BM) to fit diverse use cases, AgentGC demonstrates significant improvements in compression ratio and throughput across 9 datasets and 14 baselines, along with robustness and memory efficiency gains. The work also acknowledges high resource demands and points to future directions in CPU–GPU hybrids, targeted LLM fine‑tuning, and multi‑modal extensions.

Abstract

Lossless compression has made significant advancements in Genomics Data (GD) storage, sharing and management. Current learning-based methods are non-evolvable with problems of low-level compression modeling, limited adaptability, and user-unfriendly interface. To this end, we propose AgentGC, the first evolutionary Agent-based GD Compressor, consisting of 3 layers with multi-agent named Leader and Worker. Specifically, the 1) User layer provides a user-friendly interface via Leader combined with LLM; 2) Cognitive layer, driven by the Leader, integrates LLM to consider joint optimization of algorithm-dataset-system, addressing the issues of low-level modeling and limited adaptability; and 3) Compression layer, headed by Worker, performs compression & decompression via a automated multi-knowledge learning-based compression framework. On top of AgentGC, we design 3 modes to support diverse scenarios: CP for compression-ratio priority, TP for throughput priority, and BM for balanced mode. Compared with 14 baselines on 9 datasets, the average compression ratios gains are 16.66%, 16.11%, and 16.33%, the throughput gains are 4.73x, 9.23x, and 9.15x, respectively.

AgentGC: Evolutionary Learning-based Lossless Compression for Genomics Data with LLM-driven Multiple Agent

TL;DR

AgentGC tackles the scalability challenge of genomic data compression by introducing an evolutionary, LLM‑driven multi‑agent system that jointly optimizes algorithm, dataset, and hardware. The Leader coordinates memory availability, data representation, and few‑shot parameter tuning, while the Worker implements the AMKLCF compression framework using static and dynamic models plus arithmetic encoding. With three modes (CP, TP, BM) to fit diverse use cases, AgentGC demonstrates significant improvements in compression ratio and throughput across 9 datasets and 14 baselines, along with robustness and memory efficiency gains. The work also acknowledges high resource demands and points to future directions in CPU–GPU hybrids, targeted LLM fine‑tuning, and multi‑modal extensions.

Abstract

Lossless compression has made significant advancements in Genomics Data (GD) storage, sharing and management. Current learning-based methods are non-evolvable with problems of low-level compression modeling, limited adaptability, and user-unfriendly interface. To this end, we propose AgentGC, the first evolutionary Agent-based GD Compressor, consisting of 3 layers with multi-agent named Leader and Worker. Specifically, the 1) User layer provides a user-friendly interface via Leader combined with LLM; 2) Cognitive layer, driven by the Leader, integrates LLM to consider joint optimization of algorithm-dataset-system, addressing the issues of low-level modeling and limited adaptability; and 3) Compression layer, headed by Worker, performs compression & decompression via a automated multi-knowledge learning-based compression framework. On top of AgentGC, we design 3 modes to support diverse scenarios: CP for compression-ratio priority, TP for throughput priority, and BM for balanced mode. Compared with 14 baselines on 9 datasets, the average compression ratios gains are 16.66%, 16.11%, and 16.33%, the throughput gains are 4.73x, 9.23x, and 9.15x, respectively.
Paper Structure (35 sections, 8 equations, 5 figures, 12 tables, 1 algorithm)

This paper contains 35 sections, 8 equations, 5 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Related Work
  4. AgentGC Design
  5. The Leader Agent
  6. The Worker Agent
  7. Compression Modes for AgentGC
  8. Experimental Results
  9. Settings
  10. Compression Ratio
  11. Throughput
  12. Compression Robustness
  13. Memory Usage
  14. Ablation Study
  15. The Impact of ($s$,$k$)-mer Encoder
  16. ...and 20 more sections

Figures (5)

  • Figure 1: (a) The traditional non-evolutionary learning-based and (b) the proposed evolutionary scheme in AgentGC, which realizes collaborative modeling of "Algorithm-Dataset-Hardware".
  • Figure 2: An example of genomic data lossless compression through interaction of LLM-based AgentGC.
  • Figure 3: The pipeline of the proposed AgentGC compression system
  • Figure 4: An example of the agent system derives optimized parameter vector via LLM-based few-shot prompt learning.
  • Figure 5: The (a) compression ratio and (b) overall throughput of AgentGC ($c=32$, GPU $batchsize=320$) on PlFa dataset using different ($s$,$k$)-mer encoder.