Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CompactPrompt: A Unified Pipeline for Prompt Data Compression in LLM Workflows

Joong Ho Choi, Jiayang Zhao, Jeel Shah, Ritvika Sonawane, Vedant Singh, Avani Appalla, Will Flanagan, Filipe Condessa

TL;DR

CompactPrompt presents a training-free, end-to-end pipeline that unifies hard prompt pruning, n-gram abbreviation, and numerical quantization to compress prompts and external data in LLM workflows. By combining static and dynamic self-information for token pruning, a reversible n-gram dictionary, and dual numerical quantization schemes, the approach achieves substantial token savings while preserving, and sometimes improving, downstream QA accuracy on TAT-QA and Fin-QA. The work demonstrates practical enterprise impact with an eight-stage UnifiedCompactPromptPipeline and both GUI and library implementations, supporting cost-efficient deployment in finance-focused AI pipelines. Overall, CompactPrompt offers a principled, interpretable, and deployable path to leaner generative AI systems without model retraining, validated by cross-model experiments and human-in-the-loop fidelity assessments.

Abstract

Large Language Models (LLMs) deliver powerful reasoning and generation capabilities but incur substantial run-time costs when operating in agentic workflows that chain together lengthy prompts and process rich data streams. We introduce CompactPrompt, an end-to-end pipeline that merges hard prompt compression with lightweight file-level data compression. CompactPrompt first prunes low-information tokens from prompts using self-information scoring and dependency-based phrase grouping. In parallel, it applies n-gram abbreviation to recurrent textual patterns in attached documents and uniform quantization to numerical columns, yielding compact yet semantically faithful representations. Integrated into standard LLM agents, CompactPrompt reduces total token usage and inference cost by up to 60% on benchmark dataset like TAT-QA and FinQA, while preserving output quality (Results in less than 5% accuracy drop for Claude-3.5-Sonnet, and GPT-4.1-Mini) CompactPrompt helps visualize real-time compression decisions and quantify cost-performance trade-offs, laying the groundwork for leaner generative AI pipelines.

CompactPrompt: A Unified Pipeline for Prompt Data Compression in LLM Workflows

TL;DR

CompactPrompt presents a training-free, end-to-end pipeline that unifies hard prompt pruning, n-gram abbreviation, and numerical quantization to compress prompts and external data in LLM workflows. By combining static and dynamic self-information for token pruning, a reversible n-gram dictionary, and dual numerical quantization schemes, the approach achieves substantial token savings while preserving, and sometimes improving, downstream QA accuracy on TAT-QA and Fin-QA. The work demonstrates practical enterprise impact with an eight-stage UnifiedCompactPromptPipeline and both GUI and library implementations, supporting cost-efficient deployment in finance-focused AI pipelines. Overall, CompactPrompt offers a principled, interpretable, and deployable path to leaner generative AI systems without model retraining, validated by cross-model experiments and human-in-the-loop fidelity assessments.

Abstract

Large Language Models (LLMs) deliver powerful reasoning and generation capabilities but incur substantial run-time costs when operating in agentic workflows that chain together lengthy prompts and process rich data streams. We introduce CompactPrompt, an end-to-end pipeline that merges hard prompt compression with lightweight file-level data compression. CompactPrompt first prunes low-information tokens from prompts using self-information scoring and dependency-based phrase grouping. In parallel, it applies n-gram abbreviation to recurrent textual patterns in attached documents and uniform quantization to numerical columns, yielding compact yet semantically faithful representations. Integrated into standard LLM agents, CompactPrompt reduces total token usage and inference cost by up to 60% on benchmark dataset like TAT-QA and FinQA, while preserving output quality (Results in less than 5% accuracy drop for Claude-3.5-Sonnet, and GPT-4.1-Mini) CompactPrompt helps visualize real-time compression decisions and quantify cost-performance trade-offs, laying the groundwork for leaner generative AI pipelines.
Paper Structure (32 sections, 4 equations, 9 figures)

This paper contains 32 sections, 4 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Design
  4. Hard Prompt Compression
  5. Static Self-Information
  6. Dynamic Self-Information
  7. Combined Scoring and Phrase-Level Pruning
  8. Textual n-gram Abbreviation
  9. Extraction and Frequency Analysis
  10. Dictionary Construction
  11. Contextual Replacement and Reversibility
  12. Numerical Quantization
  13. Uniform Integer Quantization
  14. K-Means-Based Quantization
  15. Representative Example Selection for Few-Shot
  16. ...and 17 more sections

Figures (9)

  • Figure 1: CompactPrompt Workflow
  • Figure 2: CompactPrompt App Preview
  • Figure 3: Non-GUI Preview
  • Figure 4: Table showing performance of different models across varying Top N and N-gram configurations
  • Figure 5: Accuracy deltas ($\Delta$) relative to baseline on TAT-QA with n-gram size 2 and Top-3 abbreviation. Claude-3.5 and GPT-4.1-Mini show improvements up to +6 and +5 points, while GPT-4-Omni and Llama-3.3-70B remain mostly stable or decline under compression.
  • ...and 4 more figures