Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Micro-Diffusion Compression - Binary Tree Tweedie Denoising for Online Probability Estimation

Roberto Tacconelli

TL;DR

Midicoth is a lossless compression system that introduces a micro-diffusion denoising layer for improving probability estimates produced by adaptive statistical models, treating prior smoothing as a shrinkage process and applying a reverse denoising step that corrects predicted probabilities using empirical calibration statistics.

Abstract

We present Midicoth, a lossless compression system that introduces a micro-diffusion denoising layer for improving probability estimates produced by adaptive statistical models. In compressors such as Prediction by Partial Matching (PPM), probability estimates are smoothed by a prior to handle sparse observations. When contexts have been seen only a few times, this prior dominates the prediction and produces distributions that are significantly flatter than the true source distribution, leading to compression inefficiency. Midicoth addresses this limitation by treating prior smoothing as a shrinkage process and applying a reverse denoising step that corrects predicted probabilities using empirical calibration statistics. To make this correction data-efficient, the method decomposes each byte prediction into a hierarchy of binary decisions along a bitwise tree. This converts a single 256-way calibration problem into a sequence of binary calibration tasks, enabling reliable estimation of correction terms from relatively small numbers of observations. The denoising process is applied in multiple successive steps, allowing each stage to refine residual prediction errors left by the previous one. The micro-diffusion layer operates as a lightweight post-blend calibration stage applied after all model predictions have been combined, allowing it to correct systematic biases in the final probability distribution. Midicoth combines five fully online components: an adaptive PPM model, a long-range match model, a trie-based word model, a high-order context model, and the micro-diffusion denoiser applied as the final stage.

Micro-Diffusion Compression - Binary Tree Tweedie Denoising for Online Probability Estimation

TL;DR

Midicoth is a lossless compression system that introduces a micro-diffusion denoising layer for improving probability estimates produced by adaptive statistical models, treating prior smoothing as a shrinkage process and applying a reverse denoising step that corrects predicted probabilities using empirical calibration statistics.

Abstract

We present Midicoth, a lossless compression system that introduces a micro-diffusion denoising layer for improving probability estimates produced by adaptive statistical models. In compressors such as Prediction by Partial Matching (PPM), probability estimates are smoothed by a prior to handle sparse observations. When contexts have been seen only a few times, this prior dominates the prediction and produces distributions that are significantly flatter than the true source distribution, leading to compression inefficiency. Midicoth addresses this limitation by treating prior smoothing as a shrinkage process and applying a reverse denoising step that corrects predicted probabilities using empirical calibration statistics. To make this correction data-efficient, the method decomposes each byte prediction into a hierarchy of binary decisions along a bitwise tree. This converts a single 256-way calibration problem into a sequence of binary calibration tasks, enabling reliable estimation of correction terms from relatively small numbers of observations. The denoising process is applied in multiple successive steps, allowing each stage to refine residual prediction errors left by the previous one. The micro-diffusion layer operates as a lightweight post-blend calibration stage applied after all model predictions have been combined, allowing it to correct systematic biases in the final probability distribution. Midicoth combines five fully online components: an adaptive PPM model, a long-range match model, a trie-based word model, a high-order context model, and the micro-diffusion denoiser applied as the final stage.
Paper Structure (65 sections, 19 equations, 1 figure, 7 tables, 2 algorithms)

This paper contains 65 sections, 19 equations, 1 figure, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Prediction by Partial Matching (PPM)
  4. Context Mixing (PAQ/CMIX)
  5. Secondary Symbol Estimation
  6. Temperature Scaling and Tweedie's Formula
  7. Diffusion Models
  8. LLM-Based Compression
  9. Method
  10. Pipeline Overview
  11. Base Model: Adaptive PPM with Jeffreys Prior
  12. Prediction with PPMC Exclusion.
  13. Prior.
  14. Update.
  15. Micro-Diffusion Layer: Binary Tree Tweedie Denoising
  16. ...and 50 more sections

Figures (1)

  • Figure 1: The Midicoth compression pipeline. Each byte passes through PPM prediction, three sequential blending layers (Match, Word, HighCtx), and the micro-diffusion layer (3-step Tweedie denoising with James-Stein shrinkage) before arithmetic coding. All models are updated after encoding (dashed arrow).