Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Identification capacity and rate-query tradeoffs in classification systems

Tristan Simas

TL;DR

This work develops an information-theoretic framework for one-shot classification under three resources: tag rate $L$, witness cost $W$, and distortion $D$, establishing an information barrier that blocks zero-error identification using only interface observations when the attribute profile is not class-injective. The authors prove that a nominal tag of length $L=\lceil\log_2 k\rceil$ enables zero-error identification with $W=O(1)$, while any tag-free scheme incurs at least $W=\Omega(d)$ and potentially $D>0$, where $d$ is the distinguishing dimension. They show that minimal distinguishing query sets form the bases of a matroid, yielding a well-defined distinguishing dimension that lower-bounds $W$ in the absence of tags, and they formalize all results in Lean 4 with over 6,000 lines of machine-checked proofs. Throughout, the theory is illustrated via concrete instantiations in biology, libraries, databases, and programming runtimes, demonstrating the universal pattern: nominal tagging provides the optimal identification cost under the $(L,W,D)$ tradeoff. The work connects to rate-distortion and rate-distortion-perception theories, reframing semantic distortion as a burden on interactive witnesses and highlighting the fundamental role of nominal identification in practical system design.

Abstract

We study a one-shot identification analogue of rate-distortion for discrete classification under three resources: tag rate L (bits of side information stored per entity), identification cost W (attribute-membership queries per identification, excluding global preprocessing and amortized caching), and distortion D (misclassification probability). The question is to characterize achievable triples (L,W,D) when a decoder must recover an entity's class from limited observations. Zero-error barrier. If two distinct classes induce the same attribute profile, then the observation pi(V) is identical for both and no decoder can identify the class from attribute queries alone. Thus, if the profile map pi is not injective on classes, zero-error identification without tags is impossible (a zero-error feasibility threshold). Achievability and converse at D=0. With k classes, nominal tags of L = ceil(log2 k) bits enable O(1) identification cost with D=0. Conversely, any scheme with D=0 must satisfy L >= log2 k bits (tight). Without tags (L=0), identification requires Omega(n) queries in the worst case and may incur D>0. Combinatorial structure. Minimal sufficient query families form the bases of a matroid; the induced distinguishing dimension is well-defined and links to zero-error source coding via graph entropy. We illustrate implications for type systems, databases, and biological taxonomy. All results are mechanized in Lean4 (6000+ lines, 0 sorry).

Identification capacity and rate-query tradeoffs in classification systems

TL;DR

This work develops an information-theoretic framework for one-shot classification under three resources: tag rate , witness cost , and distortion , establishing an information barrier that blocks zero-error identification using only interface observations when the attribute profile is not class-injective. The authors prove that a nominal tag of length enables zero-error identification with , while any tag-free scheme incurs at least and potentially , where is the distinguishing dimension. They show that minimal distinguishing query sets form the bases of a matroid, yielding a well-defined distinguishing dimension that lower-bounds in the absence of tags, and they formalize all results in Lean 4 with over 6,000 lines of machine-checked proofs. Throughout, the theory is illustrated via concrete instantiations in biology, libraries, databases, and programming runtimes, demonstrating the universal pattern: nominal tagging provides the optimal identification cost under the tradeoff. The work connects to rate-distortion and rate-distortion-perception theories, reframing semantic distortion as a burden on interactive witnesses and highlighting the fundamental role of nominal identification in practical system design.

Abstract

We study a one-shot identification analogue of rate-distortion for discrete classification under three resources: tag rate L (bits of side information stored per entity), identification cost W (attribute-membership queries per identification, excluding global preprocessing and amortized caching), and distortion D (misclassification probability). The question is to characterize achievable triples (L,W,D) when a decoder must recover an entity's class from limited observations. Zero-error barrier. If two distinct classes induce the same attribute profile, then the observation pi(V) is identical for both and no decoder can identify the class from attribute queries alone. Thus, if the profile map pi is not injective on classes, zero-error identification without tags is impossible (a zero-error feasibility threshold). Achievability and converse at D=0. With k classes, nominal tags of L = ceil(log2 k) bits enable O(1) identification cost with D=0. Conversely, any scheme with D=0 must satisfy L >= log2 k bits (tight). Without tags (L=0), identification requires Omega(n) queries in the worst case and may incur D>0. Combinatorial structure. Minimal sufficient query families form the bases of a matroid; the induced distinguishing dimension is well-defined and links to zero-error source coding via graph entropy. We illustrate implications for type systems, databases, and biological taxonomy. All results are mechanized in Lean4 (6000+ lines, 0 sorry).
Paper Structure (71 sections, 29 theorems, 17 equations, 1 figure, 4 tables)

This paper contains 71 sections, 29 theorems, 17 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. The Identification Problem
  3. The Observation Model
  4. The Central Question
  5. The Information Barrier
  6. The Positive Result: Nominal Tagging
  7. Main Contributions
  8. Related Work and Positioning
  9. Paper Organization
  10. Compression Framework
  11. Semantic Compression: The Problem
  12. Universe of Discourse
  13. The Two-Axis Model
  14. Interface Equivalence and Observational Limits
  15. Identification Capacity
  16. ...and 56 more sections

Key Result

Theorem 1.7

Let $P: \mathcal{V} \to Y$ be any function. If $P$ is interface-computable, then $P$ is constant on $\sim$-equivalence classes: Equivalently: no interface-only observer can compute any property that varies within an equivalence class.

Figures (1)

  • Figure 1: Schematic illustration of the $(L, W, D)$ tradeoff. For a concrete example with $k = 1000$ classes and distinguishing dimension $d = 10$, the nominal-tag strategy achieves $L = 10$ bits, $W = O(1)$, $D = 0$, while the interface-only strategy requires $W = 10$ queries and incurs $D > 0$ due to attribute collisions.

Theorems & Definitions (88)

  • Definition 1.1: Entity space and attribute family
  • Remark 1.2: Terminology
  • Definition 1.3: Interface observation family
  • Definition 1.4: Interface profile
  • Definition 1.5: Interface indistinguishability
  • Definition 1.6: Interface-only observer
  • Theorem 1.7: Information barrier
  • proof
  • Remark 1.8: Information-theoretic nature
  • Corollary 1.9: Provenance is not interface-computable
  • ...and 78 more