Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rate-Distortion Analysis of Compressed Query Delegation with Low-Rank Riemannian Updates

Faruk Alpay, Bugra Kilictas

TL;DR

This paper introduces compressed query delegation (CQD), a principled approach to reasoning under bounded working memory by compressing latent state into a low-rank tensor query, delegating to a noisy oracle, and updating the latent state with Riemannian optimization on a fixed-rank manifold. It presents a math-first formulation that links CQD to rate–distortion and information bottleneck principles, proving the optimality of spectral hard-thresholding under quadratic distortion and establishing convergence guarantees for Riemannian stochastic approximation in the presence of oracle noise. The methodology relies on a Tucker/HOSVD-based low-rank tensor model and Adaptive Spectral Masking to preserve dominant spectral content while respecting a query budget; updates are performed via retractions on the Stiefel manifold. Empirically, CQD improves bounded-context performance on a 2,500-item suite against chain-of-thought baselines and reveals a drift–gain coupling in a human benchmark across multiple oracles, highlighting a principled trade-off between information gain and representation stability in constrained reasoning tasks.

Abstract

Bounded-context agents fail when intermediate reasoning exceeds an effective working-memory budget. We study compressed query delegation (CQD): (i) compress a high-dimensional latent reasoning state into a low-rank tensor query, (ii) delegate the minimal query to an external oracle, and (iii) update the latent state via Riemannian optimization on fixed-rank manifolds. We give a math-first formulation: CQD is a constrained stochastic program with a query-budget functional and an oracle modeled as a noisy operator. We connect CQD to classical rate-distortion and information bottleneck principles, showing that spectral hard-thresholding is optimal for a natural constrained quadratic distortion problem, and we derive convergence guarantees for Riemannian stochastic approximation under bounded oracle noise and smoothness assumptions. Empirically, we report (A) a 2,500-item bounded-context reasoning suite (BBH-derived tasks plus curated paradox instances) comparing CQD against chain-of-thought baselines under fixed compute and context; and (B) a human "cognitive mirror" benchmark (N=200) measuring epistemic gain and semantic drift across modern oracles.

Rate-Distortion Analysis of Compressed Query Delegation with Low-Rank Riemannian Updates

TL;DR

This paper introduces compressed query delegation (CQD), a principled approach to reasoning under bounded working memory by compressing latent state into a low-rank tensor query, delegating to a noisy oracle, and updating the latent state with Riemannian optimization on a fixed-rank manifold. It presents a math-first formulation that links CQD to rate–distortion and information bottleneck principles, proving the optimality of spectral hard-thresholding under quadratic distortion and establishing convergence guarantees for Riemannian stochastic approximation in the presence of oracle noise. The methodology relies on a Tucker/HOSVD-based low-rank tensor model and Adaptive Spectral Masking to preserve dominant spectral content while respecting a query budget; updates are performed via retractions on the Stiefel manifold. Empirically, CQD improves bounded-context performance on a 2,500-item suite against chain-of-thought baselines and reveals a drift–gain coupling in a human benchmark across multiple oracles, highlighting a principled trade-off between information gain and representation stability in constrained reasoning tasks.

Abstract

Bounded-context agents fail when intermediate reasoning exceeds an effective working-memory budget. We study compressed query delegation (CQD): (i) compress a high-dimensional latent reasoning state into a low-rank tensor query, (ii) delegate the minimal query to an external oracle, and (iii) update the latent state via Riemannian optimization on fixed-rank manifolds. We give a math-first formulation: CQD is a constrained stochastic program with a query-budget functional and an oracle modeled as a noisy operator. We connect CQD to classical rate-distortion and information bottleneck principles, showing that spectral hard-thresholding is optimal for a natural constrained quadratic distortion problem, and we derive convergence guarantees for Riemannian stochastic approximation under bounded oracle noise and smoothness assumptions. Empirically, we report (A) a 2,500-item bounded-context reasoning suite (BBH-derived tasks plus curated paradox instances) comparing CQD against chain-of-thought baselines under fixed compute and context; and (B) a human "cognitive mirror" benchmark (N=200) measuring epistemic gain and semantic drift across modern oracles.
Paper Structure (32 sections, 4 theorems, 28 equations, 1 figure, 2 tables, 2 algorithms)

This paper contains 32 sections, 4 theorems, 28 equations, 1 figure, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Notation
  4. CQD as Constrained Rate--Distortion / Information Bottleneck
  5. Low-Rank Tensor Model and Adaptive Spectral Masking
  6. Tucker/HOSVD representation
  7. ASM definition
  8. Optimality of spectral hard-thresholding under quadratic distortion
  9. Approximation bound for truncated HOSVD
  10. Query-budget scaling
  11. Oracle Model, Learning Dynamics, and Riemannian Updates
  12. Oracle as noisy operator
  13. Learning objective
  14. Riemannian update rule
  15. Algorithms
  16. ...and 17 more sections

Key Result

Theorem 5.2

Let $A\in\mathbb{R}^{m\times n}$ and $A_r$ be its truncated SVD at rank $r$. Then and the minimizer is achieved by keeping the top $r$ singular values/vectors.

Figures (1)

  • Figure 1: Benchmark B: Semantic drift vs. epistemic gain (Adjusted Layout).

Theorems & Definitions (6)

  • Definition 3.1: Multilinear rank
  • Definition 5.1: Adaptive Spectral Masking (ASM)
  • Theorem 5.2: Eckart--Young--Mirsky eckart1936mirsky1960
  • Proposition 5.3: Mode-wise quadratic optimality
  • Lemma 5.4: Truncated HOSVD tail bound kolda2009delathauwer2000
  • Theorem 6.2: Riemannian stochastic approximation convergence bonnabel2013absil2009boumal2023