Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Subjective Distortion: Achievability and Outer Bounds for Distortion Functions with Memory

Hamidreza Abin, Amin Gohari, Andrew W. Eckford

TL;DR

This work extends rate-distortion theory to subjective distortion where the cost depends on past decoder outputs, formalizing a memoryful distortion model with $d(X_i,Y_i,Y_{i-1})$. It provides computable achievability results via Markov-processStationarity and memoryless-kernel constructions, along with convexification over the distortion level $D$, and derives outer bounds using convex envelopes and relaxation techniques. The paper illustrates the bounds with binary and Gaussian examples, clarifying when single-letter bounds are tight and how convexification tightens the feasible region. The framework has potential applications in biological information processing and recommendation systems, offering fundamental limits on performance when fidelity is history-dependent.

Abstract

In some rate-distortion-type problems, the required fidelity of information is affected by past actions. As a result, the distortion function depends not only on the instantaneous distortion between a source symbol and its representation symbol, but also on past representations. In this paper, we give a formal definition of this problem and introduce both inner (achievable) and outer bounds on the rate-distortion tradeoff. We also discuss convexification of the problem, which makes it easier to find bounds. Problems of this type arise in biological information processing, as well as in recommendation engines; we provide an example applied to a simplified biological information processing problem.

Subjective Distortion: Achievability and Outer Bounds for Distortion Functions with Memory

TL;DR

This work extends rate-distortion theory to subjective distortion where the cost depends on past decoder outputs, formalizing a memoryful distortion model with . It provides computable achievability results via Markov-processStationarity and memoryless-kernel constructions, along with convexification over the distortion level , and derives outer bounds using convex envelopes and relaxation techniques. The paper illustrates the bounds with binary and Gaussian examples, clarifying when single-letter bounds are tight and how convexification tightens the feasible region. The framework has potential applications in biological information processing and recommendation systems, offering fundamental limits on performance when fidelity is history-dependent.

Abstract

In some rate-distortion-type problems, the required fidelity of information is affected by past actions. As a result, the distortion function depends not only on the instantaneous distortion between a source symbol and its representation symbol, but also on past representations. In this paper, we give a formal definition of this problem and introduce both inner (achievable) and outer bounds on the rate-distortion tradeoff. We also discuss convexification of the problem, which makes it easier to find bounds. Problems of this type arise in biological information processing, as well as in recommendation engines; we provide an example applied to a simplified biological information processing problem.
Paper Structure (16 sections, 6 theorems, 80 equations, 1 figure)

This paper contains 16 sections, 6 theorems, 80 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Problem Setup and Scientific Motivation
  3. Rate-Distortion Setup and Achievability
  4. Setting
  5. Achievability via stationarity of a Markov process
  6. Achievability via memoryless kernels
  7. Convexification over $D$
  8. Outer bounds
  9. Infeasibility via convex envelope
  10. Infeasibility via relaxation
  11. Numerical Examples
  12. Expansion of Remark 1
  13. Proof of Proposition 1
  14. Proof of Proposition 2
  15. Proof of Proposition 3
  16. ...and 1 more sections

Key Result

Theorem 1

HanBook For a uniformly integrable distortion function, we have where $\mathbf{X}=(X_1,X_2, \cdots)$ is an i.i.d. infinite-length source sequence, $\mathbf{Y}=(Y_1,Y_2, \cdots)$ is an arbitrary (infinite-length) reconstruction (jointly distributed with $\mathbf{X}$), and

Figures (1)

  • Figure 1: Example of outer bound $R_{1}(D)$ and inner bound $R_{2}(D)$ for the distortion function given in (\ref{['eqn:distortion-example-1']})-(\ref{['eqn:distortion-example-2']}), with various values of the cost $c$. A uniform prior is applied to the source. Note that the outer bound is invariant to $c$.

Theorems & Definitions (22)

  • Definition 1
  • Remark 1
  • Theorem 1
  • Theorem 2
  • proof
  • Definition 2
  • Definition 3
  • Remark 2
  • Remark 3
  • Definition 4
  • ...and 12 more