Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizing the Fano inequality further

Raghav Bongole, Tobias J. Oechtering, Mikael Skoglund

TL;DR

This work advances information-theoretic lower bounds for tail-aware performance in interactive statistical decision making (ISDM) by generalizing the interactive Fano framework to arbitrary bounded transforms of the loss through a randomized one-bit statistic. It establishes a two-sided Bernoulli $f$-divergence bound, enabling CVaR (Expected Shortfall) lower bounds via a bounded hinge transform and the Rockafellar–Uryasev representation, and provides explicit mutual-information–based CVaR guarantees under KL with a mixture reference using Pinsker’s inequality. The key contributions include (i) a general two-sided bound for $\mathbb{E}[\phi(L)]$, (ii) Bayesian CVaR lower bounds for ISDM, and (iii) a mutual-information–based CVaR bound that connects informational indistinguishability to tail risk. This framework subsumes the previous Chen et al. one-sided results as a special case and offers a principled route to understanding tail performance in bandits and MDP settings under information constraints.

Abstract

Interactive statistical decision making (ISDM) features algorithm-dependent data generated through interaction. Existing information-theoretic lower bounds in ISDM largely target expected risk, while tail-sensitive objectives are less developed. We generalize the interactive Fano framework of Chen et al. by replacing the hard success event with a randomized one-bit statistic representing an arbitrary bounded transform of the loss. This yields a Bernoulli f-divergence inequality, which we invert to obtain a two-sided interval for the transform, recovering the previous result as a special case. Instantiating the transform with a bounded hinge and using the Rockafellar-Uryasev representation, we derive lower bounds on the prior-predictive (Bayesian) CVaR of bounded losses. For KL divergence with the mixture reference distribution, the bound becomes explicit in terms of mutual information via Pinsker's inequality.

Generalizing the Fano inequality further

TL;DR

This work advances information-theoretic lower bounds for tail-aware performance in interactive statistical decision making (ISDM) by generalizing the interactive Fano framework to arbitrary bounded transforms of the loss through a randomized one-bit statistic. It establishes a two-sided Bernoulli -divergence bound, enabling CVaR (Expected Shortfall) lower bounds via a bounded hinge transform and the Rockafellar–Uryasev representation, and provides explicit mutual-information–based CVaR guarantees under KL with a mixture reference using Pinsker’s inequality. The key contributions include (i) a general two-sided bound for , (ii) Bayesian CVaR lower bounds for ISDM, and (iii) a mutual-information–based CVaR bound that connects informational indistinguishability to tail risk. This framework subsumes the previous Chen et al. one-sided results as a special case and offers a principled route to understanding tail performance in bandits and MDP settings under information constraints.

Abstract

Interactive statistical decision making (ISDM) features algorithm-dependent data generated through interaction. Existing information-theoretic lower bounds in ISDM largely target expected risk, while tail-sensitive objectives are less developed. We generalize the interactive Fano framework of Chen et al. by replacing the hard success event with a randomized one-bit statistic representing an arbitrary bounded transform of the loss. This yields a Bernoulli f-divergence inequality, which we invert to obtain a two-sided interval for the transform, recovering the previous result as a special case. Instantiating the transform with a bounded hinge and using the Rockafellar-Uryasev representation, we derive lower bounds on the prior-predictive (Bayesian) CVaR of bounded losses. For KL divergence with the mixture reference distribution, the bound becomes explicit in terms of mutual information via Pinsker's inequality.
Paper Structure (8 sections, 8 theorems, 40 equations)

This paper contains 8 sections, 8 theorems, 40 equations.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Notation
  4. Problem setup
  5. Prior results
  6. New results
  7. Relation to Prior Work
  8. Conclusion and Future Work

Key Result

Lemma 1

Let $(\mathcal{X},\mathcal{F})$, $(\mathcal{U},\mathcal{H})$, and $(\mathcal{Y},\mathcal{G})$ be measurable spaces. Let $P_X,Q_X$ be probability measures on $(\mathcal{X},\mathcal{F})$, and let $\nu$ be a probability measure on $(\mathcal{U},\mathcal{H})$. Let $\Phi:(\mathcal{X}\times\mathcal{U},\ma Let $T\circ P_X$ denote the pushforward (output) measure on $(\mathcal{Y},\mathcal{G})$ induced by

Theorems & Definitions (16)

  • Lemma 1: Data processing inequality for $f$-divergences polyanskiy2025information
  • Lemma 2: Bernoulli monotonicity
  • Theorem 1: Interactive quantile Fano
  • proof : Proof
  • Remark 1: Discussion: proof steps in the interactive quantile Fano
  • Remark 2: From tail to expectation
  • Theorem 2: Generalized Interactive Fano inequality
  • proof
  • Remark 3
  • Remark 4: Two-sided vs Chen et al.'s bound
  • ...and 6 more