Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explicit Entropic Constructions for Coverage, Facility Location, and Graph Cuts

Rishabh Iyer

TL;DR

The paper investigates whether common monotone submodular objectives used in data-centric tasks are entropic. It develops explicit entropic realizations for weighted coverage/set cover, facility location, truncations/concave-over-modular, saturated coverage, and a monotone graph-cut–type function by constructing modular random-component frameworks (shared/private variables, nested suffixes, linear-entropy constructions, and edge decompositions). For these function families, combinatorial information measures collapse to their classical Shannon counterparts, so submodular mutual information, conditional gain, and submodular conditional mutual information correspond to standard mutual information, conditional entropy, and conditional mutual information. This work bridges combinatorial information measures with traditional information theory, explains the prevalence of certain submodular primitives in practice, and opens avenues to apply entropy inequalities and decomposition arguments in SIM-based learning pipelines.

Abstract

Shannon entropy is a polymatroidal set function and lies at the foundation of information theory, yet the class of entropic polymatroids is strictly smaller than the class of all submodular functions. In parallel, submodular and combinatorial information measures (SIMs) have recently been proposed as a principled framework for extending entropy, mutual information, and conditional mutual information to general submodular functions, and have been used extensively in data subset selection, active learning, domain adaptation, and representation learning. This raises a natural and fundamental question: are the monotone submodular functions most commonly used in practice entropic? In this paper, we answer this question in the affirmative for a broad class of widely used polymatroid functions. We provide explicit entropic constructions for set cover and coverage functions, facility location, saturated coverage, concave-over-modular functions via truncations, and monotone graph-cut-type objectives. Our results show that these functions can be realized exactly as Shannon entropies of appropriately constructed random variables. As a consequence, for these functions, submodular mutual information coincides with classical mutual information, conditional gain specializes to conditional entropy, and submodular conditional mutual information reduces to standard conditional mutual information in the entropic sense. These results establish a direct bridge between combinatorial information measures and classical information theory for many of the most common submodular objectives used in applications.

Explicit Entropic Constructions for Coverage, Facility Location, and Graph Cuts

TL;DR

The paper investigates whether common monotone submodular objectives used in data-centric tasks are entropic. It develops explicit entropic realizations for weighted coverage/set cover, facility location, truncations/concave-over-modular, saturated coverage, and a monotone graph-cut–type function by constructing modular random-component frameworks (shared/private variables, nested suffixes, linear-entropy constructions, and edge decompositions). For these function families, combinatorial information measures collapse to their classical Shannon counterparts, so submodular mutual information, conditional gain, and submodular conditional mutual information correspond to standard mutual information, conditional entropy, and conditional mutual information. This work bridges combinatorial information measures with traditional information theory, explains the prevalence of certain submodular primitives in practice, and opens avenues to apply entropy inequalities and decomposition arguments in SIM-based learning pipelines.

Abstract

Shannon entropy is a polymatroidal set function and lies at the foundation of information theory, yet the class of entropic polymatroids is strictly smaller than the class of all submodular functions. In parallel, submodular and combinatorial information measures (SIMs) have recently been proposed as a principled framework for extending entropy, mutual information, and conditional mutual information to general submodular functions, and have been used extensively in data subset selection, active learning, domain adaptation, and representation learning. This raises a natural and fundamental question: are the monotone submodular functions most commonly used in practice entropic? In this paper, we answer this question in the affirmative for a broad class of widely used polymatroid functions. We provide explicit entropic constructions for set cover and coverage functions, facility location, saturated coverage, concave-over-modular functions via truncations, and monotone graph-cut-type objectives. Our results show that these functions can be realized exactly as Shannon entropies of appropriately constructed random variables. As a consequence, for these functions, submodular mutual information coincides with classical mutual information, conditional gain specializes to conditional entropy, and submodular conditional mutual information reduces to standard conditional mutual information in the entropic sense. These results establish a direct bridge between combinatorial information measures and classical information theory for many of the most common submodular objectives used in applications.
Paper Structure (13 sections, 8 theorems, 29 equations)

This paper contains 13 sections, 8 theorems, 29 equations.

Table of Contents

  1. Introduction
  2. Background: submodularity and entropy
  3. Information measures and their submodular counterparts
  4. Question and contributions
  5. Entropy Region and Polyhedral Polymatroids
  6. Examples of known entropic functions.
  7. Entropic Constructions for Common Submodular Functions
  8. Weighted Coverage / Set Cover is Entropic
  9. Facility Location is Entropic (Max-by-Nesting Construction)
  10. Truncations and Concave-over-Modular
  11. Saturated Coverage
  12. Monotone Graph-Cut--Like Function is Entropic
  13. Conclusion

Key Result

Theorem 1

Let $U$ be a universe with weights $(w_u)_{u\in U}$ where $w_u\ge 0$. Let the ground set be $\Omega=[n]$ with subsets $U_i\subseteq U$ for $i\in[n]$. Define Then $f$ is entropic.

Theorems & Definitions (17)

  • Definition 1: Entropic set function
  • Theorem 1: Weighted coverage (set cover) is entropic
  • proof
  • Lemma 1: Max is entropic via nested suffix variables
  • proof
  • Theorem 2: Facility location is entropic
  • proof
  • Theorem 3: Cardinality truncation $\min(|A|,k)$ is entropic
  • proof
  • Corollary 1: Weighted truncation $\min(w(A),k)$ is entropic (integer weights)
  • ...and 7 more