Extended Deep Submodular Functions

TL;DR

The paper tackles the limitation that Deep Submodular Functions (DSFs) cannot represent all monotone submodular functions. It introduces Extended Deep Submodular Functions (EDSFs), defined as the minimum of multiple DSFs, and proves that EDSFs can represent every monotone set function, including all monotone submodular functions, while preserving concavity for nonnegative inputs. Theoretical developments rely on polymatroid concepts and intersections, augmented by a Min-Component to maintain the desired structure. Empirically, EDSFs demonstrate dramatically lower generalization error than DSFs in learning coverage functions and show superior performance in learning cut functions and in social welfare maximization under gradient-based optimization. Collectively, EDSFs offer a powerful, more expressive neural framework for representing and learning monotone set/submodular functions with practical optimization advantages.

Abstract

We introduce a novel category of set functions called Extended Deep Submodular functions (EDSFs), which are neural network-representable. EDSFs serve as an extension of Deep Submodular Functions (DSFs), inheriting crucial properties from DSFs while addressing innate limitations. It is known that DSFs can represent a limiting subset of submodular functions. In contrast, through an analysis of polymatroid properties, we establish that EDSFs possess the capability to represent all monotone submodular functions, a notable enhancement compared to DSFs. Furthermore, our findings demonstrate that EDSFs can represent any monotone set function, indicating the family of EDSFs is equivalent to the family of all monotone set functions. Additionally, we prove that EDSFs maintain the concavity inherent in DSFs when the components of the input vector are non-negative real numbers-an essential feature in certain combinatorial optimization problems. Through extensive experiments, we illustrate that EDSFs exhibit significantly lower empirical generalization error than DSFs in the learning of coverage functions. This suggests that EDSFs present a promising advancement in the representation and learning of set functions with improved generalization capabilities.

Figures (10)

• Figure 1: An example of deep submodular function bilmes_deep_2017.
• Figure 2: The simple architecture for the representation of $g_A$ as a DSF. The function $\min(\cdot,c_A)$ is the activation function of the lower node in the hidden layer. The upper hidden node has no activation function.
• Figure 3: The architecture for the representation of the submodular function $f$.
• Figure 4: Learning coverage function with coverage probability 0.1, using the EDSF architecture, showing Training loss, Truth vs. Predicted values for train and test samples
• Figure 5: Learning coverage function with coverage probability 0.3, using the EDSF architecture, showing Training loss, Truth vs. Predicted values for train and test samples

Theorems & Definitions (22)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Definition 2.5
• Definition 2.6
• Definition 2.7
• Lemma 2.8
• Definition 2.9
• Definition 2.10