Monotone and Separable Set Functions: Characterizations and Neural Models

TL;DR

This work defines Monotone and Separable (MAS) set functions F that map multisets to vectors so that S⊆T iff F(S)≤F(T), addressing set-containment tasks with strong inductive bias. It provides existential results showing MAS embeddings exist only for finite ground sets with dimension m≥|V| (and more refined bounds when k<∞), and proves nonexistence on infinite ground sets. To cope with infinite domains, the authors propose weakly MAS functions via parametric embeddings F(S,w) that are monotone for all w and separable across w, enabling practical neural models (MASNet) using Hat activations or ReLU-based networks to achieve Hölder separability and stability. They establish universality results for MAS functions on finite ground sets, and demonstrate substantial empirical gains over DeepSets and SetTransformer on set-containment tasks across synthetic, text, and 3D point-cloud data. The MASNet framework provides a principled inductive bias for containment problems, with flexible relaxations and provable stability guarantees, offering a path to robust, monotone-set embeddings in practical applications.

Abstract

Motivated by applications for set containment problems, we consider the following fundamental problem: can we design set-to-vector functions so that the natural partial order on sets is preserved, namely $S\subseteq T \text{ if and only if } F(S)\leq F(T) $. We call functions satisfying this property Monotone and Separating (MAS) set functions. % We establish lower and upper bounds for the vector dimension necessary to obtain MAS functions, as a function of the cardinality of the multisets and the underlying ground set. In the important case of an infinite ground set, we show that MAS functions do not exist, but provide a model called our which provably enjoys a relaxed MAS property we name "weakly MAS" and is stable in the sense of Holder continuity. We also show that MAS functions can be used to construct universal models that are monotone by construction and can approximate all monotone set functions. Experimentally, we consider a variety of set containment tasks. The experiments show the benefit of using our our model, in comparison with standard set models which do not incorporate set containment as an inductive bias. Our code is available in https://github.com/yonatansverdlov/Monotone-Embedding.

Figures (4)

• Figure 1: Using a multiset model as in \ref{['eq:parametric_setfn']} with activations which are not always non-negative, like $\sigma=\mathrm{Tanh}$ will not be monotone. ReLU will be monotone, but to be weakly MAS, two layers are required. TRI and more general hat functions are weakly MAS even with a single layer.
• Figure 2: Acc vs $|T|$ for 1-layer MLP
• Figure 3: Acc vs $|T|$ for all models $\geq 2$ layer MLP
• Figure 4: Acc vs $|S|$ for all models $\geq 2$ layer MLP

Theorems & Definitions (30)

• Example 1
• Theorem 1: Dimension of MAS Function
• Theorem 2: Upper Bound on $\mm(V,k)$
• Theorem 3: Lower bounds on $\mm(V,k)$
• Corollary 4
• Definition 5: weakly MAS function
• Proposition 6
• Proposition 6
• Definition 7: The Hat activation function
• Proposition 8