Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the (Non) Injectivity of Piecewise Linear Janossy Pooling

Ilai Reshef, Nadav Dym

TL;DR

The paper investigates whether continuous piecewise linear (CPwL) $k$-ary Janossy pooling can yield injective and bi-Lipschitz multiset-to-vector mappings. It proves a negative result in full generality: for $n>k$ and a domain $C$ containing a line segment, CPwL $k$-ary Janossy pooling is not injective on multisets. It simultaneously provides a positive result on restricted domains with all elements distinct (compact $D$): there exists a CPwL construction for which 1-ary Janossy pooling is injective and bi-Lipschitz with respect to the Wasserstein distance, with complexity depending on the separation constant $R(D)$. The work clarifies the limits of CPwL pooling for injectivity and bi-Lipschitzness, guiding when standard deepsets suffice and when alternative methods (e.g., sorting-based or higher-order pooling) are warranted, particularly depending on how close multiset elements can be. It also highlights practical implications for molecular-like datasets versus highly clustered point clouds.

Abstract

Multiset functions, which are functions that map multisets to vectors, are a fundamental tool in the construction of neural networks for multisets and graphs. To guarantee that the vector representation of the multiset is faithful, it is often desirable to have multiset mappings that are both injective and bi-Lipschitz. Currently, there are several constructions of multiset functions achieving both these guarantees, leading to improved performance in some tasks but often also to higher compute time than standard constructions. Accordingly, it is natural to inquire whether simpler multiset functions achieving the same guarantees are available. In this paper, we make a large step towards giving a negative answer to this question. We consider the family of k-ary Janossy pooling, which includes many of the most popular multiset models, and prove that no piecewise linear Janossy pooling function can be injective. On the positive side, we show that when restricted to multisets without multiplicities, even simple deep-sets models suffice for injectivity and bi-Lipschitzness.

On the (Non) Injectivity of Piecewise Linear Janossy Pooling

TL;DR

The paper investigates whether continuous piecewise linear (CPwL) -ary Janossy pooling can yield injective and bi-Lipschitz multiset-to-vector mappings. It proves a negative result in full generality: for and a domain containing a line segment, CPwL -ary Janossy pooling is not injective on multisets. It simultaneously provides a positive result on restricted domains with all elements distinct (compact ): there exists a CPwL construction for which 1-ary Janossy pooling is injective and bi-Lipschitz with respect to the Wasserstein distance, with complexity depending on the separation constant . The work clarifies the limits of CPwL pooling for injectivity and bi-Lipschitzness, guiding when standard deepsets suffice and when alternative methods (e.g., sorting-based or higher-order pooling) are warranted, particularly depending on how close multiset elements can be. It also highlights practical implications for molecular-like datasets versus highly clustered point clouds.

Abstract

Multiset functions, which are functions that map multisets to vectors, are a fundamental tool in the construction of neural networks for multisets and graphs. To guarantee that the vector representation of the multiset is faithful, it is often desirable to have multiset mappings that are both injective and bi-Lipschitz. Currently, there are several constructions of multiset functions achieving both these guarantees, leading to improved performance in some tasks but often also to higher compute time than standard constructions. Accordingly, it is natural to inquire whether simpler multiset functions achieving the same guarantees are available. In this paper, we make a large step towards giving a negative answer to this question. We consider the family of k-ary Janossy pooling, which includes many of the most popular multiset models, and prove that no piecewise linear Janossy pooling function can be injective. On the positive side, we show that when restricted to multisets without multiplicities, even simple deep-sets models suffice for injectivity and bi-Lipschitzness.

Paper Structure

This paper contains 11 sections, 10 theorems, 29 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Main Results
  3. Paper Structure
  4. Problem Statement
  5. Non-injectivity of Janossy Pooling for general domains
  6. Janossy pooling when $k=n$
  7. Injectivity under restricted domains
  8. Dependence on separation
  9. Conclusion, limitations and future Work
  10. Technical Appendix
  11. Proof of Theorem \ref{['proposition:n-choose-k-points-in-one-polytope']}

Key Result

Theorem 3.1

[Non-Injectivity of $k$-ary Janossy Pooling of CPwL functions] Let $C$ be a subset of $\mathbb{R}^d$ that contains a line segment (usually this will be $[0,1]^d$ or $\mathbb{R}^d$ itself). Let $f:(\mathbb{R}^d)^k \rightarrow \mathbb{R}^m$ be a continuous piecewise linear (CPwL) function. Let $n>k$,

Figures (2)

  • Figure 1: This figure illustrates that the assumption that multisets do not have (near)-repeated points is realistic for small molecule datasets. In (a) we show an example multiset from the QM9 qm9ramakrishnan2014quantum small molecule dataset. This example shows visually that different set elements are not very close together. In (b) we see the statistics of the minimal distance within each multiset (normalized by the maximal distance), over 1,000 represetative samples from QM9. In all these instances, the minimal distance was never lower than $0.1$.
  • Figure 2: Visualization of the property from Theorem \ref{['proposition:n-choose-k-points-in-one-polytope']} for a partition of $[0,1]^2$ into four squares. The point $\mathbf{w}=(3/8,2/8,1/8)$ fulfills the conditions of the proposition, as all three $2$ dimensional ordered subvectors are in the same square (see orange dots). The vector $(7/8,6/8,1/8)$ does not fulfill the condition (see green dots)

Theorems & Definitions (22)

  • Theorem 3.1
  • Theorem 3.2
  • proof : Proof of Theorem \ref{['theorem:non-injective-janossy']}
  • Definition 4.1
  • Definition 4.2
  • Theorem 4.3
  • Proposition 4.4
  • proof
  • proof : Proof of Theorem \ref{['thm:main_injectivity']}
  • Lemma 4.5
  • ...and 12 more