Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sign Rank Limitations for Inner Product Graph Decoders

Su Hyeong Lee, Qingqi Zhang, Risi Kondor

TL;DR

This paper provides the first theoretical elucidation of this pervasive phenomenon in graph data, and suggests straightforward modifications to circumvent this issue without deviating from the inner product framework.

Abstract

Inner product-based decoders are among the most influential frameworks used to extract meaningful data from latent embeddings. However, such decoders have shown limitations in representation capacity in numerous works within the literature, which have been particularly notable in graph reconstruction problems. In this paper, we provide the first theoretical elucidation of this pervasive phenomenon in graph data, and suggest straightforward modifications to circumvent this issue without deviating from the inner product framework.

Sign Rank Limitations for Inner Product Graph Decoders

TL;DR

This paper provides the first theoretical elucidation of this pervasive phenomenon in graph data, and suggests straightforward modifications to circumvent this issue without deviating from the inner product framework.

Abstract

Inner product-based decoders are among the most influential frameworks used to extract meaningful data from latent embeddings. However, such decoders have shown limitations in representation capacity in numerous works within the literature, which have been particularly notable in graph reconstruction problems. In this paper, we provide the first theoretical elucidation of this pervasive phenomenon in graph data, and suggest straightforward modifications to circumvent this issue without deviating from the inner product framework.
Paper Structure (28 sections, 16 theorems, 20 equations, 15 figures, 1 table)

This paper contains 28 sections, 16 theorems, 20 equations, 15 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Problem motivation
  4. Notable related work
  5. Our Contributions
  6. Classifying Low Rank Graphs
  7. Graphs of rank $1$
  8. Classification of graphs of ranks $2$ or $3$
  9. Lower Bounds for Graph Rank
  10. Decoder Augmentation with Cutoffs
  11. Complex latent embeddings
  12. Generalizations
  13. Architectures and Experiments
  14. Architectures
  15. Experiments
  16. ...and 13 more sections

Key Result

Lemma 2.2

Let $k_i : = \sqrt{p_i}$ where $p_1<p_2<\dots$ are any sequence of positive integer primes. Limiting the indices to the lower triangular portion $i>j$, the periods of $\tilde{A}_{ij}$ can never match. That is, there exists no $n_1,n_2 \in \mathbb{Z}_{\neq 0}$ such that $n_1 t_1 = n_2 t_2$, where $t_

Figures (15)

  • Figure 1: The adjacency $\mathbf{A}$ of the $4$-dimensional $3\times 3\times 3\times 3$ grid graph shown on the left has matrix rank $62$, whereas the rightmost matrix has matrix rank $6$. Both matrices belong to the same equivalence class under the $\operatorname{sign}$ mapping applied elementwise.
  • Figure 2: Examples of rank $\le 2$ graphs.
  • Figure 3: Examples of rank $\le 3$ graphs.
  • Figure 4: (a) Intermediary step in the construction of a class of graphs which do not admit a latent representation of rank $2$. (b) The three node types appearing in a planar grid graph. (c) $\mathbf{w}_1$ corresponds to the center node in (b)-(ii), whereas $\mathbf{w}_i$ for $i \in [2,4]$ depicts the neighboring nodes. For each of these nodes to be disconnected, any two distinct $\mathbf{w}_i,\mathbf{w}_j$ vectors for $i,j \in [2,4]$ must form an obtuse angle.
  • Figure 5: In (b-e), the connection of node $i$ with node $j$ is severed due to $s_i$ where a cutoff value $c$ is depicted as a solid dashed line. (b-c) shows the action of small $s_i$ on $\theta_{ij}$, $\theta_i$. $\theta_{ij}$ is pushed to $\pi/2$ while $\theta_i$ is pulled toward the origin. (d-e) shows a similar phenomenon, but for larger $s_i$.
  • ...and 10 more figures

Theorems & Definitions (28)

  • Definition 2.1
  • Lemma 2.2
  • Lemma 2.3
  • Proposition 2.4
  • proof
  • Proposition 2.5
  • Theorem 3.1
  • proof
  • Remark 3.2
  • Theorem 3.3
  • ...and 18 more