Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Machine: A New Model for Learning with Tabular Data

Giannis Nikolentzos, Siyun Wang, Johannes Lutzeyer, Michalis Vazirgiannis

TL;DR

This paper shows that a traditional MLP can be viewed as an asynchronous GNN on its DAG representation, and then introduces Graph Neural Machines (GNMs) to generalize learning on tabular data by replacing the DAG with a nearly complete, cyclic directed graph and using synchronous message passing. GNMs are proven to be universal approximators and can simulate multiple MLPs, potentially offering stronger expressive power than MLPs. Empirically, GNMs achieve equal or better performance than MLPs across 15 classification and 6 regression datasets, with reduced overfitting tendencies and the ability to learn sparse, interpretable graph structures via regularization. This work suggests GNMs as a flexible, learnable architectural paradigm for tabular data, with practical implications for model design and hyperparameter tuning.

Abstract

In recent years, there has been a growing interest in mapping data from different domains to graph structures. Among others, neural network models such as the multi-layer perceptron (MLP) can be modeled as graphs. In fact, MLPs can be represented as directed acyclic graphs. Graph neural networks (GNNs) have recently become the standard tool for performing machine learning tasks on graphs. In this work, we show that an MLP is equivalent to an asynchronous message passing GNN model which operates on the MLP's graph representation. We then propose a new machine learning model for tabular data, the so-called Graph Neural Machine (GNM), which replaces the MLP's directed acyclic graph with a nearly complete graph and which employs a synchronous message passing scheme. We show that a single GNM model can simulate multiple MLP models. We evaluate the proposed model in several classification and regression datasets. In most cases, the GNM model outperforms the MLP architecture.

Graph Neural Machine: A New Model for Learning with Tabular Data

TL;DR

This paper shows that a traditional MLP can be viewed as an asynchronous GNN on its DAG representation, and then introduces Graph Neural Machines (GNMs) to generalize learning on tabular data by replacing the DAG with a nearly complete, cyclic directed graph and using synchronous message passing. GNMs are proven to be universal approximators and can simulate multiple MLPs, potentially offering stronger expressive power than MLPs. Empirically, GNMs achieve equal or better performance than MLPs across 15 classification and 6 regression datasets, with reduced overfitting tendencies and the ability to learn sparse, interpretable graph structures via regularization. This work suggests GNMs as a flexible, learnable architectural paradigm for tabular data, with practical implications for model design and hyperparameter tuning.

Abstract

In recent years, there has been a growing interest in mapping data from different domains to graph structures. Among others, neural network models such as the multi-layer perceptron (MLP) can be modeled as graphs. In fact, MLPs can be represented as directed acyclic graphs. Graph neural networks (GNNs) have recently become the standard tool for performing machine learning tasks on graphs. In this work, we show that an MLP is equivalent to an asynchronous message passing GNN model which operates on the MLP's graph representation. We then propose a new machine learning model for tabular data, the so-called Graph Neural Machine (GNM), which replaces the MLP's directed acyclic graph with a nearly complete graph and which employs a synchronous message passing scheme. We show that a single GNM model can simulate multiple MLP models. We evaluate the proposed model in several classification and regression datasets. In most cases, the GNM model outperforms the MLP architecture.
Paper Structure (27 sections, 3 theorems, 14 equations, 8 figures, 3 tables)

This paper contains 27 sections, 3 theorems, 14 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Tabular Data Learning with Graphs.
  4. Structural Learning.
  5. Neural architecture search (NAS).
  6. Preliminaries
  7. Notation
  8. Graph Neural Networks
  9. Connection between MLPs and GNNs
  10. Graph Neural Machines
  11. Expressive power.
  12. Running time.
  13. Limitations.
  14. Experimental Evaluation
  15. Datasets
  16. ...and 12 more sections

Key Result

Theorem 4.1

Let $G_{\text{MLP}}$ be the graph associated with a $K$-layer MLP. Then, the $K$-layer MLP is equivalentA model is characterized by its update mechanism. Two models are equivalent if their update mechanisms are identical. to a GNNMLP applied to $G_{\text{MLP}}$ with the aggregation function (eq:AGG_

Figures (8)

  • Figure 1: Example of the graph representation of a $3$-layer MLP which can process $2$-dimensional input data.
  • Figure 2: An illustration of the computation of a $3$-layer MLP as an asynchronous message passing procedure.
  • Figure 3: An example of the graph representation employed by the proposed GNM model.
  • Figure 4: Training curves of the GNM and MLP models on the Yeast dataset. Four instances of each model are trained consisting of approximately $500$k, $1$m, $2$m and $3$m parameters.
  • Figure 5: Average running time per epoch of the GNM model vs. number of nodes of $G_\text{GNM}$. Experiments performed on the Car dataset. Running time was measured on a NVIDIA RTX A6000 GPU (left) and on a Intel Xeon Silver 4215R CPU @ 3.20GHz (right).
  • ...and 3 more figures

Theorems & Definitions (3)

  • Theorem 4.1
  • Theorem 5.1
  • Corollary 5.2