Web Neural Network with Complete DiGraphs

TL;DR

The paper introduces a web neural network that forms a complete directed graph of $Q$ neurons to process continuous data across timesteps, aiming to more closely resemble biological neural networks by incorporating cycles and removing strict layerwise sequencing. It presents both a naive ($O(TQ)$ per step, $O(ETQ)$ total) and a vectorized ($O(ET)$) forward-pass implementation, using PyTorch matmul to exploit batched computations. Evaluations on Titanic and MNIST demonstrate functional incremental classification, with Titanic achieving about 89% validation accuracy and MNIST reaching 95% validation and 94% test accuracy, while revealing interpretable per-timestep prediction histories. The work lays foundational methodology for a brain-inspired, context-preserving network and points to future directions in wiring variations, time-series handling, and language modeling, despite notable computational demands.

Abstract

This paper introduces a new neural network model that aims to mimic the biological brain more closely by structuring the network as a complete directed graph that processes continuous data for each timestep. Current neural networks have structures that vaguely mimic the brain structure, such as neurons, convolutions, and recurrence. The model proposed in this paper adds additional structural properties by introducing cycles into the neuron connections and removing the sequential nature commonly seen in other network layers. Furthermore, the model has continuous input and output, inspired by spiking neural networks, which allows the network to learn a process of classification, rather than simply returning the final result.

Figures (7)

• Figure 1: Neuron Structure
• Figure 2: Neuron Activity
• Figure 3: Titanic Training Loss
• Figure 4: Titanic Accuracy
• Figure 5: MNIST Training Loss