Artificial Kuramoto Oscillatory Neurons

TL;DR

AKOrN introduces vector-valued, sphere-constrained oscillators that evolve via generalized Kuramoto dynamics to bind features through synchronization, enabling dynamic spatiotemporal representations. The architecture supports convolutional and attentive connectivities and interleaves Kuramoto layers with readout modules, achieving improved unsupervised object discovery, Sudoku reasoning, and robustness with well-calibrated predictions. Key findings include superior object-binding features, effectiveness on natural images, energy-guided decision reliability, and the ability to extend the number of Kuramoto steps at test time to boost OOD performance. The work suggests a fundamental shift toward dynamical neuronal representations in neural networks, with potential implications for memory, reasoning, and reliable AI systems.

Abstract

It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network. More recently, it was also hypothesized that dynamic (spatiotemporal) representations play an important role in both neuroscience and AI. Building on these ideas, we introduce Artificial Kuramoto Oscillatory Neurons (AKOrN) as a dynamical alternative to threshold units, which can be combined with arbitrary connectivity designs such as fully connected, convolutional, or attentive mechanisms. Our generalized Kuramoto updates bind neurons together through their synchronization dynamics. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, calibrated uncertainty quantification, and reasoning. We believe that these empirical results show the importance of rethinking our assumptions at the most basic neuronal level of neural representation, and in particular show the importance of dynamical representations. Code:https://github.com/autonomousvision/akorn Project page:https://takerum.github.io/akorn_project_page/

Figures (36)

• Figure 1: Our proposed artificial Kuramoto oscillatory neurons (AKOrN). The series of pictures on the left are 64$\times$ 64 oscillators evolving by the Kuramoto updates (Eq. \ref{['eq:kuramoto']}), along with a plot of the energies computed by Eq. \ref{['eq:energy']}. Each single oscillator $\mathbf{x}_i$ is an $N$-dimensional vector on the sphere and is influenced by (1) connected oscillators through the weights $\mathbf{J}_{ij}$, (2) conditional stimuli $\mathbf{c}_i$, and (3) $\boldsymbol{\Omega}_i$ that determines the natural frequency of each oscillator. See Fig. \ref{['fig:osc_timeevo']} for details on $\mathbf{C}$ and $\mathbf{J}$.
• Figure 2: Our proposed Kuramoto-based network (here, for image processing). Each block consists of a Kuramoto-layer and a readout module described in Sec \ref{['sec:block']}. $\mathbf{C}^{(L)}$ is used to make the final prediction of our model. Similar network structures are proposed in bansal2022endgeiping2025scaling.
• Figure 3: Object discovery performance on synthetic datasets.
• Figure 4: Robustness to adversarial examples by AutoAttack (Adv) and common corruptions (CC) on CIFAR10. ${}^*$The attack is done by AutoAttack with EoT athalye2018obfuscated. $\|\epsilon\|_{\infty}$ is set to 8/255. Expected Calibration Error (ECE) measures the alignment between confidence of the prediction and accuracy. The top two methods are selected from the highest-ranked methods on https://robustbench.github.io/.
• Figure 5: Visualization of clusters on (Left) PascalVOC and (Right) COCO2017.

Theorems & Definitions (4)

• Proposition G.1: sufficient conditions, non-constructive
• proof
• Proposition G.2: sufficient conditions, constructive
• proof