Ultrafast On-chip Online Learning via Spline Locality in Kolmogorov-Arnold Networks

TL;DR

This work tackles the challenge of ultrafast, model-free online learning on-chip under fixed-point constraints. It shows that Kolmogorov–Arnold Networks (KANs), with their B-spline activations, enable sparse per-sample updates and capacity growth by increasing grid size $G$, while keeping compute roughly constant. The authors prove and demonstrate both theoretical properties—sparse gradient updates and robustness to fixed-point quantization—and empirical hardware results on FPGA showing sub-100 ns latency with superior resource efficiency compared to MLP baselines. The approach yields robust online learning in drifting regression, adaptive qubit readout, and non-stationary control, offering a practical path to deterministic, deterministic-timescale adaptation in quantum, plasma, and sensing systems. Overall, the paper presents a compelling case for specialized spline-based networks as a hardware-native solution for real-time, on-device learning.

Abstract

Ultrafast online learning is essential for high-frequency systems, such as controls for quantum computing and nuclear fusion, where adaptation must occur on sub-microsecond timescales. Meeting these requirements demands low-latency, fixed-precision computation under strict memory constraints, a regime in which conventional Multi-Layer Perceptrons (MLPs) are both inefficient and numerically unstable. We identify key properties of Kolmogorov-Arnold Networks (KANs) that align with these constraints. Specifically, we show that: (i) KAN updates exploiting B-spline locality are sparse, enabling superior on-chip resource scaling, and (ii) KANs are inherently robust to fixed-point quantization. By implementing fixed-point online training on Field-Programmable Gate Arrays (FPGAs), a representative platform for on-chip computation, we demonstrate that KAN-based online learners are significantly more efficient and expressive than MLPs across a range of low-latency and resource-constrained tasks. To our knowledge, this work is the first to demonstrate model-free online learning at sub-microsecond latencies.

Figures (10)

• Figure 1: Hardware scaling under ultrafast on-chip online learning. On a non-stationary qubit readout task with fixed-point training, MLPs grow roughly linearly in on-chip resources (DSP/FF/LUT) and in forward/backward latency with parameter count. KANs leverage B-spline locality and favorable approximation scaling, increasing capacity (via grid size $G$) with near-constant resources and sub-100 ns latency. CPU/GPU (A100) lines show reference latencies for PyTorch implementations.
• Figure 2: Overview of our streaming custom hardware kernels for fully online learning. KAN (left) and MLP (right) are synthesized in Vitis HLS as single hardware kernels (200 MHz, AMD Virtex UltraScale+ XCVU13P) that process streaming inputs and perform forward inference, backward gradient propagation, and in-place on-chip parameter updates with deterministic latency. KAN: each input $x$ is mapped to a grid cell index $k$ and LUT index $u$; only the $(p{+}1)$ active B-spline basis functions are read from a small ROM LUT and accumulated, and the cached $(k,u)$ context is reused in backprop to update only active coefficients. Gradients are computed using B-spline derivatives stored in LUTs. MLP: layers compute $z=Wx+b$, $y=\sigma(z)$ with dense MACs and global weight/bias updates using cached pre-activations. LUTs and per-sample context buffers are fully partitioned, while trainable parameters reside entirely on-chip (LUTRAM/BRAM/FF depending on synthesis) with explicit array partitioning to expose parallel accesses (no external DRAM).
• Figure 3: Post-synthesis FPGA latency and resource cost for fully online learning. Models are synthesized for an AMD Virtex™ UltraScale+™ XCVU13P at 200 MHz. Across tasks, KAN achieves the best latency-resource trade-off (DSP/FF/LUT) and the highest update rate (online updates/s, forward + backward). MLPs use more resources and/or diverge under fixed-point updates. No BRAM was used for either architecture across these experiments.
• Figure 4: Adaptive function approximation under concept drift.Left: Cumulative regret with regime changes at $t=500,1000$: KAN adapts rapidly, MLP-P diverges, and MLP-L converges more slowly. Right: Final cumulative regret vs. fixed-point bitwidth (2 integer bits): KAN remains stable under quantization while MLP-L degrades at low precision.
• Figure 5: Empirical validation of approximation under quantization.Left (KAN): Error improves with grid size $G$ until reaching a quantization floor set by bitwidth. Right (MLP): Increasing parameter count $N$ amplifies sensitivity to quantization and destabilizes learning.

Theorems & Definitions (8)

• Definition 3.1: Saturated fixed-point quantization
• Lemma 3.2: Local support of B-splines KAN
• Theorem 3.3: Update complexity reduction
• Theorem 3.4: KAN Activation Bounds
• Theorem 3.5: Bounded Gradient Sensitivity
• proof
• proof
• proof