Kirchhoff-Inspired Neural Networks for Evolving High-Order Perception

Abstract

Deep learning architectures are fundamentally inspired by neuroscience, particularly the structure of the brain's sensory pathways, and have achieved remarkable success in learning informative data representations. Although these architectures mimic the communication mechanisms of biological neurons, their strategies for information encoding and transmission are fundamentally distinct. Biological systems depend on dynamic fluctuations in membrane potential; by contrast, conventional deep networks optimize weights and biases by adjusting the strengths of inter-neural connections, lacking a systematic mechanism to jointly characterize the interplay among signal intensity, coupling structure, and state evolution. To tackle this limitation, we propose the Kirchhoff-Inspired Neural Network (KINN), a state-variable-based network architecture constructed based on Kirchhoff's current law. KINN derives numerically stable state updates from fundamental ordinary differential equations, enabling the explicit decoupling and encoding of higher-order evolutionary components within a single layer while preserving physical consistency, interpretability, and end-to-end trainability. Extensive experiments on partial differential equation (PDE) solving and ImageNet image classification validate that KINN outperforms state-of-the-art existing methods.

Figures (2)

• Figure 1: KINN models temporal evolution as an intrinsic state variable and elevates representational order through cascade composition.a, Biological motivation at the neural-population level. A received neural signal can be decomposed into three complementary aspects: connection, indicating which upstream units contribute to the signal; intensity, indicating its instantaneous or short-time magnitude; and evolution, indicating its continuous temporal development. b, In many contemporary architectures, order information is commonly introduced through positional encoding, in which positional cues are externally injected into discrete feature cells rather than emerging from an internal evolving state. c, Recurrent models introduce a latent state, but a single hidden-state transition typically realizes only shallow first-order state evolution. d, KINN addresses this limitation through cascaded Kirchhoff circuit processing, where multi-stage internal state transitions progressively enrich temporal dynamics and enable higher-order evolution. e, A single Kirchhoff Neural Cell (KNC) realizes a first-order state-evolution process under Kirchhoff-inspired RC dynamics, in which latent voltage, input injection, leakage dissipation, and coupling modulation jointly determine the state update and readout. f, The full architecture instantiates KINN from key modules, combining KNC and the Cascaded Kirchhoff Block (CKB) to integrate zeroth-, first-, and higher-order evolutionary components within a unified representational hierarchy. g, We evaluate KINN across three task families spanning neural operator learning (Darcy Flow), spatiotemporal dynamics prediction (Shallow Water and Navier--Stokes), and visual recognition (ImageNet-1K). h, KINN yields consistent gains over strong baselines across these domains, supporting the view that intrinsic state evolution and cascade-induced order elevation provide a unified, stable, and interpretable route to higher-order evolution modelling.
• Figure 2: Architectural instantiation of KINN and its integration into neural operators and encoder--decoder backbones.a, Structure of a single Kirchhoff Neural Cell (KNC). The hidden representation is modeled as a latent voltage state $v(t)$ governed by Kirchhoff-inspired RC dynamics, where input injection, leakage dissipation, and state retention jointly determine the cell update and readout. b,Cascaded Kirchhoff Block (CKB). By stacking multiple KNCs in sequence, the output of each stage is propagated to the next stage, progressively increasing the effective order of the resulting dynamics and enabling higher-order state evolution. c,$N$-order Cascaded Kirchhoff Operator (CKO). KNCs are combined with lightweight projection, normalization and nonlinear transformation modules to form a trainable high-order evolution operator for deep architectures. d,FNO with KINN. The proposed Kirchhoff modules are incorporated into Fourier neural operators by inserting CKB-enhanced evolution pathways alongside spectral and convolutional transformations, enabling multi-order feature interaction in operator learning. e,U-Net with KINN. CKB is further embedded into the encoder--decoder hierarchy, where Kirchhoff-inspired evolution modules are placed along the downsampling, bottleneck and upsampling paths to enrich latent dynamics across scales. Together, these instantiations show that KINN is not restricted to a single backbone, but provides a unified and modular mechanism for introducing intrinsic and higher-order state evolution into diverse architectures.