Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Networks as Universal Finite-State Machines: A Constructive Deterministic Finite Automaton Theory

Sahil Rajesh Dhayalkar

TL;DR

The paper establishes a constructive theory that feedforward neural networks can exactly emulate deterministic finite automata by unrolling transitions across input length $T$, thereby acting as universal finite-state machines for regular languages. It proves linear separability of DFA transitions, demonstrates that binary threshold networks can achieve exponential state compression with $d=O(\log n)$, and shows Myhill–Nerode equivalence classes can be embedded into latent spaces while preserving separability. The authors also delineate the expressivity boundary, proving that fixed-depth networks cannot recognize non-regular languages such as $L_{CF}=\{a^n b^n\}$, thus aligning neural FSMs with the DFA/regular-language class in the Chomsky hierarchy. Unlike prior empirical work, the results are accompanied by constructive proofs and explicit DFA-unrolled architectures, providing a rigorous neural-symbolic blueprint for implementing discrete symbolic processes in neural systems and clarifying the limitations of finite-depth FFNs.

Abstract

We present a complete theoretical and empirical framework establishing feedforward neural networks as universal finite-state machines (N-FSMs). Our results prove that finite-depth ReLU and threshold networks can exactly simulate deterministic finite automata (DFAs) by unrolling state transitions into depth-wise neural layers, with formal characterizations of required depth, width, and state compression. We demonstrate that DFA transitions are linearly separable, binary threshold activations allow exponential compression, and Myhill-Nerode equivalence classes can be embedded into continuous latent spaces while preserving separability. We also formalize the expressivity boundary: fixed-depth feedforward networks cannot recognize non-regular languages requiring unbounded memory. Unlike prior heuristic or probing-based studies, we provide constructive proofs and design explicit DFA-unrolled neural architectures that empirically validate every claim. Our results bridge deep learning, automata theory, and neural-symbolic computation, offering a rigorous blueprint for how discrete symbolic processes can be realized in continuous neural systems.

Neural Networks as Universal Finite-State Machines: A Constructive Deterministic Finite Automaton Theory

TL;DR

The paper establishes a constructive theory that feedforward neural networks can exactly emulate deterministic finite automata by unrolling transitions across input length , thereby acting as universal finite-state machines for regular languages. It proves linear separability of DFA transitions, demonstrates that binary threshold networks can achieve exponential state compression with , and shows Myhill–Nerode equivalence classes can be embedded into latent spaces while preserving separability. The authors also delineate the expressivity boundary, proving that fixed-depth networks cannot recognize non-regular languages such as , thus aligning neural FSMs with the DFA/regular-language class in the Chomsky hierarchy. Unlike prior empirical work, the results are accompanied by constructive proofs and explicit DFA-unrolled architectures, providing a rigorous neural-symbolic blueprint for implementing discrete symbolic processes in neural systems and clarifying the limitations of finite-depth FFNs.

Abstract

We present a complete theoretical and empirical framework establishing feedforward neural networks as universal finite-state machines (N-FSMs). Our results prove that finite-depth ReLU and threshold networks can exactly simulate deterministic finite automata (DFAs) by unrolling state transitions into depth-wise neural layers, with formal characterizations of required depth, width, and state compression. We demonstrate that DFA transitions are linearly separable, binary threshold activations allow exponential compression, and Myhill-Nerode equivalence classes can be embedded into continuous latent spaces while preserving separability. We also formalize the expressivity boundary: fixed-depth feedforward networks cannot recognize non-regular languages requiring unbounded memory. Unlike prior heuristic or probing-based studies, we provide constructive proofs and design explicit DFA-unrolled neural architectures that empirically validate every claim. Our results bridge deep learning, automata theory, and neural-symbolic computation, offering a rigorous blueprint for how discrete symbolic processes can be realized in continuous neural systems.
Paper Structure (23 sections, 37 equations, 1 figure, 4 tables)

This paper contains 23 sections, 37 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Theoretical Framework
  4. Preliminaries: Deterministic Finite Automata
  5. Theorem 1: Existence of FSM-Emulating Neural Networks
  6. Lemma 1: Linear Separability of State Transitions
  7. Lemma 2: State Encodings via Binary Neurons
  8. Theorem 2: FSM Equivalence Class Representation
  9. Corollary 2.1: FSM State Compression via Embedding
  10. Theorem 3: Hardness of Non-FSM Recognition
  11. Corollary 3.1: Boundary of FSM-Expressive Neural Networks
  12. Experiments
  13. Overview and Common Setup
  14. Validation of Theorem 1: Existence of FSM-Emulating Neural Networks
  15. Validation of Lemma 1: Linear Separability of State Transitions
  16. ...and 8 more sections

Figures (1)

  • Figure 1: Figure depicting a Neural Network emulating a DFA