Inference of Deterministic Finite Automata via Q-Learning
Elaheh Hosseinkhani, Martin Leucker
TL;DR
The paper tackles passive automata inference by reframing Q-learning as a method to induce deterministic finite automata (DFAs) from observations. It introduces Q-PAI, which treats (DFA state, input) as learning states and (next state, finality) as actions, using dual Q-tables and dual automata to iteratively converge to a conforming DFA. Empirical results on Tomita grammars and BLE traces show that Q-PAI can yield accurate, minimal DFAs and often outperforms RNN-based and traditional symbolic methods in terms of compactness and robustness, albeit without formal convergence guarantees. Overall, the work advances a bridge between sub-symbolic reinforcement learning and symbolic automata, enabling interpretable automata extraction from learning dynamics.
Abstract
Traditional approaches to inference of deterministic finite-state automata (DFA) stem from symbolic AI, including both active learning methods (e.g., Angluin's L* algorithm and its variants) and passive techniques (e.g., Biermann and Feldman's method, RPNI). Meanwhile, sub-symbolic AI, particularly machine learning, offers alternative paradigms for learning from data, such as supervised, unsupervised, and reinforcement learning (RL). This paper investigates the use of Q-learning, a well-known reinforcement learning algorithm, for the passive inference of deterministic finite automata. It builds on the core insight that the learned Q-function, which maps state-action pairs to rewards, can be reinterpreted as the transition function of a DFA over a finite domain. This provides a novel bridge between sub-symbolic learning and symbolic representations. The paper demonstrates how Q-learning can be adapted for automaton inference and provides an evaluation on several examples.