Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Passive Learning of Lattice Automata from Recurrent Neural Networks

Jaouhar Slimi, Tristan Le Gall, Augustin Lemesle

TL;DR

Passive learning of Lattice Automata from Recurrent Neural Networks tackles the challenge of extracting interpretable automata from RNNs operating over very large or infinite alphabets. The authors integrate Abstract Interpretation with Grammatical Inference to produce Interval Lattice Automata that overapproximate RNN behavior using the Box abstraction on real-valued inputs. The method builds an Interval Prefix Tree Automaton from execution traces and merges states by a similarity score to yield a finite ILA, enabling automata over infinite alphabets. Experiments on Tomita languages and an extended Tomita-2.0 benchmark show fidelity comparable to state-of-the-art while enabling infinite-alphabet automata, with efficient inference and potential for verification and interpretability in time-series and other real-world tasks.

Abstract

We present a passive automata learning algorithm that can extract automata from recurrent networks with very large or even infinite alphabets. Our method combines overapproximations from the field of Abstract Interpretation and passive automata learning from the field of Grammatical Inference. We evaluate our algorithm by first comparing it with the state-of-the-art automata extraction algorithm from Recurrent Neural Networks trained on Tomita grammars. Then, we extend these experiments to regular languages with infinite alphabets, which we propose as a novel benchmark.

Passive Learning of Lattice Automata from Recurrent Neural Networks

TL;DR

Passive learning of Lattice Automata from Recurrent Neural Networks tackles the challenge of extracting interpretable automata from RNNs operating over very large or infinite alphabets. The authors integrate Abstract Interpretation with Grammatical Inference to produce Interval Lattice Automata that overapproximate RNN behavior using the Box abstraction on real-valued inputs. The method builds an Interval Prefix Tree Automaton from execution traces and merges states by a similarity score to yield a finite ILA, enabling automata over infinite alphabets. Experiments on Tomita languages and an extended Tomita-2.0 benchmark show fidelity comparable to state-of-the-art while enabling infinite-alphabet automata, with efficient inference and potential for verification and interpretability in time-series and other real-world tasks.

Abstract

We present a passive automata learning algorithm that can extract automata from recurrent networks with very large or even infinite alphabets. Our method combines overapproximations from the field of Abstract Interpretation and passive automata learning from the field of Grammatical Inference. We evaluate our algorithm by first comparing it with the state-of-the-art automata extraction algorithm from Recurrent Neural Networks trained on Tomita grammars. Then, we extend these experiments to regular languages with infinite alphabets, which we propose as a novel benchmark.

Paper Structure

This paper contains 20 sections, 1 equation, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Outline:
  3. Related work:
  4. Key Notions
  5. Elman Networks
  6. Abstract Lattice
  7. Atomistic Lattice
  8. The Box Abstraction
  9. Partition
  10. Lattice Automata
  11. Language recognition
  12. Representation of the hidden states
  13. Adding transitions
  14. Learning Lattice Automata from RNN execution traces
  15. Building the Interval Prefix Tree Automaton (IPTA)
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Example of a Lattice Automaton. $\mathbb{I} = \{q_0\}, \; \mathbb{F} = \{q_0,q_1,q_2\}$.
  • Figure 2: Example of an IPTA created given the set S from a RNN trained on language 4 of Tomita 2.0
  • Figure 3: Result of the merging of $q_0$ and $q_2$ (from the IPTA)
  • Figure 4: Fidelity and size comparison between ILA and DFA