A HAT Trick: Automatically Verifying Representation Invariants Using Symbolic Finite Automata

TL;DR

The paper tackles verifying representation invariants for clients of opaque, stateful libraries and introduces Hoare Automata Types (HATs), a refinement-type construct that encodes pre- and post-conditions as symbolic finite automata over sequences of library calls and returns. HATs enable modular reasoning by expressing invariants like the set representation invariant $I_{Set}$ with $LTL_f$ over interactions with a key-value store, and support automated verification via a bidirectional typing algorithm and SMT-backed checks, as implemented in Marple for OCaml data structures. It formalizes a trace-based specification framework, embeds it into a compositional refinement type system, and develops a bidirectional type-checking algorithm that translates declarations into efficient SMT checks, with practical evaluation on non-trivial OCaml datatype implementations. The work demonstrates practical automated verification of sophisticated representation invariants in realistic programs and highlights Marple as a first system to verify such invariants for OCaml data structures interacting with stateful libraries.

Abstract

Functional programs typically interact with stateful libraries that hide state behind typed abstractions. One particularly important class of applications are data structure implementations that rely on such libraries to provide a level of efficiency and scalability that may be otherwise difficult to achieve. However, because the specifications of the methods provided by these libraries are necessarily general and rarely specialized to the needs of any specific client, any required application-level invariants must often be expressed in terms of additional constraints on the (often) opaque state maintained by the library. In this paper, we consider the specification and verification of such representation invariants using symbolic finite automata (SFA). We show that SFAs can be used to succinctly and precisely capture fine-grained temporal and data-dependent histories of interactions between functional clients and stateful libraries. To facilitate modular and compositional reasoning, we integrate SFAs into a refinement type system to qualify stateful computations resulting from such interactions. The particular instantiation we consider, Hoare Automata Types (HATs), allows us to both specify and automatically type-check the representation invariants of a datatype, even when its implementation depends on stateful library methods that operate over hidden state. We also develop a new bidirectional type checking algorithm that implements an efficient subtyping inclusion check over HATs, enabling their translation into a form amenable for SMT-based automated verification. We present extensive experimental results on an implementation of this algorithm that demonstrates the feasibility of type-checking complex and sophisticated HAT-specified OCaml data structure implementations.