The categorical contours of the Chomsky-Schützenberger representation theorem

TL;DR

This work recasts context-free grammars and nondeterministic finite-state automata within a unified categorical framework, treating grammars as functors from free operads and automata as finitary ULF functors, thereby enabling CFGs and NFAs to be studied over arbitrary categories and operads. The key innovations are the operad of spliced arrows, the contour category with its adjunction to splicing, and the universal tree contour language, which together yield a generalized Chomsky–Schützenberger representation: a language is a CFL of arrows iff it is a functorial image of the intersection of a tree contour language with a regular language. The authors develop closure properties, a fibrational parsing view, and a pullback construction that jointly handle the intersection of CFLs with regular languages, and they extend the framework to generalized CFGs over arbitrary operads, including multiple and parallel CFGs. Collectively, the results provide a robust categorical foundation for formal languages, automata, and parsing, with potential implications for type systems, LR parsing, and cross-domain applications in category theory and computer science.

Abstract

We develop fibrational perspectives on context-free grammars and on nondeterministic finite-state automata over categories and operads. A generalized CFG is a functor from a free colored operad (aka multicategory) generated by a pointed finite species into an arbitrary base operad: this encompasses classical CFGs by taking the base to be a certain operad constructed from a free monoid, as an instance of a more general construction of an \emph{operad of spliced arrows} $\mathcal{W}\,\mathcal{C}$ for any category $\mathcal{C}$. A generalized NFA is a functor from an arbitrary bipointed category or pointed operad satisfying the unique lifting of factorizations and finite fiber properties: this encompasses classical word automata and tree automata without $ε$-transitions, but also automata over non-free categories and operads. We show that generalized context-free and regular languages satisfy suitable generalizations of many of the usual closure properties, and in particular we give a simple conceptual proof that context-free languages are closed under intersection with regular languages. Finally, we observe that the splicing functor $\mathcal{W} : Cat \to Oper$ admits a left adjoint $\mathcal{C}: Oper \to Cat$, which we call the \emph{contour category} construction since the arrows of $\mathcal{C}\,\mathcal{O}$ have a geometric interpretation as oriented contours of operations of $\mathcal{O}$. A direct consequence of the contour / splicing adjunction is that every pointed finite species induces a universal CFG generating a language of \emph{tree contour words.} This leads us to a generalization of the Chomsky-Schützenberger Representation Theorem, establishing that a subset of a homset $L \subseteq \mathcal{C}(A,B)$ is a CFL of arrows if and only if it is a functorial image of the intersection of a $\mathcal{C}$-chromatic tree contour language with a regular language.

Figures (7)

• Figure 1: Left: a constant of $\mathcal{W}\,{\mathcal{C}}$. Middle: an identity operation. Right: example of partial composition, plugging $g = u_0{-}u_1{-}u_2 : (C_1,D_1),(C_2,D_2) \to (A_2,B_2)$ into gap 1 of $f = w_0{-}w_1{-}w_2{-}w_3 : (A_1,B_1),(A_2,B_2),(A_3,B_3) \to (A,B)$ to obtain $f \circ_1 g = w_0{-}w_1u_0{-}u_1{-}u_2w_2{-}w_3 : (A_1,B_1),(C_1,D_1),(C_2,D_2),(A_3,B_3) \to (A,B)$.
• Figure 2: Example of a context-free grammar represented by a functor $\mathcal{F}_{}\,\mathbb{S} \to \mathcal{W}\,{\Sigma}$, where we have indicated the action of the functor on the generators as well as the induced action on a closed derivation tree.
• Figure 3: A sequent calculus for displayed free operads.
• Figure 4: Left: an NFA represented with a traditional state-diagram. Right: the bare NFA as a finitary ULF functor $p : \mathcal{F}\,{\mathbb{Q}} \to \mathcal{F}\,{\mathbb{B}_{\Sigma}}$. (We do not label the generators of $\mathcal{F}\,{\mathbb{Q}}$ or indicate composite arrows, and we use colors to indicate the images of the generators in $\mathcal{F}\,{\mathbb{B}_{\Sigma}}$.)
• Figure 5: Left: interpretation of the generating arrows of the contour category $\mathcal{C}\,{\mathcal{O}}$. Right: interpretation of equations \ref{['equation/contour1']} and \ref{['equation/contour2']}.

Theorems & Definitions (89)

• Definition 1.1
• Definition 1.2
• Definition 1.3
• Example 1.4
• Remark 1.5
• Example 1.6
• Remark 1.7
• Definition 1.8
• Proposition 1.9
• Example 1.10