Proposing TAGbank as a Corpus of Tree-Adjoining Grammar Derivations
Jungyeul Park
TL;DR
This work addresses the lack of large-scale corpora grounded in lexicalized grammar formalisms by proposing TAGbank, a corpus of Tree-Adjoining Grammar derivations automatically extracted from existing syntactic treebanks. It adapts the CCGbank methodology to extract TAG elementary trees (initial and auxiliary) and to reconstruct full TAG derivations via substitution and adjunction, producing derivation trees aligned to Penn Treebank annotations. Key contributions include a scalable conversion pipeline, a formal analysis of challenges in lexical anchoring and structural consistency, and a data resource designed for parsing, grammar induction, and cross-linguistic TAG research. The work lays groundwork for multilingual TAG parsing with future extensions to Chinese and Korean corpora, and for integrating semantic annotations and structure-aware neural models.
Abstract
The development of lexicalized grammars, particularly Tree-Adjoining Grammar (TAG), has significantly advanced our understanding of syntax and semantics in natural language processing (NLP). While existing syntactic resources like the Penn Treebank and Universal Dependencies offer extensive annotations for phrase-structure and dependency parsing, there is a lack of large-scale corpora grounded in lexicalized grammar formalisms. To address this gap, we introduce TAGbank, a corpus of TAG derivations automatically extracted from existing syntactic treebanks. This paper outlines a methodology for mapping phrase-structure annotations to TAG derivations, leveraging the generative power of TAG to support parsing, grammar induction, and semantic analysis. Our approach builds on the work of CCGbank, extending it to incorporate the unique structural properties of TAG, including its transparent derivation trees and its ability to capture long-distance dependencies. We also discuss the challenges involved in the extraction process, including ensuring consistency across treebank schemes and dealing with language-specific syntactic idiosyncrasies. Finally, we propose the future extension of TAGbank to include multilingual corpora, focusing on the Penn Korean and Penn Chinese Treebanks, to explore the cross-linguistic application of TAG's formalism. By providing a robust, derivation-based resource, TAGbank aims to support a wide range of computational tasks and contribute to the theoretical understanding of TAG's generative capacity.