Fine-tuning of Large Language Models for Constituency Parsing Using a Sequence to Sequence Approach

TL;DR

This work investigates fine-tuning large language models to perform constituency parsing for Spanish via a sequence-to-sequence translation task, using AnCora-ES data and the MiSintaxis educational tool. Four Hugging Face models (Bloom and GPT-2 variants) are trained with full and constrained input lengths, showing that GPT-2 large-bne achieves the best $F_1$ score while Bloom models offer favorable inference speed. The study demonstrates the viability of LLM-based syntactic analysis for language education and points to future integration with traditional parsing methods and corpus expansion to cover more classroom-relevant constructions.

Abstract

Recent advances in natural language processing with large neural models have opened new possibilities for syntactic analysis based on machine learning. This work explores a novel approach to phrase-structure analysis by fine-tuning large language models (LLMs) to translate an input sentence into its corresponding syntactic structure. The main objective is to extend the capabilities of MiSintaxis, a tool designed for teaching Spanish syntax. Several models from the Hugging Face repository were fine-tuned using training data generated from the AnCora-ES corpus, and their performance was evaluated using the F1 score. The results demonstrate high accuracy in phrase-structure analysis and highlight the potential of this methodology.

Figures (2)

• Figure 1: Evolution of the training loss
• Figure 2: Examples of syntactic analyses produced by the models.