Exploring the Feasibility of Multilingual Grammatical Error Correction with a Single LLM up to 9B parameters: A Comparative Study of 17 Models

TL;DR

The study investigates whether a single moderate-sized LLM (≤9B parameters) can perform multilingual grammatical error correction across English, German, Italian, and Swedish. It benchmarks 17 models using three prompts and the MultiGED-derived evaluation framework, combining LanguageTool-based grammaticality with semantic and syntactic similarity and preservation metrics. Gemma 9B emerges as the top overall performer, with Gemma 2B, EuroLLM variants, OpenChat, and Llama 3.1 also delivering strong multilingual results; prompt P3 consistently yields the best corrections while preserving input when correct. The findings demonstrate that appropriately trained, smaller LLMs can rival larger models for multilingual GEC, offering practical paths for deployment on consumer hardware and highlighting the value of prompt design and fine-tuning in reducing errors and language drift.

Abstract

Recent language models can successfully solve various language-related tasks, and many understand inputs stated in different languages. In this paper, we explore the performance of 17 popular models used to correct grammatical issues in texts stated in English, German, Italian, and Swedish when using a single model to correct texts in all those languages. We analyze the outputs generated by these models, focusing on decreasing the number of grammatical errors while keeping the changes small. The conclusions drawn help us understand what problems occur among those models and which models can be recommended for multilingual grammatical error correction tasks. We list six models that improve grammatical correctness in all four languages and show that Gemma 9B is currently the best performing one for the languages considered.