Greek2MathTex: A Greek Speech-to-Text Framework for LaTeX Equations Generation

TL;DR

Greek2MathTex tackles Greek-language accessibility in LaTeX by delivering an end-to-end speech-to-LaTeX system that converts spoken Greek into LaTeX equations. It combines a Greek-optimized ASR (fine-tuned XLS-R) with an LLM-based equation generator (GPT-3.5-turbo) guided by in-context learning using retrieved examples from the Gr2Tex dataset via a $k$-NN retrieval module. The model prompts are carefully selected (p1, p2, p3) and evaluated with a Levenshtein-based metric ($EL$) alongside BLEU and chrF to capture both string accuracy and linguistic quality. The work demonstrates practical accessibility gains through an open-source web app and dataset, and outlines clear directions for improving Greek-language ASR, prompting strategies, and retrieval methods.

Abstract

In the vast majority of the academic and scientific domains, LaTeX has established itself as the de facto standard for typesetting complex mathematical equations and formulae. However, LaTeX's complex syntax and code-like appearance present accessibility barriers for individuals with disabilities, as well as those unfamiliar with coding conventions. In this paper, we present a novel solution to this challenge through the development of a novel speech-to-LaTeX equations system specifically designed for the Greek language. We propose an end-to-end system that harnesses the power of Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) techniques to enable users to verbally dictate mathematical expressions and equations in natural language, which are subsequently converted into LaTeX format. We present the architecture and design principles of our system, highlighting key components such as the ASR engine, the LLM-based prompt-driven equations generation mechanism, as well as the application of a custom evaluation metric employed throughout the development process. We have made our system open source and available at https://github.com/magcil/greek-speech-to-math.

Figures (2)

• Figure 1: An overview of the proposed system's architecture.
• Figure 2: The UI when entering the application and during recording. The modal window appears when clicking on the LaTeX button.