Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks

Tianyuan Huang, Zepeng Zhu, Hangdi Xing, Zirui Shao, Zhi Yu, Chaoxiong Yang, Jiaxian He, Xiaozhong Liu, Jiajun Bu

TL;DR

BrailleLLM addresses the scarcity of Braille resources and the difficulty of mixed-content translation by introducing Braille prior knowledge-based fine-tuning (BKFT) and language-specific variants (CBKFT for Chinese, EBKFT for English). It builds CBMD and EBMD alongside a syntax-tree–driven data augmentation approach to enrich Braille-domain training data, enabling effective instruction tuning on large language models. Empirical results show that BKFT improves Braille translation quality, robustness to errors, and cross-lingual transfer, outperforming baseline LLM fine-tuning in low-resource settings. The work provides open-source Braille datasets and a reusable fine-tuning framework that advances low-resource multilingual Braille processing and lays groundwork for broader Braille-domain applications.

Abstract

Braille plays a vital role in education and information accessibility for visually impaired individuals. However, Braille information processing faces challenges such as data scarcity and ambiguities in mixed-text contexts. We construct English and Chinese Braille Mixed Datasets (EBMD/CBMD) with mathematical formulas to support diverse Braille domain research, and propose a syntax tree-based augmentation method tailored for Braille data. To address the underperformance of traditional fine-tuning methods in Braille-related tasks, we investigate Braille Knowledge-Based Fine-Tuning (BKFT), which reduces the learning difficulty of Braille contextual features. BrailleLLM employs BKFT via instruction tuning to achieve unified Braille translation, formula-to-Braille conversion, and mixed-text translation. Experiments demonstrate that BKFT achieves significant performance improvements over conventional fine-tuning in Braille translation scenarios. Our open-sourced datasets and methodologies establish a foundation for low-resource multilingual Braille research.

BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks

TL;DR

BrailleLLM addresses the scarcity of Braille resources and the difficulty of mixed-content translation by introducing Braille prior knowledge-based fine-tuning (BKFT) and language-specific variants (CBKFT for Chinese, EBKFT for English). It builds CBMD and EBMD alongside a syntax-tree–driven data augmentation approach to enrich Braille-domain training data, enabling effective instruction tuning on large language models. Empirical results show that BKFT improves Braille translation quality, robustness to errors, and cross-lingual transfer, outperforming baseline LLM fine-tuning in low-resource settings. The work provides open-source Braille datasets and a reusable fine-tuning framework that advances low-resource multilingual Braille processing and lays groundwork for broader Braille-domain applications.

Abstract

Braille plays a vital role in education and information accessibility for visually impaired individuals. However, Braille information processing faces challenges such as data scarcity and ambiguities in mixed-text contexts. We construct English and Chinese Braille Mixed Datasets (EBMD/CBMD) with mathematical formulas to support diverse Braille domain research, and propose a syntax tree-based augmentation method tailored for Braille data. To address the underperformance of traditional fine-tuning methods in Braille-related tasks, we investigate Braille Knowledge-Based Fine-Tuning (BKFT), which reduces the learning difficulty of Braille contextual features. BrailleLLM employs BKFT via instruction tuning to achieve unified Braille translation, formula-to-Braille conversion, and mixed-text translation. Experiments demonstrate that BKFT achieves significant performance improvements over conventional fine-tuning in Braille translation scenarios. Our open-sourced datasets and methodologies establish a foundation for low-resource multilingual Braille research.
Paper Structure (23 sections, 1 equation, 4 figures, 10 tables)

This paper contains 23 sections, 1 equation, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Braille Information Processing
  4. Low resource translation
  5. Large Language Models
  6. Dataset
  7. Data Collection
  8. Data Annotation
  9. Braille Data Augmentation
  10. Methodology
  11. CBKFT: Fine-Tuning Based on Chinese Braille Prior Knowledge
  12. EBKFT: Fine-Tuning Based on English Braille Prior Knowledge
  13. Model Fine-Tuning Procedure
  14. Experiment
  15. Implementation Details
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Braille translation methodologies enable the conversion of plain-text resources into Braille or Braille ASCII words (the computational representation of Braille), granting visually impaired populations access to Braille materials. Furthermore, Braille-to-text translation technology enhances educational accessibility by facilitating home tutoring for blind children and empowering Braille school educators to evaluate complex Braille examination papers efficiently.
  • Figure 2: Instruction template samples.
  • Figure 3: Example of component analysis of Braille text.
  • Figure 4: Comparison of fine-tuning based on Braille prior knowledge with conventional approaches. This study constructs a mapping knowledge base between Chinese/English Braille segments and their semantically equivalent words, enabling more rational segmentation of Braille sequences. Simultaneously, Braille segment embeddings are initialized with high-frequency equivalent word embeddings, which alleviates the model's comprehension difficulty of Braille patterns during fine-tuning.