Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OmniOCR: Generalist OCR for Ethnic Minority Languages

Bonan Liu, Zeyu Zhang, Bingbing Meng, Han Wang, Hanshuo Zhang, Chengping Wang, Daji Ergu, Ying Cai

TL;DR

OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge and a sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost.

Abstract

Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.

OmniOCR: Generalist OCR for Ethnic Minority Languages

TL;DR

OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge and a sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost.

Abstract

Optical character recognition (OCR) has advanced rapidly with deep learning and multimodal models, yet most methods focus on well-resourced scripts such as Latin and Chinese. Ethnic minority languages remain underexplored due to complex writing systems, scarce annotations, and diverse historical and modern forms, making generalization in low-resource or zero-shot settings challenging. To address these challenges, we present OmniOCR, a universal framework for ethnic minority scripts. OmniOCR introduces Dynamic Low-Rank Adaptation (Dynamic LoRA) to allocate model capacity across layers and scripts, enabling effective adaptation while preserving knowledge.A sparsity regularization prunes redundant updates, ensuring compact and efficient adaptation without extra inference cost. Evaluations on TibetanMNIST, Shui, ancient Yi, and Dongba show that OmniOCR outperforms zero-shot foundation models and standard post training, achieving state-of-the-art accuracy with superior parameter efficiency, and compared with the state-of-the-art baseline models, it improves accuracy by 39%-66% on these four datasets. Code: https://github.com/AIGeeksGroup/OmniOCR.
Paper Structure (15 sections, 2 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 15 sections, 2 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs in OCR
  4. OCR for Ethnic Minority Languages
  5. Datasets
  6. Method
  7. Overview
  8. Dynamic LoRA Adaptation
  9. Experiments
  10. Implementation Details
  11. Evaluation Metrics
  12. Main Results
  13. Ablation Study
  14. Limitation and Future Work
  15. Conclusion

Figures (4)

  • Figure 1: Accuracy comparison on Tibetan dataset across all models.
  • Figure 2: OmniOCR. A represents the processing procedure of the model's Vision Encoder; B represents the processing procedure of the model's Text Encoder; C respresents two distinct parameter-efficient fine-tuning methods: Dynamic-Rank Training and Fixed-Rank Training.
  • Figure 3: Demonstration of Recognition Performance for Tibetan Handwritten Digits via OmniOCR. The results show the accuracy and visual recognition effect of OmniOCR on the Tibetan handwritten digit dataset.
  • Figure 4: The performance of OmniOCR evaluated on four representative datasets—TibetanMNIST, Shui, Ancient Yi, and Dongba—highlighting its ability to generalize across heterogeneous scripts and writing systems.