EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation

TL;DR

EthioLLM addresses the resource gap for Ethiopian languages by training multilingual LLMs across Amharic, Ge'ez, Afan Oromo, Somali, Tigrinya, and English, and by introducing EthioBenchmark for broad downstream evaluation. The approach combines encoder-only and encoder-decoder architectures built on XLM-R and MT5 families, supplemented with language-adaptive fine-tuning and careful data curation across diverse scripts. Across news classification, MT, hate speech, NER, POS tagging, and sentiment analysis, EthioLLM variants achieve competitive or state-like performance, with notable zero-shot capabilities for Ge'ez and strong results on NER and hate speech. The work provides open-source models and datasets to accelerate NLP research for Ethiopian languages and sets a foundation for future expansion and cross-lingual transfer research.

Abstract

Large language models (LLMs) have gained popularity recently due to their outstanding performance in various downstream Natural Language Processing (NLP) tasks. However, low-resource languages are still lagging behind current state-of-the-art (SOTA) developments in the field of NLP due to insufficient resources to train LLMs. Ethiopian languages exhibit remarkable linguistic diversity, encompassing a wide array of scripts, and are imbued with profound religious and cultural significance. This paper introduces EthioLLM -- multilingual large language models for five Ethiopian languages (Amharic, Ge'ez, Afan Oromo, Somali, and Tigrinya) and English, and Ethiobenchmark -- a new benchmark dataset for various downstream NLP tasks. We evaluate the performance of these models across five downstream NLP tasks. We open-source our multilingual language models, new benchmark datasets for various downstream tasks, and task-specific fine-tuned language models and discuss the performance of the models. Our dataset and models are available at the https://huggingface.co/EthioNLP repository.