LiRA: Linguistic Robust Anchoring for Cross-lingual Large Language Models
Haolin Li, Haipeng Zhang, Mang Li, Yaohua Wang, Lijie Wen, Yu Zhang, Biqing Huang
TL;DR
LiRA introduces a theoretically grounded framework that bridges cross-lingual gaps by anchoring low-resource language representations to an English semantic space and coupling retrieval with reasoning. The ARCA module aligns multilingual and English representations via embedding and translation critics plus a policy-driven Actor--Adaptor, while LaSR fuses multilingual and English embeddings for robust ranking and reasoning. Empirical results across retrieval, semantic similarity, mathematics, and comprehension show consistent improvements, particularly for low-resource languages, and ablations validate the necessity of both ARCA and LaSR components. The authors also release LazRetrieval, a multilingual product-retrieval dataset, to spur further work in multilingual cross-lingual understanding and retrieval. These advances offer practical impact for global information access by enhancing cross-lingual robustness under few-shot and noise-rich conditions.
Abstract
As large language models (LLMs) rapidly advance, performance on high-resource languages (e.g., English, Chinese) is nearing saturation, yet remains substantially lower for low-resource languages (e.g., Urdu, Thai) due to limited training data, machine-translation noise, and unstable cross-lingual alignment. We introduce LiRA (Linguistic Robust Anchoring for Large Language Models), a training framework that robustly improves cross-lingual representations under low-resource conditions while jointly strengthening retrieval and reasoning. LiRA comprises two modules: (i) Arca (Anchored Representation Composition Architecture), which anchors low-resource languages to an English semantic space via anchor-based alignment and multi-agent collaborative encoding, preserving geometric stability in a shared embedding space; and (ii) LaSR (Language-coupled Semantic Reasoner), which adds a language-aware lightweight reasoning head with consistency regularization on top of Arca's multilingual representations, unifying the training objective to enhance cross-lingual understanding, retrieval, and reasoning robustness. We further construct and release a multilingual product retrieval dataset covering five Southeast Asian and two South Asian languages. Experiments across low-resource benchmarks (cross-lingual retrieval, semantic similarity, and reasoning) show consistent gains and robustness under few-shot and noise-amplified settings; ablations validate the contribution of both Arca and LaSR. Code will be released on GitHub and the dataset on Hugging Face.