Bridging Language Gaps with Adaptive RAG: Improving Indonesian Language Question Answering

TL;DR

This work examines question answering in a low-resource language by extending adaptive retrieval-augmented generation (RAG) to Indonesian. It builds a multi-hop Indonesian QA evaluation framework by translating HotpotQA into Indonesian and training a question-complexity classifier to guide retrieval strategy, using open-source LLMs (Gemma, Qwen) and BM25 for retrieval. Results show a reliable classifier and promising adaptive retrieval behavior, but reveal significant instability and degradation when employing multi-retrieval, especially on multi-hop tasks like HotpotQA. The study highlights both the potential and current challenges of applying Adaptive RAG in Indonesian and suggests directions for data creation and model adaptation to improve performance in low-resource settings.

Abstract

Question Answering (QA) has seen significant improvements with the advancement of machine learning models, further studies enhanced this question answering system by retrieving external information, called Retrieval-Augmented Generation (RAG) to produce more accurate and informative answers. However, these state-of-the-art-performance is predominantly in English language. To address this gap we made an effort of bridging language gaps by incorporating Adaptive RAG system to Indonesian language. Adaptive RAG system integrates a classifier whose task is to distinguish the question complexity, which in turn determines the strategy for answering the question. To overcome the limited availability of Indonesian language dataset, our study employs machine translation as data augmentation approach. Experiments show reliable question complexity classifier; however, we observed significant inconsistencies in multi-retrieval answering strategy which negatively impacted the overall evaluation when this strategy was applied. These findings highlight both the promise and challenges of question answering in low-resource language suggesting directions for future improvement.

Figures (7)

• Figure 1: Non Retrieval Method. Formatted prompt for non-retrieval method
• Figure 2: Single Retrieval Method. Formatted prompt for single-retrieval method
• Figure 3: Multi Retrieval Method. Formatted prompt for multi-retrieval method
• Figure 4: Indonesian Language Adaptive RAG system.
• Figure 5: Example of imperfect translation in the HotpotQA dataset.