Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BanglaMedQA and BanglaMMedBench: Evaluating Retrieval-Augmented Generation Strategies for Bangla Biomedical Question Answering

Sadia Sultana, Saiyma Sittul Muna, Mosammat Zannatul Samarukh, Ajwad Abrar, Tareque Mohmud Chowdhury

TL;DR

The paper introduces two Bangla biomedical MCQ datasets, BanglaMedQA and BanglaMMedBench, to evaluate reasoning and retrieval in low-resource language medical QA. It systematically benchmarks several Retrieval-Augmented Generation strategies, leveraging a Bangla textbook OCR corpus and optional web grounding, with Agentic RAG delivering the best performance (89.54% accuracy) using openai/gpt-oss-120b and showing high-quality rationales. The study also investigates translation effects by evaluating a Bangla-translated MMedBench, revealing language-specific benefits and caveats for web-grounded approaches. Collectively, the work establishes a first public Bangla biomedical QA benchmark and demonstrates the promise of dynamic, route-aware RAG pipelines for reliable multilingual medical AI.

Abstract

Developing accurate biomedical Question Answering (QA) systems in low-resource languages remains a major challenge, limiting equitable access to reliable medical knowledge. This paper introduces BanglaMedQA and BanglaMMedBench, the first large-scale Bangla biomedical Multiple Choice Question (MCQ) datasets designed to evaluate reasoning and retrieval in medical artificial intelligence (AI). The study applies and benchmarks several Retrieval-Augmented Generation (RAG) strategies, including Traditional, Zero-Shot Fallback, Agentic, Iterative Feedback, and Aggregate RAG, combining textbook-based and web retrieval with generative reasoning to improve factual accuracy. A key novelty lies in integrating a Bangla medical textbook corpus through Optical Character Recognition (OCR) and implementing an Agentic RAG pipeline that dynamically selects between retrieval and reasoning strategies. Experimental results show that the Agentic RAG achieved the highest accuracy 89.54% with openai/gpt-oss-120b, outperforming other configurations and demonstrating superior rationale quality. These findings highlight the potential of RAG-based methods to enhance the reliability and accessibility of Bangla medical QA, establishing a foundation for future research in multilingual medical artificial intelligence.

BanglaMedQA and BanglaMMedBench: Evaluating Retrieval-Augmented Generation Strategies for Bangla Biomedical Question Answering

TL;DR

The paper introduces two Bangla biomedical MCQ datasets, BanglaMedQA and BanglaMMedBench, to evaluate reasoning and retrieval in low-resource language medical QA. It systematically benchmarks several Retrieval-Augmented Generation strategies, leveraging a Bangla textbook OCR corpus and optional web grounding, with Agentic RAG delivering the best performance (89.54% accuracy) using openai/gpt-oss-120b and showing high-quality rationales. The study also investigates translation effects by evaluating a Bangla-translated MMedBench, revealing language-specific benefits and caveats for web-grounded approaches. Collectively, the work establishes a first public Bangla biomedical QA benchmark and demonstrates the promise of dynamic, route-aware RAG pipelines for reliable multilingual medical AI.

Abstract

Developing accurate biomedical Question Answering (QA) systems in low-resource languages remains a major challenge, limiting equitable access to reliable medical knowledge. This paper introduces BanglaMedQA and BanglaMMedBench, the first large-scale Bangla biomedical Multiple Choice Question (MCQ) datasets designed to evaluate reasoning and retrieval in medical artificial intelligence (AI). The study applies and benchmarks several Retrieval-Augmented Generation (RAG) strategies, including Traditional, Zero-Shot Fallback, Agentic, Iterative Feedback, and Aggregate RAG, combining textbook-based and web retrieval with generative reasoning to improve factual accuracy. A key novelty lies in integrating a Bangla medical textbook corpus through Optical Character Recognition (OCR) and implementing an Agentic RAG pipeline that dynamically selects between retrieval and reasoning strategies. Experimental results show that the Agentic RAG achieved the highest accuracy 89.54% with openai/gpt-oss-120b, outperforming other configurations and demonstrating superior rationale quality. These findings highlight the potential of RAG-based methods to enhance the reliability and accessibility of Bangla medical QA, establishing a foundation for future research in multilingual medical artificial intelligence.

Paper Structure

This paper contains 29 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Medical Question Answering (QA)
  4. Low-Resource Languages in Medical NLP
  5. Retrieval-Augmented Generation (RAG)
  6. Research Gap and Motivation
  7. Proposed Methodology
  8. Datasets
  9. BanglaMedQA- Bangla Medical Corpus
  10. BanglaMMedBench: Translated Biomedical Corpus
  11. Data Preprocessing
  12. Retrieval Augmented Generation (RAG)
  13. RAG Retrieval Source: Bangla Biology Textbook
  14. Implementation of RAG Variants
  15. Comparative Evaluation of MMedBench and BanglaMMedBench
  16. ...and 14 more sections

Figures (3)

  • Figure 1: The responses of llama-3.3-70b-versatile to a question from BanglaMedQA following different strategies.
  • Figure 2: Methodological workflow of the Bangla RAG framework.
  • Figure 6: Zero-shot and web search: MMedBench vs. translated Bangla dataset.