Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BanglaForge: LLM Collaboration with Self-Refinement for Bangla Code Generation

Mahir Labib Dihan, Sadif Ahmed, Md Nafiu Rahman

TL;DR

BanglaForge tackles Bangla-to-code generation in low-resource settings by integrating retrieval-augmented prompting, bilingual translation with a controlled glossary, and a dual-model coder–reviewer framework guided by iterative self-refinement. It grounds Bangla prompts with English context and execution feedback, using TF-IDF-based bilingual retrieval and a multi-stage generation-refinement loop. On the BLP-2025 Bangla Code Generation benchmark, BanglaForge achieves 84% Pass@1, demonstrating robust performance gains from grounding, translation, and refinement. The approach highlights practical pathways for improving code generation in underrepresented languages and suggests avenues for richer data curation, semantic retrieval, and reinforcement-driven refinement.

Abstract

Bangla is a low-resource language for code generation, lacking large-scale annotated datasets and tools to transform natural language specifications into executable programs. This makes Bangla-to-code generation a challenging task requiring innovative solutions. To address this, we introduce BanglaForge, a novel framework for generating code from Bangla function descriptions. BanglaForge leverages a retrieval-augmented dual-model collaboration paradigm with self-refinement, combining in-context learning, llm-based translation, systematic prompt engineering, and iterative self-refinement based on execution feedback, where a coder generates initial solutions and a reviewer enhances them for robustness. On the BLP-2025 Bangla Code Generation benchmark, BanglaForge achieves a competitive Pass@1 accuracy of 84.00%, demonstrating the effectiveness of retrieval, model collaboration, and self-refinement for low-resource Bangla code generation.

BanglaForge: LLM Collaboration with Self-Refinement for Bangla Code Generation

TL;DR

BanglaForge tackles Bangla-to-code generation in low-resource settings by integrating retrieval-augmented prompting, bilingual translation with a controlled glossary, and a dual-model coder–reviewer framework guided by iterative self-refinement. It grounds Bangla prompts with English context and execution feedback, using TF-IDF-based bilingual retrieval and a multi-stage generation-refinement loop. On the BLP-2025 Bangla Code Generation benchmark, BanglaForge achieves 84% Pass@1, demonstrating robust performance gains from grounding, translation, and refinement. The approach highlights practical pathways for improving code generation in underrepresented languages and suggests avenues for richer data curation, semantic retrieval, and reinforcement-driven refinement.

Abstract

Bangla is a low-resource language for code generation, lacking large-scale annotated datasets and tools to transform natural language specifications into executable programs. This makes Bangla-to-code generation a challenging task requiring innovative solutions. To address this, we introduce BanglaForge, a novel framework for generating code from Bangla function descriptions. BanglaForge leverages a retrieval-augmented dual-model collaboration paradigm with self-refinement, combining in-context learning, llm-based translation, systematic prompt engineering, and iterative self-refinement based on execution feedback, where a coder generates initial solutions and a reviewer enhances them for robustness. On the BLP-2025 Bangla Code Generation benchmark, BanglaForge achieves a competitive Pass@1 accuracy of 84.00%, demonstrating the effectiveness of retrieval, model collaboration, and self-refinement for low-resource Bangla code generation.
Paper Structure (38 sections, 10 figures, 11 tables)

This paper contains 38 sections, 10 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Dataset
  4. Methodology
  5. Problem Formulation and Input Representation
  6. Retrieval-Augmented Example Selection.
  7. Stage 1: Code Generation by Coder LLM.
  8. Stage 2: Code Review and Refinement by Reviewer LLM.
  9. Iterative Self-Refinement Protocol.
  10. Experiment
  11. Evaluation Metrics
  12. Models
  13. Results
  14. Conclusion
  15. Limitations
  16. ...and 23 more sections

Figures (10)

  • Figure 1: Workflow of the proposed BanglaForge framework. A Bangla instruction ($P_b$) is translated into English ($P_e$) and, together with unit tests, used to retrieve top-$k$ bilingual examples. The Coder LLM then generates Python code and additional test cases. The Reviewer LLM validates, refines, and re-prompts upon errors until all tests (original and generated) are passed, yielding the final code.
  • Figure 2: Example data point
  • Figure 3: System prompt for the coder model.
  • Figure 4: Main prompt template for the coder, which includes few-shot examples and the current task.
  • Figure 5: Template for providing feedback to the coder model after a failed execution attempt. This is appended to the main prompt during the self-refinement loop.
  • ...and 5 more figures