PyBangla at BLP-2025 Task 2: Enhancing Bangla-to-Python Code Generation with Iterative Self-Correction and Multilingual Agents

TL;DR

This work tackles the Bangla NL2Code challenge by introducing BanglaCodeAct, an agent-based, CodeAct-inspired framework that leverages a multilingual LLM to translate Bangla prompts into Python through an iterative Thought-Code-Observation loop. Eschewing task-specific fine-tuning, the approach relies on REPL-based execution feedback to self-correct and converge on correct solutions. On the mHumanEval Bangla NL2Code benchmark, BanglaCodeAct with Qwen3-8B attains 94.0% pass@1 on development and 71.6% on the blind test, outperforming zero-shot, few-shot, and self-consistency baselines and establishing a new benchmark for Bangla-to-Python translation. The work demonstrates the potential of agent-based reasoning and multilingual LLMs for reliable code generation in low-resource languages and highlights practical avenues for future scaling and data enrichment.

Abstract

LLMs excel at code generation from English prompts, but this progress has not extended to low-resource languages. We address Bangla-to-Python code generation by introducing BanglaCodeAct, an agent-based framework that leverages multi-agent prompting and iterative self-correction. Unlike prior approaches relying on task-specific fine-tuning, BanglaCodeAct employs an open-source multilingual LLM within a Thought-Code-Observation loop, enabling dynamic generation, testing, and refinement of code from Bangla instructions. We benchmark several small-parameter open-source LLMs and evaluate their effectiveness on the mHumanEval dataset for Bangla NL2Code. Our results show that Qwen3-8B, when deployed with BanglaCodeAct, achieves the best performance, with pass@1 accuracy of 94.0\% on the development set and 71.6\% on the blind test set. These results establish a new benchmark for Bangla-to-Python translation and highlight the potential of agent-based reasoning for reliable code generation in low-resource languages. Experimental scripts are publicly available at github.com/jahidulzaid/PyBanglaCodeActAgent.

Figures (1)

• Figure 1: Thought-Code-Observation Cycle in the BanglaCodeAct Agent Framework. This diagram illustrates the iterative process of generating code, executing it, providing feedback, and refining the solution based on self-correction, facilitating cross-lingual code generation in Bangla.