BanglaLorica: Design and Evaluation of a Robust Watermarking Algorithm for Large Language Models in Bangla Text Generation

TL;DR

This work tackles the robustness of text watermarking for Bangla LLM-generated text under cross-lingual round-trip translation attacks. It systematically evaluates embedding-time methods KGW and Exponential Sampling (EXP) and a post-generation Waterfall approach, revealing that token-level watermarks lose detectability after RTT. To mitigate this, it introduces a layered watermarking strategy that combines embedding-time signals with a post-generation layer, achieving 40–50% post-RTT detection accuracy—roughly 3–4× more robust than single-layer methods—at the cost of controlled semantic degradation. The study demonstrates that layered, training-free watermarking provides a practical, scalable solution for low-resource languages like Bangla, guiding deployment decisions in multilingual NLP and AI governance. The findings underscore a critical robustness–quality trade-off and motivate further research into multilingual benchmarks and adaptive watermarking strategies.

Abstract

As large language models (LLMs) are increasingly deployed for text generation, watermarking has become essential for authorship attribution, intellectual property protection, and misuse detection. While existing watermarking methods perform well in high-resource languages, their robustness in low-resource languages remains underexplored. This work presents the first systematic evaluation of state-of-the-art text watermarking methods: KGW, Exponential Sampling (EXP), and Waterfall, for Bangla LLM text generation under cross-lingual round-trip translation (RTT) attacks. Under benign conditions, KGW and EXP achieve high detection accuracy (>88%) with negligible perplexity and ROUGE degradation. However, RTT causes detection accuracy to collapse below RTT causes detection accuracy to collapse to 9-13%, indicating a fundamental failure of token-level watermarking. To address this, we propose a layered watermarking strategy that combines embedding-time and post-generation watermarks. Experimental results show that layered watermarking improves post-RTT detection accuracy by 25-35%, achieving 40-50% accuracy, representing a 3$\times$ to 4$\times$ relative improvement over single-layer methods, at the cost of controlled semantic degradation. Our findings quantify the robustness-quality trade-off in multilingual watermarking and establish layered watermarking as a practical, training-free solution for low-resource languages such as Bangla. Our code and data will be made public.

Figures (8)

• Figure 1: Overview of the proposed layered watermarking framework for Bangla LLMs.
• Figure 2: High-level workflow of the watermarking, RTT attack, and detection pipeline.
• Figure 3: Example prompt and generated Bangla outputs under single-layer watermarking.
• Figure 4: Detection accuracy of KGW and EXP watermarking across different generation lengths, before and after round-trip translation (RTT).
• Figure 5: Distribution of KGW detection scores for watermarked and unwatermarked text before and after round-trip translation (RTT).