Efficient Multilingual Dialogue Processing via Translation Pipelines and Distilled Language Models

TL;DR

The paper addresses multilingual dialogue understanding for low-resource Indic languages by employing a translation-first pipeline: translating dialogues to English, applying a single distilled multitask generator, and translating results back to the source languages. The core method uses a 2.55B distilled Qwen3-4B-Instruct model with a 256k context window to perform narrative and structured summarization as well as question answering, all within a three-stage pipeline that leverages high-quality translation models. Key findings show competitive performance across nine languages, with strong QnA results in Marathi, Tamil, and Hindi, and robust narrative summarization metrics (F1 up to 0.92 and BERTScore up to 0.92). The work demonstrates that translation-based multilingual processing can achieve practical, efficient results without task-specific fine-tuning, offering a viable pathway for deploying clinical dialogue systems in low-resource languages; future work may explore direct multilingual models and further efficiency optimizations.

Abstract

This paper presents team Kl33n3x's multilingual dialogue summarization and question answering system developed for the NLPAI4Health 2025 shared task. The approach employs a three-stage pipeline: forward translation from Indic languages to English, multitask text generation using a 2.55B parameter distilled language model, and reverse translation back to source languages. By leveraging knowledge distillation techniques, this work demonstrates that compact models can achieve highly competitive performance across nine languages. The system achieved strong win rates across the competition's tasks, with particularly robust performance on Marathi (86.7% QnA), Tamil (86.7% QnA), and Hindi (80.0% QnA), demonstrating the effectiveness of translation-based approaches for low-resource language processing without task-specific fine-tuning.