Add Noise, Tasks, or Layers? MaiNLP at the VarDial 2025 Shared Task on Norwegian Dialectal Slot and Intent Detection

TL;DR

The paper tackles slot and intent detection (SID) for Norwegian dialects by systematically comparing strategies to adapt pretrained models to non-standard varieties. It evaluates data-centric approaches (English xSID, MT Norwegian, NoMusic dialect data), robustness-enhancing methods (character-level noise), auxiliary NLP tasks, and Layer Swapping to assemble cross-language SID models. Key findings show noise injection consistently aids transfer, auxiliary tasks yield mixed results depending on language and task, and Layer Swapping can produce robust assemblies by combining English and dialectal experts, achieving high performance (intent accuracy about $97.6 ext extperthousand$ and slot F1 around $85.6 ext extperthousand$ on the shared task). The work demonstrates a modular and data-aware approach to cross-lingual SID, with practical implications for adapting SID systems to low-resource dialectal data. It also provides code and insights into when and how to combine diverse data sources and model components for robust dialectal NLU.

Abstract

Slot and intent detection (SID) is a classic natural language understanding task. Despite this, research has only more recently begun focusing on SID for dialectal and colloquial varieties. Many approaches for low-resource scenarios have not yet been applied to dialectal SID data, or compared to each other on the same datasets. We participate in the VarDial 2025 shared task on slot and intent detection in Norwegian varieties, and compare multiple set-ups: varying the training data (English, Norwegian, or dialectal Norwegian), injecting character-level noise, training on auxiliary tasks, and applying Layer Swapping, a technique in which layers of models fine-tuned on different datasets are assembled into a model. We find noise injection to be beneficial while the effects of auxiliary tasks are mixed. Though some experimentation was required to successfully assemble a model from layers, it worked surprisingly well; a combination of models trained on English and small amounts of dialectal data produced the most robust slot predictions. Our best models achieve 97.6% intent accuracy and 85.6% slot F1 in the shared task.

Figures (1)

• Figure 1: Overview of our approaches: pre-trained language models (PLMs) fine-tuned on English, machine-translated Norwegian data or the dialectal development set; noise injection into the Norwegian data; training on auxiliary tasks in addition to SID data (sequentially or jointly); assembling layers of models fine-tuned on different datasets.