Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss
Wei He, Marco Idiart, Carolina Scarton, Aline Villavicencio
TL;DR
This work tackles the difficulty of representing idiomatic expressions by introducing an idiomaticity-aware training objective based on adaptive contrastive triplet loss and in-batch mining, trained without explicit STS supervision. By leveraging a specialized miner and relabeling strategy, the model learns to separate literal- and idiomatic-contributions of MWEs across multiple languages, achieving state-of-the-art results on SemEval-2022 Task 2B. The approach demonstrates strong cross-language generalization and provides a practical framework for improving idiom handling in downstream tasks such as translation and simplification. Overall, the proposed adaptive triplet framework offers a scalable path to robust idiomatic representation with meaningful impact on multilingual NLP systems.
Abstract
Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP). This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics.