Bhav-Net: Knowledge Transfer for Cross-Lingual Antonym vs Synonym Distinction via Dual-Space Graph Transformers
Samyak S. Sanghvi
TL;DR
Bhav-Net addresses the cross-lingual antonym vs synonym distinction by introducing a dual-space graph-transformer architecture that transfers semantic understanding from multilingual encoders to language-specific networks. The model uses language-specific BERT encoders, separate synonym and antonym projection spaces, and a graph transformer to capture higher-order relationships, trained with a combination of binary cross-entropy and margin-based contrastive losses. Evaluated on eight languages, Bhav-Net achieves state-of-the-art performance in English and demonstrates meaningful cross-lingual transfer, with performance tied closely to the quality of language-specific embeddings. The work provides open-source implementations and highlights the need for high-quality multilingual resources and standardized multilingual benchmarks to advance cross-lingual semantic relation modeling.
Abstract
Antonym vs synonym distinction across multiple languages presents unique computational challenges due to the paradoxical nature of antonymous relationships words that share semantic domains while expressing opposite meanings. This work introduces Bhav-Net, a novel dual-space architecture that enables effective knowledge transfer from complex multilingual models to simpler, language-specific architectures while maintaining robust cross-lingual antonym--synonym distinction capabilities. Our approach combines language-specific BERT encoders with graph transformer networks, creating distinct semantic projections where synonymous pairs cluster in one space while antonymous pairs exhibit high similarity in a complementary space. Through comprehensive evaluation across eight languages (English, German, French, Spanish, Italian, Portuguese, Dutch, and Russian), we demonstrate that semantic relationship modeling transfers effectively across languages. The dual-encoder design achieves competitive performance against state-of-the-art baselines while providing interpretable semantic representations and effective cross-lingual generalization.