Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adapter Fusion for Multilingual Text2Cypher with Linear and Learned Gating

Makbule Gulcin Ozsoy

TL;DR

This work tackles scalable multilingual Text2Cypher by training language-specific LoRA adapters for English, Spanish, and Turkish and fusing them via either uniform linear merging or a fusion MLP with dynamic gating. The fusion MLP, which uses per-input routing weights and adapter previews, outperforms static merging and recovers about 75% of the gains of joint multilingual fine-tuning while using substantially less data. It demonstrates strong cross-language balance and supports incremental language expansion by requiring only one new adapter and light retraining of the gating MLP. Overall, the approach offers a practical, data-efficient alternative to full fine-tuning for multilingual Text2Cypher with scalable deployment potential.

Abstract

Large Language Models enable users to access database using natural language interfaces using tools like Text2SQL, Text2SPARQL, and Text2Cypher, which translate user questions into structured database queries. While these systems improve database accessibility, most research focuses on English with limited multilingual support. This work investigates a scalable multilingual Text2Cypher, aiming to support new languages without re-running full fine-tuning, avoiding manual hyper-parameter tuning, and maintaining performance close to joint multilingual fine-tuning. We train language-specific LoRA adapters for English, Spanish, and Turkish and combined them via uniform linear merging or learned fusion MLP with dynamic gating. Experimental results show that the fusion MLP recovers around 75\% of the accuracy gains from joint multilingual fine-tuning while requiring only a smaller subset of the data, outperforming linear merging across all three languages. This approach enables incremental language expansion to new languages by requiring only one LoRA adapter and a lightweight MLP retraining. Learned adapter fusion offers a practical alternative to expensive joint fine-tuning, balancing performance, data efficiency, and scalability for multilingual Text2Cypher task.

Adapter Fusion for Multilingual Text2Cypher with Linear and Learned Gating

TL;DR

This work tackles scalable multilingual Text2Cypher by training language-specific LoRA adapters for English, Spanish, and Turkish and fusing them via either uniform linear merging or a fusion MLP with dynamic gating. The fusion MLP, which uses per-input routing weights and adapter previews, outperforms static merging and recovers about 75% of the gains of joint multilingual fine-tuning while using substantially less data. It demonstrates strong cross-language balance and supports incremental language expansion by requiring only one new adapter and light retraining of the gating MLP. Overall, the approach offers a practical, data-efficient alternative to full fine-tuning for multilingual Text2Cypher with scalable deployment potential.

Abstract

Large Language Models enable users to access database using natural language interfaces using tools like Text2SQL, Text2SPARQL, and Text2Cypher, which translate user questions into structured database queries. While these systems improve database accessibility, most research focuses on English with limited multilingual support. This work investigates a scalable multilingual Text2Cypher, aiming to support new languages without re-running full fine-tuning, avoiding manual hyper-parameter tuning, and maintaining performance close to joint multilingual fine-tuning. We train language-specific LoRA adapters for English, Spanish, and Turkish and combined them via uniform linear merging or learned fusion MLP with dynamic gating. Experimental results show that the fusion MLP recovers around 75\% of the accuracy gains from joint multilingual fine-tuning while requiring only a smaller subset of the data, outperforming linear merging across all three languages. This approach enables incremental language expansion to new languages by requiring only one LoRA adapter and a lightweight MLP retraining. Learned adapter fusion offers a practical alternative to expensive joint fine-tuning, balancing performance, data efficiency, and scalability for multilingual Text2Cypher task.
Paper Structure (16 sections, 7 equations, 3 figures, 5 tables)

This paper contains 16 sections, 7 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs and Non-English Content
  4. Adapter Merging and Fusion Methods
  5. Multilingual Database Query Generation
  6. Methodology
  7. Per-Language Adapters
  8. Uniform Linear Merging
  9. Fusion MLP with Dynamic Gating
  10. Experimental Results
  11. Experimental Setup
  12. Performance Comparison
  13. Training Efficiency
  14. Conclusion
  15. Fusion MLP Architecture
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Incremental language expansion in multilingual Text2Cypher. At T1, only English is supported. Spanish (T2) and Turkish (T3) are added via new LoRA adapters + MLP retraining, without touching existing adapters. All speakers receive identical Cypher output.
  • Figure 2: Three approaches for multilingual Text2Cypher. (1) Joint fine-tuning requires complete retraining per language expansion. (2) Uniform linear merging applies static weights across adapters. (3) Fusion MLP dynamically routes each input to the optimal language adapter.
  • Figure 3: Architecture of fusion MLP with dynamic gating.