Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering

Yilin Wang, Yuchun Fan, Jiaoyang Li, Ziming Zhu, Yongyu Mu, Qiaozhi He, Tong Xiao, Jingbo Zhu

Abstract

Retrieval-augmented generation (RAG) systems have made significant progress in solving complex multi-hop question answering (QA) tasks in the English scenario. However, RAG systems inevitably face the application scenario of retrieving across multilingual corpora and queries, leaving several open challenges. The first one involves the absence of benchmarks that assess RAG systems' capabilities under the multilingual multi-hop (MM-hop) QA setting. The second centers on the overreliance on LLMs' strong semantic understanding in English, which diminishes effectiveness in multilingual scenarios. To address these challenges, we first construct multilingual multi-hop QA benchmarks by translating English-only benchmarks into five languages, and then we propose DaPT, a novel multilingual RAG framework. DaPT generates sub-question graphs in parallel for both the source-language query and its English translation counterpart, then merges them before employing a bilingual retrieval-and-answer strategy to sequentially solve sub-questions. Our experimental results demonstrate that advanced RAG systems suffer from a significant performance imbalance in multilingual scenarios. Furthermore, our proposed method consistently yields more accurate and concise answers compared to the baselines, significantly enhancing RAG performance on this task. For instance, on the most challenging MuSiQue benchmark, DaPT achieves a relative improvement of 18.3\% in average EM score over the strongest baseline.

DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering

Abstract

Retrieval-augmented generation (RAG) systems have made significant progress in solving complex multi-hop question answering (QA) tasks in the English scenario. However, RAG systems inevitably face the application scenario of retrieving across multilingual corpora and queries, leaving several open challenges. The first one involves the absence of benchmarks that assess RAG systems' capabilities under the multilingual multi-hop (MM-hop) QA setting. The second centers on the overreliance on LLMs' strong semantic understanding in English, which diminishes effectiveness in multilingual scenarios. To address these challenges, we first construct multilingual multi-hop QA benchmarks by translating English-only benchmarks into five languages, and then we propose DaPT, a novel multilingual RAG framework. DaPT generates sub-question graphs in parallel for both the source-language query and its English translation counterpart, then merges them before employing a bilingual retrieval-and-answer strategy to sequentially solve sub-questions. Our experimental results demonstrate that advanced RAG systems suffer from a significant performance imbalance in multilingual scenarios. Furthermore, our proposed method consistently yields more accurate and concise answers compared to the baselines, significantly enhancing RAG performance on this task. For instance, on the most challenging MuSiQue benchmark, DaPT achieves a relative improvement of 18.3\% in average EM score over the strongest baseline.
Paper Structure (11 sections, 4 equations, 1 figure, 2 tables)

This paper contains 11 sections, 4 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Parallel Planning
  4. Node Fusion
  5. Bilingual Retrieval and Answering
  6. Experiment
  7. Experiment Setups
  8. Main Results
  9. Ablation Study
  10. Conclusion
  11. Acknowledgements

Figures (1)

  • Figure 1: Our proposed framework for multilingual multi-hop QA named DaPT. The framework consists of three main stages: (i) Parallel Planning: Translating the source query into English and decomposing both into two parallel sub-question graphs. (ii) Node fusion: Fusing the two graphs into a single bilingual reasoning graph based on the embedding similarity of their node of sub-questions. (iii) Bilingual Retrieval and Answering: Sequentially solving each node on the fused graph by gathering evidence with a bilingual retrieval strategy.