Causal Retrieval with Semantic Consideration
Hyunseo Shin, Wonseok Hwang
TL;DR
CAWAI addresses knowledge-intensive retrieval by embedding causal cues into retrieval through a three-encoder architecture that separately encodes causes, effects, and semantics. It trains with a dual objective that maps causes to effects while preserving input semantics, formalized as $L_{total} = L_c + L_e + \beta (L_{sem,c} + L_{sem,e})$, and employs in-batch negative sampling to strengthen causal distinctions. Empirically, CAWAI outperforms strong baselines on causal retrieval and causal QA tasks, and shows strong zero-shot generalization on science-domain QA, while remaining competitive in general QA. This approach enables more accurate, causally informed retrieval for domains like biomedicine and law, improving the reliability of downstream knowledge-intensive AI systems.
Abstract
Recent advancements in large language models (LLMs) have significantly enhanced the performance of conversational AI systems. To extend their capabilities to knowledge-intensive domains such as biomedical and legal fields, where the accuracy is critical, LLMs are often combined with information retrieval (IR) systems to generate responses based on retrieved documents. However, for IR systems to effectively support such applications, they must go beyond simple semantic matching and accurately capture diverse query intents, including causal relationships. Existing IR models primarily focus on retrieving documents based on surface-level semantic similarity, overlooking deeper relational structures such as causality. To address this, we propose CAWAI, a retrieval model that is trained with dual objectives: semantic and causal relations. Our extensive experiments demonstrate that CAWAI outperforms various models on diverse causal retrieval tasks especially under large-scale retrieval settings. We also show that CAWAI exhibits strong zero-shot generalization across scientific domain QA tasks.