LingYi: Medical Conversational Question Answering System based on Multi-modal Knowledge Graphs

TL;DR

The paper tackles scalable medical conversational QA by leveraging a Chinese multi-modal knowledge graph (CM3KG) and a large medical dialogue dataset (CMCQA) to support knowledge-grounded, end-to-end conversations. It introduces LingYi, a pipeline system that integrates entity disambiguation, central records memory, dynamic symptom selection, entity knowledge reasoning, and generating modules, along with specialized components for triage, image-text drug recommendations, and medical record generation. The authors report state-of-the-art performance on medical entity disambiguation and competitive results for generation tasks, with open-source code and data to promote reproducibility. The approach promises to reduce clinical workload and improve access to medical guidance, while acknowledging safety, bias, and privacy considerations and outlining future directions such as federated learning for privacy-preserving personalization.

Abstract

The medical conversational system can relieve the burden of doctors and improve the efficiency of healthcare, especially during the pandemic. This paper presents a medical conversational question answering (CQA) system based on the multi-modal knowledge graph, namely "LingYi", which is designed as a pipeline framework to maintain high flexibility. Our system utilizes automated medical procedures including medical triage, consultation, image-text drug recommendation and record. To conduct knowledge-grounded dialogues with patients, we first construct a Chinese Medical Multi-Modal Knowledge Graph (CM3KG) and collect a large-scale Chinese Medical CQA (CMCQA) dataset. Compared with the other existing medical question-answering systems, our system adopts several state-of-the-art technologies including medical entity disambiguation and medical dialogue generation, which is more friendly to provide medical services to patients. In addition, we have open-sourced our codes which contain back-end models and front-end web pages at https://github.com/WENGSYX/LingYi. The datasets including CM3KG at https://github.com/WENGSYX/CM3KG and CMCQA at https://github.com/WENGSYX/CMCQA are also released to further promote future research.

Figures (4)

• Figure 1: Main processes of the LingYi system.
• Figure 2: Overview of the medical conversational QA system of LingYi.
• Figure 3: Contrastive pre-training in medical entity disambiguation.
• Figure 4: Snapshot of the proposed LingYi system.