Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning

Mayur Patidar, Riya Sawhney, Avinash Singh, Biswajit Chatterjee, Mausam, Indrajit Bhattacharya

TL;DR

FuSIC-KBQA tackles few-shot transfer learning for Knowledge Base Question Answering by fusing multiple source-trained retrieval models with large-language-model in-context learning. The system retrieves KB context with complementary retrievers, uses LLMs to re-rank and generate SPARQL, and applies execution-guided feedback to refine queries. Across multiple source–target pairs and limited target supervision, FuSIC-KBQA significantly outperforms adapted state-of-the-art supervised and FS-ICL KBQA approaches, and it also shows advantages in in-domain, low-data settings. The approach reduces annotation costs and demonstrates robust cross-domain transfer and in-domain applicability, with potential to benefit from open-source LLMs and expanded retriever sets in future work.

Abstract

Existing Knowledge Base Question Answering (KBQA) architectures are hungry for annotated data, which make them costly and time-consuming to deploy. We introduce the problem of few-shot transfer learning for KBQA, where the target domain offers only a few labeled examples, but a large labeled training dataset is available in a source domain. We propose a novel KBQA architecture called FuSIC-KBQA that performs KB-retrieval using multiple source-trained retrievers, re-ranks using an LLM and uses this as input for LLM few-shot in-context learning to generate logical forms. These are further refined using execution-guided feedback. Experiments over multiple source-target KBQA pairs of varying complexity show that FuSIC-KBQA significantly outperforms adaptations of SoTA KBQA models for this setting. Additional experiments show that FuSIC-KBQA also outperforms SoTA KBQA models in the in-domain setting when training data is limited.

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning

TL;DR

FuSIC-KBQA tackles few-shot transfer learning for Knowledge Base Question Answering by fusing multiple source-trained retrieval models with large-language-model in-context learning. The system retrieves KB context with complementary retrievers, uses LLMs to re-rank and generate SPARQL, and applies execution-guided feedback to refine queries. Across multiple source–target pairs and limited target supervision, FuSIC-KBQA significantly outperforms adapted state-of-the-art supervised and FS-ICL KBQA approaches, and it also shows advantages in in-domain, low-data settings. The approach reduces annotation costs and demonstrates robust cross-domain transfer and in-domain applicability, with potential to benefit from open-source LLMs and expanded retriever sets in future work.

Abstract

Existing Knowledge Base Question Answering (KBQA) architectures are hungry for annotated data, which make them costly and time-consuming to deploy. We introduce the problem of few-shot transfer learning for KBQA, where the target domain offers only a few labeled examples, but a large labeled training dataset is available in a source domain. We propose a novel KBQA architecture called FuSIC-KBQA that performs KB-retrieval using multiple source-trained retrievers, re-ranks using an LLM and uses this as input for LLM few-shot in-context learning to generate logical forms. These are further refined using execution-guided feedback. Experiments over multiple source-target KBQA pairs of varying complexity show that FuSIC-KBQA significantly outperforms adaptations of SoTA KBQA models for this setting. Additional experiments show that FuSIC-KBQA also outperforms SoTA KBQA models in the in-domain setting when training data is limited.
Paper Structure (42 sections, 1 figure, 7 tables)

This paper contains 42 sections, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem and Proposed Approach
  4. Few-shot Transfer Learning for KBQA:
  5. Proposed Approach:
  6. Retrieval Stage:
  7. Generation Stage:
  8. Experiments
  9. Experimental Set-up
  10. Datasets:
  11. LLM and Supervised Retrievers:
  12. Baselines:
  13. Results for Few-shot Transfer Setting
  14. Model Comparison
  15. Analysis of FuSIC-KBQA
  16. ...and 27 more sections

Figures (1)

  • Figure 1: Architecture of FuSIC-KBQA, which fuses supervised learning with LLM few-shot in-context learning for KBQA. Starting from the question, the Retrieval Stage retrieves KB elements using one or more supervised KB retrievers and re-ranks these using LLM prompting. The subsequent Generation Stage uses LLM prompting with the retrieved KB context and labeled exemplars to generate the logical form, which is refined again with LLM prompting, using feedback from execution over the KB. The question is an input to all the components. Blue arrows with compound lines show flow of source training data and target few shots to components.