Adapting Knowledge for Few-shot Table-to-Text Generation

TL;DR

The paper tackles the challenge of few-shot table-to-text generation by introducing AKG, a modular framework that leverages unlabeled domain-specific knowledge through prompt templates and a knowledge-adaptation mechanism. It decomposes the task into Generation, Knowledge-Augmentation, and Fine-Tuning modules, incorporating a Prototype-Selection Task, a knowledge-augmented language modeling objective, and a Knowledge Adapter to fuse domain knowledge with tabular cues. Using three multidomain datasets from WIKIBIO (Humans, Books, Songs), AKG demonstrates superior fluency and faithfulness versus strong baselines, aided by an information-retrieval prototype memory and unlabeled corpus prompts. The approach shows robust gains in both automatic metrics (BLEU-4, ROUGE-4, PARENT-F) and human judgments, highlighting the practical impact of modular, knowledge-aware pretraining for real-world table-to-text generation under data scarcity.

Abstract

Pretrained language models (PLMs) have made remarkable progress in table-to-text generation tasks. However, the lack of domain-specific knowledge makes it challenging to bridge the topological gap between tabular data and text, especially in real-world applications with limited resources. To mitigate the limitation of insufficient labeled data, we propose a novel framework: Adapt-Knowledge-to-Generate (AKG). The core insight of AKG is to adapt unlabeled domain-specific knowledge into the model, which brings at least three benefits: (1) it injects representation of normal table-related descriptions to bridge the topological gap between tabular data and texts; (2) it enables us to use large amounts of unlabeled domain-specific knowledge fully, which can alleviate the PLMs' inherent shortcomings of lacking domain knowledge; (3) it allows us to design various tasks to employ the domain-specific knowledge. Extensive experiments and analyses are conducted on three open-domain, few-shot natural language generation (NLG) data sets: Humans, Songs, and Books. Compared to previous state-of-the-art approaches, our model achieves superior performance in terms of both fluency and accuracy as judged by human and automatic evaluations.

Figures (5)

• Figure 1: An example of a table--text pair for few-shot table-to-text generation from the Humans data set. On the right-hand-side are a template of the key-value pair for table linearization and a description of the table. The red text indicates the content supported by the tabular data.
• Figure 2: An overview of the AKG framework. We propose a modularized pretraining strategy that targets AKG. The pretraining strategy consists of three modules: (a) Generation Module, (b) Knowledge-Augmentation Module, and (c) Fine-tuning Module. Throughout the generation module pretraining process, we divide the generation module into two tasks: a prototype-selection task and a language-modeling task. The prototype-selection task selects related prototypes to guide the generation of PLMs, and the language-modeling task employs a PLM as the backbone generator to generate fluent sentences. We pretrain these two tasks separately. We insert the Knowledge-based Adapter (KA) after the final layer of each encoder and decoder and adapt the generated prompt templates through the KA during the pretraining of the knowledge-augmentation module. All the parameters except those of the KA are frozen when pretraining the knowledge-augmentation module. The knowledge-augmentation module brings at least two benefits: (1) it enables us to use the large amounts of unlabeled domain-specific knowledge fully, which can alleviate the PLM’s inherent shortcomings of lacking domain knowledge; and (2) it allows us to design various tasks to employ the domain-specific knowledge. Finally, we fine-tune the pretrained modularized model on three data sets: Humans, Books, and Songs. Throughout the fine-tuning process, the parameters of the KA are frozen to retain the learned knowledge pretrained from the knowledge-augmentation module. The red text indicates the tabular data supporting the fact, and the blue text indicates that the (incorrect) fact conflicts with the information in the original table.
• Figure 3: Illustration of prompt generation. (a) The original tabular data, (b) illustration of the entity-detection process, and (c) the entity value mask process.
• Figure 4: Two example tables from the Human test data set, where the yielded texts from the different methods are trained with 100 training data points. The blue text denotes that the (incorrect) fact conflicts with the information in the original table. The red text indicates the fact is supported by the tabular data.
• Figure 5: Additional examples of generated results from the ablation study. Red denotes information supported by the tabular data, blue denotes information that contradicts the tabular data, and green denotes grammatical mistakes that influence fluency.