Self-Consistent Narrative Prompts on Abductive Natural Language Inference

TL;DR

<3-5 sentence high-level summary> The paper tackles abductive natural language inference (αNLI) and identifies a gap in modeling inter-sentential coherence and model consistency. It proposes α-PACE, a self-consistent prompt tuning approach that encodes discourse connectives as continuous prompts and uses multiple narrative sequences to guide reasoning. A general self-consistent narrative framework across six narrative orders is introduced, with task-specific templates and a verbalizer to unify outputs. Through extensive experiments on ART/αNLI, α-PACE achieves state-of-the-art performance, including notable gains in few-shot and ChatGPT-adapted settings, and demonstrates interpretability via learned discourse connectives. The work suggests that leveraging narrative structure and coherence can substantially improve abductive reasoning in NLP.

Abstract

Abduction has long been seen as crucial for narrative comprehension and reasoning about everyday situations. The abductive natural language inference ($α$NLI) task has been proposed, and this narrative text-based task aims to infer the most plausible hypothesis from the candidates given two observations. However, the inter-sentential coherence and the model consistency have not been well exploited in the previous works on this task. In this work, we propose a prompt tuning model $α$-PACE, which takes self-consistency and inter-sentential coherence into consideration. Besides, we propose a general self-consistent framework that considers various narrative sequences (e.g., linear narrative and reverse chronology) for guiding the pre-trained language model in understanding the narrative context of input. We conduct extensive experiments and thorough ablation studies to illustrate the necessity and effectiveness of $α$-PACE. The performance of our method shows significant improvement against extensive competitive baselines.

Figures (11)

• Figure 1: A data example from $\alpha$NLI and its corresponding narrative sequences, including linear narrative and reverse chronology. Two sequences explain the same narrative example seamlessly by utilizing the discourse connectives (i.e., " in fact," " then," " as a result,").
• Figure 2: The general self-consistent narrative prompt framework for considering varying narrative sequences. Two observations $(O^1, O^2)$ and a pair of hypotheses $(H^1, H^2)$ are permuted as six different sequence patterns, where the corresponding task-specific self-consistent prompt pattern includes two prefix prompts $P^0, P^4$, six cloze prompts $P^1, P^2, P^3, P^5, P^6, P^7$, and the manual template "It is [MASK]." and "Overall, [MASK] is plausible." The majority voting results align to label predictions finally.
• Figure 3: Performance comparison by adopting discourse connective in different settings. All models are training with 100 training instances.
• Figure 4: Fine-Tuning, Prefix-Tuning, and Prompt Tuning Templates. Two prompt tuning-based templates perform best among all designed templates in the template searching process for these baselines. The order of observations and hypotheses following the fully connected model proposed by bhagavatula2020abductive.
• Figure 5: Performance of our method with different numbers of prefix continuous prompt tokens ($p^0,p^4$) on the test dataset using 100 training instances. The red line indicates the performance of $\alpha\text{-PACE}_\text{General \& Task Consistency}$.