Plausibility Vaccine: Injecting LLM Knowledge for Event Plausibility

TL;DR

This paper tackles semantic plausibility by injecting latent knowledge from large language models into transformer representations through parameter-efficient adapters. It trains 12 adapters for physical properties and selectional associations, using adapter fusion to compose latent semantic signals on top of ALBERT-base-v2, with data automatically generated by GPT-4o. Across two plausibility datasets, property adapters yield substantial gains over direct fine-tuning, with additional improvements from including selectional association adapters, though benefits from combining datasets are mixed. The approach demonstrates scalable, automated knowledge integration that improves plausibility reasoning and offers a path toward grounding language models to reduce hallucinations, while highlighting data quality and access limitations as future challenges.

Abstract

Despite advances in language modelling, distributional methods that build semantic representations from co-occurrences fail to discriminate between plausible and implausible events. In this work, we investigate how plausibility prediction can be improved by injecting latent knowledge prompted from large language models using parameter-efficient fine-tuning. We train 12 task adapters to learn various physical properties and association measures and perform adapter fusion to compose latent semantic knowledge from each task on top of pre-trained AlBERT embeddings. We automate auxiliary task data generation, which enables us to scale our approach and fine-tune our learned representations across two plausibility datasets. Our code is available at https://github.com/Jacob-Chmura/plausibility-vaccine.

Figures (3)

• Figure 1: Our high-level model architecture
• Figure 2: Normalized Mutual Information Between Properties and Plausibility
• Figure 3: Joint correlation between SV,OV association and plausibility on PEP-3K validation dataset.