Calibrating Factual Knowledge in Pretrained Language Models

TL;DR

The paper tackles the problem of false factual knowledge stored in pretrained language models by introducing Contrastive Knowledge Assessment (CKA) to detect incorrect facts and CaliNet, a lightweight FFN-based calibration module that updates only calibration memory slots. Through paraphrase-based calibration data and a masked-language modeling objective, CaliNet corrects false facts without altering the base model parameters, achieving strong calibration while preserving language modeling ability. Empirical results show substantial reductions in false knowledge and demonstrated generalization to open-domain QA tasks, alongside analyses of calibration memory usage and interpretability. The work advances scalable, task-agnostic knowledge calibration for PLMs and provides insights into how deeper layers store and adjust factual information.

Abstract

Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivates us to explore a fundamental question: How do we calibrate factual knowledge in PLMs without re-training from scratch? In this work, we propose a simple and lightweight method CaliNet to achieve this goal. To be specific, we first detect whether PLMs can learn the right facts via a contrastive score between right and fake facts. If not, we then use a lightweight method to add and adapt new parameters to specific factual texts. Experiments on the knowledge probing task show the calibration effectiveness and efficiency. In addition, through closed-book question answering, we find that the calibrated PLM possesses knowledge generalization ability after fine-tuning. Beyond the calibration performance, we further investigate and visualize the knowledge calibration mechanism.

Figures (6)

• Figure 1: Illustration of knowledge calibration. Knowledge stored in PLMs have factual errors, which impairs model performance on question answering or generation. Knowledge calibration aims to rectifie these wrong knowledge.
• Figure 2: CKA assesses the knowledge stored in PLMs in a contrastive manner. The probing set includes the one positive probing prompt and several negative probing prompts. For simplification, we set $\alpha=0$.
• Figure 3: Illustration of CaliNet. Calibration memory slots calibrate the erroneous knowledge stored in FFN by adjusting its predicted token distributions.
• Figure 4: Calibration results for different scales of facts. Given 5000 facts, our method can calibrate more than 60% of facts in PLMs at once.
• Figure 5: Calibration ability of concatenating CaliNet in different layers.