"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models

TL;DR

This work investigates how weight editing methods unexpectedly amplify model biases after edits, and introduces a novel benchmark dataset, Seesaw-CF, for measuring bias amplification of model editing methods for demographic traits such as race, geographic origin, and gender.

Abstract

Model editing has emerged as a cost-effective strategy to update knowledge stored in language models. However, model editing can have unintended consequences after edits are applied: information unrelated to the edits can also be changed, and other general behaviors of the model can be wrongly altered. In this work, we investigate how model editing methods unexpectedly amplify model biases post-edit. We introduce a novel benchmark dataset, Seesaw-CF, for measuring bias-related harms of model editing and conduct the first in-depth investigation of how different weight-editing methods impact model bias. Specifically, we focus on biases with respect to demographic attributes such as race, geographic origin, and gender, as well as qualitative flaws in long-form texts generated by edited language models. We find that edited models exhibit, to various degrees, more biased behavior as they become less confident in attributes for Asian, African, and South American subjects. Furthermore, edited models amplify sexism and xenophobia in text generations while remaining seemingly coherent and logical. Finally, editing facts about place of birth, country of citizenship, or gender have particularly negative effects on the model's knowledge about unrelated features like field of work.

Figures (4)

• Figure 1: Example of an edit that introduces various forms of bias into GPT-J's post-edit generation.
• Figure 2: Cross-subject completion results ($D_{d}$) by racial (top) and geographic (bottom) groups. Scores lower than 0 indicate that the model becomes less confident in the correct answer after editing.
• Figure 3: Breakdown of results of $D_{d}$ ($y$-axis) on editing citizenship with MEND by continent of the target country, disaggregated by racial group. Negative scores indicate decreased model confidence post-edit.
• Figure 4: Percentage of cases per demographic trait for cross-subject cloze completions where models show decreased confidence post-edit for MEMIT. Each case is a combination of a demographic group and a property. Race includes 30 cases, gender has 8, and geographic origin has 35.