Self-Recognition in Language Models

TL;DR

This work introduces an externally applicable security-question based test to assess self recognition in ten open- and closed-source language models without accessing internal probabilities or parameters. Using a latent variable framework, the authors map observed verdict-based accuracy to a latent self-recognition strength, and analyze effects of answer length and option count through targeted interventions. Across extensive experiments, they find no evidence for general self recognition, instead observing a global preference for the most competent answers across models and notable position-bias effects that complicate benchmarking. The findings imply that current LMs do not maintain a stable self-identity signal but instead align with broader reward-driven preferences, with implications for safety, model-to-model interactions, and evaluation design; the authors provide open-source tooling to reproduce and extend the work.

Abstract

A rapidly growing number of applications rely on a small set of closed-source language models (LMs). This dependency might introduce novel security risks if LMs develop self-recognition capabilities. Inspired by human identity verification methods, we propose a novel approach for assessing self-recognition in LMs using model-generated "security questions". Our test can be externally administered to monitor frontier models as it does not require access to internal model parameters or output probabilities. We use our test to examine self-recognition in ten of the most capable open- and closed-source LMs currently publicly available. Our extensive experiments found no empirical evidence of general or consistent self-recognition in any examined LM. Instead, our results suggest that given a set of alternatives, LMs seek to pick the "best" answer, regardless of its origin. Moreover, we find indications that preferences about which models produce the best answers are consistent across LMs. We additionally uncover novel insights on position bias considerations for LMs in multiple-choice settings.

Figures (17)

• Figure 1: Graphical Model of Factors Influencing an LM's Self-recognition Decision. LM $i$ generates question $Q^k_i$, optionally intervened upon by restricting the answer length ($I$). The (intervened) question is shown as a prompt to LMs, $i, j, m, l$, each of which generates an answer. Finally, an independent discriminator LM is shown the question $Q^k_i$, its answer, and a sample of answers by other LMs according to some permutation $\sigma$. The task is to decide which of the answers the discriminator LM generated, captured in verdict $Y^k_i$.
• Figure 2: Remapping accuracy curves for $n \in \{2, 3, 5\}$
• Figure 3: Self-recognition Accuracy for Unrestricted Answers. In panel (a), we report average self-recognition with standard error bars for $n=\{2, 3, 5\}$, each mapped to $n=2$ using our latent-variable assumption. Panel (b) shows a self-recognition "confusion" matrix for $n=2$ for all models. For example, 0.88 in the upper right indicates Claude Opus chose its answer over GPT 3.5's answer 88% of the time, whereas 0.3 in the bottom left indicates GPT 3.5 chose its answer over Claude Opus' 30% of the time. Reported metrics have standard errors < $0.03$.
• Figure 4: Factors Influencing Position Bias. In panel (a), we visualize three examples of position bias: in the top row, we see that Claude Opus prefers the last choice; in the middle row, Llama 3 8B strongly prefers the first choice; and in the bottom row, Gemini 1 Pro does not display a bias given two options, yet strongly gravitates to the last options for $n \in \{3, 5\}$. In panel (b), we plot the effect of total answer lengths for $n=2$ on the position bias for selecting the first option (top) and self-recognition accuracy (bottom) for the same models as in panel (a). We note that for the selected LMs, the combined answer length amplifies whatever position bias a model displays. Additionally, we see that self-recognition accuracy varies per model for different answer lengths.
• Figure 5: MAUVE Scores for Unrestricted Answers. Rows indicate reference model representations and columns relative differences. Dark squares indicate the model answers are close in embedding space, whereas light squares indicate they are further away.