InterPol: De-anonymizing LM Arena via Interpolated Preference Learning

Abstract

Strict anonymity of model responses is a key for the reliability of voting-based leaderboards, such as LM Arena. While prior studies have attempted to compromise this assumption using simple statistical features like TF-IDF or bag-ofwords, these methods often lack the discriminative power to distinguish between stylistically similar or within-family models. To overcome these limitations and expose the severity of vulnerability, we introduce INTERPOL, a model-driven identification framework that learns to distinguish target models from others using interpolated preference data. Specifically, INTERPOL captures deep stylistic patterns that superficial statistical features miss by synthesizing hard negative samples through model interpolation and employing an adaptive curriculum learning strategy. Extensive experiments demonstrate that INTERPOL significantly outperforms existing baselines in identification accuracy. Furthermore, we quantify the real-world threat of our findings through ranking manipulation simulations on Arena battle data.

Figures (6)

• Figure 1: Overview of InterPol. (a) Hard negative synthesis via model interpolation: we construct an interpolated model to generate synthetic hard negatives, which is constructed by combining the copy model trained to mimic the target LLM and its original backbone LLM under interpolation factor $\alpha$. Together with the target response and the most similar response selected from other models, these form triplet data to train detector model. (b) Adaptive, iterative curriculum learning: depending on margin between target and non-target responses and InterPol reassigns samples to different tasks (doublet or triplet) accordingly. $\alpha$ is gradually increased, and this adaptive reassignment and training are iterated for $K$ stages, resulting in a progressively harder curriculum.
• Figure 2: Score distributions of the detector. The detector is trained to identify Gemini-Pro. Red indicates the scores of the target model. (a) Initial detector before any fine-tuning. (b) After the first triplet training stage with interpolation factor $\alpha=0.5$, where the detector begins to assign higher scores to the target model; the score range spans approximately from $-20$ to $20$. (c) After iterative curriculum training with progressively harder negatives ($\alpha=0.75$), resulting in clearer separation between target and non-target models; the score range further expands from about $-30$ to $40$.
• Figure 3: Data scalability of InterPol. Performance trend with the proposed training framework as the number of training samples increases from 250 to 1000. Accuracy shows a generally linear improvement with larger training sets across all three target LLMs. Red bars represent GPT-4o,blue bars represent Gemini-Pro, and green bars represent Claude-4.
• Figure 4: Score distributions of the detector. The detector is trained to identify GPT-4o as the target LLM. Red indicates the scores of the target model.
• Figure 5: Score distributions of the detector. The detector is trained to identify Claude4 as the target LLM. Red indicates the scores of the target model.