Metric Learning from Limited Pairwise Preference Comparisons

TL;DR

This work investigates learning a shared Mahalanobis distance from limited pairwise preference data under the ideal point model. It proves an impossibility result for generic high-dimensional data with $m=o(d)$ per user, then presents a divide-and-conquer framework that leverages low-rank subspace structure (subspace-clusterable data) to recover the metric by first learning subspace metrics and then stitching them together, with exact and approximate recovery guarantees. The approach is validated empirically on synthetic data, showing robust metric recovery under binary/noisy responses and varying numbers of subspaces, users, and comparisons. The results highlight a practical pathway for aligning high-dimensional foundation-model representations with human preferences by exploiting low-dimensional structure to reduce query costs.

Abstract

We study metric learning from preference comparisons under the ideal point model, in which a user prefers an item over another if it is closer to their latent ideal item. These items are embedded into $\mathbb{R}^d$ equipped with an unknown Mahalanobis distance shared across users. While recent work shows that it is possible to simultaneously recover the metric and ideal items given $\mathcal{O}(d)$ pairwise comparisons per user, in practice we often have a limited budget of $o(d)$ comparisons. We study whether the metric can still be recovered, even though it is known that learning individual ideal items is now no longer possible. We show that in general, $o(d)$ comparisons reveal no information about the metric, even with infinitely many users. However, when comparisons are made over items that exhibit low-dimensional structure, each user can contribute to learning the metric restricted to a low-dimensional subspace so that the metric can be jointly identified. We present a divide-and-conquer approach that achieves this, and provide theoretical recovery guarantees and empirical validation.

Figures (8)

• Figure 1.1: In our divide-and-conquer approach, users help us recover the metric $Q_\lambda$ restricted to subspaces $V_\lambda$. We stitch these together to recover the metric $M$ on $\mathbb{R}^d$. The ellipses visualize the low-dimensional unit spheres, which are "slices" of the full metric.
• Figure 3.1: Points in $\mathbb{R}^2$ with generic pairwise relations.
• Figure 5.1: (a) shows the average relative errors for varying numbers of users per subspace and preference comparisons per user, where items lie in a union of $80$$1$-dimensional subspaces of $\mathbb{R}^{10}$. (b) shows the average relative errors given increasing numbers of $1$-dimensional subspaces to reconstruct $\hat{M}$; for each subspace, $60$ users each provides $4$ preference comparisons. The dotted red curve illustrates the dimension-counting argument in Remark \ref{['rmk:min_subspaces']}. (c) shows the average relative errors for varying subspace noise levels, where items lie approximately in a union of $80$$1$-dimensional subspaces of $\mathbb{R}^{10}$; each user provides $8$ preference comparisons. The error bars in (a) and (c) represent one standard deviation from the mean.
• Figure 5.2: shows the results obtained from the three experiments with a misspecified response model. The learner is agnostic to the response noise level used to generate the data ($\beta = 4$ and $\beta = \infty$) and assumes $\beta = 1$ when recovering subspace metrics before stitching them together.
• Figure C.1: (a) Illustration of Example \ref{['example:not-generic-pairwise']}. The set of four points is in general linear position, but does not have generic pairwise relations. (b) A set of four points that has generic pairwise relations; it must also be in general linear position.

Theorems & Definitions (45)

• Definition 2.1
• Definition 3.0
• Theorem 3.1
• Definition 4.1
• Definition 4.2
• Definition 4.3
• Definition 4.4
• Lemma 4.4
• Definition 4.5
• Proposition 4.5