Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pairwise Comparisons without Stochastic Transitivity: Model, Theory and Applications

Sze Ming Lee, Yunxiao Chen

TL;DR

This work addresses the limitation of stochastic transitivity in pairwise comparison models by introducing a general framework that uses an approximately low-rank, skew-symmetric parameter matrix \(M\) with a nuclear-norm constraint to model win probabilities without a global ranking. By formulating a convex likelihood optimization under \(\|M\|_* \le C_n n\) and exploiting the spectral properties of skew-symmetric matrices, the authors derive minimax-rate optimal estimators that scale to sparse data. They provide rigorous theoretical results, including convergence rates and matching lower bounds, and develop a practical computation scheme based on a nonmonotone spectral-projected gradient method with singular-value thresholding. Empirical results on simulated data and real-world StarCraft II and ATP tennis datasets demonstrate improved predictive performance in intransitive settings and robustness when transitivity holds, with the approach offering a flexible alternative to traditional BT/ Thurstone models.

Abstract

Most statistical models for pairwise comparisons, including the Bradley-Terry (BT) and Thurstone models and many extensions, make a relatively strong assumption of stochastic transitivity. This assumption imposes the existence of an unobserved global ranking among all the players/teams/items and monotone constraints on the comparison probabilities implied by the global ranking. However, the stochastic transitivity assumption does not hold in many real-world scenarios of pairwise comparisons, especially games involving multiple skills or strategies. As a result, models relying on this assumption can have suboptimal predictive performance. In this paper, we propose a general family of statistical models for pairwise comparison data without a stochastic transitivity assumption, substantially extending the BT and Thurstone models. In this model, the pairwise probabilities are determined by a (approximately) low-dimensional skew-symmetric matrix. Likelihood-based estimation methods and computational algorithms are developed, which allow for sparse data with only a small proportion of observed pairs. Theoretical analysis shows that the proposed estimator achieves minimax-rate optimality, which adapts effectively to the sparsity level of the data. The spectral theory for skew-symmetric matrices plays a crucial role in the implementation and theoretical analysis. The proposed method's superiority against the BT model, along with its broad applicability across diverse scenarios, is further supported by simulations and real data analysis.

Pairwise Comparisons without Stochastic Transitivity: Model, Theory and Applications

TL;DR

This work addresses the limitation of stochastic transitivity in pairwise comparison models by introducing a general framework that uses an approximately low-rank, skew-symmetric parameter matrix with a nuclear-norm constraint to model win probabilities without a global ranking. By formulating a convex likelihood optimization under and exploiting the spectral properties of skew-symmetric matrices, the authors derive minimax-rate optimal estimators that scale to sparse data. They provide rigorous theoretical results, including convergence rates and matching lower bounds, and develop a practical computation scheme based on a nonmonotone spectral-projected gradient method with singular-value thresholding. Empirical results on simulated data and real-world StarCraft II and ATP tennis datasets demonstrate improved predictive performance in intransitive settings and robustness when transitivity holds, with the approach offering a flexible alternative to traditional BT/ Thurstone models.

Abstract

Most statistical models for pairwise comparisons, including the Bradley-Terry (BT) and Thurstone models and many extensions, make a relatively strong assumption of stochastic transitivity. This assumption imposes the existence of an unobserved global ranking among all the players/teams/items and monotone constraints on the comparison probabilities implied by the global ranking. However, the stochastic transitivity assumption does not hold in many real-world scenarios of pairwise comparisons, especially games involving multiple skills or strategies. As a result, models relying on this assumption can have suboptimal predictive performance. In this paper, we propose a general family of statistical models for pairwise comparison data without a stochastic transitivity assumption, substantially extending the BT and Thurstone models. In this model, the pairwise probabilities are determined by a (approximately) low-dimensional skew-symmetric matrix. Likelihood-based estimation methods and computational algorithms are developed, which allow for sparse data with only a small proportion of observed pairs. Theoretical analysis shows that the proposed estimator achieves minimax-rate optimality, which adapts effectively to the sparsity level of the data. The spectral theory for skew-symmetric matrices plays a crucial role in the implementation and theoretical analysis. The proposed method's superiority against the BT model, along with its broad applicability across diverse scenarios, is further supported by simulations and real data analysis.
Paper Structure (16 sections, 8 theorems, 84 equations, 1 figure, 1 table, 3 algorithms)

This paper contains 16 sections, 8 theorems, 84 equations, 1 figure, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Generalized Approximate Low-rank Model for Pairwise Comparisons
  3. Setting and Proposed Model
  4. Comparison with Related Work
  5. Theoretical Results
  6. Computation
  7. Simulation Results
  8. Real Data Examples
  9. Data Preparation and Parameter Tuning
  10. Evaluation Criteria
  11. StarCraft II Data
  12. Tennis Data
  13. Discussions
  14. Proofs
  15. Proof of Theorem \ref{['thm: nuclear norm convergence']}
  16. ...and 1 more sections

Key Result

Theorem 1

Under Assumptions assp: 2 and assp: 3, let $\hat{\Pi} = (\hat{\pi}_{ij})_{n\times n}$, where $\hat{\pi}_{ij} = g(\hat{m}_{ij}).$ Further let $\Pi^* = g(M^*)$. Then, with probability at least $1 - \kappa_1/n$, where $\kappa_1$ and $\kappa_2$ are constants that do not depend on n.

Figures (1)

  • Figure 1: Comparison of loss between the proposed method and the Bradley-Terry (BT) model across different sparsity levels (sparse, less sparse, dense). The x-axis represents the rank parameter $k$, while the y-axis shows the mean loss, computed as the average of the losses defined in \ref{['eq: loss']}. Results are shown for varying sample sizes ($n = 500, 1000, 1500 ,2000$)

Theorems & Definitions (13)

  • Theorem 1
  • Theorem 2
  • Proposition 1
  • proof
  • Theorem 3
  • proof
  • Lemma 1
  • proof
  • Lemma 2
  • Lemma 3
  • ...and 3 more