Pairwise Difference Learning for Classification

TL;DR

Pairwise Difference Learning (PDL) is extended from regression to multiclass classification by formulating a binary same/different pairwise task and learning a probabilistic γ over joint features $z_{i,j}=\phi(x_i,x_j)$. Final predictions are obtained by symmetrizing γ and averaging posterior probabilities across training anchors, effectively turning multiclass problems into a single binary reduction while leveraging an ensemble-like averaging effect. A Python package (pdll) enables easy integration with Scikit-learn models, and a large-scale evaluation on 99 OpenML datasets shows that the PDL classifier achieves higher macro F1 than strong baselines, with a notable unique contribution and reduced overfitting. The work highlights how combining instance-based and model-based learning through pairwise differences yields practical improvements in diverse, small-data classification tasks and suggests directions for further uncertainty-aware extensions.

Abstract

Pairwise difference learning (PDL) has recently been introduced as a new meta-learning technique for regression. Instead of learning a mapping from instances to outcomes in the standard way, the key idea is to learn a function that takes two instances as input and predicts the difference between the respective outcomes. Given a function of this kind, predictions for a query instance are derived from every training example and then averaged. This paper extends PDL toward the task of classification and proposes a meta-learning technique for inducing a PDL classifier by solving a suitably defined (binary) classification problem on a paired version of the original training data. We analyze the performance of the PDL classifier in a large-scale empirical study and find that it outperforms state-of-the-art methods in terms of prediction performance. Last but not least, we provide an easy-to-use and publicly available implementation of PDL in a Python package.

Figures (5)

• Figure 1: Illustration of the PDL classifier.
• Figure 2: Comparing learned patterns using PDL classifiers and baseline models.
• Figure 3: Distribution of key characteristics of the 99 OpenML classification datasets (minimum, mean, maximum).
• Figure 4: Comparing average Macro F1 score of optimized baseline classifiers and PDL classifiers.
• Figure 5: Effect of the anchor set size on PDC's loss relative to the baseline.