Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contrastive Learning of Preferences with a Contextual InfoNCE Loss

Timo Bertram, Johannes Fürnkranz, Martin Müller

TL;DR

Contextual preferences in drafting require comparing a single chosen action against many alternatives, which challenges standard InfoNCE formulations. The authors adapt CLIP-style contrastive learning by using a pool-card embedding with a masked, row-wise InfoNCE loss that operates on an observed context of available cards. Their Contextual InfoNCE method outperforms vanilla CLIP and triplet-based baselines on 17lands Magic: The Gathering drafting data, achieving a top-1 accuracy of 68.80% with efficient training. This approach provides a scalable, context-aware framework for contextual preference learning that can generalize to other limited-choice domains, reducing the need for extensive triplet mining.

Abstract

A common problem in contextual preference ranking is that a single preferred action is compared against several choices, thereby blowing up the complexity and skewing the preference distribution. In this work, we show how one can solve this problem via a suitable adaptation of the CLIP framework.This adaptation is not entirely straight-forward, because although the InfoNCE loss used by CLIP has achieved great success in computer vision and multi-modal domains, its batch-construction technique requires the ability to compare arbitrary items, and is not well-defined if one item has multiple positive associations in the same batch. We empirically demonstrate the utility of our adapted version of the InfoNCE loss in the domain of collectable card games, where we aim to learn an embedding space that captures the associations between single cards and whole card pools based on human selections. Such selection data only exists for restricted choices, thus generating concrete preferences of one item over a set of other items rather than a perfect fit between the card and the pool. Our results show that vanilla CLIP does not perform well due to the aforementioned intuitive issues. However, by adapting CLIP to the problem, we receive a model outperforming previous work trained with the triplet loss, while also alleviating problems associated with mining triplets.

Contrastive Learning of Preferences with a Contextual InfoNCE Loss

TL;DR

Contextual preferences in drafting require comparing a single chosen action against many alternatives, which challenges standard InfoNCE formulations. The authors adapt CLIP-style contrastive learning by using a pool-card embedding with a masked, row-wise InfoNCE loss that operates on an observed context of available cards. Their Contextual InfoNCE method outperforms vanilla CLIP and triplet-based baselines on 17lands Magic: The Gathering drafting data, achieving a top-1 accuracy of 68.80% with efficient training. This approach provides a scalable, context-aware framework for contextual preference learning that can generalize to other limited-choice domains, reducing the need for extensive triplet mining.

Abstract

A common problem in contextual preference ranking is that a single preferred action is compared against several choices, thereby blowing up the complexity and skewing the preference distribution. In this work, we show how one can solve this problem via a suitable adaptation of the CLIP framework.This adaptation is not entirely straight-forward, because although the InfoNCE loss used by CLIP has achieved great success in computer vision and multi-modal domains, its batch-construction technique requires the ability to compare arbitrary items, and is not well-defined if one item has multiple positive associations in the same batch. We empirically demonstrate the utility of our adapted version of the InfoNCE loss in the domain of collectable card games, where we aim to learn an embedding space that captures the associations between single cards and whole card pools based on human selections. Such selection data only exists for restricted choices, thus generating concrete preferences of one item over a set of other items rather than a perfect fit between the card and the pool. Our results show that vanilla CLIP does not perform well due to the aforementioned intuitive issues. However, by adapting CLIP to the problem, we receive a model outperforming previous work trained with the triplet loss, while also alleviating problems associated with mining triplets.
Paper Structure (15 sections, 5 equations, 3 figures, 1 table)

This paper contains 15 sections, 5 equations, 3 figures, 1 table.

Table of Contents

  1. Deckbuilding in Collectable Card Games
  2. Deckbuilding
  3. Limited deckbuilding
  4. Data
  5. Contextual Preference Ranking
  6. Siamese Neural Networks for Drafting
  7. InfoNCE Loss in CLIP
  8. Contextual InfoNCE for Drafting
  9. Experiments
  10. Architectures and Hyperparameters
  11. Card representation
  12. Neural Networks
  13. Results
  14. Conclusion
  15. Future Work

Figures (3)

  • Figure 1: High-level overview of the changes necessary to accommodate contextual comparisons rather than arbitrary ones.
  • Figure 2: Pseudocode for the adapted InfoNCE loss, heavily inspired by the code in CLIP radford2021learning.
  • Figure 3: Differences between the forward pass pipeline in triplet loss-based networks and CLIP-based ones. When training with the triplet loss, we use a weight-shared output layer to accommodate a Siamese network structure. When using the InfoNCE loss, we reuse the card encoder when encoding a pool, but the main network blocks and final outputs are separate.