DeepMTL2R: A Library for Deep Multi-task Learning to Rank

TL;DR

DeepMTL2R integrates heterogeneous relevance signals into a unified, context-aware model by leveraging the self-attention mechanism of transformer architectures, enabling effective learning across diverse and potentially conflicting objectives.

Abstract

This paper presents DeepMTL2R, an open-source deep learning framework for Multi-task Learning to Rank (MTL2R), where multiple relevance criteria must be optimized simultaneously. DeepMTL2R integrates heterogeneous relevance signals into a unified, context-aware model by leveraging the self-attention mechanism of transformer architectures, enabling effective learning across diverse and potentially conflicting objectives. The framework includes 21 state-of-the-art multi-task learning algorithms and supports multi-objective optimization to identify Pareto-optimal ranking models. By capturing complex dependencies and long-range interactions among items and labels, DeepMTL2R provides a scalable and expressive solution for modern ranking systems and facilitates controlled comparisons across MTL strategies. We demonstrate its effectiveness on a publicly available dataset, report competitive performance, and visualize the resulting trade-offs among objectives. DeepMTL2R is available at \href{https://github.com/amazon-science/DeepMTL2R}{https://github.com/amazon-science/DeepMTL2R}.

Figures (5)

• Figure 1: DeepMTL2R: Transformer-Based DNN in MTL2R
• Figure 2: Structure of the DeepMTL2R library. We support 9 popular LTR losses and 21 MTL algorithms.
• Figure 3: Training losses for 6 selected MTL algorithms.
• Figure 4: Validation NDCG@30 for 6 MTL algorithms.
• Figure 5: Evolution of $\Delta_m$ for NDCG@30 during validation for 6 MTL algorithms in 10 runs.