Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Maximization for Long-Tailed Semi-Supervised Domain Generalization

Leo Fillioux, Omprakash Chakraborty, Quentin Gopée, Pierre Marza, Paul-Henry Cournède, Stergios Christodoulidis, Maria Vakalopoulou, Ismail Ben Ayed, Jose Dolz

TL;DR

IMaX is proposed, a simple yet effective objective based on the well-known InfoMax principle adapted to the SSDG scenario, which mitigates the class-balance bias encoded in the standard marginal entropy term of the MI, thereby better handling arbitrary class distributions.

Abstract

Semi-supervised domain generalization (SSDG) has recently emerged as an appealing alternative to tackle domain generalization when labeled data is scarce but unlabeled samples across domains are abundant. In this work, we identify an important limitation that hampers the deployment of state-of-the-art methods on more challenging but practical scenarios. In particular, state-of-the-art SSDG severely suffers in the presence of long-tailed class distributions, an arguably common situation in real-world settings. To alleviate this limitation, we propose IMaX, a simple yet effective objective based on the well-known InfoMax principle adapted to the SSDG scenario, where the Mutual Information (MI) between the learned features and latent labels is maximized, constrained by the supervision from the labeled samples. Our formulation integrates an α-entropic objective, which mitigates the class-balance bias encoded in the standard marginal entropy term of the MI, thereby better handling arbitrary class distributions. IMaX can be seamlessly plugged into recent state-of-the-art SSDG, consistently enhancing their performance, as demonstrated empirically across two different image modalities.

Information Maximization for Long-Tailed Semi-Supervised Domain Generalization

TL;DR

IMaX is proposed, a simple yet effective objective based on the well-known InfoMax principle adapted to the SSDG scenario, which mitigates the class-balance bias encoded in the standard marginal entropy term of the MI, thereby better handling arbitrary class distributions.

Abstract

Semi-supervised domain generalization (SSDG) has recently emerged as an appealing alternative to tackle domain generalization when labeled data is scarce but unlabeled samples across domains are abundant. In this work, we identify an important limitation that hampers the deployment of state-of-the-art methods on more challenging but practical scenarios. In particular, state-of-the-art SSDG severely suffers in the presence of long-tailed class distributions, an arguably common situation in real-world settings. To alleviate this limitation, we propose IMaX, a simple yet effective objective based on the well-known InfoMax principle adapted to the SSDG scenario, where the Mutual Information (MI) between the learned features and latent labels is maximized, constrained by the supervision from the labeled samples. Our formulation integrates an α-entropic objective, which mitigates the class-balance bias encoded in the standard marginal entropy term of the MI, thereby better handling arbitrary class distributions. IMaX can be seamlessly plugged into recent state-of-the-art SSDG, consistently enhancing their performance, as demonstrated empirically across two different image modalities.
Paper Structure (7 sections, 8 equations, 2 figures, 2 tables)

This paper contains 7 sections, 8 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experiments
  5. Experimental setup
  6. Results
  7. Conclusion

Figures (2)

  • Figure 1: Performance degradation under class imbalance. Impact on the accuracy of having long-tailed class distribution in the training data using the FBCSA galappaththige2024towards trainer (left). Impact of the imbalance factor $\gamma$ (see Section \ref{['subsec:experimental_setup']}) on the performance of FBCSA and our proposed approach (right).
  • Figure 2: Ablation on the $\boldsymbol\alpha$ parameter. Impact on the $\alpha$ parameter of $\mathcal{H}_{\alpha}$ on the accuracy for the ESCA (left) Retina (right) datasets under long-tailed distribution. Results are for FreeMatch on the SSL Baseline setting.