Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneous Face Recognition Using Domain Invariant Units

Anjith George, Sebastien Marcel

TL;DR

The paper addresses Heterogeneous Face Recognition across modalities, where large domain gaps and limited paired data hinder cross-domain matching. It introduces Domain Invariant Units (DIU) trained in a teacher–student, contrastive distillation framework to adapt a subset of lower layers of a pretrained FR model, producing domain-invariant embeddings without altering the full backbone. The approach combines a cosine contrastive loss with a distillation loss from the teacher and demonstrates strong, state-of-the-art performance on multiple benchmarks (e.g., Polathermal, Tufts, SCFace) with minimal paired data. By enabling low-level feature adaptation guided by a robust RGB-trained teacher, the method enhances pretrained FR models’ robustness to cross-domain variations and is slated for public release of code and protocols.

Abstract

Heterogeneous Face Recognition (HFR) aims to expand the applicability of Face Recognition (FR) systems to challenging scenarios, enabling the matching of face images across different domains, such as matching thermal images to visible spectra. However, the development of HFR systems is challenging because of the significant domain gap between modalities and the lack of availability of large-scale paired multi-channel data. In this work, we leverage a pretrained face recognition model as a teacher network to learn domaininvariant network layers called Domain-Invariant Units (DIU) to reduce the domain gap. The proposed DIU can be trained effectively even with a limited amount of paired training data, in a contrastive distillation framework. This proposed approach has the potential to enhance pretrained models, making them more adaptable to a wider range of variations in data. We extensively evaluate our approach on multiple challenging benchmarks, demonstrating superior performance compared to state-of-the-art methods.

Heterogeneous Face Recognition Using Domain Invariant Units

TL;DR

The paper addresses Heterogeneous Face Recognition across modalities, where large domain gaps and limited paired data hinder cross-domain matching. It introduces Domain Invariant Units (DIU) trained in a teacher–student, contrastive distillation framework to adapt a subset of lower layers of a pretrained FR model, producing domain-invariant embeddings without altering the full backbone. The approach combines a cosine contrastive loss with a distillation loss from the teacher and demonstrates strong, state-of-the-art performance on multiple benchmarks (e.g., Polathermal, Tufts, SCFace) with minimal paired data. By enabling low-level feature adaptation guided by a robust RGB-trained teacher, the method enhances pretrained FR models’ robustness to cross-domain variations and is slated for public release of code and protocols.

Abstract

Heterogeneous Face Recognition (HFR) aims to expand the applicability of Face Recognition (FR) systems to challenging scenarios, enabling the matching of face images across different domains, such as matching thermal images to visible spectra. However, the development of HFR systems is challenging because of the significant domain gap between modalities and the lack of availability of large-scale paired multi-channel data. In this work, we leverage a pretrained face recognition model as a teacher network to learn domaininvariant network layers called Domain-Invariant Units (DIU) to reduce the domain gap. The proposed DIU can be trained effectively even with a limited amount of paired training data, in a contrastive distillation framework. This proposed approach has the potential to enhance pretrained models, making them more adaptable to a wider range of variations in data. We extensively evaluate our approach on multiple challenging benchmarks, demonstrating superior performance compared to state-of-the-art methods.
Paper Structure (5 sections, 4 equations, 1 figure, 6 tables)

This paper contains 5 sections, 4 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Experiments
  4. Experimental results
  5. Conclusions

Figures (1)

  • Figure 1: The proposed Domain Invariant Unit (DIU) framework. The lower layers of the student model are trained in a contrastive framework to learn invariant features, while supervision from the distillation loss prevents overfitting.