Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linking Faces and Voices Across Languages: Insights from the FAME 2026 Challenge

Marta Moscati, Ahmed Abdullah, Muhammad Saad Saeed, Shah Nawaz, Rohan Kumar Das, Muhammad Zaigham Zaheer, Junaid Mir, Muhammad Haroon Yousaf, Khalid Mahmood Malik, Markus Schedl

TL;DR

The paper addresses face-voice association in multilingual environments where test-time language may differ from training data. It introduces the FAME 2026 Grand Challenge for cross-modal verification under language variation and a baseline two-branch fusion network (FOP) that combines face embeddings from a CNN pretrained on a large facial recognition dataset with voice embeddings from an audio network trained on the spoken language, enhanced by orthogonal constraints. A new MAV-Celeb-based dataset with 58 English-German bilingual speakers enables unseen-language evaluation, and results show substantial improvements over the baseline (e.g., Simicch achieves EER 23.99% versus 41.57%). This work demonstrates the viability of multilingual cross-modal verification for real-world applications and provides a benchmark and starter kit to advance the field.

Abstract

Over half of the world's population is bilingual and people often communicate under multilingual scenarios. The Face-Voice Association in Multilingual Environments (FAME) 2026 Challenge, held at ICASSP 2026, focuses on developing methods for face-voice association that are effective when the language at test-time is different than the training one. This report provides a brief summary of the challenge.

Linking Faces and Voices Across Languages: Insights from the FAME 2026 Challenge

TL;DR

The paper addresses face-voice association in multilingual environments where test-time language may differ from training data. It introduces the FAME 2026 Grand Challenge for cross-modal verification under language variation and a baseline two-branch fusion network (FOP) that combines face embeddings from a CNN pretrained on a large facial recognition dataset with voice embeddings from an audio network trained on the spoken language, enhanced by orthogonal constraints. A new MAV-Celeb-based dataset with 58 English-German bilingual speakers enables unseen-language evaluation, and results show substantial improvements over the baseline (e.g., Simicch achieves EER 23.99% versus 41.57%). This work demonstrates the viability of multilingual cross-modal verification for real-world applications and provides a benchmark and starter kit to advance the field.

Abstract

Over half of the world's population is bilingual and people often communicate under multilingual scenarios. The Face-Voice Association in Multilingual Environments (FAME) 2026 Challenge, held at ICASSP 2026, focuses on developing methods for face-voice association that are effective when the language at test-time is different than the training one. This report provides a brief summary of the challenge.
Paper Structure (3 sections, 2 figures, 1 table)

This paper contains 3 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. The Grand Challenge: FAME 2026
  3. Acknowledgments

Figures (2)

  • Figure 1: The FAME 2026 Challenge overview: cross-modal verification to analyze the impact of multiple languages.
  • Figure 2: (a) Overview of participant activity across grand challenge phases, showing the temporal distribution of submissions. (b) Aggregation of team submissions, emphasizing differences in total participation and the phase allocation between Registration/Progress and Evaluation phase. (c) Evaluation phase performance is summarized by team-wise mean EER, with error bars indicating standard deviation.