Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plan

Marta Moscati, Muhammad Saad Saeed, Marina Zanoni, Mubashir Noman, Rohan Kumar Das, Monorama Swain, Yufang Hou, Elisabeth Andre, Khalid Mahmood Malik, Markus Schedl, Shah Nawaz

Abstract

Multimodal speaker identification systems typically assume the availability of complete and homogeneous audio-visual modalities during both training and testing. However, in real-world applications, such assumptions often do not hold. Visual information may be missing due to occlusions, camera failures, or privacy constraints, while multilingual speakers introduce additional complexity due to linguistic variability across languages. These challenges significantly affect the robustness and generalization of multimodal speaker identification systems. The POLY-SIM Grand Challenge 2026 aims to advance research in multimodal speaker identification under missing-modality and cross-lingual conditions. Specifically, the Grand Challenge encourages the development of robust methods that can effectively leverage incomplete multimodal inputs while maintaining strong performance across different languages. This report presents the design and organization of the POLY-SIM Grand Challenge 2026, including the dataset, task formulation, evaluation protocol, and baseline model. By providing a standardized benchmark and evaluation framework, the challenge aims to foster progress toward more robust and practical multimodal speaker identification systems.

POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plan

Abstract

Multimodal speaker identification systems typically assume the availability of complete and homogeneous audio-visual modalities during both training and testing. However, in real-world applications, such assumptions often do not hold. Visual information may be missing due to occlusions, camera failures, or privacy constraints, while multilingual speakers introduce additional complexity due to linguistic variability across languages. These challenges significantly affect the robustness and generalization of multimodal speaker identification systems. The POLY-SIM Grand Challenge 2026 aims to advance research in multimodal speaker identification under missing-modality and cross-lingual conditions. Specifically, the Grand Challenge encourages the development of robust methods that can effectively leverage incomplete multimodal inputs while maintaining strong performance across different languages. This report presents the design and organization of the POLY-SIM Grand Challenge 2026, including the dataset, task formulation, evaluation protocol, and baseline model. By providing a standardized benchmark and evaluation framework, the challenge aims to foster progress toward more robust and practical multimodal speaker identification systems.

Paper Structure

This paper contains 13 sections, 2 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Grand Challenge Objective
  3. Grand Challenge Description
  4. Challenge Setup
  5. Evaluation Protocol
  6. Dataset
  7. Baseline Method & Starter Kit
  8. Evaluation Metric
  9. Baseline Results
  10. Submission Template
  11. Timeline
  12. Registration Process
  13. Rules for System Development

Figures (2)

  • Figure 1: POLY-SIM: Polyglot Speaker Identification with Missing Modality. The model is trained on paired face images and audio segments in a specific language (e.g., English). At test time, the face modality is missing and the input consists of audio segments in another language (e.g., Urdu) only.
  • Figure 2: Audio-visual samples randomly selected from the MAV-Celeb nawaz2021crosssaeed2024synopsismoscati2025linking. The visual data contains different variations such as pose, lighting condition, and motion. (Left) The block shows data of speakers speaking English. (Right) The block shows data of the same speakers speaking Urdu language.