Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The TMU System for the XACLE Challenge: Training Large Audio Language Models with CLAP Pseudo-Labels

Ayuto Tsutsumi, Kohei Tanaka, Sayaka Shiota

TL;DR

This work targets predicting semantic alignment scores for text-to-audio pairs in the XACLE challenge using a large audio language model (LALM) that fuses BEATs-based audio encoding with a Qwen2.5-0.5B-Instruct LLM. The authors introduce a three-stage training pipeline—AAC pretraining, CLAP pseudo-label pretraining, and XACLE fine-tuning—to learn robust alignment scoring, and demonstrate that CLAP pseudo-label pretraining is the primary driver of performance, capable of approaching or surpassing teacher models. On the XACLE validation set, the full pipeline achieves an SRCC of 0.674, with the pseudo-label stage reaching 0.598 and laterfinetuning providing further gains; the test-set ensemble reaches 0.632 SRCC, significantly beating the baseline. The results suggest that LALM-based approaches offer architectural advantages for alignment scoring in audio-text tasks and validate the utility of pseudo-label-based weak supervision in data-scarce regimes.

Abstract

In this paper, we propose a submission to the x-to-audio alignment (XACLE) challenge. The goal is to predict semantic alignment of a given general audio and text pair. The proposed system is based on a large audio language model (LALM) architecture. We employ a three-stage training pipeline: automated audio captioning pretraining, pretraining with CLAP pseudo-labels, and fine-tuning on the XACLE dataset. Our experiments show that pretraining with CLAP pseudo-labels is the primary performance driver. On the XACLE test set, our system reaches an SRCC of 0.632, significantly outperforming the baseline system (0.334) and securing third place in the challenge team ranking. Code and models can be found at https://github.com/shiotalab-tmu/tmu-xacle2026

The TMU System for the XACLE Challenge: Training Large Audio Language Models with CLAP Pseudo-Labels

TL;DR

This work targets predicting semantic alignment scores for text-to-audio pairs in the XACLE challenge using a large audio language model (LALM) that fuses BEATs-based audio encoding with a Qwen2.5-0.5B-Instruct LLM. The authors introduce a three-stage training pipeline—AAC pretraining, CLAP pseudo-label pretraining, and XACLE fine-tuning—to learn robust alignment scoring, and demonstrate that CLAP pseudo-label pretraining is the primary driver of performance, capable of approaching or surpassing teacher models. On the XACLE validation set, the full pipeline achieves an SRCC of 0.674, with the pseudo-label stage reaching 0.598 and laterfinetuning providing further gains; the test-set ensemble reaches 0.632 SRCC, significantly beating the baseline. The results suggest that LALM-based approaches offer architectural advantages for alignment scoring in audio-text tasks and validate the utility of pseudo-label-based weak supervision in data-scarce regimes.

Abstract

In this paper, we propose a submission to the x-to-audio alignment (XACLE) challenge. The goal is to predict semantic alignment of a given general audio and text pair. The proposed system is based on a large audio language model (LALM) architecture. We employ a three-stage training pipeline: automated audio captioning pretraining, pretraining with CLAP pseudo-labels, and fine-tuning on the XACLE dataset. Our experiments show that pretraining with CLAP pseudo-labels is the primary performance driver. On the XACLE test set, our system reaches an SRCC of 0.632, significantly outperforming the baseline system (0.334) and securing third place in the challenge team ranking. Code and models can be found at https://github.com/shiotalab-tmu/tmu-xacle2026
Paper Structure (6 sections, 2 figures, 2 tables)

This paper contains 6 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System
  3. Model Architecture
  4. Training Pipeline
  5. Experiments and Results
  6. Conclusion

Figures (2)

  • Figure 1: Overview of the proposed system. Frozen BEATs features are projected into the LLM, and the alignment score is regressed from the Score Token.
  • Figure 2: Three-stage training pipeline. We first pretrain the projection and LLM for AAC, then introduce a score head and pretrain with CLAP pseudo-labels, and finally fine-tune on XACLE human alignment scores.