MOSS Transcribe Diarize: Accurate Transcription with Speaker Diarization

TL;DR

This work presents MOSS Transcribe Diarize, a unified audio–text multimodal LLM that performs end-to-end Speaker-Attributed, Time-Stamped Transcription (SATS) in a single pass. It leverages a $128k$-token long-context window to process up to $90$ minutes of multi-speaker audio, producing transcripts with speaker labels and precise timestamps. Trained on diverse real-world data (e.g., AISHELL-4) and simulated mixtures to capture overlap and noise, the model achieves robust long-form diarization and timing without chunking. Evaluations on AISHELL-4, Podcast, and Movies show state-of-the-art cpCER and $\Delta$cp, demonstrating strong attribution consistency and practical applicability for meeting-transcription tasks. This approach reduces error propagation typical of modular pipelines and enables retrieval-friendly, timestamped transcripts at meeting scale.

Abstract

Speaker-Attributed, Time-Stamped Transcription (SATS) aims to transcribe what is said and to precisely determine the timing of each speaker, which is particularly valuable for meeting transcription. Existing SATS systems rarely adopt an end-to-end formulation and are further constrained by limited context windows, weak long-range speaker memory, and the inability to output timestamps. To address these limitations, we present MOSS Transcribe Diarize, a unified multimodal large language model that jointly performs Speaker-Attributed, Time-Stamped Transcription in an end-to-end paradigm. Trained on extensive real wild data and equipped with a 128k context window for up to 90-minute inputs, MOSS Transcribe Diarize scales well and generalizes robustly. Across comprehensive evaluations, it outperforms state-of-the-art commercial systems on multiple public and in-house benchmarks.

Figures (2)

• Figure 1: Overview of key capabilities of MOSS Transcribe Diarize
• Figure 2: Overall architecture of the MOSS Transcribe Diarize model.