FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning

TL;DR

Chroma 1.0 tackles the challenge of real-time, personalized voice dialogue by delivering an open-source end-to-end S2S system with sub-second latency. It introduces a streaming architecture that tightly couples speech understanding and generation via an interleaved $1:2$ text-audio token schedule, powered by the Reasoner and a lightweight Backbone/Decoder/Codec Decoder pipeline. The model achieves a notable $10.96\%$ relative improvement in speaker similarity over the human baseline, with an efficient Real-Time Factor of $0.43$ and a Time-to-First-Token of $146.87$ ms, while maintaining solid reasoning and dialogue performance at only 4B parameters. Trained on synthetic speech data, Chroma demonstrates strong practical viability for personalized voice AI with open-source availability for reproducibility and extension.

Abstract

Recent end-to-end spoken dialogue systems leverage speech tokenizers and neural audio codecs to enable LLMs to operate directly on discrete speech representations. However, these models often exhibit limited speaker identity preservation, hindering personalized voice interaction. In this work, we present Chroma 1.0, the first open-source, real-time, end-to-end spoken dialogue model that achieves both low-latency interaction and high-fidelity personalized voice cloning. Chroma achieves sub-second end-to-end latency through an interleaved text-audio token schedule (1:2) that supports streaming generation, while maintaining high-quality personalized voice synthesis across multi-turn conversations. Our experimental results demonstrate that Chroma achieves a 10.96% relative improvement in speaker similarity over the human baseline, with a Real-Time Factor (RTF) of 0.43, while maintaining strong reasoning and dialogue capabilities. Our code and models are publicly available at https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma and https://huggingface.co/FlashLabs/Chroma-4B .

Figures (2)

• Figure 1: System workflow of Chroma 1.0. Chroma takes speech as input and produces speech as output, maintaining consistent speaker identity throughout the conversation.
• Figure 2: Overall architecture of Chroma 1.0. The Reasoner outputs text tokens and hidden states. These form an interleaved text–audio embedding sequence (1:2) consumed by the Backbone to generate coarse acoustic codes $c_t^{0}$ and hidden states $h_t$. The Decoder predicts the remaining RVQ levels $c_t^{1:N-1}$, and the Codec Decoder reconstructs the full codebook sequence into a continuous waveform, enabling high-fidelity S2S generation.