Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

J-CHAT: Japanese Large-scale Spoken Dialogue Corpus for Spoken Dialogue Language Modeling

Wataru Nakata, Kentaro Seki, Hitomi Yanaka, Yuki Saito, Shinnosuke Takamichi, Hiroshi Saruwatari

TL;DR

This work introduces J-CHAT, a large-scale open-source Japanese spoken dialogue corpus built via a language-independent, automated pipeline that collects in-the-wild web audio, filters for Japanese, extracts dialogue turns with speaker labels, and cleans noise with speech enhancement. It analyzes dataset size and phonetic diversity, demonstrating that J-CHAT covers broad acoustic regions and scales to about 69k hours across YouTube and podcast sources. The authors validate the corpus by training generative dialogue SLMs (dGSLMs) and show that multi-domain data from J-CHAT improves naturalness and meaningfulness of dialogue generation, with the full J-CHAT model outperforming domain-specific variants. The study provides detailed methodology, rigorous evaluation, and resources to enable reproducibility, highlighting the practical impact for advancing dialogue-focused speech language modeling.

Abstract

Spoken dialogue plays a crucial role in human-AI interactions, necessitating dialogue-oriented spoken language models (SLMs). To develop versatile SLMs, large-scale and diverse speech datasets are essential. Additionally, to ensure hiqh-quality speech generation, the data must be spontaneous like in-wild data and must be acoustically clean with noise removed. Despite the critical need, no open-source corpus meeting all these criteria has been available. This study addresses this gap by constructing and releasing a large-scale spoken dialogue corpus, named Japanese Corpus for Human-AI Talks (J-CHAT), which is publicly accessible. Furthermore, this paper presents a language-independent method for corpus construction and describes experiments on dialogue generation using SLMs trained on J-CHAT. Experimental results indicate that the collected data from multiple domains by our method improve the naturalness and meaningfulness of dialogue generation.

J-CHAT: Japanese Large-scale Spoken Dialogue Corpus for Spoken Dialogue Language Modeling

TL;DR

This work introduces J-CHAT, a large-scale open-source Japanese spoken dialogue corpus built via a language-independent, automated pipeline that collects in-the-wild web audio, filters for Japanese, extracts dialogue turns with speaker labels, and cleans noise with speech enhancement. It analyzes dataset size and phonetic diversity, demonstrating that J-CHAT covers broad acoustic regions and scales to about 69k hours across YouTube and podcast sources. The authors validate the corpus by training generative dialogue SLMs (dGSLMs) and show that multi-domain data from J-CHAT improves naturalness and meaningfulness of dialogue generation, with the full J-CHAT model outperforming domain-specific variants. The study provides detailed methodology, rigorous evaluation, and resources to enable reproducibility, highlighting the practical impact for advancing dialogue-focused speech language modeling.

Abstract

Spoken dialogue plays a crucial role in human-AI interactions, necessitating dialogue-oriented spoken language models (SLMs). To develop versatile SLMs, large-scale and diverse speech datasets are essential. Additionally, to ensure hiqh-quality speech generation, the data must be spontaneous like in-wild data and must be acoustically clean with noise removed. Despite the critical need, no open-source corpus meeting all these criteria has been available. This study addresses this gap by constructing and releasing a large-scale spoken dialogue corpus, named Japanese Corpus for Human-AI Talks (J-CHAT), which is publicly accessible. Furthermore, this paper presents a language-independent method for corpus construction and describes experiments on dialogue generation using SLMs trained on J-CHAT. Experimental results indicate that the collected data from multiple domains by our method improve the naturalness and meaningfulness of dialogue generation.
Paper Structure (22 sections, 6 figures, 3 tables)

This paper contains 22 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Spoken Language Models
  4. Existing Speech Corpora
  5. Corpus Construction Methodology
  6. Data Collection
  7. Data Selection
  8. Extracting Japanese Speech Data
  9. Extracting Dialogue Speech Data
  10. Data Cleansing
  11. Corpus Analysis
  12. Dataset Size
  13. Phonetic Diversity
  14. Experiments
  15. Experimental Conditions
  16. ...and 7 more sections

Figures (6)

  • Figure 1: Corpus construction methodology proposed in this work.
  • Figure 2: Distribution of HuBERT hsu2021hubert features extracted from J-CHAT (ours), STUDIES (simulated dialogue), and JNV (non-verbal expression).
  • Figure 3: Screenshot of instruction given to the participants on the naturalness MOS evaluation.
  • Figure 4: Screenshot of instruction given to the participants on the meaningfulness MOS evaluation.
  • Figure 5: P-values from the subjective evaluation on the naturalness of the dGSLM output samples
  • ...and 1 more figures