Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepASMR: LLM-Based Zero-Shot ASMR Speech Generation for Anyone of Any Voice

Leying Zhang, Tingxiao Zhou, Haiyang Sun, Mengxiao Bi, Yanmin Qian

TL;DR

DeepASMR targets zero-shot ASMR speech generation, addressing the gap where current TTS models struggle with ASMR's unvoiced, low-intensity textures. It introduces a two-stage framework: an LLM-based text-to-semantic encoder and a flow-matching acoustic decoder, leveraging token-level soft factorization to separate style from timbre. The authors release DeepASMR-DB, a 670-hour bilingual ASMR corpus with 35 speakers, and propose a rigorous evaluation protocol integrating objective metrics, subjective MOS, LLM-based style scoring, and unvoiced speech analysis. Experimental results show state-of-the-art naturalness and ASMR style fidelity for unseen speakers while maintaining competitive performance on normal speech, with a scalable Task Prompt Selection mechanism through a Virtual Speaker Pool. The work advances practical, personalized ASMR synthesis and sets a foundation for safer deployment and broader research into unvoiced speech synthesis and multi-style voice generation.

Abstract

While modern Text-to-Speech (TTS) systems achieve high fidelity for read-style speech, they struggle to generate Autonomous Sensory Meridian Response (ASMR), a specialized, low-intensity speech style essential for relaxation. The inherent challenges include ASMR's subtle, often unvoiced characteristics and the demand for zero-shot speaker adaptation. In this paper, we introduce DeepASMR, the first framework designed for zero-shot ASMR generation. We demonstrate that a single short snippet of a speaker's ordinary, read-style speech is sufficient to synthesize high-fidelity ASMR in their voice, eliminating the need for whispered training data from the target speaker. Methodologically, we first identify that discrete speech tokens provide a soft factorization of ASMR style from speaker timbre. Leveraging this insight, we propose a two-stage pipeline incorporating a Large Language Model (LLM) for content-style encoding and a flow-matching acoustic decoder for timbre reconstruction. Furthermore, we contribute DeepASMR-DB, a comprehensive 670-hour English-Chinese multi-speaker ASMR speech corpus, and introduce a novel evaluation protocol integrating objective metrics, human listening tests, LLM-based scoring and unvoiced speech analysis. Extensive experiments confirm that DeepASMR achieves state-of-the-art naturalness and style fidelity in ASMR generation for anyone of any voice, while maintaining competitive performance on normal speech synthesis.

DeepASMR: LLM-Based Zero-Shot ASMR Speech Generation for Anyone of Any Voice

TL;DR

DeepASMR targets zero-shot ASMR speech generation, addressing the gap where current TTS models struggle with ASMR's unvoiced, low-intensity textures. It introduces a two-stage framework: an LLM-based text-to-semantic encoder and a flow-matching acoustic decoder, leveraging token-level soft factorization to separate style from timbre. The authors release DeepASMR-DB, a 670-hour bilingual ASMR corpus with 35 speakers, and propose a rigorous evaluation protocol integrating objective metrics, subjective MOS, LLM-based style scoring, and unvoiced speech analysis. Experimental results show state-of-the-art naturalness and ASMR style fidelity for unseen speakers while maintaining competitive performance on normal speech, with a scalable Task Prompt Selection mechanism through a Virtual Speaker Pool. The work advances practical, personalized ASMR synthesis and sets a foundation for safer deployment and broader research into unvoiced speech synthesis and multi-style voice generation.

Abstract

While modern Text-to-Speech (TTS) systems achieve high fidelity for read-style speech, they struggle to generate Autonomous Sensory Meridian Response (ASMR), a specialized, low-intensity speech style essential for relaxation. The inherent challenges include ASMR's subtle, often unvoiced characteristics and the demand for zero-shot speaker adaptation. In this paper, we introduce DeepASMR, the first framework designed for zero-shot ASMR generation. We demonstrate that a single short snippet of a speaker's ordinary, read-style speech is sufficient to synthesize high-fidelity ASMR in their voice, eliminating the need for whispered training data from the target speaker. Methodologically, we first identify that discrete speech tokens provide a soft factorization of ASMR style from speaker timbre. Leveraging this insight, we propose a two-stage pipeline incorporating a Large Language Model (LLM) for content-style encoding and a flow-matching acoustic decoder for timbre reconstruction. Furthermore, we contribute DeepASMR-DB, a comprehensive 670-hour English-Chinese multi-speaker ASMR speech corpus, and introduce a novel evaluation protocol integrating objective metrics, human listening tests, LLM-based scoring and unvoiced speech analysis. Extensive experiments confirm that DeepASMR achieves state-of-the-art naturalness and style fidelity in ASMR generation for anyone of any voice, while maintaining competitive performance on normal speech synthesis.
Paper Structure (42 sections, 8 equations, 4 figures, 7 tables)

This paper contains 42 sections, 8 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM based Text-to-Speech
  4. ASMR Speech Conversion
  5. ASMR Speech Synthesis
  6. Methodology
  7. Task Formulation
  8. Framework Overview
  9. Analysis of Token-level Soft Factorization
  10. Task Prompt Selection via Virtual Speaker Pool
  11. Pool Construction
  12. Similarity-based Retrieval
  13. Training and Inference Pipeline
  14. Two-Stage Training
  15. Inference Process
  16. ...and 27 more sections

Figures (4)

  • Figure 1: DeepASMR Framework Overview
  • Figure 2: tSNE visualization of speaker embedding extracted from S3 tokenizer hidden features
  • Figure 3: Illustration of speaker pool construction and task prompt selection
  • Figure 4: A comprehensive pipeline for constructing the DeepASMR-DB dataset