Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework

Yuhang Jia, Yang Chen, Jinghua Zhao, Shiwan Zhao, Wenjia Zeng, Yong Chen, Yong Qin

TL;DR

This paper introduces AudioEditor, a training-free audio editing framework built on the pretrained diffusion-based TTA model that incorporates Null-text Inversion and EOT-suppression methods, enabling the model to preserve original audio features while executing accurate edits.

Abstract

Diffusion-based text-to-audio (TTA) generation has made substantial progress, leveraging latent diffusion model (LDM) to produce high-quality, diverse and instruction-relevant audios. However, beyond generation, the task of audio editing remains equally important but has received comparatively little attention. Audio editing tasks face two primary challenges: executing precise edits and preserving the unedited sections. While workflows based on LDMs have effectively addressed these challenges in the field of image processing, similar approaches have been scarcely applied to audio editing. In this paper, we introduce AudioEditor, a training-free audio editing framework built on the pretrained diffusion-based TTA model. AudioEditor incorporates Null-text Inversion and EOT-suppression methods, enabling the model to preserve original audio features while executing accurate edits. Comprehensive objective and subjective experiments validate the effectiveness of AudioEditor in delivering high-quality audio edits. Code and demo can be found at https://github.com/NKU-HLT/AudioEditor.

AudioEditor: A Training-Free Diffusion-Based Audio Editing Framework

TL;DR

This paper introduces AudioEditor, a training-free audio editing framework built on the pretrained diffusion-based TTA model that incorporates Null-text Inversion and EOT-suppression methods, enabling the model to preserve original audio features while executing accurate edits.

Abstract

Diffusion-based text-to-audio (TTA) generation has made substantial progress, leveraging latent diffusion model (LDM) to produce high-quality, diverse and instruction-relevant audios. However, beyond generation, the task of audio editing remains equally important but has received comparatively little attention. Audio editing tasks face two primary challenges: executing precise edits and preserving the unedited sections. While workflows based on LDMs have effectively addressed these challenges in the field of image processing, similar approaches have been scarcely applied to audio editing. In this paper, we introduce AudioEditor, a training-free audio editing framework built on the pretrained diffusion-based TTA model. AudioEditor incorporates Null-text Inversion and EOT-suppression methods, enabling the model to preserve original audio features while executing accurate edits. Comprehensive objective and subjective experiments validate the effectiveness of AudioEditor in delivering high-quality audio edits. Code and demo can be found at https://github.com/NKU-HLT/AudioEditor.
Paper Structure (12 sections, 6 equations, 1 figure, 5 tables)

This paper contains 12 sections, 6 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. METHOD
  3. Diffusion based Text-to-Audio Model
  4. DDIM Inversion and Null-text Optimization
  5. EOT-suppression
  6. Attention Loss
  7. EXPERIMENTS AND RESULTS
  8. Dataset
  9. Baselines and Experimental Setup
  10. Evaluation Metrics
  11. Main Results and Analysis
  12. Ablation Study

Figures (1)

  • Figure 1: The overall workflow of AudioEditor. The workflow can be divided into 5 parts: A) audio space processing, B) spectrogram space processing, C) latent space processing, D) performing DDIM Inversion and Null-text Optimization, and E) performing EOT-suppression and Attention Loss updating.