Coarse-to-Fine Proposal Refinement Framework for Audio Temporal Forgery Detection and Localization

TL;DR

The paper tackles the challenge of detecting and localizing partial audio forgery by introducing CFPRF, a two-stage framework with a frame-level detection network and a proposal refinement network. It leverages difference-aware feature learning and boundary-aware feature enhancement to capture subtle manipulations and transition artifacts, followed by a two-stage training regime and Soft-NMS-based post-processing for precise localization. Across three benchmarks, CFPRF achieves state-of-the-art performance in both frame-level forgery detection and temporal forgery localization, notably excelling on challenging multi-segment scenarios. The work offers a practical, plug-and-play localization solution that extends beyond binary classification to provide robust timestamps and segment-level forgery analysis, with implications for forensic reliability and media integrity.

Abstract

Recently, a novel form of audio partial forgery has posed challenges to its forensics, requiring advanced countermeasures to detect subtle forgery manipulations within long-duration audio. However, existing countermeasures still serve a classification purpose and fail to perform meaningful analysis of the start and end timestamps of partial forgery segments. To address this challenge, we introduce a novel coarse-to-fine proposal refinement framework (CFPRF) that incorporates a frame-level detection network (FDN) and a proposal refinement network (PRN) for audio temporal forgery detection and localization. Specifically, the FDN aims to mine informative inconsistency cues between real and fake frames to obtain discriminative features that are beneficial for roughly indicating forgery regions. The PRN is responsible for predicting confidence scores and regression offsets to refine the coarse-grained proposals derived from the FDN. To learn robust discriminative features, we devise a difference-aware feature learning (DAFL) module guided by contrastive representation learning to enlarge the sensitive differences between different frames induced by minor manipulations. We further design a boundary-aware feature enhancement (BAFE) module to capture the contextual information of multiple transition boundaries and guide the interaction between boundary information and temporal features via a cross-attention mechanism. Extensive experiments show that our CFPRF achieves state-of-the-art performance on various datasets, including LAV-DF, ASVS2019PS, and HAD.

Figures (3)

• Figure 1: The schematic diagram of a novel partial audio forgery and its countermeasures: the partial forgery detection (PFD) network identifies the content authenticity at different detection resolutions, while the temporal forgery localization (TFL) network predicts proposal regions (confidence score, start timestamp, duration length) for forgery segments.
• Figure 2: The structure of the proposed coarse-to-fine proposal refinement framework (CFPRF), involving a frame-level detection network and a proposal refinement network for audio temporal forgery detection and localization.
• Figure 3: Qualitative examples of our proposed CFPRF ablation experiments. From left to right: temporal forgery localization results with different PRN components and t-SNE results on the corresponding sample with different loss functions.