Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech

TL;DR

The paper tackles robust attribution of synthetic speech to known generators while detecting unseen sources. It introduces SIGNAL, a hybrid framework that fuses a Graph Attention Network over generator prototypes with a KNN-based open-set module, using a query-conditioned graph and ensemble fusion $p_{ens} = \alpha p_{GNN} + (1-\alpha) p_{KNN}$, with open-set routing governed by a threshold $\tau$. Evaluations on DiffSSD and SingFake show that SIGNAL consistently improves both attribution and open-set detection, with MAMBA-based embeddings often delivering the strongest performance; the combination outperforms standalone baselines across seen and unseen generators. This work demonstrates a principled approach to combine structured graph reasoning with local instance-based inference for forensic speech analysis, offering improved reliability for origin tracing and unseen-source detection in real-world deployment.

Abstract

We propose a unified framework for not only attributing synthetic speech to its source but also for detecting speech generated by synthesizers that were not encountered during training. This requires methods that move beyond simple detection to support both detailed forensic analysis and open-set generalization. To address this, we introduce SIGNAL, a hybrid framework that combines speech foundation models (SFMs) with graph-based modeling and open-set-aware inference. Our framework integrates Graph Neural Networks (GNNs) and a k-Nearest Neighbor (KNN) classifier, allowing it to capture meaningful relationships between utterances and recognize speech that doesn`t belong to any known generator. It constructs a query-conditioned graph over generator class prototypes, enabling the GNN to reason over relationships among candidate generators, while the KNN branch supports open-set detection via confidence-based thresholding. We evaluate SIGNAL using the DiffSSD dataset, which offers a diverse mix of real speech and synthetic audio from both open-source and commercial diffusion-based TTS systems. To further assess generalization, we also test on the SingFake benchmark. Our results show that SIGNAL consistently improves performance across both tasks, with Mamba-based embeddings delivering especially strong results. To the best of our knowledge, this is the first study to unify graph-based learning and open-set detection for tracing synthetic speech back to its origin.

Figures (3)

• Figure 1: Proposed framework: SIGNAL. The model extracts representations, followed by parallel reasoning via a Graph Attention Network (GAT) and a K-Nearest Neighbors (KNN) module. The outputs are fused via an ensemble head. The final routing decision is based on a confidence threshold ($\tau = 0.5$), directing samples to either seen ($S_1 \ldots S_n$) or unseen class predictions.
• Figure 2: Subfigure a and b depict t-SNE : (a) shows the raw embedding space on the DiffSSD test set, while (b) illustrates enhanced class separation after GNN-based refinement. While subfigure c and d present confusion matrices (c) shows ID/OOD separation on DiffSSD using the GNN+KNN ensemble, and (d) shows full test set attribution performance.
• Figure 3: Threshold $\tau$ sensitivity of SIGNAL (KNN+GNN) with Mamba-B.