An approach to optimize inference of the DIART speaker diarization pipeline

TL;DR

The paper addresses the need for low-latency online speaker diarization by targeting the embedding component of the DIART pipeline. It systematically applies classical inference-optimization techniques—knowledge distillation, pruning, layer fusion, quantization, and ONNX export—to the pyannote/embedding model and evaluates effects on latency and diarization error. Key findings show that knowledge distillation reduces latency but increases DER, while layer fusion and int8 quantization reduce latency with minimal DER impact; pruning offers little latency gain, and ONNX conversion can worsen latency. The results guide practical deployment by highlighting favorable trade-offs (distillation, fusion, quantization) and indicate future work in improving distillation data and understanding ONNX-related latency behavior.

Abstract

Speaker diarization answers the question "who spoke when" for an audio file. In some diarization scenarios, low latency is required for transcription. Speaker diarization with low latency is referred to as online speaker diarization. The DIART pipeline is an online speaker diarization system. It consists of a segmentation and an embedding model. The embedding model has the largest share of the overall latency. The aim of this paper is to optimize the inference latency of the DIART pipeline. Different inference optimization methods such as knowledge distilation, pruning, quantization and layer fusion are applied to the embedding model of the pipeline. It turns out that knowledge distillation optimizes the latency, but has a negative effect on the accuracy. Quantization and layer fusion also have a positive influence on the latency without worsening the accuracy. Pruning, on the other hand, does not improve latency.

Figures (3)

• Figure 1: Pyannote embedding structure
• Figure 2: Pyannote segmentation structure
• Figure 3: Pyannote embedding reduced structure