What is Learnt by the LEArnable Front-end (LEAF)? Adapting Per-Channel Energy Normalisation (PCEN) to Noisy Conditions

TL;DR

This paper analyzes what LEAF learns during training across keyword spotting, emotion recognition, and language identification, and finds that only the Per-Channel Energy Normalization (PCEN) layer actually adapts, while the spectral decomposition filters and smoothing remain effectively fixed. PCEN is defined as $PCEN[n,i] = \left( \frac{E[n,i]}{(M[n,i]+epsilon)^{alpha_i}} + delta_i \right)^{gamma_i} - delta_i^{gamma_i}$ with $M[n,i] = s_i E[n,i] + (1-s_i) M[n-1,i]$, highlighting its role as the dynamic range compressor. The authors then show that adapting only the PCEN layer with a small amount of noisy data can improve performance when deploying LEAF trained on clean speech in noisy environments, under both Gaussian and babble noise, with varying effectiveness by noise type. This points to a practical, low-dimensional adaptation strategy for robust front-ends and suggests that LEAF's learning is concentrated in a narrow subspace corresponding to PCEN. The work thus provides both mechanistic insight into LEAF and a scalable path to noise-robust speech processing systems.

Abstract

There is increasing interest in the use of the LEArnable Front-end (LEAF) in a variety of speech processing systems. However, there is a dearth of analyses of what is actually learnt and the relative importance of training the different components of the front-end. In this paper, we investigate this question on keyword spotting, speech-based emotion recognition and language identification tasks and find that the filters for spectral decomposition and the low pass filter used to estimate spectral energy variations exhibit no learning and the per-channel energy normalisation (PCEN) is the key component that is learnt. Following this, we explore the potential of adapting only the PCEN layer with a small amount of noisy data to enable it to learn appropriate dynamic range compression that better suits the noise conditions. This in turn enables a system trained on clean speech to work more accurately on noisy test data as demonstrated by the experimental results reported in this paper.

Figures (5)

• Figure 1: An overview of LEAF (the input speech frame contains $M$ samples).
• Figure 2: Visualising learnt information of LEAF across three tasks.
• Figure 3: Data partitions in the CREMA-D dataset for PCEN adaptation experiments.
• Figure 4: Overview of the PCEN adaptation experimental setup.
• Figure 5: Accuracy of PCEN adaptation experiments on the CREMA-D dataset for different levels of Gaussian noise (left) and babble noise (right).