Gradient-based Optimisation of Modulation Effects

TL;DR

The paper addresses the challenge of faithfully emulating analog modulation units (phaser, flanger, chorus) with differentiable DSP while enabling zero-latency, time-domain deployment. It proposes a frame-based, time–frequency training framework that learns a frame-dependent frequency response $h_m$, driven by a learnable LFO via a LUT+MLP, and then runs a time-domain model at inference. Key contributions include a gradient-based analysis of delay learning and all-pass pole estimation, a DDSP-based architecture that achieves zero-latency inference, and perceptual evaluations showing close similarity to analog references across several pedals (e.g., BF-2, SV-1, Small Stone), with identified limitations for long delays with feedback. This work demonstrates practical, real-time virtual-analog modulation modelling with robust perceptual fidelity and highlights areas for future improvement in long-delay/feedback regimes. It advances real-time audio processing by integrating differentiable priors with time-domain execution for modulation effects.

Abstract

Modulation effects such as phasers, flangers and chorus effects are heavily used in conjunction with the electric guitar. Machine learning based emulation of analog modulation units has been investigated in recent years, but most methods have either been limited to one class of effect or suffer from a high computational cost or latency compared to canonical digital implementations. Here, we build on previous work and present a framework for modelling flanger, chorus and phaser effects based on differentiable digital signal processing. The model is trained in the time-frequency domain, but at inference operates in the time-domain, requiring zero latency. We investigate the challenges associated with gradient-based optimisation of such effects, and show that low-frequency weighting of loss functions avoids convergence to local minima when learning delay times. We show that when trained against analog effects units, sound output from the model is in some cases perceptually indistinguishable from the reference, but challenges still remain for effects with long delay times and feedback.

Figures (13)

• Figure 1: Loss surface $\Gamma(\hat{D}, k)$ (top) and its mean over bin index $\mathcal{L}(\hat{D})$ for a spectrally flat input signal (bottom) for DFT domain estimation of a delay of $D=100$ samples.
• Figure 2: Triangular kernels of length $N'$ (top); their $N=256$ point DFTs (middle); and the corresponding loss surface as a function of $\hat{D}$ for a target delay of $D=100$ samples as in Eq. \ref{['eq:dft_delay_loss']} (bottom).
• Figure 3: The loss surface as a function of pole location of a cascade of $K = 4$ first order all-pass filer sections for a spectrally flat input signal (length $N'=1$) and a triangular pulse of length $N' = 128$ samples, for a DFT length of $N=256$.
• Figure 4: Proposed model structure as it appears during training (a) and at inference (b). Training uses the frequency sampling method over short frames of length $N$ samples. Dashed lines indicate the flow of gradients from the loss function to the learnable modules/parameters (coloured yellow). Inference operates in the time-domain. The switch in position (ii) moves the SVF2 filter within the feedback loop.
• Figure 5: Time-domain ESR for models trained on the digital flanger and phaser effects. Error bars show median and 95% CI.