Active Restoration of Lost Audio Signals Using Machine Learning and Latent Information

TL;DR

This paper tackles the problem of reconstructing lost audio segments by embedding latent side information via steganography and halftoning, enabling active restoration with machine learning. It introduces ARLAS, a framework that integrates halftone-based compression (HCR) with sequence-to-sequence and RF models to recover erased audio from heavily compressed embedded data. Through extensive experiments on diverse instrument audio and gap lengths, ARLAS shows superior reconstruction quality (via SNR and ODG) for longer gaps and demonstrates robustness beyond traditional inpainting baselines and prior learning-based methods like D2WGAN and SG. The work highlights practical benefits for self-healing audio systems and security applications, while acknowledging room for improvement in inverse-halftoning and cross-domain extension to image reconstruction.

Abstract

Digital audio signal reconstruction of a lost or corrupt segment using deep learning algorithms has been explored intensively in recent years. Nevertheless, prior traditional methods with linear interpolation, phase coding and tone insertion techniques are still in vogue. However, we found no research work on reconstructing audio signals with the fusion of dithering, steganography, and machine learning regressors. Therefore, this paper proposes the combination of steganography, halftoning (dithering), and state-of-the-art shallow and deep learning methods. The results (including comparing the SPAIN, Autoregressive, deep learning-based, graph-based, and other methods) are evaluated with three different metrics. The observations from the results show that the proposed solution is effective and can enhance the reconstruction of audio signals performed by the side information (e.g., Latent representation) steganography provides. Moreover, this paper proposes a novel framework for reconstruction from heavily compressed embedded audio data using halftoning (i.e., dithering) and machine learning, which we termed the HCR (halftone-based compression and reconstruction). This work may trigger interest in optimising this approach and/or transferring it to different domains (i.e., image reconstruction). Compared to existing methods, we show improvement in the inpainting performance in terms of signal-to-noise ratio (SNR), the objective difference grade (ODG) and Hansen's audio quality metric. In particular, our proposed framework outperformed the learning-based methods (D2WGAN and SG) and the traditional statistical algorithms (e.g., SPAIN, TDC, WCP).

Figures (8)

• Figure 1: HCR Visual Inspection: (a) Original audio data reshaped using Eq. \ref{['eq1']} and visualised (b) Halftone of (a) (binary image), (c) Reconstructed (a) from (b) and (d) Small patches cropped from each image left to right, respectively.
• Figure 2: A dot plot depicting the correlation between the original data and its estimated version. Note that the reconstruction is made from merely binary values (without ML and signal drop). This figure shows how much of the information is lost in the compression process; however, we rely on our deep learning model to learn reconstructing the original signal from this approximation.
• Figure 3: Audio track segments to train (red) and test (green) machine learning models.
• Figure 4: Reconstruction of a short audio signal using RF and LSTM.
• Figure 5: Phase I: Determining the best performing non-learning methods. From the results of 20 tests on each of these methods, we have two competitive methods (CP and TDC) that will be tested in Phase II. Although ASPAIN has had no winning cases, since it is a recent algorithm, we opt to upvote it for relevance to phase II (see subsection \ref{['sec:Dataset']}). The X-axis The X-axis denotes the number of cases a given algorithm outperforms other algorithms in terms of SNR or ODG (20 tests were measured). Detailed numerical results, on which this figure is based, are furnished in the supplementary files (https://github.com/ARDISDataset/ARLAS/tree/main/Excel_Sheet).