Denoising of photogrammetric dummy head ear point clouds for individual Head-Related Transfer Functions computation

TL;DR

Comparison of HRTFs derived from original and denoised scans against reference data shows that the best-performing DNN marginally reduces the deviation of photogrammetric dummy head HRTFs to levels closer to accurately measured ones.

Abstract

Individual Head-Related Transfer Functions (HRTFs), crucial for realistic virtual audio rendering, can be efficiently numerically computed from precise three-dimensional head and ear scans. While photogrammetry scanning is promising, it generally lacks accuracy, leading to HRTFs showing significant perceptual deviation from reference data, mainly due to scanning errors affecting the most occluded pinna structures. This paper examines the application of Deep Neural Networks (DNNs) for denoising photogrammetric ear scans. Several DNNs, fine-tuned on pinna samples corrupted with synthetic error modelled to mimic that observed in photogrammetric dummy head scans, are tested and benchmarked against a classical denoising method. One DNN is further modified and retrained to enhance its denoising performance. The comparison of HRTFs derived from original and denoised scans against reference data shows that the best-performing DNN marginally reduces the deviation of photogrammetric dummy head HRTFs to levels closer to accurately measured ones. Additionally, correlation analysis between geometric and HRTF metrics, computed on the scanned point clouds and their corresponding HRTFs, is used to identify key measures for evaluating the deviation between target and reference scans. These findings are expected to guide the selection of relevant loss functions and foster improvements in this and similar DNN models.

Figures (11)

• Figure 1: KU1 right ear point cloud derived from the laser scan. The colour shows the ambient occlusion at each point.
• Figure 2: KU1 right ear point clouds derived from the photogrammetric scan with chalk marks (a) and scanning spray (b), and laser scan corrupted with synthetic photogrammetric error (c). The colour shows the signed distance from the laser scan, cropped at $\pm2.5mm$.
• Figure 3: Example of denoised query point ($\widetilde{y_i}$) and points ($x_i$) of its related local reference patch ($\mathcal{X}_{y_i}$). The lines show the minimum and maximum $L_2$ norm between $\widetilde{y_i}$ and $\mathcal{X}_{y_i}$, used in the loss function ($L$).
• Figure 4: Absolute magnitude difference between analytical and FEM Adaptive Order solutions for the scattering of a point source by a rigid sphere. The results are shown on the horizontal plane from contralateral to ipsilateral side, centred at $\theta = \SIlist{-90; 90}{\degree}$, respectively.
• Figure 5: Noise reduction obtained with the denoising algorithms on the testingset (a) and scanset (b). The bars show the interquartile range, while the markers show the median.