Time delay embeddings to characterize the timbre of musical instruments using Topological Data Analysis: a study on synthetic and real data

TL;DR

The paper addresses timbre characterization by leveraging Topological Data Analysis on time-delay embeddings of audio signals to detect harmonic structure beyond standard spectral descriptors. It analyzes how the embedding delay $\tau$ influences the persistent diagrams of the 2D embedding $X_2(x_t; \tau)$ and uses the Wasserstein distance $W$ to quantify timbre as the difference between the diagram of a signal and that of a sine wave with the same fundamental frequency $f_0$. Key contributions include a systematic study identifying delays related to fractions of the fundamental period, showing that delays around $T_0/2$ emphasize integer harmonics while delays around $T_0/4$ highlight both integer and non-integer harmonics, validated on synthetic and real data (e.g., guitar, NSynth). The results suggest a practical, topology-based pathway for timbre analysis, with extensions to higher-dimensional embeddings and richer persistence statistics.

Abstract

Timbre allows us to distinguish between sounds even when they share the same pitch and loudness, playing an important role in music, instrument recognition, and speech. Traditional approaches, such as frequency analysis or machine learning, often overlook subtle characteristics of sound. Topological Data Analysis (TDA) can capture complex patterns, but its application to timbre has been limited, partly because it is unclear how to represent sound effectively for TDA. In this study, we investigate how different time delay embeddings affect TDA results. Using both synthetic and real audio signals, we identify time delays that enhance the detection of harmonic structures. Our findings show that specific delays, related to fractions of the fundamental period, allow TDA to reveal key harmonic features and distinguish between integer and non-integer harmonics. The method is effective for synthetic and real musical instrument sounds and opens the way for future works, which could extend it to more complex sounds using higher-dimensional embeddings and additional persistence statistics.

Figures (15)

• Figure 1: Examples of $n$-simplices. From left to right: a $0$-simplex, a $1$-simplex, a $2$-simplex, and a $3$-simplex. Note that in a simplicial complex one cannot find an isolated $n$-simplex (for $n\geq 1$), as the presence of an $n$-simplex implies the existence of all the $(n-1)$-simplices.
• Figure 2: Examples of topological structures and their Betti numbers. From left to right: a dot ($0$-dimensional structure), whose only non-zero Betti number is $\beta_0=1$ (i.e., one connected component); a line ($1$-dimensional), with $\beta_0=1$ (i.e., one connected component); a circle ($2$-dimensional, with $\beta_0=1$ and $\beta_1=1$ (i.e., one connected component and one hole); an empty sphere ($3$-dimensional, with $\beta_0=1$, $\beta_1=0$, and $\beta_2=1$ (i.e., one connected component, no holes, and one empty volume).
• Figure 3: Topology of a simplicial complex. Example of a simplicial complex made of $6$ nodes ($0$-simplices) $[v_0],[v_1],[v_2],[v_3],[v_4],[v_5]$, $6$ links ($1$-simplices) $[v_0,v_1],[v_0,v_2],[v_1,v_2],[v_1,v_3],[v_2,v_3],[v_3,v_4]$, and $1$ triangle ($2$-simplex) $[v_0,v_1,v_2]$. There are $2$ connected components, so $\beta_0=2$, and one hole, so that $\beta_1=1$. Note that this simplicial complex is topologically equivalent to another simplicial complex obtained with only $[v_1],[v_2],[v_3],[v_5]$ as $0$-simplices, and $[v_1,v_2],[v_1,v_3],[v_2,v_3]$ as $1$-simplices, as the latter would have the same Betti numbers $\beta_0$ and $\beta_1$.
• Figure 4: Example of how a simplicial complex is extracted from data. We start with a radius $r_0=0$, so all the points ($0$-simplices) are separated (left panel) and $\beta_0$ is equal to the number of points. Then we increase the radius making the circles around the point larger. Once two circles intersect, the two corresponding $0$-simplices are connected via a $1$-simplex (middle panel), and $\beta_0$ decreases. If three circles intersect simultaneously, then the corresponding $0$-simplices are connected via a $2$-simplex (right panel). In the latter case, we have that $\beta_0=1$ and $\beta_1=1$. In general, when $(n+1)$ circles intersect simultaneously, the corresponding $0$-simplices are connected via an $n$-simplex. The results of such analysis can be visualized through the persistence diagram (see Fig. \ref{['fig_E']}).
• Figure 5: $2$-dimensional data and persistence diagram. On the left, example of data points in a $2$-dimensional domain. On the right, example of a persistence diagram, which summarizes the topological features of the dataset on the left panel across scales. Each point represents a topological feature. The $x$ axis indicates the radius at which the feature appears (“birth”), and the $y$ axis the radius at which it disappears (“death”). Topological features that persist with respect to the radius, i.e., far from the diagonal, correspond to meaningful structures in the data, while those close to the diagonal are usually caused by noise in the data.