SOMson -- Sonification of Multidimensional Data in Kohonen Maps

TL;DR

Instead of a user study, this work presents an interactive online example, so readers can explore SOMson themselves, an interactive sonification of the underlying data, as a data augmentation technique.

Abstract

Kohonen Maps, aka. Self-organizing maps (SOMs) are neural networks that visualize a high-dimensional feature space on a low-dimensional map. While SOMs are an excellent tool for data examination and exploration, they inherently cause a loss of detail. Visualizations of the underlying data do not integrate well and, therefore, fail to provide an overall picture. Consequently, we suggest SOMson, an interactive sonification of the underlying data, as a data augmentation technique. The sonification increases the amount of information provided simultaneously by the SOM. Instead of a user study, we present an interactive online example, so readers can explore SOMson themselves. Its strengths, weaknesses, and prospects are discussed.

Figures (6)

• Figure 1: The U-matrix is the main output of a Self Organizing Map. Instead of visualizing the $4$-dimensional pointer at each of the $16\times 16$ nodes of the unit layer, the U-matrix indicates the mean distance between each node's pointer and the pointers of all its neighboring nodes. Nodes in the corners have $3$ neighbors, the other nodes along the fringe have $5$, and nodes in the middle have $8$ neighbors. The single items used to train the SOM (techno music) are shown as colored dots, whereby different colors represent different techno music styles.
• Figure 2: The component planes plot the magnitude of each one feature at each unit from dark blue (minimum) to yellow (maximum).
• Figure 3: The SOMson interface with the visualizations on the left and the sonification parameters on the right. Buttons allow switching between U-matrix and the component planes, and showing/hiding the training data.
• Figure 4: Bezold effect: Colors and shades may appear different depending on their surrounding colors. Here, all three notes have the same (single) color, even though the one on the left may appear darker, and the one in the middle may seem to have a color gradient.
• Figure 5: The Bezold effect also holds for lightness: All notes have the exact same (single) color and lightness level, even though the one on the left may appear darker and the one in the middle appears as if it had a lightness gradient.