Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mixer is more than just a model

Qingfeng Ji, Yuxin Wang, Letong Sun

TL;DR

The paper extends the Mixer paradigm to audio by introducing ASM-RH, a model that fuses time-domain and frequency-domain information through Roll-Time-mixing and Hermit-Frequency-mixing. It streamlines the architecture by removing unnecessary tokens and adding RollBlock-based time rolling, integrated with Hermit FFT operations, to form RH-MixerBlocks followed by an MLP head. Empirical results across SpeechCommands, UrbanSound8K, CASIA Chinese Sentiment, and RAVDESS demonstrate improved accuracy and AUC over baselines and prior Mixer-based audio models, with ablations confirming the contribution of each mixing component. The work argues for a broader interpretation of Mixer as a data-mixing mindset applicable beyond vision, signaling potential for energy-efficient, cross-domain architectures that blend information from multiple domains.

Abstract

Recently, MLP structures have regained popularity, with MLP-Mixer standing out as a prominent example. In the field of computer vision, MLP-Mixer is noted for its ability to extract data information from both channel and token perspectives, effectively acting as a fusion of channel and token information. Indeed, Mixer represents a paradigm for information extraction that amalgamates channel and token information. The essence of Mixer lies in its ability to blend information from diverse perspectives, epitomizing the true concept of "mixing" in the realm of neural network architectures. Beyond channel and token considerations, it is possible to create more tailored mixers from various perspectives to better suit specific task requirements. This study focuses on the domain of audio recognition, introducing a novel model named Audio Spectrogram Mixer with Roll-Time and Hermit FFT (ASM-RH) that incorporates insights from both time and frequency domains. Experimental results demonstrate that ASM-RH is particularly well-suited for audio data and yields promising outcomes across multiple classification tasks. The models and optimal weights files will be published.

Mixer is more than just a model

TL;DR

The paper extends the Mixer paradigm to audio by introducing ASM-RH, a model that fuses time-domain and frequency-domain information through Roll-Time-mixing and Hermit-Frequency-mixing. It streamlines the architecture by removing unnecessary tokens and adding RollBlock-based time rolling, integrated with Hermit FFT operations, to form RH-MixerBlocks followed by an MLP head. Empirical results across SpeechCommands, UrbanSound8K, CASIA Chinese Sentiment, and RAVDESS demonstrate improved accuracy and AUC over baselines and prior Mixer-based audio models, with ablations confirming the contribution of each mixing component. The work argues for a broader interpretation of Mixer as a data-mixing mindset applicable beyond vision, signaling potential for energy-efficient, cross-domain architectures that blend information from multiple domains.

Abstract

Recently, MLP structures have regained popularity, with MLP-Mixer standing out as a prominent example. In the field of computer vision, MLP-Mixer is noted for its ability to extract data information from both channel and token perspectives, effectively acting as a fusion of channel and token information. Indeed, Mixer represents a paradigm for information extraction that amalgamates channel and token information. The essence of Mixer lies in its ability to blend information from diverse perspectives, epitomizing the true concept of "mixing" in the realm of neural network architectures. Beyond channel and token considerations, it is possible to create more tailored mixers from various perspectives to better suit specific task requirements. This study focuses on the domain of audio recognition, introducing a novel model named Audio Spectrogram Mixer with Roll-Time and Hermit FFT (ASM-RH) that incorporates insights from both time and frequency domains. Experimental results demonstrate that ASM-RH is particularly well-suited for audio data and yields promising outcomes across multiple classification tasks. The models and optimal weights files will be published.
Paper Structure (13 sections, 4 figures, 6 tables)

This paper contains 13 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. ASM-RH Model
  3. Removal of useless structures
  4. RollBlock and Roll-Time-mixing
  5. Hermit-Frequency-mixing
  6. Experiments
  7. SpeechCommands
  8. UrbanSound8K
  9. CASIA Chinese Sentiment Corpus
  10. RAVDESS
  11. Ablation Study
  12. Discussion
  13. Mixer is more than just a model

Figures (4)

  • Figure 1: Audio Spectrogram Mixer with Roll-Time and Hermit FFT
  • Figure 2: RollBlock
  • Figure 3: Roll-Time-mixing
  • Figure 4: Hermit-Frequency-mixing