Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Models

Xiaoyu Yi, Qi He, Gus Xia, Ziyu Wang

TL;DR

This work identifies instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and proposes ViTex, a visual representation of instrumental texture, enabling explicit texture-level control while maintaining strong unconditional generation quality.

Abstract

In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.

ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Models

TL;DR

This work identifies instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and proposes ViTex, a visual representation of instrumental texture, enabling explicit texture-level control while maintaining strong unconditional generation quality.

Abstract

In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.
Paper Structure (16 sections, 5 equations, 3 figures, 2 tables)

This paper contains 16 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Control in Symbolic Music Generation
  4. Multitrack Symbolic Music Generation
  5. Diffusion Techniques
  6. Method
  7. Data Representation
  8. Discrete Diffusion Model
  9. Guidance Mechanism
  10. UNet Architecture
  11. Experiment
  12. Dataset and Training Details
  13. Baselines
  14. Objective Evaluation
  15. Subjective Evaluation
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Visualization of ViTex. Left: piano rolls of each track in a digital audio workstation (DAW). Right: the corresponding ViTex visualization. Different glyphs represent local textures. Horizontal axis indicates time; vertical positions within each track indicate the pitch.
  • Figure 2: Illustration of the UNet architecture with optional control from ViTex and chord progressions.
  • Figure 3: Subjective evaluation results with mean ratings and within-subject confidence intervals.