Maestoso: An Intelligent Educational Sketching Tool for Learning Music Theory

TL;DR

Maestoso tackles the lack of feedback-rich, sketch-based learning tools for music theory by enabling novice students to sketch and have their input automatically recognized. The system combines a hierarchical sketch recognition pipeline (template matching, staff/classification, and note components) with an instructional interface that provides instructor-emulated feedback and a guided lesson builder. Evaluations show high recognition accuracy for novice sketches (over 95% for simpler components and above 90% for more complex symbols) and that short sessions can introduce introductory concepts effectively. The work demonstrates a practical, scalable approach to bridge novice learning with the capabilities of professional notation tools, potentially broadening access to early music theory education.

Abstract

Learning music theory not only has practical benefits for musicians to write, perform, understand, and express music better, but also for both non-musicians to improve critical thinking, math analytical skills, and music appreciation. However, current external tools applicable for learning music theory through writing when human instruction is unavailable are either limited in feedback, lacking a written modality, or assuming already strong familiarity of music theory concepts. In this paper, we describe Maestoso, an educational tool for novice learners to learn music theory through sketching practice of quizzed music structures. Maestoso first automatically recognizes students' sketched input of quizzed concepts, then relies on existing sketch and gesture recognition techniques to automatically recognize the input, and finally generates instructor-emulated feedback. From our evaluations, we demonstrate that Maestoso performs reasonably well on recognizing music structure elements and that novice students can comfortably grasp introductory music theory in a single session.

Figures (7)

• Figure 1: A sample screenshot of the Maetoso interface: (a) with the original raw strokes displayed, and (b) with the generated beautified strokes displayed.
• Figure 2: A user with novice music theory knowledge writing music component responses.
• Figure 3: Hierarchy of classifiers used in Maestoso, in order of execution.
• Figure 4: Interface for practicing on prompted music theory questions. The quiz interface removes the text hints and graphic solution.
• Figure 5: Output window displaying personalized feedback to users of the indivdiual question feedback.