Mixed Reality Strategies for Piano Education

Publikation: Beitrag in FachzeitschriftForschungsartikelBeigetragenBegutachtung

Beitragende

Abstract

Head-mounted mixed reality interfaces, in which a user perceives a seamless blend of real, virtual or remote content, have great potential in a wide range of educational contexts. In this paper, we explore the use of mixed reality (MR) in piano education. We begin with a review of previous examples of virtual and mixed reality for piano teaching and learning, identifying four major categories of functionality: remote teaching, learning to read musical notation, providing alternative notation systems to Western musical notation, and enhancing users' understanding and experience of music. Following this review, we present an application designed to demonstrate the use of MR for facilitating remote student-teacher piano lessons. Additionally, hand-tracking enables real-time, bi-directional visualization of both the student's and teacher's hand movements while playing, which is a crucial communication channel in piano instruction. We also present the Piano Theory Hub, an interactive area for independent learning and practice designed to compliment remote instructions. The Piano Theory Hub uses MR to provide spatial in situ explanations and exercises on notes, intervals, scales, and chords. User studies were conducted to evaluate both remote and solo aspects of the application. The first study with 10 participants revealed a strong sense of immersion and co-presence with the teacher during remote lessons.Trials also revealed that, in addition to virtual hands, visual highlighting of key-presses was found to improve visualisation of the remote users' play. The second study, with 15 participants found the Piano Theory Hub to be helpful for beginners and some intermediate players, with feedback suggesting improvements in user experience and highlighting the limitations of replacing traditional piano teachers. Our experiments demonstrate that MR can be successfully employed for the following strategies for piano education: hand interaction coaching, flexible virtual hand and piano visibility, augmented feedback including key highlighting, gamified learning elements, and flexible teacher positioning options. Overall, the findings suggest that mixed reality holds promise as an effective tool for remote piano learning and music education, offering immersive and engaging learning experiences.

Details

OriginalspracheEnglisch
Aufsatznummer1397154
FachzeitschriftFrontiers in Virtual Reality
Jahrgang5
PublikationsstatusVeröffentlicht - 2024
Peer-Review-StatusJa

Externe IDs

ORCID /0000-0002-8923-6284/work/170106987
Scopus 85209381692
ORCID /0000-0002-9268-4854/work/173987970

Schlagworte

Forschungsprofillinien der TU Dresden

Fächergruppen, Lehr- und Forschungsbereiche, Fachgebiete nach Destatis

ASJC Scopus Sachgebiete