Towards Learning to Play Piano with Dexterous Hands and Touch.
Research output: Contribution to book/Conference proceedings/Anthology/Report › Conference contribution › Contributed › peer-review
Contributors
Abstract
As Liszt once said '(a virtuoso) must call up scent and blossom, and breathe the breath of life', a virtuoso plays the piano with passion, poetry, and extraordinary technical ability. Hence, piano playing, being a task that is quintessentially human, becomes a hallmark for roboticians and artificial intelligence researchers to pursue. In this paper, we advocate an end-to-end reinforcement learning (RL) paradigm to demonstrate how an agent can learn directly from machine-readable music score to play the piano with touch-augmented dexterous hands on a simulated piano. To achieve the desired tasks, we design useful touch- and audio-based reward functions and a series of tasks. Empirical results show that the RL agent can not only find the correct key position but also deal with the various rhythmic, volume, and fingering requirements. As a result, the agent demonstrates its effectiveness in playing simple pieces that have different musical requirements which show the potential of leveraging reinforcement learning approach for the piano playing tasks.
Details
Original language | English |
---|---|
Title of host publication | 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan |
Pages | 10410-10416 |
Number of pages | 7 |
ISBN (electronic) | 9781665479271 |
Publication status | Published - 2022 |
Peer-reviewed | Yes |
External IDs
Scopus | 85146332550 |
---|---|
ORCID | /0000-0001-9430-8433/work/146646294 |