Co-imagination of Behaviour and Morphology of Agents
Research output: Contribution to book/conference proceedings/anthology/report › Conference contribution › Contributed › peer-review
Contributors
Abstract
The field of robot learning has made great advances in developing behaviour learning methodologies capable of learning policies for tasks ranging from manipulation to locomotion. However, the problem of combined learning of behaviour and robot structure, here called co-adaptation, is less studied. Most of the current co-adapting robot learning approaches rely on model-free algorithms or assume to have access to an a-priori known dynamics model, which requires considerable human engineering. In this work, we investigate the potential of combining model-free and model-based reinforcement learning algorithms for their application on co-adaptation problems with unknown dynamics functions. Classical model-based reinforcement learning is concerned with learning the forward dynamics of a specific agent or robot in its environment. However, in the case of jointly learning the behaviour and morphology of agents, each individual agent-design implies its own specific dynamics function. Here, the challenge is to learn a dynamics model capable of generalising between the different individual dynamics functions or designs. In other words, the learned dynamics model approximates a multi-dynamics function with the goal to generalise between different agent designs. We present a reinforcement learning algorithm that uses a learned multi-dynamics model for co-adapting robot’s behaviour and morphology using imagined rollouts. We show that using a multi-dynamics model for imagining transitions can lead to better performance for model-free co-adaptation, but open challenges remain.
Details
Original language | English |
---|---|
Title of host publication | Machine Learning, Optimization, and Data Science - 9th International Conference, LOD 2023 |
Editors | Giuseppe Nicosia, Varun Ojha, Emanuele La Malfa, Gabriele La Malfa, Panos M. Pardalos, Renato Umeton |
Publisher | Springer Science and Business Media B.V. |
Pages | 318-332 |
Number of pages | 15 |
ISBN (print) | 9783031539688 |
Publication status | Published - 2024 |
Peer-reviewed | Yes |
Publication series
Series | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 14505 LNCS |
ISSN | 0302-9743 |
Conference
Title | 9th International Conference on Machine Learning, Optimization, and Data Science, LOD 2023 |
---|---|
Duration | 22 - 26 September 2023 |
City | Grasmere |
Country | United Kingdom |
External IDs
ORCID | /0000-0001-9430-8433/work/158768043 |
---|
Keywords
ASJC Scopus subject areas
Keywords
- Co-Adaptation, Co-Design, Evolutionary Robotics, Reinforcement Learning