Co-imagination of Behaviour and Morphology of Agents

Research output: Contribution to book/conference proceedings/anthology/reportConference contributionContributedpeer-review

Contributors

Abstract

The field of robot learning has made great advances in developing behaviour learning methodologies capable of learning policies for tasks ranging from manipulation to locomotion. However, the problem of combined learning of behaviour and robot structure, here called co-adaptation, is less studied. Most of the current co-adapting robot learning approaches rely on model-free algorithms or assume to have access to an a-priori known dynamics model, which requires considerable human engineering. In this work, we investigate the potential of combining model-free and model-based reinforcement learning algorithms for their application on co-adaptation problems with unknown dynamics functions. Classical model-based reinforcement learning is concerned with learning the forward dynamics of a specific agent or robot in its environment. However, in the case of jointly learning the behaviour and morphology of agents, each individual agent-design implies its own specific dynamics function. Here, the challenge is to learn a dynamics model capable of generalising between the different individual dynamics functions or designs. In other words, the learned dynamics model approximates a multi-dynamics function with the goal to generalise between different agent designs. We present a reinforcement learning algorithm that uses a learned multi-dynamics model for co-adapting robot’s behaviour and morphology using imagined rollouts. We show that using a multi-dynamics model for imagining transitions can lead to better performance for model-free co-adaptation, but open challenges remain.

Details

Original languageEnglish
Title of host publicationMachine Learning, Optimization, and Data Science - 9th International Conference, LOD 2023
EditorsGiuseppe Nicosia, Varun Ojha, Emanuele La Malfa, Gabriele La Malfa, Panos M. Pardalos, Renato Umeton
PublisherSpringer Science and Business Media B.V.
Pages318-332
Number of pages15
ISBN (print)9783031539688
Publication statusPublished - 2024
Peer-reviewedYes

Publication series

SeriesLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14505 LNCS
ISSN0302-9743

Conference

Title9th International Conference on Machine Learning, Optimization, and Data Science, LOD 2023
Duration22 - 26 September 2023
CityGrasmere
CountryUnited Kingdom

External IDs

ORCID /0000-0001-9430-8433/work/158768043

Keywords

Keywords

  • Co-Adaptation, Co-Design, Evolutionary Robotics, Reinforcement Learning