Automated Curriculum Learning for Dynamic Robot Behaviours

The School of EECS is hosting the following PhD Progress Review 1 Confirmation Seminar:

Automated Curriculum Learning for Dynamic Robot Behaviours

Speaker: Humphrey Munn
Host: A/Prof Marcus Gallagher

Abstract: Recent advancements in robotics and deep reinforcement learning  have opened new avenues for intricate, dynamic robot behaviours. The current research landscape involves two kinds of curriculum strategies for training robots: highly engineered hand-crafted curricula, and  autonomous curricula with implicit and untested assumptions about how  robots learn. While these techniques have excelled at training robots for tasks with non-sparse rewards like walking over terrain, initial investigation suggests they struggle to meet the challenges posed by intricate, sparse-reward manipulation tasks like full-body throwing and catching of a ball with a humanoid robot, where the reward cannot be meaningfully determined until the task has been executed.

In this research, we explore the under-researched and challenging area of full-body manipulation tasks for robotics. Our primary focus is on evaluating the efficacy of the existing and proposed autonomous curriculum learning methods, which assist a standard reinforcement learning algorithm (e.g. distributed- proximal policy optimisation) to acquire full-body manipulation skills. Unlike existing approaches which require careful curricula design, our goal is to develop general algorithm(s) that alleviate this burden, offering a more scalable and adaptable approach to learning complex manipulation skills. Furthermore, we hope to improve on existing autonomous curriculum learning methods by analysing the effect of curriculum variants on robot performance and using this analysis to better inform our methods. This investigation aims to both advance the performance of robots in complex tasks, and to illuminate the relationship between curriculum design and the  performance of agents more generally in complex, sparse-reward tasks.