Autonomous Generation of Sub-goals for Lifelong Learning in Robots

TL;DR

The paper tackles autonomous discovery of sub-goals for lifelong robot learning by proposing a two-pronged approach: a top-down mechanism driven by intrinsic motivation to derive sub-goals from general goals, and a bottom-up mechanism that uncovers latent relationships among perceptual classes and goals to chain sub-goals efficiently. Implemented within the e-MDB cognitive architecture, the method leverages a graph-based memory of perceptual classes and goals to enable generalization and reuse of skills. Robotic experiments on a TIAGo platform show that combining top-down and bottom-up sub-goal generation yields learning efficiency approaching a reward-based baseline, with clear demonstrations of reusable sub-goals and latent link exploitation. These results suggest that integrating intrinsic motivation with latent relational reasoning enhances lifelong open-ended learning in dynamic, unknown environments.

Abstract

One of the challenges of open-ended learning in robots is the need to autonomously discover goals and learn skills to achieve them. However, when in lifelong learning settings, it is always desirable to generate sub-goals with their associated skills, without relying on explicit reward, as steppingstones to a goal. This allows sub-goals and skills to be reused to facilitate achieving other goals. This work proposes a two-pronged approach for sub-goal generation to address this challenge: a top-down approach, where sub-goals are hierarchically derived from general goals using intrinsic motivations to discover them, and a bottom-up approach, where sub-goal chains emerge from making latent relationships between goals and perceptual classes that were previously learned in different domains explicit. These methods help the robot to autonomously generate and chain sub-goals as a way to achieve more general goals. Additionally, they create more abstract representations of goals, helping to reduce sub-goal duplication and make the learning of skills more efficient. Implemented within an existing cognitive architecture for lifelong open-ended learning and tested with a real robot, our approach enhances the robot's ability to discover and achieve goals, generate sub-goals in an efficient manner, generalize learned skills, and operate in dynamic and unknown environments without explicit intermediate rewards.

Figures (8)

• Figure 1: Memory structure of the e-MDB cognitive architecture.
• Figure 2: Top-down sub-goal generation.
• Figure 3: Bottom-up sub-goal generation. $G_y$ is identified as sub-goal of $G_x$.
• Figure 4: Experimental setup with a TIAGo robot.
• Figure 5: Performance comparison of the different approaches of sub-goal generation.