Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robots

Ali Ayub, Chrystopher Nehaniv, Kerstin Dautenhahn

TL;DR

The paper tackles why home service robots struggle to maintain semantic knowledge in changing environments and presents an Interactive Continual Learning (ICL) architecture that merges continual learning, interactive machine learning, and semantic reasoning within a ROS-based robot stack. It enables real-time, few-shot learning of objects and contexts through human guidance and stores knowledge in a dual-memory system (LTM and STM) with memory fading to reflect dynamics in homes. Demonstrated on a Fetch robot over two months, learning 20 objects and 2 contexts and performing repeated object-fetch tasks, ICL shows partial forgetting but no catastrophic forgetting and competitive performance compared with joint training. The work advances long-term personalization for home service robots, supporting robust object search and manipulation in dynamic environments with minimal user input and real-time learning capabilities.

Abstract

For robots to perform assistive tasks in unstructured home environments, they must learn and reason on the semantic knowledge of the environments. Despite a resurgence in the development of semantic reasoning architectures, these methods assume that all the training data is available a priori. However, each user's environment is unique and can continue to change over time, which makes these methods unsuitable for personalized home service robots. Although research in continual learning develops methods that can learn and adapt over time, most of these methods are tested in the narrow context of object classification on static image datasets. In this paper, we combine ideas from continual learning, semantic reasoning, and interactive machine learning literature and develop a novel interactive continual learning architecture for continual learning of semantic knowledge in a home environment through human-robot interaction. The architecture builds on core cognitive principles of learning and memory for efficient and real-time learning of new knowledge from humans. We integrate our architecture with a physical mobile manipulator robot and perform extensive system evaluations in a laboratory environment over two months. Our results demonstrate the effectiveness of our architecture to allow a physical robot to continually adapt to the changes in the environment from limited data provided by the users (experimenters), and use the learned knowledge to perform object fetching tasks.

Interactive Continual Learning Architecture for Long-Term Personalization of Home Service Robots

TL;DR

The paper tackles why home service robots struggle to maintain semantic knowledge in changing environments and presents an Interactive Continual Learning (ICL) architecture that merges continual learning, interactive machine learning, and semantic reasoning within a ROS-based robot stack. It enables real-time, few-shot learning of objects and contexts through human guidance and stores knowledge in a dual-memory system (LTM and STM) with memory fading to reflect dynamics in homes. Demonstrated on a Fetch robot over two months, learning 20 objects and 2 contexts and performing repeated object-fetch tasks, ICL shows partial forgetting but no catastrophic forgetting and competitive performance compared with joint training. The work advances long-term personalization for home service robots, supporting robust object search and manipulation in dynamic environments with minimal user input and real-time learning capabilities.

Abstract

For robots to perform assistive tasks in unstructured home environments, they must learn and reason on the semantic knowledge of the environments. Despite a resurgence in the development of semantic reasoning architectures, these methods assume that all the training data is available a priori. However, each user's environment is unique and can continue to change over time, which makes these methods unsuitable for personalized home service robots. Although research in continual learning develops methods that can learn and adapt over time, most of these methods are tested in the narrow context of object classification on static image datasets. In this paper, we combine ideas from continual learning, semantic reasoning, and interactive machine learning literature and develop a novel interactive continual learning architecture for continual learning of semantic knowledge in a home environment through human-robot interaction. The architecture builds on core cognitive principles of learning and memory for efficient and real-time learning of new knowledge from humans. We integrate our architecture with a physical mobile manipulator robot and perform extensive system evaluations in a laboratory environment over two months. Our results demonstrate the effectiveness of our architecture to allow a physical robot to continually adapt to the changes in the environment from limited data provided by the users (experimenters), and use the learned knowledge to perform object fetching tasks.
Paper Structure (17 sections, 5 equations, 2 figures, 1 table)

This paper contains 17 sections, 5 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Continual Learning
  4. Semantic Reasoning Architectures
  5. Interactive Continual Learning Architecture
  6. Perception Layer
  7. Interaction Layer
  8. Continual Learning and Memory
  9. Continual Learning
  10. Long-Term Memory
  11. Short-Term Memory
  12. Reasoning, Planning, and Action
  13. Experiments
  14. Experimental Setup
  15. Continual Learning Experiment
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of the ICL architecture. It is composed of 1) perception, 2) interaction, 3) continual learning and memory, and 4) reasoning, planning, and action.
  • Figure 2: (a) An example of the kitchen context in our experimental setup. Note the variation in object size, the closeness of objects, and distractions, such as robots and transparent walls behind. See the supplementary video (https://youtu.be/9y7zf4JUQGU) for examples of other contexts. (b) SLAM map of the indoor environment.