Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recursive Distillation for Open-Set Distributed Robot Localization

Kenta Tsukahara, Kanji Tanaka

TL;DR

This work tackles open-world self-localization for distributed robots where annotated training data is unavailable. It introduces data-free recursive distillation (DFRD), which reconstructs a pseudo-training dataset $D=\{(x,y)\}$ from a teacher model using a trainable synthesizer $g$ and uses it to train a student without accessing original data. The approach relies on a minimal assumption that the teacher's self-localization model can be used as a communication channel for knowledge transfer and employs a ranking-function as a generic teacher model, with sampling controlled by oracle/random mixes and regularized by the reciprocal-rank feature. Experiments on the NCLT dataset demonstrate robust, privacy-preserving continual learning across 10 seasons with a 100-place grid, showing that the open-set KT pipeline remains effective as the teacher set grows.

Abstract

A typical assumption in state-of-the-art self-localization models is that an annotated training dataset is available for the target workspace. However, this is not necessarily true when a robot travels around the general open world. This work introduces a novel training scheme for open-world distributed robot systems. In our scheme, a robot (``student") can ask the other robots it meets at unfamiliar places (``teachers") for guidance. Specifically, a pseudo-training dataset is reconstructed from the teacher model and then used for continual learning of the student model under domain, class, and vocabulary incremental setup. Unlike typical knowledge transfer schemes, our scheme introduces only minimal assumptions on the teacher model, so that it can handle various types of open-set teachers, including those uncooperative, untrainable (e.g., image retrieval engines), or black-box teachers (i.e., data privacy). In this paper, we investigate a ranking function as an instance of such generic models, using a challenging data-free recursive distillation scenario, where a student once trained can recursively join the next-generation open teacher set.

Recursive Distillation for Open-Set Distributed Robot Localization

TL;DR

This work tackles open-world self-localization for distributed robots where annotated training data is unavailable. It introduces data-free recursive distillation (DFRD), which reconstructs a pseudo-training dataset from a teacher model using a trainable synthesizer and uses it to train a student without accessing original data. The approach relies on a minimal assumption that the teacher's self-localization model can be used as a communication channel for knowledge transfer and employs a ranking-function as a generic teacher model, with sampling controlled by oracle/random mixes and regularized by the reciprocal-rank feature. Experiments on the NCLT dataset demonstrate robust, privacy-preserving continual learning across 10 seasons with a 100-place grid, showing that the open-set KT pipeline remains effective as the teacher set grows.

Abstract

A typical assumption in state-of-the-art self-localization models is that an annotated training dataset is available for the target workspace. However, this is not necessarily true when a robot travels around the general open world. This work introduces a novel training scheme for open-world distributed robot systems. In our scheme, a robot (``student") can ask the other robots it meets at unfamiliar places (``teachers") for guidance. Specifically, a pseudo-training dataset is reconstructed from the teacher model and then used for continual learning of the student model under domain, class, and vocabulary incremental setup. Unlike typical knowledge transfer schemes, our scheme introduces only minimal assumptions on the teacher model, so that it can handle various types of open-set teachers, including those uncooperative, untrainable (e.g., image retrieval engines), or black-box teachers (i.e., data privacy). In this paper, we investigate a ranking function as an instance of such generic models, using a challenging data-free recursive distillation scenario, where a student once trained can recursively join the next-generation open teacher set.
Paper Structure (4 sections, 4 figures)

This paper contains 4 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Open-Set Distributed Robot Localization
  3. Data-Free Recursive Distillation
  4. Experiments

Figures (4)

  • Figure 1: Recursive distillation scheme. Teacher/student models and teacher-to-student relationships are represented by circles and line segments. A student can ask an open teacher set to transfer knowledge. A trained student can recursively join the next-generation open teacher set.
  • Figure 2: NCLT dataset. Top: A bird's eye view of the robot workspace. Bottom: Sample view images from the robot's onboard camera.
  • Figure 3: Visual embedding used in the experimental setup. A spatial-semantic scene graph is generated from the view image. For details, please refer to irosw2022yoshida.
  • Figure 4: Performance results. Self-localization performance of the student model at each generation is evaluated. The vertical and horizontal axes are Top-1 accuracy performance and generation ID, respectively.