Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Active Learning of Robot Vision Using Adaptive Path Planning

Julius Rückin, Federico Magistri, Cyrill Stachniss, Marija Popović

TL;DR

A recent adaptive planning framework for efficient training data collection to substantially reduce human labelling requirements in semantic terrain monitoring missions is presented, which combines high-quality human labels with automatically generated pseudo labels.

Abstract

Robots need robust and flexible vision systems to perceive and reason about their environments beyond geometry. Most of such systems build upon deep learning approaches. As autonomous robots are commonly deployed in initially unknown environments, pre-training on static datasets cannot always capture the variety of domains and limits the robot's vision performance during missions. Recently, self-supervised as well as fully supervised active learning methods emerged to improve robotic vision. These approaches rely on large in-domain pre-training datasets or require substantial human labelling effort. To address these issues, we present a recent adaptive planning framework for efficient training data collection to substantially reduce human labelling requirements in semantic terrain monitoring missions. To this end, we combine high-quality human labels with automatically generated pseudo labels. Experimental results show that the framework reaches segmentation performance close to fully supervised approaches with drastically reduced human labelling effort while outperforming purely self-supervised approaches. We discuss the advantages and limitations of current methods and outline valuable future research avenues towards more robust and flexible robotic vision systems in unknown environments.

Active Learning of Robot Vision Using Adaptive Path Planning

TL;DR

A recent adaptive planning framework for efficient training data collection to substantially reduce human labelling requirements in semantic terrain monitoring missions is presented, which combines high-quality human labels with automatically generated pseudo labels.

Abstract

Robots need robust and flexible vision systems to perceive and reason about their environments beyond geometry. Most of such systems build upon deep learning approaches. As autonomous robots are commonly deployed in initially unknown environments, pre-training on static datasets cannot always capture the variety of domains and limits the robot's vision performance during missions. Recently, self-supervised as well as fully supervised active learning methods emerged to improve robotic vision. These approaches rely on large in-domain pre-training datasets or require substantial human labelling effort. To address these issues, we present a recent adaptive planning framework for efficient training data collection to substantially reduce human labelling requirements in semantic terrain monitoring missions. To this end, we combine high-quality human labels with automatically generated pseudo labels. Experimental results show that the framework reaches segmentation performance close to fully supervised approaches with drastically reduced human labelling effort while outperforming purely self-supervised approaches. We discuss the advantages and limitations of current methods and outline valuable future research avenues towards more robust and flexible robotic vision systems in unknown environments.

Paper Structure

This paper contains 13 sections, 1 equation, 3 figures.

Table of Contents

  1. Introduction
  2. Our Approach
  3. Probabilistic Semantic Environment Mapping
  4. Adaptive Informative Path Planning
  5. Efficient Labelling
  6. Experimental Results
  7. Discussion & Future Directions
  8. Faster to Answer Human Labelling Queries
  9. Novel Embodied Self-supervised Learning Methods
  10. Improved Uncertainty Quantification
  11. Towards Continual Active Learning
  12. Improved Model Re-training Efficiency
  13. Conclusion

Figures (3)

  • Figure 1: Our semi-supervised active learning approach in an unknown environment. During a mission, a semantic segmentation network predicts pixel-wise semantics and model uncertainties from an RGB-D image. Both are fused into an uncertainty-aware semantic map, which is used by our adaptive planner to guide the robot towards areas of informative training data where model uncertainty is high. After a mission, the collected data is labelled using two sources of labels: (i) human pixel labelling and (ii) self-supervised pseudo label generation from the semantic map.
  • Figure 2: Our global map-based adaptive planners (blue, orange, green) compared to state-of-the-art local planning (purple) and classical non-adaptive coverage paths (yellow). Our map-based planners require substantially fewer pixel-wise human-labelled images to reach the same performance as coverage and local planning.
  • Figure 3: Our semi-supervised adaptive frontier planning compared to fully and self-supervised adaptive frontier planning. Our semi-supervised approach almost reaches the fully supervised performance while clearly outperforming the self-supervised approach.