IMOST: Incremental Memory Mechanism with Online Self-Supervision for Continual Traversability Learning

TL;DR

IMOST tackles online continual traversability learning by integrating an Incremental Dynamic Memory that builds scene-aware, balanced clusters with an information expansion criterion and a Self-Supervised Annotation pipeline using FastSAM to provide complete, real-time traversability masks. An anomaly-learning module based on a VAE-MLP framework learns from memory-derived supervision with reconstruction, traversability, and KL-divergence losses, enabling robust online adaptation. The approach, validated on a quadruped robot across public and self-collected datasets, outperforms the prior online method while maintaining low memory usage and strong boundary delineation. The combination of IDM and SSA enables scalable, real-time continual learning for traversability in dynamic, unstructured environments, with practical impact on robotic navigation and autonomy.

Abstract

Traversability estimation is the foundation of path planning for a general navigation system. However, complex and dynamic environments pose challenges for the latest methods using self-supervised learning (SSL) technique. Firstly, existing SSL-based methods generate sparse annotations lacking detailed boundary information. Secondly, their strategies focus on hard samples for rapid adaptation, leading to forgetting and biased predictions. In this work, we propose IMOST, a continual traversability learning framework composed of two key modules: incremental dynamic memory (IDM) and self-supervised annotation (SSA). By mimicking human memory mechanisms, IDM allocates novel data samples to new clusters according to information expansion criterion. It also updates clusters based on diversity rule, ensuring a representative characterization of new scene. This mechanism enhances scene-aware knowledge diversity while maintaining a compact memory capacity. The SSA module, integrating FastSAM, utilizes point prompts to generate complete annotations in real time which reduces training complexity. Furthermore, IMOST has been successfully deployed on the quadruped robot, with performance evaluated during the online learning process. Experimental results on both public and self-collected datasets demonstrate that our IMOST outperforms current state-of-the-art method, maintains robust recognition capabilities and adaptability across various scenarios. The code is available at https://github.com/SJTU-MKH/OCLTrav.

Figures (7)

• Figure 1: Continual traversability learning. Under limited resource, previous experience replace methods (ERM) did not take into account the semantic distribution across multiple scenarios, leading to imbalanced data and catastrophic forgetting.Ours IMOST incrementally and evenly preserves data related to different scenes.
• Figure 2: The overview of IMOST. FastSAM generates the traversability mask, with the traversable regions highlighted in purple in the image. Stego produces the feature map. The mask and feature map are combined to compute the feature distribution of the traversable areas. Then IDM updates the cluster according to expansion criterion and heuristically selects data to be input into the inference network for training.
• Figure 3: Comparison performance of self-supervised methods for 3 example images. WVN expands sparse labels by associating similar semantics.But the boundary segmentation between traversable and non-traversable areas is still not clear enough. Our SSA achieves nearly complete traversability semantic annotation.
• Figure 4: Detailed data flow. There are multiple scenes, and the data is imbalanced across different scenes.
• Figure 5: Qualitative results on public dataset.