CON: Continual Object Navigation via Data-Free Inter-Agent Knowledge Transfer in Unseen and Unfamiliar Places

TL;DR

This work proposes a framework in which a traveler robot (student) communicates with local robots (teachers) to obtain ON knowledge through minimal interactions, and develops a query-based occupancy map that dynamically represents target object locations, serving as an effective and communication-friendly knowledge representation.

Abstract

This work explores the potential of brief inter-agent knowledge transfer (KT) to enhance the robotic object goal navigation (ON) in unseen and unfamiliar environments. Drawing on the analogy of human travelers acquiring local knowledge, we propose a framework in which a traveler robot (student) communicates with local robots (teachers) to obtain ON knowledge through minimal interactions. We frame this process as a data-free continual learning (CL) challenge, aiming to transfer knowledge from a black-box model (teacher) to a new model (student). In contrast to approaches like zero-shot ON using large language models (LLMs), which utilize inherently communication-friendly natural language for knowledge representation, the other two major ON approaches -- frontier-driven methods using object feature maps and learning-based ON using neural state-action maps -- present complex challenges where data-free KT remains largely uncharted. To address this gap, we propose a lightweight, plug-and-play KT module targeting non-cooperative black-box teachers in open-world settings. Using the universal assumption that every teacher robot has vision and mobility capabilities, we define state-action history as the primary knowledge base. Our formulation leads to the development of a query-based occupancy map that dynamically represents target object locations, serving as an effective and communication-friendly knowledge representation. We validate the effectiveness of our method through experiments conducted in the Habitat environment.

Figures (3)

• Figure 1: CON formulation: We consider a continual learning (CL) problem where knowledge for object navigation (ON) tasks (e.g., object feature maps, neural state-action maps) is transferred from existing black-box models (teachers) to a new model (student). Unlike typical CL setups, a teacher-side plug-and-play knowledge transfer module called "student proxy" (2) is allowed to engage in high-frequency question-and-answer sessions with the teacher, while the questions sent (1) and the responses received (3) are required to be lightweight and beneficial messages (e.g., language prompts). The figure is adapted from tsukahara2024trainingselflocalizationmodelsunseen.
• Figure 2: Experimental setup. Top: A bird's-eye view of the workspace. Bottom left: An illustration of the map generation process. Bottom right: 100 target images.
• Figure 3: Performance comparison of two ON scenarios. The performance of the proposed method is demonstrated for different settings $X$ of the self-localization failure rate $P^E$: "prob $X$", as well as for two baseline methods: "frontier" and "w/o merge". Vertical axis: SPL performance. Horizontal axis: Target object ID. Higher SPL indicates better performance. For clarity, target object IDs are independently sorted for each individual curve.