Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Embodied Perception in Dynamic Environments via Disentangled Weight Fusion

Juncen Guo, Xiaoguang Zhu, Jingyi Wu, Jingyu Zhang, Jingnan Cai, Zhenghao Niu, Liang Song

Abstract

Embodied perception systems face severe challenges of dynamic environment distribution drift when they continuously interact in open physical spaces. However, the existing domain incremental awareness methods often rely on the domain id obtained in advance during the testing phase, which limits their practicability in unknown interaction scenarios. At the same time, the model often overfits to the context-specific perceptual noise, which leads to insufficient generalization ability and catastrophic forgetting. To address these limitations, we propose a domain-id and exemplar-free incremental learning framework for embodied multimedia systems, which aims to achieve robust continuous environment adaptation. This method designs a disentangled representation mechanism to remove non-essential environmental style interference, and guide the model to focus on extracting semantic intrinsic features shared across scenes, thereby eliminating perceptual uncertainty and improving generalization. We further use the weight fusion strategy to dynamically integrate the old and new environment knowledge in the parameter space, so as to ensure that the model adapts to the new distribution without storing historical data and maximally retains the discrimination ability of the old environment. Extensive experiments on multiple standard benchmark datasets show that the proposed method significantly reduces catastrophic forgetting in a completely exemplar-free and domain-id free setting, and its accuracy is better than the existing state-of-the-art methods.

Robust Embodied Perception in Dynamic Environments via Disentangled Weight Fusion

Abstract

Embodied perception systems face severe challenges of dynamic environment distribution drift when they continuously interact in open physical spaces. However, the existing domain incremental awareness methods often rely on the domain id obtained in advance during the testing phase, which limits their practicability in unknown interaction scenarios. At the same time, the model often overfits to the context-specific perceptual noise, which leads to insufficient generalization ability and catastrophic forgetting. To address these limitations, we propose a domain-id and exemplar-free incremental learning framework for embodied multimedia systems, which aims to achieve robust continuous environment adaptation. This method designs a disentangled representation mechanism to remove non-essential environmental style interference, and guide the model to focus on extracting semantic intrinsic features shared across scenes, thereby eliminating perceptual uncertainty and improving generalization. We further use the weight fusion strategy to dynamically integrate the old and new environment knowledge in the parameter space, so as to ensure that the model adapts to the new distribution without storing historical data and maximally retains the discrimination ability of the old environment. Extensive experiments on multiple standard benchmark datasets show that the proposed method significantly reduces catastrophic forgetting in a completely exemplar-free and domain-id free setting, and its accuracy is better than the existing state-of-the-art methods.

Paper Structure

This paper contains 18 sections, 7 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Disentangled Feature Representation Learning
  4. Anti-forgetting via Intrinsic Weight Fusion
  5. Domain-invariant Feature Augmentation
  6. Experiments
  7. Experimental Setup
  8. Datasets
  9. Evaluation Metrics
  10. Implementation Details
  11. Comparison with Existing Methods
  12. Results on Dataset CDDB
  13. Results on Dataset CORe50
  14. Results on Dataset DomainNet
  15. Ablation studies
  16. ...and 3 more sections

Figures (3)

  • Figure 1: The performance of method S-iPromptsp12 under different experimental conditions. Left: the test performance of the model on domain S2 after training only on S2, and after training on both S1 and S2 (covering three datasets). Right: the model undergoes training with S1-S3 and is then tested in the subsequent unseen domain with zero samples on the CDDBp3 dataset, the model undergoes training with S1-S4 and is then tested in the subsequent unseen domain on the CORe50p4 dataset.
  • Figure 2: The framework of disentangled representation and weight fusion for DIL.
  • Figure 3: The sensitivity analysis on $q$ and $\lambda$ with CDDB