Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ExPrIS: Knowledge-Level Expectations as Priors for Object Interpretation from Sensor Data

Marian Renz, Martin Günther, Felix Igelbrink, Oscar Lima, Martin Atzmueller

TL;DR

This work tackles semantic consistency in robotic perception by injecting knowledge-level expectations as priors into an incremental 3D scene understanding framework. The method builds and updates a dynamic $3DSSG$, fusing contextual priors and ConceptNet-based knowledge through a heterogeneous $GNN$ to bias inference over time. Key contributions include the formalization of context and knowledge expectations, a modular hierarchical graph architecture that separates spatial and semantic relations, and an expectation-biased learning objective that uses a global prior graph to guide local predictions. The approach promises more robust, semantically coherent scene interpretation in real-world robotics, with planned integration on a mobile platform and future work on larger knowledge graphs and open-vocabulary reasoning.

Abstract

While deep learning has significantly advanced robotic object recognition, purely data-driven approaches often lack semantic consistency and fail to leverage valuable, pre-existing knowledge about the environment. This report presents the ExPrIS project, which addresses this challenge by investigating how knowledge-level expectations can serve as to improve object interpretation from sensor data. Our approach is based on the incremental construction of a 3D Semantic Scene Graph (3DSSG). We integrate expectations from two sources: contextual priors from past observations and semantic knowledge from external graphs like ConceptNet. These are embedded into a heterogeneous Graph Neural Network (GNN) to create an expectation-biased inference process. This method moves beyond static, frame-by-frame analysis to enhance the robustness and consistency of scene understanding over time. The report details this architecture, its evaluation, and outlines its planned integration on a mobile robotic platform.

ExPrIS: Knowledge-Level Expectations as Priors for Object Interpretation from Sensor Data

TL;DR

This work tackles semantic consistency in robotic perception by injecting knowledge-level expectations as priors into an incremental 3D scene understanding framework. The method builds and updates a dynamic , fusing contextual priors and ConceptNet-based knowledge through a heterogeneous to bias inference over time. Key contributions include the formalization of context and knowledge expectations, a modular hierarchical graph architecture that separates spatial and semantic relations, and an expectation-biased learning objective that uses a global prior graph to guide local predictions. The approach promises more robust, semantically coherent scene interpretation in real-world robotics, with planned integration on a mobile platform and future work on larger knowledge graphs and open-vocabulary reasoning.

Abstract

While deep learning has significantly advanced robotic object recognition, purely data-driven approaches often lack semantic consistency and fail to leverage valuable, pre-existing knowledge about the environment. This report presents the ExPrIS project, which addresses this challenge by investigating how knowledge-level expectations can serve as to improve object interpretation from sensor data. Our approach is based on the incremental construction of a 3D Semantic Scene Graph (3DSSG). We integrate expectations from two sources: contextual priors from past observations and semantic knowledge from external graphs like ConceptNet. These are embedded into a heterogeneous Graph Neural Network (GNN) to create an expectation-biased inference process. This method moves beyond static, frame-by-frame analysis to enhance the robustness and consistency of scene understanding over time. The report details this architecture, its evaluation, and outlines its planned integration on a mobile robotic platform.
Paper Structure (6 sections, 4 figures)

This paper contains 6 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Semantic Mapping with 3D Semantic Scene Graphs
  3. Knowledge-Based Expectation Generation
  4. Expectation-Biased Machine Learning
  5. Planned Integration and Evaluation on a Robot Platform
  6. Discussion and Outlook

Figures (4)

  • Figure 1: The ExPrIS/LIEREx semantic mapping architecture. This paper focuses on the ExPrIS and semantic map parts.
  • Figure 2: Hierarchical scene graph based on contact and support relationships.
  • Figure 3: model for expectation integration
  • Figure 4: The Mobipick robot