Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FOGMACHINE -- Leveraging Discrete-Event Simulation and Scene Graphs for Modeling Hierarchical, Interconnected Environments under Partial Observations from Mobile Agents

Lars Ohnemus, Nils Hantke, Max Weißer, Kai Furmans

TL;DR

FOGMACHINE addresses the challenge of modeling dynamic, hierarchical environments under partial observability for embodied AI. It fuses dynamic scene graphs with discrete-event simulation to capture stochastic object dynamics, agent observations, and interactions at scale. The framework enables uncertainty propagation, planning under incomplete perception, and emergent multi-agent behavior, with an open-source implementation and OpenStreetMap-based urban scenarios. This work provides a scalable, configurable tool for benchmarking, model training, and advancing embodied AI in complex, uncertain environments.

Abstract

Dynamic Scene Graphs (DSGs) provide a structured representation of hierarchical, interconnected environments, but current approaches struggle to capture stochastic dynamics, partial observability, and multi-agent activity. These aspects are critical for embodied AI, where agents must act under uncertainty and delayed perception. We introduce FOGMACHINE , an open-source framework that fuses DSGs with discrete-event simulation to model object dynamics, agent observations, and interactions at scale. This setup enables the study of uncertainty propagation, planning under limited perception, and emergent multi-agent behavior. Experiments in urban scenarios illustrate realistic temporal and spatial patterns while revealing the challenges of belief estimation under sparse observations. By combining structured representations with efficient simulation, FOGMACHINE establishes an effective tool for benchmarking, model training, and advancing embodied AI in complex, uncertain environments.

FOGMACHINE -- Leveraging Discrete-Event Simulation and Scene Graphs for Modeling Hierarchical, Interconnected Environments under Partial Observations from Mobile Agents

TL;DR

FOGMACHINE addresses the challenge of modeling dynamic, hierarchical environments under partial observability for embodied AI. It fuses dynamic scene graphs with discrete-event simulation to capture stochastic object dynamics, agent observations, and interactions at scale. The framework enables uncertainty propagation, planning under incomplete perception, and emergent multi-agent behavior, with an open-source implementation and OpenStreetMap-based urban scenarios. This work provides a scalable, configurable tool for benchmarking, model training, and advancing embodied AI in complex, uncertain environments.

Abstract

Dynamic Scene Graphs (DSGs) provide a structured representation of hierarchical, interconnected environments, but current approaches struggle to capture stochastic dynamics, partial observability, and multi-agent activity. These aspects are critical for embodied AI, where agents must act under uncertainty and delayed perception. We introduce FOGMACHINE , an open-source framework that fuses DSGs with discrete-event simulation to model object dynamics, agent observations, and interactions at scale. This setup enables the study of uncertainty propagation, planning under limited perception, and emergent multi-agent behavior. Experiments in urban scenarios illustrate realistic temporal and spatial patterns while revealing the challenges of belief estimation under sparse observations. By combining structured representations with efficient simulation, FOGMACHINE establishes an effective tool for benchmarking, model training, and advancing embodied AI in complex, uncertain environments.

Paper Structure

This paper contains 18 sections, 10 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Need for dsg-Generative Models
  4. Existing Generative Models for dsg
  5. Methods
  6. Dynamic Scene Graph
  7. Processes
  8. Mobile Agents, Tasks and Observations
  9. Discrete-Event Simulation
  10. Stochastic Models
  11. Monte Carlo Sampling
  12. Application: Autonomous Delivery Robots
  13. Scene Graph Generation
  14. Modeling and Parametrization
  15. Scenarios
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Overview of FOGMACHINE : The environment is represented as a DSG (c), updated by arrival and deletion events from processes (a). Mobile agents (b) perform tasks in this dsg-structured environment and, in doing so, observe parts of it, yielding the observed DSG (d).
  • Figure 2: The local subgraph around place , located near a school, is shown in (a). Panel (b) depicts an observation heatmap, where small black circles indicate few and large red-to-white circles indicate many agent observations, highlighting uneven coverage along routes. Panels (c–e) report bicycle arrival rates at nodes – , comparing true distributions obtained from long-run Monte Carlo sampling (pink) with observed beliefs averaged over five replications (green, error bars are standard deviation). While node is in the vicinity of a school, nodes and are located in purely residential areas.
  • Figure 3: Daily Trends: Number of objects alive in the simulation for a) Bruchsal, b) Wenningstedt, and c) Trier.