A Neurosymbolic Approach to Adaptive Feature Extraction in SLAM

TL;DR

This work targets robust SLAM in dynamic environments by adapting the feature extraction stage through a neurosymbolic program (nFEX). It introduces a domain-specific language (DSL) and a knowledge-graph framework to jointly select feature extractors (e.g., ORB, SIFT) and optimize their parameters in response to environmental context, using a two-phase training regime that couples neural parameter learning with symbolic reasoning. Empirical results across KITTI, EuRoC, and Holoset show substantial reductions in absolute trajectory error compared with fixed extractors, with ORB achieving up to $90 ext{\%}$ and SIFT up to $66 ext{\%}$ improvements in pose accuracy, and nFEX performing best in more challenging indoor/outdoor settings. The approach demonstrates improved adaptability and efficiency, reducing hand-tuning and enabling better generalization to novel environments, while also highlighting areas for future work in data efficiency, interpretability, and cross-domain generalization.

Abstract

Autonomous robots, autonomous vehicles, and humans wearing mixed-reality headsets require accurate and reliable tracking services for safety-critical applications in dynamically changing real-world environments. However, the existing tracking approaches, such as Simultaneous Localization and Mapping (SLAM), do not adapt well to environmental changes and boundary conditions despite extensive manual tuning. On the other hand, while deep learning-based approaches can better adapt to environmental changes, they typically demand substantial data for training and often lack flexibility in adapting to new domains. To solve this problem, we propose leveraging the neurosymbolic program synthesis approach to construct adaptable SLAM pipelines that integrate the domain knowledge from traditional SLAM approaches while leveraging data to learn complex relationships. While the approach can synthesize end-to-end SLAM pipelines, we focus on synthesizing the feature extraction module. We first devise a domain-specific language (DSL) that can encapsulate domain knowledge on the important attributes for feature extraction and the real-world performance of various feature extractors. Our neurosymbolic architecture then undertakes adaptive feature extraction, optimizing parameters via learning while employing symbolic reasoning to select the most suitable feature extractor. Our evaluations demonstrate that our approach, neurosymbolic Feature EXtraction (nFEX), yields higher-quality features. It also reduces the pose error observed for the state-of-the-art baseline feature extractors ORB and SIFT by up to 90% and up to 66%, respectively, thereby enhancing the system's efficiency and adaptability to novel environments.

Figures (6)

• Figure 1: Our contribution (blue box) within SLAM pipeline.
• Figure 2: A high-level overview of our approach.
• Figure 3: Illustration of feature extraction knowledge graph.
• Figure 4: Example DSL program
• Figure 5: Grammar to parse the DSL for nFEX.