Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safe and Trustworthy Robot Pathfinding with BIM, MHA*, and NLP

Mani Amani, Reza Akhavian

TL;DR

This paper addresses safe path planning for construction robots operating in dynamic job sites by fusing BIM-based spatial and semantic information with a multi-heuristic A* search (MHA*) and Artificial Potential Fields (APF). It introduces a BIM-to-graph pipeline, APF-based heuristics, and an LLM-driven weighting mechanism to adapt obstacle avoidance while preserving path efficiency, achieving improved obstacle clearance and explainability. The approach is validated in a 2D BIM model, showing that MHA* with LLM integration can substantially increase clearance to obstacles with modest or acceptable changes to path length, and that explanations generated by the LLM enhance trust. The work highlights practical benefits like reduced sensor burden, flexible risk tuning by per-object or per-room coefficients, and a framework extendable to 3D and online SLAM for real-time adaptability in construction environments.

Abstract

Construction robots have gained significant traction in recent years in research and development. However, the application of industrial robots has unique challenges. Dynamic environments, domain-specific tasks, and complex localization and mapping are significant obstacles in their development. In construction job sites, moving objects and complex machinery can make pathfinding a difficult task due to the possibility of object collisions. Existing methods such as simultaneous localization and mapping are viable solutions to this problem, however, due to the precision and data quality required by the sensors and the processing of the information, they can be very computationally expensive. We propose using spatial and semantic information in building information modeling (BIM) to develop domain-specific pathfinding strategies. In this work, we integrate a multi-heuristic A* (MHA*) algorithm using APFs from the BIM spatial information and process textual information from the BIM using large language models (LLMs) to adjust the algorithm for dynamic object avoidance. We show increased robot object proximity by 80% while maintaining similar path lengths.

Safe and Trustworthy Robot Pathfinding with BIM, MHA*, and NLP

TL;DR

This paper addresses safe path planning for construction robots operating in dynamic job sites by fusing BIM-based spatial and semantic information with a multi-heuristic A* search (MHA*) and Artificial Potential Fields (APF). It introduces a BIM-to-graph pipeline, APF-based heuristics, and an LLM-driven weighting mechanism to adapt obstacle avoidance while preserving path efficiency, achieving improved obstacle clearance and explainability. The approach is validated in a 2D BIM model, showing that MHA* with LLM integration can substantially increase clearance to obstacles with modest or acceptable changes to path length, and that explanations generated by the LLM enhance trust. The work highlights practical benefits like reduced sensor burden, flexible risk tuning by per-object or per-room coefficients, and a framework extendable to 3D and online SLAM for real-time adaptability in construction environments.

Abstract

Construction robots have gained significant traction in recent years in research and development. However, the application of industrial robots has unique challenges. Dynamic environments, domain-specific tasks, and complex localization and mapping are significant obstacles in their development. In construction job sites, moving objects and complex machinery can make pathfinding a difficult task due to the possibility of object collisions. Existing methods such as simultaneous localization and mapping are viable solutions to this problem, however, due to the precision and data quality required by the sensors and the processing of the information, they can be very computationally expensive. We propose using spatial and semantic information in building information modeling (BIM) to develop domain-specific pathfinding strategies. In this work, we integrate a multi-heuristic A* (MHA*) algorithm using APFs from the BIM spatial information and process textual information from the BIM using large language models (LLMs) to adjust the algorithm for dynamic object avoidance. We show increased robot object proximity by 80% while maintaining similar path lengths.

Paper Structure

This paper contains 19 sections, 5 equations, 12 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. BIM localization
  4. A* Search
  5. Artificial Potential Fields
  6. Trust and Reasoning Capabilities in LLMs
  7. Methodology
  8. Graph Generation
  9. Attractive Potential
  10. Repulsive Potential
  11. MHA* Search
  12. LLM
  13. Experiments and Validation
  14. MHA* vs classic A*
  15. LLM Integration
  16. ...and 4 more sections

Figures (12)

  • Figure 1: Graph Generation using Unity's Navigation Package
  • Figure 2: Moore's Neighborhood Graph
  • Figure 3: Generated Grid in Unity with the Moore's Neighborhood Node Connections
  • Figure 4: Potential field before Gaussian Smoothing
  • Figure 5: Potential Field after Gaussian Smoothing
  • ...and 7 more figures