Semantic Intelligence: Integrating GPT-4 with A Planning in Low-Cost Robotics

TL;DR

Classical robot navigation relies on hardcoded state machines and purely geometric planners, which limits semantics. This paper presents a hybrid GPT-4 and A* pipeline that uses LLM reasoning to interpret high-level instructions and environmental cues while retaining A* for feasible path generation on a low-cost robot. Empirical results on a Petoi Bittle system show strong semantic task performance (approximately 96.7% overall) and effective sequential planning with dynamic obstacle buffering, albeit with slower planning times compared to pure A*. The work demonstrates that accessible hardware can achieve context-aware navigation by offloading semantic reasoning to an external LLM, enabling flexible task execution and prompting future improvements in safety monitoring and real-time performance.

Abstract

Classical robot navigation often relies on hardcoded state machines and purely geometric path planners, limiting a robot's ability to interpret high-level semantic instructions. In this paper, we first assess GPT-4's ability to act as a path planner compared to the A* algorithm, then present a hybrid planning framework that integrates GPT-4's semantic reasoning with A* on a low-cost robot platform operating on ROS2 Humble. Our approach eliminates explicit finite state machine (FSM) coding by using prompt-based GPT-4 reasoning to handle task logic while maintaining the accurate paths computed by A*. The GPT-4 module provides semantic understanding of instructions and environmental cues (e.g., recognizing toxic obstacles or crowded areas to avoid, or understanding low-battery situations requiring alternate route selection), and dynamically adjusts the robot's occupancy grid via obstacle buffering to enforce semantic constraints. We demonstrate multi-step reasoning for sequential tasks, such as first navigating to a resource goal and then reaching a final destination safely. Experiments on a Petoi Bittle robot with an overhead camera and Raspberry Pi Zero 2W compare classical A* against GPT-4-assisted planning. Results show that while A* is faster and more accurate for basic route generation and obstacle avoidance, the GPT-4-integrated system achieves high success rates (96-100%) on semantic tasks that are infeasible for pure geometric planners. This work highlights how affordable robots can exhibit intelligent, context-aware behaviors by leveraging large language model reasoning with minimal hardware and no fine-tuning.

Figures (6)

• Figure 1: Diagram of our evaluations and system design. Evaluation 1 tested GPT-4's performance in generating a route around obstacles to a desired goal against an A* path planning algorithm. Both systems were given identical routes for consistency to measure accuracy and time. Three courses were made for this assessment, increasing in difficulty from one obstacle to a max of three. Evaluations 2 and 3 enabled an A* path planner to generate routes to all possible goals, then giving GPT-4 a task to select the appropriate route that achieved the desired result of the prompt.
• Figure 2: Flowchart of the system design and ROS2 nodes demonstrating the flow of information to the GPT-4 Node for semantic evaluation and route selection given a prompted task.
• Figure 3: Three courses were created to measure the effectiveness of GPT-4 for path planning compared to A*. The courses increased in complexity from one obstacle to three, measuring GPT-4's ability to understand the location of multiple obstacles while generating a safe route to the goal.
• Figure 4: Figure 4: LLM executes sequential tasks where the first task is to go to the goal with a "long skinny object that is a resource" (challenging GPT-4 to reinterpret an obstacle as a resource) and the second task is to travel safely to the destination and apply a buffer if the route contains obstacles. Notice the buffer increase for the obstacles between stage 1 and stage 2. This indicates GPT-4 recognized the second task of travelling to the final goal was more hazardous, resulting in the declaration of an increased buffer size to force A* to calculate a safer route and avoid contact with the obstacles.
• Figure 5: COMPARISON OF CLASSICAL A* VS GPT-4 AUGMENTED PLANNING ON OBSTACLE COURSES