NSP: A Neuro-Symbolic Natural Language Navigational Planner

TL;DR

A neuro-symbolic framework for path planning from natural language inputs called NSP is proposed that leverages the neural reasoning abilities of LLMs to craft symbolic representations of the environment and a symbolic path planning algorithm is obtained.

Abstract

Path planners that can interpret free-form natural language instructions hold promise to automate a wide range of robotics applications. These planners simplify user interactions and enable intuitive control over complex semi-autonomous systems. While existing symbolic approaches offer guarantees on the correctness and efficiency, they struggle to parse free-form natural language inputs. Conversely, neural approaches based on pre-trained Large Language Models (LLMs) can manage natural language inputs but lack performance guarantees. In this paper, we propose a neuro-symbolic framework for path planning from natural language inputs called NSP. The framework leverages the neural reasoning abilities of LLMs to i) craft symbolic representations of the environment and ii) a symbolic path planning algorithm. Next, a solution to the path planning problem is obtained by executing the algorithm on the environment representation. The framework uses a feedback loop from the symbolic execution environment to the neural generation process to self-correct syntax errors and satisfy execution time constraints. We evaluate our neuro-symbolic approach using a benchmark suite with 1500 path-planning problems. The experimental evaluation shows that our neuro-symbolic approach produces 90.1% valid paths that are on average 19-77% shorter than state-of-the-art neural approaches.

Figures (6)

• Figure 1: Overview of the NSP framework. In Step 1, an arbitrary free-form path planning problem $\mathcal{D} = \braket{\mathcal{E+S+C}}$ is mapped to a symbolic representation consisting of graph, a path planning algorithm, and the parameters accepted by the algorithm. These objects are passed to Step 2 to be interpreted as $\mathcal{F(G, I)}$. In the event that this triggers a timeout or interpreter error, the feedback loop is triggered to improve the plan (repeat Step 1). If the feedback loop is not triggered, the symbolic path plan is returned.
• Figure 2: The prompt template used within the neuro-symbolic translation is shown above. The template captures the problem description, the expect outputs, and the API of graph library.
• Figure 3: The flow of information in the NSP framework. The Neuro-Symbolic Translation can be found in Section \ref{['subsec:4c']}.
• Figure 4: An error message detected by the interpreter is appended to the prompt prior to repeating neuro-symbolic translation (step 1).
• Figure 5: Python code resulting from example in Section \ref{['subsec:4c']}. Segment 1: Returns $\mathcal{G}$, a graph that is perfectly consistent with the natural language description of the house contained in the input. Segment 2: Takes $\braket{\mathcal{G, A}}$ as input and returns the symbolic path plan $\mathcal{P}$ as output. Segment 3: $\braket{\mathcal{A}}$, the parameters defined by the model, which are passed to the function defined in segment 2. Segment 4: The code contained in segments 1 through 3 is executed, and its output is stored in the variable generated_path = $\mathcal{P}$ .