LLMPC: Large Language Model Predictive Control

TL;DR

This work reframes LLM-based planning within Model Predictive Control by treating LLMs as approximate optimizers of a planning objective $C(s_t,\dots,s_{t+H},a_t,\dots,a_{t+H})$. It proposes LLMPC, which samples multiple LL-generated plans, evaluates them with a cost function, and selects the best plan for execution before replanning, enabling explicit optimization over a horizon $H$. The authors validate the approach on a mass-spring control problem and two planning benchmarks (trip planning and meeting planning), showing that plan sampling and iterative replanning consistently improve performance over single-round prompting, with larger gains as problem complexity increases. The results underscore the practical value of combining the flexibility of LLMs with the rigor of MPC for structured planning.

Abstract

Recent advancements in prompting techniques for Large Language Models (LLMs) have improved their reasoning, planning, and action abilities. This paper examines these prompting techniques through the lens of model predictive control (MPC). We show that LLMs act as implicit planning cost function minimizers when planning prompts are used. We propose a unified MPC framework for planning with LLMs and demonstrate improved performance over few shot prompting on several planning benchmarks.

Figures (4)

• Figure 1: State, control and cost trajectories for MPC and LLMPC algorithm on spring-mass problem.
• Figure 2: Ratio of LLMPC optimized planning objective value to MPC optimized planning objective value for mass-spring system over a range of states.
• Figure 3: Trip planning success rate grouped by number of cities
• Figure 4: Comparison of LLMPC against GPT-4o with few shot prompt on meeting planning problem, success rate segmented by number of meetings.