BaB-ND: Long-Horizon Motion Planning with Branch-and-Bound and Neural Dynamics

TL;DR

BaB-ND tackles the challenge of planning long trajectories with non-linear neural dynamics by introducing a GPU-accelerated branch-and-bound framework that decomposes the action space into subdomains, uses a modified bound propagation mechanism inspired by alpha-beta-CROWN to bound subproblems, and incorporates a search component to find high-quality feasible action sequences. Distinguishing itself from verification-only approaches, BaB-ND focuses on producing near-optimal trajectories for complex, contact-rich manipulation tasks and scales to large neural dynamics models and architectures, including MLPs and GNNs. The method demonstrates superior open-loop planning performance and improved real-world control across non-prehensile pushing with obstacles, object merging, rope routing, and object sorting, while outperforming standard baselines and MIP-based planners in scalability. Key contributions include (1) a general BaB-based framework for long-horizon planning over neural dynamics, (2) novel branching, bounding, and searching components adapted from neural network verification, and (3) demonstrations of applicability and scalability to deformable objects and graph-based dynamics. The results suggest BaB-ND offers a principled, scalable alternative for planning with learned dynamics in real-world manipulation.

Abstract

Neural-network-based dynamics models learned from observational data have shown strong predictive capabilities for scene dynamics in robotic manipulation tasks. However, their inherent non-linearity presents significant challenges for effective planning. Current planning methods, often dependent on extensive sampling or local gradient descent, struggle with long-horizon motion planning tasks involving complex contact events. In this paper, we present a GPU-accelerated branch-and-bound (BaB) framework for motion planning in manipulation tasks that require trajectory optimization over neural dynamics models. Our approach employs a specialized branching heuristics to divide the search space into subdomains, and applies a modified bound propagation method, inspired by the state-of-the-art neural network verifier alpha-beta-CROWN, to efficiently estimate objective bounds within these subdomains. The branching process guides planning effectively, while the bounding process strategically reduces the search space. Our framework achieves superior planning performance, generating high-quality state-action trajectories and surpassing existing methods in challenging, contact-rich manipulation tasks such as non-prehensile planar pushing with obstacles, object sorting, and rope routing in both simulated and real-world settings. Furthermore, our framework supports various neural network architectures, ranging from simple multilayer perceptrons to advanced graph neural dynamics models, and scales efficiently with different model sizes.

Figures (13)

• Figure 1: Framework overview. (a) Our framework takes scene observations and applies a branch-and-bound (BaB) method to generate robot trajectories using the neural dynamics model (ND). The BaB-ND planner constructs a search tree by branching the problem into sub-domains and then systematically searching only in promising sub-domains by evaluating nodes with a bounding procedure. (b) BaB-ND demonstrates superior long-horizon planning performance compared to existing sampling-based methods and achieves better closed-loop control in the real-world scenarios. We evaluate our framework on various complex planning tasks, including non-prehensile planar pushing with obstacles, object merging, rope routing, and object sorting.
• Figure 2: Seeking $f^*$ with Branch-and-Bound.Sample on input space ${\mathcal{C}}$. $\bullet$: sampled points. ★: the optimal value $f^*$. : the current best upper bound of $f^*$ from sampling. Branch ${\mathcal{C}}$ into ${\mathcal{C}}_1$ and ${\mathcal{C}}_2$. : the linear lower bounds of $f^*$ in subdomains. Discard ${\mathcal{C}}_1$ since its lower bound is larger than $\IfNoValueTF {-NoValue-} {\overline{f}^*} { \IfNoValueTF {-NoValue-} {\overline{f}^{(*)(-NoValue-)}}{\overline{f}^{(*)(-NoValue-)}_{-NoValue-}} }$. : the remaining subdomain to be searched. Search on only ${\mathcal{C}}_2$ and upper bound of $f^*$ is improved. : the previous upper bound. Continue to branch ${\mathcal{C}}_2$ and bound on ${\mathcal{C}}_3$ and ${\mathcal{C}}_4$. Search on ${\mathcal{C}}_3$. The upper bound approaches $f^*$.
• Figure 3: Illustration of the branch-and-bound process. (a) Configuration: we visualize a simplified case of pushing an object toward the target using a 1D action ${\bm{u}}$. We select two keypoints on the object and target and denote the distances as $d_1$ and $d_2$. Then we define our objective function $f({\bm{u}})$ and seek ${\bm{u}}^*$ to minimize it. (b) Branching: we iteratively construct the search tree by splitting, queuing, and pruning nodes (subdomains). In every iteration, only the most promising nodes are prioritized for splitting, cooperating with bounding and searching. (c) Bounding: In every subdomain ${\mathcal{C}}_i$, we obtain the linear lower bound of $f^*$ ($\IfNoValueTF {-NoValue-} {\underline{f}^*} { \IfNoValueTF {-NoValue-} {\underline{f}^{(*)(-NoValue-)}}{\underline{f}^{(*)(-NoValue-)}_{-NoValue-}} }$) via bound propagation. (d) Searching: we search better solutions with smaller objective ($\IfNoValueTF {-NoValue-} {\overline{f}^*} { \IfNoValueTF {-NoValue-} {\overline{f}^{(*)(-NoValue-)}}{\overline{f}^{(*)(-NoValue-)}_{-NoValue-}} }$) on selected subdomains. indicates the most promising subdomain in every iteration. The search space progressively shrinks within the original input domain ${\mathcal{C}}$ with better solutions found and more subdomains pruned. A detailed illustration of our BaB-ND in a simplified robotic manipulation task is provided in Section \ref{['app_subsec:illu']}.
• Figure 4: Optimization result on a synthetic $f({\bm{u}})$ over increasing dimensions $d$. BaB-ND outperforms all baselines in terms of the optimized objective. We run all methods multiple times and visualize the median values with 25$^\text{th}$ and 75$^\text{th}$ percentiles in the shaded area.
• Figure 5: Qualitative results on real-world manipulation tasks. We evaluate our BaB-ND across four complex robotic manipulation tasks, involving non-convex feasible regions, requiring long-horizon planning, and interactions between multiple objects and the deformable rope. For each task, we visualize the initial and target configurations and one successful trajectory. Please refer to our https://robopil.github.io/bab-nd/ for video demonstrations.

Theorems & Definitions (6)

• Lemma B.1: Relaxation of a ReLU layer in CROWN
• proof
• Theorem B.2: CROWN bound propagation on neural network
• proof
• Theorem B.3: Bound Concretization under $\ell_p$ ball Perturbations
• proof