AnyNav: Visual Neuro-Symbolic Friction Learning for Off-road Navigation

TL;DR

AnyNav addresses off-road navigation by learning continuous terrain friction from vision and grounding it in physics via neuro-symbolic reasoning. It decouples perception from dynamics, enabling self-supervised, sim-to-real transfer through bilevel optimization. The system builds a physics-informed terrain map and uses it to plan feasible, efficient paths and speed profiles across diverse terrains and vehicle platforms. Experiments in simulation and on real robots demonstrate improved trajectory accuracy and robust navigation under unknown, challenging environments.

Abstract

Off-road navigation is critical for a wide range of field robotics applications from planetary exploration to disaster response. However, it remains a longstanding challenge due to unstructured environments and the inherently complex terrain-vehicle interactions. Traditional physics-based methods struggle to accurately capture the nonlinear dynamics underlying these interactions, while purely data-driven approaches often overfit to specific motion patterns, vehicle geometries, or platforms, limiting their generalization in diverse, real-world scenarios. To address these limitations, we introduce AnyNav, a vision-based friction estimation and navigation framework grounded in neuro-symbolic principles. Our approach integrates neural networks for visual perception with symbolic physical models for reasoning about terrain-vehicle dynamics. To enable self-supervised learning in real-world settings, we adopt the imperative learning paradigm, employing bilevel optimization to train the friction network through physics-based optimization. This explicit incorporation of physical reasoning substantially enhances generalization across terrains, vehicle types, and operational conditions. Leveraging the predicted friction coefficients, we further develop a physics-informed navigation system capable of generating physically feasible, time-efficient paths together with corresponding speed profiles. We demonstrate that AnyNav seamlessly transfers from simulation to real-world robotic platforms, exhibiting strong robustness across different four-wheeled vehicles and diverse off-road environments.

Figures (24)

• Figure 1: We tackle the challenges of off-road navigation by developing AnyNav, a neuro-symbolic framework for predicting friction coefficient and integrating these predictions as physical constraints in path and speed planning for reliable navigation.
• Figure 2: AnyNav comprises a neuro-symbolic friction learning module and a physics-informed navigation module. The neuro-symbolic module has a neural network to predict friction coefficients from visual inputs, guided by symbolic physics reasoning for self-supervised training. The inputs are captured by camera and LiDAR before entering a terrain patch, while training occurs afterward using the dynamics observed on that patch. The navigation module builds terrain property maps online and leverages the predicted friction information to generate physically feasible and efficient path and speed profiles for off-road driving.
• Figure 3: An example of the Stribeck curve. The plot illustrates the friction coefficient of a terrain material with respect to tire sliding speed. The shape of the curve is governed by four coefficients, which vary across different terrain types.
• Figure 4: The configurable vehicle model. The blue dots represent the mass distribution, with their size proportional to the magnitude of the mass at each point. $M$ is the total mass, $\mathbf{pt}_{\text{COM}}$ is the center of mass, $\mathbf{I}$ is the rotational inertia matrix, $\mathbf{c}$ are wheel contact points, and $r$ is wheel radius.
• Figure 5: Architecture of the AnyNav system. Blue boxes represent ROS nodes, each operates in parallel processes. The odometry and mapping stacks run at 10Hz, while the planning stack runs 2Hz path and speed planning and 10Hz PID control.