Neural Force Field: Few-shot Learning of Generalized Physical Reasoning

TL;DR

Neural Force Field (NFF) introduces a physics-grounded, force-field representation that extends Neural ODEs to learn explicit continuous interactions via a neural operator on a relation graph. By integrating learned force fields with a second-order ODE solver, NFF achieves accurate trajectory prediction and enables forward and backward planning with few-shot learning, transferring across unseen scenarios. The approach demonstrates strong generalization on I-PHYRE, N-body, and PHYRE benchmarks, outperforming state-of-the-art baselines in both prediction and planning tasks while requiring far less training data. This work suggests that explicit physics-inspired representations can bridge human-like intuitive physics and data-driven learning, offering efficient, interpretable, and adaptable physical world models. The results also highlight the value of precise ODE grounding and neural operators for robust cross-domain generalization and rapid interactive refinement.

Abstract

Physical reasoning is a remarkable human ability that enables rapid learning and generalization from limited experience. Current AI models, despite extensive training, still struggle to achieve similar generalization, especially in Out-of-distribution (OOD) settings. This limitation stems from their inability to abstract core physical principles from observations. A key challenge is developing representations that can efficiently learn and generalize physical dynamics from minimal data. Here we present Neural Force Field (NFF), a framework extending Neural Ordinary Differential Equation (NODE) to learn complex object interactions through force field representations, which can be efficiently integrated through an Ordinary Differential Equation (ODE) solver to predict object trajectories. Unlike existing approaches that rely on discrete latent spaces, NFF captures fundamental physical concepts such as gravity, support, and collision in continuous explicit force fields. Experiments on three challenging physical reasoning tasks demonstrate that NFF, trained with only a few examples, achieves strong generalization to unseen scenarios. This physics-grounded representation enables efficient forward-backward planning and rapid adaptation through interactive refinement. Our work suggests that incorporating physics-inspired representations into learning systems can help bridge the gap between artificial and human physical reasoning capabilities.

Figures (12)

• Figure 1: The framework of nff. (a) nff models complex physical interactions by constructing dynamic interaction graphs from scenes and performing continuous integration on force fields. (b) The force fields are inferred by a neural operator that takes object interactions as input. (c) After learning from a few examples, nff can capture various physical concepts represented by force fields and generalize to unseen ood scenarios.
• Figure 2: Comparison between discrete latent transition in traditional interaction modeling and continuous explicit transition in nff. The figures in the first line compare the predicted discrete steps and continuous trajectories, in which the discrete decoding cannot explain how the green ball goes through the black wall. In the second line, the black arrows indicate velocity vectors and the red and green arrows indicate force vectors. Traditional interaction models such as in and SlotFormer decode discrete frames from a learnable decoder. In contrast, nff decodes trajectories by integrating the continuous force using an ODE solver, which benefits learning detailed interactions such as collisions.
• Figure 3: Visualization of learned force fields after few-shot learning. (a) nff successfully inverts force fields across different templates on PHYRE, consisting of physical behaviors like falling under gravity, sliding with friction, and colliding with momentum transfer. (b) The learned gravitational field distributions in N-body also show accurate force field reconstruction.
• Figure 4: Trajectory predictions on unseen scenarios after few-shot learning. After learning from 100 and 200 trajectories on I-PHYRE and N-body systems, respectively, our nff predictions closely match the ground truth behaviors across diverse scenarios, from rigid body interactions to gravitational dynamics. In contrast, other baselines fail to predict physically plausible dynamics. Additional visualizations are provided in \ref{['sec:supp:vis']}.
• Figure 5: Trajectory Predictions on PHYRE. We compare the vision-based nff, trained on $12{,}000$ trajectories, with SOTA methods trained on $3.2$ million trajectories. Prior methods decoding objects from latent spaces often suffer from object inconsistency. For example, in the last column, RPIN mistakenly transforms a gray cup into a gray ball, indicating overfitting to training shapes. SlotFormer also shows object disappearance in cross-scenario tasks. In contrast, nff produces more accurate predictions while maintaining object consistency.