Neural Implicit Flow Fields for Spatio-Temporal Motion Mapping

TL;DR

This work tackles the challenge of modeling site-specific human motion for robot operation by introducing NeMo-map, a continuous spatio-temporal Map of Dynamics (MoD) built with implicit neural representations. It outputs parameters of a Semi-Wrapped Gaussian Mixture Model (SWGMM) for any (location, time) query, enabling smooth, multimodal motion priors without discretization. Empirical results on the ATC dataset show that NeMo-map achieves lower negative log-likelihood and substantially faster map construction than discretized baselines like CLiFF-map and STeF-map, while preserving temporal variability and alignment with environmental topology. The approach offers a practical, scalable prior for planning and long-term prediction in dynamic environments, with potential extensions to online updating and real-time adaptation.

Abstract

Safe and efficient robot operation in complex human environments can benefit from good models of site-specific motion patterns. Maps of Dynamics (MoDs) provide such models by encoding statistical motion patterns in a map, but existing representations use discrete spatial sampling and typically require costly offline construction. We propose a continuous spatio-temporal MoD representation based on implicit neural functions that directly map coordinates to the parameters of a Semi-Wrapped Gaussian Mixture Model. This removes the need for discretization and imputation for unevenly sampled regions, enabling smooth generalization across both space and time. Evaluated on a large public dataset with long-term real-world people tracking data, our method achieves better accuracy of motion representation and smoother velocity distributions in sparse regions while still being computationally efficient, compared to available baselines. The proposed approach demonstrates a powerful and efficient way of modeling complex human motion patterns.

Figures (5)

• Figure 1: Indicative applications of Maps of dynamics (MoD) for motion planning and human motion prediction. Left: Illustration of how MoDs can support socially aware motion planning. The robot (red diamond) navigates toward the goal (green cross) in the presence of two opposing human flows, with the underlying MoD shown as colored arrows. In this scenario, the oncoming pedestrian flow moving in the opposing direction is initially outside the robot’s observation radius (grey dashed circle). Without guidance from the MoD, the planner initially takes a direct path to the goal but eventually becomes stuck and collides with the oncoming flow. In contrast, when informed by the MoD, the robot aligns its trajectory with the motion patterns and reaches the goal efficiently and safely. Right: Human motion prediction with a 60s horizon. The red line represents the ground truth trajectory and the green line represents the observed trajectory. With MoD guidance, CLiFF-LHMP zhu2023clifflhmp makes more accurate and realistic predictions than deep learning methods. While the trajectories predicted by Social LSTM alahi2016social, TUTR shi2023tutr and MID gu2022mid often are unfeasible (e.g., crossing the walls), CLiFF-LHMP predictions implicitly follow the topology of the environment.
• Figure 2: Probability density of a Semi-Wrapped Gaussian Mixture Model (SWGMM) with two components, visualized on a cylinder. Orientation $\theta$ is wrapped around the circular axis, while speed $\rho$ extends along the vertical axis. The representation allows joint modeling of angular (orientation) and linear (speed) variables, capturing multimodality in motion patterns.
• Figure 3: Method overview. A spatio-temporal query $(\mathbf{x},t)$ is mapped to parameters of a Semi-Wrapped Gaussian Mixture Model (SWGMM). The spatial coordinate $\mathbf{x}$ is used to interpolate features from a learnable spatial grid $\mathbf{G}_s$, and the temporal coordinate $t$ is encoded using a SIREN network. The spatial features $\mathbf{f}_s(\mathbf{x})$, temporal encoding $\mathbf{f}_t(t)$, and raw coordinates are concatenated and passed through an MLP, which outputs the parameters of a SWGMM, providing a continuous, multimodal probabilistic representation of motion dynamics at the queried location and time.
• Figure 4: Layout of the ATC dataset environment brscic2013person, showing the main east corridor and open areas annotated with semantic information such as entry and exit points, shops, seating areas, and stairs.
• Figure 5: NeMo-map in the ATC dataset, for 09:00 (left), 11:00 (middle) and 18:00 (right), showing changes of motion patterns throughout the day. Predicted Semi-Wrapped Gaussian Mixture Models (SWGMMs) are visualized. At each location, arrow color encodes orientation and arrow length encodes speed. The top row shows multimodality by rendering all SWGMM components with transparency proportional to their weights, while the bottom row more clearly shows the dominant flow, only displaying the mixture component with the largest weight.