Towards Arbitrary Motion Completing via Hierarchical Continuous Representation

TL;DR

The paper addresses the limitation of fixed-frame-rate motion representations by proposing PA-HiRes, a continuous implicit representation for human motion across arbitrary frame rates. It introduces a physics-informed architecture that combines Multi-Scale Temporal Encoding (MSTE) with a Parametric Activation Induced Decoding (PAID), employing Fourier-based activations within an MLP to capture high-frequency motion dynamics. A skeleton-aware embedding and a progressive cross-scale attention mechanism enable robust, multi-scale temporal modeling, while a velocity-consistency loss ensures physically plausible motion. Empirical results on HumanML3D, LaFAN1, and CMU Mocap demonstrate strong interpolation and inbetweening performance, with added capability for forward and backward extrapolation at unseen temporal inputs, highlighting practical benefits for high-fidelity motion synthesis at any frame rate.

Abstract

Physical motions are inherently continuous, and higher camera frame rates typically contribute to improved smoothness and temporal coherence. For the first time, we explore continuous representations of human motion sequences, featuring the ability to interpolate, inbetween, and even extrapolate any input motion sequences at arbitrary frame rates. To achieve this, we propose a novel parametric activation-induced hierarchical implicit representation framework, referred to as NAME, based on Implicit Neural Representations (INRs). Our method introduces a hierarchical temporal encoding mechanism that extracts features from motion sequences at multiple temporal scales, enabling effective capture of intricate temporal patterns. Additionally, we integrate a custom parametric activation function, powered by Fourier transformations, into the MLP-based decoder to enhance the expressiveness of the continuous representation. This parametric formulation significantly augments the model's ability to represent complex motion behaviors with high accuracy. Extensive evaluations across several benchmark datasets demonstrate the effectiveness and robustness of our proposed approach.

Figures (7)

• Figure 1: Illustration of complex physical characteristics of human motion sequence across different frame rates. Human motion sequences inherently exhibit rich and intricate physical properties, such as velocity and acceleration, which vary significantly with changes in frame rate. While the positional data of motions sampled at different FPS (frames per second) may appear visually similar, their underlying physical dynamics, particularly temporal derivatives like velocity and acceleration, can differ substantially.
• Figure 2: (a) The overall framework of our PA-HiRes. (b) The simple illustration and visualization of the proposed parametric activation function.
• Figure 3: Visualization results under $\times$2, $\times$3, $\times$4, and $\times$5 interpolation. The blue motions represent the known input, while the red motions indicate the results generated by our PA-HiRes.
• Figure 4: Visualization of the frequency distribution trends of activation functions across MLPs.
• Figure 5: Ablation study between different choices of $\lambda$.