CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow

TL;DR

CacheFlow tackles the bottleneck of density estimation in stochastic 3D human motion prediction by separating the modeling into an unconditional flow-based transformation and a lightweight conditional base density. By precomputing and caching the results of the unconditional flow, inference reduces to evaluating a compact base density and retrieving from a stored triplet cache, achieving about $1$ ms latency. Empirically, CacheFlow attains density estimation accuracy comparable to or better than state-of-the-art methods while being markedly faster than VAE- or diffusion-based approaches on Human3.6M and AMASS. The method enables real-time, probabilistic motion prediction with explicit density, and the authors plan to release code and models for reproducibility. A notable limitation is reliance on precomputed coverage, suggesting future work to broaden the precomputation strategy and extend to other regression tasks.

Abstract

Many density estimation techniques for 3D human motion prediction require a significant amount of inference time, often exceeding the duration of the predicted time horizon. To address the need for faster density estimation for 3D human motion prediction, we introduce a novel flow-based method for human motion prediction called CacheFlow. Unlike previous conditional generative models that suffer from time efficiency, CacheFlow takes advantage of an unconditional flow-based generative model that transforms a Gaussian mixture into the density of future motions. The results of the computation of the flow-based generative model can be precomputed and cached. Then, for conditional prediction, we seek a mapping from historical trajectories to samples in the Gaussian mixture. This mapping can be done by a much more lightweight model, thus saving significant computation overhead compared to a typical conditional flow model. In such a two-stage fashion and by caching results from the slow flow model computation, we build our CacheFlow without loss of prediction accuracy and model expressiveness. This inference process is completed in approximately one millisecond, making it 4 times faster than previous VAE methods and 30 times faster than previous diffusion-based methods on standard benchmarks such as Human3.6M and AMASS datasets. Furthermore, our method demonstrates improved density estimation accuracy and comparable prediction accuracy to a SOTA method on Human3.6M. Our code and models will be publicly available.

Figures (4)

• Figure 1: Previous methods vs. Our CacheFlow. Previous methods of stochastic motion prediction generate multiple future motions by sampling noises from the fixed source in an ad hoc manner. In contrast, CacheFlow uses the precomputed and cached latent-motion pairs from an unconditional flow-based generative model. Thus, the computation of the unconditional flow can be skipped at inference. One can achieve fast inference by selecting predictions from these cached pairs.
• Figure 2: Overview of our CacheFlow. Our method utilizes the unconditional flow-based model $f_\theta$. This $f_\theta$ maps the lightweight conditional base density $q_\phi(\bm{z}|\bm{c})$ into future motion density $p(\bm{x}|\bm{c})$. In this formulation, the flow-based model is independent of past motions. Thus, we can precompute the unconditional flow-based model. These results are cached as K triplets as shown in (a). Due to the precomputation, we can skip the inference of $f_\theta$ and omit a large portion of the entire computation. At inference, density estimation is achieved by only evaluating the lightweight conditional base density $q_\phi(\bm{z}_k|\bm{c})$ and combining it with the stored K triplets as shown in (b).
• Figure 3: Qualitative Comparison on AMASS dataset.
• Figure 4: Visualization of future motion densities by CacheFlow. The estimated densities for four different motion sequences are visualized. We used UMAP to project these future motions onto a 2D space. Each dot represents an evaluated future motion, and the color of each dot indicates its probability, as shown in the side color bar. The red stars represent the projected ground truth future motions.