Temporal Dynamics Decoupling with Inverse Processing for Enhancing Human Motion Prediction

TL;DR

Problem: improve human motion prediction by leveraging historical information without conflicting auxiliary reconstruction signals. Approach: Temporal Decoupling Decoding with Inverse Processing ($TD^2IP$) uses two specialized decoders for reconstruction and prediction and adds an inverse processing path to strengthen bidirectional temporal correlations, all operating on a shared encoded representation $M$. Contributions: (1) a dual-decoder TD^2IP architecture, (2) a novel inverse processing training signal, and (3) extensive experiments on standard benchmarks showing improved $MPJPE$ across multiple baselines. Significance: the method is straightforward to integrate with existing models and yields robust gains in prediction accuracy, advancing practical human motion prediction systems.

Abstract

Exploring the bridge between historical and future motion behaviors remains a central challenge in human motion prediction. While most existing methods incorporate a reconstruction task as an auxiliary task into the decoder, thereby improving the modeling of spatio-temporal dependencies, they overlook the potential conflicts between reconstruction and prediction tasks. In this paper, we propose a novel approach: Temporal Decoupling Decoding with Inverse Processing (\textbf{$TD^2IP$}). Our method strategically separates reconstruction and prediction decoding processes, employing distinct decoders to decode the shared motion features into historical or future sequences. Additionally, inverse processing reverses motion information in the temporal dimension and reintroduces it into the model, leveraging the bidirectional temporal correlation of human motion behaviors. By alleviating the conflicts between reconstruction and prediction tasks and enhancing the association of historical and future information, \textbf{$TD^2IP$} fosters a deeper understanding of motion patterns. Extensive experiments demonstrate the adaptability of our method within existing methods.

Figures (5)

• Figure 1: T-SNE visualization of motion features in H3.6M under different networks (Left: GCN, Middle: LSTM, Right: Transformer). Purple points denote the ground truth motion features, while green points indicate the predicted features. Incorporating the reconstruction task effectively enhances the alignment between predicted and ground truth motion features.
• Figure 2: Comparison of predictive performance (test loss) in H3.6M under different networks. "LSTM", "Transformer", and "GCN" simultaneously perform both reconstruction and prediction tasks with a shared decoder. "-P" indicates models solely performing the prediction task.
• Figure 3: Illustration of the $TD^2IP$.
• Figure 4: T-SNE visualization of human motion. Green represents the ground truth motion features, and blue depicts the motion features predicted by the model.
• Figure 5: Prediction samples on H3.6M for 80, 160, 320, 400 and 1000 ms. The purple dotted lines indicates the predictions and the grey lines indicate the ground truth actions.