Cross-Camera Human Motion Transfer by Time Series Analysis

TL;DR

Cross-Camera Human Motion Transfer by Time Series Analysis addresses motion transfer across heterogeneous camera systems by exploiting seasonality in HR motion data and transferring a learned additive pattern to LR sequences. The method uses a five-step, training-free time-series framework that decomposes LR pose sequences into short-term, long-term, and noise components and transfers the HR additive pattern to refine LR poses. Validation on real-world dual-camera data demonstrates improved 3D human mesh reconstruction from LR video and reduced pose jitter in downstream tasks like texture transfer, without requiring large training datasets. The approach emphasizes interpretability and computational efficiency, and code is publicly available at the project repository.

Abstract

With advances in optical sensor technology, heterogeneous camera systems are increasingly used for high-resolution (HR) video acquisition and analysis. However, motion transfer across multiple cameras poses challenges. To address this, we propose an algorithm based on time series analysis that identifies motion seasonality and constructs an additive model to extract transferable patterns. Validated on real-world data, our algorithm demonstrates effectiveness and interpretability. Notably, it improves pose estimation in low-resolution videos by leveraging patterns derived from HR counterparts, enhancing practical utility. Code is available at: https://github.com/IndigoPurple/TSAMT

Figures (9)

• Figure 1: Our framework utilizes a multi-camera system to capture human-centric videos at multiple scales. By transfering extracted human motion patterns from high-resolution (HR) videos, we enhance pose estimation in low-resolution (LR) videos across different camera feeds. This approach ensures superior pose estimation irrespective of the video resolution.
• Figure 2: For surveillance and security, a multi-camera system captures pedestrian trajectories in a large FoV video. Meanwhile, local-view cameras capture HR details of pedestrians.
• Figure 3: HR and LR motion data. (a) and (b) represent the $\theta_{1, 1}$ value of the HR and LR motion data, respectively.
• Figure 4: (a) Auto-correlation of HR motion data shows periodic variation with a maximum value during predefined intervals, gradually decreasing to zero. (b) Fourier analysis of the HR motion data reveals a dominant response at $f = 5$, indicating a reference period of $16$ (derived from $80 / 5$).
• Figure 5: (a) Crossover points of the blue and dark green curves indicate the locations of the periods, marked as red circles. (b) To refine LR pose values using HR ones, we estimate the additive factor $\mathbf{A}$ by averaging all the periods. (c) By adding the additive factor to the long-term trend $\mathbf{T}^L$, we generate the final poses for refining LR pose values with HR ones.