Nonlinear Methods for Analyzing Pose in Behavioral Research

Abstract

Advances in markerless pose estimation have made it possible to capture detailed human movement in naturalistic settings using standard video, enabling new forms of behavioral analysis at scale. However, the high dimensionality, noise, and temporal complexity of pose data raise significant challenges for extracting meaningful patterns of coordination and behavioral change. This paper presents a general-purpose analysis pipeline for human pose data, designed to support both linear and nonlinear characterizations of movement across diverse experimental contexts. The pipeline combines principled preprocessing, dimensionality reduction, and recurrence-based time series analysis to quantify the temporal structure of movement dynamics. To illustrate the pipeline's flexibility, we present three case studies spanning facial and full-body movement, 2D and 3D data, and individual versus multi-agent behavior. Together, these examples demonstrate how the same analytic workflow can be adapted to extract theoretically meaningful insights from complex pose time series.

Figures (13)

• Figure 1: Effects of gap length on RQA stability and recurrence structure. Top Left: Raw time series showing original signal used. Bottom Left: Relative error in Recurrence Rate (RR) and Determinism (DET) as a function of the embedding delay $\tau$. Curves show the mean error across 30 trials, with error bars indicating standard deviation. The dashed line marks the theoretical threshold $(m-1)\tau$, beyond which embedding vectors begin to span the gap; the dotted line marks a 5% error reference. RR error increases markedly once gaps exceed $(m-1)\tau$, while DET remains comparatively stable. Right: Recurrence plots illustrating how interpolation affects the underlying recurrence structure. The top-left panel shows the baseline RP (no gaps). Remaining panels show discrete difference maps (gaps-baseline) for centered gaps of 0.5$\tau$, 2$\tau$, and 4$\tau$. Colors denote recurrence changes: blue = lost recurrence, white = unchanged, red = gained recurrence. Small gaps introduce minimal distortions, whereas larger gaps generate structured artifacts, including diagonal streaks and localized regions of false recurrence.
• Figure 2: Procrustes alignment of 2D pose data. Synthetic pose skeletons illustrate the three stages of Procrustes alignment. (A) Raw: Two participants with different positions, scales, and orientations relative to the template (black). (B) After Translation: Poses are centered to the template centroid. (C) After Scale: Poses are normalized to match the template size. (D) After Rotation: Poses are rotated to align with the template orientation. This alignment procedure removes between-participant differences in position, body size, and camera angle, enabling direct comparison of pose dynamics.
• Figure 3: Illustration of recurrence quantification analysis (RQA) workflows. Each approach begins with a time series (left), which is reconstructed into a state space via time-delay embedding (middle), and then converted into a recurrence plot (right). Auto-RQA (top row) quantifies recurrent structure within a single trajectory, capturing the temporal organization of one system’s dynamics. Cross-RQA (middle row) embeds two trajectories into a shared phase space and identifies when one system revisits states previously visited by the other, allowing coordinated patterns to be quantified. Multidimensional RQA (bottom row) embeds multiple time series simultaneously, treating them as components of a single dynamical system to assess collective recurrence structure across signals. Together, these methods extend recurrence analysis from individual dynamics to coupled and high-dimensional systems, providing flexible tools for movement and coordination research.
• Figure 4: Illustrative Average Mutual Information (AMI) curves demonstrating common patterns encountered when selecting the embedding delay $\tau$ for pose-derived time series. (A) A clear first minimum, the canonical textbook case, where $\tau$ may be chosen directly at the first local minimum. (B) A broad plateau or shallow trough, characteristic of many natural movement signals; here a range of delays provides comparable information content, and robustness should be assessed across this interval. (C) An oscillatory or quasi-periodic AMI profile with multiple local minima; in such cases, the earliest stable minimum or plateau region is typically preferred. (D) A slow, almost flat decay indicating strong temporal redundancy at the sampling rate; delays may be selected at the onset of the plateau or based on a fixed AMI threshold. Across all panels, shaded bands denote plausible delay ranges, emphasizing that for non-rhythmic pose data, selecting a single $\tau$ is rarely justified. Robust analyses should verify that substantive results are stable across a small set of reasonable delays. The shaded bands are illustrative rather than prescriptive and are intended to denote plausible regions rather than precise optima.
• Figure 5: Analysis pipeline for pose-based behavioral research. The five-stage workflow is applied across three case studies to demonstrate adaptability across recording configurations and movement scales. Stages 1–3 represent the common preprocessing sequence, including low-pass filtering (LPF), confidence-based masking, and Procrustes alignment to ensure spatial and temporal standardization. Stage 4 extracts linear kinematic features to quantify movement magnitude and variability. Stage 5 employs recurrence quantification analysis (RQA/CRQA) to characterize the temporal organization and coordination dynamics of the behavioral signal.