Portable Biomechanics Laboratory: Clinically Accessible Movement Analysis from a Handheld Smartphone

TL;DR

Handheld smartphone video can deliver accurate, scalable, and low-burden biomechanical measurement, enabling greatly increased monitoring of movement impairments, and is validated on the first clinically-validated method for measuring whole-body kinematics from handheld smartphone video.

Abstract

Movement directly reflects neurological and musculoskeletal health, yet objective biomechanical assessment is rarely available in routine care. We introduce Portable Biomechanics Laboratory (PBL), a secure platform for fitting biomechanical models to video collected with a handheld, moving, smartphone. We validate this approach on over 15 hours of data synchronized to ground truth motion capture, finding mean joint-angle errors < 3$°$ and pelvis-translation errors of a few centimeters across patients with neurological-injury, lower-limb prosthesis users, pediatric in-patients, and controls. In > 5 hours of prospective deployments to neurosurgery and sports-medicine clinics, PBL was easy to setup, yielded highly reliable gait metrics (ICC > 0.9), and detected clinically relevant differences. For cervical-myelopathy patients, its measurement of gait quality correlated with modified Japanese Orthopedic Association (mJOA) scores and were responsive to clinical intervention. Handheld smartphone video can therefore deliver accurate, scalable, and low-burden biomechanical measurement, enabling greatly increased monitoring of movement impairments. We release the first clinically-validated method for measuring whole-body kinematics from handheld smartphone video at https://IntelligentSensingAndRehabilitation.github.io/MonocularBiomechanics/.

Figures (10)

• Figure 1: Study Design Overview. We first validated the Portable Biomechanics Laboratory (PBL) against an optical motion capture dataset in which control participants performed a range of upper- and lower-limb activities. We then evaluated PBL in clinical populations by recording participants performing lower-limb tasks using both multi-camera markerless motion capture and PBL in hospital laboratories and on clinical floors. Finally, we deployed PBL in clinic hallways and therapy gyms, recording participants after clinical visits to assess the relationship between PBL-derived gait quality scores and standard clinical measures.
• Figure 1: Joint Angle Error Distributions.A) In the MMMC dataset, Median Absolute Error distributions remain relatively stable when aggregated across activity types, participant populations (B), as well as between handheld (moving) and static cameras. C) Joint-specific Median Absolute Error distributions from both the MMMC and BML-MoVi datasets.
• Figure 2: Biomechanical Reconstruction Overview. We introduce a method for biomechanically grounded movement analysis in clinical settings using a handheld smartphone. A) Researchers held a smartphone (optionally with gimbal) while following a participant walking. Our system has no specific requirements regarding viewing angle, distance to subject, or therapist assistance. B) Recorded smartphone video and optional wearable sensor data are stored in the cloud, and processed using PosePipe, an open-source package implementing computer vision models for person tracking and keypoint detection. C) To reconstruct movement, we represent movement as a function that outputs joint angles, which—combined with body scaling parameters and evaluated through forward kinematics—generate a posed biomechanical model in 3D space. This untrained model is compared to video-extracted joint locations and optionally smartphone sensor data to compute a loss. This loss guides backpropagation to iteratively refine both the kinematic trajectory and body scale. D) Initially, the representation lacks knowledge of the person’s movements and scale (e.g., height, limb proportions), but after optimization, it typically tracks joint locations within 15 mm in 3D and 5 pixels in 2D.
• Figure 2: SPM1D Comparison of Multi Camera and Monocular. Raw kinematic traces (n=1971) compared between the Multi-Camera (MMMMC) and Single Camera (PBL) modalities of the MMMC dataset are compared using paired SPM1D t-tests, revealing significant differences at almost every timepoint in the gait cycle. Dashed lines in second row represent p < 0.05 significance threshold.
• Figure 3: Biomechanical Reconstructions from Handheld Smartphone. Using only a handheld smartphone, our PBL and end-to-end biomechanical fitting approach robustly captures movement across varied clothing types, assistive devices, and clinical environments. A) We first validate this approach on clinical outpatients in our lab, including participants using assistive devices or receiving support from a clinician as needed. B) We next deploy this approach in an outpatient clinic for patients with gait impairments, finding this method minimally disrupting clinical workflow while capturing relevant gait features. C) This technique allows deployable biomechanical capture in dynamic clinical settings as well as outdoors.