Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiomechAgent: AI-Assisted Biomechanical Analysis Through Code-Generating Agents

R. James Cotton, Thomas Leonard

TL;DR

BomechAgent, a code-generating AI agent that enables biomechanical analysis through natural language and allows users to querying databases, generating visualizations, and even interpret data without requiring users to write code, is presented.

Abstract

Markerless motion capture is making quantitative movement analysis increasingly accessible, yet analyzing the resulting data remains a barrier for clinicians without programming expertise. We present BiomechAgent, a code-generating AI agent that enables biomechanical analysis through natural language and allows users to querying databases, generating visualizations, and even interpret data without requiring users to write code. To evaluate BiomechAgent's capabilities, we developed a systematic benchmark spanning data retrieval, visualization, activity classification, temporal segmentation, and clinical reasoning. BiomechAgent achieved robust accuracy on data retrieval and visualization tasks and demonstrated emerging clinical reasoning capabilities. We used our dataset to systematically evaluate several of our design decisions. Biomechanically-informed, domain-specific instructions significantly improved performance over generic prompts, and integrating validated specialized tools for gait event detection substantially boosted accuracy on challenging spatiotemporal analysis where the base agent struggled. We also tested BiomechAgent using a local open-weight model instead of a frontier cloud based LLM and found that perform was substantially diminished in most domains other than database retrieval. In short, BiomechAgent makes the data from accessible motion capture and much more useful and accessible to end users.

BiomechAgent: AI-Assisted Biomechanical Analysis Through Code-Generating Agents

TL;DR

BomechAgent, a code-generating AI agent that enables biomechanical analysis through natural language and allows users to querying databases, generating visualizations, and even interpret data without requiring users to write code, is presented.

Abstract

Markerless motion capture is making quantitative movement analysis increasingly accessible, yet analyzing the resulting data remains a barrier for clinicians without programming expertise. We present BiomechAgent, a code-generating AI agent that enables biomechanical analysis through natural language and allows users to querying databases, generating visualizations, and even interpret data without requiring users to write code. To evaluate BiomechAgent's capabilities, we developed a systematic benchmark spanning data retrieval, visualization, activity classification, temporal segmentation, and clinical reasoning. BiomechAgent achieved robust accuracy on data retrieval and visualization tasks and demonstrated emerging clinical reasoning capabilities. We used our dataset to systematically evaluate several of our design decisions. Biomechanically-informed, domain-specific instructions significantly improved performance over generic prompts, and integrating validated specialized tools for gait event detection substantially boosted accuracy on challenging spatiotemporal analysis where the base agent struggled. We also tested BiomechAgent using a local open-weight model instead of a frontier cloud based LLM and found that perform was substantially diminished in most domains other than database retrieval. In short, BiomechAgent makes the data from accessible motion capture and much more useful and accessible to end users.
Paper Structure (35 sections, 2 figures, 3 tables)

This paper contains 35 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. BiomechAgent System
  4. Agent Architecture
  5. Language Model Backends
  6. Custom Instructions
  7. Data Sources
  8. Chat Interface
  9. Dataset Generation
  10. Privileged Data Sources
  11. Database Tasks
  12. Activity Classification Tasks
  13. Spatiotemporal Tasks
  14. Clinical Tasks
  15. Segmentation Tasks
  16. ...and 20 more sections

Figures (2)

  • Figure 1: BiomechAgent chat interface screenshots from a sequence of reasoning about the side with hemiparesis, including plotting and feeding the result into the VLM.
  • Figure 2: Performance comparison across task categories for different model configurations. Baseline+GT replaces the spatiotemporal results from the baseline evaluation with those when the GaitTransformer tool was also included.