SportsBuddy: Designing and Evaluating an AI-Powered Sports Video Storytelling Tool Through Real-World Deployment

TL;DR

This paper tackles the difficulty of producing informative sports highlights without advanced video editing skills by introducing SportsBuddy, an AI-driven, web-based tool that integrates player tracking, embedded visualizations, and narrative timelines. It details a three-branch video processing pipeline (player detection/pose estimation, foreground segmentation, and AI captioning), a canvas-based rendering engine with 13 render objects, and a cloud-backed distributed backend for scalable use. Real-world deployment since August 2024 and case studies with Harvard Athletics and content creators demonstrate substantial improvements in editing efficiency, coaching communication, and fan engagement, validating the platform’s practical impact. The work highlights the value of domain-specific, human-centered visualization tools in extending the reach of advanced sports analytics and calls for broader, real-world deployment to guide iterative design improvements.

Abstract

Video storytelling is essential for sports performance analysis and fan engagement, enabling sports professionals and fans to effectively communicate and interpret the spatial and temporal dynamics of gameplay. Traditional methods rely on manual annotation and verbal explanations, placing significant demands on creators for video editing skills and on viewers for cognitive focus. However, these approaches are time-consuming and often struggle to accommodate individual needs. SportsBuddy addresses this gap with an intuitive, interactive video authoring tool. It combines player tracking, embedded interaction design, and timeline visualizations to seamlessly integrate narratives and visual cues within game contexts. This empowers users to effortlessly create context-driven video stories. Since its launch, over 150 sports users, including coaches, athletes, content creators, parents and fans, have utilized SportsBuddy to produce compelling game highlights for diverse use cases. User feedback highlights its accessibility and ease of use, making video storytelling and insight communication more attainable for diverse audiences. Case studies with collegiate teams and sports creators further demonstrate SportsBuddy's impact on enhancing coaching communication, game analysis, and fan engagement.

Figures (4)

• Figure 1: Nine visualization features are provided under Highlight Tab. Users can directly interact with the video to apply effects, such as clicking on a player (a) or drawing on canvas (b-d). These features support communicating sports insights, including highlighting players (a)(e), illustrating tactics (b-d)(f), and annotating actions (g).
• Figure 2: A demonstration of a coach adding insights with visualization features on SportsBuddy: starting with uploading a clip (a), highlighting player movements and tactics (b-d), and annotating insights with a video freeze frame, background filter, and text labels (e-f).
• Figure 3: Narrative Tab provides captioning features. Users can (a) input captions manually, (b) create AI-generated captions, and (c) see instant caption update in the video.
• Figure 4: SportsBuddy enhances the effectiveness and diversity of video creation for athletic communication, game analysis, and storytelling, supporting use cases such as highlighting (a) player synergy, (b) tactics, (c) spatial actions, and (d) player performance.