iBall: Augmenting Basketball Videos with Gaze-moderated Embedded Visualizations

TL;DR

The paper tackles the challenge that casual basketball fans struggle to interpret game footage due to knowledge gaps. It introduces iBall, a system that fuses a CV pipeline for player detection, pose estimation, and foreground-background segmentation with gaze-moderated embedded visualizations to adaptively reveal players’ importance and abilities. Through computational evaluation and a user study involving 24 participants (16 casual fans and 8 die-hard fans), the authors demonstrate that gaze-driven overlays increase usefulness, engagement, and learning, with FULL mode (embedding plus gaze interactions) delivering the strongest benefits. The work offers design guidelines for attention-driven, synchronized, and adaptive visualizations in sports video, and discusses broader implications for live game viewing and AR-enabled experiences.

Abstract

We present iBall, a basketball video-watching system that leverages gaze-moderated embedded visualizations to facilitate game understanding and engagement of casual fans. Video broadcasting and online video platforms make watching basketball games increasingly accessible. Yet, for new or casual fans, watching basketball videos is often confusing due to their limited basketball knowledge and the lack of accessible, on-demand information to resolve their confusion. To assist casual fans in watching basketball videos, we compared the game-watching behaviors of casual and die-hard fans in a formative study and developed iBall based on the fndings. iBall embeds visualizations into basketball videos using a computer vision pipeline, and automatically adapts the visualizations based on the game context and users' gaze, helping casual fans appreciate basketball games without being overwhelmed. We confrmed the usefulness, usability, and engagement of iBall in a study with 16 casual fans, and further collected feedback from 8 die-hard fans.

Figures (8)

• Figure 3: Our CV pipeline takes a raw video as the input, outputs the bounding box, identity, and key points of each player, and separates the image frame into the foreground (humans) and background (all others). The bounding boxes, identities, and key points are used to create visualizations, which are then composited with the foreground and background to form the augmented video.
• Figure 4: The system takes positional tracking data and historical stats as input to calculate the players' importance and offensive and defensive abilities. Only the important players and their offensive and defensive abilities will be highlighted and visualized in the video. The user can use gaze points to adjust the players' importance levels, as well as controlling whose abilities to show.
• Figure 5: Visualization of various importance levels: Lv3, key offensive players, highlighted by a sportlight; Lv2.5, players of interest to the user, triggered by Gaze Focus, highlighted by a glowing effect (will be introduced in Sec. \ref{['sec:gaze_int']}); Lv2, key defensive players, highlighted by extra brightness; Lv1, other players, no highlighting.
• Figure 7: Three embedded visualizations for in-game data: a) Offense Ring shows the offensive performance of an offensive player. The darker, larger, the better. b) Defense Shield shows the defensive performance of the defender. The thicker, longer, the better. c) One-on-one Line shows the one-on-one relationship between the offensive player with the ball and the defenders.
• Figure 8: Left: Three design alternatives for Offense Ring. a) Displaying the data on top of the player can occlude other players. b) Moving the visualization higher (e.g., the design in CourtVision courtvision) can make it hard to connect to the target player. c) Displaying the data aside of the players (e.g., the shot meter in NBA 2K nba2k) can also occlude other players. Right: An experimental EPV map of Steven Curry encodes his shooting frequency and EPV by using the size and divergent color scale. Different from \ref{['fig:epv_map']}, the bins in this EPV map are not grouped by regions.