Multi-Stage Generative Upscaler: Reconstructing Football Broadcast Images via Diffusion Models

TL;DR

This work addresses the challenge of reconstructing and upscaling severely degraded football broadcast frames by presenting a multi-stage diffusion pipeline that progresses from an image-to-image reconstruction to conditional refinement with ControlNet and domain-specific fine-tuning via LoRA on football data. The approach leverages a football-focused dataset and a KohyaSS-based training regimen to recover domain-specific details such as jersey textures and logos, achieving higher fidelity than conventional upscaling methods. Key contributions include a robust multi-stage architecture, a football-dedicated LoRA, and an empirical demonstration of improved structural and textural quality under realistic broadcast degradations, albeit with high VRAM requirements. The findings have practical implications for automated video enhancement and real-time sports analytics, and the authors outline future work on larger datasets, GUI development, and potential 3D representations to broaden applicability.

Abstract

The reconstruction of low-resolution football broadcast images presents a significant challenge in sports broadcasting, where detailed visuals are essential for analysis and audience engagement. This study introduces a multi-stage generative upscaling framework leveraging Diffusion Models to enhance degraded images, transforming inputs as small as $64 \times 64$ pixels into high-fidelity $1024 \times 1024$ outputs. By integrating an image-to-image pipeline, ControlNet conditioning, and LoRA fine-tuning, our approach surpasses traditional upscaling methods in restoring intricate textures and domain-specific elements such as player details and jersey logos. The custom LoRA is trained on a custom football dataset, ensuring adaptability to sports broadcast needs. Experimental results demonstrate substantial improvements over conventional models, with ControlNet refining fine details and LoRA enhancing task-specific elements. These findings highlight the potential of diffusion-based image reconstruction in sports media, paving the way for future applications in automated video enhancement and real-time sports analytics.

Figures (8)

• Figure 1: An example of raw image frames segmented from a 1080p interlaced broadcast. These snapshots, manually cut by technical operators during the match, exhibit varying resolutions, often starting as low as $60 \times 60$ pixels.
• Figure 2: Complete pipeline flowchart
• Figure 3: Comparison of image enhancement techniques applied to raw broadcast frames. From left to right, each column represents: the raw image frames resized to $256 \times 256$ using Lanczos resampling, the result of applying Real-ESRGAN for $1024 \times 1024$ upscaling, the result from the latent x4 upscaler (Stable Diffusion x4 Upscaler), and finally, the Lanczos resampling upscaled to $1024 \times 1024$.
• Figure 4: Comparison of outputs from image-to-image pipelines using various models. From left to right, each coloumn represents: the original raw frames, the outputs of Stable Diffusion XL, RealVis_v4.0, Stable Diffusion 3.5-Medium, Stable Diffusion 3.5-Large, and FLUX.1-dev.
• Figure 5: Comparison of Control-Net pipeline outputs. From left to right, each coloumn represents: the original raw frames, the output images without the first stage of the reconstruction, the output images with the first stage of the reconstruction.