RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression

TL;DR

RL-RC-DoT introduces a block-level reinforcement learning agent that optimizes task-aware video compression by controlling MB QP deltas within standard encoders such as x264. Framing encoding as an MDP, the agent learns via PPO to balance downstream task performance with bitrate constraints, using encoder statistics as state and a self-supervised task reward. The method achieves substantial BD-rate reductions for car detection (~-24.7%) and ROI saliency encoding (~-25.6%) with minimal PSNR impact, and demonstrates robustness to task/model shifts and transfer across related tasks. It is codec-agnostic and designed for real-time deployment, suggesting broad applicability to autonomous systems and data collection, though it requires careful consideration of training complexity and generalization to different resolutions and codecs.

Abstract

Video encoders optimize compression for human perception by minimizing reconstruction error under bit-rate constraints. In many modern applications such as autonomous driving, an overwhelming majority of videos serve as input for AI systems performing tasks like object recognition or segmentation, rather than being watched by humans. It is therefore useful to optimize the encoder for a downstream task instead of for perceptual image quality. However, a major challenge is how to combine such downstream optimization with existing standard video encoders, which are highly efficient and popular. Here, we address this challenge by controlling the Quantization Parameters (QPs) at the macro-block level to optimize the downstream task. This granular control allows us to prioritize encoding for task-relevant regions within each frame. We formulate this optimization problem as a Reinforcement Learning (RL) task, where the agent learns to balance long-term implications of choosing QPs on both task performance and bit-rate constraints. Notably, our policy does not require the downstream task as an input during inference, making it suitable for streaming applications and edge devices such as vehicles. We demonstrate significant improvements in two tasks, car detection, and ROI (saliency) encoding. Our approach improves task performance for a given bit rate compared to traditional task agnostic encoding methods, paving the way for more efficient task-aware video compression.

Figures (14)

• Figure 1: RL-RC-DoT workflow. Our proposed solution to the block-level control for a downstream task. RL-RC-DoT takes encoder statistics as input and outputs a block-level delta QP map. We then evaluate the difference in downstream task performance between the reconstructed frame and the raw frame. The reward contains both global score as reward and block-level score.
• Figure 2: Car detection precision and recall of YOLO5, and PSNR. Value are mean and s.e.m. calculated across all frames from the test set.
• Figure 3: RD curves for Car detection task for different streams (color). Comparing standard x264 (dashed lines) with RL-RC-DoT (solid lines). Curves show 3 example streams, demonstrating how RL-RC-DoT improves quality across the range of bit-rate values. (a) Car detection precision (b) recall (c) PSNR.
• Figure 4: Car detection example result. (a) detection output on x264 reconstructed frame, (b) output on raw frame and (c) output on RL-RC-DoT reconstructed frame. Notice that both RL-RC-DoT and x264 used the same target bit-rate
• Figure 5: Example frame analysis. Red areas denote more bits allocate in the QP map ((a) x264 allocated bits,(c) RL-RC-DoT allocated bits ) and (b) Eigen-CAM values (middle).