Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling
Qi Zhang, Shanshe Wang, Xinfeng Zhang, Chuanmin Jia, Zhao Wang, Siwei Ma, Wen Gao
TL;DR
This work addresses the limitation of machine vision coding that optimizes for a narrow set of machines by introducing Satisfied Machine Ratio (SMR), a statistical measure of how well compressed visual data preserves machine analysis across many machines. It builds two large machine libraries and a-scale SMR dataset to capture diverse MVS perceptual characteristics, and proposes deep-learning-based SMR predictors that relate deep feature differences to SMR, including an auxiliary task leveraging SMR differences between quality tiers. Extensive experiments show that SMR-driven coding achieves substantial gains in BD-rate for unseen machines, codecs, and datasets, with strong generalization for both classification and detection tasks. The approach advances VCM from machine-specific to generalizable perceptual coding, with broad practical implications for machine-centric video compression and related perception tasks.
Abstract
Video Coding for Machines (VCM) aims to compress visual signals for machine analysis. However, existing methods only consider a few machines, neglecting the majority. Moreover, the machine's perceptual characteristics are not leveraged effectively, resulting in suboptimal compression efficiency. To overcome these limitations, this paper introduces Satisfied Machine Ratio (SMR), a metric that statistically evaluates the perceptual quality of compressed images and videos for machines by aggregating satisfaction scores from them. Each score is derived from machine perceptual differences between original and compressed images. Targeting image classification and object detection tasks, we build two representative machine libraries for SMR annotation and create a large-scale SMR dataset to facilitate SMR studies. We then propose an SMR prediction model based on the correlation between deep feature differences and SMR. Furthermore, we introduce an auxiliary task to increase the prediction accuracy by predicting the SMR difference between two images in different quality. Extensive experiments demonstrate that SMR models significantly improve compression performance for machines and exhibit robust generalizability on unseen machines, codecs, datasets, and frame types. SMR enables perceptual coding for machines and propels VCM from specificity to generality. Code is available at https://github.com/ywwynm/SMR.