Few-shot Semantic Encoding and Decoding for Video Surveillance

TL;DR

The paper tackles the growing bandwidth and storage burden of surveillance video by introducing a sketch-based semantic encoding and decoding framework that requires only a few training samples per scene. It compresses sketch information, reconstructs sketches with a few-shot semantic decoder, and translates sketches into frames using a reference-image conditioned image translation network, with an optical-flow–guided fusion to improve quality. The approach demonstrates superior reconstruction metrics (KID, LPIPS, PSNR, SSIM) compared with baselines and achieves substantial bitrate reduction through masked sketch compression, enhancing the practicality of semantic surveillance video transmission. Overall, it provides a scalable path toward practical semantic communication for surveillance systems by reducing training data needs while maintaining high reconstruction fidelity.

Abstract

With the continuous increase in the number and resolution of video surveillance cameras, the burden of transmitting and storing surveillance video is growing. Traditional communication methods based on Shannon's theory are facing optimization bottlenecks. Semantic communication, as an emerging communication method, is expected to break through this bottleneck and reduce the storage and transmission consumption of video. Existing semantic decoding methods often require many samples to train the neural network for each scene, which is time-consuming and labor-intensive. In this study, a semantic encoding and decoding method for surveillance video is proposed. First, the sketch was extracted as semantic information, and a sketch compression method was proposed to reduce the bit rate of semantic information. Then, an image translation network was proposed to translate the sketch into a video frame with a reference frame. Finally, a few-shot sketch decoding network was proposed to reconstruct video from sketch. Experimental results showed that the proposed method achieved significantly better video reconstruction performance than baseline methods. The sketch compression method could effectively reduce the storage and transmission consumption of semantic information with little compromise on video quality. The proposed method provides a novel semantic encoding and decoding method that only needs a few training samples for each surveillance scene, thus improving the practicality of the semantic communication system.

Figures (5)

• Figure 1: Overall structure of the proposed semantic encoding and decoding method
• Figure 2: Sketch compression method
• Figure 3: The image translation network
• Figure 4: Sketch compression compare
• Figure 5: Reconstructed frames of different video decoding method