Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradient

TL;DR

RL-V2V-GAN is proposed, a new deep neural network approach based on RL for unsupervised conditional video-to-video synthesis that can produce temporally coherent video results and is particularly effective when dealing with limited videos in the target domain.

Abstract

The development of sophisticated models for video-to-video synthesis has been facilitated by recent advances in deep reinforcement learning and generative adversarial networks (GANs). In this paper, we propose RL-V2V-GAN, a new deep neural network approach based on reinforcement learning for unsupervised conditional video-to-video synthesis. While preserving the unique style of the source video domain, our approach aims to learn a mapping from a source video domain to a target video domain. We train the model using policy gradient and employ ConvLSTM layers to capture the spatial and temporal information by designing a fine-grained GAN architecture and incorporating spatio-temporal adversarial goals. The adversarial losses aid in content translation while preserving style. Unlike traditional video-to-video synthesis methods requiring paired inputs, our proposed approach is more general because it does not require paired inputs. Thus, when dealing with limited videos in the target domain, i.e., few-shot learning, it is particularly effective. Our experiments show that RL-V2V-GAN can produce temporally coherent video results. These results highlight the potential of our approach for further advances in video-to-video synthesis.

Figures (8)

• Figure 1: The source video sequence is depicted in the first row and serves as the input to the model. The target video, shown in the second row, is characterized by a blue background in its first half and a red background in its second half.
• Figure 2: The diagram presents the RL-V2V-GAN model, which integrates sequence generators $G_x$, $G_y$, predictors $P_x$, $P_y$, and discriminators $D_x$. $D_y$ for video style transfer. It captures the workflow where $G$ networks transform videos between domains, $P$ networks forecast future frames and $D$ networks assess authenticity and style. The model operates under various losses—adversarial, recurrent, ReCycle, and video—to ensure high-quality, coherent video generation, addressing the challenge of data scarcity in style-specific videos.
• Figure 3: This figure showcases the reinforcement learning mechanism of RL-V2V-GAN, involving Q-networks ($Q_x$, $Q_y$) and policy networks ($\mu$). States ($s$) are videos, and actions ($a$) are potential frames. Q-networks assess the flow of frame sequences, guiding the model to produce coherent and stylistically accurate videos. The system collects transitions in a replay buffer, optimizing for actions that yield realistic sequences and updates policy and Q-network with policy gradient.
• Figure 4: (\ref{['fig:v2v_r']}) shows the structure of block R. (\ref{['fig:v2v_rbp']}) shows the structure of block RPB. (\ref{['fig:v2v_block_urb']}) shows the structure of block URB.
• Figure 5: (\ref{['fig:v2v_dis']}) shows the structure of a discriminator. This instance contains three layers of RPB blocks and a 3D convolutional layer with a sigmoid activation function. (\ref{['fig:v2v_gen']}) shows the structure of a generator. This instance contains three layers of RPB blocks in the encoder and three layers of URB blocks in the decoder, respectively.