Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uniformly Accelerated Motion Model for Inter Prediction

Zhuoyuan Li, Yao Li, Chuanbo Tang, Li Li, Dong Liu, Feng Wu

TL;DR

The paper addresses nonuniform temporal motion in inter prediction by introducing a Uniformly Accelerated Motion Model (UAMM) that incorporates velocity and acceleration into motion representation. It derives UAMM parameters from reference frames using $a = \frac{2\left(\mathbf{mv}_1 \cdot T_0 - \mathbf{mv}_0 \cdot T_1\right)}{T_0 T_1 (T_0+T_1)}$ and $v_0 = \frac{\mathbf{mv}_0 - \frac{a T_0^2}{2}}{T_0}$, and extrapolates future motion via $\mathbf{mv}_2 = \mathbf{v}_0 T_2 + a T_2 (T_0+T_1) + \tfrac{1}{2} a T_2^2$. The UAMM is integrated into existing VVC inter modes (Merge, MMVD, CIIP) with per-$4\times4$ sub-block parameter maintenance, enabling sub-block motion extrapolation and MV correction without extra signaling. Experimental results on VTM-12.0 under Low-delay P show BD-rate reductions up to $0.38\%$ (avg $0.13\%$) with only a small increase in encoding/decoding time, notably improving performance on sequences with moving objects while highlighting limitations in scenes with tiny objects or camera motion. Overall, UAMM advances temporal motion modeling in inter prediction and offers practical gains for video coding in real-world motion conditions.

Abstract

Inter prediction is a key technology to reduce the temporal redundancy in video coding. In natural videos, there are usually multiple moving objects with variable velocity, resulting in complex motion fields that are difficult to represent compactly. In Versatile Video Coding (VVC), existing inter prediction methods usually assume uniform speed motion between consecutive frames and use the linear models for motion estimation (ME) and motion compensation (MC), which may not well handle the complex motion fields in the real world. To address these issues, we introduce a uniformly accelerated motion model (UAMM) to exploit motion-related elements (velocity, acceleration) of moving objects between the video frames, and further combine them to assist the inter prediction methods to handle the variable motion in the temporal domain. Specifically, first, the theory of UAMM is mentioned. Second, based on that, we propose the UAMM-based parameter derivation and extrapolation schemes in the coding process. Third, we integrate the UAMM into existing inter prediction modes (Merge, MMVD, CIIP) to achieve higher prediction accuracy. The proposed method is implemented into the VVC reference software, VTM version 12.0. Experimental results show that the proposed method achieves up to 0.38% and on average 0.13% BD-rate reduction compared to the VTM anchor, under the Low-delay P configuration, with a slight increase of time complexity on the encoding/decoding side.

Uniformly Accelerated Motion Model for Inter Prediction

TL;DR

The paper addresses nonuniform temporal motion in inter prediction by introducing a Uniformly Accelerated Motion Model (UAMM) that incorporates velocity and acceleration into motion representation. It derives UAMM parameters from reference frames using and , and extrapolates future motion via . The UAMM is integrated into existing VVC inter modes (Merge, MMVD, CIIP) with per- sub-block parameter maintenance, enabling sub-block motion extrapolation and MV correction without extra signaling. Experimental results on VTM-12.0 under Low-delay P show BD-rate reductions up to (avg ) with only a small increase in encoding/decoding time, notably improving performance on sequences with moving objects while highlighting limitations in scenes with tiny objects or camera motion. Overall, UAMM advances temporal motion modeling in inter prediction and offers practical gains for video coding in real-world motion conditions.

Abstract

Inter prediction is a key technology to reduce the temporal redundancy in video coding. In natural videos, there are usually multiple moving objects with variable velocity, resulting in complex motion fields that are difficult to represent compactly. In Versatile Video Coding (VVC), existing inter prediction methods usually assume uniform speed motion between consecutive frames and use the linear models for motion estimation (ME) and motion compensation (MC), which may not well handle the complex motion fields in the real world. To address these issues, we introduce a uniformly accelerated motion model (UAMM) to exploit motion-related elements (velocity, acceleration) of moving objects between the video frames, and further combine them to assist the inter prediction methods to handle the variable motion in the temporal domain. Specifically, first, the theory of UAMM is mentioned. Second, based on that, we propose the UAMM-based parameter derivation and extrapolation schemes in the coding process. Third, we integrate the UAMM into existing inter prediction modes (Merge, MMVD, CIIP) to achieve higher prediction accuracy. The proposed method is implemented into the VVC reference software, VTM version 12.0. Experimental results show that the proposed method achieves up to 0.38% and on average 0.13% BD-rate reduction compared to the VTM anchor, under the Low-delay P configuration, with a slight increase of time complexity on the encoding/decoding side.
Paper Structure (16 sections, 12 equations, 4 figures, 3 tables)

This paper contains 16 sections, 12 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Theory of Uniformly Accelerated Motion Model
  4. Uniformly Accelerated Motion Model-based Parameter Derivation and Extrapolation
  5. Derivation of the UAMM-related parameters
  6. Extrapolation of to-be-predicted motion information
  7. Integration of Uniformly Accelerated Motion Model and Inter Modes in VVC
  8. Experimental Results
  9. Experimental Settings
  10. Performance
  11. Overall Performance Under Common Test Conditions
  12. Comparison with the Other Temporal Motion Modeling Methods
  13. Performance on the Specific Scenes
  14. Bi-directional Prediction Exploration
  15. Limitation
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Illustration of temporal motion vector prediction (TMVP) in VVC.
  • Figure 2: Exploiting the uniformly accelerated model for temporal-domain motion modeling. Here we illustrate the different motion models. For the uniform speed model, the object motion can only be modeled on the linear trajectory (straight line). Compared to the uniform speed model, the uniformly accelerated model is used to model the motion with the velocity and acceleration elements in the $x$ and $y$ directions. The different accelerations of $x$ and $y$ directions enable it to model the motion on the nonlinear trajectory (curve), which makes the uniformly accelerated model enable the stronger representation ability.
  • Figure 3: Illustration of the motion trajectory between the coding (to-be-coded) block and reference (reconstructed) blocks in previous reference frames. The coding block and its motion information are colorized in red, the reference blocks and their motion information are colorized in black.
  • Figure 4: Illustration of the proposed uniformly accelerated motion model-integrated inter prediction (UAMM) framework from the perspective of decoding. Boxes represent the sub-modules of UAMM, and arrows indicate the direction of the data flow. The coding block and its motion information are colorized in red, the reference (reconstructed) blocks and their motion information are colorized in black.