Attention-Based Bi-Prediction Network for Versatile Video Coding (VVC) over 5G Network

Abstract

As the demands of various network-dependent services such as Internet of things (IoT)applications, autonomous driving, and augmented and virtual reality (AR/VR) increase, the fifthgeneration(5G) network is expected to become a key communication technology. The latest videocoding standard, versatile video coding (VVC), can contribute to providing high-quality services byachieving superior compression performance. In video coding, inter bi-prediction serves to improvethe coding efficiency significantly by producing a precise fused prediction block. Although block-wisemethods, such as bi-prediction with CU-level weight (BCW), are applied in VVC, it is still difficult forthe linear fusion-based strategy to represent diverse pixel variations inside a block. In addition, apixel-wise method called bi-directional optical flow (BDOF) has been proposed to refine bi-predictionblock. However, the non-linear optical flow equation in BDOF mode is applied under assumptions,so this method is still unable to accurately compensate various kinds of bi-prediction blocks. Inthis paper, we propose an attention-based bi-prediction network (ABPN) to substitute for the wholeexisting bi-prediction methods. The proposed ABPN is designed to learn efficient representations ofthe fused features by utilizing an attention mechanism. Furthermore, the knowledge distillation (KD)-based approach is employed to compress the size of the proposed network while keeping comparableoutput as the large model. The proposed ABPN is integrated into the VTM-11.0 NNVC-1.0 standardreference software. When compared with VTM anchor, it is verified that the BD-rate reduction of thelightweighted ABPN can be up to 5.89% and 4.91% on Y component under random access (RA) andlow delay B (LDB), respectively.