2021 IEEE International Conference on Image Processing
19-22 September 2021 • Anchorage, Alaska, USA
Imaging Without Borders
| Paper ID | SS-IVC-DL.1 | ||
| Paper Title | A DEEPLY MODULATED SCHEME FOR VARIABLE-RATE VIDEO COMPRESSION | ||
| Authors | Jianping Lin, Dong Liu, Unversity of Science and Technology of China, China; Jie Liang, Simon Fraser University, Canada; Houqiang Li, Feng Wu, Unversity of Science and Technology of China, China | ||
| Session | SS-IVC-DL: Special Session: Optimized Image and Video Coding Using Deep Learning | ||
| Location | Area B | ||
| Session Time: | Wednesday, 22 September, 08:00 - 09:30 | ||
| Presentation Time: | Wednesday, 22 September, 08:00 - 09:30 | ||
| Presentation | Poster | ||
| Topic | Special Sessions: Optimized image and video coding schemes using deep learning | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Rate adaption is one of the decisive factors for the applications of video compression. Previous deep video compression methods are usually optimized for a single fixed rate-distortion (R-D) tradeoff. While they can achieve multiple bitrates by training multiple independent models, the achievable bitrates are limited to several discrete points on the R-D curve and the storage cost increases proportionally to the number of models. We propose a variable-rate scheme for deep video compression, which can achieve continuously variable rate by a single model, i.e., reaching any point on the R-D curve. In our scheme, two deep auto-encoders are used to compress the residual and the motion vector field respectively, which directly generate the final bitstream. The basic rate adaptation can be achieved by using the R-D tradeoff parameter to deeply modulate all the internal feature maps of the auto-encoders. In addition, other modules in our scheme, notably motion estimation and motion compensation, also affect the final bitrate indirectly. We further use the R-D tradeoff parameter to modulate them via a conditional map, thereby effectively improving the compression efficiency. We use a multi-rate-distortion loss function together with a step-by-step training strategy to optimize the entire scheme. The experimental results show the proposed scheme achieves continuously variable rate by a single model with almost the same compression efficiency as multiple fixed-rate models. The additional parameters and computation of our model are negligible when compared with a single fixed-rate model. | ||