Rendering Distortion Estimation Model for 3D High Efficiency Depth Coding

A depth map represents three-dimensional (3D) scene geometry information and is used for depth image based rendering (DIBR) to synthesize arbitrary virtual views. Since the depth map is only used to synthesize virtual views and is not displayed directly, the depth map needs to be compressed in a certain way that can minimize distortions in the rendered views. In this paper, a modified distortion estimation model is proposed based on view rendering distortion instead of depth map distortion itself and can be applied to the high efficiency video coding (HEVC) rate distortion cost function process for rendering view quality optimization. Experimental results on various 3D video sequences show that the proposed algorithm provides about 31% BD-rate savings in comparison with HEVC simulcast and 1.3？dB BD-PSNR coding gain for the rendered view. 1. Introduction 3D video has gained increasing interest recently. It provides viewers with the illusion of 3D depth perception. The typical 3D video is represented using the multiview video plus depth (MVD) format [1, 2], in which few captured texture videos as well as associated depth maps are used. The depth maps provide per-pixel with depth corresponding to the texture video that can be used to render arbitrary virtual views by using depth image based rendering (DIBR) [3, 4]. For such depth enhanced 3D formats, high efficiency 3D video coding solutions are currently being developed in joint collaborative team on 3D video coding extension development (JCT-3V). Since depth enhanced 3D video MVD representation causes huge amount of data to be stored or transmitted, it is essential to develop efficient 3D video coding techniques. The most straightforward approach to compress 3D video is using conventional video compression algorithms. The next generation video compression standard HEVC developed by Joint video team (JVT) is joined by both ISO/IEC motion picture experts (MPEG) and ITU-T video coding experts group (VCEG). It provides about 50% bit rate reduction as compared to H.264/AVC achieving the same subjective video quality [5]. Therefore even simulcast HEVC compression of multiview video is more efficient than multiview video coding (MVC). For 3D video coding, a simple extension is to apply two HEVC codecs: one for all texture videos and the other for all depth maps. However, compared to conventional 2D video images, depth maps have very different characteristics. One of the major differences is that the depth maps are only used to render virtual views but not directly used to display, so depth map coding errors