Improvements for H.264 Intra Mode Selection Based on SSE and SSIM

The distortion metrics Sum of Square Error (SSE) and Sum of Absolute Difference (SAD), adopted in RDO on H.264 reference software, have been proved not correlating well with Human Visual System (HVS). Referring to JM16.2, this paper proposes to Combine SSE and Structural SIMilarity (SSIM) as the distortion metric in RDO (CSSRDO) instead of SSE. Firstly, the algorithm finds the appropriate relationship between SSIM and rate. After utilizing the RDO function where distortion is measured as SSE and considering the HVS, the RDO based on combined SSE and SSIM as the distortion metric is proposed. Further more, CSSRDO is introduced into JM16.2 for intra-mode selection. Results show that the proposed algorithm can achieve better coding efficiency and image quality.