Continuous Seam Carving Algorithm Based on Just Noticeable Distortion Algorithm in Image Retargeting
-
摘要: 图像缩放技术要求对图像缩放的同时保证重要信息不丢失且物体边缘不发生扭曲。近年来,Seam Carving及其改进算法得到了广泛的关注和研究。由于采用了离散式最小能量线迭代搜索策略,缩放信息无法在迭代过程中传递导致扭曲现象普遍存在。该文针对上述问题提出最小位移可视差(JND)检测算法,能够有效地检测每一次迭代中出现的潜在扭曲信息。能量权重
$ {E}_{w} $ 能够将JND信息累加传递给后续的迭代过程,从而抑制缩放过程中的边缘扭曲现象。通过JND算法和能量权重,该文首次将离散的Seam Carving模型转变为连续缩放模型。最后,在公共数据集RetargetMe上与最新的图像缩放算法进行多组对比实验,验证了所提方法的有效性和先进性。-
关键词:
- 图像缩放 /
- Seam Carving /
- 最小位移可视差 /
- 平均场近似
Abstract: Image retargeting technologies require important information preservation and less edge distortion during increasing/decreasing image size. The seam carving based algorithms, as the classic retargeting model, receive widespread attention in recent years. However, because of the discrete least energy seam searching strategy, the retargeting information can not be passed generation by generation, which causes retargeting distortions to prevail. To solve this problem, the Just Noticeable Distortion (JND) algorithm is proposed to detect the potential distribution of distortion information. Through the proposed energy weight Ew, the JND information can be passed to the following retargeting iteration for distortion reduction. According to the best knowledge, it is the first time to propose the seam carving algorithm in continuous way by the JND algorithm and energy weight, are the promising results also demonstrated compared with several new approaches at public database ‘Retarget Me’, qualitatively and quantitatively.-
Key words:
- Image Retargeting /
- Seam Carving /
- Just Noticeable Distortion (JND) /
- Mean Approximation
-
表 1 连续Seam Carving图像缩放算法
输入:目标图像, Iin,缩放比例参数, n 输出:缩放结果图像,Iout 1: repeat 2: 根据式(11)计算${f_{{\rm{en}}}}$ 3: 根据式(12)计算$E_{{w}}^{\rm{i}}$ 4: 微小缩放I2和I1至相同大小 5: 根据式(9)和式(10)计算ui 6: 进行缝合计算 7: 更新缩放参数 8: until seamnum=n 表 2 图像缩放定量对比
Crop SC SNS WARP DISJND CONJND Sift-flow 0.37 0.57 0.69 0.38 0.59 0.79 ARS 0.94 0.89 0.90 0.89 0.89 0.91 Vssm 0.25 0.68 0.68 0.75 0.65 0.42 -
梁晓萍, 郭振军, 朱昌洪. 基于头脑风暴优化算法的BP神经网络模糊图像复原[J]. 电子与信息学报, 2019, 41(12): 2980–2986. doi: 10.11999/JEIT190261LIANG Xiaoping, GUO Zhenjun, and ZHU Changhong. BP neural network fuzzy image restoration basedon brain storming optimization algorithm[J]. Journal of Electronics &Information Technology, 2019, 41(12): 2980–2986. doi: 10.11999/JEIT190261 罗洪艳, 朱子岩, 林睿, 等. 基于掩盖效应和梯度信息的无参考噪声图像质量评价改进算法[J]. 电子与信息学报, 2019, 41(1): 210–218. doi: 10.11999/JEIT180195LUO Hongyan, ZHU Ziyan, LIN Rui, et al. Improved no-reference noisy image quality assessment based on masking effect and gradient information[J]. Journal of Electronics &Information Technology, 2019, 41(1): 210–218. doi: 10.11999/JEIT180195 LIU Hong, XU Bin, LU Dianjie, et al. A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm[J]. Applied Soft Computing, 2018, 68: 360–376. doi: 10.1016/j.asoc.2018.04.015 LIU Hong, LIU Baoxi, and ZHANG Hao. Crowd evacuation simulation approach based on navigation knowledge and two-layer control mechanism[J]. Information Sciences, 2018, 436/437: 247–267. doi: 10.1016/j.ins.2018.01.023 AVIDAN S and SHAMIR A. Seam carving for content-aware image resizing[C]. ACM SIGGRAPH 2007, San Diego, USA, 2007. doi: 10.1145/1275808.1276390. ZHANG Lixia, LI Kangshun, QU Zhaoming, et al. Seam warping: A new approach for image retargeting for small displays[J]. Soft Computing, 2017, 21(2): 447–457. doi: 10.1007/s00500-015-1795-1 SHAO Feng, LIN Wenchong, LIN Weisi, et al. QoE-guided warping for stereoscopic image retargeting[J]. IEEE Transactions on Image Processing, 2017, 26(10): 4790–4805. doi: 10.1109/TIP.2017.2721546 HASHEMZADEH M, ASHEGHI B, and FARAJZADEH N. Content-aware image resizing: An improved and shadow-preserving seam carving method[J]. Signal Processing, 2019, 155: 233–246. doi: 10.1016/j.sigpro.2018.09.037 LI Chenyang, HU Ruimin, LIANG Chao, et al. Faster seam carving for video retargeting[C]. The 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018: 823–827. doi: 10.1109/ICIP.2018.8451794. ZHOU Bin, WANG Xuanyin, CAO Songxiao, et al. Optimal bi-directional seam carving for compressibility-aware image retargeting[J]. Journal of Visual Communication and Image Representation, 2016, 41: 21–30. doi: 10.1016/j.jvcir.2016.09.002 SHAFIEYAN F, KARIMI N, MIRMAHBOUB B, et al. Image retargeting using depth assisted saliency map[J]. Signal Processing: Image Communication, 2017, 50: 34–43. doi: 10.1016/j.image.2016.10.006 LU Hongju, YUE Pengfei, ZHAO Yanhui, et al. Contour sensitive saliency and depth application in image retargeting[J]. SPIE Ninth International Conference on Graphic and Image Processing, 2017: 10615. doi: 10.1117/12.2303573. 岳朋飞, 王化雨, 郑元杰, 等. 结合边缘模糊的景深显著性在图像缩放中的研究[J]. 计算机辅助设计与图形学学报, 2018, 30(3): 415–423.YUE Pengfei, WANG Huayu, ZHENG Yuanjie, et al. Image retargeting using blur based depth saliency descriptor[J]. Journal of Computer-Aided Design &Computer Graphics, 2018, 30(3): 415–423. RUBINSTEIN M, GUTIERREZ D, SORKINE O, et al. A comparative study of image retargeting[J]. ACM Transactions on Graphics, 2010, 29(6): 160. doi: 10.1145/1866158.1866186 HAREL J, KOCH C, and PERONA P. Graph-based visual saliency[C]. The 19th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2006: 545–552. ITTI L, KOCH C, and NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11): 1254–1259. doi: 10.1109/34.730558 YAN Qiong, XU Li, SHI Jianping, et al. Hierarchical saliency detection[C]. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 1155–1162. ITO I. Gradient-based global features for seam carving[J]. Eurasip Journal on Image and Video Processing, 2016, 2016(1): 27. doi: 10.1186/s13640-016-0130-9 FANG Yuming, FANG Zhijun, YUAN Feiniu, et al. Optimized multioperator image retargeting based on perceptual similarity measure[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(1): 2956–2966. doi: 10.1109/TSMC.2016.2557225 WEI Daiyan, CHOU Y C, and SU P C. A multi-operator retargeting scheme for compressed videos[C]. 2018 IEEE International Conference on Consumer Electronics, Taichung, China, 2018: 1–12. doi: 10.1109/ICCE-China.2018.8448819. HAN Rong, KE Yongzhen, DU Ling, et al. Exploring the location of object deleted by seam-carving[J]. Expert Systems with Applications, 2018, 95: 162–171. doi: 10.1016/j.eswa.2017.11.023 XU Jinlan, KANG Hongmei, and CHEN Falai. Content-aware image resizing using quasi-conformal mapping[J]. The Visual Computer, 2018, 34(3): 431–442. doi: 10.1007/s00371-017-1350-4 XU Li, JIA Jiaya, and MATSUSHITA Y. Motion detail preserving optical flow estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1744–1757. doi: 10.1109/TPAMI.2011.236 GEIGER D and GIROSI F. Parallel and deterministic algorithms from MRFs: Surface reconstruction and integration[C]. The 1st European Conference on Computer Vision, Antibes, France, 1990: 89–98. doi: 10.1007/BFb0014854. SHI Jianping, XU Li, and JIA Jiaya. Just noticeable defocus blur detection and estimation[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 657–665. WANG Y S, TAI C L, SORKINE O, et al. Optimized scale-and-stretch for image resizing[J]. ACM Transactions on Graphics, 2008, 27(5): 118. doi: 10.1145/1457515.1409071 LIU Ce, YUEN J, and TORRALBA A. SIFT flow: Dense correspondence across scenes and its applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978–994. doi: 10.1109/TPAMI.2010.147 ZHANG Yabin, FANG Yuming, LIN Weisi, et al. Backward registration-based aspect ratio similarity for image retargeting quality assessment[J]. IEEE Transactions on Image Processing, 2016, 25(9): 4286–4297. doi: 10.1109/TIP.2016.2585884 ZHANG Yabin, LIN Weisi, LI Qiaohong, et al. Multiple-level feature-based measure for retargeted image quality[J]. IEEE Transactions on Image Processing, 2018, 27(1): 451–463. doi: 10.1109/TIP.2017.2761556 ZHOU Zeqi, DUAN Chenda, and CUI Jia. The multi-modality content-aware retargeting algorithm and variable scale similarity measurement for image retargeting[C]. The 2nd International Conference on Advances in Image Processing, Chengdu, China, 2018: 73–77.