基于最小位移可视差的连续Seam Carving算法在图像缩放中的研究

崔嘉; 宋磊; 陆宏菊; 唐明晰; 戚萌

doi:10.11999/JEIT191050

基于最小位移可视差的连续Seam Carving算法在图像缩放中的研究

doi: 10.11999/JEIT191050

崔嘉^{1, 2},
宋磊^{1, 2},
陆宏菊³,
唐明晰⁴,
戚萌^{1, 2, ,}

1.
山东师范大学信息科学与工程学院济南 250358
2.
山东师范大学智能信息计算与安全实验室济南 250358
3.
济南技师学院济南 250031
4.
香港理工大学设计学院香港 999077

基金项目: 国家自然科学基金(61902225, 61502285)，山东省青年基金(ZR2014FQ013)

详细信息

作者简介:
崔嘉：男，1982年生，副教授，研究方向为图像处理、计算机辅助设计、智能设计

宋磊：男，1993年生，硕士生，研究方向为图像处理

陆宏菊：女，1982年生，讲师，研究方向为图像处理、语义分析

唐明晰：男，1956年生，教授，研究方向为智能设计、设计推理、计算机图形学

戚萌：女，1983年生，讲师，研究方向为虚拟现实、计算机图形学

通讯作者:
戚萌　qimeng@sdnu.edu.cn

中图分类号: TP391.41
计量
- 文章访问数: 1179
- HTML全文浏览量: 589
- PDF下载量: 50
- 被引次数: 0
出版历程
- 收稿日期: 2019-12-30
- 修回日期: 2020-10-23
- 网络出版日期: 2020-12-11
- 刊出日期: 2021-04-20

Continuous Seam Carving Algorithm Based on Just Noticeable Distortion Algorithm in Image Retargeting

Jia CUI^{1, 2},
Lei SONG^{1, 2},
Hongju LU³,
Mingxi TANG⁴,
Meng QI^{1, 2
, ,}

1.
School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
2.
Key Laboratory of Intelligent Information Processing, Shandong Normal University, Jinan 250358, China
3.
College of Jinan Institute of Technology, Jinan 250031, China
4.
School of Design, The Hong Kong Polytechnic University, Hong Kong 999077, China

Funds: The National Natural Science Foundation of China (61902225, 61502285), The Natural Science Foundation of Shandong Province (ZR2014FQ013)

摘要

摘要: 图像缩放技术要求对图像缩放的同时保证重要信息不丢失且物体边缘不发生扭曲。近年来，Seam Carving及其改进算法得到了广泛的关注和研究。由于采用了离散式最小能量线迭代搜索策略，缩放信息无法在迭代过程中传递导致扭曲现象普遍存在。该文针对上述问题提出最小位移可视差(JND)检测算法，能够有效地检测每一次迭代中出现的潜在扭曲信息。能量权重$ {E}_{w} $能够将JND信息累加传递给后续的迭代过程，从而抑制缩放过程中的边缘扭曲现象。通过JND算法和能量权重，该文首次将离散的Seam Carving模型转变为连续缩放模型。最后，在公共数据集RetargetMe上与最新的图像缩放算法进行多组对比实验，验证了所提方法的有效性和先进性。
- 图像缩放 /
- Seam Carving /
- 最小位移可视差 /
- 平均场近似
Abstract: Image retargeting technologies require important information preservation and less edge distortion during increasing/decreasing image size. The seam carving based algorithms, as the classic retargeting model, receive widespread attention in recent years. However, because of the discrete least energy seam searching strategy, the retargeting information can not be passed generation by generation, which causes retargeting distortions to prevail. To solve this problem, the Just Noticeable Distortion (JND) algorithm is proposed to detect the potential distribution of distortion information. Through the proposed energy weight E_w, the JND information can be passed to the following retargeting iteration for distortion reduction. According to the best knowledge, it is the first time to propose the seam carving algorithm in continuous way by the JND algorithm and energy weight, are the promising results also demonstrated compared with several new approaches at public database ‘Retarget Me’, qualitatively and quantitatively.
- Image Retargeting /
- Seam Carving /
- Just Noticeable Distortion (JND) /
- Mean Approximation

HTML全文

图 1 传统SC模型与本文模型对比图

下载: 全尺寸图片幻灯片

图 2 连续SC缩放模型处理流程

下载: 全尺寸图片幻灯片

图 3 SC改进算法定量对比实验

下载: 全尺寸图片幻灯片

图 4 像素级扭曲度对比

下载: 全尺寸图片幻灯片

图 5 图像缩放算法定量对比实验

下载: 全尺寸图片幻灯片

表 1 连续Seam Carving图像缩放算法

输入：目标图像, I_in，缩放比例参数, n
输出：缩放结果图像，I_out
1: repeat
2: 根据式(11)计算${f_{{\rm{en}}}}$
3: 根据式(12)计算$E_{{w}}^{\rm{i}}$
4: 微小缩放I₂和I₁至相同大小
5: 根据式(9)和式(10)计算u_i
6: 进行缝合计算
7: 更新缩放参数
8: until seam_num=n

下载: 导出CSV

表 2 图像缩放定量对比

	Crop	SC	SNS	WARP	DISJND	CONJND
Sift-flow	0.37	0.57	0.69	0.38	0.59	0.79
ARS	0.94	0.89	0.90	0.89	0.89	0.91
Vssm	0.25	0.68	0.68	0.75	0.65	0.42

下载: 导出CSV

参考文献(30)

梁晓萍, 郭振军, 朱昌洪. 基于头脑风暴优化算法的BP神经网络模糊图像复原[J]. 电子与信息学报, 2019, 41(12): 2980–2986. doi: 10.11999/JEIT190261

LIANG Xiaoping, GUO Zhenjun, and ZHU Changhong. BP neural network fuzzy image restoration basedon brain storming optimization algorithm[J]. Journal of Electronics &Information Technology, 2019, 41(12): 2980–2986. doi: 10.11999/JEIT190261

罗洪艳, 朱子岩, 林睿, 等. 基于掩盖效应和梯度信息的无参考噪声图像质量评价改进算法[J]. 电子与信息学报, 2019, 41(1): 210–218. doi: 10.11999/JEIT180195

LUO Hongyan, ZHU Ziyan, LIN Rui, et al. Improved no-reference noisy image quality assessment based on masking effect and gradient information[J]. Journal of Electronics &Information Technology, 2019, 41(1): 210–218. doi: 10.11999/JEIT180195

LIU Hong, XU Bin, LU Dianjie, et al. A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm[J]. Applied Soft Computing, 2018, 68: 360–376. doi: 10.1016/j.asoc.2018.04.015

LIU Hong, LIU Baoxi, and ZHANG Hao. Crowd evacuation simulation approach based on navigation knowledge and two-layer control mechanism[J]. Information Sciences, 2018, 436/437: 247–267. doi: 10.1016/j.ins.2018.01.023

AVIDAN S and SHAMIR A. Seam carving for content-aware image resizing[C]. ACM SIGGRAPH 2007, San Diego, USA, 2007. doi: 10.1145/1275808.1276390.

ZHANG Lixia, LI Kangshun, QU Zhaoming, et al. Seam warping: A new approach for image retargeting for small displays[J]. Soft Computing, 2017, 21(2): 447–457. doi: 10.1007/s00500-015-1795-1

SHAO Feng, LIN Wenchong, LIN Weisi, et al. QoE-guided warping for stereoscopic image retargeting[J]. IEEE Transactions on Image Processing, 2017, 26(10): 4790–4805. doi: 10.1109/TIP.2017.2721546

HASHEMZADEH M, ASHEGHI B, and FARAJZADEH N. Content-aware image resizing: An improved and shadow-preserving seam carving method[J]. Signal Processing, 2019, 155: 233–246. doi: 10.1016/j.sigpro.2018.09.037

LI Chenyang, HU Ruimin, LIANG Chao, et al. Faster seam carving for video retargeting[C]. The 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018: 823–827. doi: 10.1109/ICIP.2018.8451794.

ZHOU Bin, WANG Xuanyin, CAO Songxiao, et al. Optimal bi-directional seam carving for compressibility-aware image retargeting[J]. Journal of Visual Communication and Image Representation, 2016, 41: 21–30. doi: 10.1016/j.jvcir.2016.09.002

SHAFIEYAN F, KARIMI N, MIRMAHBOUB B, et al. Image retargeting using depth assisted saliency map[J]. Signal Processing: Image Communication, 2017, 50: 34–43. doi: 10.1016/j.image.2016.10.006

LU Hongju, YUE Pengfei, ZHAO Yanhui, et al. Contour sensitive saliency and depth application in image retargeting[J]. SPIE Ninth International Conference on Graphic and Image Processing, 2017: 10615. doi: 10.1117/12.2303573.

岳朋飞, 王化雨, 郑元杰, 等. 结合边缘模糊的景深显著性在图像缩放中的研究[J]. 计算机辅助设计与图形学学报, 2018, 30(3): 415–423.

YUE Pengfei, WANG Huayu, ZHENG Yuanjie, et al. Image retargeting using blur based depth saliency descriptor[J]. Journal of Computer-Aided Design &Computer Graphics, 2018, 30(3): 415–423.

RUBINSTEIN M, GUTIERREZ D, SORKINE O, et al. A comparative study of image retargeting[J]. ACM Transactions on Graphics, 2010, 29(6): 160. doi: 10.1145/1866158.1866186

HAREL J, KOCH C, and PERONA P. Graph-based visual saliency[C]. The 19th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2006: 545–552.

ITTI L, KOCH C, and NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11): 1254–1259. doi: 10.1109/34.730558

YAN Qiong, XU Li, SHI Jianping, et al. Hierarchical saliency detection[C]. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, 1155–1162.

ITO I. Gradient-based global features for seam carving[J]. Eurasip Journal on Image and Video Processing, 2016, 2016(1): 27. doi: 10.1186/s13640-016-0130-9

FANG Yuming, FANG Zhijun, YUAN Feiniu, et al. Optimized multioperator image retargeting based on perceptual similarity measure[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(1): 2956–2966. doi: 10.1109/TSMC.2016.2557225

WEI Daiyan, CHOU Y C, and SU P C. A multi-operator retargeting scheme for compressed videos[C]. 2018 IEEE International Conference on Consumer Electronics, Taichung, China, 2018: 1–12. doi: 10.1109/ICCE-China.2018.8448819.

HAN Rong, KE Yongzhen, DU Ling, et al. Exploring the location of object deleted by seam-carving[J]. Expert Systems with Applications, 2018, 95: 162–171. doi: 10.1016/j.eswa.2017.11.023

XU Jinlan, KANG Hongmei, and CHEN Falai. Content-aware image resizing using quasi-conformal mapping[J]. The Visual Computer, 2018, 34(3): 431–442. doi: 10.1007/s00371-017-1350-4

XU Li, JIA Jiaya, and MATSUSHITA Y. Motion detail preserving optical flow estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1744–1757. doi: 10.1109/TPAMI.2011.236

GEIGER D and GIROSI F. Parallel and deterministic algorithms from MRFs: Surface reconstruction and integration[C]. The 1st European Conference on Computer Vision, Antibes, France, 1990: 89–98. doi: 10.1007/BFb0014854.

SHI Jianping, XU Li, and JIA Jiaya. Just noticeable defocus blur detection and estimation[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 657–665.

WANG Y S, TAI C L, SORKINE O, et al. Optimized scale-and-stretch for image resizing[J]. ACM Transactions on Graphics, 2008, 27(5): 118. doi: 10.1145/1457515.1409071

LIU Ce, YUEN J, and TORRALBA A. SIFT flow: Dense correspondence across scenes and its applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978–994. doi: 10.1109/TPAMI.2010.147

ZHANG Yabin, FANG Yuming, LIN Weisi, et al. Backward registration-based aspect ratio similarity for image retargeting quality assessment[J]. IEEE Transactions on Image Processing, 2016, 25(9): 4286–4297. doi: 10.1109/TIP.2016.2585884

ZHANG Yabin, LIN Weisi, LI Qiaohong, et al. Multiple-level feature-based measure for retargeted image quality[J]. IEEE Transactions on Image Processing, 2018, 27(1): 451–463. doi: 10.1109/TIP.2017.2761556

ZHOU Zeqi, DUAN Chenda, and CUI Jia. The multi-modality content-aware retargeting algorithm and variable scale similarity measurement for image retargeting[C]. The 2nd International Conference on Advances in Image Processing, Chengdu, China, 2018: 73–77.

施引文献

资源附件(0)

访问统计