Multi-focus Image Fusion Algorithm Based on Super Pixel Level Convolutional Neural Network

Xixi NIE; Bin XIAO; Xiuli BI; Weisheng LI

doi:10.11999/JEIT191053

Volume 43 Issue 4

Apr. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(4): 965-973

Xixi NIE, Bin XIAO, Xiuli BI, Weisheng LI. Multi-focus Image Fusion Algorithm Based on Super Pixel Level Convolutional Neural Network[J]. Journal of Electronics & Information Technology, 2021, 43(4): 965-973. doi: 10.11999/JEIT191053

Citation:

Xixi NIE, Bin XIAO, Xiuli BI, Weisheng LI. Multi-focus Image Fusion Algorithm Based on Super Pixel Level Convolutional Neural Network[J]. Journal of Electronics & Information Technology, 2021, 43(4): 965-973. doi: 10.11999/JEIT191053

Citation:

PDF( 5281 KB)

Multi-focus Image Fusion Algorithm Based on Super Pixel Level Convolutional Neural Network

doi: 10.11999/JEIT191053

Chongqing Key Laboratory of Computational Intelligence, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Key Research and Development Project of China (2016YFC1000307-3), The National Natural Science Foundation of China (61976031, 61806032)

Received Date: 2019-12-30
Rev Recd Date: 2020-10-28

Available Online: 2020-12-12

Publish Date: 2021-04-20

Abstract

Abstract

This paper proposes a multi-focus image fusion algorithm based on super pixel-level Convolutional Neural Network (sp-CNN). In this method, multi-scale super pixel segmentation is firstly applied to the source image to obtain the super pixels. Secondly, the sp-CNN is proposed to acquire the initial decision maps. Thirdly, according to the similarities and differences of the multiple initial decision maps, the uncertain region is reclassified by spatial frequency to obtain the phase decision map. At last, the final decision map is achieved to fuse the source images by post-processing the phase decision graph with morphology. Experimental results show that the proposed method achieves the goal of reducing time complexity and attains better fusion effect compared with the state-of-the-art fusion methods which utilize overlapping blocks.
- Multi-focus image fusion,
- Convolutional Neural Network (CNN),
- Super pixel segmentation,
- Spatial pyramid pooling

FullText(HTML)

References(24)

References

RAO Yizhou, HE Lin, and ZHU Jiawei. A residual convolutional neural network for pan-shaprening[C]. 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP), Shanghai, China, 2017: 1–4. doi: 10.1109/RSIP.2017.7958807.

YIN Ming, LIU Xiaoning, LIU Yu, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1): 49–64. doi: 10.1109/TIM.2018.2838778

朱浩然, 刘云清, 张文颖. 基于灰度变换与两尺度分解的夜视图像融合[J]. 电子与信息学报, 2019, 41(3): 640–648. doi: 10.11999/JEIT180407

ZHU Haoran, LIU Yunqing, and ZHANG Wenying. Night-vision image fusion based on intensity transformation and two-scale decomposition[J]. Journal of Electronics &Information Technology, 2019, 41(3): 640–648. doi: 10.11999/JEIT180407

PETROVIC V S and XYDEAS C S. Gradient-based multiresolution image fusion[J]. IEEE Transactions on Image Processing, 2004, 13(2): 228–237. doi: 10.1109/TIP.2004.823821

LEWIS J J, O’CALLAGHAN R J, NIKOLOV S G, et al. Pixel- and region-based image fusion with complex wavelets[J]. Information Fusion, 2007, 8(2): 119–130. doi: 10.1016/j.inffus.2005.09.006

ZHANG Qiang and GUO Baolong. Multifocus image fusion using the nonsubsampled contourlet transform[J]. Signal Processing, 2009, 89(7): 1334–1346. doi: 10.1016/j.sigpro.2009.01.012

LI Shutao, KANG Xudong, FANG Leyuan, et al. Pixel-level image fusion: A survey of the state of the art[J]. Information Fusion, 2017, 33: 100–112. doi: 10.1016/j.inffus.2016.05.004

LI Shutao, KANG Xudong, and HU Jianwen. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864–2875. doi: 10.1109/TIP.2013.2244222

LI Shutao, KANG Xudong, HU Jianwen, et al. Image matting for fusion of multi-focus images in dynamic scenes[J]. Information Fusion, 2013, 14(2): 147–162. doi: 10.1016/j.inffus.2011.07.001

LIU Yu, LIU Shuping, and WANG Zengfu. Multi-focus image fusion with dense SIFT[J]. Information Fusion, 2015, 23: 139–155. doi: 10.1016/j.inffus.2014.05.004

WANG Zhaobin, MA Yide, and GU J. Multi-focus image fusion using PCNN[J]. Pattern Recognition, 2010, 43(6): 2003–2016. doi: 10.1016/j.patcog.2010.01.011

LIU Yu, CHEN Xun, PENG Hu, et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 2017, 36: 191–207. doi: 10.1016/j.inffus.2016.12.001

TANG Han, XIAO Bin, LI Weisheng, et al. Pixel convolutional neural network for multi-focus image fusion[J]. Information Sciences, 2018, 433/434: 125–141. doi: 10.1016/j.ins.2017.12.043

ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274–2282. doi: 10.1109/TPAMI.2012.120

REN Xiaofeng and MALIK J. Learning a classification model for segmentation[C]. The 9th IEEE International Conference on Computer Vision, Nice, France, 2003: 10–17. doi: 10.1109/ICCV.2003.1238308.

ESKICIOGLU A M and FISHER P S. Image quality measures and their performance[J]. IEEE Transactions on Communications, 1995, 43(12): 2959–2965. doi: 10.1109/26.477498

SHELHAMER E, LONG J, and DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640–651. doi: 10.1109/TPAMI.2016.2572683

HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904–1916. doi: 10.1109/TPAMI.2015.2389824

肖斌, 唐翰, 徐韵秋, 等. 基于Hess矩阵的多聚焦图像融合方法[J]. 电子与信息学报, 2018, 40(2): 255–263. doi: 10.11999/JEIT170497

XIAO Bin, TANG Han, XU Yunqiu, et al. Multi-focus image fusion based on Hess matrix[J]. Journal of Electronics &Information Technology, 2018, 40(2): 255–263. doi: 10.11999/JEIT170497

CAO Liu, JIN Longxu, TAO Hongjiang, et al. Multi-focus image fusion based on spatial frequency in discrete cosine transform domain[J]. IEEE Signal Processing Letters, 2015, 22(2): 220–224. doi: 10.1109/LSP.2014.2354534

LIU Zheng, BLASCH E, XUE Zhiyun, et al. Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(1): 94–109. doi: 10.1109/TPAMI.2011.109

WANG Zhou and LI Qiang. Information content weighting for perceptual image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(5): 1185–1198. doi: 10.1109/TIP.2010.2092435

ZHAO Yuxin, JIA Renfeng, and SHI Peng. A novel combination method for conflicting evidence based on inconsistent measurements[J]. Information Sciences, 2016, 367/368: 125–142. doi: 10.1016/j.ins.2016.05.039

LIU Anmin, LIN Weisi, and NARWARIA M. Image quality assessment based on gradient similarity[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1500–1512. doi: 10.1109/TIP.2011.2175935

Relative Articles

Supplements(0)

Cited By

Proportional views