Feature Fusion Classification for Optical Image and SAR Image Based on Spatial-spectral Attention

JIANG Wen; PAN Jie; ZHU Jinbiao; YUE Xijuan

doi:10.11999/JEIT220063

Volume 45 Issue 3

Mar. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(3): 987-995

JIANG Wen, PAN Jie, ZHU Jinbiao, YUE Xijuan. Feature Fusion Classification for Optical Image and SAR Image Based on Spatial-spectral Attention[J]. Journal of Electronics & Information Technology, 2023, 45(3): 987-995. doi: 10.11999/JEIT220063

Citation:

JIANG Wen, PAN Jie, ZHU Jinbiao, YUE Xijuan. Feature Fusion Classification for Optical Image and SAR Image Based on Spatial-spectral Attention[J]. Journal of Electronics & Information Technology, 2023, 45(3): 987-995. doi: 10.11999/JEIT220063

Citation:

PDF( 24578 KB)

Feature Fusion Classification for Optical Image and SAR Image Based on Spatial-spectral Attention

doi: 10.11999/JEIT220063

JIANG Wen¹,
PAN Jie^{1
,
,},
ZHU Jinbiao^{1, 2},
YUE Xijuan¹

1.
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China
2.
Northwestern Polytechnical University, Xi’an 710072, China

Received Date: 2022-01-13
Rev Recd Date: 2022-05-28

Available Online: 2022-06-10

Publish Date: 2023-03-10

Abstract

Abstract

Considering the issue of difference and complementarity of multi-source remote sensing images, this paper proposes a feature fusion classification method for optical image and SAR image based on spatial-spectral attention. Firstly, features of optical image and SAR image are extracted by the convolutional neural network, and an attention module composed of spatial attention and spectral attention is designed to analyze the importance of features. Features can be enhanced by the weights of the attention module, which can reduce the attention to irrelevant information, and thus improve the accuracy of fusion classification for optical and SAR images. Experimental results on two datasets of optical image and SAR image demonstrate that the proposed method is able to yield higher fusion classification accuracy.
- SAR image,
- Deep learning,
- Feature fusion,
- Attention mechanism

FullText(HTML)

References(24)

References

[1]	SUKAWATTANAVIJIT C, CHEN Jie, and ZHANG Hongsheng. GA-SVM algorithm for improving land-cover classification using SAR and optical remote sensing data[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(3): 284–288. doi: 10.1109/LGRS.2016.2628406
[2]	李璐, 杜兰, 何浩男, 等. 基于深度森林的多级特征融合SAR目标识别[J]. 电子与信息学报, 2021, 43(3): 606–614. doi: 10.11999/JEIT200685 LI Lu, DU Lan, HE Haonan, et al. Multi-level feature fusion SAR automatic target recognition based on deep forest[J]. Journal of Electronics &Information Technology, 2021, 43(3): 606–614. doi: 10.11999/JEIT200685
[3]	ALONSO-GONZÁLEZ A, LÓPEZ-MARTÍNEZ C, PAPATHANASSIOU K P, et al. Polarimetric SAR time series change analysis over agricultural areas[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(10): 7317–7330. doi: 10.1109/TGRS.2020.2981929
[4]	ZHANG Hongsheng and XU Ru. Exploring the optimal integration levels between SAR and optical data for better urban land cover mapping in the Pearl River Delta[J]. International Journal of Applied Earth Observation and Geoinformation, 2018, 64: 87–95. doi: 10.1016/j.jag.2017.08.013
[5]	KUSSUL N, LAVRENIUK M, SKAKUN S, et al. Deep learning classification of land cover and crop types using remote sensing data[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(5): 778–782. doi: 10.1109/LGRS.2017.2681128
[6]	ZHANG Xiangrong, WANG Xin, TANG Xu, et al. Description generation for remote sensing images using attribute attention mechanism[J]. Remote Sensing, 2019, 11(6): 612. doi: 10.3390/rs11060612
[7]	XIE Jie, HE Nanjun, FANG Leyuan, et al. Scale-free convolutional neural network for remote sensing scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6916–6928. doi: 10.1109/TGRS.2019.2909695
[8]	KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. Proceedings of the 25th International Conference on neural Information Processing Systems, Lake Tahoe, USA, 2012: 1097–1105.
[9]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv: 1409.1556, 2014.
[10]	SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 1–9.
[11]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Identity mappings in deep residual networks[C]. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 630–645.
[12]	周顺杰, 杨学志, 董张玉, 等. 面向特征识别的SAR与可见光图像融合算法研究[J]. 合肥工业大学学报:自然科学版, 2018, 41(7): 900–907. doi: 10.3969/j.issn.1003-5060.2018.07.008 ZHOU Shunjie, YANG Xuezhi, DONG Zhangyu, et al. Fusion algorithm of SAR and visible images for feature recognition[J]. Journal of Hefei University of Technology, 2018, 41(7): 900–907. doi: 10.3969/j.issn.1003-5060.2018.07.008
[13]	雷俊杰, 杨武年, 李红, 等. 哨兵光学及SAR卫星影像协同分类研究[J]. 现代电子技术, 2022, 45(2): 135–139. doi: 10.16652/j.issn.1004-373x.2022.02.026 LEI Junjie, YANG Wunian, LI Hong, et al. Research on cooperative classification of sentinel optical and SAR satellite images[J]. Modern Electronics Technique, 2022, 45(2): 135–139. doi: 10.16652/j.issn.1004-373x.2022.02.026
[14]	KONG Yingying, YAN Biyuan, LIU Yanjuan, et al. Feature-level fusion of polarized SAR and optical images based on random forest and conditional random fields[J]. Remote Sensing, 2021, 13(7): 1323. doi: 10.3390/rs13071323
[15]	XU Zhe, ZHU Jinbiao, GENG Jie, et al. Triplet attention feature fusion network for SAR and optical image land cover classification[C]. Proceedings of 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 2021: 4256–4259.
[16]	XU Xiaodong, LI Wei, RAN Qiong, et al. Multisource remote sensing data classification based on convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2): 937–949. doi: 10.1109/TGRS.2017.2756851
[17]	HONG Danfeng, GAO Lianru, YOKOYA N, et al. More diverse means better: Multimodal deep learning meets remote-sensing imagery classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(5): 4340–4354. doi: 10.1109/TGRS.2020.3016820
[18]	HU Jie, SHEN Li, and SUN Gang. Squeeze-and-excitation networks[C]. Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 7132–7141.
[19]	WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 3–19.
[20]	PARK J, WOO S, LEE J Y, et al. A simple and light-weight attention module for convolutional neural networks[J]. International Journal of Computer Vision, 2020, 128(4): 783–798. doi: 10.1007/s11263-019-01283-0
[21]	FU Jun, LIU Jing, TIAN Haijie, et al. Dual attention network for scene segmentation[C]. Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 3141–3149.
[22]	MISRA D. Mish: A self regularized non-Monotonic neural activation function[J]. arXiv: 1908.08681, 2019.
[23]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
[24]	徐从安, 吕亚飞, 张筱晗, 等. 基于双重注意力机制的遥感图像场景分类特征表示方法[J]. 电子与信息学报, 2021, 43(3): 683–691. doi: 10.11999/JEIT200568 XU Cong’an, LÜ Yafei, ZHANG Xiaohan, et al. A discriminative feature representation method based on dual attention mechanism for remote sensing image scene classification[J]. Journal of Electronics &Information Technology, 2021, 43(3): 683–691. doi: 10.11999/JEIT200568