Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data

Fei ZHAO; Wenkai ZHANG; Zhiyuan YAN; Hongfeng YU; Wenhui DIAO

doi:10.11999/JEIT190047

Volume 41 Issue 10

Oct. 2019

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2019 > 41(10): 2525-2531

Fei ZHAO, Wenkai ZHANG, Zhiyuan YAN, Hongfeng YU, Wenhui DIAO. Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data[J]. Journal of Electronics & Information Technology, 2019, 41(10): 2525-2531. doi: 10.11999/JEIT190047

Citation:

Fei ZHAO, Wenkai ZHANG, Zhiyuan YAN, Hongfeng YU, Wenhui DIAO. Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data[J]. Journal of Electronics & Information Technology, 2019, 41(10): 2525-2531. doi: 10.11999/JEIT190047

Citation:

PDF( 1193 KB)

Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data

doi: 10.11999/JEIT190047

Fei ZHAO^{1, 2},
Wenkai ZHANG^{1, 3, 4
,
,},
Zhiyuan YAN^{1, 3, 4},
Hongfeng YU^{1, 3, 4},
Wenhui DIAO^{1, 3, 4}

1.
University of Chinese Academy of Sciences, Beijing 100049, China
2.
Beijing Institute of Tracking and Telecommunications Technology, Beijing 100049, China
3.
Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China
4.
Key Laboratory of Spatial Information Processing and Application System Technology, Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China

Funds: The National Natural Science Foundation of China (41701508)

Received Date: 2019-01-17
Rev Recd Date: 2019-04-08

Available Online: 2019-04-20

Publish Date: 2019-10-01

Abstract

Abstract

Utilizing multiple data (elevation information) to assist remote sensing image segmentation is an important research topic in recent years. However, the existing methods usually directly use multivariate data as the input of the model, which fails to make full use of the multi-level features. In addition, the target size varies in remote sensing images, for some small targets, such as vehicles, houses, etc., it is difficult to achieve detailed segmentation. Considering these problems, a Multi-Feature map Pyramid fusion deep Network (MFPNet) is proposed, which utilizes optical remote sensing images and elevation data as input to extract multi-level features from images. Then the pyramid pooling structure is introduced to extract the multi-scale features from different levels. Finally, a multi-level and multi-scale feature fusion strategy is designed, which utilizes comprehensively the feature information of multivariate data to achieve detailed segmentation of remote sensing images. Experiment results on the Vaihingen dataset demonstrate the effectiveness of the proposed method.
- Semantic segmentation,
- Deep Convolutional Neural Network(DCNN),
- Feature map fusion,
- Pyramid pooling

FullText(HTML)

References(16)

References

DALAL N and TRIGGS B. Histograms of oriented gradients for human detection[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, USA, 2005: 886–893.

LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91–110. doi: 10.1023/B:VISI.0000029664.99615.94

SHOTTON J, JOHNSON M, and CIPOLLA R. Semantic texton forests for image categorization and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 1–8.

KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. The 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, 2012: 1097–1105.

LONG J, SHELHAMER E, and DARRELL T. Fully convolutional networks for semantic segmentation[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 3431–3440.

KAMPFFMEYER M, SALBERG A B, and JENSSEN R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[C]. The IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, USA, 2016: 1–9.

MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Convolutional neural networks for large-scale remote-sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2): 645–657. doi: 10.1109/TGRS.2016.2612821

SHELHAMER E, LONG J, and DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640–651. doi: 10.1109/TPAMI.2016.2572683

MARMANIS D, WEGNER J D, GALLIANI S, et al. Semantic Segmentation of Aerial Images with an Ensemble of CNNS[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, III-3: 473–480. doi: 10.5194/isprsannals-III-3-473-2016

SHERRAH J. Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery[J]. arXiv: 1606.02585, 2016.

ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2016: 6230–6239.

HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.

HAZIRBAS C, MA L N, DOMOKOS C, et al. FuseNet: Incorporating depth into semantic segmentation via fusion-based CNN architecture[C]. The 13th Asian Conference on Computer Vision, Taipei, China, 2016.

ISPRS 2D semantic labeling contest[EB/OL]. http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html, 2019.

ABADI M, BARHAM P, CHEN Jianmin, et al. TensorFlow: A system for large-scale machine learning[C]. The 12th USENIX Conference on Operating Systems Design and Implementation, Savannah, USA, 2016.

CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834–848. doi: 10.1109/TPAMI.2017.2699184

Relative Articles

Supplements(0)

Cited By

Proportional views