高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于多特征图金字塔融合深度网络的遥感图像语义分割

赵斐 张文凯 闫志远 于泓峰 刁文辉

赵斐, 张文凯, 闫志远, 于泓峰, 刁文辉. 基于多特征图金字塔融合深度网络的遥感图像语义分割[J]. 电子与信息学报, 2019, 41(10): 2525-2531. doi: 10.11999/JEIT190047
引用本文: 赵斐, 张文凯, 闫志远, 于泓峰, 刁文辉. 基于多特征图金字塔融合深度网络的遥感图像语义分割[J]. 电子与信息学报, 2019, 41(10): 2525-2531. doi: 10.11999/JEIT190047
Fei ZHAO, Wenkai ZHANG, Zhiyuan YAN, Hongfeng YU, Wenhui DIAO. Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data[J]. Journal of Electronics & Information Technology, 2019, 41(10): 2525-2531. doi: 10.11999/JEIT190047
Citation: Fei ZHAO, Wenkai ZHANG, Zhiyuan YAN, Hongfeng YU, Wenhui DIAO. Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data[J]. Journal of Electronics & Information Technology, 2019, 41(10): 2525-2531. doi: 10.11999/JEIT190047

基于多特征图金字塔融合深度网络的遥感图像语义分割

doi: 10.11999/JEIT190047
基金项目: 国家自然科学基金(41701508)
详细信息
    作者简介:

    赵斐:男,1974年生,高级工程师,研究方向为遥感图像目标检测

    张文凯:男,1990年生,助理研究员,研究方向为图像集视觉总结,遥感图像分类

    闫志远:女,1994年生,硕士,研究方向为遥感图像语义分割

    于泓峰:男,1991年生,助理研究员,研究方向为遥感图像智能解译

    刁文辉:男,1988年生,助理研究员,研究方向为遥感图像目标检测

    通讯作者:

    张文凯 iecas_wenkai@yahoo.com

  • 中图分类号: TP391.41

Multi-feature Map Pyramid Fusion Deep Network for Semantic Segmentation on Remote Sensing Data

Funds: The National Natural Science Foundation of China (41701508)
  • 摘要: 在遥感图像语义分割中,利用多元数据(如高程信息)进行辅助是一个研究重点。现有的基于多元数据的分割方法通常直接将多元数据作为模型的多特征输入,未能充分利用多元数据的多层次特征,此外,遥感图像中目标尺寸大小不一,对于一些中小型目标,如车辆、房屋等,难以做到精细化分割。针对以上问题,提出一种多特征图金字塔融合深度网络(MFPNet),该模型利用光学遥感图像和高程数据作为输入,提取图像的多层次特征,然后针对不同层次的特征,分别引入金字塔池化结构,提取图像的多尺度特征,最后,设计了一种多层次、多尺度特征融合策略,综合利用多元数据的特征信息,实现遥感图像的精细化分割。基于Vaihingen数据集设计了相应的对比实验,实验结果证明了所提方法的有效性。
  • 图  1  多元特征图融合网络模型框架图

    图  2  金字塔池化结构

    图  3  不同方法分割结果对比图

    表  1  特征编码网络结构

    ResNet卷积层光学遥感图像分支输出高程数据分支输出多元特征融合融合输出输出尺寸
    7×7,64,步幅2L1-imgL1-ele1/2
    3×3,最大值池化,步幅2
    $\left. \begin{aligned}& \ \, 1 \times 1,\;64\\ & \ \, 3 \times 3,\;64\;\;\;\; \times 3\\ & \ \, 1 \times 1,\;256 \end{aligned} \right\}$
    L2-imgL2-eleC21/4
    $\left. \begin{aligned} & 1 \times 1,\;128\\ & 3 \times 3,\;128\;\;\;\; \times 4\\ & 1 \times 1,\;512 \end{aligned} \right\}$L3-imgL3-eleC31/8
    $\left. \begin{aligned} & 1 \times 1,\;128\\ & 3 \times 3,\;128\;\; \times 23\\ & 1 \times 1,\;512 \end{aligned} \right\}\left( {{\text{带孔卷积}} } \right)$L4-imgL4-eleC41/8
    $\left. \begin{aligned}& \ \, 1 \times 1,\;512\\ & \ \, 3 \times 3,\;512\;\; \times 3\\ & \ \, 1 \times 1,\;2048 \end{aligned} \right\}\left( {{\text{带孔卷积}} } \right)$L5-imgL5-eleC51/8
    下载: 导出CSV

    表  2  MFPNet模型消融实验结果

    模型mIOUOAF1
    道路建筑物草地树木车辆其它
    Color-E68.9681.770.850.880.720.830.500.59
    MFFNet75.8184.750.890.910.790.870.620.68
    MFPNet77.1085.950.910.960.820.880.760.75
    下载: 导出CSV

    表  3  MFPNet与其他方法的对比结果

    方法mIoUOAF1
    道路建筑物草地树木车辆其它
    FCN59.6579.670.820.860.690.810.560.59
    Deeplab70.8582.750.860.890.720.820.600.61
    PSPNet74.9683.920.900.930.740.810.650.63
    MFPNet77.1085.950.910.960.820.880.760.75
    下载: 导出CSV
  • DALAL N and TRIGGS B. Histograms of oriented gradients for human detection[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, USA, 2005: 886–893.
    LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91–110. doi: 10.1023/B:VISI.0000029664.99615.94
    SHOTTON J, JOHNSON M, and CIPOLLA R. Semantic texton forests for image categorization and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 1–8.
    KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. The 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, 2012: 1097–1105.
    LONG J, SHELHAMER E, and DARRELL T. Fully convolutional networks for semantic segmentation[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 3431–3440.
    KAMPFFMEYER M, SALBERG A B, and JENSSEN R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[C]. The IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, USA, 2016: 1–9.
    MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Convolutional neural networks for large-scale remote-sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(2): 645–657. doi: 10.1109/TGRS.2016.2612821
    SHELHAMER E, LONG J, and DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640–651. doi: 10.1109/TPAMI.2016.2572683
    MARMANIS D, WEGNER J D, GALLIANI S, et al. Semantic Segmentation of Aerial Images with an Ensemble of CNNS[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, III-3: 473–480. doi: 10.5194/isprsannals-III-3-473-2016
    SHERRAH J. Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery[J]. arXiv: 1606.02585, 2016.
    ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2016: 6230–6239.
    HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
    HAZIRBAS C, MA L N, DOMOKOS C, et al. FuseNet: Incorporating depth into semantic segmentation via fusion-based CNN architecture[C]. The 13th Asian Conference on Computer Vision, Taipei, China, 2016.
    ISPRS 2D semantic labeling contest[EB/OL]. http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html, 2019.
    ABADI M, BARHAM P, CHEN Jianmin, et al. TensorFlow: A system for large-scale machine learning[C]. The 12th USENIX Conference on Operating Systems Design and Implementation, Savannah, USA, 2016.
    CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834–848. doi: 10.1109/TPAMI.2017.2699184
  • 加载中
图(3) / 表(3)
计量
  • 文章访问数:  8093
  • HTML全文浏览量:  2891
  • PDF下载量:  275
  • 被引次数: 0
出版历程
  • 收稿日期:  2019-01-17
  • 修回日期:  2019-04-08
  • 网络出版日期:  2019-04-20
  • 刊出日期:  2019-10-01

目录

    /

    返回文章
    返回