高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于一致性生成对抗的遥感多时相建筑物变化检测数据对生成技术

陈昊 周光尧 王乾通 高斌 王文志 唐皓

陈昊, 周光尧, 王乾通, 高斌, 王文志, 唐皓. 基于一致性生成对抗的遥感多时相建筑物变化检测数据对生成技术[J]. 电子与信息学报. doi: 10.11999/JEIT240720
引用本文: 陈昊, 周光尧, 王乾通, 高斌, 王文志, 唐皓. 基于一致性生成对抗的遥感多时相建筑物变化检测数据对生成技术[J]. 电子与信息学报. doi: 10.11999/JEIT240720
HAO Chen, GUANGYAO Zhou, QIANTONG Wang, BIN Gao, WENZHI Wang, HAO Tang. Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240720
Citation: HAO Chen, GUANGYAO Zhou, QIANTONG Wang, BIN Gao, WENZHI Wang, HAO Tang. Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240720

基于一致性生成对抗的遥感多时相建筑物变化检测数据对生成技术

doi: 10.11999/JEIT240720
详细信息
    作者简介:

    陈昊:男,副研究员,研究方向为遥感图像处理,变化检测

    周光尧:男,研究员,研究方向为遥感信息处理系统,遥感图像解译

    王乾通:男,助理研究员,研究方向为光学遥感影像解译

    高斌:男,助理研究员,研究方向为遥感信息处理系统

    王文志:男,助理研究员,研究方向为遥感信息处理系统,图像解译,多源信息融合处理

    唐皓:男,高级工程师,研究方向为遥感信息处理系统,高光谱图像处理,目标智能解译

    通讯作者:

    周光尧 zhougy@aircas.an.cn

  • 中图分类号: TN991.73; TP751.2

Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery

  • 摘要: 虽然目前可以获取海量的多时相遥感数据,但是由于建筑物变化时间周期过长,难以获取充足的建筑物变化数据对儿来支撑数据驱动的深度学习变化检测模型构建,呈现多时相遥感建筑物变化检测处理精度差的问题。因此,为提升变化检测算法模型处理性能,该文从建筑物变化检测训练数据对生成开展研究,基于一致性对抗生成机理提出了多时相建筑物变化检测数据对生成网络(BAG-GAN)。其主要在多时相图像生成过程中采用对抗一致性损失函数约束,在保证生成图像和输入图像关联性的同时,保证了生成模型的多模态输出能力。此外,还通过重组原数据集中的变化标签和多时相遥感图像来进一步提升建筑物变化信息生成的多样性,解决了训练数据中有效建筑物变化信息占比少的问题,为变化监测算法模型的充分训练奠定了基础。最后,在LEVIR-CD和WHU-CD建筑变化检测数据集上进行了数据生成实验,并使用生成扩充后的数据集训练了多种较为经典的遥感图像变化检测模型,实验结果表明该文提出的BAG-GAN多时相建筑物变化检测数据对生成网络及相应的生成策略可以有效提升变化检测模型的处理精度。
  • 图  1  本文所提一致性生成对抗BAG-GAN的数据生成方法

    图  2  循环一致性和对抗一致性对比原理

    图  3  本文所提BAG-GAN变化检测数据生成可视化结果

    图  4  LEVIR-CD(a)原始数据集(未经生成增广)变化检测结果;(b) 数据生成增广后的变化检测结果

    图  5  WHU-CD (a)原始数据集(未经生成增广)变化检测结果;(b)数据生成增广后的变化检测结果

    图  6  CD模型性能随数据集类不平衡率变化折线图

    表  1  WHU-CD模型性能提升(20%与100%)

    变化检测模型LEVIR-CD(20%)
    Prec/Rec/IoU
    LEVIR-CD(100%)
    Prec/Rec/IoU
    FC-EF
    +数据增强变换
    0.689/0.696/0.595
    0.683/0.654/0.511
    0.769/0.682/0.620
    0.771/0.665/0.593
    +BAG-GAN0.863/0.641/0.6110.875/0.757/0.701
    FC-Siam-Conc
    +数据增强变换
    0.615/0.709/0.541
    0.609/0.698/0.531
    0.696/0.802/0.628
    0.667/0.735/0.609
    +BAG-GAN0.894/0.711/0.6680.922/0.741/0.691
    FC-Siam-Diff
    +数据增强变换
    0.581/0.690/0.495
    0.573/0.634/0.487
    0.654/0.787/0.586
    0.647/0.772/0.557
    +BAG-GAN0.866/0.618/0.5640.889/0.781/0.737
    SNUNet
    +数据增强变换
    0.940/0.916/0.872
    0.913/0.920/0.865
    0.956/0.951/0.914
    0.903/0.944/0.887
    +BAG-GAN0.933/0.938/0.8760.961/0.958/0.924
    下载: 导出CSV

    表  2  LEVIR-CD模型性能提升(20%与100%)

    变化检测模型LEVIR-CD(20%)
    Prec/Rec/IoU
    LEVIR-CD(100%)
    Prec/Rec/IoU
    FC-EF
    +数据增强变换
    0.689/0.696/0.595
    0.683/0.654/0.511
    0.769/0.682/0.620
    0.771/0.665/0.593
    +BAG-GAN0.863/0.641/0.6110.875/0.757/0.701
    FC-Siam-Conc
    +数据增强变换
    0.615/0.709/0.541
    0.609/0.698/0.531
    0.696/0.802/0.628
    0.667/0.735/0.609
    +BAG-GAN0.894/0.711/0.6680.922/0.741/0.691
    FC-Siam-Diff
    +数据增强变换
    0.581/0.690/0.495
    0.573/0.634/0.487
    0.654/0.787/0.586
    0.647/0.772/0.557
    +BAG-GAN0.866/0.618/0.5640.889/0.781/0.737
    SNUNet
    +数据增强变换
    0.940/0.916/0.872
    0.913/0.920/0.865
    0.956/0.951/0.914
    0.903/0.944/0.887
    +BAG-GAN0.933/0.938/0.8760.961/0.958/0.924
    下载: 导出CSV
  • [1] NASTASE I N A, MOLDOVANU S, and MORARU L. Deep learning-based segmentation of breast masses using convolutional neural networks[J]. Journal of Physics: Conference Series, 2024, 2701(1): 012005. doi: 10.1088/1742-6596/2701/1/012005.
    [2] 刘美琴, 王子麟. 基于域适应的图像语义分割综述[J]. 北京交通大学学报, 2024, 48(2): 1–9. doi: 10.11860/j.issn.1673-0291.20230120.

    LIU Meiqin and WANG Zilin. A review on image semantic segmentation based on domain adaptation[J]. Journal of Beijing Jiaotong University, 2024, 48(2): 1–9. doi: 10.11860/j.issn.1673-0291.20230120.
    [3] EL-MAGD S A A, MASOUD A M, HASSAN H S, et al. Towards understanding climate change: Impact of land use indices and drainage on land surface temperature for valley drainage and non-drainage areas[J]. Journal of Environmental Management, 2024, 350: 119636. doi: 10.1016/j.jenvman.2023.119636.
    [4] HOSSAIN M S, KHAN M A H, OLUWAJUWON T V, et al. Spatiotemporal change detection of land use land cover (LULC) in Fashiakhali wildlife sanctuary (FKWS) impact area, Bangladesh, employing multispectral images and GIS[J]. Modeling Earth Systems and Environment, 2023, 9(3): 3151–3173. doi: 10.1007/s40808-022-01653-7.
    [5] 何自芬, 史本杰, 张印辉, 等. 多注意力融合的环高原湖泊遥感影像分割[J]. 电子学报, 2023, 51(4): 885–895. doi: 10.12263/DZXB.20220085.

    HE Zifen, SHI Benjie, ZHANG Yinhui, et al. Remote sensing image segmentation of around plateau lakes based on multi-attention fusion[J]. Acta Electronica Sinica, 2023, 51(4): 885–895. doi: 10.12263/DZXB.20220085.
    [6] 张冬梅, 李石磊. 一种显著性检测提取高分遥感影像建筑物的方法[J]. 测绘与空间地理信息, 2024, 47(6): 97–101. doi: 10.3969/j.issn.1672-5867.2024.06.029.

    ZHANG Dongmei and LI Shilei. A saliency detection method for extracting buildings from high-resolution remote sensing images[J]. Geomatics & Spatial Information Technology, 2024, 47(6): 97–101. doi: 10.3969/j.issn.1672-5867.2024.06.029.
    [7] MEI Jie, ZHENG Yibo, and CHENG Mingming. D2ANet: Difference-aware attention network for multi-level change detection from satellite imagery[J]. Computational Visual Media, 2023, 9(3): 563–579. doi: 10.1007/s41095-022-0325-1.
    [8] LIAO Cheng, HU Han, YUAN Xuekun, et al. BCE-Net: Reliable building footprints change extraction based on historical map and up-to-date images using contrastive learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 201: 138–152. doi: 10.1016/j.isprsjprs.2023.05.011.
    [9] KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, USA, 2012: 1097–1105.
    [10] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
    [11] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269. doi: 10.1109/CVPR.2017.243.
    [12] SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. 3rd International Conference on Learning Representations, San Diego, USA, 2015. doi: 10.48550/arXiv.1409.1556.
    [13] DEMIRCI M Y, BEŞLI N, and GÜMÜŞÇÜ A. An improved hybrid solar cell defect detection approach using generative adversarial networks and weighted classification[J]. Expert Systems with Applications, 2024, 252: 124230. doi: 10.1016/j.eswa.2024.124230.
    [14] 刘少鹏, 赵慧民, 洪佳明, 等. 面向医学图像生成的鲁棒条件生成对抗网络[J]. 电子学报, 2023, 51(2): 427–437. doi: 10.12263/DZXB.20210051.

    LIU Shaopeng, ZHAO Huimin, HONG Jiaming, et al. Medical image synthesis using robust conditional GAN[J]. Acta Electronica Sinica, 2023, 51(2): 427–437. doi: 10.12263/DZXB.20210051.
    [15] STANCIU D C and IONESCU B. Autoencoder-based data augmentation for deepfake detection[C]. Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation, Thessaloniki, Greece, 2023: 19–27. doi: 10.1145/3592572.3592840.
    [16] YANG Suorong, XIAO Weikang, ZHANG Mengchen, et al. Image data augmentation for deep learning: A survey[EB/OL]. https://arxiv.org/abs/2204.08610, 2022.
    [17] OUBARA A, WU Falin, AMAMRA A, et al. Survey on remote sensing data augmentation: Advances, challenges, and future perspectives[C]. 5th Conference on Computing Systems and Applications, Cham, Switzerland, 2022: 95–104. doi: 10.1007/978-3-031-12097-8_9.
    [18] CRESWELL A, WHITE T, DUMOULIN V, et al. Generative adversarial networks: An overview[J]. IEEE Signal Processing Magazine, 2018, 35(1): 53–65. doi: 10.1109/MSP.2017.2765202.
    [19] BOWLES C, CHEN Liang, GUERRERO R, et al. GAN augmentation: Augmenting training data using generative adversarial networks[EB/OL]. https://arxiv.org/abs/1810.10863, 2018.
    [20] DOERSCH C. Tutorial on variational autoencoders[EB/OL]. https://arxiv.org/abs/1606.05908, 2016.
    [21] WANG Yinglin, ZHAO Jianhui, GUO Zhengwei, et al. Soil moisture inversion based on data augmentation method using multi-source remote sensing data[J]. Remote Sensing, 2023, 15(7): 1899. doi: 10.3390/rs15071899.
    [22] HUANG Rui, WANG Ruofei, GUO Qing, et al. Background-mixed augmentation for weakly supervised change detection[C]. Proceedings of the 37th AAAI Conference on Artificial Intelligence, Washington, USA, 2023: 7919–7927. doi: 10.1609/aaai.v37i7.25958.
    [23] 但志平, 方帅领, 孙航, 等. 基于双判别器异构CycleGAN框架下多阶通道注意力校准的室外图像去雾[J]. 电子学报, 2023, 51(9): 2558–2571. doi: 10.12263/DZXB.20211337.

    DAN Zhiping, FANG Shuailing, SUN Hang, et al. Outdoor image dehazing based on multi-order channel attention calibration using a dual-discriminator heterogeneous CycleGAN framework[J]. Acta Electronica Sinica, 2023, 51(9): 2558–2571. doi: 10.12263/DZXB.20211337.
    [24] 马宾, 王一利, 徐健, 等. 基于双向生成对抗网络的图像感知哈希算法[J]. 电子学报, 2023, 51(5): 1405–1412. doi: 10.12263/DZXB.20221224.

    MA Bin, WANG Yili, XU Jian, et al. An image perceptual hash algorithm based on bidirectional generative adversarial network[J]. Acta Electronica Sinica, 2023, 51(5): 1405–1412. doi: 10.12263/DZXB.20221224.
    [25] 贾童瑶, 卓力, 李嘉锋, 等. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231–245. doi: 10.12263/DZXB.20220838.

    JIA Tongyao, ZHUO Li, LI Jiafeng, et al. Research advances on deep learning based single image dehazing[J]. Acta Electronica Sinica, 2023, 51(1): 231–245. doi: 10.12263/DZXB.20220838.
    [26] 李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128–138. doi: 10.12263/DZXB.20210659.

    LI Tao, DONG Xiucheng, and LIN Hongwei. Depth map super-resolution reconstruction based on deeply supervised cross-scale attention network[J]. Acta Electronica Sinica, 2023, 51(1): 128–138. doi: 10.12263/DZXB.20210659.
    [27] JI Shunping, WEI Shiqing, and LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574–586. doi: 10.1109/tgrs.2018.2858817.
    [28] LI Xuan, DUAN Haibin, ZHANG Hui, et al. Data augmentation using image generation for change detection[C]. 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence (DTPI), Beijing, China, 2021: 188–191. doi: 10.1109/DTPI52967.2021.9540199.
    [29] VARGHESE A, GUBBI J, RAMASWAMY A, et al. ChangeNet: A deep learning architecture for visual change detection[C]. Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, 2018: 129–145. doi: 10.1007/978-3-030-11012-3_10.
    [30] SHAFIQUE A, CAO Guo, KHAN Z, et al. Deep learning-based change detection in remote sensing images: A review[J]. Remote Sensing, 2022, 14(4): 871. doi: 10.3390/rs14040871.
    [31] 张良培, 武辰. 多时相遥感影像变化检测的现状与展望[J]. 测绘学报, 2017, 46(10): 1447–1458. doi: 10.11947/j.AGCS.2017.20170340.

    ZHANG Liangpei and WU Chen. Advance and future development of change detection for multi-temporal remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10): 1447–1459. doi: 10.11947/j.AGCS.2017.20170340.
    [32] 眭海刚, 冯文卿, 李文卓, 等. 多时相遥感影像变化检测方法综述[J]. 武汉大学学报: 信息科学版, 2018, 43(12): 1885–1898. doi: 10.13203/j.whugis20180251.

    SUI Haigang, FENG Wenqing, LI Wenzhuo, et al. Review of change detection methods for multi-temporal remote sensing imagery[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 1885–1898. doi: 10.13203/j.whugis20180251.
    [33] 佟国峰, 李勇, 丁伟利, 等. 遥感影像变化检测算法综述[J]. 中国图象图形学报, 2015, 20(12): 1561–1571. doi: 10.11834/jig.20151201.

    TONG Guofeng, LI Yong, DING Weili, et al. Review of remote sensing image change detection[J]. Journal of Image and Graphics, 2015, 20(12): 1561–1571. doi: 10.11834/jig.20151201.
    [34] JI Shunping, SHEN Yanyun, LU Meng, et al. Building instance change detection from large-scale aerial images using convolutional neural networks and simulated samples[J]. Remote Sensing, 2019, 11(11): 1343. doi: 10.3390/rs11111343.
    [35] NEMOTO K, HAMAGUCHI R, SATO M, et al. Building change detection via a combination of CNNs using only RGB aerial imageries[C]. Proceedings of SPIE 10431, Remote Sensing Technologies and Applications in Urban Environments II, Warsaw, Poland, 2017: 104310J. doi: 10.1117/12.2277912.
    [36] LIU Ruoyun, KUFFER M, and PERSELLO C. The temporal dynamics of slums employing a CNN-based change detection approach[J]. Remote Sensing, 2019, 11(23): 2844. doi: 10.3390/rs11232844.
    [37] LIU Yi, PANG Chao, ZHAN Zongqian, et al. Building change detection for remote sensing images using a dual-task constrained deep Siamese convolutional network model[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5): 811–815. doi: 10.1109/LGRS.2020.2988032.
    [38] CHEN Hao and SHI Zhenwei. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J]. Remote Sensing, 2020, 12(10): 1662. doi: 10.3390/rs12101662.
    [39] JIANG Huiwei, HU Xiangyun, LI Kun, et al. PGA-SiamNet: Pyramid feature-based attention-guided Siamese network for remote sensing orthoimagery building change detection[J]. Remote Sensing, 2020, 12(3): 484. doi: 10.3390/rs12030484.
    [40] ZHAN Yang, FU Kun, YAN Menglong, et al. Change detection based on deep Siamese convolutional network for optical aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1845–1849. doi: 10.1109/LGRS.2017.2738149.
    [41] SUN Chen, SHRIVASTAVA A, SINGH S, et al. Revisiting unreasonable effectiveness of data in deep learning era[C]. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 843–852. doi: 10.1109/ICCV.2017.97.
    [42] ALBAHLI S and ALBATTAH W. Deep transfer learning for COVID-19 prediction: Case study for limited data problems[J]. Current Medical Imaging, 2021, 17(8): 973–980. doi: 10.2174/1573405616666201123120417.
    [43] HAO Hanxiang, BAIREDDY S, BARTUSIAK E R, et al. An attention-based system for damage assessment using satellite imagery[C]. 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 2021: 4396–4399. doi: 10.1109/IGARSS47720.2021.9554054.
    [44] SEO M, LEE H, JEON Y, et al. Self-pair: Synthesizing changes from single source for object change detection in remote sensing imagery[C]. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, USA, 2023: 6363–6372. doi: 10.1109/WACV56688.2023.00631.
    [45] CHEN Hao, LI Wenyuan, and SHI Zhenwei. Adversarial instance augmentation for building change detection in remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5603216. doi: 10.1109/TGRS.2021.3066802.
    [46] KUMDAKCI H, ÖNGÜN C, and TEMIZEL A. Generative data augmentation for vehicle detection in aerial images[C]. Pattern Recognition. ICPR International Workshops and Challenges, 2021: 19–31. doi: 10.1007/978-3-030-68793-9_2. (查阅网上资料,未找到本条文献出版地信息,请确认) .
    [47] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2242–2251. doi: 10.1109/ICCV.2017.244.
    [48] JIANG Yuchen, ZHU Bin, and XIE Bo. Remote sensing images data augmentation based on style transfer under the condition of few samples[J]. Journal of Physics: Conference Series, 2020, 1653(1): 012039. doi: 10.1088/1742-6596/1653/1/012039.
    [49] ZHAO Yihao, WU Ruihai, and DONG Hao. Unpaired image-to-image translation using adversarial consistency loss[C]. 16th European Conference on Computer Vision, Glasgow, UK, 2020: 800–815. doi: 10.1007/978-3-030-58545-7_46.
    [50] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976. doi: 10.1109/CVPR.2017.632.
    [51] DAUDT R C, LE SAUX B, and BOULCH A. Fully convolutional Siamese networks for change detection[C]. 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018: 4063–4067. doi: 10.1109/ICIP.2018.8451652.
    [52] FANG Sheng, LI Kaiyu, SHAO Jinyuan, et al. SNUNet-CD: A densely connected Siamese network for change detection of VHR images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8007805. doi: 10.1109/LGRS.2021.3056416.
  • 加载中
图(6) / 表(2)
计量
  • 文章访问数:  12
  • HTML全文浏览量:  6
  • PDF下载量:  3
  • 被引次数: 0
出版历程
  • 收稿日期:  2024-08-19
  • 修回日期:  2025-02-10
  • 网络出版日期:  2025-02-24

目录

    /

    返回文章
    返回