Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery

HAO Chen; GUANGYAO Zhou; QIANTONG Wang; BIN Gao; WENZHI Wang; HAO Tang

doi:10.11999/JEIT240720

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

HAO Chen, GUANGYAO Zhou, QIANTONG Wang, BIN Gao, WENZHI Wang, HAO Tang. Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240720

Citation:

HAO Chen, GUANGYAO Zhou, QIANTONG Wang, BIN Gao, WENZHI Wang, HAO Tang. Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240720

Citation:

HAO Chen, GUANGYAO Zhou, QIANTONG Wang, BIN Gao, WENZHI Wang, HAO Tang. Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240720

PDF( 5520 KB)

Consistent Generative Adversarial Based on Building Change Detection Data Generation Technology for Multi-temporal Remote Sensing Imagery

doi: 10.11999/JEIT240720

1.
Beijing Institute of Tracking and Communication Technology, Beijing, 100094
2.
Aerospace Information Innovation Research Institute, Chinese Academy of Sciences, Beijing, 100094

Received Date: 2024-08-19
Rev Recd Date: 2025-02-10

Available Online: 2025-02-24

Abstract

Abstract

Objective Building change detection is an essential task in urban planning, disaster management, environmental monitoring, and other critical applications. Advances in multi-temporal remote sensing technology have provided vast amounts of data, enabling the monitoring of changes over large geographic areas and extended time frames. Despite this, significant challenges persist, particularly in acquiring sufficient labeled data pairs for training deep learning models. Building changes are typically characterized by long temporal cycles, leading to a scarcity of annotated data that is critical for training data-driven deep learning models. This scarcity severely limits the models' capacity to generalize and achieve high accuracy, particularly in complex and diverse scenarios. The performance of existing methods often suffers from poor generalization due to insufficient training data, reducing their applicability to practical tasks. To address these challenges, this study proposes a novel solution: the development of a multi-temporal building change detection data pair generation network, referred to as BAG-GAN. This network leverages a consistency adversarial generation mechanism to create diverse and semantically consistent data pairs. The aim is to enrich training datasets, thereby enhancing the learning capacity of deep learning models for detecting building changes. By addressing the bottleneck of insufficient labeled data, BAG-GAN provides a new pathway for improving the accuracy and robustness of multi-temporal building change detection. Methods BAG-GAN integrates Generative Adversarial Networks (GANs) with a specially designed consistency constraint mechanism, tailored for the generation of data pairs in multi-temporal building change detection tasks. The core innovation of this network lies in its adversarial consistency loss function. This loss function ensures that the generated images maintain semantic consistency with the corresponding input images while reflecting realistic and diverse changes. The consistency constraint is crucial for preserving the integrity of the generated data and ensuring its relevance to real-world scenarios. The network is composed of two main components: a generator and a discriminator, which work in tandem through an adversarial learning process. The generator aims to produce realistic and semantically consistent multi-temporal image pairs, while the discriminator evaluates the quality of the generated data, guiding the generator to improve iteratively. Additionally, BAG-GAN is equipped with multimodal output capabilities, enabling the generation of diverse building change data pairs. This diversity enhances the robustness of deep learning models by exposing them to a wider range of scenarios during training. To address the issue of limited training data, the study incorporates a data augmentation strategy. Original datasets, such as LEVIR-CD and WHU-CD, were reorganized by combining change labels with multi-temporal remote sensing images to create new synthetic datasets. These augmented datasets, along with the data generated by BAG-GAN, were used to train and evaluate several widely recognized deep learning models, including FC-EF, FC-Siam-Conc, and others. Comparative experiments were conducted to assess the effectiveness of BAG-GAN and its contribution to improving model performance in multi-temporal building change detection. Results and Discussions The experimental results demonstrate that BAG-GAN effectively addresses the challenges of insufficient labeled data in building change detection tasks. Models trained on the augmented datasets, which included BAG-GAN-generated data, achieved significant improvements in detection accuracy and robustness. For instance, classic models like FC-EF and FC-Siam-Conc showed substantial performance gains when trained on augmented datasets compared to their performance on the original datasets. These improvements validate the effectiveness of BAG-GAN in generating high-quality training data. BAG-GAN also excelled in producing diverse and multimodal building change data pairs Visual comparisons between the generated data and the original datasets highlighted the network's ability to create realistic and varied data, effectively enhancing the diversity of training datasets. This diversity is critical for addressing the imbalance in existing datasets, where effective building change information is underrepresented. By increasing the proportion of relevant change information in the training data, BAG-GAN improves the learning conditions for deep learning models, enabling them to better generalize across different scenarios. Further analysis revealed that BAG-GAN significantly enhances the ability of detection models to localize changes and recover fine-grained details of building modifications. This is particularly evident in complex scenarios involving subtle or small-scale changes. The adversarial consistency loss function played a pivotal role in ensuring the semantic relevance of the generated data, making BAG-GAN a reliable tool for data augmentation in remote sensing applications. Moreover, the network's ability to generate data pairs with high-quality and multimodal characteristics ensures its applicability to a wide range of remote sensing tasks beyond building change detection. Conclusions This study introduces BAG-GAN, a novel multi-temporal building change detection data pair generation network designed to overcome the limitations of insufficient labeled data in remote sensing. The network incorporates an adversarial consistency loss function, which ensures that the generated data is both semantically consistent and diverse. By leveraging a consistency adversarial generation mechanism, BAG-GAN enhances the quality and diversity of training datasets, addressing key bottlenecks in multi-temporal building change detection tasks. Through experiments on the LEVIR-CD and WHU-CD datasets, BAG-GAN demonstrated its ability to significantly improve the performance of classic remote sensing change detection models, such as FC-EF and FC-Siam-Conc. The results highlight the network’s effectiveness in generating high-quality data pairs that enhance model training and detection accuracy. This research not only provides a robust methodological framework for improving multi-temporal building change detection but also offers a foundational tool for broader applications in remote sensing. The findings pave the way for future advancements in change detection techniques, offering valuable insights for researchers and practitioners in the field.
- Remote Sensing,
- Change Detection,
- Adversarial Generative Network,
- Data Generation

FullText(HTML)

References(52)

References

[1]	NASTASE I N A, MOLDOVANU S, and MORARU L. Deep learning-based segmentation of breast masses using convolutional neural networks[J]. Journal of Physics: Conference Series, 2024, 2701(1): 012005. doi: 10.1088/1742-6596/2701/1/012005.
[2]	刘美琴, 王子麟. 基于域适应的图像语义分割综述[J]. 北京交通大学学报, 2024, 48(2): 1–9. doi: 10.11860/j.issn.1673-0291.20230120. LIU Meiqin and WANG Zilin. A review on image semantic segmentation based on domain adaptation[J]. Journal of Beijing Jiaotong University, 2024, 48(2): 1–9. doi: 10.11860/j.issn.1673-0291.20230120.
[3]	EL-MAGD S A A, MASOUD A M, HASSAN H S, et al. Towards understanding climate change: Impact of land use indices and drainage on land surface temperature for valley drainage and non-drainage areas[J]. Journal of Environmental Management, 2024, 350: 119636. doi: 10.1016/j.jenvman.2023.119636.
[4]	HOSSAIN M S, KHAN M A H, OLUWAJUWON T V, et al. Spatiotemporal change detection of land use land cover (LULC) in Fashiakhali wildlife sanctuary (FKWS) impact area, Bangladesh, employing multispectral images and GIS[J]. Modeling Earth Systems and Environment, 2023, 9(3): 3151–3173. doi: 10.1007/s40808-022-01653-7.
[5]	何自芬, 史本杰, 张印辉, 等. 多注意力融合的环高原湖泊遥感影像分割[J]. 电子学报, 2023, 51(4): 885–895. doi: 10.12263/DZXB.20220085. HE Zifen, SHI Benjie, ZHANG Yinhui, et al. Remote sensing image segmentation of around plateau lakes based on multi-attention fusion[J]. Acta Electronica Sinica, 2023, 51(4): 885–895. doi: 10.12263/DZXB.20220085.
[6]	张冬梅, 李石磊. 一种显著性检测提取高分遥感影像建筑物的方法[J]. 测绘与空间地理信息, 2024, 47(6): 97–101. doi: 10.3969/j.issn.1672-5867.2024.06.029. ZHANG Dongmei and LI Shilei. A saliency detection method for extracting buildings from high-resolution remote sensing images[J]. Geomatics & Spatial Information Technology, 2024, 47(6): 97–101. doi: 10.3969/j.issn.1672-5867.2024.06.029.
[7]	MEI Jie, ZHENG Yibo, and CHENG Mingming. D2ANet: Difference-aware attention network for multi-level change detection from satellite imagery[J]. Computational Visual Media, 2023, 9(3): 563–579. doi: 10.1007/s41095-022-0325-1.
[8]	LIAO Cheng, HU Han, YUAN Xuekun, et al. BCE-Net: Reliable building footprints change extraction based on historical map and up-to-date images using contrastive learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 201: 138–152. doi: 10.1016/j.isprsjprs.2023.05.011.
[9]	KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, USA, 2012: 1097–1105.
[10]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
[11]	HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269. doi: 10.1109/CVPR.2017.243.
[12]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. 3rd International Conference on Learning Representations, San Diego, USA, 2015. doi: 10.48550/arXiv.1409.1556.
[13]	DEMIRCI M Y, BEŞLI N, and GÜMÜŞÇÜ A. An improved hybrid solar cell defect detection approach using generative adversarial networks and weighted classification[J]. Expert Systems with Applications, 2024, 252: 124230. doi: 10.1016/j.eswa.2024.124230.
[14]	刘少鹏, 赵慧民, 洪佳明, 等. 面向医学图像生成的鲁棒条件生成对抗网络[J]. 电子学报, 2023, 51(2): 427–437. doi: 10.12263/DZXB.20210051. LIU Shaopeng, ZHAO Huimin, HONG Jiaming, et al. Medical image synthesis using robust conditional GAN[J]. Acta Electronica Sinica, 2023, 51(2): 427–437. doi: 10.12263/DZXB.20210051.
[15]	STANCIU D C and IONESCU B. Autoencoder-based data augmentation for deepfake detection[C]. Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation, Thessaloniki, Greece, 2023: 19–27. doi: 10.1145/3592572.3592840.
[16]	YANG Suorong, XIAO Weikang, ZHANG Mengchen, et al. Image data augmentation for deep learning: A survey[EB/OL]. https://arxiv.org/abs/2204.08610, 2022.
[17]	OUBARA A, WU Falin, AMAMRA A, et al. Survey on remote sensing data augmentation: Advances, challenges, and future perspectives[C]. 5th Conference on Computing Systems and Applications, Cham, Switzerland, 2022: 95–104. doi: 10.1007/978-3-031-12097-8_9.
[18]	CRESWELL A, WHITE T, DUMOULIN V, et al. Generative adversarial networks: An overview[J]. IEEE Signal Processing Magazine, 2018, 35(1): 53–65. doi: 10.1109/MSP.2017.2765202.
[19]	BOWLES C, CHEN Liang, GUERRERO R, et al. GAN augmentation: Augmenting training data using generative adversarial networks[EB/OL]. https://arxiv.org/abs/1810.10863, 2018.
[20]	DOERSCH C. Tutorial on variational autoencoders[EB/OL]. https://arxiv.org/abs/1606.05908, 2016.
[21]	WANG Yinglin, ZHAO Jianhui, GUO Zhengwei, et al. Soil moisture inversion based on data augmentation method using multi-source remote sensing data[J]. Remote Sensing, 2023, 15(7): 1899. doi: 10.3390/rs15071899.
[22]	HUANG Rui, WANG Ruofei, GUO Qing, et al. Background-mixed augmentation for weakly supervised change detection[C]. Proceedings of the 37th AAAI Conference on Artificial Intelligence, Washington, USA, 2023: 7919–7927. doi: 10.1609/aaai.v37i7.25958.
[23]	但志平, 方帅领, 孙航, 等. 基于双判别器异构CycleGAN框架下多阶通道注意力校准的室外图像去雾[J]. 电子学报, 2023, 51(9): 2558–2571. doi: 10.12263/DZXB.20211337. DAN Zhiping, FANG Shuailing, SUN Hang, et al. Outdoor image dehazing based on multi-order channel attention calibration using a dual-discriminator heterogeneous CycleGAN framework[J]. Acta Electronica Sinica, 2023, 51(9): 2558–2571. doi: 10.12263/DZXB.20211337.
[24]	马宾, 王一利, 徐健, 等. 基于双向生成对抗网络的图像感知哈希算法[J]. 电子学报, 2023, 51(5): 1405–1412. doi: 10.12263/DZXB.20221224. MA Bin, WANG Yili, XU Jian, et al. An image perceptual hash algorithm based on bidirectional generative adversarial network[J]. Acta Electronica Sinica, 2023, 51(5): 1405–1412. doi: 10.12263/DZXB.20221224.
[25]	贾童瑶, 卓力, 李嘉锋, 等. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231–245. doi: 10.12263/DZXB.20220838. JIA Tongyao, ZHUO Li, LI Jiafeng, et al. Research advances on deep learning based single image dehazing[J]. Acta Electronica Sinica, 2023, 51(1): 231–245. doi: 10.12263/DZXB.20220838.
[26]	李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128–138. doi: 10.12263/DZXB.20210659. LI Tao, DONG Xiucheng, and LIN Hongwei. Depth map super-resolution reconstruction based on deeply supervised cross-scale attention network[J]. Acta Electronica Sinica, 2023, 51(1): 128–138. doi: 10.12263/DZXB.20210659.
[27]	JI Shunping, WEI Shiqing, and LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574–586. doi: 10.1109/tgrs.2018.2858817.
[28]	LI Xuan, DUAN Haibin, ZHANG Hui, et al. Data augmentation using image generation for change detection[C]. 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence (DTPI), Beijing, China, 2021: 188–191. doi: 10.1109/DTPI52967.2021.9540199.
[29]	VARGHESE A, GUBBI J, RAMASWAMY A, et al. ChangeNet: A deep learning architecture for visual change detection[C]. Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, 2018: 129–145. doi: 10.1007/978-3-030-11012-3_10.
[30]	SHAFIQUE A, CAO Guo, KHAN Z, et al. Deep learning-based change detection in remote sensing images: A review[J]. Remote Sensing, 2022, 14(4): 871. doi: 10.3390/rs14040871.
[31]	张良培, 武辰. 多时相遥感影像变化检测的现状与展望[J]. 测绘学报, 2017, 46(10): 1447–1458. doi: 10.11947/j.AGCS.2017.20170340. ZHANG Liangpei and WU Chen. Advance and future development of change detection for multi-temporal remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10): 1447–1459. doi: 10.11947/j.AGCS.2017.20170340.
[32]	眭海刚, 冯文卿, 李文卓, 等. 多时相遥感影像变化检测方法综述[J]. 武汉大学学报: 信息科学版, 2018, 43(12): 1885–1898. doi: 10.13203/j.whugis20180251. SUI Haigang, FENG Wenqing, LI Wenzhuo, et al. Review of change detection methods for multi-temporal remote sensing imagery[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 1885–1898. doi: 10.13203/j.whugis20180251.
[33]	佟国峰, 李勇, 丁伟利, 等. 遥感影像变化检测算法综述[J]. 中国图象图形学报, 2015, 20(12): 1561–1571. doi: 10.11834/jig.20151201. TONG Guofeng, LI Yong, DING Weili, et al. Review of remote sensing image change detection[J]. Journal of Image and Graphics, 2015, 20(12): 1561–1571. doi: 10.11834/jig.20151201.
[34]	JI Shunping, SHEN Yanyun, LU Meng, et al. Building instance change detection from large-scale aerial images using convolutional neural networks and simulated samples[J]. Remote Sensing, 2019, 11(11): 1343. doi: 10.3390/rs11111343.
[35]	NEMOTO K, HAMAGUCHI R, SATO M, et al. Building change detection via a combination of CNNs using only RGB aerial imageries[C]. Proceedings of SPIE 10431, Remote Sensing Technologies and Applications in Urban Environments II, Warsaw, Poland, 2017: 104310J. doi: 10.1117/12.2277912.
[36]	LIU Ruoyun, KUFFER M, and PERSELLO C. The temporal dynamics of slums employing a CNN-based change detection approach[J]. Remote Sensing, 2019, 11(23): 2844. doi: 10.3390/rs11232844.
[37]	LIU Yi, PANG Chao, ZHAN Zongqian, et al. Building change detection for remote sensing images using a dual-task constrained deep Siamese convolutional network model[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5): 811–815. doi: 10.1109/LGRS.2020.2988032.
[38]	CHEN Hao and SHI Zhenwei. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J]. Remote Sensing, 2020, 12(10): 1662. doi: 10.3390/rs12101662.
[39]	JIANG Huiwei, HU Xiangyun, LI Kun, et al. PGA-SiamNet: Pyramid feature-based attention-guided Siamese network for remote sensing orthoimagery building change detection[J]. Remote Sensing, 2020, 12(3): 484. doi: 10.3390/rs12030484.
[40]	ZHAN Yang, FU Kun, YAN Menglong, et al. Change detection based on deep Siamese convolutional network for optical aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1845–1849. doi: 10.1109/LGRS.2017.2738149.
[41]	SUN Chen, SHRIVASTAVA A, SINGH S, et al. Revisiting unreasonable effectiveness of data in deep learning era[C]. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 843–852. doi: 10.1109/ICCV.2017.97.
[42]	ALBAHLI S and ALBATTAH W. Deep transfer learning for COVID-19 prediction: Case study for limited data problems[J]. Current Medical Imaging, 2021, 17(8): 973–980. doi: 10.2174/1573405616666201123120417.
[43]	HAO Hanxiang, BAIREDDY S, BARTUSIAK E R, et al. An attention-based system for damage assessment using satellite imagery[C]. 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 2021: 4396–4399. doi: 10.1109/IGARSS47720.2021.9554054.
[44]	SEO M, LEE H, JEON Y, et al. Self-pair: Synthesizing changes from single source for object change detection in remote sensing imagery[C]. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, USA, 2023: 6363–6372. doi: 10.1109/WACV56688.2023.00631.
[45]	CHEN Hao, LI Wenyuan, and SHI Zhenwei. Adversarial instance augmentation for building change detection in remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5603216. doi: 10.1109/TGRS.2021.3066802.
[46]	KUMDAKCI H, ÖNGÜN C, and TEMIZEL A. Generative data augmentation for vehicle detection in aerial images[C]. Pattern Recognition. ICPR International Workshops and Challenges, 2021: 19–31. doi: 10.1007/978-3-030-68793-9_2. (查阅网上资料,未找到本条文献出版地信息,请确认) .
[47]	ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2242–2251. doi: 10.1109/ICCV.2017.244.
[48]	JIANG Yuchen, ZHU Bin, and XIE Bo. Remote sensing images data augmentation based on style transfer under the condition of few samples[J]. Journal of Physics: Conference Series, 2020, 1653(1): 012039. doi: 10.1088/1742-6596/1653/1/012039.
[49]	ZHAO Yihao, WU Ruihai, and DONG Hao. Unpaired image-to-image translation using adversarial consistency loss[C]. 16th European Conference on Computer Vision, Glasgow, UK, 2020: 800–815. doi: 10.1007/978-3-030-58545-7_46.
[50]	ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976. doi: 10.1109/CVPR.2017.632.
[51]	DAUDT R C, LE SAUX B, and BOULCH A. Fully convolutional Siamese networks for change detection[C]. 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018: 4063–4067. doi: 10.1109/ICIP.2018.8451652.
[52]	FANG Sheng, LI Kaiyu, SHAO Jinyuan, et al. SNUNet-CD: A densely connected Siamese network for change detection of VHR images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8007805. doi: 10.1109/LGRS.2021.3056416.