Difference-aware Adaptive Prompt Learning and Dense Alignment for Weakly Supervised Building Change Detection
-
摘要: 遥感影像建筑物变化检测可为城市规划、土地资源管理和违章建筑排查等提供重要信息，但像素级标注成本高昂，限制了其在大规模场景中的应用。针对现有图像级弱监督建筑物变化检测方法存在语义先验利用不足、变化区域缺乏空间约束以及定位结果不完整等问题，该文设计了一种差异感知提示学习与密集语义对齐的弱监督建筑物变化检测方法(DAPL−CD)。为了在降低像素级标注依赖的同时提升变化区域定位能力，该方法利用对比语言-图像预训练模型(CLIP)的跨模态表征能力，将文本语义与双时相影像差异特征映射至统一匹配空间，为变化区域定位提供语义参照。设计差异感知可学习文本提示自适应双时相影像中的复杂差异语义，缓解固定文本模板难以适配遥感场景的问题。针对CLIP全局图文对齐缺乏局部定位约束的问题，引入像素−文本密集对齐机制，约束局部视觉特征与前景、背景文本特征之间的匹配关系，增强变化区域与未变化区域的语义判别能力。实验结果表明，在仅依赖图像级标签的情况下，所提方法在WHU−CD和LEVIR−CD两个数据集上均取得了较优结果，能获得更完整、准确的变化区域定位效果，为大规模城市监测与违建排查等应用提供了一种低标注成本的技术思路。Abstract:
Objective Building change detection from bi-temporal high-resolution remote sensing images is important for urban planning, land resource management, illegal construction monitoring, and disaster damage assessment. Existing fully supervised change detection methods generally achieve high detection accuracy but require pixel-level annotations. However, obtaining pixel-level labels for large-scale remote sensing images is labor-intensive and time-consuming, which limits their application to large-scale monitoring scenarios. Image-level weakly supervised change detection reduces annotation costs by using only image-level labels indicating whether an image pair contains changes. However, the lack of spatial supervision makes accurate localization of changed regions difficult. Existing weakly supervised methods generally rely on Class Activation Maps (CAMs) to generate pseudo labels. CAMs tend to highlight only the most discriminative regions, resulting in incomplete coverage of changed areas or background noise. Vision-language models provide semantic priors for weakly supervised learning. However, directly applying Contrastive Language-Image Pre-training (CLIP) to change detection remains challenging. Fixed text prompts are difficult to adapt to the difference semantics of bi-temporal images, and the original CLIP objective mainly focuses on global image-text alignment rather than local pixel-level localization. To address these problems, a Difference-aware Adaptive Prompt Learning and Dense Alignment method for weakly supervised building change detection, termed DAPL-CD, is proposed. Methods The proposed framework introduces CLIP-based cross-modal semantic knowledge into image-level weakly supervised building change detection. For a pair of bi-temporal remote sensing images, a shared CLIP visual encoder is first used to extract visual representations from the two temporal images. The local visual features are fused along the channel dimension to obtain bi-temporal difference features containing semantic information related to changed and unchanged regions. Based on the difference characteristics of building change detection, a difference-aware adaptive prompt learning strategy is designed. Instead of using manually designed fixed text templates, learnable context vectors are inserted into the text prompts while preserving category-related semantic words. The resulting foreground and background text embeddings are used as foreground and background text prototypes to provide adaptive semantic guidance for change localization. Furthermore, a pixel-text dense alignment mechanism is introduced to extend CLIP’s global image-text alignment capability to local feature matching. The initial CAM generated by the classification branch is used to obtain preliminary foreground and background regions. Visual-text positive and negative sample pairs are then constructed between local difference features and the foreground and background text prototypes. An InfoNCE-based dense alignment loss is used to pull matched visual and textual features closer and push mismatched features apart. Finally, the classification and segmentation branches are jointly optimized using the classification loss, global alignment loss, dense alignment loss, and segmentation loss, with the segmentation loss introduced only after the quality of the generated CAMs has stabilized. Results and Discussions Experiments are conducted on WHU-CD and LEVIR-CD, two public benchmark datasets for building change detection. Only image-level labels are used during training, whereas pixel-level annotations are used only for evaluation. Overall Accuracy (OA), F1-score, and Intersection over Union (IoU) are adopted as the main evaluation metrics. Because changed buildings usually occupy a small proportion of remote sensing images, OA can be strongly affected by the dominant unchanged background pixels. Therefore, F1-score and IoU are emphasized for evaluating the detection quality of changed regions. Quantitative comparisons show that DAPL-CD achieves an OA of 94.7%, an F1-score of 82.8%, and an IoU of 70.6% on WHU-CD, and an OA of 92.3%, an F1-score of 68.0%, and an IoU of 51.5% on LEVIR-CD. The method achieves the best F1-score and IoU among the compared weakly supervised change detection methods ( Table 1 ). Visual comparisons further show that the proposed method produces more complete responses for large-scale building changes and more continuous predictions for small and scattered changed buildings (Figs. 3 and4 ). Ablation experiments verify the effectiveness of difference-aware adaptive prompt learning and pixel-text dense alignment. The baseline model using fixed text prompts without foreground or background alignment achieves an F1-score of 63.2% and an IoU of 46.2%. Introducing both foreground and background alignment increases these metrics to 66.3% and 49.6%, respectively, indicating that dense semantic matching between local visual features and text prototypes improves the discrimination of changed regions. After difference-aware adaptive prompt learning is incorporated, the F1-score and IoU increase to 68.0% and 51.5%, respectively, indicating that learnable context vectors reduce the semantic mismatch between fixed text descriptions and bi-temporal difference features. Under the adaptive-prompt setting, foreground alignment alone achieves an F1-score of 64.4% and an IoU of 47.5%, whereas background alignment alone achieves 65.3% and 48.5%, respectively. Combining the two alignment branches yields the best F1-score and IoU, indicating that foreground and background semantic constraints provide complementary guidance for change localization (Table 2 ). The CAM results further show that pixel-text dense alignment produces stronger and more complete responses over actual changed regions while suppressing irrelevant background activations (Fig. 5 ).Conclusions A weakly supervised building change detection framework based on difference-aware adaptive prompt learning and pixel-text dense alignment is proposed. By introducing CLIP-based cross-modal semantic priors, the proposed method converts text-level semantic knowledge into local change localization capability. The difference-aware adaptive prompt learning strategy improves the representation of change-related semantic descriptions, whereas the pixel-text dense alignment mechanism establishes direct correspondence between local difference features and foreground and background text prototypes. Experimental results on WHU-CD and LEVIR-CD demonstrate that DAPL-CD achieves high performance under image-level supervision and improves the completeness and accuracy of changed building localization. The proposed framework provides an effective approach for reducing annotation requirements in large-scale remote sensing change detection. Future research will focus on improving pseudo-label reliability, reducing dependence on large-scale pre-trained models, and extending the method to multi-temporal and multi-spectral remote sensing data. -
表 2 重要模块的消融实验结果
前景对齐 背景对齐 可学习文本提示 上下文长度M OA(%) F1(%) IoU(%) × × × − 89.6 63.2 46.2 × × √ 16 89.5 63.7 46.7 √ × √ 16 89.6 64.4 47.5 × √ √ 16 90.1 65.3 48.5 √ √ × − 91.2 66.3 49.6 √ √ √ 8 92.7 67.6 51.0 √ √ √ 16 92.3 68.0 51.5 √ √ √ 32 92.2 67.1 50.5 
下载: 导出CSV
-
[1] 肖振久, 田昊, 张杰浩, 等. 融合动态特征增强的遥感建筑物分割[J]. 光电工程, 2025, 52(3): 240231. doi: 10.12086/oee.2025.240231.XIAO Zhenjiu, TIAN Hao, ZHANG Jiehao, et al. Fusion of dynamic features enhances remote sensing building segmentation[J]. Opto-Electronic Engineering, 2025, 52(3): 240231. doi: 10.12086/oee.2025.240231. [2] 陈昊, 周光尧, 王乾通, 等. 基于一致性生成对抗的遥感多时相建筑物变化检测数据对生成技术[J]. 电子与信息学报, 2025, 47(3): 825–838. doi: 10.11999/JEIT240720.CHEN Hao, ZHOU Guangyao, WANG Qiantong, et al. Building change detection data generation technology for multi-temporal remote sensing imagery based on consistent generative adversarial[J]. Journal of Electronics & Information Technology, 2025, 47(3): 825–838. doi: 10.11999/JEIT240720. [3] MALILA W A. Change vector analysis: An approach for detecting forest changes with Landsat[C]. The 6th Annual Symposium on Machine Processing of Remotely Sensed Data, West Lafayette, USA, 1980: 326–335. [4] DENG Jinsong, WANG Ke, DENG Yuhong, et al. PCA-based land‐use change detection and analysis using multitemporal and multisensor satellite data[J]. International Journal of Remote Sensing, 2008, 29(16): 4823–4838. doi: 10.1080/01431160801950162. [5] NIELSEN A A. The regularized iteratively reweighted MAD method for change detection in multi-and hyperspectral data[J]. IEEE Transactions on Image Processing, 2007, 16(2): 463–478. doi: 10.1109/TIP.2006.888195. [6] 李希, 曾怀恩, 韦朋成. 基于增强超分辨率的异源遥感影像双路径短期密集连接度量变化检测[J]. 电子与信息学报, 2025, 47(12): 4728–4741. doi: 10.11999/JEIT250328.LI Xi, ZENG Huaien, and WEI Pengcheng. Enhanced super-resolution-based dual-path short-term dense concatenate metric change detection network for heterogeneous remote sensing images[J]. Journal of Electronics & Information Technology, 2025, 47(12): 4728–4741. doi: 10.11999/JEIT250328. [7] 翁星星, 庞超, 许博文, 等. 面向遥感图像解译的增量深度学习[J]. 电子与信息学报, 2024, 46(10): 3979–4001. doi: 10.11999/JEIT240172.WENG Xingxing, PANG Chao, XU Bowen, et al. Incremental deep learning for remote sensing image interpretation[J]. Journal of Electronics & Information Technology, 2024, 46(10): 3979–4001. doi: 10.11999/JEIT240172. [8] JI Shunping, WEI Shiqing, and LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574–586. doi: 10.1109/TGRS.2018.2858817. [9] DAUDT R C, LE SAUX B, and BOULCH A. Fully convolutional Siamese networks for change detection[C]. 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018: 4063–4067. doi: 10.1109/ICIP.2018.8451652. [10] FANG Sheng, LI Kaiyu, SHAO Jinyuan, et al. SNUNet-CD: A densely connected Siamese network for change detection of VHR images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8007805. doi: 10.1109/LGRS.2021.3056416. [11] ZHANG Haotian, CHEN Hao, ZHOU Chenyao, et al. BiFA: Remote sensing image change detection with bitemporal feature alignment[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5614317. doi: 10.1109/TGRS.2024.3376673. [12] NOMAN M, FIAZ M, CHOLAKKAL H, et al. Remote sensing change detection with transformers trained from scratch[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 4704214. doi: 10.1109/TGRS.2024.3383800. [13] ZHOU Jiting, ZHANG Pu, and ZHOU Zhihao. CIMF-Net: A change indicator-enhanced multiscale fusion network for remote sensing change detection[J]. IEEE Access, 2025, 13: 66843–66854. doi: 10.1109/ACCESS.2025.3560591. [14] DING Lei, HONG Danfeng, ZHAO Maofan, et al. A survey of sample-efficient deep learning for change detection in remote sensing: Tasks, strategies, and challenges[J]. IEEE Geoscience and Remote Sensing Magazine, 2025, 13(3): 164–189. doi: 10.1109/MGRS.2025.3533605. [15] ZHOU Bolei, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016: 2921–2929. doi: 10.1109/CVPR.2016.319. [16] ZHAO Zhenghui, RU Lixiang, WU Chen, et al. TransWCD: Scene-adaptive joint constrained framework for weakly supervised change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 4702112. doi: 10.1109/TGRS.2025.3545051. [17] LIU Wenhao, YU Zhuoyuan, and LUO Bin. ACWCD: Utilizing inherent transformers information and prior knowledge for weakly supervised change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 4402614. doi: 10.1109/TGRS.2025.3527009. [18] LI Zhenglai, TANG Chang, LIU Xinwang, et al. MS-Former: Memory-supported transformer for weakly supervised change detection with patch-level annotations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5625213. doi: 10.1109/TGRS.2024.3399215. [19] FANG Leyuan, JIANG Yiqi, YU Hongfeng, et al. Point label meets remote sensing change detection: A consistency-aligned regional growth network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5603911. doi: 10.1109/TGRS.2023.3348459. [20] WU Chen, DU Bo, and ZHANG Liangpei. Fully convolutional change detection framework with generative adversarial network for unsupervised, weakly supervised and regional supervised change detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 9774–9788. doi: 10.1109/TPAMI.2023.3237896. [21] LI Kaiyu, CAO Xiangyong, DENG Yupeng, et al. SemiCD-VL: Visual-language model guidance makes better semi-supervised change detector[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5601613. doi: 10.1109/TGRS.2024.3512548. [22] QIU Junlong, LIU Wei, ZHANG Hui, et al. A novel change detection method based on visual language from high-resolution remote sensing images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025, 18: 4554–4567. doi: 10.1109/JSTARS.2024.3524382. [23] 田澍, 张秉熙, 曹林, 等. 基于目标语义提示与双注意力感知的遥感图像文本检索方法[J]. 电子与信息学报, 2025, 47(6): 1734–1746. doi: 10.11999/JEIT240946.TIAN Shu, ZHANG Bingxi, CAO Lin, et al. Remote sensing image text retrieval method based on object semantic prompt and dual-attention perception[J]. Journal of Electronics & Information Technology, 2025, 47(6): 1734–1746. doi: 10.11999/JEIT240946. [24] RADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language supervision[C]. The 38th International Conference on Machine Learning, 2021: 8748–8763. [25] DONG Sijun, WANG Libo, DU Bo, et al. ChangeCLIP: Remote sensing change detection with multimodal vision-language representation learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2024, 208: 53–69. doi: 10.1016/j.isprsjprs.2024.01.004. [26] VAN DEN OORD A, LI Yazhen, and VINYALS O. Representation learning with contrastive predictive coding[EB/OL]. https://arxiv.org/abs/1807.03748, 2018. doi: 10.48550/arXiv.1807.03748. [27] LONG J, SHELHAMER E, and DARRELL T. Fully convolutional networks for semantic segmentation[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 3431–3440. doi: 10.1109/CVPR.2015.7298965. [28] MILLETARI F, NAVAB N, and AHMADI S A. V-Net: Fully convolutional neural networks for volumetric medical image segmentation[C]. 2016 Fourth International Conference on 3D Vision (3DV), Stanford, USA, 2016: 565–571. doi: 10.1109/3DV.2016.79. [29] CHEN Hao and SHI Zhenwei. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J]. Remote Sensing, 2020, 12(10): 1662. doi: 10.3390/rs12101662. [30] ANDERMATT P and TIMOFTE R. A weakly supervised convolutional network for change segmentation and classification[C]. 15th Asian Conference on Computer Vision on Computer Vision, Kyoto, Japan, 2020: 103–119. doi: 10.1007/978-3-030-69756-3_8. [31] HUANG Rui, WANG Ruofei, GUO Qing, et al. Background-mixed augmentation for weakly supervised change detection[C]. The 37th AAAI Conference on Artificial Intelligence, Washington, USA, 2023: 7919–7927. doi: 10.1609/aaai.v37i7.25958. [32] CAO Yinxia, HUANG Xin, and WENG Qihao. A multi-scale weakly supervised learning method with adaptive online noise correction for high-resolution change detection of built-up areas[J]. Remote Sensing of Environment, 2023, 297: 113779. doi: 10.1016/j.rse.2023.113779. -
下载:
图(5) / 表(2)
计量
- 文章访问数: 260
- HTML全文浏览量: 66
- PDF下载量: 19
- 被引次数: 0
