Earth Surface Anomaly Detection Using Graph Neural Network-based Representation and Reasoning of Remote Sensing Geographic Object Relationships
-
摘要: 遥感地物间的语义关系可以表征地物间的相互影响与结构信息，对地表的灾害检测与应急响应具有重要意义。然而，现有的遥感地物关系提取方法多依赖于目标检测，定位精度有限，且关系预测网络主要局限于注意力机制、卷积网络，难以有效建模复杂拓扑关系。此外，公开规范的遥感地物关系数据集的缺乏也进一步制约了该领域的发展。为了解决上述问题，该文建立了遥感地物语义关系数据集，并采用了一种基于图神经网络的关系预测模型，准确提取遥感场景中蕴含的地物关系。具体而言，首先针对地物实例定义了遥感地物关系描述体系，结合地物类别和拓扑信息标注地物间的语义关系，构建了遥感地物语义关系数据集。其次，引入先进的图神经网络模型进行关系预测，通过子图采样和超参数优化，有效提升了模型在遥感场景下的性能。通过上述方法，该文建立了一个小型的遥感地物语义关系数据集，探索了图神经网络在遥感地表异常场景中地物关系提取的应用。在遥感地物关系描述数据集上进行的实验结果表明，模型不仅在验证集的评估指标中表现出较强的竞争力，还在灾害异常场景中的实验中检测到灾害前后地物关系的显著变化，加强了对灾害场景地表异常的理解能力。Abstract:
Objective The increasing frequency and severity of surface anomalies induced by natural processes and human activities has raised the demand for real-time, intelligent remote sensing systems for disaster monitoring and emergency response. Existing approaches to extracting geographic object relationships in remote sensing images primarily rely on object detection models. These approaches often lack sufficient localization precision and fail to capture topological dependencies between objects. Moreover, the absence of standardized, high-quality datasets restricts progress in model development. To address these limitations, this study proposes a framework that integrates graph-based representation with a Graph Neural Network (GNN) architecture to reason over geographic object relationships. The main objectives are to: (1) construct a semantically annotated dataset of geographic object relationships in remote sensing imagery; (2) develop a GNN-based model to improve relationship prediction accuracy; and (3) evaluate the model’s effectiveness in detecting and interpreting surface anomalies by analyzing pre- and post-disaster relationship patterns across a range of scenarios. Methods The methodology comprises three primary components: dataset construction, model development, and performance evaluation. To address the scarcity of labeled data, a semantic relationship dataset is constructed. Thirty high-resolution remote sensing images from the OpenEarthMap dataset are manually annotated using EISeg software, resulting in 17 object categories ( Table 1 ) and five semantic relationships—contain, connect, on, along, and beside—defined through analysis of topological interactions (Table 2 ). Instance-level annotations are generated using connected component labeling, and relationship labels are assigned based on both topological configuration and object category. The resulting dataset includes 7,063 annotated entities and 13,273 relationship triplets. A GNN-based model is developed to predict semantic relationships, incorporating subgraph sampling and hyperparameter optimization. The model employs the Personalized PageRank (PPR) algorithm to extract query-relevant subgraphs, thereby reducing computational complexity while preserving essential topological structure. Message passing mechanisms from RED-GNN are used to propagate node features, and Bayesian optimization is applied to tune hyperparameters. Model performance is assessed using standard metrics: Mean Reciprocal Rank (MRR), HITS@1, and HITS@10.Results and Discussions Extensive experiments demonstrate the high performance of the proposed framework. On the constructed dataset, the model achieves an MRR of 0.9879 on the test set, with HITS@1 and HITS@10 scores of 97.03% and 99.96%, respectively, outperforming baseline methods such as RED-GNN and Grail (Table 5 ). Ablation studies confirm the effectiveness of the PPR sampling strategy, which outperforms random walk, breadth-first search, and standard PageRank in terms of both accuracy and efficiency (Table 6 ). Model generalizability is further assessed using pre- and post-disaster images from the xBD dataset. In hurricane-affected regions (Fig. 6 ,Fig. 7 ), abnormal relationships—such as “sea lake pond contain residential area”—emerge, reflecting the submergence of buildings and roads due to flooding. Frequency histograms (Fig. 8 ,Fig. 9 ) indicate a post-disaster decrease in relationship diversity and a shift toward water-related spatial associations. In wildfire scenarios (Fig. 10 –Fig. 13 ), relationships such as "bareland contain rangeland" replace "tree beside rangeland," suggesting vegetation loss and soil exposure. These findings demonstrate the model’s capacity to detect spatial and semantic shifts in geographic object relationships caused by disasters. Coarse anomaly localization is achieved through centroid-based node mapping, enabling interpretation of surface anomaly dynamics over time.Conclusions This study contributes to remote sensing-based surface anomaly detection through three main innovations. First, a high-quality semantic relationship dataset is constructed with pixel-level annotations and standardized relationship definitions, addressing the lack of labeled data in this area. The dataset includes 17 object categories and five topologically defined relationship types, offering a valuable benchmark for future research. Second, a novel GNN-based model is developed that advances relationship prediction by integrating PPR-based subgraph sampling with optimized message passing mechanisms. Third, the framework is extensively validated using real-world disaster scenarios, demonstrating its practical utility in detecting and interpreting surface anomalies through changes in object relationships. The model’s ability to produce interpretable relationship graphs while maintaining computational efficiency supports its application in time-sensitive emergency response contexts. Future work will focus on expanding image diversity, refining relationship definitions, and incorporating real-world noise to improve robustness. -
表 1 地物类别的定义
类别索引 类别名称 类别颜色 类别描述 1 Tree [34, 97, 38] 树木，有明显的高度信息 2 Rangeland [0, 255, 36] 草地，纹理较为平滑 3 Bareland [128, 0, 0] 裸土，一般是水边的未开发区域 4 AgricType [75, 181, 73] 耕地，植物多、纹理较多 5 Road [255, 255, 255] 道路，车辆行驶的主干道 6 Sea Lake Pond [0, 69, 255] 海水、湖水、池塘 7 Residential Area [222, 31, 7] 居民区建筑 8 Bridge [139, 69, 19] 桥梁、立交桥 9 River [255, 215, 0] 河流 10 Ship [255, 128, 128] 船只 11 Fallow [128, 128, 128] 休耕地，暂时停耕的土地 12 Industrial Area [255, 160, 222] 工业区建筑，一般面积较大 13 Car [198, 226, 255] 车辆 14 SportsField [225, 228,196] 体育场 15 CarPark [0, 139,139] 停车场，可用于停车的空地 16 SandyLand [230, 230,250] 沙地，居民区附近的泥土地等 17 Path [0,255,255] 小路，一般仅供人行走 
下载: 导出CSV
表 2 地物间语义关系词的定义
关系名称 关系示例图 关系描述 contain 
实例A的边界完全包含在实例B中
(包含、被包含都统一用此关系描述)connect 
实例A与实例B几乎以0像素相连 on 
实例A在实例B上，反映层次性关系 along 
条状实例A沿着块状实例B beside 
实例A在实例B附近，但没有相连
下载: 导出CSV
表 4 模型在知识图谱数据集以及本文数据集上的指标对比
数据集名称 验证集MRR指标 HITS@1 HITS@10 WN18RR 0.589 0 50.91 61.25 NELL 0.535 1 47.65 60.23 YAGO 0.621 7 53.66 70.08 遥感地物语义关系数据集 0.987 9 97.03 99.96
下载: 导出CSV
表 5 不同模型在本文数据集上的指标对比
模型名称 验证集MRR指标 HITS@1 HITS@10 推理时间(s) RED-GNN(逐层) 0.536 4 52.48 54.87 100.82 Grail(逐子图) 0.964 2 92.84 97.83 56.94 Ours(单子图) 0.987 9 97.03 99.96 20.62
下载: 导出CSV
表 6 不同采样方法在本文数据集上的指标对比
采样方法 验证集MRR指标 HITS@1 HITS@10 推理时间(s) RW 0.845 6 83.72 85.19 8.72 BFS 0.978 8 96.66 98.82 25.73 PR 0.9792 96.85 98.93 31.65 PPR(本文) 0.987 9 97.03 99.96 20.62
下载: 导出CSV
-
[1] XU Jianming, YAN Kai, FAN Zaiwang, et al. Toward a novel method for general on-orbit earth surface anomaly detection leveraging large vision models and lightweight priors[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 4706321. doi: 10.1109/TGRS.2024.3432749. [2] 王桥. 地表异常遥感探测与即时诊断方法研究框架[J]. 测绘学报, 2022, 51(7): 1141–1152. doi: 10.11947/j.AGCS.2022.20220124.WANG Qiao. Research framework of remote sensing monitoring and real-time diagnosis of earth surface anomalies[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1141–1152. doi: 10.11947/j.AGCS.2022.20220124. [3] JOHNSON J, KRISHNA R, STARK M, et al. Image retrieval using scene graphs[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 3668–3678. doi: 10.1109/CVPR.2015.7298990. [4] LU Xiaoqiang, WANG Binqiang, ZHENG Xiangtao, et al. Exploring models and data for remote sensing image caption generation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(4): 2183–2195. doi: 10.1109/TGRS.2017.2776321. [5] QU Bo, LI Xuelong, TAO Dacheng, et al. Deep semantic understanding of high resolution remote sensing image[C]. 2016 International Conference on Computer, Information and Telecommunication Systems, Kunming, China, 2016: 1–5. doi: 10.1109/CITS.2016.7546397. [6] CHANG Xiaojun, REN Pengzhen, XU Pengfei, et al. A comprehensive survey of scene graphs: Generation and application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1): 1–26. doi: 10.1109/TPAMI.2021.3137605. [7] CAI Qi, PAN Yingwei, NGO C W, et al. Exploring object relation in mean teacher for cross-domain detection[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 11449–11458. doi: 10.1109/CVPR.2019.01172. [8] CHEN Jie, ZHOU Xing, ZHANG Yi, et al. Message-passing-driven triplet representation for geo-object relational inference in HRSI[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 8003205. doi: 10.1109/LGRS.2020.3038569. [9] LIN Zhiyuan, ZHU Feng, KONG Yanzi, et al. SRSG and S2SG: A model and a dataset for scene graph generation of remote sensing images from segmentation results[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4707411. doi: 10.1109/TGRS.2022.3185678. [10] LI Yansheng, WANG Linlin, WANG Tingzhu, et al. STAR: A first-ever dataset and a large-scale benchmark for scene graph generation in large-size satellite imagery[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025, 47(3): 1832–1849. doi: 10.1109/TPAMI.2024.3508072. [11] 陈杰, 戴欣宜, 周兴, 等. 双LSTM驱动的高分遥感影像地物目标空间关系语义描述[J]. 遥感学报, 2021, 25(5): 1085–1094. doi: 10.11834/jrs.20210340.CHEN Jie, DAI Xinyi, ZHOU Xing, et al. Semantic understanding of geo-objects’ relationship in high resolution remote sensing image driven by dual LSTM[J]. National Remote Sensing Bulletin, 2021, 25(5): 1085–1094. doi: 10.11834/jrs.20210340. [12] 伍丝琪. 基于目标空间关系和多源地理数据的高分辨率遥感城市场景语义分类方法研究[D]. [硕士论文], 武汉大学, 2019.WU Siqi. Research on high spatial resolution remote sensing urban scene classification using spatial relationship of objects and multi-source geographic data[D]. [Master dissertation], Wuhan University, 2019. [13] 周兴, 孙庚, 戴欣怡, 等. 基于语句模板的遥感影像地物空间关系语义理解[J]. 测绘工程, 2021, 30(6): 27–33,39. doi: 10.19349/j.cnki.issn1006-7949.2021.06.005.ZHOU Xing, SUN Geng, DAI Xinyi, et al. Template-based spatial relationship semantic understanding of remote sensing images[J]. Engineering of Surveying and Mapping, 2021, 30(6): 27–33,39. doi: 10.19349/j.cnki.issn1006-7949.2021.06.005. [14] XIA Junshi, YOKOYA N, ADRIANO B, et al. OpenEarthMap: A benchmark dataset for global high-resolution land cover mapping[C]. 2023 IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, USA, 2023: 6243–6253. doi: 10.1109/WACV56688.2023.00619. [15] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]. The 32nd AAAI Conference on Artificial Intelligence, New Orleans, USA, 2018: 1811–1818. doi: 10.1609/aaai.v32i1.11573. [16] ZHOU Zhanke, ZHANG Yongqi, YAO Jiangchao, et al. Less is more: One-shot subgraph reasoning on large-scale knowledge graphs[C]. The 12th International Conference on Learning Representations, Vienna, Austria, 2024. [17] ZHANG Yongqi and YAO Quanming. Knowledge graph reasoning with relational digraph[C]. The ACM Web Conference 2022, Lyon, France, 2022: 912–924. doi: 10.1145/3485447.3512008. [18] TERU K K, DENIS E G, and HAMILTON W L. Inductive relation prediction by subgraph reasoning[C]. The 37th International Conference on Machine Learning, 2020: 876. [19] XIONG Wenhan, HOANG T, and WANG W Y. DeepPath: A reinforcement learning method for knowledge graph reasoning[C]. The 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017. doi: 10.18653/v1/D17-1060. [20] SUCHANEK F M, KASNECI G, and WEIKUM G. Yago: A core of semantic knowledge[C]. The 16th International Conference on World Wide Web, Banff, Canada, 2007: 697–706. doi: 10.1145/1242572.1242667. [21] GUPTA R, GOODMAN B, PATEL N, et al. Creating xBD: A dataset for assessing building damage from satellite imagery[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, USA, 2019: 10–17. doi: 10.1184/R1/8135576.v1. -
图(13) / 表(6)
计量
- 文章访问数: 283
- HTML全文浏览量: 134
- PDF下载量: 36
- 被引次数: 0
下载:
